Commit Graph

16 Commits

Author SHA1 Message Date
Sebastian Hengst
3174048fe9 Backed out changeset 54e4545b0fe0 (bug 1380338) for breaking spidermonkey pkg due to failing to process taskgraph/test/python.ini. r=backout 2017-07-12 19:18:46 +02:00
Andrew Halberstadt
7ec26c47a2 Bug 1380338 - Convert taskgraph unit tests to the |mach python-test| framework, r=dustin
These tests can now be run with:
./mach python-test taskcluster/taskgraph

or:
./mach python-test taskcluster

They can now run in parallel by passing in -j.

MozReview-Commit-ID: JXeZV8B04Sf

--HG--
extra : rebase_source : a8bb8c21d4c9722bf630c426a780f856fffc8cdb
2017-07-12 10:47:14 -04:00
Dustin J. Mitchell
449d528cf3 Bug 1333255: rename taskgraph.task to taskgraph.loader; r=jonasfj
MozReview-Commit-ID: 6aPdGbxn0D1

--HG--
rename : taskcluster/taskgraph/task/__init__.py => taskcluster/taskgraph/loader/__init__.py
rename : taskcluster/taskgraph/task/balrog.py => taskcluster/taskgraph/loader/balrog.py
rename : taskcluster/taskgraph/task/beetmover.py => taskcluster/taskgraph/loader/beetmover.py
rename : taskcluster/taskgraph/task/beetmover_checksums.py => taskcluster/taskgraph/loader/beetmover_checksums.py
rename : taskcluster/taskgraph/task/checksums_signing.py => taskcluster/taskgraph/loader/checksums_signing.py
rename : taskcluster/taskgraph/task/post_build.py => taskcluster/taskgraph/loader/post_build.py
rename : taskcluster/taskgraph/task/repacks.py => taskcluster/taskgraph/loader/repacks.py
rename : taskcluster/taskgraph/task/signing.py => taskcluster/taskgraph/loader/signing.py
rename : taskcluster/taskgraph/task/test.py => taskcluster/taskgraph/loader/test.py
rename : taskcluster/taskgraph/task/transform.py => taskcluster/taskgraph/loader/transform.py
rename : taskcluster/taskgraph/task/base.py => taskcluster/taskgraph/task.py
extra : rebase_source : 42a183bae9aedfa04876d99a59119fd08bbf7d73
2017-03-09 19:00:49 -05:00
Dustin J. Mitchell
d46264d259 Bug 1333255: and then there was only one Task class; r=jonasfj
Note that the to_json method prefers the taskgraph's dependencies information
(edges) to that from the task.dependencies entries. At a few points in
task-graph generation, these values differ, although that is expected (for
example, the full task set contains no edges, but that information is still in
task.dependencies). Unifying that representation leads to some difficulty with
task transforms that reach into the dependency tree (beetmover), so the
different representations are left as-is.

MozReview-Commit-ID: GeW8HNwFA9Z

--HG--
extra : rebase_source : 549773e05e18371a399612d9bceccffc29be8cf2
2017-03-09 23:14:30 -05:00
Dustin J. Mitchell
b8d8f8a712 Bug 1252948: support for periodic taskgraphs; r=Callek,jonasfj,kmoir
This adds `.cron.yml` and a new mach command to interpret it.  While
functionality is limited to nightlies right now, there is room to expand to
more diverse periodic tasks.  Let your imagination run wild!

MozReview-Commit-ID: KxQkaUbsjQs

--HG--
extra : rebase_source : ddf0a1eadae5a1169c0ead7bcb7b9ce61b255fbf
2017-01-18 19:45:53 +00:00
Dustin J. Mitchell
1045a329b8 Bug 1302227: set the taskGroupId to match the decision task; r=garndt
In this case, the tasks must have the same schedulerId as the existing task,
so this is calculated using parameters['level'].

MozReview-Commit-ID: G8EE2kvFstT

--HG--
extra : rebase_source : 214885da9b58520727d5f80b9a31bb8a206f9279
2016-09-22 21:30:56 +00:00
Dustin J. Mitchell
c6a548be78 Bug 1280231: rename types.py to taskgraph.py; r=Callek
MozReview-Commit-ID: Jf6CVyFCGMI

--HG--
rename : taskcluster/taskgraph/types.py => taskcluster/taskgraph/taskgraph.py
extra : rebase_source : 12e30313eb1fd4062c0be9d869460644ae949019
extra : intermediate-source : 4b1d94901b4d09c13031982c28eec6a48f938c71
extra : source : e85765be39d61d7c0df9831e16930328effad554
2016-06-20 19:11:52 +00:00
Dustin J. Mitchell
49f6131a63 Bug 1280231: refactor task kinds to task classes; r=jonasfj
MozReview-Commit-ID: 1cNukxBgfey

--HG--
extra : rebase_source : 4f0fcce2bcea0fb78ba70e7c052638ca2c5b8a3d
extra : intermediate-source : ba5cbf4e06a550993e5216f816dcf0ccd3938b2e
extra : source : f744bd2fbcd3ae9b90851dcd12307c15d04f8bea
2016-06-27 22:57:44 +00:00
Wes Kocher
c9059e6a53 Backed out 3 changesets (bug 1280231) for decision task failures CLOSED TREE
Backed out changeset 4b1d94901b4d (bug 1280231)
Backed out changeset 7898d1ab1afc (bug 1280231)
Backed out changeset ba5cbf4e06a5 (bug 1280231)

--HG--
rename : taskcluster/taskgraph/taskgraph.py => taskcluster/taskgraph/types.py
2016-06-27 15:45:44 -07:00
Dustin J. Mitchell
1a1a14c19c Bug 1280231: rename types.py to taskgraph.py; r=Callek
MozReview-Commit-ID: Jf6CVyFCGMI

--HG--
rename : taskcluster/taskgraph/types.py => taskcluster/taskgraph/taskgraph.py
extra : source : e85765be39d61d7c0df9831e16930328effad554
extra : histedit_source : 732dc9a6b9206be1bb2a3e4f1d507d0001588f6c
2016-06-20 19:11:52 +00:00
Dustin J. Mitchell
4cfe2b182c Bug 1280231: refactor task kinds to task classes; r=jonasfj
MozReview-Commit-ID: 1cNukxBgfey

--HG--
extra : source : f744bd2fbcd3ae9b90851dcd12307c15d04f8bea
2016-06-20 20:45:15 +00:00
Justin Wood
15b23fced1 Bug 1280956 - Use in-tree linter job to flake8 test taskcluster directory. r=dustin
MozReview-Commit-ID: FsWmAnnycZ2

--HG--
extra : rebase_source : 04a32cea2de133cb75472092cffb8a215f7dc603
2016-06-20 21:06:55 -04:00
Justin Wood
d57a9292af Bug 1280914 - Fix test_create after "Bug 1276216 - All tasks with no other dependencies should depend on the decision task"
MozReview-Commit-ID: CCdXafMIDUD
2016-06-20 10:22:15 -04:00
Dustin J. Mitchell
69471c3482 Bug 1276216: make any tasks without dependencies depend on the decision task; r=ahal
MozReview-Commit-ID: 5k5jC7CCHNW

--HG--
extra : rebase_source : 0f77e208b1947b2082481b7ceeab57055c13797e
2016-06-06 17:23:03 +00:00
Dustin J. Mitchell
722b95f979 Bug 1274611: Implement task-graph optimization; r=gps
* Implement & document optimization (although legacy kind doesn't do much of it)
 * Introduce `optimize_target_tasks` parameter to control whether tasks in the
   target set can be optimized (no for try, yes for most other branches)
 * Refactor to include resolved taskIds in the optimized task graph
 * Include a `label-to-taskid.json` artifact.
 * Introduce {'task-reference': '... <dependency-name> ...'} for referring to
   parent tasks' taskId.

MozReview-Commit-ID: LWvlWNz49U5

--HG--
extra : rebase_source : 780e0e23d24b268ade33ecdcbccb5081f32aac48
2016-06-05 19:49:41 +00:00
Dustin J. Mitchell
cea2249ca3 Bug 1258497: Implement a new taskgraph generation system; r=gps
The `taskgraph` package generates TaskCluster task graphs based on collections
of task "kinds".  Initially, there is only one kind, the "legacy" kind, which
reads the YAML files from `testing/taskcluster/tasks` to generate the task
graph.

Try syntax is implemented by filtering the tasks in the taskgraph after it has
been created, then extending the result to include any prerequisite tasks.

A collection of `mach taskgraph` subcommands are provided for developers to
extend or debug the task-graph generation process.

MozReview-Commit-ID: 1TJCns4XxZ8

--HG--
rename : testing/taskcluster/docs/index.rst => taskcluster/docs/index.rst
extra : rebase_source : 7b9125281d66044db9bd8e4a1fade16136f384b9
extra : histedit_source : 47640d27080acda0279270babbcf33f5badb0d1c
2016-05-16 22:53:22 +00:00