Commit Graph

178 Commits

Author SHA1 Message Date
Dustin J. Mitchell
271fc2bc85 Bug 1275409: remove taskcluster_graph.image_builder; r=wcosta
MozReview-Commit-ID: 21HBtcDVPXC

--HG--
extra : rebase_source : 985d158fdd3c326f422ce6feea1a59c688d48b01
2016-06-06 18:55:10 +00:00
Dustin J. Mitchell
a8e677a2f0 Bug 1275409: remove taskcluster_graph.transform; r=wcosta
MozReview-Commit-ID: B5Ee70AXFLP

--HG--
extra : rebase_source : 2d5b989d62d35452ae720c6231088b49f398a5c9
2016-06-04 19:31:44 +00:00
Dustin J. Mitchell
6c587676b0 Bug 1275409: move taskcluster_graph.from_now to taskgraph.util.time; r=wcosta
MozReview-Commit-ID: 3GFO4f34dg8

--HG--
rename : testing/taskcluster/tests/test_from_now.py => taskcluster/taskgraph/test/test_util_time.py
rename : testing/taskcluster/taskcluster_graph/from_now.py => taskcluster/taskgraph/util/time.py
extra : rebase_source : 5dfe5c92fa42c2837381232d3f1b343e93f6422e
2016-06-04 19:20:07 +00:00
Dustin J. Mitchell
c1ccda957f Bug 1275409: move templates to taskgraph.util; r=wcosta
MozReview-Commit-ID: 3vdnm20W4OD

--HG--
rename : taskcluster/taskgraph/test/test_util.py => taskcluster/taskgraph/test/test_util_docker.py
rename : testing/taskcluster/tests/test_templates.py => taskcluster/taskgraph/test/test_util_templates.py
rename : taskcluster/taskgraph/util.py => taskcluster/taskgraph/util/__init__.py
rename : testing/taskcluster/taskcluster_graph/templates.py => taskcluster/taskgraph/util/templates.py
extra : rebase_source : 6d098d87e715b82c0dcd5bf03beb7646bbd50fe2
2016-06-05 18:34:22 +00:00
Dustin J. Mitchell
8e8b2fa3df Bug 1275409: remove testing/taskcluster/mach_commands.py, except 'mach taskcluster-load-image'; r=wcosta
MozReview-Commit-ID: BrhXVWT5r8D

--HG--
extra : rebase_source : a6f6296405ed88cb2fda8cb96c229992830d4374
2016-06-04 19:40:35 +00:00
Rob Thijssen
ca75277c21 Bug 1273549 - Split tc build task inheritance (docker/windows). r=pmoore
--HG--
extra : rebase_source : 6bc3cef8857fb87d222809a8d98318271e59c43f
2016-06-08 19:31:58 +01:00
Dustin J. Mitchell
6ea3546a3e Bug 1277417: output task information in JSON or just labels; r=ahal
The JSON output is suitable for processing with `jq` to extract features of
interest.

MozReview-Commit-ID: 5wpV7sXlOz3

--HG--
extra : rebase_source : 4ffb78ab7a85b32e64d10218a4a8841c22e689f8
2016-06-07 03:09:48 +00:00
Gregory Szorc
f1ac718545 Bug 1278406 - Use a thread pool for submitting tasks to queue; r=dustin
Currently, Gecko decision tasks spend ~8 minutes making HTTP requests to the
queue API. Let's throw a thread pool at it so tasks are submitted faster.

This could possibly be faster if there were a single batch submit API in
the queue. However, that isn't implemented for technical reasons described
in the bug.

MozReview-Commit-ID: MeHItEVBbk

--HG--
extra : rebase_source : ebe182c5906df70cd2e4cbd545f7b7b9189882da
2016-06-07 11:06:12 -07:00
Dustin J. Mitchell
69471c3482 Bug 1276216: make any tasks without dependencies depend on the decision task; r=ahal
MozReview-Commit-ID: 5k5jC7CCHNW

--HG--
extra : rebase_source : 0f77e208b1947b2082481b7ceeab57055c13797e
2016-06-06 17:23:03 +00:00
Dustin J. Mitchell
bf9e498725 Bug 1274611: set schedulerId for image task, too; r=wcosta
MozReview-Commit-ID: 7ThMbpT5BOW

--HG--
extra : rebase_source : d1874694e249237973f05883ebf5ff566e5ebb87
2016-06-05 22:31:01 +00:00
Dustin J. Mitchell
3c13ffbd14 Bug 1274611: spell target-tasks.json with -; r=wcosta
MozReview-Commit-ID: Aij7wL3onop

--HG--
extra : rebase_source : a22fe4849b8494bf2bc460e2f7ce9be08f76ec69
2016-06-01 20:29:08 +00:00
Dustin J. Mitchell
d788a52aeb Bug 1274611: implement docker image builds as a distinct kind; r=wcosta
MozReview-Commit-ID: 81Ad9LcBdx6

--HG--
extra : rebase_source : 3a2c06040f7af8b757eec2991360a7a4d3cf61cb
2016-06-05 19:15:53 +00:00
Dustin J. Mitchell
722b95f979 Bug 1274611: Implement task-graph optimization; r=gps
* Implement & document optimization (although legacy kind doesn't do much of it)
 * Introduce `optimize_target_tasks` parameter to control whether tasks in the
   target set can be optimized (no for try, yes for most other branches)
 * Refactor to include resolved taskIds in the optimized task graph
 * Include a `label-to-taskid.json` artifact.
 * Introduce {'task-reference': '... <dependency-name> ...'} for referring to
   parent tasks' taskId.

MozReview-Commit-ID: LWvlWNz49U5

--HG--
extra : rebase_source : 780e0e23d24b268ade33ecdcbccb5081f32aac48
2016-06-05 19:49:41 +00:00
Dustin J. Mitchell
05baf69879 Bug 1269340: add support for try -t option; r=jmaher
This adds support for the `-t`/`--talos` option, matching such jobs against
`talos_try_name`.  There are no such tasks just yet.

MozReview-Commit-ID: FTEx7Nyyi9Z

--HG--
extra : rebase_source : 64f289ed18a90c4d2c6988935a5865b41367f976
2016-05-18 17:55:33 +00:00
Andrew Halberstadt
7f3e0d8258 Bug 1275613 - Always schedule a 'file_patterns' job when its task definition has changed, r=dustin
Currently, jobs which has 'file_patterns' set will only be run if a file matching one
of those patterns has been modified. This means that unless the task definition of the
job was explicitly included in those 'file_patterns', the job won't run when we modify
it!

I think it is safe to assume that when making changes to a job's task, we want it to be
run no matter what. So let's add the task definition to 'file_patterns' automatically.

Here's a try run which modifies the marionette harness task:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=01e6cf9d8599c64d61ac8ecb5320c1c4209e0a8b

MozReview-Commit-ID: AUSfUFzhiA1

--HG--
extra : rebase_source : de9d66d1d17de90dc83681e51134637616a0652e
2016-05-25 11:56:38 -04:00
Steve Fink
c9eb80e36c Bug 1273673 followup follwup - fix string concatenation that broke precedence, r=dustin
--HG--
extra : rebase_source : b8a9d9ad49e6abd5eb3f58ead54742a204821945
2016-05-19 11:08:16 -07:00
Dustin J. Mitchell
d99040fe43 Bug 1274645: generate task labels for docker image builds; r=ahal
This monkey-patches the old taskcluster_graph code to avoid impacting branches
that are still using that support for decision graphs.  All of this will go
away when docker image builds become their own kind.

MozReview-Commit-ID: KukC3qqZDYA

--HG--
extra : rebase_source : 7bc4e89ad586120fdea78dfd7539633014ccc300
extra : source : 5e8b6de669628bf838432b93d1e1cff783b7ff62
2016-05-20 21:04:49 +00:00
Steve Fink
c2977b87db Bug 1273673 followup: bad logging syntax, r=dustin on a CLOSED TREE
--HG--
extra : rebase_source : d74730a644796510d52a994493c5065d9cf5a110
extra : amend_source : 1e4538b179fa72c35fed09d8b4def3724fd325d6
2016-05-19 10:35:02 -07:00
Dustin J. Mitchell
47d764d976 Bug 1273673: log task creation, too; r=gps
MozReview-Commit-ID: K2Pn4WDXsxo

--HG--
extra : rebase_source : 76860de71812c3c0a2725bd2cbe213f71201ab49
2016-05-18 15:58:21 +00:00
Dustin J. Mitchell
ef1616806f Bug 1273673: log phases of taskgraph generation; r=gps
MozReview-Commit-ID: 1raUmGXSS8b

--HG--
extra : rebase_source : e3b48d7df28fcd4e8b97610a818f303e7cc12559
2016-05-18 15:54:30 +00:00
Dustin J. Mitchell
120576c2a9 Bug 1273673: use Python logging for taskgraph generation; r=gps
MozReview-Commit-ID: EYI6hGFUWP8

--HG--
extra : rebase_source : 06095357cec152790c6025184e4c7c49e5d03835
2016-05-18 18:02:51 +00:00
Dustin J. Mitchell
e98ea9da30 Bug 1273633: support per-branch config in the decision task; r=garndt
MozReview-Commit-ID: LXQIaSzPpr4

--HG--
extra : rebase_source : ebdbc008d0547b84edef5f92e12325a1b90a437a
2016-05-17 21:27:11 +00:00
Dustin J. Mitchell
05ef600e1c Bug 1258497: wrap mach Commands in try/except; r=gps
MozReview-Commit-ID: FTPDTjw4gR

--HG--
extra : rebase_source : a922f03418d9c03f4018940554a23ed341a4e807
extra : histedit_source : 439fda8e499696cca39dff7e1b436f74541d17bb
2016-05-16 17:44:24 +00:00
Dustin J. Mitchell
b79ac66e13 Bug 1258497: use taskgraph decision in decision task; r=gps
MozReview-Commit-ID: 9STbp4EPMrT

--HG--
extra : rebase_source : e46a41df3b0159fbbae2fc26ded576a3261fc870
extra : histedit_source : 4fe675a1bc33929e809ba3e2fe9373140eb4279d
2016-05-16 22:42:10 +00:00
Dustin J. Mitchell
cea2249ca3 Bug 1258497: Implement a new taskgraph generation system; r=gps
The `taskgraph` package generates TaskCluster task graphs based on collections
of task "kinds".  Initially, there is only one kind, the "legacy" kind, which
reads the YAML files from `testing/taskcluster/tasks` to generate the task
graph.

Try syntax is implemented by filtering the tasks in the taskgraph after it has
been created, then extending the result to include any prerequisite tasks.

A collection of `mach taskgraph` subcommands are provided for developers to
extend or debug the task-graph generation process.

MozReview-Commit-ID: 1TJCns4XxZ8

--HG--
rename : testing/taskcluster/docs/index.rst => taskcluster/docs/index.rst
extra : rebase_source : 7b9125281d66044db9bd8e4a1fade16136f384b9
extra : histedit_source : 47640d27080acda0279270babbcf33f5badb0d1c
2016-05-16 22:53:22 +00:00
Kim Moir
7f85dca70f Bug 1174206 - Migrate Android x86 builds to TaskCluster and make Tier 2 r=gbrown 2016-06-29 11:10:30 -04:00
Steve Fink
595768c9f4 Bug 1280637 - Make sm-{asan,tsan,msan} try-only for now, r=garndt
MozReview-Commit-ID: Dx58G6O3B2Y

--HG--
extra : rebase_source : 37889988fcc06ae2011839925dcb53e1de9f2063
2016-07-08 13:51:11 -07:00
Steve Fink
1fca5e3aec Bug 1280637 - Implement tsan (thread sanitizer), asan (address sanitizer), and msan (uninitialized memory) jobs, r=terrence
MozReview-Commit-ID: JuFVqAb9C4s

--HG--
rename : browser/config/tooltool-manifests/linux64/asan.manifest => browser/config/tooltool-manifests/linux64/msan.manifest
extra : rebase_source : 1a5e77992388b0323bb639ee2fd4285a68b94505
2016-07-15 12:33:40 -07:00