Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
How Tos
|
|
|
|
=======
|
|
|
|
|
|
|
|
All of this equipment is here to help you get your work done more efficiently.
|
|
|
|
However, learning how task-graphs are generated is probably not the work you
|
|
|
|
are interested in doing. This section should help you accomplish some of the
|
|
|
|
more common changes to the task graph with minimal fuss.
|
|
|
|
|
|
|
|
.. important::
|
|
|
|
|
|
|
|
If you cannot accomplish what you need with the information provided here,
|
|
|
|
please consider whether you can achieve your goal in a different way.
|
|
|
|
Perhaps something simpler would cost a bit more in compute time, but save
|
|
|
|
the much more expensive resource of developers' mental bandwidth.
|
|
|
|
Task-graph generation is already complex enough!
|
|
|
|
|
|
|
|
If you want to proceed, you may need to delve into the implementation of
|
|
|
|
task-graph generation. The documentation and code are designed to help, as
|
|
|
|
are the authors - ``hg blame`` may help track down helpful people.
|
|
|
|
|
|
|
|
As you write your new transform or add a new kind, please consider the next
|
|
|
|
developer. Where possible, make your change data-driven and general, so
|
|
|
|
that others can make a much smaller change. Document the semantics of what
|
|
|
|
you are changing clearly, especially if it involves modifying a transform
|
|
|
|
schema. And if you are adding complexity temporarily while making a
|
|
|
|
gradual transition, please open a new bug to remind yourself to remove the
|
|
|
|
complexity when the transition is complete.
|
|
|
|
|
|
|
|
Hacking Task Graphs
|
|
|
|
-------------------
|
|
|
|
|
|
|
|
The recommended process for changing task graphs is this:
|
|
|
|
|
2017-08-23 19:22:10 +00:00
|
|
|
1. Run one of the ``mach taskgraph`` subcommands (see :doc:`mach`) to
|
|
|
|
generate a baseline against which to measure your changes.
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
|
|
|
|
.. code-block:: none
|
|
|
|
|
2017-08-23 19:22:10 +00:00
|
|
|
./mach taskgraph tasks --json > old-tasks.json
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
|
2017-08-23 19:22:10 +00:00
|
|
|
2. Make your modifications under ``taskcluster/``.
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
|
2017-08-23 19:22:10 +00:00
|
|
|
3. Run the same ``mach taskgraph`` command, sending the output to a new file,
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
and use ``diff`` to compare the old and new files. Make sure your changes
|
2017-08-23 19:22:10 +00:00
|
|
|
have the desired effect and no undesirable side-effects. A plain unified
|
|
|
|
diff should be useful for most changes, but in some cases it may be helpful
|
|
|
|
to post-process the JSON to strip distracting changes.
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
|
2017-08-23 19:22:10 +00:00
|
|
|
4. When you are satisfied with the changes, push them to try to ensure that the
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
modified tasks work as expected.
|
|
|
|
|
2017-07-20 17:11:28 +00:00
|
|
|
Hacking Actions
|
|
|
|
...............
|
|
|
|
|
|
|
|
If you are working on an action task and wish to test it out locally, use the
|
|
|
|
``./mach taskgraph test-action-callback`` command:
|
|
|
|
|
|
|
|
.. code-block:: none
|
|
|
|
|
|
|
|
./mach taskgraph test-action-task \
|
|
|
|
--task-id I4gu9KDmSZWu3KHx6ba6tw --task-group-id sMO4ybV9Qb2tmcI1sDHClQ \
|
2017-08-23 19:22:10 +00:00
|
|
|
--input input.yml hello_world_action
|
2017-07-20 17:11:28 +00:00
|
|
|
|
|
|
|
This invocation will run the hello world callback with the given inputs and
|
|
|
|
print any created tasks to stdout, rather than actually creating them.
|
|
|
|
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
Common Changes
|
|
|
|
--------------
|
|
|
|
|
|
|
|
Changing Test Characteristics
|
|
|
|
.............................
|
|
|
|
|
|
|
|
First, find the test description. This will be in
|
|
|
|
``taskcluster/ci/*/tests.yml``, for the appropriate kind (consult
|
|
|
|
:doc:`kinds`). You will find a YAML stanza for each test suite, and each
|
|
|
|
stanza defines the test's characteristics. For example, the ``chunks``
|
|
|
|
property gives the number of chunks to run. This can be specified as a simple
|
|
|
|
integer if all platforms have the same chunk count, or it can be keyed by test
|
|
|
|
platform. For example:
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
chunks:
|
|
|
|
by-test-platform:
|
|
|
|
linux64/debug: 10
|
|
|
|
default: 8
|
|
|
|
|
|
|
|
The full set of available properties is in
|
|
|
|
``taskcluster/taskgraph/transform/tests/test_description.py``. Some other
|
|
|
|
commonly-modified properties are ``max-run-time`` (useful if tests are being
|
|
|
|
killed for exceeding maxRunTime) and ``treeherder-symbol``.
|
|
|
|
|
|
|
|
.. note::
|
|
|
|
|
|
|
|
Android tests are also chunked at the mozharness level, so you will need to
|
|
|
|
modify the relevant mozharness config, as well.
|
|
|
|
|
|
|
|
Adding a Test Suite
|
|
|
|
...................
|
|
|
|
|
|
|
|
To add a new test suite, you will need to know the proper mozharness invocation
|
|
|
|
for that suite, and which kind it fits into (consult :doc:`kinds`).
|
|
|
|
|
|
|
|
Add a new stanza to ``taskcluster/ci/<kind>/tests.yml``, copying from the other
|
|
|
|
stanzas in that file. The meanings should be clear, but authoritative
|
|
|
|
documentation is in
|
|
|
|
``taskcluster/taskgraph/transform/tests/test_description.py`` should you need
|
|
|
|
it. The stanza name is the name by which the test will be referenced in try
|
|
|
|
syntax.
|
|
|
|
|
|
|
|
Add your new test to a test set in ``test-sets.yml`` in the same directory. If
|
|
|
|
the test should only run on a limited set of platforms, you may need to define
|
|
|
|
a new test set and reference that from the appropriate platforms in
|
|
|
|
``test-platforms.yml``. If you do so, include some helpful comments in
|
|
|
|
``test-sets.yml`` for the next person.
|
|
|
|
|
|
|
|
Greening Up a New Test
|
|
|
|
......................
|
|
|
|
|
|
|
|
When a test is not yet reliably green, configuration for that test should not
|
|
|
|
be landed on integration branches. Of course, you can control where the
|
|
|
|
configuration is landed! For many cases, it is easiest to green up a test in
|
|
|
|
try: push the configuration to run the test to try along with your work to fix
|
|
|
|
the remaining test failures.
|
|
|
|
|
|
|
|
When working with a group, check out a "twig" repository to share among your
|
|
|
|
group, and land the test configuration in that repository. Once the test is
|
|
|
|
green, merge to an integration branch and the test will begin running there as
|
|
|
|
well.
|
|
|
|
|
2016-09-09 21:29:27 +00:00
|
|
|
Adding a New Task
|
|
|
|
.................
|
|
|
|
|
|
|
|
If you are adding a new task that is not a test suite, there are a number of
|
|
|
|
options. A few questions to consider:
|
|
|
|
|
|
|
|
* Is this a new build platform or variant that will produce an artifact to
|
|
|
|
be run through the usual test suites?
|
|
|
|
|
|
|
|
* Does this task depend on other tasks? Do other tasks depend on it?
|
|
|
|
|
|
|
|
* Is this one of a few related tasks, or will you need to generate a large
|
|
|
|
set of tasks using some programmatic means (for example, chunking)?
|
|
|
|
|
|
|
|
* How is the task actually excuted? Mozharness? Mach?
|
|
|
|
|
|
|
|
* What kind of environment does the task require?
|
|
|
|
|
|
|
|
Armed with that information, you can choose among a few options for
|
|
|
|
implementing this new task. Try to choose the simplest solution that will
|
|
|
|
satisfy your near-term needs. Since this is all implemented in-tree, it
|
|
|
|
is not difficult to refactor later when you need more generality.
|
|
|
|
|
|
|
|
Existing Kind
|
|
|
|
`````````````
|
|
|
|
|
|
|
|
The simplest option is to add your task to an existing kind. This is most
|
|
|
|
practical when the task "makes sense" as part of that kind -- for example, if
|
|
|
|
your task is building an installer for a new platform using mozharness scripts
|
|
|
|
similar to the existing build tasks, it makes most sense to add your task to
|
|
|
|
the ``build`` kind. If you need some additional functionality in the kind,
|
|
|
|
it's OK to modify the implementation as necessary, as long as the modification
|
|
|
|
is complete and useful to the next developer to come along.
|
|
|
|
|
2017-07-01 20:40:39 +00:00
|
|
|
Tasks in the ``build`` kind generate Firefox installers, and the ``test`` kind
|
|
|
|
will add a full set of Firefox tests for each ``build`` task.
|
|
|
|
|
2016-09-09 21:29:27 +00:00
|
|
|
New Kind
|
|
|
|
````````
|
|
|
|
|
|
|
|
The next option to consider is adding a new kind. A distinct kind gives you
|
|
|
|
some isolation from other task types, which can be nice if you are adding an
|
|
|
|
experimental kind of task.
|
|
|
|
|
2017-07-01 20:40:39 +00:00
|
|
|
Kinds can range in complexity. The simplest sort of kind uses the transform
|
|
|
|
loader to read a list of jobs from the ``jobs`` key, and applies the standard
|
|
|
|
``job`` and ``task`` transforms:
|
2016-09-09 21:29:27 +00:00
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
implementation: taskgraph.task.transform:TransformTask
|
|
|
|
transforms:
|
|
|
|
- taskgraph.transforms.job:transforms
|
|
|
|
- taskgraph.transforms.task:transforms
|
|
|
|
jobs:
|
|
|
|
- ..your job description here..
|
|
|
|
|
2017-07-01 20:40:39 +00:00
|
|
|
Job descriptions are defined and documented in
|
|
|
|
``taskcluster/taskgraph/transforms/job/__init__.py``.
|
|
|
|
|
|
|
|
Custom Kind Loader
|
|
|
|
``````````````````
|
2016-09-09 21:29:27 +00:00
|
|
|
|
|
|
|
If your task depends on other tasks, then the decision of which tasks to create
|
2017-07-01 20:40:39 +00:00
|
|
|
may require some code. For example, the ``test`` kind iterates over
|
|
|
|
the builds in the graph, generating a full set of test tasks for each one. This specific
|
|
|
|
post-build behavior is implemented as a loader defined in ``taskcluster/taskgraph/loader/test.py``.
|
|
|
|
|
|
|
|
A custom loader is useful when the set of tasks you want to create is not
|
|
|
|
static but based on something else (such as the available builds) or when the
|
|
|
|
dependency relationships for your tasks are complex.
|
2016-09-09 21:29:27 +00:00
|
|
|
|
|
|
|
Custom Transforms
|
|
|
|
`````````````````
|
|
|
|
|
2017-07-01 20:40:39 +00:00
|
|
|
Most loaders apply a series of ":doc:`transforms <transforms>`" that start with
|
|
|
|
an initial human-friendly description of a task and end with a task definition
|
|
|
|
suitable for insertion into a Taskcluster queue.
|
|
|
|
|
|
|
|
Custom transforms can be useful to apply defaults, simplifying the YAML files
|
|
|
|
in your kind. They can also apply business logic that is more easily expressed
|
|
|
|
in code than in YAML.
|
|
|
|
|
|
|
|
Transforms need not be one-to-one: a transform can produce zero or more outputs
|
|
|
|
for each input. For example, the test transforms perform chunking by producing
|
|
|
|
an output for each chunk of a given input.
|
|
|
|
|
|
|
|
Ideally those transforms will produce job descriptions, so you can use the
|
|
|
|
existing ``job`` and ``task`` transforms:
|
2016-09-09 21:29:27 +00:00
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
transforms:
|
|
|
|
- taskgraph.transforms.my_stuff:transforms
|
|
|
|
- taskgraph.transforms.job:transforms
|
|
|
|
- taskgraph.transforms.task:transforms
|
|
|
|
|
2017-07-01 20:40:39 +00:00
|
|
|
Try to keep transforms simple, single-purpose and well-documented!
|
2016-09-09 21:29:27 +00:00
|
|
|
|
|
|
|
Custom Run-Using
|
|
|
|
````````````````
|
|
|
|
|
|
|
|
If the way your task is executed is unique (so, not a mach command or
|
|
|
|
mozharness invocation), you can add a new implementation of the job
|
|
|
|
description's "run" section. Before you do this, consider that it might be a
|
|
|
|
better investment to modify your task to support invocation via mozharness or
|
|
|
|
mach, instead. If this is not possible, then adding a new file in
|
|
|
|
``taskcluster/taskgraph/transforms/jobs`` with a structure similar to its peers
|
|
|
|
will make the new run-using option available for job descriptions.
|
|
|
|
|
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
|
|
|
Something Else?
|
|
|
|
...............
|
|
|
|
|
|
|
|
If you make another change not described here that turns out to be simple or
|
|
|
|
common, please include an update to this file in your patch.
|
2017-06-27 20:33:20 +00:00
|
|
|
|
|
|
|
|