Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
Task Kinds
==========
This section lists and documents the available task kinds.
2016-09-12 18:34:06 +00:00
build
2017-06-15 17:47:22 +00:00
-----
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
2016-09-12 18:34:06 +00:00
Builds are tasks that produce an installer or other output that can be run by
users or automated tests. This is more restrictive than most definitions of
"build" in a Mozilla context: it does not include tasks that run build-like
actions for static analysis or to produce instrumented artifacts.
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
2016-11-25 19:52:46 +00:00
build-signing
2017-06-15 17:47:22 +00:00
-------------
Many builds must be signed. The build-signing task takes the unsigned `build`
2017-06-29 20:50:13 +00:00
kind artifacts and passes them through signingscriptworker to a signing server
2017-06-15 17:47:22 +00:00
and returns signed results.
2016-11-25 19:52:46 +00:00
2016-09-07 01:01:52 +00:00
artifact-build
--------------
This kind performs an artifact build: one based on precompiled binaries
discovered via the TaskCluster index. This task verifies that such builds
continue to work correctly.
2016-09-12 18:04:09 +00:00
hazard
------
Hazard builds are similar to "regular' builds, but use a compiler extension to
extract a bunch of data from the build and then analyze that data looking for
hazardous behaviors.
2016-09-09 21:06:51 +00:00
l10n
----
2016-12-13 16:06:48 +00:00
The l10n kind takes the last published nightly build, and generates localized builds
from it. You can read more about how to trigger these on the `wiki
<https://wiki.mozilla.org/ReleaseEngineering/TryServer#Desktop_l10n_jobs_.28on_Taskcluster.29> `_.
2016-09-09 21:06:51 +00:00
Bug 1322041 - Nightly l10n support, and docs. r=dustin
Involved work which landed on the date project branch with:
* No Bug, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/e7af20515b1c,
https://hg.mozilla.org/projects/date/rev/d7f0f1c6d564,
https://hg.mozilla.org/projects/date/rev/db530d7a4672)
* Bug 1171738, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/a906f6997731,
https://hg.mozilla.org/projects/date/rev/ada87315d4c0,
https://hg.mozilla.org/projects/date/rev/fbee52f145e9,
https://hg.mozilla.org/projects/date/rev/2939b68e59fc)
* Bug 1312000, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/79a2f66ff5c0)
* Bug 1312396, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/dc06c58a5663)
* Bug 1312585, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/7cf267b61752)
* Bug 1316527, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/cc2d9365d9b9)
* Bug 1318068, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/7cb11cb76a09)
* Bug 1326093, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/f64a03d94ac7)
MozReview-Commit-ID: Fhom5OcIWru
--HG--
extra : rebase_source : 4e3c56d72ea7b27a7ad930534be26d19f512fa2b
2017-01-06 14:13:41 +00:00
nightly-l10n
------------
The nightly l10n kind repacks a specific nightly build (from the same source code)
in order to provide localized versions of the same source.
Bug 1322041 - Sign nightly l10n jobs. r=dustin
Involved work which landed on the date project branch with:
* Bug 1312000, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/79a2f66ff5c0)
* Bug 1312500, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/6ae07fa4b011)
* Bug 1312585, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/7cf267b61752)
* Bug 1313662, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/92f531d1cd77)
* Bug 1314008, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/bbc7608106c0)
* Bug 1314847, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/4a1231655fbb)
* Bug 1316214, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/edae37481cab,
https://hg.mozilla.org/projects/date/rev/62bd3371e954,
https://hg.mozilla.org/projects/date/rev/2cc558343951)
* Bug 1319189, by Callek@gmail.com (https://hg.mozilla.org/projects/date/rev/4c33f8ccecf5)
MozReview-Commit-ID: 3hadIi757jn
--HG--
extra : rebase_source : 5ff13afd2f6235a0007aeb7f750d30deeb2db8aa
2016-12-13 16:43:47 +00:00
nightly-l10n-signing
--------------------
The nightly l10n signing kind takes artifacts from the nightly-l10n kind and
passes them to signing servers to have their contents signed appropriately, based
on an appropriate signing format. One signing job is created for each nightly-l10n
job (usually chunked).
2017-02-02 21:16:06 +00:00
source-test
2017-06-15 17:47:22 +00:00
-----------
2016-09-12 18:39:05 +00:00
2017-02-02 21:16:06 +00:00
Source-tests are tasks that run directly from the Gecko source. This can include linting,
unit tests, source-code analysis, or measurement work. While source-test tasks run from
a source checkout, it is still possible for them to depend on a build artifact, though
often they do not.
2016-09-12 18:39:05 +00:00
2016-09-07 15:08:28 +00:00
upload-symbols
--------------
Upload-symbols tasks run after builds and upload the symbols files generated by
build tasks to Socorro for later use in crash analysis.
2017-08-09 18:32:05 +00:00
upload-generated-sources
--------------
Upload-generated-sources tasks run after builds and upload source files that were generated as part of the build process to an s3 bucket for later use in links from crash reports or when debugging shipped builds.
2016-09-07 00:48:31 +00:00
valgrind
--------
Valgrind tasks produce builds instrumented by valgrind.
2016-09-08 00:31:35 +00:00
static-analysis
---------------
Static analysis builds use the compiler to perform some detailed analysis of
the source code while building. The useful output from these tasks are their
build logs, and while they produce a binary, they do not upload it as an
artifact.
2016-09-12 18:04:31 +00:00
toolchain
---------
Toolchain builds create the compiler toolchains used to build Firefox. These
will eventually be dependencies of the builds themselves, but for the moment
are run manually via try pushes and the results uploaded to tooltool.
2016-09-12 18:04:24 +00:00
spidermonkey
------------
Spidermonkey tasks check out the full gecko source tree, then compile only the
spidermonkey portion. Each task runs specific tests after the build.
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
Tests
-----
Test tasks for Gecko products are divided into several kinds, but share a
common implementation. The process goes like this, based on a set of YAML
files named in `` kind.yml `` :
* For each build task, determine the related test platforms based on the build
platform. For example, a Windows 2010 build might be tested on Windows 7
2016-12-04 20:38:14 +00:00
and Windows 10. Each test platform specifies "test sets" indicating which
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
tests to run. This is configured in the file named
`` test-platforms.yml `` .
* Each test set is expanded to a list of tests to run. This is configured in
2016-12-04 20:38:14 +00:00
the file named by `` test-sets.yml `` . A platform may specify several test
sets, in which case the union of those sets is used.
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
* Each named test is looked up in the file named by `` tests.yml `` to find a
test description. This test description indicates what the test does, how
it is reported to treeherder, and how to perform the test, all in a
platform-independent fashion.
* Each test description is converted into one or more tasks. This is
performed by a sequence of transforms defined in the `` transforms `` key in
`` kind.yml `` . See :doc: `transforms` : for more information on these
transforms.
* The resulting tasks become a part of the task graph.
.. important ::
This process generates *all* test jobs, regardless of tree or try syntax.
It is up to a later stage of the task-graph generation (the target set) to
select the tests that will actually be performed.
2016-12-23 22:12:17 +00:00
test
....
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
2016-12-23 22:12:17 +00:00
The `` desktop-test `` kind defines tests for builds. Its `` tests.yml `` defines
the full suite of desktop tests and their particulars, leaving it to the
transforms to determine how those particulars apply to the various platforms.
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
2016-12-23 22:12:17 +00:00
This kind includes both unit tests and talos.
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
docker-image
------------
Tasks of the `` docker-image `` kind build the Docker images in which other
Docker tasks run.
The tasks to generate each docker image have predictable labels:
`` build-docker-image-<name> `` .
2016-10-20 12:55:34 +00:00
Docker images are built from subdirectories of `` taskcluster/docker `` , using
Bug 1281004: Specify test tasks more flexibly; r=gps; r=gbrown
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
2016-07-11 23:27:14 +00:00
`` docker build `` . There is currently no capability for one Docker image to
depend on another in-tree docker image, without uploading the latter to a
Docker repository
The task definition used to create the image-building tasks is given in
`` image.yml `` in the kind directory, and is interpreted as a :doc:`YAML
Template <yaml-templates>`.
2016-11-25 19:52:46 +00:00
android-stuff
2017-06-15 17:47:22 +00:00
-------------
2017-01-12 22:45:36 +00:00
2017-01-13 02:01:54 +00:00
balrog
------
Balrog is the Mozilla Update Server. Jobs of this kind are submitting information
which assists in telling Firefox that an update is available for the related job.
balrog-l10n
-----------
Balrog is the Mozilla Update Server. Jobs of this kind are submitting information
which assists in telling Firefox that an update is available for the localized
job involved.
2017-01-12 22:45:36 +00:00
beetmover
---------
Beetmover, takes specific artifacts, "Beets", and pushes them to a location outside
of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the
process determines the final location and a "pretty" name (versioned product name)
beetmover-l10n
--------------
Beetmover L10n, takes specific artifacts, "Beets", and pushes them to a location outside
of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the
process determines the final location and a "pretty" name (versioned product name)
This separate kind uses logic specific to localized artifacts, such as including
the language in the final artifact names.
2017-02-08 03:44:12 +00:00
2017-04-19 19:49:55 +00:00
beetmover-repackage
2017-06-15 17:47:22 +00:00
-------------------
2017-04-19 19:49:55 +00:00
Beetmover-repackage is beetmover but for tasks that need an intermediate step
between signing and packaging, such as OSX. For more details see the definitions
of the Beetmover kind above and the repackage kind below.
2017-02-08 03:44:12 +00:00
checksums-signing
2017-06-15 17:47:22 +00:00
-----------------
2017-02-08 03:44:12 +00:00
Checksums-signing take as input the checksums file generated by beetmover tasks
and sign it via the signing scriptworkers. Returns the same file signed and
additional detached signature.
beetmover-checksums
-------------------
Beetmover, takes specific artifact checksums and pushes it to a location outside
of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the
process determines the final location and "pretty" names it (version product name)
2017-04-04 09:21:07 +00:00
push-apk-breakpoint
-------------------
Decides whether or not APKs should be published onto Google Play Store. Jobs of this
kind depend on all the signed multi-locales (aka "multi") APKs for a given release,
in order to make the decision.
push-apk
--------
PushApk publishes Android packages onto Google Play Store. Jobs of this kind take
all the signed multi-locales (aka "multi") APKs for a given release and upload them
all at once. They also depend on the breakpoint.
2017-04-19 19:18:44 +00:00
repackage
---------
Repackage tasks take a signed output and package them up into something suitable
for shipping to our users. For example, on OSX we return a tarball as the signed output
and this task would package that up as an Apple Disk Image (.dmg)
repackage-l10n
--------------
Repackage-L10n is a `` ` Repackage ` `` task split up to be suitable for use after l10n repacks.
2017-06-27 21:55:09 +00:00
repackage-signing
-----------------
Repackage-signing take the repackaged installers (windows) and update packaging (with
the signed internal bits) and signs them.