mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-10-26 11:45:37 +00:00
ffca84ff7d
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
--HG--
rename : taskcluster/taskgraph/kind/base.py => taskcluster/taskgraph/task/base.py
rename : taskcluster/taskgraph/kind/docker_image.py => taskcluster/taskgraph/task/docker_image.py
rename : taskcluster/taskgraph/kind/legacy.py => taskcluster/taskgraph/task/legacy.py
extra : rebase_source : 03e70902c2d3a297eb9e3ce852f8737c2550d5a6
extra : histedit_source : d4d9f4b192605af21f41d83495fc3c923759c3cb
84 lines
2.8 KiB
ReStructuredText
84 lines
2.8 KiB
ReStructuredText
Task Kinds
|
|
==========
|
|
|
|
This section lists and documents the available task kinds.
|
|
|
|
Builds
|
|
------
|
|
|
|
Builds are currently implemented by the ``legacy`` kind.
|
|
|
|
Tests
|
|
-----
|
|
|
|
Test tasks for Gecko products are divided into several kinds, but share a
|
|
common implementation. The process goes like this, based on a set of YAML
|
|
files named in ``kind.yml``:
|
|
|
|
* For each build task, determine the related test platforms based on the build
|
|
platform. For example, a Windows 2010 build might be tested on Windows 7
|
|
and Windows 10. Each test platform specifies a "test set" indicating which
|
|
tests to run. This is configured in the file named
|
|
``test-platforms.yml``.
|
|
|
|
* Each test set is expanded to a list of tests to run. This is configured in
|
|
the file named by ``test-sets.yml``.
|
|
|
|
* Each named test is looked up in the file named by ``tests.yml`` to find a
|
|
test description. This test description indicates what the test does, how
|
|
it is reported to treeherder, and how to perform the test, all in a
|
|
platform-independent fashion.
|
|
|
|
* Each test description is converted into one or more tasks. This is
|
|
performed by a sequence of transforms defined in the ``transforms`` key in
|
|
``kind.yml``. See :doc:`transforms`: for more information on these
|
|
transforms.
|
|
|
|
* The resulting tasks become a part of the task graph.
|
|
|
|
.. important::
|
|
|
|
This process generates *all* test jobs, regardless of tree or try syntax.
|
|
It is up to a later stage of the task-graph generation (the target set) to
|
|
select the tests that will actually be performed.
|
|
|
|
desktop-test
|
|
............
|
|
|
|
The ``desktop-test`` kind defines tests for Desktop builds. Its ``tests.yml``
|
|
defines the full suite of desktop tests and their particulars, leaving it to
|
|
the transforms to determine how those particulars apply to Linux, OS X, and
|
|
Windows.
|
|
|
|
android-test
|
|
............
|
|
|
|
The ``android-test`` kind defines tests for Android builds.
|
|
|
|
It is very similar to ``desktop-test``, but the details of running the tests
|
|
differ substantially, so they are defined separately.
|
|
|
|
legacy
|
|
------
|
|
|
|
The legacy kind is the old, templated-yaml-based task definition mechanism. It
|
|
is still used for builds and generic tasks, but not for long!
|
|
|
|
docker-image
|
|
------------
|
|
|
|
Tasks of the ``docker-image`` kind build the Docker images in which other
|
|
Docker tasks run.
|
|
|
|
The tasks to generate each docker image have predictable labels:
|
|
``build-docker-image-<name>``.
|
|
|
|
Docker images are built from subdirectories of ``testing/docker``, using
|
|
``docker build``. There is currently no capability for one Docker image to
|
|
depend on another in-tree docker image, without uploading the latter to a
|
|
Docker repository
|
|
|
|
The task definition used to create the image-building tasks is given in
|
|
``image.yml`` in the kind directory, and is interpreted as a :doc:`YAML
|
|
Template <yaml-templates>`.
|