mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-11-23 21:01:08 +00:00
3c131b2865
This links directly to the taskgraph howto section in most cases: https://taskcluster-taskgraph.readthedocs.io/en/latest/howto/index.html Then removes documentation that duplicates that section. Jira: RELENG-709 Differential Revision: https://phabricator.services.mozilla.com/D151937
80 lines
3.3 KiB
ReStructuredText
80 lines
3.3 KiB
ReStructuredText
.. taskcluster_index:
|
|
|
|
Firefox CI and Taskgraph
|
|
========================
|
|
|
|
Firefox's `continuous integration`_ (CI) system is made up of three parts.
|
|
|
|
First there's `Taskcluster`_, a task execution framework developed by Mozilla.
|
|
Taskcluster is capable of building complex, scalable and highly customizable CI
|
|
systems. Taskcluster is more like a set of building blocks that can be used to
|
|
create CI systems, rather than a fully-fledged CI system itself.
|
|
|
|
The second part is the `Firefox CI`_ instance of Taskcluster. This is the
|
|
Taskcluster deployment responsible for running most of Mozilla's CI needs. This
|
|
component is comprised of a Kubernetes cluster to run the Taskcluster services
|
|
(maintained by SRE Services), some customizations to support Taskcluster
|
|
on ``hg.mozilla.org`` and access control in the `ci-configuration`_ repo
|
|
(maintained by Release Engineering).
|
|
|
|
The third part is `Taskgraph`_. Taskgraph is a Python library that can generate
|
|
a `DAG`_ of tasks and submit them to a Taskcluster instance. This is the
|
|
component that most Gecko and Mozilla developers will interact with when
|
|
working with tasks, and will be the focal point of the rest of this
|
|
documentation.
|
|
|
|
.. note::
|
|
|
|
Historically Taskgraph started out inside ``mozilla-central``. It was then
|
|
forked to standalone `Taskgraph`_ in order to support projects on Github.
|
|
Over time maintaining two forks grew cumbersome so they are in the process
|
|
of being merged back together.
|
|
|
|
Today the version of Taskgraph under ``taskcluster/gecko_taskgraph`` depends
|
|
on the standalone version, which is vendored under
|
|
``third_party/python/taskcluster-taskgraph``. There is still a lot of
|
|
duplication between these places, but ``gecko_taskgraph`` is slowly being
|
|
re-written to consume standalone Taskgraph.
|
|
|
|
The ``taskcluster`` directory contains all the files needed to define the graph
|
|
of tasks that must be executed to build and test the Gecko tree. This is more
|
|
complex than you think! There are 30,000+ tasks and counting in Gecko's CI
|
|
graphs.
|
|
|
|
Taskgraph supports:
|
|
|
|
* A huge array of tasks
|
|
* Different behavior for different repositories
|
|
* "Try" pushes, with special means to select a subset of the graph for execution
|
|
* Optimization -- skipping tasks that have already been performed
|
|
* Extremely flexible generation of a variety of tasks using an approach of
|
|
incrementally transforming job descriptions into task definitions.
|
|
|
|
The most comprehensive resource on Taskgraph is `Taskgraph's documentation`_. These docs
|
|
will refer there when appropriate and expand on topics specific to ``gecko_taskgraph``
|
|
where necessary.
|
|
|
|
If you are reading this with a particular goal in mind and would rather avoid
|
|
becoming a task-graph expert, check out the :doc:`how-to section <howto/index>`.
|
|
|
|
.. _continuous integration: https://en.wikipedia.org/wiki/Continuous_integration
|
|
.. _Taskcluster: https://taskcluster.net/
|
|
.. _Firefox CI: https://firefox-ci-tc.services.mozilla.com/
|
|
.. _ci-configuration: https://hg.mozilla.org/ci/ci-configuration/
|
|
.. _Taskgraph: https://github.com/taskcluster/taskgraph
|
|
.. _DAG: https://en.wikipedia.org/wiki/Directed_acyclic_graph
|
|
.. _Taskgraph's documentation: https://taskcluster-taskgraph.readthedocs.io/en/latest/
|
|
|
|
.. toctree::
|
|
|
|
taskgraph
|
|
howto/index
|
|
transforms/index
|
|
optimization/index
|
|
cron
|
|
try
|
|
release-promotion
|
|
versioncontrol
|
|
config
|
|
reference
|