mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-11-28 07:13:20 +00:00
99 lines
3.7 KiB
ReStructuredText
99 lines
3.7 KiB
ReStructuredText
.. taskcluster_caches:
|
|
|
|
Caches
|
|
======
|
|
|
|
There are various caches used by the in-tree tasks. This page attempts to
|
|
document them and their appropriate use.
|
|
|
|
Caches are essentially isolated filesystems that are persisted between
|
|
tasks. For example, if 2 tasks run on a worker - one after the other -
|
|
and both tasks request the same cache, the subsequent task will be
|
|
able to see files in the cache that were created by the first task.
|
|
It's also worth noting that TaskCluster workers ensure a cache can only
|
|
be used by 1 task at a time. If a worker is simultaneously running
|
|
multiple tasks requesting the same named cache, the worker will
|
|
have multiple caches of the same name on the worker.
|
|
|
|
Caches and ``run-task``
|
|
-----------------------
|
|
|
|
``run-task`` is our generic task wrapper script. It does common activities
|
|
like ensure a version control checkout is present.
|
|
|
|
One of the roles of ``run-task`` is to verify and sanitize caches.
|
|
It does this by storing state in a cache on its first use. If the recorded
|
|
*capabilities* of an existing cache don't match expectations for the
|
|
current task, ``run-task`` bails. This ensures that caches are only
|
|
reused by tasks with similar execution *profiles*. This prevents
|
|
accidental cache use across incompatible tasks. It also allows run-task
|
|
to make assumptions about the state of caches, which can help avoid
|
|
costly operations.
|
|
|
|
In addition, the hash of ``run-task`` is used to derive the cache name.
|
|
So any time ``run-task`` changes, a new set of caches are used. This
|
|
ensures that any backwards incompatible changes or bug fixes to
|
|
``run-task`` result in fresh caches.
|
|
|
|
Some caches are reserved for use with run-task. That property will be denoted
|
|
below.
|
|
|
|
Common Caches
|
|
-------------
|
|
|
|
Version Control Caches
|
|
::::::::::::::::::::::
|
|
|
|
``level-{{level}}-checkouts-{{hash}}``
|
|
This cache holds version control checkouts, each in a subdirectory named
|
|
after the repo (e.g., ``gecko``).
|
|
|
|
Checkouts should be read-only. If a task needs to create new files from
|
|
content of a checkout, this content should be written in a separate
|
|
directory/cache (like a workspace).
|
|
|
|
This cache name pattern is managed by ``run-task`` and must only be
|
|
used with ``run-task``.
|
|
|
|
``level-{{level}}-checkouts-sparse-{{hash}}``
|
|
This is like the above but is used when the checkout is sparse (contains
|
|
a subset of files).
|
|
|
|
``level-{{level}}-checkouts-{{version}}`` (deprecated)
|
|
This cache holds version control checkouts, each in a subdirectory named
|
|
after the repo (e.g., ``gecko``).
|
|
|
|
Checkouts should be read-only. If a task needs to create new files from
|
|
content of a checkout, this content should be written in a separate
|
|
directory/cache (like a workspace).
|
|
|
|
A ``version`` parameter appears in the cache name to allow
|
|
backwards-incompatible changes to the cache's behavior.
|
|
|
|
The ``hg-store`` contains a `shared store <https://www.mercurial-scm.org/wiki/ShareExtension>`
|
|
that is is used by ``hg robustcheckout``. If you are using ``run-task`` you
|
|
should set the ``HG_STORE_PATH`` environment variable to point to this
|
|
directory. If you are using ``hg robustcheckout``, pass this directory to the
|
|
``--sharebase`` option.
|
|
|
|
Workspace Caches
|
|
::::::::::::::::
|
|
|
|
``level-{{level}}-*-workspace``
|
|
These caches (of various names typically ending with ``workspace``)
|
|
contain state to be shared between task invocations. Use cases are
|
|
dependent on the task.
|
|
|
|
Other
|
|
:::::
|
|
|
|
``level-{{level}}-tooltool-cache-{{hash}}
|
|
Tooltool invocations should use this cache. Tooltool will store files here
|
|
indexed by their hash.
|
|
|
|
This cache name pattern is reserved for use with ``run-task`` and must only
|
|
be used by ``run-task``
|
|
|
|
``tooltool-cache`` (deprecated)
|
|
Legacy location for tooltool files. Use the per-level one instead.
|