mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-11-01 06:35:42 +00:00
b547239226
ff5a4bab0813 (bug 1311791) and 332a08725ed0 (bug 1292071) changed behavior of the VCS caches. First, the store cache / path was merged into the checkouts cache. Then the path of the Mercurial shared store was moved within the cache to always be rooted at the cache root. Caches are shared across tasks. Tasks can execute on any revision configured to use a cache. So, when interacting with caches, it is important to consider how every revision configured to use that cache will interact with it. Take this scenario for example. A worker executes a task where the hg shared store is rooted at /home/worker/checkouts/src/hg-shared. Then the worker executes a task where the hg shared store is rooted at /home/worker/checkouts/hg-store. `hg robustcheckout` will see the checkout from the first task. But then it sees that the store it is pointing to is at an unexpected location (checkouts/src/hg-shared instead of checkouts/hg-store). `hg robustcheckout` aggressively normalizes state to ensure consistency. So when it sees this mismatch, it blows away the checkout and creates one from checkouts/hg-store to replace it. That's a lot of overhead. And this cycle can repeat itself if the right combination of revisions run on the worker! A solution to this problem is to create a clean break from caches when cache semantics change. In TaskCluster, that means using a different cache. This commit introduces a "version" component to the checkouts cache name. By doing so, we create a clean break from all previous caches, ensuring all revisions this point forward won't encounter an hg shared store at an unexpected location. This also paves the road for easily making additional clean breaks in the future. MozReview-Commit-ID: JT8yuULKpch --HG-- extra : rebase_source : 2e5d321e4314ff7543423d3c7837b770b7c8a3a3
44 lines
1.4 KiB
ReStructuredText
44 lines
1.4 KiB
ReStructuredText
.. taskcluster_caches:
|
|
|
|
=============
|
|
Common Caches
|
|
=============
|
|
|
|
There are various caches used by the in-tree tasks. This page attempts to
|
|
document them and their appropriate use.
|
|
|
|
Version Control Caches
|
|
======================
|
|
|
|
``level-{{level}}-checkouts-{{version}}``
|
|
This cache holds version control checkouts, each in a subdirectory named
|
|
after the repo (e.g., ``gecko``).
|
|
|
|
Checkouts should be read-only. If a task needs to create new files from
|
|
content of a checkout, this content should be written in a separate
|
|
directory/cache (like a workspace).
|
|
|
|
A ``version`` parameter appears in the cache name to allow
|
|
backwards-incompatible changes to the cache's behavior.
|
|
|
|
``level-{{level}}-{{project}}-tc-vcs`` (deprecated)
|
|
This cache is used internally by ``tc-vcs``. This tool is deprecated and
|
|
should be replaced with ``hg robustcheckout``.
|
|
|
|
Workspace Caches
|
|
================
|
|
|
|
``level-{{level}}-*-workspace``
|
|
These caches (of various names typically ending with ``workspace``)
|
|
contain state to be shared between task invocations. Use cases are
|
|
dependent on the task.
|
|
|
|
Other
|
|
=====
|
|
|
|
``tooltool-cache``
|
|
Tooltool invocations should use this cache. Tooltool will store files here
|
|
indexed by their hash, and will verify hashes before copying files from
|
|
this directory, so there is no concern with sharing the cache between jobs
|
|
of different levels.
|