mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-12-02 18:08:58 +00:00
78 lines
3.4 KiB
ReStructuredText
78 lines
3.4 KiB
ReStructuredText
Optimization Process
|
|
====================
|
|
|
|
Optimization proceeds in three phases: removing tasks, replacing tasks,
|
|
and finally generating a subgraph containing only the remaining tasks.
|
|
|
|
Assume the following task graph as context for these examples::
|
|
|
|
TC1 <--\ ,- UP1
|
|
, B1 <--- T1a
|
|
I1 <-| `- T1b
|
|
` B2 <--- T2a
|
|
TC2 <--/ |- T2b
|
|
`- UP2
|
|
|
|
Removing Tasks
|
|
--------------
|
|
|
|
This phase begins with tasks on which nothing depends and follows the
|
|
dependency graph backward from there -- right to left in the diagram above. If
|
|
a task is not removed, then nothing it depends on will be removed either.
|
|
Thus if T1a and T1b are both removed, B1 may be removed as well. But if T2b is
|
|
not removed, then B2 may not be removed either.
|
|
|
|
For each task with no remaining dependencies, the decision whether to remove is
|
|
made by calling the optimization strategy's ``should_remove_task`` method. If
|
|
this method returns True, the task is removed.
|
|
|
|
The optimization process takes a ``do_not_optimize`` argument containing a list
|
|
of tasks that cannot be removed under any circumstances. This is used to
|
|
"force" running specific tasks.
|
|
|
|
Replacing Tasks
|
|
---------------
|
|
|
|
This phase begins with tasks having no dependencies and follows the reversed
|
|
dependency graph from there -- left to right in the diagram above. If a task is
|
|
not replaced, then anything depending on that task cannot be replaced.
|
|
Replacement is generally done on the basis of some hash of the inputs to the
|
|
task. In the diagram above, if both TC1 and I1 are replaced with existing
|
|
tasks, then B1 is a candidate for replacement. But if TC2 has no replacement,
|
|
then replacement of B2 will not be considered.
|
|
|
|
It is possible to replace a task with nothing. This is similar to optimzing
|
|
away, but is useful for utility tasks like UP1. If such a task is considered
|
|
for replacement, then all of its dependencies (here, B1) have already been
|
|
replaced and there is no utility in running the task and no need for a
|
|
replacement task. It is an error for a task on which others depend to be
|
|
replaced with nothing.
|
|
|
|
The ``do_not_optimize`` set applies to task replacement, as does an additional
|
|
``existing_tasks`` dictionary which allows the caller to supply as set of
|
|
known, pre-existing tasks. This is used for action tasks, for example, where it
|
|
contains the entire task-graph generated by the original decision task.
|
|
|
|
Subgraph Generation
|
|
-------------------
|
|
|
|
The first two phases annotate each task in the existing taskgraph with their
|
|
fate: removed, replaced, or retained. The tasks that are replaced also have a
|
|
replacement taskId.
|
|
|
|
The last phase constructs a subgraph containing the retained tasks, and
|
|
simultaneously rewrites all dependencies to refer to taskIds instead of labels.
|
|
To do so, it assigns a taskId to each retained task and uses the replacement
|
|
taskId for all replaced tasks.
|
|
|
|
The `soft-dependencies` are then solved for each task, by adding all the
|
|
remaining tasks in the subgraph from that list to its `dependencies`.
|
|
|
|
The result is an optimized taskgraph with tasks named by taskId instead of
|
|
label. At this phase, the edges in the task graph diverge from the
|
|
``task.dependencies`` attributes, as the latter may contain dependencies
|
|
outside of the taskgraph (for replacement tasks).
|
|
|
|
As a side-effect, this phase also expands all ``{"task-reference": ".."}`` and
|
|
``{"artifact-reference": ".."}`` objects within the task definitions.
|