It's often useful to nest 'by-*' statements in the CI configs. However, calling
'resolve_keyed_by' will keep drilling down and resolving these statements all
at once. Normally this is fine, but sometimes we may want to use, e.g 'by-foo'
when 'foo' hasn't been resolved yet (i.e, it is being set by a later transform
or something like that).
Currently, this means that we'll hit the 'default' case as 'foo' doesn't exist.
This revision adds the ability to 'defer' evaluation of 'foo' until a later
time.
While running into this scenario might be a code smell, re-organizing
transforms to untangle these kinds of ordering dependencies will be a major
undertaking. Plus, there are legitimate reasons we might want one kind of
ordering with certain tasks, then another ordering for other tasks.
Differential Revision: https://phabricator.services.mozilla.com/D88191
We define a value `run.workdir` to point to the defaut directory to use for
all sorts of paths, including checkouts, in the job transform. However, that
directory only makes sense for docker-worker tasks, and relative paths should
be used elsewhere. Adjust the paths on non-docker-worker linux tasks to match
macOS paths.
Differential Revision: https://phabricator.services.mozilla.com/D86668
This is needed as future revisions will refactor the optimization process to be
two pass. So we can't be sure if a task is optimized or not until the very end.
Depends on D87771
Differential Revision: https://phabricator.services.mozilla.com/D87772
I got tripped up by the docs for Graph.visit_preorder/visit_postorder which
talks about nodes linking to and from other nodes, but makes no mention of
which direction the linking happens practically (in my mind I thought "linking
to" meant going from task -> dependent).
This blurb would have helped me catch my mistake.
Differential Revision: https://phabricator.services.mozilla.com/D87771
taskgraph-gen.py dies on try.yml currently.
I'm not sure what the underlying issue is, but this gets things working again.
Differential Revision: https://phabricator.services.mozilla.com/D87691
We define a value `run.workdir` to point to the defaut directory to use for
all sorts of paths, including checkouts, in the job transform. However, that
directory only makes sense for docker-worker tasks, and relative paths should
be used elsewhere. Adjust the paths on non-docker-worker linux tasks to match
macOS paths.
Differential Revision: https://phabricator.services.mozilla.com/D86668
We define a value `run.workdir` to point to the defaut directory to use for
all sorts of paths, including checkouts, in the job transform. However, that
directory only makes sense for docker-worker tasks, and relative paths should
be used elsewhere. Adjust the paths on non-docker-worker linux tasks to match
macOS paths.
Differential Revision: https://phabricator.services.mozilla.com/D86668
This resolves a long-standing issue in development where `mach artifact` (and therefore `mach bootstrap`) would fail unpredictably if you had dirty, but ignored, files in your checkout. Resolving this problem often required unwieldy `hg purge`/`git ignore` incantations that are easy to get wrong.
This patch addresses the problem by doing what we "should" have been doing all along, and consulting the VCS to list tracked files rather than listing EVERY file on disk and applying heuristics to determine whether they should be included in the hash.
Differential Revision: https://phabricator.services.mozilla.com/D86780
I've left the monitor disabled for now, so that we can have a smaller pushes for enabling and disabling it if needed. It should allow more fine grained control.
We may also want to include extracting the monitor tool from a github version instead, and also removing the assumption and it being forked from the parent, so that it's instead given a process ID to treat as the parent it should watch.
Differential Revision: https://phabricator.services.mozilla.com/D84374
In bug 1626058, I changed how the docker image digest was generated:
- I used the same directory structure to generate the digest as was used for generating the context
- I moved context generation to the decision task, and used the hash of that as part of the digest.
Unfortunately, it turns out the file name in the gzip header of the context
.tar.gz differed between when we are creating a context to write out, and when
were just generating the hash.
This adjust the name used in the gzip header to be consistent.
Differential Revision: https://phabricator.services.mozilla.com/D84753
For all fission web-platform tests currently running on mozilla-central as tier 2,
promote to tier 1 and run on all trunk trees.
Differential Revision: https://phabricator.services.mozilla.com/D85512
This patch fixes an issue where the metric settings were not being used because they don't use the test name. It also handles some changes (from a bad copy-paste) that didn't make it into the last live-site patch series.
Differential Revision: https://phabricator.services.mozilla.com/D85609
