The new debian9 build image appears to fix the errors connecting to the
X server when running Firefox to generate profile data for PGO builds
(bug 1516114). By updating the image here and leaving 3-tier PGO
disabled, we can see if the generate-profile tasks intermittents go away
while 1-tier intermittents persist to know for sure that this fixes the
problem.
Differential Revision: https://phabricator.services.mozilla.com/D16321
--HG--
extra : moz-landing-system : lando
This produces the same executables (produced for the same platforms) as
those currently pulled from tooltool (modulo timestamps, maybe changes
since last manifest change, etc.). Unfortunately, as of currently, the
Windows variant needs to be cross-built with mingw because it doesn't
compile without some POSIX APIs that MSVC/Windows SDK don't provide.
One thing that is left out of this change is whether to be completely
accurate with the toolchain cache hash (requiring a large list of files
as resources, and making those built very frequently), whether we'd
rely on manual updates, or if we should go with completely uncached
tasks. This can be left for a followup, the tasks not being hooked up
to be actually used by other tasks yet.
Differential Revision: https://phabricator.services.mozilla.com/D16302
--HG--
extra : moz-landing-system : lando
With 3-tier PGO enabled, we seem to hit bug 1516114 more frequently than
we do with the standard all-in-one PGO. It probably makes sense for us
to wait until that bug is fixed before fully switching it over,
otherwise we will have to mess with retriggering parts of the taskgraph.
The bulk of the 3-tier PGO work has been to support macOS and Android,
so we can revisit Linux after those are completed.
Differential Revision: https://phabricator.services.mozilla.com/D15751
--HG--
extra : moz-landing-system : lando
Two new kinds are introduced - 'instrumented-build' and
'generate-profile'. The instrumented-build kind is almost identical to
the build kind, except it can be used earlier in the task graph. The
3-tier PGO process becomes:
instrumented-build -> generate-profile -> build
The final build stage is identical to any other build, except it has
the 'use-pgo' flag set to True in its task definition. This flag causes
the transforms to add the instrumented-build and generate-profile tasks
to the taskgraph.
Differential Revision: https://phabricator.services.mozilla.com/D15750
--HG--
extra : moz-landing-system : lando
This will be used by |mach try chooser| to help pull 'android-stuff' tasks
(which aren't really builds) apart from the main build tasks.
Differential Revision: https://phabricator.services.mozilla.com/D14902
--HG--
extra : moz-landing-system : lando
The staging balrog server got reset to match production, so update the rules to match.
Differential Revision: https://phabricator.services.mozilla.com/D15764
--HG--
extra : moz-landing-system : lando
There's a guard to prevent action, but it's confusing to ask for it
and then not take the action. Let's not ask. I'm leaving the guard,
pointless as it should be, just in case.
Differential Revision: https://phabricator.services.mozilla.com/D14828
--HG--
extra : moz-landing-system : lando
This changes a bunch of symbols, notably:
* Group name BMR-L10n(*) ==> BMR(*)
* Group name BM-L10n(*) ==> BM(*)
* Groups the BMR symbol itself for en-US Nightly [so BMR ==> BMR(N)]
* Groups the BM symbol itself for en-US Nightly [so BM ==> BM(N)]
* Changes Source Signing fron symbol Bs to Srcs
* Groups Source signing beetmoving as BM(Srcs) instead of symbol BM-S
* Balrog complete update submission changes from c-Up(N) to c-Up(Ns) [Fennec Only]
* Beetmover en-US changes from symbol BM-S to BM(Ns) [used in Fennec]
* Beetmover Checksums (en-US) changes from BMcs(N) => BMcs(Ns) [fennec]
* Checksums signing (en-US, fennec) changes from cs(N) => cs(Ns)
* [Asan] Balrog changed [c-UP(N) ==> c-UP(BoR)]
* [Asan] signing changed from Ns to BoRs
* [Asan] checksums signing changed from cs(N) to cs(BoR)
* [Asan] Change generated source uploading from Ugs ==> UgsBoR
Differential Revision: https://phabricator.services.mozilla.com/D4178
--HG--
extra : moz-landing-system : lando
The nightly partial generation code generates URLs with https:// so they should
be accepted as well.
Differential Revision: https://phabricator.services.mozilla.com/D15311
--HG--
extra : moz-landing-system : lando
We create a minimal wrapper function to call collect_vcs_options()
and vcs_checkout().
We could consolidate this logic into vcs_checkout(). But I don't have
strong feelings about doing that.
Differential Revision: https://phabricator.services.mozilla.com/D13821
--HG--
extra : moz-landing-system : lando
One step closer to making all state gathering and normalization in one
place.
Differential Revision: https://phabricator.services.mozilla.com/D13819
--HG--
extra : moz-landing-system : lando
This makes behavior consistent across all VCS checkouts. I'm still not
a fan of using environment variables here. But at least this gets us
1 step closer to being able to plug alternate logic in without having
to update use of environment variables outside a single function.
Differential Revision: https://phabricator.services.mozilla.com/D13816
--HG--
extra : moz-landing-system : lando
We currently manage VCS checkout arguments as one-offs for each
project. This isn't scalable and results in a bit of copy-pasta.
Let's make the VCS checkout arguments generic so we can have the
same control for all repositories.
This commit focuses on consolidating the existing argument parser
code. It stops short of further unification, which will be done in
subsequent commits.
Differential Revision: https://phabricator.services.mozilla.com/D13815
--HG--
extra : moz-landing-system : lando
This code was used by mozharness jobs to check out the tools repo.
However, those jobs aren't actually using the repository.
Differential Revision: https://phabricator.services.mozilla.com/D13855
--HG--
extra : moz-landing-system : lando
We have multiple source checkouts. --sparse-profile is ambiguous
as to which one it could refer to. Let's rename the argument so it
is prefixed with the repo/project we are checking out.
Differential Revision: https://phabricator.services.mozilla.com/D13814
--HG--
extra : moz-landing-system : lando
We now have multiple things we may check out. "vcs" meaning "gecko"
is not obvious. Let's change the terminology to be more specific.
Differential Revision: https://phabricator.services.mozilla.com/D13813
--HG--
extra : moz-landing-system : lando
We create a minimal wrapper function to call collect_vcs_options()
and vcs_checkout().
We could consolidate this logic into vcs_checkout(). But I don't have
strong feelings about doing that.
Differential Revision: https://phabricator.services.mozilla.com/D13821
--HG--
extra : moz-landing-system : lando
One step closer to making all state gathering and normalization in one
place.
Differential Revision: https://phabricator.services.mozilla.com/D13819
--HG--
extra : moz-landing-system : lando
This makes behavior consistent across all VCS checkouts. I'm still not
a fan of using environment variables here. But at least this gets us
1 step closer to being able to plug alternate logic in without having
to update use of environment variables outside a single function.
Differential Revision: https://phabricator.services.mozilla.com/D13816
--HG--
extra : moz-landing-system : lando
We currently manage VCS checkout arguments as one-offs for each
project. This isn't scalable and results in a bit of copy-pasta.
Let's make the VCS checkout arguments generic so we can have the
same control for all repositories.
This commit focuses on consolidating the existing argument parser
code. It stops short of further unification, which will be done in
subsequent commits.
Differential Revision: https://phabricator.services.mozilla.com/D13815
--HG--
extra : moz-landing-system : lando
This code was used by mozharness jobs to check out the tools repo.
However, those jobs aren't actually using the repository.
Differential Revision: https://phabricator.services.mozilla.com/D13855
--HG--
extra : moz-landing-system : lando
We have multiple source checkouts. --sparse-profile is ambiguous
as to which one it could refer to. Let's rename the argument so it
is prefixed with the repo/project we are checking out.
Differential Revision: https://phabricator.services.mozilla.com/D13814
--HG--
extra : moz-landing-system : lando
We now have multiple things we may check out. "vcs" meaning "gecko"
is not obvious. Let's change the terminology to be more specific.
Differential Revision: https://phabricator.services.mozilla.com/D13813
--HG--
extra : moz-landing-system : lando
The only branch where it set differently is `try`/`try-comm-central`, but it gets
overriden there in `set_try_config`.
Differential Revision: https://phabricator.services.mozilla.com/D15478
--HG--
extra : moz-landing-system : lando
This provides an easy way to encode an artifact URL in static data such as
taskcluster/ci/nightly-l10n/kind.yml, without knowing in advance the format of
the URL.
--HG--
extra : rebase_source : 5b5229d96aad2916b1b3c8e72045c1d461fc1c02
extra : source : 9c35dd209c6b407bc3a45ce7b4c27272ef1bb486
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015). But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere in production, and
TASKCLUSTER_PROXY_URL is set wherever the proxy is active.
The taskgraph Taskcluster utils module gets a `get_root_url()` that gets the
root URL for the current run, either from an environment variable in production
or, on the command line, defaulting to https://taskcluster.net for user
convenience. When the production instance's URL changes, we can simply change
that default.
Other changes to use this function are reserved for later commits.
This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.
--HG--
extra : rebase_source : 4f50e9d066da62a1887baabd8603844c85a32ee6
extra : source : 5ea6f03f845e49d503f5d0283557f54561c41654
This allows the target tasks to include cached tasks, which by default, get
optimized out of the graph on on-`try` branches.
Differential Revision: https://phabricator.services.mozilla.com/D15278
--HG--
extra : moz-landing-system : lando
Supports partner builds on esr60 by re-arranging how we specify the manifest repositories, which are different for esr60 compared to beta+release. The defaults for enabling partners and EME-free now depend on the partner-urls definition in taskcluster/ci/config.yml.
This patch also adds support for partner & EMEfree repacks in try staging releases, for all the various release branches. It makes release_level available early in the decision task (for the release promotion action) where a Parameters object is not available. Signing worker type is no longer hard-coded, and we use level-1 secrets for querying Github (via bug 1513375).
Differential Revision: https://phabricator.services.mozilla.com/D15185
--HG--
extra : moz-landing-system : lando
Sets MOZ_SOURCE_CHANGESET to be the comm-* changeset when building
Thunderbird on Windows. This mirrors a change at line 196 that sets the
same when building for Linux and OSX.
Differential Revision: https://phabricator.services.mozilla.com/D15282
--HG--
extra : moz-landing-system : lando
The original code include several extraneous newlines. Use a template instead,
to get a stable output format.
Differential Revision: https://phabricator.services.mozilla.com/D15309
--HG--
extra : moz-landing-system : lando
Now that the upload-* kinds inherit `run-on-projects`, we don't need to special
case handling of them in target-tasks.
Differential Revision: https://phabricator.services.mozilla.com/D15153
--HG--
extra : moz-landing-system : lando
The optimization doesn't actually do what it claims. For the optimization to be
considered, a task needs to be in the target graph, but then all the
dependencies will be too, which inhibits this optimization.
Differential Revision: https://phabricator.services.mozilla.com/D15151
--HG--
extra : moz-landing-system : lando
This patch adds a cron task to regularly schedule an update to the custom d8/v8 version in use in jsshell benchmarks.
Differential Revision: https://phabricator.services.mozilla.com/D15256
--HG--
extra : moz-landing-system : lando
Rap-P treeherder group for Raptor power tests.
Emit separate PERFHERDER_DATA for power in addition to the performance measurements.
Use magic --host HOST_IP value to have framework load host ip from environment
variable HOST_IP.
