The original globs were only finding C++ files in the root of the repository.
Differential Revision: https://phabricator.services.mozilla.com/D25046
--HG--
extra : moz-landing-system : lando
Run gv-docs as tier 2 rather than tier 3, to make the task visible by default
and get the benefit of at least some sheriffing.
Differential Revision: https://phabricator.services.mozilla.com/D24956
--HG--
extra : moz-landing-system : lando
Makes most kinds that reference nightly attribute reference the shippable attribute.
Also makes most transforms that use nightly use shippable
Transfers dependencies/ownership for some things to shippable from nightly where it was harder to support both.
In no particular order, full list:
Send shippable attribute down to dep tasks.
Set tests as shippable attribute
Add new signing routes
Add shippable routes to repackage_routes transform
Adjust target tasks
Add shippable nightly-l10n
Add nightly-signing and as a side affect add repackage and repackage-signing
Add support for proper balrog platforms for shippable
Add shippable to the nightly sccache guard
Fix TC_PLATFORM_PER_FTP in partners.py to use shippable
Add shippable to mozharness_test variants
Only actually used for android which doesn't have shippable at this time.
Add shippable variant to beetmover transforms
Do nightly signing for mars on shippable
Support shippable in partner-repack transform
Support shippable in amo langpacks transform
Use proper signing for shippable tasks in repackage transforms
Set upload symbols to use shippable too
Use shippable as deps for geckodriver extraction
Use shippable as dep for autograph-stage signing
Do not run beetmover-l10n for shippable
Run shippable style jobs for repackage signing
Set build_platform for update verify and uvc to be shippable
Run repackage-msi for shippable
Add shippable to osx partner repack signing
add shippable to emefree repackage
add shippable to emefree repackage signing
add shippable to beetmover checksums
Add shippable to partner repack repackage signing
add partner repack beetmover
Add shippable to mar signing
Add shippable to mar-signing-l10n
add shippable to eme free beetmover checksums
Add shippable to upload-generated-sources
Add beetmover langpacks to shippable
Add repackage-l10n to shippable
Add shippable to partner repack chunk-dummy
Do eme free builds with shippable
Add signing of language packs to shippable
Add snap repackage for shippable
Add shippable for release-eme-free repack signing
Add partials for shippable
Add partner repack repackage for shippable
Add emefree beetmover for shippable
Add checksums-signing to shippable
Switch partner repacks to shippable
Add shippable to beetmover-repackage
Add secondary update verify configs for shippable
secondary update verify for shippable
Differential Revision: https://phabricator.services.mozilla.com/D24699
--HG--
extra : moz-landing-system : lando
Jmaher indicated we do not have the test capacity to incur this as a duplicated set
Differential Revision: https://phabricator.services.mozilla.com/D23382
--HG--
extra : moz-landing-system : lando
This also relates to Bug 1522111 where we turned off opt tests in favor of pgo,
shippable is like the new pgo so do that.
This has a side affect of adding talos-tp6-stylo-threads to inbound for
osx-shippable (previously was only on autoland).
This has no practical affect after D23382 is applied (because of lack of OSX capacity)
Differential Revision: https://phabricator.services.mozilla.com/D23129
--HG--
extra : moz-landing-system : lando
Should there end up being a need we can back out this patch and let them run, but :jmaher indicated
he was happy with dropping them entirely and not duplicating.
Differential Revision: https://phabricator.services.mozilla.com/D23383
--HG--
extra : moz-landing-system : lando
This is useful in order to not have to run linux64/opt on push, especially on autoland/inbound
when we need a source test. It is also required if we remove the linux64-pgo build type entirely.
Differential Revision: https://phabricator.services.mozilla.com/D23124
--HG--
extra : moz-landing-system : lando
I did a bunch of manual testing with this, the biggest uncertainties lie around beta and central/nightly.
We are adding shippable-qr to beta because of replacing nightly too.
Autoland and inbound should have the same sets of tasks.
beta -
- adds mochitest-plain-headless-{1..4} to beta (not currently run)
- adds raptor to run on shippable for beta
- currently runs on opt on beta, and for nightly tasks on beta only webaudio-chrome runs.
- adds talos to shippable tasks, on beta talos only runs against opt.
central -
- adds browser-screenshots to nightly graph
- adds mochitest-plain-headless-{1..4} to nightly graph
- adds browser-instrumentation to shippable
Differential Revision: https://phabricator.services.mozilla.com/D23122
--HG--
extra : moz-landing-system : lando
Effectively back out much of the run on projects from D22710
This also has the added affect of scheduling the shippable builds to run because of dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D22833
--HG--
extra : moz-landing-system : lando
raptor-chrome is Google Chrome and only needs to run once per day, so mozilla-central pushes and try.
raptor-profiling is primarily for devs to have up to date profile information and it too only needs to run once per day.
TODO is to try and find a clean way to make them only run when we trigger Nightlies rather than every m-c push.
Differential Revision: https://phabricator.services.mozilla.com/D22832
--HG--
extra : moz-landing-system : lando
Makes most kinds that reference nightly attribute reference the shippable attribute.
Also makes most transforms that use nightly use shippable
Transfers dependencies/ownership for some things to shippable from nightly where it was harder to support both.
In no particular order, full list:
Send shippable attribute down to dep tasks.
Set tests as shippable attribute
Add new signing routes
Add shippable routes to repackage_routes transform
Adjust target tasks
Add shippable nightly-l10n
Add nightly-signing and as a side affect add repackage and repackage-signing
Add support for proper balrog platforms for shippable
Add shippable to the nightly sccache guard
Fix TC_PLATFORM_PER_FTP in partners.py to use shippable
Add shippable to mozharness_test variants
Only actually used for android which doesn't have shippable at this time.
Add shippable variant to beetmover transforms
Do nightly signing for mars on shippable
Support shippable in partner-repack transform
Support shippable in amo langpacks transform
Use proper signing for shippable tasks in repackage transforms
Set upload symbols to use shippable too
Use shippable as deps for geckodriver extraction
Use shippable as dep for autograph-stage signing
Do not run beetmover-l10n for shippable
Run shippable style jobs for repackage signing
Set build_platform for update verify and uvc to be shippable
Run repackage-msi for shippable
Add shippable to osx partner repack signing
add shippable to emefree repackage
add shippable to emefree repackage signing
add shippable to beetmover checksums
Add shippable to partner repack repackage signing
add partner repack beetmover
Add shippable to mar signing
Add shippable to mar-signing-l10n
add shippable to eme free beetmover checksums
Add shippable to upload-generated-sources
Add beetmover langpacks to shippable
Add repackage-l10n to shippable
Add shippable to partner repack chunk-dummy
Do eme free builds with shippable
Add signing of language packs to shippable
Add snap repackage for shippable
Add shippable for release-eme-free repack signing
Add partials for shippable
Add partner repack repackage for shippable
Add emefree beetmover for shippable
Add checksums-signing to shippable
Switch partner repacks to shippable
Add shippable to beetmover-repackage
Add secondary update verify configs for shippable
secondary update verify for shippable
Differential Revision: https://phabricator.services.mozilla.com/D24699
--HG--
extra : moz-landing-system : lando
Jmaher indicated we do not have the test capacity to incur this as a duplicated set
Differential Revision: https://phabricator.services.mozilla.com/D23382
--HG--
extra : moz-landing-system : lando
This also relates to Bug 1522111 where we turned off opt tests in favor of pgo,
shippable is like the new pgo so do that.
This has a side affect of adding talos-tp6-stylo-threads to inbound for
osx-shippable (previously was only on autoland).
This has no practical affect after D23382 is applied (because of lack of OSX capacity)
Differential Revision: https://phabricator.services.mozilla.com/D23129
--HG--
extra : moz-landing-system : lando
Should there end up being a need we can back out this patch and let them run, but :jmaher indicated
he was happy with dropping them entirely and not duplicating.
Differential Revision: https://phabricator.services.mozilla.com/D23383
--HG--
extra : moz-landing-system : lando
This is useful in order to not have to run linux64/opt on push, especially on autoland/inbound
when we need a source test. It is also required if we remove the linux64-pgo build type entirely.
Differential Revision: https://phabricator.services.mozilla.com/D23124
--HG--
extra : moz-landing-system : lando
I did a bunch of manual testing with this, the biggest uncertainties lie around beta and central/nightly.
We are adding shippable-qr to beta because of replacing nightly too.
Autoland and inbound should have the same sets of tasks.
beta -
- adds mochitest-plain-headless-{1..4} to beta (not currently run)
- adds raptor to run on shippable for beta
- currently runs on opt on beta, and for nightly tasks on beta only webaudio-chrome runs.
- adds talos to shippable tasks, on beta talos only runs against opt.
central -
- adds browser-screenshots to nightly graph
- adds mochitest-plain-headless-{1..4} to nightly graph
- adds browser-instrumentation to shippable
Differential Revision: https://phabricator.services.mozilla.com/D23122
--HG--
extra : moz-landing-system : lando
Effectively back out much of the run on projects from D22710
This also has the added affect of scheduling the shippable builds to run because of dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D22833
--HG--
extra : moz-landing-system : lando
raptor-chrome is Google Chrome and only needs to run once per day, so mozilla-central pushes and try.
raptor-profiling is primarily for devs to have up to date profile information and it too only needs to run once per day.
TODO is to try and find a clean way to make them only run when we trigger Nightlies rather than every m-c push.
Differential Revision: https://phabricator.services.mozilla.com/D22832
--HG--
extra : moz-landing-system : lando
This task runs on wpt metadata changes and uploads an artifact
containing the summarised metadata.
Depends on D24178
Differential Revision: https://phabricator.services.mozilla.com/D24179
--HG--
extra : moz-landing-system : lando
Taskcluster has authorative information about the repository being built, so
pass that to mozharness, rather than have mozharness reconstruct it from
hand-maintained mozharness config.
Differential Revision: https://phabricator.services.mozilla.com/D24561
--HG--
extra : moz-landing-system : lando
Bug 1534522 added win64-aarch64-eme/opt builds, which are artifact builds
that glue together a win64-aarch64/opt build and a win32/opt build.
This enables EME on the corresponding nightlies in a slightly different
way:
- this adds a no-eme build that corresponds to win64-aarch64/opt.
- this turns the existing nightly into an artifact build that glues
together that no-eme build and the win32 nightly.
The no-eme build cannot have the nightly attribute set, first because
the beetmover transform fails in that case, and because that would imply
shipping those builds, but they're not meant to be shipped this way.
It also has run-on-projects set to an empty list so that it doesn't
appear by default in `mach try fuzzy`, while still being triggered when
needed due to being a dependency of the nightly build.
It is preferable to keep the win64-aarch64{,-eme}/opt builds untouched
to make things easier for try (the win64-aarch64 ones being the main
ones to try; also, the -eme builds currently fail with --artifacts).
Ideally, like in bug 1534522, we'd add a diffoscope build to ensure
the variations between the nightly and its base no-eme build are within
control, but currently, that would trigger nightlies on every push,
which is not desirable. Ideally, they'd trigger whenever both their
dependencies are in the target task graph. We leave that to a followup.
Differential Revision: https://phabricator.services.mozilla.com/D23640
The code that actually downloads it is behind a condition that isn't set
anywhere.
Differential Revision: https://phabricator.services.mozilla.com/D24215
--HG--
extra : moz-landing-system : lando
Currently, if the files match and you try to look at the diff, you get
{
"reason": "file-missing-on-worker",
"message": "Artifact \"public/diff.html\" not found at \"/builds/worker/diff.html\""
}
which makes it hard to tell if there was an error generating a diff, or if the files matched.
This changes things to ask diffoscope to always output a diff, to remedy that confusion.
Differential Revision: https://phabricator.services.mozilla.com/D24316
--HG--
extra : moz-landing-system : lando
Searchfox relies on the bugzilla component job running on the same push
as the indexing jobs, but there's nothing that actually guarantees that.
Thus far pushes to m-c pretty much always have source changes so the
bugzilla component job gets run, but on beta/release branches it's
possible to get pushes with just tag changes and no source changes, so
the bugzilla component job would get optimized away. This patch ensures
that the job gets run along with the other indexing jobs that searchfox
needs.
Differential Revision: https://phabricator.services.mozilla.com/D24507
--HG--
extra : moz-landing-system : lando
Without this the shippable builds take >20 seconds to compute the prune through new_as_old_is_high_value, locally. With this it is near instant.
Differential Revision: https://phabricator.services.mozilla.com/D22711
--HG--
extra : moz-landing-system : lando
This sets all of the shippable tests to not run in the places where they would otherwise.
This patch will be effectively undone later in the patchset.
Differential Revision: https://phabricator.services.mozilla.com/D22710
--HG--
extra : moz-landing-system : lando
When we set the nightly attribute the tasks don't run on-push, so we use a new attribute.
Differential Revision: https://phabricator.services.mozilla.com/D22709
--HG--
extra : moz-landing-system : lando
This commit adds a new build for OSX since there is no current PGO build type for OSX.
And calls it shippable, this mirrors that of the nightly.
Differential Revision: https://phabricator.services.mozilla.com/D22225
--HG--
extra : moz-landing-system : lando
This was needed since when we have job-defaults and later on a test set for
``
run-on-projects:
by-test-platform:
...
``
We were ending up with both the list of by-* being extended but also
any array in that list also being extended (like `default: ['a', 'b']`
was getting extended to also have the new values for default)
This is not only usually wrong but very very likely not what the author wanted.
Differential Revision: https://phabricator.services.mozilla.com/D22708
--HG--
extra : moz-landing-system : lando
Bug 1534522 added win64-aarch64-eme/opt builds, which are artifact builds
that glue together a win64-aarch64/opt build and a win32/opt build.
This enables EME on the corresponding nightlies in a slightly different
way:
- this adds a no-eme build that corresponds to win64-aarch64/opt.
- this turns the existing nightly into an artifact build that glues
together that no-eme build and the win32 nightly.
The no-eme build cannot have the nightly attribute set, first because
the beetmover transform fails in that case, and because that would imply
shipping those builds, but they're not meant to be shipped this way.
It also has run-on-projects set to an empty list so that it doesn't
appear by default in `mach try fuzzy`, while still being triggered when
needed due to being a dependency of the nightly build.
It is preferable to keep the win64-aarch64{,-eme}/opt builds untouched
to make things easier for try (the win64-aarch64 ones being the main
ones to try; also, the -eme builds currently fail with --artifacts).
Ideally, like in bug 1534522, we'd add a diffoscope build to ensure
the variations between the nightly and its base no-eme build are within
control, but currently, that would trigger nightlies on every push,
which is not desirable. Ideally, they'd trigger whenever both their
dependencies are in the target task graph. We leave that to a followup.
Differential Revision: https://phabricator.services.mozilla.com/D23640
This introduces a mozharness script, android_emulator_pgo.py, to run the
profileserver suite with the PGO-instrumented Android build, and collect
the profile data and jarlog.
The mozharness script contains some redundancy with
build/pgo/profileserver.py, but the additional requirements for Android
to use adb and existing mozharness classes to control the emulator made
it difficult to share the desktop profileserver implementation.
Differential Revision: https://phabricator.services.mozilla.com/D22825
--HG--
extra : moz-landing-system : lando
This is the first stage of the Android PGO task pipeline to generate an
instrumented build.
Differential Revision: https://phabricator.services.mozilla.com/D22824
--HG--
extra : moz-landing-system : lando
