This is similar to the same-named commit in bug 1619837.
For those few configurations that don't run xpcshell tests, we need a
`fix-stacks` dependency in the build job. This is because xpcshell self tests
run during the build job, but the existing xpcshell test dependency doesn't
cover that case.
Differential Revision: https://phabricator.services.mozilla.com/D66927
--HG--
extra : moz-landing-system : lando
First, with `fix_stacks.py` now being used on most test suites, those test
suites need a dependency on `fix-stacks`.
Second, for those few configurations that don't run xpcshell tests, we need a
`fix-stacks` dependency in the build job. This is because xpcshell self tests
run during the build job, but the existing xpcshell test dependency doesn't
cover that case.
Differential Revision: https://phabricator.services.mozilla.com/D65943
--HG--
extra : moz-landing-system : lando
This moves the object directory and source directories around in all mozharness
jobs, to allow enabling caching on windows builders.
This makes a number of changes that all need to land at once:
- Move the source checkout for the workspace cache mount, to the checkouts
cache mount.
- Makes the object directoy from underneath the source directory, to directly
in the work directory (which is still under `workspace`).
- Sets the object directory to `obj-build` instead of `obj-firefox`.
- Stops caching the workspace directory (it is still a volume in docker workers,
so writes perform well; a followup revision add some checks around this).
- Removes one level of directory in the mozharness workdir (things were under
`workspace/build`, but are now just under `workspace/`.
- Adjust paths in environment variables and artifact specifications to match
the above changes.
Differential Revision: https://phabricator.services.mozilla.com/D62482
--HG--
extra : moz-landing-system : lando
Define and use fuzzing-builds optimization strategy that allows either
skip-unless-schedules or seta strategies.
Differential Revision: https://phabricator.services.mozilla.com/D64628
--HG--
extra : moz-landing-system : lando
Future work should hopefully find a way to not assume tests are packaged with beetmover, and to have a solution for w64's emefree design.
Differential Revision: https://phabricator.services.mozilla.com/D64429
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky area is the win64-aarch64 builds, which have a workaround
by specifying the new skip-verify-test-packaging attribute.
In one case, win64-aarch64-shippable has tests that run against it, but
it copies those tests from a win64-aarch64-shippable-no-eme task, which
itself has no tests. Both of those tasks need to skip the verify check
as a result.
In another case, the win64-aarch64-eme task is an artifact build that
grabs test packages from the win64-aarch64 build. Since the win64-aarch64
build doesn't have tests, it needs to skip the verify check.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky area is the win64-aarch64 builds, which have a workaround
by specifying the new skip-verify-test-packaging attribute.
In one case, win64-aarch64-shippable has tests that run against it, but
it copies those tests from a win64-aarch64-shippable-no-eme task, which
itself has no tests. Both of those tasks need to skip the verify check
as a result.
In another case, the win64-aarch64-eme task is an artifact build that
grabs test packages from the win64-aarch64 build. Since the win64-aarch64
build doesn't have tests, it needs to skip the verify check.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
In GCP we have the double the number of core compared to AWS
counterparts, but we use the same amount of memory. Request the builds
to be less parallel to avoid OOM.
Differential Revision: https://phabricator.services.mozilla.com//D62514
We are migrating the worker used by builds from AWS to GCP. We have had tier- 3 GCP builds for initial testing. We are replacing those with tier-3 AWS builds to ensure they keep working so we are able to switch back.
Differential Revision: https://phabricator.services.mozilla.com/D60042
--HG--
extra : moz-landing-system : lando
We are migrating the worker used by builds from AWS to GCP. We have had tier- 3 GCP builds for initial testing. We are replacing those with tier-3 AWS builds to ensure they keep working so we are able to switch back.
Differential Revision: https://phabricator.services.mozilla.com/D60042
--HG--
extra : moz-landing-system : lando
Changes:
Diffoscope linux32 (diff-reproducible-linux32) was triggering linux-shippable jobs on autoland.
For good measure, specify for `geckodriver-repack` that it should not run on autoland.
Also restrict `linux-shippable` build on gcp from taking place on autoland. It is a tier 3 job so not visible normally.
Differential Revision: https://phabricator.services.mozilla.com/D59444
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky task is win64-aarch64-shippable/opt, which copies tests
from another build rather than building them itself. For this reason, it
explicitly sets MOZ_AUTOMATION_PACKAGE_TESTS: '0' in the environment
even though that is now the default. This is why the exception is only
raised if MOZ_AUTOMATION_PACKAGE_TESTS is not set at all, rather than
checking that it is set to 1.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
We're going to build `lucetc` for other platforms; let's prefix this
task with the platform name like we do for other toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D59498
--HG--
extra : moz-landing-system : lando
Changes:
Do not run `linux32` builds on `try` or `autoland` by default, but ensure it is able to run on all other repos (eg. `mozilla-beta`).
Filter out all tasks with `linux-shippable` in the name, including builds, auxiliary tasks (balrog, etc).
Differential Revision: https://phabricator.services.mozilla.com/D58982
--HG--
extra : moz-landing-system : lando
Changes:
Revert the changes to the treeherder name since the partials script does depend on the name as it appears in treeherder to the pre-D58406 state.
Differential Revision: https://phabricator.services.mozilla.com/D58897
--HG--
extra : moz-landing-system : lando
Changes:
Demote linux32-shippable/opt tasks to tier 2 on all repositories. Other linux32 jobs are defined but do not appear to run.
Limit the build of linux32-shippable/opt to `try`/`mozilla-central`.
Differential Revision: https://phabricator.services.mozilla.com/D58207
--HG--
extra : moz-landing-system : lando
Changes:
Demote linux32-shippable/opt tasks to tier 2 on all repositories. Other linux32 jobs are defined but do not appear to run.
Limit the build of linux32-shippable/opt to `try`/`mozilla-central`.
Differential Revision: https://phabricator.services.mozilla.com/D58207
--HG--
extra : moz-landing-system : lando
Changes:
Make available all variants of linux1804 for general use. Where appropriate, each variant is populated with `linux1804-tests`, which contain the shared tests that should run on all appropriate variants.
To deal with linux64-asan/opt and linux1804-64-asan/opt clashing in the taskgraph due to the translation service, remove `linux64/asan` entry from the translation dictionary and manually check in the `if/else` conditional later on.
Differential Revision: https://phabricator.services.mozilla.com/D58406
--HG--
extra : moz-landing-system : lando
xvfb was used to create a virtual framebuffer for running Firefox during
build jobs to support PGO profile generation. That now runs in a
separate task, so we don't need this flag for builds anymore.
Note that other Linux builds still need xvfb in order to run xpcshell in
'make check'.
Differential Revision: https://phabricator.services.mozilla.com/D56111
--HG--
extra : moz-landing-system : lando
GCC8 happens not to generate the code that causes the crash, so do that for now
to unblock fuzzers from hitting this.
We still need to figure out what to do about the more general issue of course...
Differential Revision: https://phabricator.services.mozilla.com/D55985
--HG--
extra : moz-landing-system : lando
