Commit Graph

8056 Commits

Author SHA1 Message Date
Andrew Halberstadt
4c80f3f36b Bug 1640890 - [ci] Use 'push-interval-10' optimization on fuzzing builds, r=bc
This has the same effect as the 'backstop' optimization except it *doesn't*
optimize tasks on non-autoland branches.

Differential Revision: https://phabricator.services.mozilla.com/D76842
2020-05-26 13:45:03 +00:00
Gregory Mierzwinski
008827b6ac Bug 1638376 - Run recorded sites as live sites once a week. r=perftest-reviewers,alexandru.irimovici
This patch enables a new cron job that runs our recorded sites as live sites once a week through browsertime so we can continuously monitor how well our recordings match with what page the user is seeing.

Differential Revision: https://phabricator.services.mozilla.com/D75575
2020-05-26 11:20:27 +00:00
Butkovits Atila
f167b5c275 Backed out 6 changesets (bug 1602832) for causing perma failure at test_trr_case_sensitivity.js. CLOSED TREE
Backed out changeset 0e9c378df995 (bug 1602832)
Backed out changeset 534fedb3836e (bug 1602832)
Backed out changeset db012c05d64e (bug 1602832)
Backed out changeset ca0c207dca21 (bug 1602832)
Backed out changeset bea1f3aeea49 (bug 1602832)
Backed out changeset 2d54acd78124 (bug 1602832)
2020-05-26 13:09:07 +03:00
Kershaw Chang
917b0cce29 Bug 1602832 - P2: Create 'socketprocess_networking' test variant r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D58112
2020-05-26 08:35:22 +00:00
Andrew Halberstadt
f5b5676618 Bug 1633866 - [taskgraph] Implement the ability for tasks to dynamically specify chunks, r=gbrown,jmaher
This allows tasks to opt into dynamic chunking. This means rather than define
the chunks ahead of time, in-tree manifest runtime data is used to guess how
many chunks are needed for a reasonable average task runtime (currently 20
min).

Only suites that are chunked in the taskgraph, and therefore whose manifests
are known ahead of time, can opt into this feature.

Differential Revision: https://phabricator.services.mozilla.com/D76497
2020-05-25 18:52:12 +00:00
Kartikaya Gupta
4e7d703303 Bug 1638664 - Run the windows searchfox build as cross-compilation on Linux. r=asuth
This updates the task definition for the win64 searchfox job to closely match
the win64/debug task definition in taskcluster/ci/build/windows.yml. So, instead
of running the build with the mozsearch-plugin on a windows worker, it runs
on a Linux worker and does a cross-compile of windows code. The end result in
terms of searchfox artifacts is equivalent, except for absolute filename paths.
I verified that with mozsearch/mozsearch#299 and mozsearch/mozsearch-mozilla#87
in place, searchfox correctly indexes windows-only C++ and Rust code.

Differential Revision: https://phabricator.services.mozilla.com/D76721
2020-05-25 18:53:08 +00:00
Andrew Halberstadt
3901f12775 Bug 1640580 - [taskgraph.util.bugbug] Translate WPT paths from relative source directory to WPT ids, r=marco
Differential Revision: https://phabricator.services.mozilla.com/D76719
2020-05-25 15:44:58 +00:00
egao
8e54528b9d Bug 1637463 - support loading of runtime information for a specific suite r=ahal
Changes:
  - permit existing code in chunking.py and mochitests/runtests.py to load manifest runtime information specifict to the suite

Differential Revision: https://phabricator.services.mozilla.com/D75592
2020-05-24 00:24:05 +00:00
Mihai Alexandru Michis
74ff68a9a5 Backed out 3 changesets (bug 1637463) for causing bc jobs to timeout.
CLOSED TREE

Backed out changeset 93efb53380c2 (bug 1637463)
Backed out changeset 5c8ccda5f1b1 (bug 1637463)
Backed out changeset a5fc1b604238 (bug 1637463)
2020-05-23 02:35:26 +03:00
egao
d40fc31a1b Bug 1637463 - support loading of runtime information for a specific suite r=ahal
Changes:
  - permit existing code in chunking.py and mochitests/runtests.py to load manifest runtime information specifict to the suite

Differential Revision: https://phabricator.services.mozilla.com/D75592
2020-05-22 18:31:47 +00:00
Marco Castelluccio
30cfe57424 Bug 1640047 - Don't take into account bugbug's known tasks if they are likely wrong. r=ahal DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D76418
2020-05-22 15:28:43 +00:00
Andrew Halberstadt
0591e31c1f Bug 1639349 - [ci] Set up a SCHEDULES rule to exclude non-webrender tests on webrender-only pushes, r=gbrown
Depends on D76433

Differential Revision: https://phabricator.services.mozilla.com/D76434
2020-05-22 14:03:46 +00:00
Andrew Halberstadt
e6fc2d3678 Bug 1639349 - [taskgraph] Set schedules-component for test kind in an independent transform, r=gbrown
Depends on D76306

Differential Revision: https://phabricator.services.mozilla.com/D76433
2020-05-22 14:03:46 +00:00
Ben Hearsum
d7624bd580 Bug 1638014: blacklist shippable tests from try when there is an opt version of the test available r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D76012
2020-05-22 17:45:06 +00:00
Rob Wu
27981af497 Bug 1636158 - Update the addons-mlbf.bin attachment via periodic_file_updates.sh r=sfraser
Tested by running the following code and verifying that the script runs
successfully and generates a reasonable diff with the latest metadata
and matching attachment content:

```
$ cd taskcluster/docker/periodic-updates/
$ tar -czf- -s "#$(git rev-parse --show-cdup)#topsrcdir/#" . $(awk -v d="$(git rev-parse --show-cdup)" '/^# %include/{print d$3}' Dockerfile) | docker build - -t hsts-local
$ docker run -e DO_REMOTE_SETTINGS=1 -e PRODUCT="firefox" -e BRANCH="mozilla-central" -e USE_MOZILLA_CENTRAL=1 hsts-local
```

After that, I opened a shell in the container, built Firefox and ran the
following test, to confirm that the updated records are used correctly:

```
mach test toolkit/mozapps/extensions/test/xpcshell/rs-blocklist/test_blocklist_mlbf_dump.js
```

Differential Revision: https://phabricator.services.mozilla.com/D76158
2020-05-22 01:37:37 +00:00
Tom Prince
2bf24e6359 Bug 1639955: Use prefix for geckodriver toolchain artifacts; r=aki
We originally switched it to use just `public/` becuase iscript broke with the
extra path, but that broke out-of-tree consumers. Now that iscript is fixed,
switch it back.

Differential Revision: https://phabricator.services.mozilla.com/D76409
2020-05-22 03:38:32 +00:00
Tom Prince
9703222150 Bug 1639955: Include toolchain-artifact in the cached-task digest; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D76410
2020-05-22 03:38:29 +00:00
Chris AtLee
4c1292f8f8 Bug 1637381: Fix reference to old tarball for win64-clang-tidy r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D76420
2020-05-22 01:12:56 +00:00
Justin Wood
d582323fcb Bug 1640057 - Make condition profile fetch use the correct directory (en-US from shippable). r=aki
Fixes bustage from 1623355.

Differential Revision: https://phabricator.services.mozilla.com/D76424
2020-05-22 01:47:28 +00:00
Sylvestre Ledru
27f8422c60 Bug 1635826 - platform docs: improve the list of platforms r=marco DONTBUILD
* document more jobs
* document when some are running
* remove some old jobs
  - stylo seq a9c044b771
  - webrender pgo https://github.com/mozilla/treeherder/pull/6437

Differential Revision: https://phabricator.services.mozilla.com/D76184
2020-05-21 09:16:12 +00:00
Bob Clary
5d2b7338b8 Bug 1639886 - Allow the Backstop optimization strategy to be used outside of the test optimization, r=ahal.
Differential Revision: https://phabricator.services.mozilla.com/D76350
2020-05-21 20:53:20 +00:00
Andrew Halberstadt
5ad7af589e Bug 1633866 - [taskgraph] Refactor logic around manifest chunking, r=egao
This sets things up to be a little bit easier and cleaner to modify going
forward. The taskgraph before/after this change is identical.

Differential Revision: https://phabricator.services.mozilla.com/D74452
2020-05-21 15:58:24 +00:00
Justin Wood
e49ff70dad Followup to Bug 1623355 - Part 1, specifically fix typo in 'nightly'. CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D76352
2020-05-21 18:07:22 +00:00
Chris AtLee
b34b110320 Bug 1637381: Install zstandard on linting images on a CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D76348
2020-05-21 17:31:01 +00:00
Gregory Mierzwinski
c4ae7d3f70 Bug 1639374 - Disable majority of browsertime desktops. r=perftest-reviewers,Bebe
This patch disables the majority of browsertime desktop tests. First, we disable all of the tests on autoland so no more browsertime tests run there. These benchmark tests that were running there will be enabled on mozilla-central along with the rest of the tests. Finally, all browsertime tests will be further restricted to running on the linx64-shippable platform.

Differential Revision: https://phabricator.services.mozilla.com/D76212
2020-05-21 14:07:15 +00:00
Justin Wood
6c368aa658 Bug 1617481 - Remove most of nightly attribute from taskcluster/taskgraph. r=aki
We leave the nightly index in place for now in order to support Geckodriver until Bug 1577110 sticks.

Differential Revision: https://phabricator.services.mozilla.com/D74510
2020-05-21 16:31:04 +00:00
Justin Wood
3f24c1f0f0 Bug 1617481 - Remove nightly index's. r=aki
They only really worked for promoted builds when the build was generated on the nightly cron anyway.

Differential Revision: https://phabricator.services.mozilla.com/D74509
2020-05-21 16:30:11 +00:00
Justin Wood
f8829d41ab Bug 1636637 - Remove now-redundant geckoview fat aar (non-shippable). r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74508
2020-05-21 16:30:03 +00:00
Justin Wood
b2ef4dc0b7 Bug 1636637 - Make geckoview shippable its own treeherder line and update its index. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74507
2020-05-21 16:30:01 +00:00
Justin Wood
4cff028d95 Bug 1636635 - Swap devedition to shippable index and use non-Nightly treeherder symbols. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74506
2020-05-21 16:29:59 +00:00
Justin Wood
ec5c1d56da Bug 1623355 - Part 9/9 - Remove 'pgo' support from performance testing target tasks. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74961
2020-05-21 16:29:51 +00:00
Justin Wood
27e43125b2 Bug 1623355 - Part 8/9 - Set new shippable index's for Android shippable builds. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74360
2020-05-21 16:29:48 +00:00
Justin Wood
66aebeae94 Bug 1623355 - Part 7/9 - Set treeherder symbols appropriately for Android Shippable. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74359
2020-05-21 16:29:46 +00:00
Justin Wood
d02f213610 Bug 1623355 - Part 6/9 - Remove Android PGO build types. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74309
2020-05-21 16:29:44 +00:00
Justin Wood
897aafaf58 Bug 1623355 - Part 5/9 - Rename pgo instrumentation steps to shippable. And make condition profiles use shippable. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74308
2020-05-21 16:29:41 +00:00
Justin Wood
15cec74666 Bug 1623355 - Part 4/9 - Point tests based on pgo builds to use shippable builds. r=aki
This needs to remove nightly attribute from android shippable.

Differential Revision: https://phabricator.services.mozilla.com/D74307
2020-05-21 16:29:37 +00:00
Justin Wood
7633ba28c6 Bug 1623355 - Part 3/9 - Rename android pgo tests to -shippable/opt and adjust run-on-platforms to cope. r=aki
This keeps all tests that were running as pgo, continuing to run in the same places.

Differential Revision: https://phabricator.services.mozilla.com/D74306
2020-05-21 16:29:34 +00:00
Justin Wood
f9bb21ade0 Bug 1623355 - Part 2/9 - Add shippable as a valid target for performance test cron targets. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D74960
2020-05-21 16:31:23 +00:00
Tom Prince
24d507a5bc Bug 1623355 - Part 1/9 - Set geckoview artifact names to match mozharness on non-release branches; r=Callek
For shippable builds on non-release branches, mozharness defaults to using
`nightly-{project}`. The update channel is encoded in the artifact filenames
for non release-channel builds, so update taskgraph to generate the correct
artifact names on non-release branches.

Differential Revision: https://phabricator.services.mozilla.com/D76274
2020-05-21 16:32:06 +00:00
Marco Castelluccio
39d348c1c5 Bug 1639812 - Don't apply 'skip-unless-schedules' when overriding 'build-fuzzing' strategy. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D76306
2020-05-21 13:38:57 +00:00
Chris AtLee
62f699319d Bug 1637381: Use zstd for clang toolchains r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D74930
2020-05-21 13:31:55 +00:00
Chris AtLee
b1c32e06b2 Bug 1637381: Use zstd for rust toolchains r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D74929
2020-05-21 13:30:56 +00:00
Chris AtLee
72cc360dcd Bug 1637381: Use python3 for repack_rust.py r=glandium
repack_rust.py was the only toolchain build using a python script directly. we
don't need it to use the `mach python` wrapper, so remove that special casing
from the transform.

Differential Revision: https://phabricator.services.mozilla.com/D75702
2020-05-21 13:30:54 +00:00
Chris AtLee
d0a9a8b30b Bug 1637381: Use zstd for android toolchains r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D74928
2020-05-21 13:29:13 +00:00
Chris AtLee
3befa9e65b Bug 1637381: Add zstdpy script r=glandium
This is a simple command line utility for doing zstd compression/decompression
of stdin to stdout. It depends on python's zstandard module being available.

Differential Revision: https://phabricator.services.mozilla.com/D75358
2020-05-21 13:28:55 +00:00
Chris AtLee
ac6909cd9a Bug 1637381: Add python-zstandard to ubuntu test images so they can unpack toolchains r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D75356
2020-05-21 13:28:45 +00:00
Chris AtLee
cd35328624 Bug 1637381: Add support for extracting zst files to mozbuild tooltool r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D75203
2020-05-21 13:28:33 +00:00
Chris AtLee
ef685f3549 Bug 1637379: Optimize android toolchains r=gbrown
Create an emulator toolchain
Stop including mozboot/sdk-*.zip in our tarball

Differential Revision: https://phabricator.services.mozilla.com/D74931
2020-05-21 13:28:10 +00:00
Sylvestre Ledru
b39fdab18d Bug 1638050 - Taskcluster config: Do not run mingw jobs on m-b or m-r r=jmaher,tjr
mingw is used by Tor, we don't ship it
as they are based on esr, no need to run mingw on every uplift in m-b or m-r

Differential Revision: https://phabricator.services.mozilla.com/D76132
2020-05-20 20:38:01 +00:00
Tom Prince
ca0e268b05 No bug: Rename list of labels to ignore on try to UNCOMMON_TRY_TASK_LABELS; r=ahal
Give it a more descriptive name, and avoid using `blacklist` in the name.

Differential Revision: https://phabricator.services.mozilla.com/D76031
2020-05-20 13:18:19 +00:00