Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-10-20 00:35:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
711,499 Commits 615 Branches 98 Tags 5.9 GiB
ddaf15c72f
Commit Graph

8134 Commits

Author SHA1 Message Date
Andrew Halberstadt
ddaf15c72f Bug 1633866 - [taskgraph] Implement a multiplier to tweak the duration for certain platforms / suites, r=gbrown 
This is needed because xpcshell runs in parallel. Therefore we need to know the number
of threads are used (which varies by platform) in order to accurately determine how
many chunks are needed.

This allows us to specify a platform / suite specific multiplier to compensate via trial
and error.

Differential Revision: https://phabricator.services.mozilla.com/D78153
 2020-06-04 15:34:51 +00:00
Andrew Halberstadt
d4001d553e Bug 1633866 - [taskgraph] Truncate runtimes that exceed DYNAMIC_CHUNK_DURATION so they don't artificially increase chunks, r=gbrown 
For example, assume we have three manifests:

foo -> 98 minutes
bar -> 1 minute
baz -> 1 minute

And let's say we want average chunk lengths of ~20 minutes. The calculation
would currently say we need 5 chunks ((98 + 1 + 1) / 20). Even if set the max
chunks to the # of manifests, it would still return 3. Clearly this only needs
at most 2 chunks to run.

By truncating the runtime, we'd have:

foo -> 20 minutes (truncated)
bar -> 1 minute
baz -> 1 minute

So we'd actually only assign one chunk (round((20 + 1 + 1) / 20)). Despite
having a single 100 minute chunk, this outcome is desirable as we now avoid the
overhead of a second chunk that only runs 2 minutes worth of tests.

Depends on D77871

Differential Revision: https://phabricator.services.mozilla.com/D77873
 2020-06-03 21:00:43 +00:00
Andrew Halberstadt
b3fbd0cf70 Bug 1633866 - [taskgraph] Always use chunk number as symbol if it would otherwise be empty, r=jmaher 
Depends on D76523

Differential Revision: https://phabricator.services.mozilla.com/D77871
 2020-06-02 18:56:54 +00:00
Andrew Halberstadt
27d78d7059 Bug 1633866 - [taskgraph] Create a 'bugbug' based test manifest loader, r=marco 
Loads manifests using bugbug's push/schedules endpoint. For now we use as low
confidence threshold to select manifests. This is to try and both:

1) Improve regression detection rate
2) Save resources

at the same time. This way we theoretically shouldn't regress either dimension.

Note that we still optimize with CT_MEDIUM. My thinking is that at least all
of the "extra" stuff is still guaranteed to be relevant. Though we may want to
consider making these two thresholds match.

Differential Revision: https://phabricator.services.mozilla.com/D76523
 2020-06-04 18:45:09 +00:00
Andrew Halberstadt
be4e9089ec Bug 1633866 - [taskgraph] Simplify how 'suites' are handled in manifest resolving and chunking, r=egao 
This change is a straight refactor and results in identical full taskgraphs.

Differential Revision: https://phabricator.services.mozilla.com/D77543
 2020-06-02 20:00:46 +00:00
Razvan Maries
e46cf863d7 Backed out changeset 447cbc28fda3 (bug 1641948) for Android 7 perma failures. CLOSED TREE 2020-06-04 18:55:43 +03:00
Razvan Maries
179572857b Backed out changeset d51c3c722d65 (bug 1643335) for breaking Gecko Decision Task. CLOSED TREE 2020-06-04 18:29:19 +03:00
Sylvestre Ledru
0f57f34c18 Bug 1643335 - Run hazard-linux64-haz/debug less often r=jmaher,sfink 
Differential Revision: https://phabricator.services.mozilla.com/D78275
 2020-06-04 15:19:38 +00:00
Ben Hearsum
dd5a8a4ba7 Bug 1642713: allow diff-artifact tasks to be optimized by push interval r=tomprince 
Differential Revision: https://phabricator.services.mozilla.com/D77859
 2020-06-04 14:27:42 +00:00
Ben Hearsum
3dca5e3f67 Bug 1641948: remove android opt builds from non-mozilla-central branches; reduce frequency of fat-aar builds r=snorp,Callek 
snorp - I'm not sure if you want/need to review this yourself so I'm flagging you as optional.

Differential Revision: https://phabricator.services.mozilla.com/D77834
 2020-06-03 20:52:38 +00:00
Ed Lee
96ed60b4bd Bug 1641863 - Upgrade Node 10.x (in preparation for ESR 78) r=Standard8. CLOSED TREE 
Node v10.21.0 (LTS) just released https://nodejs.org/en/blog/release/v10.21.0/

Differential Revision: https://phabricator.services.mozilla.com/D77904
 2020-06-03 18:00:52 +00:00
Tom Prince
9eac413f54 Bug 1642446: Add an attribute to annotate toolchain tasks used by mach bootstrap; r=firefox-build-system-reviewers,rstewart 
Toolchains that are used for local development need to be built on a level-3
branch to installable via `mach bootstrap`. Add an attribute to track the fact
that a toolchain is used that way, and:
- ensure that everything installed via `mach boostrap` has that attribute set
- ensure that everything with that attribute set is built on trunk projects

We could additionally verify that attribute is only set on things used by
bootstrap, but bootstrap doesn't currently have an exhaustive list of things
that it might install, making that difficult.

Differential Revision: https://phabricator.services.mozilla.com/D77706
 2020-06-03 15:55:26 +00:00
Nathan Froyd
40348063b3 Bug 1643029 - remove build-.*-aarch64 from being hidden behind --full; r=ahal 
It's not good to hide Android AArch64 builds, a tier 1 platform, behind `--full`.

Differential Revision: https://phabricator.services.mozilla.com/D78070
 2020-06-03 16:15:43 +00:00
Andrew Halberstadt
3428cd2244 Bug 1601782 - [ci] Enable crashtest-fis on same platforms where reftest-fis is running, r=jmaher 
This adds the following task:
test-linux1804-64-qr/debug-crashtest-fis-e10s

Differential Revision: https://phabricator.services.mozilla.com/D78030
 2020-06-03 14:09:42 +00:00
Kartikaya Gupta
b086844bd9 Bug 1642720 - Disable reftests on win7/mingw builds. r=tjr 
These reftests seem to hang quite frequently, caused by the content process
hanging on file create/open operations. See dependencies of bug 1642719 for
some examples.

Differential Revision: https://phabricator.services.mozilla.com/D77875
 2020-06-03 14:17:08 +00:00
Marco Castelluccio
6b213d3c19 Bug 1637810 - Define a new strategy that optimizes builds away if they are not selected by bugbug. r=ahal DONTBUILD 
Differential Revision: https://phabricator.services.mozilla.com/D75621
 2020-06-03 09:53:17 +00:00
Tom Prince
8073961f7e Bug 1641971: [taskgraph] Use task-reference to refer to the decision task in task; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77552
 2020-06-03 02:35:00 +00:00
Tom Prince
ca6c9020de Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77551
 2020-06-03 02:35:03 +00:00
Tom Prince
9f3450be95 Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77550
 2020-06-03 02:35:05 +00:00
Tom Prince
86711ecee8 Bug 1641971: Use task-reference to pass the decision task id to shadow schedulers; r=ahal 
Differential Revision: https://phabricator.services.mozilla.com/D77549
 2020-06-03 02:35:07 +00:00
Tom Prince
581eba974b Bug 1641971: Pass mach perftest options explicitly, rather than inpsect parameters; r=tarek 
Differential Revision: https://phabricator.services.mozilla.com/D77548
 2020-06-03 02:35:09 +00:00
Tom Prince
6617f03f05 Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77547
 2020-06-03 02:43:34 +00:00
Tom Prince
01e244f62c Bug 1641971: Allow making references to a task itself in the task definition; r=Callek 
This is to support linking to the a specific task in taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D77529
 2020-06-03 02:35:14 +00:00
Henrik Skupin
12f6324f02 Bug 1623128 - [remote] Run Puppeteer unit tests on ubuntu18.04. r=egao 
Differential Revision: https://phabricator.services.mozilla.com/D77765
 2020-06-03 04:07:43 +00:00
Mike Hommey
98b889cf83 Bug 1622963 - Install python 3.6 in build docker images. CLOSED TREE 
Differential Revision: https://phabricator.services.mozilla.com/D77753
 2020-06-02 03:51:14 +00:00
Narcis Beleuzu
479c45bbea Backed out 7 changesets (bug 1641971) for breaking Gecko Decision Task opt . CLOSED TREE 
Backed out changeset a039e60a26d2 (bug 1641971)
Backed out changeset e7d4c490c31a (bug 1641971)
Backed out changeset f2322981b294 (bug 1641971)
Backed out changeset d123b08d2c20 (bug 1641971)
Backed out changeset 57772c0b9b3e (bug 1641971)
Backed out changeset 11a887f73dbb (bug 1641971)
Backed out changeset e9c7a59d4bc5 (bug 1641971)
 2020-06-03 05:30:51 +03:00
Mitchell Hentges
658f5f0eeb Bug 1642411: Removes unused test-macosx.sh script r=ahal 
A grep through the repository and a brief discussion with #releng seems to point towards this script not being used.

Differential Revision: https://phabricator.services.mozilla.com/D77689
 2020-06-02 18:55:35 +00:00
Tom Prince
6f0a7bbba6 Bug 1641971: [taskgraph] Use task-reference to refer to the decision task in task; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77552
 2020-06-02 18:23:23 +00:00
Tom Prince
f32e4a4f4d Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77551
 2020-06-02 18:23:19 +00:00
Tom Prince
7c6a413ca9 Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77550
 2020-06-02 18:23:20 +00:00
Tom Prince
73b2e6f745 Bug 1641971: Use task-reference to pass the decision task id to shadow schedulers; r=ahal 
Differential Revision: https://phabricator.services.mozilla.com/D77549
 2020-06-02 18:23:11 +00:00
Tom Prince
11aedca9b2 Bug 1641971: Pass mach perftest options explicitly, rather than inpsect parameters; r=tarek 
Differential Revision: https://phabricator.services.mozilla.com/D77548
 2020-06-02 18:23:07 +00:00
Tom Prince
602da9cdcd Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77547
 2020-06-02 18:50:10 +00:00
Tom Prince
de6126516d Bug 1641971: Allow making references to a task itself in the task definition; r=Callek 
This is to support linking to the a specific task in taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D77529
 2020-06-02 18:22:55 +00:00
Chris AtLee
4925d4d08e Bug 1638990: Use requests for fetching parameters r=tomprince 
Differential Revision: https://phabricator.services.mozilla.com/D76133
 2020-05-28 19:01:42 +00:00
Chris AtLee
b6edad89bf Bug 1638990: Sort values in taskgraph in order to make comparing output easier r=rstewart 
Differential Revision: https://phabricator.services.mozilla.com/D77320
 2020-05-29 15:01:50 +00:00
Sylvestre Ledru
273a7905e6 Bug 1635826 - platform docs: Document WebRender tasks r=jbonisteel 
Differential Revision: https://phabricator.services.mozilla.com/D77677
 2020-06-01 15:28:13 +00:00
Sylvestre Ledru
c5994e6bba Bug 1628289 - platform docs: the Documentation jobs moved away from the linting section 
Depends on D76185

Differential Revision: https://phabricator.services.mozilla.com/D77657
 2020-06-01 15:28:13 +00:00
Sylvestre Ledru
58f742cd8b Bug 1635826 - platform docs: Remove the nightly platforms r=Callek 
Remove in treeherder in this patch:
1568eaf479

Differential Revision: https://phabricator.services.mozilla.com/D76185
 2020-06-01 11:11:22 +00:00
Edwin Takahashi
3efe2edb5c Bug 1641395 - install amd64 variants of existing i386 packages and insert additional startup parameters for pulseaudio r=jmaher 
Changes:
  - install the `amd64` variants of various packages alongside `i386` packages
  - install `amd64` version of `fontconfig` instead
  - additional command line arguments for pulseaudio initialization
  - do not add configuration files to `~/.pulse` in the docker image

Differential Revision: https://phabricator.services.mozilla.com/D77512
 2020-06-01 20:44:47 +00:00
Agi Sferro
4e005a2cbd Bug 1641711 - Update artifacts for mozlint-android. r=nalexander 
Differential Revision: https://phabricator.services.mozilla.com/D77544
 2020-06-01 15:57:58 +00:00
Tom Prince
98dd864200 No bug: Don't use proxy for accessing taskcluster index; r=Callek 
The index is not protected by scopes, so don't use the proxy to access it

Differential Revision: https://phabricator.services.mozilla.com/D77537
 2020-05-29 21:37:46 +00:00
emilfs
7e13ebcb0a Bug 1628289 - move documentation jobs from lint/opt to doc/opt. r=sylvestre 
Differential Revision: https://phabricator.services.mozilla.com/D77481
 2020-05-29 21:12:13 +00:00
Mitchell Hentges
2a66b39bc4 Bug 1641962: Defer imports of distutils.util to ensure bootstrap can run r=rstewart,perftest-reviewers,sparky 
Differential Revision: https://phabricator.services.mozilla.com/D77516
 2020-05-29 20:17:27 +00:00
Andrew Halberstadt
f0e5f6ed7b Bug 1633866 - [taskgraph] Create a system to choose how test manifests are loaded via a parameter, r=tomprince 
Currently test manifests are loaded by instantiating a TestResolver and
traversing moz.build files to find manifests.

With 'manifest-scheduling', we'll want to grab the manifests directly from the
bugbug service instead. Initially we'll want to enable manifest-scheduling with
|mach try auto|, but not on autoland yet.

This patch will allow |mach try auto| to set the parameter that causes bugbug
to be used (see future commits in this bug).

Differential Revision: https://phabricator.services.mozilla.com/D76522
 2020-05-29 18:51:14 +00:00
Bogdan Tara
c72f5589b1 Backed out changeset 2c18e64584af (bug 1641131) for gecko decision task bustage CLOSED TREE 2020-05-29 18:33:23 +03:00
Andrew Halberstadt
ec4749227c Bug 1641131 - [taskgraph] Make 'repackage' and 'repackage-signing' tasks copy their dependency's optimization, r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77346
 2020-05-29 14:43:29 +00:00
Andrew Halberstadt
12b0e45ee9 Bug 1641131 - [taskgraph] Make 'upload-symbols' tasks copy their dependency's optimization r=Callek 
Differential Revision: https://phabricator.services.mozilla.com/D77048
 2020-05-29 14:42:51 +00:00
Andrew Halberstadt
ad57d3a5df Bug 1641131 - [taskgraph] Make 'upload-generated-sources' tasks copy their dependency's optimization r=Callek 
This should ensure that they are never the cause of their dependency being scheduled
and only run when they do.

Differential Revision: https://phabricator.services.mozilla.com/D76988
 2020-05-29 14:42:44 +00:00
Andrew Halberstadt
bd5660af34 Bug 1641131 - [taskgraph] Fix the 'push-interval-*' schema to not require an argument, r=jmaher 
The Backstop optimization doesn't take any arguments, yet the schema for 'push-optimization-*'
requires a 'schedules-component' that goes ignored. Fix this.

Differential Revision: https://phabricator.services.mozilla.com/D77058
 2020-05-29 14:48:07 +00:00