Commit Graph

964 Commits

Author SHA1 Message Date
Mike Hommey
3854c3f71d Bug 1341214 - Add a small API to handle taskcluster queue and index requests. r=dustin
Various modules under taskcluster are doing ad-hoc url formatting or
requests to taskcluster services. While we could use the taskcluster
client python module, it's kind of overkill for the simple requests done
here. So instead of vendoring that module, create a smaller one with
a limited set of functions we need.

This changes the behavior of the get_artifact function to return a
file-like object when the file is neither a json nor a yaml, but that
branch was never used (and was actually returning an unassigned
variable, so it was broken anyways).

At the same time, make the function that does HTTP requests more
error-resistant, using urllib3's Retry with a backoff factor.

Also add a function that retrieves the list of artifacts, that while
currently unused, will be used by `mach artifact` shortly.

--HG--
extra : rebase_source : d7ef633e8e5041dc8450f3ff2f3751c85f144cdc
2017-02-17 12:04:48 +09:00
Mike Hommey
28cdcbdebb Bug 1341214 - Define GECKO in a single location. r=dustin
Instead of every file trying to get the top source directory having an
ad-hoc definition that gets wrong if the files gets moved around for
some reason, define it in a more central location.

--HG--
extra : rebase_source : 1a0cbea267193f6b388b88b36166353e20ac8569
2017-02-17 11:56:12 +09:00
Wes Kocher
f448516b0f Merge autoland to central, a=merge
MozReview-Commit-ID: I3hgM0ng45o
2017-02-27 13:12:25 -08:00
Mihai Tabara
035e4968ce Bug 1338186 - tweak the bucket scope to new form. r=aki a=release DONTBUILD
MozReview-Commit-ID: Fbjk6xTZwmZ

--HG--
extra : source : e6b4a6513789a8fa3a2950bb9f6c65d5f5afc5f2
extra : intermediate-source : e00f58c31a85ad03e6eccfa99926f75687041254
2017-02-27 18:55:03 +00:00
Justin Wood
c61a67c094 Bug 1337240: Cleanup, re-sort JOB_NAME_WHITELIST. r=dustin
MozReview-Commit-ID: CTpW53dAN6P

--HG--
extra : rebase_source : 3535fd6d19332881939034ec9c8b6db9038288ca
2017-02-05 22:24:26 -05:00
Justin Wood
f7a102bbb2 Bug 1337240: Cleanup, remove unused entries from gecko_v2_whitelist. r=dustin
MozReview-Commit-ID: GZDui8mNWlu

--HG--
extra : rebase_source : 4492d3197959373eb65470dce877df793783eddb
2017-02-05 22:22:11 -05:00
Wander Lairson Costa
15db056695 Bug 1330310: Switch macosx tests to buildbot-bridge. r=catlee,dustin
We use buildbot-bridge to schedule macosx tests in buildbot, and disable
scheduling on buildbot. Also, schedule a subset of unittests in
taskcluster-worker Tier 3 machines.

MozReview-Commit-ID: Dbn6U4V2NLp

--HG--
extra : rebase_source : 4f2e5edb4da73975d6634e6313f76a1a9c7651a8
2017-02-27 09:52:22 -03:00
Dustin J. Mitchell
daecfa79ae Bug 1342510: fix -u mochitest-e10s-foo aliases to point to nonempty sets of tests; r=jmaher
MozReview-Commit-ID: 4P8WQraSe9U

--HG--
extra : rebase_source : 4562cc3f7ddd47dbeaa8e88e9904ed3a8d223e1b
2017-02-24 22:03:26 +00:00
Wes Kocher
14a9e6ef12 Merge m-c to autoland a=merge
MozReview-Commit-ID: DAlTWWRlpmp
2017-02-24 16:59:05 -08:00
Wes Kocher
e84fc624ff Merge inbound to central, a=merge
MozReview-Commit-ID: DpCZgRV1csS
2017-02-24 16:46:12 -08:00
Jonas Finnemann Jensen
b77f334748 Bug 1341727 - Setup logging in ./mach taskgraph action-callback. r=dustin,wlach
MozReview-Commit-ID: LLSU87IofRh

--HG--
extra : rebase_source : 81450062c2b7fa37708d6cdba05a91db91018db9
2017-02-23 18:33:38 -08:00
Jonas Finnemann Jensen
abaaf7cd38 Bug 1341727 - Removed invalid arguments in action callback template. r=dustin,wlach
MozReview-Commit-ID: 1rIhRMjbewL

--HG--
extra : rebase_source : b712dca7aaa923fd3424971d89ef88605ceba6a9
2017-02-24 13:27:20 -08:00
Mihai Tabara
c822738661 Bug 1338186 - assign bucket specific scopes per type of release. r=aki
MozReview-Commit-ID: 7OAIvkaFF2a

--HG--
extra : rebase_source : c85a88fbff77e360531578709de4a39490fcf245
2017-02-24 17:30:46 +02:00
Andrew Halberstadt
2d35842ffc Bug 1336559 - Add ability to depend on build artifacts to 'run_task' based tasks, r=dustin
Currently 'run_task' tasks have no easy way to depend on a build task. For example, some
python unittests need a Firefox binary for their tests, like the mozrunner tests and future
test harness selftests (like mochitest tests).

This patch allows kinds to add a new key to the kind config which maps test platforms to
build-labels. Then 'run_task' jobs can add a 'requires-build': true field to get a build
dependency automatically added. The build artifact url will also be stored in the
$GECKO_INSTALLER_URL environment variable on the test host.

MozReview-Commit-ID: Jqyhbj7nC6z

--HG--
extra : rebase_source : 2f44b6c94f35a0d2e11464cf773e821ae6fe8538
2017-02-24 09:04:22 -05:00
Andrew Halberstadt
d9d7976260 Bug 1336559 - Refactor keyed-by matching algorithm into standalone utility function, r=dustin
MozReview-Commit-ID: Jqyhbj7nC6z

--HG--
extra : rebase_source : 59d0066ca4d02071331a27e9b8e84eb6e675f917
2017-02-24 08:56:11 -05:00
Alin Selagea
a11c397c44 Bug 1339179 - Set up pine for activity-stream builds & tests, r=kmoir 2017-02-24 17:19:04 +02:00
Andrew Halberstadt
2d476494f7 Bug 1336559 - Rename source-check kind to source-test, r=dustin
The name 'source-check' is a bit of a misnomer, because it already includes a bunch
of tasks that are proper unittests, not lints. Some of these unittests will soon
depend on a build task, which makes 'source-check' feel even more wrong.

They still have a lot in common with the lint tasks though, so it's probably not
worth splitting this into two separate kinds. Instead, let's just rename this kind
to 'source-test', which means, any task that tests stuff and is run from the source
directory (instead of a tests.zip). I think both lints and python-tests fall squarely
under this definition.

MozReview-Commit-ID: K0gZ5rVLyeD

--HG--
rename : taskcluster/ci/source-check/doc.yml => taskcluster/ci/source-test/doc.yml
rename : taskcluster/ci/source-check/kind.yml => taskcluster/ci/source-test/kind.yml
rename : taskcluster/ci/source-check/mozlint.yml => taskcluster/ci/source-test/mozlint.yml
rename : taskcluster/ci/source-check/python-tests.yml => taskcluster/ci/source-test/python-tests.yml
rename : taskcluster/ci/source-check/webidl.yml => taskcluster/ci/source-test/webidl.yml
extra : rebase_source : a683b5b6e243849de57f2681993046c776d6a8f2
2017-02-02 16:16:06 -05:00
William Lachance
5b60333fe6 Bug 1322433 - Make it possible to add tags + add a mochitest tag to mochitest jobs r=jonasfj
This will be used to restrict mochitest actions to mochitest jobs only.


MozReview-Commit-ID: DbFb9V6s9Rb

--HG--
extra : rebase_source : 16ebd751bf7048fd46d71bd350119ca3f8a68302
2017-01-25 16:04:37 -05:00
Brian Stack
fe3c438717 Bug 1342264 - Fix case of backfilling with no previous builds r=dustin
MozReview-Commit-ID: FXF7EW8Nygl

--HG--
extra : rebase_source : c28c1d12c37fba2b8633b4cc79a75d15feab7406
2017-02-23 15:13:07 -08:00
Carsten "Tomcat" Book
e6b82618f4 merge mozilla-inbound to mozilla-central a=merge 2017-02-23 14:59:37 +01:00
Wander Lairson Costa
90611046e2 Bug 1339038 - Replicate BB environment variables. r=dustin
There are some environment variables necessary to run test jobs in BB
which are also necessary for taskcluster-worker. They were copied from
[1].

Additionally, native-engine was broken by bug 1325701. We fix it in this
commit.

[1] https://github.com/mozilla-releng/build-buildbotcustom/blob/master/env.py#L89-L96
2017-02-22 14:47:00 -05:00
Aki Sasaki
580bb7d424 bug 1340609 - adjust beetmover+balrog scopes until we're ready. r=mtabara 2017-02-22 16:04:42 -08:00
Wes Kocher
29443ed98c Merge m-c to inbound, a=merge
MozReview-Commit-ID: HDHJC8X9xWG
2017-02-22 15:06:46 -08:00
Wes Kocher
b7294d3167 Merge inbound to central, a=merge
MozReview-Commit-ID: 2R3yE5OIznC
2017-02-22 15:01:04 -08:00
Wes Kocher
5a7f5284cd Merge autoland to central, a=merge
MozReview-Commit-ID: 5l8P3DzMNXz
2017-02-22 14:55:21 -08:00
Aki Sasaki
13a088f827 bug 1340609 - promotable beta linux builds on push. r=dustin a=release
MozReview-Commit-ID: JAqifpgD3vW

--HG--
extra : rebase_source : 4570e7a304ac9fc208a0c2267e352e6cf4e1c80a
2017-02-22 10:24:58 -08:00
Aki Sasaki
344d405df7 bug 1340609 - toggle nightly scopes on-demand. r=dustin a=release
MozReview-Commit-ID: JiBpEWfGefh

--HG--
extra : rebase_source : f78b4bf97a90890d941141cd147f6ab8541ae0ef
2017-02-22 10:50:38 -08:00
Carsten "Tomcat" Book
197fbb6eb9 Backed out changeset d8122277e9fd (bug 1341214) for causing on m-c problems in vp tc tasks 2017-02-22 15:33:42 +01:00
Carsten "Tomcat" Book
42bc96ae3d Backed out changeset 5ffd6c7264ce (bug 1341214) 2017-02-22 15:33:17 +01:00
Carsten "Tomcat" Book
a7cadef96e Merge mozilla-central to mozilla-inbound 2017-02-22 14:40:19 +01:00
Johan Lorenzo
d1cbd41a41 Bug 1337825 - Jamun: Change android-api-15-{l10n,nightly} to l10n bumped file r=aki
Also adds missing files

MozReview-Commit-ID: C0G6bE6wmK4
2017-02-16 18:51:25 +01:00
Wes Kocher
13e1cccb3f Merge inbound to central, a=merge
MozReview-Commit-ID: EEMZdLCheAm
2017-02-21 17:35:01 -08:00
Brian Stack
2d939c8b59 Bug 1340668 - Fix repo scopes in action.json generation r=garndt
MozReview-Commit-ID: CprCcIiqsbH

--HG--
extra : rebase_source : 4df66bf8dea629e78cd878b078355ba9e6b4f0a2
2017-02-17 14:43:59 -08:00
Joel Maher
e1f3393ad5 Bug 1288993 - adjust target tasks to use correct taskcluster attributes. r=dustin
MozReview-Commit-ID: 8g04JU31JLC
2017-02-21 08:43:50 -05:00
Kartikaya Gupta
b10e2495e8 Bug 1341312 - Split jsreftest jobs on Linux64-QR builds into 4 chunks to avoid timeout. r=jmaher
MozReview-Commit-ID: 2sbxIj3JgpW

--HG--
extra : rebase_source : 4540391ac1a2368004dc744f47f253990deab2b4
2017-02-21 10:36:48 -05:00
Wes Kocher
2859ffc86f Merge m-c to autoland, a=merge
MozReview-Commit-ID: E64mguPbLgc
2017-02-21 17:40:31 -08:00
Mike Hommey
3e79befa4b Bug 1341214 - Add a small API to handle taskcluster queue and index requests. r=dustin
Various modules under taskcluster are doing ad-hoc url formatting or
requests to taskcluster services. While we could use the taskcluster
client python module, it's kind of overkill for the simple requests done
here. So instead of vendoring that module, create a smaller one with
a limited set of functions we need.

This changes the behavior of the get_artifact function to return a
file-like object when the file is neither a json nor a yaml, but that
branch was never used (and was actually returning an unassigned
variable, so it was broken anyways).

At the same time, make the function that does HTTP requests more
error-resistant, using urllib3's Retry with a backoff factor.

Also add a function that retrieves the list of artifacts, that while
currently unused, will be used by `mach artifact` shortly.

--HG--
extra : rebase_source : 06777dea62e884f546a5b951baad80fd8aec1f1e
2017-02-17 12:04:48 +09:00
Mike Hommey
f99718e272 Bug 1341214 - Define GECKO in a single location. r=dustin
Instead of every file trying to get the top source directory having an
ad-hoc definition that gets wrong if the files gets moved around for
some reason, define it in a more central location.

--HG--
extra : rebase_source : 06fa06d47732223e19b0201f8791fdbffdc9ee03
2017-02-17 11:56:12 +09:00
Mike Hommey
0982410db2 Bug 1341213 - Make the index path for toolchain jobs independent of the source path. r=dustin
When I refactored hash_paths to add caching, I mixed things up such that
for each file, we end up hashing:
  (u'$sha256sum', u'$topsrcdir/$relpath') $relpath

when the intent was to hash:
  $sha256sum $relpath

This change fixes it, such that now the index paths are independent of
the source path, as originally intended.

--HG--
extra : rebase_source : 8ff7b49927d2365ed87fa06d8e6fca157faddc7d
2017-02-16 17:33:35 +09:00
Wes Kocher
72ecfb5fd6 Merge m-c to autoland, a=merge
MozReview-Commit-ID: 7gVISUXnm2Q
2017-02-22 15:07:12 -08:00
Brian Stack
6d1d9fbf92 Bug 1341704 - Fix assumption of uniqeness of treeherder jobs r=jmaher
MozReview-Commit-ID: EDpsIvF2IEE

--HG--
extra : rebase_source : 8ad736c151553bfd7df935b999aad93536e8603f
2017-02-22 09:33:32 -08:00
Greg Mierzwinski
ba5aaf53e6 Bug 1301984 - Add code coverage to mochitest-plain tests. r=jmaher
This patch allows the use of the flag '--jscov-dir-prefix' for mochitest plain tests to enable code coverage collection with the JS Debugger. It also enables the mochitest-plain tests for the linux64-jsdcov build platform.

MozReview-Commit-ID: 6RqMEZ1I0D7

--HG--
extra : rebase_source : 351754541801f69f7c54807f6bdd3a3d1baf9222
2017-02-20 22:55:59 -05:00
Chris Manchester
8909da0ecd Bug 1340410 - Distinguish gradle builds as their own platform in Treeherder. r=nalexander
MozReview-Commit-ID: LjIp3tY6bjh

--HG--
extra : rebase_source : 53abe6221cb1b063b8d5708d22ad66b788e271bf
2017-02-17 15:47:35 -08:00
Wes Kocher
dda9a2cfb8 Merge m-c to autoland a=merge
MozReview-Commit-ID: 7m1sarjIoLM
2017-02-17 13:50:13 -08:00
Kim Moir
4ecbdfe247 Bug 1339604 - stylo builds + tests should only run on m-c and stylo branch to reduce budget impact r=kmoir a=merge
MozReview-Commit-ID: Lwph8g1ALVt

--HG--
extra : source : 4f8aecb0f44f4801e6c277077e788516e6dd31de
extra : amend_source : 2a26aa2b1904bc1058b41d75283c275570be5e66
2017-02-17 16:38:30 -05:00
Kim Moir
ab418078b0 Bug 1339604 - stylo builds + tests should only run on m-c and stylo branch to reduce budget impact r=bholley a=merge
MozReview-Commit-ID: 2YlYAeoRHXC

--HG--
extra : source : 2fe83738221a0d438ccd33ffe8c9ee3c980c1ab3
2017-02-17 15:29:47 -05:00
Kim Moir
ff5a946da7 Bug 1339604 - stylo builds + tests should only run on limited branches to reduce budget impact r=dustin a=merge
MozReview-Commit-ID: 8SOmDuN6TTL

--HG--
extra : source : 5fbcbdd6849379c48ce869d4ed0e8083af9b8d23
2017-02-17 13:31:39 -05:00
Johan Lorenzo
d1082915e4 Bug 1337825 - Schedule l10n repacks for fennec with specified locale revisions using an intree changesets file r=aki
MozReview-Commit-ID: 1DEIjFYAx74

--HG--
extra : histedit_source : fe73d536f0ae6bf1d60279fec210e16b7ffb913f%2Ca439f62086a4b85a8d46edd0fcf0000bc376f2f0
2017-02-17 15:57:11 +01:00
Carsten "Tomcat" Book
5f96998164 Backed out changeset c7c9f7712163 (bug 1337825) for flake8 failure 2017-02-17 15:52:10 +01:00
Johan Lorenzo
903b044aae Bug 1337825 - Schedule l10n repacks for fennec with specified locale revisions using an intree changesets file r=aki
MozReview-Commit-ID: 1DEIjFYAx74
2017-02-14 15:04:54 +01:00