Commit Graph

2878 Commits

Author SHA1 Message Date
Mike Hommey
34e619d1c1 Bug 1357323 - Remove support for gonk in the build system. r=gps
Everything depending on the widget being gonk can go away, as well as
everything depending on MOZ_AUDIO_CHANNEL_MANAGER, which was only
defined on gonk builds under b2g/ (which goes away in bug 1357326).

--HG--
extra : rebase_source : 9f0aeeb7eea8417fa4e06d662d566d67ecaf2a24
2017-04-18 16:56:09 +09:00
Mike Hommey
75fd170e4c Bug 1357326 - Remove things relying on the b2g* MOZ_BUILD_APP in the build system. r=gps
--HG--
extra : rebase_source : 2471f47ad07d8b90098f90db3ead2e26648cea9f
2017-04-18 18:39:53 +09:00
Mike Hommey
9620fd514e Bug 1356937 - Avoid mach artifact toolchain failing to get taskcluster artifacts when they are not directly under public/. r=gps
--HG--
extra : rebase_source : aeb7a54dea393a97c8597412e45520a2215f9cc5
2017-04-17 09:47:56 +09:00
Chris Manchester
c0c9bf46a0 Bug 1247567 - Include per-source flags in the CompileDB. r=glandium
MozReview-Commit-ID: Ivm4DeG8z94

--HG--
extra : rebase_source : 036629e4aefed04be78452d4c1cb88e8009da35e
2017-04-14 16:02:25 -07:00
Mike Hommey
81ecb91908 Bug 1356976 - Don't make mach artifact toolchain return an error code when there is nothing to download. r=gps
There can be cases where there is simply nothing to download, especially
with the --for-job argument. So we just stop erroring out when nothing
was downloaded.

However, if the user explicitly requested a particular file(s) via the
command line and there is nothing to download, we still emit an error
code.

--HG--
extra : rebase_source : 143c15c9711bbfbbfdc110da14f3738132d53ed4
2017-04-17 16:02:25 +09:00
Mike Hommey
59c62bd61c Bug 1356934 - Don't assume the path to the taskcluster config is relative to topsrcdir. r=gps
--HG--
extra : rebase_source : 4f9f30f563b60649f4fe26836c766e16d4b54491
2017-04-17 09:05:13 +09:00
Alex Gaynor
fcd28c5b2a Bug 1294641 - whitelist reads from the .app directory in the macOS sandbox r=froydnj,haik
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.

r=haik,froydnj

MozReview-Commit-ID: KNvAoUs5Ati

--HG--
extra : rebase_source : 81ba8bfee0ca96979cf8e30d75cdd47f06bc10ea
2017-04-07 14:53:19 -04:00
Sebastian Hengst
e7317789ad Backed out changeset 225683fed1d1 (bug 1294641) for breaking crashtests, reftests and Windows builds. r=backout on a CLOSED TREE 2017-04-17 16:29:52 +02:00
Alex Gaynor
fd11319799 Bug 1294641 - whitelist reads from the .app directory in the macOS sandbox r=froydnj,haik
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.

r=haik,froydnj

MozReview-Commit-ID: KNvAoUs5Ati

--HG--
extra : rebase_source : f637acff32fc8582732de932503dd696abc57877
2017-04-07 14:53:19 -04:00
Mike Hommey
d9e412fe4d Bug 1356524 - Add a mach artifact toolchain option to get artifacts from taskcluster builds. r=gps
Now that we have automated build jobs that produce toolchains, we want
to avoid the burden of uploading them to tooltool and then update the
tooltool manifests. But we don't have build jobs for all the possible
toolchains, so we allow `mach artifact toolchain` to get a mix of
tooltool and taskcluster artifacts.

For taskcluster artifacts, we can give a list of job names (conveniently
automatically normalized to begin with 'toolchain-' and end with '/opt')
for which the artifacts will be downloaded, in place of any tooltool
package with the same name (if a tooltool manifest is given).

The taskcluster artifacts that we download are the ones matching the
contents of the tree the command is run with, per the resources declared
for the corresponding toolchain build job (in
taskcluster/ci/toolchain*.yml)

So for example, a linux64 build could call the following command:
  mach artifact toolchain --tooltool-manifest \
    browser/config/tooltool-manifests/linux64/releng.manifest \
    --from-build linux64-gcc

and get the right gcc corresponding to the build-gcc script in tree,
along with the other non-gcc files from the tooltool manifest.

Things are however planned to be even more convenient, but some commands
can already benefit from this form (even without a tooltool manifest).
See e.g. bug 1328454.

--HG--
extra : rebase_source : b12ed77bef529eb8d67476aceac0166bdfe2eeed
2017-04-14 11:34:53 +09:00
Mike Hommey
29eb575e7e Bug 1356524 - Only add Authorization header when sending requests to the tooltool url. r=gps
We're going to potentially use the same download manager for tooltool
and taskcluster artifacts, and we don't want to send the tooltool
authentication header to the taskcluster requests.

--HG--
extra : rebase_source : 79a0afdbf06cb05d7792f413ab1f6715b2a9fe2e
2017-04-14 10:28:42 +09:00
Gregory Szorc
81cd3ec73b Bug 1334525 - Handle tuple types and WPT manifests correctly; r=chmanchester
beb43155b7a6 changed WPT items to be 3-tuples instead of 2-tuples.
This broke test_defaults_for_path().

MozReview-Commit-ID: 7M0RcQ7bOIU

--HG--
extra : rebase_source : 28e44e5206abb7939d540ba809ec71325ead341c
2017-04-12 15:15:36 -07:00
Mike Hommey
a8cb87ea48 Bug 1356142 - Use mozbuild.util.mkdir instead of os.makedirs for the artifact cache directory. r=gps
At the same time, make the artifact cache directory not indexed, and
move the directory creation to the base classes that actually use it.

--HG--
extra : rebase_source : 62994499afceb5166c0041148dcc702f87166fdc
2017-04-13 16:35:20 +09:00
Mike Hommey
7b4adcc548 Bug 1355731 - Add a mach command wrapping tooltool. r=gps
The ultimate goal is to have a generic command that pulls relevant
toolchains from either tooltool or taskcluster artifacts.

This introduces the command and makes it work to allow to wrap tooltool
in most places where it's used currently, with the ability to replace
tooltool_wrapper.sh as well.

--HG--
extra : rebase_source : f63178282f00f8698a148db484f0276c99d61f7b
2017-04-12 17:42:17 +09:00
Mike Hommey
f3a3f6081e Bug 1355731 - For artifacts cache, use the file name from the url if it looks like a hash digest. r=gps
--HG--
extra : rebase_source : ef48b50df8f1063c8ed9c27ecf04b17388b69512
2017-04-12 16:26:20 +09:00
Mike Hommey
4d1881e4dd Bug 1355731 - Move tooltool to mozbuild.action. r=gps
--HG--
rename : taskcluster/docker/recipes/tooltool.py => python/mozbuild/mozbuild/action/tooltool.py
extra : rebase_source : 6278d8692956a67f37fd788df894657d6eeb53e8
2017-02-22 17:20:41 +09:00
Wes Kocher
20dd5f52c2 Merge m-c to autoland, a=merge 2017-04-13 17:35:34 -07:00
Wes Kocher
514e230373 Merge inbound to central, a=merge 2017-04-13 17:24:01 -07:00
Mike Hommey
aedebf3f44 Bug 1356064 - Remove the mach artifact print-cache subcommand. r=nalexander
--HG--
extra : rebase_source : b044acc420285ec050196efdf737e59d60269991
2017-04-13 09:50:57 +09:00
Mike Hommey
a3c516ed4e Bug 1356064 - Remove the mach artifact last subcommand. r=nalexander
It has been broken for a while (bug 1341205).

--HG--
extra : rebase_source : 85feaf001545642342c54af856fd4dfe6828b35d
2017-04-13 09:47:27 +09:00
Gregory Szorc
f2e9cb2310 Bug 1355630 - Inline specialized version of relpath(); r=chmanchester
Profiling revealed that mozpath.relpath() accounted for a lot of CPU
time when operating on an input of ~42,000 paths.

Due to the nature of the paths we're operating on, we don't need the
full power of mozpath.relpath() here. Instead, we can implement a
specialized version that works given already normalized paths and the
knowledge that context paths must be ancestors of the current path
being examined.

This change drops execution time of a mach command feeding ~42,000
paths to this function from ~90s to ~24s. On an input with 9131 paths,
execution time dropped from ~8.8s to ~3.7s.

MozReview-Commit-ID: EGLiJa10Zj2

--HG--
extra : rebase_source : 8e9d7a957ea3d865a6f18a3b99cca2b22f7a0385
2017-04-11 16:46:55 -07:00
Gregory Szorc
009ab65975 Bug 1355630 - Memoize tests_defaults_for_path call; r=chmanchester
Calling self.test_defaults_for_path() from files_info() with tens
of thousands of paths resulted in a CPU explosion in various path
normalization functions. I don't think it was so much the complexity
of the operations as much as the volume.

For an input with 9131 elements, this reduces execution time of a
mach command from ~25.7s to ~8.8s. With ~42,000 inputs, execution time
drops from <it took too long and I gave up> to ~90s.

MozReview-Commit-ID: pjQQByi2Bc

--HG--
extra : rebase_source : b8307a92e494f128ba3cf29348bc7996cfea4221
2017-04-11 16:24:42 -07:00
Gregory Szorc
4a47739a31 Bug 1355630 - Use a set for membership testing; r=chmanchester
With ~42,000 entries in relpaths, this change drops execution time
of this loop from ~23s to ~0.01s.

MozReview-Commit-ID: Afm245tjWUQ

--HG--
extra : rebase_source : 86ee525b0539c80e47e4326a3f111a3fc84d0a6e
2017-04-11 15:25:13 -07:00
Nathan Froyd
2989feb7d7 Bug 1353810 - add a --enable-rust-debug option; r=chmanchester
For people working on Rust code, compiling in debug mode (Cargo's "dev"
profile) is convenient: debug assertions are turned on, optimization is
turned off, and parallel compilation inside of rustc itself can be
used.  These things make the build faster and the debugging experience
more pleasant.

To obtain that currently, one needs to --enable-debug at the Gecko
toplevel, which turns on debug assertions for the entire browser, which
makes things run unreasonably slowly.  So it would be desirable to be
able to turn *off* debug mode for the entirety of the browser, but turn
on debug mode for the Rust code only.

Hence this added switch, --enable-rust-debug, which does what it
suggests and defaults to the value of --enable-debug.  For our own
sanity and because we judge it a non-existent use case, we do not
support --enable-debug --disable-rust-debug.
2017-04-12 21:49:25 -04:00
Wes Kocher
2986c817dc Merge autoland to central, a=merge CLOSED TREE 2017-04-12 15:13:30 -07:00
Axel Hecht
d5c1373ef0 bug 1344782, update compare-locales to 1.2.3, r=flod
Dropped support for python 2.6
Removed support for bookmarks.html
Better internal APIs

Intermediate versions had incompatibilities with old versions
of python 2.7, but 1.2.3 does not.

MozReview-Commit-ID: LfKhr8qWe28

--HG--
extra : rebase_source : 21fc39d45f22f4a6ae10523a87b0e6068f268439
2017-04-12 15:26:40 +02:00
Petr Sumbera
0e17adee7c Bug 1354510 - Firefox build should recognize Solaris. r=glandium
--HG--
extra : rebase_source : 2bec2709a26a5295e723b0ecc424b0d648715bb7
2017-04-11 00:01:32 -07:00
Mike Shal
5b62b8c036 Bug 1337145 - Remove python components used for EME generation; r=glandium
MozReview-Commit-ID: G8OhcbStQkG

--HG--
extra : rebase_source : ad7434cf729c3257bcc3911a4d25ba0145384939
2017-04-10 13:40:03 -04:00
Chris Manchester
5c47bb8d2e Bug 1337986 - Dump symbols during the compile tier. r=ted
This commit moves symbol dumping to the compile tier, to be run via "syms"
targets. Tracking files are used for the sake of incremental builds, because
dump_syms may genearate multiple outputs whose paths are not known ahead of
time.

Minimal changes to symbolstore.py are made here. More extensive
simplifications will be made in a future commit on the basis of symbolstore.py
handling one file at a time.

MozReview-Commit-ID: 3mOP8A6Y7iM

--HG--
extra : rebase_source : bfe97afcdfc05b9e79f01577701c83e8b00eb4e9
2017-02-10 08:34:08 -08:00
Chris Manchester
4db8e0b54a Bug 1337986 - Generate "syms" targets for directories containing programs or shared libraries. r=glandium
This creates "syms" targets that depend on the corresponding "target" for
directories containing shared libraries or programs. These targets are added
to the main compile graph in automation, and can be invoked through a special
"symbols" target. A future commit will use these targets to dump symbols for
shared libraries and programs during the compile tier.

MozReview-Commit-ID: KLuvmqsK4Zj

--HG--
extra : rebase_source : 8d76b999cb6fac8f11168ac6ebfb58774dfc2d3c
2017-04-10 10:27:49 -07:00
Simon Sapin
8f7b8a3663 Bug 1352815 - "codegen-units = 4" for Rust code in debug mode r=froydnj
MozReview-Commit-ID: IIFVmijY4B3

--HG--
extra : rebase_source : 1722dba55a1759f15b82532c13f1e61affd28e9b
2017-04-02 14:17:44 +02:00
Sebastian Hengst
548852b3a4 merge mozilla-central to autoland. r=merge a=merge 2017-04-07 22:16:26 +02:00
Carsten "Tomcat" Book
3a762886de Merge mozilla-central to mozilla-inbound 2017-04-07 15:33:03 +02:00
Michael Layzell
981693a30f Bug 1320179 - Part 2: Add the nserror rust crate and generate NS_ERROR codes, r=froydnj
MozReview-Commit-ID: FFU6WfEqev
2017-04-06 18:37:25 -04:00
Ralph Giles
1d688bb86c Bug 1354221 - Remove dead code in _process_rust_library. r=froydnj
These variables are ignored in favour or reading the same values
from the passed in libdef object.

MozReview-Commit-ID: 8Xkkd68clNN

--HG--
extra : rebase_source : 767c15f1b59a6f48d09411c4f2e64eff9f8b4cc6
2017-04-06 10:59:46 -07:00
Locke Chen
29b6845e6b Bug 1350229 - Removing Preview side panel. r=rickychien
MozReview-Commit-ID: aDksj7VKAp

--HG--
extra : rebase_source : b0c19cc8fc7b41fed7909f3a1ee56a16ee0f4468
2017-04-07 23:31:18 +08:00
Michael Layzell
fb7949544e Bug 1347290 - Set sgr0 to '' if blessings.tigetstr('sgr0') returns None, r=gps
MozReview-Commit-ID: vP4izM2RwP
2017-04-06 17:44:30 -07:00
Utkarsh Anand
ecd3979b25 Bug 1353460 Asked the user to restart the shell before building from source. r=rillian
When downloading rust manually, it's mandatory to restart the shell,
as the script does not export the PATH environment variable.
This fix also ensures that the rust version in PATH is modern enough for a successful build.

MozReview-Commit-ID: HdLpiLPBLW7

--HG--
extra : rebase_source : dc087c36e89fd6d2957fd1e03af42229de914d5a
2017-04-06 08:27:57 +05:30
Ting-Yu Chou
e529a580e1 Bug 1333003 part 5 - Include ASan runtime dll in common.tests.zip. r=ted
MozReview-Commit-ID: AzPdxaWwJn4

--HG--
extra : rebase_source : ce80d8569b7abc3afe3ca4bba653f030d16d8ffc
2017-03-10 12:09:17 +08:00
Chris AtLee
b7af545070 Bug 1351071: Get rid of pre-generated startup cache r=glandium
MozReview-Commit-ID: BcWcqEKwGBv

--HG--
extra : rebase_source : 4bd21805a89359a5a2ed7ffd2ee2b76cf1e9e634
2017-04-04 09:26:25 -04:00
Carsten "Tomcat" Book
14e0b51ace merge mozilla-inbound to mozilla-central a=merge 2017-04-04 12:35:03 +02:00
Justin Wood
71acec2f03 Bug 1185666 - Move DMG unpack logic to a python script, support linux. r=mshal
MozReview-Commit-ID: inKT2BWof4

--HG--
extra : rebase_source : 3fa0861bd363a8b22002111fe55ab2d248d85d6c
extra : source : e1df7b730858ba0170004a5b705f2bc484f9e2b0
2017-03-28 21:46:19 -04:00
Axel Hecht
b1ae66a84b bug 1336406, update http -> https in compare-locales docs, r=aryx
Ride-along fix to change http://mxr to https://dxr, too.

MozReview-Commit-ID: 2H6aGv9nuKv

--HG--
extra : rebase_source : 63e83e193747f6b56d7bee9a368d200eac15bc86
2017-04-03 14:55:28 +02:00
Chris Manchester
4ada59b662 Bug 1349640 - Upload a mapping for headers in dist/include for the benefit of code coverage builds. r=mshal
MozReview-Commit-ID: 5q9I5S1QOt9

--HG--
extra : rebase_source : e893e9b2ea1ef2cbfe1cfd8e92af71e79f1615a2
2017-03-29 11:43:16 -07:00
Mike Hommey
d559ffb37a Bug 1343718 - Limit artifact downloads by size rather than by number. r=chmanchester
This adds a unit test for the expected behavior, and adds the modules
from mach_bootstrap.py that are missing in the virtualenv for the test
to run.

--HG--
extra : rebase_source : e34d0474cfb6c8c5ce9cd847b96de88906191923
2017-04-12 16:17:52 +09:00
Mike Hommey
a07b022268 Bug 1343718 - Don't display download progress when we don't know the download size in advance. r=chmanchester
In most cases, the HTTP response for the download will contain the
content-length, but if some error happens (e.g. authentication error),
there might not be one, and the download fails with a TypeError for a
division by None, instead of failing with a more friendly error message
about the HTTP error.

--HG--
extra : rebase_source : f179142e69c9ca09b05ad2ff942753fd84da6a69
2017-03-01 11:16:27 +09:00
Mike Hommey
2f47399ef6 Bug 1343718 - Avoid requiring an objdir when simply importing mozbuild.artifacts. r=chmanchester
--HG--
extra : rebase_source : 99daff7ae1e2d1072f19c96a3c1777498e63342f
2017-03-01 11:15:15 +09:00
Mike Shal
306a525fa2 Bug 1347576 - Add a 'mach repackage' command, with OSX dmg support; r=chmanchester
This is the initial support of 'mach repackage', which can take an
existing tarball and create a DMG on either an OSX host or on a Linux
host with cross-OSX tools. Configure is needed in order to find the
tools necessary to create the DMG. On a Linux cross-compiled environment
with tooltool, this can be as simple as:

export MKFSHFS=$topsrcdir/hfsplus-tools/newfs_hfs
export DMG_TOOL=$topsrcdir/dmg/dmg
export HFS_TOOL=$topsrcdir/dmg/hfsplus
ac_add_options --disable-compile-environment

MozReview-Commit-ID: 6t2rlXpwUvu

--HG--
extra : rebase_source : 39f2f9ecac9d7e5af197f1b5dea70cd307acf488
2017-03-10 18:36:23 -05:00
Dan Minor
961b4550db Bug 964133 - Add mach command for webrtc.org unit tests; r=chmanchester
MozReview-Commit-ID: HJ1eNdaqIQM

--HG--
extra : rebase_source : 85dfdfe736aa35a11ac5c70e1076f59744714b63
2017-01-26 13:23:44 -05:00
Wes Kocher
93d11e3441 Backed out 7 changesets (bug 1333003) for windows asan failures a=backout
Backed out changeset 3d2b2eeda8d3 (bug 1333003)
Backed out changeset 400d409ba4ca (bug 1333003)
Backed out changeset 1ba027abdfc9 (bug 1333003)
Backed out changeset 70114135bd8c (bug 1333003)
Backed out changeset 5715b15e33c0 (bug 1333003)
Backed out changeset 375e952bd738 (bug 1333003)
Backed out changeset d5d4112599f2 (bug 1333003)

MozReview-Commit-ID: DZUHJTdjX7V
2017-03-23 11:01:44 -07:00
Arshad Khan
47d2c3993b Bug 906177 - Added distro support for Elementary OS Luna;r=gps
MozReview-Commit-ID: FtHmXIFtzIz

--HG--
extra : rebase_source : 1d37d38f7635cbe6c1d30669240a613b52a62ed2
2017-03-23 16:39:40 +05:30
Carsten "Tomcat" Book
187105652e Merge mozilla-central to autoland 2017-03-23 13:59:20 +01:00
Ting-Yu Chou
6e4ca4a367 Bug 1333003 part 5 - Include ASan runtime dll in common.tests.zip. r=ted
MozReview-Commit-ID: AzPdxaWwJn4

--HG--
extra : rebase_source : ce80d8569b7abc3afe3ca4bba653f030d16d8ffc
2017-03-10 12:09:17 +08:00
Steve Fink
1d35364a0b Bug 1330746 - Disable terminal-style output when running within emacs, r=qdot
MozReview-Commit-ID: CgjgL91alUa

--HG--
extra : rebase_source : ba00d397e310d1f3159ff2239ebc3c158184305c
2017-03-22 14:03:53 -07:00
Carsten "Tomcat" Book
947874f380 Backed out changeset 42cf5ddabc8a (bug 1333003) 2017-03-23 10:38:09 +01:00
Ting-Yu Chou
d0375ffc60 Bug 1333003 part 5 - Include ASan runtime dll in common.tests.zip. r=ted
MozReview-Commit-ID: AzPdxaWwJn4

--HG--
extra : rebase_source : d2e68f52cc9580a02790513733222405d1e17b56
2017-03-10 12:09:17 +08:00
Edmund Wong
f2596e1f77 Bug 1345422 - Do not hardcode the sfx package and installer path. r=mshal 2017-03-22 08:52:35 +08:00
Carsten "Tomcat" Book
023d5aecd1 Merge mozilla-central to mozilla-inbound 2017-03-16 15:39:58 +01:00
Nathan Froyd
53fd6ce968 Bug 1316990 - part 2b - add initial set of approved licenses for Rust packages; r=gerv
This commit adds the initial whitelist of (mostly) SPDX license
identifiers and the initial whitelist of separately-specified license
files for those packages that require it.  For gamma-lut, the license
that the hash identifies is:

https://hg.mozilla.org/mozilla-central/file/47611a305c44/third_party/rust/gamma-lut/LICENSE

For deque, the license that the hash identifies is:

https://hg.mozilla.org/mozilla-central/file/47611a305c44/third_party/rust/deque/LICENSE-MIT

Integrating any new licenses into about:license is left as separate work
for the time being.
2017-03-16 10:00:02 -04:00
Nathan Froyd
6374bdf4f1 Bug 1316990 - part 2a - check licenses of vendored Cargo packages; r=rillian
This commit adds the core support to `mach vendor rust` to verify that
the licenses of vendored packages are OK for distribution in Firefox.
2017-03-16 10:00:02 -04:00
Nathan Froyd
8234066ae9 Bug 1316990 - part 1 - split out an _ensure_cargo function; r=rillian
We're going to be adding some more code to the vendor function, and
splitting this function out helps make vendor a little more manageable.
2017-03-16 10:00:02 -04:00
David Major
ed12ea611f Bug 1346078: Remove nsAString_internal and just use the nsAString name directly. r=bsmedberg
MozReview-Commit-ID: DWDEDMIVKm7

--HG--
extra : rebase_source : 7fe8dc07c816dca234c67761e61cdee372a87e97
2017-03-10 15:17:23 +13:00
Paul Yang
8d0339f0a5 Bug 1272113: Packaging awsy in build job r=jmaher
MozReview-Commit-ID: 9VhsVIaaWqt

--HG--
extra : rebase_source : ce4768033b0f3a912be9790b5ebe06fa6a2a0803
2017-03-11 02:33:19 +08:00
Mike Hommey
1b1fb10914 Bug 1346381 - Strip executables in the jsshell package. r=mshal
In bug 1335309, FileFinder was made to default to not find executables,
and zip.py was made to use the default instead. Which made sense for
most uses of zip.py, except for the jsshell package.

So we add a flag to make zip.py able to strip executable (which happens
when the FileFinder is made to find them), and use that flag for the
jsshell package only.

--HG--
extra : rebase_source : 0202f9acd5e6175d3790aaef026e18c6913cf0c6
2017-03-14 15:22:39 +09:00
suman gupta
0539e340b8 Bug 1346284 Updated ndk r11c to be installed by mach bootstrap; r=nalexander
MozReview-Commit-ID: KgM9ecZR0fB

--HG--
extra : rebase_source : 9dd666e1fc12d455b4e0c58b5af17d5bd7f35930
2017-03-13 23:47:23 +05:30
Wes Kocher
3842d8fcef Merge inbound to central, a=merge
MozReview-Commit-ID: 62e9XDudUlc
2017-03-08 17:07:55 -08:00
Joel Maher
209b6df381 Bug 1344891 - add BUG_COMPONENT to python/* files, r=ted
MozReview-Commit-ID: Kv8icpC7B4x
2017-03-08 11:17:12 -05:00
Chris Manchester
885ee4b986 Bug 1314678 - Use hfs and dmg tools rather than mozinstall to unpack downloaded builds during cross artifact builds. r=mshal
Mozinstall uses hdiutil, which doesn't exist on cross osx-linux builders.

MozReview-Commit-ID: A4KAIFUp4Gj

--HG--
extra : rebase_source : 6d9ea01ec8bbe37d22c07146a1285f128f5217cd
2017-03-07 10:21:34 -08:00
Andrew Halberstadt
430f043d9e Bug 1345109 - Delete 'JS_STANDALONE' from EmptyConfig when generating test metadata, r=mshal
EmptyConfig objects set JS_STANDALONE=1 by default. However, test tasks that need to run without an objdir
need to be behind an "if not CONFIG['JS_STANDALONE']" condition to avoid causing bustage to sm-pkg task (js
packaging). This patch explicitly deletes that default value, only when generating the TestManifestBackend.

Ideally, the js/src packaging should have their own moz.build instead of re-using the root moz.build. But this
is an easier fix in the short term to get the marionette-harness tests working again.

MozReview-Commit-ID: 26lHLY6WlZK

--HG--
extra : rebase_source : 9c2ffdd938f2f2d6ead7d2aead610a7028e18d97
2017-03-07 10:56:22 -05:00
Andrew Halberstadt
a9859900b2 Bug 1345109 - Ensure |mach python-test| errors out if no tests are found, r=mshal
Because test_objects was a generator, using it in the condition always returned True,
even if no tests were found. But extending test_objects to the manifest, converts it
to a list. So this patch simply moves the 'no tests' check a bit later on.

MozReview-Commit-ID: JpETWD1WQWH

--HG--
extra : rebase_source : 874d7a17d4dfa4a9de4f9daf54b51d1763d55044
2017-03-07 10:55:07 -05:00
Rob Thijssen
5c5c4d5a75 bug 1287881 - windows repackage refactored as actions; r=mshal
MozReview-Commit-ID: 2vI6LlMVvLE

--HG--
extra : rebase_source : e9c812e2bbf77ff8d6aaafd3dba9ff23368d47e1
2017-02-20 16:11:47 +00:00
Wes Kocher
9746f81e53 Merge m-c to autoland, a=merge
MozReview-Commit-ID: FzykkWPQCrq

--HG--
rename : browser/modules/test/browser_UsageTelemetry_content.js => browser/modules/test/browser/browser_UsageTelemetry_content.js
rename : browser/modules/test/browser_UsageTelemetry_content_aboutHome.js => browser/modules/test/browser/browser_UsageTelemetry_content_aboutHome.js
rename : browser/modules/test/browser_UsageTelemetry_searchbar.js => browser/modules/test/browser/browser_UsageTelemetry_searchbar.js
rename : browser/modules/test/browser_UsageTelemetry_urlbar.js => browser/modules/test/browser/browser_UsageTelemetry_urlbar.js
2017-03-01 17:48:44 -08:00
Andrew Halberstadt
6b9bf4adbf Bug 1335873 - Convert marionette harness unittests to standard python unittests, r=maja_zf
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette

They will also run if running the full suite.

The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.

MozReview-Commit-ID: Ata99evHxbd

--HG--
extra : rebase_source : 22382e3d65ce8454a1682cfced0d03477762e8fe
2017-02-15 16:38:45 -05:00
Bob Owen
d30aee57bf Bug 1339729: Remove wow_helper from Windows process sandboxing. r=glandium 2017-03-01 10:41:07 +00:00
Mike Hommey
2be7f1e067 Bug 1341215 - Remove the dependency on the python taskcluster client module for mach artifact. r=chmanchester
This also removes the need for a virtualenv.

--HG--
extra : rebase_source : 867a1b12bf6b57a8d9b6b2428bde637306efe31f
2017-02-21 15:32:28 +09:00
Mike Hommey
142fc24229 Bug 1341215 - Remove the dependency on the mozregression module for mach artifact. r=chmanchester
This imports two modules from mozregression in the tree to do so. They
are imported from current trunk on github, rather than the version we
were getting from pypi.

Note we take six from testing/web-platform/tests/tools/six) instead of
moving it to python/six because it's there by coming from a copy of
https://github.com/w3c/wpt-tools, which contains it as a submodule, and
moving it would make updates there harder.

--HG--
extra : rebase_source : 16619d1c3f38f6493fb490a64361b3f4e8fecc1f
2017-02-21 16:10:54 +09:00
Mike Hommey
f98d09ab4c Bug 1341215 - Vendor dlmanager. r=chmanchester
from https://github.com/parkouss/dlmanager

Note this technically should come before the first patch, but mozreview
won't show useful interdiffs if I do that, so I'll reorder the patches
before landing.

--HG--
extra : rebase_source : dedca9393783623461509c7c85e35302f4b08a2a
2017-02-23 15:30:46 +09:00
Kartikaya Gupta
c4984e8f16 Bug 1336555 - Ensure that cargo-vendor is new enough when running mach vendor rust. r=ted
MozReview-Commit-ID: 3wQ5bA7ZpS5

--HG--
extra : rebase_source : d50302bad066b19afb88bf022b36a8ac6ff67cde
2017-02-24 13:14:16 -05:00
Carsten "Tomcat" Book
7256146318 merge mozilla-inbound to mozilla-central a=merge 2017-02-24 11:46:38 +01:00
Kris Maglione
66ed43fcda Bug 1341502: Fix automatic regeneration of backend when an included test manifest changes. r=mshal
MozReview-Commit-ID: 1ZQNQpqboz9

--HG--
extra : rebase_source : 100dea07ac8f6457dc7c2a189830bf05d834b4c0
extra : source : 4188ae2c8c0ced71fd13dbd78a1bdf46fefa2b9c
2017-02-21 17:31:21 -08:00
Nathan Froyd
50ad8956c0 Bug 1302704 - part 6 - enforce identical profiles for gkrust and gkrust-gtest; r=chmanchester
Cargo hashes various compilation settings into the dependency graph for
dependent libraries.  So if the compilation settings for gkrust and
gkrust-gtest are different, their dependencies will likewise be
different.  The setup we've created in the previous patches depends on
the compilation settings being identical, so we should enforce that at
the moz.build level.
2017-02-23 10:35:06 -05:00
Nathan Froyd
a13c8b0b74 Bug 1302704 - part 3 - propagate information about CARGO_TARGET_DIR from the frontend into the backend; r=chmanchester
Rust libraries can set RUST_LIBRARY_TARGET_DIR so that they can share
compilation artifacts with other libraries.  This setting needs to be
propagated to the backend so it can be communicated to Cargo.
2017-02-23 10:35:07 -05:00
Nathan Froyd
22a91bfe53 Bug 1302704 - part 2 - rename cargo_target_directory; r=chmanchester
We're going to be introducing a value for CARGO_TARGET_DIR in a later
patch, we we should rename this function to not conflict with that
concept.
2017-02-23 10:35:07 -05:00
Nathan Froyd
1cbbd567b5 Bug 1302704 - part 1 - define CARGO_TARGET_DIR in the backend; r=chmanchester
CARGO_TARGET_DIR is currently hardcoded to the current directory, but
we'd like the ability to choose a value for Rust libraries.
2017-02-23 10:35:07 -05:00
Mike Hommey
4f2c73ffd9 Bug 1341207 - Use MachCommandConditions.is_{git,hg} for mach clobber python. r=chmanchester
--HG--
extra : rebase_source : 4b78d3857cca67455343847ab27902c5877e0a6e
2017-02-21 16:04:08 +09:00
Mike Hommey
3a2a54e7d2 Bug 1339240 - Detect git worktrees in MachCommandConditions.is_git. r=chmanchester
--HG--
extra : rebase_source : 1763953383ead9f9985fdadc0e862ace23cb2b21
2017-02-21 15:53:27 +09:00
Sebastian Hengst
11514f09be Backed out changeset e4e02bf4f094 (bug 1341502) for failing TestTestManifestBackend.test_test_manifest_sources on Windows. r=backout on a CLOSED TREE 2017-02-23 21:42:37 +01:00
Kris Maglione
a46cc7117e Bug 1341502: Fix automatic regeneration of backend when an included test manifest changes. r=mshal
MozReview-Commit-ID: 1ZQNQpqboz9

--HG--
extra : rebase_source : 6e5a4f6f037d00ea59a28d392060059f1b38b365
2017-02-21 17:31:21 -08:00
KuoE0
2fb7487fe1 Bug 1340379 - Remove hardcoded Homebrew URL of autoconf 2.13 for macOS. r=nalexander
MozReview-Commit-ID: 2QRkVCOFTUL

--HG--
extra : rebase_source : 0544def007eeb6ec1ada1cbd0a58aa297bc5b711
2017-02-17 11:14:29 +08:00
Wes Kocher
dda9a2cfb8 Merge m-c to autoland a=merge
MozReview-Commit-ID: 7m1sarjIoLM
2017-02-17 13:50:13 -08:00
Nathan Froyd
6bf34c2a33 Bug 1340401 - add back definition of FILESIZE_LIMIT; r=obvious-bustage 2017-02-17 10:18:48 -05:00
Andrew Halberstadt
8d7db75ee7 Bug 1340162 - Add task for running mozlint unitttests on Linux (and pull old tests out of make check), r=smacleod
MozReview-Commit-ID: 3XCWMJtQMvZ

--HG--
extra : rebase_source : 290c14cda5b7abf6aea43315fc91c0ac8fc869f0
2017-02-02 11:08:41 -05:00
Jan Beich
5e185e524c Bug 1320940 - rustup is N/A on Tier3 platforms, so prefer downstream Rust packages. r=rillian
MozReview-Commit-ID: NsRGdweHuF

--HG--
extra : rebase_source : 1cdc69999b5047b8e03bc12006b9fb8d09cfaa20
2016-11-29 14:27:46 +00:00
Sebastian Hengst
964fc701a8 Backed out changeset 3c0a7527608a (bug 1335873) 2017-03-01 17:35:06 +01:00
Andrew Halberstadt
0fa9f19bdf Bug 1335873 - Convert marionette harness unittests to standard python unittests, r=maja_zf
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette

They will also run if running the full suite.

The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.

MozReview-Commit-ID: Ata99evHxbd

--HG--
extra : rebase_source : 16ed70edd38a53c3279d8632d7cba3df4d5216c3
2017-02-15 16:38:45 -05:00
Ting-Yu Lin
cca57e34fe Bug 1341962 - Update ccache stats parser for version 3.3.3. r=glandium
The "unsupported code directive" is added to the 'ccache -s' output in
b6d7cf5502

We need to teach our parser for it.

MozReview-Commit-ID: IrrJv7I7BVa

--HG--
extra : rebase_source : 025167f75ce8486287d408ccdb3d8113450354cb
2017-02-23 11:35:39 +08:00
Iris Hsiao
cde2364468 Backed out changeset 2a15e34e2d22 (bug 1341962) for build bustage 2017-03-01 11:54:27 +08:00
Ting-Yu Lin
d34b8cebe3 Bug 1341962 - Update ccache stats parser for version 3.3.3. r=glandium
The "unsupported code directive" is added to the 'ccache -s' output in
b6d7cf5502

We need to teach our parser for it.

MozReview-Commit-ID: IrrJv7I7BVa

--HG--
extra : rebase_source : b3bb6de344e26e7a62fc59f899c45168bafca4ef
2017-02-23 11:35:39 +08:00
Mike Shal
ad087668b7 Bug 1339182 - Remove OSX universal support in the build system; r=glandium
This removes the UNIFY_DIST and UNIFIED_BUILD variables, as well as the
--unify flag from the packager and UnifiedBuildFinder from mozpack. As a
result the STAGEPATH variable is never defined anymore, so its uses can
be removed as well.

test_unify.py is currently the only mozbuild/mozpack test that fails
without running configure first, and there isn't much point in fixing
tests for things that we don't actually use anymore.

MozReview-Commit-ID: F5q1FPW3Did

--HG--
extra : rebase_source : cadbd237f51c23ea1983135294521d628d16f0df
2017-02-10 16:52:17 -05:00
Mike Shal
e86326d270 Bug 1338559 - Use the tier system to recurse for 'make check'; r=glandium
This helps us avoid recursing over every directory when we only need to
run 'make check' in a select few.

MozReview-Commit-ID: BJ3hJBOneIz

--HG--
extra : rebase_source : 2493f924b9ccba3c779e512d7a8b7a2c26f43797
2017-02-03 16:47:28 -05:00
Andrew Halberstadt
a2025d4c9c Bug 1003417 - Add ability to run subsuites to |mach python-test|, r=ted
This adds the ability to use manifestparser subsuites to |mach python-test|.
Subsuites are based on the premise of a "default" set that gets run when no
subsuites are explicitly specified. When a test is labelled with a subsuite,
that test is removed from the default set and will only run if that subsuite
is explicitly specified. This will allow us to chunk python unittests out of
'make check' piecemeal. The default set will run in 'make check', and
individual tasks (e.g mozbase), will specify a subsuite explicitly.

The |mach python-test| implementation is slightly different. By default,
subsuites are not considered if developers do not pass in --subsuite. This
means running |mach python-test| without arguments will still run the full set
of tests, and similarly, passing in test paths will *just work*.

If for some reason a developer needs to actually run the default set, a special
"default" subsuite has been create, so they can use
|mach python-test --subsuite default|. This default subsuite is also what 'make
check' will explicitly invoke.

MozReview-Commit-ID: FaHb4nvuoK9

--HG--
extra : rebase_source : 2ecbc902bb6bafa7cd78ac0e380a10bad7c14351
2016-11-17 16:30:33 -05:00
Andrew Halberstadt
ea89582f61 Bug 1320194 - Generate all-tests.pkl and related files when resolving tests r=mshal
This replaces the 'run-tests-deps' make target with a python function that will directly
read moz.build files, emit them with TestManifestEmitter, then consume them with
TestManifestBackend. Because the TestResolver is the only place that actually reads the
test metadata files, we can remove this logic from the CommonBackend as well.

MozReview-Commit-ID: DXgMoeH5dKf



MozReview-Commit-ID: HstZ57qkqf2

--HG--
extra : rebase_source : f377fa6863ef66d3adb86ed64f844e346686862f
2017-02-01 09:56:09 -05:00
Andrew Halberstadt
eb228a736f Bug 1320194 - Add ability to specify custom emitter function in TreeMetadataEmitter r=gps
Currently, the only way to emit objects after reading moz.build, is to emit everything. Though, sometimes
it may be desirable to only emit certain types of objects. This adds a new argument that allows consumers
to specify a custom emitter function. This gives them the flexibility to do whatever they want.

This will be used when resolving tests, so only TestManifest objects are emitted.

MozReview-Commit-ID: DPGgNmn2JvE

--HG--
extra : rebase_source : 044f8c8cad1bf7fd1dfbed05c89fbd114508182a
2017-01-27 11:54:09 -05:00
Andrew Halberstadt
ffc74847e7 Bug 1320194 - Fix bug preventing more than two backends in a HybridBackend r=gps
This is a drive by fix that is not relevant to the rest of the commit series.

MozReview-Commit-ID: Bwrb74o3Qh8

--HG--
extra : rebase_source : 34706a6640a6c704f9c03f24aede71e4c1c5ac1c
2017-01-27 16:31:15 -05:00
Andrew Halberstadt
1f4ea1e485 Bug 1320194 - Refactor test metadata related backend code into a partial TestManifestBackend r=gps
Currently the CommonBackend is responsible for processing TestManifest objects and using them to generate
the test metadata files (e.g all-tests.pkl et al). This patch pulls that logic out into a partial backend
specifically for test manifests.

This patch is solely a refactoring and shouldn't change any build behaviour. CommonBackend has a
TestManifestBackend instance and calls consume_object directly on it. However, this is just a temporary
measure to avoid checking in a broken commit.

This commit also adds a test for the 'test-defaults.pkl' file which was previously missing.

MozReview-Commit-ID: HOr2QVT8CJ1

--HG--
extra : rebase_source : e4b9cb982937381346c0821971191277173b165b
2017-01-27 11:47:34 -05:00
Carsten "Tomcat" Book
ed3d8d8286 merge mozilla-inbound to mozilla-central a=merge 2017-02-15 12:21:44 +01:00
Nathan Froyd
a0e43ae419 Bug 1306078 - part 3 - make mach vendor rust check for overly-large files; r=ted.mielczarek
When vendoring third-party files, we'd like an explicit notice/review
when said files contain a "large file".  This commit adds such checks
for files vendored via `mach vendor rust`.

As we don't yet have a server-side hook in place to prevent large files
from being added, we just have a command-line flag that people are
expected to use, on the honor system, to permit large files to be added
when vendoring.
2017-02-14 16:12:19 -05:00
Nathan Froyd
f466efaa92 Bug 1306078 - part 2 - add Repository.get_added_files; r=ted.mielczarek
Figuring out the files that have been added is also something that you
want to do with a source code repository.
2017-02-14 16:12:19 -05:00
Nathan Froyd
7e71a9a29e Bug 1306078 - part 1 - add forget_add_remove_files to mozversioncontrol Repository objects; r=gps
This command is useful for users who wish to perform something like the
following:

1. Add/remove a bunch of files;
2. Examine the additions/removals/modifications for interesting changes;
3. Reject the add/remove if it doesn't meet some set of conditions.
2017-02-14 16:12:19 -05:00
Wes Kocher
5583be3ddf Merge m-c to autoland, a=merge
MozReview-Commit-ID: KkIFx9ndC8y
2017-02-14 14:46:45 -08:00
Ralph Giles
e44edf88cc Bug 1339409 - mozboot: Fix undefined FileNotFoundError. r=froydnj
The FileNotFoundError built-in exception is only present in
python 3. Emulate its behaviour in python 2 with a conditional
OSError.

MozReview-Commit-ID: 4b8THPG7jph

--HG--
extra : rebase_source : 718bf3659f14bd349d052d43bf3197dfbb4a016f
2017-02-14 11:41:04 -08:00
Gabriele Svelto
276f7423bb Bug 1310703 - Introduce the pingsender executable; r=ted
MozReview-Commit-ID: 2SEwwGywahQ
2017-01-16 17:39:22 +01:00
Ralph Giles
d9052fefa0 Bug 1337153 - Require rust 1.15.1. r=Ehsan
Bump the minimum version of the rust toolchain we require to
build. The 1.15 release includes support for custom #[derive]
directives, letting us use the serde serialization crate without
checking in a lot of generated code.

This is primarily motivated by webrender and the audio remoting
work, and lets us drop the heavy syntex dependency.

MozReview-Commit-ID: 6IObHhouPAn

--HG--
extra : rebase_source : 4be8b148fb653a48f6df4309811ab1d8755f7edf
2017-02-09 09:38:48 -08:00
Wes Kocher
30d9ef4d29 Merge m-c to autoland, a=merge
MozReview-Commit-ID: D76dMYwFRUQ
2017-02-09 17:06:59 -08:00
Ehsan Akhgari
471a55162b Bug 1337874 - Include files that participate in unified builds in the compilation database; r=glandium
Without this, clang-tidy cannot check any diffs including such files.
In the future, when checking the entire tree using clang-tidy, we will
ignore these entries in the compilation database.
2017-02-09 10:14:47 -05:00
Justin Wood
6d726fa81c Bug 1197325 - Remove now unused genisoimage. r=ted
MozReview-Commit-ID: AO3RgE4SdsH

--HG--
extra : rebase_source : 5afb3583c4426d3903b00e879a248a19cb02bd31
2017-02-02 10:10:27 -05:00
Justin Wood
9feb80031e Bug 1197325 -- Set volume icon for DMG in Linux->Mac cross compiles. r=ted
MozReview-Commit-ID: C4LFZB6msmL

--HG--
extra : rebase_source : 18d4e2fa3af4a387bf5e08f987d44a0a658ac1d3
2017-01-30 17:32:32 -05:00
Mike Hommey
f2c9fad61b Bug 1337391 - Don't skip all directories that aren't traversed with a DIRS in a moz.build file. r=chmanchester
The way directory traversal is computed relies on the
RecursiveMakeTraversal class, which is used to reproduce the old
traversal order from the old entirely-in-make traversal with DIRS,
PARALLEL_DIRS, etc. because of the undeclared intra-directory
dependencies that are looming here and there.

It's fed through DirectoryTraversal objects emitted by the frontend.
Normally, DirectoryTraversal objects are emitted for a directory,
possibly giving the subdirectories defined in DIRS/TEST_DIRS its
moz.build. But in the case of gyp processing, nothing places the gyp
objdirs in some virtual DIRS of some parent moz.build since bug 1308982.

As a consequence, the corresponding entries in the
RecursiveMakeTraversal instance attached to the backend are not attached
to any parent directory. When subsequently traversing the tree from the
root, they are never found, and end up being skipped, irregarding of
their actual _no_skip status.

It would probably be possible to revert the changes from bug 1308992,
but we might as well not rely on remains from the old ways. So instead,
we make the RecursiveMakeTraversal consider directories without a
declared parent attached directly to the root directory. They don't need
to depend on any other directory anyways.

--HG--
extra : rebase_source : 17403922322a71d490fdea8db0ff16b04983ed7a
2017-02-09 15:22:34 +09:00
Phil Ringnalda
0bf37ead29 Backed out 3 changesets (bug 1197325) for adding a burning Cc(hfsplus) job
CLOSED TREE

Backed out changeset 158233bce738 (bug 1197325)
Backed out changeset b5ac3fa0bbe7 (bug 1197325)
Backed out changeset 55a8ad127517 (bug 1197325)
2017-02-06 20:04:55 -08:00
Justin Wood
a39754cb97 Bug 1197325 - Remove now unused genisoimage. r=ted
MozReview-Commit-ID: AO3RgE4SdsH

--HG--
extra : rebase_source : 2a49430e1eb0809b191435c6986e076f9cac7b96
2017-02-02 10:10:27 -05:00
Justin Wood
214e9da5c5 Bug 1197325 -- Set volume icon for DMG in Linux->Mac cross compiles. r=ted
MozReview-Commit-ID: C4LFZB6msmL

--HG--
extra : rebase_source : a302c9d12537ca59d3233504abede89bdace15d0
2017-01-30 17:32:32 -05:00
Chris Manchester
b9cc7edb0f Bug 1336231 - Install the gmp plugins to the correct directory during artifact builds. r=mshal
We had been installing them to dist/plugins with the rest of the test plugins,
but tests are actually expecting them to be in dist/bin.

MozReview-Commit-ID: 9qYFgZA4Fni

--HG--
extra : rebase_source : 52a4cbf70ffe7998cfe81ab5e508b9f802dd3a35
2017-02-03 10:59:06 -08:00
Michael Kaply
77e5098faa Bug 1328713 - Allow region specific overrides of search engines. r=florian 2017-02-03 16:48:44 -06:00
Ralph Giles
4efb911890 Bug 1284816 - Remove --disable-rust. r=ted
Remove the option to build without rust code. We are not testing
this configuration and expect to land non-optional rust code in
the near future, so it doesn't make sense to maintain this option.

MozReview-Commit-ID: CwTlMXGvr5n

--HG--
extra : rebase_source : 080a9df5b4828c66aa2452ad1c16a503bcd5e689
2017-02-01 15:38:49 -08:00
Chris Manchester
6a74c9ea61 Bug 1330421 - Update artifact code to populate generated test support files from the tests archive. r=gps
This updates the client artifact code to locate test support files in the
common test archive and populate the objdir with these files appropriately.

MozReview-Commit-ID: GuXjwUtsl

--HG--
extra : rebase_source : 3560efee22533f60be1e7394fa0841005991cca6
2017-01-12 15:30:51 -08:00
Chris Manchester
47674ccc56 Bug 1330421 - Change the structure of test archives include generated test support files in common.tests.zip for the benefit of artifact builds. r=gps
Generated support files for specific tests generally end up in a harness specific
test archive, but artifact builds, which might not want to generate these files
themselves, get all of their test related files from the common.tests.zip. This
effectively moves a small number of these support files from harness specific
archives to the common test archive, making it possible to run tests that
depend on these files against artifact builds.


MozReview-Commit-ID: BvEj1ot1OTw

--HG--
extra : rebase_source : 06e958adc7b5667c50ca5ddbf06c1bf878d75737
2017-01-12 15:29:48 -08:00
Chris Manchester
d3afe369a1 Bug 1333135 - Ensure test support binaries are tracked by install manifests in the build backend. r=mshal
Install manifests influence test packaging, allowing artifact builds to consume
generated test support files and run tests that depend on them. This commit
tracks binaries with an install target under "_tests" so they will be correctly
installed in artifact builds.

This could similarly be achieved by installing the binaries via TEST_HARNESS_FILES
rather than setting FINAL_TARGET, however, PGO builds will fail attempting to install
the files during the profile generation step, because SIMPLE_PROGRAMS are not built
for this step.

MozReview-Commit-ID: ES4bTxOoqMN

--HG--
extra : rebase_source : f41d8c47711fc15247fa336e2e9cbdedf57c2daf
2017-01-27 14:21:29 -08:00
Mike Hommey
dc5d1e6adf Bug 1335309 - Change the default for find_executables to False. r=mshal
Back when the class was written, for the packaging code, it made sense
that the default was True. But now that it's used all over the place,
and that the vast majority of uses are with find_executables=False, it
makes more sense for that to be the default.

--HG--
extra : rebase_source : ff813735fc0d53093f348f20eb77ee03e9b09d4e
2017-01-31 14:06:15 +09:00
Mike Hommey
dbdbf959b8 Bug 1335309 - Add explicit find_executables arguments to every use of FileFinder. r=mshal
And make it an error not to give it. While the default is True, we do
pass a value of False wherever it makes sense, which happens to be, in
most places.

This is an intermediate step to flip the default from True to False,
ensuring that we don't unwantedly switch some calls to False.

--HG--
extra : rebase_source : 432e03f032fef378af482581685583262e5d2661
2017-01-31 13:01:34 +09:00
Benjamin Smedberg
ca77995f5d Bug 1333826 - Remove SDK_FILES, SDK_LIBRARY, and related is_sdk support in the build goop, r=mshal
MozReview-Commit-ID: 52vPyDXdFte

--HG--
extra : rebase_source : c3217730bb70eb7319152dd07536b12f49d6a597
2017-01-30 11:24:10 -05:00
Wes Kocher
18e735dcef Merge m-c to inbound, a=merge
MozReview-Commit-ID: EfMOcZbAiSe
2017-01-27 16:39:29 -08:00
Francesco Pischedda
8a2906dd4f Bug 1313194 - Add --outgoing argument to mozlint. r=ahal
Was: Replace mozlint's --rev with a --outgoing argument

MozReview-Commit-ID: 5YcRWbc1dR4

--HG--
extra : rebase_source : e13fca582d37fb40ba9d7782413190da4c1fa8b3
2017-01-26 17:50:34 +01:00
Nathan Froyd
deb4dfaf07 Bug 1334556 - make GitRepository.get_modified_files slightly more idiomatic; r=gps
Having to munge the output of `git status --porcelain` is unseemly when
there are approaches that will give us what we want directly.
2017-01-27 13:51:58 -05:00
Mike Hommey
9982192790 Bug 1322025 - Provide variants of host and target that depend on --help. r=chmanchester
We want to avoid giving --help dependencies to host and target, so that
they we don't spawn config.guess and config.sub when running configure
--help, and don't need to reach out to the which module to find a
suitable shell to execute them.

So, when --help is given, we return a fake host/target namespace, and
avoid the config.guess/config.sub-invoking code being executed.

Then, by giving the --help option to the linter, it can properly find
that the config.guess/config.sub-invoking code doesn't need the
dependency on --help.

This effectively unbreaks configure --help after bug 1313306.

--HG--
extra : rebase_source : 9ed4cbe5a81121b139d0d86aff6af542c2dff53b
2017-01-25 17:54:16 +09:00
Mike Hommey
c480e73bf9 Bug 1322025 - Allow to combine two DependsFunctions with "|". r=chmanchester
Ideally, it would have been better if it were "or", but it's not
possible to override "or" in python ; __or__ is for "|".

This does feel magic, but it's also shorter than adding something like
@depends_any(), and while we're only adding "|" as of this change, we
can add other operations such as "&" in the future, or __getattr__ for
things like milestone.is_nightly.

An alternative form in moz.configure could require the @depends function
to be called, e.g. "a() | b()" instead of "a | b", but I'm not
particularly convinced that one is less magic than the other.

This feature is hooked up such that b is not resolved if a is true,
although in practice, it will still be resolved in Sandbox.run... but
not when --help is passed. In the long run, the forced resolution of
@depends functions will be removed from Sandbox.run.

--HG--
extra : rebase_source : 2389153acfbc3930786d5495ba4a04640325613e
2017-01-25 17:42:33 +09:00
Mike Hommey
023fca1d5c Bug 1322025 - Don't automatically add --help dependencies to CombinedDependsFunctions. r=chmanchester
Adding those dependencies, retrospectively, only worked around the poor
handling of --help requirements by the linter, that we fixed a few
commits ago. This is now not necessary anymore.

--HG--
extra : rebase_source : 0ba95ab2d246315be097fed7897db1c20666060a
2017-01-25 17:10:03 +09:00
Mike Hommey
30cbb26b96 Bug 1322025 - Don't wrap the combination function in CombinedDependsFunction. r=chmanchester
Several things were wrong with the wrapping:
- the equality test on functions was actually comparing the memoized
  functions, which have a type memoize, which inherits from dict. So
  they weren't comparing actual functions, but the dict used to store
  the cache of their invocation.
- each CombinedDependsFunction created for the same combination function
  used a different wrapped function, so even if the dict problem wasn't
  there, the equality test still wouldn't work, except if the function
  wrapping itself was memoized.
- the memoization was not particularly useful.

Also, for upcoming changes, we'd actually like the combination function to
take an iterable instead of a variable argument list, so that items of
the iterable can be skipped.

--HG--
extra : rebase_source : 2c91889315b49695a2e2b709a9264de9ff237598
2017-01-25 16:50:29 +09:00
Mike Hommey
c15f7ef485 Bug 1322025 - Make DependsFunction.func less public. r=chmanchester
We're going to change the function signature for CombinedDependsFunction,
so make it visible in the API that the function member is not meant to
be used directly. The linter still does, though, because it needs to
look in their guts.

At the same time, avoid setting DependsFunction names via the function
name itself, because in upcoming changes, it will not be modifiable in
some cases.

--HG--
extra : rebase_source : a46ec8d3d2e2511dbb44a48cf2c02ab7ad190510
2017-01-25 16:32:03 +09:00
Mike Hommey
9ab594f516 Bug 1322025 - Enforce --help requirement on option's when argument. r=chmanchester
Options with a `when` argument (either directly, or inherited through
only_when() or an include) require --help per _value_for_option, but
that code path is not exercised during a lint pass.

With this change, along the previous one, we now correctly detect that
bug 1316957 was not supposed to work as is.

--HG--
extra : rebase_source : 0961c132bb200a90814b948aff7d2e80a1b21a30
2017-01-25 14:37:34 +09:00
Mike Hommey
feaac39b5f Bug 1322025 - Enforce --help requirement on indirect dependencies. r=chmanchester
Bug 1313306 relaxed the --help dependency requirement in some cases, but
while doing so, the requirement was also removed in other, unexpected
cases. Specifically, the --help dependency ended up not being required
on indirect dependencies that should have had it, had the --help
dependency been explicit on the direct dependency.

--HG--
extra : rebase_source : 81755988754798590482dfb297b7ddfc14ee88c7
2017-01-25 14:25:58 +09:00
Chris Manchester
c2144deeea Bug 1332517 - Add an 'artifact' key to mozinfo. r=mshal
MozReview-Commit-ID: GPq52LMg23Q

--HG--
extra : rebase_source : 7b61110ab5d8d901c351a1824b3825b5a57f6601
2017-01-19 17:05:02 -08:00
Andrew Halberstadt
30e80e125a Bug 1333564 - [manifestparser] Stop supporting ';' as a valid comment character, r=jmaher
It turns out there are shockingly few cases of manifestparser manifests that actually use the ';'
character as a comment. Because we will soon allow inline comments, deprecating the use of ';' will
ensure that values are allowed to have semicolons in them.

Even without inline comments, might as well enforce consistency across manifests.

MozReview-Commit-ID: AEPPQFdNXG0

--HG--
extra : rebase_source : 3540fa385f328bffb020c0a6debc4d2b3a90ed39
2017-01-25 14:38:37 -05:00
Andrew Halberstadt
b994a9eca5 Bug 1318597 - Explicitly call manager.shutdown() after running mozlint, r=mshal
This is an educated guess in an attempt to solve a hard to reproduce intermittent. This
also makes sure the original SIGINT handler is restored after running the linters.

MozReview-Commit-ID: HRo0GRlSN5L

--HG--
extra : rebase_source : 7b8bf4b5818d741b5ab6a38c8c56f4eb53a801c8
2017-01-23 17:35:03 -05:00
Phil Ringnalda
1b60bc19eb Merge m-c to autoland 2017-01-24 18:56:54 -08:00
Phil Ringnalda
bf98cd4315 Merge m-i to m-c, a=merge 2017-01-24 18:53:29 -08:00
Boris Zbarsky
e007abaff9 Bug 1333478. preprocessor.py should preserve line numbers in webidl files. r=gps 2017-01-24 16:11:39 -05:00
Andrzej Hunt
7ae844d24a Bug 1332911 - Update android-opt artifact test archive filename r=nalexander
MozReview-Commit-ID: 3xeXGcfsqKn

--HG--
extra : rebase_source : fa3540aa020e6ca1764b1c3cdd3393527d7f153b
extra : source : aa4c7481d59906ffa4c79c7e14f53df0a4522006
2017-01-22 15:42:37 +01:00
Chris Manchester
26adeacc8a Bug 1312916 - Add support for host rust libraries in moz.build. r=froydnj
MozReview-Commit-ID: It8VH5Obnx5

--HG--
extra : rebase_source : bc4dabb18ff51c45ac8df5a409881d0d4a7279fa
2017-01-19 14:58:10 -08:00
Ralph Giles
5f0a2f9de1 Bug 1324435 - mozboot: ensure win32 rust support. r=ted
If we have rustup installed, use it to check the available
target platforms and install 32-bit windows support if we're
on the (default 64-bit) windows platform.

This catches systems where the mozillabuild bootstrapper was
run before it installed this, so rustup is available, but the
i686 target library isn't.

MozReview-Commit-ID: 9bE2OQnmvxs

--HG--
extra : rebase_source : 0915fef85755718c5524b13b6b7d0d8b6dbe05b4
2017-01-11 11:30:47 -08:00
Ralph Giles
d8025db2ea Bug 1324435 - mozboot: Install rust i686-msvc target. r=ted
Windows devs often want to target 32-bit windows. Make this
easier by installing the target for them at mozboot time.

MozReview-Commit-ID: 6gFbFBOqMz8

--HG--
extra : rebase_source : 12933c6cb7895cef859c9022efa87c62f321219e
2017-01-11 10:19:28 -08:00