Commit Graph
3315 Commits
Author SHA1 Message Date
dluca 935c41c8bf Backed out 1 changesets (bug 1393817) for failing to repackage the nightly build on Windows a=backout.
Backed out changeset 25f21f22a34b (bug 1393817)
2017-11-09 15:34:00 +02:00
Sebastian Hengst 52ea9f8769 merge mozilla-inbound to mozilla-central. r=merge a=merge 2017-11-09 00:00:16 +02:00
Zibi Braniecki 505ab04c54 Bug 1415432 - .ftl files repackaged multiple times. r=glandium 2017-11-07 22:16:16 -08:00
Andrew Halberstadt 2730a80a6b Bug 1414399 - [mozbuild/moztest] Move mozbuild.testing.TestResolver to moztest.resolve r=gps
The TestMetadata and TestResolver classes aren't technically part of the build
system. The only connection is that they consume some build system output.

The next patch in this series is going to be merging in a bunch of other test
resolving logic from other parts of the tree. Moving this out first allows us
to keep that extra logic out of mozbuild.

MozReview-Commit-ID: 1eq4SjFVCyW
2017-11-06 08:41:42 -05:00
Tom Ritter 1625276bf4 Bug 1393817 Use buildconfig to get the path for 7z rather than hardcoding it in exe_7z_archive.py r=ted
With that fixed, Re-enable MOZ_AUTOMATION_L10N_CHECK for MinGW

MozReview-Commit-ID: BDTtY1J7sBj
2017-10-27 20:13:21 -05:00
Jonathan Watt 096bcbef0b Bug 1413227 - Turn off Eclipse CDT's "Limit console output" option for debug output. r=botond
The default debug console line limit is far too short, especially when
debugging a debug build or a build with some logging turned on.  I've never
had any issues with turning off the limit, so hopefully this is fine for
everyone else too.

MozReview-Commit-ID: Krlo7XPEjE0
2017-10-16 11:53:26 +01:00
Jonathan Watt 679962db59 Bug 1413227 - Increase the line limit for Eclipse CDT's build console to 10,000. r=botond
By default the build console is limited to 500 lines.  This can be far too
short, especially when building using `mach build --verbose`.  I've run with
this value set to 1,000,000 before without issue, so 10,000 should be fine.

MozReview-Commit-ID: GxBUjW7nOYJ
2017-10-16 09:36:09 +01:00
Gregory Szorc cdf807962b Bug 1412932 - Switch to PGO build in Makefile.in; r=ted
Previously, client.mk made the decision of whether to perform a PGO
build. This required passing around MOZ_PGO and invoking a separate
make target if this variable was set.

In this commit, we move this logic to Makefile.in. We employ a special
mechanism in rules.mk to override the default make target so
`make` evaluates "profiledbuild" if MOZ_PGO is set. This also
required using an explicit target for $(MAKE) invocations inside
the "profiledbuild" rule to avoid infinite recursion.

MozReview-Commit-ID: 8sHiVspMisM
2017-11-07 16:38:39 -08:00
Marco Castelluccio 56a9cc11e1 Bug 1317041 - Calculate crc32 incrementally; r=gps 2017-11-07 15:34:20 -08:00
Chris Manchester 226e4b9e9a Bug 1414064 - Remove LD_VERSION_SCRIPT from the build system. r=mshal
MozReview-Commit-ID: 6KOLZh6aQmc
2017-11-07 14:35:05 -08:00
Tom Prince 5c1e459dac Bug 1415199: Move some logic for loading taskgraph kinds from mozbuild.mach_bootstrap to taskgraph. r=dustin
This fixes the fallout from the mach command not being updated for 69c60c8fc528d8d038e22a2ae62f5ff7b1131231.

MozReview-Commit-ID: 7WZ8wW4gnCY
2017-11-07 10:08:49 -07:00
Chris Manchester 4888eb56c9 Bug 1414370 - Emit ldflags in directories containing only rust programs for the benefit of cargo. r=froydnj
MozReview-Commit-ID: 6gD7kAIQbtB
2017-11-06 19:57:12 -08:00
Sebastian Hengst 3e4740d424 Backed out changeset ec6b9fda1868 (bug 1415199) for breaking gecko decision task. r=backout on a CLOSED TREE 2017-11-07 19:50:30 +02:00
Tom Prince b146a4c9e0 Bug 1415199: Move some logic for loading taskgraph kinds from mozbuild.mach_bootstrap to taskgraph. r=dustin
This fixes the fallout from the mach command not being updated for 69c60c8fc528d8d038e22a2ae62f5ff7b1131231.

MozReview-Commit-ID: 7WZ8wW4gnCY
2017-11-07 10:08:49 -07:00
Andreas Tolfsen 44220465d3 Bug 1413195 - Vendor webdriver's dependencies through geckodriver r=jgraham
The webdriver crate's dependencies are accounted for through geckodriver's
Cargo.lock file, because the webdriver dependency is given by path.

This commit reverts 540273e5cf22 made in bug 1412037.

MozReview-Commit-ID: F0q9bDN3pqd
2017-11-03 14:02:09 +00:00
Sylvestre Ledru 80acd52ec8 Bug 1414781 - Also install clang-format r=glandium
MozReview-Commit-ID: A6Jlc5RXVuP
2017-11-07 11:45:58 +01:00
Zibi Braniecki 5c0a86e2e7 Bug 1413985 - Package .ftl files on repackage for both, source and AB_CD locales. r=glandium
When repackaging, we want to take all localization resources from the source package,
and add all localization resources from the repackage target locale.

This patch makes packager.l10n scan for `localization` directories in l10n_finder and
add them.

MozReview-Commit-ID: CRLP3bOAyDx
2017-11-02 12:10:26 -07:00
Alex Gaynor e84c51c81f Bug 1414842 - remove reliance on a CPython implementation detail in mozbuild; r=gps
MozReview-Commit-ID: SGrqK6NpEY
2017-11-06 10:36:15 -05:00
Chris Manchester 29f24a04fd Bug 1414370 - Emit ldflags in directories containing only rust compilation for the benefit of cargo. r=froydnj
MozReview-Commit-ID: 9AuU3mjKR8V
2017-11-03 12:16:39 -07:00
Sebastian Hengst 6de3048253 merge mozilla-inbound to mozilla-central. r=merge a=merge
MozReview-Commit-ID: Ai6Y5GGfkfT
2017-11-04 10:58:24 +01:00
David Major a19b4ac9ef Bug 1412952 - Use VS2017 with clang-cl builds. r=froydnj 2017-11-03 12:29:58 -04:00
Chris Manchester a6454325e7 Bug 1411712 - Move LDFLAGS to mozbuild. r=mshal
MozReview-Commit-ID: A8cZ2FUveAf
2017-11-02 11:47:48 -07:00
Zibi Braniecki d436f04b61 Bug 1411951 - Use mozpack.path.normsep when generating chrome entry paths for langpack manifest.json. r=mshal
MozReview-Commit-ID: L8kVm1ISkzD
2017-10-28 21:40:12 -07:00
Gregory Szorc 2ef96dd47a Backed out changesets f47abc8062fc, 73df5a910488, 75807d1ab22e, and 53a93e195f91 (bug 1412460); a=bustage
CLOSED TREE
2017-11-01 15:49:04 -07:00
Gregory Szorc 40617a6b64 Bug 1412460 - Always evaluate config environment; r=mshal
I /think/ the only scenario where we wouldn't be able to evaluate
a config environment after configure was the case where
MOZ_BUILD_PROJECTS was in play. Since we removed support for that
feature, let's drop support for treating the config environment
as optional.

MozReview-Commit-ID: 4sz9dOwaA3y
2017-10-27 17:14:00 -07:00
Gregory Szorc 96f32f105d Bug 1412460 - Move configure invocation out of mach_commands.py; r=mshal
While we're here, also move the low-level code to invoke configure
to a proper Python module.

We needed to muck around with the log manager because the underlying
logger for output changed from "mach" (which is registered by default)
to something under the "mozbuild" hierarchy.

MozReview-Commit-ID: 4rlCxOwcVu1
2017-11-01 15:01:21 -07:00
Gregory Szorc 9198e068be Bug 1412460 - Move code for invoking build system out of mach_commands.py; r=mshal
The goal is to move this lower-level code somewhere where it can be
reused.

The only meaningful code change as part of the move was handling
of the mach context. Before, we could access self._mach_context.
After, we need to pass it as an argument. This also uncovered a bug
where "telemetry_handler" could be None but we called it
unconditionally. We now guard this call with an "if."

MozReview-Commit-ID: FWw4l6H1ayS
2017-10-27 10:58:30 -07:00
Gregory Szorc aa7cbc905d Bug 1412460 - Move build output processing code out of mach_commands.py; r=mshal
In general, we shouldn't have so much business logic in mach_commands.py
files.

I'd like to move more of the low-level "perform a build" logic out of
mach_commands.py to facilitate some future work. That code makes use of
this output processing code. So as the first step in moving the build
code, we move its dependencies.

As part of this, I also cleaned up some random unused imports around
lines that were touched.

No meaningful code changes were performed as part of the code move.

MozReview-Commit-ID: 96mGWUJ7oLb
2017-10-27 10:45:41 -07:00
Nathan Froyd b869ae1f4f Bug 1413232 - move the default value for FILES_PER_UNIFIED_FILE to a more logical location; r=nalexander
Putting it in emitter.py increases the obviousness of its default value
when searching for things.
2017-11-01 10:16:39 -04:00
Nick Fitzgerald e9c771c5d0 Bug 1414349 - Re-sync the list of top level crates for mach cargo check; r=mbrubeck
Because there isn't a single, canonical source of truth, the `mach cargo
check`'s list got out of date.

MozReview-Commit-ID: EGNc2eNSumT
2017-11-03 10:49:34 -07:00
Nick Fitzgerald 23264cab5f Bug 1414349 - Add mozjs_sys to the top level crate set; r=mbrubeck
This is necessary because it runs as its own taskcluster job, and therefore has
its own Cargo.lock file.

MozReview-Commit-ID: HjB60bQnMeO
2017-11-03 10:49:13 -07:00
J.C. Jones 50ba3c2a6a Bug 1409259 - Add xpcshell tests for the Symantec distrust r=keeler
This commit adds two new xpcshell tests, both of them testing whether the
security state in TransportSecurityInfo includes the new
STATE_CERT_DISTRUST_IMMINENT flag under the correct circumstances.

The first test, test_symantec_apple_google.js, tests the four combinations of
certs that chain to an affected Symantec root: with/without a whitelisted
intermediate, and before/after the notBefore cutoff date.

The second test, test_symantec_apple_google_unaffected.js, tests an unrelated
ca->intermediate->ee chain that does not chain to an affected root, and ensures
the flag is not set.

This patch adds SymantecSanctionsServer to the mozbuild and xpcshell test
infrastructure files to ensure it runs properly on TaskCluster, too.

MozReview-Commit-ID: GtUXH2VFFh
2017-11-01 11:12:11 -07:00
Gregory Szorc f8d0e741b3 Bug 1413254 - Write a JSON file with normalized data; r=jmaher
The new file contains changes that optimize for final file size:

* The components are stored as an integer index/key into a map
  instead of strings.
* Paths are stored in hierarchical dicts.
* No indentation is used.

These changes significantly redude redundancy in string data. For every
file in the repo, the raw JSON size is decreased from 27,074,607 to
7,643,529 bytes.

MozReview-Commit-ID: 58FWSct8W39
2017-10-31 10:57:14 -07:00
Gregory Szorc a8a9f360b0 Bug 1413254 - Fail Bugzilla CI task if file is missing component annotation; r=jmaher
We want all files in the repo to be annotated with a Bugzilla
component. This change will cause the CI task to fail if files are
missing annotations.

MozReview-Commit-ID: Kn6dheQqEsk
2017-10-31 10:37:18 -07:00
Gregory Szorc b4a5c368b3 Bug 1413254 - Establish dedicated mach command for Bugzilla metadata validation; r=jmaher
Previously, the CI task invoked 2 mach commands and performed manual
gzip compressed. This was sub-optimal for a few reasons:

* Redundant moz.build evaluation made execution slower
* Potential for shell expansion to interfere with wildcards
* Made it harder to change exit code

By moving all logic into a mach command, we eliminate all the issues
around using shell.

Behavior of the new command and the CI task should be backwards
compatible.

MozReview-Commit-ID: HUFvLteKLOO
2017-10-31 10:30:15 -07:00
Chris Manchester c7fb052be1 Bug 1405811 - Move host compilation flags to mozbuild. r=mshal
MozReview-Commit-ID: 7I5IHM65eaU
2017-10-02 11:41:41 -07:00
Gregory Szorc 0355371eb8 Bug 1411994 - Ignore Unicode errors when logging; r=glandium
LineIO currently attempts to convert bytes to unicode using the
system preferred encoding. This can obviously fail for some byte
sequences.

In many cases, the conversion is happening for purposes of logging.
In these cases, Unicode validation shouldn't be that important to
us.

So, this commit teaches LineIO to accept an argument defining its
Unicode error mode. We also teach relevant users of LineIO doing
logging to switch the default mode from "strict" to "replace" so
invalid Unicode byte sequences can be logged with the Unicode
replacement character.

During review, glandium pointed out that it may make better sense
to defer Unicode decoding to the logging layer instead of in LineIO.
I agree with this idea. But it can be implemented in another bug:
for now we should unbust the build system.

MozReview-Commit-ID: 7miuSDY8Xzv
2017-10-26 15:27:01 -07:00
Kartikaya Gupta bdaacf12e0 Bug 1413185 - Remove obsolete license whitelist entry. r=froydnj
DONTBUILD because this only gets run as part of a manual `mach vendor rust`
invocation and won't run in a regular CI run.

MozReview-Commit-ID: FMym1BsEAFe
2017-10-31 09:39:50 -04:00
Chris Manchester 7cbf6f501b Bug 1411787 - Convert MOZBUILD_CFLAGS from passthru variables to computed flags. r=mshal
MozReview-Commit-ID: HOMf4vfFE1n
2017-10-27 10:38:10 -07:00
Chris Manchester d2ec086a84 Bug 1412267 - Fix the inclusion of MOZ_DEBUG_FLAGS in the compile command line to depend on MOZ_DEBUG_SYMBOLS and MOZ_DEBUG. r=gps
MozReview-Commit-ID: 4MgzRQCX5iW
2017-10-27 09:39:44 -07:00
Andreas Tolfsen b816825470 Bug 1412037 - Include testing/webdriver as vendored dependency. r=jgraham
This is required for "./mach vendor rust" to pick up on
testing/webdriver's dependency tree.  This change should have been
made when the webdriver crate was vendored in the tree in the first place.

MozReview-Commit-ID: 5sqRojm0bGD
2017-10-30 15:02:19 +00:00
Gregory Szorc a1e4fe3b39 Bug 1412431 - Remove support for MOZ_BUILD_PROJECTS; r=nalexander
This was mainly used to support Universal MacOS builds, which were
removed several months ago.

In theory, someone could be using this feature to build multiple
applications with one build system invocation. But given that client.mk
is no longer the preferred interface to the build system and multiple
applications can be built by running `mach build` with different
mozconfigs, I don't think support for this feature is worth keeping.

This commit removes support for MOZ_BUILD_PROJECTS and related
functionality from client.mk. Support for recognizing
MOZ_CURRENT_PROJECT in configure and mozconfig evaluation has also
been removed. This includes support for the ac_add_app_options
mozconfig function.

Good riddance.

MozReview-Commit-ID: 7xI2jYxDFFr
2017-10-27 15:20:34 -07:00
James Willcox d00277d1a3 Bug 1412677 - Install NDK r15c as part of 'mach bootstrap' r=froydnj
MozReview-Commit-ID: A1aEUTWSDVm
2017-10-30 08:51:15 -05:00
Sebastian Hengst 274ffbb5a5 merge mozilla-central to mozilla-inbound. r=merge a=merge 2017-10-27 00:00:25 +02:00
Mike Shal 94d53aefec Bug 1407403 - Handle IPDL code generation in the tup backend; r=gps
IPDL generation consists of a single ipdl.py invocation on all the
*.ipdl and *.ipdlh files in the tree. Unfortunately the headers that are
generated are placed into directories corresponding to the namespaces
declared within the IDPL, and can't be determined solely based on the
input filename. As such, we have to do a quick parsing of the AST in
order to grab the namespaces and calculate the filenames for the .h
files.

The downside of this approach is that all *.ipdl/*.ipdlh files are
considered inputs for the tup backend, so editing one will perform
moz.build generation before running ipdl.py. This could be fixed by
either standardizing the generated header layout, or fixing tup to
somehow allow unspecified outputs.

MozReview-Commit-ID: DzF2ryLEsrg
2017-10-23 10:44:11 -04:00
Sebastian Hengst 5a55d163c7 Backed out changeset 26c1d50db1c1 (bug 1407403) for py-compat failure: python/mozbuild/mozbuild/backend/tup.py:470:124 | inconsistent use of tabs and spaces in indentation. r=backout 2017-10-26 16:23:40 +02:00
Mike Shal bf2f2be908 Bug 1407403 - Handle IPDL code generation in the tup backend; r=gps
IPDL generation consists of a single ipdl.py invocation on all the
*.ipdl and *.ipdlh files in the tree. Unfortunately the headers that are
generated are placed into directories corresponding to the namespaces
declared within the IDPL, and can't be determined solely based on the
input filename. As such, we have to do a quick parsing of the AST in
order to grab the namespaces and calculate the filenames for the .h
files.

The downside of this approach is that all *.ipdl/*.ipdlh files are
considered inputs for the tup backend, so editing one will perform
moz.build generation before running ipdl.py. This could be fixed by
either standardizing the generated header layout, or fixing tup to
somehow allow unspecified outputs.

MozReview-Commit-ID: DzF2ryLEsrg
2017-10-23 10:44:11 -04:00
Attila Craciun a7a4b9e712 Merge inbound to mozilla-central r=merge a=merge 2017-10-26 12:20:45 +03:00
David Major 2f857c3fba Bug 1411661: Push Speedometer files down a level so they can refer to "../resources". r=froydnj,jmaher 2017-10-25 18:05:34 -04:00
Nathan Froyd f7fd01caad Bug 1411680 - fix clang toolchain downloaded by mach bootstrap on Linux; r=nalexander
We renamed this toolchain a while back, but never updated bootstrap.
2017-10-25 15:10:31 -04:00