Commit Graph

1500 Commits

Author SHA1 Message Date
Ehsan Akhgari
b73f093357 Bug 904572 - Add support for generating clang compilation database; r=glandium,r=gps 2015-09-29 09:23:50 +09:00
Mike Hommey
86a76ef43b Bug 1207897 - Add a configure option to build multiple build backends. r=gps
Also record what backends were requested at configure time so that running
`mach build-backend` or `config.status` uses the same set.
2015-09-29 09:23:50 +09:00
Mike Hommey
c038211ac4 Bug 1207893 - Allow to create multiple build backends at once. r=gps
When running mach `build-backend` or `config.status`, it is now possible to
pass multiple backends to the --backend/-b option, so that they can share
moz.build reading and object emitting.

The command line syntax is however maybe a little awkward:
  mach build-backend -b Backend1 Backend2

but supporting with `-b Backend1 -b Backend2` requires more argument parser
twiddling (action='append' doesn't work out of the box with choices, we'd
need a custom action class)
2015-09-29 09:23:49 +09:00
Mike Hommey
d873d2729a Bug 1207893 - Change how we track object consumption from the build backend. r=gps
Currently, we set a flag on each object to know whether it has been consumed
by the backend. This doesn't work nicely when multiple backends try to consume
the same objects.
2015-09-29 09:23:48 +09:00
Mike Hommey
64341c67d8 Bug 1207893 - Refactor how build backend execution is summarized. r=gps
- Make all backends report the time spent in their own execution
- Change how the data is collected for the reader and emitter such that
  each of them is aware of its own data, instead of everything being
  tracked by the backend.

This is meant to open the door to multiple backends running from the
same execution of config.status.
2015-09-29 09:23:47 +09:00
Mike Hommey
0e156c41d5 Bug 1207882 - Add an initial partial implementation of a new, faster, build backend. r=gps 2015-09-29 09:23:46 +09:00
Mike Hommey
75bdaecdfd Bug 1207882 - Associate an install target with every ContextDerived object. r=gps
Since reading context['FINAL_TARGET'] can have side effects, use it lazily.
2015-09-29 09:23:45 +09:00
Mike Hommey
a756617866 Bug 1207882 - Ensure chrome manifests are created in a directory that exists when processing jar manifests. r=gps
This currently works because other things in the build system are creating
those directories, but it's not a safe thing to rely on.
2015-09-29 09:23:45 +09:00
Chris Manchester
933886fcaf Bug 1184405 - Take matches by wildcard pattern into account in the test resolver. r=gps
--HG--
extra : commitid : 20bUC0s5hyx
2015-09-25 07:33:11 -07:00
Chris Manchester
c5c1b11a93 Bug 1184405 - Add a mach command to expose test-deps file info. r=gps
This commit exposes test-deps file info as a mach command, and
modifies the test scheme reader to make it filter out unsuitable
contexts when generating TestManifest objects for metadata context.

--HG--
extra : commitid : 7QOoHkfvWOF
2015-09-25 07:33:11 -07:00
Chris Manchester
9e57cd5781 Bug 1184405 - Add annotations for tags, file patterns, and test flavors to moz.build to specify tests potentially impacted by source files. r=gps
--HG--
extra : commitid : JbsAq6sNQ9f
2015-09-25 07:33:11 -07:00
Chris Manchester
442573a6f8 Bug 1184405 - Add a container type to mozbuild with a namedtuple-like interface and typed, mutable fields. r=gps
--HG--
extra : commitid : C983a05SWwh
2015-09-25 07:33:11 -07:00
Nick Alexander
faa87abc9b Bug 1194365 - Bump Andoid NDK to r10e; manually install Android SDK and NDK on Mac OS X. r=mcomella
This bumps the NDK version to r10e.

Previously, we used brew to install android-sdk and a custom version
of android-ndk.  That makes it hard to control the installed versions.
This installs from downloaded archives, which unifies the Mac OS X
approach with the straight-forward Linux approach.

--HG--
extra : commitid : E7hEqsyy8Gw
extra : rebase_source : 9ea27e7d2ae3fbaaa3efbabdd701521981bec877
extra : histedit_source : c07c80c50ac066dc6808e7ccf96f0bc14dc09df2
2015-09-22 13:56:17 -07:00
Nick Alexander
4fd10538cf Bug 1171232 - Depend on platform-tools-preview; don't check for missing and failing Android packages. r=mcomella
The 'tools' package depends on 'platform-tools-preview' now.  Roll
with it until Google breaks us back again.

The behaviour of the |android| tool has changed; recent versions don't
reveal what packages are installed.  That means we can't skip already
installed packages; and we can't really tell if our installation
attempts succeeded.  But we have faith!

--HG--
extra : commitid : 341NxbHTJXC
extra : rebase_source : 945e8018effc0b417fc3fedb7220455fabeaedb3
extra : histedit_source : e42fb06e176d5b9e9ebb6553af7045f3a061105f
2015-09-22 13:54:17 -07:00
Nick Alexander
14f61c3434 Bug 1204260 - Post: remove platforms/android-* from |mach bootstrap|. r=glandium
--HG--
extra : commitid : 8ty9AOUtBe2
extra : rebase_source : 3eb41ec78617f82c80da194d709053ada16afc78
extra : histedit_source : 565b7dbf6a1014c70e2251616471f3bb072c5036
2015-09-22 11:40:06 -07:00
Nick Alexander
416f8c64aa Bug 1108782 - Part 2: Explode AAR files at configure time. r=glandium
This gets us a limited version of AAR support: we can consume static
AAR libraries, where here static does not refer to linking, but to
static assets that are fixed at build-backend time and not modified
(or produced) during the build.  This lets us pin our dependencies
(and move to Google's versioned Maven repository packages, away from
Google's unversioned ad-hoc packages).

By restricting to static AAR libraries, we avoid having to handle
truly complicated dependency trees, as changing parts of generated AAR
files require delicate rebuilding of the APKs (and internal libraries)
that depend on the AAR files.

It is possible that we will generate AARs in the tree at some time.
Right now, we don't do that, even for GeckoView: the AARs produced are
assembled as artifacts at package time and are intended for external
consumption.  We might want this for GeckoView and Fennec at some
time; we should consider using Gradle everywhere at that point.

The patch itself does the simplest possible thing (which has precedent
from Gradle and other build systems): it simply "explodes" the AAR
into the object directory and uses existing mechanisms to refer to the
exploded pieces.

AARs have both required and optional components.  Each component is
defined with an expected and required flag. If a component is expected
and not present, or not expected and is present, an error is raised.
If the component is expected and present, autoconf's ifelse() macro is
used to define the relevant AAR_* component variables.  If the
component is not expected and not present, no action is taken.  A
consuming build backend therefore can guard all AAR_* component
variables with just the top-level AAR variable.

Many AAR files have empty assets/ directories.  This patch doesn't
explode empty assets/ directories, protecting against trivial changes
to AAR files that don't impact the build.

There's a lot not to like in this approach, including:

* We need to manually reference internal AAR libs;
* I haven't separated the pinned version numbers out of configure.in.

However, it's closer to what we want than what we have!

--HG--
extra : commitid : 11kUhDAkCn5
extra : rebase_source : 2454c9842ab3296d53ca5fa394a5a962aa382c8d
extra : histedit_source : e2f97502d215016925e93500b8fd93f8b32fba3a
2015-09-22 10:04:26 -07:00
J. Ryan Stinnett
4f1af0e003 Bug 912121 - Update misc. DevTools paths and comments. rs=devtools
--HG--
extra : commitid : BSf4D59s9HF
extra : rebase_source : 4d14e1aa3ad93bbb426861336e8b703bc1e0b3af
2015-09-21 12:07:31 -05:00
Xidorn Quan
c13216ec53 Bug 1205144 - Remove 'REMOVED' commands from output of mach-commands and mach-debug-commands. r=gps DONTBUILD
--HG--
extra : source : 7d40827d58f699b57668ad80573e93b3b0a8d0b0
extra : amend_source : 98c5b7adbd1a330528d70a8f41096cd26fe49bdd
2015-09-21 11:32:03 +10:00
Mike Hommey
624c5580a9 Bug 1204719 - Don't create interfaces.manifest files from the build backend. r=gps 2015-09-16 17:21:27 +09:00
Mike Hommey
b8c10aca4a Bug 1204712 - Handle wildcards properly for localized content in jar manifests. r=gps 2015-09-16 17:21:12 +09:00
Ted Mielczarek
15800ee8cb bug 935237 - use libdmg-hfsplus to create DMG files during packaging on Linux. r=gps
--HG--
extra : commitid : 2d19BPHSECP
extra : rebase_source : 8121f9f1a929fdf2bdbf53817b7f98c26d9163ec
2015-08-21 15:33:03 -04:00
Ted Mielczarek
62ae78337e bug 1198226 - Add HOST_{C,CXX}FLAGS recursive make varible blacklist. r=mshal
--HG--
extra : commitid : FoL068jTSIs
extra : rebase_source : c520079216e1d53e2bfca01a14d391d0e86e25a0
2015-08-25 09:17:30 -04:00
Ted Mielczarek
15063c8111 bug 1198226 - Add HOST_{CFLAGS,CXXFLAGS,DEFINES} support to mozbuild frontend+recursive make backend. r=mshal
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/defines/moz.build => python/mozbuild/mozbuild/test/backend/data/host-defines/moz.build
rename : python/mozbuild/mozbuild/test/frontend/data/defines/moz.build => python/mozbuild/mozbuild/test/frontend/data/host-defines/moz.build
extra : commitid : BqqF9IWK2hy
extra : rebase_source : 4beab63be2ffca5c0a43cd14039ffeb259ea74fe
2015-09-04 13:33:04 -04:00
Peter Moore
0cc8144421 Bug 1194767 - use slugid 1.0.6 in ./mach taskcluster-graph command. r=wcosta
This included adding the slugid 1.0.6 python source code in /python since slugid
is now a dependency of the ./mach taskcluster-graph command, as well us updating
references that used it. Previously the implementation was in-tree.
2015-09-04 17:59:34 +02:00
Gregory Szorc
84631939a9 Bug 1174037 - Support installing Mercurial from pip; r=glandium
Previously, we used a PPA on Ubuntu to install Mercurial. The PPA has
proved to be unreliable. Furthermore, we didn't have a mechanism for
installing a modern Mercurial on Debian and other derived distros.

In this commit, we remove the PPA from Ubuntu. We add the ability to
install Mercurial from pip to the Debian bootstrapper. However, since
some people may not want <not apt> installing package-like things,
we add a prompt explaining the situation and giving users a choice.
We recommend installing a modern Mercurial via pip. But we also given
the option to install a likely legacy version via apt. And, for Git
users, we give the option to not install Mercurial at all.

Since the new version of the Ubuntu bootstrapper is empty, it doesn't
need to exist, so it has been removed.

DONTBUILD (NPOTB)

--HG--
extra : commitid : 1n8D8hTG6Fm
extra : rebase_source : 666d9945650c0ce27a32353d98886e2336628a36
extra : amend_source : bb9e7538157fdac0203e2ecc3165185f95717098
2015-09-04 10:37:29 -07:00
Nick Alexander
9c3e48627a Bug 1137898 - Migrate to android:versionCode scheme v1. r=rnewman,snorp
Android version codes serve multiple masters.  They indicate newer
versions, yes; but they also break ties between versions with
different features and requirements. High order bits effectively
partition the space of versions and are valuable.  Since Android
version codes are signed Java integers, we have 31 bits to work with.
Mozilla's traditional build ID is YYYYMMDDhhmmss.  This was chopped to
ten characters (YYYYMMDDhh, i.e., hourly build IDs) and then converted
to a decimal.  This took many high order bits.  We will lose another
high order bit in the 36th month of 2015 -- i.e., as soon as the year
rolls over to 2016.  If we waited to lose the next higher order bit,
we'd lose that one in the 46th month of 2017 -- i.e., as soon as the
year rolls over to 2018.

The following patch sacrifices a high order bit to change the version
scheme, winning us roughly 15 years of hourly build IDs before we are
forced to lose another high order bit.  So it's clearly to our
advantage to change the scheme sooner rather than later -- we will
sacrifice 1 bit for 15 years of build IDs, rather than keeping the
current scheme and sacrificing (say) 2 bits for 3 years of build IDs.

The resulting scheme produces build IDs that look like (in binary):

0111 1000 0010 tttt tttt tttt tttt txpg

The meaning of these build IDs is documented in the Python source code
that generates them.

--HG--
extra : commitid : 7BEkkGHsYVL
extra : rebase_source : bbc0ead8e7a383c320e838b023b02b1fb0d94ff3
extra : amend_source : dab8c737e3274694aad40123e77884a3239908de
extra : histedit_source : 530bedde3695d534805465cdf8bcb9cd23f7b031
2015-08-26 18:34:48 -07:00
Geoff Brown
2015a6ea72 Bug 1164596 - Add mach android-emulator command; r=ahal 2015-09-01 15:07:53 -06:00
Nicholas Nethercote
f44287005f Bug 1198334 (part 1) - Replace the opt-in FAIL_ON_WARNINGS with the opt-out ALLOW_COMPILER_WARNINGS. r=glandium.
The patch removes 455 occurrences of FAIL_ON_WARNINGS from moz.build files, and
adds 78 instances of ALLOW_COMPILER_WARNINGS. About half of those 78 are in
code we control and which should be removable with a little effort.

--HG--
extra : rebase_source : 82e3387abfbd5f1471e953961d301d3d97ed2973
2015-08-27 20:44:53 -07:00
Xidorn Quan
dbaa85ce62 Bug 1188468 - Allow script to force updating a generated file even if the file is actually not changed. r=gps
--HG--
extra : source : 47b56f2495030d77c446215d8822c31fc32f23b7
2015-08-25 10:07:43 +10:00
Birunthan Mohanathas
bc29413b87 Bug 1197527 - Consolidate obtaining hg path into mozversioncontrol.get_hg_path; r=gps
DONTBUILD

--HG--
extra : commitid : LsREOYJSAa7
extra : amend_source : 6f19087188d9c6c009a817c3499c7fd9165bfc6a
extra : histedit_source : 36aa6d1a3e78367d50a3e65ac6f61872b7a635d4
2015-08-24 14:37:42 -04:00
Philipp Kewisch
757af5060a Bug 1167986 - TEST-UNEXPECTED-FAIL | xpcshell-unpack.ini:toolkit/mozapps/extensions/test/xpcshell/test_signed_install.js (and signing failures). r=gps 2015-08-23 11:54:44 +02:00
Wes Kocher
fa57aa7153 Merge m-c to inbound, a=merge 2015-08-20 12:51:40 -07:00
Wes Kocher
d8fffe1765 Merge inbound to central, a=merge 2015-08-20 12:42:21 -07:00
Joel Maher
02ece3e234 Bug 1196253 - update in-tree psutil to 3.1.1. r=gps 2015-08-20 08:03:31 -04:00
Wes Kocher
96e3c3623b Merge m-c to fx-team, a=merge CLOSED TREE 2015-08-19 17:13:38 -07:00
Nick Alexander
ad27580b77 Bug 1195388 - Part 2: Add ANDROID_EXTRA_{PACKAGES,RES_DIRS} moz.build variables. r=gps
This moves a little bit more of mobile/android/base/Makefile.in into
moz.build, and gets closer to moving that aapt invocation into
java-build.mk.

There are no other extra package consumers in the tree.  (There should
be a new one shortly: b2gdroid.)

--HG--
extra : commitid : AaYqXYReOSX
extra : rebase_source : d41368ff0bd0736221fdc04ed8299b70c2488c8b
extra : histedit_source : 845efd5ba9f99f4e186c3a5c66affe69eac7fec7
2015-08-14 18:13:11 -07:00
Nick Alexander
543a450286 Bug 1195388 - Part 1: Make ANDROID_APK_{NAME,PACKAGE} moz.build variables. r=gps
This paves the way for defining additional Android packages in
moz.build, which is a step toward moving the special
mobile/android/base/Makefile.in aapt invocations into the generic
java-build.mk framework.

The new variables are both passthru variables for now: in the future,
we'll roll them into some aggregate Android APK definition.

It's worth noting that references to the variables in Makefile.in
files are only defined after including rules.mk (and thereby
backend.mk).  This only required a few changes in the tree but it
confused me for some time.

--HG--
extra : commitid : G5mEvm8Ng4F
extra : rebase_source : 7ba05f2e53554549ffb5cefe270925e3e2025b6a
extra : histedit_source : eacd22f4b7edddab67147c413fea45a3ba292c0c
2015-08-19 12:34:58 -07:00
Andrew Halberstadt
f22ebea9f6 Bug 1194415 - Refactor BuildProgressFooter.draw() for minor |mach build| perf improvement, r=gps
Over a 19 minute build, |mach build| was spending ~45s drawing the TIER footer. This brings it down to ~30s.

--HG--
extra : commitid : 2sLWbNL853A
extra : rebase_source : ab8452dfa6ac39bc40a8f22b542e155595f18881
2015-08-13 14:08:12 -04:00
Niels De Graef
582ad89c13 Bug 1195182 - Fixed broken AUR link for uuid; r=gps
DONTBUILD (NPOTB)

--HG--
extra : commitid : GVD1noGQI0w
extra : amend_source : 3e46667094f8bb51bdd74cc2615443bc4db3d0ba
2015-08-18 09:26:37 -07:00
Aaron Raimist
2ed988f853 Bug 1171783 - Fix bootstrap.py by using DNF instead of Yum if possible; r=mshal
DONTBUILD (NPOTB)

--HG--
extra : commitid : KbAGQkESbla
extra : rebase_source : b171bdb997ae94905254a111cd550431e680d3f4
extra : amend_source : 2b2354cb623688359d0dfb1f6b548a19140551d6
2015-08-19 12:56:22 -07:00
David Keeler
a7f29f4b91 bug 1190603 - import PyECC library r=gps,gerv
Obtained from https://github.com/amintos/PyECC.git
(commit 1bfd3a41410c3650e57d58f7bd016bb0819af250)
2015-08-03 15:17:00 -07:00
Nathan Froyd
a8f3b221c3 Bug 1186444 - part 2 - remove MODULE_OPTIMIZE_FLAGS from config.mk; r=mshal 2015-07-22 11:17:49 -04:00
Andrew Halberstadt
9cbfe3373d Bug 1184696 - Add clobber targets to |mach clobber|; Ability to clobber compiled python files, r=gps
This change adds a 'what' parameter to |mach clobber| which facilitates clobbering other things as well.
E.g, |mach clobber python| now clobbers all .pyc and .pyo files in the tree. Multiple clobber targets can
be specified, e.g |mach clobber objdir python|. By default |mach clobber| without arguments will only
remove the currently active objdir, the same as before.

--HG--
extra : commitid : 4mJqdnWPRbH
extra : rebase_source : cf56de2d51c57747047936f7e742bafbf3f91c9f
2015-07-20 14:38:18 -04:00
Ryan VanderMeulen
efce95618e Merge fx-team to m-c. a=merge 2015-08-13 11:00:54 -04:00
Nick Alexander
61a6ae674e Bug 1160563 - Part 2: Make ANDROID_ASSETS_DIRS a moz.build variable. r=gps
We have had singular ANDROID_ASSETS_DIR in Makefile.in for a while.
Fennec itself does not use the existing Makefile.in Android code, for
complicated historical reasons.

This makes the existing variable moz.build-only; generalizes the
existing variable to an ordered list; and adds the equivalent use of
the new list to the Fennec build, with a simple example asset.

This patch also updates the packager to include assets packed into the
gecko.ap_.  Without the packager change, the assets/ directory in the
ap_ gets left out of the final apk.  This whole approach is totally
non-standard but is more or less required to support our single-locale
repack scheme.

--HG--
extra : commitid : 4EAh1UNGNWT
extra : rebase_source : 5e5b4c4a120c3b4cc776c9f9380ddd2f9b63587e
extra : source : 0ddce3eb833e6d6180a19928a9b45d5d12f1d7fa
2015-08-12 11:04:03 -07:00
Nick Alexander
549c28f40f Bug 1160563 - Part 1: Make ANDROID_RES_DIRS a moz.build variable. r=gps
This patch does a few things.  First, it adds an AbsolutePath data
type, sibling to SourcePath and ObjDirPath.  (Existing Path consumers
that accept an open set of Path subtypes, and that only use full_path,
should function fine with the new AbsolutePath subtype.)

Second, it moves ANDROID_RES_DIRS to a moz.build list of Paths
(ordered).  We test, but don't use in tree, the new AbsolutePath.

--HG--
extra : commitid : DMLy1ogTJ0Y
extra : rebase_source : cb9ac47e8bf7c893a0284adc7a42eccb78ccae3d
2015-08-12 11:03:44 -07:00
Eric Rahm
e2ce0103de Bug 1193914 - Properly check ide type when deciding whether or not to package. r=benwa 2015-08-12 13:59:05 -07:00
Ted Mielczarek
a116bbbe7d bug 1164816 - Import concurrent.futures into the tree. r=gps
--HG--
extra : commitid : AjJH6cEqLlp
extra : rebase_source : 6a5befe8804e0297fd18af57643ac0061c570c5a
2015-05-12 14:52:06 -04:00
Ted Mielczarek
de9027084a bug 1190522 - Rewrite pkg-dmg as a Python script. r=gps
--HG--
extra : commitid : 8CKCtjFc6wl
extra : rebase_source : bc51508f3708a7a7af8fe48bdbcb0d787bf77362
extra : source : bca1ad6cc3781af4b8c8667ccb3c29e5bcb50ea9
2015-08-03 15:40:04 -04:00
Kyle Huey
7f17047f2a No bug: Make ./mach clobber work on windows without winrm.exe. r=me 2015-08-11 06:10:46 -07:00