The hglib Mercurial finder was nice. But it is somewhat slow, as it
involves a separate Mercurial process.
This commit introduces a native Mercurial finder that speaks directly
to a Mercurial repository instance. It is significantly faster.
The new code is isolated to its own file because it imports Mercurial
code, which is GPL.
--HG--
extra : commitid : 8CzDFt3lQmx
extra : rebase_source : 1540ad20d795086926b316612062e2a1f10c4958
This commit adds support for specifying a Mercurial revision with `mach
file-info`. Aside from being a potentially useful feature, it proves
that MercurialRevisionFinder works with BuildReader.
--HG--
extra : commitid : 7143XT9ENqb
extra : rebase_source : d8c0c98d536d07db76653b648fd4b7d74d8f43ae
The moz.build reader uses absolute paths when referencing moz.build
files. *Finder classes expect paths to be relative to a base. When we
switched the reader to use *Finder instances for I/O, we simply provided
a default FileFinder based at the filesystem root. This was quick and
easy. Things worked.
Unfortunately, that solution isn't ideal because not all *Finder
instances can accept absolute paths like that. The
MercurialRevisionFinder is one of them.
Changing the moz.build reader to talk in terms of relative paths is a
lot of work. While this would be ideal, it is significantly easier to
defer the problem until later and hack up MercurialRevisionFinder to
accept absolute paths. This commit does exactly that.
Bug 1171069 has been filed to track converting BuildReader to relative
paths.
--HG--
extra : commitid : 2JmzOBldBLy
extra : rebase_source : a8af6ce88dd9e2b98f131c92b45c3ece852e13d2
Now that moz.build files use finders for I/O, we can start reading
moz.build data from other sources.
Version control is essentially a filesystem. We implement a finder
that speaks to Mercurial to obtain file data. It is able to obtain
file data from a specific revision in the repository.
We use the hglib package (which uses the Mercurial command server) for
speaking with Mercurial. This adds overhead compared to consuming the
raw Mercurial APIs. However, it also avoids GPL side-effects of
importing Mercurial's Python modules.
Testing shows that performance is good but not great. A follow-up
commit will introduce a GPL licensed Mercurial finder. For now, get
the base functionality in place.
--HG--
extra : commitid : BkwaQOW9MiR
extra : rebase_source : 915d6015317ccc79c228a76eed861d9f43e2fd17
Sometimes moz.build data may not come from the local filesystem. To
support defining moz.build data in other backing stores, we switch
moz.build reading to use mozpack's *Finder classes for I/O.
The use of a FileFinder bound to / is sub-optimal. We should ideally
be creating a finder bound to topsrcdir. However, doing this would
require refactoring a lot of path handling in the reader, as that code
makes many assumptions that paths are absolute. This would be excellent
follow-up work.
While I was here, I cleaned up some linter warnings for unused imports.
On my machine, this commit results in a slight slowdown of moz.build
reading. Before, `mach build-backend` was consistently reporting 1.99s
for reading 2,572 moz.build files. After, it is consistently reporting
2.07s, an increase of 4%. This is likely due to a) function call
overhead b) the cost of instantiating a few thousand File instances
c) FileFinder performing an os.path.exists() before returning File
instances. I believe the regression is tolerable.
--HG--
extra : commitid : 1WDcrSU3SQD
extra : rebase_source : a23aa5a4063c6f7080ee00b6f0fe0ee532da3ce7
Passing raw file handles around is a bit dangerous because it could lead
to leaking file descriptors. Add a read() method that handles the simple
case of obtaining the full contents of a File instance.
This should ideally be a method on BaseFile. But this would require
extra work and isn't needed. So we've deferred it until bug 1170329.
--HG--
extra : commitid : 82qw76XmpjC
extra : rebase_source : 422b16c5a3b1577f080097925aeaeb560aa3e798
Today, the *Finder classes are optimized for doing matching and
returning multiple results. However, sometimes only looking for a
single file is wanted.
This commit implements the "get" method on all the *Finder classes.
It returns a BaseFile or None.
FileFinder._find_file has been refactored into FileFinder.get
to avoid code duplication.
--HG--
extra : commitid : K9It2ZJ3Rbo
extra : rebase_source : a56f8f70aa1902d26373a7196eae2847cce653d3
The current mode of execution of templates doesn't allow them to more advanced
control flow, like returning early, returning or yielding values, because that
mode of execution is equivalent to running the code at the top level of a .py
file.
Making the templates executed through a function call, although trickier,
allows those control flows, which will be useful for template as context
managers.
inspect.getsourcelines() and inspect.getfile() involve I/O out of our control.
Our use of those functions, however, doesn't require all their smarts. In fact,
we only use them on function objects, for which we can just do the work
ourselves without involving inspect functions that trigger I/O.
`mach help <command>` currently only displays a brief description of the
command along with its arguments. Sometimes more detailed help text is
needed.
With this commit, the docstrings of mach command handlers will appear in
the output of `mach help <command>` if they are defined.
I've implemented basic docstrings for the three flavors of mach commands
(normal command, main subcommand, subcommand) to demonstate things work.
My hope is others will start to fill in docstrings once this feature
lands so the output for `mach help` can serve as a better learning guide
for new contributors.
--HG--
extra : commitid : Hx6ZkHDxbCK
extra : rebase_source : 01ced5a044442e370a45cd3fb245ac6283316925
extra : amend_source : fceb771e0e1ffa4e6f3f1b7c22eae6e25cf82034
The old way of writing scripts for generated files would invoke the script thusly:
python script.py arg1...
Invoking the script this way means that the script's directory is
automatically added to sys.path, and importing modules from that
directory is easy. Let's make it equally easy in the new world for
GENERATED_FILES, too.
Document the role of l10n.ini, filter.py, file paths and checks.
Also add a glossary for the jargon used in the doc.
--HG--
extra : rebase_source : ec71f9b7123ba76370d34d2349b2f078f14dd1de
Add generic support for different forms of paths in moz.build:
'/topsrcdir/relative/paths'
'srcdir/relative/paths'
'!/topobjdir/relative/paths'
'!objdir/relative/paths'
This drops the use of UserString for performance reasons, which required
going around with a meta class.
This code has been commented since it landed, and enabling it requires
adding the proper conditionals around all non_unified_sources in gyp
configurations, which is a daunting task. OTOH, the commented code is
already outdated (it would need updates to work) and the related code
that is not commented gets in the way of upcoming changes, so remove
it.
This helps upcoming changes, and relieves backends from path resolution.
This has the side effect of chaning the order of some unified sources,
which consequently breaks nsTextFormatter because it declares snprintf
methods after nsStringAPI #defines it.
A host elfhack binary is only built when there is a compile environment.
--HG--
extra : rebase_source : 4f1da429c581dfd81cbe3d5164c7586066cf6e79
extra : amend_source : f26fe7f3b44291f38459a81b9ff31bd6dbd220aa
extra : histedit_source : 93c628a500c66c46d522bfe678500bf5b5bf0de9
With TemplateContexts keeping the name of the associated template, and the
Gyp context being declared as a TemplateContext, it is now possible to know
the equivalent of IS_GYP_DIR just by looking at the template name.
Add .rs as a recognized file extension in SOURCES.
Propagate that through to the Makefile backend and add a dependency
generated and an explicit rule to call $(RUSTC) to compile them.
rustc builds static libraries, not obj files. At least, if one
asks it to output an obj file, I'm not clear how to get all the
compiler-specific runtime libraries the code will expect to link
to. Therefore we generate a static library for each rust source
file (which must be a complete crate for the time being) and link
that. Because of the extension it ends up on the LIBS line in the
the corresponding .desc file.
Note that the static library does still depend on some system
libraries, e.g. -ldl -lpthread -lm on linux. Gecko already
links to all of those, so we don't keep track of it here.
Should we need to add explicit linkage for other targets,
rustc does print a list to stderr which can be parsed.
--HG--
extra : rebase_source : 9f66101fd15a649a952d5481cf9561416204272a
Formerly, running |./mach test image/| would result in running a number of devtools tests
in addition to running tests under the image/ subdirectory. With this change, only tests
under the image/ directory would be run. Note that ./mach test animations or similar will
still run a variety of tests across the tree, because this input does not match a directory.
--HG--
extra : rebase_source : 8a0482796ccb51cec8cd873cf8cd9933e30d46d1
mach dispatch makes separate, independent calls to construct build
system state. Part of this resolution is determining the object
directory. For environments without an object directory defined, we must
execute config.guess to determine the object directory. This redundant
execution of config.guess can result in significant execution overhead.
Before this patch, `mach help` with no mozconfig took ~1.5s on my OS X
machine. After this patch, it goes down to ~0.750s. On Windows, the
difference is even more pronounced, with execution time dropping from
8.5s to 0.930s.
--HG--
extra : rebase_source : e8c0f49ceb71e1371e174e31f6e1f354d1e51b65
extra : amend_source : 133b013d5dd8f3817f92555aed1717534769585d
`mercurial version` doesn't always print the version number. `mercurial
--version` does. Use the latter.
--HG--
extra : rebase_source : 6f3c305ce8b38599a60c01d9cee410b724bfc37f
extra : amend_source : 430a3cf66094a62fabee59dbf4a486d4392a8c47
Now that moz.build can see EXTRA_*COMPONENTS and NO_JS_MANIFEST, we can
move some logic from rules.mk (executed every build) to moz.build's
emitter.py (executed only at build-backend time).
Interactive prompts make automation more difficult for developers looking to stand up machines using bootstrap en masse.
Two new options have been added to allow users with such needs to bypass all prompts: one for selecing an application
(desktop/mobile) and another for assuming yes to all questions posed by package managers (apt-get/yum).
This avoids duplicating the logic from SimplePackager to find base
directories, and fixes some cases where the l10n repack code wouldn't
find them properly.
Bug 910660 added a consistency check that rejects cases where a manifest
inside a directory detected as being the base of an addon is included from
outside the addon. Unfortunately, this triggered false positives when
a manifest is included from within the addon, but just happens to be at
the top-level of that addon.
This avoids duplicating the logic from SimplePackager to find base
directories, and fixes some cases where the l10n repack code wouldn't
find them properly.
Some file types, such as XPTFile, read their associated data during the
copy. In the case of XPTFiles, several original files are linked together
in one destination file. When a FileCopier is used in-place, those
original files are removed before XPTFile.copy is invoked, so XPTFile.copy
fails.
Generally speaking, there are many other reasons why FileCopier can fail
in-place, but the only active use of this mode is l10n repack code, which
actually doesn't do much of the dangerous uses. However, it can end up
linking XPTFiles for some reason, which fails for the reasons given above.
Back when mozpack.path was added, it was used as:
import mozpack.path
mozpack.path.func()
Nowadays, the common idiom is:
import mozpack.path as mozpath
mozpath.func()
because it's shorter.
$ git grep mozpath\\. | wc -l
423
$ git grep mozpack.path\\. | wc -l
123
This change was done with:
$ git grep -l mozpack.path\\. | xargs sed -i 's/mozpack\.path\./mozpath./g'
$ git grep -l 'import mozpack.path$' | xargs sed -i 's/import mozpack.path$/\0 as mozpath/'
$ (pat='import mozpack.path as mozpath'; git grep -l "$pat" | xargs sed -i "1,/$pat/b;/$pat/d")
We're using as many defaults from the configure step as we can. We're also opinionated upon the defaults, but obviously allow most compare-locales options to be specified.
There are two exceptions:
Reference language is specified to be en-US, without optional argument. This is our in-tree command, and the reference language is known.
We always clobber the merge dir, and don't give an option not to. We default to a merge dir in the objdir, so we don't need to be that paranoid as in the standalone version.
Also, compare-locales clobbers merge-dir/browser etc, so you're not going to get / removed.
--HG--
extra : rebase_source : c0f63e566779e83201708d05966f3583ae82e4ee
This change includes a few fixes to other issues in mach file-info, such
as error handling for invalid paths not working properly.
--HG--
extra : rebase_source : fca43650678d944730273353a5a9154a63247c58
extra : histedit_source : 314e7702ca061005525bd4e054360f79db68935b
Now that we have a mechanism for defining file-based metadata, let's add
a mach command to interface with it.
Currently, we limit ourselves to simple Bugzilla data dumping. Features
will be added over time.
--HG--
extra : rebase_source : 20fa72cac6e6d5aff7973b7dd200c03c8f238639
The Files sub-context allows us to attach metadata to files based on
pattern matching rules.
Patterns are matched against files in a last-write-wins fashion.
The sub-context defines the BUG_COMPONENT variable, which is a 2-tuple
(actually a named tuple) defining the Bugzilla product and component for
files. There are no consumers yet. But an eventual use case will be to
suggest a bug component for a patch/commit. Another will be to
automatically suggest a bug component for a failing test.
--HG--
extra : rebase_source : 9489738136d929a53db7f54bbe6acf3186e0a47c
We want the ability to read data from any moz.build file without needing
a full build configuration (running configure). This will enable tools
to consume metadata by merely having a copy of the source code and
nothing more.
This commit creates the EmptyConfig object. It is a config object that -
as its name implies - is empty. It will be used for reading moz.build
files in "no config" mode.
Many moz.build files make assumptions that variables in CONFIG are
defined and that they are strings. We create the EmptyValue type that
behaves like an empty unicode string. Since moz.build files also do some
type checking, we carve an exemption for EmptyValue, just like we do for
None.
We add a test to verify that reading moz.build files in "no config" mode
works. This required some minor changes to existing moz.build files to
make them work in the new execution mode.
--HG--
extra : rebase_source : f701417f83dfa4e196e39182f8d0a6fea46c6fbb
extra : source : af07351bf2d6e85293ae3edf0fe4ae6cbc0ce246
Building on top of the API to retrieve relevant moz.build files for a
given path, we introduce a moz.build reading API that reads all
moz.build files relevant to a given set of paths. We plan to use this
new API to read metadata from moz.build files relevant to a set of
files.
This patch changes the generator behavior of read_mozbuild to emit the
main context before any processing occurs. This allows downstream
consumers to manipulate state of the context before things like
directory processing occurs. We utilize this capability in the new
reading API to forcefully declare the directory traversal order for
processed moz.build files, overriding DIRS and similar variables.
Since variable exporting doesn't work reliably in this new traversal
mode, variable exporting no-ops when this mode is activated.
--HG--
extra : rebase_source : aa5c67f275e8eb83516dc97648cc8fea62b52815
extra : source : 6c44edc8208a54a9d5d830266cded7b409a776e1
Currently, MozSandbox assumes that the FUNCTIONS, SPECIAL_VARIABLES, and
SUBCONTEXTS data structures are the instances that should be associated
with the sandbox. As we introduce new moz.build processing modes that
wish to change processing behavior, it is necessary for them to have
control over these special symbols.
This patch moves the declaration of these types to the special metadata
dictionary which is inherited during recursion. The "read_topsrcdir" API
now explicitly passes the initial metadata into "read_mozbuild".
--HG--
extra : rebase_source : 59ad9ac7c288a5b610d7e3b513aaa0f5b2ec2009
extra : source : ed135df395751194bf379584a4d210f14ac849b4
An upcoming patch introduces a use case for a strongly typed named
tuple. So, we introduce a generic factory function that can produce these
types.
--HG--
extra : rebase_source : 7f4d17ff28925fbe8d850c036605aa03a38f0ef2
extra : source : acdd5491f10ecf8ea4e1a14150f9a2e282e2cf5d
We have an eventual goal to store file-level metadata in moz.build files
and to have this metadata "cascade" down directory hierarchies. e.g.
metadata in the root directory will apply to all children directories.
A prerequisite for this feature is a way to query which moz.build files
are relevant to a given file. In this patch, we implement an API that
returns this information.
--HG--
extra : rebase_source : 70726926aeddf89df27c46dc2f503dc8a3194633
extra : source : 4013d256b5910404cc04bb3caaf696b8ee551fc5
The inputs to scripts for GENERATED_FILES are restricted to filenames
only. We have several examples in the tree, however, where a script
takes non-filename arguments. For converting those cases to use
GENERATED_FILES, we first need to provide some way of "injecting"
non-filename arguments into the script.
This commit adds a method for doing that, by extending the .script flag
on GENERATED_FILES to include an optional method name:
f = GENERATED_FILES['foo']
f.script = 'script.py:make_foo'
will invoke the make_foo function found in script.py instead of the
function named main.
The inputs to scripts for GENERATED_FILES are restricted to filenames
only. We have several examples in the tree, however, where a script
takes non-filename arguments. For converting those cases to use
GENERATED_FILES, we first need to provide some way of "injecting"
non-filename arguments into the script.
This commit adds a method for doing that, by extending the .script flag
on GENERATED_FILES to include an optional method name:
f = GENERATED_FILES['foo']
f.script = 'script.py:make_foo'
will invoke the make_foo function found in script.py instead of the
function named main.
As content in moz.build files has grown, it has become clear that
storing everything in one global namespace (the "context") per moz.build
file will not scale. This approach (which is carried over from
Makefile.in patterns) limits our ability to do things like declare
multiple instances of things (like libraries) per file.
A few months ago, templates were introduced to moz.build files. These
started the process of introducing separate contexts / containers in
each moz.build file. But it stopped short of actually emitting multiple
contexts per container. Instead, results were merged with the main
context.
This patch takes sub-contexts to the next level.
Introduced is the "SubContext" class. It is a Context derived from
another context. SubContexts are special in that they are context
managers. With the context manager is entered, the SubContext becomes
the main context associated with the executing sandbox, temporarily
masking the existence of the main context. This means that UPPERCASE
variable accesses and writes will be handled by the active SubContext.
This allows SubContext instances to define different sets of variables.
When a SubContext is spawned, it is attached to the sandbox executing
it. The moz.build reader will now emit not only the main context, but
also every SubContext that was derived from it.
To aid with the creation and declaration of sub-contexts, we introduce
the SUBCONTEXTS variable. This variable holds a list of classes that
define sub-contexts.
Sub-contexts behave a lot like templates. Their class names becomes the
symbol name in the sandbox.
--HG--
extra : rebase_source : 5df4bcf073ce46605b972021f1e918ce4affa6f3
The regular expression cache for mozpack.path.match was keyed off the
original pattern. However, that variable was mutated as part of the
function and the mutated result was subsequently stored as the cache
key. This effectively resulted in a 0% cache hit rate.
On some tests being written for bug 1132111 which involve a full
filesystem traversal for moz.build files and subsequent execution of
those files, the following timings are indicative of the impact of this
patch.
Before:
real 16.082s
user 14.760s
sys 1.318s
After:
real 6.345s
user 5.085s
sys 1.257s
--HG--
extra : rebase_source : caf4a9f37fda9b43b444059f647535e1b085d422
Support for a callback to be executed post sandbox evaluation was added
in 24b43ecb4cad (bug 949906) to unbust Sphinx as a result of some GYP
processing changes. e93c40d4344f and bug 1071012 subsequently changed
how Sphinx variables are extracted from moz.build, removing the only
consumer of this feature.
Since there are no consumers of this feature left, remove it and make
the code simpler.
--HG--
extra : rebase_source : 95bb619aba0bb00719e060a4cf346b9dd51eb541
extra : amend_source : ef195e478b122a18f89ad0b2b2b1aab4a4090e42
extra : histedit_source : 36a464ace2fe261b1315544c9ee6a23fc0ab71a2
The value of cache files or cache size might decrease under these
circumstances.
* The user manually changes the max cache size to a value smaller than
the current cache size.
* The cache size is reaching the cache max size.
We should not assume both values will be increased after the build is
finished.
The Android ARchive contains the compiled Gecko libraries that Firefox
for Android interfaces to. It does not contain the Gecko resources
(the omnijar, omni.ja) nor the compiled Java code (classes.dex).
This also uploads metadata and sha1 hashes for future consumption by
Maven and/or Ivy dependency managers. In some brave future world,
we'll work out exactly what that looks like; for now, this solves a
storage problem (each .aar file is ~20MB) and it's possible to point
Gradle directly at the uploaded Ivy metadata and artifacts.
--HG--
extra : rebase_source : 0c12b44f587d4a027ca5258bae8fcbb6f6028c24
Previously, we raised errors attempting to register a mach sub-command
if the parent command was not defined on the Registrar. This broke B2G
because its mach command Registrar imposes restrictions on which
commands it exposes. Commands there were not getting registered on the
Registrar, leading mach to give a false positive that the parent command
was never defined.
We change the verification logic to take present but unregistered
commands into consideration and to skip registering sub-commands if the
parent isn't present in the Registrar.
--HG--
extra : rebase_source : 150eea0ca85d9f2c9cd9e98f5bf905695da53bf4
Now that we have proper moz.build objects for GENERATED_FILES, we can
add 'script' flags and 'args' flags in moz.build for select
GENERATED_FILES. We restrict 'args' to being filenames for ease of
implementing checks for file existence, and many (all?) of the examples
of file generation throughout the tree don't need arbitrary strings or
Python data.
This patch is mostly useful for being able to see these changes
independently of the major changes to GENERATED_FILES. We are going to
need proper moz.build objects for GENERATED_FILES when we add the
ability to define scripts and arguments for them, so we might as well do
that first.
With the previous changes, we can now tell Visual Studio about the
actual unified files, rather than the files that are #include'd in them.
I believe this is much closer to what Visual Studio wants to see, and
enables things like Intellisense to work properly.
CommonBackend is where the writing of the unified files belong, since
that's an operation common to all backends. A special hook into
subclasses is used to enable subclass-specific processing of
UnifiedSources.
UnifiedSources will be processed outside of consume_object, so we need
some way of accessing the backend file for an object outside of
consume_object as well.
UnifiedSources, not the recursivemake backend, should be responsible for
figuring out what unified files to generate, and what those unified
files should contain.
Generating the list of idl deps to generate an xpt from its dependency list
makes us give all _previous_ dependencies, inherited from the .deps makefiles.
This leads to removed files being listed on xpidl-process.py command line, and
the command subsequently failing.
Instead, use generated lists of idl dependencies. At the same time, lighten the
generated Makefile further by not emitting xpt dependencies on their containing
directory, and instead generating it from the $xpt_files list.
This brings down the Makefile size from 100k to 38k.
The previous debugger was setting a breakpoint in the mach dispatcher.
This required users to step into the main function to be called into.
Using pdb.runcall(), the debugger starts at the first line in the
executed command, which is far more useful.
--HG--
extra : rebase_source : 10734805ad40ec85eedbb089b11d618dc32398e7
extra : amend_source : da89da1af1f137db2f5902bcb5a69312f4636a4b
Similar to the changes made for IPDL files, this commit moves all of the
non-makefile related logic for WebIDL files out of the recursive make
backend and into the common build backend. Derivative backends that
would like to do interesting things with WebIDL files now need to
implement _handle_webidl_build, which takes more parameters, but should
ideally require less duplication of logic.
After a bunch of tiny changes, we're finally ready to make real
progress. We can now move the grouping of the generated IDPL C++ files
and the actual writing of the unified files for them into the common
build backend. Derivative backends now only have to concern themselves
with adding the particular logic that compiling those files requires.
We'll need to write out unified files for multiple backends, not just
the recursive make one. Put that logic someplace where all build
backends can access it.
_add_unified_build_rules shouldn't be in the business of determining how
to group files into their unified files. That logic belongs in the
caller of _add_unified_build_rules. Once that's done, the logic for
determining how to group files can migrate out of the recursive make
bakend.
Nothing about writing unified files is specific to the recursive make
backend, and if we want to write the unified files for IPDL and WebIDL
files, we'll need this functionality available in the common build
backend.
RecursiveMakeBackend._group_unified_files doesn't contain any
functionality specific to the recursive make backend. We would also
like to move the unification of generated IPDL and WebIDL source files
into the common build backend. Moving _group_unified_files into the
common build backend would be the logical place for it, but the frontend
should also be able to handle unifying files so that backends don't have
to duplicate logic for UNIFIED_FILES. Therefore, we choose to move it
to mozbuild.util as its final resting place.
Pushing on a CLOSED TREE since Android build only.
--HG--
extra : rebase_source : cc99efa734d1f4738d3f026c930a4d1955723783
extra : amend_source : 52c07807a77f263d2eb2593826dc0285928d9be4
This handles:
list.0=A
list.1=B
list.sublist.0=C
so that
list=>[A, B]
list.sublist=>[C]
and
dict=default
dict.key1=A
dict.key2=B
so that
dict=>{key1:A, key2:B}
dict=>default
--HG--
extra : rebase_source : 2c8f5941d2fca9c56b3858d3e98078a43fa69090
DONTBUILD NPOTB on a CLOSED TREE
This was tested by ally and mfinkle in #mobile.
--HG--
extra : amend_source : 5dd061d6ef98f49cb232baa7221ec514a32eeeee
This does two things. First, it aligns the brew formula name
(AndroidNdk) with the brew file name (android-ndk.rb). Second, it
makes sure that we actually find the android-ndk.rb file. I think
|mach bootstrap| always worked, due to a felicity where the working
directory always contained android-ndk.rb; but |python bootstrap.py|
failed because android-ndk.rb was downloaded to a temporary location
that was not included in the |brew install| command.
--HG--
rename : python/mozboot/mozboot/android-ndk-r8e.rb => python/mozboot/mozboot/android-ndk.rb
extra : rebase_source : 44f164d3d5916bc2faf4c64285e232047daea35e
This addresses issues with offline builds - due to pip latest version checking
Source archive downloaded from
https://pypi.python.org/packages/source/v/virtualenv/virtualenv-12.0.5.tar.gz
and uncompressed into python/virtualenv. The egg-info directory was
removed.
--HG--
extra : amend_source : 30a79486d91d9f79a36535eab438c0cd6f81daf3
Writing the unified files is another thing that will have to be moved
out of recursivemake.py eventually. And it doesn't belong inline amidst
makefile rules and variables. Move its logic to a separate function as
well.
_add_unified_build_rules does quite a lot of work besides adding
makefile rules and variables. The divying up of source files into
unified files is one part of that, so move it out into its own function.
When we eventually move that computation out of recursivemake.py, this
refactoring will make it easier to verify that's what we've done.
terminal.py had an ambiguous |import logging| that was importing
mach.logging from Sphinx. We fix it.
There was also a poorly formed link in the mach commands documentation.
We fix it.
--HG--
extra : rebase_source : 12783c69027989ac031d29e4ecbc1ee2f465ffa4
extra : histedit_source : 4283c6cdecc4de8aa7636d0c4cc566daf5142b50
Python API documentation requires the ability to import modules. So, we
set up a virtualenv in our Sphinx environment so module loading works.
This solution isn't perfect: a number of modules fail to import when run
under sphinx-build.
--HG--
extra : rebase_source : fce732e0b8aefe0e9a2ee594b8a08ac02e27579a
extra : histedit_source : bef27c947b95c3182fbc6cd656ae8c96acaaa6f4
Previously, code for staging the Sphinx documentation from moz.build
metadata lived in a mach command and in the moztreedocs module. This
patch moves the invocation to the Sphinx extension.
When the code is part of the Sphinx extension, it will run when executed
with sphinx-build. This is a prerequisite to getting RTD working, since
sphinx-build is the only supported entrypoint for generating
documentation there.
With this patch, we can now invoke sphinx-build to build the
documentation. The `mach build-docs` command is no longer needed.
--HG--
extra : rebase_source : 86e76c7d598ffa23dae858254eecedbdd12706a4
extra : histedit_source : 1826aa5ddfafdff62847cc293d1f0950b236caac
The recursivemake backend knows how to do several things with the IPDL
sources:
1) Determine the C++ sources that will be generated from given IPDL
sources.
2) Write out all the makefile rules and variables for said sources.
The first part isn't unique to the recursivemake backend; other backends
would eventually like to know what C++ sources come from IPDL source
files for easier cross-referencing purposes, etc. Let's take a first
cut at moving things into CommonBackend. (This may not be the best
interface, since it relies on consume_finished being invoked, and not
all backends call CommonBackend.consume_finished. Still, it's a start.)
psutil 2.1.3 is replacing psutil 1.0.1. There are numerous bug fixes and
feature enhancements in psutil worth obtaining.
Source code was obtained from
https://pypi.python.org/packages/source/p/psutil/psutil-2.1.3.tar.gz and
uncompressed into python/psutil without modification except for the
removal of the egg-info directory and the .travis.yml file.
--HG--
extra : source : 697eb6db7d96dc21e817cd27a7e46ed4ab00f9bb
psutil 2.1.3 is replacing psutil 1.0.1. There are numerous bug fixes and
feature enhancements in psutil worth obtaining.
Source code was obtained from
https://pypi.python.org/packages/source/p/psutil/psutil-2.1.3.tar.gz and
uncompressed into python/psutil without modification except for the
removal of the egg-info directory and the .travis.yml file.
--HG--
extra : rebase_source : 90b02e514e5dd1249d97d83223ef8fcf20c9da94
extra : histedit_source : 2e96344e1796b6156ccf279684687b2e8995be92
Source archive downloaded from
https://pypi.python.org/packages/source/v/virtualenv/virtualenv-12.0.2.tar.gz
and uncompressed into python/virtualenv. The egg-info directory was
removed.
--HG--
extra : rebase_source : e96572c694b2e780ed143a08b3c6670273a39fdb
extra : histedit_source : 17d490c968fdadd4dacad3b880bfcb1fc7b3e148
Pushing on a CLOSED TREE because this is NPOTB.
The custom brew formula is a lightly edited version of an earlier
revision of brew's android-ndk.rb. It's not clear that using a custom
brew formula for the Android SDK version r8e is better than using the
existing android Python module for installing the SDK and the NDK, but
it's done now and works locally. If we really wanted to avoid shipping
it, we could probably arrange to land it in
https://github.com/Homebrew/homebrew-versions.
I see no easy way to install the Java 1.7 JDK with macports, so in the
spirit of the good now being better than the perfect later, I've
punted. (I don't see an Android NDK package either, but that
functionality exists in Python.) Patches wanted!
--HG--
extra : rebase_source : d3ca836951b05eb30fabc17e523676d31c1ec0ef
extra : amend_source : d34e9eb5cd840b827a7ce65a0d22c1c275efff09
This prepares the way for implementing install_mobile_android_packages
on OS X.
--HG--
extra : rebase_source : b52b7080f857e351e71f169f09a42dfd61b43a43
This adds a generic android Python module that handles:
* downloading and unpacking Google's Android SDK and NDK bundles;
* using the |android| tool to install additional Android packages;
* printing a mozconfig snippet suitable for mobile/android builds.
--HG--
extra : rebase_source : abaffcee70eacb391f53d622fba4a445277818db
This lays the foundation for adding support for bootstrapping
mobile/android.
* Queries for which application to bootstrap, currently browser or
mobile/android;
* Adds call to install_APPLICATION_packages after
install_system_packages;
* Adds call to suggest_APPLICATION_mozconfig after bootstrapping
everything;
* and splits install_browser_packages out of install_system_packages
throughout (essentially untested, but generally simple).
To implement a new application (b2g?), just add it to the list of
applications and implement install_b2g_packages throughout.
--HG--
extra : rebase_source : 8c9b4485dbe5ceb64436fd73ea0258592a2b818f
Various bits of the test harnesses key off of mozinfo.info.get('asan');
we will need a similar switch for finding out whether this build
supports tsan.
Now that the mozbuild backend knows about FINAL_TARGET, we are able to
install generated xpt files into their final location. This saves us
from copying xpt files into their final location on every build.
Original patch by gps, rebased and comments addressed by Ms2ger
--HG--
extra : transplant_source : %E2%DC%0F%E0%AD%C2%25%A1%B8%A9%FE%B0%8C%60%FF%CB%02G%25%E5
mopack.BaseFile.copy() performs a generic read/write file copy. Windows
has an explicit CopyFile() call that tests have shown to be
significantly faster. Let's use that instead via the magic of ctypes.
Having SOURCES and its close relatives go through VariablePassthru
objects clutters the handling of VariablePassthru in build backends and
makes it less obvious how to handle things that actually get compiled.
Therefore, this patch introduces four new moz.build objects
corresponding to the major variants of SOURCES. It looks like a large
patch, but there's an ample amount of new tests included, which accounts
for about half of the changes.
An upcoming patch will reuse this code. Split it out into its own
function.
--HG--
extra : rebase_source : 2057266a1ed4d86b1ca857712cd279bdfbd32148
extra : histedit_source : 8f2b7ec06e5b72ab08586902c51c3025a53259e6
With this patch, we now have the ability to declare mach sub-commands.
Subsequent patches will implement dispatching for sub-commands.
--HG--
extra : rebase_source : 32a672ff3ba7a1a1c8c27dff3fe657dec08e3569
extra : histedit_source : 2bb2c27ae0e2dab052a6ef93b0a70f488f8cd56e
We will soon have proper sub-commands. Let's fix function names.
--HG--
extra : rebase_source : 1cb0c008ecc0d2159dd799e76adad4892d65f119
extra : histedit_source : 9472b2d2609224b011eca04385cf9d5a49986864
mach conditions are implemented elsewhere. This comment can die.
--HG--
extra : rebase_source : c956d5af7858d997a1f49f3b24bb54e5f8a52240
extra : histedit_source : 76b8155b12ab0ea86e16e807b1c6f691cb8f3640
Documentation was referencing sub commands. The proper terminology is
"command."
--HG--
extra : rebase_source : aa2fc73c6141850109324fd5299491e49ef29e91
extra : histedit_source : da075bea5a36a9de81527d93a9fc751cd6cae9b2
There were a few parts of the docs that were linking to Python types
incorrectly. Let's fix that.
--HG--
extra : rebase_source : 374be660084727eba1ce9181c2abf384f40eeb82
extra : histedit_source : bfa8de9b7a277666d92ebe71cd64ffed96628518
The main mach docs page is a bit long. Let's split it into multiple
articles to increase readability going forward.
--HG--
rename : python/mach/docs/index.rst => python/mach/docs/commands.rst
rename : python/mach/docs/index.rst => python/mach/docs/driver.rst
rename : python/mach/docs/index.rst => python/mach/docs/logging.rst
extra : rebase_source : 484d60327568333fcb0069e1f3444dc6db4322c0
extra : histedit_source : 18d09ac2e2e93565661763b6567f7a46226735f5
Now that mach's docs are part of a proper Sphinx install, link to
Python type docs. While we're here, we also clean up formatting of code
samples.
--HG--
extra : rebase_source : 974337316dc87c04aca79a9e4a0d464490e5cdd4
extra : histedit_source : db87e8e5cb7c9c1969fd89e58c39623862684a6a
mach has restructured text documentation. Let's expose it to the in-tree
Sphinx config so we can increase visibility of the documentation.
--HG--
rename : python/mach/README.rst => python/mach/docs/index.rst
extra : rebase_source : 68a08bebaa5b99ca39e250fa1ed60ca4d7d0af68
extra : histedit_source : 6afda2fece486547192c1c8eba97a2474e3d99c1
Now that defining $DMD is no longer necessary to run DMD, this patch does the
following.
- Removes all the places where we set DMD=1 (test harnesses, etc.)
- Still handles DMD=1, for backwards compatibility.
- Prints "$DMD is undefined" at DMD start-up if appropriate.
- Writes a |null| value for |dmdEnvVar| in the JSON if $DMD is undefined. Bumps
the DMD output version number accordingly.
- Changes a bunch of the test files accordingly, including changing the mode of
script-ignore-alloc-fns.json in order to test a case where $DMD is undefined.
--HG--
extra : rebase_source : eb1ef5722410734ce6d7658465ff6f442ee4ed49
Various os.path attributes are being used in tight loops. Having local
variables prevents extra dictionary lookups.
This appears to shave 10-20ms off of the tests install manifest
processing time.
--HG--
extra : rebase_source : de941f2978cf0b1fd7c4f7401c848b61d406a2c8
extra : amend_source : e33c896856fa559197496b8227e10ab8149d146e
