Some manifest entries (e.g. skin or locale) have an attached identifier,
and there can be different entries with different identifiers for the
same chrome name. The change from bug 1366169 would consider those as
errors, while they are the expected configuration.
--HG--
extra : rebase_source : ceb08da909121a2ac0a2cdaba7970e4594dde09f
This allows us to write files coming from a finder or other source
that isn't directly the filesystem.
MozReview-Commit-ID: KhPSD0JYzsQ
--HG--
extra : rebase_source : 10d494fc910982c3e34f4744592edca906d3a85d
As described in changeset c94e87a18096, chrome manifest entries are
reordered and don't necessarily appear in the order they have in jar.mn.
And in some cases, the order does matter: when entries with flags are
followed by entries with more broad flags or no flags at all. Nobody
should be writing entries in that order on purpose, so error out during
packaging when we detect the pattern.
--HG--
extra : rebase_source : d9617bbcbd8560503c532a13c10c8afb0fd49411
This is the initial support of 'mach repackage', which can take an
existing tarball and create a DMG on either an OSX host or on a Linux
host with cross-OSX tools. Configure is needed in order to find the
tools necessary to create the DMG. On a Linux cross-compiled environment
with tooltool, this can be as simple as:
export MKFSHFS=$topsrcdir/hfsplus-tools/newfs_hfs
export DMG_TOOL=$topsrcdir/dmg/dmg
export HFS_TOOL=$topsrcdir/dmg/hfsplus
ac_add_options --disable-compile-environment
MozReview-Commit-ID: 6t2rlXpwUvu
--HG--
extra : rebase_source : 39f2f9ecac9d7e5af197f1b5dea70cd307acf488
This removes the UNIFY_DIST and UNIFIED_BUILD variables, as well as the
--unify flag from the packager and UnifiedBuildFinder from mozpack. As a
result the STAGEPATH variable is never defined anymore, so its uses can
be removed as well.
test_unify.py is currently the only mozbuild/mozpack test that fails
without running configure first, and there isn't much point in fixing
tests for things that we don't actually use anymore.
MozReview-Commit-ID: F5q1FPW3Did
--HG--
extra : rebase_source : cadbd237f51c23ea1983135294521d628d16f0df
CLOSED TREE
Backed out changeset 158233bce738 (bug 1197325)
Backed out changeset b5ac3fa0bbe7 (bug 1197325)
Backed out changeset 55a8ad127517 (bug 1197325)
Back when the class was written, for the packaging code, it made sense
that the default was True. But now that it's used all over the place,
and that the vast majority of uses are with find_executables=False, it
makes more sense for that to be the default.
--HG--
extra : rebase_source : ff813735fc0d53093f348f20eb77ee03e9b09d4e
And make it an error not to give it. While the default is True, we do
pass a value of False wherever it makes sense, which happens to be, in
most places.
This is an intermediate step to flip the default from True to False,
ensuring that we don't unwantedly switch some calls to False.
--HG--
extra : rebase_source : 432e03f032fef378af482581685583262e5d2661
Make the symlinks in the dist directory relative instead of absolute.
MozReview-Commit-ID: HS7KL4JwSbV
--HG--
extra : rebase_source : 5dca673cc17423d47e6707d8800f7ee9693a9c48
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
As of Python 3, decimal notations of octal values for permission modes
are no longer permitted and will result in a `SyntaxError` exception
(`invalid token`).
Using the proper octal notation which is also Python 2.7 compatible will
fix this issue.
--HG--
extra : rebase_source : 2e897c51f04ad0ee69071f84b98df224f3af72d3
Mingw python has a different os.path setup from native python, and has
os.sep and os.altsep reversed. In that case, the normsep function was
doing the wrong thing, leading to all sorts of problems.
While fixing this, also ensure the corresponding unit test covers this
peculiarity, even when running under the native win32 python.
--HG--
extra : rebase_source : 8fb18e0d4dc669c1d7e069f73fc44c22d419d43c
I have a need to create tar archives deterministically
and reproducibly. Since we already have similar functionality in
mozpack for producting zip/jar archives, I figured it made sense for
this functionality to live in mozpack.
I made the functionality as simple as possible: we only accept
files from the filesystem and the set of files must be known in
advance. No class to hold/buffer state: just a simple function
that takes a mapping of files and writes to a stream.
MozReview-Commit-ID: If0NTcA7wpc
--HG--
extra : rebase_source : 9cbea36347ba2840dd5bff9dffefd994a73a0725
This will be needed to teach artifact builds to extract files from
omni.ja files whose content is loaded into memory (from a tar
archive).
MozReview-Commit-ID: LH2HkKx5Zj3
--HG--
extra : rebase_source : 7f6b176f0ef9fc87889151d1d02da62de8a455d8
extra : source : 266928b5a7615fa054c70adf0f649cbb3f085e8d
Generally speaking, the configuration needs forward-slashes in paths.
We might as well make it hard(er) to set configuration items with
backslash separators on Windows by exposing mozpath.* functions in place
of os.path functions. The downside is that functions explicitly
importing os will still get the real os.path functions.
As previous measurements have shown, creating/appending files
on Windows/NTFS is slow because the CloseHandle() Win32 API takes
1-3ms to complete. This is apparently due to a fundamental issue
with NTFS extents. A way to work around this slowness is to use
multiple threads for I/O so file closing doesn't block execution
as much.
This commit updates the file copier to use a thread pool of 4
threads when processing file copies. Additional threads appear
to have diminishing returns.
On my i7-6700K, this reduces the time for processing the tests install
manifest (24,572 files) on Windows from ~22.0s to ~12.5s in the best
case.
Using the thread pool globally resulted in a performance regression
on Linux. Given the performance sensitivity of manifest copying,
I thought it best to implement a slightly redundant non-Windows
branch to preserve performance. For the record, that same machine
running Linux is capable of processing nearly the same install
manifest (24,616 files) in ~2.2s in the best case.
MozReview-Commit-ID: B9LbKaOoO1u
--HG--
extra : rebase_source : e9fee3861a70e1da9d18448f8435f4bd3e28c647
This is two straightforward optimizations in FileCopier: avoiding a redundant iteration
over the directory structure to find destination files (which includes a
call to normpath) and avoiding redundant calls to determine directories to preserve
when remove_unaccounted is not specified (which include a call to dirname).
Running a no-op install of _tests with this patch results in a reduction of about
25,000 calls to normpath and remove about 220,000 calls to dirname, resulting in
an overall speedup of 10-20%.
--HG--
extra : commitid : 8nyTo489q8X
