In order to meet the addon signing requirement for tests, specialpowers
needs to be installed at gecko runtime. This means it must be restartless.
This patch packages specialpowers as a restartless addon, but it does not
yet install it at runtime.
--HG--
rename : testing/specialpowers/components/SpecialPowersObserver.js => testing/specialpowers/content/SpecialPowersObserver.jsm
extra : commitid : Bbg8gFten8S
extra : rebase_source : 99d0b841a1f8f9eb6b6ca846363a158836482e57
This builds the Robocop tests with |mach build mobile/android|, making
it easier for developers to build Fennec and the tests at the same
time.
--HG--
rename : build/mobile/robocop/AndroidManifest.xml.in => mobile/android/tests/browser/robocop/AndroidManifest.xml.in
rename : build/mobile/robocop/Makefile.in => mobile/android/tests/browser/robocop/Makefile.in
rename : build/mobile/robocop/README => mobile/android/tests/browser/robocop/README
rename : build/mobile/robocop/moz.build => mobile/android/tests/browser/robocop/moz.build
rename : build/mobile/robocop/res/values/strings.xml => mobile/android/tests/browser/robocop/res/values/strings.xml
rename : build/mobile/robocop/robotium-solo-4.3.1.jar => mobile/android/tests/browser/robocop/robotium-solo-4.3.1.jar
extra : commitid : BuNBjgXdm1d
extra : rebase_source : c36b8bf0183d8f5821b7f7839668ca963065d894
extra : histedit_source : a86fef3b834420ea496a9c2644ca72786a2d7da9
Mozharness isn't part of the test package, so shouldn't really be packaged during 'package-tests'.
--HG--
extra : commitid : LIK7qJMSPHi
extra : rebase_source : 644c8dd6e0e993b063709ba06e9634d52cb6596d
Won't impact performance much. But fewer make foo makes porting the C++
unit tests (which are the largest remaining tests) to the Python
archiver easier to grok.
This conversion did change behavior slightly. Previously, startup
cache files weren't being packaged if startup cache was disabled. Now,
we always package them since their presence in the test archive should
be harmless. The original change to guard their inclusion in
ee82e0ae5488 was probably unnecessary.
--HG--
extra : commitid : AzU65j0E1q0
extra : rebase_source : 9b8a15dc1a5f3c3d3e453cefb3a99b05f5a77711
This prevents copying of 447 files adding to ~4 MB.
--HG--
extra : commitid : 7zTbiQeMQSQ
extra : rebase_source : b3ac223835ba7289ace45aa7d02c5a050d54cc0d
This saves copying of ~100 files comprising ~1 MB. Not significant. But
it gets us a little closer to no staging.
--HG--
extra : commitid : 6Hjnhv4Yi5R
extra : rebase_source : 291c89682a23cde957b3c68f2efe3b6dc3d3d543
This is slightly more involved than earlier changes because reftests
have a one-off mechanism for finding files. Essentially, the master
reftest manifest is loaded, directories are discovered, and every file
in those directories is packaged.
We add support to our test archive generation tool to read sources from
reftest manifests and tell it where the reftest manifests are.
print-manifest-dirs.py was only being used for staging reftest files.
Since we don't do that any more, the functionality doesn't need to exist
in a standalone file, so it has been moved inline into test_archive.py.
This change avoids copying ~26,000 tests consuming 131 MB during test
packaging. This is a majority of the file count that was remaining in
the stage directory at this point. On my machine (which hasn't typically
seen major wall time wins from not staging files due to its fast SSD),
this change made test packaging ~20% faster, reducing wall time from
~50s to ~40s!
A Try push seemed to indicate drastic results with the series up to this
point. Including the already landed changes to generate test archives
concurrently, test packaging times on OS X builders dropped from ~18:40
to 6:29! Times on Linux x64 remained about the same (~2:46). This is
possibly due to these machines already having SSDs and due to normal
variance in performance of builders and EC2 instances.
--HG--
extra : commitid : 34E8V8lSGg7
extra : rebase_source : 720afcd35f6a2b6cb1217df23ae981408a88cb94
This avoids copying 5000+ files consuming ~37 MB on my build
configuration.
--HG--
extra : commitid : 6DmsjUYgjXq
extra : rebase_source : 123dd42a7d0b9cc244a3ab7773010dfc5769a4ac
With this change, all test ZIP archives are now generated via Python and
mozpack.
This change does not change I/O or file copy behavior at all. There is
still a lot of room for eliminating extra file copies.
--HG--
extra : commitid : 9mWdtDK6wAb
extra : rebase_source : 0f19c627d64d22bf9d65161d4f7df7c9778dea3c
This doesn't change I/O or copy behavior at all. But it does remove a
one-off make rule.
--HG--
extra : commitid : X0efdFHA0k
extra : rebase_source : c7cb8616461eccd1ff7f8eb3b409bd4944c9e1ec
This is pretty straightforward. This saves ~26 MB of file copies.
--HG--
extra : commitid : ItghoP73zS8
extra : rebase_source : 9656719a6459c1e6fa28165591722fe00d6d9b1d
The web-platform test archive now builds without any staging at all.
This saves ~103 MB of file copies on my machine.
The testing/web-platform/Makefile.in serves no purpose after this
change, so it and all references to it have been removed.
--HG--
extra : commitid : HDHGG3QGVBH
extra : rebase_source : dd7302aad96b46932aa00e4e66918c8077475b10
This is very similar to what we did for xpcshell. Like xpcshell, there
are still some staged files. However, about 73MB of copies are
eliminated with this change. On my machine, overall execution time of
test packaging appears to decrease, although CPU usage is up slightly.
--HG--
extra : commitid : 5dy340X80J9
extra : rebase_source : d37be29367b17e6c1d9c885ab4705932b7a42b39
This commit produces the xpcshell test archive without staging 5000+
xpcshell test files first.
We teach the archiver to ignore .mkdir.done files.
The xpcshell Makefile.in still stages some files. This is less than
ideal. However, it is a small handful of files and shouldn't add too
much overhead.
This appears to not impact overall CPU usage significantly on my
machine, despute using Python instead of `zip`. It does reduce I/O
by ~25MB by avoiding the staging copy.
--HG--
extra : commitid : IwvLaYvAbFt
extra : rebase_source : a690ae4b1adbabd491851a2479fa66d81241601b
Test archive generation currently copies a bunch of files into a staging
area then runs `zip` to produce ZIP files. There are 2 concerns with
this approach:
1) We incur a lot of extra I/O to copy files so everything is
rooted in a single tree so the `zip` invocation and paths are
simple.
2) ZIP files inherit properties from the local filesystem (including
mtime), making ZIP files non-deterministic.
This commit introduces a new mozbuild action for producing test
archives. It does so using the mozpack file finder and JAR writer,
which are used throughout the build to deterministically
produce ZIP/JAR files from files in multiple source directories.
We implement support for producing the mozharness archive. This archive
does not involve files that are staged, so no I/O is saved. In fact,
the switch from `zip` to Python likely makes this slightly slower.
However, we do have deterministic archives now.
Additional archives will be ported over in subsequent commits.
--HG--
extra : commitid : H1BOidPDZST
extra : rebase_source : 120e2bfea921e5fb3a8d97b2dd0227edce452cfd
Previously, we had a single make target and rule for generating all test
archives. These tasks can be performed in parallel. This commit
refactors the make file to add multiple targets for each archive and
thus enables test archives to be generated concurrently.
On my MacBook Pro, this reduces `make package-tests -j8` from ~78s to
~50s, a reduction of ~28s, or ~36%. Reduction on machines without SSDs
(like many builders in automation) will likely be less. Although, the
page cache should service most file reads during archiving since these
files were just staged, so hopefully the gains are in the same ballpark.
--HG--
extra : commitid : 1PsEuE8KS7J
extra : rebase_source : b4ed9e593775d96a994b7076c6814d18ca277d8e
extra : histedit_source : f33af390df36d8ffe98df4f4329fa50a7cde6b0c
Upcoming work will introduce multiple targets for building test
archives. To prepare for this, we introduce a phony target that
tracks the staging of all test files so each target can gate on a common
prerequisite.
--HG--
extra : commitid : KcdyQv7sZqk
extra : rebase_source : a64336881990d65e11451b3f3858e4ff5c200eca
extra : histedit_source : 3cbbbd73d31f82108125dd6593c5974f2f42184c
This makes reftest command line arguments behave more like other test suites,
so we can use a simple unified syntax for e.g. |mach try|. The patch also
reworks the command line argument parsing to use argparse rather than optparse,
and causes mach to reuse the same parser as the suite.
Backed out changeset 58331e57de1c (bug 917999)
Backed out changeset 50f9123412c7 (bug 917999)
Backed out changeset 3b19643ec039 (bug 917999)
CLOSED TREE
