Add configurations for building and uploading AArch64 Nightly builds, in
tier 1 and without artifact support for now.
As for not denoting AArch64 builds as "api-21", I don't really think we
will split AArch64 the way we split ARMv7 before. Originally, we split
into API 9 and API 11+ because of lots of "constrained" devices that
were stuck with API 9. We made an API 9 APK in order to lower our
footprint on those devices. That probably will not be a problem for
AArch64, because devices with API 21+ and AArch64 support are usually
more than capable for running Fennec. Secondly, it was a big change for
Android going from API 9 to API 11+, so we saved quite a bit of
code/resources when we stripped out API 11+. I don't see such drastic
changes going from API 21 to upcoming versions, so even if we did split,
I don't think it'll get us much benefit.
MozReview-Commit-ID: 7N7Slv1pPgb
Several years ago there was a single zip file for all test files. Clients
would only extract the files they needed. Thus, zip was a reasonable
archive format because it allowed direct access to members without
having to decompress the entirety of the stream.
We have since split up that monolithic archive into separate,
domain-specific archives. e.g. 1 archive for mochitests and one
for xpcshell tests. This drastically cut down on network I/O
required on testers because they only fetched archives/data that
was relevant. It also enabled parallel generation of test archives,
we shaved dozens of seconds off builds due to compression being
a long pole.
Despite the architectural changes to test archive management, we
still used zip files. This is not ideal because we no longer access
specific files in test archives and thus don't care about single/partial
member access performance.
This commit implements support for generating tar.gz test archives.
And it switches the web-platform archive to a tar.gz file.
The performance implications for archive generation are significant:
before: 48,321,250 bytes; 6.05s
after: 31,844,267 bytes; 4.57s
The size is reduced because we have a single compression context
so data from 1 file can benefit compression in a subsequent file.
CPU usage is reduced because the compressor has to work less with
1 context than it does with N. While I didn't measure it, decompression
performance should also be improved for the same reasons. And of course
network I/O will be reduced.
mozharness consumers use a generic method for handling unarchiving.
This method automagically handles multiple file extensions. So as long
as downstream consumers aren't hard coding ".zip" this change should
"just work."
MozReview-Commit-ID: LQa5MIHLsms
--HG--
extra : rebase_source : cd029cdbbcccc1d16f03d63a5f1fdf60be5db4fd
extra : source : a0e257e346ccf3c1db332ec5903241f4eeb9a7ee
The current indexed task has a big rank number, we blocks a new indexed
task update.
Add an index rank to uploader task big enough to replace the old indexed
task.
MozReview-Commit-ID: Hg5Ya4D0qYt
--HG--
extra : rebase_source : c08386e5fe62ba4ee4e2832539b6aebf015d6799
This patch will allow jittest, jsreftest, and any other test suite to run on linux64-ccov regardless of whether or not the files in 'files-changed' have been matched with changes in a patch.
MozReview-Commit-ID: 78CWUTWBhV
--HG--
extra : rebase_source : 4e8bc8a43a6be2b15e2b51c986da9e5ba5adc039
Tasks should be assigned a priority based on the branch they originated from. It
is important that certain branches receive preferential treatment, such as a release
branch task being executed before a task from Try. Branch priority mirrors
the priorities defined within buildbot.
MozReview-Commit-ID: 8qR9F34lzzc
--HG--
extra : source : 6a2176a264959e19e4a4cbcb18fd47b9653745f8
Several years ago there was a single zip file for all test files. Clients
would only extract the files they needed. Thus, zip was a reasonable
archive format because it allowed direct access to members without
having to decompress the entirety of the stream.
We have since split up that monolithic archive into separate,
domain-specific archives. e.g. 1 archive for mochitests and one
for xpcshell tests. This drastically cut down on network I/O
required on testers because they only fetched archives/data that
was relevant. It also enabled parallel generation of test archives,
we shaved dozens of seconds off builds due to compression being
a long pole.
Despite the architectural changes to test archive management, we
still used zip files. This is not ideal because we no longer access
specific files in test archives and thus don't care about single/partial
member access performance.
This commit implements support for generating tar.gz test archives.
And it switches the web-platform archive to a tar.gz file.
The performance implications for archive generation are significant:
before: 48,321,250 bytes; 6.05s
after: 31,844,267 bytes; 4.57s
The size is reduced because we have a single compression context
so data from 1 file can benefit compression in a subsequent file.
CPU usage is reduced because the compressor has to work less with
1 context than it does with N. While I didn't measure it, decompression
performance should also be improved for the same reasons. And of course
network I/O will be reduced.
mozharness consumers use a generic method for handling unarchiving.
This method automagically handles multiple file extensions. So as long
as downstream consumers aren't hard coding ".zip" this change should
"just work."
MozReview-Commit-ID: LQa5MIHLsms
--HG--
extra : rebase_source : 19ec875917546abc147b234a815e1a64c204b92a
The purpose of this parameter has been superseded by the `include_nightly`
property.
MozReview-Commit-ID: 4iXQsv9Drqg
--HG--
extra : rebase_source : c94142282909a88c29fe6809d87721bef1f198c2
This simplifies the return value of optimize functions:
* False -- don't optimize
* True -- optimize away
* <taskId> -- replace with this task
Original patch by ayodeji.oyewole, with test updates by dustin.
MozReview-Commit-ID: HKoWcINVSnV
--HG--
extra : rebase_source : 1d22d61030eb2bf95d4b9d0803a40e0be96c8830
extra : source : ae32a34822c40dbf7f9129edd701b945f6c95b3a
This allows `make check` to pass when run against these builds.
MozReview-Commit-ID: BVs6BPifbfW
--HG--
extra : rebase_source : ebad40260b160744e0d2f9dfc52a74b7e5e5ae0e
Tasks should be assigned a priority based on the branch they originated from. It
is important that certain branches receive preferential treatment, such as a release
branch task being executed before a task from Try. Branch priority mirrors
the priorities defined within buildbot.
MozReview-Commit-ID: 8qR9F34lzzc
--HG--
extra : rebase_source : 29f61899bfad6cdf1eeb902e5a5779ab782690c4
