b227363d66bb added a config item and code for disabling package
metrics, which don't work everywhere.
The previous logic was faulty.
MozReview-Commit-ID: BGxWgmWxXyH
Pushing on a CLOSED TREE
--HG--
extra : amend_source : 99ca472e4c0fca306deec4038e9c7cbfe5412b0a
mozharness is Python. self.query_exe('python') could resolve to a
different Python interpreter from what mozharness is running as.
In order to promote consistency, always invoke python processes with
the Python being used to run mozharness.
In some cases, this may cause former `python` processes to run as
Python 2.7 instead of 2.6 (since `python` resolves to a 2.6 interpreter
on many systems). It may also result in slightly different Python
binaries being used. But I think sharing interpreters between the
mozharness script and launched processes is logical. So if this causes
problems, I'd like to flush those out.
MozReview-Commit-ID: KfawUvT5jgW
--HG--
extra : source : b6f04897fdda51e42612617a89a93f696edbdf92
extra : amend_source : 32dafc7c9dc2cec80bc289bd1a17cdbb8cde5025
Previously, mozharness defined a separate action to collect build
metrics. This required the script and/or config to define that
action.
Metrics collection for CI is important. So it should be enabled by
default.
This commit changes the "build" action/method to always call the
metrics collection function after successful build. References to
the "generate-build-stats" action have been removed because it is
redundant.
A side-effect of this change is we may generate build metrics where
we weren't before. This could lead to e.g. duplicate entries in some
Perfherder series. Let's see what breaks ;)
MozReview-Commit-ID: 42UQI5YQTMC
--HG--
extra : rebase_source : c57dc9ec6ac46003384edff098a0ad81c75539b7
extra : source : c9812dd7d27a174c0ee46d44ec595fbe29c9e1db
We're about to enable metrics collection for all builds. There are some
Android build configurations that use buildbase.py but don't create a
package. So we need a way to conditionally obtain package metrics.
We could change package metrics collection to no-op if a package file
can't be found. However, that has a risk that a future change could
break metrics collection and we wouldn't necessarily find out. I like
things that fail fast.
MozReview-Commit-ID: CzByf7yHVS8
--HG--
extra : rebase_source : 99ee18ed4dd61e5ea8f5eda1b810b573fe254158
A subsequent commit will make all this code conditional. Rather than
indent the world, it is easier to conditionally call a function.
A benefit of the new code is that we skip some code for debug builds,
which is one less thing that can break.
MozReview-Commit-ID: fiUNBbikmy
--HG--
extra : rebase_source : aeb151ea5864d0f97db20bee921b60afc00aee61
Previously, this ran during postflight_build(). The magic postflight_*
methods are called automagically by BaseScript.run_action() and are
only called if the main action method didn't raise. So there should
be no functional difference with this commit.
The reason I changed this is that a subsequent commit will perform
metrics generation from build() and without the build properties
file loaded, at least the OS X 64 opt buildbot build doesn't have
packageFilename defines, which breaks metrics collection.
MozReview-Commit-ID: 54ftuQqGKVi
--HG--
extra : rebase_source : c3c28426468474a7aa51a10787d01ebbba10dd82
extra : source : 387d8415d05e7f1dc96ed3adb441c54f232baf0d
mozharness is Python. self.query_exe('python') could resolve to a
different Python interpreter from what mozharness is running as.
In order to promote consistency, always invoke python processes with
the Python being used to run mozharness.
In some cases, this may cause former `python` processes to run as
Python 2.7 instead of 2.6 (since `python` resolves to a 2.6 interpreter
on many systems). It may also result in slightly different Python
binaries being used. But I think sharing interpreters between the
mozharness script and launched processes is logical. So if this causes
problems, I'd like to flush those out.
MozReview-Commit-ID: KfawUvT5jgW
--HG--
extra : rebase_source : 8babadc464ea4d8971e091d5446d86d2630e07b9
A number of developers find it convenient to build with
--disable-optimize --enable-debug for an improved debugging experience.
We don't currently have a configuration in CI that ensures this
combination of options works, so various changes break builds with this
configuration every so often. We should test such configurations to
ensure they build to provide a smooth experience for developers.
This patch enables linux64-ccov to run JSVM code coverage collection at the same time GCOV code coverage is being collected. It uploads the code coverage that was created in a file called 'code-coverage-jsvm.zip'.
MozReview-Commit-ID: 4x5GrjRJRo4
--HG--
extra : rebase_source : 461eb6a0d6bf9f16a83899a1147c0f9cf2c02823
The main motivation behind this change is that going towards toolchain
tasks hooked up in the task graph (bug 1313111), we're going to end up
with jobs using both taskcluster toolchain job and tooltool artifacts
for their toolchain needs. With the current setup, this means the
toolchain dependencies will be spread between taskcluster task graph
definition and mozharness configuration.
It also makes things more complex to provide a command that pulls the
right toolchains from both taskcluster and tooltool (bug 1356529),
because one needs to find and parse the mozharness config (which also
happens to be python code that uses platform-specific things, so e.g.
reading windows mozharness config fails on other platforms).
All in all, moving the tooltool manifest path to the taskcluster task
definitions would make things simpler, and would also allow make patches
switching from tooltool to taskcluster artifacts more straightforward to
validate.
But since some build types still run on buildbot, we'll have to keep
part of the current setup using mozharness configs. So we allow to
override the tooltool manifest path from the environment, and we'll rely
on taskcluster task definitions being able to set environment variables.
Actually moving the relevant tooltool manifest paths from mozharness
config to taskcluster task definitions is left for a followup.
Another followup is to move the tooltool manifest paths declared in
some ad-hoc build scripts to taskcluster task definitions as well.
The immediate need for this, though, is to allow to have duplicated jobs
that only differ in their tooltool manifest, without duplicating a
complete mozharness config that will end up stale (the goal being that
really only the tooltool manifest differs, even when the original jobs
change).
--HG--
extra : rebase_source : 3622779926b1b5e86e809c1f6422bd55ef64eed7
In addition to saving the log level of 'process_output' messages, this will also start passing 'log'
messages through the error lists. This means mozharness will start using 'log' errors when determining
the tbpl_status and worst_log_level.
MozReview-Commit-ID: CZnH6aI1Wo0
--HG--
extra : rebase_source : 28c00105d3fa14d16b637b4275b29e6604f4691d
As of bug 1342503 being fixed, all of our desktop firefox builds have
webrender compiled in by default. Webrender can therefore be enabled at
runtime either by a pref or environment variable on any desktop firefox
build. The old builds that we originally used to stand up webrender are
no longer needed, as the *only* difference between them and the regular
builds are that they build with the pref turned on instead of turned
off. This doesn't warrant keeping around these extra builds, and this
patch removes them along with all the associated goop that was needed to
configure them.
MozReview-Commit-ID: 5wlOWo11fEk
--HG--
extra : rebase_source : 696afdd2d9fb5f7932d0737a7d71c3aa6af0bd64
The `remove_exes.py` config left mozharness searching for exes, but we do need
a path to tooltool, since it is not in $PATH.
MozReview-Commit-ID: I9gk8rNOmda
--HG--
extra : rebase_source : 9482321128b5d8da085bd38e088649c88ea6294c
This flag can be used to allow gcov for c++ code coverage to run with '--code-coverage' but it prevents 'gcda' artifacts from being uploaded after the tests are done. The 'gcno' upload is allowed with another flag in the code-coverage config called 'MOZ_CODE_COVERAGE'. Removing it's definition will prevent the 'gcno' file from being uploaded.
MozReview-Commit-ID: 1XkH0P4Bh5A
--HG--
extra : rebase_source : 4340e83de39709a7068e469d7f7d6b1e434c9afd
In addition to saving the log level of 'process_output' messages, this will also start passing 'log'
messages through the error lists. This means mozharness will start using 'log' errors when determining
the tbpl_status and worst_log_level.
MozReview-Commit-ID: CZnH6aI1Wo0
--HG--
extra : rebase_source : 55c74bfa2afdf6d7b510b351ad657ecd615d4347
Add configurations for building and uploading AArch64 Nightly builds, in
tier 1 and without artifact support for now.
As for not denoting AArch64 builds as "api-21", I don't really think we
will split AArch64 the way we split ARMv7 before. Originally, we split
into API 9 and API 11+ because of lots of "constrained" devices that
were stuck with API 9. We made an API 9 APK in order to lower our
footprint on those devices. That probably will not be a problem for
AArch64, because devices with API 21+ and AArch64 support are usually
more than capable for running Fennec. Secondly, it was a big change for
Android going from API 9 to API 11+, so we saved quite a bit of
code/resources when we stripped out API 11+. I don't see such drastic
changes going from API 21 to upcoming versions, so even if we did split,
I don't think it'll get us much benefit.
MozReview-Commit-ID: 7N7Slv1pPgb
We have a few places that we have scheduled talos tests as part of a sendchange, but have since disbled those tests (android, linux32), or replaced with taskcluster (linux64, macosx). This was discovered when artifact builds were scheduling all talos tests in duplicate for osx.
MozReview-Commit-ID: 6Ze4Ic0GQrY
