We now vendor pip via the vendored copy of virtualenv. Stop trying to install a
specific version of pip elsewhere, particularly stop downgrading it.
Differential Revision: https://phabricator.services.mozilla.com/D962
--HG--
extra : rebase_source : abb5c076747f229aa4379fa3989c9abf36b45d0b
extra : source : 747d0b814dc1be3e5ac04e080361ab0b0fc034f9
To ease investigation of failures the gecko log should be streamed
to stdout so it will be part of the default log. It helps with
correlating tracing output with the appropriate test.
MozReview-Commit-ID: JnH64bhhtgk
--HG--
extra : rebase_source : b50707189c181a865ab66dac8b3cb4e258a8e427
This makes the changes necessary to use TestRunnerActivity when geckoview
is installed and requested, but we do not yet attempt to run any such
test tasks in automation.
Android jit tests are not ready to run yet: They may not run green, there are concerns about
capacity, etc. I am adding this support now to make it more convenient to debug on try.
--HG--
extra : rebase_source : 00bc5bf21fec3c130133b0de322b1f37250893c3
You can still run them on a --disable-stylo build, as long as that works
(presumably not for long).
I think I haven't missed anything, but please double-check.
MozReview-Commit-ID: 3BIAEjgTLo5
This backs out the source readme changes, and gets us to the original source tarball, but massaged for taskcluster's signing+beetmover model.
MozReview-Commit-ID: QIaeQ9LdLb
--HG--
extra : rebase_source : 4677497da550e98a4d07c16169c0949c3ec495b9
The configuration doesn't need to be dynamic, so removed the code that existed
to support it.
Differential Revision: https://phabricator.services.mozilla.com/D542
--HG--
extra : rebase_source : 34b317b846ed4d8c54344b2379bbd4100a8623e5
extra : source : 36171867cedd5160dc230470033a60d31a7fbee5
The upload is handled by taskcluster, which is handled by copying files around,
so remove support for specifying alternative commands.
Differential Revision: https://phabricator.services.mozilla.com/D540
--HG--
extra : rebase_source : 968249c0a308178b62af77d48e6aa307d4192e5a
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
In general, there is no simple mapping between apk name (geckoview_example.apk) and
package name (org.mozilla.geckoview_example). With bug 1434411, it will be relatively
easy to add or modify tasks to use a geckoview apk in taskcluster tests. At the
mozharness level, scripts are expected to expand mozharness configurations containing
"%(app)" into package names. For Firefox, android_emulator_unittest extracts and
reads package_name.txt, but there is no such file in the geckoview apk. In future
we might add package_name.txt to the geckoview apk, or possibly use a tool like aapt,
but for our immediate needs this simple hack does the job: If "geckoview" is in
the apk name, assume we are installing org.mozilla.geckoview_example.
When the emulator is in good shape, emulator verification on shutdown
only takes a few seconds. However, if the emulator is hung or otherwise
unresponsive, emulator verification can take several minutes. In the
pathological worst case, verification slows down the task enough to
exceed the task timeout, resulting in a failure that would otherwise
have been handled as a task retry. Emulator verification on shutdown
provides little value, so it is simply eliminated.
It is no longer used as part of the build, now that publishing is handled in
separate tasks.
Differential Revision: https://phabricator.services.mozilla.com/D373
--HG--
extra : rebase_source : a6e31fe5d554076d4da584f0993c21b6c1a8ebd0
When MOZHARNESS_TEST_PATHS is set, the test suite mozharness scripts
will run the paths specified there instead of the normal chunking
and/or default manifest. Paths should be separated by a ':' character.
In the case of web_platform_tests.py, we have to make the test paths
relative to 'testing/web-platform'.
MozReview-Commit-ID: IHRXXi5mB4G
--HG--
extra : rebase_source : 17b31ec19a64ab16918d0bd80d19d9bb496cbe37
We use bogomips as a convenient indicator or emulator performance/health and
make that available in the android-performance.log artifact. When the task
times out, that artifact is not uploaded, leaving me curious about what the
bogomips were; let's dump it to the test log.
Check /proc/cpuinfo on android and extract the "bogomips" reading: If it is < 250, retry
the task, since there appears to be a higher probability of tests running too slowly
in such environments.
OSX (cross) repackages are currently using a tooltool manifest to get
libdmg and hfsplus. Change those jobs to use the toolchain artifacts
instead.
At the same time, modify the repackage mozharness script's _run_tooltool
so that it doesn't fail with MOZ_TOOLCHAINS being set but without a
tooltool_manifest_src, matching the similar function in buildbase.py.
--HG--
extra : rebase_source : d128d4709c5d1d28d1a6b9c585fde82e99f725c7
We're about to remove some tooltool manifests, so we need those calls to
work properly when TOOLTOOL_MANIFEST is not set.
--HG--
extra : rebase_source : 89d41021a87915dc9133e61543352e3bda1dace4
In bug 749312, we were given permission to create a source readme
instead of a source tarball. This will save us cycles, disk, and
human configuration time.
We still need to address the missing balrog_props.json for
beetmover-source for that task to turn green.
MozReview-Commit-ID: wnyPoNXCsH
--HG--
extra : rebase_source : 843751523e1fce5743849f43796788dbba5115d3
extra : histedit_source : 2993eb186dc7bd71ad35af48d4393803b0b147dc
Updates the mozharness script to run the Marionette command by using
the test folder as current working directory. This will make sure
that the relative path to the tests is reported. It's identical to
the location in the tree.
MozReview-Commit-ID: 6hOQnJSqfv0
--HG--
extra : rebase_source : b54f2a928d47b369b4102a920532aee0503534df
All of these options only exist in the configuration files and a referenced
nowhere else in-tree.
MozReview-Commit-ID: CMeu3H1hqdx
--HG--
extra : rebase_source : df366b686557e474037534335a757ace445ba8b9
It seems psutil can throw a wide range of exceptions when accessing system
information on aws...intermittently, of course. Let's expect and discard
such exceptions so that test jobs are not dependent on creating system-info.log.
According to sfink, mozharness is no longer used to drive hazard
builds. That means a lot of dead code that can be removed.
After this commit, there are no more references to "hazard" or
"spidermonkey" in testing/mozharness.
MozReview-Commit-ID: 8MWl8dMwRTD
--HG--
extra : rebase_source : 2156fbd13dffb22bb08b10fec2a66a9eebde8d57
These scripts are included by hazard-analysis.sh. That's their only
reference in repo.
We could probably inline these scripts. But let's start by moving them
out of mozharness since no active mozharness based task is using them.
MozReview-Commit-ID: 13oen42Txmh
--HG--
rename : testing/mozharness/scripts/spidermonkey/build.browser => taskcluster/scripts/builder/hazard-browser.sh
rename : testing/mozharness/scripts/spidermonkey/build.shell => taskcluster/scripts/builder/hazard-shell.sh
extra : rebase_source : 782f7b3f3537cfefb51b0e5f1b459c8ad0daca5b
I'm seeing "try" in my repacks, when the underlying build has
"nightly-try". This should make the two agree.
MozReview-Commit-ID: 45yE9Qwz0v7
--HG--
extra : rebase_source : ff1ae4e50203ea032032069203558d75d348ff21
Single-locale repacks need to run aapt (--without-gradle) or Gradle
(--with-gradle). When running --with-gradle, they need to compile the
Java source code again (in order to produce a fresh R.java with
correct IDs). That compile will be part of the shipping APK, so it
needs to be configured "the same" as the underlying repacked. *This
is a significant change in behaviour, but necessary to support newer
Gradle/aapt versions, which do not maintain R.java ID mappings across
invocations.*
Part of the configuration are the secret keys and features that are
gated on them. This commit makes those secrets available to
single-locale repacks.
MozReview-Commit-ID: 4REPsIb5TgN
--HG--
extra : rebase_source : 2d23e0e0c51a61e50acf24123b316bdbb0b579ff
extra : source : a721890f7573140ca6a926e722bd3538c732dae7
I'm seeing "try" in my repacks, when the underlying build has
"nightly-try". This should make the two agree.
MozReview-Commit-ID: 45yE9Qwz0v7
--HG--
extra : rebase_source : 8b524470680e248c649bf3e4e532cdd5487ec538
Single-locale repacks need to run aapt (--without-gradle) or Gradle
(--with-gradle). When running --with-gradle, they need to compile the
Java source code again (in order to produce a fresh R.java with
correct IDs). That compile will be part of the shipping APK, so it
needs to be configured "the same" as the underlying repacked. *This
is a significant change in behaviour, but necessary to support newer
Gradle/aapt versions, which do not maintain R.java ID mappings across
invocations.*
Part of the configuration are the secret keys and features that are
gated on them. This commit makes those secrets available to
single-locale repacks.
MozReview-Commit-ID: 4REPsIb5TgN
--HG--
extra : rebase_source : 9a74ea5f6633d1a4bd52d6116b60edaf974d11eb
extra : source : a721890f7573140ca6a926e722bd3538c732dae7
The old code was simply running configure and manually invoking some
make targets via client.mk. These can both be done via `mach`.
As part of the change, the build targets have been consolidated. There
is still an abstraction leak here. But at least we aren't using client.mk.
MozReview-Commit-ID: 7oMXPWPZS6V
--HG--
extra : rebase_source : 6d632dc086d79a17e577da66336c77003d963f67
Upgrade from optparse to argparse:
1. 'type' field now needs to be callable (deleted if type was 'string' as that is the default)
2. 'extend' action re-implemented for argparse
3. 'callback' action no longer exists, re-implemented as a custom argparse action (only used in buildbase.py)
4. minor api changes, e.g 'add_option' -> 'add_argument'
MozReview-Commit-ID: HcKowF13Da3
--HG--
extra : rebase_source : e5e8160d91263fb273f790dbda5e2c2b2e02eaf6
Bug 1382564 made the `mach artifact toolchain` invocations from
mozharness use the MOZ_TOOLCHAINS environment variable when it's set by
taskcluster through the decision task, so that toolchain dependencies
from the task graph are used, but the mozharness code is still skipping
mach artifact toolchain when MOZ_TOOLCHAINS is set but there is no
tooltool manifest set. Most jobs today still have a tooltool manifest
set, but jobs shouldn't need a dummy tooltool manifest to use toolchain
dependencies automatically.
--HG--
extra : rebase_source : 0437a8f3d43a83ffe32c4192f86ee9a621977e3e
This patch:
- removes the obsolete mozilla-aurora l10n-bumper config.
- adds both central and beta format desktop bumper configs to jamun for testing.
- updates the central and beta configs to add desktop.
- updates the script to support the desktop configs.
We now support an `ignore_config` which acts like the `ignore-platforms` attribute.
MozReview-Commit-ID: KGwo0bRibw4
--HG--
extra : rebase_source : 1014c8d46104fc3b05586aa64f207cf38f37f98f
This is to avoid hitting https://github.com/pypa/pip/issues/510.
MozReview-Commit-ID: 7TK4DdbpKRD
--HG--
extra : rebase_source : ffef899d80864a81a81a1a332a9da1f949a05551
Complications:
- had to copy ReftestManifest into a test zip
- reftest harness was emitting multiple suite_start log entries with --repeat
- some extra path manipulation required to find reftests
This uploads a JSON summary of the results, without extra logging or
expectation data or anything. It is mostly useful for comparing the
results from two runs e.g. in a dashboard.
MozReview-Commit-ID: Ac45NVBxhy8
--HG--
extra : rebase_source : 2c7a82d6e46b8c9f7af8ad34559e51d84ba4e4ba
Notable changes
* ensure we run dump-symbols and upload actions on all platforms
* On android:
* add configuration and support for aarch64
* set min_sdk levels to match Fennec builds
* use a full copy of the r11c ndk (our truncated one was missing toolchains we needed) and set NDKROOT when calling build
* ensure the tooltool provided sdk is on the PATH
* on linux copy tooltool.py into the mock environment, so we can get dump_syms from tt
* remove macosx32 config as we've deprecated that in Firefox builds
* update dump_syms to recent m-c, notably for aarch64 support on linux
* on linux rev e365137fa61bfd729617ba1ebf9f1ed79facd1f2 (via try 0f72a5c28be1cdc2f3bdfaafdf3826254f6ba077)
* on mac rev e365137fa61bfd729617ba1ebf9f1ed79facd1f2 (via compile on a bld-lin-r5)
* on windows rev a4a448ba7f187069fce916ee234a06cbb0d06f80 (via try dc8b121e3c08e8022d62c0fa1951dd3dc4d6f7cc)
* switch to Visual Studio 2015 Update 3 on win32/win64 to match Firefox
* many updates to environement variables
* painful to get win64 right to run win32 dump_syms.exe, but that's why the x86 redist is on teh PATH
* unwind the changes to get_output_from_command() in v1.6 patch to avoid affecting other builds, and use query_env() which has this support already
* add a scp_upload_directory since we don't have rsync on windows, use that to talk to the ffxbld upload host (not a long term solution but OK for now)
Applies on top of https://reviewboard.mozilla.org/r/64022/diff/4#index_header
MozReview-Commit-ID: B3NiWFvr2oR
This will change all build symbols to 'Ba' and set the USE_ARTIFACTS=1 environment variable.
Mozharness will detect this env to decide whether to perform an artifact build or not.
MozReview-Commit-ID: J8HVZzOt4mX
--HG--
extra : rebase_source : 453028d9be5cb2ad07e9a2a8b769cb6aac9893fe
Ahem.ttf is copied to $(DIST)/bin/firefox/fonts, but this file is broken due to text mode copy. So we should use binary mode instead.
MozReview-Commit-ID: KP7yNyPiejU
--HG--
extra : rebase_source : 2de749f458a6d4650f9044f1912ff97835c5b795
