This patch adds the ability to skip tests to Talos. This lets us prevent tests from running depending on which branch we are on. It also adds some code that will skip tests on the pine branch.
Differential Revision: https://phabricator.services.mozilla.com/D152513
Now that 102.1.0esr has shipped the Firefox-esr-next-msix-latest-SSL
product is populated in bouncer so we can enable the check.
Differential Revision: https://phabricator.services.mozilla.com/D152764
This patch adds a new set of tests (essential only) to run in CI at tier 2. A command line argument is added to turn bytecode cache testing on in warm pageloads. This also disables the testing in our primary pageload tests to fix the variance and increases in runtime.
Differential Revision: https://phabricator.services.mozilla.com/D151757
This greens up the openh264 build tasks and gets them running with python 3.
I had a look at modernizing gittool.py, but that seemed complicated and unnecessary for our needs: we just need to checkout the git repo.
There's some discussion about the Windows CFLAGS. I had a look and tried various approaches, but this was really the only way that worked for me (thanks jcristau!). Certainly I'm open to other suggestions...or we can revisit later.
This change does not affect the signing format, which still requires work for OSX.
Differential Revision: https://phabricator.services.mozilla.com/D149578
updated patch for android_hardware_unittests.py, asking for a review- please look at the interdiff to see recent changes.
Differential Revision: https://phabricator.services.mozilla.com/D144985
On Android long lines are truncated by logcat, meaning the base64
images output by wrench's reftest harness get truncated. The glue code
used by the old version of cargo-apk automatically split long lines,
which we then stitched back together again when parsing the
log. However, the glue used by the new cargo-apk no longer does this.
While we could manually split the long strings ourselves, this method
has always been unreliable. Instead, this patch makes us additionally
write stdout and stderr to a file as well as logcat. The test runner
script then pulls this file from the device and scrapes it rather than
the logcat.
Differential Revision: https://phabricator.services.mozilla.com/D145076
This patch forces us to re-install browsertime if FFMPEG is not found in the cache. This fixes an issue where deleting the `.mozbuild/browsertime` folder would go undetected and cause failures later when running browsertime.
Alongside these changes, the patch also enables detecting for FFMPEG on Linux, and Windows machines. It works in the same way as mac, but it also requires using `browsertime_ffmpeg` since the `os.environ` settings don't get transferred to the required environment.
Depends on D143603
Differential Revision: https://phabricator.services.mozilla.com/D144464
This patch runs the visual-metrics processing within the test task rather than the (deleted) `-vismet` tasks. It also updates the browsertime version to v15. It requires an update to node16 so new code was added to pull, and install the *-node-16 toolchain artifacts from CI.
This patch is possible because the browsertime update gives us the new `visualmetrics-portable.py` script which doesn't require ImageMagick anymore (they've been converted to Python dependencies). We did this conversion here: https://github.com/sitespeedio/browsertime/pull/1741
Furthermore, some changes are made to handle timeouts better, and increase the timeout for an interactive test.
Depends on D143502
Differential Revision: https://phabricator.services.mozilla.com/D142838
This patch adds the ability to raptor-browsertime for downloading, and installing a node binary from our Taskcluster CI. This makes it possible to seamlessly use raptor-browsertime with node-16, while using node-12 for the rest of the mozilla-central tooling.
Depends on D142837
Differential Revision: https://phabricator.services.mozilla.com/D143502
This uses a patched version of glutin 0.28 which builds successfully
on Android. It has the caveat that the application is now responsible
to ensure we only create a GL context when the application has been
resumed and the window is valid. This patch does so by spinning an
event loop on startup until we receive a Resume event. This is a bit
of a hack, and will break if the app is minimised, but it is good
enough for wrench's use case.
Cargo-apk no longer supports specifying a separate target_sdk_version
and android_version, meaning we must use a target_sdk_version of
31. This means we no longer have permission to read from "/sdcard", so
wrench and its scripts have been updated to use the application's
"external data dir".
Finally, when running on CI we use a patched version of cargo-apk
which allows building with SDK version 31 and NDK r21d. We should be
able to switch to the upstream git version once we update to NDK r23.
Differential Revision: https://phabricator.services.mozilla.com/D144418
This patch adds the ability to provide custom browsertime arguments through the command line option `--browsertime-arg`. It is used like so:
`
./mach raptor --browsertime -t browsertime --browsertime-arg test_script=/home/sparky/mozilla-source/mozilla-central/testing/raptor/browsertime/browsertime_pageload.js --browsertime-arg browsertime.url=https://www.sitespeed.io --browsertime-arg iterations=3
`
Differential Revision: https://phabricator.services.mozilla.com/D144166
The idea is to use this configuration to pass a DAMP suite name to the DAMP tests without having to create separate talos test classes.
Rather than passing a hardcoded `damp_suite` value, `extra_prefs` might be useful for other tests and at least should be generic.
Differential Revision: https://phabricator.services.mozilla.com/D141966
The idea is to use this configuration to pass a DAMP suite name to the DAMP tests without having to create separate talos test classes.
Rather than passing a hardcoded `damp_suite` value, `extra_prefs` might be useful for other tests and at least should be generic.
Differential Revision: https://phabricator.services.mozilla.com/D141966
