From 39c859e008beafcc291bb0f230e6af886ad00ba8 Mon Sep 17 00:00:00 2001 From: Andreas Tolfsen Date: Tue, 24 Apr 2018 08:16:24 +0100 Subject: [PATCH] NO BUG - Eliminate one header level in Taskcluster docs. r=me MozReview-Commit-ID: GLZowUaSJ3i --- testing/marionette/doc/Taskcluster.md | 65 ++++++++++++++------------- 1 file changed, 33 insertions(+), 32 deletions(-) diff --git a/testing/marionette/doc/Taskcluster.md b/testing/marionette/doc/Taskcluster.md index d0d387c1fd72..eb18dbbfafed 100644 --- a/testing/marionette/doc/Taskcluster.md +++ b/testing/marionette/doc/Taskcluster.md @@ -1,4 +1,5 @@ -# Taskcluster +Testing with one-click loaners +============================== [Taskcluster] is the task execution framework that supports Mozilla's continuous integration and release processes. @@ -6,19 +7,14 @@ continuous integration and release processes. Build and test jobs (like Marionette) are executed across all supported platforms, and job results are pushed to [Treeherder] for observation. -[Taskcluster]: https://docs.taskcluster.net/ -[Treeherder]: https://treeherder.mozilla.org - -## Testing with One Click Loaners - The best way to debug issues for intermittent test failures of Marionette tests for Firefox and Fennec (Android) is to use a one-click loaner as provided by Taskcluster. Such a loaner creates an interactive task you can interact with via a shell and VNC. -To create an interactive task for a Marionette job which is -shown as failed on Treeherder, follow the Taskcluster -documentation for [Debugging a task]. +To create an interactive task for a Marionette job which is shown +as failed on Treeherder, follow the Taskcluster documentation for +[Debugging a task]. Please note that you need special permissions to actually request such a loaner. @@ -29,40 +25,45 @@ provide some options. Best here is to choose the second option, which will run all the setup steps, installs the Firefox or Fennec binary, and then exits. +[Taskcluster]: https://docs.taskcluster.net/ +[Treeherder]: https://treeherder.mozilla.org [Debugging a task]: https://docs.taskcluster.net/tutorial/debug-task#content -### Setup the Marionette environment +Setting up the Marionette environment +------------------------------------- Best here is to use a virtual environment, which has all the necessary packages installed. If no modifications to any Python package will be done, the already created environment by the wizard can be used: - % cd /builds/worker/workspace/build - % source venv/bin/activate + % cd /builds/worker/workspace/build + % source venv/bin/activate Otherwise a new virtual environment needs to be created and populated with the mozbase and marionette packages installed: - % cd /builds/worker/workspace/build && rm -r venv - % virtualenv venv && source venv/bin/activate - % cd tests/mozbase && ./setup_development.py - % cd ../marionette/client && python setup.py develop - % cd ../harness && python setup.py develop - % cd ../../../ + % cd /builds/worker/workspace/build && rm -r venv + % virtualenv venv && source venv/bin/activate + % cd tests/mozbase && ./setup_development.py + % cd ../marionette/client && python setup.py develop + % cd ../harness && python setup.py develop + % cd ../../../ -### Running Marionette tests +Running Marionette tests +------------------------ -#### Firefox +### Firefox To run the Marionette tests execute the `runtests.py` script. For all the required options as best search in the log file of the failing job -the interactive task has been created from. Then copy the complete +the interactive task has been created from. Then copy the complete command and run it inside the already sourced virtual environment: - % /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/marionette/harness/marionette_harness/runtests.py --gecko-log=- -vv --binary=/builds/worker/workspace/build/application/firefox/firefox --address=localhost:2828 --symbols-path=https://queue.taskcluster.net/v1/task/GSuwee61Qyibujtxq4UV3A/artifacts/public/build/target.crashreporter-symbols.zip /builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit-tests.ini + % /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/marionette/harness/marionette_harness/runtests.py --gecko-log=- -vv --binary=/builds/worker/workspace/build/application/firefox/firefox --address=localhost:2828 --symbols-path=https://queue.taskcluster.net/v1/task/GSuwee61Qyibujtxq4UV3A/artifacts/public/build/target.crashreporter-symbols.zip /builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit-tests.ini + #### Fennec @@ -74,20 +75,20 @@ The following lines set necessary environment variables before starting the emulator in the background, and to let Marionette know of various Android SDK tools. - % export ADB_PATH=/builds/worker/workspace/build/android-sdk-linux/platform-tools/adb - % export ANDROID_AVD_HOME=/builds/worker/workspace/build/.android/avd/ - - % /builds/worker/workspace/build/android-sdk-linux/tools/emulator -avd test-1 -show-kernel -debug init,console,gles,memcheck,adbserver,adbclient,adb,avd_config,socket & + % export ADB_PATH=/builds/worker/workspace/build/android-sdk-linux/platform-tools/adb + % export ANDROID_AVD_HOME=/builds/worker/workspace/build/.android/avd/ + % /builds/worker/workspace/build/android-sdk-linux/tools/emulator -avd test-1 -show-kernel -debug init,console,gles,memcheck,adbserver,adbclient,adb,avd_config,socket & The actual call to `runtests.py` is different per test job because those are using chunks on Android. As best search for the command and its options in the log file of the failing job the interactive -task has been created from. Then copy the complete command and run it -inside the already sourced virtual environment. +task has been created from. Then copy the complete command and run +it inside the already sourced virtual environment. -Here an example for chunk 1 which runs all the tests in the current chunk -with some options for logs removed: +Here an example for chunk 1 which runs all the tests in the current +chunk with some options for logs removed: - % /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/marionette/harness/marionette_harness/runtests.py --emulator --app=fennec --package=org.mozilla.fennec_aurora --address=localhost:2828 /builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit-tests.ini --disable-e10s --gecko-log=- --symbols-path=/builds/worker/workspace/build/symbols --startup-timeout=300 --this-chunk 1 --total-chunks 10 + % /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/marionette/harness/marionette_harness/runtests.py --emulator --app=fennec --package=org.mozilla.fennec_aurora --address=localhost:2828 /builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit-tests.ini --disable-e10s --gecko-log=- --symbols-path=/builds/worker/workspace/build/symbols --startup-timeout=300 --this-chunk 1 --total-chunks 10 -To execute a specific test only simply replace `unit-tests.ini` with its name. +To execute a specific test only simply replace `unit-tests.ini` +with its name.