00d0eac6d3
B2G updates can be of multiple types: - OTA, applied without rebooting the device, - FOTA with only Gecko/Gaia, - FOTA with whole system partition files, - FOTA dumping partitions images. Each type of updates has its advantages and drawbacks. There is an extensive documentation maintained on MDN about each and the options: https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Firefox_OS_update_packages All those updates are being packaged as a MAR file that gets injected into the classical Firefox update mechanism, submitted to Balrog and downloaded by the client. The content of the MAR will however depend on the type of update: an OTA update will packate a Gecko and Gaia set of files to update those parts; while any FOTA package is just an update.zip that will get applied in recovery mode on the device. So one fundamental difference is that OTA will not reboot your device (just Gecko) while FOTA requires a working recovery mode and will reboot your device. But OTA needs more system partition space to get applied, and it can only update files that are within the /system/b2g/ directory. FOTA on the other hand can update anything since the payload will contain an update script written in Edify (Android recovery update scripting language). For each device we might need to produce several types of updates that will be pushed to users depending on the context: for some users we want to push just a Gecko/Gaia update, for some we know that we need to update more content and thus we need to send some partitions. Previously, the b2g_build.py script would only allow one kind of update payload to be produced for each device available: we would need to have a device "flame-kk-ota" and "flame-kk-fota" just to produce the OTA and FOTA packages for the same device, thus resulting in a waste of computing power and storage. This commit introduces a new field "update_types" that can take an array of values: - ota, to produce an OTA package as before - fota, to produce a FOTA package with only Gecko/Gaia - fota:full, to produce a FOTA package of all files of the system partition - fota:fullimg, to produce a FOTA package dumping partitions The old "update_type" will be used in the absence of "update_types". And if none are present, we will keep defaulting to generating OTA as previously. |
||
|---|---|---|
| .. | ||
| scripts | ||
| taskcluster_graph | ||
| tasks | ||
| tests | ||
| design.md | ||
| mach_commands.py | ||
| README.md | ||
| requirements.txt | ||
| routes.json | ||
| setup.py | ||
Taskcluster + Gecko Integration
Directory structure:
-
tasks/ : All task definitions
-
tests/ : Tests for the mach target internals related to task graph generation
-
scripts : Various scripts used by taskcluster docker images and utilities these exist in tree primarily to avoid rebuilding docker images.
Task conventions
In order to properly enable task reuse there are a small number of conventions and parameters that are specialized for build tasks vs test tasks. The goal here should be to provide as much of the power as taskcluster but not at the cost of making it easy to support the current model of build/test.
All tasks are in the YAML format and are also processed via mustache to allow for greater customizations. All tasks have the following templates variables:
-
docker_image: Helper for always using the latest version of a docker image that exist in tree.{{#docker_image}}base{{/docker_image}}Will produce something like (see the docker folder):
quay.io/mozilla.com/base:0.11 -
from_now: Helper for crafting a JSON date in the future.{{#from_now}}1 year{{/from_now}}Will produce:
2014-10-19T22:45:45.655Z -
now: Current time as a json formatted date.
Build tasks
By convention build tasks are stored in tasks/builds/ the location of
each particular type of build is specified in job_flags.yml (and more
locations in the future), which is located in the appropriate subdirectory
of branches/.
Task format
To facilitate better reuse of tasks there are some expectations of the build tasks. These are required for the test tasks to interact with the builds correctly but may not effect the builds or indexing services.
# This is an example of just the special fields. Other fields that are
# required by taskcluster are omitted and documented on http://docs.taskcluster.net/
task:
payload:
# Builders usually create at least two important artifacts the build
# and the tests these can be anywhere in the task and also may have
# different path names to include things like arch and extension
artifacts:
# The build this can be anything as long as its referenced in
# locations.
'public/name_i_made_up.tar.gz': '/path/to/build'
'public/some_tests.zip': '/path/to/tests'
extra:
# Build tasks may put their artifacts anywhere but there are common
# resources that test tasks need to do their job correctly so we
# need to provide an easy way to lookup the correct aritfact path.
locations:
build: 'public/name_i_made_up.tar.gz'
tests: 'public/some_tests.zip' or test_packages: 'public/test_packages.json'
Templates properties
-
repository: Target HG repository (ex: https://hg.mozilla.org/mozilla-central)
-
revision: Target HG revision for gecko
-
owner: Email address of the committer
Test Tasks
By convention test tasks are stored in tasks/tests/ the location of
each particular type of build is specified in job_flags.yml (and more
locations in the future)
Template properties
-
repository: Target HG repository (ex: https://hg.mozilla.org/mozilla-central)
-
revision: Target HG revision for gecko
-
owner: Email address of the committer
-
build_url: Location of the build
-
tests_url: Location of the tests.zip package
-
chunk: Current chunk
-
total_chunks: Total number of chunks
Developing
Running commands via mach is the best way to invoke commands testing works a little differently (I have not figured out how to invoke python-test without running install steps first)
mach python-test tests/
Examples:
Requires taskcluster-cli.
mach taskcluster-trygraph --message 'try: -b do -p all' \
--head-rev=33c0181c4a25 \
--head-repository=http://hg.mozilla.org/mozilla-central \
--owner=jlal@mozilla.com | taskcluster run-graph
Creating only a build task and submitting to taskcluster:
mach taskcluster-build \
--head-revision=33c0181c4a25 \
--head-repository=http://hg.mozilla.org/mozilla-central \
--owner=user@domain.com tasks/builds/b2g_desktop.yml | taskcluster run-task --verbose
mach taskcluster-tests --task-id=Mcnvz7wUR_SEMhmWb7cGdQ \
--owner=user@domain.com tasks/tests/b2g_mochitest.yml | taskcluster run-task --verbose