Bug 1277810 - Part 1 - Restructure Telemetry client documentation. r=dexter, f=chutten

--HG--
rename : toolkit/components/telemetry/docs/scalars.rst => toolkit/components/telemetry/docs/collection/scalars.rst
rename : toolkit/components/telemetry/docs/crashes.rst => toolkit/components/telemetry/docs/concepts/crashes.rst
rename : toolkit/components/telemetry/docs/pings.rst => toolkit/components/telemetry/docs/concepts/pings.rst
rename : toolkit/components/telemetry/docs/common-ping.rst => toolkit/components/telemetry/docs/data/common-ping.rst
rename : toolkit/components/telemetry/docs/core-ping.rst => toolkit/components/telemetry/docs/data/core-ping.rst
rename : toolkit/components/telemetry/docs/crash-ping.rst => toolkit/components/telemetry/docs/data/crash-ping.rst
rename : toolkit/components/telemetry/docs/deletion-ping.rst => toolkit/components/telemetry/docs/data/deletion-ping.rst
rename : toolkit/components/telemetry/docs/environment.rst => toolkit/components/telemetry/docs/data/environment.rst
rename : toolkit/components/telemetry/docs/heartbeat-ping.rst => toolkit/components/telemetry/docs/data/heartbeat-ping.rst
rename : toolkit/components/telemetry/docs/main-ping.rst => toolkit/components/telemetry/docs/data/main-ping.rst
rename : toolkit/components/telemetry/docs/sync-ping.rst => toolkit/components/telemetry/docs/data/sync-ping.rst
rename : toolkit/components/telemetry/docs/uitour-ping.rst => toolkit/components/telemetry/docs/data/uitour-ping.rst
rename : toolkit/components/telemetry/docs/preferences.rst => toolkit/components/telemetry/docs/internals/preferences.rst

browser/docs/UITelemetry.rst

@@ -1,3 +1,5 @@
+.. _uitelemetry:
+
 =======================
 UITelemetry data format
 =======================

toolkit/components/telemetry/docs/collection/index.rst

@@ -0,0 +1,10 @@
+===============
+Data collection
+===============
+
+.. toctree::
+   :maxdepth: 2
+   :titlesonly:
+   :glob:
+
+   *

toolkit/components/telemetry/docs/collection/scalars.rst

@@ -6,7 +6,7 @@ Historically we started to overload our histogram mechanism to also collect scal
 such as flag values, counts, labels and others.
 The scalar measurement types are the suggested way to collect that kind of scalar data.
 We currently only support recording of scalars from the parent process.
-The serialized scalar data is submitted with the :doc:`main pings <main-ping>`.
+The serialized scalar data is submitted with the :doc:`main pings <../data/main-ping>`.
 
 The API
 =======

toolkit/components/telemetry/docs/concepts/crashes.rst

@@ -7,8 +7,8 @@ There are many different kinds of crashes for Firefox, there is not a single sys
 Main process crashes
 ====================
 
-If the Firefox main process dies, that should be recorded as an aborted session. We would submit a :doc:`main ping <main-ping>` with the reason ``aborted-session``.
+If the Firefox main process dies, that should be recorded as an aborted session. We would submit a :doc:`main ping <../data/main-ping>` with the reason ``aborted-session``.
-If we have a crash dump for that crash, we should also submit a :doc:`crash ping <crash-ping>`.
+If we have a crash dump for that crash, we should also submit a :doc:`crash ping <../data/crash-ping>`.
 
 The ``aborted-session`` information is first written to disk 60 seconds after startup, any earlier crashes will not trigger an ``aborted-session`` ping.
 Also, the ``aborted-session`` is updated at least every 5 minutes, so it may lag behind the last session state.

toolkit/components/telemetry/docs/concepts/index.rst

@@ -0,0 +1,11 @@
+========
+Concepts
+========
+
+.. toctree::
+   :maxdepth: 2
+   :titlesonly:
+   :glob:
+
+   pings
+   crashes

toolkit/components/telemetry/docs/concepts/pings.rst

@@ -8,8 +8,8 @@ A *Telemetry ping* is the data that we send to Mozillas Telemetry servers.
 
 That data is stored as a JSON object client-side and contains common information to all pings and a payload specific to a certain *ping types*.
 
-The top-level structure is defined by the :doc:`common-ping` format.
+The top-level structure is defined by the :doc:`../data/common-ping` format.
-It contains some basic information shared between different ping types, the :doc:`environment` data (optional) and the data specific to the *ping type*, the *payload*.
+It contains some basic information shared between different ping types, the :doc:`../data/environment` data (optional) and the data specific to the *ping type*, the *payload*.
 
 Submission
 ==========
@@ -21,7 +21,7 @@ If a ping fails to successfully submit to the server immediately (e.g. because
 of missing internet connection), Telemetry will store it on disk and retry to
 send it until the maximum ping age is exceeded (14 days).
 
-*Note:* the :doc:`main pings <main-ping>` are kept locally even after successful submission to enable the HealthReport and SelfSupport features. They will be deleted after their retention period of 180 days.
+*Note:* the :doc:`main pings <../data/main-ping>` are kept locally even after successful submission to enable the HealthReport and SelfSupport features. They will be deleted after their retention period of 180 days.
 
 Sending of pending pings starts as soon as the delayed startup is finished. They are sent in batches, newest-first, with up
 to 10 persisted pings per batch plus all unpersisted pings.
@@ -42,15 +42,21 @@ The telemetry server team is working towards `the common services status codes <
 Ping types
 ==========
 
-* :doc:`main <main-ping>` - contains the information collected by Telemetry (Histograms, hang stacks, ...)
+We send Telemetry with different ping types. The :doc:`main <../data/main-ping>` ping is the ping that contains the bulk of the Telemetry measurements for Firefox. For more specific use-cases, we also send other custom ping types.
-* :doc:`saved-session <main-ping>` - has the same format as a main ping, but it contains the *"classic"* Telemetry payload with measurements covering the whole browser session. This is only a separate type to make storage of saved-session easier server-side. This is temporary and will be removed soon.
+
-* :doc:`crash <crash-ping>` - a ping that is captured and sent after Firefox crashes.
+Examples are:
-* :doc:`uitour-ping` - a ping submitted via the UITour API
+
+* :doc:`main <../data/main-ping>` - contains the information collected by Telemetry (Histograms, hang stacks, ...)
+* :doc:`saved-session <../data/main-ping>` - has the same format as a main ping, but it contains the *"classic"* Telemetry payload with measurements covering the whole browser session. This is only a separate type to make storage of saved-session easier server-side. This is temporary and will be removed soon.
+* :doc:`crash <../data/crash-ping>` - a ping that is captured and sent after Firefox crashes.
 * ``activation`` - *planned* - sent right after installation or profile creation
 * ``upgrade`` - *planned* - sent right after an upgrade
-* :doc:`heartbeat-ping` - contains information on Heartbeat surveys
+* :doc:`deletion <../data/deletion-ping>` - sent when FHR upload is disabled, requesting deletion of the data associated with this user
-* :doc:`deletion <deletion-ping>` - sent when FHR upload is disabled, requesting deletion of the data associated with this user
+* :doc:`uitour <../data/uitour-ping>` - a ping submitted via the UITour API
-* :doc:`sync <sync-ping>` - sent after a sync is completed or fails, contains information on sync errors and performance.
+* :doc:`heartbeat <../data/heartbeat-ping>` - contains information on Heartbeat surveys
+* :doc:`sync <../data/sync-ping>` - sent after a sync is completed or fails, contains information on sync errors and performance.
+
+Pings sent from code that ships with Firefox are listed in the :doc:`data documentation <../data/index>`.
 
 Archiving
 =========

toolkit/components/telemetry/docs/data/common-ping.rst

@@ -7,7 +7,7 @@ It contains basic information shared between different ping types, which enables
 
 It also contains optional further information:
 
-* the :doc:`environment data <environment>`, which contains important info to correlate the measurements against
+* the :doc:`environment data <../data/environment>`, which contains important info to correlate the measurements against
 * the ``clientId``, a UUID identifying a profile and allowing user-oriented correlation of data
 
 *Note:* Both are not submitted with all ping types due to privacy concerns. This and the data it that can be correlated against is inspected under the `data collection policy <https://wiki.mozilla.org/Firefox/Data_Collection>`_.

toolkit/components/telemetry/docs/data/core-ping.rst

@@ -68,7 +68,7 @@ vs. "Galaxy Note") than we are for shortened manufacturer names so we provide
 more characters for the model than the manufacturer.
 
 distributionId
-~~~~~~
+~~~~~~~~~~~~~~
 The ``distributionId`` contains the distribution ID as specified by
 preferences.json for a given distribution. More information on distributions
 can be found `here <https://wiki.mozilla.org/Mobile/Distribution_Files>`_.

toolkit/components/telemetry/docs/data/environment.rst

@@ -279,11 +279,11 @@ The object contains:
   engine is currently set as the default.
 
 - a ``loadPath`` property: an anonymized path of the engine xml file, e.g.
- jar:[app]/omni.ja!browser/engine.xml
+  jar:[app]/omni.ja!browser/engine.xml
   (where 'browser' is the name of the chrome package, not a folder)
- [profile]/searchplugins/engine.xml
+  [profile]/searchplugins/engine.xml
- [distribution]/searchplugins/common/engine.xml
+  [distribution]/searchplugins/common/engine.xml
- [other]/engine.xml
+  [other]/engine.xml
 
 - an ``origin`` property: the value will be ``default`` for engines that are built-in or from distribution partners, ``verified`` for user-installed engines with valid verification hashes, ``unverified`` for non-default engines without verification hash, and ``invalid`` for engines with broken verification hashes.
 

toolkit/components/telemetry/docs/data/index.rst

@@ -0,0 +1,17 @@
+==================
+Data documentation
+==================
+
+.. toctree::
+   :maxdepth: 2
+   :titlesonly:
+   :glob:
+
+   common-ping
+   environment
+   main-ping
+   deletion-ping
+   crash-ping
+   *-ping
+
+The `mozilla-pipeline-schemas repository <https://github.com/mozilla-services/mozilla-pipeline-schemas/>`_ contains schemas for some of the pings.

toolkit/components/telemetry/docs/data/main-ping.rst

@@ -2,6 +2,9 @@
 "main" ping
 ===========
 
+.. toctree::
+   :maxdepth: 2
+
 This is the "main" Telemetry ping type, whose payload contains most of the measurements that are used to track the performance and health of Firefox in the wild.
 It includes the histograms and other performance and diagnostic data.
 
@@ -101,7 +104,7 @@ Structure::
 
 scalars
 ~~~~~~~
-This section contains the :doc:`scalars` that are valid for the current platform. Scalars are not created nor submitted if no data was added to them, and are only reported with subsession pings. Scalar data is only currently reported for the main process. Their type and format is described by the ``Scalars.yaml`` file. Its most recent version is available `here <https://dxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/Scalars.yaml>`_. The ``info.revision`` field indicates the revision of the file that describes the reported scalars.
+This section contains the :doc:`../collection/scalars` that are valid for the current platform. Scalars are not created nor submitted if no data was added to them, and are only reported with subsession pings. Scalar data is only currently reported for the main process. Their type and format is described by the ``Scalars.yaml`` file. Its most recent version is available `here <https://dxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/Scalars.yaml>`_. The ``info.revision`` field indicates the revision of the file that describes the reported scalars.
 
 childPayloads
 -------------
@@ -129,7 +132,7 @@ Only available in the extended set of measures, it contains a set of counters re
 
 UITelemetry
 ~~~~~~~~~~~
-Only available in the extended set of measures. See the documentation for :doc:`/browser/docs/UITelemetry <UITelemetry>`.
+Only available in the extended set of measures. For more see :ref:`uitelemetry`.
 
 startupInterrupted
 ~~~~~~~~~~~~~~~~~~

toolkit/components/telemetry/docs/fhr/architecture.rst

@@ -6,7 +6,7 @@ Architecture
 
 ``healthreporter.jsm`` contains the main interface for FHR, the
 ``HealthReporter`` type. An instance of this is created by the
-:ref:`data_reporting_service`.
+``data_reporting_service`.
 
 ``providers.jsm`` contains numerous ``Metrics.Provider`` and
 ``Metrics.Measurement`` used for collecting application metrics. If you

toolkit/components/telemetry/docs/fhr/index.rst

@@ -1,5 +1,3 @@
-.. _healthreport:
-
 ================================
 Firefox Health Report (Obsolete)
 ================================
@@ -11,7 +9,7 @@ Firefox Health Report is a background service that collects application
 metrics and periodically submits them to a central server. The core
 parts of the service are implemented in this directory. However, the
 actual XPCOM service is implemented in the
-:ref:`data_reporting_service`.
+``data_reporting_service`.
 
 The core types can actually be instantiated multiple times and used to
 power multiple data submission services within a single Gecko

toolkit/components/telemetry/docs/index.rst

@@ -1,3 +1,5 @@
+.. _telemetry:
+
 =========
 Telemetry
 =========
@@ -6,25 +8,18 @@ Telemetry is a feature that allows data collection. This is being used to collec
 
 Client-side, this consists of:
 
-* data collection in `Histograms <https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Adding_a_new_Telemetry_probe>`_, :doc:`scalars` and other data structures
+* data collection in `Histograms <https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Adding_a_new_Telemetry_probe>`_, :doc:`collection/scalars` and other data structures
-* assembling :doc:`pings` with the general information and the data payload
+* assembling :doc:`concepts/pings` with the general information and the data payload
 * sending them to the server and local ping retention
 
 *Note:* the `data collection policy <https://wiki.mozilla.org/Firefox/Data_Collection>`_ documents the process and requirements that are applied here.
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 5
+   :titlesonly:
 
-   pings
+   concepts/index
-   common-ping
+   collection/index
-   environment
+   data/index
-   scalars
+   internals/index
-   main-ping
+   fhr/index
-   core-ping
-   deletion-ping
-   crash-ping
-   uitour-ping
-   heartbeat-ping
-   sync-ping
-   preferences
-   crashes

toolkit/components/telemetry/docs/internals/index.rst

@@ -0,0 +1,9 @@
+=========
+Internals
+=========
+
+.. toctree::
+   :maxdepth: 2
+   :titlesonly:
+
+   preferences

toolkit/components/telemetry/moz.build

@@ -101,7 +101,6 @@ LOCAL_INCLUDES += [
 ]
 
 SPHINX_TREES['telemetry'] = 'docs'
-SPHINX_TREES['healthreport'] = 'docs/fhr'
 
 if CONFIG['GNU_CXX']:
     CXXFLAGS += ['-Wno-error=shadow']