It was observed that FHR was sending invalid JSON payloads to the
server. Specifically, JSON payloads contained invalid Unicode strings.
Investigation revealed that the culprint was CommonUtils.convertString()
silently swallowing high bytes. When the Bagheera client went to gzip
the JSON payload, the input buffer into gzip was missing high bytes.
This patch changes the bagheera client to UTF-8 encode strings before
gzip, thus ensuring all data is preserved. A corresponding change was
also added to the mock bagheera server implementation.
Alternatively, we could have changed CommonUtils.convertString() to
be high byte aware. However, many consumers rely on this function.
This patch is written with the intent of being uplifted and the change
performed is targeted at the specific problem.
Tests for Unicode preserving behavior have been added to both the
generic Bagheera client and to FHR. The latter test is arguably
not necessary, but peace of mind is a good thing, especially with
FHR.
See also bug 915850.
--HG--
extra : rebase_source : 4efddea7767c2e5f8cf19df247c3aba07c40eec6
extra : amend_source : ae3b6d89efa54fc9ed1794404476622946ad4b22
The data reporting notification was over-complicated. It wasn't
displayed for +24hr after first run and it had a weird, non-required
policy around what constituted acceptance of the policy.
The notification is now shown shortly after first startup.
The logic around "notification accepted" has been greatly simplified by
rolling it into "notification shown." Where we once were checking
whether the notification has been "accepted," we now check whether it
has been displayed. The overly complicated logic around the implicit
acceptance of the policy has also been removed.
The end result is the code for managing the state of the notification is
greatly simplified.
If recording FHR data during uploading raised an exception, it could
potentially abort the upload. This would appear to Mozilla as clients
that suddenly stopped using Firefox.
This patch adds explicit exception trapping around event record to
ensure this doesn't happen.
--HG--
extra : rebase_source : 7cd207b08a4f62be55093c71cb56e28832fd39d8
extra : amend_source : 9144ecea16a013370fffa5c2e833af4ec528ef5b
The data reporting notification was over-complicated. It wasn't
displayed for +24hr after first run and it had a weird, non-required
policy around what constituted acceptance of the policy.
The notification is now shown shortly after first startup.
The logic around "notification accepted" has been greatly simplified by
rolling it into "notification shown." Where we once were checking
whether the notification has been "accepted," we now check whether it
has been displayed. The overly complicated logic around the implicit
acceptance of the policy has also been removed.
The end result is the code for managing the state of the notification is
greatly simplified.
--HG--
extra : rebase_source : 808efdf1edd103552f6aa10b5c4309b64e514773
extra : amend_source : e4252e6a850a348d1b5aca733121dd07cbc6a70a
extra : histedit_source : 10ec20a07677674a8c9a705a3ffb4dc46a22b890%2Ca9442934d5964f16e9ad1101b786b4d094ac228d
The upcoming build system patches don't support hypthens in path names.
Changing this for that reason is kind of silly, but it's the easiest
way. Besides, nothing else uses hyphens in directory names.
--HG--
extra : rebase_source : 42dda2b1f16a3c0bfe17397a70092362e400530f
Up to this point, Firefox Health Report has generated and submitted a
random UUID with each upload. Generated UUIDs were stored on the client.
During upload, the client asked the server to delete all old UUIDs.
Well-behaving clients thus left at most one record/ID on the server.
Unfortunately, clients in the wild have not been behaving properly. We
are seeing multiple documents on the server that appear to come from the
same client. Clients are uploading new records but failing to delete the
old ones. These old, undeleted "orphan" records are severely impacting
the ability to derive useful knowledge from FHR data because it is
difficult, resource intensive, and error prone to filter the records on
the server. This is undermining the ability for FHR data to be put to
good use.
This patch introduces a persistent client identifier. When the client is
initialized, it generates a random UUID. That UUID is persisted to the
profile and sent as part of every upload.
For privacy reasons, if a client opts out of data submission, the client
ID will be reset as soon as all remote data has been deleted.
We still issue and send upload IDs. They exist mostly for forensics
purposes so we may log client behavior and more accurately determine
what exactly misbehaving, orphan-producing clients are doing.
It is worth noting that this persistent client identifier will not solve
all problems of branching and orphaned records. For example, profile
copying will result in multiple clients sharing a client identifier. A
"client ID version" field has been added to facilitate an upgrade path
towards client IDs with different generation semantics.
--HG--
extra : rebase_source : b761daab39fb07b6ab8883819d68bf53462314a0
Previously, it was technically possible for the FHR client to have
multiple simultaneous uploads. While this should never occur in
well-behaving systems, server logs have indicated that this behavior
might be occurring.
This patch adds a lock around uploading to ensure only 1 upload
operation may be active at a given time.
To measure the impact of this change in the wild, we added a counter
that increments whenever a held lock is encountered.
--HG--
extra : rebase_source : f0bb5065a3618cd335b6b6f31e4e68850f31f151
