Commit Graph

4825 Commits

Author SHA1 Message Date
Thom Chiovoloni
edeaf1bc14 Bug 1415012 - Don't trigger a resync if the previous sync wasn't completely successful r=kitcambridge
MozReview-Commit-ID: DcNsFUxXR1O

--HG--
extra : rebase_source : 2632aa8d6db8e9e101e949c237b559015444f808
2017-11-06 20:56:57 -05:00
Sebastian Hengst
af670f4147 merge mozilla-central to mozilla-inbound. r=merge a=merge 2017-11-07 02:47:30 +02:00
Manish Kumar
53383c5b3f Bug 1408108 - Rename "Sync ID" to "record ID" in the bookmarks engine and PlacesSyncUtils. r=kitcambridge
--HG--
extra : rebase_source : da3acfc4f7393a615c44f238c624bb826ff5b766
2017-11-06 12:17:07 -08:00
Thom Chiovoloni
935734388d Bug 1412351 - Make sync behave better during shutdown r=kitcambridge
MozReview-Commit-ID: 4zWNph8Zdld

--HG--
extra : rebase_source : 12e9f82222a4bcfe2161ad02669619b57f0442fe
2017-10-27 13:37:49 -04:00
Sebastian Hengst
d61954557d merge mozilla-inbound to mozilla-central. r=merge a=merge
MozReview-Commit-ID: Ai6Y5GGfkfT
2017-11-04 10:58:24 +01:00
Nicholas Nethercote
b9c6a1d730 Bug 1413413 (part 2) - Remove support for extensions having their own prefs file. r=glandium,kmag.
It's no longer needed, now that legacy extensions aren't supported.

Pieces removed include the following.

- The "load-extension-default" observer notification.

- The code for reading defaults/preferences/*.js from extensions.

- The unit test for this stuff.

- A crash reporter annotation relating to very long prefs set by add-ons.

- All references to "ExtPrefDL".

MozReview-Commit-ID: KMBoYn3uZ3x

--HG--
extra : rebase_source : 4dc8ffd425c6cdf06806409090c4f9d04a64930b
2017-10-31 16:34:57 +11:00
Luciano I
e4007fdbe7 Bug 1375223 - Remove Async.querySpinningly. r=kitcambridge
MozReview-Commit-ID: bMo1jyIY5g

--HG--
extra : rebase_source : ac69fde2cb8216300bdb9e7d19528c15cdceb7c8
2017-10-31 18:50:31 -04:00
Kit Cambridge
604673b3d3 Bug 1366067 - Use JSONFile.jsm to persist the backlog and previously failed IDs. r=tcsc
MozReview-Commit-ID: DAwMrtWI8eh

--HG--
extra : rebase_source : f6713745b09de2e6b0993539cce9ed4e8226dbbf
2017-11-01 11:10:47 -07:00
Thom Chiovoloni
4a77d5a93f Bug 1388224 - Remove SyncStorageRequest HTTP wrapper class, which is no longer used r=kitcambridge
MozReview-Commit-ID: EgExfizNll5

--HG--
extra : rebase_source : 85a6098a5b985990bd3a8483936bf6cf768f1f0b
2017-11-01 17:17:46 -04:00
Kit Cambridge
064ed56076 Bug 1368209 - Refactor Engine::_processIncoming into three stages. r=eoger,tcsc
* In the first stage, we fetch changed records, newest first, up to the
  download limit. We keep track of the oldest record modified time we
  see.
* Once we've fetched all records, we reconcile, noting records that
  fail to decrypt or reconcile for the next sync. We then ask the store
  to apply all remaining records. Previously, `applyIncomingBatchSize`
  specified how many records to apply at a time. I removed this because
  it added an extra layer of indirection that's no longer necessary,
  now that download batching buffers all records in memory, and all
  stores are async.
* In the second stage, we fetch IDs for all remaining records changed
  between the last sync and the oldest modified time we saw in the
  first stage. We *don't* set the download limit here, to ensure we
  add *all* changed records to our backlog, and we use the `"oldest"`
  sort order instead of `"index"`.
* In the third stage, we backfill as before. We don't want large deltas
  to delay other engines from syncing, so we still only take IDs up to
  the download limit from the backlog, and include failed IDs from the
  previous sync. On subsequent syncs, we'll keep fetching from the
  backlog until it's empty.

Other changes to note in this patch:

* `Collection::_rebuildURL` now allows callers to specify both `older`
  and `newer`. According to :rfkelly, this is explicitly and
  intentionally supported.
* Tests that exercise `applyIncomingBatchSize` are gone, since that's
  no longer a thing.
* The test server now shuffles records if the sort order is
  unspecified.

MozReview-Commit-ID: 4EXvNOa8mIo

--HG--
extra : rebase_source : f382f0a883c5aa1f6a4466fefe22ad1a88ab6d20
2017-11-01 11:09:57 -07:00
Kit Cambridge
96593594d3 Bug 1368209 - Add a test for fetching backlogged history records. r=tcsc
The test captures the existing logic in `_processIncoming`, even though
it's not quite correct:

* First, we fetch all records changed since the last sync, up to the
  download limit, and without an explicit sort order. This happens to
  work correctly now because the Python server uses "newest" by
  default, but can change in the future.
* If we reached the download limit fetching records, we request
  IDs for all records changed since the last sync, also up to the
  download limit, and sorted by index. This is likely to return IDs
  for records we've already seen, since the index is based on the
  frecency. It's also likely to miss IDs for other changed records,
  because the number of changed records might be higher than the
  download limit.
* Since we then fast-forward the last sync time, we'll never download
  any remaining changed records that we didn't add to our backlog.
* Finally, we backfill previously failed and backlogged records.

MozReview-Commit-ID: 7uQLXMseMIU

--HG--
extra : rebase_source : 719ee2d9e46102195251b410f093da3247095c22
2017-10-27 17:54:48 -07:00
Kit Cambridge
432850c0aa Bug 1368209 - Add support for sortindex and older to the mock Sync server. r=tcsc
MozReview-Commit-ID: 6YI1OEE8x7R

--HG--
extra : rebase_source : 2f0da0cd2a1554685fff5af3de100e4b01564447
2017-10-27 17:55:30 -07:00
Edouard Oger
3e7662202d Bug 1413685 - Unify AsyncResource and Resource. r=tcsc
MozReview-Commit-ID: 1yEeAwpbGxJ

--HG--
extra : rebase_source : 8c87b0d16138acf2b55be70767fe0547f4ac1d42
2017-11-02 14:30:59 -04:00
NARCIS BELEUZU
1b5408583f Backed out 3 changesets (bug 1368209)xpcshell failures services/sync/tests/unit/test_syncengine_sync.js on CLOSED TREE
Backed out changeset 1b868efa368f (bug 1368209)
Backed out changeset b88c681ccdc1 (bug 1368209)
Backed out changeset 4b8e56844ae9 (bug 1368209)
2017-11-03 11:40:06 +02:00
Kit Cambridge
4105f1cb3a Bug 1368209 - Refactor Engine::_processIncoming into three stages. r=eoger,tcsc
* In the first stage, we fetch changed records, newest first, up to the
  download limit. We keep track of the oldest record modified time we
  see.
* Once we've fetched all records, we reconcile, noting records that
  fail to decrypt or reconcile for the next sync. We then ask the store
  to apply all remaining records. Previously, `applyIncomingBatchSize`
  specified how many records to apply at a time. I removed this because
  it added an extra layer of indirection that's no longer necessary,
  now that download batching buffers all records in memory, and all
  stores are async.
* In the second stage, we fetch IDs for all remaining records changed
  between the last sync and the oldest modified time we saw in the
  first stage. We *don't* set the download limit here, to ensure we
  add *all* changed records to our backlog, and we use the `"oldest"`
  sort order instead of `"index"`.
* In the third stage, we backfill as before. We don't want large deltas
  to delay other engines from syncing, so we still only take IDs up to
  the download limit from the backlog, and include failed IDs from the
  previous sync. On subsequent syncs, we'll keep fetching from the
  backlog until it's empty.

Other changes to note in this patch:

* `Collection::_rebuildURL` now allows callers to specify both `older`
  and `newer`. According to :rfkelly, this is explicitly and
  intentionally supported.
* Tests that exercise `applyIncomingBatchSize` are gone, since that's
  no longer a thing.
* The test server now shuffles records if the sort order is
  unspecified.

MozReview-Commit-ID: 4EXvNOa8mIo

--HG--
extra : rebase_source : 13605dd3a43569a6d83dc2eb15a578a7bbd5c1ca
2017-11-01 11:09:57 -07:00
Kit Cambridge
481bdb6dd6 Bug 1368209 - Add a test for fetching backlogged history records. r=tcsc
The test captures the existing logic in `_processIncoming`, even though
it's not quite correct:

* First, we fetch all records changed since the last sync, up to the
  download limit, and without an explicit sort order. This happens to
  work correctly now because the Python server uses "newest" by
  default, but can change in the future.
* If we reached the download limit fetching records, we request
  IDs for all records changed since the last sync, also up to the
  download limit, and sorted by index. This is likely to return IDs
  for records we've already seen, since the index is based on the
  frecency. It's also likely to miss IDs for other changed records,
  because the number of changed records might be higher than the
  download limit.
* Since we then fast-forward the last sync time, we'll never download
  any remaining changed records that we didn't add to our backlog.
* Finally, we backfill previously failed and backlogged records.

MozReview-Commit-ID: 7uQLXMseMIU

--HG--
extra : rebase_source : 5742474889845b934c3d2e8b479d26d719cd03c0
2017-10-27 17:54:48 -07:00
Kit Cambridge
8f60eeed17 Bug 1368209 - Add support for sortindex and older to the mock Sync server. r=tcsc
MozReview-Commit-ID: 6YI1OEE8x7R

--HG--
extra : rebase_source : 2f0da0cd2a1554685fff5af3de100e4b01564447
2017-10-27 17:55:30 -07:00
Thom Chiovoloni
4ee948935f Bug 1408176 - Use trace logs when logging PII in sync passwords engine r=kitcambridge
MozReview-Commit-ID: KueHgdTH02Z

--HG--
extra : rebase_source : a8f5814c5d9baf8e6c370c65da8472f9c7c6213e
2017-10-27 11:32:22 -04:00
Mark Banner
4de6bf22b1 Bug 1411368 - Automatically fix no-multi-spaces issues raised when using ESLint 4. r=mossop
MozReview-Commit-ID: H5YVp3rnzGo

--HG--
extra : rebase_source : 5b45b6c0df834131812d094e975047eaad374e06
2017-10-26 11:47:01 +01:00
Kit Cambridge
3b78d2f449 Bug 1412139 - Prepare existing bookmark engine tests for new buffered engine. r=kitcambridge
MozReview-Commit-ID: 2HIkdPOuWOy

--HG--
extra : rebase_source : 6c8108f707f59e8d795dfd489453c614f00fd096
2017-09-11 14:17:24 -07:00
Edouard Oger
76547f53cc Bug 1395460 p1 - Remove usages of about:accounts. r=tcsc
MozReview-Commit-ID: APdGc9avJDw

--HG--
extra : rebase_source : aaa788be5251607ebedcb5b690da0e2a1a64e2cb
2017-10-24 14:05:52 -04:00
Sebastian Hengst
93903f0306 Backed out 2 changesets (bug 1395460) for eslint failure at browser/components/uitour/UITour.jsm:556 (no return value from arrow function). r=backout on a CLOSED TREE
Backed out changeset 0bc6d186d609 (bug 1395460)
Backed out changeset 6c028123001d (bug 1395460)
2017-10-26 23:33:04 +02:00
Edouard Oger
58876ecbe9 Bug 1395460 p1 - Remove usages of about:accounts. r=tcsc
MozReview-Commit-ID: APdGc9avJDw

--HG--
extra : rebase_source : 9da1cea9fb6494c057bc6587a7bcbdb4ac127448
2017-10-24 14:05:52 -04:00
Kit Cambridge
1225809c1d Bug 1411690 - Floor the repair advance and most recent repair timestamps. r=tcsc
The timestamps are automatically truncated when they're stored in
prefs, which is fine because we don't need millisecond precision.
However, the truncation raises a warning, so we need to explicitly
floor the values.

MozReview-Commit-ID: BRflL6s0b1

--HG--
extra : rebase_source : 6724a1ad05cb8aee4ab8c666545784960c23a0f3
2017-10-25 13:54:44 -07:00
Edouard Oger
4a0a0ce970 Bug 1411714 - Handle action=email in about:accounts. r=tcsc
MozReview-Commit-ID: 5U6Qy78S1Nl

--HG--
extra : rebase_source : 69368eca5243d418c9705959be81f724351291c6
2017-10-25 16:41:37 -04:00
Sebastian Hengst
1ead0a96a9 Backed out changeset dab1c5badddb (bug 1407679) for failing server startup in mochitests on Android. r=backout on a CLOSED TREE
--HG--
extra : amend_source : 3dc4cb2bbaf55f3f6329687ce64d4e862fc3800b
2017-10-24 21:18:26 +02:00
Valentin Gosu
9dd47ebf50 Bug 1407679 - Merge nsIIOService and nsIIOService2 r=nwgh
Removes deprecated nsIIOService2.newChannelFromURIWithProxyFlags

MozReview-Commit-ID: Gr3nA25kcre

--HG--
extra : rebase_source : 3407b5ed0f8f7067523abee28c137db3cac86b14
2017-10-20 16:41:07 +02:00
Kit Cambridge
c31dc99e96 Bug 890217 - Add a test for syncing bookmarks with IMAP URLs. r=tcsc
MozReview-Commit-ID: KBVInzYk1Rr

--HG--
extra : rebase_source : 9ba1b55c154a9a19f24dcf8b859189123abcad1c
2017-10-20 08:55:52 -07:00
Tom Prince
0a687f249e Bug 1404577: Actually disable loading blocklist dump in test_cert_blocklist.js. r=leplatrem
The dump file isn't in the expected location in firefox tests, but is in
thunderbird tests, so the preference to disable loading wasn't originally
implemented.

MozReview-Commit-ID: HvFqfC69yMQ

--HG--
extra : rebase_source : 1d358292f0ab94299e444f4d3e3454a2259d1a64
2017-10-19 21:26:20 -06:00
Dan Banner
7caa92d5d8 Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8
MozReview-Commit-ID: Jm8BRgt6mIv
2017-10-15 20:50:39 +01:00
Thom Chiovoloni
2c51427c6d Bug 1408180 - Ensure LoginRec.toString doesn't contain the password. r=kitcambridge
MozReview-Commit-ID: 5mV0g9LH4vE

--HG--
extra : rebase_source : 33a9c246053448450e6107c3b244d0bf86f7decb
2017-10-12 18:15:48 -04:00
Sebastian Hengst
87cd48824a merge mozilla-central to autoland. r=merge a=merge on a CLOSED TREE 2017-10-13 00:53:56 +02:00
Sebastian Hengst
f7efb5fc2c Merge mozilla-central to mozilla-inbound. r=merge a=merge on a CLOSED TREE 2017-10-12 12:03:15 +02:00
Edouard Oger
d8c3cc3c0b Bug 1401700 - Prevent incoming tabs from opening multiple times if client sync fails. r=tcsc
MozReview-Commit-ID: DhrZ1oFY2WG

--HG--
extra : rebase_source : d8d35bdb08edbad9412d6953a671c3e38e75711c
2017-10-03 14:45:11 -04:00
Kevin Pellet (Ilphrin)
87b8d9ecb9 Bug 1406488 - Use a set instead of array to store current visits in _recordToPlaceInfo r=kitcambridge
MozReview-Commit-ID: LW6Gg7i5lun
2017-10-11 19:06:10 -04:00
Sebastian Hengst
30ccd917b0 Backed out changeset 8d5cc47b248f (bug 1401700) for failing xpcshell services/sync/tests/unit/test_bookmark_repair.js on OS X 10.10 opt. r=backout 2017-10-11 23:44:26 +02:00
Edouard Oger
51d18ff941 Bug 1401700 - Prevent incoming tabs from opening multiple times if client sync fails. r=tcsc
MozReview-Commit-ID: DhrZ1oFY2WG

--HG--
extra : rebase_source : e98e4ee0a3e376840ce89f0908d7210b167154fa
2017-10-03 14:45:11 -04:00
Edouard Oger
4397201c90 Bug 1407726 - Add reason field to push messages sent with /notify. r=tcsc
MozReview-Commit-ID: 3sRpheND9Wa

--HG--
extra : rebase_source : 23f7dffb920f9f9f14ccdd16f171c489f36745cc
2017-10-11 14:58:21 -04:00
Thom Chiovoloni
7555b605f1 Bug 1407082 - Handle more kinds of errors in SyncTelemetry.transformError. r=kitcambridge
MozReview-Commit-ID: CjwY7w7vUqs

--HG--
extra : rebase_source : 4030ab88c3eccd755272499ff0527e9e316998b1
2017-10-10 12:31:42 -04:00
Thom Chiovoloni
da5797466a Bug 1403276 - Dedupe sync devices with the same fxaDeviceId by picking the one with the newer last modified date r=eoger
MozReview-Commit-ID: 3Lq7vuPpF6

--HG--
extra : rebase_source : f16c5fe0d29551ab829ba8f0c8c92e78109ace07
2017-10-10 13:41:32 -04:00
Thom Chiovoloni
83f6879b5d Bug 1052247 - Enforce that OAuth is done over HTTPS in FxAccountsOAuthClient. r=markh
It now follows the setting of the identity.fxaccounts.allowHttp preference.

MozReview-Commit-ID: 9646Xi48QMP

--HG--
extra : rebase_source : 630e65bebc00e755ca3be1d159e08fec738d590f
2017-10-04 16:22:45 -04:00
Thom Chiovoloni
0c48a189bc Bug 1405833 - Ensure SyncEngine uses CommonUtils.namedTimer properly. r=kitcambridge
MozReview-Commit-ID: 6YnhcSjKW9U

--HG--
extra : rebase_source : 3e0e520516f06efa50aa8e3fccb77214daca419a
2017-10-04 17:26:20 -04:00
Kit Cambridge
e9d579fe2a Bug 1405566 - Clamp future and far past synced history visit dates. r=markh
MozReview-Commit-ID: Gs0DpTTu1Ab

--HG--
extra : rebase_source : b836f4ae366eaf127935dee593dc51d2222e3aca
2017-10-04 14:16:29 -07:00
Robert Helmer
c42f57923a Bug 1348981 - only load system add-ons from a built-in list r=kmag
MozReview-Commit-ID: A6c5kaLmNPP

--HG--
extra : rebase_source : e20ec54eabb5ecb2cc0b85efb687dbed06f1d16a
2017-10-02 18:41:20 -07:00
Thom Chiovoloni
e6050e53aa Bug 1401990 - Use max_request_bytes instead of max_post_bytes if sync server provides both r=markh
MozReview-Commit-ID: GJadUIuSGKt

--HG--
extra : rebase_source : be8da0b14c0e1522bc1dc025871fc34830d9ae58
2017-10-02 16:42:22 -04:00
Sebastian Hengst
bd61777c6b Backed out changeset fe9b8761d51f (bug 1348981) 2017-10-04 15:43:46 +02:00
Robert Helmer
8d488926ef Bug 1348981 - only load system add-ons from a built-in list r=kmag
MozReview-Commit-ID: A6c5kaLmNPP

--HG--
extra : rebase_source : 9ad775a652affc0d8d6c7ce4153dddc8a13c4ca4
2017-10-02 18:41:20 -07:00
Sebastian Hengst
d8e232c88a Backed out changeset 8dda8bbcf385 (bug 1348981) 2017-10-04 01:00:09 +02:00
Robert Helmer
c8714b1c7d Bug 1348981 - only load system add-ons from a built-in list r=kmag
MozReview-Commit-ID: A6c5kaLmNPP

--HG--
extra : rebase_source : 058e6011c8ce953059e674eea032ba79c19f8eb8
2017-10-02 18:41:20 -07:00
Thom Chiovoloni
d39e820403 Bug 1403052 - Limit tab sync max_record_payload_size to 512k. r=markh
Also fixes an issue where we wouldn't encode to utf8 when comparing the actual
size to the limit after the first time.

MozReview-Commit-ID: Cf3byjI1FTZ

--HG--
extra : rebase_source : 272ec3b3ad85f8b44c4d69950be83419054abdab
2017-10-02 19:27:54 -04:00