gecko-dev/services
Kit Cambridge 064ed56076 Bug 1368209 - Refactor Engine::_processIncoming into three stages. r=eoger,tcsc
* In the first stage, we fetch changed records, newest first, up to the
  download limit. We keep track of the oldest record modified time we
  see.
* Once we've fetched all records, we reconcile, noting records that
  fail to decrypt or reconcile for the next sync. We then ask the store
  to apply all remaining records. Previously, `applyIncomingBatchSize`
  specified how many records to apply at a time. I removed this because
  it added an extra layer of indirection that's no longer necessary,
  now that download batching buffers all records in memory, and all
  stores are async.
* In the second stage, we fetch IDs for all remaining records changed
  between the last sync and the oldest modified time we saw in the
  first stage. We *don't* set the download limit here, to ensure we
  add *all* changed records to our backlog, and we use the `"oldest"`
  sort order instead of `"index"`.
* In the third stage, we backfill as before. We don't want large deltas
  to delay other engines from syncing, so we still only take IDs up to
  the download limit from the backlog, and include failed IDs from the
  previous sync. On subsequent syncs, we'll keep fetching from the
  backlog until it's empty.

Other changes to note in this patch:

* `Collection::_rebuildURL` now allows callers to specify both `older`
  and `newer`. According to :rfkelly, this is explicitly and
  intentionally supported.
* Tests that exercise `applyIncomingBatchSize` are gone, since that's
  no longer a thing.
* The test server now shuffles records if the sort order is
  unspecified.

MozReview-Commit-ID: 4EXvNOa8mIo

--HG--
extra : rebase_source : f382f0a883c5aa1f6a4466fefe22ad1a88ab6d20
2017-11-01 11:09:57 -07:00
..
blocklists Bug 1345556 - adjust BUG_COMPONENT for services/blocklist/* files. r=leplatrem 2017-03-10 12:26:58 -05:00
common Bug 1375223 - Remove Async.querySpinningly. r=kitcambridge 2017-10-31 18:50:31 -04:00
crypto Bug 1408777 - Automatically fix instances of missing semicolons in the tree. r=Standard8 2017-10-15 20:50:39 +01:00
fxaccounts Bug 1411368 - Automatically fix no-multi-spaces issues raised when using ESLint 4. r=mossop 2017-10-26 11:47:01 +01:00
sync Bug 1368209 - Refactor Engine::_processIncoming into three stages. r=eoger,tcsc 2017-11-01 11:09:57 -07:00
.eslintrc.js Bug 1375212 - Wrap thrown strings in Error objects r=eoger,markh 2017-07-22 18:55:43 -07:00
moz.build Bug 1361171 - Remove CloudSync. r=markh 2017-05-01 14:43:00 -07:00