Read-only Git mirror of the Mercurial gecko repositories at https://hg.mozilla.org. How to contribute: https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html
Go to file
Grisha Kruglov 283d1a3450 Bug 1291821 - Allow BatchingDownloader to resume downloads using offset r=rnewman
BatchingDownloader uses provided RepositoryStateProvider instance in order to track
offset and high water mark as it performs batching.

The state holder objects are initialized by individual ServerSyncStages, and prefixes are used to ensure keys
won't clash.

Two RepositoryStateProvider implementations are used: persistent and non-persistent. Non-persistent
state provider does not allow for resuming after a sync restart, while persistent one does.

Persistent state provider is used by the history stage. It is fetched oldest-first, and records are applied
to live storage as they're downloaded. These conditions let use resume downloads. It's also possible to
resume downloads for stages which use a persistent buffer, but currently we do not have any.

Offset value and its context is reset if we hit a 412 error; it is maintained if we hit a sync deadline, allowing us to
minimize number of records we'll redownload. BatchingDownloaderController owns resuming and context checking logic.

High water mark (h.w.m.) is maintained across syncs and used instead of stage's "last-synced" timestamp if said stage is
set to fetch oldest-first and explicitely allows use of a h.w.m. Server15RepositorySession provides correct timestamp
to RecordsChannel, decoupling BatchingDownloader from this logic.

MozReview-Commit-ID: IH28YrDU4vW

--HG--
extra : rebase_source : 63bd7daaa1fd2a63e10289d6d4cd198aaf81498b
2017-01-19 13:11:18 -08:00
.cargo Bug 1277338 - Part 1: Turn js/src into the mozjs-sys crate; r=sfink 2016-10-13 16:12:42 -07:00
accessible Backed out changeset c1544abd34c2 (bug 1340903) for frequent failure of docshell/test/unit/test_setUsePrivateBrowsing.js on Linux opt and pgo. r=backout 2017-02-22 09:59:29 +01:00
addon-sdk Bug 1339483: Fix tab frame lookup performance issues. r=mossop 2017-02-14 13:24:07 -08:00
b2g Bug 1338311 - Update builders to rust 1.15.1. r=mshal 2017-02-09 14:43:26 -08:00
browser Merge m-c to autoland a=merge 2017-02-24 16:59:05 -08:00
build Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
caps Backed out changeset a6267555a244 (bug 1341218) for various test failures, e.g. xpcshell test dom/push/test/xpcshell/test_service_parent.js. r=backout 2017-02-22 11:21:37 +01:00
chrome merge mozilla-inbound to mozilla-central a=merge 2017-02-03 13:25:24 +01:00
config Backed out 5 changesets (bug 1318370) for mozilla-central Windows static-analysis bustage 2017-02-21 21:22:01 -08:00
db/sqlite3 Bug 1340028 - Remove the CLANG_VERSION macro from the in-tree SQLite. r=mak 2017-02-16 09:46:41 -05:00
devtools Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
docshell Merge inbound to m-c a=merge 2017-02-17 13:38:51 -08:00
dom Merge m-c to autoland a=merge 2017-02-24 16:59:05 -08:00
editor merge mozilla-inbound to mozilla-central a=merge 2017-02-23 14:59:37 +01:00
embedding Bug 1332392 - embedding/* should have BUG_COMPONENTS specified in moz.build files. r=ted 2017-02-06 09:45:58 -05:00
extensions Merge m-c to inbound, a=merge 2017-02-17 13:51:01 -08:00
gfx Merge m-c to autoland a=merge 2017-02-24 16:59:05 -08:00
gradle/wrapper Bug 1300094 - Update Gradle to 2.14.1 and Gradle Plugin to 2.1.3 r=sebastian 2016-09-22 14:31:01 -07:00
hal Bug 1060419 - make log_print use Printf.h, r=froydnj 2016-12-15 20:16:31 -07:00
image Reenable several crashtests that no longer crash or leak or time out with stylo. No bug, r=bzbarsky 2017-02-24 15:22:37 -05:00
intl Bug 1340924 - Remove Iterator consumer from intl/. r=Pike 2017-02-22 16:11:34 +09:00
ipc merge mozilla-inbound to mozilla-central a=merge 2017-02-23 14:59:37 +01:00
js Merge m-c to autoland a=merge 2017-02-24 16:59:05 -08:00
layout Merge m-c to autoland a=merge 2017-02-24 16:59:05 -08:00
media Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
memory Bug 1341621 - Avoid deadlocks in DMD when forking. r=njn 2017-02-23 17:47:02 -08:00
mfbt Bug 1338574 - Part 1: Add MOZ_CRASH_UNSAFE_OOL and MOZ_CRASH_UNSAFE_PRINTF to crash with a runtime generated explanation string. r=froydnj 2017-02-21 18:01:41 +01:00
mobile Bug 1291821 - Allow BatchingDownloader to resume downloads using offset r=rnewman 2017-01-19 13:11:18 -08:00
modules merge mozilla-inbound to mozilla-central a=merge 2017-02-17 13:24:55 +01:00
mozglue Merge inbound to m-c a=merge 2017-02-17 13:38:51 -08:00
netwerk Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
nsprpub Bug 1331810: enable sending of 0 size UDP packets in NSPR. r=ted 2017-01-18 11:50:12 -08:00
other-licenses Bug 1324617 - Allow any of multiple installations to be set as the Windows default browser; r=jimm,rstrong 2017-01-09 08:14:00 -08:00
parser Bug 1334290 - Remove useless fields from nsScanner. r=mrbkap. 2017-02-17 08:44:49 +02:00
probes Bug 1286877 - do not set c-basic-offset for python-mode; r=gps 2016-07-14 10:16:42 -06:00
python merge mozilla-inbound to mozilla-central a=merge 2017-02-24 11:46:38 +01:00
rdf Bug 1060419 - make log_print use Printf.h, r=froydnj 2016-12-15 20:16:31 -07:00
release/docker bug 1314574 replace sys.path with site.addsitedir so that .pth files are processed. r=rail a=release DONTBUILD 2016-11-02 10:43:57 +00:00
security No bug, Automated HPKP preload list update from host bld-linux64-spot-025 - a=hpkp-update 2017-02-24 08:12:13 -08:00
services merge mozilla-inbound to mozilla-central a=merge 2017-02-20 14:05:25 +01:00
servo servo: Merge #14740 - Add support for keyword values for min-width and max-width (from glasserc:extremum-length); r=Manishearth 2017-02-24 14:08:58 -08:00
startupcache Bug 1341794 - annotate startupcache/* moz.build file with BUG_COMPONENT. r=froydnj 2017-02-23 08:01:45 -05:00
storage Merge inbound to m-c a=merge 2017-02-17 13:38:51 -08:00
taskcluster Merge m-c to autoland a=merge 2017-02-24 16:59:05 -08:00
testing Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
third_party/rust Merge m-c to graphics 2017-02-23 09:51:44 -05:00
toolkit merge mozilla-inbound to mozilla-central a=merge 2017-02-24 11:46:38 +01:00
tools Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
uriloader Bug 1322576 - [4.0] Fix some coding style issues. r=smaug 2017-02-22 14:57:41 +01:00
view Bug 1275347 - Move the display list profiler marker to the right place. r=mattwoodrow 2017-02-16 19:18:37 -05:00
widget Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
xpcom merge mozilla-inbound to mozilla-central a=merge 2017-02-24 11:46:38 +01:00
xpfe Bug 1060419 - remove unneeded includes of prprf.h, r=froydnj 2016-12-09 10:00:01 -10:00
.clang-format Bug 1322321 - Update the clang format file to match more our coding style r=ehsan 2016-12-06 17:43:17 -10:00
.clang-format-ignore Bug 1332797 - Move mfbt/double-conversion source files into a new source/ subdirectory, to segregate upstream files from update.sh and our local patches. r=froydnj 2017-01-23 15:11:50 -08:00
.clang-tidy Bug 1319396 - Update the list of checkers in the .clang-tidy file r=Ehsan 2016-11-22 12:30:19 +01:00
.cron.yml Bug 1338385 - Add cron job for nightly coverage tests run; r=jmaher 2017-02-08 11:16:25 -08:00
.eslintignore Merge inbound to central, a=merge 2017-02-24 16:46:12 -08:00
.eslintrc.js Bug 1338585 - Add an eslint rule to require using .ownerGlobal instead of .ownerDocument.defaultView, r=jaws. 2017-02-13 13:01:32 +01:00
.flake8 Bug 1321956 - Disable flake8 check for 'visually indented line with same indent as next logical line', r=ahal 2016-12-08 09:53:21 -10:00
.gdbinit Bug 1317802 - don't stop for SIGSYS in .gdbinit; r=jld 2016-11-16 01:13:22 -05:00
.gdbinit_python
.gitignore Bug 1301790 - Bundle codemirror into a single script;r=gl 2016-09-11 09:38:51 -07:00
.hgignore Bug 1332691: Add script to regenerate tests, update the WPT manifest and expectations; r=luke 2017-02-17 20:13:54 +01:00
.hgtags No bug - Tagging mozilla-central f80dc9fc34680105b714a49b4704bb843f5f7004 with FIREFOX_AURORA_53_BASE a=release DONTBUILD CLOSED TREE 2017-01-23 15:31:28 +00:00
.lldbinit
.taskcluster.yml Bug 1334810: temporarily revert {{owner}}; r=garndt 2017-02-07 16:18:19 +00:00
.ycm_extra_conf.py Bug 1293534: Fix .ycm_extra_conf after bug 1195748. r=gps 2016-08-09 10:30:58 -07:00
aclocal.m4 Bug 1262155 - Use moz.build instead of libffi's build system. r=ted,glandium 2016-08-08 16:58:54 -07:00
Android.mk
AUTHORS Bug 1302763 - Move docker images out of testing/docker into taskcluster/docker; r=dustin r=CuriousLearner 2016-10-20 18:25:34 +05:30
build.gradle Bug 1318823 - Use com.getkeepsafe.dexcount in Gradle builds. r=sebastian 2016-11-18 18:28:58 -08:00
client.mk Bug 1317778 - Emit a depfile with python configure dependencies so Make will know when to re-run configure. r=glandium 2016-12-02 10:05:57 -08:00
client.py
CLOBBER Backed out 5 changesets (bug 1318370) for mozilla-central Windows static-analysis bustage 2017-02-21 21:22:01 -08:00
configure.in Bug 1295751 - Make it possible to copy configure.in to configure. r=gps 2016-08-16 19:35:00 -07:00
configure.py Bug 1316140 - Allow use of multiprocessing from config.status on windows. r=mshal 2016-12-21 16:28:28 -08:00
GNUmakefile
gradle.properties
gradlew
LEGAL
LICENSE
mach Bug 1316140 - Allow use of multiprocessing from config.status on windows. r=mshal 2016-12-21 16:28:28 -08:00
Makefile.in Bug 1339182 - Remove OSX universal support in the build system; r=glandium 2017-02-10 16:52:17 -05:00
moz.build Bug 1003417 - Include testing/mozbase/moz.build from root moz.build file, r=ted 2017-02-01 09:56:33 -05:00
moz.configure Bug 1197325 - Remove now unused genisoimage. r=ted 2017-02-02 10:10:27 -05:00
mozilla-config.h.in
old-configure.in Backed out 5 changesets (bug 1318370) for mozilla-central Windows static-analysis bustage 2017-02-21 21:22:01 -08:00
README.txt
settings.gradle Bug 1291363 - Add geckoview and geckoview_example Gradle projects. r=jchen,sebastian 2016-10-05 20:23:38 -07:00
test.mozbuild

An explanation of the Mozilla Source Code Directory Structure and links to
project pages with documentation can be found at:

    https://developer.mozilla.org/en/Mozilla_Source_Code_Directory_Structure

For information on how to build Mozilla from the source code, see:

    http://developer.mozilla.org/en/docs/Build_Documentation

To have your bug fix / feature added to Mozilla, you should create a patch and
submit it to Bugzilla (https://bugzilla.mozilla.org). Instructions are at:

    http://developer.mozilla.org/en/docs/Creating_a_patch
    http://developer.mozilla.org/en/docs/Getting_your_patch_in_the_tree

If you have a question about developing Mozilla, and can't find the solution
on http://developer.mozilla.org, you can try asking your question in a
mozilla.* Usenet group, or on IRC at irc.mozilla.org. [The Mozilla news groups
are accessible on Google Groups, or news.mozilla.org with a NNTP reader.]

You can download nightly development builds from the Mozilla FTP server.
Keep in mind that nightly builds, which are used by Mozilla developers for
testing, may be buggy. Firefox nightlies, for example, can be found at:

    https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/
            - or -
    http://nightly.mozilla.org/