Commit Graph

5650 Commits

Author SHA1 Message Date
Ciure Andrei
9a123fb597 Backed out changeset 81156ff3f4fa (bug 1571656) for causing browser_PageActions.js and test_insertTree_fixupOrSkipInvalidEntries.js to perma fail CLOSED TREE 2019-08-19 13:00:44 +03:00
Moritz Birghan
240771fa29 Bug 1571656 - Use Assert.jsm numeric comparison functions in tests r=mixedpuppy,MattN
Differential Revision: https://phabricator.services.mozilla.com/D40614

--HG--
extra : moz-landing-system : lando
2019-08-19 07:17:12 +00:00
Mark Banner
b1970e6a2f Bug 1571466 - Cleanup unnecessary ESLint global definitions. r=mossop
These are raised as redeclares or unused variables by ESLint 6.

Differential Revision: https://phabricator.services.mozilla.com/D37268

--HG--
extra : moz-landing-system : lando
2019-08-19 07:11:56 +00:00
Lina Cambridge
a17323e9d8 Bug 1573709 - Return an aborted watchdog for testing interrupted bookmark merges. a=testonly
Differential Revision: https://phabricator.services.mozilla.com/D41876

--HG--
extra : moz-landing-system : lando
2019-08-14 00:28:52 +00:00
Lina Cambridge
65dfa84ccd Bug 1573305 - Remove the check for unmerged synced bookmark changes. r=markh
When a local or remote item changed, we'd potentially scan three tables
(with an expensive `LEFT JOIN`!) to check if anything changed...then
scan the same tables again to build the local and remote trees. This
check was originally meant to avoid unnecessary merges. However, the
bottleneck isn't merging now; it's reading from the database.

Since the merger has been rewritten in Rust, is synchronous, doesn't
keep a transaction open for the entire merge (see the
`total_sync_changes` check), and only emits ops for items that actually
changed, it's more efficient to build and merge optimistically, and
bail before applying if nothing changed.

This commit also moves `validateLocalRoots` into Rust.

Differential Revision: https://phabricator.services.mozilla.com/D41690

--HG--
extra : moz-landing-system : lando
2019-08-13 22:58:16 +00:00
Lina Cambridge
19557885f6 Bug 1572615 - Abort bookmark merges that take longer than 5 minutes. r=markh
This commit introduces a new `Watchdog` class that signals an abort,
either after a delay or at shutdown, and wires up the buffered engine
to use it.

Differential Revision: https://phabricator.services.mozilla.com/D41311

--HG--
extra : moz-landing-system : lando
2019-08-13 18:48:25 +00:00
ffxbld
21d02cb6fe No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings tld-suffixes - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D41541

--HG--
extra : moz-landing-system : lando
2019-08-12 13:08:23 +00:00
Nicholas Nethercote
41a012cce7 Bug 1572621 - Merge services-sync.js into all.js. r=markh
Differential Revision: https://phabricator.services.mozilla.com/D41323

--HG--
extra : moz-landing-system : lando
2019-08-09 09:18:41 +00:00
Nicholas Nethercote
73d5a12400 Bug 1572621 - Merge services-common.js into all.js. r=markh
Differential Revision: https://phabricator.services.mozilla.com/D41322

--HG--
extra : moz-landing-system : lando
2019-08-09 09:18:27 +00:00
Lina Cambridge
0d63b1e98f Bug 1498073 - Use centisecond precision for modified times in Sync tests. a=testonly
Differential Revision: https://phabricator.services.mozilla.com/D41266

--HG--
extra : moz-landing-system : lando
2019-08-08 19:19:47 +00:00
Jared Wein
22ae3511f6 Bug 1571567 - Fix no-fallthrough errors in /services. r=markh
Differential Revision: https://phabricator.services.mozilla.com/D40878

--HG--
extra : moz-landing-system : lando
2019-08-08 15:18:36 +00:00
Bogdan Tara
844afcfb06 Backed out 10 changesets (bug 1571567) complementary backout after es lint failure on the patch CLOSED TREE
Backed out changeset ce83fa75ae32 (bug 1571567)
Backed out changeset 7aa97ba7cce9 (bug 1571567)
Backed out changeset 777d79076e99 (bug 1571567)
Backed out changeset fbdf6b75a484 (bug 1571567)
Backed out changeset e2ed4620f232 (bug 1571567)
Backed out changeset 2c67015f12c6 (bug 1571567)
Backed out changeset 7ec086bb5bd5 (bug 1571567)
Backed out changeset 42df735c8556 (bug 1571567)
Backed out changeset 7d5fc57b2809 (bug 1571567)
Backed out changeset 606bafb8211c (bug 1571567)
2019-08-08 18:13:54 +03:00
Jared Wein
9cf685253f Bug 1571567 - Fix no-fallthrough errors in /services. r=markh
Differential Revision: https://phabricator.services.mozilla.com/D40878

--HG--
extra : moz-landing-system : lando
2019-08-08 01:38:51 +00:00
Mark Hammond
05df0254b8 Bug 1568353 - set the default rest.js timeout to be 5 minutes. r=rfkelly
Differential Revision: https://phabricator.services.mozilla.com/D40927

--HG--
extra : moz-landing-system : lando
2019-08-07 05:25:49 +00:00
Mark Hammond
40d28c338b Bug 1561762 - better handle concurrent requests for FxA oauth tokens. r=eoger,rfkelly
Differential Revision: https://phabricator.services.mozilla.com/D38036

--HG--
extra : moz-landing-system : lando
2019-08-06 15:25:05 +00:00
Lina Cambridge
9320172d8f Bug 1567238 - Refactor the bookmarks mirror merge triggers to do less work. r=tcsc,markh
This commit reduces the number of database writes and table scans
needed to merge synced bookmarks.

* Remove `fetchNew{Local, Remote}Contents`. Fetching the tree already
  scans the table, so we can piggyback on it to fetch content info for
  deduping.
* Store completion ops in temp tables to only update changed parts of
  the local tree, and remove the `mergeStates` table and views.
* Replace the `itemsToMerge` view with an indexed `itemsToApply` temp
  table.
* Replace the `updateGuidsAndSyncFlags` trigger with a `changeGuidOps`
  table and a `changeGuids` trigger.
* Replace the `updateLocalItems` trigger with an `apply_remote_items`
  function to bulk upsert new and updated items.
* Replace the `structureToMerge` view with an
  `applyNewLocalStructureOps` table that holds parents and positions
  for moved items, and an `applyNewLocalStructure` trigger to update
  them.
* Remove tombstones for revived items, update change counters, and flag
  mirror items as merged directly in `update_local_items_in_places`,
  instead of indirecting through temp tables.
* Don't mark items flagged for reupload as merged, since we'll write
  them back to the mirror after upload.
* Use a scalar subquery instead of a join in the `localTags` view to
  look up the tags root ID.
* Replace `relatedIdsToReupload` with a `Store::prepare` method that
  flags all bookmarks with keyword-URL mismatches for reupload.
* Trigger frecency updates for origins once, not for every item.
* Remove two extra scans on `itemsAdded` and `itemsChanged` when
  recording observer notifications for changed keywords.
* Notify all `bookmark-added` listeners in a single batch.

This also fixes some edge cases:

* Update root positions correctly after deleting a non-syncable root
  or item.
* Keyword-URL mismatches may reupload more items than before, but now
  ensure that all bookmarks with the same URL have the same keyword.
* Only set items with deduped GUIDs to `SYNC_STATUS_NORMAL` after
  merging.
* Bump the last modified time for modified items only.

Differential Revision: https://phabricator.services.mozilla.com/D39889

--HG--
extra : moz-landing-system : lando
2019-08-06 23:48:03 +00:00
ffxbld
6dfb6ea6a1 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D40198

--HG--
extra : moz-landing-system : lando
2019-08-01 13:24:01 +00:00
Ehsan Akhgari
f172f95ac0 Bug 1568341 - Part 4: Pre-package the anti-tracking-url-decoration.json remote settings bucket for new profiles or fresh upgrades; r=leplatrem
Differential Revision: https://phabricator.services.mozilla.com/D39922

--HG--
extra : moz-landing-system : lando
2019-07-31 19:32:07 +00:00
Luke Crouch
940333a24b Bug 1566591: catch and log RemoteSettings errors r=jaws
Differential Revision: https://phabricator.services.mozilla.com/D39824

--HG--
extra : moz-landing-system : lando
2019-07-30 19:39:03 +00:00
Mathieu Leplatre
68a314f460 Bug 1569894 - Upgrade kinto-http-client.js to 4.7.3 r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D39852

--HG--
extra : moz-landing-system : lando
2019-07-30 13:32:45 +00:00
ffxbld
9ee911f801 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D39676

--HG--
extra : moz-landing-system : lando
2019-07-29 13:29:43 +00:00
ffxbld
b7828720b0 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings tld-suffixes - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D39331

--HG--
extra : moz-landing-system : lando
2019-07-25 13:20:25 +00:00
Edouard Oger
684e965f89 Bug 1556276 p2 - Build metrics-flow and legal FxA URLs. r=markh
Differential Revision: https://phabricator.services.mozilla.com/D39033

--HG--
extra : moz-landing-system : lando
2019-07-25 01:12:31 +00:00
Edouard Oger
1c060fbfab Bug 1556276 p1 - Allow extraParams to be passed when constructing FxA URLs. r=markh
Differential Revision: https://phabricator.services.mozilla.com/D39032

--HG--
extra : moz-landing-system : lando
2019-07-24 18:34:06 +00:00
Mathieu Leplatre
5cb0d4bf75 Bug 1563180 - Prevent synchronization to be ran in parallel multiple times r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D36921

--HG--
extra : moz-landing-system : lando
2019-07-22 13:50:04 +00:00
Mark Hammond
fb1ef8ec44 Bug 1567376 - remove support for the old custom FxA server prefs. r=eoger
Differential Revision: https://phabricator.services.mozilla.com/D38616

--HG--
extra : moz-landing-system : lando
2019-07-19 17:31:21 +00:00
ffxbld
28af937c05 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings tld-suffixes - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D38478

--HG--
extra : moz-landing-system : lando
2019-07-18 12:58:04 +00:00
Mark Hammond
60f4926fea Bug 1564131 - re-enable syncing of builtin themes. r=rpl
Differential Revision: https://phabricator.services.mozilla.com/D37837

--HG--
extra : moz-landing-system : lando
2019-07-18 01:25:50 +00:00
ffxbld
4ede446d10 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D38063

--HG--
extra : moz-landing-system : lando
2019-07-15 17:06:26 +00:00
Luca Greco
55b47893cf Bug 1541496 - Ensure storage.sync does sync the data for previously synced extensions without an active extension context. r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D37413

--HG--
extra : moz-landing-system : lando
2019-07-11 14:30:20 +00:00
ffxbld
7c0acde6a7 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings tld-suffixes - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D37716

--HG--
extra : moz-landing-system : lando
2019-07-11 12:55:28 +00:00
Narcis Beleuzu
8478c8b076 Merge autoland to mozilla-central. a=merge 2019-07-09 06:43:17 +03:00
ffxbld
6ca9b306bd Update configs. IGNORE BROKEN CHANGESETS CLOSED TREE NO BUG a=release ba=release 2019-07-08 17:27:36 +00:00
Coroiu Cristina
6e3840ddec Backed out changeset 7aa8180f36eb (bug 1563180) for ESlint failure 2019-07-06 19:43:30 +03:00
Mathieu Leplatre
4039e268d2 Bug 1563180 - Prevent synchronization to be ran in parallel multiple times r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D36921

--HG--
extra : moz-landing-system : lando
2019-07-06 16:05:12 +00:00
Mathieu Leplatre
326c302870 Bug 1563213 - Upgrade kinto-http-client.js to 4.7.2 r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D36752

--HG--
extra : moz-landing-system : lando
2019-07-05 13:45:56 +00:00
Jonathan Kingston
31441f82ea Bug 1560455 - rename CodebasePrincipal to ContentPrincipal. r=ckerschb
Differential Revision: https://phabricator.services.mozilla.com/D35504

--HG--
extra : moz-landing-system : lando
2019-07-08 16:37:45 +00:00
ffxbld
18132fb6de No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D37242

--HG--
extra : moz-landing-system : lando
2019-07-08 12:47:22 +00:00
Victor Porof
34c33b48c4 Bug 1561435 - Fix linting errors for services/, r=standard8
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D35931

--HG--
extra : source : f441e31c307bc815c7f05e99c0cd8ee77344dc80
extra : intermediate-source : 7e366ab79479e3a9bbdf9e61f04cfa7b6c271990
2019-06-21 13:38:54 -07:00
Victor Porof
9d673f1f0f Bug 1561435 - Format services/, a=automatic-formatting
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D35930

--HG--
extra : source : 073567e8ded785e46382ba7e402d8da939a69d7b
2019-07-05 10:58:22 +02:00
Victor Porof
b5a4cb3848 Bug 1558517 - Pre 3.0: Remove conflicting eslint rules, and turn on "curly: all" everywhere, r=standard8
Differential Revision: https://phabricator.services.mozilla.com/D34535

--HG--
extra : source : 74ed7ee773393d305c4e948a57a1b1e32b1f12e8
extra : intermediate-source : 403d0757d61683e0a85d0bb07768eb39fbd0af72
2019-06-28 17:14:01 +02:00
Mathieu Leplatre
56afa0ab7c Bug 1559514 - Emit sync event when up-to-date if JSON dump is loaded r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D35240

--HG--
extra : moz-landing-system : lando
2019-07-04 15:15:48 +00:00
ffxbld
c3dff17808 No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings tld-suffixes - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D36908

--HG--
extra : moz-landing-system : lando
2019-07-04 13:06:12 +00:00
Mathieu Leplatre
e661f9a03d Bug 1558710 - Make sure clients never try to push local changes r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D36718

--HG--
extra : moz-landing-system : lando
2019-07-03 13:41:21 +00:00
ffxbld
bba7a6fe2b No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D36484

--HG--
extra : moz-landing-system : lando
2019-07-01 13:15:39 +00:00
Mathieu Leplatre
3e347218c9 Bug 1560451 - Fix duplicates in list of created on sync event r=glasserc
Depends on D36171

Differential Revision: https://phabricator.services.mozilla.com/D36182

--HG--
extra : moz-landing-system : lando
2019-06-27 13:40:24 +00:00
Mathieu Leplatre
0cf601cfff Bug 1559158 - Improve performance of sync for large collections r=glasserc
Differential Revision: https://phabricator.services.mozilla.com/D36171

--HG--
extra : moz-landing-system : lando
2019-06-27 13:25:57 +00:00
ffxbld
ab68b05adf No Bug, mozilla-central repo-update HSTS HPKP blocklist remote-settings tld-suffixes - a=repo-update r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D36187

--HG--
extra : moz-landing-system : lando
2019-06-27 12:57:42 +00:00
Marco Bonardo
eb9461aa93 Bug 1508973 - Remove getAnnotationsForItem and unused bookmark annotations APIs. r=Standard8,lina
Differential Revision: https://phabricator.services.mozilla.com/D35009

--HG--
extra : moz-landing-system : lando
2019-06-25 15:21:53 +00:00
Mark Hammond
adf91feaf8 Bug 1538015 (part 2) - restrict when incoming synced prefs are applied. r=tcsc,pauljt,Gijs
Specifically, a "control pref" for a pref must already exist locally, or
a new preference, `services.sync.prefs.dangerously_allow_arbitrary` must
be set to true.

This also removes a few preferences from the set we sync by default based
due to potential harm which can be caused syncing inappropriate values.

Differential Revision: https://phabricator.services.mozilla.com/D29775

--HG--
extra : moz-landing-system : lando
2019-06-25 04:40:58 +00:00