This test uses the Kinto client to access the RemoteSettings database.
However, due to the database version bump from D72416, the kinto client
is no longer compatible with the RemoteSettings database.
This will be fixed in bug 1634203.
Differential Revision: https://phabricator.services.mozilla.com/D73160
The MLBF (addons-mlbf.bin) itself is 64 KB.
Together with the metadata, this is 65 KB.
In contrast, the current JSON-based dump (addons.json) is 913 KB.
Differential Revision: https://phabricator.services.mozilla.com/D73159
With this piece, it is now possible for RemoteSettings clients to always
have a valid attachment.
This adds the client APIs that could support bug 1542177.
Differential Revision: https://phabricator.services.mozilla.com/D72417
The current RemoteSettings API has two methods for downloading:
- `download(record)` to download an attachment to disk, with a filename
given by the record. The file is read and returned before downloading.
A new download attempt overwrites the file, with no recovery mechanism
if the download fails.
- `downloadAsBytes(record)` to download an attachment in memory.
The `download` method does have a cache, but it is only useful for
reducing bandwidth usage, not for availability of data. Moreover, if its
associated record is removed from the collection, then callers do not
have a way to delete the file since its identifier (filename) originates
from the record.
The `downloadAsBytes` method does not have this file tracking issue
since it does not persist the data, but it forces callers to implement
their own attachment storing mechanism.
This commit adds the `useCache` option to the `download()` method to
enable callers to use an IndexedDB-based cache instead of the
filesystem. The following options unlock significant features:
- `fallbackToCache` - If the requested attachment is not available, the
last known attachment is returned.
- `fallbackToDump` - If the requested attachment is not available, nor
cached, then the attachment (dump) that is packaged with the client is
returned instead. This is implemented in the next commit (D72417).
The original record is cached along with the attachment, to allow
callers to use the file (identified by the given `attachmentId`) and
its metadata, even when the original record has been removed from the
collection.
This is particularly useful for scenarios where one wants to keep a file
(and metadata) up to date via RemoteSettings, without having to develop
a separate storage and synchronization mechanism.
The deprecation of the old behavior will be handled in bug 1634127.
Differential Revision: https://phabricator.services.mozilla.com/D72416
This adds abort handlers to Database.jsm and RemoteSettingsWorker.js's
indexedDB transactions, so we handle transaction aborts appropriately
and do not leave consumers waiting forever.
It also adds explicit error passing to places where we continue operating on a
store via a live transaction after control flow passes back to executeIDB,
because we use the `onsuccess` handler of earlier IDBRequests to run more
requests in the transaction.
In this case, in theory exceptions get handled by IndexedDB by invoking the
abort handler on the transaction (which we do not have right now...), but as a
belt-and-braces approach, it's probably better to do this explicitly.
Differential Revision: https://phabricator.services.mozilla.com/D71326
These two tests use the Kinto client to access the RemoteSettings
database. However, due to the database version bump from D72416,
the kinto client is no longer compatible with the RemoteSettings
database.
This will be fixed in bug 1634203.
Differential Revision: https://phabricator.services.mozilla.com/D73160
The MLBF (addons-mlbf.bin) itself is 64 KB.
Together with the metadata, this is 65 KB.
In contrast, the current JSON-based dump (addons.json) is 913 KB.
Differential Revision: https://phabricator.services.mozilla.com/D73159
With this piece, it is now possible for RemoteSettings clients to always
have a valid attachment.
This adds the client APIs that could support bug 1542177.
Differential Revision: https://phabricator.services.mozilla.com/D72417
The current RemoteSettings API has two methods for downloading:
- `download(record)` to download an attachment to disk, with a filename
given by the record. The file is read and returned before downloading.
A new download attempt overwrites the file, with no recovery mechanism
if the download fails.
- `downloadAsBytes(record)` to download an attachment in memory.
The `download` method does have a cache, but it is only useful for
reducing bandwidth usage, not for availability of data. Moreover, if its
associated record is removed from the collection, then callers do not
have a way to delete the file since its identifier (filename) originates
from the record.
The `downloadAsBytes` method does not have this file tracking issue
since it does not persist the data, but it forces callers to implement
their own attachment storing mechanism.
This commit adds the `useCache` option to the `download()` method to
enable callers to use an IndexedDB-based cache instead of the
filesystem. The following options unlock significant features:
- `fallbackToCache` - If the requested attachment is not available, the
last known attachment is returned.
- `fallbackToDump` - If the requested attachment is not available, nor
cached, then the attachment (dump) that is packaged with the client is
returned instead. This is implemented in the next commit (D72417).
The original record is cached along with the attachment, to allow
callers to use the file (identified by the given `attachmentId`) and
its metadata, even when the original record has been removed from the
collection.
This is particularly useful for scenarios where one wants to keep a file
(and metadata) up to date via RemoteSettings, without having to develop
a separate storage and synchronization mechanism.
The deprecation of the old behavior will be handled in bug 1634127.
Differential Revision: https://phabricator.services.mozilla.com/D72416
This commit wires up `StorageSyncArea::teardown()` to call the new
`webext_storage::Store::close()` method.
It also changes `teardown` to drop the `LazyStore`, and thus close its
database connection, on the main thread if dispatching to the task
queue fails. Dispatch should only fail at shutdown, and putting the
owned reference back only adds indirection, since the `StorageSyncArea`
will still drop its `LazyStore` later in shutdown.
Finally, it includes an xpcshell test fix for
https://github.com/mozilla/application-services/pull/3050.
Differential Revision: https://phabricator.services.mozilla.com/D72992
Now that `BridgedEngine` has been moved to `sync15_traits`, we can
remove `golden_gate_traits` from the tree, and change Golden Gate to
depend on `sync15_traits` directly.
This commit also adds a Cargo feature, `services_sync`, which reflects
the `MOZ_SERVICES_SYNC` config option. In the future, we'll use this
feature to gate implementations of `mozIBridgedSyncEngine`.
Differential Revision: https://phabricator.services.mozilla.com/D72784
This commit adds a new crate for bridging Rust Sync engines to Desktop,
and a `mozIBridgedSyncEngine` for accessing the bridge via JS.
Naturally, the bridge is called Golden Gate. 😊 For more information
on how to use it, please see `golden_gate/src/lib.rs`.
Other changes include:
* Ensuring the test Sync server uses UTF-8 for requests and responses.
* Renaming `mozISyncedBookmarksMirrorLogger` to `mozIServicesLogger`,
and moving it into the shared Sync interfaces.
The `BridgedEngine` trait lives in its own crate, called
`golden_gate_traits`, to make it easier to eventually move into a-s.
`Interruptee` and `Interrupted` already exist in a-s, and are
duplicated in this crate for now.
Differential Revision: https://phabricator.services.mozilla.com/D65268
--HG--
extra : moz-landing-system : lando
This commit adds a new crate for bridging Rust Sync engines to Desktop,
and a `mozIBridgedSyncEngine` for accessing the bridge via JS.
Naturally, the bridge is called Golden Gate. 😊 For more information
on how to use it, please see `golden_gate/src/lib.rs`.
Other changes include:
* Ensuring the test Sync server uses UTF-8 for requests and responses.
* Renaming `mozISyncedBookmarksMirrorLogger` to `mozIServicesLogger`,
and moving it into the shared Sync interfaces.
The `BridgedEngine` trait lives in its own crate, called
`golden_gate_traits`, to make it easier to eventually move into a-s.
`Interruptee` and `Interrupted` already exist in a-s, and are
duplicated in this crate for now.
Differential Revision: https://phabricator.services.mozilla.com/D65268
--HG--
extra : moz-landing-system : lando
This commit adds a new crate for bridging Rust Sync engines to Desktop,
and a `mozIBridgedSyncEngine` for accessing the bridge via JS.
Naturally, the bridge is called Golden Gate. 😊 For more information
on how to use it, please see `golden_gate/src/lib.rs`.
Other changes include:
* Ensuring the test Sync server uses UTF-8 for requests and responses.
* Renaming `mozISyncedBookmarksMirrorLogger` to `mozIServicesLogger`,
and moving it into the shared Sync interfaces.
The `BridgedEngine` trait lives in its own crate, called
`golden_gate_traits`, to make it easier to eventually move into a-s.
`Interruptee` and `Interrupted` already exist in a-s, and are
duplicated in this crate for now.
Differential Revision: https://phabricator.services.mozilla.com/D65268
--HG--
extra : moz-landing-system : lando
By and large, this change accomplishes two things:
1. Run db.close() in finally clauses so that even if db access fails, we close
our connections. It also tries to avoid waiting on other, non-DB operations
before calling close, to avoid the DB connection needlessly hanging around.
2. Intercept all async database operations from the remote settings client to
kinto and ensuring they complete before the end of `profile-before-change`.
Any operations started after Services.startup.isShuttingDown (so after
quit/restart is initiated by the user) will throw. Operations started
beforehand are put in a set of operations, and remove themselves once
complete. We AsyncShutdown block on that set of operations completing.
Differential Revision: https://phabricator.services.mozilla.com/D66995
--HG--
extra : moz-landing-system : lando
By and large, this change accomplishes two things:
1. Run db.close() in finally clauses so that even if db access fails, we close
our connections. It also tries to avoid waiting on other, non-DB operations
before calling close, to avoid the DB connection needlessly hanging around.
2. Intercept all async database operations from the remote settings client to
kinto and ensuring they complete before the end of `profile-before-change`.
Any operations started after Services.startup.isShuttingDown (so after
quit/restart is initiated by the user) will throw. Operations started
beforehand are put in a set of operations, and remove themselves once
complete. We AsyncShutdown block on that set of operations completing.
Differential Revision: https://phabricator.services.mozilla.com/D66995
--HG--
extra : moz-landing-system : lando
When the signature verification fails after retry we don't want to apply the changes, and if possible we want to leave the local data as it was before sync.
Differential Revision: https://phabricator.services.mozilla.com/D67139
--HG--
extra : moz-landing-system : lando
A significant amount of clients report OFFLINE_ERROR during shutdown.
Instead of adding a new uptake status to be reported when detecting offline during shutdown (eg. INTERRUPTED, not *_ERROR), we do nothing and give up immediately.
Differential Revision: https://phabricator.services.mozilla.com/D66271
--HG--
extra : moz-landing-system : lando
The test passes on nightly because the condition actually checks specifically for nightly. On non-nightly the test fails because although it mocks the channel with withFakeChannel, the code under test doesn't respect that.
Differential Revision: https://phabricator.services.mozilla.com/D65303
--HG--
extra : moz-landing-system : lando
This commit changes `Store::local_row_to_item` to validate local URLs,
and flags items with malformed URLs as invalid. These items are either
replaced with valid remote copies, if they exist, or deleted if not.
Additionally, `BaseBookmarksStore#_calculateIndex` no longer throws
when determining the sort index for an item with an invalid URL.
As an aside, we use the `url` crate to parse URLs. This is the same
crate as used by `MozURL`, which, in turn, backs the JS `URL`
constructor...so URLs should be validated the same way in Rust and JS.
Differential Revision: https://phabricator.services.mozilla.com/D64695
--HG--
extra : moz-landing-system : lando
This patch adds a scenario that ends up syncing the generated profile into FxSync.
Differential Revision: https://phabricator.services.mozilla.com/D53789
--HG--
extra : moz-landing-system : lando
This patch adds a scenario that ends up syncing the generated profile into FxSync.
Differential Revision: https://phabricator.services.mozilla.com/D53789
--HG--
extra : moz-landing-system : lando
This patch was generated with a script. It doesn't include all files:
- Files that use the preprocessor or fail to parse are skipped
- Files that are loaded as JSMs but don't use the .jsm extension are skipped (those will be renamed in Bug 1609269)
It was generated with the following command using d855222aa2/no-this-property-read.js:
```
hg revert --all &&
cp .gitignore .rgignore &&
rg --files-without-match -g '*.jsm' '^#endif|^#include|^#filter' | jscodeshift --stdin --transform ~/Code/jsm-rewrites/no-this-property-read.js --ignore-pattern ./mobile/android/modules/Sanitizer.jsm --ignore-pattern ./js/xpconnect/tests/unit/syntax_error.jsm &&
./mach eslint `hg st | rg '^M ' | sed 's/^M //'`
```
Differential Revision: https://phabricator.services.mozilla.com/D60187
--HG--
extra : moz-landing-system : lando
We need to stop relying on the global `this` in order to support ES Modules.
This instance was harder to fix by a script than others since it's in a try/catch,
so this patch is a manual change.
Differential Revision: https://phabricator.services.mozilla.com/D60080
--HG--
extra : moz-landing-system : lando
Legacy extensions are no longer loaded, so we can drop the build config for it. We
still need flags for handling experimental APIs since what we require differs between builds
and distributions.
Differential Revision: https://phabricator.services.mozilla.com/D57413
--HG--
extra : moz-landing-system : lando