There's currently EnsureTemporaryStorageIsInitializedInternal which can only be
called on the QuotaManager IO thread. That method shouldn't be exposed to quota
clients and origin operations. There should be a public method callable from
the PBackground thread returning a MozPromise instead. Such method will
guarantee that proper directory locking is acquired before temporary storage
initialization is started.
Differential Revision: https://phabricator.services.mozilla.com/D192135
One of the goals of the asynchronous temporary storage initialization is to
call EnsureTemporaryStorageIsInitialized only from InitTemporaryStorageOp.
Calling from other places including quota clients will be disallowed by
changing the method to a private method. The private nature of the method
should be emphasized by adding the Internal suffix.
Differential Revision: https://phabricator.services.mozilla.com/D188332
QuotaManagerDependencyFixture::IsStorageInitialized now uses
QuotaManager::StorageInitialized, so it makes sense to remove "Is" from all
storage initialization checking methods as well.
Differential Revision: https://phabricator.services.mozilla.com/D191933
QuotaManager::StorageInitialized is the primary source of truth.
QuotaManager::IsStorageInitialized only synchronously checks a flag which
shadows the primary state on the QuotaManager IO thread. Calling an async
method better matches other QuotaManagerDependencyFixture methods.
Differential Revision: https://phabricator.services.mozilla.com/D191932
This patch also adds QuotaManager::StorageInitialized and
QuotaManager::TemporaryStorageInitialized which can be then used by the
QuotaManagerDependencyFixture.
Differential Revision: https://phabricator.services.mozilla.com/D191930
There's now a dedicated method QuotaManager::ClearStoragesForOrigin which can
be called in the parent process directly, so the fixture doesn't have to use
IPC for clearing origins anymore.
Differential Revision: https://phabricator.services.mozilla.com/D191915
CachingDatabaseConnection relies on a global macro to toggle thread
ownership checks. The checks do not work with thread pool event targets
and by this change we ensure that the global macro users are not
impacted by such event target changes.
Depends on D190598
Differential Revision: https://phabricator.services.mozilla.com/D190601
Operations are shown as "pending" in the QM shutdown hang annotations until they are fully executed. Given that executing them often requires acquiring a directory lock first and also involves some thread hopping, we want to understand better, where we hang.
In particular many crashes show `ShutdownStorageOp` as pending but there is no sign in the stack trace that `DoDirectoryWork` ever started. In addition, most if not all of those crashes show either an open IDB connection or an unclosed Cache manager, which might interfere with directory locking here.
The annotation should help to confirm that `ShutdownStorageOp` is endlessly waiting to acquire the directory lock in those cases.
Differential Revision: https://phabricator.services.mozilla.com/D190425
ClearRequestBase::DeleteFiles currently provides a way to clear origins for any
combination of persistence type, origin scope and client type. All origin
directories need to be traversed to find relevant matches. This can be slow if
there are many origin directories. Fortunately, the traversal can be completely
avoided when exact origin is being cleared.
Changes done in this patch:
- added a dedicated ClearRequestBase::DeleteFiles method for clearing of exact
origins
- changed ClearOriginOp::DoDirectoryWork to use the new variant of
ClearRequestBase::DeleteFiles
Differential Revision: https://phabricator.services.mozilla.com/D186781
The removing of an origin directory became a bit more complex because when the
app shutdown already started, origin directories shouldn't be fully removed
from disk. They need to be only moved to a special directory. All this
complexity should be covered by a dedicated QuotaManager method.
Changes done in this patch:
- added a new method QuotaManager::RemoveOriginDirectory
- added a prefilled string for the special to-be-removed directory
- adjusted ClearRequestBase::DeleteFiles to use the new
QuotaManager::RemoveOriginDirectory method
Differential Revision: https://phabricator.services.mozilla.com/D186780
There are now no callers of nsIQuotaManagerService::ClearStoragesForPrincipal
which would request clearing of all storages for given prefix. The support for
that can be removed.
Changed done in this patch:
- removed the aClearAll argument from
nsIQuotaManagerService::ClearStoragesForPrincipal
- removed the aClearAll argument from the async IPC message
- removed the aClearAll argument from QuotaManager::ClearStoragesForOrigin
- changed ClearOriginOp to support clearing of exact origins only
Differential Revision: https://phabricator.services.mozilla.com/D186779
nsIQuotaManagerService::ClearStoragesForPrincipal currently supports both
clearing storages for a specific origin only and clearing storages for a group
of origins sharing the same prefix. It would be better to have separate methods
for this.
Changes done in this patch:
- added nsIQuotaManagerService::ClearStoragesForOriginPrefix
- added a new helper for testing clearOriginsByPrefix
- changed verifyStorage to accept an optional shared key name
- added thorough testing for the new method
Differential Revision: https://phabricator.services.mozilla.com/D186777
QuotaManagerService::ClearStoragesForOriginAttributesPattern and
QuotaManagerService::ClearStoragesForPrincipal still create sub actors which
makes it hard to add new clearing operations which would use async IPC messages
and which would inherit from the ClearRequestBase class as well.
Changes done in this patch:
- added QuotaManager::ClearStoragesForOrigin
- added QuotaManager::ClearStoragesForOriginAttributesPattern
- changed ClearRequestBase to inherit from ResolvableNormalOriginOp
- QuotaManagerService::ClearStoragesForOriginAttributesPattern reworked to use
an asynchronous message instead of a sub actor
- QuotaManagerService::ClearStoragesForPrincipal reworked to use an
asynchronous message instead of a sub actor
- added a new mactor QM_CUF_AND_IPC_FAIL similar to QM_IPC_FAIL
creating a sub actor
Differential Revision: https://phabricator.services.mozilla.com/D186628
ClearRequestBase currently provides generic directory locking based on generic
member variables. It's not totally clear what derived classes do in terms of
directory locking and work on the QuotaManager IO thread. It would be better if
ClearRequestBase only provided a generic DeleteFiles function instead.
Changed done in this patch:
- moved implementation of directory locking methods to derived classes of
ClearRequestBase
- moved implementation of DoDirectoryWork to derived classes as well
- adjusted member variables
Differential Revision: https://phabricator.services.mozilla.com/D186627
The goal to call QuotaManager::EnsureStorageIsInitializedInternal only from
InitOp has been achieved. It's now easy to change the method to be a private
method.
Changes done in this patch
- moved InitOp to the mozilla::dom::quota namespace
- added InitOp as a friend to QuotaManager
- changed QuotaManager::EnsureStorageIsInitializedInternal to be a private
method
Differential Revision: https://phabricator.services.mozilla.com/D186208
TestFileOutputStream currently doesn't create any client directory lock and
just directly bounces to the QuotaManager I/O thread where it ensures that
storage is initialized. This should be now replaced by calling
QuotaManager::OpenClientDirectory on the PBackground thread.
Changes done in this patch:
- added a PBackground task which is executed first
- added QuotaManager::OpenClientDirectory call on the PBackground thread
- removed QuotaManager::EnsureStorageIsInitializedInternal call on the IO
thread
Differential Revision: https://phabricator.services.mozilla.com/D186207
Origin operations which require storage initialization currently create a
universal directory lock first and then when the universal directorylock is
acquired, they bounce to the QuotaManager I/O thread where they ensure that
storage is initialized. This can be now replaced by just calling
QuotaManager::OpenStorageDirectory.
Changes done in this patch:
- replaced QuotaManager::CreateDirectoryLockInternal call with
QuotaManager::OpenStorageDirectory in corresponding OpenDirectory
implementations
- removed QuotaManager::EnsureStorageIsInitializedInternal call from
corresponding DoDirectoryWork implementations
- added QuotaManager::AssertStorageIsInitializedInternal to corresponding
DoDirectoryWork implementations
Differential Revision: https://phabricator.services.mozilla.com/D186206
SaveOriginAccessTimeOp::DoDirectoryWork currently doesn't call
QuotaManager::EnsureStorageIsInitializedInternal and just expects that
something else initialized storage previously. This seems to work, but it would
be cleaner to always make sure that storage is initialized. However, adding
QuotaManager::EnsureStorageIsInitializedInternal revealed another problem.
Storage shudown or storage clearing acquires an exlusive lock over entire
storage area which essentially forces that all existing directory locks are
released first. When the last directory lock for an origin is released, saving
of origin access time is scheduled. The problem is that it's scheduled after
the exclusive lock for storage shutdown or storage clearing, so storage would
be initialized again in the end or access time wouldn't be saved at all due
to quota manager shutdown being already in progress.
Changes done in this patch:
- added QuotaManager::EnsureStorageIsInitializedInternal call to
SaveOriginAccessTimeOp::DoDirectoryWork
- changed QuotaManager::UnregisterDirectoryLock to work with already cleared
directory lock tables
- added a new QuotaManager::ClearDirectoryLockTables method
- added QuotaManager::ClearDirectoryLockTables call to
ShutdownStorageOp::OpenDirectory and ClearStorageOp::OpenDirectory
- made ClearStorageOp and ShutdownStorageOp friend classes of QuotaManager
Differential Revision: https://phabricator.services.mozilla.com/D187877
