It seems that sometimes the cache purge task may fail to remove the
cache folder. When that happens, we might end up with multiple cache
folders in the profile folder. Since the task is unable to remove it,
we should remove the folders on a background thread (check once a day)
Differential Revision: https://phabricator.services.mozilla.com/D194051
Purge HTTP disk cache using backgroundtasks
Motivation:
In the History settings preferences panel you may choose to
`Clear History when Firefox closes` - which sets the
`privacy.sanitize.sanitizeOnShutdown` pref.
If the `Cache` checkbox is also checked it sets the
`privacy.clearOnShutdown.cache` pref to true.
When both of these prefs are true `CacheObserver::ClearCacheOnShutdown()`
will return true, which will cause Firefox to clear the HTTP disk cache
during shutdown. This will block shutdown for howeverlong this takes. If the
user has a slow disk, or the user is on Windows and something is blocking the
deletion of those files, this will trigger a shutdown hang making Firefox
crash. This leads to a bad user experience as trying to start Firefox again
during this time will show the "Firefox is already running message".
Eventually a crash dump is produced. Bug 1356853 and bug 1682899 are caused
by this specific issue.
In order to avoid these crashes and a bad user experience we have
a few options:
1. Completely disable the disk cache when a user checks this box.
This option will degrade the user's browsing experience.
2. Don't delete the folder at shutdown
Whether we do this by removing the checkbox or simply not respecting
the pref value, users who already have this setting would be surprised
if the cache folder stops being deleted.
3. Use a thread pool to delete the files
While it's likely to speed up the deletion at least a little bit,
this would introduce additional complexity while not completely
fixing the issue. A slow disk will still block shutdown.
4. Delete the cache at shutdown using a separate process
This is likely the best option. It has the advantage of not blocking
shutdown while at the same time maintaining similar properties to the
existing functionality:
- The cache folder is deleted at shutdown.
- Has the same behaviour if Firefox gets killed or crashes for different issues.
- Behaves similarly if the OS is forcefully shutdown during or before we begin to purge.
- Additionaly, because we rename the folder prior to dispatching the background task, if the purging isn't completed we
- don't rebuild the cache from an incomplete folder on next restart
- are able to resume the purging after Firefox startup
A particularly special case is the Windows shutdown.
If the user shuts down windows that will try to close Firefox. If the shutdown
takes too long, the user will see the "Close anyway" button and may click
it thus preventing the cache purge to complete.
When using a background task we have a similar situation, but the button won't
even appear. So after the next Firefox restart we will check if the cache
needs to be purged.
Largely, the new behaviour will be:
- At shutdown we conditionally dispatch a background task to delete the folder
- If creating the process fails (for some reason) we fallback to the old way
of synchronously deleting the folder.
- The task will then try to delete the folder in the background
- If for some reason it fails, we will dispatch a new background task shortly
after Firefox restarts to clean up the old folders.
Differential Revision: https://phabricator.services.mozilla.com/D126339
Like in case of bug 1506534, this telemetry is sampled periodically every time 2GB of data is written to the cache, i.e. when the cache was used for some time and there is a chance that its content has changed significantly.
Differential Revision: https://phabricator.services.mozilla.com/D29678
Whenever a cache entry is accessed during a document load, eTLD+1 of the top level document is added to the entry's metadata. Number of accessing sites is also stored in cache index. So we know how many copies of each entry would we have if we did a first party isolation without data deduplication. The telemetry is sent every time we write 2GB to the cache and then the data is reset. Telemetry report ID is an identifier of the telemetry cycle and it's used to invalidate eTLD+1 hashes in all cache entries.
Differential Revision: https://phabricator.services.mozilla.com/D26425
This patch changes all size limits in CacheObserver to kilobytes. The same unit is used at most places when checking these limits. This avoids uint32_t overflow when converting to bytes and back.
We can hit the limit very easily when writing javascript bytecode as alternative data to the cache entry because all data is written at once but CacheFileOutputStream splits it into chunks which are then written on a backgound thread. 40MB was chosen because bytecode is usually 4x-10x larger than the original data, so it can occupy most of the cache entry which is limited to 50MB.