Commit Graph
93 Commits
Author SHA1 Message Date
Valentin Gosu 0e41566d01 Bug 1848542 - Do a cache cleanup when cache purging is enabled r=necko-reviewers,kershaw
It seems that sometimes the cache purge task may fail to remove the
cache folder. When that happens, we might end up with multiple cache
folders in the profile folder. Since the task is unable to remove it,
we should remove the folders on a background thread (check once a day)

Differential Revision: https://phabricator.services.mozilla.com/D194051
2023-11-21 17:29:08 +00:00
Valentin Gosu 3bb8425e41 Bug 1705676 - Purge HTTP disk cache using backgroundtasks (out-of-process) r=nalexander,necko-reviewers,dragana
Purge HTTP disk cache using backgroundtasks

Motivation:

In the History settings preferences panel you may choose to
`Clear History when Firefox closes` - which sets the
`privacy.sanitize.sanitizeOnShutdown` pref.
If the `Cache` checkbox is also checked it sets the
`privacy.clearOnShutdown.cache` pref to true.

When both of these prefs are true `CacheObserver::ClearCacheOnShutdown()`
will return true, which will cause Firefox to clear the HTTP disk cache
during shutdown. This will block shutdown for howeverlong this takes. If the
user has a slow disk, or the user is on Windows and something is blocking the
deletion of those files, this will trigger a shutdown hang making Firefox
crash. This leads to a bad user experience as trying to start Firefox again
during this time will show the "Firefox is already running message".
Eventually a crash dump is produced. Bug 1356853 and bug 1682899 are caused
by this specific issue.

In order to avoid these crashes and a bad user experience we have
a few options:

1. Completely disable the disk cache when a user checks this box.
  This option will degrade the user's browsing experience.
2. Don't delete the folder at shutdown
  Whether we do this by removing the checkbox or simply not respecting
  the pref value, users who already have this setting would be surprised
  if the cache folder stops being deleted.
3. Use a thread pool to delete the files
  While it's likely to speed up the deletion at least a little bit,
  this would introduce additional complexity while not completely
  fixing the issue. A slow disk will still block shutdown.
4. Delete the cache at shutdown using a separate process
  This is likely the best option. It has the advantage of not blocking
  shutdown while at the same time maintaining similar properties to the
  existing functionality:
  - The cache folder is deleted at shutdown.
  - Has the same behaviour if Firefox gets killed or crashes for different issues.
  - Behaves similarly if the OS is forcefully shutdown during or before we begin to purge.
  - Additionaly, because we rename the folder prior to dispatching the background task, if the purging isn't completed we
    - don't rebuild the cache from an incomplete folder on next restart
    - are able to resume the purging after Firefox startup

A particularly special case is the Windows shutdown.
If the user shuts down windows that will try to close Firefox. If the shutdown
takes too long, the user will see the "Close anyway" button and may click
it thus preventing the cache purge to complete.
When using a background task we have a similar situation, but the button won't
even appear. So after the next Firefox restart we will check if the cache
needs to be purged.

Largely, the new behaviour will be:
- At shutdown we conditionally dispatch a background task to delete the folder
- If creating the process fails (for some reason) we fallback to the old way
  of synchronously deleting the folder.
- The task will then try to delete the folder in the background
- If for some reason it fails, we will dispatch a new background task shortly
  after Firefox restarts to clean up the old folders.

Differential Revision: https://phabricator.services.mozilla.com/D126339
2022-08-15 07:07:45 +00:00
Valentin Gosu 200b910915 Bug 1729717 - Remove unused CleanupCacheDirectories r=necko-reviewers,dragana
This was used previously when switching between cache backend versions.

Differential Revision: https://phabricator.services.mozilla.com/D124922
2021-09-13 12:07:37 +00:00
Csoregi Natalia 0f56f9b8f3 Backed out 2 changesets (bug 1729717) for causing assertion failures on CacheObserver.cpp. CLOSED TREE
Backed out changeset 7123efe825f0 (bug 1729717)
Backed out changeset 92e06bd625c9 (bug 1729717)
2021-09-13 12:40:11 +03:00
Valentin Gosu 8cfa7af727 Bug 1729717 - Remove unused CleanupCacheDirectories r=necko-reviewers,dragana
This was used previously when switching between cache backend versions.

Differential Revision: https://phabricator.services.mozilla.com/D124922
2021-09-13 09:12:34 +00:00
Valentin Gosu 56cd5036af Bug 1714307 - Autofix static-analysis warnings in netwerk/ r=necko-reviewers,kershaw
Differential Revision: https://phabricator.services.mozilla.com/D116701
2021-06-04 12:35:16 +00:00
Butkovits Atila 1f3e4dc865 Backed out changeset c4b44c4ef15b (bug 1714307) for causing build bustages on nsHTTPCompressConv.h. CLOSED TREE 2021-06-04 13:53:37 +03:00
Valentin Gosu 5b4ead87fd Bug 1714307 - Autofix static-analysis warnings in netwerk/ r=necko-reviewers,kershaw
Differential Revision: https://phabricator.services.mozilla.com/D116701
2021-06-04 10:00:55 +00:00
David Major 9829f29105 Bug 1688292 - Fix non-unified build errors in netwerk/cache2/ r=necko-reviewers,valentin
This is a drive-by fix while I was looking into something else.

Differential Revision: https://phabricator.services.mozilla.com/D102795
2021-01-25 12:23:13 +00:00
Alexis Beingessner d494005666 Bug 1682932 - ensure sAutoMemoryCacheCapacity is properly synchronized. r=valentin,necko-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D100027
2020-12-17 17:16:14 +00:00
Michal Novotny eabbd45270 Bug 1662012 - Remove filesystem reporting from the cache, r=valentin,necko-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D88740
2020-08-31 15:16:07 +00:00
Michal Novotny 4c6e01b44e Bug 1662011 - Remove hash stats code from the cache, r=valentin,necko-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D88739
2020-08-31 06:53:06 +00:00
Michal Novotny 885022c12f Bug 1645052 - Do not store number of bytes written to the cache in preferences, r=valentin,necko-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D88488
2020-08-28 09:22:45 +00:00
Michal Novotny 036dcbbc63 Bug 1645051 - Calculated smart max cache size shouldn't overwrite browser.cache.disk.capacity pref r=necko-reviewers,mayhemer
Differential Revision: https://phabricator.services.mozilla.com/D83377
2020-07-14 16:54:22 +00:00
Michal Novotny 6092b2983f Bug 1606207 - Remove first-party cache isolation telemetry r=mayhemer
Differential Revision: https://phabricator.services.mozilla.com/D59352
2020-01-14 13:53:01 +00:00
Tom Tung 7c1b52bb1f Bug 1474608 - P2 - Clear origin attributes data directly on cache2; r=asuth,michal,baku
Differential Revision: https://phabricator.services.mozilla.com/D34324
2019-12-02 19:47:05 +00:00
Nicholas Nethercote 16a032cb38 Bug 1562305 - Make privacy.sanitize.sanitizeOnShutdown and privacy.clearOnShutdown.cache static prefs. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37195
2019-07-11 04:38:36 +00:00
Nicholas Nethercote bfb9b0862f Bug 1562305 - Make browser.cache.max_shutdown_io_lag a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37192
2019-07-11 04:38:36 +00:00
Nicholas Nethercote 093d032335 Bug 1562305 - Remove browser.cache.compression_level pref. r=michal
It's unused.

Differential Revision: https://phabricator.services.mozilla.com/D37191
2019-07-11 04:13:45 +00:00
Nicholas Nethercote f96bc38248 Bug 1562305 - Make browser.cache.disk.max_{,priority_}chunks_memory_usage static prefs. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37190
2019-07-11 04:13:21 +00:00
Nicholas Nethercote 7b6c4f99f0 Bug 1562305 - Make browser.cache.memory.max_entry_size a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37189
2019-07-11 04:12:59 +00:00
Nicholas Nethercote a1a2ab010d Bug 1562305 - Make browser.cache.disk.max_entry_size a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37187
2019-07-11 04:12:36 +00:00
Nicholas Nethercote 07340690a2 Bug 1562305 - Make browser.cache.disk.preload_chunk_count a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37186
2019-07-11 04:12:19 +00:00
Nicholas Nethercote d2a45e9587 Bug 1562305 - Make browser.cache.disk.free_space_hard_limit a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37185
2019-07-11 04:11:59 +00:00
Nicholas Nethercote bcf1710f22 Bug 1562305 - Make browser.cache.disk.free_space_soft_limit a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37183
2019-07-11 04:11:41 +00:00
Nicholas Nethercote 5c35048f5a Bug 1562305 - Make browser.cache.memory.capacity a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37182
2019-07-11 04:11:39 +00:00
Nicholas Nethercote 35c169ed23 Bug 1562305 - Make browser.cache.disk.smart_size.enabled a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37181
2019-07-11 04:40:17 +00:00
Nicholas Nethercote 63503652ee Bug 1562305 - Make browser.cache.disk.metadata_memory_limit a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37179
2019-07-11 04:40:17 +00:00
Nicholas Nethercote ccb6e1fae3 Bug 1562305 - Make browser.cache.memory.enable a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37178
2019-07-11 04:10:13 +00:00
Nicholas Nethercote b4f4715df4 Bug 1562305 - Make browser.cache.disk.enable a static pref. r=michal
Differential Revision: https://phabricator.services.mozilla.com/D37177
2019-07-11 04:09:50 +00:00
Michal Novotny cd57829945 Bug 1548472 - Collect telemetry about how is the cache split by the content type, r=mayhemer
Like in case of bug 1506534, this telemetry is sampled periodically every time 2GB of data is written to the cache, i.e. when the cache was used for some time and there is a chance that its content has changed significantly.

Differential Revision: https://phabricator.services.mozilla.com/D29678
2019-05-08 13:15:40 +00:00
Michal Novotny 53f5f245cb Bug 1506534 - Collect telemetry to measure how much penalty we will experience with first-party cache isolation, r=mayhemer, data-r=chutten
Whenever a cache entry is accessed during a document load, eTLD+1 of the top level document is added to the entry's metadata. Number of accessing sites is also stored in cache index. So we know how many copies of each entry would we have if we did a first party isolation without data deduplication. The telemetry is sent every time we write 2GB to the cache and then the data is reset. Telemetry report ID is an identifier of the telemetry cycle and it's used to invalidate eTLD+1 hashes in all cache entries.

Differential Revision: https://phabricator.services.mozilla.com/D26425
2019-04-25 12:21:17 +00:00
Valentin Gosu ea3eb469ce Bug 1532324 - Remove manual NS_RELEASE from netwerk/* r=JuniorHsu
Differential Revision: https://phabricator.services.mozilla.com/D22005
2019-03-05 18:43:02 +00:00
Michal Novotny 44506e93f2 Bug 1479357 - Specific values for browser.cache.disk.capacity do break the cache, r=mayhemer
This patch changes all size limits in CacheObserver to kilobytes. The same unit is used at most places when checking these limits. This avoids uint32_t overflow when converting to bytes and back.
2019-01-08 16:28:00 +02:00
Sylvestre Ledru e5a134f73a Bug 1511181 - Reformat everything to the Google coding style r=ehsan a=clang-format
# ignore-this-changeset
2018-11-30 11:46:48 +01:00
Michal Novotny 0e7835c51e Bug 1457084 - Increase max chunk memory usage limit, r=mayhemer
We can hit the limit very easily when writing javascript bytecode as alternative data to the cache entry because all data is written at once but CacheFileOutputStream splits it into chunks which are then written on a backgound thread. 40MB was chosen because bytecode is usually 4x-10x larger than the original data, so it can occupy most of the cache entry which is limited to 50MB.
2018-05-23 05:03:00 +03:00
Honza Bambas 24b85237ae Bug 1430197 - Remove the HTTP cache half-live experiment, r=michal 2018-01-12 11:51:00 +02:00
Nicholas Nethercote 45164b6a68 Bug 1416638 - Add a PrefValueKind arg to Preferences::Get*(), and remove Preferences::GetDefault*(). r=glandium
MozReview-Commit-ID: LgnZAf1pAl1
2017-11-14 19:06:01 +11:00
Michal Novotny 31c97d265e Bug 1382688 - Do not allow to enable old disk cache backend, r=honzab 2017-08-24 10:25:38 +02:00
Bill McCloskey ce42826bdf Bug 1372405 - Provide names for all runnables in the tree (r=froydnj)
MozReview-Commit-ID: DKR6ROiHRS7
2017-06-26 14:19:58 -07:00
Carsten "Tomcat" Book 238bf154d5 Backed out changeset 4f6302a98ae4 (bug 1372405) 2017-06-21 13:59:26 +02:00
Bill McCloskey 67e8af4720 Bug 1372405 - Provide names for all runnables in the tree (r=froydnj)
MozReview-Commit-ID: DKR6ROiHRS7
2017-06-20 21:44:11 -07:00
Carsten "Tomcat" Book bbe9441993 Backed out changeset 9846de3bd954 (bug 1372405) 2017-06-20 08:27:02 +02:00
Bill McCloskey f69608368b Bug 1372405 - Provide names for all runnables in the tree (r=froydnj)
MozReview-Commit-ID: DKR6ROiHRS7
2017-06-19 22:25:47 -07:00
Andrea Marchesini 46a9207d60 Bug 1328653 - Merging all the various *OriginAttributes to just one, r=huseby 2017-01-12 17:38:48 +01:00
Yoshi Huang 85db56b906 Bug 1237152 - rename clear-origin-data to clear-origin-attributes-data, r=smaug
find \( -name '*.cpp' -o -name '*.h' -o -name '*.js' -o -name '*.jsm' -o -name '*.idl' -o -name '*.html' \) \
-a -type f -exec sed -i 's/clear-origin-data/clear-origin-attributes-data/g' {} \;
2016-10-07 17:45:10 +08:00
James Andreou ec9db2689b Bug 1282882 - Remove LoadContextInfo and LoadContext private browsing flags; r=jdm,mayhemer 2016-08-26 13:06:48 -04:00
Honza Bambas 2e8472e065 Bug 1276930 - Bypass HTTP cache data read right after shutdown, r=michal 2016-06-22 16:18:01 -07:00
Michal Novotny 911d646438 Bug 1267980 - Leak buffers in CacheFileChunk and CacheFileMetadata during shutdown, r=honzab 2016-05-11 11:46:54 +02:00
Kyle Huey b972c94d0f Bug 1268313: Part 7 - Move NS_NewRunnableMethod and friends to mozilla::NewRunnableMethod. r=froydnj 2016-05-05 01:45:00 -07:00