Scan from the command line: exit status, Manual Scan options, playlist refresh (#19977)

* file_path: stop at the end of the path after '~'

fill_pathname_expand_special() skipped two characters after a leading
'~' or ':', so a bare "~" read past its NUL and "~x" lost the x. It
now skips the prefix and a slash after it, if there is one.

* scan: hand the callback user data and what the scan did

task_push_dbscan() and task_push_manual_content_scan() take user data
for their callback, and the callback's task_data is now a
task_dbscan_result_t: how many entries the scan added, how many its
playlists already held, and how many files matched nothing. A file is
counted by what became of it, so an archive that is expanded is not
counted as unmatched, and a loose scan's unmatched files count as
added. Callers pass NULL for now.

* scan: say in the result when a scan failed

A scan that ends before it runs (bad setup, no valid content, a DAT
that will not load) or ends with a task error now sets the result's
failed flag. A database folder that is missing or unset is treated as
an empty one, so it reports the same "no database" error instead of
finishing as if it had scanned.

* scan: count the files a scan could not read

A file whose scan fails (an unreadable file, a failed lookup) was in
no count. The result now carries an errors count for them. They do
not fail the scan.

* scan: print --scan errors on stderr and totals on stdout

A command-line scan now ends with a totals line on stdout, or with the
reason it failed on stderr. A file it could not read is listed on
stderr. The ++/==/?? marks are coloured only when their stream is a
terminal, so pipes and redirected output get plain text; on Windows
redirected output keeps its marks instead of losing them to
WriteConsole.

* scan: the manual scan finder skips a finished task

A finished scan stays in the task queue while its callback runs, and
the finder that refuses a second scan for the same playlist still
matched it, so a scan pushed from that callback was refused.

* retroarch: run every --scan, then exit with what the scans did

Each --scan is collected while the command line is parsed and run in
turn once it is done, so later options and further --scan flags are
no longer dropped. A path is expanded and made absolute first; one
that does not exist, or a scan that cannot start, is reported on
stderr and the next path still runs. --scan with --cloudsync or
--command is refused.

Without --menu the exit status is 1 if any scan failed, else 0 if any
added an entry, else 2, and a content path given with the scans is
ignored, with a line on stderr. With --menu each finished scan starts
the next, as a second scan is refused while one runs.

* manual_content_scan: clear the database name a refresh did not save

A playlist with no saved database name left the name buffer
uninitialised, and the refresh compared it against every installed
database, so whatever was on the stack could pick a database the
playlist never named.

* scan: Skip Database References names the scan's one playlist

The option was meant to put the playlist's name in each entry's
db_name, even for a database match (#18641), for users who build
their own thumbnail sets. When the scan picks the playlists, db_name
also picks the playlist an entry goes to, so with the option on every
match went to one playlist named after the content folder.

It now applies only when the scan writes one playlist, and gives
that playlist's name, a loose scan's unmatched entries included. With
the target on <Automatic>, or Fully Automatic with the option left on
from a Custom scan, matches go to their databases' playlists as with
it off; the entries of a custom playlist look their thumbnails up
under its name, not the folder's.

* scan: name the scanned folder in the start line

The "Scanning content" line, in the log and in --scan's output, has
named the database folder since the manual scan and the database scan
were merged (2fdf736d51); before, it named the path being scanned. It
names the content folder again.

* manual_content_scan: reset, scan overrides and their apply

The command line's scan options need each scan to start from the
defaults and to lay their values over them, or over a playlist's saved
settings. Add a reset, a record of those options and a function that
applies it through the existing setters: a core named by path, file
name or display name, a database, a playlist, the database check or a
DAT, extensions and the switches, stopping with a reason at the first
that does not resolve.

A core path is normalised before it is compared. A core's file name
matches in the case given first, then ignoring case, on every
platform; a display name matches exactly. Given another database, a
playlist named after its database keeps that name as a custom one, so
a refresh still writes the same playlist.

The core setter's copy step becomes its own function, so a core found
by path or file name can be set. Nothing calls the new functions yet.

* scan: run --scan items through one chain

The --scan paths ran through a loop in retroarch.c, and with --menu
through a second chain of their own. The scan options and playlist
refreshes to come need one runner for every item, from the command
line and from the menu.

task_push_dbscan_chain() runs a list of items one task at a time: a
prep task lists the databases, then each step is pushed from the last
task's callback on the main thread, so nothing waits and nothing
locks. An item that fails is reported and the run goes on; a
cancelled task ends it. retroarch.c builds the items in the first
argument pass and hands them to the chain; without --menu it waits
for the chain's end and exits as before.

Running out of memory prints "out of memory" on stderr: for an item's
path the item fails, and for the chain itself nothing runs, which
without --menu exits 1.

The chain also prints a scan's override errors, and the warnings for
an option that needs one playlist or does nothing for a single file,
which the scan options of the next commit can trigger.

* retroarch: Manual Scan options for --scan

--scan-playlist, --scan-database, --scan-match, --scan-dat, --scan-core,
--scan-exts and the five switches set the Manual Scan settings of the
--scan before them; a --scan without any stays Fully Automatic.

Values are checked in the first pass, so an option before any --scan,
an empty or bad value, or one too long for its setting ends the run
with its reason on stderr and status 1, before the config loads and
before anything is scanned. Cores, databases and DAT files are
resolved when their scan runs, and one that does not resolve fails
only that scan.

* scan: refresh playlists in the chain

A refresh item rescans one playlist with its saved scan settings, as
the menu's Refresh Playlist does, with the item's scan options laid
over them and written back by the scan. A refresh-all item does the
same for every playlist in the playlist folder that has saved
settings, in the order Manage Playlists lists them, histories and
favorites left out.

A refresh item names its playlist by name or by a path in the playlist
folder. A bare "name.lpl" is the playlist folder's file, and an
absolute path is normalised before it is checked against the folder.
The name is looked up in the folder's listing on every platform, the
exact name first, then ignoring case, since macOS and FAT/exFAT cards
ignore case; the refresh then takes the file's own name, so a
database's playlist stays one. Two or more playlists that match only
in other cases fail the item.

The playlist folder walk and each playlist load run in prep tasks: on
the worker with a threaded queue, under the shared I/O window without
one, each pushed from the previous step's callback. A playlist whose
saved content folder is gone is skipped rather than failed. A listing
that runs out of memory fails the refresh-all; a playlist folder that
is missing or cannot be read lists nothing, and the refresh-all ends
with "no playlists with saved scan settings". A refresh's start line
names its content folder; its totals and failure lines name the
playlist.

* retroarch: --scan-refresh and --scan-refresh-all

--scan-refresh=NAME|PATH rescans a playlist with its saved scan
settings, as the menu's Refresh Playlist does, and --scan-refresh-all
rescans every playlist that has them. Both are items of the same run
as --scan: the scan options after one are laid over the saved
settings and written back, and the exit status follows the same rule.

--scan-playlist after a refresh is refused, so a refresh always
updates its own playlist, and an empty --scan-refresh is refused as
the other empty values are. Like --scan, the refresh options cannot be
combined with --cloudsync or --command; the error names whichever of
the three comes first on the command line.

* menu: Refresh All Playlists

Manage Playlists gets Refresh All Playlists as its first entry when it
lists any playlist, so an empty list still says there are no entries.
It rescans every playlist that has saved scan settings, with those
settings, through the same chain as --scan-refresh-all, and ends with
one notification saying how many playlists were refreshed, skipped
(content folder gone) and failed.

While a content scan runs it is refused with a new message; a refused
scan had none. Other languages come through Crowdin.

While a scan chain runs (Refresh All, or a command-line scan with
--menu), Start Scan, a playlist's Refresh, Scan Directory and Scan
File are refused with the same message, as they would run with the
settings the chain last loaded. task_push_dbscan() returns false then
too, which covers the companion and the iOS caller.

* docs: --scan, refresh and the Manual Scan options in the man page

--help lists the options but not what a script needs: the order the
scans run in, the exit status and what the options set. The man page
now has them, next to --cloudsync, and CHANGES.md has the entry.

* scan chain: writes from workers reach the playlist cache by note

A playlist written by a task - the chain's prep parse, a scan, a
playlist-manager task, whatever thread it runs on - leaves a note on a
lock-free stack instead of reading the cached playlist; the main
thread applies the notes when it next decides whether the cache may
stand in, so no worker reads playlist_cached while the main thread
frees or installs it. Without threads or pointer atomics a write
applies at once.

The chain's prep tasks set their handlers by name, and
thread_read_audit.py fails on a handler set from a
retro_task_handler_t parameter, which would hide every handler passed
in. The chain puts the Manual Scan settings it found back when it
ends, and is no longer running when its done callback runs, so that
callback may start another. task_push_dbscan() says why it refuses,
so the menu and the companions show the same reason, and the
companions' status line takes it from companion_core. --scan and
--scan-refresh set the verbosity override, so a config with logging
off cannot detach the console the totals go to, and refuse a path too
long for the chain's buffers. The Refresh All notification passes the
length written, and fill_pathname_expand_special() stops at the end of
a buffer shorter than the home or application directory.

Lanes: playlist_parity_test writes the cached file from a worker,
once at the same size and then while the main thread reuses and frees
the cache; database_scan_test checks the refusal message, the restored
settings and a done callback that starts the next chain;
fill_pathname_test expands into a buffer shorter than the home
directory.

* playlist: header-only parse for the scan chain's prep

The prep task needs a playlist's saved scan settings and nothing else,
yet parsed every entry and, through autofix, could write the file back
from a worker. playlist_parse_begin_header() stops at "items", rebases
the scan record in memory when the base content directory moved, and
marks the playlist so playlist_write_file() refuses it.

Lane: playlist_parity_test header_only (entries skipped, scan dir
rebased, file bytes untouched, old format).

* scan chain: ask the file system nothing on the main thread

The chain's steps ran on the main thread between its tasks and stat'd
and realpath'd there: a --scan path's resolve and stat, a
--scan-refresh path's two realpaths, the content folder's stat, the
saved and override DATs' stat and size, an override core path's
realpath.  On a network share each of those can stall a frame.

- manual_content_scan_fs_read() reads them on any thread, touching
  nothing but its arguments; the _fs variants of the content-folder,
  playlist and override setters take its answers instead of asking.
  The plain setters pass NULL and behave as before, so the menu is
  unchanged.
- A --scan path gets its own prep task (dbscan_path_handler); a
  --scan-refresh resolves its path inside the load task, and the load
  reads the folder, the saved DAT and the overrides' paths once the
  header is parsed.  The worker gets copies of the item's strings.
- The prep's state is freed in one place.

Lane: database_scan_test "fs off main": six items that each stop
before their scan, on a threaded queue, with path_stat, path_is_valid,
path_is_directory, path_get_size and path_resolve_realpath wrapped:
no call on the main thread.

Not moved: the scan task's own push still stats the content folder
and the DAT (manual_content_scan_get_task_config), shared with the
menu.

* samples: the scan chain's lanes hold on Windows and without realpath

From a native Windows run of both tests:

- database_scan_test "fs off main" refreshed f_gone.lpl through
  "pl/../pl/", which only a realpath folds; the sample builds without
  RARCH_INTERNAL, where path_resolve_realpath() folds nothing, so the
  item was rightly not in the playlist folder.  It names the playlist
  by its plain path now, which the check still realpaths on the
  worker.
- "another case, a database's playlist" read the system name type
  after the chain ended, and the chain puts the Manual Scan settings
  back then.  The item callback records each scan's type instead.
- playlist_parity_test's header lane expected "/moved/snes"; autofix
  converts the suffix to the local form, "/moved\snes" on Windows.
- The worker lane's stress read required a core name of every read.
  Windows replaces the file with MoveFileExW, and an open that meets
  the replace fails: that read is an empty playlist, which has none.

---------

Co-authored-by: LibretroAdmin <[email protected]>

CHANGES.md

@@ -147,6 +147,7 @@
 - SCAN: Interpret M3U files during scanning
 - SCAN: Merge manual and automatic scan into a more flexible setup
 - SCAN: Improve GDI content recognition
+- SCAN: --scan runs every path and exits with what the scans did; Manual Scan options, --scan-refresh and --scan-refresh-all on the command line; Refresh All Playlists in Manage Playlists
 - SHADER: Display on-screen error when preset load fails
 - SHADER: Add gyroscope / accelerometer uniforms for shaders
 - SHADER: Add aggressive shader caching to slang shader backend

core_info.c

@@ -1648,7 +1648,7 @@ static bool core_info_path_is_standalone_exempt(
    return false;
 }
 
-static size_t core_info_get_file_id(const char *core_filename,
+size_t core_info_get_file_id(const char *core_filename,
       char *s, size_t len)
 {
    size_t _len;

core_info.h

@@ -200,6 +200,11 @@ bool core_info_list_update_missing_firmware(core_info_ctx_firmware_t *info);
 bool core_info_find(const char *core_path,
       core_info_t **core_info);
 
+/* Writes the file id of 'core_filename' (no extension or
+ * platform suffix) to 's'; returns its length */
+size_t core_info_get_file_id(const char *core_filename,
+      char *s, size_t len);
+
 bool core_info_load(const char *core_path);
 
 /* The extensions of every core that claims the database at

docs/retroarch.6

@@ -65,6 +65,68 @@ Prints available features compiled into RetroArch, then exits.
 \fB--cloudsync\fR
 Runs a cloud sync with the configured cloud sync driver, then exits.
 
+.TP
+\fB--scan PATH\fR
+Scans the folder or file PATH for content and adds it to playlists, as Manual Scan does.
+Can be given more than once.
+
+.TP
+\fB--scan-refresh NAME|PATH\fR
+Rescans one playlist with the scan settings it saved, as Refresh Playlist does.
+NAME matches a playlist's name exactly first, then ignoring case. PATH must be in the playlist folder.
+Like Refresh Playlist, it drops entries whose files are gone or whose extension the saved settings no longer include.
+A playlist whose saved content folder is missing is skipped instead, so an unplugged drive does not empty it.
+
+.TP
+\fB--scan-refresh-all\fR
+Does --scan-refresh for every playlist that has saved scan settings, as Refresh All Playlists does.
+
+.IP
+These run in the order given, and one that fails does not stop the rest.
+Each prints a totals line on stdout, or its error on stderr.
+Without --menu, RetroArch exits when they are done and does not load content given on the command line.
+The exit status is then 1 if any failed, 0 if any added an entry, and 2 if none added anything; a skipped playlist does not count.
+With --menu, they run while the menu does.
+They cannot be combined with --cloudsync or --command.
+
+.IP
+The options below set the Manual Scan settings of the --scan, --scan-refresh or --scan-refresh-all before them.
+On a refresh they override the saved settings, and the refresh saves them.
+A --scan without any is Fully Automatic.
+
+.TP
+\fB--scan-playlist NAME\fR
+Sets Target Playlist: a database's name, or a custom playlist name. Not allowed on a refresh.
+
+.TP
+\fB--scan-database NAME|first\fR
+Sets Database To Match: the database NAME, or the first database that matches.
+
+.TP
+\fB--scan-match strict|loose|none\fR
+Sets Database Check.
+
+.TP
+\fB--scan-dat FILE\fR
+Sets Arcade DAT File. Matching is strict unless --scan-match loose is given.
+
+.TP
+\fB--scan-core CORE\fR
+Sets Default Core: a core's path, its file name (with or without _libretro and the extension) or its display name.
+
+.TP
+\fB--scan-exts EXTS\fR
+Sets File Extensions, separated by commas or spaces.
+
+.TP
+\fB--scan-no-recursive, --scan-no-archives\fR
+Turn off Scan Recursively and Scan Inside Archives.
+
+.TP
+\fB--scan-overwrite, --scan-validate, --scan-omit-db-ref\fR
+Turn on Overwrite Existing Playlist, Validate Existing Entries and Skip Database References From Playlist.
+They act only when the scan has one playlist, and --scan-overwrite only on a folder; otherwise they print a warning and are ignored.
+
 .TP
 \fB-L PATH, --libretro PATH\fR
 Path to a libretro implementation which is to be used.

intl/msg_hash_lbl.h

@@ -1955,6 +1955,10 @@ MSG_HASH(
    MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST,
    "playlist_manager_refresh_playlist"
    )
+MSG_HASH(
+   MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS,
+   "playlist_manager_refresh_all_playlists"
+   )
 MSG_HASH(
    MENU_ENUM_LABEL_PLAYLIST_SETTINGS_BEGIN,
    "playlist_settings_begin"

intl/msg_hash_us.h

@@ -23216,6 +23216,14 @@ MSG_HASH(
    MENU_ENUM_SUBLABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST,
    "Add new content and remove invalid entries by repeating the content scan operation last used to create or edit the playlist."
    )
+MSG_HASH(
+   MENU_ENUM_LABEL_VALUE_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS,
+   "Refresh All Playlists"
+   )
+MSG_HASH(
+   MENU_ENUM_SUBLABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS,
+   "Rescan every playlist that has saved scan settings, with those settings."
+   )
 MSG_HASH(
    MENU_ENUM_LABEL_VALUE_DELETE_PLAYLIST,
    "Delete Playlist"
@@ -32548,6 +32556,10 @@ MSG_HASH(
    MSG_PLAYLIST_MANAGER_REFRESH_DAT_FILE_TOO_LARGE,
    "Refresh failed - arcade DAT file too large (insufficient memory): "
    )
+MSG_HASH(
+   MSG_PLAYLIST_MANAGER_REFRESH_ALL_DONE,
+   "Playlists refreshed: %u, skipped: %u, failed: %u"
+   )
 MSG_HASH(
    MSG_ADDED_TO_FAVORITES,
    "Added to favorites"
@@ -34344,6 +34356,10 @@ MSG_HASH(
    MSG_MANUAL_CONTENT_SCAN_END,
    "Scan complete: "
    )
+MSG_HASH(
+   MSG_MANUAL_CONTENT_SCAN_ALREADY_RUNNING,
+   "A content scan is already running."
+   )
 MSG_HASH(
    MSG_CORE_BACKUP_SCANNING_CORE,
    "Scanning core: "

libretro-common/file/file_path.c

@@ -1132,6 +1132,7 @@ size_t fill_pathname_join_delim(char *s, const char *dir,
 size_t fill_pathname_expand_special(char *s, const char *in_path, size_t len)
 {
 #if !defined(RARCH_CONSOLE) && defined(RARCH_INTERNAL)
+   char *start = s;
    if (in_path[0] == '~' || in_path[0] == ':')
    {
       char app_dir[DIR_MAX_LENGTH];
@@ -1146,8 +1147,13 @@ size_t fill_pathname_expand_special(char *s, const char *in_path, size_t len)
 
       if (*app_dir)
       {
+         /* strlcpy() reports the source's length whether or not it
+          * fit: a directory longer than @s leaves the result cut
+          * there, with nothing appended past the end. */
          size_t _len  = strlcpy(s, app_dir, len);
 
+         if (_len >= len)
+            return len ? len - 1 : 0;
          s           += _len;
          len         -= _len;
 
@@ -1155,15 +1161,21 @@ size_t fill_pathname_expand_special(char *s, const char *in_path, size_t len)
          {
             _len      = strlcpy(s, PATH_DEFAULT_SLASH(), len);
 
+            if (_len >= len)
+               return (size_t)(s - start) + (len ? len - 1 : 0);
             s        += _len;
             len      -= _len;
          }
 
-         in_path += 2;
+         in_path++;
+         if (PATH_CHAR_IS_SLASH(*in_path))
+            in_path++;
       }
    }
-#endif
+   return (size_t)(s - start) + strlcpy(s, in_path, len);
+#else
    return strlcpy(s, in_path, len);
+#endif
 }
 
 size_t fill_pathname_abbreviate_special(char *s,

libretro-common/samples/file/file_path/fill_pathname_test.c

@@ -259,6 +259,16 @@ static void test_join_truncation(void)
    free(region);
 }
 
+/* mingw-w64 has no setenv(). */
+static void set_home(const char *home)
+{
+#ifdef _WIN32
+   _putenv_s("HOME", home);
+#else
+   setenv("HOME", home, 1);
+#endif
+}
+
 /* fill_pathname_abbreviated_or_relative() resolves a relative @in_path
  * against @in_refpath's directory before choosing between the
  * relative and abbreviated forms.  That join still carries its '..'
@@ -273,7 +283,7 @@ static void test_abbreviated_or_relative_long_join(void)
    size_t _len     = 0;
    size_t i;
 
-   setenv("HOME", "/nonexistent-home", 1);
+   set_home("/nonexistent-home");
 
    _len = strlcpy(refpath, "/r/s/t/u/v/w/", sizeof(refpath));
    for (i = 0; i < 230; i++)
@@ -315,6 +325,80 @@ static void test_abbreviated_or_relative_long_join(void)
       printf("[SUCCESS] absolute reference preferred over deeper relative\n");
 }
 
+/* Equal but for which slash separates. */
+static int path_same(const char *a, const char *b)
+{
+   for (; *a && *b; a++, b++)
+      if (*a != *b && !(PATH_CHAR_IS_SLASH(*a) && PATH_CHAR_IS_SLASH(*b)))
+         return 0;
+   return *a == *b;
+}
+
+/* fill_pathname_expand_special() replaces '~' and the slash after it.
+ * The input is a heap copy, so ASan sees a read past its NUL. */
+static void test_expand_special_home(void)
+{
+   static const char *const in[]   =
+         { "~",   "~/",  "~/x",  "~x",   "", "a/~" };
+   static const char *const want[] =
+         { "/h/", "/h/", "/h/x", "/h/x", "", "a/~" };
+   char   out[64];
+   size_t i;
+
+   set_home("/h");
+   for (i = 0; i < sizeof(in) / sizeof(in[0]); i++)
+   {
+      size_t n   = strlen(in[i]) + 1;
+      char *path = (char*)malloc(n);
+      if (!path)
+      {
+         printf("[FAILED] expand \"%s\": out of memory\n", in[i]);
+         failures++;
+         continue;
+      }
+      memcpy(path, in[i], n);
+      fill_pathname_expand_special(out, path, sizeof(out));
+      free(path);
+      if (!path_same(out, want[i]))
+      {
+         printf("[FAILED] expand \"%s\" gave \"%s\" (expected \"%s\")\n",
+               in[i], out, want[i]);
+         failures++;
+      }
+      else
+         printf("[SUCCESS] expand \"%s\"\n", in[i]);
+   }
+}
+
+/* A home directory longer than the buffer: the expansion is cut at the
+ * buffer's end and nothing is written past it.  The buffer is a heap
+ * block of exactly its size, so ASan sees a write past the end. */
+static void test_expand_special_truncation(void)
+{
+   const size_t len = 8;
+   char  *out       = (char*)malloc(len);
+   size_t got;
+
+   if (!out)
+   {
+      printf("[FAILED] expand truncation: out of memory\n");
+      failures++;
+      return;
+   }
+   set_home("/a-home-directory-longer-than-the-buffer");
+   got = fill_pathname_expand_special(out, "~/file", len);
+   if (got != len - 1 || strlen(out) != len - 1
+         || strncmp(out, "/a-home", len - 1))
+   {
+      printf("[FAILED] expand truncation gave \"%s\" (%u)\n",
+            out, (unsigned)got);
+      failures++;
+   }
+   else
+      printf("[SUCCESS] expand truncation\n");
+   free(out);
+}
+
 int main(void)
 {
    /* Documented semantics. */
@@ -335,6 +419,8 @@ int main(void)
    test_overlong_input_no_dot();
    test_join_truncation();
    test_abbreviated_or_relative_long_join();
+   test_expand_special_home();
+   test_expand_special_truncation();
 
    if (failures)
    {

manual_content_scan.c

@@ -58,6 +58,20 @@ typedef struct
    bool validate_entries;
 } scan_settings_t;
 
+#define MANUAL_CONTENT_SCAN_SETTINGS_DEFAULT { \
+   MANUAL_CONTENT_SCAN_METHOD_AUTOMATIC,        /* scan method */ \
+   MANUAL_CONTENT_SCAN_SYSTEM_NAME_AUTO,        /* system_name_type */ \
+   MANUAL_CONTENT_SCAN_CORE_DETECT,             /* core_type */ \
+   MANUAL_CONTENT_SCAN_USE_DB_STRICT, \
+   MANUAL_CONTENT_SCAN_SELECT_DB_AUTO, \
+   true,                                        /* search_recursively */ \
+   true,                                        /* search_archives */ \
+   false, \
+   false,                                       /* filter_dat_content */ \
+   false,                                       /* overwrite_playlist */ \
+   false                                        /* validate_entries */ \
+}
+
 /* TODO/FIXME - static public global variables */
 /* Static settings object
  * > Provides easy access to settings parameters
@@ -68,19 +82,7 @@ typedef struct
  *   but this should not be an issue (i.e. regular configuration_settings
  *   are not thread safe, but we only access them when pushing a
  *   task, not in the task thread itself, so all is well) */
-static scan_settings_t scan_settings = {
-   MANUAL_CONTENT_SCAN_METHOD_AUTOMATIC,        /* scan method */
-   MANUAL_CONTENT_SCAN_SYSTEM_NAME_AUTO,        /* system_name_type */
-   MANUAL_CONTENT_SCAN_CORE_DETECT,             /* core_type */
-   MANUAL_CONTENT_SCAN_USE_DB_STRICT,
-   MANUAL_CONTENT_SCAN_SELECT_DB_AUTO,
-   true,                                        /* search_recursively */
-   true,                                        /* search_archives */
-   false,
-   false,                                       /* filter_dat_content */
-   false,                                       /* overwrite_playlist */
-   false                                        /* validate_entries */
-};
+static scan_settings_t scan_settings = MANUAL_CONTENT_SCAN_SETTINGS_DEFAULT;
 
 /* The path and name buffers live outside the struct: three
  * non-zero defaults were dragging eleven kilobytes of zero buffers
@@ -268,24 +270,30 @@ void manual_content_scan_scrub_file_exts_custom(void)
  * in, structs out - so both the extension whitelist
  * and the file system checks live here, with the scan
  * configuration that owns the path. */
-bool manual_content_scan_dat_path_is_valid(
-      const char *path, uint64_t *file_size)
+/* The extensions a Logiqx XML DAT may legitimately carry:
+ * .dat and .xml */
+static bool manual_content_scan_dat_ext_is_valid(const char *path)
 {
    const char *file_ext = NULL;
-   int64_t file_size_int;
 
    if (!path || !*path)
       return false;
 
-   /* Check file extension (.dat / .xml, the extensions
-    * a Logiqx XML DAT may legitimately carry) */
    file_ext = path_get_extension(path);
 
    if (!file_ext || !*file_ext)
       return false;
 
-   if (   !string_is_equal_noncase(file_ext, "dat")
-       && !string_is_equal_noncase(file_ext, "xml"))
+   return string_is_equal_noncase(file_ext, "dat")
+       || string_is_equal_noncase(file_ext, "xml");
+}
+
+bool manual_content_scan_dat_path_is_valid(
+      const char *path, uint64_t *file_size)
+{
+   int64_t file_size_int;
+
+   if (!manual_content_scan_dat_ext_is_valid(path))
       return false;
 
    /* Ensure file exists */
@@ -305,9 +313,11 @@ bool manual_content_scan_dat_path_is_valid(
 }
 
 /* Checks 'dat_file_path' string and resets it
- * if invalid */
-enum manual_content_scan_dat_file_path_status
-      manual_content_scan_validate_dat_file_path(void)
+ * if invalid.  @known_size, when not NULL, is the file's
+ * size as read already (<= 0: not there), so the file
+ * system is not asked again. */
+static enum manual_content_scan_dat_file_path_status
+      manual_content_scan_validate_dat(const int64_t *known_size)
 {
    enum manual_content_scan_dat_file_path_status dat_file_path_status =
          MANUAL_CONTENT_SCAN_DAT_FILE_UNSET;
@@ -315,10 +325,22 @@ enum manual_content_scan_dat_file_path_status
    /* Check if 'dat_file_path' has been set */
    if (*scan_dat_file_path)
    {
-      uint64_t file_size;
+      uint64_t file_size = 0;
+      bool valid;
 
       /* Check if path itself is valid */
-      if (manual_content_scan_dat_path_is_valid(scan_dat_file_path, &file_size))
+      if (known_size)
+      {
+         valid = manual_content_scan_dat_ext_is_valid(scan_dat_file_path)
+              && *known_size > 0;
+         if (valid)
+            file_size = (uint64_t)*known_size;
+      }
+      else
+         valid = manual_content_scan_dat_path_is_valid(scan_dat_file_path,
+               &file_size);
+
+      if (valid)
       {
          uint64_t free_memory = mem_stats_free();
          dat_file_path_status = MANUAL_CONTENT_SCAN_DAT_FILE_OK;
@@ -359,12 +381,91 @@ enum manual_content_scan_dat_file_path_status
    return dat_file_path_status;
 }
 
+enum manual_content_scan_dat_file_path_status
+      manual_content_scan_validate_dat_file_path(void)
+{
+   return manual_content_scan_validate_dat(NULL);
+}
+
 /* Menu setters */
 
+/* Sets every setting back to its default */
+void manual_content_scan_reset_settings(void)
+{
+   static const scan_settings_t d = MANUAL_CONTENT_SCAN_SETTINGS_DEFAULT;
+
+   scan_settings                   = d;
+   scan_core_path[0]               = '\0';
+   scan_file_exts_core[0]          = '\0';
+   scan_file_exts_custom[0]        = '\0';
+   scan_dat_file_path[0]           = '\0';
+   scan_content_dir[0]             = '\0';
+   scan_system_name_content_dir[0] = '\0';
+   scan_system_name_database[0]    = '\0';
+   scan_system_name_custom[0]      = '\0';
+   scan_core_name[0]               = '\0';
+}
+
+struct manual_content_scan_saved
+{
+   scan_settings_t settings;
+   char core_path[PATH_MAX_LENGTH];
+   char file_exts_core[PATH_MAX_LENGTH];
+   char file_exts_custom[PATH_MAX_LENGTH];
+   char dat_file_path[PATH_MAX_LENGTH];
+   char content_dir[PATH_MAX_LENGTH];
+   char system_name_content_dir[DIR_MAX_LENGTH];
+   char system_name_database[NAME_MAX_LENGTH];
+   char system_name_custom[NAME_MAX_LENGTH];
+   char core_name[NAME_MAX_LENGTH];
+};
+
+manual_content_scan_saved_t *manual_content_scan_save_settings(void)
+{
+   manual_content_scan_saved_t *sv = (manual_content_scan_saved_t*)
+      malloc(sizeof(*sv));
+   if (!sv)
+      return NULL;
+   sv->settings = scan_settings;
+   memcpy(sv->core_path,               scan_core_path,               sizeof(sv->core_path));
+   memcpy(sv->file_exts_core,          scan_file_exts_core,          sizeof(sv->file_exts_core));
+   memcpy(sv->file_exts_custom,        scan_file_exts_custom,        sizeof(sv->file_exts_custom));
+   memcpy(sv->dat_file_path,           scan_dat_file_path,           sizeof(sv->dat_file_path));
+   memcpy(sv->content_dir,             scan_content_dir,             sizeof(sv->content_dir));
+   memcpy(sv->system_name_content_dir, scan_system_name_content_dir, sizeof(sv->system_name_content_dir));
+   memcpy(sv->system_name_database,    scan_system_name_database,    sizeof(sv->system_name_database));
+   memcpy(sv->system_name_custom,      scan_system_name_custom,      sizeof(sv->system_name_custom));
+   memcpy(sv->core_name,               scan_core_name,               sizeof(sv->core_name));
+   return sv;
+}
+
+void manual_content_scan_restore_settings(manual_content_scan_saved_t *sv)
+{
+   if (!sv)
+      return;
+   scan_settings = sv->settings;
+   memcpy(scan_core_path,               sv->core_path,               sizeof(scan_core_path));
+   memcpy(scan_file_exts_core,          sv->file_exts_core,          sizeof(scan_file_exts_core));
+   memcpy(scan_file_exts_custom,        sv->file_exts_custom,        sizeof(scan_file_exts_custom));
+   memcpy(scan_dat_file_path,           sv->dat_file_path,           sizeof(scan_dat_file_path));
+   memcpy(scan_content_dir,             sv->content_dir,             sizeof(scan_content_dir));
+   memcpy(scan_system_name_content_dir, sv->system_name_content_dir, sizeof(scan_system_name_content_dir));
+   memcpy(scan_system_name_database,    sv->system_name_database,    sizeof(scan_system_name_database));
+   memcpy(scan_system_name_custom,      sv->system_name_custom,      sizeof(scan_system_name_custom));
+   memcpy(scan_core_name,               sv->core_name,               sizeof(scan_core_name));
+   free(sv);
+}
+
 /* Sets content directory for next manual scan
  * operation.
  * Returns true if content directory is valid. */
 bool manual_content_scan_set_menu_content_dir(const char *content_dir)
+{
+   return manual_content_scan_set_menu_content_dir_fs(content_dir, NULL);
+}
+
+bool manual_content_scan_set_menu_content_dir_fs(const char *content_dir,
+      const manual_content_scan_fs_t *fs)
 {
    size_t _len;
    int path_stats;
@@ -426,7 +527,7 @@ bool manual_content_scan_set_menu_content_dir(const char *content_dir)
 
    /* A file takes the name of the folder that holds it,
     * as a scan of that folder would */
-   path_stats = path_stat(scan_content_dir);
+   path_stats = fs ? fs->content_stat : path_stat(scan_content_dir);
    if (      (path_stats & RETRO_VFS_STAT_IS_VALID)
          && !(path_stats & RETRO_VFS_STAT_IS_DIRECTORY)
          &&  fill_pathname_parent_dir_name(scan_system_name_content_dir,
@@ -555,6 +656,53 @@ error:
    return false;
 }
 
+/* Copies the core's path, name and extensions
+ * to the settings.
+ * Returns false if the core has no path. */
+static bool manual_content_scan_set_core(const core_info_t *core_info)
+{
+   /* Copy core path to settings struct */
+   if (!core_info->path || !*core_info->path)
+      return false;
+
+   strlcpy(
+         scan_core_path,
+         core_info->path,
+         sizeof(scan_core_path));
+
+   /* Copy core name to settings struct */
+   strlcpy(
+         scan_core_name,
+         core_info->display_name,
+         sizeof(scan_core_name));
+
+   /* Copy supported extensions to settings
+    * struct, if required */
+   if (    core_info->supported_extensions
+       && *core_info->supported_extensions)
+   {
+      strlcpy(
+            scan_file_exts_core,
+            core_info->supported_extensions,
+            sizeof(scan_file_exts_core));
+
+      /* Core info extensions are delimited by
+       * vertical bars. For internal consistency,
+       * replace them with spaces */
+      string_replace_all_chars(scan_file_exts_core, '|', ' ');
+
+      /* Apply standard scrubbing/clean-up
+       * (should not be required, but must handle the
+       * case where a core info file is incorrectly
+       * formatted) */
+      manual_content_scan_scrub_file_exts(scan_file_exts_core);
+   }
+   else
+      scan_file_exts_core[0] = '\0';
+
+   return true;
+}
+
 /* Sets core name for the next manual scan
  * operation (+ core path and other associated
  * parameters).
@@ -607,45 +755,9 @@ bool manual_content_scan_set_menu_core_name(
                /* Core has been found */
                core_found = true;
 
-               /* Copy core path to settings struct */
-               if (!core_info->path || !*core_info->path)
+               if (!manual_content_scan_set_core(core_info))
                   goto error;
 
-               strlcpy(
-                     scan_core_path,
-                     core_info->path,
-                     sizeof(scan_core_path));
-
-               /* Copy core name to settings struct */
-               strlcpy(
-                     scan_core_name,
-                     core_info->display_name,
-                     sizeof(scan_core_name));
-
-               /* Copy supported extensions to settings
-                * struct, if required */
-               if (    core_info->supported_extensions 
-                   && *core_info->supported_extensions)
-               {
-                  strlcpy(
-                        scan_file_exts_core,
-                        core_info->supported_extensions,
-                        sizeof(scan_file_exts_core));
-
-                  /* Core info extensions are delimited by
-                   * vertical bars. For internal consistency,
-                   * replace them with spaces */
-                  string_replace_all_chars(scan_file_exts_core, '|', ' ');
-
-                  /* Apply standard scrubbing/clean-up
-                   * (should not be required, but must handle the
-                   * case where a core info file is incorrectly
-                   * formatted) */
-                  manual_content_scan_scrub_file_exts(scan_file_exts_core);
-               }
-               else
-                  scan_file_exts_core[0] = '\0';
-
                break;
             }
          }
@@ -674,8 +786,17 @@ error:
  * Returns MANUAL_CONTENT_SCAN_PLAYLIST_REFRESH_OK
  * if playlist contains a valid scan record. */
 enum manual_content_scan_playlist_refresh_status
-      manual_content_scan_set_menu_from_playlist(playlist_t *playlist,
-            const char *path_content_database, bool show_hidden_files)
+      manual_content_scan_set_menu_from_playlist_list(playlist_t *playlist,
+            const struct string_list *rdb_list)
+{
+   return manual_content_scan_set_menu_from_playlist_list_fs(playlist,
+         rdb_list, NULL);
+}
+
+enum manual_content_scan_playlist_refresh_status
+      manual_content_scan_set_menu_from_playlist_list_fs(
+            playlist_t *playlist, const struct string_list *rdb_list,
+            const manual_content_scan_fs_t *fs)
 {
    const char *playlist_path    = NULL;
    const char *content_dir      = NULL;
@@ -689,9 +810,6 @@ enum manual_content_scan_playlist_refresh_status
    bool overwrite_playlist      = false;
    bool omit_db_ref             = false;
    int db_usage                 = MANUAL_CONTENT_SCAN_USE_DB_NONE;
-#ifdef HAVE_LIBRETRODB
-   struct string_list *rdb_list = NULL;
-#endif
    enum manual_content_scan_system_name_type
          system_name_type       = MANUAL_CONTENT_SCAN_SYSTEM_NAME_CONTENT_DIR;
    enum manual_content_scan_core_type
@@ -729,13 +847,15 @@ enum manual_content_scan_playlist_refresh_status
 
    if (database_name_lpl && *database_name_lpl)
       fill_pathname(database_name,database_name_lpl,"",sizeof(database_name));
+   else
+      database_name[0] = '\0';
    /* Cannot happen, but would constitute a
     * 'system name' error */
    if (!*system_name)
       return MANUAL_CONTENT_SCAN_PLAYLIST_REFRESH_INVALID_SYSTEM_NAME;
 
    /* Set content directory */
-   if (!manual_content_scan_set_menu_content_dir(content_dir))
+   if (!manual_content_scan_set_menu_content_dir_fs(content_dir, fs))
       return MANUAL_CONTENT_SCAN_PLAYLIST_REFRESH_INVALID_CONTENT_DIR;
 
    if (!manual_content_scan_set_menu_scan_use_db((enum manual_content_scan_db_usage)db_usage))
@@ -744,13 +864,6 @@ enum manual_content_scan_playlist_refresh_status
    scan_settings.db_selection = MANUAL_CONTENT_SCAN_SELECT_DB_AUTO;
 
    /* Set system name */
-#ifdef HAVE_LIBRETRODB
-   /* > If platform has database support, get names
-    *   of all installed database files */
-   rdb_list = dir_list_new_special(
-         path_content_database,
-         DIR_LIST_DATABASES, NULL, show_hidden_files);
-
    if (rdb_list && rdb_list->size)
    {
       size_t i;
@@ -786,9 +899,6 @@ enum manual_content_scan_playlist_refresh_status
       }
    }
 
-   string_list_free(rdb_list);
-#endif
-
    /* > If system name does not match a database
     *   file, then check whether it matches the
     *   content directory name */
@@ -853,7 +963,7 @@ enum manual_content_scan_playlist_refresh_status
       fill_pathname_expand_special(scan_dat_file_path, dat_file_path,
             sizeof(scan_dat_file_path));
 
-      switch (manual_content_scan_validate_dat_file_path())
+      switch (manual_content_scan_validate_dat(fs ? &fs->dat_size : NULL))
       {
          case MANUAL_CONTENT_SCAN_DAT_FILE_INVALID:
             return MANUAL_CONTENT_SCAN_PLAYLIST_REFRESH_INVALID_DAT_FILE;
@@ -880,6 +990,466 @@ enum manual_content_scan_playlist_refresh_status
    return MANUAL_CONTENT_SCAN_PLAYLIST_REFRESH_OK;
 }
 
+enum manual_content_scan_playlist_refresh_status
+      manual_content_scan_set_menu_from_playlist(playlist_t *playlist,
+            const char *path_content_database, bool show_hidden_files)
+{
+   enum manual_content_scan_playlist_refresh_status status;
+   struct string_list *rdb_list = NULL;
+#ifdef HAVE_LIBRETRODB
+   /* > If platform has database support, get names
+    *   of all installed database files */
+   rdb_list = dir_list_new_special(
+         path_content_database,
+         DIR_LIST_DATABASES, NULL, show_hidden_files);
+#endif
+   status   = manual_content_scan_set_menu_from_playlist_list(playlist,
+         rdb_list);
+   string_list_free(rdb_list);
+   return status;
+}
+
+/* Whether @name is an installed database: the basename,
+ * without extension, of a path in @rdb_list */
+static bool manual_content_scan_rdb_list_has(
+      const struct string_list *rdb_list, const char *name)
+{
+   size_t i;
+
+   if (!rdb_list)
+      return false;
+
+   for (i = 0; i < rdb_list->size; i++)
+   {
+      char rdb_name[NAME_MAX_LENGTH];
+      const char *rdb_path = rdb_list->elems[i].data;
+      if (!rdb_path || !*rdb_path)
+         continue;
+      fill_pathname(rdb_name, path_basename(rdb_path), "",
+            sizeof(rdb_name));
+      if (string_is_equal(name, rdb_name))
+         return true;
+   }
+
+   return false;
+}
+
+enum manual_content_scan_core_key
+{
+   MANUAL_CONTENT_SCAN_CORE_KEY_PATH = 0,
+   MANUAL_CONTENT_SCAN_CORE_KEY_FILE_ID_EXACT,
+   MANUAL_CONTENT_SCAN_CORE_KEY_FILE_ID,
+   MANUAL_CONTENT_SCAN_CORE_KEY_DISPLAY_NAME
+};
+
+/* Counts the cores whose @key is @s; *found gets the first */
+static unsigned manual_content_scan_match_cores(
+      core_info_list_t *core_info_list,
+      enum manual_content_scan_core_key key, const char *s,
+      const core_info_t **found)
+{
+   size_t i;
+   unsigned count = 0;
+#ifdef _WIN32
+   /* Either slash: core_info joins its folder and file with '/' */
+   char *core_path = NULL;
+
+   if (   key == MANUAL_CONTENT_SCAN_CORE_KEY_PATH
+       && !(core_path = (char*)malloc(PATH_MAX_LENGTH)))
+      return 0;
+#endif
+
+   for (i = 0; i < core_info_list->count; i++)
+   {
+      const core_info_t *core_info = core_info_get(core_info_list, i);
+      bool match                   = false;
+
+      if (!core_info)
+         continue;
+
+      switch (key)
+      {
+         case MANUAL_CONTENT_SCAN_CORE_KEY_PATH:
+#ifdef _WIN32
+            if (core_info->path)
+            {
+               strlcpy(core_path, core_info->path, PATH_MAX_LENGTH);
+               pathname_conform_slashes_to_os(core_path);
+               match = string_is_equal_case_insensitive(core_path, s);
+            }
+#else
+            match = string_is_equal(core_info->path, s);
+#endif
+            break;
+         case MANUAL_CONTENT_SCAN_CORE_KEY_FILE_ID_EXACT:
+            match = string_is_equal(core_info->core_file_id.str, s);
+            break;
+         case MANUAL_CONTENT_SCAN_CORE_KEY_FILE_ID:
+            match = string_is_equal_case_insensitive(
+                  core_info->core_file_id.str, s);
+            break;
+         case MANUAL_CONTENT_SCAN_CORE_KEY_DISPLAY_NAME:
+            match = string_is_equal(core_info->display_name, s);
+            break;
+      }
+
+      if (match && !count++)
+         *found = core_info;
+   }
+
+#ifdef _WIN32
+   free(core_path);
+#endif
+   return count;
+}
+
+/* Whether @core is written as a path: display names hold
+ * slashes too, so a path that matches no core falls through
+ * to the names */
+static bool manual_content_scan_core_is_path(const char *core)
+{
+   return strchr(core, '/')
+#ifdef _WIN32
+       || strchr(core, '\\')
+#endif
+       || *core == '~';
+}
+
+/* @core, a path, as the core list holds paths: expanded, and
+ * "//" and ".." folded in an absolute path too; "" when it
+ * resolves to nothing.  Asks the file system. */
+static void manual_content_scan_core_path_resolve(char *s,
+      const char *core, size_t len)
+{
+   fill_pathname_expand_special(s, core, len);
+#ifndef RARCH_CONSOLE
+   if (   *s
+       && !path_resolve_realpath(s, len, false)
+       && !path_is_absolute(s))
+      *s = '\0';
+#endif
+#ifdef _WIN32
+   pathname_conform_slashes_to_os(s);
+#endif
+}
+
+/* An override DAT path as the scan opens it: expanded, and a
+ * relative one resolved; "" when it resolves to nothing.
+ * Asks the file system. */
+static void manual_content_scan_dat_path_resolve(char *s,
+      const char *dat, size_t len)
+{
+   fill_pathname_expand_special(s, dat, len);
+#ifndef RARCH_CONSOLE
+   if (   *s && !path_is_absolute(s)
+       && !path_resolve_realpath(s, len, false))
+      *s = '\0';
+#endif
+}
+
+/* Finds the core @core names: a path, else a file name or
+ * file id, else a display name. The first of these that
+ * matches decides, and *count gets its matches.
+ * > @resolved: @core resolved as a path already, or NULL to
+ *   resolve it here */
+static const core_info_t *manual_content_scan_find_core(
+      const char *core, const char *resolved, unsigned *count)
+{
+   core_info_list_t *core_info_list = NULL;
+   const core_info_t *found         = NULL;
+   char *s                          = NULL;
+   char *lower                      = NULL;
+
+   *count = 0;
+   core_info_get_list(&core_info_list);
+
+   if (!core_info_list || !(s = (char*)malloc(PATH_MAX_LENGTH)))
+      return NULL;
+
+   if (manual_content_scan_core_is_path(core))
+   {
+      if (resolved)
+         strlcpy(s, resolved, PATH_MAX_LENGTH);
+      else
+         manual_content_scan_core_path_resolve(s, core, PATH_MAX_LENGTH);
+      if (*s)
+         *count = manual_content_scan_match_cores(core_info_list,
+               MANUAL_CONTENT_SCAN_CORE_KEY_PATH, s, &found);
+   }
+
+   /* Lower case, as core_info finds "_libretro" in lower case only */
+   if (!*count && (lower = strdup(core)))
+   {
+      size_t i, _len;
+      string_to_lower(lower);
+      /* core_info's rule would cut a bare id's last "_" part */
+      if (strstr(lower, "_libretro"))
+         _len = core_info_get_file_id(lower, s, PATH_MAX_LENGTH);
+      else
+      {
+         _len = fill_pathname(s, lower, "", PATH_MAX_LENGTH);
+         strlcpy(s + _len, "_libretro", PATH_MAX_LENGTH - _len);
+      }
+      /* The case typed, for an exact match first */
+      for (i = 0; i < _len; i++)
+         if (s[i] == lower[i])
+            s[i] = core[i];
+      free(lower);
+      *count = manual_content_scan_match_cores(core_info_list,
+            MANUAL_CONTENT_SCAN_CORE_KEY_FILE_ID_EXACT, s, &found);
+      if (!*count)
+         *count = manual_content_scan_match_cores(core_info_list,
+               MANUAL_CONTENT_SCAN_CORE_KEY_FILE_ID, s, &found);
+   }
+
+   if (!*count)
+      *count = manual_content_scan_match_cores(core_info_list,
+            MANUAL_CONTENT_SCAN_CORE_KEY_DISPLAY_NAME, core, &found);
+
+   free(s);
+   return found;
+}
+
+manual_content_scan_overrides_t *manual_content_scan_overrides_new(void)
+{
+   return (manual_content_scan_overrides_t*)calloc(1,
+         sizeof(manual_content_scan_overrides_t));
+}
+
+void manual_content_scan_overrides_free(manual_content_scan_overrides_t *ov)
+{
+   if (!ov)
+      return;
+   free(ov->playlist);
+   free(ov->database);
+   free(ov->dat);
+   free(ov->core);
+   free(ov->exts);
+   free(ov);
+}
+
+manual_content_scan_fs_t *manual_content_scan_fs_read(
+      const char *content_dir, const char *dat,
+      const char *ov_core, const char *ov_dat)
+{
+   manual_content_scan_fs_t *fs = (manual_content_scan_fs_t*)calloc(1,
+         sizeof(*fs));
+   char *s                      = (char*)malloc(PATH_MAX_LENGTH);
+
+   if (!fs || !s)
+      goto error;
+
+   /* The folder as manual_content_scan_set_menu_content_dir()
+    * stats it: expanded, one trailing slash gone */
+   if (content_dir && *content_dir)
+   {
+      size_t _len = fill_pathname_expand_special(s, content_dir,
+            PATH_MAX_LENGTH);
+      if (_len > 0 && _len < PATH_MAX_LENGTH
+            && s[_len - 1] == PATH_DEFAULT_SLASH_C())
+         s[_len - 1] = '\0';
+      if (*s)
+         fs->content_stat = path_stat(s);
+   }
+
+   /* The saved DAT, as a refresh expands it */
+   fs->dat_size = -1;
+   if (dat && *dat)
+   {
+      fill_pathname_expand_special(s, dat, PATH_MAX_LENGTH);
+      if (*s && path_is_valid(s))
+         fs->dat_size = path_get_size(s);
+   }
+
+   fs->ov_dat_size = -1;
+   if (ov_dat)
+   {
+      manual_content_scan_dat_path_resolve(s, ov_dat, PATH_MAX_LENGTH);
+      if (!(fs->ov_dat = strdup(s)))
+         goto error;
+      if (*s && path_is_valid(s))
+         fs->ov_dat_size = path_get_size(s);
+   }
+
+   if (ov_core && manual_content_scan_core_is_path(ov_core))
+   {
+      manual_content_scan_core_path_resolve(s, ov_core, PATH_MAX_LENGTH);
+      if (!(fs->ov_core = strdup(s)))
+         goto error;
+   }
+
+   free(s);
+   return fs;
+
+error:
+   free(s);
+   manual_content_scan_fs_free(fs);
+   return NULL;
+}
+
+void manual_content_scan_fs_free(manual_content_scan_fs_t *fs)
+{
+   if (!fs)
+      return;
+   free(fs->ov_dat);
+   free(fs->ov_core);
+   free(fs);
+}
+
+/* Lays @ov over the current settings and sets the Custom
+ * method, stopping at the first override that does not
+ * resolve; NULL @ov changes nothing.
+ * > @rdb_list: paths of the installed databases, or NULL
+ * > @count: the cores @ov->core matched, when not NULL */
+enum manual_content_scan_override_status
+      manual_content_scan_apply_overrides(
+            const manual_content_scan_overrides_t *ov,
+            const struct string_list *rdb_list, unsigned *count)
+{
+   return manual_content_scan_apply_overrides_fs(ov, rdb_list, count,
+         NULL);
+}
+
+enum manual_content_scan_override_status
+      manual_content_scan_apply_overrides_fs(
+            const manual_content_scan_overrides_t *ov,
+            const struct string_list *rdb_list, unsigned *count,
+            const manual_content_scan_fs_t *fs)
+{
+   if (count)
+      *count = 0;
+
+   if (!ov)
+      return MANUAL_CONTENT_SCAN_OVR_OK;
+
+   if (ov->core)
+   {
+      unsigned matches             = 0;
+      const core_info_t *core_info = manual_content_scan_find_core(
+            ov->core, fs ? fs->ov_core : NULL, &matches);
+
+      if (count)
+         *count = matches;
+      if (matches > 1)
+         return MANUAL_CONTENT_SCAN_OVR_AMBIGUOUS_CORE;
+      if (!core_info || !manual_content_scan_set_core(core_info))
+         return MANUAL_CONTENT_SCAN_OVR_NO_SUCH_CORE;
+      scan_settings.core_type = MANUAL_CONTENT_SCAN_CORE_SET;
+   }
+
+   if (ov->database)
+   {
+      if (!manual_content_scan_rdb_list_has(rdb_list, ov->database))
+         return MANUAL_CONTENT_SCAN_OVR_NO_SUCH_DATABASE;
+      /* A refreshed database playlist's name shares this
+       * buffer: it keeps that name as a custom one */
+      if (   scan_settings.system_name_type
+               == MANUAL_CONTENT_SCAN_SYSTEM_NAME_DATABASE
+          && !string_is_equal(scan_system_name_database, ov->database))
+      {
+         strlcpy(scan_system_name_custom, scan_system_name_database,
+               sizeof(scan_system_name_custom));
+         manual_content_scan_set_menu_system_name(
+               MANUAL_CONTENT_SCAN_SYSTEM_NAME_CUSTOM, NULL);
+      }
+      manual_content_scan_set_menu_scan_db_select(
+            MANUAL_CONTENT_SCAN_SELECT_DB_SPECIFIC, ov->database);
+   }
+   else if (ov->flags & MANUAL_CONTENT_SCAN_OVR_FIRST_DB)
+      manual_content_scan_set_menu_scan_db_select(
+            MANUAL_CONTENT_SCAN_SELECT_DB_AUTO_FIRST_MATCH, NULL);
+
+   /* A database playlist's name and the database to match
+    * share one buffer: next to another database, the
+    * playlist is custom */
+   if (ov->playlist)
+   {
+      if (   manual_content_scan_rdb_list_has(rdb_list, ov->playlist)
+          && (!ov->database || string_is_equal(ov->database, ov->playlist)))
+         manual_content_scan_set_menu_system_name(
+               MANUAL_CONTENT_SCAN_SYSTEM_NAME_DATABASE, ov->playlist);
+      else
+      {
+         strlcpy(scan_system_name_custom, ov->playlist,
+               sizeof(scan_system_name_custom));
+         manual_content_scan_scrub_system_name_custom();
+         manual_content_scan_set_menu_system_name(
+               MANUAL_CONTENT_SCAN_SYSTEM_NAME_CUSTOM, NULL);
+      }
+   }
+
+   if (ov->dat)
+   {
+      if (   (ov->flags & MANUAL_CONTENT_SCAN_OVR_MATCH)
+          && ov->match == MANUAL_CONTENT_SCAN_USE_DB_NONE)
+         return MANUAL_CONTENT_SCAN_OVR_DAT_WITHOUT_MATCH;
+
+      if (fs && fs->ov_dat)
+         strlcpy(scan_dat_file_path, fs->ov_dat,
+               sizeof(scan_dat_file_path));
+      else
+         manual_content_scan_dat_path_resolve(scan_dat_file_path, ov->dat,
+               sizeof(scan_dat_file_path));
+      switch (manual_content_scan_validate_dat(
+            (fs && fs->ov_dat) ? &fs->ov_dat_size : NULL))
+      {
+         case MANUAL_CONTENT_SCAN_DAT_FILE_OK:
+            break;
+         case MANUAL_CONTENT_SCAN_DAT_FILE_TOO_LARGE:
+            return MANUAL_CONTENT_SCAN_OVR_DAT_TOO_LARGE;
+         default:
+            return MANUAL_CONTENT_SCAN_OVR_NOT_A_DAT;
+      }
+
+      manual_content_scan_set_menu_scan_use_db(
+               (   (ov->flags & MANUAL_CONTENT_SCAN_OVR_MATCH)
+                && ov->match == MANUAL_CONTENT_SCAN_USE_DB_LOOSE)
+            ? MANUAL_CONTENT_SCAN_USE_DB_DAT_LOOSE
+            : MANUAL_CONTENT_SCAN_USE_DB_DAT_STRICT);
+   }
+   else if (ov->flags & MANUAL_CONTENT_SCAN_OVR_MATCH)
+   {
+      enum manual_content_scan_db_usage usage = ov->match;
+
+      /* A refresh keeps its saved DAT unless the match is none */
+      if (   scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_DAT_STRICT
+          || scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_DAT_LOOSE)
+      {
+         if (ov->match == MANUAL_CONTENT_SCAN_USE_DB_STRICT)
+            usage = MANUAL_CONTENT_SCAN_USE_DB_DAT_STRICT;
+         else if (ov->match == MANUAL_CONTENT_SCAN_USE_DB_LOOSE)
+            usage = MANUAL_CONTENT_SCAN_USE_DB_DAT_LOOSE;
+         else
+            scan_dat_file_path[0] = '\0';
+      }
+
+      manual_content_scan_set_menu_scan_use_db(usage);
+   }
+
+   if (ov->exts)
+   {
+      strlcpy(scan_file_exts_custom, ov->exts,
+            sizeof(scan_file_exts_custom));
+      manual_content_scan_scrub_file_exts_custom();
+   }
+
+   if (ov->flags & MANUAL_CONTENT_SCAN_OVR_NO_RECURSIVE)
+      scan_settings.search_recursively = false;
+   if (ov->flags & MANUAL_CONTENT_SCAN_OVR_NO_ARCHIVES)
+      scan_settings.search_archives    = false;
+   if (ov->flags & MANUAL_CONTENT_SCAN_OVR_OVERWRITE)
+      scan_settings.overwrite_playlist = true;
+   if (ov->flags & MANUAL_CONTENT_SCAN_OVR_VALIDATE)
+      scan_settings.validate_entries   = true;
+   if (ov->flags & MANUAL_CONTENT_SCAN_OVR_OMIT_DB_REF)
+      scan_settings.omit_db_ref        = true;
+
+   manual_content_scan_set_menu_scan_method(
+         MANUAL_CONTENT_SCAN_METHOD_CUSTOM);
+
+   return MANUAL_CONTENT_SCAN_OVR_OK;
+}
+
 /* Menu getters */
 
 /* Fetches content directory for next manual scan
@@ -1340,6 +1910,32 @@ error:
 /* Task Helpers */
 /****************/
 
+/* The system name type a scan of the current settings takes: an
+ * automatic name is filled when the settings predict it. */
+static enum manual_content_scan_system_name_type
+      manual_content_scan_task_name_type(void)
+{
+   if (scan_settings.system_name_type == MANUAL_CONTENT_SCAN_SYSTEM_NAME_AUTO)
+   {
+      if (scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_DAT_STRICT ||
+          scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_DAT_LOOSE ||
+          scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_NONE)
+         return MANUAL_CONTENT_SCAN_SYSTEM_NAME_CONTENT_DIR;
+      if (scan_settings.db_selection == MANUAL_CONTENT_SCAN_SELECT_DB_SPECIFIC)
+         return MANUAL_CONTENT_SCAN_SYSTEM_NAME_DATABASE;
+   }
+   return scan_settings.system_name_type;
+}
+
+/* True if a scan of the current settings writes one playlist
+ * (target_is_single_determined_playlist). */
+bool manual_content_scan_is_single_playlist(void)
+{
+   return    scan_settings.scan_method != MANUAL_CONTENT_SCAN_METHOD_AUTOMATIC
+          && manual_content_scan_task_name_type()
+                != MANUAL_CONTENT_SCAN_SYSTEM_NAME_AUTO;
+}
+
 /* Parses current manual content scan settings,
  * and extracts all information required to configure
  * a manual content scan task.
@@ -1386,15 +1982,7 @@ bool manual_content_scan_get_task_config(
          sizeof(task_config->content_dir));
 
    /* If name is set as automatic, but is predictable, fill it now. */
-   if (scan_settings.system_name_type == MANUAL_CONTENT_SCAN_SYSTEM_NAME_AUTO)
-   {
-      if (scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_DAT_STRICT ||
-          scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_DAT_LOOSE ||
-          scan_settings.db_usage == MANUAL_CONTENT_SCAN_USE_DB_NONE)
-         scan_settings.system_name_type = MANUAL_CONTENT_SCAN_SYSTEM_NAME_CONTENT_DIR;
-      else if (scan_settings.db_selection == MANUAL_CONTENT_SCAN_SELECT_DB_SPECIFIC)
-         scan_settings.system_name_type = MANUAL_CONTENT_SCAN_SYSTEM_NAME_DATABASE;
-   }
+   scan_settings.system_name_type = manual_content_scan_task_name_type();
    /* Get target playlist ("system name")
     *
     * The system name is not a leaf value: the scan pipeline appends a

manual_content_scan.h

@@ -104,6 +104,43 @@ enum manual_content_scan_playlist_refresh_status
    MANUAL_CONTENT_SCAN_PLAYLIST_REFRESH_DAT_FILE_TOO_LARGE
 };
 
+/* Defines all possible return values for
+ * manual_content_scan_apply_overrides() */
+enum manual_content_scan_override_status
+{
+   MANUAL_CONTENT_SCAN_OVR_OK = 0,
+   MANUAL_CONTENT_SCAN_OVR_NO_SUCH_CORE,
+   MANUAL_CONTENT_SCAN_OVR_AMBIGUOUS_CORE,
+   MANUAL_CONTENT_SCAN_OVR_NO_SUCH_DATABASE,
+   MANUAL_CONTENT_SCAN_OVR_NOT_A_DAT,
+   MANUAL_CONTENT_SCAN_OVR_DAT_TOO_LARGE,
+   MANUAL_CONTENT_SCAN_OVR_DAT_WITHOUT_MATCH
+};
+
+enum manual_content_scan_override_flags
+{
+   MANUAL_CONTENT_SCAN_OVR_MATCH        = (1 << 0), /* 'match' is set */
+   MANUAL_CONTENT_SCAN_OVR_FIRST_DB     = (1 << 1),
+   MANUAL_CONTENT_SCAN_OVR_NO_RECURSIVE = (1 << 2),
+   MANUAL_CONTENT_SCAN_OVR_NO_ARCHIVES  = (1 << 3),
+   MANUAL_CONTENT_SCAN_OVR_OVERWRITE    = (1 << 4),
+   MANUAL_CONTENT_SCAN_OVR_VALIDATE     = (1 << 5),
+   MANUAL_CONTENT_SCAN_OVR_OMIT_DB_REF  = (1 << 6)
+};
+
+/* Settings laid over a scan's own; a NULL string or a clear
+ * flag leaves its setting as it is */
+typedef struct manual_content_scan_overrides
+{
+   char *playlist;  /* --scan-playlist */
+   char *database;  /* --scan-database=NAME */
+   char *dat;       /* --scan-dat, as typed */
+   char *core;      /* --scan-core, as typed */
+   char *exts;      /* --scan-exts, space separated */
+   enum manual_content_scan_db_usage match; /* STRICT, LOOSE or NONE */
+   uint16_t flags;  /* MANUAL_CONTENT_SCAN_OVR_* */
+} manual_content_scan_overrides_t;
+
 /* Holds all configuration parameters required
  * for a manual content scan task */
 typedef struct
@@ -223,11 +260,47 @@ enum manual_content_scan_dat_file_path_status
 
 /* Menu setters */
 
+/* Sets every setting back to its default */
+void manual_content_scan_reset_settings(void);
+
+/* The Manual Scan settings as they are, for a scan chain to put back
+ * once it has run its own: NULL out of memory. Restoring frees it. */
+typedef struct manual_content_scan_saved manual_content_scan_saved_t;
+manual_content_scan_saved_t *manual_content_scan_save_settings(void);
+void manual_content_scan_restore_settings(manual_content_scan_saved_t *saved);
+
 /* Sets content directory for next manual scan
  * operation.
  * Returns true if content directory is valid. */
 bool manual_content_scan_set_menu_content_dir(const char *content_dir);
 
+/* What the file system says of one scan's paths, read off the main
+ * thread by manual_content_scan_fs_read(), so that the _fs setters
+ * below ask it nothing.  Only the paths it was read for: it holds no
+ * reference to the settings. */
+typedef struct manual_content_scan_fs
+{
+   char *ov_dat;         /* @ov_dat expanded and resolved, or NULL */
+   char *ov_core;        /* @ov_core resolved when it is a path, or NULL */
+   int64_t dat_size;     /* the saved DAT's size; <= 0: not there */
+   int64_t ov_dat_size;  /* likewise, @ov_dat's */
+   int content_stat;     /* path_stat() of the content folder */
+} manual_content_scan_fs_t;
+
+/* Any thread: reads nothing but its arguments and the file system.
+ * @content_dir and @dat as the settings hold them, before expansion;
+ * @ov_core and @ov_dat an item's overrides, each NULL when unset.
+ * NULL out of memory.  Free with manual_content_scan_fs_free(). */
+manual_content_scan_fs_t *manual_content_scan_fs_read(
+      const char *content_dir, const char *dat,
+      const char *ov_core, const char *ov_dat);
+void manual_content_scan_fs_free(manual_content_scan_fs_t *fs);
+
+/* As manual_content_scan_set_menu_content_dir(), the folder's stat
+ * taken from @fs (NULL: asked now) */
+bool manual_content_scan_set_menu_content_dir_fs(const char *content_dir,
+      const manual_content_scan_fs_t *fs);
+
 bool manual_content_scan_set_menu_scan_method(
       enum manual_content_scan_method method);
 
@@ -269,6 +342,40 @@ enum manual_content_scan_playlist_refresh_status
       manual_content_scan_set_menu_from_playlist(playlist_t *playlist,
             const char *path_content_database, bool show_hidden_files);
 
+/* As manual_content_scan_set_menu_from_playlist(), with the
+ * paths of the installed databases in @rdb_list (NULL: none). */
+enum manual_content_scan_playlist_refresh_status
+      manual_content_scan_set_menu_from_playlist_list(playlist_t *playlist,
+            const struct string_list *rdb_list);
+
+/* As manual_content_scan_set_menu_from_playlist_list(), the folder
+ * and the DAT as @fs read them (NULL: asked now) */
+enum manual_content_scan_playlist_refresh_status
+      manual_content_scan_set_menu_from_playlist_list_fs(
+            playlist_t *playlist, const struct string_list *rdb_list,
+            const manual_content_scan_fs_t *fs);
+
+manual_content_scan_overrides_t *manual_content_scan_overrides_new(void);
+void manual_content_scan_overrides_free(manual_content_scan_overrides_t *ov);
+
+/* Lays @ov over the current settings and sets the Custom
+ * method, stopping at the first override that does not
+ * resolve; NULL @ov changes nothing.
+ * > @rdb_list: paths of the installed databases, or NULL
+ * > @count: the cores @ov->core matched, when not NULL */
+enum manual_content_scan_override_status
+      manual_content_scan_apply_overrides(
+            const manual_content_scan_overrides_t *ov,
+            const struct string_list *rdb_list, unsigned *count);
+
+/* As manual_content_scan_apply_overrides(), the core path and the
+ * DAT as @fs read them for @ov (NULL: asked now) */
+enum manual_content_scan_override_status
+      manual_content_scan_apply_overrides_fs(
+            const manual_content_scan_overrides_t *ov,
+            const struct string_list *rdb_list, unsigned *count,
+            const manual_content_scan_fs_t *fs);
+
 /* Menu getters */
 
 /* Fetches content directory for next manual scan
@@ -318,6 +425,10 @@ struct string_list *manual_content_scan_get_menu_core_name_list(void);
 /* Task Helpers */
 /****************/
 
+/* True if a scan of the current settings writes one playlist
+ * (target_is_single_determined_playlist). */
+bool manual_content_scan_is_single_playlist(void);
+
 /* Parses current manual content scan settings,
  * and extracts all information required to configure
  * a manual content scan task.

menu/cbs/menu_cbs_ok.c

@@ -9161,6 +9161,8 @@ static int action_ok_manual_content_scan_start(const char *path,
 {
    playlist_config_t playlist_config;
    settings_t *settings                = config_get_ptr();
+   if (task_dbscan_chain_refuse())
+      return 0;
    /* Note: playlist_config.path will set by the
     * task itself */
    playlist_config.capacity            = COLLECTION_SIZE;
@@ -9171,7 +9173,7 @@ static int action_ok_manual_content_scan_start(const char *path,
          settings->bools.playlist_portable_paths ?
                settings->paths.directory_menu_content : NULL);
 
-   task_push_manual_content_scan(true, NULL);
+   task_push_manual_content_scan(true, NULL, NULL);
    return 0;
 }
 
@@ -9637,6 +9639,8 @@ static int action_ok_playlist_refresh(const char *path,
    playlist_config = playlist_get_config(playlist);
    if (!playlist_config || !*playlist_config->path)
       return -1;
+   if (task_dbscan_chain_refuse())
+      return 0;
    stat = manual_content_scan_set_menu_from_playlist(playlist,
          settings->paths.path_content_database,
          settings->bools.show_hidden_files);
@@ -9721,7 +9725,35 @@ static int action_ok_playlist_refresh(const char *path,
             settings->bools.playlist_portable_paths ?
             settings->paths.directory_menu_content : NULL);
 
-      task_push_manual_content_scan(false, NULL);
+      task_push_manual_content_scan(false, NULL, NULL);
+   }
+   return 0;
+}
+
+static void action_ok_playlist_refresh_all_cb(
+      const task_dbscan_chain_result_t *res, void *user_data)
+{
+   char msg[128];
+   int  n = snprintf(msg, sizeof(msg),
+         msg_hash_to_str(MSG_PLAYLIST_MANAGER_REFRESH_ALL_DONE),
+         res->ran, res->skipped, res->failed);
+   /* the length written, not the one a translation asked for */
+   size_t _len = n < 0 ? 0 : (size_t)n >= sizeof(msg) ? sizeof(msg) - 1 : (size_t)n;
+   runloop_msg_queue_push(msg, _len, 1, 180, true, NULL,
+         MESSAGE_QUEUE_ICON_DEFAULT, MESSAGE_QUEUE_CATEGORY_INFO);
+}
+
+static int action_ok_playlist_refresh_all(const char *path,
+      const char *label, unsigned type, size_t idx, size_t entry_idx)
+{
+   task_dbscan_item_t *item = NULL;
+   if (task_dbscan_chain_refuse_start())
+      return 0;
+   if ((item = (task_dbscan_item_t*)calloc(1, sizeof(*item))))
+   {
+      item->kind = TASK_DBSCAN_ITEM_REFRESH_ALL;
+      task_push_dbscan_chain(item, 1, NULL,
+            action_ok_playlist_refresh_all_cb, NULL);
    }
    return 0;
 }
@@ -10086,6 +10118,7 @@ static int menu_cbs_init_bind_ok_compare_label(menu_file_list_cbs_t *cbs,
          {MENU_ENUM_LABEL_PLAYLIST_MANAGER_RESET_CORES,        action_ok_playlist_reset_cores},
          {MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST,     action_ok_playlist_clean},
          {MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST,   action_ok_playlist_refresh},
+         {MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS, action_ok_playlist_refresh_all},
          {MENU_ENUM_LABEL_ACCOUNTS_RETRO_ACHIEVEMENTS,         action_ok_push_accounts_cheevos_list},
 #ifdef HAVE_LAKKA
          {MENU_ENUM_LABEL_EJECT_DISC,                          action_ok_push_eject_disc},

menu/cbs/menu_cbs_scan.c

@@ -69,7 +69,7 @@ int action_scan_file(const char *path,
 
    fill_pathname_join_special(fullpath, menu_path, path, sizeof(fullpath));
 
-   task_push_dbscan(fullpath, handle_dbscan_finished);
+   task_push_dbscan(fullpath, handle_dbscan_finished, NULL);
 
    return 0;
 }
@@ -96,7 +96,7 @@ int action_scan_directory(const char *path,
    else
       strlcpy(fullpath, menu_path, sizeof(fullpath));
 
-   task_push_dbscan(fullpath, handle_dbscan_finished);
+   task_push_dbscan(fullpath, handle_dbscan_finished, NULL);
 
    return 0;
 }

menu/cbs/menu_cbs_sublabel.c

@@ -2755,6 +2755,7 @@ int menu_cbs_init_bind_sublabel(menu_file_list_cbs_t *cbs,
       { MENU_ENUM_LABEL_PLAYLIST_MANAGER_SORT_MODE, MENU_ENUM_SUBLABEL_PLAYLIST_MANAGER_SORT_MODE },
       { MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST, MENU_ENUM_SUBLABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST },
       { MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST, MENU_ENUM_SUBLABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST },
+      { MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS, MENU_ENUM_SUBLABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS },
       { MENU_ENUM_LABEL_SAVE_AS_CONFIG, MENU_ENUM_SUBLABEL_SAVE_AS_CONFIG },
       { MENU_ENUM_LABEL_SAVE_MAIN_CONFIG, MENU_ENUM_SUBLABEL_SAVE_MAIN_CONFIG },
       { MENU_ENUM_LABEL_ONSCREEN_DISPLAY_SETTINGS, MENU_ENUM_SUBLABEL_ONSCREEN_DISPLAY_SETTINGS },

menu/drivers/materialui.c

@@ -12746,6 +12746,7 @@ static void materialui_list_insert(void *userdata,
                   || string_is_equal(label, MENU_ENUM_LABEL_UPDATE_GLSL_SHADERS_STR)
                   || string_is_equal(label, MENU_ENUM_LABEL_UPDATE_SLANG_SHADERS_STR)
                   || string_is_equal(label, MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST_STR)
+                  || string_is_equal(label, MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS_STR)
                   )
                   {
                      node->icon_texture_index = MUI_TEXTURE_UPDATER;

menu/drivers/ozone.c

@@ -2460,6 +2460,7 @@ static uintptr_t ozone_entries_icon_get_texture(
       case MENU_ENUM_LABEL_FRAME_TIME_COUNTER_SETTINGS:
       case MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST:
       case MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST:
+      case MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS:
       case MENU_ENUM_LABEL_CLOUD_SYNC_SETTINGS:
             return icons_tex[OZONE_ENTRIES_ICONS_TEXTURE_RELOAD];
       case MENU_ENUM_LABEL_SHUTDOWN:

menu/drivers/xmb.c

@@ -4393,6 +4393,7 @@ static uintptr_t xmb_icon_get_id(xmb_handle_t *xmb,
       case MENU_ENUM_LABEL_FRAME_TIME_COUNTER_SETTINGS:
       case MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST:
       case MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST:
+      case MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS:
       case MENU_ENUM_LABEL_CLOUD_SYNC_SETTINGS:
          return GFX_SURFACE_HANDLE(xmb->textures.list[XMB_TEXTURE_RELOAD]);
       case MENU_ENUM_LABEL_VRR_RUNLOOP_ENABLE:

menu/menu_displaylist.c

@@ -6378,6 +6378,17 @@ static unsigned menu_displaylist_parse_playlist_manager_list(
          count++;
 #endif
 
+   /* None keeps the "No entries" fallback */
+   if (count)
+   {
+      menu_entries_prepend(list,
+            msg_hash_to_str(MENU_ENUM_LABEL_VALUE_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS),
+            MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS_STR,
+            MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS,
+            MENU_SETTING_ACTION, 0, 0);
+      count++;
+   }
+
    return count;
 }
 

msg_hash.h

@@ -29964,6 +29964,9 @@ enum msg_hash_enums
    MSG_PLAYLIST_MANAGER_REFRESH_INVALID_DAT_FILE,
    MSG_PLAYLIST_MANAGER_REFRESH_DAT_FILE_TOO_LARGE,
 
+   MENU_LABEL(PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS),
+   MSG_PLAYLIST_MANAGER_REFRESH_ALL_DONE,
+
    MENU_LABEL(CORE_UPDATER_SETTINGS),
    MENU_LABEL(SAVE_AS_CONFIG),
    MENU_LABEL(SAVE_MAIN_CONFIG),
@@ -34353,6 +34356,7 @@ enum msg_hash_enums
    MSG_MANUAL_CONTENT_SCAN_IN_PROGRESS,
    MSG_MANUAL_CONTENT_SCAN_M3U_CLEANUP,
    MSG_MANUAL_CONTENT_SCAN_END,
+   MSG_MANUAL_CONTENT_SCAN_ALREADY_RUNNING,
 
    MSG_3DS_BOTTOM_MENU_DEFAULT,
    MSG_3DS_BOTTOM_MENU_ASSET_NOT_FOUND,

msg_hash_lbl_str.h

@@ -486,6 +486,7 @@ extern const struct msg_hash_us_lbl_blob_s msg_hash_us_lbl_blob;
 #define MENU_ENUM_LABEL_PLAYLIST_MANAGER_SORT_MODE_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_PLAYLIST_MANAGER_SORT_MODE)
 #define MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST)
 #define MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST)
+#define MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS)
 #define MENU_ENUM_LABEL_PLAYLIST_SETTINGS_BEGIN_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_PLAYLIST_SETTINGS_BEGIN)
 #define MENU_ENUM_LABEL_RDB_ENTRY_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_RDB_ENTRY)
 #define MENU_ENUM_LABEL_RDB_ENTRY_DETAIL_STR (msg_hash_us_lbl_blob.m_MENU_ENUM_LABEL_RDB_ENTRY_DETAIL)
@@ -2239,6 +2240,7 @@ extern const struct msg_hash_us_lbl_blob_s msg_hash_us_lbl_blob;
 #define MENU_ENUM_LABEL_PLAYLIST_MANAGER_SORT_MODE_STR "playlist_manager_sort_mode"
 #define MENU_ENUM_LABEL_PLAYLIST_MANAGER_CLEAN_PLAYLIST_STR "playlist_manager_clean_playlist"
 #define MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_PLAYLIST_STR "playlist_manager_refresh_playlist"
+#define MENU_ENUM_LABEL_PLAYLIST_MANAGER_REFRESH_ALL_PLAYLISTS_STR "playlist_manager_refresh_all_playlists"
 #define MENU_ENUM_LABEL_PLAYLIST_SETTINGS_BEGIN_STR "playlist_settings_begin"
 #define MENU_ENUM_LABEL_RDB_ENTRY_STR "rdb_entry"
 #define MENU_ENUM_LABEL_RDB_ENTRY_DETAIL_STR "rdb_entry_detail"

playlist.c

@@ -32,6 +32,8 @@
 #include <formats/rjson.h>
 #include <formats/rjson_stream.h>
 #include <array/rbuf.h>
+#include <retro_atomic.h>
+#include <queues/mpsc_stack.h>
 
 #include "playlist.h"
 #include "verbosity.h"
@@ -75,7 +77,9 @@ enum content_playlist_flags
    CNT_PLAYLIST_FLG_MOD        = (1 << 0),
    CNT_PLAYLIST_FLG_OLD_FMT    = (1 << 1),
    CNT_PLAYLIST_FLG_COMPRESSED = (1 << 2),
-   CNT_PLAYLIST_FLG_CACHED_EXT = (1 << 3)
+   CNT_PLAYLIST_FLG_CACHED_EXT = (1 << 3),
+   /* read by playlist_parse_begin_header(): no entries, never written */
+   CNT_PLAYLIST_FLG_HEADER_ONLY = (1 << 4)
 };
 
 struct content_playlist
@@ -111,7 +115,9 @@ enum json_ctx_flags
    JSON_CTX_FLG_IN_ITEMS             = (1 << 0),
    JSON_CTX_FLG_IN_SUBSYSTEM_CONTENT = (1 << 1),
    JSON_CTX_FLG_CAPACITY_EXCEEDED    = (1 << 2),
-   JSON_CTX_FLG_OOM                  = (1 << 3)
+   JSON_CTX_FLG_OOM                  = (1 << 3),
+   JSON_CTX_FLG_HEADER_ONLY          = (1 << 4),  /* stop at "items" */
+   JSON_CTX_FLG_HEADER_DONE          = (1 << 5)
 };
 
 /* The runtime and last-played values live packed inside the entry and
@@ -156,9 +162,27 @@ typedef struct
 static playlist_t *playlist_cached = NULL;
 /* Set when the cached playlist's file has been rewritten by something
  * else, so the next request re-reads it.  See
- * playlist_cached_after_write for why this is a flag and not a free. */
+ * playlist_cached_note_write for why this is a flag and not a free. */
 static bool playlist_cached_stale  = false;
 
+/* A write reaches the cache through a note, from whatever thread made
+ * it: the main thread applies the notes when it next decides whether
+ * the cache may stand in, so no writer reads or writes playlist_cached.
+ * Where the pointer atomics or threads are missing the write applies
+ * at once, as every write then comes from the one thread. */
+#if defined(HAVE_THREADS) && defined(RETRO_ATOMIC_HAS_PTR)
+#define PLAYLIST_WRITE_NOTES 1
+typedef struct playlist_write_note
+{
+   mpsc_stack_node_t node;
+   const playlist_t *written;  /* compared, never dereferenced */
+   int64_t           size;     /* the file's size after the write */
+   char              path[1];  /* allocated to fit */
+} playlist_write_note_t;
+
+static mpsc_stack_t playlist_write_notes;
+#endif
+
 typedef int (playlist_sort_fun_t)(
       const struct playlist_entry *a,
       const struct playlist_entry *b);
@@ -2168,40 +2192,72 @@ end:
    }
 }
 
-/* A playlist file has just been written.  If the cached playlist is a
- * different object reading the same path, its contents are now stale -
- * a scan or a playlist-manager task writing the file the menu happens
- * to have open is the ordinary way this happens.  Drop it rather than
- * let a later reuse serve what is on disk no longer.  When the cached
- * playlist is the one that was written, it is still authoritative; only
- * its size stamp needs to catch up. */
-static void playlist_cached_after_write(playlist_t *written)
+/* What one write means for the cache: a write of the cached playlist
+ * itself moves its size stamp; any other write to its file leaves it
+ * stale, flagged rather than freed so nothing holding the pointer
+ * playlist_get_cached returned loses it (the next playlist_init_cached
+ * re-reads, at a point where nothing holds the old one).  Main thread. */
+static void playlist_cached_note_write(const playlist_t *written,
+      const char *path, int64_t size)
 {
-   if (!playlist_cached || !written || !*written->config.path)
+   if (!playlist_cached || !string_is_equal(playlist_cached->config.path, path))
       return;
    if (playlist_cached == written)
+      playlist_cached->file_size = size;
+   else
+      playlist_cached_stale = true;
+}
+
+/* The notes since the last look, newest first: only the newest write of
+ * the cached playlist sets its stamp. Main thread. */
+static void playlist_cached_apply_writes(void)
+{
+#ifdef PLAYLIST_WRITE_NOTES
+   bool sized              = false;
+   mpsc_stack_node_t *node = mpsc_stack_drain(&playlist_write_notes);
+   while (node)
    {
-      playlist_cached->file_size = path_get_size(written->config.path);
+      mpsc_stack_node_t     *next = node->next;
+      playlist_write_note_t *n    = (playlist_write_note_t*)node;
+      if (playlist_cached && n->written == playlist_cached)
+      {
+         if (!sized)
+            playlist_cached_note_write(n->written, n->path, n->size);
+         sized = true;
+      }
+      else
+         playlist_cached_note_write(n->written, n->path, n->size);
+      free(n);
+      node = next;
+   }
+#endif
+}
+
+/* A playlist file has just been written, possibly by a task on a
+ * worker.  A scan or a playlist-manager task writing the file the menu
+ * has open is the ordinary way the cache goes stale. */
+static void playlist_cached_after_write(playlist_t *written)
+{
+#ifdef PLAYLIST_WRITE_NOTES
+   size_t                 len;
+   playlist_write_note_t *n;
+   if (!written || !*written->config.path)
+      return;
+   len = strlen(written->config.path);
+   if (!(n = (playlist_write_note_t*)malloc(sizeof(*n) + len)))
+   {
+      /* no note to leave: a later size check still catches a change */
       return;
    }
-   if (string_is_equal(playlist_cached->config.path, written->config.path))
-   {
-      /* Mark it, do not free it.
-       *
-       * This runs from whatever happened to write the file, and the
-       * commonest case is the history list: launching an entry pushes
-       * it onto g_defaults.content_history, which is a different
-       * playlist_t for the same path as the one the menu is displaying
-       * from.  Freeing here would pull that object out from under
-       * every menu_displaylist caller still holding what
-       * playlist_get_cached returned - a use after free whose symptoms
-       * appear later and somewhere else entirely.
-       *
-       * The staleness is what matters, not the memory: flagging it
-       * makes the next playlist_init_cached re-read, which is a point
-       * where nothing is holding the old pointer. */
-      playlist_cached_stale = true;
-   }
+   n->written = written;
+   n->size    = path_get_size(written->config.path);
+   memcpy(n->path, written->config.path, len + 1);
+   mpsc_stack_push(&playlist_write_notes, &n->node);
+#else
+   if (written && *written->config.path)
+      playlist_cached_note_write(written, written->config.path,
+            path_get_size(written->config.path));
+#endif
 }
 
 void playlist_write_file(playlist_t *playlist)
@@ -2226,6 +2282,8 @@ void playlist_write_file(playlist_t *playlist)
 
    if (   !playlist
        || !*playlist->config.path
+       /* a header alone would replace the file's entries with none */
+       || (playlist->flags & CNT_PLAYLIST_FLG_HEADER_ONLY)
        || !( (playlist->flags & CNT_PLAYLIST_FLG_MOD)
 #if defined(HAVE_COMPRESSION)
           || (pl_compressed != playlist->config.compress)
@@ -3113,7 +3171,13 @@ static bool JSONObjectMemberHandler(void *context, const char *pValue, size_t le
             break;
          case 'i':
             if (!strcmp(pValue, "items"))
-               pCtx->flags |= JSON_CTX_FLG_IN_ITEMS;
+            {
+               /* the writer puts every header field before the items */
+               if (pCtx->flags & JSON_CTX_FLG_HEADER_ONLY)
+                  pCtx->flags |= JSON_CTX_FLG_HEADER_DONE;
+               else
+                  pCtx->flags |= JSON_CTX_FLG_IN_ITEMS;
+            }
             break;
          case 'l':
             if (!strcmp(pValue, "label_display_mode"))
@@ -3215,6 +3279,7 @@ struct playlist_parse
    enum playlist_parse_phase phase;
    bool res;
    bool autofix_scan_done;
+   bool header_only;
 };
 
 static void playlist_parse_close_io(playlist_parse_t *p)
@@ -3253,6 +3318,35 @@ static void playlist_parse_close_io(playlist_parse_t *p)
 /* Decide what follows the parse: the autofix pass when it applies,   */
 /* completion otherwise.  Mirrors the condition playlist_init         */
 /* historically evaluated after playlist_read_file returned.          */
+/* The scan record's paths under the base content directory @p's
+ * config names, as autofix moves them. */
+static void playlist_rebase_scan_record(playlist_t *playlist,
+      size_t oldref_len, size_t newref_len)
+{
+   char tmp[PATH_MAX_LENGTH];
+
+   if (playlist->scan_record.content_dir && *playlist->scan_record.content_dir)
+   {
+      tmp[0] = '\0';
+      path_replace_base_path_and_convert_to_local_file_system(tmp,
+            playlist->scan_record.content_dir,
+            playlist->base_content_directory, oldref_len,
+            playlist->config.base_content_directory, newref_len,
+            sizeof(tmp));
+      playlist_entry_replace_str(&playlist->scan_record.content_dir, tmp);
+   }
+   if (playlist->scan_record.dat_file_path && *playlist->scan_record.dat_file_path)
+   {
+      tmp[0] = '\0';
+      path_replace_base_path_and_convert_to_local_file_system(tmp,
+            playlist->scan_record.dat_file_path,
+            playlist->base_content_directory, oldref_len,
+            playlist->config.base_content_directory, newref_len,
+            sizeof(tmp));
+      playlist_entry_replace_str(&playlist->scan_record.dat_file_path, tmp);
+   }
+}
+
 static void playlist_parse_enter_autofix(playlist_parse_t *p)
 {
    playlist_t *playlist = p->playlist;
@@ -3265,6 +3359,23 @@ static void playlist_parse_enter_autofix(playlist_parse_t *p)
       return;
    }
 
+   /* The header alone: its scan record moves as autofix would move
+    * it, in memory - there are no entries to fix and nothing is
+    * saved. */
+   if (p->header_only)
+   {
+      if (    playlist->config.autofix_paths
+          && !string_is_equal(playlist->base_content_directory,
+               playlist->config.base_content_directory)
+          && playlist->base_content_directory
+          && *playlist->base_content_directory)
+         playlist_rebase_scan_record(playlist,
+               strlen(playlist->base_content_directory),
+               strlen(playlist->config.base_content_directory));
+      p->phase = PLAYLIST_PARSE_PHASE_DONE;
+      return;
+   }
+
    if (    playlist->config.autofix_paths
        && !string_is_equal(playlist->base_content_directory,
             playlist->config.base_content_directory)
@@ -3369,6 +3480,11 @@ static int playlist_parse_step_json(playlist_parse_t *p,
          stop = t;
          goto stopped;
       }
+      if (p->context.flags & JSON_CTX_FLG_HEADER_DONE)
+      {
+         stop = RJSON_DONE;
+         goto stopped;
+      }
    }
 
 stopped:
@@ -3678,35 +3794,8 @@ static int playlist_parse_step_autofix(playlist_parse_t *p,
       {
          p->autofix_scan_done = true;
 
-         /* Fix scan record content directory */
-         if (playlist->scan_record.content_dir && *playlist->scan_record.content_dir)
-         {
-            tmp_entry_path[0] = '\0';
-            path_replace_base_path_and_convert_to_local_file_system(
-                  tmp_entry_path,
-                  playlist->scan_record.content_dir,
-                  playlist->base_content_directory, p->oldref_len,
-                  playlist->config.base_content_directory, p->newref_len,
-                  sizeof(tmp_entry_path));
-
-            playlist_entry_replace_str(&playlist->scan_record.content_dir,
-                  tmp_entry_path);
-         }
-
-         /* Fix scan record arcade DAT file */
-         if (playlist->scan_record.dat_file_path && *playlist->scan_record.dat_file_path)
-         {
-            tmp_entry_path[0] = '\0';
-            path_replace_base_path_and_convert_to_local_file_system(
-                  tmp_entry_path,
-                  playlist->scan_record.dat_file_path,
-                  playlist->base_content_directory, p->oldref_len,
-                  playlist->config.base_content_directory, p->newref_len,
-                  sizeof(tmp_entry_path));
-
-            playlist_entry_replace_str(&playlist->scan_record.dat_file_path,
-                  tmp_entry_path);
-         }
+         /* Fix the scan record's content directory and DAT file */
+         playlist_rebase_scan_record(playlist, p->oldref_len, p->newref_len);
       }
    }
 
@@ -3729,7 +3818,8 @@ static int playlist_parse_step_autofix(playlist_parse_t *p,
    return 1;
 }
 
-playlist_parse_t *playlist_parse_begin(const playlist_config_t *config)
+static playlist_parse_t *playlist_parse_begin_ex(
+      const playlist_config_t *config, bool header_only)
 {
    playlist_parse_t *p  = NULL;
    playlist_t *playlist = (playlist_t*)malloc(sizeof(*playlist));
@@ -3770,6 +3860,11 @@ playlist_parse_t *playlist_parse_begin(const playlist_config_t *config)
    p->playlist = playlist;
    p->res      = true;
    p->phase    = PLAYLIST_PARSE_PHASE_DONE;   /* until proven otherwise */
+   if ((p->header_only = header_only))
+   {
+      playlist->flags  |= CNT_PLAYLIST_FLG_HEADER_ONLY;
+      p->context.flags |= JSON_CTX_FLG_HEADER_ONLY;
+   }
 
 #if defined(HAVE_COMPRESSION)
       /* Always use RZIP interface when reading playlists
@@ -3860,6 +3955,13 @@ playlist_parse_t *playlist_parse_begin(const playlist_config_t *config)
       return p;
    }
 
+   /* The old format has no scan settings: a header read is done. */
+   if (p->header_only)
+   {
+      playlist_parse_enter_autofix(p);
+      return p;
+   }
+
    /* Heap-held: at six entries this is 3 KiB even at the console
     * path length, on the playlist load path that runs from task
     * threads.  On allocation failure the legacy-format file simply
@@ -3879,6 +3981,16 @@ error:
    return NULL;
 }
 
+playlist_parse_t *playlist_parse_begin(const playlist_config_t *config)
+{
+   return playlist_parse_begin_ex(config, false);
+}
+
+playlist_parse_t *playlist_parse_begin_header(const playlist_config_t *config)
+{
+   return playlist_parse_begin_ex(config, true);
+}
+
 int playlist_parse_step(playlist_parse_t *p,
       bool (*budget_cb)(void *), void *budget_ud)
 {
@@ -3950,6 +4062,8 @@ void playlist_parse_abort(playlist_parse_t *p)
 
 void playlist_free_cached(void)
 {
+   /* what was written since no longer has a cache to tell */
+   playlist_cached_apply_writes();
    if (playlist_cached && !(playlist_cached->flags & CNT_PLAYLIST_FLG_CACHED_EXT))
       playlist_free(playlist_cached);
    playlist_cached       = NULL;
@@ -3980,6 +4094,7 @@ playlist_t *playlist_get_cached(void)
  * size is the one signal the VFS layer offers portably. */
 static bool playlist_cached_is_reusable(const playlist_config_t *config)
 {
+   playlist_cached_apply_writes();
    if (!playlist_cached || playlist_cached_stale)
       return false;
    if (!string_is_equal(playlist_cached->config.path, config->path))

playlist.h

@@ -259,6 +259,12 @@ playlist_t *playlist_init(const playlist_config_t *config);
 typedef struct playlist_parse playlist_parse_t;
 
 playlist_parse_t *playlist_parse_begin(const playlist_config_t *config);
+/* The same, for what comes ahead of the entries only - the defaults
+ * and the scan record, which the writer puts before them - as a
+ * playlist with no entries that playlist_write_file() never writes.
+ * The scan record's paths move to the configured base content
+ * directory as a full read's autofix would move them, in memory. */
+playlist_parse_t *playlist_parse_begin_header(const playlist_config_t *config);
 int playlist_parse_step(playlist_parse_t *p,
       bool (*budget_cb)(void *), void *budget_ud);
 playlist_t *playlist_parse_end(playlist_parse_t *p);

retroarch.c

@@ -231,6 +231,7 @@
 #include "core.h"
 #include "configuration.h"
 #include "list_special.h"
+#include "manual_content_scan.h"
 #ifdef HAVE_CHEATS
 #include "cheat_manager.h"
 #endif
@@ -390,6 +391,19 @@ enum
    RA_OPT_MAX_FRAMES_SCREENSHOT_PATH,
    RA_OPT_SET_SHADER,
    RA_OPT_DATABASE_SCAN,
+   RA_OPT_SCAN_REFRESH,
+   RA_OPT_SCAN_REFRESH_ALL,
+   RA_OPT_SCAN_PLAYLIST,
+   RA_OPT_SCAN_DATABASE,
+   RA_OPT_SCAN_MATCH,
+   RA_OPT_SCAN_DAT,
+   RA_OPT_SCAN_CORE,
+   RA_OPT_SCAN_EXTS,
+   RA_OPT_SCAN_NO_RECURSIVE,
+   RA_OPT_SCAN_NO_ARCHIVES,
+   RA_OPT_SCAN_OVERWRITE,
+   RA_OPT_SCAN_VALIDATE,
+   RA_OPT_SCAN_OMIT_DB_REF,
    RA_OPT_ACCESSIBILITY,
    RA_OPT_LOAD_MENU_ON_ERROR,
    RA_OPT_CLOUDSYNC,
@@ -7941,9 +7955,59 @@ static void retroarch_print_help(const char *arg0)
          "  how a particular subsystem wants content to be loaded.\n");
 
 #ifdef HAVE_LIBRETRODB
+   fputs(buf, stdout);
+   buf[0] = '\0';
+   _len   = 0;
+
    strlcpy_append(buf, sizeof(buf), &_len,
          "      --scan=PATH|FILE           "
-         "Import content from path.\n");
+         "Import content from path. Repeatable.\n"
+         "                                 "
+         "  The options from --scan-playlist on apply\n"
+         "                                 "
+         "  to the --scan, --scan-refresh or\n"
+         "                                 "
+         "  --scan-refresh-all before them. Without\n"
+         "                                 "
+         "  any, the scan is Fully Automatic.\n");
+   strlcpy_append(buf, sizeof(buf), &_len,
+         "      --scan-refresh=NAME|PATH   "
+         "Refresh a playlist by its saved scan settings.\n"
+         "      --scan-refresh-all         "
+         "Refresh each playlist that has saved settings.\n");
+   strlcpy_append(buf, sizeof(buf), &_len,
+         "      --scan-playlist=NAME       "
+         "Playlist to add to.\n"
+         "      --scan-database=NAME|first "
+         "Database to match: one by name, or the first\n"
+         "                                 "
+         "  that matches.\n"
+         "      --scan-match=MODE          "
+         "Database check: strict, loose or none.\n"
+         "      --scan-dat=FILE            "
+         "Arcade DAT file to match instead, strict unless\n"
+         "                                 "
+         "  --scan-match=loose.\n");
+   strlcpy_append(buf, sizeof(buf), &_len,
+         "      --scan-core=CORE           "
+         "Core: a path, a file name or a display name.\n"
+         "      --scan-exts=EXTS           "
+         "File extensions, separated by commas or spaces.\n"
+         "      --scan-no-recursive        "
+         "Do not scan subfolders.\n"
+         "      --scan-no-archives         "
+         "Do not look inside archives.\n");
+   strlcpy_append(buf, sizeof(buf), &_len,
+         "      --scan-overwrite           "
+         "Replace the playlist instead of adding to it.\n"
+         "      --scan-validate            "
+         "Drop entries for missing or unsupported files.\n"
+         "      --scan-omit-db-ref         "
+         "Look up thumbnails by the playlist name.\n");
+
+   fputs(buf, stdout);
+   buf[0] = '\0';
+   _len   = 0;
 #endif
 #ifdef HAVE_CLOUDSYNC
    strlcpy_append(buf, sizeof(buf), &_len,
@@ -8274,6 +8338,253 @@ void handle_dbscan_finished(retro_task_t *task,
       void *task_data, void *user_data, const char *err);
 #endif
 
+#ifdef HAVE_LIBRETRODB
+/* Appends a @kind item for @arg; false if out of memory. */
+static bool retroarch_cli_scan_add(task_dbscan_item_t **items,
+      size_t *count, size_t *cap, enum task_dbscan_item_kind kind,
+      const char *arg)
+{
+   task_dbscan_item_t *item;
+
+   if (*count == *cap)
+   {
+      size_t n                  = *cap ? *cap * 2 : 4;
+      task_dbscan_item_t *grown = (task_dbscan_item_t*)realloc(*items,
+            n * sizeof(*grown));
+
+      if (!grown)
+         return false;
+      *items = grown;
+      *cap   = n;
+   }
+   item            = &(*items)[*count];
+   item->kind      = kind;
+   item->overrides = NULL;
+   item->arg       = NULL;
+   if (arg && !(item->arg = strdup(arg)))
+      return false;
+   (*count)++;
+   return true;
+}
+
+/* Sets scan option @c, one of @opts, on the latest of the @count
+ * items; a bad one ends the run before the config loads. */
+static void retroarch_cli_scan_option(task_dbscan_item_t *items,
+      size_t count, const struct option *opts, int c, const char *value)
+{
+   manual_content_scan_overrides_t *ov = NULL;
+   const char *err                     = NULL;
+   char **field                        = NULL;
+   char *copy                          = NULL;
+   size_t max_len                      = PATH_MAX_LENGTH - 1;
+
+   while (opts->val != c)
+      opts++;
+
+   if (!count)
+      err = "must follow a --scan";
+   else
+   {
+      if (!(ov = items[count - 1].overrides))
+         ov = items[count - 1].overrides =
+               manual_content_scan_overrides_new();
+      if (opts->has_arg)
+         copy = strdup(value);
+      if (!ov || (opts->has_arg && !copy))
+      {
+         fprintf(stderr, "out of memory\n");
+         exit(1);
+      }
+
+      switch (c)
+      {
+         case RA_OPT_SCAN_PLAYLIST:
+            /* A refresh writes its own playlist. */
+            if (items[count - 1].kind == TASK_DBSCAN_ITEM_REFRESH)
+               err     = "cannot follow --scan-refresh";
+            else if (items[count - 1].kind == TASK_DBSCAN_ITEM_REFRESH_ALL)
+               err     = "cannot follow --scan-refresh-all";
+            else
+            {
+               field   = &ov->playlist;
+               /* <name>.lpl.tmp fits a file name */
+               max_len = NAME_MAX_LENGTH - 9;
+            }
+            break;
+         case RA_OPT_SCAN_DATABASE:
+            if (string_is_equal(copy, "first"))
+            {
+               free(ov->database);
+               ov->database  = NULL;
+               ov->flags    |= MANUAL_CONTENT_SCAN_OVR_FIRST_DB;
+            }
+            else
+            {
+               ov->flags    &= ~MANUAL_CONTENT_SCAN_OVR_FIRST_DB;
+               field         = &ov->database;
+               max_len       = NAME_MAX_LENGTH - 1;
+            }
+            break;
+         case RA_OPT_SCAN_MATCH:
+            if (!*copy)
+               err       = "needs a value";
+            else if (string_is_equal(copy, "strict"))
+               ov->match = MANUAL_CONTENT_SCAN_USE_DB_STRICT;
+            else if (string_is_equal(copy, "loose"))
+               ov->match = MANUAL_CONTENT_SCAN_USE_DB_LOOSE;
+            else if (string_is_equal(copy, "none"))
+               ov->match = MANUAL_CONTENT_SCAN_USE_DB_NONE;
+            else
+               err       = "must be strict, loose or none";
+            ov->flags   |= MANUAL_CONTENT_SCAN_OVR_MATCH;
+            break;
+         case RA_OPT_SCAN_DAT:
+            field = &ov->dat;
+            break;
+         case RA_OPT_SCAN_CORE:
+            field = &ov->core;
+            break;
+         case RA_OPT_SCAN_EXTS:
+            {
+               /* Commas become spaces, runs of spaces one, ends trimmed */
+               char *out = copy;
+               const char *in;
+
+               for (in = copy; *in; in++)
+               {
+                  char ch = (*in == ',') ? ' ' : *in;
+                  if (ch != ' ' || (out > copy && out[-1] != ' '))
+                     *out++ = ch;
+               }
+               if (out > copy && out[-1] == ' ')
+                  out--;
+               *out  = '\0';
+               field = &ov->exts;
+            }
+            break;
+         case RA_OPT_SCAN_NO_RECURSIVE:
+            ov->flags |= MANUAL_CONTENT_SCAN_OVR_NO_RECURSIVE;
+            break;
+         case RA_OPT_SCAN_NO_ARCHIVES:
+            ov->flags |= MANUAL_CONTENT_SCAN_OVR_NO_ARCHIVES;
+            break;
+         case RA_OPT_SCAN_OVERWRITE:
+            ov->flags |= MANUAL_CONTENT_SCAN_OVR_OVERWRITE;
+            break;
+         case RA_OPT_SCAN_VALIDATE:
+            ov->flags |= MANUAL_CONTENT_SCAN_OVR_VALIDATE;
+            break;
+         case RA_OPT_SCAN_OMIT_DB_REF:
+            ov->flags |= MANUAL_CONTENT_SCAN_OVR_OMIT_DB_REF;
+            break;
+      }
+
+      if (field)
+      {
+         if (!*copy)
+            err = "needs a value";
+         else if (strlen(copy) > max_len)
+            err = "is too long";
+         else
+         {
+            free(*field);
+            *field = copy;
+            copy   = NULL;
+         }
+      }
+   }
+
+   free(copy);
+   if (err)
+   {
+      /* Before any --scan, no console yet on Windows. */
+      frontend_driver_attach_console();
+      fprintf(stderr, "--%s %s\n", opts->name, err);
+      exit(1);
+   }
+}
+
+/* What the chain told a run without --menu. */
+typedef struct
+{
+   task_dbscan_chain_result_t totals;
+   bool done;
+} retroarch_cli_scan_state_t;
+
+static void retroarch_cli_scan_done(
+      const task_dbscan_chain_result_t *res, void *user_data)
+{
+   retroarch_cli_scan_state_t *st = (retroarch_cli_scan_state_t*)user_data;
+
+   st->totals = *res;
+   st->done   = true;
+}
+
+static bool retroarch_cli_scan_running(void *data)
+{
+   return !((retroarch_cli_scan_state_t*)data)->done;
+}
+
+static void retroarch_cli_scan_menu_item(const char *name,
+      enum task_dbscan_outcome outcome,
+      const task_dbscan_result_t *res, void *user_data)
+{
+#ifdef HAVE_MENU
+   if (res)
+      handle_dbscan_finished(NULL, NULL, NULL, NULL);
+#endif
+}
+
+/* Runs the --scan @items through the scan chain, which owns them.
+ * With --menu the chain is pushed and the menu goes on.  Without, it
+ * is the whole run: the process exits 1 if any scan failed, else 0 if
+ * any added an entry, else 2. */
+static void retroarch_cli_scan(task_dbscan_item_t *items, size_t count,
+      bool menu, const char *content)
+{
+   retroarch_cli_scan_state_t st;
+   settings_t *settings = config_get_ptr();
+   int reinit_flags     = DRIVERS_CMD_ALL &
+         ~(DRIVER_VIDEO_MASK | DRIVER_AUDIO_MASK | DRIVER_MICROPHONE_MASK | DRIVER_INPUT_MASK | DRIVER_MIDI_MASK);
+
+   drivers_init(settings, reinit_flags, (enum driver_lifetime_flags)0, false);
+   retroarch_init_task_queue();
+
+   if (menu)
+   {
+      if (!task_push_dbscan_chain(items, count,
+            retroarch_cli_scan_menu_item, NULL, NULL))
+         fprintf(stderr, "out of memory\n");
+      return;
+   }
+
+#ifdef HAVE_THREADS
+   /* Threaded for this run whatever the setting: only there does the
+    * scan get its readers. */
+   if (!task_queue_is_threaded())
+   {
+      task_queue_set_threaded();
+      task_queue_check();
+   }
+#endif
+
+   if (content)
+      fprintf(stderr, "without --menu: ignoring \"%s\"\n",
+            content);
+
+   memset(&st, 0, sizeof(st));
+   if (task_push_dbscan_chain(items, count, NULL,
+         retroarch_cli_scan_done, &st))
+      task_queue_wait(retroarch_cli_scan_running, &st);
+   else
+      fprintf(stderr, "out of memory\n");
+
+   driver_uninit(DRIVERS_CMD_ALL, (enum driver_lifetime_flags)0);
+   task_queue_deinit();
+   exit((!st.done || st.totals.failed) ? 1 : st.totals.added ? 0 : 2);
+}
+#endif
+
 #ifdef HAVE_CLOUDSYNC
 /* The sync's verdict, where the waiting caller reads it. */
 static void retroarch_cloud_sync_finished(retro_task_t *task,
@@ -8327,8 +8638,11 @@ static bool retroarch_parse_input_and_config(
    access_state_t *access_st       = access_state_get_ptr();
 #endif
 #ifdef HAVE_LIBRETRODB
-   retro_task_callback_t cb_task_dbscan
-                                   = NULL;
+   task_dbscan_item_t   *scan_items = NULL;
+   size_t                scan_count = 0;
+   size_t                  scan_cap = 0;
+   const char        *scan_conflict = NULL;
+   bool                    cli_scan = false;
 #endif
 
    const struct option opts[]      = {
@@ -8391,6 +8705,19 @@ static bool retroarch_parse_input_and_config(
       { "entryslot",          1, NULL, 'e' },
 #ifdef HAVE_LIBRETRODB
       { "scan",               1, NULL, RA_OPT_DATABASE_SCAN },
+      { "scan-refresh",       1, NULL, RA_OPT_SCAN_REFRESH },
+      { "scan-refresh-all",   0, NULL, RA_OPT_SCAN_REFRESH_ALL },
+      { "scan-playlist",      1, NULL, RA_OPT_SCAN_PLAYLIST },
+      { "scan-database",      1, NULL, RA_OPT_SCAN_DATABASE },
+      { "scan-match",         1, NULL, RA_OPT_SCAN_MATCH },
+      { "scan-dat",           1, NULL, RA_OPT_SCAN_DAT },
+      { "scan-core",          1, NULL, RA_OPT_SCAN_CORE },
+      { "scan-exts",          1, NULL, RA_OPT_SCAN_EXTS },
+      { "scan-no-recursive",  0, NULL, RA_OPT_SCAN_NO_RECURSIVE },
+      { "scan-no-archives",   0, NULL, RA_OPT_SCAN_NO_ARCHIVES },
+      { "scan-overwrite",     0, NULL, RA_OPT_SCAN_OVERWRITE },
+      { "scan-validate",      0, NULL, RA_OPT_SCAN_VALIDATE },
+      { "scan-omit-db-ref",   0, NULL, RA_OPT_SCAN_OMIT_DB_REF },
 #endif
 #ifdef HAVE_CLOUDSYNC
       { "cloudsync",          0, NULL, RA_OPT_CLOUDSYNC },
@@ -8585,13 +8912,79 @@ static bool retroarch_parse_input_and_config(
                explicit_menu = true;
                break;
             case RA_OPT_DATABASE_SCAN:
+            case RA_OPT_SCAN_REFRESH:
+            case RA_OPT_SCAN_REFRESH_ALL:
 #ifdef HAVE_LIBRETRODB
-               verbosity_enable();
-               retroarch_override_setting_set(RARCH_OVERRIDE_SETTING_DATABASE_SCAN, NULL);
+               {
+                  enum task_dbscan_item_kind kind = TASK_DBSCAN_ITEM_PATH;
+                  const char *arg                 = optarg;
+
+                  if (c == RA_OPT_SCAN_REFRESH)
+                  {
+                     if (!*arg)
+                     {
+                        frontend_driver_attach_console();
+                        fprintf(stderr, "--scan-refresh needs a value\n");
+                        exit(1);
+                     }
+                     kind = TASK_DBSCAN_ITEM_REFRESH;
+                  }
+                  else if (c == RA_OPT_SCAN_REFRESH_ALL)
+                  {
+                     kind = TASK_DBSCAN_ITEM_REFRESH_ALL;
+                     arg  = NULL;
+                  }
+                  /* Cut to fit the chain's path buffers, a path could
+                   * name a folder above the one asked for. */
+                  if (arg && strlen(arg) >= PATH_MAX_LENGTH)
+                  {
+                     frontend_driver_attach_console();
+                     fprintf(stderr, "--%s is too long\n",
+                           c == RA_OPT_SCAN_REFRESH ? "scan-refresh" : "scan");
+                     exit(1);
+                  }
+                  cli_scan = true;
+                  /* The totals and errors are the command's output: the
+                   * config's log_verbosity must not turn them off, which
+                   * on Windows detaches the console. */
+                  verbosity_enable();
+                  retroarch_override_setting_set(RARCH_OVERRIDE_SETTING_VERBOSITY, NULL);
+                  retroarch_override_setting_set(RARCH_OVERRIDE_SETTING_DATABASE_SCAN, NULL);
+                  if (!retroarch_cli_scan_add(&scan_items, &scan_count,
+                        &scan_cap, kind, arg))
+                  {
+                     fprintf(stderr, "out of memory\n");
+                     exit(1);
+                  }
+               }
 #endif
                break;
+            case RA_OPT_SCAN_PLAYLIST:
+            case RA_OPT_SCAN_DATABASE:
+            case RA_OPT_SCAN_MATCH:
+            case RA_OPT_SCAN_DAT:
+            case RA_OPT_SCAN_CORE:
+            case RA_OPT_SCAN_EXTS:
+            case RA_OPT_SCAN_NO_RECURSIVE:
+            case RA_OPT_SCAN_NO_ARCHIVES:
+            case RA_OPT_SCAN_OVERWRITE:
+            case RA_OPT_SCAN_VALIDATE:
+            case RA_OPT_SCAN_OMIT_DB_REF:
+#ifdef HAVE_LIBRETRODB
+               retroarch_cli_scan_option(scan_items, scan_count, opts, c,
+                     optarg);
+#endif
+               break;
+#if defined(HAVE_LIBRETRODB) && defined(HAVE_NETWORKING) && defined(HAVE_NETWORK_CMD) && defined(HAVE_COMMAND)
+            case RA_OPT_COMMAND:
+               scan_conflict = "--command";
+               break;
+#endif
             case RA_OPT_CLOUDSYNC:
 #ifdef HAVE_CLOUDSYNC
+#ifdef HAVE_LIBRETRODB
+               scan_conflict = "--cloudsync";
+#endif
                /* The sync's summary is the command's output. */
                verbosity_enable();
                retroarch_override_setting_set(RARCH_OVERRIDE_SETTING_VERBOSITY, NULL);
@@ -8618,6 +9011,21 @@ static bool retroarch_parse_input_and_config(
          }
       }
    }
+#ifdef HAVE_LIBRETRODB
+   /* Each of them is the whole run. */
+   if (cli_scan && scan_conflict)
+   {
+      const char *first = "--scan";
+
+      if (scan_items[0].kind == TASK_DBSCAN_ITEM_REFRESH)
+         first = "--scan-refresh";
+      else if (scan_items[0].kind == TASK_DBSCAN_ITEM_REFRESH_ALL)
+         first = "--scan-refresh-all";
+      fprintf(stderr, "%s cannot be combined with %s\n", first,
+            scan_conflict);
+      exit(1);
+   }
+#endif
    verbosity_enabled = verbosity_is_enabled();
    /* Enable logging to file if verbosity and log-file arguments were passed.
     * RARCH_OVERRIDE_SETTING_LOG_TO_FILE is set by the RA_OPT_LOG_FILE case above
@@ -9039,42 +9447,20 @@ static bool retroarch_parse_input_and_config(
                }
                break;
             case RA_OPT_DATABASE_SCAN:
-#ifdef HAVE_LIBRETRODB
-               {
-                  settings_t *settings           = config_get_ptr();
-                  int reinit_flags               = DRIVERS_CMD_ALL &
-                        ~(DRIVER_VIDEO_MASK | DRIVER_AUDIO_MASK | DRIVER_MICROPHONE_MASK | DRIVER_INPUT_MASK | DRIVER_MIDI_MASK);
-
-                  drivers_init(settings, reinit_flags, (enum driver_lifetime_flags)0, false);
-                  retroarch_init_task_queue();
-#ifdef HAVE_THREADS
-                  /* Threaded for this run whatever the setting: only
-                   * there does the scan get its readers. */
-                  if (!explicit_menu && !task_queue_is_threaded())
-                  {
-                     task_queue_set_threaded();
-                     task_queue_check();
-                  }
-#endif
-
-#ifdef HAVE_MENU
-                  if (explicit_menu)
-                     cb_task_dbscan = handle_dbscan_finished;
-#endif
-
-                  task_push_dbscan(optarg, cb_task_dbscan);
-
-                  if (!explicit_menu)
-                  {
-                     /* Command-line scan with no menu: the scan is
-                      * the whole run, so run the queue dry and exit.
-                      * Nothing is drawn meanwhile. */
-                     task_queue_wait(NULL, NULL);
-                     driver_uninit(DRIVERS_CMD_ALL, (enum driver_lifetime_flags)0);
-                     exit(0);
-                  }
-               }
-#endif
+            case RA_OPT_SCAN_REFRESH:
+            case RA_OPT_SCAN_REFRESH_ALL:
+            case RA_OPT_SCAN_PLAYLIST:
+            case RA_OPT_SCAN_DATABASE:
+            case RA_OPT_SCAN_MATCH:
+            case RA_OPT_SCAN_DAT:
+            case RA_OPT_SCAN_CORE:
+            case RA_OPT_SCAN_EXTS:
+            case RA_OPT_SCAN_NO_RECURSIVE:
+            case RA_OPT_SCAN_NO_ARCHIVES:
+            case RA_OPT_SCAN_OVERWRITE:
+            case RA_OPT_SCAN_VALIDATE:
+            case RA_OPT_SCAN_OMIT_DB_REF:
+               /* Taken in the first pass. */
                break;
             case RA_OPT_CLOUDSYNC:
 #ifdef HAVE_CLOUDSYNC
@@ -9119,6 +9505,12 @@ static bool retroarch_parse_input_and_config(
       }
    }
 
+#ifdef HAVE_LIBRETRODB
+   if (scan_items)
+      retroarch_cli_scan(scan_items, scan_count, explicit_menu,
+            (optind < argc) ? argv[optind] : NULL);
+#endif
+
 #ifdef HAVE_GIT_VERSION
    RARCH_LOG("RetroArch %s (Git %s)\n",
          PACKAGE_VERSION, retroarch_git_version);

samples/playlist/playlist_parity_test.c

@@ -57,6 +57,7 @@
 
 #include "../../core_info.h"
 #include <compat/strl.h>
+#include <rthreads/rthreads.h>
 
 bool core_info_find(const char *core_path, core_info_t **core_info)
 {
@@ -812,6 +813,209 @@ static void check_cache_reuse(const char *path, bool compressed,
    playlist_free_cached();
 }
 
+
+/* A task on a worker writes the playlist file the menu has cached.
+ * The writer leaves a note for the main thread rather than touching
+ * the cached playlist, which the main thread frees and installs at
+ * will; the main thread applies the note when it next asks the cache.
+ * The worker's write keeps the file's size, so the note alone must
+ * keep the cache from serving what is on disk no longer.  A stress
+ * round follows: the worker rewrites while the main thread reuses and
+ * frees the cache (under ASan, and TSan with SANITIZER=thread). */
+struct cache_writer
+{
+   const char *path;
+   unsigned    rounds;
+};
+
+static void cache_writer_name(const char *path, const char *name)
+{
+   playlist_config_t config;
+   playlist_t *pl;
+   config_defaults(&config, path);
+   if (!(pl = playlist_init(&config)))
+      return;
+   playlist_set_default_core_name(pl, name);
+   playlist_write_file(pl);
+   playlist_free(pl);
+}
+
+static void cache_writer_run(void *arg)
+{
+   struct cache_writer *w = (struct cache_writer*)arg;
+   unsigned i;
+   for (i = 0; i < w->rounds; i++)
+      cache_writer_name(w->path, (i & 1) ? "Core X" : "Core Y");
+}
+
+static void lane_cache_worker_write(void)
+{
+   char path[512];
+   playlist_config_t config;
+   struct cache_writer w;
+   sthread_t *t;
+   playlist_t *pl;
+   unsigned had = failures;
+   unsigned i;
+
+   snprintf(path, sizeof(path), "%s/worker.lpl", fixture_dir);
+   CHECK(write_whole(path, json_full), "worker lane: fixture write");
+   /* the writer's own form, so every later write has its size */
+   cache_writer_name(path, "Core Z");
+   config_defaults(&config, path);
+
+   playlist_free_cached();
+   CHECK(playlist_init_cached(&config), "worker lane: cached init");
+   pl = playlist_get_cached();
+   CHECK(pl && streq(playlist_get_default_core_name(pl), "Core Z"),
+         "worker lane: cached playlist");
+
+   w.path   = path;
+   w.rounds = 1;
+   CHECK((t = sthread_create(cache_writer_run, &w)) != NULL,
+         "worker lane: writer thread");
+   if (t)
+      sthread_join(t);
+
+   CHECK(playlist_init_cached(&config), "worker lane: cached init after");
+   pl = playlist_get_cached();
+   CHECK(pl && streq(playlist_get_default_core_name(pl), "Core Y"),
+         "worker lane: a same-size write from a worker was served "
+         "from the stale cache");
+
+   /* concurrent: the worker writes while the main thread reuses,
+    * reads and frees */
+   w.rounds = 200;
+   if ((t = sthread_create(cache_writer_run, &w)))
+   {
+      for (i = 0; i < 400; i++)
+      {
+         /* Windows replaces the file with MoveFileExW, and an open
+          * that meets it fails: that read is an empty playlist. */
+         if (playlist_init_cached(&config) && (pl = playlist_get_cached()))
+            CHECK(   playlist_get_default_core_name(pl) != NULL
+#ifdef _WIN32
+                  || !playlist_size(pl)
+#endif
+                  , "worker lane: stress read");
+         if (i & 1)
+            playlist_free_cached();
+      }
+      sthread_join(t);
+   }
+   CHECK(playlist_init_cached(&config), "worker lane: final init");
+   pl = playlist_get_cached();
+   /* the last of 200 writes, round 199, wrote "Core X" */
+   CHECK(pl && streq(playlist_get_default_core_name(pl), "Core X"),
+         "worker lane: the cache missed the worker's last write");
+   playlist_free_cached();
+   if (failures == had)
+      fprintf(stderr, "[pass] worker-write cache lane\n");
+}
+
+/* The whole file, for comparing it before and after; caller frees. */
+static char *slurp(const char *path, long *len)
+{
+   char *buf = NULL;
+   FILE *f   = fopen(path, "rb");
+   *len      = -1;
+   if (!f)
+      return NULL;
+   if (     !fseek(f, 0, SEEK_END)
+         && (*len = ftell(f)) >= 0
+         && !fseek(f, 0, SEEK_SET)
+         && (buf = (char*)malloc((size_t)*len + 1)))
+   {
+      if (fread(buf, 1, (size_t)*len, f) != (size_t)*len)
+      {
+         free(buf);
+         buf = NULL;
+      }
+   }
+   fclose(f);
+   return buf;
+}
+
+static playlist_t *header_read(const playlist_config_t *config)
+{
+   playlist_parse_t *p = playlist_parse_begin_header(config);
+   if (!p)
+      return NULL;
+   if (playlist_parse_step(p, NULL, NULL) < 0)
+   {
+      playlist_parse_abort(p);
+      return NULL;
+   }
+   return playlist_parse_end(p);
+}
+
+/* playlist_parse_begin_header(): what comes ahead of the entries and
+ * nothing after; under a moved base content directory the scan record
+ * moves in memory while the file stays as it was; the result is never
+ * written; the old format, which has no scan settings, reads empty. */
+static void lane_header_only(void)
+{
+   char path[512];
+   playlist_config_t config;
+   playlist_t *pl;
+   char *before, *after;
+   long blen, alen;
+   unsigned had = failures;
+
+   snprintf(path, sizeof(path), "%s/header.lpl", fixture_dir);
+   CHECK(write_whole(path, json_full), "header lane: fixture write");
+   config_defaults(&config, path);
+
+   pl = header_read(&config);
+   CHECK(pl != NULL, "header lane: read");
+   if (pl)
+   {
+      CHECK(playlist_size(pl) == 0, "header lane: %u entries read",
+            (unsigned)playlist_size(pl));
+      CHECK(streq(playlist_get_default_core_name(pl), "Core A"),
+            "header lane: default core name");
+      CHECK(streq(playlist_get_scan_content_dir(pl), "/games/snes"),
+            "header lane: scan content dir");
+      playlist_free(pl);
+   }
+
+   /* a base moved since the save: autofix's move, in memory only */
+   before = slurp(path, &blen);
+   playlist_config_set_base_content_directory(&config, "/moved");
+   pl = header_read(&config);
+   /* the suffix in the local form, as autofix converts it */
+   CHECK(pl && streq(playlist_get_scan_content_dir(pl),
+#ifdef _WIN32
+            "/moved\\snes"
+#else
+            "/moved/snes"
+#endif
+            ), "header lane: scan content dir under the moved base");
+   if (pl)
+   {
+      /* and never written, even when asked */
+      playlist_set_default_core_name(pl, "Core Changed");
+      playlist_write_file(pl);
+      playlist_free(pl);
+   }
+   after = slurp(path, &alen);
+   CHECK(before && after && blen == alen && !memcmp(before, after, (size_t)blen),
+         "header lane: the file changed under a header read");
+   free(before);
+   free(after);
+
+   CHECK(write_whole(path, old_format), "header lane: old format write");
+   config_defaults(&config, path);
+   pl = header_read(&config);
+   CHECK(pl && playlist_size(pl) == 0
+         && !playlist_get_scan_content_dir(pl),
+         "header lane: the old format reads empty");
+   playlist_free(pl);
+
+   if (failures == had)
+      fprintf(stderr, "[pass] header-only lane\n");
+}
+
 static void lane_cache_reuse(void)
 {
    char path[512];
@@ -1660,6 +1864,8 @@ int main(int argc, char *argv[])
    lane_budgeted_autofix();
    lane_abort_midway();
    lane_cache_reuse();
+   lane_cache_worker_write();
+   lane_header_only();
    lane_switch_playlist_never_shows_previous();
    lane_switch_supersedes_pending();
    lane_saf_slow_reads();

samples/tasks/database/Makefile

@@ -250,8 +250,11 @@ $(INDEX_TEST)$(EXE_EXT): $(INDEX_TEST_OBJECTS)
 # count and refuse reader pushes, count queue worker starts, count an
 # archive's stored-CRC and directory reads, fail a chosen serial's
 # query, order the database and content lists, count reader passes,
-# hold chosen lookups, count an archive's member hits a step and hold
-# a chosen index build in one step.
+# hold chosen lookups, count an archive's member hits a step, hold
+# a chosen index build in one step, tell which thread the refresh
+# walks and playlist loads ran on, fail a chosen task_init, fail the
+# playlist folder walk, and count the file system calls the main
+# thread makes.
 SCAN_TEST_WRAPS := -Wl,--wrap=libretrodb_open -Wl,--wrap=task_queue_park \
 	-Wl,--wrap=intfstream_crc_step -Wl,--wrap=rm3u_is_m3u \
 	-Wl,--wrap=task_nbio_slice_open -Wl,--wrap=task_nbio_slice_close \
@@ -265,7 +268,12 @@ SCAN_TEST_WRAPS := -Wl,--wrap=libretrodb_open -Wl,--wrap=task_queue_park \
 	-Wl,--wrap=database_info_dir_init_from_list \
 	-Wl,--wrap=database_info_index_job_step \
 	-Wl,--wrap=task_nbio_slice_within_budget \
-	-Wl,--wrap=fill_pathname_join_delim
+	-Wl,--wrap=fill_pathname_join_delim \
+	-Wl,--wrap=dir_list_iter_new -Wl,--wrap=dir_list_iter_step \
+	-Wl,--wrap=playlist_parse_begin_header -Wl,--wrap=task_init \
+	-Wl,--wrap=path_stat -Wl,--wrap=path_is_valid \
+	-Wl,--wrap=path_is_directory -Wl,--wrap=path_get_size \
+	-Wl,--wrap=path_resolve_realpath
 ifeq ($(HAVE_THREADS), 1)
 SCAN_TEST_WRAPS += -Wl,--wrap=sthread_create
 endif

samples/tasks/database/main.c

@@ -240,7 +240,7 @@ int main(int argc, char *argv[])
 
    /* The return value matters: false means no task exists, so nothing
     * will ever call main_db_cb and the loop below would never end. */
-   if (!task_push_dbscan(input_dir, main_db_cb))
+   if (!task_push_dbscan(input_dir, main_db_cb, NULL))
    {
       fprintf(stderr, "task_push_dbscan refused to start a scan\n");
       goto done;

samples/tasks/database/scan_begin_budget_test.c

@@ -527,7 +527,7 @@ static bool run_scan(const char *content_dir, const char *dat_path,
       return false;
 
    t0 = cpu_features_get_time_usec();
-   if (!task_push_manual_content_scan(false, scan_cb))
+   if (!task_push_manual_content_scan(false, scan_cb, NULL))
    {
       fprintf(stderr, "task_push_manual_content_scan refused\n");
       return false;
@@ -628,7 +628,7 @@ static bool run_db_scan(const char *content_dir, const char *db_dir,
       return false;
 
    t0 = cpu_features_get_time_usec();
-   if (!task_push_manual_content_scan(false, scan_cb))
+   if (!task_push_manual_content_scan(false, scan_cb, NULL))
    {
       fprintf(stderr, "database scan: push refused\n");
       return false;
@@ -693,7 +693,7 @@ static bool run_scan_threaded(const char *content_dir, const char *dat_path,
 
    task_queue_init(true, msgq_push);
 
-   if (!task_push_manual_content_scan(false, scan_cb))
+   if (!task_push_manual_content_scan(false, scan_cb, NULL))
    {
       fprintf(stderr, "threaded: task_push_manual_content_scan refused\n");
       task_queue_deinit();
@@ -742,7 +742,7 @@ static bool run_scan_cancelled(const char *content_dir, const char *dat_path,
 
    task_queue_init(false, msgq_push);
 
-   if (!task_push_manual_content_scan(false, scan_cb))
+   if (!task_push_manual_content_scan(false, scan_cb, NULL))
    {
       fprintf(stderr, "cancel lane: push refused\n");
       task_queue_deinit();

tasks/tasks_internal.h

@@ -336,16 +336,90 @@ bool task_push_image_load_set(const char *const *paths,
  * task. Safe from a task_queue_find predicate. */
 bool task_image_set_tag(retro_task_t *task, uint64_t *tag);
 
+/* A content scan's outcome, handed to its callback as task_data and
+ * valid only during the call.  Counts cover every playlist the scan
+ * writes; errors are files it could not read. */
+typedef struct
+{
+   unsigned added;
+   unsigned duplicates;
+   unsigned unmatched;
+   unsigned errors;
+   bool     failed;
+} task_dbscan_result_t;
+
 #ifdef HAVE_LIBRETRODB
 /* Scans @fullpath, a directory or a single file, against the content
  * databases; the database and playlist directories come from the
- * settings. */
-bool task_push_dbscan(const char *fullpath, retro_task_callback_t cb);
+ * settings.  @cb gets @user_data.  False while a chain runs. */
+bool task_push_dbscan(const char *fullpath, retro_task_callback_t cb,
+      void *user_data);
 #endif
 
+struct manual_content_scan_overrides;
+
+enum task_dbscan_item_kind
+{
+   TASK_DBSCAN_ITEM_PATH = 0,     /* --scan */
+   TASK_DBSCAN_ITEM_REFRESH,      /* --scan-refresh */
+   TASK_DBSCAN_ITEM_REFRESH_ALL   /* --scan-refresh-all, Refresh All */
+};
+
+typedef struct
+{
+   char *arg;  /* as typed; NULL for REFRESH_ALL */
+   struct manual_content_scan_overrides *overrides; /* NULL: none */
+   enum task_dbscan_item_kind kind;
+} task_dbscan_item_t;
+
+enum task_dbscan_outcome
+{
+   TASK_DBSCAN_RAN = 0,   /* res holds the counts */
+   TASK_DBSCAN_FAILED,
+   TASK_DBSCAN_SKIPPED
+};
+
+typedef struct
+{
+   unsigned ran;     /* scans that ran and did not fail */
+   unsigned added;   /* of those, the ones that added an entry */
+   unsigned failed;
+   unsigned skipped;
+   bool     cancelled;
+} task_dbscan_chain_result_t;
+
+typedef void (*task_dbscan_item_cb_t)(const char *name,
+      enum task_dbscan_outcome outcome,
+      const task_dbscan_result_t *res, void *user_data);
+typedef void (*task_dbscan_chain_cb_t)(
+      const task_dbscan_chain_result_t *res, void *user_data);
+
+/* Runs @items in order, one task at a time: each step is pushed from
+ * the last one's callback, and a cancelled task ends the run.  The
+ * chain owns @items from the call, whatever it returns.  @item_cb: once
+ * per scan, failed item or skipped playlist; @done_cb: once at the end,
+ * from a task callback.  @res is valid during the call only.  False,
+ * calling neither, if out of memory or while a chain or a manual scan
+ * runs.  Main thread only. */
+bool task_push_dbscan_chain(task_dbscan_item_t *items, size_t count,
+      task_dbscan_item_cb_t item_cb, task_dbscan_chain_cb_t done_cb,
+      void *user_data);
+
+/* Main thread only. */
+bool task_dbscan_chain_is_running(void);
+
+/* False while task_push_dbscan_chain() would refuse: a chain or a
+ * manual scan runs.  Main thread only. */
+bool task_dbscan_chain_can_start(void);
+/* A scan asked for while a chain runs would use the settings the chain
+ * last loaded: true, with the message shown, when it must be refused. */
+bool task_dbscan_chain_refuse(void);
+/* The same for a new chain, which also waits for a running scan. */
+bool task_dbscan_chain_refuse_start(void);
+
 bool task_push_manual_content_scan(
       bool do_menu_refresh,
-      retro_task_callback_t user_cb);
+      retro_task_callback_t user_cb, void *user_data);
 
 #ifdef HAVE_OVERLAY
 bool task_push_overlay_load_default(

tools/thread_read_audit.py

@@ -292,7 +292,7 @@ def enclosing_function(text, pos):
     return text[start if start >= 0 else 0:end if end >= 0 else len(text)]
 
 
-def source_entries(root, main_thread=None):
+def source_entries(root, main_thread=None, forwarded=None):
     entries = {}
     # handler name -> [path, every assignment flagged main-thread]
     handlers = {}
@@ -320,6 +320,14 @@ def source_entries(root, main_thread=None):
             entries.setdefault(m.group(1), rel)
         clears = MAIN_CLEAR_RE.search(text) is not None
         for m in HANDLER_RE.finditer(text):
+            # A handler set from a retro_task_handler_t parameter names
+            # no function, and the handlers its callers pass are never
+            # entries: the audit cannot see them, so it says so.
+            if re.search(r"\bretro_task_handler_t\s+%s\b"
+                         % re.escape(m.group(1)), text):
+                if forwarded is not None:
+                    forwarded.append((rel, m.group(1)))
+                continue
             main = (not clears and MAIN_SET_RE.search(
                 enclosing_function(text, m.start())) is not None)
             site = handlers.setdefault(m.group(1), [rel, True])
@@ -417,7 +425,8 @@ def load_allow(path):
 def run(binary, root, allow_path, list_unaudited=False,
         objdump="objdump"):
     main_thread = {}
-    entries = source_entries(root, main_thread)
+    forwarded = []
+    entries = source_entries(root, main_thread, forwarded)
     try:
         out = subprocess.run([objdump, "-d", binary],
                              capture_output=True, text=True, check=True)
@@ -436,6 +445,12 @@ def run(binary, root, allow_path, list_unaudited=False,
         print("tools/thread_read_allow.list.")
         for entry, fn, reader in findings:
             print("%s -> %s calls %s" % (entry, fn, reader))
+    if forwarded:
+        print("%d task handler(s) set from a parameter, so the handlers"
+              % len(forwarded))
+        print("passed in are not audited. Assign each by name:")
+        for rel, name in forwarded:
+            print("  %s: task->handler = %s" % (rel, name))
     if list_unaudited:
         absent = sorted(e for e in entries if e not in defined)
         print("%d source entr%s absent from this binary - compiled out"
@@ -452,7 +467,7 @@ def run(binary, root, allow_path, list_unaudited=False,
              len(boundaries), "y" if len(boundaries) == 1 else "ies",
              len(main_thread)),
           file=sys.stderr)
-    return 1 if findings else 0
+    return 1 if findings or forwarded else 0
 
 
 FIXTURE = """
@@ -520,11 +535,22 @@ def selftest():
                     "   task_set_flags(task, RETRO_TASK_FLG_MAIN_THREAD,"
                     " false);\n"
                     "}\n")
+        # A helper that sets whatever handler it is given hides it.
+        with open(os.path.join(td, "fwd.c"), "w") as f:
+            f.write("static void push_any(retro_task_t *task,\n"
+                    "      retro_task_handler_t handler)\n"
+                    "{\n"
+                    "   task->handler = handler;\n"
+                    "}\n")
         if subprocess.run(["gcc", "-O0", src, "-o", binp,
                            "-lpthread"]).returncode:
             print("selftest: fixture build failed")
             return 1
-        entries = source_entries(td)
+        forwarded = []
+        entries = source_entries(td, None, forwarded)
+        if forwarded != [("fwd.c", "handler")] or "handler" in entries:
+            print("selftest: FAIL forwarded handler %r" % forwarded)
+            return 1
         out = subprocess.run(["objdump", "-d", binp],
                              capture_output=True, text=True, check=True)
         calls, defined = call_graph(out.stdout.splitlines())

ui/companion/companion_core.c

@@ -81,8 +81,9 @@ static void companion_core_download_orphan(companion_core_t *core);
 #endif
 
 #ifdef HAVE_LIBRETRODB
-/* task_push_dbscan() carries no user data, so the requesting core is
- * kept here. Only one desktop companion is active at a time. */
+/* The requesting core, kept here rather than passed as the scan's
+ * user data so that freeing the core clears it. Only one desktop
+ * companion is active at a time. */
 static companion_core_t *companion_core_scan_owner = NULL;
 #endif
 
@@ -1848,6 +1849,15 @@ static void companion_core_scan_finished(retro_task_t *task,
 }
 #endif
 
+const char *companion_core_scan_refusal(void)
+{
+#ifdef HAVE_LIBRETRODB
+   if (task_dbscan_chain_is_running())
+      return msg_hash_to_str(MSG_MANUAL_CONTENT_SCAN_ALREADY_RUNNING);
+#endif
+   return "Scanning is not available in this build.";
+}
+
 bool companion_core_request_scan(companion_core_t *core, const char *path)
 {
 #ifdef HAVE_LIBRETRODB
@@ -1855,7 +1865,7 @@ bool companion_core_request_scan(companion_core_t *core, const char *path)
       return false;
 
    companion_core_scan_owner = core;
-   return task_push_dbscan(path, companion_core_scan_finished);
+   return task_push_dbscan(path, companion_core_scan_finished, NULL);
 #else
    (void)core;
    (void)path;

ui/companion/companion_core.h

@@ -272,6 +272,8 @@ void companion_core_event_command(companion_core_t *core,
  * list has been reset. Returns false when no scan could be started
  * (no libretrodb in this build, or the task could not be pushed). */
 bool companion_core_request_scan(companion_core_t *core, const char *path);
+/* Why companion_core_request_scan() refused, for the status line. */
+const char *companion_core_scan_refusal(void);
 
 /* --- Companion settings (retroarch.cfg) ------------------------------- */
 

ui/companion/test/companion_core_stubs.c

@@ -209,14 +209,18 @@ bool task_push_load_content_with_current_core_from_companion_ui(
 
 bool task_push_start_current_core(content_ctx_info_t *ci) { (void)ci; stub_calls_start_core++; return true; }
 
-bool task_push_dbscan(const char *path, retro_task_callback_t cb)
+bool task_push_dbscan(const char *path, retro_task_callback_t cb,
+      void *user_data)
 {
    (void)cb;
+   (void)user_data;
    stub_calls_dbscan++;
    strlcpy(stub_last_scan_dir, path ? path : "", sizeof(stub_last_scan_dir));
    return true;
 }
 
+bool task_dbscan_chain_is_running(void) { return false; }
+
 /* Entries and macros (input_driver.c): kept by number as their text. An
  * entry reads when it has a colon, a macro when it has a digit, as a
  * stand-in for the real parsers; sixteen of each at most. */

ui/drivers/ui_cocoa_companion.m

@@ -4227,7 +4227,7 @@ static const char *cc_thumb_subdir(int t)
    if (companion_core_request_scan(wimp->core, [path UTF8String]))
       [self setStatus:"Scanning..."];
    else
-      [self setStatus:"Scanning is not available in this build."];
+      [self setStatus:companion_core_scan_refusal()];
 }
 
 /* The companion's own Load Core window (like Qt's LoadCoreWindow):

ui/drivers/ui_cocoatouch.m

@@ -1507,7 +1507,7 @@ bool cocoa_audio_session_begin_record(unsigned preferred_rate,
                   sel_registerName("stopAccessingSecurityScopedResource"));
       }
    }
-   task_push_dbscan(fullpath, NULL);
+   task_push_dbscan(fullpath, NULL, NULL);
    return true;
 }
 

ui/drivers/ui_win32_companion.c

@@ -4606,7 +4606,7 @@ static void cw_scan_directory(ui_companion_win32_wimp_t *w)
    if (companion_core_request_scan(w->core, dir))
       cw_status_set(w, "Scanning...");
    else
-      cw_status_set(w, "Scanning is not available in this build.");
+      cw_status_set(w, companion_core_scan_refusal());
 }
 
 static void cw_context_menu(ui_companion_win32_wimp_t *w, HWND from,