task_database: ask the core-info gate before walking the database

task_database_iterate_crc_lookup() consulted the size gate before the
core-info gate.  The size gate is not free: the first time a database
is consulted, task_database_fill_db_min_max() walks the whole .rdb to
build its crc index and derive the size range.  The core-info gate is
an in-memory string comparison.  Doing them in that order meant every
database in the directory was read end to end once per scan, whether
or not any installed core claims it - 201 MB of .rdb reads to scan a
71 MB Master System set against the shipped 147-file database
directory, most of it the 47 MB J2ME database that can never match a
.zip of SMS ROMs.

Run the cheap gate first.  Databases no core claims are now skipped
without being opened.

Measured, --scan of the reporter's sets against all 147 rdbs, release
build, same machine:

              before    after
  SMS  (620)  6.76 s    0.19 s
  PCE  (412)  6.70 s    0.16 s
  Lynx  (95)  1.28 s    0.05 s

Playlist output is byte-identical to the pre-change build.  The gain
scales with how many databases the installed cores do not claim; a
setup with every core installed still walks each claimed database
once, as before.  ASan+UBSan+LSan clean.
This commit is contained in:
LibretroAdmin authored and libretroadmin committed 2026-09-10 22:28:56 +02:00
1 parent ee37994dc3
commit fbe3d2b246
1 file changed
+22 -19
+22 -19
View File
@@ -1470,6 +1470,28 @@ static enum scan_verdict task_database_iterate_crc_lookup(
return database_info_list_iterate_next(db_state);
}
/* The core-info gate is an in-memory string check; the size gate
* below costs a full walk of the database the first time it is
* consulted, because task_database_fill_db_min_max() builds the
* crc index in that walk. Running the walk first meant every
* database in the directory was read end to end once per scan,
* including the ones no installed core claims and which can never
* match - 200 MB of .rdb reads to scan a 70 MB set with the
* shipped database directory. Ask the cheap question first. */
if (!(_db->flags & DB_HANDLE_FLAG_SCAN_WITHOUT_CORE_MATCH))
{
if (!core_info_database_supports_content_path(
db_state->list->elems[db_state->list_index].data, name))
return database_info_list_iterate_next(db_state);
if (!path_contains_compressed_file)
{
if (core_info_database_match_archive_member(
db_state->list->elems[db_state->list_index].data))
return database_info_list_iterate_next(db_state);
}
}
/* If size boundaries are not filled for this DB, run the queries */
if (!(db_state->flags[db_state->list_index] & DB_STATE_FLAG_SIZE_CHECKED))
task_database_fill_db_min_max(db_state);
@@ -1520,25 +1542,6 @@ static enum scan_verdict task_database_iterate_crc_lookup(
query[0] = '\0';
if (!(_db->flags & DB_HANDLE_FLAG_SCAN_WITHOUT_CORE_MATCH))
{
/* don't scan files that can't be in this database.
*
* Could be because of:
* - A matching core missing
* - Incompatible file extension */
if (!core_info_database_supports_content_path(
db_state->list->elems[db_state->list_index].data, name))
return database_info_list_iterate_next(db_state);
if (!path_contains_compressed_file)
{
if (core_info_database_match_archive_member(
db_state->list->elems[db_state->list_index].data))
return database_info_list_iterate_next(db_state);
}
}
/* Answer from this database's crc index when we can. Building
* it costs one walk - about what a single probe used to cost -
* and every later content file is then a binary search instead