mirror of
https://github.com/rommapp/romm.git
synced 2026-10-12 02:21:58 +00:00
* feat(converto): read per-file title ids with rom-converto during scans The title-id half of #5002: rom-converto's info runs fill rom_files.title_id and title_version, sigil fills what rom-converto can't read and keeps owning save targets, and roms in a scan batch share info runs of up to 64 files. A converto_read_at marker lets quick scans backfill files rom-converto has not read yet. converto.scan_metadata (config.yml only) turns it off. The other per-file fields and embedded images stay out until a UI reads them. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(converto): name the same-content check that keeps unread title ids Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(converto): queue unread files in one pass over the built rows Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(converto): address Greptile review on #5031 - A file refresh now persists converto_read_at when rom-converto's read confirms a reused row's stored title id, so the file isn't re-read on every refresh. - RomConvertoService.read_infos splits its paths into info runs of at most INFO_BATCH_SIZE, so a rom with many files can't send them all through one run. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(converto): apply code-review fixes - A stored file title id counts as rom-converto's only once rom-converto has read the row, so a stale sigil id can't override a fresh extraction. - Fresh hashes decide whether unread title ids still describe a file, ahead of a size and mtime a copy can preserve. - One rom's failed candidate listing no longer aborts its scan batch. - The info paths file's cleanup can't raise out of read_infos. - _rom_file_order sorts by Path like _TitleIdSource.order. - Restored the in-place reconcile test for _identify_rom. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(converto): apply simplify pass - sync_rom_files reads 'rom-converto read this pass' off converto_read_at instead of an inspected set threaded through ParsedRomFiles, FSRom, the scan socket and the refresh. - One _converto_may_read check serves the scan and the batch candidates. - converto_candidates reuses _list_rom_dir, validates paths off the event loop and is best-effort itself, so the scan socket gathers it plainly. - get_rom_files builds rom-converto paths from the validated rom dir instead of validating each file again. - _rom_level_identity returns early on sigil's extraction. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * chore(converto): apply review-polish pass Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(converto): drop the shared info batch Sharing rom-converto info runs across a scan batch saved about 2 ms per rom (one process start) in exchange for a second directory walk per rebuilt rom, the RomConvertoInfoBatch bookkeeping and the scan socket wiring. Each rom now reads its own files; read_infos still caps a run at INFO_RUN_MAX_FILES. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(converto): name the rom-converto columns inline Drops ROM_FILE_CONVERTO_COLUMNS and ROM_FILE_CONVERTO_KEPT_COLUMNS, so _copy_scanned_columns keeps master's frozenset keep_when_unset. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
61 lines
2.2 KiB
Python
61 lines
2.2 KiB
Python
"""Switch title-id knowledge: recognising a base-game id, recovering one from an
|
|
update or DLC id, and mapping a content type to a file category."""
|
|
|
|
import re
|
|
from typing import Final
|
|
|
|
from models.rom import RomFileCategory, RomIdentity, SaveTargetLayout
|
|
|
|
# Maps sigil's Switch CNMT content type to the RomFile category. Authoritative
|
|
# over folder-derived categories when the binary parse succeeds.
|
|
CONTENT_TYPE_CATEGORIES: Final[dict[str, RomFileCategory]] = {
|
|
"application": RomFileCategory.GAME,
|
|
"patch": RomFileCategory.UPDATE,
|
|
"addon": RomFileCategory.DLC,
|
|
}
|
|
|
|
TITLE_ID_REGEX: Final = re.compile(r"[0-9A-Fa-f]{16}")
|
|
# An already-embedded id, e.g. [0100F4700B2E0000].
|
|
TITLE_ID_BRACKET_REGEX: Final = re.compile(rf"\[{TITLE_ID_REGEX.pattern}\]")
|
|
|
|
|
|
def derive_base_title_id(title_id: str) -> str | None:
|
|
"""Derive the base-game title id from an update/DLC id: clear the low 12
|
|
bits and decrement the program-index nibble when it is odd."""
|
|
if len(title_id) < 4:
|
|
return None
|
|
nibble_char = title_id[-4]
|
|
try:
|
|
nibble = int(nibble_char, 16)
|
|
except ValueError:
|
|
return None
|
|
if nibble % 2 == 1:
|
|
nibble -= 1
|
|
formatted = format(nibble, "x" if nibble_char.islower() else "X")
|
|
return f"{title_id[:-4]}{formatted}000"
|
|
|
|
|
|
def is_base_title_id(title_id: str) -> bool:
|
|
"""A base-game id is its own derived base."""
|
|
return derive_base_title_id(title_id) == title_id
|
|
|
|
|
|
def normalize_identity(is_switch: bool, identity: RomIdentity) -> RomIdentity:
|
|
"""Force a Switch identity onto the base game's, leaving others untouched.
|
|
|
|
Every writer of `Rom.title_id` goes through here: reassociating a renamed
|
|
non-hashable ROM matches on that column, which only works while it holds
|
|
the base game's id. A non-derivable id is left as read rather than dropped.
|
|
"""
|
|
title_id = identity.title_id
|
|
derived = derive_base_title_id(title_id) if is_switch and title_id else None
|
|
if derived is None or (derived == title_id and identity.save_target):
|
|
return identity
|
|
|
|
# Switch saves are keyed by the base title id itself.
|
|
return RomIdentity(
|
|
title_id=derived,
|
|
save_target=derived,
|
|
save_target_layout=SaveTargetLayout.FOLDER_EXACT,
|
|
)
|