Files
romm/backend/utils/switch.py
T
Georges-Antoine AssiandClaude Opus 5.5 05bf29fe09 feat(converto): read per-file title ids with rom-converto during scans (#5031)
* feat(converto): read per-file title ids with rom-converto during scans

The title-id half of #5002: rom-converto's info runs fill rom_files.title_id
and title_version, sigil fills what rom-converto can't read and keeps owning
save targets, and roms in a scan batch share info runs of up to 64 files.
A converto_read_at marker lets quick scans backfill files rom-converto has
not read yet. converto.scan_metadata (config.yml only) turns it off.

The other per-file fields and embedded images stay out until a UI reads them.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(converto): name the same-content check that keeps unread title ids

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(converto): queue unread files in one pass over the built rows

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(converto): address Greptile review on #5031

- A file refresh now persists converto_read_at when rom-converto's read
  confirms a reused row's stored title id, so the file isn't re-read on
  every refresh.
- RomConvertoService.read_infos splits its paths into info runs of at
  most INFO_BATCH_SIZE, so a rom with many files can't send them all
  through one run.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(converto): apply code-review fixes

- A stored file title id counts as rom-converto's only once rom-converto
  has read the row, so a stale sigil id can't override a fresh extraction.
- Fresh hashes decide whether unread title ids still describe a file,
  ahead of a size and mtime a copy can preserve.
- One rom's failed candidate listing no longer aborts its scan batch.
- The info paths file's cleanup can't raise out of read_infos.
- _rom_file_order sorts by Path like _TitleIdSource.order.
- Restored the in-place reconcile test for _identify_rom.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(converto): apply simplify pass

- sync_rom_files reads 'rom-converto read this pass' off converto_read_at
  instead of an inspected set threaded through ParsedRomFiles, FSRom, the
  scan socket and the refresh.
- One _converto_may_read check serves the scan and the batch candidates.
- converto_candidates reuses _list_rom_dir, validates paths off the event
  loop and is best-effort itself, so the scan socket gathers it plainly.
- get_rom_files builds rom-converto paths from the validated rom dir
  instead of validating each file again.
- _rom_level_identity returns early on sigil's extraction.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* chore(converto): apply review-polish pass

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(converto): drop the shared info batch

Sharing rom-converto info runs across a scan batch saved about 2 ms per
rom (one process start) in exchange for a second directory walk per
rebuilt rom, the RomConvertoInfoBatch bookkeeping and the scan socket
wiring. Each rom now reads its own files; read_infos still caps a run
at INFO_RUN_MAX_FILES.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(converto): name the rom-converto columns inline

Drops ROM_FILE_CONVERTO_COLUMNS and ROM_FILE_CONVERTO_KEPT_COLUMNS, so
_copy_scanned_columns keeps master's frozenset keep_when_unset.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 13:49:40 -04:00

61 lines
2.2 KiB
Python

"""Switch title-id knowledge: recognising a base-game id, recovering one from an
update or DLC id, and mapping a content type to a file category."""
import re
from typing import Final
from models.rom import RomFileCategory, RomIdentity, SaveTargetLayout
# Maps sigil's Switch CNMT content type to the RomFile category. Authoritative
# over folder-derived categories when the binary parse succeeds.
CONTENT_TYPE_CATEGORIES: Final[dict[str, RomFileCategory]] = {
"application": RomFileCategory.GAME,
"patch": RomFileCategory.UPDATE,
"addon": RomFileCategory.DLC,
}
TITLE_ID_REGEX: Final = re.compile(r"[0-9A-Fa-f]{16}")
# An already-embedded id, e.g. [0100F4700B2E0000].
TITLE_ID_BRACKET_REGEX: Final = re.compile(rf"\[{TITLE_ID_REGEX.pattern}\]")
def derive_base_title_id(title_id: str) -> str | None:
"""Derive the base-game title id from an update/DLC id: clear the low 12
bits and decrement the program-index nibble when it is odd."""
if len(title_id) < 4:
return None
nibble_char = title_id[-4]
try:
nibble = int(nibble_char, 16)
except ValueError:
return None
if nibble % 2 == 1:
nibble -= 1
formatted = format(nibble, "x" if nibble_char.islower() else "X")
return f"{title_id[:-4]}{formatted}000"
def is_base_title_id(title_id: str) -> bool:
"""A base-game id is its own derived base."""
return derive_base_title_id(title_id) == title_id
def normalize_identity(is_switch: bool, identity: RomIdentity) -> RomIdentity:
"""Force a Switch identity onto the base game's, leaving others untouched.
Every writer of `Rom.title_id` goes through here: reassociating a renamed
non-hashable ROM matches on that column, which only works while it holds
the base game's id. A non-derivable id is left as read rather than dropped.
"""
title_id = identity.title_id
derived = derive_base_title_id(title_id) if is_switch and title_id else None
if derived is None or (derived == title_id and identity.save_target):
return identity
# Switch saves are keyed by the base title id itself.
return RomIdentity(
title_id=derived,
save_target=derived,
save_target_layout=SaveTargetLayout.FOLDER_EXACT,
)