fix(test): cart matrix must never publish a missing core as 123 dead ROMs

A sweep run against a missing core wrote LOAD_FAIL for 123 of 154 titles.
Nothing was wrong with those ROMs -- the dylib was absent (an ABI-mode
switch deletes it, and a stray iOS-built .o had made the relink fail), so
every worker dlopen-failed and the classifier read 'no CARTPROBE line' as
'the ROM would not load'. Rows are cached per build id, so the next
invocation reused the poisoned rows instead of re-running.

Three coupled faults, all fixed:

1. CORE fell back to ./virtualjaguar_libretro.so when the dylib was
   missing, without checking that the .so existed either. Now exits with
   a build hint if neither is present.
2. classify() attributed a dlopen failure to the title. It now emits
   '? (core_error)' -- a harness fault, not a title result -- so a broken
   core can no longer manufacture cacheable LOAD_FAIL rows.
3. The preflight's core check was gated behind 'if a CARTPROBE line
   exists', so a first ROM that legitimately fails to load (an alpha dump,
   in this case Alien vs Predator (Alpha), which is first alphabetically)
   skipped the whole check. It now fails hard on dlopen, and otherwise
   walks the corpus until some ROM loads, so the sweep never starts on an
   unverified core.

The file's own comment already warned that 'a stale or mismatched core
once turned an entire sweep into 153 LOAD_FAIL rows -- fail loudly and
write nothing instead'. The guard existed; it just had a hole.

Verified: a missing core and a non-mach-o core both abort before any row
is written.
This commit is contained in:
Joseph Mattiello committed 2026-08-13 17:15:49 -04:00
1 parent a556fef699
commit 7e6975fb11
1 file changed
+48
+48
View File
@@ -45,6 +45,16 @@ MAX_RUNS="${CART_MATRIX_MAX_RUNS:-0}"
CORE="${CART_MATRIX_CORE:-./virtualjaguar_libretro.dylib}"
[ -f "$CORE" ] || CORE=./virtualjaguar_libretro.so
# Refuse to start without a core. The fallback above silently selected a
# nonexistent .so when the dylib was missing (an ABI-mode switch deletes it,
# and a stray iOS-built .o makes the relink fail), and every worker then
# dlopen-failed -- which the classifier read as "the ROM would not load" and
# wrote 123 false LOAD_FAIL rows. Fail here instead.
if [ ! -f "$CORE" ]; then
echo "error: no core at ./virtualjaguar_libretro.{dylib,so}" >&2
echo "build one with: make TEST_EXPORTS=1" >&2
exit 1
fi
PROBE=./test/tools/cart_boot_probe
BUILD_ID="$(bash scripts/build-id.sh)"
@@ -114,6 +124,14 @@ classify_mode() {
echo "? (timeout)|no probe line within ${TIMEOUT_SECS}s${sigs:+; $sigs}"
return
fi
# A dlopen failure is a HARNESS fault, not a title result. Writing it as
# LOAD_FAIL is what let a missing core masquerade as 123 unloadable ROMs,
# and because rows are cached per build id, the bad rows were then reused
# by the next invocation. Never let that reach the matrix.
if grep -q 'dlopen(' "$logfile" 2>/dev/null; then
echo "? (core_error)|core failed to load — row invalid, not a title result"
return
fi
if [ -z "$line" ] || printf '%s' "$line" | grep -q 'load_fail=1'; then
echo "LOAD_FAIL|probe could not load the ROM"
return
@@ -236,6 +254,36 @@ if [ "$count" -gt 0 ]; then
echo "rebuild with: make TEST_EXPORTS=1 (and rebuild the probe if it changed)" >&2
exit 1
fi
# The core must actually load. This check used to be gated behind
# "if a CARTPROBE line exists", so a first ROM that legitimately fails to
# load (e.g. an alpha dump) skipped the whole preflight and let a broken
# core through -- exactly how the 123-LOAD_FAIL sweep got started.
if grep -q 'dlopen(' "$preflight_log"; then
echo "error: core failed to dlopen: $CORE" >&2
grep -m1 'dlopen(' "$preflight_log" >&2
echo "rebuild with: make clean && make TEST_EXPORTS=1" >&2
exit 1
fi
# A first ROM that cannot load is not itself an error, but it means the
# preflight proved nothing -- walk forward until one loads, so the sweep is
# never started on an unverified core.
if ! grep -q '^CARTPROBE ' "$preflight_log"; then
probe_ok=0
while read -r cand; do
run_bounded "$TIMEOUT_SECS" "$preflight_log" \
"$PROBE" "$CORE" "$cand" --frames 1
if grep -q 'dlopen(' "$preflight_log"; then
echo "error: core failed to dlopen: $CORE" >&2
exit 1
fi
if grep -q '^CARTPROBE ' "$preflight_log"; then probe_ok=1; break; fi
done < "$FRESH"
if [ "$probe_ok" -eq 0 ]; then
echo "error: no ROM in the corpus produced a CARTPROBE line" >&2
echo "the core or the probe is broken -- refusing to write a matrix" >&2
exit 1
fi
fi
if grep -q '^CARTPROBE ' "$preflight_log" &&
! grep -q ' pc_valid=1 ' "$preflight_log"; then
echo "error: probe could not read m68k_get_reg from the core" >&2