From d8f73d3553e5b7420d970983ac70f90fa950c8a0 Mon Sep 17 00:00:00 2001 From: Joe Mattiello Date: Fri, 30 Oct 2026 00:30:01 -0400 Subject: [PATCH] perf(dsp): idle skip admits Iron Soldier 2's wait loop (#952) (#1121) * perf(dsp): idle skip admits Iron Soldier 2's wait loop (#952) Iron Soldier 2's DSP spends nearly all of its time in a 12-word loop at $F1B0A6 that waits for the I2S interrupt handler to move a buffer pointer in the alternate register bank: nop ; addq #1,r19 ; movei #$F1B730,r14 ; movefa r28,r28 sub r31,r28 ; jr NN,+2 ; sharq #2,r28 ; addq #31,r28 moveq #0,r30 ; cmpq #8,r28 ; jr C,head ; nop The idle skip rejected it three ways over; each is now admitted, no wider than needed: - Reach. The probe only ran for a closing jr at most 7 words back. It now covers the jr's full reach (15 words). - An internal forward jr (`jr NN,+2`). A body may hold ONE jr whose target is a decoded instruction after its delay slot and at most the closing jr, with a delay slot that is neither movei nor jr, and no exit jr alongside it. That gives exactly two routes: not taken (1 taken jr, F = idleBodyCount + 1 opcodes) or taken (2 taken jrs, F - K, K = instructions skipped). Both probe iterations must show the same pair, and it must be one of these. Any detour has to fall through the closing jr and come back, costing at least 4 more than the legal route with the same taken-jr count (leaving can only add taken jrs), so it can never match. The dataflow walk still checks the skippable instructions and forgets what they wrote at the join. - A counter on the flag-setting addq (`addq #1,r19`). A counter may now use addq/subq onto itself when its flags are dead: overwritten in full (Z, N and C, per the dsp_opcode_* bodies) on every route before anything reads a flag. Flags no instruction reads cannot carry the counter's growth into a branch. Probing every jr up to 15 words back means loops that never qualify reach the probe more often (Doom +0.25% host instructions per frame), so the probe now remembers the reject-memo entry it hit or added last and returns at once, with no probe in flight, when the same loop comes round again -- exactly what the memo scan would conclude. Doom's cost is then +0.11%. dsp_idle_probe_falsify gains six programs: E/E2 (Iron Soldier 2's loop word for word, internal jr taken and not taken) fire and still leave when the polled register moves; F (counter Z read by the closing jr), G (counter carry read by addc), H (taken internal jr, fall-through, undecoded store and a jump back, matching F opcodes with two taken jrs) and I (exit and internal jr together) must not. With the taken-jr pairing, the dead-flag rule or the exit/internal ban removed, H, F+G and I fire respectively. Co-Authored-By: Claude Opus 5.5 * fix(dsp): dead-flag scan stops before the closing jr's delay slot (#952) The closing jr has no entry of its own in idleBodyIdx, but its delay slot is the last entry, so dsp_idle_flags_dead() walked into that slot without seeing the branch read the flags first. A flag killer there (`addq #1,r19 ; jr NE,head ; cmpq #0,r0`) made the counter's Z look dead, and the skip extrapolated straight past the loop's exit. The scan now stops short of the delay slot. dsp_idle_probe_falsify gains program J, that loop at its own address (F's reject-memo entry would otherwise mask it); it fires without this fix and is rejected with it. Reported by the Codex review on #1121. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- src/jerry/dsp.c | 239 +++++++++++++++++++++---- src/jerry/dsp_exec_untimed.h | 9 +- test/tools/dsp_idle_probe_falsify.c | 266 +++++++++++++++++++++++++++- 3 files changed, 471 insertions(+), 43 deletions(-) diff --git a/src/jerry/dsp.c b/src/jerry/dsp.c index 361814f2..b065ae40 100644 --- a/src/jerry/dsp.c +++ b/src/jerry/dsp.c @@ -1774,7 +1774,8 @@ void DSPSyncToM68K(void) * ---- ADMISSION RULE ------------------------------------------------ * * Candidate: a TAKEN backward (or self-targeting) `jr` whose target is - * at most 8 words behind, with the whole body inside DSP local RAM. + * at most 15 words behind (the jr's full reach), with the whole body + * inside DSP local RAM. * Body = target .. jr inclusive, plus the inlined delay-slot * instruction. Static decode (register-independent) requires: * - every opcode on the whitelist below (no store of any kind, no @@ -1786,8 +1787,15 @@ void DSPSyncToM68K(void) * body straight-line; the probe proves it was not taken (see the * EXIT-BRANCH CHECK in DSPIdleLoopProbe) and the fixed-point proof * below then proves it never will be; - * - the decode walk lands exactly on the jr, so the body really is - * straight-line (movei carries a 32-bit immediate, i.e. 3 words); + * - or ONE internal forward jr instead of exits (#952, Iron Soldier 2: + * `sub r31,r28 ; jr NN,+2 ; sharq #2,r28 ; addq #31,r28`), whose + * target is an instruction after its delay slot and at most the + * closing jr, and whose delay slot is neither a movei nor a jr. It + * splits the body into two routes, which the probe tells apart (see + * the INTERNAL-BRANCH CHECK); the fixed-point proof below makes it + * go the same way on every iteration; + * - the decode walk lands exactly on the jr, so every instruction is + * decoded (movei carries a 32-bit immediate, i.e. 3 words); * - no movei in the delay slot (its immediate would be fetched from * past the branch). * Register-dependent checks are deliberately deferred to the third @@ -1812,20 +1820,23 @@ void DSPSyncToM68K(void) * as the delta of `cycles`, never recomputed from * dsp_opcode_cycles[] -- the inlined delay slot is not charged); * - likewise the measured dsp_exec_opcode_count delta; - * - that measured opcode delta equals idleBodyCount + 1 EXACTLY, which - * is the count only a straight-line traversal of the decoded body - * can produce -- this is the executed-path check, see the comment at - * the test itself; + * - that measured opcode delta, together with the number of taken jrs + * per iteration, matches a route through the decoded body EXACTLY + * (idleBodyCount + 1 with one taken jr; with an internal jr, also + * K fewer with two) -- this is the executed-path check, see the + * comment at the test itself; * - every register with a NONZERO per-iteration delta is read and * written only by an addqt/subqt whose destination is that same - * register. + * register, or by an addq/subq onto itself whose flags are dead: + * fully overwritten on every route before anything reads a flag + * (dsp_idle_flags_dead). * * That last rule is what makes the branch unable to flip. It is * strictly stronger than "the last flag-setting instruction reads only - * zero-delta registers": addqt/subqt are the only admitted opcodes that - * write no flags (dsp_opcode_addqt/subqt above), so any flag-setting - * instruction is forced to read zero-delta operands, AND so is every - * load base and every shift/rotate count. Proof that the loop is then + * zero-delta registers": addqt/subqt write no flags, and a dead-flag + * addq/subq writes flags no instruction ever reads, so every flag value + * that IS read comes from zero-delta operands, AND so does every load + * base and every shift/rotate count. Proof that the loop is then * affine forever: memory is constant by the theorem, so with all * zero-delta registers holding their S1 values and the flags holding * their S1 values, every instruction that does not touch a counter @@ -1835,7 +1846,8 @@ void DSPSyncToM68K(void) * S1 + (k-1)*delta for all k, and the branch condition never changes. * The same holds at every point INSIDE the body, so an exit jr sees the * same flags on every iteration: not taken from S1 to S2 (the probe - * checks that) means not taken on any extrapolated iteration either. + * checks that) means not taken on any extrapolated iteration either, and + * an internal jr goes the same way as it did from S1 to S2. * * Then n = (cycles_remaining / cost) - 1 (always leave one full * iteration to execute normally), reg += n*delta, cycles -= n*cost, @@ -1850,12 +1862,13 @@ void DSPSyncToM68K(void) * state is re-derived from scratch inside each DSPExec call. */ -/* Longest body accepted, in 16-bit words (jr offset range [-8,-1]). - * Belt-and-braces only: the caller's `pcThis - dsp_pc <= 14` filter - * already caps [head, jr) at 7 words, so this bound never binds. */ -#define DSP_IDLE_MAX_BODY 8 -/* Body instruction slots: <= 7 before the jr, plus the delay slot. */ -#define DSP_IDLE_MAX_INSN 10 +/* Longest body accepted, in instructions before the closing jr. The + * caller's `pcThis - dsp_pc <= 30` filter is the jr's full backward reach + * (offset -16), which caps [head, jr) at 15 words (#952: Iron Soldier 2's + * wait loop is 12), so this bound never binds. */ +#define DSP_IDLE_MAX_BODY 15 +/* Body instruction slots: <= 15 before the jr, plus the delay slot. */ +#define DSP_IDLE_MAX_INSN 16 /* Per-slice reject memo: without it a loop that fails the fixed-point * test would be re-probed every three iterations for the whole slice. */ #define DSP_IDLE_MEMO 16 @@ -1910,11 +1923,23 @@ static uint8_t idleBodyP1[DSP_IDLE_MAX_INSN]; static uint8_t idleBodyP2[DSP_IDLE_MAX_INSN]; static uint32_t idleBodyImm[DSP_IDLE_MAX_INSN]; static int idleBodyCount; +/* The body's one INTERNAL forward jr (#952), or -1: idleIntJr is its body + * index, and [idleIntFrom, idleIntTo) the body indices it jumps over when + * taken (its delay slot, idleIntJr + 1, runs on both routes). */ +static int idleIntJr; +static int idleIntFrom, idleIntTo; static uint32_t idleMemoHead[DSP_IDLE_MEMO]; static uint32_t idleMemoJr[DSP_IDLE_MEMO]; static int idleMemoCount; static int idleMemoNext; +/* The memo entry hit or added last. Only trusted while idleMemoCount is + * nonzero: it is written together with a memo insert or hit, so after the + * per-slice memo reset it cannot name a loop until the memo holds that + * loop again. Lets the common case -- the same rejected loop coming + * round again -- skip the scan (#952: probing every jr up to 15 words + * back cost Doom +0.25% host instructions without it). */ +static uint32_t idleMemoLastHead, idleMemoLastJr; /* Opcode whitelist. Admit only what has been reasoned about; anything * absent is rejected, including every store (45/46/47/49/50/60/61), the @@ -1948,9 +1973,10 @@ static int dsp_idle_op_admitted(uint32_t idx) * PC-modifying opcode but the loop-closing jr itself is load-bearing * for the executed-path check below (idleBodyCount forces exactly * one route from head to jrAddr), not only for the store-freedom - * this whitelist was originally written to guarantee. The one - * exception, a forward exit jr, is admitted by dsp_idle_decode - * itself and backed by the probe's EXIT-BRANCH CHECK. */ + * this whitelist was originally written to guarantee. The two + * exceptions, forward exit jrs and one internal forward jr, are + * admitted by dsp_idle_decode itself and backed by the probe's + * EXIT-BRANCH CHECK. */ default: return 0; } @@ -2039,7 +2065,11 @@ static int dsp_idle_memo_hit(uint32_t head, uint32_t jr) for (i = 0; i < idleMemoCount; i++) if (idleMemoHead[i] == head && idleMemoJr[i] == jr) + { + idleMemoLastHead = head; + idleMemoLastJr = jr; return 1; + } return 0; } @@ -2049,6 +2079,8 @@ static void dsp_idle_memo_reject(uint32_t head, uint32_t jr) return; idleMemoHead[idleMemoNext] = head; idleMemoJr[idleMemoNext] = jr; + idleMemoLastHead = head; + idleMemoLastJr = jr; idleMemoNext = (idleMemoNext + 1) % DSP_IDLE_MEMO; if (idleMemoCount < DSP_IDLE_MEMO) idleMemoCount++; @@ -2059,11 +2091,14 @@ static void dsp_idle_memo_reject(uint32_t head, uint32_t jr) static int dsp_idle_decode(uint32_t head, uint32_t jrAddr) { uint32_t pc = head; + uint32_t pcAt[DSP_IDLE_MAX_INSN]; + uint32_t intTarget = 0; uint16_t op; uint32_t idx; - int n = 0; + int n = 0, exits = 0, j; idleBodyCount = 0; + idleIntJr = -1; while (pc < jrAddr) { @@ -2073,17 +2108,31 @@ static int dsp_idle_decode(uint32_t head, uint32_t jrAddr) idx = (uint32_t)(op >> 10); if (idx == 53) { - /* Exit branch: only a jr that leaves the loop entirely, past - * the closing jr's delay slot. Anything landing inside - * [head, jr+2] would make a second route around the loop. */ + /* A jr before the closing one is either an EXIT branch, + * landing past the closing jr's delay slot, or the body's one + * INTERNAL forward branch (#952, Iron Soldier 2), landing on + * an instruction after its own delay slot and no later than + * the closing jr. Anything else -- backward, onto its own + * delay slot or onto the closing jr's -- would make a route + * around the loop the probe cannot account for. */ int32_t off = (int32_t)((op >> 5) & 0x1F); + uint32_t target; if (off & 0x10) off -= 0x20; - if (pc + 2 + (uint32_t)(off * 2) <= jrAddr + 2) + target = pc + 2 + (uint32_t)(off * 2); + if (target > jrAddr + 2) + exits++; + else if (idleIntJr < 0 && target >= pc + 6 && target <= jrAddr) + { + idleIntJr = n; + intTarget = target; + } + else return 0; } else if (!dsp_idle_op_admitted(idx)) return 0; + pcAt[n] = pc; idleBodyIdx[n] = (uint8_t)idx; idleBodyP1[n] = (uint8_t)((op >> 5) & 0x1F); idleBodyP2[n] = (uint8_t)(op & 0x1F); @@ -2104,6 +2153,32 @@ static int dsp_idle_decode(uint32_t head, uint32_t jrAddr) if (pc != jrAddr) /* decode fell out of step */ return 0; + if (idleIntJr >= 0) + { + /* Exit and internal branches together would let a taken exit + * plus a `jump` back show the same taken-jr count as a taken + * internal skip, which is what the probe tells them apart by. */ + if (exits) + return 0; + /* Its delay slot must be in the body and be neither a movei + * (immediate fetched past the branch) nor another jr. */ + if (idleIntJr + 1 >= n || idleBodyIdx[idleIntJr + 1] == 38 + || idleBodyIdx[idleIntJr + 1] == 53) + return 0; + /* The target must start an instruction, not a movei immediate; + * the closing jr itself counts as index n (the join then falls + * just before its delay slot, which is the same thing). */ + idleIntFrom = idleIntJr + 2; + idleIntTo = -1; + if (intTarget == jrAddr) + idleIntTo = n; + for (j = idleIntFrom; j < n; j++) + if (pcAt[j] == intTarget) + idleIntTo = j; + if (idleIntTo <= idleIntFrom) /* not a start, or skips nothing */ + return 0; + } + /* The inlined delay slot at jr+2 runs on every iteration too. A * movei there would fetch its immediate from past the branch, so * reject it rather than model it. */ @@ -2121,6 +2196,56 @@ static int dsp_idle_decode(uint32_t head, uint32_t jrAddr) return 1; } +/* Is the Z/N/C written by body instruction j dead -- overwritten in full + * on every route before any instruction reads a flag? Lets a counter + * use the flag-setting addq/subq (#952: Iron Soldier 2's `addq #1,r19`, + * whose flags `sub r31,r28` replaces two instructions later), since flags + * that are never read cannot carry the counter's growth anywhere. + * + * Flag behavior is taken from the dsp_opcode_* bodies, not assumed: + * - write all of Z, N and C without reading a flag ("killers"): + * add addq sub subq neg abs shlq shrq sharq ror rorq cmp cmpq; + * - read no flag, so the scan passes them: addqt subqt (write none), + * and or xor not bset bclr (Z and N only), btst (Z only), the moves, + * movei, the loads and nop; + * - everything else ends the scan as a reader: the internal jr, + * addc / subc (read C), and sat16s, treated as a reader rather than + * audited. + * The closing jr reads the flags too. It has no entry of its own in + * idleBodyIdx, but its delay slot is the last entry, so the scan stops + * short of that slot: a killer there runs after the branch has already + * read the counter's flags, and the scan never wraps. + * A killer inside the internal jr's skippable region does not count for + * an instruction ahead of that region: the taken route never runs it. */ +static int dsp_idle_flags_dead(int j) +{ + int k, overRegion = (idleIntJr >= 0 && j < idleIntFrom); + + for (k = j + 1; k < idleBodyCount - 1; k++) + { + if (k == idleIntJr) + return 0; + switch (idleBodyIdx[k]) + { + case 0: case 2: case 4: case 6: case 8: + case 22: case 24: case 25: case 27: + case 28: case 29: case 30: case 31: + if (overRegion && k >= idleIntFrom && k < idleIntTo) + break; + return 1; + case 3: case 7: case 9: case 10: case 11: + case 12: case 13: case 14: case 15: + case 34: case 35: case 36: case 37: case 38: + case 39: case 40: case 41: case 43: case 44: + case 57: case 58: case 59: + break; + default: + return 0; + } + } + return 0; +} + /* Register-dependent half of the admission rule, run at S2 where the * head state is provably steady. Walks the body tracking each * register's known value so a load base written earlier in the same body @@ -2130,8 +2255,9 @@ static int dsp_idle_check_body(const uint32_t *delta, int cur, int alt) { uint32_t kval[64]; uint8_t kok[64]; + uint64_t skipped = 0; /* registers written in the skippable region */ int src[3]; - int nsrc, dst, s, j, i; + int nsrc, dst, s, j, i, counter; uint32_t idx, p1, p2, ea; for (i = 0; i < 32; i++) @@ -2148,20 +2274,34 @@ static int dsp_idle_check_body(const uint32_t *delta, int cur, int alt) p1 = idleBodyP1[j]; p2 = idleBodyP2[j]; + /* Join after the internal jr's skippable region: the taken route + * did not run it, so nothing it wrote is known any more. Its own + * instructions are still checked below as if they ran -- the + * not-taken route does run them. */ + if (j == idleIntTo && idleIntJr >= 0) + for (i = 0; i < 64; i++) + if (skipped & ((uint64_t)1 << i)) + kok[i] = 0; + if (!dsp_idle_operands(idx, p1, p2, cur, alt, src, &nsrc, &dst)) return 0; + if (dst >= 0 && idleIntJr >= 0 && j >= idleIntFrom && j < idleIntTo) + skipped |= (uint64_t)1 << dst; /* A register whose per-iteration delta is nonzero is a pure * counter: it may only be read, and only be written, by an * addqt/subqt targeting itself. Those two are the only * admitted opcodes that touch no flag, so this forbids a * growing value from reaching a compare, a load address, a - * shift count or anything else. */ + * shift count or anything else. addq/subq onto itself also + * qualify when the flags they write are dead + * (dsp_idle_flags_dead, #952). */ + counter = (idx == 3 || idx == 7) + || ((idx == 2 || idx == 6) && dsp_idle_flags_dead(j)); for (s = 0; s < nsrc; s++) - if (delta[src[s]] != 0 - && !((idx == 3 || idx == 7) && src[s] == dst)) + if (delta[src[s]] != 0 && !(counter && src[s] == dst)) return 0; - if (dst >= 0 && delta[dst] != 0 && !(idx == 3 || idx == 7)) + if (dst >= 0 && delta[dst] != 0 && !counter) return 0; /* Loads: the effective address must be provably plain RAM. */ @@ -2271,9 +2411,15 @@ static int32_t DSPIdleLoopProbe(int32_t cycles, uint32_t head, uint32_t jrAddr) { uint32_t delta[64]; int32_t cost, n; - uint32_t opcost; + uint32_t opcost, jrt; int cur, alt, i; + /* The same rejected loop again, with no probe in flight: exactly what + * the memo check below would conclude, minus the scan. */ + if (idleProbeStage == 0 && idleMemoCount != 0 + && head == idleMemoLastHead && jrAddr == idleMemoLastJr) + return cycles; + /* Whole body -- including the delay slot and its trailing byte -- * must sit in DSP local SRAM, so every fetch is a pure dsp_ram_8 * read and the PC-escape check in DSPExec is a no-op for it. */ @@ -2430,10 +2576,29 @@ static int32_t DSPIdleLoopProbe(int32_t cycles, uint32_t head, uint32_t jrAddr) * program D is such a route. So also require that each probe * iteration took exactly ONE jr, the closing one: a taken exit is a * taken jr and would make it two. With every exit untaken, the body - * is straight-line again and the count argument above applies. */ - if (opcost != (uint32_t)idleBodyCount + 1 - || idleProbeJrt1 - idleProbeJrt0 != 1 - || dsp_jr_taken - idleProbeJrt1 != 1) + * is straight-line again and the count argument above applies. + * + * INTERNAL-BRANCH CHECK (#952). With the body's one internal jr + * (exits then excluded by the decoder) there are exactly two routes, + * F = idleBodyCount + 1: + * internal not taken: 1 taken jr, F opcodes + * internal taken: 2 taken jrs, F - K opcodes + * (K = idleIntTo - idleIntFrom; the delay slot is charged once either + * way, by DSPExec's loop or inline by dsp_opcode_jr). Both iterations + * must show the same pair, and it must be one of these two. Any + * other route has to fall through the closing jr (the only way out + * without a taken jr), so it runs one full pass (at least F - K, or F + * with the internal untaken), a transfer back plus its delay slot + * (+2), and the closing jr plus its slot again (+2) -- and leaving can + * only ADD taken jrs. So a detour showing 1 taken jr took no internal + * branch and costs >= F + 4, and one showing 2 costs >= F - K + 4: + * never equal to the legal pair with the same count. */ + jrt = idleProbeJrt1 - idleProbeJrt0; + if (dsp_jr_taken - idleProbeJrt1 != jrt + || !((jrt == 1 && opcost == (uint32_t)idleBodyCount + 1) + || (jrt == 2 && idleIntJr >= 0 + && opcost == (uint32_t)(idleBodyCount + 1 + - (idleIntTo - idleIntFrom))))) { dsp_idle_memo_reject(head, jrAddr); return cycles; diff --git a/src/jerry/dsp_exec_untimed.h b/src/jerry/dsp_exec_untimed.h index 665517c4..bc0ada5b 100644 --- a/src/jerry/dsp_exec_untimed.h +++ b/src/jerry/dsp_exec_untimed.h @@ -257,11 +257,12 @@ static DSP_NOINLINE void DSP_UNTIMED_NAME(int32_t cycles) } /* Idle-loop fast-forward (issue #569). A taken `jr` that landed - * on or behind its own address, at most 8 words back, is the only - * candidate; the unsigned difference rejects a not-taken jr - * (pc = pcThis + 2) and every forward branch in one compare. */ + * on or behind its own address, up to its full reach of 15 words + * back (#952), is the only candidate; the unsigned difference + * rejects a not-taken jr (pc = pcThis + 2) and every forward + * branch in one compare. */ if (idleSkipActive && index == 53 - && (uint32_t)(pcThis - dsp_pc) <= 14) + && (uint32_t)(pcThis - dsp_pc) <= 30) cycles = DSPIdleLoopProbe(cycles, dsp_pc, pcThis); /* Age out a D_FLAGS store once the instruction that was already diff --git a/test/tools/dsp_idle_probe_falsify.c b/test/tools/dsp_idle_probe_falsify.c index 056cf9e9..3035ae45 100644 --- a/test/tools/dsp_idle_probe_falsify.c +++ b/test/tools/dsp_idle_probe_falsify.c @@ -54,6 +54,55 @@ * at every arrival, so only the exit-branch check (one taken jr per * iteration) rejects it. * + * E. Iron Soldier 2's DSP wait loop, word for word (#952) -> the + * probe MUST fire, and the loop must leave once the polled + * alternate-bank register moves: + * head: nop ; addq #1,r19 ; movei #$F1B730,r14 ; movefa r28,r28 + * sub r31,r28 ; jr NN,+2 ; sharq #2,r28 ; addq #31,r28 + * moveq #0,r30 ; cmpq #8,r28 ; jr C,head ; nop + * Three relaxations at once: a 12-word body, an INTERNAL forward jr + * (taken here: 2 taken jrs, one opcode fewer), and a counter written + * by the flag-setting addq, whose flags `sub` overwrites unread. + * E2 is the same loop with the internal jr NOT taken (1 taken jr). + * + * F. a counter whose flags ARE read -> MUST NOT fire. + * head: addq #1,r19 ; jr NE,head ; nop (r19 from $FFFF0000) + * The loop ends when r19 wraps to 0; extrapolating r19 would jump + * straight over the exit. The wrap lies beyond this test's cycle + * budget, so the loop never reaches its exit park loop (which is + * idle and would fire itself): any fire is this loop's. + * + * G. a counter whose carry is read -> MUST NOT fire. + * head: moveq #0,r2 ; addq #1,r19 ; addc r0,r2 ; jr EQ,head ; nop + * r2 picks up addq's carry, so the loop ends when r19 wraps; every + * register addc reads has zero delta, so only the dead-flag rule + * (addc reads C) rejects it. + * + * H. the internal-branch exploit -> MUST NOT fire. + * head: moveq #0,r1 ; or r1,r1 ; jr always,+5 ; nop + * nop x4 <- skipped, K = 4 + * jr: jr NE,head ; or r5,r5 <- (delay slot) r5 = 1 + * jump (r6) ; store r3,(r4) <- NEVER DECODED, r6 = jr + * The internal jr is always taken, so the closing jr arrives with + * Z set and falls through; its delay slot clears Z, the jump runs an + * undecoded store in ITS delay slot and lands back on the closing + * jr, now taken. Two taken jrs and F = idleBodyCount + 1 opcodes: + * exactly the not-taken route's count, so only pairing the opcode + * count with the taken-jr count (2 taken means F - K) rejects it. + * + * I. an exit jr and an internal jr in one body -> MUST NOT fire. + * head: btst #0,r22 ; jr NE,out ; nop ; jr EQ,+2 ; nop ; nop + * jr always,head ; nop + * A taken exit plus a `jump` back would show the same taken-jr + * count as a taken internal skip, so the decoder refuses the mix. + * + * J. F with a flag killer in the closing jr's delay slot -> MUST NOT + * fire. + * head: addq #1,r19 ; jr NE,head ; cmpq #0,r0 (r19 from $FFFF0000) + * The jr reads addq's Z before its delay slot overwrites it, so the + * dead-flag scan must stop short of that slot. At its own address: + * F's reject-memo entry would otherwise mask it. + * * Build: cc -O2 -Wall -std=c99 $(INCFLAGS) -o test/tools/dsp_idle_probe_falsify \ * test/tools/dsp_idle_probe_falsify.c test/harness/harness.c -ldl -lm * Needs the wide test ABI (make TEST_EXPORTS=1). Uses test/roms/yarc.j64, @@ -68,14 +117,19 @@ #include "../harness/harness.h" /* RISC opcode indices (src/jerry/dsp.c dsp_opcode[]) */ +#define OP_ADDC 1 +#define OP_ADDQ 2 #define OP_ADDQT 3 #define OP_SUBQ 6 #define OP_BTST 13 +#define OP_CMPQ 31 #define OP_STORE 47 #define OP_JUMP 52 #define OP_JR 53 #define OP_NOP 57 #define OP_MOVEI 38 +#define OP_MOVEQ 35 +#define OP_OR 10 /* Branch condition codes: 0 = always, 1 = NE (fails when Z is set). */ #define CC_ALWAYS 0 @@ -90,6 +144,13 @@ #define PROG_C_OUT 0x0320 /* $F1B320 -- C's exit target */ #define SCRATCH_OFF 0x0400 /* $F1B400 -- the exploit's store target */ #define PROG_D_OFF 0x0500 /* $F1B500 */ +#define PROG_E_OFF 0x0600 /* $F1B600 */ +#define PROG_E_OUT 0x061C /* $F1B61C -- E's exit park loop */ +#define PROG_F_OFF 0x0700 /* $F1B700 */ +#define PROG_J_OFF 0x0740 /* $F1B740 */ +#define PROG_G_OFF 0x0780 /* $F1B780 */ +#define PROG_H_OFF 0x0880 /* $F1B880 */ +#define PROG_I_OFF 0x0800 /* $F1B800 */ static uint8_t *dsp_ram; static uint32_t *p_dsp_pc; @@ -189,6 +250,104 @@ static void build_prog_d(void) setreg(6, DSP_RAM_BASE + PROG_D_OFF + 14); /* jump target = closing jr */ } +/* Park loop for a program's exit path, so the PC stays readable. */ +static void put_park(uint32_t off) +{ + put16(off + 0, enc(OP_ADDQT, 1, 7)); /* addqt #1,r7 */ + put16(off + 2, enc(OP_JR, 0x1E /* -2 */, CC_ALWAYS)); + put16(off + 4, enc(OP_NOP, 0, 0)); +} + +static void build_prog_e(uint32_t alt_r28) +{ + /* The words of Iron Soldier 2 (World) at $F1B0A6-$F1B0C0. */ + static const uint16_t is2[] = { + 0xE400, /* nop */ + 0x0833, /* addq #1,r19 */ + 0x980E, 0xB730, 0x00F1, /* movei #$00F1B730,r14 */ + 0x979C, /* movefa r28,r28 */ + 0x13FC, /* sub r31,r28 */ + 0xD454, /* jr NN,+2 */ + 0x6C5C, /* sharq #2,r28 */ + 0x0BFC, /* addq #31,r28 */ + 0x8C1E, /* moveq #0,r30 */ + 0x7D1C, /* cmpq #8,r28 */ + 0xD668, /* jr C,-13 (head) */ + 0xE400 /* nop (delay slot) */ + }; + unsigned i; + + for (i = 0; i < sizeof is2 / sizeof is2[0]; i++) + put16(PROG_E_OFF + 2 * i, is2[i]); + put_park(PROG_E_OUT); + + setreg(19, 0); + setreg(31, 0x40); + /* movefa reads the ALTERNATE bank's r28, the one the IS2 interrupt + * handler maintains; setreg writes both, and the loop only ever + * overwrites the current bank's copy. */ + setreg(28, alt_r28); +} + +/* F at `off` with `slot` in the closing jr's delay slot (J: cmpq). */ +static void build_prog_f(uint32_t off, uint16_t slot) +{ + put16(off + 0, enc(OP_ADDQ, 1, 19)); /* addq #1,r19 */ + put16(off + 2, enc(OP_JR, 0x1E /* -2 */, CC_NE)); /* jr ne,head */ + put16(off + 4, slot); + put_park(off + 6); + + setreg(19, 0xFFFF0000u); +} + +static void build_prog_g(void) +{ + put16(PROG_G_OFF + 0, enc(OP_MOVEQ, 0, 2)); /* moveq #0,r2 */ + put16(PROG_G_OFF + 2, enc(OP_ADDQ, 1, 19)); /* addq #1,r19 */ + put16(PROG_G_OFF + 4, enc(OP_ADDC, 0, 2)); /* addc r0,r2 */ + put16(PROG_G_OFF + 6, enc(OP_JR, 0x1C /* -4 */, CC_EQ)); /* jr eq,head */ + put16(PROG_G_OFF + 8, enc(OP_NOP, 0, 0)); + put_park(PROG_G_OFF + 10); + + setreg(0, 0); + setreg(19, 0xFFFF0000u); +} + +static void build_prog_h(void) +{ + unsigned i; + + put16(PROG_H_OFF + 0, enc(OP_MOVEQ, 0, 1)); /* moveq #0,r1 */ + put16(PROG_H_OFF + 2, enc(OP_OR, 1, 1)); /* or r1,r1: Z=1 */ + put16(PROG_H_OFF + 4, enc(OP_JR, 5, CC_ALWAYS)); /* jr +5 (internal) */ + for (i = 6; i <= 14; i += 2) /* delay slot + 4 */ + put16(PROG_H_OFF + i, enc(OP_NOP, 0, 0)); + put16(PROG_H_OFF + 16, enc(OP_JR, 0x17 /* -9 */, CC_NE)); /* jr ne,head */ + put16(PROG_H_OFF + 18, enc(OP_OR, 5, 5)); /* (ds) or r5,r5 */ + put16(PROG_H_OFF + 20, enc(OP_JUMP, 6, CC_ALWAYS)); /* jump (r6) */ + put16(PROG_H_OFF + 22, enc(OP_STORE, 4, 3)); /* (ds) store */ + + setreg(3, 0xDEADBEEFu); + setreg(4, DSP_RAM_BASE + SCRATCH_OFF); + setreg(5, 1); + setreg(6, DSP_RAM_BASE + PROG_H_OFF + 16); /* jump target = closing jr */ +} + +static void build_prog_i(void) +{ + put16(PROG_I_OFF + 0, enc(OP_BTST, 0, 22)); /* btst #0,r22 */ + put16(PROG_I_OFF + 2, enc(OP_JR, 10, CC_NE)); /* jr ne,out */ + put16(PROG_I_OFF + 4, enc(OP_NOP, 0, 0)); + put16(PROG_I_OFF + 6, enc(OP_JR, 2, CC_EQ)); /* jr eq,+2 (internal) */ + put16(PROG_I_OFF + 8, enc(OP_NOP, 0, 0)); /* (delay slot) */ + put16(PROG_I_OFF + 10, enc(OP_NOP, 0, 0)); /* skipped */ + put16(PROG_I_OFF + 12, enc(OP_JR, 0x19 /* -7 */, CC_ALWAYS)); /* jr head */ + put16(PROG_I_OFF + 14, enc(OP_NOP, 0, 0)); /* (delay slot) */ + put_park(PROG_I_OFF + 24); /* out */ + + setreg(22, 0); +} + /* Run one hand-assembled program from `off` for `cycles`, returning how * many times the fast-forward fired. */ static uint32_t run_prog(uint32_t off, int32_t cycles, @@ -210,13 +369,19 @@ static uint32_t run_prog(uint32_t off, int32_t cycles, int main(int argc, char **argv) { harness_config cfg = HARNESS_CONFIG_DEFAULT; - harness_result res[6]; + harness_result res[14]; unsigned nres = 0; uint32_t fires_a, fires_b, rej_a, rej_b, ops_a, ops_b; uint32_t fires_c, fires_d, rej_c, rej_d, ops_c, ops_d, pc_c; + uint32_t fires_e, fires_e2, fires_f, fires_g, fires_h, fires_i, pc_e; + uint32_t fires_j, rej_e, rej_e2, rej_f, rej_g, rej_h, rej_i, rej_j; static char detail_a[192], detail_b[192], detail_c[192]; static char detail_d[192], detail_e[192], detail_f[192]; + static char detail_g[192], detail_h[192], detail_i[192]; + static char detail_j[192], detail_k[192], detail_l[192]; + static char detail_m[192], detail_n[192]; int ok_a, ok_b, ok_c, ok_d, ok_e; + int ok_f, ok_g, ok_h, ok_i, ok_j, ok_k, ok_l, ok_m; cfg.frames = 2; cfg.quiet = 1; @@ -259,6 +424,27 @@ int main(int argc, char **argv) pc_c = *p_dsp_pc; build_prog_d(); fires_d = run_prog(PROG_D_OFF, 20000, &rej_d, &ops_d); + /* E: alt r28 $50 -> r28 = $10 >= 0, internal jr taken, $10>>2 = 4 < 8. */ + build_prog_e(0x50); + fires_e = run_prog(PROG_E_OFF, 20000, &rej_e, NULL); + /* The interrupt handler's pointer moves: $60 -> $20>>2 = 8, not < 8. */ + setreg(28, 0x60); + p_dspexec(200); + pc_e = *p_dsp_pc; + /* E2: alt r28 $40-100 -> r28 = -100 < 0, internal jr not taken, + * -100>>2 + 31 = 6 < 8. */ + build_prog_e(0x40u - 100u); + fires_e2 = run_prog(PROG_E_OFF, 20000, &rej_e2, NULL); + build_prog_f(PROG_F_OFF, enc(OP_NOP, 0, 0)); + fires_f = run_prog(PROG_F_OFF, 20000, &rej_f, NULL); + build_prog_f(PROG_J_OFF, enc(OP_CMPQ, 0, 0)); /* cmpq #0,r0 */ + fires_j = run_prog(PROG_J_OFF, 20000, &rej_j, NULL); + build_prog_g(); + fires_g = run_prog(PROG_G_OFF, 20000, &rej_g, NULL); + build_prog_h(); + fires_h = run_prog(PROG_H_OFF, 20000, &rej_h, NULL); + build_prog_i(); + fires_i = run_prog(PROG_I_OFF, 20000, &rej_i, NULL); ok_a = (fires_a > 0); ok_b = (fires_b == 0); @@ -266,6 +452,15 @@ int main(int argc, char **argv) ok_d = (pc_c >= DSP_RAM_BASE + PROG_C_OUT && pc_c <= DSP_RAM_BASE + PROG_C_OUT + 4); ok_e = (fires_d == 0); + ok_f = (fires_e > 0); + ok_g = (pc_e >= DSP_RAM_BASE + PROG_E_OUT + && pc_e <= DSP_RAM_BASE + PROG_E_OUT + 4); + ok_h = (fires_e2 > 0); + ok_i = (fires_f == 0); + ok_j = (fires_g == 0); + ok_k = (fires_i == 0); + ok_l = (fires_h == 0); + ok_m = (fires_j == 0); snprintf(detail_a, sizeof detail_a, "genuinely idle loop (addqt/jr/nop) fired %u time(s), " @@ -318,7 +513,74 @@ int main(int argc, char **argv) res[nres].detail = detail_f; nres++; + snprintf(detail_g, sizeof detail_g, + "Iron Soldier 2 loop, internal jr taken, fired %u time(s), " + "%u reject(s)", fires_e, rej_e); + res[nres].status = ok_f ? "PASS" : "FAIL"; + res[nres].name = "internal_branch_loop_fires"; + res[nres].detail = detail_g; + nres++; + + snprintf(detail_h, sizeof detail_h, + "after alt r28 moved, DSP PC $%06X (must be in $%06X..$%06X)", + pc_e, DSP_RAM_BASE + PROG_E_OUT, DSP_RAM_BASE + PROG_E_OUT + 4); + res[nres].status = ok_g ? "PASS" : "FAIL"; + res[nres].name = "internal_branch_loop_still_exits"; + res[nres].detail = detail_h; + nres++; + + snprintf(detail_i, sizeof detail_i, + "Iron Soldier 2 loop, internal jr not taken, fired %u time(s), " + "%u reject(s)", fires_e2, rej_e2); + res[nres].status = ok_h ? "PASS" : "FAIL"; + res[nres].name = "internal_branch_untaken_fires"; + res[nres].detail = detail_i; + nres++; + + snprintf(detail_j, sizeof detail_j, + "counter addq whose Z the closing jr reads fired %u time(s) " + "(must be 0), %u reject(s)", fires_f, rej_f); + res[nres].status = ok_i ? "PASS" : "FAIL"; + res[nres].name = "live_flag_counter_rejected"; + res[nres].detail = detail_j; + nres++; + + snprintf(detail_k, sizeof detail_k, + "counter addq whose carry addc reads fired %u time(s) " + "(must be 0), %u reject(s)", fires_g, rej_g); + res[nres].status = ok_j ? "PASS" : "FAIL"; + res[nres].name = "live_carry_counter_rejected"; + res[nres].detail = detail_k; + nres++; + + snprintf(detail_n, sizeof detail_n, + "counter addq whose Z the closing jr reads before its delay " + "slot's cmpq fired %u time(s) (must be 0), %u reject(s)", + fires_j, rej_j); + res[nres].status = ok_m ? "PASS" : "FAIL"; + res[nres].name = "delay_slot_flag_kill_rejected"; + res[nres].detail = detail_n; + nres++; + + snprintf(detail_m, sizeof detail_m, + "taken internal jr + fall-through + undecoded store + jump " + "back, F opcodes with 2 taken jrs, fired %u time(s) (must be 0), " + "%u reject(s)", fires_h, rej_h); + res[nres].status = ok_l ? "PASS" : "FAIL"; + res[nres].name = "internal_branch_detour_rejected"; + res[nres].detail = detail_m; + nres++; + + snprintf(detail_l, sizeof detail_l, + "body with an exit jr and an internal jr fired %u time(s) " + "(must be 0), %u reject(s)", fires_i, rej_i); + res[nres].status = ok_k ? "PASS" : "FAIL"; + res[nres].name = "exit_plus_internal_rejected"; + res[nres].detail = detail_l; + nres++; + harness_report(&cfg, res, nres); harness_shutdown(&cfg); - return (ok_a && ok_b && ok_c && ok_d && ok_e) ? 0 : 1; + return (ok_a && ok_b && ok_c && ok_d && ok_e + && ok_f && ok_g && ok_h && ok_i && ok_j && ok_k && ok_l && ok_m) ? 0 : 1; }