A flat NAOMI image whose header says "Naomi2" is run as a NAOMI 2, with
the BIOS from naomi2.zip (or .7z). Until now every such image was taken
for a NAOMI 1, and the BIOS answered "this game is not acceptable by
main board" - which is what upstream flycast does with one as well.
What a NAOMI 2 has that a NAOMI does not:
- The geometry processor of the last patch, in area 2: its registers
at 08800000, its command port at 09000000, its 32 MB at 0a000000,
and at 085f6800 and 085f8000 the holly's and the PowerVR's registers
written to both PowerVRs at once (core/hw/pvr/elan_host.cpp). What
it sends goes into the tile accelerator the way a store queue write
does.
- A second PowerVR. The two are one here, as in upstream: one set of
registers, one video memory. What the second has of its own is its
normal interrupt status, at 025f6900: the end of a list and the end
of a render are raised in both, and the game clears each.
- The second PowerVR's video memory is the mirror 32 MB above the
first's, and games write every texture there too. The texture
cache did not watch the mirror ("the mirrors don't seem to be
used"), so a texture changed through it kept its old picture. On a
NAOMI 2 it is watched.
- Frames several times the size of anything a NAOMI sends: the tile
accelerator's buffer and the lists a frame is parsed into are four
times as big on a NAOMI 2, and a buffer kept from another machine
is not reused.
Save states: version V20 adds, on a NAOMI 2 only, the second interrupt
status, the chip's state and memory, and the time its texture transfer
ends. Older states load as before.
"Is this a NAOMI" is asked in a dozen places: SYSTEM_IS_NAOMI() answers
for both.
Tested with a flat image of Virtua Fighter 4 and naomi2.7z, in the
RetroArch nightly of 7 October under Xvfb (Mesa llvmpipe):
- The BIOS starts the game; the attract mode's temple scene and the
demonstration fight are drawn - letterbox, coin text, the stage, the
fighters and their shadows - by the OpenGL and the Vulkan renderer,
per triangle and per pixel, and look alike in all four.
- Against upstream flycast (which lights in vertex shaders), made to
run the same image as a NAOMI 2 by a change of three lines: the same
scenes, the same picture to the eye. The two do not reach a scene on
the same frame, so they were not compared pixel by pixel.
- What the chip sends the tile accelerator over 1500 frames is the
same, byte for byte, from run to run.
- Threaded rendering on and off: 138 pixels of a 1440x1080 frame
differ, where two runs with the same setting differ in 54. Those
are Mesa's: a texture with mirrored repeat comes out differently
from run to run on llvmpipe, and identically with plain repeat.
- A state saved, the game run 50 frames on, the state loaded and the
game run on: 100 frames later its state differs from a straight
run's in no more bytes than two straight runs differ from each
other (about 900 of 92 MB).
- tools/threads/headless.sh, the Dreamcast test disc: passes.
Not done here: the chip's memory is not in the recompiler's direct
mapping, so its accesses go by the handlers; NAOMI 2 games in MAME's
zip form are not in the game table; starting without a BIOS.
Two things a loaded state did to the sound:
- A game starts sounds in two steps: it sets the key-on bit of each
channel it wants, then starts them all with one write. Loading a
state put every channel's envelope back through the function that
changes its state - which, for a channel that is released, as every
idle one is, also clears its key-on bit. A state loaded between the
two steps lost the mark, and the sound never started. The envelope
is put back without that. Upstream loads states the same way.
- The drive's audio status was read back as "playing or not": a drive
that was paused, or had played to its end, came back playing. That
reading is for states from when the status was a flag, and is kept
for those only.
This is what the live test's "sound, with states saved and loaded"
run has been failing on, now and then and lately every time: it saves
and loads by the clock, and whether a save fell between the disc's
two steps depended on how fast the machine ran.
tools/threads/headless.c can now save a state before one frame and
load it before another (HEADLESS_SAVE, HEADLESS_LOAD), and
headless.sh has a sixth run that does, at fixed frames, with a channel
marked and not yet started: the sound from the load on has to be the
first run's from where the state was saved. On the old code it is not
(the marked channel stays silent); with this all six runs pass, and
the live test passes on gl and glcore.
When the frontend put a disc in, the drive had it at once: ready, of
its kind, with "the medium may have changed" already in its sense
data. A real drive spins the disc up and reads its table of contents
first, and games that have the player change discs wait for what it
says while it does. Upstream gave it a second, and lists Skies of
Arcadia, Shenmue II, Alone in the Dark and Dancing Blade as games
whose disc change that fixed.
So now, as upstream has it:
- for a second after a disc goes in, the drive is busy and its sense
data says "not ready, becoming ready" (2, 04h/01h); the kind of disc
it reports is still what it reported with the lid open;
- then it has the disc, and says once "medium may have changed"
(6, 28h), or 29h if there turned out to be nothing to read;
- unit attention sets CHECK in the status of the commands that follow,
which upstream changed for D2's disc change;
- opening the lid no longer puts unit attention in the sense data.
A disc that is there when the game starts is ready at once, as before.
The second is in the save state (V19, eight bytes); a reset or the lid
opening again ends it.
The headless frontend can open and shut the lid (HEADLESS_SWAP), and
the headless test gains a fifth run that does. The disc's program
watches the drive every frame: it has to be busy for 55 to 65 frames,
then ready, then answer REQ_ERROR with 6/28h. With the second cut to
nothing the run fails (bad00c01).
Not tried with any of the games named: none is at hand.
Read against upstream's, which changed all of this for games:
- The answers were written into the game's memory the moment a
transfer was started, and the interrupt came later. A game that
looks before the interrupt found them there already. They are now
kept until the transfer is over and written then, with the
interrupt. (Upstream: "Maple dma xfer must not be executed
immediately. Delay until interrupt is raised.")
- An answer with nowhere to go, its address not in main memory, was
written to some other address made out of it. It is now not
written, and the overrun interrupt is raised when the transfer is
over. (Upstream: fixes Cheats 'N Codes Volume 1.)
- The time a transfer takes was every byte at 2 Mbit/s. Only the
console sends that fast: devices answer at 724 to 738 kbit/s,
measured on the wire, and each frame has its start, parity and stop
as well. Now 2 Mbit/s out and 740 kbit/s back, as upstream has it.
(Upstream: fixes Silent Scope's memory card load at boot. Its commit
also notes that Power Drift then only boots with one controller
connected; that is not looked into here.)
- In the mode where the bus sends the most significant byte first
(SB_MMSEL 0), the last word of a frame was not byte-swapped on its
way to the device.
The pending answers are in the save state, which becomes V18 and
16 KB larger; older states load as before. They are dropped on reset.
The disc gains a test: a controller's answer must not be in memory
right after the transfer is started and must be when it is over. With
answers written at once it fails (bad00901).
The SH4 has two store queues, and each has a register (QACR0, QACR1)
naming the area its 32 bytes are sent to. With the MMU off the core
picked one routine for both queues from the area of whichever of the
two registers was written last, and the general routine read QACR0
for either queue. Games nearly always set the two alike, and then it
makes no difference. One that aims a queue at the Tile Accelerator
and the other at main memory had both go where the register written
second said.
Upstream goes by QACR0 for both, which is right for the first queue
and wrong for the second in the same case.
The routine is now picked from both registers: the quick ones for
main memory and for the TA when the two agree, as before, and the
general one when they do not, which now reads the register of the
queue being flushed. It is picked again when a save state is loaded,
from the registers, not from what the state says was in use. A QACR
reads back as its area bits only, as upstream has it.
The live test's disc has a test of it: the second queue aimed at main
memory, then the first at the TA, 32 bytes through the second, which
have to arrive in memory with nothing either side disturbed. If not,
the background is orange and the verdict word says so. Run headless on
the core as it was, the disc reports exactly that (0xBAD00006); with
this, it passes, under both SH4 timings, and the live test passes.
The sound chip's ARM7 was interpreted on x86-64, one instruction at a
time; only ARM hosts had a recompiler for it. This brings in the one
flycast has upstream for x86-64: arm7_rec.cpp turns the instructions
at the ARM's program counter into a block of simple operations, and
arm7_rec_x64.cpp turns the block into machine code. Instructions it
has no code for - block transfers of several registers, multiplies,
anything odd - are run by the interpreter from inside the block.
The recompiler ARM hosts use is the old one, untouched.
On the Soul Calibur state the ARM7 costs 690 cycles a sample this way
against 1049 interpreted, a third less. The driver's compiled code is
21 KB.
It is not upstream's as it stands. Run against the interpreter on
random programs (tools/arm7, next patch) it stopped on some valid
instructions and got others wrong. Fixed here:
- The two-operand instructions worked on their operands' own
registers, and for some pairings of result and operand overwrote
one before using it (SUB and SBC with the result in the second
operand changed the first operand's register; RSC with the result
in the first operand subtracted the second from itself). With r15
as the first operand most could not be assembled at all and threw.
Both operands are now taken by value before anything is written.
- RSB was done as negate and add, which has another carry.
- A shift by a register took the whole register for the count, where
it is the low byte: a count of 0x102 was "32 or more".
- A flag-setting logical instruction with a shift by a register left
the carry alone; so did one with a rotated constant under 256, and
one with r15 as second operand took its carry from r15. These go
to the interpreter. The carry of a constant was also looked for in
the first operand.
- r15 shifted by 32 (LSR #0, ASR #0) was folded as a shift by 0, and
r15 with a rotate, or with a shift whose carry is wanted, lost the
shift.
- A block load or store of one register is rewritten as a plain load
or store, and wrote the base back whether asked to or not; and the
load turned the word by the address's low bits, as a plain load
does and a block load does not.
- An instruction after a status register write with the same
condition was run or skipped on the flags from before the write.
- Filling the code cache threw. It is flushed instead.
With these, the recompiler and the interpreter leave the same
registers, flags and memory behind on 26,391 random programs.
As upstream: code is compiled as it is first reached and kept until
the ARM is reset or a state is loaded, so a driver that rewrote its
own instructions while running would keep running the old ones; and
interrupts are taken between blocks of up to 32 instructions, not
between instructions.
The registers gain a scratch slot after the last, for the recompiler.
It is not counted with them, so a save state is what it was.
Compiled for Windows x64 and, for the parts shared with the old
recompiler, ARM64; run on Linux x86-64 only.
The sound chip mixes the sector it is playing straight out of a mapped
track file through TrackFile::Lend(), which lends a sector for the
disc's lifetime; a decoded hunk cannot be lent and is still read into
the AICA's buffer. libCore_CDDA_Detach() copies a lent sector into
that buffer before the disc goes away and around a save state, so a
lid opening mid-sector or a state saved then is safe.
The live test's disc now starts its audio track playing, looping, so
the saves, loads, resets and lid cycles all happen on a lent sector.
The palette's conversion to host formats, and the hashes paletted
textures are keyed by, were redone when pal_needs_update said so: a
flag set by the register writes and cleared by palette_update(), which
the emulation thread called when the game started a render.
With Threaded Rendering that is the wrong thread at the wrong time. The
frame before can still be drawing, on the frontend's thread, when the
game starts the next render, and palette_update() rewrote the very
tables that frame's texture decoding was reading. It also converted
from the live registers, which by then may hold the palette of a frame
the renderer is not on yet.
palette_update() is now called by the renderer, for each frame, before
the frame's textures are processed, and reads the copy of the registers
the frame is drawn with; TexCache.cpp reads that copy throughout, as
the renderers do. Whether the tables are stale is decided by comparing
the 4 KB of palette and the control register with what was converted
last, so there is no flag for two threads to share, and a palette
written back unchanged costs no conversion and no upload.
pal_needs_update is gone.
The live test's disc now draws its background with an 8-bit paletted
texture and, after the texture's last repaint, changes the palette
entry it uses to yellow: the yellow screen takes the texture write and
the palette write both. With the conversion taken out the screen is
black and the test fails.
Whether to upload the fog table was decided by fog_needs_update, set by
the register write on the emulation thread and cleared by the renderer.
With Threaded Rendering those are two threads, and a frame is rendered
from a copy of the registers taken when the game started it. A table
written after that copy was taken set the flag; the renderer, still on
the earlier frame, uploaded the old table from its copy and cleared the
flag. The new table was then never uploaded, until the game wrote the
table again.
The table is 256 bytes as a texture. The renderers now build those
bytes from the registers the frame is rendered with, every frame, and
upload them when they differ from what the GPU has. There is no flag,
so nothing to lose, and a game that rewrites the same table every frame
no longer has it uploaded every frame. OpenGL, OpenGL per-pixel and
Vulkan all do it this way; fog_needs_update is gone.
register_serialize() wrote data32 out for every register. For one that
is read through a function that field is the low half of the function's
address, which is not state - register_unserialize() reads it into a
dummy - and is different in every process. About 1,300 of them went into
each state, so two states of the same machine at the same moment were
never the same file.
A register read through a function is now written as zero. The format
does not change and older states load as before.
With this and the read-only registers no longer holding an address
either, two runs of the same content for the same number of frames
leave states that differ in the real-time clock and nothing else: 4
places instead of 2,500 bytes. tools/threads/live_same.py now requires
that - more than a few bytes of difference between two identical runs
fails - and still compares the threaded state with the non-threaded one
everywhere but the clock.
retro_unserialize() called CalculateSync() after dc_unserialize(). Besides
recomputing the derived video timing, CalculateSync() also reset
pvr_cur_scanline to 0 and re-armed the vblank scheduler event at
now + Line_Cycles, discarding the raster position and vblank schedule that
had just been restored from the state.
Because the running instance never does this, a saved-then-reloaded run
diverges from an uninterrupted one: the vblank event fires up to one
scanline early right after the load, shifting frame timing and cascading
into a full audio/CPU desync. It is most visible on the reios (HLE BIOS)
path, where the attract loop polls GD-ROM and is sensitive to frame
timing; a real BIOS happened to hide it. This breaks runahead and netplay
determinism when no BIOS is present.
Add a reschedule flag to CalculateSync() (default true, so register-write
and reset callers are unchanged) and pass false on the unserialize path so
only the derived timing is recomputed. Serialize pvr_cur_scanline as well,
bumping the savestate version to V15, so the raster line round-trips.
Pre-V15 states still load and keep the previous reset-to-0 behaviour.
Two host-clock dependencies in the emulation path:
QueueRender() had a wall-clock speed limiter under SynchronousRendering: it
measured emulated-cycles-per-real-second via os_GetSeconds() and waited if the
SH4 was running faster than real time. A libretro core must not throttle itself
against the host clock -- the frontend paces via audio/video sync -- and doing
so is non-deterministic and fights that pacing. Drop the wall-clock measurement;
keep only the render-to-texture wait (a queued RTT frame must finish before the
SH4 reads it back), which is a correctness sync, not a timer.
The AICA RTC increments deterministically off the SH4 scheduler (DreamcastSecond
every SH4_MAIN_CLOCK cycles), but its initial value is seeded from the host
clock and RealTimeClock was never serialized, so two netplay peers -- or a
runahead rollback -- diverged the moment a game read the clock. Serialize it
(V14) so it is synced with the rest of the state. The host-clock seed is kept:
it is the correct behaviour for a real-time clock, and is now deterministic
across the state transfer that netplay and runahead both perform.
Left in place (not emulation timing): log timestamps, the vendored Vulkan
allocator's QueryPerformanceCounter, the dead #if 0 fastmmu benchmark, the SH4
guest-side GetTickCount/QPC address decode, the NAOMI multiboard network thread's
pacing sleep, rumble duration (output-only), and the threaded serialize/reset
handshake timeouts.
retro_unserialize() receives the frontend buffer length but dc_unserialize()
only ever used it to disambiguate two legacy save-state versions, never to
bound the reads. ra_unserialize() did an unconditional memcpy from the input
cursor, so a truncated or malformed state (netplay peer, runahead, corrupt
file) walked straight off the end of the input buffer, and several length
fields taken verbatim from the state were used as memcpy destinations:
- UnserializeTAContext: 'size' copied into the fixed TA_DATA_SIZE buffer and
'render_pass_count' used to index the fixed render_passes[10] array.
- sys_nvmem_sram/flash: stored .size overwrote the live size and drove the
copy length into the fixed data[] allocation.
Track the input base/limit in dc_unserialize() and reject any read that would
run past the end, latching a failure that propagates to the retro_unserialize()
return value. Clamp the attacker-controlled TA size, render_pass_count, and
nvmem copy lengths to their real destination capacities. Legitimate states
round-trip byte-for-byte unchanged.
set SB_ADST to 1 when dma is in progress. Fixes Carrier audio glitches
(Issue #927)
move aica and rtc scheduler callbacks to aica
implement internal aica dma
slow mmu: missing instruction tlb protection violation. don't flush mmu
tables when changing state
sh4: sr.RB isn't set to 0 in user mode, simply ignored
various non-functional changes
Better IP.BIN parsing (wince, vga)
Use wince flag and boot filename for wince detection
Switch to TV composite if vga unsupported
Switch to supported BIOS region if current is unsupported or default
Add default ISP information to flashrom
Initialize the emulator in retro_load_game() instead of in its thread or
retro_run(). If init fails, an error is returned to the frontend.
Add more data to save states, V3 introduced.
Initialize modem at init to have a fixed size scheduler list at runtime
Fixed init race conditions when using auto load state.
Don't serialize sh4 scheduler ids. Don't depend on their value for ser/deser.
LibretroAdmin