The vblank clock was shared under an slock: the kernel-event thread
took it once per vblank to feed the clock, and the frame thread took it
two or three times a frame to read the beam. Uncontended that costs
little, but it lets the frame thread wait behind the time-critical
clock thread, and the house pattern for a one-writer snapshot read by
another thread is a seqlock - video_driver.c's frame cache, the
viewport, the shader sensor uniforms.
The clock has exactly one writer at a time: the kernel-event thread
while it runs, the frame thread otherwise (the DWM and DirectDraw
sources, and every reset). Every source switch stops the thread before
the frame thread writes and starts it after, and thread start and join
order that handover, so the writer's clock is plain memory. What the
frame thread reads is a published copy:
- vblank_clock_publish() / vblank_clock_snapshot(), in vblank_clock.h:
a stamp made odd around four relaxed 64-bit and int stores - the
period and nominal rate as the bits of their doubles - and a reader
that retries against a publish in progress up to 64 times, then
takes "no clock this once". The same shape and bound as the frame
cache's. The writer never waits and the reader never blocks.
- Available where retro_atomic is lock-free and 64 bits wide, which
both MinGW targets are - on 32-bit x86 the 64-bit fields ride
cmpxchg8b, which every processor Windows 2000 runs on has. A build
without them keeps the lock, as the frame cache does.
The DirectDraw source, whose writer is the frame thread itself, reads
its own copy instead of the snapshot.
samples/gfx/vblank_clock/vblank_clock_pub_test: one writer publishing
two million clocks flat out, each built so its fields agree with each
other, and three readers checking every snapshot is one whole publish
and that none goes backwards. Five to six million snapshots, none torn
or backwards; clean under ThreadSanitizer, ASan and UBSan, and run
under TSan in the samples/gfx job. (Under TSan's slowdown about 5% of
reads hit the retry bound against a writer that never pauses; the real
one publishes once a vblank.)
The vblank clock had one source, a thread waiting on
D3DKMTWaitForVerticalBlankEvent, which exists only on WDDM. Scanline
Sync now picks its source per display, best first, and all three feed
the same clock; the frame side - beam by arithmetic, one timer to the
target line - is unchanged.
Read from the binaries first. DXGI's WaitForVBlank resolves
D3DKMTWaitForVerticalBlankEvent by name (dxgi 10.0.26100), and
DirectDraw on WDDM builds WaitForVerticalBlank as a user-mode loop over
D3DKMTGetScanLine - no sleep, no bound, under DirectDraw's process-wide
lock (ddraw 6.1.7600, DdWaitForVerticalBlank, the loop at 71065e12). So
on Vista and later neither is a way in that is better than calling the
export, and the kernel-event tier keeps calling it directly. On XP
(ddraw 5.3.2600.5512, symbols through the OMAP of its BBT layout),
WaitForVerticalBlank makes one call into the display driver and blocks
or spins as that driver sees fit, holding the same lock, and its event
mode returns E_NOTIMPL; GetScanLine is one call into the driver.
- DWM (Vista and later with composition on; always from Windows 8):
DwmGetCompositionTimingInfo's last vblank and refresh period, read on
the frame thread - no thread, no kernel wait. With composition on a
frame reaches the screen at DWM's next composition, which runs on
this clock. Used only while DWM's period matches the window's
monitor, since DWM paces to one display; and only while its
timestamps advance, since exclusive fullscreen stops them. It takes
over from the kernel event after 60 advancing frames, so a window
flicking in and out of exclusive fullscreen does not keep restarting
the clock.
- The kernel vblank event (WDDM): the existing thread, now one source
among three.
- DirectDraw readings (XDDM - Windows 2000 and XP, where no D3DKMT
exists): one IDirectDraw::GetScanLine per call on the frame thread,
on the DirectDraw object DirectDrawEnumerateEx gives for the
window's monitor. WaitForVerticalBlank is not used.
vblank_sampler_feed(), new beside the clock, turns each reading into
the time of the vblank it implies, measuring the beam's line rate
from pairs of readings in the same frame and the total line count
from the rate and the period, which XDDM has no API to report. The
mode's whole-Hz rate from EnumDisplaySettings is only a start - a
driver reporting "hardware default" starts from 60 Hz - and the clock
learns the true period. A driver without GetScanLine stops being
asked after 60 failures.
Until the clock has settled the frame is told there is no scanline, so
Scanline Sync idles instead of polling the counter while it settles.
Only a display none of the three serves - Vista, with no
QueryDisplayConfig for the line counts - still reads the counter.
ddraw.dll and dwmapi.dll are loaded at run time, so nothing is linked
that Windows 2000 lacks; a 32-bit build for _WIN32_WINNT 0x0500
compiles.
samples/gfx/vblank_clock gains the sampler: a counter read twice a
frame at varying points, each read taking 2-30 us, blanking reported
as an error, a 59.94 Hz display started from 60. The total line count
comes out exact and the beam within 2 lines. samples/gfx/vblank_clock_ddraw
runs the tier against a real DirectDraw - wine's, which reports the
counter as an XDDM driver does - and checks the clock's beam against
GetScanLine: within 2-5 lines over 115-119 checks. It runs under wine
in the samples/gfx job.
Not run on real hardware: the DWM and kernel-event tiers need Windows,
and the XDDM tier's cost per GetScanLine and each driver's report of
blanking are unmeasured on XP itself.
Scanline Sync waited for its target line by calling D3DKMTGetScanLine
in a loop, after a sleep that did not look at the beam at all. Read
from dxgkrnl.sys (10.0.26100.7462, its own PDB), that call is a system
call that probes its input, looks the adapter up by handle, takes the
adapter's DDI lock, checks IRQL and dispatches into the display
miniport's DxgkDdiGetScanLine - about 223 us a call on the panel the
tree's comment measured, some 15 lines at 1080p60. And the tuner
measured those lines against RetroArch's viewport height, which is the
display's only fullscreen at the desktop resolution.
D3DKMTWaitForVerticalBlankEvent, measured in the same comment within
30 us of the real vblank with its period accurate to 0.02%, was already
resolved and never called. In dxgkrnl the wait is a KeWaitForSingleObject
on the adapter's vblank event with a fixed 80 ms timeout, so it cannot
hang.
- A vblank clock thread waits on that event for the window's monitor,
at time-critical priority, and feeds each timestamp to
gfx/common/vblank_clock.h. A failed wait is never taken for a vblank;
an error that returns at once backs off on the thread's stop event.
Stopping it joins a wait the kernel bounds.
- The mode's active and total lines, and its exact rate, come from
QueryDisplayConfig, matched to the counter's output by the adapter
LUID and VidPn source ID that D3DKMTOpenAdapterFromHdc reports
(win32_display_signal_timing, in dispserv_win32.c beside the
existing QueryDisplayConfig code). The tuner measures in those lines
now, and takes the total outright instead of watching a frame to
count it. Interlaced modes are left out.
- vblank_clock.h is the arithmetic, pure and on its own. A waiter only
ever wakes late, so it tracks the lower edge of the timestamps: an
early arrival pulls the phase by half, a late one by a sixty-fourth,
and the period is corrected from the same error. A whole number of
periods is missed vblanks and keeps the settle count; anything else
starts over; the same vblank reported twice is ignored. The beam is
reported only after 120 good vblanks, and never once none has come
for two periods - and a period that wanders, variable refresh, never
settles, so it is never used there.
- D3DKMTGetScanLine answers from the clock when it is running, and
reads the counter only when it is not. After a frame, the wait for
the target line is one high-resolution waitable timer aimed at the
moment the clock says the beam gets there, with a rebind event - the
window moved, the mode changed - ending it at once, and a bound. No
polling, no sleep. The counter loop is kept, and bounded, for when
there is no clock.
samples/gfx/vblank_clock drives vblank_clock.h against 50 simulated
displays up to 600 ppm off their nominal rate, with a waiter waking up
to 460 us late. Beam and wait landing stay within 4 lines (6 is
asserted); the period within 1.8 us. Getting there took the test: a
first version, smoothing the timestamps as if lateness were symmetric,
was 16 lines out, and reporting after 8 vblanks let it be 54 lines out
while converging. Missed and duplicate vblanks, a stale clock, variable
refresh and a target already passed are covered. It runs under C89,
ASan and UBSan in the samples/gfx job.
Not run on hardware: wine has no D3DKMTGetScanLine, and nothing here
has a display. The Windows side is syntax-checked with MinGW and the
linux-c89 flags.
Two faults in Scanline Sync, the D3DKMT beam-racing path on Windows.
The wait after a frame spun on D3DKMTGetScanLine until the beam reached
the target line, and that was its only way out. The before-frame tuner
checks once per frame that the query works, but a query that starts
failing during the wait - the display going to sleep, a mode change,
the lock screen, an adapter reset - returns -1, which never reaches the
target: the frame thread spun at full CPU for good. The wait now leaves
at once on a failed read, keeping the tuner's state as it was, and is
bounded to two frames, which covers reaching the target and the
calibration pass that measures a whole frame.
And the scanline counter was opened on whichever display device
answered first at startup - usually the primary - whatever monitor the
window was on, so on a second screen the sync raced another display's
beam, possibly at another refresh rate. It is now opened on the adapter
output driving the window's monitor. WM_MOVE and WM_DISPLAYCHANGE flag
a possible change, and the thread that reads the counter rebinds it
before its next read, so the adapter state is only touched by that
thread; with nothing changed that costs one atomic load. The adapter
handle being replaced is closed with D3DKMTCloseAdapter where gdi32
has it; they were never closed before.
Syntax-checked with MinGW and the linux-c89 flags. Not run: wine has no
D3DKMTGetScanLine, and nothing here has a display to race.
The comment above the upload path still described the ticket the
surface replaced, and the surface submit still fell through to a
video_driver_texture_load() of its own. That load is reached only
when no surface could be made, and if it ever ran it would leave the
thumbnail's texture owned by the thumbnail and by a surface at once -
the second owner this path exists to remove. A thumbnail with no
surface has nothing to show, which is what the fade already reports,
so the fall-through goes and the comment says what the path does.
* fix achievement badge display
* fix linux build
* cheevos badges: any thread may ask, the main thread does the work
Under the threaded video wrapper the achievement widgets are iterated
and drawn on the video thread, and the badge cache answered 0 to
anything but the main thread, so their badges never arrived. Letting
the video thread in whenever video is threaded brings them back, but
the slot table is then shared with the main thread's task callbacks
with nothing between them, and the file check and the download
request run on the video thread.
rcheevos_get_badge_texture() now only looks the badge up: a ready
handle is handed over, anything else answers 0, and a badge not in the
table is written into it as a request. That is all an asking thread
does, under a lock word held for a 17-slot scan, so any thread may ask
and the video_driver_is_threaded() test is gone. The main thread does
the rest in rcheevos_badge_cache_service() - the file check, the
download request, the image task - at once when it is the asker, from
the runloop otherwise, where it costs one atomic load with nothing
asked for. Texture unloads happen outside the lock.
Also:
- a reset keeps the slots' sequence numbers, so a load in flight
across it is unloaded instead of landing in whatever reuses its slot
- a failed load stays failed until the slot is reused or the file is
downloaded again; a missing file asked for without a download can
still be revived by an asker that wants one, once
- a full table forgets a failed slot before one that is loading
- the default badge's shared slot is only reached through
rcheevos_get_default_badge_texture(), whose handle is lent; asking
for "00000" by name gives an owned handle like any other, as the
widgets, which unload what they are given, expect
- the badge download helpers move to cheevos_client.c, so
samples/cheevos/badge_cache links again; that file includes the
compat/strl.h it was already using
badge_cache_test covers the new contract, and builds with HAVE_THREADS
so the lock is compiled in.
* cheevos menu: borrow the default badge, load an entry's badge when drawn
An entry without its own badge yet shows the server default badge.
That handle is the cache's: the entry marks itself pending, never
unloads it, and keeps asking for its own badge each time it is drawn.
Before, the pending mark was set only when the default badge was not
there either, so an entry that did get it stopped asking, and later
unloaded a handle other entries were still showing.
Achievement entries are marked pending when the list is built rather
than asked for there: a set has far more achievements than fit on
screen or in the cache, and the menu drivers only ask for the entries
they draw.
---------
Stills had a ticket mechanism of their own - a record of which list
and which request an upload belonged to, checked on delivery to
decide whether the texture was still wanted - beside the surface the
animation frames go through. They take the surface now:
gfx_surface_new_static() holds the image the load task decoded,
submit_external() hands it to the video thread or uploads it here,
and the release frees the image and installs the texture. A reset
frees the surface, which is what makes a superseded upload harmless,
so the ticket, its sequence counter and the case where an animation
frame lands before a still and has to defend its texture all go away
with it.
gfx_surface_t carries a user_img pointer for a producer whose release
has something to free; the surface never touches it.
A Bluetooth scan ran inside one task handler call, and both drivers
waited out the ten-second discovery window in it: bluez in a
retro_sleep(10000) between StartDiscovery and StopDiscovery,
bluetoothctl in a pclose() of "bluetoothctl --timeout 10 scan on".
That is the task queue's one worker thread, so for the whole window
every other task - downloads, thumbnails, cloud sync - sat queued
behind the scan.
The driver interface's scan() becomes scan_begin() and scan_end(),
BLUETOOTH_SCAN_WINDOW_US apart. The scan task calls begin, sets its own
'when' to the end of the window and returns unfinished; the task queue
keeps scheduled tasks behind ready ones, so everything else runs in the
meantime, and the task comes back at the end of the window to call end
and finish. A cancel ends the window early.
- bluez: begin powers the adapter and starts discovery on a connection
it keeps open - BlueZ ends a client's discovery when its connection
goes - and end stops discovery, reads the devices and disconnects.
A begin that failed part way leaves end only disconnecting. The
driver's free() now releases a connection a scan still holds, and
the device list, which it leaked.
- bluetoothctl: begin starts "scan on" with popen() and returns; end
pcloses it, which by the end of the window has already exited, then
lists the devices as before. free() closes a scan still running.
samples/tasks/bluetooth_scan runs the scan task on the real threaded
task queue against stand-in driver calls, with the window shortened to
300 ms. It pins that begin and end run once each a window apart, that
another task pushed during the window runs within 100 ms, and that the
scan's callback fires. Clean under ASan, UBSan and TSan; added to the
samples/tasks job.
The animation thread pushed a frame and then slept its duration before
looking again. For a GIF or APNG that can be seconds, and switching to
another thumbnail's animation, stopping, or closing the companion all
waited it out: the new animation did not start, and free() did not
return, until the frame's time was up.
The hold is now a wait on anim_cond with the frame's remaining time as
the timeout, ended at once by quit or by the animation changing.
animate() and free() already signalled it; animate_stop() now does too,
and so does cancel(), whose emptied queues are part of what a thread
waiting for a video's still re-checks. Frames are also held on a
schedule - each one's time counted from the last one's, not from after
its decode and scale - so decode time no longer stretches every frame.
companion_thumbs_test gains a case with an APNG of 5 s frames: during a
frame, switching to another animation must show it within a second and
free() must return within a second. On the old code these took 3.1 s
and 5.0 s. The suite passes plain, under TSan and under ASan.
Where AL_SOFT_events is available, a writer waiting for a free buffer
already parks on an eventcount the event callback notifies. Everywhere
else - core OpenAL, Apple's - it slept 1 ms and asked again whether a
buffer had been processed, up to 200 times.
A buffer is freed when the source reaches the end of the one it is
playing, and how far that is is known: AL_SAMPLE_OFFSET into the
current buffer against the buffer's length, at the rate. The fallback
now sleeps exactly that long, then unqueues, keeping the 200 ms bound
for a device that has stopped processing.
Against OpenAL Soft's null device with the events path compiled out,
a second of blocking writes made 250 sleep calls in the whole process,
against 546 before; the writes and shutdown are unchanged.
When the DirectSound buffer had no room, or the staging ring had nothing
queued with time left before an underrun, the pump thread slept 1 ms and
looked again - a thousand wake-ups a second whenever the writer was
ahead, which with audio sync on is most of the time. The comment said
position notifications are not reliable on every driver, which is true,
so it polled.
The pump does not need notifications to know how long to wait. What it
is waiting for is the play cursor moving a known distance: far enough
to free a chunk, or to the point where silence has to go in. That
distance at the device's byte rate is the wait. It waits that long on a
new eventcount, feed, which dsound_write notifies after each block it
queues and dsound_stop_thread notifies on stop, so queued audio or a
stop ends the wait at once. feed is gated like park beside it: a writer
with nobody waiting pays no syscall. The next pass re-reads the cursor
either way, so a driver whose cursor moves in coarser steps than the
arithmetic assumes costs a re-wait, not a missed deadline.
samples/audio/dsound_pump runs the driver end to end on wine's
DirectSound, fed by an ALSA null PCM playing at real-time pace. It pins
that blocking writes still go through and wait on the device, that the
pump keeps the device fed with silence once the writer stops, and that
free() returns at once while the pump is waiting. The CI Linux
samples/audio job builds it with MinGW and runs it there.
Stopping the btstack thread signalled a run-loop source and joined.
Signalling a CFRunLoopSource marks it ready but does not wake a run
loop that is asleep, so the stop request sat there until something
else woke the loop - and the join with it. Then the source's handler
only sent HCI power-off; the loop was stopped by the packet handler
when the daemon reported the new power state. A daemon that never did
kept the thread, and the join, forever.
The thread records its run loop, and stopping it now signals the source
and wakes the loop with CFRunLoopWakeUp, so the handler runs at once.
The handler sends power-off as before and also arms a two-second timer
that stops the loop if the report does not come; the timer is released
when the thread ends.
Also include verbosity.h directly: the file used RARCH_LOG without it,
which only compiled because griffin includes it first.
Syntax-checked against the real CoreFoundation run-loop signatures
(stand-in header, off macOS); no new warnings. Not run: no device with
btstack here.
The Wii USB HID poll thread slept 10 ms between passes over the
adapters. Every report waited for the next pass before its read was
reissued - up to 10 ms of added latency per report, on top of waking a
hundred times a second with nothing to do.
It now sleeps until woken by what gives it work: a read completing, a
device arriving, a control message being queued, or free() asking it
to stop. The read and device-change callbacks are IOS callbacks, which
libogc runs in interrupt context, so the wake is an LWP thread queue
and a pending flag - no mutex - and the wait checks the flag and sleeps
with interrupts disabled, exactly as libogc's own synchronous IPC does,
so a wake cannot fall between the check and the sleep.
Syntax-checked with a PowerPC compiler against the in-tree libogc with
Makefile.wii's defines; no new warnings. Not run: no Wii here.
The Wii U HID polling thread ran
while (!quit) { handle attach events; reissue finished reads; }
with no wait of any kind, so it spun one of the console's three cores
for as long as the driver was loaded, most of it taking and releasing
the adapter list's mutex.
It now waits on an auto-reset OSEvent between passes, signalled by
everything that gives it work: a read completing (the read callback),
a device attaching or detaching (the HID attach callback), the frontend
readying or retiring an adapter (synchronized_process_adapters), and
stop. A signal that lands while the thread is mid-pass leaves the event
set, so the next wait returns at once and nothing is lost.
Shutdown's wait for reads still in flight polled every 5 ms, up to a
thousand times. It now waits on the same event, which each completing
read signals, with the same five-second total bound; past it the
adapters are left, as before, rather than freed under a read.
Syntax-checked with a PowerPC compiler against the tree's Wii U
headers, with Makefile.wiiu's defines; no new warnings. Not run: no
Wii U here.
The extension-init sequence slept 100 ms after each of its two register
writes and before reading the calibration - about 300 ms on whatever
thread delivered the packet. That is the main thread under
iohidmanager, btstack's packet callback, and libusb's event thread,
which every libusb pad now shares, so one remote plugging in an
extension stalled all of them. The note above the handshake recorded
this and what the fix should look like.
The sleeps stood in for the remote's answer. A Wiimote acknowledges
every register write with report 0x22, which nothing handled. The
handshake now sends the first write and returns; the acknowledgement of
it sends the second; the acknowledgement of that reads the extension
id. The read of the calibration that follows a classic controller id
goes out at once: it answers the previous read, so there is nothing to
wait for. Nothing sleeps on the packet thread any more.
A remote that never acknowledges - some clones may not - is not left
stuck: any later packet that arrives once the old 100 ms have passed
moves the handshake on, as the sleep used to. An acknowledgement of a
different output report is not taken for the write's.
Also, the packet parsing loaded big-endian fields through int16_t and
int32_t casts of byte pointers, which UBSan reports as misaligned on
real report buffers. The button words, the extension id and the
address and length of read and write requests are now assembled from
bytes.
samples/input/wii_handshake feeds reports to the handshake and captures
what it sends. It fails on the old code - all three steps went out at
once after 200 ms of sleep - and pins the acknowledged sequence, the
fallback's timing, and that no packet takes more than a few
milliseconds. Clean under ASan and UBSan; added to the samples/input
job. Checked against the protocol, not a Wiimote: nobody here has one.
The task queue's worker claims its state with retro_atomic_cas_int,
which every backend defines except the volatile fallback, so a build
that lands on that fallback - what the portability lane compiles with
RETRO_ATOMIC_FORCE_VOLATILE - fails on an undeclared function. The
fallback gets the CAS its neighbours have: a read, a compare and a
write, which is what its loads and stores already are and is correct
under the same conditions they are.
Every pad had its own thread doing blocking interrupt transfers with a
one-second timeout: a read, then any queued command. So:
- A command (rumble, LEDs) waited behind the read - for the pad's next
report, or for the second to run out on a pad that was sending
nothing.
- Removing a pad, or freeing the driver, joined that thread, which
took up to a second per pad, two if a send was in flight too.
- A read that failed outright (device gone, stalled endpoint) returned
at once and was retried at once, spinning a core until hotplug got
round to removing the pad.
There is no thread per pad now. Each pad keeps one IN transfer
submitted, with no timeout; it completes on the poll thread when the
pad sends a report, which is handed to the pad handler, and is
resubmitted from its callback. Commands go into the same queue as
before and are sent one at a time on a single OUT transfer: a command
is submitted as soon as it is queued if nothing is being sent, and the
next from the previous one's callback. They go out in order and never
wait on a read. A read that ends in anything but success or a timeout
stops, with a warning unless the device simply left.
Removal cancels whatever is in flight and parks the adapter on a
retire list; it frees nothing and waits for nothing, which it could
not do anyway from inside the hotplug callback on the very thread that
delivers the cancellations. The poll thread frees retired adapters
whose transfers have all come back, between event batches rather than
from a callback. libusb_hid_free() stops the poll thread, deregisters
hotplug so no pad arrives mid-teardown, removes every adapter, and
hands the cancellations over itself; each call returns as soon as
libusb has them. A two-second deadline only covers a device stack that
never answers, whose adapters are then left allocated rather than
freed with a transfer still in flight.
Also: a pad whose setup failed after pad_connection_pad_init() now
releases its slot, and callbacks are declared LIBUSB_CALL with a
fallback for headers that do not define it.
Nobody here has a libusb pad, so samples/input/libusb_hid runs the
driver against a fake libusb built on the real <libusb.h>. It pins
that a report reaches the pad handler and the read is resubmitted,
that a command goes out at once with no report pending and three go
out in order one at a time, that unplugging frees the adapter, closes
the handle, releases the slot and disconnects the pad, and that
freeing the driver with a send still pending returns at once, with
every transfer freed, none while in flight and none submitted twice.
Clean under ASan, UBSan and TSan; added to the samples/input job,
which now installs the libusb headers.
The poll thread handled libusb events with a 100 ms timeout so that it
would come back to look at its quit flag. That woke it ten times a
second for as long as the driver was loaded, with nothing to do, and
held libusb_hid_free() for up to 100 ms waiting for the next lap.
With libusb 1.0.21 or later the thread now blocks in
libusb_handle_events_completed() with no timeout, and libusb_hid_free()
wakes it with libusb_interrupt_event_handler() after setting the flag.
If the thread has not reached the wait yet, the interrupt stays pending
and its next call returns at once, so there is no window where the
wake is lost. Checked against the system libusb: the join returns in
under 0.1 ms whether the thread is already blocked, not yet waiting,
or racing the stop (200 runs).
Older libusb, and FreeBSD's own implementation whose API version does
not track upstream's features, keep the 100 ms timeout.
The IIO illuminance poller slept a whole period after every reading and
was stopped by pthread_cancel() into that sleep. A core that raised the
rate had to wait out the rest of the old period first - up to a second
at 1 Hz - and the only way to end the thread was cancellation, with a
cancel-enable dance around the file read and an EINTR check on a sleep
to tell a cancel from a wakeup.
The thread now waits between readings on a condition that close and
rate changes signal, with a timeout to the next scheduled reading:
- Close sets done and signals; the thread wakes at once and leaves, so
the join lasts no longer than a read already in progress. No
cancellation.
- A rate change signals too; the thread reads at once and schedules
from there at the new rate.
- Readings are scheduled a period apart rather than a period after the
previous read finished, so the rate holds however long a read takes;
a reader that falls behind skips ahead instead of bursting.
poll_rate, done and the new rate_changed flag are guarded by the
sensor's lock. The reading itself is published with retro_atomic
acquire/release instead of relying on aligned int writes being atomic.
Without HAVE_THREADS nothing changes: one reading at open.
IIO_DEVICES_DIR can now be overridden at build time, which is how the
new samples/input/illuminance_sensor test points the sensor at a fake
sysfs tree. It pins that a raised rate applies at once (it fails on the
old code: the change waited out the 1 s sleep), that close at 1 Hz
returns at once, that readings follow a changing value, and that open
reports no sensor when there is none. Clean under ASan, UBSan and
TSan; run by a new CI Linux samples/input job.
xmb, materialui and ozone took gfx_surface.h inside a conditional
block - HAVE_CHEEVOS for two of them - so a build without that define
reached gfx_surface_wants_rgba() with no declaration, which the
Emscripten job treats as the error it is. The include belongs with the
unconditional ones, since the call it declares is unconditional too.
The enumeration job was handed back to the driver by its task callback.
That tied the driver to the task queue retiring the task: a walk the
queue let go of would never report in, leaving g_dinput_enum_inflight
set for good - the win32 hotplug timer re-arming forever and the
hybrid driver never probing XInput ports again.
The job no longer uses a task callback. The walk marks itself done when
EnumDevices() returns, and the driver's poll() - on the main thread,
every frame - moves the pads of a finished walk into g_pads[], ends the
in-flight state and autoconfigures. Ownership is a two-count reference,
one for the driver and one for the walk: destroy() marks the job
abandoned and drops the driver's, the walk drops its own when it
returns, and whichever is last frees the job. So destroy() still never
waits, and a walk that outlives the driver cleans up after itself.
That makes the walk fit the task queue's new detachable contract, and
it is now pushed with RETRO_TASK_FLG_DETACHABLE: a walk stuck in the
device stack no longer holds quit or a reinit of the task queue. To
keep to the contract, the walk touches only the job and its task's
flags:
- The rumble "unavailable" warnings were logged from the walk. The walk
now just leaves the effect NULL, and dinput_pad_report_rumble() logs
per pad on the main thread when the pads are published.
- The hybrid driver's RAWINPUT HID snapshot was a static global built
and freed around each walk. It lives in the job now: a walk left
behind may still be running when the next one starts.
- The abandon flag is a retro_atomic int, read by the walk and written
by destroy().
The synchronous fallback, when no task can be allocated, walks and
publishes inside init(), as before.
task_queue_deinit() joined the worker with no limit. The exit path
already drains tasks with a bound (retroarch_drain_tasks_for_exit),
but a handler that is blocked inside an OS call outlives that drain,
and then held the join in task_queue_deinit() - and with it quit, and
every reinit of the queue on content load - for as long as it stayed
blocked. A DirectInput device walk held by a Bluetooth stack still
coming up after boot is one such handler.
Deinit now waits for the worker with a bound (one second). A worker
that is idle or doing the queue's own bookkeeping sees worker_continue
and leaves long before that, and says so on worker_cond, so the common
case wakes exactly when it exits rather than polling. Only a handler
that has not returned can run the bound out, and then:
- If the task was pushed with the new RETRO_TASK_FLG_DETACHABLE, deinit
stops waiting. The task is taken off the running queue, the worker
thread is detached, and the queue's globals are cleared for the next
init. The worker, as soon as its handler returns, finds itself
orphaned, takes its own generation's running_lock once - which deinit
releases as its last step, so that is an exact wait, not a poll -
then frees that generation's primitives and the task and exits
without touching any global. The task's callback and cleanup never
run.
- Otherwise deinit keeps waiting, as before: a handler that did not
promise otherwise may be using something teardown frees next
(net_http's pool is released right after task_queue_deinit() on
exit, for one).
Detachable is a promise by the pusher that the handler touches only its
own task: its state, and the task_set_* / task_get_* accessors. No
logging, no pushing other tasks, nothing the frontend tears down. The
accessors take property_lock, so once a worker has been left behind
that lock is kept and reused by every later init instead of being freed.
Where the worker is, and which primitives are its own, now live in a
per-worker record passed as the thread's userdata instead of in
globals: a worker left behind must not read the state of the one the
next init starts.
samples/queues/task_queue_detach_test pins it: an idle worker stops at
once; a detachable handler stuck past the bound is left behind, a new
queue runs tasks meanwhile, the handler finishing later touches nothing
of it and its callback never fires; a handler that is not detachable
is still waited for past the bound. Clean under ASan, UBSan and TSan,
and added to the libretro-common samples job.
The reservation a fastmem window lives in has two implementations on
Windows and a runner only ever takes one. From 10 1803 it is a
placeholder that mapping splits and unmapping coalesces; below that,
back to Windows 2000, it is one reservation per allocation-granularity
slot, released and mapped over a slot at a time. Every hosted runner is
years past 1803, so the legacy path has never run anywhere.
It is worth running because its failures are quiet. A placeholder split
that loses a range, or a slot released and not put back, does not
crash: it maps the wrong memory, or maps nothing and lets a later map
land elsewhere. A core with a guest RAM behind it then reads and writes
addresses that are not the ones it computed.
Two jobs on windows-2022, the same sample built twice -- once as the
runner would take it, once with MEMSHM_AREA_FORCE_LEGACY, which selects
the pre-1803 path on a system that has the placeholder APIs. Both run
it and both fail on a report of no area at all: that is a pass for a
platform which cannot have one and a silent no-op on a runner that can.
The compiler environment comes from vswhere and vcvars64, not a new
third-party action, since this repository uses only setup-msbuild and
one job is not a reason to widen that.
The sample and memmap.c are warning-free as errors under C89 for both
configurations; the paths filter keeps the job to changes that can
affect it.
Pad enumeration runs IDirectInput8_EnumDevices() on the task queue, and
the walk wrote straight into g_pads[] with the driver's own DirectInput
context. That made it impossible for destroy() to let go of a walk that
was still running: freeing the pads or the context would pull them out
from under it. So destroy() joined it, in a loop that pumped the task
queue and slept 1 ms at a time with no limit. A Bluetooth stack still
coming up after boot can hold EnumDevices() for seconds, and a video
reinit (fullscreen toggle, resolution change) or a joypad driver switch
during that window froze the frontend until it let go.
Give each enumeration its own job instead. The walk fills the job's pad
array with a reference to the context that the job holds itself, and
touches nothing the driver owns. Its main-thread callback moves the
pads into g_pads[] in one step and fires autoconfiguration. destroy()
no longer waits at all: it marks the job abandoned and tears the driver
down immediately. The walk stops at its next device, and the callback,
whenever it runs, releases what the job found and its context
reference instead of publishing them. A reinit started meanwhile pushes
a fresh job, which queues behind the old one and publishes on its own.
The hybrid XInput driver gets the same treatment. Its walk read the
XInput connection state and the blocking flag from driver globals; the
job now takes a snapshot of both at init, and carries the per-pad
XInput user mapping back to the callback.
Along the way:
- Pads are now published when the walk completes, not one at a time
as they are found. Autoconfiguration already waited for completion.
- Moving a pad copies its rumble parameters, which point into the pad
itself; dinput_pad_rebind_rumble() re-aims them after the copy.
- The hybrid driver's copy of dinput_create_rumble_effects() still
pointed its effect parameters at stack variables of a function that
had returned, the bug dinput_joypad.c fixed in its own copy. It now
uses the same pad-owned storage.
- A pad dropped on disconnect in poll() left its two rumble effects
unreleased. dinput_pad_release() stops and releases them along with
the device, and both destroy() and the disconnect path use it.
- The cooperative-level window is captured on the main thread when the
job is created instead of being read from the worker.
The win32 hotplug settle timer still waits for an enumeration in flight
to finish before reinitialising, but only to avoid discarding a walk
and queueing another behind it; its comment says so now.
Sixteen call sites across the menu drivers, gfx_display, the widgets
and the menu's own loaders read the display flag to decide what a
decode should produce; they ask gfx_surface_wants_rgba() now, a
shorthand over the requirements query for the many producers that
decode 32-bit images and have no other question. The two widget
samples that link gfx_widgets.c answer it like the rest of the
frontend stubs.
That leaves retroarch.c and runloop.c on the allowlist in
tools/surface_requirements_check.py, and they stay: one clears the
flag while tearing a driver down and the other answers a core's
environment query with it, so neither is a producer deciding what to
decode into. Every producer that is one now has a single place to
ask.
WM_DEVICECHANGE arrives once per HID interface, so a single composite
device - a USB headset with volume keys, a keyboard with a media or
vendor collection - delivers a burst of arrivals. The dinput and
winraw input drivers reinitialised the joypad driver on every one of
them: a full destroy, init and DirectInput EnumDevices() per event,
back to back. Because destroy() joins an enumeration still in flight,
each event after the first also parked the window thread until the
previous walk of the HID/PnP tree finished.
Arm a one-shot window timer on each event instead. Re-arming restarts
it, so the reinit runs once, 250 ms after the last event of the burst,
woken by the timer rather than by polling or by re-sampling a clock.
When the timer fires while an enumeration from the previous reinit is
still running (a slow Bluetooth stack after boot), do not block on it:
re-arm and look again one settle period later - the same "skip the
pass, retry on a later wake" the audio drivers use in place of an
open-ended wait. The hotplug path can no longer block the window
thread at all.
The dinput and winraw wndprocs route the timer to their input driver's
handler, which only ever sees its own live driver data. Any other
wndproc that receives it kills it in wnd_proc_common, so a timer armed
by a driver that has since gone away cannot tick forever.
This is not the EnumDevices()-every-frame problem some games have (the
one DInput8_CacheFix works around); RetroArch never enumerated outside
init and hotplug. It removes the remaining burst case.
cheevos_badge.c asks the surface layer what the driver wants before
it queues a badge decode, and this sample links cheevos_badge.o
against its own frontend stubs, so it needs an answer: no driver
here, so ARGB words, 8 bits a channel, no in-place update.
tools/sample_link_check.sh grew a --all mode. Selecting samples by
the sources a diff names catches a call added to a file they link,
which is the common case, but not a sample that reaches a changed
file through a header; when a change is wide enough that guessing is
the wrong move, build them all.
The exception-port call that keeps a debugger out of an EXC_BAD_ACCESS
loop is macOS and iOS only: tvOS marks the mach task API unavailable and
its SDK ships no headers for it, so a core that reaches this file failed
to build there. The includes and the call are behind TARGET_OS_TV now,
and nothing else on the tvOS path needs them -- the filter itself is
sigaction and the slot table.
The test is written defined-first rather than as a bare !TARGET_OS_TV.
Both forms select the same branch on every platform, including a
non-Apple one, where the short circuit on __APPLE__ stops the
expansion. They differ where TargetConditionals.h is not in scope: the
bare form warns twice under -Wundef and quietly evaluates the name as
zero, which is the wrong answer for tvOS and the right one for
everything else, so it would go unnoticed. Asking whether the name is
defined says what the code means without depending on an include having
happened first.
Checked by preprocessing: the tvOS shape emits no reference to
task_set_exception_ports and macOS emits the four it had. C89 -Werror
clean on Linux and both Win32 targets, and the sample passes.
Reported against the PS2 core, which carries this file.
task_image.c asks the surface layer what the driver wants before it
decodes, and this sample links task_image.o against its own frontend
stubs, so it needs an answer like the rest of them: no driver here,
so ARGB words, 8 bits a channel, no in-place update.
tools/sample_link_check.sh builds every sample whose Makefile names a
source the working tree has changed. The samples link subsets of the
tree against their own stubs, so a call added to a file they link is
a link error there and nowhere else - compiling that file alone never
shows it, and neither does building the tree. Run it before sending a
patch.
Every other parameter in video_driver_update_viewport() comes off the
seqlock'd snapshot; the overlay override's fit branch still read the
live aspect ratio, which the main thread owns, from the video thread.
The overlay id check read `image > gl->overlays`, one past the page's
last entry, and the tex_geom setters had no check at all - a write
through a NULL coordinate array before a page is loaded, or into the
neighbouring region of the block the three coordinate arrays share.
gl3_render_overlay() also drew entries whose texture name was 0. That
is black on a core context, but a GLES driver may sample whatever the
unit last held, which on the alpha blend pipeline's unit is the
content texture - the page's art then shows the core image.
gl3_pass_build_commands() bound the pass framebuffer only when it was
complete, and drew either way. Every pass ends by binding the default
framebuffer, so a non-final pass that fell through drew the core image
onto the backbuffer at its own size - full window, every frame,
underneath whatever the final pass put in the viewport.
A framebuffer only becomes complete once something calls set_size on
it: gl3_framebuffer_new() reserves the name and seeds the size at 1x1
with nothing attached. A pass whose first computed size was also 1x1 -
what a viewport-scaled pass asks for when the viewport is not yet
real - therefore skipped the build and stayed incomplete.
Build an unbuilt framebuffer whatever its size says, and skip the pass
entirely when it still has no target.
Candidate fix for #19573.
A bool for "10-bit" said the only question a producer could ever have
about depth was whether ten bits were wanted. The requirements now
carry the set of formats the driver samples without the frontend
converting first - a bit each for 8888, XRGB2101010 and FP16 scRGB,
with names reserved for FP32, 565 and 4444 - and which of them is
worth decoding into. A context driver already advertises an FP16
scRGB framebuffer (GFX_CTX_FLAGS_SCRGB_FRAMEBUFFER), so that path is
now a decoder change rather than another change to this contract, and
a producer asks whether a bit is in the set instead of assuming a
depth.
The cheevos badge loader, the image viewer core, the image task and
the overlay loader ask the query for their channel order, and the
image task asks it for the 10-bit decision too - both answers at one
moment from one place, where they were two lookups of two different
flags. The image decoders emit 8-bit or 10-bit, so the task asks for
the widest format it can itself produce rather than the driver's
preference, which may be wider than anything upstream of it. All four
come off the allowlist in tools/surface_requirements_check.py.
Both still-image requests and all three animation job sites read the
display flag themselves; they ask gfx_surface_query_requirements()
now, through one helper, so the file has a single answer for what a
decode should produce and comes off the allowlist in
tools/surface_requirements_check.py. The helper sits outside the
threaded block: the still paths are compiled without HAVE_THREADS
too, and only linking the samples shows it.
Two build breaks from 3ac0a0b.
MXE's mingw-w64 predates the Windows 10 1803 SDK and does not define
MEM_RESERVE_PLACEHOLDER or its siblings, so the placeholder path failed
to compile on the Windows jobs even though it only ever runs behind a
runtime check. The flags are now a compile-time condition of their own:
without them only the legacy slot-reservation path is built, which
reaches every Windows this file supports anyway, and with them the
runtime check decides as before. Each placeholder-only local is
declared with the branch that uses it rather than unconditionally.
memshm_area_free was gated on HAVE_MMAN alone where the rest of the
area requires MAP_FIXED and MAP_ANONYMOUS as well. DJGPP defines
HAVE_MMAN and ships a stub header with neither, so every other function
compiled to its no-area stub and that one emitted a munmap the DOS libc
does not have -- a link error rather than a compile one, which is why
it surfaced last. All four now share one condition.
Checked against a header shape that undefines the four flags after
windows.h, which reproduces the reported errors exactly, and against
the DJGPP shape, where no mmap or munmap call now survives the
preprocessor. Eight configurations are warning-free: Linux, an Apple
shape, Win64 and Win32 each with and without the flags, the forced
legacy path, and DOS.
The other half of what a fastmem window needs. memshm gives a region
that can be mapped more than once; this gives the address space those
mappings live in: one contiguous reservation that stays reserved while
pieces of a guest RAM are mapped into it at the offsets a recompiler
computes and unmapped again as the guest remaps. Neither mmap nor
VirtualAlloc does that directly, and every core with a window has
carried its own.
POSIX reserves PROT_NONE anonymous memory, maps MAP_FIXED over it and
puts PROT_NONE back -- calls with no floor on Linux or Apple, with
MAP_ANON aliased for the latter, which never took the MAP_ANONYMOUS
spelling. All three functions require the same two macros, so a
platform that has only one of them reaches the stub in every one of
them rather than compiling a body that names a macro it lacks.
Windows has two paths. From 10 1803 the reservation is a placeholder
that mapping splits and unmapping restores and coalesces, through
VirtualAlloc2, MapViewOfFile3 and UnmapViewOfFile2. Below that -- back
to Windows 2000, on functions that have been kernel32 imports since NT
-- the span is reserved one allocation-granularity slot at a time,
because MEM_RELEASE frees a whole reservation and never part of one;
mapping releases exactly the slots it covers and maps a view over them,
and unmapping restores those reservations immediately, since an
unreserved hole is an address something else can take. The brief window
between release and map is retried, and a second failure puts the
reservations back and fails cleanly rather than leaving the span holed.
Runtime is identical on all of them: a mapped view is a mapped view and
a load through the window is the same instruction. The legacy path
costs one extra system call per map, on a path that runs at setup and
on a guest remap.
Which Windows path is decided by resolving all three placeholder entry
points, not one: Windows 10 1709 exports UnmapViewOfFile2 and neither
of the others, so resolving one and assuming the rest calls a null
pointer on a shipping system. Checked against the binaries -- Windows
7, 8, 8.1, 1511 and 1607 export none of the three, 1709 exports that
one, and every build from 1803 to 2004 exports all three with
byte-identical argument validation. Those functions are thin shims that
validate flags and forward to Nt*; the placeholder split and coalesce
are the kernel's, so a disagreement between the tracked ranges and the
real mappings comes back as a failed VirtualFree, and the coalesce
calls check it.
On 64-bit Windows the base is placed above 4 GB by sweeping a ladder of
floors: a window based below that crosses into the system's own
reservations or aliases mapped pages, and a fault there cannot be told
from a real one. A single constrained request fails outright on a
fragmented address space where a ladder still finds room. None of it
applies to a 32-bit process and it is compiled out there.
memshm_create gains a Darwin case with it: a shm name there is capped
at 31 characters where Linux takes one ten times longer, so a name that
will not fit is shortened from the front, keeping the tail that
distinguishes one process from another. The same caller now works on
both instead of failing at runtime on one.
The test maps one region at three slots, writes through the first and
last and reads through the others, unmaps the middle with live mappings
either side, maps the freed slot again, and creates a region under a
name too long for Darwin -- the failure modes that corrupt silently or
fail on one platform only. MEMSHM_AREA_FORCE_LEGACY selects the
pre-1803 path on a system that has the placeholder APIs, which is how
that path is testable without a Windows 7 machine, and the sample
builds a second binary that way. C89 -pedantic clean on Linux, an Apple
shape, and both Win32 targets in both configurations; ASan and UBSan
clean.
Deriving the emulated mailbox's waits from the refresh rate put
video_driver_get_refresh_rate() and cpu_features_get_time_usec() in
vulkan_common.c, and this is the one harness that links it. Stub the
first as the other video_driver entry points here are stubbed, and link
features_cpu.c for the second.
alsa_microphone_read() waited on snd_pcm_wait(mic->pcm, -1), so a
capture device that stopped delivering parked its caller for good. It
was the last unbounded wait in the driver: the playback side lost its
own in 48394abbb2, and alsa_microphone_wait_readable() has always been
bounded.
Wait the same two periods that one does, and give up after the same
number of laps, returning what the read got. The derivation both use is
one function now.
The lifecycle harness gains the lane: its scripted wait already fails a
run on an unbounded timeout, so the read only had to be driven through
it, with a microphone that never delivers and one that does.
The producer waiting on a full ring during a limited fast-forward,
rather than dropping, had no test: the ff_speedup lanes mark the pipe
stalled before submitting, so the producer never reaches the park.
Drive it here, where there is a live consumer. The device is made the
slower of the two so the ring fills, and a limited hold is compared
against an unlimited one over the same source: the first keeps it, the
second drops almost all of it. A second lane holds the limiter against
a device that has stopped and requires the frames after the stall is
recorded to be immediate.
Core options were only written automatically at the very end of core
teardown, in runloop_deinit_core_options(): after the auto save-state,
retro_unload_game(), retro_deinit() and dylib_close(). A crash anywhere
in that sequence, or the process being killed before it, silently lost
every option change made during the session. Per-game "Save Game
Options" was unaffected because it writes immediately.
- Split the write logic out of core_options_flush() into a quiet helper,
runloop_core_options_save(), which writes to whichever options file
is active without freeing anything or showing a notification.
- Call it at the start of CMD_EVENT_UNLOAD_CORE (Close Content) and
CMD_EVENT_CORE_DEINIT (Quit), before anything calls back into the
core. The late save in runloop_deinit_core_options() is kept; it is
a no-op unless the core changed an option during teardown, since
config_file_write() only writes a modified config and clears the
flag afterwards.
- Call it from android_input_flush_pending_state(), next to the SRAM
and config flush, so option changes survive Android killing the
process in the background (e.g. swiped away from Recents). Not gated
on config_save_on_exit, matching the unload-time save.
Verified with a test core that crashes in retro_unload_game(): an
option change made at runtime is lost before this change and saved
after it. Normal (non-crashing) shutdown behaviour is unchanged.
Fixes#19215
The producer's wait for a consumer pass was a flat second, so a device
that stopped draining cost the frontend all of it before the stall was
latched - and fast-forward reaches this wait now. The pass is paced by
the device draining what the ring holds, so bound it by that ring's own
playing time at the speed it is being pulled at: shorter in
fast-forward, floored so a small ring still leaves a usable wait. The
consumer's own waits keep the flat bound; they wait on a producer, not
a device.
The ring-fill trim only had a floor, so the published multiplier could
leave the range audio_driver_fastforward_ratio_mult() clamps to. Cap it
there too.
The blocking acquire re-armed its full timeout on every wake, and the
condition carries the request and the dead flag as well as the acquire,
so a wake that was neither would extend the wait without limit. Take
one deadline before the loop and pass the remainder, as the audio
pipeline's producer does.
The timeout itself was half a second flat, unrelated to what is being
waited on: a swapchain image coming free, which the display paces. It
is also what teardown costs, since sthread_join cannot interrupt the
thread's acquire, and teardown runs on the fast-forward release edge.
Derive it from the refresh rate, sampled at init so the thread never
reads video state, floored and capped so a missing or absurd rate still
leaves a usable bound.
The pitch was a plain width * 4. No image reaches a width that
overflows it, but this is the contract every producer's layout comes
through and it should not begin with an unchecked multiply: the query
now returns false for a width whose row size_t cannot express, leaves
the requirements untouched, and the thumbnail caller acts on that.
A rule that lives only in a comment decays, so
tools/surface_requirements_check.py holds the other half: a source
file outside its allowlist may not read VIDEO_FLAG_USE_RGBA or
GFX_CTX_FLAGS_SCREEN_10BPC_SOURCE directly, and a file on the list
that has stopped reading them fails the check too, so the list
shrinks as producers migrate rather than sitting there. The video
drivers answer those flags rather than consume them and are not
listed. It carries a self-test for both directions and runs as its
own job beside the gfx samples.
Android dispatches into a player after Destroy has returned, so no
amount of stopping, clearing or confirming the play state makes the
handle safe to free: the AudioTrack thread has already latched the
callback and its context by then, and unregistering afterwards changes
nothing. The harness reproduces it as a use-after-free in
opensl_callback.
Move what the callback reaches - the block count, the played-frame
count, the block size and the eventcount - into a static context the
driver never frees, and have the callback drop a dispatch whose player
is gone or replaced. A stale dispatch then costs nothing instead of
touching freed memory. The handle and its audio buffers are freed as
before; on Android the eventcount is a bare futex word, so a driver
that is never used allocates nothing and one that is re-initialized
does not grow.
This is the driver's half. The poisoned mutex in the reported stack is
libwilhelm's own object, which this cannot reach.
The mock destroyed the player the way the spec says to - clearing the
registration and waiting for any callback already inside - so the
harness modelled a stronger guarantee than Android gives and passed the
teardown hardening that did not hold on a phone.
Add the platform's behaviour as a mode: the player goes without waiting
and its registration outlives it, with the pump parked at the callback
so a test owns the window. The new case tears the driver down there and
finds opensl_callback on freed memory.
Since 8618e55115 the scanner collapses a multi-disc game's disc
entries into its M3U, labelling the M3U entry with the first disc's
label minus its disc indicator - and the playlist flush then
deliberately replaces the entry the M3U got under its own name.
That only helps when the disc label is a database or DAT title. When
the disc matched nothing (hacks, undubs, a plain manual scan), its
label is just its file name, so
Game Name (Undub).m3u
Game Name (USA) [Hack by Person v2] (Undub) (Disc 1).chd
came out as 'Game Name (USA) [Hack by Person v2] (Undub)' instead of
the 'Game Name (Undub)' that 1.22.2 gave, breaking thumbnail setups
and anything else keyed to the M3U's name.
When the first disc's label is its own file name, label the M3U entry
with the M3U's file name. A real title is still collapsed as before.
Fixes#19376.
5f9c76f3fc moved update_viewport's aspect ratio onto the seqlock'd
snapshot and seeded it at the top of video_driver_init_internal().
That seed runs before init stores the aspect ratio from
aspectratio_lut, so the snapshot carries the previous aspect - and on
a cold start, launching content straight from the command line, it
carries zero.
Drivers compute their viewport from inside init, so every init-time
viewport came out zero pixels wide until the first video_driver_frame
republished. Vulkan built the shader preset during init: any pass
scaled to the viewport asked for a 0xH framebuffer, was refused, and
the whole preset fell back to stock (#19569 - 'Creating framebuffer
0x2168'). Starting from the menu hid it, since an earlier init had
already stored a real aspect; switching shaders afterwards worked
because the per-frame publish had caught up.
Publish once the aspect ratio is stored instead, still ahead of the
driver's init.
Fixes#19569. Likely also #19573 (glcore overlay missing on CLI
launch, first frames at the wrong aspect).
What a producer should emit was spread across three unrelated
lookups - a display flag for channel order, a context flag for
whether a 10-bit source is worth decoding, a poke for compressed
formats - and every decoder that cared had to know all three and
where they lived. gfx_surface_query_requirements() answers them
together, before the decode, for the driver that is actually up, with
the row pitch and alignment the upload paths take; it is safe to ask
before any surface exists and gives a software path's answers when no
driver is. gfx_surface_supports_compressed() covers the fourth, over
a new video_driver_supports_texture_format().
The thumbnail decode path asks it rather than reading the display
flag, and the remaining producers can follow one at a time. The
struct is also where a backend that wants a pitch or a layout nothing
in the tree currently offers would be described.
Emulating the mailbox skipped the block that resolves
VK_DATA_FLAG_USE_WSI_SEMAPHORE, so on every platform that compiles the
emulation in, the flag was never written and neither log line was
reachable: the acquire path a report comes from is invisible, and the
flag's value rests on the caller having zeroed the struct.
Resolve it either way and log it once, with whether the emulation is
available.
With the threaded pipeline and audio sync the ring sits full at 1x: the
producer's wait on it is the frontend's audio throttle. A fast-forward
engage ended that wait, and the tempo it requested was positioned behind
the few frames of 1x source already in the ring, so while those played
at 1x the core ran ahead into a full ring and every publish was dropped,
heard as a crackle at each press on Windows with WASAPI, in FF7 and
Ocarina of Time alike.
Wait for the consumer during fast-forward as at 1x when audio sync is
on, the transport follows the speed and the ratio is limited: the
consumer pulls at that speed, so the wait paces the core instead of
losing source. Unlimited, waiting would hold the core to the fastest
tempo the audio plays, so that still drops. Publish the seed multiplier
with the first frame of a hold, and have the runloop's engage position
its tempo at the consumer's ring position, so what the ring holds plays
at the new tempo at once.
A core that waits runs at the consumer's pull. The published multiplier
is trimmed by the ring fill toward half full, so while the ring is full
the pull runs above the estimate, the core held to it speeds up and is
measured faster, and the estimate climbs to the limiter's speed instead
of settling where it started.
On the inline pipeline the first flush of a fast-forward played at the
configured ratio, although its audio spans the frame before the limiter
lifted, which left the device a frame short at the edge. The inline
flushes run with the frame they play, so that first flush plays at 1.0.
The threaded producer still publishes the configured ratio: its consumer
applies it to what the ring already holds.
Toggling fast-forward flips the swap interval. On Linux that resolved to
a different present mode, so the swapchain was recreated on every toggle
and the driver then re-initialised everything built on it, the whole
shader chain included: a 15-25 ms stall one frame after each edge, long
enough to run a 64 ms audio device dry and heard as a crackle at both
ends of every fast-forward.
Emulate the mailbox on desktop Linux, as Windows does in fullscreen and
macOS always, so the swapchain is kept through the toggle, and in every
Vulkan context stop a resize that kept its swapchain from invalidating
it: only a replaced swapchain has anything to rebuild. As on those
platforms, the emulated mailbox also moves integrated GPUs from WSI
semaphores to fences for the acquire.
scond_wait_timeout() takes microseconds, so the blocking mailbox acquire
waited 0.5 ms instead of the intended 500 ms. Leaving the emulated
mailbox without an acquired image then timed out whenever the image was
not ready within 0.5 ms and recreated the swapchain, the stall the
emulated mailbox is there to avoid.
The mailbox thread also cleared its request before acquiring and, when
the acquire timed out, went back to waiting for a new one. The main
thread only asks again once it has been handed an image, so none came
and every frame skipped the swapchain until the mailbox was torn down.
An overlay image and an animated preview frame now reach the GPU
through one path. gfx_surface_new_static() makes a surface with no
slots of its own, for an image whose pixels the caller keeps, and
gfx_surface_submit_external() uploads from where they are: an overlay
pack already owns its decoded pixels, so slots would have added a
copy the abstraction is not allowed to cost. Each unique image of a
pack becomes one such surface, the pages draw from the surfaces'
handles, and a release frees surfaces rather than tracking textures
of its own - so an upload still in flight when a pack goes away frees
itself through the completion, the way a preview frame's does.
Under threaded video the pack's submits are posted and collected with
one poll, because a page needs its handles before it is shown; a
preview frame may be dropped when the video thread is busy, an asset
that uploads once may not.
Eight public headers included <windows.h> or <winsock2.h> bare --
retro_atomic.h, retro_miscellaneous.h, retro_timers.h, retro_math.h,
memmap.h, encodings/utf.h, net/net_compat.h and
glsym/rglgen_private_headers.h. MSVC's minwindef.h defines min and max
as macros in C++ as well as C, where MinGW's is guarded by
#ifndef __cplusplus, which is why no MinGW build shows this. Once they
are defined, every std::numeric_limits<>::max() in any C++ translation
unit that reaches one of these headers is a syntax error: a core hit it
in glslang's arrays.h and in its own texture cache, several include
levels below the header that leaked.
All eight define NOMINMAX and WIN32_LEAN_AND_MEAN before the include,
as a public header must, and in net_compat.h the guard goes above
winsock2.h, which pulls windows.h in itself. Checked by compiling each
one alone and asserting neither macro is left defined; C89 -pedantic
warning-free on Linux and both Win32 targets.
A COM interface may only be released while the apartment it was
created in is still initialised. The frontend's own CoInitialize is
undone before the drivers are torn down, so the taskbar interface was
released through a vtable belonging to a module that had already
unloaded: the exit crash the threaded-video harness hit on the Windows
runner after every lane passed, at "call *0x10(%rax)" - Release - with
rax pointing into a gone module. Every Windows session that reached
the display server took that path out.
The server takes a reference of its own on the apartment when it
creates the interface and gives it back after the release, so the
apartment outlives what it holds. CoInitializeEx returns S_FALSE when
the apartment is already initialised, which still takes the reference.
Every claim the surface work rests on is a statement about numbers
nobody was counting: a streaming frame allocates nothing, copies
nothing and creates no texture; an overlay page switch uploads
nothing; the direct path queues nothing. gfx_instrument.c gives those
numbers - texture loads, in-place updates and the refusals, unloads,
wrapper posts and how many of them allocated a node, completions,
synchronous commands, surface creations and slot bytes, submits by
result, animation frames decoded in place against copied out of a
canvas, overlay uploads and page switches - counted where each thing
happens. They are relaxed atomics and nothing there logs, allocates
or waits, so a counted path keeps the timing that makes its numbers
worth having. GFX_INSTRUMENT=1 builds them; without it no counter
exists and no call to one survives the preprocessor.
The threaded-video harness reads them in its surface and overlay
lanes, printing what a run cost on the driver it is pointed at and
holding the budgets the paths are written to: no post may allocate,
no submit may copy into a slot, a streaming surface keeps one
texture, and a page switch never uploads. A submit a driver drops
because its own staging is still in flight counts as neither an
update nor a load, which is the latest-frame policy working rather
than a miss. The thumbnail animation sample gains the same kind of
lane for the decode side.
A crash on a runner is the only place some of these happen, and a
backtrace names the function that stopped rather than the access that
faulted: a release build with one field read, one vtable call and one
function pointer in the same frame leaves nothing to tell them apart.
The GDB=1 runner prints the faulting instruction, the symbol it
belongs to, the registers it used and the four instructions leading
to it, ahead of the backtraces it already printed.
The lane submitted into alternating slots and expected each submit to
be queued after a single frame, which holds on the null driver and not
on a real one: a queued submit belongs to the video thread until it
has taken it, the slot reads BUSY until the release comes back, and on
the Vulkan lane a dozen of the thirty submits landed in that window.
Each submit now gives the release the frames it needs before it is
called a failure, and the release count is checked against the submits
that were actually queued rather than a fixed number.
main_exit() freed the settings and then tore the drivers down, and
the teardown reads them: the Win32 display server restores the
display mode it changed, which goes through win32_monitor_info() for
the monitor index, so a run that had changed the mode ended with a
segfault in win32_display_server_destroy(). The threaded-video
harness found it on the Windows runner, after every lane on d3d11 had
passed. The settings are now freed after the drivers, and
win32_monitor_info() treats their absence as "the monitor the window
is on", since it is also reachable while a shutdown is further along.
The MSYS2 job becomes a two-driver matrix, so the harness lanes - the
surface lane's in-place texture updates and the overlay page lane
among them - run on both explicit-API backends on WARP. D3D12's
update is fence-tagged against the frame that last copied from the
texture's upload buffer, which nothing but a running driver
exercises, and its overlay page draws from copies of the pack's
textures.
Two lanes join the harness, so the paths the surface work added run on
whatever driver it is pointed at rather than only on stubs. The
surface lane streams 31 frames through a gfx_surface under the wrapper
and 11 without it: a threaded submit must answer QUEUED and hand its
slot back through release() on a later frame, a second submit while
one is in flight must answer BUSY, a direct submit must answer DONE at
once, a driver with an in-place update must keep one texture handle
across every frame, and a surface freed with a frame still in flight
must complete quietly. The overlay lane makes textures, shows them as
a page through the driver's load_textures, switches pages with no
upload, and disables the page before unloading them - the order
input_overlay_free keeps.
Both lanes run on the null driver as before and on a real one when the
harness is given a driver: on the Vulkan lavapipe job they exercise
vulkan_update_texture and its stream state under the validation layer.
A recompiler with a fastmem window reads guest memory with ordinary
loads into a reserved region and lets the MMU say when a page is not
there; the handler commits it, or rewrites the access to a slow path,
and the instruction re-executes. That is a normal part of running, on
the recompiler's own threads, while every other thread's faults must go
on meaning what they meant -- and every core that does it has carried
its own filter. This is the one to carry: a SIGSEGV and SIGBUS
sigaction, or a vectored exception handler on Windows, that identifies
the faulting thread, hands the pc and the faulting page to a callback,
resumes when it says it handled the fault, and chains to whatever was
installed before when it does not.
Everything the filter touches is lock-free, because it runs in a signal
handler: the registry is a small fixed table of atomics that a fault
linear-scans, the identity is the platform's thread id rather than a
thread-local -- a core is a shared object, whose thread-locals are
global-dynamic, and the first access on a thread can allocate at
exactly the wrong moment -- and the callback pointer is published
behind an acquire flag rather than cast through an atomic void*, which
C89 does not allow. A per-slot flag catches a fault inside the callback
and chains out, so a genuine crash still produces its core dump.
The test commits a reserved page from inside the handler and checks the
store completed, then the three faults the filter must not claim, each
with a chain handler underneath to prove it ran: an unregistered
thread, a callback that declines, and after removal. C89 -pedantic
warning-free on Linux and both Win32 targets; ASan and UBSan clean, and
TSan reports no data race. All three run with handle_segv=0, which is
how a fastmem core runs -- a sanitizer's own SEGV handler would take
the faults first -- and the workflow sets it for this target.
video_display_server_destroy() freed the server's data through its
destroy and left the pointer standing, so the next destroy handed the
Win32 server a freed struct - retroarch_deinit_drivers() at exit,
after a driver reinit had already replaced the server - and it read
the original display mode and released a taskbar interface out of
that memory. The threaded-video harness found it on the Windows
runner as a segfault at exit, after every lane on d3d11 had passed.
The pointers are cleared after the destroy: a second destroy does
nothing, and the next init makes a new server.
With that, the d3d11 lane on WARP passes every lane, so it stops
being advisory: it fails the workflow like the others.
The interface said the core must not touch the image until
wait_sync_index; it did not say the image must stay alive that long,
and a core that kept only its newest image let each previous one be
destroyed as soon as the next frame replaced it. A frontend reading
on its own schedule - the threaded video path, and under fast-forward
many frames behind - then sampled a destroyed image, which is a GPU
page fault and a lost device. That was parallel-GS on every
fast-forward. The contract now says it: the image, and the reference
that keeps it, are held until wait_sync_index for the index it was
handed at returns, one image per set bit of get_sync_index_mask.
VIDEO_FLAG_GPU_DEVICE_LOST existed, the D3D9 drivers set it, and
nothing read it, so a device lost to a TDR or a GPU reset left the
frontend running against a dead device: every submission failing,
the window showing whatever the last good present left, and the core
- on the same device - emulating on with sound and input. The runloop
now acts on the flag once per iteration, on the thread that owns the
drivers: it reinitialises the video, input and menu drivers, which
brings the context back on the recovered device and gives a hardware
core its context_reset.
The Vulkan driver reports the loss where it sees it: the frame
submissions, the present and the acquire, once, and from whichever
thread runs them - the flag is atomic, and under the wrapper that is
the video thread, which can only ask.
tpool_create_with_stack_size() makes a pool whose threads have the
stack the caller names, through sthread_create_with_stack_size;
tpool_create() keeps the platform default. The thumbnail decode
worker's pool now asks for 512 KiB a thread: its threads decode VP9
tile columns and HEVC CTB rows as well as converting bands, which
recurse through the decoders, and a platform's default thread stack
is not a number those decoders were written against.
Every recorded upload now tags its texture with the fence value the
frame being recorded signals at its end - the present signals it
behind the frame's command list, without waiting - and an in-place
update between frames writes the texture's upload buffer only once
the fence has reached that value; while the frame that copied from
it is still executing, the frame is dropped and the texture keeps
what it shows. A texture still marked dirty has had no copy recorded
since its last write and is written straight away, as before.
Freeing a pack disables the driver's overlay first and unloads the
pack's textures after, where it was the other way round: the driver's
page keeps copies of the pack's textures, and under threaded video a
frame can be drawn between the two calls - on D3D11 and D3D12 from
released objects.
The D3D11 in-place texture update maps its staging texture without
waiting: the last update's copy from it may still be the GPU's, and a
streaming frame with a newer one behind it is dropped in that case,
the texture keeping what it shows, rather than the decode worker
stalling on the map.
The HDR constant buffer is created with the HDR swapchain, only while
HDR is enabled, but the six HDR pokes - menu nits, paper white,
expand gamut, scanlines, subpixel layout, inverse tonemap - and the
hdr10 switch mapped it unconditionally, and Map() on a NULL resource
is a crash inside d3d11.dll: the threaded-video harness hit it on the
video thread from its every-poke lane, on WARP with HDR off, which is
also every user who reaches an HDR setting with HDR off. The pokes
keep their value in hdr.ubo_values, which the buffer is created from
when HDR comes on, and one guarded d3d11_hdr_ubo_update() writes the
buffer when there is one.
getopt_long() scanned for the next option with `for (; argv[idx]; )`,
and took an option's argument from argv[1] unchecked: it relied on the
NULL a real main() places after the last argument. An argv a caller
builds itself may not have one - the threaded-video harness's did
not - and then the scan reads whatever the stack held, which under
gdb on the Windows runner was not a NULL: a segfault in
__getopt_long_retro() from retroarch_main_init(), before the first
lane. The scan and the argument reads are now bounded by argc, so
the parser never looks past what it was given; the harness also
terminates its argv, as a main()'s is.
The d3d11 lane's first run on the Windows runner passed five lanes
and died with a segfault in the sixth, the present-repeat lane, and
a Windows runner leaves no core to read afterwards. GDB=1 runs the
harness under gdb - the exit status is still the harness's own - so
the stacks of every thread land in the job log at the crash, and a
failing run now also prints the last sixty lines of the run log,
driver messages included, so what the driver logged just before is
there too. The d3d11 job installs gdb and uses it.
The two passes it already compares are timed, and the milliseconds
per frame on one thread and on the pool are printed beside the
result, so a run by hand shows the ratio for that machine and that
stream - the tile columns, the wavefront rows and the conversion
bands together. The check itself is still the comparison alone.
Disabling an overlay no longer unloads the pack's textures: the driver
they were made on is still there, and a pack that comes back from the
cache - the common case, a content load without a video reinit - now
shows its pages with no upload at all, where it used to upload every
image of the pack again. The one place the textures must go is the
video driver's teardown, which now unloads the active and the cached
pack's textures through input_overlay_video_teardown() while the
driver can still do so; both packs keep their decoded pixels and
upload them again on the next enable, on whatever driver comes up.
check-driver and check-vulkan skipped with exit 77 when DISPLAY was
unset, which is right on a Linux runner without Xvfb and wrong on
Windows, where nothing sets DISPLAY and the desktop is always there:
the d3d11 lane built the tree and the harness on the runner and then
skipped itself. The skip now also requires OS to be something other
than Windows_NT, the variable Windows itself sets.
The D3D11 and D3D12 drivers create an IDXGISwapChain and hold it as
an IDXGISwapChain4, which every swapchain is on the DXGI runtime of
Windows 10 and later. On DXGI 1.1 or 1.2 - Windows 7 and 8, and Wine
- the object ends at IDXGISwapChain or IDXGISwapChain1, and
CheckColorSpaceSupport, SetColorSpace1 and SetHDRMetaData, called
unconditionally at init, went through slots past its vtable: the
d3d11 driver crashed in dxgi_set_hdr_metadata() the first time the
threaded-video harness ran a Windows build against it. Both helpers
now ask the object for the interface those calls belong to and do
nothing where it is not one.
The harness builds and runs as a Windows program: its scratch
directory comes from path_mkdir() under TMPDIR, TEMP or /tmp rather
than a shell's mkdir -p (cmd.exe has neither the flag nor the path),
argv[0] may carry backslashes, the harness core's two extra exports
are RETRO_API so a DLL exports them, and build.sh compiles the core
with the compiler the tree's own compile line names, so a cross build
gets a core it can load. A d3d11 lane on a Windows runner over WARP
through the MSYS2 MinGW64 build joins the workflow, advisory until
its first green run.
rh265_video_set_thread_pool() gives the decoder an rthreads pool and
a thread count, and a single-slice picture coded with
entropy_coding_sync - x265's default - then has its CTB rows decoded
as the wavefront the syntax was made for: the calling thread and the
pool claim rows in order from a shared counter, a row takes its CABAC
contexts from the row above once that has passed its second CTB, and
each CTB waits until the row above is two CTBs ahead, which is what
the intra neighbours, the motion-vector candidates, the SAO merge and
the deblocking edge marks all fall inside. Extra threads decode on
shadows of the decoder, a shallow copy per picture, so the planes,
metadata and reference lists are shared and only the CABAC engine,
scratch and per-CU state are private. Rows claimed in order wait only
on rows already running, so no pool size can deadlock; a row that
fails stops the others. Pictures without WPP, with several slices or
one row, and decoders without a pool, take the serial path as before.
The MP4 stream hands its conversion pool to the decoder, so the
Animated Thumbnail Threads setting covers it. The blit-bands oracle
gains an HEVC fixture coded with WPP (six CTB rows) and requires every
picture byte-identical to the serial decode; the gfx samples workflow
runs it under ASan and TSan, and the compile matrix lanes for the
threaded decoders take rh265 in.
The eight-point and six-point ramps that interpolate BC3's alpha and
BC4's and BC5's channels divided by 7 and by 5 and truncated. Measured
against a real-arithmetic decoder rounded once -- what the D3D
reference produces -- that was off by one on a quarter of the values,
mean error 0.26. Half the divisor is added before each division; the
numerator is non-negative on this path and the divisor odd, so that is
exactly round-half-up, and the same measurement now reads zero, mean
and maximum, over three million values. The signed BC4 and BC5 variants
take the separate 16.16 ramp, which already rounded, and are unchanged.
The colour block was measured the same way and was already exact.
The block test gains the case that tells: a0 = 255, a1 = 0, index 2 is
six sevenths, 218.57, which rounds to 219 and truncated to 218.
The Vulkan workflow had the validation-layer run and the TSan run of
the threaded-video harness, and no address,undefined build of it; the
GL matrix had none either. Both get one, the same check-vulkan and
check-driver targets with SAN=address, ubsan-vulkan.supp covering the
vendored SPIRV-Cross downcast as it already did for the sample runs.
The context-only vulkan_validation sample under samples/gfx, which
brings vulkan_common.c up and down under the layer and selects every
memory type, has never run in CI; it gets a job of its own, with its
exit 77 (no device) taken as a skip.
The DDS loader decodes its blocks with static functions -- bcdec's BC1
through BC7. A core that has compressed blocks from somewhere other
than a DDS file, a replacement texture its GPU cannot take as-is, has
been carrying a second copy of the same decoders to unpack them itself.
The four it needs are public now: rdds_decode_block_bc1, bc2, bc3 and
bc7, each taking one block, a 4x4 RGBA8 output with R first, and the
row pitch, over the same static functions the loader uses.
The first rdds test comes with them, on blocks whose result is known by
hand: BC1 with red and blue endpoints and every index 0, then every
index 1, then the four-colour row 0 1 2 3, which must land on exactly
two-thirds and one-third of each endpoint; BC2 with explicit alpha 255
then 0 over that colour block; BC3 with a0 everywhere; BC7 mode 6
writing every byte of the output over a sentinel; and a pitch of 32
leaving the gap between rows untouched. ASan and UBSan clean, C89 on
Linux and Win32, the test builds for Windows, and it joins Makefile.test
and the samples workflow.
When the acquire came back out of date, the swapchain was rebuilt and
the acquire tried again, and if that came back out of date too the
frame thread slept 10 ms and went round again, for as long as the
window kept changing under it. Now it is rebuilt once: a swapchain
that is out of date the moment it is made means the window is still
moving, so the frame goes without an image the way a frame with no
swapchain does, and the next frame's acquire builds a new one. The
runloop already paces a window with nothing to present.
memdecommit() spelt the advice MADV_DONTNEED, which a libc built to a
strict POSIX profile does not have; it uses posix_madvise() with
POSIX_MADV_DONTNEED where that is the one available, and skips the
advice where neither is, which keeps the pages and nothing else.
A recompiler that has to leave a block it emitted cannot use the C
library's longjmp everywhere. On Windows x64 that longjmp walks the
unwind tables, and a JIT block has none, so it faults; on POSIX the
plain setjmp also saves and restores the signal mask, a system call on
the path an interpreter takes at every state change. fastjmp_set stores
the return address, the caller's stack pointer and the ABI's
callee-saved set, fastjmp_jmp restores them and returns from
fastjmp_set a second time with the value; nothing is unwound and no
mask is touched. Cores with a JIT have each carried their own; this is
the one to carry.
Three sequences, one per ABI, as file-scope assembly in one C file:
x86-64 System V, x86-64 Win64 under MinGW -- which must also preserve
xmm6-xmm15, so its buffer is 240 bytes and 16-aligned for movaps --
and AArch64 on every OS, with the Mach-O underscore where Apple wants
it. MSVC has no file-scope assembly on x64, so the Win64 sequence is
also fastjmp_msvc_x64.asm for ml64, beside the C file, and under
_MSC_VER the C file compiles to nothing. Anywhere else the two names are
setjmp and longjmp, correct without the two properties, so a core
compiles on every architecture and gets the fast form where it exists;
FASTJMP_FORCE_FALLBACK selects that form on purpose, for comparing.
The test jumps from three frames down and gets its value back, holds
eight live values across the set at -O2 -- where a wrong register list
is what would fail -- and does a hundred thousand round trips. It runs
natively and with the fallback forced. ASan and UBSan are clean with
detect_stack_use_after_return off, which the workflow's RUN_ENV sets
for this one target: a longjmp that is not libc's cannot be followed by
the fake-stack check, as libco's switch cannot. C89 -pedantic
warning-free on Linux, Win32 x64 and x86, where it takes the fallback;
the Win64 object carries the xmm saves.
rvp9_set_tile_pool() gives the decoder an rthreads pool and a thread
count, and a frame coded in more than one tile column then has its
columns decoded side by side: contiguous column ranges, the calling
thread taking the first and the pool the rest, joined before the loop
filter. Each extra thread decodes on a shadow of the decoder, a
shallow copy taken per frame, so every buffer and all frame-level
state stay shared and only the tile-local fields - bool decoder, left
contexts, scratch, counts, edges, the corrupt flag - are private;
after the join the shadows' counts are summed into the frame's, so
backward adaptation is unchanged, and their corrupt flags merged. VP9
tile columns share nothing else: no entropy state, and neither the
intra predictor's above-right nor the motion-vector candidate scan
crosses a column edge, so this changes when pixels are written, never
what they are. The tile loop moves into rvp9_decode_tile_cols(), each
thread walking the tile size prefixes to its own columns; frames coded
as one column, and decoders without a pool, take the old path.
The WEBM and MP4 streams hand their conversion pool to the decoder, so
the Animated Thumbnail Threads setting covers tile decoding too. The
blit-bands oracle now takes WebM and a new four-column VP9 fixture,
requiring every frame byte-identical to the single-threaded decode;
the gfx samples workflow runs it under ASan and TSan, and the compile
matrix gains threaded and threadless lanes for the decoder and the
video streams.
memshm_create() moved a descriptor 0 with F_DUPFD_CLOEXEC, which is
POSIX 2008 and absent from the Orbis libc and the older Apple SDKs, so
the PS4 and PowerPC macOS builds stopped compiling memmap.c. F_DUPFD
and a separate F_SETFD are available on every one of them, and the
gap between the two cannot leak the descriptor to a child this
process has not forked yet.
tools/companion_thumbs_test.sh links rmp4_video.c, which now blits
through image_blit_bands.c on a tpool; both join its source list, in
all three of its sanitizer variants.
memmap.h declared its functions with no extern "C" guard, so a C++ TU
that included it looked for mangled names and failed to link against
memmap.c's C symbols. Every other libretro-common header a core includes
from C++ has RETRO_BEGIN_DECLS; this one has it now, around the
declarations and after the platform include block. Checked with a C++
TU that calls memshm_create and links; both memmap samples unchanged;
C89 -pedantic warning-free.
vulkan_create_swapchain() and vulkan_destroy_swapchain() drained the
whole device, without the queue lock, on every resize, fullscreen
toggle and out-of-date acquire: a threaded hardware core's queued work
was waited for by a window event, unsynchronised with its submissions.
The only work that references the swapchain images from this side is
the context's own frames, each with a fence, so a rebuild or teardown
now waits on those and nothing else; the old swapchain retires through
oldSwapchain as before, and the video driver rebuilds its framebuffers
after waiting on the same fences. The one drain left in the context is
the teardown before the device itself is destroyed.
gfx_widgets_deinit() cleared the worker flag and freed the widget
fonts while the video thread could still be inside a frame drawing
with them: the frames the wrapper had queued carried the main
thread's widgets-active flag from before the deinit, and nothing
between them and font_driver_free() held the state lock - a font
read on the video thread against a free on the main thread, which the
threaded-video harness under ThreadSanitizer reports on the reinit
lane. The worker is now taken away under the state lock, so a frame
being drawn finishes first, and gfx_widgets_frame() draws nothing
once the worker is gone while the wrapper is up, so no later frame
touches the fonts the deinit then frees.
The harness targets print the whole sanitizer report when a run
fails, not just the lines matching the lane results.
memshm_create returned the file descriptor plus one, to keep 0 out of a
pointer-sized handle. That made the handle opaque in a way the Windows
side is not -- there it is the HANDLE itself -- and a core that maps the
region some way memshm_map does not offer, MAP_FIXED into a reservation
it owns for a fastmem mirror, has to hand the descriptor to mmap and was
handing it the wrong one.
The handle is the descriptor now, as the header says. Descriptor 0 is
kept out of it the proper way: a region that lands there -- only if
stdin was closed -- is moved with F_DUPFD_CLOEXEC before the handle is
made. Checked that the returned handle is a live descriptor; the sample
passes; C89 on Linux and both Win32 targets, and the iOS and DOS shapes,
are unchanged.
memsync exists for a JIT that has just written code, and it was only
right on 32-bit ARM. Its test was __arm__, which aarch64 does not
define, so on 64-bit ARM Linux and macOS it fell through to msync --
which flushes data, not instructions -- and on Windows it did nothing.
It names __aarch64__ now beside __arm__, uses sys_icache_invalidate on
Apple and __builtin___clear_cache elsewhere on ARM, and on Windows calls
FlushInstructionCache, which ARM64 Windows requires and x86 ignores.
Checked by preprocessing: aarch64 selects the clear_cache branch, x86
Linux msync, Win32 FlushInstructionCache.
memreserve_at(hint, len) is memreserve with a preferred base -- a fastmem
window that has to sit where the recompiler expects it. One VirtualAlloc
at the hint on Windows, an mmap of PROT_NONE at the hint on mman
platforms; the hint is never forced, and the caller compares.
The memshm sample gains a reservation at an 8 GiB hint, committed,
written at both ends, memsynced and released, reporting whether the
hint was honoured rather than asserting it. Linux and both Win32
targets are warning-free under C89 -pedantic; the iOS and DOS shapes
compile with their stub headers; ASan and UBSan clean.
The WEBM and MP4 streams gain set_blit_pool(): an rthreads pool and a
band count, and each decoded frame's YUV-to-pixel conversion then
runs as row bands on the pool, the decoding thread taking one band
and joining the rest, through the new image_blit_bands() splitter.
Bands start on chroma-row boundaries; an untagged matrix is resolved
from the frame's full height before the split so a band's height
cannot pick BT.601 over BT.709; the HDR blits' lazily built tables
are warmed before any band can race the build; and with no pool, one
band, or too few rows for a split to pay, the conversion is the one
call on the decoding thread it always was.
The thumbnail decode worker owns the pool: the new Animated Thumbnail
Threads setting (menu_thumbnail_preview_threads, 1 by default, which
is exactly the previous path) says how many bands, the poll publishes
it, and the worker re-sizes the pool between frames and destroys it
after its join. tpool moves from the FFmpeg block to the threads block
in both build systems. A new preview sample decodes a fixture on one
thread and in eight bands and requires every frame byte-identical.
On the Ubuntu 24.04 runner lavapipe's rasteriser lives in
libgallium-24.0.x.so, which the ICD dlopens, so ThreadSanitizer's
frames carry that name and tsan-vulkan.supp - which named the ICD,
the layer and LLVM from a Mesa 25 host where the code sits in the ICD
- did not cover them: the first internal atomic there failed the lane
after its first pass. The loader, gallium and libGL join the
suppressions for the Vulkan and GL lanes; every entry is still a
library name, so a race in the frontend still fails the run.
check-vulkan also prints sanitizer reports, runtime errors and X
errors from the run log along with the lane results, as check-driver
does: a lane that fails with exit 66 and nothing else in the log is
exactly what the runner showed.
When a frame acquired a swapchain image but never submitted, the next
acquire found the previous acquire semaphore still pending: it could
be neither acquired with again nor destroyed, so the whole device was
drained - without the queue lock - to destroy it. Now it goes on a
stale list, and the next submission waits on it alongside its own
acquire semaphore, which consumes the signal; it recycles with that
frame's fence like any other. Each frame's waited semaphores are kept
as a list for that, the recycle pool grows to hold them all, and the
hardware-core submission reserves room for them. The drain remains
only as the fallback once a whole swapchain's worth of frames in a row
went unsubmitted.
font_driver_rebuild() only marshalled to the video thread for a font
made with a threading hint. The OSD font is made on the thread that
owns the context and carries none, but a font-size change rebuilds it
from the settings path on the main thread, and a GL backend then makes
its context current on a thread that does not hold it - an X
BadAccess on GLX, which is how the threaded-video harness found it on
its first run on the gl driver. A rebuild now goes through the video
thread whenever the wrapper is up, and video_thread_font_init() runs
the backend directly when it is already on that thread, so a worker
that rebuilds a font while it initialises the driver does not send a
command to itself.
The harness gets a check-driver target for any real driver named in
DRIVER, and the workflow a gl/glcore matrix over llvmpipe, release and
ThreadSanitizer (tsan-gl.supp names Mesa's uninstrumented libraries),
so the frontend's driver calls run on the wrong thread nowhere without
CI seeing it.
The WEBM and MP4 streams gain set_output(): the caller's frame to blit
the decoded planes into instead of the stream's own, returned by
stream_next as before. image_transfer_anim_stream_set_output() carries
it through the type switch and reports which types take it - the video
streams, whose blit has one destination either way; APNG and WEBP
compose on a persistent canvas the next frame is built from, so their
frames still come out of the canvas.
The thumbnail decode worker points each job's stream at the surface
slot the job owns, so a video frame is uploaded from where it was
decoded and the per-frame copy into the slot goes with it; the
synchronous decode path does the same into slot 0, first making sure
the slot is not still on its way to the video thread. Canvas streams
keep the one copy that decouples them from the decoder.
Two lanes 73462ad broke.
pthread_jit_write_protect_np exists on macOS and nowhere else, and the
gate was __APPLE__ && __aarch64__, which iOS and tvOS satisfy too. It is
TARGET_OS_OSX now, from TargetConditionals.h, so those two get the empty
memjit stubs and macOS on Apple Silicon keeps the toggle.
DJGPP defines __unix__ and so HAVE_MMAN, but ships a stub <sys/mman.h>
with no MAP_SHARED and no shm_open; the memshm mman branch took it and
failed. It is gated on MAP_SHARED being defined, the way reserve/commit
in this file already gates on its own constants, and DOS falls through
to the stubs that report NULL.
Checked with header shapes for both: an Apple arm64 build with
TARGET_OS_OSX 0 reaches no pthread_jit symbol, a __DJGPP__ build with a
stub mman.h reaches no shm symbol, and a TARGET_OS_OSX 1 build still
calls pthread_jit_write_protect_np. Linux and both Win32 targets are
unchanged; both memmap samples pass.
Two compatibility holes, one older than 73462ad and one from it.
FILE_MAP_EXECUTE is XP SP2 / Server 2003 SP1. The kernel rejects it
before that, which is fine -- MapViewOfFileEx fails and the call reports
NULL rather than mapping without execute -- but an SDK old enough may not
spell the macro, and memmap.c's mmap shim has used it since before
memshm existed. It is SECTION_MAP_EXECUTE_EXPLICIT, 0x0020, declared at
the top of the Windows block when the headers do not, so the VS2005 job
at _WIN32_WINNT=0x0400 builds either way. Checked by undefining it after
<windows.h> and compiling: the shim and memshm both find it.
memshm_create passed its length to ftruncate through a cast to off_t.
libretro-common does not set _FILE_OFFSET_BITS, so on 32-bit Linux off_t
is 32 bits and a length of 2 GB or more wrapped to a negative one.
Refused now, before the cast: a caller on a 32-bit off_t that needs more
has no way to get it through this call, and a NULL says so.
The rest of the surface, for the record: the read/write shared-memory
path is NT 3.1 on Windows, glibc 2.2 with -lrt on Linux, any macOS;
memfd on Android is kernel 3.17; pthread_jit_write_protect_np is macOS
11.0 and is only reached on arm64, where nothing older exists.
Two things a core with a fastmem window needs from the host that memmap
did not offer, both taken from what such a core had to carry itself.
memshm_create, memshm_map, memshm_unmap and memshm_destroy: an
anonymous-backed region that can be mapped at several addresses, which
is how a guest RAM is mirrored into a large reserved window. Windows uses
a pagefile-backed file mapping and MapViewOfFileEx at the hint; Android a
memfd, since Bionic has no shm_open; everything else shm_open with the
name unlinked at once, so it never outlives the process. The hint is
never MAP_FIXED -- that silently replaces a neighbour -- and a caller that
wanted a particular address compares the result. The fd rides in the
handle as fd + 1, so the encoding never depends on 0 being stdin. On the
consoles with no mman every call reports NULL.
memjit_write_begin and memjit_write_end: on Apple Silicon, where MAP_JIT
pages are write-xor-execute per thread, they switch the calling thread
to writing and back with pthread_jit_write_protect_np, depth-counted per
thread so one thread's nesting never flips another's pages. Everywhere
else they are empty.
The test maps one region twice, writes a pattern through the first
address and reads it through the second, changes a word through the
second and sees it through the first, then unmaps both and destroys the
handle. memjit is called nested and returns. C89 -pedantic warning-free
on Linux and both Win32 targets at 0x0400, the test builds for Windows,
ASan and UBSan are clean, and it joins Makefile.test and the samples
workflow.
Fast-forwarding a core that renders through its own audio callback
sped the audio up and crackled, on every driver (#19541, second
half). Stable never touched callback audio during fast-forward, and
that was right: the callback is paced by the device draining, not by
retro_run, so running the video faster produces no audio surplus for
the speed machinery to absorb.
Worse than needless, the machinery misfires there. The speedup
multiplier is measured from flush cadence, and on the callback's
audio thread the cadence IS the device draining: the measurement can
only confirm whatever multiplier it last applied, seeded at the
configured fast-forward ratio. A feedback loop wearing a
measurement's clothes - the same pathology the threaded pipeline
already sidesteps by measuring on the producer's side, except a
callback core has no producer-side delivery to measure at.
Every speed consumer reads the published runloop snapshot, and a
callback core never runs the threaded pipeline, so one gate covers
them all: audio_driver_publish_runloop() withholds the fastmotion
bit while a callback core owns audio. The callback loop's cadence
reset then arms itself, reentry starts at unity, and the flush,
transport tempo and multiplier sites all see plain realtime. The
configure-time tempo derivation reads the runloop directly rather
than the snapshot, so it carries the same gate.
For anyone who wants the opposite, "Fast-Forward Affects Callback
Audio" (audio_fastforward_callback, default off) opts a callback
core into the speed handling - with a measurement that can work.
What fast-forward changes for such a core is retro_run cadence, so
audio_driver_ff_callback_frame_end() measures exactly that at the
frame boundary: one video frame's worth of audio at the core's input
rate against the wall clock between frame ends, through the same EMA,
published in pipe_ff_mult_q16 as the pipeline's producer measurement
is. The audio thread's flush reads that word and never measures its
own cadence. Ownership of the inline cadence state thereby moves to
the main thread's frame boundary; the audio thread's idle reset
remains for the menu and pause, where no frames end, and the two
only ever race zero against zero.
The ff_speedup suite is updated to the new contract - opted out, a
callback core's cadence resets in every mode and fast-forward stays
at unity; opted in, the multiplier follows the frame cadence and
release returns to unity.
Fixes the fast-forward half of what remains of #19541.
The null-driver TSan job never runs driver code on the video thread,
so a plain field shared between the frame thread and a driver's frame
- the shader chains' FrameTimeDelta was one - only shows up with a
real driver. A third job in the Vulkan workflow builds the tree with
-fsanitize=thread and runs check-vulkan on lavapipe. The validation
layer, the ICD and its LLVM are not instrumented and their internal
atomics read as races, so they are suppressed through
tsan-vulkan.supp; ubsan-vulkan.supp covers the vendored SPIRV-Cross C
API's opaque-handle downcast for the address,undefined build. The
suppressions name libraries only, so a race or an undefined operation
in the frontend still fails the lane.
frame_time_delta_us is written by the frame thread in
video_driver_frame() and read by the video thread from inside the
driver's frame, where the shader chains take their FrameTimeDelta - a
plain unsigned crossing threads, which the threaded-video harness on a
real driver under ThreadSanitizer reports on its first frame. It is now
a retro_atomic_int_t stored and loaded relaxed: every read is a whole
value, and no ordering is claimed since none is needed - a pass that
sees the previous frame's delta draws the previous frame's motion.
Controllers kept driving RetroArch after another window took focus
(issue #5004): "Pause Content When Not Active" stops the core, but the
menu still navigated and the controller hotkeys (menu toggle, quit,
close content, reset, ...) still fired, and with that pause off the
core took the input as well. The keyboard is already ignored while
unfocused; controllers had no equivalent.
New setting input_joypad_background, "Background Controller Input"
(Settings > Input), on by default so nothing changes for anyone who
drives several instances from one controller. With it off and the
window unfocused, every controller read sees an idle pad: the core,
the hotkey/menu input gathered by input_driver_collect_system_input(),
menu_event() and input_key_pressed().
The gate is one flag, INP_FLAG_JOYPAD_UNFOCUSED, published once per
poll, and one accessor, input_driver_joypad_for_read(), which hands
the read paths an idle stand-in joypad driver while the flag is set.
The real drivers are still polled every frame, so nothing held before
focus was lost is replayed afterwards. Rumble, sensors, device names,
binding and driver lifetime keep using the real drivers. Joypad and
input drivers that have their own notion of background input can
follow the same flag and setting.
tools/settings_table.reference is regenerated; it also picks up three
rows master already emitted but had not recorded
(audio_resampler_hq_oversampling, input_sdl3_system_keyboard,
audio_latency_floor).
Menu text dialogs on the Vita now open the native SceImeDialog
instead of the built-in on-screen keyboard (issue #3435), through the
native-keyboard contract the SDL3, Android and iOS backends already
use.
psp_input.c drives the dialog once per poll: it opens when a menu
dialog wants text, and on Enter the typed text is converted from
UTF-16 and handed to the keyboard line followed by a Return, the same
way the libnx swkbd path delivers it. Close sends the Return with the
line empty, which the menu already treats as cancel. Password and
integer dialogs get the password text box and the number keypad. If
the menu drops the dialog while it is up, the panel is aborted and its
result discarded.
The dialog is only drawn when the app calls sceCommonDialogUpdate()
each frame, so gxm_frame() now does that, after its scene ends, while
the panel is up. Only the GXM driver makes that call: under any other
video driver (the VITAGLES build) the dialog is not opened and the
built-in keyboard stays in use, as it does if sceImeDialogInit fails.
While the panel is up it owns the input: the pad reads as idle, USB/BT
keyboard presses are not typed into the line behind it, and touches
are held back from the menu until every finger lifts after it closes,
so the tap on its Enter key does not land on a menu entry.
vkQueuePresentKHR shared the graphics queue and so its lock, and on
some drivers the exclusive-fullscreen present waits for the display
inside the call - holding the lock a hardware core's submissions go
through. The frame fence wait before the present kept that from
stopping the frontend, at the cost of a wait on the video thread.
The present now goes to a second queue of the graphics family when the
device offers one: requested on the default path, and taken from a
core that creates the device through the negotiation interface when
its presentation_queue is a different queue of the same family (a
different family would need an image ownership transfer around every
frame, and is still refused). On that queue the present holds no lock,
the core's lock_queue never waits behind the display, and the fence
wait before the present is not needed and not done. A device with one
queue in the family keeps the shared queue, its lock and the wait.
Presentation on the second queue is synchronised with the render on the
first by the swapchain semaphore that already ordered them.
The affinity lane compares the mask a thread ends with against the one
it inherited, but read the inherited one in a race with the main
thread's pin: on a multi-core host the pin can land first, the thread
records the pin as its inheritance, and clearing back to every CPU
then reads as a failure. The thread now reports the read before main
pins, and both waits yield.
Every vkQueueWaitIdle and vkDeviceWaitIdle in the video driver drained
the core's work along with the driver's, and had to run under
queue_lock - so a hardware core parked in lock_queue with work still
to submit, work the queue may be waiting on, could never let the queue
go idle, and the frontend stopped. The driver's objects are only ever
referenced by the driver's own submissions, each of which carries a
fence: vulkan_wait_own_submissions() waits on those, lock free, and
retires the staging uploads on the way. Overlay reload, font free and
the ribbon VBO rebuild no longer wait at all - the old objects park on
the deferred list. The slang chain takes the same wait through a new
wait_submissions callback for its rebuild and teardown, falling back to
the device drain only for a driver that gives none. The idle waits that
legitimately remain are named in tools/vulkan_idle_wait_allow.list, and
tools/vulkan_idle_wait_check.py fails on any other, and on a stale
entry.
The threaded-video harness now runs on a real Vulkan context on request
(HARNESS_VIDEO_DRIVER, HARNESS_VERBOSE), with a check-vulkan target that
fails on any validation error, and a CI lane runs it on lavapipe under
VK_LAYER_KHRONOS_validation. What the layer found on the first run is
fixed here: a frame texture replaced on a geometry change was destroyed
and its memory reused while a pending frame still sampled it (the write
into freed memory that a reallocated wrapper then inherited), so
vulkan_create_texture() now parks the old texture instead of pilfering
it; a zero-extent frame asked for a zero-sized image and buffer every
frame, and is skipped, as is the lend of a zero-sized software
framebuffer; streamed textures are moved to GENERAL layout by a one-shot
submission at creation, so no image barrier lands inside the render
pass (not allowed on a sampled image) and no per-draw host barrier is
issued (host writes before a submission are visible without one); and
the wrapper hands textures it still held at teardown - the retire lists
and completed, undelivered async uploads - back to the driver on the
worker before freeing it, instead of assuming the driver takes them.
vulkan_context_t's debug messenger member is no longer under
VULKAN_DEBUG, since a mixed build shifted every field after it, and
x11_get_refresh_rate() reports 0 rather than NaN on a server without
XF86VidMode.
The seqlock lane needs a reader to see the odd stamp at least once,
which a writer that finishes all its passes before any reader is
scheduled never gives it: the pass count is now a floor, and the
writer goes on, up to a bound and yielding every so often, until a
reader has retried. The spsc test's spin waits yield the core, so a
side waiting on the other is not spending its whole slice finding
that out; with them it completes in seconds where it timed out.
Give up the rest of the timeslice to any runnable thread. It is for the
back-off in a bounded spin once the bound is reached and the caller is
not yet ready to park, and for the one place a lock cannot be taken -- a
fault handler, where a mutex is not signal-safe and a spinlock with a
yield fallback is the only shape that works. It is never a substitute
for a real wait, and the header says so.
One call per backend, each the name that backend's own SDK declares:
SwitchToThread, LWP_YieldThread, svcSleepThread(0) on both 3DS and
Switch, sceKernelDelayThread(0) on PSP and Vita, OSYieldThread,
RotateThreadReadyQueue(0), and on PS3 through the existing per-SDK shim,
sysThreadYield under PSL1GHT and sys_ppu_thread_yield otherwise;
sched_yield elsewhere. The console names were checked against the SDK
headers, since the console lanes are the only place they compile.
The sthread_ex sample calls it a hundred thousand times; there is no
property of a yield to observe beyond it returning. C89 -pedantic
warning-free on Linux and both Win32 targets; ASan, UBSan and TSan clean.
bionic declares cpu_set_t and sched_setaffinity only under _GNU_SOURCE,
which must be set before the first system header - not something a
unity build's include order provides. The Android arm now builds the
kernel's plain bit mask itself and makes the sched_setaffinity syscall
directly, for another thread by its tid (API 21) and for the calling
thread, leaving cpu_set_t to the other Linux builds. The compile matrix
gains the two Android arms of the file.
sthread_create_with_stack_size takes a size_t, and the header is
included from translation units that have not seen stddef.h first;
it now brings the type with it.
Two things a caller could not ask rthreads for, both of which an
emulator's threads need: a stack larger than the platform default, for a
recompiler that runs the guest on the host stack, and a pin to a CPU.
sthread_create_with_stack_size(func, data, size) joins sthread_create and
sthread_create_with_priority; all three now call one internal
sthread_create_ex, so the size reaches every backend whose create takes
one. Windows passes it to CreateThread; pthread sets it on the attr,
keeping Apple's existing 2 MB floor and going above it when asked; each
console backend uses it in place of its STACKSIZE default. 0 means the
default everywhere, so every existing caller is unchanged.
sthread_set_affinity(thread, mask) and sthread_set_current_affinity(mask)
take a bit-per-CPU mask, 0 meaning every CPU, which is how a pin is
undone. Windows takes the mask as is on the thread's real handle; glibc
builds a cpu_set and uses pthread_setaffinity_np; Android has no
pthread_setaffinity_np, so it names the thread by pthread_gettid_np to
sched_setaffinity, which is API 21 -- below that only the calling thread
can pin itself, and pinning another reports false. Darwin has a
scheduler hint and no hard affinity, and every console backend has none;
all report false rather than pretend.
The test uses both rather than reading them back: a thread on a 16 MB
stack recurses 12 MB deep and returns, which the 8 MB default would
fault on; a thread pinned to CPU 0 reads its mask from the kernel, then
clears it and reads again against what it inherited -- on one CPU those
coincide and the test says so. It builds for Win32 at 0x0400 on both
targets, and is clean under ASan, UBSan and TSan; the other four rthreads
samples still pass, and it joins them in Makefile.test and the samples
workflow. rthreads.c compiles C89 -pedantic warning-free.
id is Objective-C's object type, so a parameter of that name does not
parse in metal.m; the in-place texture update takes its handle under
the name the file's other texture entry points use.
The overlay interface gains load_textures(): a page's textures by
handle, from which the driver takes only its per-image geometry and
alpha, uploading and owning nothing. When the driver has it, the
frontend uploads every unique image of the pack once through
video_driver_texture_load() on enable - the loader already
deduplicates by path, so a page's entry shares its pixels with one
unique image - and every page gets a list of handles into that set,
so a page switch decodes nothing, uploads nothing, creates and
destroys no texture and waits for no GPU. The handles are unloaded by
the disable, while the driver is still there to do so; a driver that
declines the handles (the wrapper answers for any driver) puts the
pack back on load() for that enable, and drivers without the entry
are unchanged.
gl2 and gl3 keep the pack's names in their overlay block behind an
OVERLAY_BORROWED flag so the page's free deletes nothing, gl3 no
longer generating a spare set of names it then overwrote. Vulkan,
D3D11 and D3D12 draw from copies of the pack's texture structs, again
never released with the page: Vulkan's vkQueueWaitIdle and D3D12's
queue drain stay on the pixel path only, and both D3D drivers reuse
the page's sprite buffer while it is big enough and free the sprite
array a switch used to leak. Metal's Overlay retains the pack's
MTLTextures in its array. The threaded wrapper marshals the new call
beside CMD_OVERLAY_LOAD.
A gfx_surface owns one GPU texture and the pixel slots a producer
writes into. The first submit creates the texture and every later
same-size submit rewrites it in place through the new update_texture
poke (gl2, gl3, Vulkan, D3D11, D3D12, Metal), so a playing preview
costs one upload per frame with no texture creation or destruction
and no allocation. Under threaded video the submit posts the surface's
own embedded node to the video thread - a descriptor, never a copy of
the pixels - and the slot stays the surface's until the completion is
delivered on the main thread; without the wrapper the same call
collapses to a direct driver call. Drivers without an in-place path,
or a handle they decline, get a replacement texture per frame, and a
surface freed mid-flight swallows its own completion.
Animated thumbnails decode straight into the surface's two slots: the
job block no longer carries frames, a job whose slot is on its way to
the video thread is HELD until the surface releases it, and the
surface lives until gfx_thumbnail_reset so the last frame stays as
the still. Vulkan keeps two persistent mapped staging buffers and two
fences per updated texture and records each copy as a one-shot submit
that queue order places ahead of the next frame; with both copies in
flight the frame is dropped rather than waited for. The
gfx_thumbnail_anim sample exercises both the in-place and the
replacement route, and the compile matrix gains gfx_surface lanes with
and without threads.
vkQueuePresentKHR runs under queue_lock, and on some drivers it does
not return until the image has finished rendering: the NVIDIA exclusive
fullscreen path spins inside the call until then. A hardware core under
the video thread wrapper submits from the main thread through
lock_queue, and its work is not all on the queue when the frame's is -
a renderer that flushes per queue can have a graphics batch ahead of
the frame waiting on a compute or transfer batch it has not submitted
yet. The present then holds the lock waiting for a submission that is
blocked on the lock, and the frontend stops with the core parked in
lock_queue and the video thread inside the driver.
For a hardware core under the wrapper the frame's fence is now waited
out before the present, with the lock free, so the core submits, the
queue drains and the present has nothing left to wait for. Software
cores and hardware cores presenting on their own thread have no other
submitter to wait on and keep the asynchronous present unchanged.
Nineteen waits in this driver had no timeout. Seventeen were the same
signal-fence-then-wait sequence written out at each call site, which is
now d3d12_queue_drain(), and two were the hardware ring's capture
fence.
The GPU signals a fence when the work submitted before it has retired.
One that has been reset or removed never signals, and these run on the
frontend thread, so waiting without a bound freezes the application
rather than the renderer. Two seconds is the bound; on expiry the
driver says the device is most likely lost and carries on, which is
the same shape d3d11.c already uses for its frame-latency wait.
The remaining INFINITE is in the hardware ring's own wait helper,
where the caller asks for it by passing HW_RING_WAIT_FOREVER.
The three pre-existing mingw header errors in this file are unchanged
in count, and the helper is defined above all seventeen of its callers.
wdmks_free() waits for outstanding packets to come back before freeing
the buffers the kernel is using. The wait is bounded at one second
across all four packets and spent on each packet's own event, which
the kernel signals on completion or cancellation. A device that has
stopped answering therefore costs a second at a driver switch rather
than freezing the frontend thread indefinitely.
A packet that does not come back keeps its pending flag, and its
buffer, OVERLAPPED and event are left allocated rather than freed:
the kernel may still be using them, and a few kilobytes lost to
hardware that has stopped responding is cheaper than freeing memory in
use.
metal.m was written against ARC, but only the command-line Makefile
(per-file -fobjc-arc) and the Xcode configurations inheriting
BaseConfig.xcconfig compile it that way: the griffin_objc.m unity TU
takes whatever mode the enclosing configuration sets, and the QT,
Steam, App Store and CI configurations of RetroArch_Metal.xcodeproj
set no CLANG_ENABLE_OBJC_ARC, so they build the driver as MRC -- a
mode where __weak is a hard compile error and __bridge_retained /
__bridge_transfer silently compile to no-op casts, turning every
ownership transfer in the file into a use-after-free (drawables,
command buffers and encoders stored from autoreleased factory
results and read after the pool drains; the overlay image array) or
a leak (all RARCH_STRUCT_ASSIGN teardown, the metal_free /
unload_texture / raster_font_free handle transfers, every [X new]
temporary, the BufferChain tail trim, and the alloc'd instance on
every failed init). Route every transfer through the RARCH_* ARC/MRR
ownership layer in libretro-common's defines/cocoa_defines.h --
extending it with the value and transfer forms this file needs
(RARCH_AUTORELEASE_R, RARCH_RELEASE_NIL, RARCH_ASSIGN, the bridge
transfer forms, RARCH_STRUCT_ASSIGN, RARCH_WEAK,
RARCH_RETURN_INIT_FAILURE), documented next to their definitions --
give every class a dealloc that releases
its objects under MRC, release self on init-failure paths, wrap the
driver entry points that run on the video thread outside
renderFrame's pool in @autoreleasepool, and spell out the ownership
of the Texture and BufferNode properties MRC would synthesize as
assign. This also fixes three bugs present under ARC: FrameView
teardown never freed the shader chain's bridge-retained engine slots
or the calloc'd shader struct, STRUCT_ASSIGN on the __strong HDR
texture ivars pinned an extra reference per texture, and
MetalRaster's init-failure path leaked the font driver allocation.
Fetch viewport readback rows in bounded chunks instead of one
getBytes call per scanline: readBackBuffer and readViewportHDR pull
as many whole rows per Metal call as fit in a 1 MiB scratch buffer
(stack for small captures), collapsing the 1000+ API calls per 1080p
frame the recording path pays through read_viewport into a handful
while preserving the issue #19038 overscan clamping exactly. Stop
dispatching non-mipmapped texture loads to the video thread: only
mipmap generation encodes into the shared blitCommandBuffer; plain
loads touch just the thread-safe MTLDevice, a fresh texture and the
immutable sampler table, so menu icon and thumbnail loads no longer
pay a per-icon video-thread round-trip. Both modes compile the file
warning-free against a stubbed SDK; the clang static analyzer's
retain-count checker reports no ownership defects in either mode;
the MRC macro expansions pass a GNUstep retain-count/dealloc test
under ASan+UBSan+LeakSanitizer; the chunked loop matches the per-row
implementation byte-for-byte across ten viewport/overscan cases
under the same sanitizers; and concurrent readbacks plus the
inline-load/video-thread interleaving run clean under
ThreadSanitizer.
Rewrite BufferNode/BufferChain as C89 structs with static functions,
embedded by value in Context: the per-frame transient-buffer
allocator is the one piece of driver-internal state whose accessor
dispatch multiplies with scene complexity (an allocRange per draw
batch plus node walks per frame in all three chains), and as plain
aggregates it inlines into its callers and messages only the Metal
objects themselves. Measured on the extracted allocator (clang -O2,
arm64): 556 -> 248 instructions, 38 -> 10 objc_msgSend sites (the
survivors are the irreducible MTLBuffer/MTLDevice calls), 92 -> 28
calls, and over a third fewer memory operations, since a msgSend
boundary forces every field to be reloaded after each opaque call.
The nodes lose their per-object isa and refcount words, the chains
their heap allocations, and MTLBuffer ownership runs through the
same RARCH_STRUCT_ASSIGN discipline in both modes; the extracted allocator
passes a GNUstep behavioral test (alignment stepping, node growth,
tail trim, quiescent-frame retention, zero live buffers at teardown)
under ASan+UBSan+LeakSanitizer. This also removes the unused
kConstantAlignment constant, the file's one standing warning.
The last availability check in the file guarded a dylib_load of
MoltenVK behind macOS 10.15 / iOS 13 / tvOS 12. It guarded no symbol
use: on an older OS the load simply fails and falls through to the
versioned framework name, which is what the check produced too. The
three loads are now attempted in order - the loader, MoltenVK, then
MoltenVK-v1.2.7.framework - and a plain C file no longer carries
clang's availability runtime. Apple-only code under __APPLE__; the
linux build and its declaration gate confirm nothing else moved.
The comment in video_shader_dir_get_anchor() said the menu writes
retroarch.* to the config directory. Apply Changes, and appending or
prepending a preset, write it into the Video Shaders directory first,
falling back to the menu config and config directories only when that
fails. The code was already right: video_shader_dir_has_preset()
ignores retroarch.*, so a folder holding only that file does not
anchor. Comment only, no functional change.
6d88945 (shaders: the menu copies the driver's loaded preset, not the
disk) made video_shader_apply_shader() call
video_shader_driver_get_current_shader() and
menu_shader_manager_set_preset_from_live(). The sample links
gfx/video_shader_parse.c for real, so without stubs for the two it no
longer links against master.
The driver stub reports no loaded shader, which keeps the apply on the
menu_shader_manager_set_preset() fallback the sample already stubs.
The map constructor checked string_list_new and nothing after it: each
append went unchecked and was followed by elems[size - 1], which on a
failed append indexes the previous entry or underflows an empty list,
and each strdup went unchecked, leaving an appended entry whose
userdata - the directory the manifest walks - is NULL. Every entry
now goes through one helper that copies the path first, appends
second, and attaches the copy only once both exist; any failure frees
the whole list, userdata included, and the constructor returns NULL,
which its caller already handles.
Verified: the cloudsync suite under plain, ASan+UBSan and TSan, the
full build with the declaration gate, and the 91-lane matrix.
--list-unaudited reported thread_func in rthreads.c: the scanner saw
sthread_create_with_priority(thread_func, ...) inside sthread_create
itself and took the forwarded parameter for a thread entry. A
create-call whose entry argument is declared as a function-pointer
parameter - (*name)( - in the same file names no function, and is
skipped. The selftest's ghost entry, a prototype with no definition,
is still listed as it should be; the selftest passes and the binary
audit's numbers are unchanged.
The download HTTP callback release-stored http_task_complete at its
top and then went on writing - the decompress task it pushed, and on
failure download_handle->status - so a worker that acquired the flag
could move to WAIT_DECOMPRESS while the callback was still writing
behind it, and a status the worker owns was being written from the
main thread. The failure is now a payload field, http_task_error,
written before the flag; the publish moves to the end of the
callback, after decompress_task and the decompress flag; and the
worker, on acquiring the flag, moves itself to the error state when
the field is set. status has one writer.
The refresh-flags harness had a race of its own that TSan reached
once the product one was gone: its mock server's listening socket
was a plain int read by the accept loop and retired by server_stop
from the main thread. It is an atomic now, published before the
thread starts and replaced once to stop it, with the accept loop
reading it fresh each lap.
Verified: the core_updater suites under plain, ASan+UBSan and TSan,
the refresh-flags test eight runs per pass with zero failures where
master fails two of eight under TSan; the full build with the
declaration gate and the ELF audit steady; the 91-lane matrix.
The Apple idle_wait shipped on a false premise. rarch_draw_observer
calls CFRunLoopWakeUp after every iteration unconditionally on macOS
and tvOS, so the run loop between callbacks is not a wait: the
caller's ten-millisecond sleep was the only thing pacing a paused
RetroArch on a non-Qt macOS build, and returning true without
waiting turned pause into a spin of the main thread. A nested
CFRunLoopRunInMode from inside the observer is no answer either -
it re-enters the observer and recurses. The OS-driven builds now
return false and keep the sleep; the HAVE_QT owned loop keeps its
pump, which runs from plain C and nests in nothing. The driver's
comment names where the real Apple wait belongs: an observer in
cocoa_common.m that does not wake the loop while paused, which
wants a Mac to verify.
UICO_ST_FLAG_IS_ON_FOREGROUND is removed. Its setters went in 2022
as companion-UI remnants and no reader was updated, so the flag has
been false ever since; each remaining test is folded to what it
has evaluated to for four years - focus follows pause_nonactive
and is_focused alone, the companion's render_messagebox and the
resume-time companion toggle were never reached, and the touch
build's frame limiter never skipped pacing - so behavior is
unchanged and the code says what it does.
Verified: the full linux build with the declaration gate on both
touched C files and the ELF audit steady, the mingw build and link,
and the 91-lane matrix. The Apple driver is inspection-class here.
TSan on the get-list refresh-flags suite: the HTTP callback on the
main thread writes the handle's http_data and http_task_success
and sets http_task_complete, all as plain fields, and the worker
polls http_task_complete plainly and then reads the payload - with
http_task_success written after the flag, so a worker that saw the
flag could read the result before it existed. The three completion
flags (the list handle's http_task_complete, the download handle's
http_task_complete and decompress_task_complete) are now
retro_atomic_int_t: every callback stores them with release after
the payload it is publishing, every worker poll loads with acquire
before reading it, and the list callback writes success before the
flag. The worker-private http_task_finished bools stay plain.
Verified: the suite under plain, ASan+UBSan and TSan passes clean
(master reproduces the race in two of five TSan runs), the full
build with the declaration gate and the ELF audit steady, and the
91-lane matrix.
runloop_idle_wait had grown a platform ladder - Win32 message queue,
X11 socket, Android looper - inside runloop.c, and Apple had no
entry because runloop.c cannot know which of Apple's two loop
models it is running under. The wait source is a property of the
windowing system, so it becomes a method of the display server,
which already has one driver per windowing system: idle_wait(data,
ms) blocks until the event transport is readable or ms pass,
dispatches nothing, and returns false when the server has no
waitable source, in which case runloop sleeps as before. runloop.c
keeps one call and no ifdefs.
Win32 waits on the calling thread's message queue. X11 polls the
connection fd, moved here from x11_common. Android polls the
ident-based looper. SDL2 and SDL3 use SDL_WaitEventTimeout with a
NULL event, which removes nothing. Apple: under HAVE_QT RetroArch
owns the main loop and the wait runs the default CFRunLoop mode
until a source is handled or the deadline; on every other build the
OS drives runloop_iterate from an observer or a display link and
the run loop between callbacks is the wait, so the method returns
at once - except a backgrounded iOS or tvOS app, which keeps the
sleep so a suspended loop cannot become a spin. The COCOATOUCH
branch at the call site goes with it. Wayland stays NULL, and the
driver says why: this server holds its own wl_display, input rides
the video context's, and the real wait needs that display plus the
prepare-read pairing verified against a live compositor. KMS has no
window; UWP awaits a CoreDispatcher wait in a later round.
Two latent header dependencies surfaced by the new lane are fixed:
platform_unix.h sizes struct android_app by DEFAULT_MAX_PADS without
including input_driver.h, and dispserv_android.c uses PROP_VALUE_MAX
without sys/system_properties.h - both hidden until now by griffin's
include order. dispserv_android.c now compiles in the matrix under
-DANDROID against a JNI stub carrying the interface slots RetroArch
calls; the lane was shown failing on a misspelled looper call before
this commit.
Verified: the 91-lane matrix; the linux build compiling the x11,
wayland, kms and sdl2 servers with the declaration gate on every
touched C file and the ELF audit steady; the mingw build compiling
the win32 server and linking with the PE audit steady. The Apple
driver has no toolchain here and is inspection-class, and says so.
@available has two problems for RetroArch's range of targets:
1. It needs a clang new enough to know the syntax, so the ancient
toolchains building the legacy iOS/macOS targets cannot compile
any file that uses it.
2. It lowers to a call to __isPlatformVersionAtLeast() every time
it is evaluated, including in per-frame and per-input paths.
Introduce apple_runtime.h with apple_runtime_available(mac, ios,
tvos), a drop-in replacement that resolves the running OS version
exactly once per process and caches it in a static int; after
warm-up every check is a single load and integer compare, cheaper
than @available. Platforms the original check did not list are
passed as 0, preserving @available's semantics of returning true
on unlisted platforms.
The version is read from UIDevice systemVersion on iOS/tvOS (never
shimmed, works on any version), and on macOS from the
kern.osproductversion sysctl - which reports the real OS version
even for binaries linked against pre-11.0 SDKs, where the
NSProcessInfo APIs (and @available's own plist source) are subject
to the 10.16 compatibility shim - with an
operatingSystemVersionString fallback for pre-10.13.4 systems.
This also fixes the broken iOS 9 / iOS 6 CI builds: cocoa_common.m
used UIWindowScene.effectiveGeometry, an iOS 16 API that older SDKs
(e.g. iPhoneOS 14.2) do not declare at all, so @available could not
save compilation. That call chain is now resolved entirely at
runtime via objc_msgSend (identical cost to a compiled property
access), with the capability probe cached in a static.
All 84 live @available sites across 13 files are converted.
Behavior on every OS version is unchanged: the same paths are taken
at the same version thresholds. Swift #available sites are
untouched; Swift enforces availability at compile time and those
files only build on modern toolchains.
Existing compile-time SDK gates (e.g. around NSWorkspace
isVoiceOverEnabled and the Metal HDR blocks) are kept, since they
guard symbol declarations, not runtime dispatch; comments updated
accordingly.
runloop_idle_wait had every desktop branch and no Android one, on
the platform that reaches the fallback most: unfocused but
foreground. It now polls the thread's looper for ten milliseconds.
The looper is ident-based - prepared with
ALOOPER_PREPARE_ALLOW_NON_CALLBACKS - so the poll reports readiness
and consumes nothing; the wake returns to the loop and the next
android_input_poll dispatches the ident as it always did. Input,
app commands and the sensor queue all arrive through that one
object. The IDLE case is unchanged: the input poll itself blocks
there and the call site exempts it.
The wait stays in runloop rather than folding into
android_input_poll, whose own timeout is input_block_timeout - a
running-latency setting unrelated to pause. The comment at the
branch records the reasoning.
runloop.c now compiles in the matrix under -DANDROID against
hermetic NDK stubs (jni, looper, native_activity, sensor,
configuration, window) with the host identity shed; the lane was
shown failing on a misspelled looper call before this commit.
Verified: the 90-lane matrix, the linux build with the declaration
gate and the ELF audit steady, and the mingw build and link.
cb_task_core_updater_get_list() runs at retrieval on the main thread
and reads refresh_menu through task->state to clear
MENU_ST_FLAG_ENTRIES_NONBLOCKING_REFRESH / _NEED_REFRESH - the clear
the Core Downloader entry point depends on before its pending
displaylist can populate. The handler freed the handle and nulled
task->state at FINISHED, so the callback's guard skipped the clear
every time: one visit to Core Downloader left NONBLOCKING_REFRESH set
for the session, MENU_ENTRIES_NEEDS_REFRESH() pinned false, and every
deeper menu wedged until restart, on every platform. The handle now
stays attached until task_core_updater_get_list_cleanup(), which the
queue calls at retirement after the callback - the shape task_http.c
already uses, and the point the finder-safety concern wanted all
along: the handle stays alive exactly as long as the task is
findable. free_core_updater_list_handle() also frees the headers
list that arrives with the transfer data it owns.
samples/tasks/core_updater/get_list_refresh_flags_test links the
shipping task_core_updater.c, task_http.c, task_queue.c and
net_http.c, drives the real push against a loopback .index-extended
server in both refresh_menu lanes, and asserts the flags clear and
the refresh gate returns: red on the early-free shape, green here.
Wired into Linux-samples-tasks.yml with ASan+UBSan and TSan lanes.
The libretro-common samples workflow builds every sample and refuses
one that is classified in neither RUN_TARGETS nor NORUN_TARGETS; the
retro_sleep_until test was added without that entry. It runs: never
early is the contract it checks, and its late bound is loose enough
for a loaded runner.
The salamander launchers compile rtime.c for rtime_localtime alone
and link no features_cpu.c, and every one of them failed to link a
deadline wait that had no caller there. The wait was in the wrong
file: it is a wait on cpu_features_get_time_usec()'s clock, and it
now lives in features_cpu.c next to that clock, declared in
features_cpu.h. rtime.c and retro_timers.h return to exactly what
they were before it - rtime.c neither includes retro_timers.h nor
names features_cpu, which is the dependency that should never have
been added to a file the launchers build.
The matrix lanes move with it: features_cpu.c compiles under each
console's defines and emscripten's, shedding the host's __linux__,
against hermetic stubs of every SDK header the file pulls (3ds
os/svc/cfgu, ogc lwp_watchdog, wiiu os/time, psprtc, psp2 rtc,
sys_time, lv2 systime, emscripten.h). A new lane links the
salamander object set - rtime.c with a host main that calls
rtime_localtime and nothing else - because a syntax pass cannot see
the link error an unlinked TU's symbol produces. Both lane kinds
were proven against their bugs before this commit: the link lane
run on the previous rtime.c fails with CI's exact undefined
reference, and the eight compile lanes fail with the macro missing
once retro_timers.h is taken out of features_cpu.c; with the fixes
in place all pass.
Verified: the full 89-lane matrix, the linux build with the
declaration gate on both touched files, the mingw build and link
with the PE audit steady, and the retro_sleep_until sample green.
retro_sleep_until_us's generic loop calls retro_sleep_us, which is a
per-platform macro from retro_timers.h on every console and on
emscripten. rtime.c never included that header. Windows and Darwin
compiled because they define the function in this file; 3DS,
GameCube, Wii, WiiU, PSP, Vita, PS3 and emscripten all broke, as
implicit declarations where the build allows them and link errors
where it does not. One include fixes every one of them.
The class this belongs to is the one the matrix exists to catch and
did not: a shared file whose #if ladder has a branch only the
console toolchains compile. Eight lanes now compile rtime.c under
each platform's defines against hermetic stubs of the SDK headers
retro_timers.h pulls (3ds.h, pspthreadman.h, wiiu/os/thread.h,
sys/timer.h are new; the Vita stub gains the one call it names). A
first draft of those lanes passed with the include removed: the
host compiler predefines __linux__, so every lane took the Linux
branch of the ladder and proved nothing. Console lanes now shed
__linux__ and its aliases; with the include removed all eight fail
with CI's exact error, and with it restored all eight pass. The
lanes were proven against the bug before the fix was committed.
Verified: the full 88-lane matrix, the linux build, and the
retro_sleep_until sample green.
The last runloop sleep the audit still calls a poll interval. What
POLLED_AND_SLEEP waits for is user input - the pause key, focus
coming back - and a fixed ten-millisecond sleep answered it five
milliseconds late on average while waking the process a hundred
times a second for nothing. Where the platform has a waitable
source of input, the wait moves there, with the same ten
milliseconds as the bound rather than the schedule; the state
machine above is untouched, so the worst case is exactly the old
behavior and the common case is a wake at the event.
Windows blocks on the calling thread's own message queue -
MsgWaitForMultipleObjectsEx, QS_ALLINPUT, removing nothing, so the
normal poll consumes as before. X11 blocks on readiness of the
connection socket: a plain poll() on the fd, implemented in
x11_common.c where Xlib's headers belong (runloop.c repeats the
one declaration rather than inheriting Status, None and Bool), no
Xlib call in the path, so it is safe against a threaded video
context using the same Display - and events a dispatcher already
drained were someone's to act on. Wayland's fd is only safely
watchable through the prepare-read protocol, so it keeps the
sleep rather than gaining a deadlock; so does every platform
without a source. The Android looper exemption, the Cocoa
foreground gate, and the emscripten deferred path stay exactly as
they were.
Verified: full linux build with the standing gates and the ELF
audit at its numbers, the declaration gate on both touched files
with their real compile lines, the mingw build and link with the
PE audit steady, and the full 80-lane matrix.
The audit's longest-standing non-audio entry: the limiter slept a
relative interval short of its deadline and spun the rest, so two
things it never asked for landed in its adaptive margin - the time
between reading the clock and entering the kernel, and whatever an
early or interrupted wake left over. The margin ate both, which is
why it worked, and why it was bigger than it needed to be: every
microsecond of margin is a microsecond of spin, every frame.
libretro-common grows retro_sleep_until_us(deadline), an absolute
wait on cpu_features_get_time_usec()'s own clock. On Linux and
Android that is clock_nanosleep with TIMER_ABSTIME on
CLOCK_MONOTONIC - the exact tool, and EINTR re-arms against the
unchanged deadline by definition. On Darwin it is mach_wait_until
through one paired read, absolute in Mach terms so nothing
accumulates across a wake. Everywhere else it re-arms the
platform's best relative wait against the deadline until the clock
agrees - on desktop Windows that wait is rtime.c's own per-thread
high-resolution timer, so each lap is microsecond-grained, and on
a platform whose sleep rounds up the loop ends one lap past the
deadline, no worse than the relative call was. Never early against
that clock, on any backend; a new libretro-common sample proves
exactly that - twenty laps at four durations, an early return
fails at any load - and measures 149 us worst overshoot on this
container.
The limiter now sleeps until deadline minus margin, and its margin
update measures the distance past that instant - purely the
kernel's own overshoot now, which is what the margin was for. The
spin tail stays, doing only its real job, and shrinks as the
margin converges to the smaller number. The emscripten branch, the
Cocoa background exemption, and the schedule arithmetic are
untouched.
Verified: full linux build with the standing gates and the ELF
audit at its numbers, the declaration gate on runloop.c and
rtime.c with their real compile lines, the mingw build and link
with the PE audit steady, the new sample green, and the full
80-lane matrix.
The last two names in the still-old column turn out to belong in
the other column, and each file now says so where its primitives
live. Pulse runs entirely on pa_threaded_mainloop - including the
timed wait an earlier round built from a mainloop time event - and
that rendezvous is not machinery the driver chose but libpulse's
contract: every pa_* call holds the mainloop lock, callbacks
dispatch under it, and the signal is only meaningful inside it.
Nothing of ours sits in front of pa_stream_write to park on.
PipeWire is the same contract with pw_thread_loop, and its data
path already runs on the library's own lock-free spa_ringbuffer.
Both take the ALSA column: the wait is the API's.
The web driver's sleeps are asyncify yields to the browser's event
loop - the platform's model, not a poll to convert. The one real
item left was ctr_csnd's sleep(1) poll of the hardware cursor,
which carried a todo asking for the computed period: it now waits
what the arithmetic says - samples until whichever half of the
predicate can flip, the smaller of the two, over the fixed rate,
floored at a millisecond - through retro_sleep_us, which on the
3DS is nanosecond-precise svcSleepThread. Hardware cursor, no
notifier, so the wdmks rule applies: a computed timed wait is the
honest one.
The suite checker flagged live_drivers on pulse, and honestly: it
fails identically on master in this container, which has no sound
server for it to drive - environment, not regression, verified by
stashing the change and running it again.
Verified: pulse compiles in the full build with the standing gates
and the ELF audit at its numbers, a -Wall syntax pass at zero on
pulse.c, the sinc_hq suite green through the checker's three
passes, and the full 80-lane matrix. ctr_csnd compiles nowhere in
this container and says so here rather than pretending: the edit
is one arithmetic block, C89-placed, in a file only devkitARM
builds.
ThreadSanitizer, on the pacing suite's second pass, caught a race
the render conversion created: w->fed lived under fifo_lock, the
conversion removed the lock, and the flag stayed a plain bool -
written by the writer on its first ring feed, read by the pump to
keep pre-first-write silence out of the underrun count. A
release/acquire atomic now, initialized with the others.
The checker missed it because it ran only the plain build, while
the CI runs the suite under address+undefined and then under
thread. tools/audio-suite-check.sh now runs all three passes for
any matching suite whose Makefile takes SANITIZER - the same
matrix the workflow runs, before a commit instead of after it.
Verified: the pacing suite green under ThreadSanitizer with zero
reports, green under address+undefined, the sibling multichannel
suite green, the mingw build and link clean, and the PE
thread-read audit at its numbers.
The next entry on the audit's still-old column, and the earlier
rounds had already done half of it: the buffer counter was an
atomic and OnBufferEnd decremented it lock-free. What remained was
the DSound shape - an auto-reset event, one SetEvent syscall on
XAudio2's engine thread for every buffer completed, parked writer
or none. Both callback variants, the C++ class and the CINTERFACE
table, now notify the eventcount instead: an atomic bump and a
load in the steady state where the writer keeps ahead, a gated
wake when one is parked, and never anything else on the engine
thread. Both writer waits - the full-queue throttle in write and
the wait_writable loop - become the prepare/re-check/commit window
at the same XAUDIO_TIMEOUT bound with the same short-write and lap
semantics.
The verification is real for the first time, and making it so took
two build fixes worth recording. The audit lane carried
HAVE_XAUDIO=0, so nothing had ever compiled this file there;
mingw-w64 ships a genuine xaudio2.h, so it now carries
HAVE_XAUDIO=1 for good. And Makefile.common only linked the
XAudio2 import library inside its MSYS2 detection, which a Linux
cross build never enters - a cross toolchain gets its own case
now, against the 2.8 import library mingw-w64 actually ships,
leaving the MSYS2 path exactly as it was.
Verified against real headers and a real link: the file compiles
with zero diagnostics, the declaration gate at zero on its real
compile line, the exe links with the driver in it, the PE
thread-read audit steady at forty-seven entries and zero findings,
no sample suite references the file (the checker says so, rather
than silence), and the full 80-lane matrix.
The follow-up the audit drew as a table: the WaitForSingleObjects
stay - packet IRP completion and the notification event are the
device, and nothing here points a different primitive at them -
but the three Sleeps were clocks, and Sleep is a bad clock.
Sleep(1) is not a millisecond; it is the global timer period,
~15.6 ms by default, longer than a typical 8 ms WaveRT loop - the
very hole the old yield-spin existed to avoid, reopened with a
nicer idle. The replacement already ships:
retro_sleep_us on desktop Windows is the per-thread
high-resolution waitable timer in rtime.c.
Three sites. The no-event wait with a mapped register wakes when
the cursor has moved or a millisecond has passed, whichever is
sooner, against the real clock, early exit on room kept. The
no-register wait sleeps the whole computed interval in one wait
rather than a tick-rounded one - the deadline exists so nothing
is asked on the way. The refill thread's pump cadence on an
event-less pin becomes half the loop's duration, floored at the
same 500 microseconds as the room wait's own interval, so the
register is read at least twice per wrap and the wrap accounting
holds by construction; the residual on join is at most one slice.
The stale comment above the room wait, which still described the
yield design two designs back, now describes this one.
Verified: the file compiles with zero diagnostics under
HAVE_WDMKS=1, the declaration gate at zero on its real compile
line, the exe links, the PE thread-read audit steady at
forty-seven entries and zero findings, the wdmks_abi suite via
the checker, and the full 80-lane matrix.
The audit called the last commit the cheap slice, correctly, so
here is the rest of the note. Threaded looped pins get a refill
thread and a frontend retro_spsc, and every piece lands where the
note put it: the ring is the one frontend buffer (the mapped loop
stays the device's - no third ring); the thread waits the
notification event under the stream-scaled watchdog, or a
millisecond slice against the register where the driver offers
none; the eventcount parks the frontend on ring room and never
touches the pin's waits; and the packet path, IOCTL completion
events included, is not changed by one byte.
What the thread owns, it owns whole: the register and presentation
reads, rt_write and the mapped-loop copies, the wrap accounting
and the clock fit - sampled every period now, so the wrap window
is never outrun however long the frontend is descheduled, which
was the lie the audit named in rt_played. The frontend gets the
absolute frame count and the fitted ppm back as published
atomics; frames_consumed and the ppm query on a threaded looped
pin load and touch nothing. The frontend write and wait_writable
park on ring room with the watchdog's own units as the bound. The
dead flag becomes an atomic, as it is now read across threads.
Stop and free join the thread before the pin goes down - it reads
the register and writes the driver's buffer, so its lifetime is
strictly inside the pin's - and wake it through the event so the
join does not ride out a watchdog. The no-threads build keeps the
present synchronous path unchanged.
A confession the verification owed: the previous wdmks commit
claimed the mingw build compiled and linked the driver, and it did
not - HAVE_WDMKS was never set in the audit lane, so nothing
compiled it and the claim was vacuous. The lane script now passes
HAVE_WDMKS=1 permanently; this commit is the first whose wdmks
verification is real, and the binary growing by twenty-six
kilobytes says the same thing.
Verified, all against the flag actually on: the file compiles with
zero diagnostics, the declaration gate at zero on its real compile
line, the exe links, the PE thread-read audit counts the refill
thread as its forty-seventh entry with zero findings, the
wdmks_abi suite via tools/audio-suite-check.sh, and the full
80-lane matrix.
The wasapi render conversion moved the driver onto retro_spsc and
retro_eventcount and the pacing suite's Makefile was not taught to
link them - the same omission that broke the opensl suite, so the
omission now gets a tool instead of a memory:
tools/audio-suite-check.sh takes one driver file, finds only the
suites that reference it, and builds and runs exactly those. That
is the standing pre-commit step for every audio-driver change from
here on: targeted, not a sweep.
Verified: the pacing suite builds and passes - every period
answered, pump wakes in tolerance - via the checker itself.
First item landed from the WDMKS notes, and deliberately the
narrow one: the packet OVERLAPPED waits and the registered
notification event are the kernel's period and stay exactly as
they are - the eventcount is for a ring of ours, which this driver
does not have yet - but the no-event WaveRT path burned a core.
SwitchToThread with nothing else runnable returns immediately, so
a six-millisecond deadline became thousands of no-op scheduler
calls, and a backgrounded process kept spinning at full speed
doing nothing.
Both arms of wdmks_rt_wait_room now sleep in the kernel. With a
mapped position register the deadline is walked in millisecond
slices with the register re-read after each, so room found early
is still taken early - the wait resolution moves from a yield to a
millisecond, noise against a period already floored at half of
one. Without a register the deadline is slept out whole and the
device asked once at the end, as before, since the deadline was
computed precisely so it needs no checking on the way.
The larger items in the notes - the WaveRT refill thread with a
frontend retro_spsc and an eventcount park, rt_played advanced by
a thread the scheduler cannot starve into a repeating loop -
are their own round: that is a real thread added to a
three-thousand-line kernel-streaming driver, not an evening's
swap, and it will get the harness treatment.
Verified: the mingw build compiles and links the driver clean at
the same binary size, and the PE thread-read audit holds its
numbers.
The census's answer for this driver came out split, and the split
is the same line the WDMKS notes draw: the worker's wait is the
device - sceAudioSRCOutputBlocking, one hardware period per
iteration, exactly the DMA pacing the console sinks are left with -
and it stays untouched. Everything in front of the worker was ours:
a hand-rolled power-of-two ring under one lock, and a second lock
with a condition variable that parked the writer while the worker
signalled through it every single period, parked writer or none.
The ring keeps its shape rather than becoming a retro_spsc, for a
reason worth recording: the worker consumes in place, handing
psp->buffer + read_pos straight to the output syscall, and a
retro_spsc would add a bounce copy per period on hardware with the
least memory bandwidth of anything this tree runs on. Instead the
index pair itself takes the SPSC discipline - producer publishes
write_pos with release after the samples land, consumer publishes
read_pos with release after the syscall returns, each side reads
the other's index with acquire - which is all fifo_lock ever
guarded. The condition pair becomes the eventcount: the worker's
per-period notify is an atomic bump and a load with no writer
parked, both writer waits are the prepare/re-check/commit window at
the same microsecond bound and lap counts, and stop's notify after
dropping running releases a parked writer the departed worker can
no longer wake. The running flag joins the other drivers as a
release/acquire atomic in place of volatile.
No PSP or Vita toolchain reaches this container, so the lane is the
verification: compile-only sce stubs in the SLES pattern, PSP and
Vita arms both, strict declaration flags and -Wall clean; the
matrix's standing count becomes eighty. The Orbis arm shares the
Vita code path shape and compiles nowhere here, which the stubs do
not pretend otherwise.
Verified: both new lanes green, the full 80-lane matrix, and
structural asserts on the wait windows (balanced, every prepare
answered on each path).
The mic capture path was already halfway there from the earlier
round - a retro_spsc ring with the capture callback as sole
producer, room measured against the requested size - and the lock
and condition variable that remained were pure parking, the exact
shape sdl_audio's mic already converted. The same conversion, by
that template: the callback's signal becomes a notify, an atomic
bump and a load with no reader parked and never a lock on the
capture thread; both of the reader's timed waits become the
prepare/re-check/commit window at the same ten-millisecond bound.
The old comment's accounting - a slice landing between the check
and the wait costs at most one timeout - improves to costing
nothing: the epoch moves, the commit falls through.
No macOS toolchain reaches this container, so verification is
inspection-class, as metal's was: every call shape is verbatim from
the drivers the eventcount compiles green in, the two wait windows
are structurally asserted balanced with their prepare answered on
every path, and the declarations sit at block top for C89. The full
78-lane matrix is unaffected and passes; a mock suite in the style
of the opensl one is the honest follow-up if this driver ever needs
harness coverage.
The last render-side driver from the census. The ring was already a
retro_spsc from the earlier conversion; what remained was the win32
auto-reset event the writer slept on, and its cost sat on the wrong
side: SetEvent is a syscall on every block the pump moves, waiter or
none, which is the steady state where the writer keeps ahead and
never waits. The eventcount's notify is gated - an atomic bump and a
load with nobody parked - so the pump's per-block cost disappears
exactly where it was pure waste, and the ntdll waiter-list backend
carries the parked case without a mutex on any tier.
Both waits become the documented prepare/re-check/commit window over
the same predicate (ring space and thread_alive) with the same
DSOUND_TIMEOUT bound and the same stalled behavior, and the pump's
final notify - the one that clears thread_alive when the buffer is
lost for good - can no longer fall between the writer's test and its
park.
Verified: the mingw build compiles and links the file clean and the
PE thread-read audit holds its numbers across the dsound pump and
writer.
Two breaks from the same class of mistake: verification that
depended on the machine it ran on. The openal matrix lane compiled
against system AL headers, which the container that wrote it had
and the CI runner does not; and the opensl teardown hardening
started calling GetPlayState through a vtable the opensl sample
suite's mock had never needed to define.
The openal lane now compiles against compile-only AL stubs
(tools/platform_stubs/openal, first in the include order), the same
pattern as the SLES and gekko stubs: the lane needs no
libopenal-dev anywhere and behaves the same on every runner. The
driver declares its AL_SOFT_events surface itself and probes the
multichannel formats by name at runtime, so al.h and alc.h cover
the whole compile surface.
The opensl suite's mock SLES header gains GetPlayState in the
SLPlayItf vtable, in the driver macro's slot order, and the mock
implements it against its own play state under the queue lock - so
the teardown's STOPPED confirmation exercises a real transition
rather than a canned answer. The suite links
retro_eventcount.c now that the driver parks on it.
Verified: the opensl suite builds and runs green, the openal lane
compiles through the stubs with strict flags and -Wall clean, and
the full 78-lane matrix passes.
The census's most urgent entry, for a reason beyond style: JACK's
process callback runs on the server's realtime thread, where taking
a lock risks priority inversion against the very writer it is
waking, and the old scond_signal takes one on the portable backend.
The data path was already clean - JACK's own lock-free ringbuffer,
the shutdown flag and underrun counter already atomics from earlier
rounds - so the lock and condition variable existed purely to park
the writer, the same shape as every conversion before it.
Both callbacks now notify the eventcount: the process callback's
per-period wake is an atomic bump and a gated wake with a writer
parked, nothing at all with none, and never a lock on the realtime
thread on the lock-free backend. The shutdown callback's notify -
the one that is never repeated, whose loss the old code managed
with timed waits and a long comment - lands reliably now: shutdown
and write space are re-checked inside the eventcount's
prepare/commit window, so a notify between the test and the park
makes the commit fall through instead of being lost. The timed
waits stay, repurposed honestly as the lap clock for the
graph-stalled bounds, at the same period and the same lap counts.
Verified: full build clean (jack is in the audit configure), the
build-flags declaration gate on jack.c, a -Wall pass against real
jack headers with zero diagnostics, the standing thread-read gate
at its numbers, and the full 78-lane matrix.
The audio-driver census after openal: sdl_audio and rsound were
already converted in the earlier rounds - rsound is the template,
a lock-free retro_spsc ring with an eventcount park - and wasapi
was the largest remaining holder, thirty-four lock sites around one
fifo, one condition variable, and the counters that lived under
them. The render path now follows the template exactly.
The fifo becomes a retro_spsc: the writer produces, the pump
consumes, and the roles never swap - the no-threads build is
single-threaded, where both roles are one thread and the ring is
trivially safe. retro_spsc rounds capacity to a power of two, so
the requested size is kept and every room figure is capped to it
through one helper, exactly as rsound does, keeping the rate
control's latency arithmetic intact; the buffer-size report loses
the fifo's one-reserved-byte while it is at it. room_cond becomes
the eventcount: the pump notifies after freeing a period, the
writer and the wait-for-room path park with the documented
prepare/re-check/commit loop, same one-millisecond step, same lap
bounds. consumed and released are single-writer counters and become
load/store atomics; ac3_inflight has two writers and becomes a real
RMW atomic, its underflow clamp kept as a CAS loop even though the
ring's release/acquire pairing means it can no longer trigger. The
lock, the condition variable, and every fifo call on the render
path are gone; the mic capture path keeps its own fifo untouched,
its examination being its own round.
Putting the file under the real-flags declaration gate surfaced one
pre-existing C89 violation - wasapi_sh_engine_period reset above
the declarations in wasapi_init_client_sh, which no lane compiled
with the error enabled - moved below them with its semantics
restored intact.
Verified: the mingw build compiles and links the file clean, the
declaration gate at zero on wasapi.c with its real compile line,
the PE audit at zero findings across the wasapi pump and device
threads, and the full 78-lane matrix.
Same question as opensl, same answer, and a cleaner one: the
driver's event path already spelled out the eventcount's exact
protocol by hand. The mixer-thread callback bumped a generation
counter under a mutex and signalled; the waiter read the
generation, made its AL calls with the lock released, and slept
only if the generation was unchanged - which is prepare_wait,
work, commit_wait with a mutex where the epoch should be. The
completed counter was never consumed as a count (al_unqueue_buffers
asks the device directly); it existed purely as that generation, so
it does not survive: the eventcount's own epoch is the generation,
and the prepare/commit window skips the wait an event raced past,
exactly as the old comment promised.
The callback becomes one release-store (the disconnect flag, now a
plain atomic read on the waiter's side, no lock) and one notify -
no AL calls, as AL_SOFT_events requires, and no syscall while
nobody is parked. The wait keeps its step timeout and its total
budget with the same accounting: only timed-out steps are charged,
so there is still no wall clock in the path. The no-events
sleep-poll fallback stays as it was - a library without the
extension offers nothing to park on. Lock, condition variable and
both plain shared fields all go; on lock-free backends the driver
now owns no mutex, and elsewhere the eventcount's scond backend
centralizes the one that remains instead of hand-rolling it here.
OpenAL is not in the audit build's configure, so the file gets a
matrix lane - real system AL headers, no stubs - with
declaration-after-statement strictness; the matrix's standing count
becomes seventy-eight.
Verified: the new lane green, a -Wall syntax pass against real AL
headers with zero diagnostics, the full 78-lane matrix, full build
clean, and the standing thread-read gate at its numbers.
Three questions answered at once. The driver is Android-only in
practice - it builds under HAVE_SL, registered from the Android
build alone, and the Android buffer-queue extension is load-bearing
on every path - so raw futex was not wrong there. But it was a
local reinvention: libretro-common carries retro_eventcount, from
the earlier queue work, and its lock-free backend is the same
futex(FUTEX_WAIT_PRIVATE) with no lock at all, next to a Windows
waiter-list backend and an scond fallback for everything else. So
the futex approach was available through a construct, the construct
unties the file from Linux should it ever compile elsewhere, and
going through it does not break lock-free - that backend is the
point of it.
It is also more correct than the hand-rolled version: the wake
gating paired a relaxed waiter bump with an acquire load, which
leaves the classic window where a notify reads zero waiters while a
registering waiter still holds the pre-notify value; the
eventcount's seq_cst pairing closes exactly that, per the comment
in its own notify. Both waits become the construct's documented
prepare/re-check/commit loop with the same stall timeout and the
same stalled behavior; the callback's notify is one atomic bump and
one load with no writer parked; the raw futex helpers, the waiters
counter and the linux includes all go. Semaphores were the wrong
turn for the reasons that prompted the question: bionic's
sem_timedwait wants an absolute realtime deadline, and the portable
one is a mutex and condvar inside.
Verified: the android opensl lane green with strict flags, a -Wall
syntax pass at zero warnings, eventcount already linked wherever
this driver builds (griffin, under HAVE_THREADS), and the full
77-lane matrix.
Answering whether the sleep and wall-clock reads the lock-free
conversion introduced can go too, without falling back to
semaphores: bionic's sem_timedwait wants an absolute CLOCK_REALTIME
deadline - wall clock again, and jump-sensitive - and the portable
semaphore is a mutex and condition variable inside, which brings
back the very object the conversion removed.
This driver is Android-only and Android is Linux, so the precise
tool applies: the writer parks in the kernel on buffered_blocks'
own address. FUTEX_WAIT takes a relative timeout, so the stall
bound needs no deadline arithmetic and the hot path reads no clock;
its compare-and-block removes even the lost-wakeup window the old
condition variable had to tolerate, since a callback landing
between the caller's test and the kernel's compare makes the wait
return immediately. Both waits - sl_write's throttle and
sl_wait_writable - convert, with the same timeout and the same
stalled behavior. The callback stays syscall-free in the common
case: it wakes the futex only when the new waiters counter says a
writer is actually parked. EINTR re-arms the stall window, which
detects dead devices and owes no precision.
The one poll left is teardown's STOPPED confirmation: a foreign
API's state transition has nothing to park on, so that single cold
loop is now iteration-bounded rather than clock-bounded, once per
session, normally satisfied on the first read.
Also corrects the stub SLES header's outputMix locator to the real
SLObjectItf type, so the file passes -Wall clean against the stubs.
Verified: the android opensl matrix lane green with
declaration-after-statement strictness, a -Wall -fsyntax-only pass
with zero warnings, and the full 77-lane matrix.
Investigating #19561 - libwilhelm's AudioTrack thread crashing in
pthread_mutex_lock on 0x55-poisoned memory, every stack frame
inside libwilhelm itself - and answering whether the driver's locks
can be atomics instead. They can, entirely: the shared state
already was. Both counters the callback and writer exchange are
atomics from earlier work; the mutex survived only as the condition
variable's mandatory companion, guarding nothing but the wait.
Both waits - sl_write's throttle and sl_wait_writable - become
bounded millisecond polls on the atomic block counter, keeping the
exact stall semantics the condition path had (same timeout, same
partial-write bail; the bound exists because a dead device sends no
callback and an unbounded wait parked the core's thread for good).
A millisecond of wake grain is noise against a block period of tens
of milliseconds. The callback loses its one remaining syscall and
runs pure atomics on libwilhelm's thread; the lock, the condition
variable, their creation and their frees all go. The driver now
owns no mutex any thread could touch after free, which settles the
issue's accusation structurally: the poisoned mutex is libwilhelm's
own object lock, freed by Destroy while its AudioTrack thread was
still in flight.
For that actual race, teardown is hardened to the spec-clean
recipe: stop the player, Clear the buffer queue, and confirm the
STOPPED transition with a bounded GetPlayState poll before any
Destroy runs, so the AudioTrack thread has nothing left to execute
when the objects go. The spec makes Destroy synchronize with
callbacks; Android's implementation has raced that when torn down
from PLAYING with buffers queued, which is exactly the reported
signature.
The driver compiles nowhere in the existing lanes, so a stub SLES
header set lands in tools/platform_stubs/android and a matrix lane
compiles the driver with declaration-after-statement strictness;
the matrix's standing count becomes seventy-seven.
Verified: the new lane green (strict flags included), the full
77-lane matrix, and the standing thread-read gate at its numbers.
The rgui driver carries console-specific shapes - GEKKO and DINGUX
framebuffer dimensions, aspect handling, and the gx pixel
converters - that no lane compiled, so a refactor could break them
silently. Two syntax lanes cover both configurations; GEKKO needs
a single tiny libogc stub header, now in tools/platform_stubs/gekko
so the lane is reproducible from the repo. The matrix's standing
count becomes seventy-six lanes.
Verified: the full matrix green at 76, both new lanes passing.
Two kinds of drift a direct mingw build tripped over, found while
building the audit lane. DEFINES has carried -DHAVE_OVERLAY since
before the object list moved to Makefile.common, which keys off the
make variables: the define without the variable compiled overlay
callers against overlay objects that were never built, and
screenshots without HAVE_RPNG could not encode - both surfacing as
undefined references at link. The variables are now set where the
other HAVE_ ones are, so definitions and objects agree again.
And win32_common.c pulls imm32 with #pragma comment(lib), which
MSVC honours and GNU ld silently ignores, leaving the IME
composition calls undefined; the library is now on LIBS explicitly.
The audit lane script drops both of its compensations - the
variable overrides and the manual relink that appended imm32 - and
its plain make now compiles and links the audit subject in one
step, which is the fix demonstrating itself.
Verified: a clean cross build linking through the makefile's own
rule, and the PE audit at its standing zero findings.
Alternative shape for the fix in #19565, keeping the diagnosis and
moving the guard where the knowledge lives. The bug is real and the
PR's analysis of it is right: loading content over running content
deinits the core, which tears the drivers down, and unloading the
old content's override then fires CMD_EVENT_REINIT (the override
changed video_fullscreen, CORE_RUNNING still set). The instance
created by that reinit is replaced by retroarch_main_init's
drivers_init without being freed, and the orphan's window and
device stay alive under the widget fonts until the new content
crashes.
The PR guards the call site in config_unload_override with
video_driver_get_ptr() and replicates the grab release with
input_state_get_ptr(). Guarded instead at the top of
command_event_reinit: configuration.c keeps its bare
command_event(), no getter reaches into video or input state from
the config layer, every present and future caller that fires a
reinit while video is down is covered rather than one, and the
check spends only the function's own locals - command_event_reinit
already holds settings, video_st and input_st for the work it does.
The grab handling carries over on the PR's reasoning: nothing can
be ungrabbed with the drivers down, but the flag is bookkeeping
that the win32 focus pump and the grab toggle read later, so the
early-out leaves it as the skipped reinit's game-focus reapply
would have - released, unless exclusive fullscreen, auto-grab or
game focus keeps it.
The build-flags C89 gate caught the guard's first placement sitting
above the function's static snapshot declarations; it now sits
below the declaration block.
Verified: that gate green on command.c, full build clean, the
standing thread-read gate at its numbers, playlist_nav green, and
the 74-lane compile matrix.
The six findings the mingw audit lane surfaced on its first run,
plus two the flags could not see, all the same classes the campaign
closed in GL and Vulkan.
Rotation: the frames' uniform values and both init_render_targets
(reached from the frames on resize) called retroarch_get_rotation()
live; all five sites now read video_driver_get_rotation_snapshot(),
the seqlock accessor the glsl/cg round added.
HDR uniforms: both frames re-read paper-white, scanlines, subpixel
layout and expand-gamut from live settings every frame - redundantly,
since the HDR pokes already store all four into hdr.ubo_values as
blocking commands and init seeds them. The pass loops read the
latches; the per-frame ubo scanlines re-read is dropped outright,
and d3d12's frame-wide settings hoist goes with it, having no
remaining users.
Swapchain bit depth: the resize handling read the setting live, and
its change command is APPLY_STATE_CHANGES, not a reinit, so an
init-only latch would go stale. swapchain_bit_depth_latched is
seeded at init and refreshed inside apply_state_changes - a
blocking command, so the settings read there cannot race main - and
the resize path consumes it.
dxgi_check_display_hdr_support was the worst of the six: on an
unsupported display it wrote settings from the frame path -
video_hdr_mode to zero plus SETTINGS_FLG_MODIFIED, a video-thread
mutation of main-owned state. The check now release-stores an
atomic request; dxgi_hdr_process_deferred_disable() applies the
write on the main thread from video_driver_frame, declared locally
there to keep the windows headers out of the shared file. The
disp-flags clear stays in the check: those flags are atomic.
Verified: the PE audit at zero findings through the mingw lane, the
Linux gate untouched at its numbers, both builds clean,
declaration-after-statement checked with the real mingw compile
lines for all three Windows files, the build-flags gate on
video_driver.c, hot-function budgets, and the 74-lane matrix.
The audit's coverage note has listed the Windows-only entries as
unauditable since the tool landed: ten thread entrypoints - wasapi's
two, dsound, the dinput and xinput enumeration workers, winraw's
name handler, the win32 UI browser thread, the modeline resync
thread - plus the d3d11 and d3d12 frame contexts, all compiled out
of the Linux binary the gate runs on.
tools/mingw-audit-build.sh cross-builds an audit subject with
mingw-w64: not a shippable configuration - freetype, Cg and XAudio
are off for missing import libraries, tools/mingw_audit_stubs/
papers over headers older mingw-w64 lacks (d3dkmthk.h, the
unversioned d3d12 serialize PFN, three interface names mapped to
IUnknown; layout is irrelevant when only call edges are read), and
imm32 is appended by relinking from the `make -n` line because
Makefile.win assigns LIBS with := - but a complete one for walking.
The audit itself gains --objdump so the PE binary can be read with
x86_64-w64-mingw32-objdump.
First run over the PE: the ten Windows thread entries come back
clean, and the two frame contexts surface six findings - the exact
classes the campaign closed in GL and Vulkan, alive in the D3D
drivers: both frames read config_get_ptr directly, both reach
dxgi_check_display_hdr_support's per-frame settings read, and both
still call retroarch_get_rotation, which the rotation round only
converted for gl3 and vulkan. Those six are the fix queue the lane
was built to find; they are the next change, not this one.
Verified: selftest, the standing Linux gate untouched at its
numbers, and the PE audit reproducible through the script.
The last unexamined note on the threading ledger: shader_load_begin
is called on the driver straight from the main thread, with no
wrapper command in sight. Chased, the mechanism turns out to be
doubly guarded by design. The parse-side path gates on
!video_st->threaded, with the exact race named in its comment - the
tick runs on the main thread while gl3_frame() runs on the video
thread, and they would race on the filter_chain pointer - so the
direct driver call only ever happens when the main thread is the
render thread. And the wrapper's vtable slot is NULL, which fails
the capability test independently even if that gate were ever
bypassed; threaded video takes the synchronous fallback, whose
driver calls are blocking wrapper commands. The tick and the
teardown cancellation re-check capability and state on the same
main-side terms and cannot leave IDLE under the wrapper.
The NULL slot now carries that contract in a comment, so the next
reader chasing the same question finds the answer at the wrapper
instead of re-deriving it. This also settles the one deferred
verdict that leaned on it: vulkan_shader_load_begin's live settings
read runs only on an unthreaded main thread, which is the render
context.
Verified: full build clean, build-flags C89 gate on the wrapper,
and the standing thread-read gate at its numbers.
The one remaining live read in the shader backends, on the ledger
since the rotation round: the glsl and Cg uniform paths called
retroarch_get_rotation() from the video thread for the Rotation and
OriginalAspectRotated uniforms - five sites - reading config and
system state the main thread owns, with no ordering. Invisible to
the thread-read audit because gl2 reaches these backends through a
function pointer the walker cannot follow, which is why it stayed
open on the ledger rather than in the findings.
The planned fix was a shader_backend vtable member with six
initializers to touch. The viewport-snapshot work made that
obsolete: video_driver_get_core_aspect(), called two lines away in
both files, already reads the seqlock'd viewport-parameter
snapshot, and the snapshot already carries the composed rotation.
video_driver_get_rotation_snapshot() exposes it - same value, same
freshness class as everything else in the snapshot, exact across
set_rotation's ordered publish - and the five sites call it. No
vtable change, no driver churn.
Verified: full build clean, build-flags C89 gate on video_driver.c
and shader_glsl.c, shader_gl_cg.c syntax-checked against stub Cg
headers (the Cg toolchain does not exist in the audit container;
the swaps are same-signature), the standing thread-read gate at its
numbers, playlist_nav green, hot-function budgets, and the 74-lane
compile matrix.
The rewind/runahead ownership sweep, the auditor's last unswept
territory. Its verdict: both are main-confined by construction -
the rewind ring is touched only from check_rewind on the main
thread (no task ever reaches state_manager state), and runahead's
only threading primitives are acquire-side consumers of the
already-atomic video flags - with exactly one real crossing.
state_manager_frame_is_reversed() read the FRAME_IS_REVERSED bit
straight off the rewind flags word. Its writer is check_rewind on
the main thread, per frame while rewinding; its readers include
every FrameDirection uniform - gl3, d3d12, gx2, metal, glsl, cg,
all on the video thread per frame - and task_save on the worker.
A plain uint8_t bit with no ordering in either direction, invisible
to the thread-read audit because the accessor reads the runloop
global directly rather than through a tracked getter.
The flags word stays main-only rewind logic; a retro_atomic mirror
alongside it is release-stored at all five transitions - the two
set sites (rewind pop and the bsv-movie hold), the per-frame clear,
and both deinit clears - and the accessor acquire-loads the mirror.
No caller changes anywhere: every reader already goes through the
accessor.
Verified: build-flags C89 gate on state_manager.c and runloop.c,
full build clean, the standing thread-read gate at its numbers,
playlist_nav green, hot-function budgets, and the 74-lane compile
matrix including the no-threads lanes the new retro_atomic include
must survive.
The last fifteen findings were the widgets, menu, font and
display-text stack - five callees across the three frame roots -
and every one verified safe under two arguments now recorded in the
allowlist. The disp_get_ptr calls in font_driver_render_msg,
menu_driver_frame and gfx_widgets_frame touch only gfx_display's
quad-batch state, which is filled, flushed and latched exclusively
on the rendering thread; the main-written scalars on the same
struct are not read by these callees, and the menu drivers take
their dimensions from their own snapshots per the menu sweep. The
video_state reads in gfx_widgets_frame and gfx_display_draw_text
are current_video, poke and data - vtable dispatch through pointers
that change only at video init/free, which a frame cannot execute
across.
With that, the extension's opening backlog of twenty-nine findings
stands at zero - latches and snapshots for the settings, rotation,
input-sensor, spirv-toggle and HDR-recreate classes, allowlist
entries with their reasons for the rest - so the frame-context
entries become part of the standing gate rather than an opt-in.
--frame-context remains accepted as a no-op. The gate's standing
numbers are now fifty-seven entries, zero findings, eight
allowlisted pairs, one boundary, and any future read that leaks
into a driver's frame closure fails the default audit run.
Verified: selftest, the audit identical with and without the
compatibility flag, hot-function budgets, and the 74-lane compile
matrix.
Three more classes off the frame-context backlog, nineteen findings
down to fifteen.
gl3_spirv_binary_supported() read the direct-SPIR-V user toggle
from live settings on every call - deliberately, its comment said,
so toggling takes effect on the next preset load. The lazy pass
builds call it from the video thread with the main thread free, so
the toggle is now a latch refreshed at init and at set_shader: the
same blocking windows "next preset load" already meant, identical
semantics, no live read.
The Vulkan filter-chain builders - default and preset - run on the
video thread when a swapchain recreate rebuilds the chain
mid-session, and read five HDR settings live: mode, paper-white,
expand-gamut, scanlines, subpixel layout. Every one already has a
blocking write path: hdr_mode cannot change without CMD_EVENT_REINIT
(latched at init), paper-white and expand-gamut land in the
driver's ubo_values through their pokes, and the scanlines and
subpixel pokes - which previously pushed only into live chains, so
a rebuilt chain re-read settings and kept the values current only
by that accident - now latch driver-side too. Both builders consume
latches exclusively. The deferred shader_load_begin keeps its live
read: it runs on the main thread.
video_thread_swap_count() is correct as written - the video
thread's writes to the counter happen under thr->lock in the
wrapper's frame handler, and the accessor either runs on the video
thread or takes the same lock - so it is allowlisted with that
reason. Doing so exposed that the allowlist only matched at
(entry, reader) grain, which would suppress a reader across an
entire frame root; the audit now honors (callee, reader) pairs -
the surgical form the list's entries were written as - alongside
the root-level form, with the selftest passing.
Remaining fifteen: the widgets, menu, font and display-text stack,
whose verification is the next pass.
Verified: both audit modes and the selftest as stated, the
build-flags C89 gate on the touched sources, full build clean,
hot-function budgets, and the 74-lane compile matrix.
The viewport-snapshot commit placed video_driver_publish_vp_params()
in video_driver_init_internal() above a declaration that only exists
under HAVE_VIDEO_FILTER - a statement before a declaration, exactly
what the C89 coding guideline forbids and the release build's
-Werror=declaration-after-statement rejects. It shipped anyway
because the per-file C89 gate that was supposed to catch it compiles
with a hand-picked define set that does not include
HAVE_VIDEO_FILTER: the preprocessor deleted the very declaration
that breaks the real build, and the gate passed a file the compiler
never sees. The seed now sits below the full declaration block,
gates included.
That gate design is the actual defect, and this is its third escape
(the enum trailing commas and the frame-function placement were the
same class), so it is retired: tools/c89-decl-gate.sh touches the
given sources, takes each one's compile line from `make -n` - every
HAVE_ define, include path and gate exactly as the release compiler
sees them - and re-runs it with -fsyntax-only and the
declaration-after-statement error added. Run against every file
this campaign has touched: all pass, including this fix, so no
further members of this class are latent in the merged work.
Verified: the new gate green across the campaign's files, full
build clean, thread-read audit at its standing numbers, and the
74-lane compile matrix.
The frame-context audit's input class, four of its findings: the
slang pass builders in the gl3 and vulkan chains (and the glsl
uniform path the audit's roots do not reach on this config) read
the sensor caches - three vec3s - straight off input state from the
video thread, and set the shader-demand latch by writing a plain
bool into input state from the same context. The caches were
already a deliberate poll-side design ("a consistent per-frame
snapshot", its comment says), but the publication was plain float
stores: a reader could assemble a vector mixing two polls, and
nothing established happens-before in either direction.
input_driver_poll() now publishes the nine floats behind a seqlock
after computing the caches, with the same discipline as the video
viewport snapshots; input_driver_read_sensor_snapshot() is the read
side, used by all three shader backends, coherent as a set. The
demand latch becomes an atomic: the backends release-store it
through input_driver_set_shader_uses_sensors(), the poll's demand
check acquire-loads it. Both helpers reach input state through the
getter by construction, so their pairs are allowlisted with their
reasons - the audit's designed flow for verified-safe reads - and
the standing gate's numbers become fifty-one entries, zero
findings, two allowlisted pairs, one boundary.
The flagged audit drops from twenty-three findings to nineteen with
zero input_state_get_ptr rows remaining.
Verified: both audit modes as stated, the threaded-video suite and
playlist_nav green, C89 gate clean on input_driver.c (shader_glsl.c
stays at its pre-existing baseline, verified against a stash),
full build clean, hot-function budgets, and the 74-lane compile
matrix.
The frame-context audit's two biggest classes, six of its
twenty-nine findings: video_driver_update_viewport() and
video_driver_get_core_aspect() read config_get_ptr() and
retroarch_get_rotation() from the drivers' frame closures - resize
handling on the video thread, main running free - and
video_viewport_get_scaled_integer() and _get_scaled_aspect2()
compounded it by reading seven more settings fields plus the custom
viewport rectangle through a passed settings pointer, invisible to
the getter-based audit but the same live reads.
Everything those functions need now travels in one seqlock-published
snapshot on video_driver_state, fifteen values wide: the
scale-integer flag, both rotations, the aspect ratio, aspect index,
integer-scaling mode and axis, both bias pairs, and the custom
rectangle. Published on the main thread at four points, each
ordered before its consumers: video init (drivers call
update_viewport from inside init, and the reader must never spin on
an unpublished seq), set_aspect_ratio and set_rotation (the driver
pokes they precede are blocking commands, so the video thread
observes the new snapshot before it can act on the change), and
once per video_driver_frame as the catch-all that lands plain
settings toggles within a frame. Readers follow the overlay
viewport's seq discipline a few lines above - the same retry loop,
converging immediately on the main thread. The portrait bias fields
exist only in mobile builds; elsewhere the slots mirror the
landscape values so they are never unpublished. The integer-scaling
helper takes the snapshot instead of a settings pointer;
get_scaled_aspect2 reads it internally, so its console caller keeps
its signature and gets the same frame-fresh values.
The flagged audit drops from twenty-nine findings to twenty-three:
the rotation and update_viewport classes are gone, on top of the
driver-side rotation latches from the previous change. The flagless
gate stays at its standing fifty-one entries, zero findings.
Verified: both audit modes as stated, playlist_nav green, the
overlay seqlock harness green (the neighbouring discipline this
copies), C89 gate on video_driver.c, full build clean, hot-function
budgets, stack-frame sweep, and the 74-lane compile matrix -
including the no-threads and feature-gate lanes that caught the
snapshot fields initially sitting inside HAVE_OVERLAY.
Two results from triaging the frame-context audit's rotation class.
First, a live bug the triage walked into: the SET_ROTATION
environment handler passed the core's raw rotation to
video_driver_set_rotation, while the menu path and
retroarch_get_rotation() itself compose it with the user's
configured rotation. A core rotating with video_rotation set
silently dropped the config offset until the user next touched the
rotation setting, and the driver's stored rotation disagreed with
retroarch_get_rotation until then. The env path now passes the same
(core + config) % 4 sum the menu path always has - which is also
what makes the drivers' stored rotation trustworthy as a latch.
Second, the frame-side hygiene that latch enables: gl3 and vulkan
keep the raw 0-3 value alongside their transformed degrees, written
only inside blocking wrapper commands (set_rotation, and a seed in
init), and their frame paths read the latch instead of calling
retroarch_get_rotation() - a config_get_ptr read - from the video
thread. The filter chains' pass-sizing code reads the rotation the
chain already stores per pass (pushed to it each frame) instead of
reaching for the global.
The audit's three rotation findings do not clear yet, and the
commit does not claim they do: the surviving edges run through the
shared helper tier in video_driver.c - get_core_aspect,
set_aspect_ratio, update_viewport, init_internal and
video_driver_frame itself all call retroarch_get_rotation, and the
frame closures reach them. That tier is where the update_viewport
settings-read finding lives too; converting it (latch or frame-info
snapshot) is the next triage target, with the five sites now
enumerated. glsl and cg still read the global for their rotation
uniforms - moving them onto the latch needs an optional
set_rotation member on the shader backend vtable, deferred with
this note rather than half-done.
Verified: full build clean, flagless audit at its standing 51/0,
C89 gate on runloop.c, hot-function budgets, and the 74-lane
compile matrix.
Closes the gap recorded in the gl2/gl3 frame-path commit: the
binary walk follows no indirect edges, so the wrapper's
driver->frame dispatch made every driver's frame code invisible to
the audit, and its zero never covered the one context where a read
races - the video thread running free while the main thread
mutates. The feared modeling problem dissolves on the same fact:
because no indirect edges are followed, listing only the
free-running functions (frame, set_texture_frame) as direct entries
yields exactly the frame closures without the blocking command
handlers, whose init-time reads are legal by the send-and-wait
protocol and must not become findings.
First run over gl2, gl3 and vulkan surfaces twenty-nine findings in
ten callee groups, recorded next to the entry list: per-frame
settings reads (retroarch_get_rotation, video_driver_update_
viewport, gl3_spirv_binary_supported), live input state read by the
shader pass builders for input-driven uniforms, the widgets, menu,
font and display-text stack reading dispwidget and video state on
the video thread, video_thread_swap_count, and a settings read
inside the swapchain-recreate path reached mid-frame. Some of these
will be latches or snapshot plumbing, some are protected by
existing locks and will be allowlisted with their reason; none of
that triage belongs in this change.
The extension is opt-in via --frame-context until the backlog is
triaged to zero: the flagless run - fifty-one entries, zero
findings - remains the standing green gate, unchanged, and the
selftest passes in both modes.
Verified: flagless audit at its standing numbers, flagged audit
reporting the documented twenty-nine, tool selftest, and the
74-lane compile matrix untouched by a tools-only change.
The Vulkan/D3D submission-sync sweep of the discovery pass: all
twenty-six vkQueueSubmit/vkQueuePresentKHR/vkQueueWaitIdle sites
across vulkan.c, vulkan_common.c and the slang filter chain were
put on a site-by-lock matrix. Twenty-five hold the discipline -
context queue_lock in the driver and swapchain code, the negotiated
lock_queue/unlock_queue pair in the filter chain, with the
drop-before-fence-wait and hold-across-device-idle choices already
reasoned in place. The one gap was vulkan_font_free(): a bare
vkQueueWaitIdle on the shared queue.
Font frees are dispatched to the video thread, which serialises
them against command-buffer recording - but the queue is also
shared with HW-render cores submitting from their own threads
through the negotiated lock, and no dispatch parks those. A
vkQueueWaitIdle concurrent with a core's vkQueueSubmit violates the
queue's external-synchronisation requirement outright. The wait now
sits under queue_lock with the same HAVE_THREADS guards
vulkan_free() uses.
The race needs a Vulkan device and an HW-render core mid-submit,
which the container cannot run; the site-by-lock matrix (search
census over every queue-access spelling, each classified against
its lock evidence, comment and symbol-table false positives
discarded by hand) is the verification, stated as such.
Verified: full build clean including the vulkan objects,
thread-read audit 0 findings, hot-function budgets, and the 74-lane
compile matrix including its no-threads lanes against the new
guards.
The threaded-video suite's AddressSanitizer lane caught the flush
predicate overreading: rarch_log_tag_wants_flush() compared the tag
with memcmp and the pattern's sizeof, so matching "[WARN]" - seven
bytes with its terminator - against "[ERROR]"'s eight-byte pattern
read one byte past the tag's global. The very first line of any
boot (config load's RARCH_WARN) trips it under ASan; on ordinary
builds it is a silent one-byte overread whose result is correct by
accident, since the compare has already differed by then. The
Android priority mapping carried the identical sized-memcmp pattern
against core-supplied tags of any length, and was the precedent the
predicate copied - both now use strcmp, which stops at the first
difference or terminator and cannot overread either operand.
Verified: the threaded-video harness runs green through the exact
boot path the CI trace aborted in, the log-atomicity harness
(including the flush-durability lane) green three consecutive runs,
the logger-macros suite green in all four configurations, Android
stub syntax pass, C89 gate, full build clean, thread-read audit 0
findings, and the 74-lane compile matrix.
Actions the TODO above the synchronous fallback asked for: applying
a preset paid three walks of the preset-reference chain - the
driver's own parse-and-compile, then, to mirror the result into the
menu, video_shader_load_preset_into_shader's root-chain crawl plus
its re-read of the original file for reference validation. The
deferred load's completion paid the same two menu-side walks. On
slow storage with chained presets, every apply read every file in
the chain three times.
The struct audit says a copy can replace the re-parse:
struct video_shader is self-contained inline arrays with exactly one
owned-pointer exception, pass[i].source.string.{vertex,fragment},
which driver compilation (glsl, d3d) fills and owns and menu code
never reads. video_shader_copy_for_menu() is a struct copy that
clears those two pointers per pass and runs the same file-watch
registration the parse path runs. menu_shader_manager_set_preset_
from_live() wraps it in set_preset's success-path bookkeeping, and
both call sites - the synchronous fallback and the deferred
completion - use it when the driver exposes get_current_shader,
parsing from disk as before when it does not. Reading the live
struct there is main-thread reader against video-thread reader:
after set_shader's blocking window the struct's only mutating field
has one sanctioned writer, and it is this same main thread.
The copy also retires a latent index hazard the parse path merely
got away with: the slang backends rebuild the parameter list by
reflection, and the parameter menu indexes the driver's live struct
with positions taken from the menu's own copy. A copy of the
driver's struct agrees with it by construction; a fresh parse only
happens to.
samples/gfx/shader_copy links the shipping objects and drives what
the copy replaces: a child preset referencing a parent whose pass
source carries a #pragma parameter, parsed through the real
reference-chain walk with the referenced value applied; heap source
strings planted the way compilation does; then the copy must be
byte-identical modulo the two cleared pointers, leave the driver's
strings unstolen (the ASan-clean single free is the no-aliasing
proof), and be independent under mutation.
Verified: the parity harness green under ASan, playlist_nav green,
full build clean, C89 gates on the three touched translation units,
thread-read audit 0 findings, hot-function budgets, and the 74-lane
compile matrix.
The shader-preset lifetime check of the discovery pass: menu code
obtains the driver's live video_shader through the get_current_shader
poke - which the threaded wrapper forwards as a direct call, handing
the main thread a raw pointer into the video thread's world - and
four menu actions wrote parameters[i].current through that pointer:
the parameter left/right steps, the keyboard-entry OK path, and the
start-button reset. The video thread's frame path reads the same
field every frame for uniform upload. A single-writer aligned-float
race - it never tears on any supported target - but main-thread
stores into video-thread-owned memory outside any blocking window is
exactly the crossing the campaign eliminates.
The struct's other fields (name, range, step, initial, count) mutate
only inside set_shader's own blocking command, so reading them from
the main thread stays sound; parameters[i].current was the one live
crossing. It now has one sanctioned writer:
video_shader_driver_set_parameter() stores on the owning thread as a
blocking round trip under the wrapper and directly otherwise. The
wrapper's generic run-on-video-thread machinery behind
video_thread_texture_handle() is exported under its true name,
video_thread_run_blocking(), and texture_handle delegates to it. The
four menu actions step or clamp on a local copy of the parameter and
submit the result; the keyboard path also gains the range clamp the
stepped paths always had, rather than trusting typed input verbatim.
Display-side reads of current elsewhere in the menu remain direct:
the main thread is the only writer, so it reads back its own stores
coherently.
Verified: full build clean, C89 gates on all five touched files,
thread-read audit 0 findings / 0 allowlisted pairs, hot-function
budgets, and the 74-lane compile matrix. The race itself needs a GL
context and the threaded wrapper live, which the container cannot
run - the blocking-window argument and the writer census (all four
sites converted, no other writer of the live struct outside blocking
commands) are the verification, stated as such.
The VFS/I-O ownership sweep of the discovery pass came up clean, and
this records why in the one place a future task author will look.
What the sweep verified, surface by surface: the VFS vtable handed
to cores is a static table of the frontend's own implementations,
installed before any thread exists and never swapped; task_save
serializes through its documented capture/spare-buffer design;
screenshots copy the frame at capture time on the pushing thread
precisely because the cached frame's lifetime can end under a
worker; interval autosave reads core SRAM under its sequence/dirty
protocol. And every mutable file-scope static across tasks/ -
pending flags, result caches, handoff pointers in task_movie,
task_save, task_content, task_powerstate, task_database_info and
the rest - is written only from push-side code and callbacks, never
from handlers.
That last pattern is an architectural invariant, not a habit: the
threaded runner has exactly one worker, so handlers never race each
other, and callbacks and cleanups retire on the thread that pumps
the queue - RetroArch's main thread - so the statics are main-affine
and safe without a lock. Nothing stated it, so nothing defended it:
a handler that wrote one of those statics would compile fine and
race the main thread. The contract now sits above the handler and
callback typedefs, where the pattern is inherited from.
No behavior changes; comment only.
Verified: full build clean, thread-read audit 0 findings, and the
74-lane compile matrix.
First find of the shader/filter-chain lifetime sweep: the softfilter
worker pool's free() tore resources down in the wrong order. Workers
execute the plugin's work functions with impl_data and read the
packet array, yet rarch_softfilter_free() freed the packets,
destroyed impl_data and dylib_closed the plugin libraries before
setting die and joining a single worker. The order is latent today -
process() waits for every worker before returning and free is only
reached with the pool idle - but it inverts the ownership it relies
on, and one branch of it was live: when create fails after
filt->threads is counted but the thread_data allocation is what
failed, free walked the NULL array. The pool now goes down first,
guarded on thread_data, and only then do packets, impl_data and the
plugin libraries follow.
Second find, courtesy of the new harness's leak checking:
free(filt->plugs) sat inside #ifdef HAVE_DYLIB, but
append_softfilter_plugs allocates the plug table in the builtin
build too - every static-filter build (consoles among them) leaked
one table per filter lifecycle, a couple hundred bytes on every
filter change or video reinit with a filter active. The dylib_close
loop stays guarded; the free is unconditional.
samples/gfx/filter_lifecycle compiles the shipping video_filter.c
with the full builtin filter set under AddressSanitizer and walks
the lifecycle eight times at four worker threads: create Normal2x,
verify the output size and that every sampled source pixel became
the exact 2x2 block Normal2x promises (covering all worker slices),
run a second frame through the reused pool, free. A final lane
creates from a nonexistent path and expects a clean NULL - the
error-path free. The plug-table leak was found by this harness's
first run and its fix is verified by the same run coming back
leak-clean.
Verified: harness green under ASan with no leak report, syntax
passes across the HAVE_THREADS/HAVE_DYLIB/HAVE_FILTERS_BUILTIN
define combinations, C89 gate clean, full build clean, thread-read
audit 0 findings, hot-function budgets, and the 74-lane compile
matrix.
The re-audit's remaining dominant cost with file logging on was the
unconditional per-line fflush, kept in the correctness pass for
crash durability. This takes the middle it prescribed: lines whose
tag is [ERROR] or [WARN] still reach the OS before the logging call
returns - they must survive the crash whose report they may be -
and informational traffic rides stdio's buffer, becoming durable at
the flush points: any later flushed line on the same stream, and
rarch_log_file_deinit()'s fclose. The traded tail is documented at
the decision: a hard crash can lose buffered INFO/DEBUG lines
written after the last error or warning. Tags are matched by exact
label, so a core-supplied custom tag rides the buffer like INFO.
rarch_log_line_emit() carries the verdict as a parameter, computed
from the tag by every caller - the generic branch, both Apple file
writes, and the Android file arm - so the policy is one predicate,
not one per region. The LibNX mutex still wraps the write and any
flush as one unit.
The log-atomicity harness grows the durability lane: after the
racing writers finish, a RARCH_ERR line must be readable from the
file before any deinit runs - that is the flush promise - while
informational lines are deliberately not asserted present or absent
mid-run, since stdio may flush them whenever it likes; the whole-
file scan still counts every writer line after deinit, which is the
buffered path's own promise, and skips the marker line whose shape
is not a writer line's.
Verified: the extended harness green three consecutive runs, the
logger-macros suite green in all four configurations, LibNX,
Android and Apple-macOS stub syntax passes, C89 gate, full build
clean, thread-read audit 0 findings, and the 74-lane compile
matrix.
The re-audit's two named format costs, one change: the Apple arm
formatted every line with a heap vasprintf and handed
("%s %s", tag, buffer) separately to each sink, and the desktop
companion mirror formatted a second bounded copy of its own - body
only, no tag, overflow unhandled (r >= 1024 silently truncated
without the newline repair the r < 0 path had).
rarch_log_line_write splits into its two halves.
rarch_log_line_format() renders "tag body" once into the caller's
stack line with the existing exact-sized heap detour for wide lines;
rarch_log_line_emit() is the single write plus flush under the LibNX
mutex. The non-Xbox, non-Android region now formats once up front
and hands the same buffer to everything: the companion (which
therefore shows the tag and receives wide lines whole), printf,
stderr, os_log, asl_log - all as ("%s", out), since the tag is
already inside - and the log file through emit. The vasprintf
machinery, its allocation-failure fallback and the apple_log_done
label are gone; so is the companion's private copy. One formatting
pass per line where the Apple file case previously paid three (heap
vasprintf, tag+body into the terminal, tag+body into the file) plus
the companion's fourth.
rarch_log_line_write itself remains for its one caller, the Android
file arm, and is compiled only there.
Verified: the log-atomicity harness green three consecutive runs and
the logger-macros suite green in all four HAVE_LOGGER
configurations; LibNX, Android and Apple-macOS syntax passes against
the platform stubs (the iOS os_log/asl arms compile against SDKs the
container does not have - their change is the mechanical
("%s %s", tag, buffer) to ("%s", out) rewrite, read closely in
place); C89 gate, five atomic-backend passes, full build with no
unused-function residue, thread-read audit 0 findings, hot-function
budgets, and the 74-lane compile matrix.
The threaded-video verification pass of the discovery list found the
one surface the campaigns had not reached: the GL drivers' own frame
code, running as driver->frame on the wrapper's thread while the
main thread runs free. Two live-settings reads lived there:
- gl2_frame()/gl3_frame() (and set_texture_frame) consult
*_core_context_is_mains(), whose short-circuit fell through to
*_hw_ring_expected() whenever the HW_RING flag was clear - the
common case - and that read settings->arrays.video_driver through
config_get_ptr() every frame, racing any main-thread write of the
driver string. The answer is init-time-stable, so it is now
latched: *_FLAG_HW_RING_EXPECTED is set during init - inside the
wrapper's blocking CMD_INIT, where the main thread is parked in
send-and-wait and the settings read is race-free by the same
protocol that makes the wrapper's borrowed command payloads safe -
and is_mains reads flags only. gl3_hw_ring_expected() keeps its
init-time caller; gl2's had no other caller and is gone.
- *_encode_pq_to_sdr() fetched settings and then used the pointer
only as a null-check ternary whose value side was
gl->scrgb.paper_white_nits - driver-owned and already latched by
the HDR poke path. A vestige of the fetch that once supplied the
value; the fetch is deleted and the field read directly. No
behavior change: config_get_ptr() is never NULL there.
Alongside, the wrapper protocol itself was audited for this pass:
every command is a blocking round trip (CMD_CUSTOM_COMMAND spells
its wait as send-plus-wait_reply with a running-check fallback), so
borrowed pointer payloads are safe by construction - recorded here
because it is the invariant the latch relies on.
Verified at source level with the closure analyzer, comment-stripped
before definition extraction as well as call extraction (a phantom
definition named "up", born from a comment's "bring-up)", had
swallowed a real function and hidden the portal from the sweep):
pre-fix, *_hw_ring_expected sits in both drivers' frame closures;
post-fix both closures carry no config_get_ptr and no portal. The
race itself needs a GL context to exercise, which the container does
not have - the analyzer red-green and the protocol argument are the
verification, stated as such.
Known gap, deliberately not papered over here: the thread-read audit
walks thread entries from thread-creation sites and cannot follow
the wrapper's indirect driver->frame edge, which is why its zero
never covered driver frame code. Modeling that properly needs the
blocking-window distinction (settings reads inside blocking command
handlers are safe; frame-path reads are not) and gets its own
change.
Verified: full build clean including no unused-function residue,
thread-read audit 0 findings / 0 allowlisted pairs, C89 gates at
their pre-existing baselines on both drivers (the trailing-comma
slips this change briefly introduced in the flag enums are fixed and
covered by the matrix's C89 lane), hot-function and stack budgets,
and the 74-lane compile matrix including the no-threads gl2 lane
that caught the unguarded latch.
The discovery pass's shutdown-ordering thread, opened by the mixer
retirement fix: main_exit() tore drivers down and freed global state
before task_queue_deinit(), and deinit abandons whatever is still
queued - the worker breaks out mid-queue (its own comment asked
whether it should), so remaining handlers never run, finish
callbacks never fire, and cleanup callbacks never release their
owners. A mixer load, a screenshot, a database scan chunk in flight
at quit just leaked, and anything that did retire in that stretch
retired into torn subsystems - the class the MIXER_INITED latch
guards one door of.
main_exit() now runs retroarch_drain_tasks_for_exit() first:
task_queue_reset() cancels everything so cancellation-aware handlers
wind down instead of completing their work, and a bounded
task_queue_wait_timeout() retires the queue - handlers, finish
callbacks and cleanups all run on the exiting thread while every
subsystem they can reach is still alive. The bound (three seconds)
keeps a stuck handler from hanging quit; whatever survives it meets
the old abandonment, demoted from rule to fallback. Tasks scheduled
for later (a set 'when') do not hold the wait and are dropped as
designed. The worker's questioning comment now states the contract.
samples/tasks/exit_drain links the shipping objects with only main()
replaced, boots headless with the threaded task runner, and puts the
exit-time population in flight: a cancellation-aware handler
mid-run, a task completing on its own, and one scheduled thirty
seconds out. The drain must retire the first two - handler wound
down, finish and cleanup callbacks run - in well under the bound
(432 microseconds measured) without the deferred task holding it.
The MODE=old lane runs the pre-change order, straight to
task_queue_deinit(): the aware task's callbacks never run, which is
the abandonment this change demotes.
Verified: both harness lanes green as stated, playlist_nav green,
thread-read audit 0 findings / 0 allowlisted pairs, C89 gate clean
on task_queue.c, full build scanned clean for the warning classes,
hot-function budgets and the 74-lane compile matrix.
The logger audit asked whether verbosity_get_ptr() could go; walking
its one consumer found the setting bound to it broken outright. The
menu's bool actions write the bound flag first and run the change
handler second, and write_handler_logging_verbosity() read
verbosity_is_enabled() - the same memory, post-write - to pick the
transition. Every press therefore ran the opposite one: from off,
toggling on saw the freshly written true, called verbosity_disable()
and rarch_log_file_deinit(), and snapped the flag back; from a
config-loaded on, toggling off re-initialized the log file and
re-enabled. The setting was stuck in both directions, churning the
log file against the display, and because verbosity_enable() and
verbosity_disable() re-assert the flag, the snap-back was invisible
to everything but the person pressing the button.
The handler now reads the freshly written value - the transition the
person asked for - and re-drives enable/disable from it, which also
answers the audit's bypass concern: the direct write through the
bound pointer is immediately reconciled, so the console
attach/detach side effect always follows.
On removing the getter: not worth it. The setting framework needs a
bool target to bind, the flag must stay in verbosity.c (logging
works before settings exist), and the alternative is a shadow bool
in menu code synced by hand. The getter stays with its contract
stated at the definition: one consumer, every write reconciled by
the change handler, nothing else writes through it.
samples/settings/verbosity_toggle links the shipping objects with
only main() replaced, boots the menu headless, finds the
LOG_VERBOSITY setting, verifies it is bound to the verbosity flag,
and drives its own right-press action: from off, press one lands
enabled with the log file open, press two lands disabled with it
closed, press three lands enabled again. Against the pre-fix handler
the probe fails exactly as the defect predicts - press one snaps
back to off with no file, and only the accidental parity of an
already-broken press one lets press two look right.
Verified: the toggle harness green (and red pre-fix as stated),
playlist_nav green, thread-read audit 0 findings / 0 allowlisted
pairs, C89 gate clean on menu_setting.c, full build scanned clean
for the warning classes, hot-function budgets and the 74-lane
compile matrix.
The whole-tree logger audit's actionable findings, the ones that are
defects rather than policy:
- Xbox1 wrote the message body with wvsprintf, which takes no size:
any expansion past the 256-byte stack buffer wrote off its end.
Both Xbox1 and WinRT now go through bounded vsnprintf (the msvc
compat header maps it for that toolchain) into a 1024-byte line,
matching the generic branch's budget; truncation stays the policy
on this heapless path.
- The ORBIS HAVE_LOGGER va_list macros handed debugNetPrintf
(tag, fmt, vp): the tag became the format string and a raw va_list
rode as a value argument, and RARCH_WARN_V mapped to a different
debugnet level than RARCH_WARN. The non-ORBIS logger_send spelling
was two calls per line - prefix plus tag, then body - which
interleave under concurrent writers, with one branch carrying a
stray "[WARN] :: " prefix. All eight *_V sites now route through
one bounded format-then-send helper per backend: one call per
line, body formatted, levels matching the non-V spellings
(debugnet has no WARN level, so warnings are INFO on both
spellings).
- LibNX ran mutexInit on every retro_main_log_file_init: a
file->console->file cycle re-initialized a mutex another thread
may hold for a write. Once per process now, and deinit still never
destroys it, for the same reason.
- The Android file arm logged the bare body with no tag - the one
line-format inconsistency across platforms - through its own
vfprintf pair. The generic branch's format-once single-write is
now a shared helper, and Android's file arm goes through it: tag
included, line whole under stdio's per-call lock (logcat keeps
receiving the tag as a priority).
- RARCH_LOG_BUFFER computed data + len before any NULL check - UB on
a NULL buffer - and carried a dead 'end' from an older loop. NULL
with a length returns; the leftover is gone.
samples/frontend/logger_macros compiles verbosity.h under all four
HAVE_LOGGER configurations (ORBIS x IS_SALAMANDER) against capturing
sinks and drives the *_V macros with real va_lists: one sink call
per line, body formatted, tag and prefix present, WARN_V at
RARCH_WARN's level. Against the pre-fix header the logger
configuration fails on exactly the audited shapes (two calls per
line, tag lost, prefix lost); the harness also documents that the
old macros only ever compiled with literal tags, since they pasted
the tag into a string constant.
The audit's policy items stay open deliberately: per-line fflush vs
crash durability, macro early-outs and a compile-time RARCH_DBG,
printf format attributes (a tree-wide -Wformat sweep of its own),
the verbosity_get_ptr writer walk, and asl_close on the Apple
client. Each changes behavior or blast radius and gets its own
decision.
Verified: the logger-macro harness green on all four configurations
and red as stated on the pre-fix header, log_atomicity green, host
syntax passes under -DHAVE_LIBNX (stubbed mutex) and -DANDROID
(stubbed logcat) with implicit-function-declaration promoted to an
error, five forced atomic backends, C89 gate at baseline, full build
scanned clean for the warning classes, thread-read audit 0 findings
/ 0 allowlisted pairs, hot-function budgets and the 74-lane compile
matrix.
2277ad2a58 made audio_driver_mixer_add_stream() refuse streams
outside the mixer's init..done window, and this suite's fixture -
which includes audio_driver.c directly and hand-builds only the
state lock - was modelling the pre-latch world, so every add it
drives was refused and all its lanes failed.
The fixture now establishes the window the way the shipping
lifecycle does, raising AUDIO_FLAG_MIXER_INITED where
audio_driver_init_internal() would - and the guard itself becomes
the suite's stage 0 first: before the latch is raised, an add with a
counting owner must be refused with the owner released exactly once,
and lock balance after the refusal is proven the suite's own way,
by the next add having to take the lock under the watchdog. That
gives the boundary a deterministic unit lane alongside
samples/audio/mixer_retirement's full-boot one.
Verified: the mixer-lock suite green under ASan/UBSan with the new
stage first, mixer_retirement green unchanged, thread-read audit 0
findings / 0 allowlisted pairs on the relinked binary.
The SYNC backend's load-acquire is the canonical fetch_and_add of
zero, and OSAtomic's int load is the same idiom - RMWs, so a
const-qualified pointer to the state trips -Wdiscarded-qualifiers.
audio_driver_dev_frame_bytes() reads the flags through a const view
and has been carrying exactly that warning on the shipping DINGUX
lanes (miyoo, rs90), where the SYNC backend is what the toolchain
selects. The object behind an atomic is never const - only the
caller's view is - so the macros now cast to the volatile atomic
type themselves, the same shape the int64 variants and OSAtomic's
size loads in this header already use. Every other backend's loads
(C11, __atomic, volatile, PS2) already accept const views.
Verified: the retro_atomic behavior suite passes under forced SYNC,
GCC_NEW and VOLATILE backends; the forced-C11 run trips a
pre-existing coverage assertion on this single-CPU container
("no reader ever landed inside a write") on clean master at the same
rate, which is the seqlock stress lane reporting an unopened
scheduling window, not a correctness failure - noted for a test-side
fix. All five forced backends now pass -Wall -Werror syntax on
audio_driver.c, video_driver.c and rgui.c; thread-read audit 0
findings / 0 allowlisted pairs; full build scanned clean for
discarded-qualifier, implicit-declaration and static-follows
warnings; hot-function budgets and the 74-lane compile matrix.
RARCH_LOG_V's generic branch - Linux, Windows, Switch and every
console - wrote each line as a tag fprintf followed by a body
vfprintf, under a comment claiming it was one call. Between those
two calls sits a window where a thread switch interleaves one
writer's tag into another's message; RetroArch logs from task
workers and, under threaded video, from the video thread, so the
window is real. The Switch-only mutex around the pair existed to
paper over exactly this on a platform whose newlib stdio locking
was not trusted.
The branch now formats tag and message into one stack buffer and
ships it with a single fputs: stdio's per-call lock keeps concurrent
lines whole on every platform, which is what the other branches
already had (the Xbox/WinRT head formats into one buffer, Apple goes
through vasprintf, Android hands the va_list to the OS logger).
Lines wider than the 1024-byte stack buffer take an exact-sized heap
detour rather than truncating; if that allocation fails, the
truncated stack line ships - a shortened message over a dropped one.
The libnx mutex remains, shrunk around exactly the one write and its
flush, since trusting devkitA64's stdio lock retargeting is a
platform verification this change does not make.
Two unreachable blocks fall out of the same read of the function: an
inner __WINRT__ arm nested inside the head branch's #else (only
compiled when __WINRT__ is undefined, so never), and the HAVE_LIBNX
mutex lines inside the Apple ASL branch, which referenced a
g_verbosity symbol that exists nowhere in the tree - only an
impossible platform combination kept them from being a compile
error.
samples/frontend/log_atomicity links the shipping objects with only
main() replaced and hammers RARCH_LOG from concurrent threads:
fixed-width payloads from racing writers, plus a lane wider than the
stack buffer so the heap detour is raced too. Every line of the
resulting file must be a tag followed by one complete payload, at
full count. Against the split-write shape the tear reproduced in
three of six runs on a single CPU; the lane documents that the red
is probabilistic by nature - its standing value is the invariant it
hammers on.
Verified: thread-read audit 0 findings / 0 allowlisted pairs, the
harness green three consecutive runs, a host syntax pass under
-DHAVE_LIBNX with stubbed mutex primitives, C89 gate at its
pre-existing baseline, five forced atomic backends, the full build
scanned clean for implicit-declaration and static-follows warnings,
hot-function budgets and the 74-lane compile matrix.
First find of the discovery pass the whole-tree audit pointed at
next (task retirement against subsystem teardown):
audio_driver_mixer_add_stream() trusted its caller's timing. The
mixer's lifecycle is the audio driver's - audio_mixer_init() runs
inside audio_driver_init_internal(), audio_mixer_done() inside
audio_driver_mixer_deinit() - and done() releases every voice and
frees its lock. Two reachable paths deliver a stream outside that
window:
- shutdown tears the drivers down before the task queue, so a
mixer-load task that finishes late retires its callback into a
done mixer;
- every failure exit of audio_driver_init_internal() precedes
audio_mixer_init(), so a session whose audio driver failed to
come up funnels each menu sound into the same door.
In both cases add_stream claims a voice whose lock is gone -
slock_lock(NULL) returns silently, so nothing crashes; the voice is
written unlocked, its per-voice resources are orphaned when the next
audio_mixer_init() wipes voice types, and a phantom stream entry
survives in mixer_streams[].
AUDIO_FLAG_MIXER_INITED now latches the window: raised right after
audio_mixer_init(), dropped with MIXER_ACTIVE before
audio_mixer_done(), and checked at the top of add_stream - the one
injector; the other public mixer entry points walk mixer_streams[],
which teardown has already emptied. The refusal keeps the ownership
contract: buf_owner is released exactly as on every other failure
path, and out_slot reports -1.
samples/audio/mixer_retirement links the shipping objects with only
main() replaced, boots headless against ALSA's null PCM device (the
"null" audio driver is a stub vtable that cannot initialize), and
drives the seam: a baseline add lands in a slot with the WAV
conversion releasing its owner once; an add after
audio_driver_deinit() is refused with the owner released exactly
once and out_slot -1; an add after CMD_EVENT_AUDIO_REINIT lands
again. With the guard removed the refusal lane fails exactly as the
defect predicts (the stream lands in slot 16 of the torn mixer).
Noted while here, untouched: the SYNC atomic backend's load on a
const-qualified state pointer trips -Wdiscarded-qualifiers at
audio_driver_dev_frame_bytes() on shipping DINGUX lanes; that is
pre-existing and gets its own change.
Verified: thread-read audit 0 findings / 0 allowlisted pairs, the
mixer_retirement harness green and red as stated, playlist_nav
green, host syntax passes under __PS3__, PSP and VITA (the file's
platform gates) with implicit-function-declaration promoted to an
error, the full build scanned clean for implicit-declaration and
static-follows warnings, hot-function and stack budgets, the
74-lane compile matrix, forced atomic backends (SYNC carries the
pre-existing const warning above; the other four clean) and the C89
gate.
The whole-tree audit at 96ee694ae9 closed every tracked finding and
left one P4: config_file_stream_push() accumulated stream->total_in
with an unchecked addition. The counter never sizes an allocation or
a copy - that is stream->len, checked separately at the top of
push() - it only feeds the RHMAP_FIT pre-size hint, so a wrap was
never memory-unsafe. But a stream large enough to reach the ceiling
wants the largest hint, not a wrapped small one, and an honest
counter costs one compare: total_in now saturates at (size_t)-1,
and its comment states the trust boundary so the next audit doesn't
have to re-derive it.
No new test lane: the counter is private to the translation unit and
saturation is externally unobservable by construction - the existing
config_file suite (which drives the streaming parser across packet
sizes, plain and under ASan/UBSan) is the regression coverage, and
it passes unchanged.
Verified: the config_file suite plain and sanitized, a host syntax
pass on the __PS3__ side of the file's one platform gate with
implicit-function-declaration promoted to an error, the full build
scanned clean for implicit-declaration and static-follows warnings,
thread-read audit 0 findings / 0 allowlisted pairs, the 74-lane
compile matrix, five forced atomic backends and the C89 gate at its
pre-existing baseline.
96ee694ae9 broke three console lanes in two ways, both invisible to
an x86 Linux build:
- rgui_flush_video_config() had no forward declaration, and
rgui_render() calls it before the definition. Host gcc let the
implicit declaration through as a warning and the later
static-follows-non-static as another; devkitPPC makes the pair a
hard error, failing ngc and wii. The declaration joins
rgui_apply_video_config()'s forward block.
- Under DINGUX, rgui_set_aspect_ratio() carried platform-local
overrides ('unsigned aspect_ratio = RGUI_DINGUX_ASPECT_RATIO' and
the lock) that had shadowed the settings reads the staging split
turned into parameters - now a redeclaration of the arguments,
failing miyoo and rs90. The fixed-aspect override now assigns to
the arguments instead of declaring over them, which is also the
honest semantics: Dingux runs a fixed aspect regardless of what
the caller supplies.
While in the exact lines: the 'vp' viewport local is only used on
platforms that query the viewport (VITA, DJGPP and the generic
branch), so its declaration takes the matching guard and the
unused-variable warning GEKKO and DINGUX logs carried is gone.
Verified beyond the usual board: menu/drivers/rgui.c passes a host
syntax check under -DDINGUX (the redeclaration is gone) and under
-DGEKKO -DHW_RVL against stubbed libogc headers (no implicit
declaration, no static-follows, no unused vp), and the full Linux
build now runs with implicit-function-declaration promoted to an
error and reports zero. Thread-read audit 0 findings / 0 allowlisted
pairs, the rgui_aspect_defer and playlist_nav harnesses green, the
74-lane compile matrix, five forced atomic backends and the C89
gate.
rgui_frame() runs inside the video driver's frame(), which under
threaded video is the video thread. Its frame closure held the
driver's last two live-settings functions, and one of them wrote:
rgui_set_video_config() stored five settings fields (the aspect
index and the custom viewport) plus the global custom-aspect LUT
entry, and the frame path's aspect-lock and resize handling called
it from the video thread. The old delay_update flag already deferred
CMD_EVENT_VIDEO_SET_ASPECT_RATIO to rgui_render() because firing it
from the frame deadlocks; the settings writes now take the same
road.
- rgui_set_video_config() splits into rgui_stage_video_config()
(copy the configuration onto the rgui handle and raise
RGUI_FLAG_ASPECT_UPDATE_PENDING; latest staged wins) and
rgui_set_video_config_now() (main thread: apply and fire).
rgui_render()'s pending consumer applies the staged copy - on the
thread that owns settings writes - before firing the command that
consumes it.
- rgui_set_aspect_ratio() is dual-context (render, init, populate
and toggle on the main thread; the frame path's aspect and resize
checks on the video thread), so its two settings arrive as
arguments from each caller's own context, and it always stages:
the immediate-mode callers that need the configuration in place
before proceeding (video init, menu toggle, render's framebuffer
resize) call the new rgui_flush_video_config() on return, so the
apply side stays structurally outside the frame closure rather
than dead-in-a-branch inside it.
- rgui_frame()'s config_get_ptr() root is gone; the frame closure
(16 functions, walked over comment-stripped call edges) contains
no settings-> read and no config_get_ptr.
samples/menu/rgui_aspect_defer links the shipping objects with only
main() replaced, boots rgui headless on the null video driver, flips
the aspect-ratio lock on the main thread, and drives
menu_driver_frame() with the real video_driver_build_info() snapshot
followed by the render slot: the detecting frame must leave the
settings untouched, the render must land the staged configuration
(custom aspect index, populated custom viewport), and further rounds
must leave it alone. On the pre-fix shape - the frame path writing
the settings itself - the first lane fails; here all pass.
This closes the menu class sweep: materialui, xmb, ozone and rgui
all read the per-frame snapshot on the frame path, and none of them
touch the live settings from the thread that draws.
Verified: thread-read audit 0 findings / 0 allowlisted pairs, the
defer harness green (and red on the pre-fix shape), the playlist_nav
harness (which boots this driver) fully green, hot-function and
stack budgets within limits, the 74-lane compile matrix, five forced
atomic backends and the C89 gate on menu/drivers/rgui.c.
ozone_frame() runs inside the video driver's frame(), which under
threaded video is the video thread. Its config_get_ptr() was the
frame path's live settings root; the render vtable slot and the
navigation handlers run from runloop_check_state() on the main
thread and keep their live reads legitimately - the call-edge walk
that separates the two matters, because the textual analyzer pulls
ozone_render into the frame closure through a comment mention alone.
- ozone_draw_entries, ozone_draw_header, ozone_draw_footer and
ozone_draw_thumbnail_bar take const video_frame_info_t* in place
of settings_t*, with all 25 field reads on the snapshot;
ozone_draw_messagebox gains the parameter for its swap-ok-cancel
read.
- The dual-context helpers - ozone_compute_entries_position (render
and frame), ozone_update_content_metadata (navigation and frame
via compute), ozone_manage_available (entry actions and the
footer) - take their settings as value arguments; each caller
supplies them from its own context, the live settings on the main
paths and the snapshot on the frame path.
- The snapshot gains the ten fields the paths consume:
show_sublabels_current_selection_only, disable_search_button,
playlist_show_entry_idx, kiosk_mode_enable, content_runtime_log,
content_runtime_log_aggregate, ozone_scroll_content_metadata,
left_thumbnails, ozone_header_icon and ozone_header_separator.
The conversion also surfaced a shipping regression: an earlier
mechanical snapshot pass aimed configuration_set_uint()'s write
target at the frame's snapshot copy in the "change theme on the fly"
block, so the preferred-system-theme value was discarded with the
frame - the setting never persisted, and the stale snapshot re-fired
the block every frame. The write was also a frame-thread settings
write in the first place. The frame path now stores the value on the
ozone handle under OZONE_FLAG2_COLOR_THEME_WRITE_PENDING, and
ozone_render() - the main thread, which owns settings writes -
performs the configuration_set_uint and clears the flag before
anything below it reads the setting.
samples/menu/ozone_color_theme_defer links the shipping objects with
only main() replaced (the playlist_nav pattern), boots headless with
ozone on the null video driver, raises the menu, flips the
preference on the main thread, and drives menu_driver_frame() with
the real video_driver_build_info() snapshot followed by the render
slot: the system theme must persist, marking the configuration
modified, and the block must quiesce once the setting catches up.
On the broken shape the setting never moves and the block re-marks
the configuration every frame; both lanes fail there and pass here.
Verified: thread-read audit 0 findings / 0 allowlisted pairs, a
comment-stripped closure sweep of ozone_frame showing no settings->
or config_get_ptr remaining in the 48-function frame closure, the
defer harness green (and red on the regression shape), hot-function
and stack budgets within limits, the 74-lane compile matrix, five
forced atomic backends and the C89 gate on the touched files.
The title mailbox published by strdup on the frame path - once per
fps_update_interval, video_driver_frame() allocated, and the
threaded_video heap lane (which predates the mailbox) has been red
since: an allocation on the frame path is a lock, a possible syscall
and a fragmentation source on the one path that must not have any.
The pointer mailbox becomes a three-slot inline buffer published by
index. window_title_slot holds (slot + 1), or 0 for empty; publish
fills a free slot and exchanges its index in, take exchanges the
word to 0 and copies the slot out. Nothing allocates, nothing frees,
neither side takes a lock, and the consumer's empty fast path stays
one acquire load.
Three slots, not two, so the guarantee needs no timing argument: at
most one slot is published and at most one is held by a consumer
mid-copy (takes are sequential, so the consumer holds only the slot
it exchanged out last). The producer's exchange return decides its
next slot - nonzero is the untaken previous publish, which no
consumer can hold, so it is reused; zero means the consumer took the
previous publish and may still be copying that slot, so the producer
writes the remaining third. A held slot is therefore never rewritten
no matter how long the copy stalls. With two slots a lapping
producer overwrites the held slot mid-copy; the suite's new stall
lane shows exactly that (38 of 40 stalled holds torn at
TITLE_SLOTS=2) and holds the shipping shape to zero.
The window_title_mailbox suite models the new protocol byte for
byte, keeps the checksum/order/final-title claims, and adds the
stall lane: a consumer that copies its held slot one byte at a time
while hundreds of publishes land. The heap-traffic claim is now
structural - there is nothing to leak.
Verified: threaded_video 18/18 with the frame-path heap lane at 0
calls in 300 frames both threaded and not, the mailbox suite green
at 3 slots and red at 2, thread-read audit 0 findings / 0 allowlisted
pairs, hot-function and stack budgets within limits, the 74-lane
compile matrix, five forced atomic backends and the C89 gate on
gfx/video_driver.c.
xmb_frame() runs inside the video driver's frame(), which under
threaded video is the video thread; its config_get_ptr() was the
frame path's one remaining live settings root, plumbed through
xmb_draw_items into the per-item context and onward to every draw
helper. The whole path now reads the video_info snapshot the main
thread assembles under MENU_ST_FLAG_ALIVE, and the live pointer is
gone from the frame path entirely.
- xmb_draw_text, xmb_draw_items, xmb_draw_fullscreen_thumbnails,
xmb_draw_no_thumbnail_available and xmb_render_messagebox_internal
take (or gain) const video_frame_info_t*; xmb_draw_ctx_t carries
video_info in place of settings, so xmb_draw_item and its sublabel
helper read the snapshot through the context they already receive.
- The snapshot gains the fields the path actually consumes:
menu_font_color_{red,green,blue}, menu_xmb_entry_icons,
menu_xmb_switch_icons, playlist_show_history_icons,
ozone_sort_after_truncate_playlist_name, input_turbo_bind and
input_turbo_button, and xmb_font_is_default - the last a derived
bool because the one frame-path consumer of path_menu_xmb_font
only ever asks whether it is the FILE_PATH_UNKNOWN placeholder;
the string itself stays out of the per-frame snapshot.
- xmb_icon_get_id() is dual-context (xmb_set_title on the main
thread, xmb_draw_item on the frame path), so the turbo values it
needs arrive as arguments and each caller supplies them from its
own context: the live settings on the main path, the snapshot on
the frame path.
Verified: thread-read audit at 0 findings / 0 allowlisted pairs, a
textual sweep of every frame-path function showing no settings-> or
config_get_ptr remaining, hot-function and stack budgets unchanged
within limits, the 74-lane compile matrix, five forced atomic
backends and the C89 gate on the touched files.
video_state_get_ptr() stays - the main thread and the wrapper's
finder-entries locate ra-video state through it - but nothing that
runs on a task worker or on the video thread calls it any longer,
and the audit now proves that with no allowlisted exceptions: both
entries (video_thread_loop's owner-domain reads and
task_load_handler's frame-count poll) are deleted from
tools/thread_read_allow.list, and the audit reports 0 findings with
0 allowlisted pairs.
Every conversion is a stable-pointer capture taken on the main
thread before the consuming thread exists:
- task_load_handler polls core readiness through a frame-counter
pointer captured into save_task_state_t at both load-task push
sites; the read keeps its documented single-benign-poll-per-tick
semantics.
- thread_video_t carries the singleton's address from init; the
loop's statistics publishes, video_thread_pace_hold(), and the
loop-called video_thread_convert()/video_thread_filter() (which
now take the wrapper instance as their first argument) go through
it. A file-scope capture of the wrapper instance, set at init and
cleared first thing in video_thread_free(), serves the entries
callable from the video thread itself - call_on_waiter, font_init,
texture_handle - where NULL now covers both "no wrapper" and the
driver-reinit window the thread_wrapper_active check used to
guard.
- The font driver binds the singleton's address once, from
video_driver_init_internal() before any driver or wrapper exists,
because under threaded video the OSD fonts live on the video
thread. Unbound degrades exactly as pre-video-init always has: no
shared OSD font, lifecycle calls no-op, width queries return the
no-font -1.
- gfx_widgets captures the address at init, before the draw worker
exists, for the worker's state-lock dispatch.
The font_lifecycle harness gains an unbound-phase probe and binds
the capture the way production does, so the degradation contract
stays pinned.
The menu half of the draw-thread settings sweep. menu_driver_frame
is invoked from inside the video drivers' frame() callbacks, which
under threaded video run on the video thread - which is exactly why
the main thread assembles the video_info snapshot with the menu
display preferences before handing the frame over. A per-file call
graph walked from each driver's frame() function against every
config_get_ptr site shows materialui's whole exposure to be one
function: materialui_render_header took the live settings pointer
from the frame and read five display preferences - battery, core
name, timedate enable, style and separator - on the draw thread.
All five already have snapshot fields. The header render now takes
video_info instead of settings, reads the snapshot, and the
frame's live settings local is gone; the frame path of this driver
touches no live settings at all.
The same walk maps the remaining drivers for the rounds ahead: xmb
routes its live pointer through xmb_draw_items and two more draw
helpers plus an inline config_get_ptr in a call argument, and
ozone and rgui show wider reachable sets that need their call
edges verified before their counts are believed - navigation
handlers appear reachable from frame only through shared helper
names. Full build, per-file C89, the 74-lane matrix, all four
budgets and the fifty-one-entry audit zero hold.
With Remember Last Used Shader Directory off and presets in the root
of the Video Shaders directory, Next/Previous Shader cycle those
presets, starting from the loaded preset when it is one of them. With
the setting on, or when the root holds only RetroArch's own
retroarch.* preset, they step from the loaded preset into neighbouring
folders, as before.
A retroarch.* or auto preset stands in for the preset it references,
so a core shader override in the config directory steps from the
preset that defines the shader rather than from itself. Cycling the
root is the exception: a retroarch.* preset there is one of the
presets cycled, so the reference is followed a single hop. A preset
loaded as itself anchors where it sits, whatever it references, as
Save Preset As writes a simple preset too and following those would
cycle a preset's source folder instead of the folder it was saved
into.
Once a hotkey has applied a preset from the list, the press after it
steps from that list rather than anchoring on the preset again: an
entry that is itself a simple preset would otherwise resolve back to
the entry it references, and the cycling would never leave the pair.
The list is dropped when the setting changes, so turning it on follows
a retroarch.* preset out of the root that was cycled while it was off.
The setting defaults to on. It also opens the shader file browser at
the last used folder.
samples/gfx/shader_dir_cycle covers which folder the hotkeys cycle for
a core override that references a preset in the Video Shaders root,
and for the setting changing mid-session.
The rest of the draw-thread settings sweep the widgets round
opened. Every animation callback in gfx/ is clean after the
screenshot latch, and the widget frames take their settings from
the pre-snapshotted video_info as designed - but
gfx_display_texture_filter() read menu_texture_mipmapping live,
and two of its eighteen callers run off the main thread: the badge
loader, reached from the widget appliers and frame-time badge
retries on the draw thread, and the screenshot widget's iterate.
Both were invisible to the audit behind their callback pointers.
The function now refreshes a relaxed atomic latch on every
main-thread call - menu icon loads alone refresh it at least per
menu rebuild - and gfx_display_texture_filter_latched() serves the
off-main sites from the latch, tracking the setting to within one
texture's filter mode. The badge loader and the screenshot iterate
use it; the sixteen main-thread callers keep the live read, which
is what keeps the latch fresh. The badge_cache sim's stub set
gains the new symbol - checked against every sample that links the
touched files, per the standing lesson.
Full build, the badge_cache suite, all four budgets, the 74-lane
matrix, per-file C89 and the fifty-one-entry audit zero all
pass.
The gfx/widgets settings survey's finding: of the directory's six
config_get_ptr sites, four run on the main thread - the screenshot
task callback, the state-slot show, the flash starter they call,
and the cheevos appearance update - but two are animation
callbacks, and animation callbacks fire on the draw thread, where
a live settings read is the same worker-thread singleton class the
audit hunts, hidden from it behind the callback pointer.
The flash-speed and duration settings are latched into the
widget's state as atomics at its two main-thread entry points, and
the fadeout and end callbacks read the latch, never the live
settings. Staleness across a mid-animation settings change costs
one animation's timing. The main-side sites keep their live reads:
they are the latch's writers.
gfx/widgets lock survey, second conversion - and this one was not
an optimization. The leaderboard display's array_lock is used at
ten sites and created at none: no slock_new for it exists anywhere,
so every acquisition has been slock_lock(NULL), a silent no-op,
and the cheevos thread's in-place mutations of the tracker and
challenge arrays - memmove compaction included - have raced the
draw thread's render unguarded in shipping builds. On top of the
phantom lock, the producers called the font driver for width
metrics, fetched and unloaded badge textures, and stamped
animation-adjacent state from the cheevos thread.
All seven mutators now cross as whole command nodes on a lock-free
MPSC stack - set/hide tracker, set/hide challenge, the two clears
riding the same ordered stream, progress, and the two flags - and
the widget's iterate drains them in arrival order on the draw
thread. From there the arrays, the progress tracker and the flags
are draw-thread-only plain state: the frame renders and the
visibility test reads with nothing to guard, and the badge
fetches, unloads and font-width queries run on the thread those
calls belong to. The phantom lock's field, sites and teardown are
gone.
samples/gfx/leaderboard_cmd_mpsc pins the handoff - the only
concurrent part - across 4000 mixed commands: whole checksummed
payloads in issue order, every node freed exactly once, and the
applied end-state byte-equal to a single-threaded oracle running
the same script. Clean under ASan/UBSan and ThreadSanitizer. Five
backends compile the file, C89 holds, and the full build, the
popup and widgets-race suites, the 74-lane matrix, the budgets and
the fifty-one-entry audit zero all pass.
--enable-vulkan brings vulkan_emulated_mailbox_loop into the
audited set: fifty-one entries in the full Linux build, still zero
findings - the acquire worker touches no singleton. The recipe
script gains the flag, the package, and the count, and the blind
spot names one fewer entry.
The survey of the driver's three locks, for the record here:
queue_lock is the Vulkan specification's own external-
synchronization requirement on VkQueue and is not a lock this
codebase gets to remove; the emulated mailbox's lock and condition
variable are the worker's request-wake rendezvous, where a signal
outside the lock races the predicate wait, and its flags share one
word under that single discipline; and the third site is the
queue_lock's per-reinit lifetime, not a lock of its own. All
three stay.
The answer to whether gfx_thumb_worker_lock can go atomic splits
down the middle of what it guarded. The queue links and the
release() rendezvous stay under it: unlinking a queued job from
the middle of an intrusive list and waiting out an in-flight
decode before freeing the job are what the lock and its two
condition variables are for, and both run cold - submit and close,
not per frame.
The hot side was neither of those. The per-vsync poll took the
lock twice per animated thumbnail purely to read and write status
words, and status is release/acquire shaped: the worker's
READY/FINISHED store now comes as a release after the frame,
duration and loops_left are written, so the poll's acquire load
seeing READY sees the decode's results whole - formalising the
ownership the code already documented ('READY and not queued: the
worker holds no reference'). The two per-poll lock sections are
gone; each status field's answer stands on its own, since a job
the poll sees as IDLE is not queued and cannot be touched by the
worker. Status writes on the still-locked paths become atomic
stores like the rest, and the RUNNING wait's predicate reads the
atomic under the lock exactly as before.
The gfx_thumbnail_anim harness passes plain and under Clang
ThreadSanitizer - the direct test of the removed sections - and
gfx_thumbnail_preview passes over its generated fixtures, as does
the thumbnail status-atomic harness next door. Five backends
compile the file, C89 holds, and the full build, the 74-lane
matrix, all four budgets and the fifty-entry audit zero pass.
The tree-wide census flagged this as its biggest fish -
audio_driver_flush holds it on every core sample batch - and the
survey of all twenty-three sections says: keep it, and say why at
the declaration. Everything cheap to publish already crosses
without it - runloop_snapshot, the AUDIO_FLAGS word, the SPSC pipe
ring, the eventcount wakeups - and what remains under the lock is
what atomics cannot express: the driver's own write() and control
calls serialized against device reinit, resampler and DSP
instances against their replacement, the mixer's voice table
against task-thread play() calls, and the nesting contract the
mixer_lock sample pins. The per-batch cost is one uncontended
acquire around microseconds of resampling and device I/O; the
audio_driver_sample_batch budget holds at 24 of 94 through it.
The doc block now carries that reasoning so the next lock hunt
starts from the answer instead of re-deriving it - or, worse,
converting one section and quietly breaking the ownership
contract of the other twenty-two.
The tree-wide lock census's first conversion. The popup queue lock
covered the widest critical section left in the widgets: the entire
popup draw - scissored GPU calls included - held it against a rare
push from the cheevos thread, and the push, when the queue was
empty, started the slide-in animation from that thread, a
cross-thread poke into animation state the lock never made safe.
The push also lazily created the lock itself on first use, a race
of its own.
The cheevos thread now builds a whole node - title, subtitle,
badge name, owned by the node - and pushes it onto a lock-free
MPSC stack, nothing else; the badge texture prefetch stays ahead
of it, since that call can round-trip to the video thread. The
popup widget's iterate, on the draw thread, drains the stack in
arrival order into the ring, which from then on is draw-thread-only
plain state: render, badge retry, advance and free all run there,
lock-free because single-threaded. A popup arriving at a full ring
is dropped whole, as before, and the slide-in starts at drain
time, on the one thread animation state may be touched from. The
widget-state lock wrapper around the push is gone too: the push
touches only the popup's own stack.
samples/gfx/achievement_popup_mpsc holds the handoff across 3000
pushes: whole checksummed nodes in posting order, every node freed
exactly once whether consumed or dropped (ASan), the slide-in
starting on empty-to-nonempty transitions consumer-side, clean
under ThreadSanitizer; the write-after-push sabotage is caught at
2991 torn nodes. Full build, the widgets race sample, the 74-lane
matrix, all four budgets and the fifty-entry audit zero hold.
Census verdicts for the rest, recorded: core_info_list_mutex stays
(task readers walk a list the main thread swaps and frees - the
lock is the grace mechanism, on scan-cold paths); the thumbnail
worker lock stays (a wake/done rendezvous); audio_driver's
state_lock sits under audio_driver_flush on every core sample
batch and is the census's biggest fish - it needs its own survey
sequence, not a drive-by.
The threaded-video wrapper survey, with per-lock verdicts:
Kept, as rendezvous rather than data: 'lock' with cond_reply and
cond_user (the command slot - callers must block for replies, and
the ownership/nesting semantics are the protocol, not contention),
cond_ring and cond_thread (ring progress - the waits are the
point, and a lock-free ring plus a wakeup primitive is the same
machine with more parts), frame.lock (the menu texture and
apply_state_changes handoff - a multi-field staged buffer whose
buffer resizes freely from the main thread; converting trades one
lock for a full texture copy per update), and waiter_call.cond
(the cross-thread call service, welded to the reply waits).
Converted: alpha_lock. Its resize path was already quiescent - the
alpha array is reallocated only inside the CMD_OVERLAY_LOAD
handler, with the main thread blocked in that command's reply wait
and the video thread out of its frame call - so the lock's live
concurrency was one fire-and-forget setter against the per-frame
apply. The values are float bits in atomic ints now; the setter
stores a value relaxed and store-releases alpha_update, and the
apply clears the flag with an acquire fetch_and BEFORE reading the
values, so a set landing mid-apply re-raises the flag and lands
whole next frame. Clearing first is what makes an update
impossible to lose, and the clear-after order is the trap.
samples/gfx/overlay_alpha_flag holds both claims: 4000
fire-and-forget writes against a paced applier converge to zero
stale slots, clean under ThreadSanitizer; the clear-after sabotage,
with the final write choreographed into the read-to-clear window,
loses it deterministically. All five backends compile the wrapper
(fetch_and where one backend lacks exchange, void casts where
atomic ints are volatile-qualified), C89 holds, the 74-lane
matrix, the video_thread_frame budget and the fifty-entry audit
zero all pass.
What context_lock still covered: video_driver_is_hw_context()
reading hw_render.context_type against video_driver_free_hw_context()
running context_destroy() and zeroing the struct - one word of
protection, while every other hw_render field read in the tree goes
unlocked as main-thread state. The race window is real
(core_unload_game frees the hw context while threaded video still
runs), so the word's coherence and its ordering keep mattering; the
lock does not.
hw_context_type is that word as an atomic mirror. Both writers run
on the main thread and store-release it after their edit lands -
SET_HW_RENDER after copying the callback in,
video_driver_free_hw_context() after the destroy and the memset -
so an acquire reader observing RETRO_HW_CONTEXT_NONE is guaranteed
the teardown completed, exactly the ordering the lock's reader got.
video_driver_is_hw_context() is one acquire load on every backend
and build, threaded or not. The lock's field, lifecycle and both
teardown sites go, along with the stale lock-ordering notes in gl1,
gdi, d3d9hlsl and d3d9cg - the last still naming both dead locks.
Also swept up: runloop_task_msg_queue_push() held a runloop_st
local whose last use left with RUNLOOP_MSG_QUEUE_LOCK; widgets
builds now compile it warning-free.
samples/gfx/hw_context_mirror holds the ordering claim across 3000
set/free cycles with a mid-memset stall: a reader that observes
NONE finds every struct byte zero, clean under ThreadSanitizer
through the release/acquire pair; releasing the mirror before the
memset instead is caught at 53 dirty observations. Clean
C89_BUILD=1 rebuild, all five backends over video_driver.c and
runloop.c, the 74-lane matrix, all four budgets, the runloop and
gfx suites, and the fifty-entry audit zero all hold.
NOT BUILT FOR ANDROID LOCALLY: this machine has no NDK, so
android_input.c could not be compiled whole here. The rewritten
region compiles clean standalone - gnu99 and pedantic C89 with
declaration-after-statement - against micro-stubs of the JNI and
glue surface it touches, and the protocol carries a harness, but
the file needs a real Android build before this merges anywhere
that matters.
android_kbd_lock is gone, with the staging buffer and the dirty
flag. The UI thread's JNI callback publishes an immutable snapshot
of the whole keyboard line - text plus finished/cancel - into a
one-slot atomic mailbox: publish is exchange-in, freeing whatever
snapshot was never taken, and android_keyboard_poll() on the input
thread exchanges it out. Last-wins is the semantics, not a
compromise: the Java side sends the whole line on every event and
finished arrives on the last one. The open gate is atomic, written
at start/end on the input thread and load-acquire checked by the
producer; a snapshot racing past a closing gate sits unread and is
drained at end and at the next start. Everything else the lock
covered - the line buffer, its cursors, the callback - is
input-thread-only state and is now plainly that.
samples/input/android_kbd_mailbox reproduces the protocol and
holds four claims across 3000-event sessions: snapshots are whole
(checksummed), the session's final state equals the last published
snapshot with finished never lost, a cancel session ends cancelled
through the NULL-line callback path, and a snapshot published as
the gate closes is drained rather than delivered into the next
session. Clean under ASan/UBSan and Clang ThreadSanitizer; a
sabotage lane that publishes before filling is caught at 3001 torn
snapshots. The Linux build and fifty-entry audit zero are
unaffected - this file is not in them - which is exactly why the
harness and the Android build note carry the weight here.
CODING-GUIDELINES line 275: explain how the code works moving
forward, not how it worked historically - and the rule applies to
code comments. A sweep of this engagement's comments for
diff-narration - 'used to', 'old code', 'is gone', 'moved out of',
'has been removed' - rewritten as what each site must do and what
breaks otherwise, with the measurements and failure modes kept:
the invariant is the point, the archaeology is not. Touches
video_driver, retroarch, runloop, the save/database/updater/backup/
image tasks, the companion latch, the widgets producer, gl1's
swizzle staging and the audit allowlist. Lifecycle wording that
happens to read as history ('the driver itself is gone', 'no
longer valid') describes present teardown states and stands.
Comments only; the build, the fifty-entry audit zero and every
touched file's syntax check all hold.
What display_lock still guarded after the title left it: the
VIDEO_FLAG_* word, and the overlay viewport tuple. Both are atomic
now, and the lock object survives only on the one backend without
pointer atomics, purely as the window-title fallback's lock, gated
with the field itself.
The flags word becomes a retro_atomic int. Every site outside the
funnel was already a snapshot read - those become relaxed loads,
across the context drivers, the frontend and the companion - and
video_driver_modify_disp_flags keeps its combined clear-then-set
indivisible with a CAS loop, falling back to two RMWs only on the
single-core volatile backend where the seam cannot be observed.
This also closes a set of latent plain-access races the lock never
covered: the wgl, x11 and sdl3 context drivers read the word on
the video thread without ever taking it. get_disp_flags is one
acquire load; the threaded branch that took the lock for the same
read is gone, as is the lock-ordering note in gl1, gdi and d3d9hlsl.
The overlay viewport crosses as float bits in atomic ints under a
generation counter: bumped odd before a rewrite, even after, with
readers retrying while it is odd or changed across the copy - and
the read loop shaped so no comparison ever sees an unset counter,
the same -Wmaybe-uninitialized shape task_save just taught.
samples/gfx/overlay_vp_seqlock holds the protocol across 4000
rounds with a deliberately stalled writer mid-tuple: no torn
snapshot (round-encoded fields must agree), progress, clean under
ThreadSanitizer since every shared field is an atomic int; with the
counter sabotaged away the stall hands readers 174 torn fields and
the lane fails as designed.
All five backends compile video_driver.c - VOLATILE keeps the
title lock and the two-step funnel - C89 holds, the 74-lane
matrix, all four budgets, the title mailbox harness and the
fifty-entry audit zero all pass.
The guard on core_status_msg compared against a priority that was
only ever zeroed, so it admitted every message - preserved
deliberately through the lock removal so that change stayed
threading-only, and flagged then as a separate question. Answered:
the set arm now stores the message's priority, on the direct path
and in the deferral drain identically, so a held status keeps its
priority for its lifetime and lower-priority updates - clears
included - bounce until it expires or a same-or-higher write
lands. Every clear path already zeroed the held priority, so an
expired slot accepts anything.
The core_status_defer harness's shared rule gains the same store,
keeping its drain-vs-oracle agreement meaningful, and a
hand-computed lane pins the new semantics step by step: a priority-3
status survives a priority-1 update and a priority-1 clear, falls
to an equal-priority clear, and the emptied slot takes a
priority-0 message. Clean C89_BUILD=1 rebuild, all five backends,
the 74-lane matrix, budgets, pacing and content-closing suites,
and the audit's fifty-entry zero all hold.
Opening the menu on a core that renders through its own audio
callback left the quick menu barely responsive, and disabling audio
outright was the only way to get it back (#19541: Dinothawr, EasyRPG,
ChaiLove, WASM-4 with callback audio on - and every audio driver,
because no driver is involved). Plain pause had the same hole.
The callback loop on the audio thread decides whether the core is
paused from the published runloop snapshot. The snapshot is
republished around retro_run and at driver start and stop - and a
pause is exactly when retro_run stops being called, so the pause is
the one state the snapshot cannot be trusted to carry. Keeping the
device running behind the menu (d428b0af) removed the driver stop
that used to park the audio thread on menu entry, and from then on
the loop read a stale "running" snapshot for the whole visit: it
invoked the core's callback, the delivery gates dropped the rendered
audio against core_silenced, the missing sink progress parked it for
the 1 ms idle wait, and it asked again - the core's audio callback at
the idle-wait rate on a priority-raised thread, contending with the
menu's own silence feed for the state lock.
audio_driver_pause_fade() already stores core_silenced on every
pause, resume and menu transition, on the thread that makes them, and
the delivery gates already trust it over the snapshot. Let the
callback loop treat it as paused too.
Closes#19541.
Reported as libretro/RetroArch#19562: launching content with cheevos
enabled fails to login on recent builds. This adds a permanent CI
guard around the login transport so that once the regression is fixed
it cannot come back silently, and so the conditions a login meets in
the wild are pinned down one by one.
cheevos_login_test drives libretro-common's net_http exactly the way
cheevos_client.c does for login -- POST /dorequest.php, urlencoded
body, user agent set -- against a local mock server that validates
every request byte-for-byte and answers the real login2 JSON shape,
over plain HTTP and over TLS with the real SSL backend linked in
(builtin BearSSL from deps/, and system mbedTLS, the two backends a
Linux build ships with). Scenarios:
- login POST over TLS on a fresh connection (the first request
cheevos makes after content load)
- BearSSL only: the trust store is loaded through the real
append_certs_pem_x509() parser from a realistic bundle fixture --
comment lines, blank lines, a cert with an invalid base64 body
that must be skipped rather than abort the bundle, CRLF line
endings, the valid CA last. The TLS handshake only succeeds if
the CA survived that parse, guarding the trust-store-ends-up-
empty failure mode fixed in 3a253df478
- two logins reusing one pooled keep-alive connection, asserted
against the server's accepted-connection count
- login on a pooled connection the server closed while it sat idle
(200 answered with keep-alive advertised, then closed): must be
replayed once on a fresh connection and succeed (#19457)
- the same idle-close recovery over TLS
- a pooled connection that dies after response bytes arrived must
NOT be replayed: status -1, prompt termination, and the pool must
have shed the dead connection so the next login still works
Runs in the samples/network CI job under ASan+UBSan (both backends)
and TSan (BearSSL), a few seconds per variant. Fixture PKI is a
throwaway CA and localhost leaf generated once by openssl; the mock
server exits with the test via its stdin pipe, so nothing outlives
the job.
As of current master all scenarios pass over both backends, so the
transport layer handles these conditions on x86-64 Linux; the #19562
root cause is not yet reproduced by this harness and remains open.
br_ssl_engine_recvapp_buf() returns NULL with a zero length when the
engine has no decrypted bytes ready, and both receive paths passed
that straight to memcpy(). A zero-length memcpy() from NULL is
undefined behaviour (the pointer parameters are declared nonnull),
flagged by UBSan on every empty poll of a TLS socket. Fold the copy
under the same length guard the ack already uses, in both the
nonblocking and the blocking receive path.
The Linux dev build's blind spot shrinks from thirty-three entries
to twenty-five: wifi and bluetooth task handlers, the libusb HID
adapter and poll threads, and the ffmpeg decode, record and camera
workers all join the audited set once their features are enabled -
fifty entries in the full Linux build, zero findings, no fixes
needed; they were already clean. With the mingw build's fifty-two
that leaves only the entries needing platform toolchains this
machine lacks: Android, emscripten, GX, PSP, sunxi and hub75,
btstack, Steam.
tools/thread_read_audit_configs.sh captures both recipes - the full
Linux configure line with its Debian package list, and the mingw
cross build with its xaudio note - so covering every entry a Linux
box can see is one hand-run command per config, and
--list-unaudited keeps naming what neither binary contains.
The mmdevice-watcher spawn in wasapi.c now captures a pointer to
audio_state_get_ptr()->reinit_request on the main thread - which
put the getter's symbol into a translation unit the
wasapi_exclusive_pacing harness compiles as its sim, and that
harness's stub set never needed it before. The stub returns a
static audio_driver_state_t, same shape as its config_get_ptr
neighbor; the scenarios never spawn the watcher, the linker just
wants the symbol. discrete_multichannel, the other consumer of
wasapi.c-as-sim, already carried the stub and still builds and
passes.
The first run of tools/thread_read_audit.py against a Windows
binary - a mingw cross build, which host objdump reads natively -
found 52 worker entries and 27 findings where the Linux build shows
42 and zero. Twenty-five shared one reader: on Windows RARCH_LOG
routes through the desktop companion, and
ui_companion_driver_log_active read desktop_menu_enable through
config_get_ptr on whatever thread logged - every worker that logs a
line was a worker-thread singleton read, invisible on Linux where
logging never touches the companion.
desktop_menu_enable is now latched into the companion state as an
atomic, stored on the main thread at init_first and toggle - both
already receive the live value from their callers' settings read -
and the log sink, the message push and the refresh notification
read the latch with relaxed loads. Staleness across a toggle costs
at most one log line. The manual scan's completion notify moves
from the handler to the task callback, at retrieval on the main
thread, alongside the read it carried. The WASAPI pump captures its
two MMCSS bools into the driver at init and takes them as
parameters. And the mmdevice watcher thread, which raised exactly
one flag through audio_state_get_ptr, is handed a pointer to that
flag at spawn instead - its previously unused thread argument.
Windows: 52 entries, 0 findings. Linux: 42 entries, 0 findings,
unchanged budgets, the 74-lane matrix, C89 on the shared files and
the database suite all pass. The 33-entry --list-unaudited blind
spot is now ten entries smaller in the one build that can see
them.
RUNLOOP_MSG_QUEUE_LOCK is gone. The queue, core_status_msg and
their decay were already main-thread-only in every path but one:
off-main runloop_msg_queue_push callers ride the mpsc deferral, the
task system's progress pushes run at gather and retirement on the
main thread, and video_driver_frame's pulls, extraction and status
decay run there too. The one remaining producer that could arrive
on another thread was a rogue core calling SET_MESSAGE_EXT(STATUS)
off retro_run's thread - the libretro API forbids it, cores do it
anyway - and that case now rides the same deferral: the update is
captured whole into a node flagged core_status, and the drain at
the top of the next iterate applies it under the direct path's own
overwrite rule, mirrored exactly - including that path's quirk of
never storing a priority, kept deliberately so this change is
threading only. With every toucher on the main thread by
construction, the lock serialized nothing, and the seven lock
sites, the field and the lifecycle go; the macro block in runloop.h
becomes the invariant statement, and the stale claims in the win32
companion, gfx_widgets.h and the widgets race sample - all written
against the world where any thread pushed under this lock - are
brought up to date. The widget ring's own msg_queue_lock is
untouched: it never depended on the runloop's.
The deferral struct and its off-main test move above the
environment callback, their new first user - defined before use,
the lesson the mailbox gate taught positionally last week.
samples/runloop/core_status_defer reproduces the STATUS deferral
and holds three claims across 4000 posted updates from a worker,
scripted clears included: applied exactly once, whole and in
order; the drained final status agrees byte for byte with an
oracle running the direct path's rule over the same script; every
node freed exactly once. Clean under ASan/UBSan and Clang
ThreadSanitizer. A clean C89_BUILD=1 rebuild passes, all five
atomic backends compile runloop.c and the gfx lane, threads-off
builds hold, the 74-lane matrix and all four hot-function budgets
pass, the runloop pacing and content-closing suites pass, and the
thread-read audit stays at zero.
Reported by LibretroAdmin against 0cb8a462cf: video_driver_lock_new
sits far above the title protocol block, so its #ifdef
VIDEO_TITLE_MAILBOX evaluated before the macro was defined - and an
#ifdef on a not-yet-defined macro is silently false, so the drain
that empties the mailbox on driver re-init was compiled out of the
build without a single diagnostic. The gate now moves to the top of
the file, before every user, with a comment saying why it cannot
live with the rest of the title machinery; the protocol comment and
the fallback macros stay where they were. Verified positionally in
the preprocessor output: the drain's exchange is present in
lock_new on pointer-atomic backends and correctly absent under the
VOLATILE backend, all five backends compile, C89 holds, the
mailbox harness passes, and the hot-function budgets are
unchanged.
GCC's -Wmaybe-uninitialized was right that it could not prove
size.total_size initialized at the assignment after the serialize
block: when the sizing call returns zero the struct is never
filled, and only the state->data guard - an implication outside
the compiler's reach - kept the read on the safe arm. The size now
starts at zero and takes total_size inside the one branch where
the sizing call has necessarily succeeded, so the proof is local
and the warning is gone by construction rather than by a dummy
initializer that would also mask a real future miss. Behavior is
identical on every arm; the save suite, including the
background-serialize timing lane, passes.
The title handoff was the last thing the video thread took
display_lock for on its frame path: the main thread assembles the
title once per fps interval and every driver's frame() offers to
apply it, with the 512-byte string uncopyable in one atomic step.
The lock's job was only to prevent a torn copy - so the copy stops
being shared at all. Where the atomics have pointer ops the title
crosses as an immutable heap copy through a one-slot mailbox:
publish is exchange-in, freeing whatever title was never taken, and
take is exchange-out. A copy either is not published or is
complete, because nothing writes it after publication; both sides
are wait-free; a slow reader costs a superseded title, never a
leak; and the read-side fast path stays one acquire load. The win
is not the lock's cycles - it was uncontended - but the lock-order
edge it removes: display_lock also guards the disp flags and the
viewport rectangle, and the context drivers already document
context_lock + display_lock ordering on the video thread, so the
title no longer participates in that graph at all.
On the one backend without pointer atomics, and in builds without
threads, the old protocol stands unchanged behind the same macros,
per retro_atomic.h's documented gate - the mpsc deferral's
precedent. samples/gfx/window_title_mailbox reproduces the protocol
and holds three claims across 20000 paced publishes: no torn copy
(embedded checksums), supersession only forward with the final
title always taken, and every copy freed exactly once under ASan.
Clean under Clang ThreadSanitizer; a write-after-publish sabotage
lights up all three checks. All five atomic backends compile the
new video_driver.c, VOLATILE on the lock path; C89 holds; the
74-lane matrix and all four hot-function budgets pass.
Zero findings is a statement about one binary. Thirty-three of the
seventy-five worker entry points in the tree are compiled out of a
Linux dev build - Windows joypad enumeration, HID adapter threads,
console audio loops, the companion UI workers - and no run here
says anything about them. The flag prints them with their source
files, so a port maintainer knows exactly which entries need a run
against their own build before claiming the same zero. The
selftest grows a ghost entry: declared and spawned in source,
defined nowhere in the fixture binary, discovered but not audited.
The last eight findings, and with them the campaign: the audit that
opened at thirty-nine findings closes at none, with two allowlisted
pairs carrying their justification in the list and one reviewed
boundary. Every task handler that ran settings or override reads on
the threaded task queue's worker now reads a capture its push made
on the main thread.
The per-entry thumbnail task captures the five thumbnail settings
into its handle and resolves paths through a _cfg variant of
gfx_thumbnail_update_path that reads nothing live; the enable check
splits the same way, into a values-parameterized core the live
wrapper feeds. The manual scan's task config carries the CLI-scan
flag and the serial/CRC toggle from the moment the config is built.
The core updater list task captures its three paths at push - and
because the update-installed-cores handler pushes that task from
the worker, the push itself splits into a captured-only core and a
live wrapper, with the worker passing captures made at its own
push. The core updater and core backup handlers stop clearing and
setting menu flags from the worker - a read-modify-write racing
every writer on the main thread - and move that work into task
callbacks, which run at retrieval on the main thread. The disc dump
task captures the core assets directory. The load handler's single
frame-count poll is allowlisted with its justification: a benign
monotonic read, documented at the site, holding the load until the
core has rendered.
Audit: 42 entries, 0 findings. All five atomic backends and C89
hold on every touched file; the task suites, the 74-lane matrix,
and the hot-function budgets pass.
Two reds on the libretro-common samples lane. The first is
bookkeeping: mpsc_stack_test built but was never classified, so it
joins RUN_TARGETS, and it gets the same dedicated Clang+TSan step
the other lock-free queue carries - the stack's commit claimed TSan
cleanliness and the generic ASan pass was never going to hold that.
The second looked like the barrier failing and is the brand-new
harness failing itself. The park counter mixed a blind store at
round entry with fetch_add claim arithmetic: the producer's
stale-claim un-claim is a -1/+1 transient, and a store of 1 landing
between the halves leaves the counter at 2 - a value from which
fetch_add(-1) never returns 1, so no claim can succeed and no post
can come while the producer publishes millions of sequence steps.
That is exactly the CI signature: seq jumping from thousands to
millions across each two-second timeout, three sporadic hits in
five thousand rounds, each self-healing on the next round's store.
Round entry is now the same arithmetic as everything else touching
the counter; an exhaustive interleaving check of the entry racing
the un-claim shows the old protocol reaching the postless 2 and the
new one only ever reaching states a pending post squares. All tiers
reachable here pass 5000/5000; the barrier itself was never wrong.
Two lanes the deferred-message push broke, both mine to have run.
The C89 lane: three functions grew statements ahead of declarations
- the off-main check now sits after the declaration block and the
settings hoists it must precede became assignments, the iterate's
drain moved below its declarations, and the deinit's cleanup sits
under the function's own declaration. make C89_BUILD=1 links from
clean.
The atomic-backend lane: mpsc_stack.h used the pointer atomics
unconditionally, and the volatile fallback backend omits them by
design - the gate is RETRO_ATOMIC_HAS_PTR, as retro_atomic.h's own
contract says. Without it the stack keeps its shape with plain
volatile operations and the guarantees that backend itself makes:
none beyond the single-context targets it exists for. All five
forced backends build the gfx sources and the task queue; the mpsc
lane and its TSan run stay green.
A background save's handler ran the whole rastate write on the
threaded task queue's worker: the core's serialize - which is what
the core's SET_SAVE_STATE_IN_BACKGROUND request vouches for - but
also the replay block, the rewind direction and the achievement
block, live frontend state read while the main thread mutates it.
The push captures the frontend blocks on the main thread, and the
worker's sizing and writing run on a captured-only path with no
live frontend read in it at all: the core serializes live, the
capture fills in around it. The foreground path is untouched.
The save suite pins the timing from the feature's side: a
background push serializes nothing and the handler serializes the
core exactly once, a foreground push pays at push - so the deferral
the env request exists for cannot quietly become a push-time
serialize. tools/thread_read_audit.py --binary retroarch: 11
findings become 8, and task_save_handler no longer appears.
The deferred-message list took the message queue lock to push and,
worse, probed its head with a plain read every iterate - a data race
against the workers' locked writes, benign in effect and illegal in
form. The list is now queues/mpsc_stack.h, a many-producer
single-consumer intrusive stack in libretro-common: a worker's push
is a compare-exchange loop that never touches the message queue
lock, the iterate's emptiness probe is an acquire load, and the
drain takes the whole chain in one exchange. No ABA arises in this
shape - nodes are only pushed and the consumer takes everything -
and message frequency never made the lock slow; what the stack buys
is a legal probe and workers that cannot contend with the main
thread's own message work.
The primitive carries its own lane, mpsc_stack_test in the
libretro-common samples: four producers against a draining consumer,
every push drained exactly once with per-producer order and payload
held, clean under TSan and ASan - and a push weakened to a blind
store fails it under TSan on both the race report and the count. The
samples binary ignore no longer swallows new *_test directories, so
the lane is trackable.
runloop_msg_queue_push() renders widgets, measures text, speaks the
accessibility line and reads the settings - the main thread's work,
and six of the threaded task handlers were doing it on the worker.
An off-main caller now queues the message, whole, under the message
queue lock, and the iterate's drain replays it on the main thread a
frame late at most - the cadence messages show at anyway. The
threaded task worker also stops pushing progress after a handler:
the gather pushes for every running task each check and retirement
pushes the final state, both on the main thread, so the worker-side
push was a duplicate doing widget work from the wrong thread through
the queue's message callback - a function pointer the audit cannot
see, which is worth remembering.
The audit learns what this fix is: a boundary - a function that
defers off-main callers before touching anything - declared as
@boundary in tools/thread_read_allow.list and held by review, where
the walk stops. runloop_msg_queue_push is the first.
tools/thread_read_audit.py --binary retroarch: 23 findings become
11, across every handler that showed messages; the remaining 11 are
direct reads. All twenty-one task sample suites pass.
The retry task's handler runs on the threaded task queue's worker,
and it rebuilt the badge path from the config and the menu theme on
every poll, then started the widget popup - text measurement and
widget state - from that worker. The badge path is captured into the
retry's own state when the task is armed, so the wait is a
path_is_valid() on a string the task owns; the popup moves to the
task's callback, which the queue runs on the main thread at
retrieval, where the achievement lookup and the widget belong. A
game unloaded during the wait is still detected - by the callback,
where reading the client races nothing.
tools/thread_read_audit.py --binary retroarch: 30 findings become
23, and rcheevos_retry_achievement_popup no longer appears.
The handler runs on the threaded task queue's worker, and it read
the sync toggles, the destructive flag, the core-assets directory,
the save and state directories and the config path from the live
settings and runloop mid-sync - torn reads if the menu writes a path
while a sync runs. The push captures all of it into the task's state
on the main thread: the destructive flag and the core-assets
directory as fields, the directory map as a list the task owns.
The per-task list also retires the process-wide directory-map cache
and its staleness handling: a fresh list per sync is a handful of
strdups, and it cannot disagree with the toggles the sync was pushed
under.
tools/thread_read_audit.py --binary retroarch: 39 findings become
30, and task_cloud_sync_task_handler no longer appears.
video_st->data and current_video have independent lifetimes. driver_uninit()
releases the handle and leaves the vtable installed, and VIDEO_FLAG_ACTIVE is
raised before any video driver exists: retroarch_main_init() sets it, loads the
core through CMD_EVENT_CORE_INIT, and only reaches drivers_init() after that. A
frame produced anywhere inside that window arrives at the driver call with a
live vtable and no handle, and every GL, Vulkan, D3D and Metal frame()
dereferences its handle on entry, so it faults instead of declining the frame.
video_driver_frame() now tests the handle alongside the vtable and drops such a
frame, leaving VIDEO_FLAG_ACTIVE as it is: no driver is a gap rather than a
failing driver, and the first frame after drivers_init() presents normally.
The deferred shader load pump takes the same pairing for the same reason. It
steps through current_video->shader_load_step(video_st->data, ...), so a
compile whose driver has gone ends there instead of stepping into it.
* wayland: wait for the fullscreen configure before the first frame
Toggling fullscreen recreates the window, and RetroArch drew its first
frame before the compositor had confirmed the fullscreen request, so
the window was first mapped as a normal 240x256 window. Tiling window
managers such as Krohnkite on KWin place it in that state, and the
window then goes fullscreen at that placement, offset from the monitor
it should cover. On a scaled output the fullscreen swapchain was also
sized from the placeholder.
Dispatch events until a configure reports the fullscreen state, for at
most 500 ms, before continuing.
* wayland: harden the fullscreen-configure wait
Examine poll revents and bail out on POLLERR/POLLHUP/POLLNVAL so a
dead display fd cannot spin the wait for the rest of the timeout
budget, and log when the wait ends without the compositor reporting
fullscreen or when libdecor fails to dispatch, matching the logging
of the window-creation loop.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The samples lane killed retro_procbarrier_test at its sixty-second limit
with no output at all -- not even the first line. That was the test's
own design hiding its own finding, and it is the first run on real SMP
hardware, so the finding matters.
Four things about the test made a failure invisible. Stdout was
block-buffered, so a run killed by the harness lost everything it had
printed. The fence loop counted lost wakes but never stopped, and each
lost wake is a two-second condvar timeout, so on a machine where the
barrier did not fence the loop needed hours and was killed at a minute
instead. The withdraw path's wait for the producer's post had no timeout
at all. And the producer was unpaced: on its own core it moved the
sequence every few nanoseconds, the consumer never found it unchanged,
and the park -- the path under test -- was never taken; the withdraw path
got all five thousand rounds.
Now: stdout is unbuffered; the producer spins briefly between publishes
so parks happen on real cores; every wait is bounded; and the loop stops
after three lost wakes, prints so, and fails. Injecting failures every
seventh round shows three lines, "stopping after 3 lost wakes", FAILED,
in under a second. Worst case on the runner is three timeouts, six
seconds, output intact.
One change to the primitive, prompted by the same run. The membarrier
probe accepted registration succeeding as proof the tier works, and a
sandbox can allow the query and the registration and still refuse the
expedited call -- which would give a tier that fences nothing while
reporting itself as membarrier. The probe now issues one expedited call
and believes its return, so a refused barrier falls through to the page
flip rather than being trusted.
What this commit does not do is explain the runner's result. Either the
barrier did not fence there, or something in the harness stalled;
on one core neither can be told apart and the yield-injected runs here
pass every tier. The next run on the runner will say which, in its own
output, in under ten seconds.
The PowerPC cross lane rejected the binary: pthread_from_mach_thread_np
is not in the 10.4u SDK. It arrived in 10.6, and RetroArch's PowerPC
floor is 10.4.
The function is a walk of libpthread's private thread list under its
private lock, matching a Mach port against each thread's cached
mach_thread_self slot -- Apple's source says exactly that and nothing
more -- so there is no reimplementing it against an older libpthread
that would not be a guess at a struct layout. The Darwin tier needed it
only to turn a Mach port into a pthread_t for pthread_kill, so the fix is
to not need a pthread_t.
thread_suspend on the port is already a synchronous barrier. XNU's
thread_wait loops while the target's state has TH_RUN, blocking while it
is the active thread on any processor, so thread_suspend does not return
until the target has been switched off its CPU -- and a context switch
drains its store buffer. That holds in the current kernel and in
xnu-1228, the last PowerPC one, where the loop is
while (thread->state & TH_RUN) ...
if (processor->active_thread == thread) assert_wait(...)
The Darwin tier now suspends and resumes each running thread by port:
no pthread_t, no signal handler, no signal number to take from the
application. Every call it makes -- task_threads, thread_info,
thread_suspend, thread_resume, mach_thread_self, mach_task_self,
mach_port_deallocate, vm_deallocate -- is in 10.4u's libSystem, checked
against the SDK's export strings. Nothing is called between the suspend
and the resume: a suspended thread may hold any lock, malloc's included,
and that window has to be one in which this thread needs none of them.
Makefile.test gains darwin-floor-check: it strips each source's comments,
pulls every function called in its Darwin-only paths, and checks each
against the 10.4u SDK's libSystem exports, fetched once and cached. The
PPC lane audits the whole binary and is the authority; this answers the
one question -- is this symbol in 10.4 -- in seconds on the Linux job
rather than at the end of a cross-build. Run against the version the
lane rejected it names pthread_from_mach_thread_np and nothing else;
against this it passes.
The suspend/resume loop was driven against a stub Mach on Linux -- four
threads, one of them self and one waiting -- and suspended and resumed
exactly the two running ones. That is the control flow executed, not a
Darwin build, which the PPC lane still provides. Linux and both Windows
targets are unchanged and warning-free, and all four Linux tiers pass.
The thing retro_procbarrier was built for. Same shape and contract as
retro_eventcount -- notify on one side; prepare, cancel and commit on the
other; the epoch as the handshake -- with the ordering paid for on the
cold side instead of the hot one.
retro_eventcount's notify is a sequentially-consistent read-modify-write
and a sequentially-consistent load, a locked instruction on every publish,
so that a notify and a registering waiter can never each miss the other.
Here notify is a release store and a relaxed load: three ordinary moves,
no lock prefix, no fence. prepare_wait registers and then calls
retro_procbarrier(), which makes every other running thread pass through
a full fence, so either the producer's epoch store executed before the
barrier and the waiter's re-check sees it, or it executed after and the
producer's load finds the registration and wakes. Measured on the
producer with the consumer awake: 2.6 ns against 6.2 ns per publish.
Two conditions, both in the header and both the caller's. It is single
producer: a plain store cannot serve concurrent notifiers. And the
barrier is a syscall at best and a signal per running thread at worst,
so this primitive calls it in prepare_wait and nowhere else; a consumer
that parks at a high rate can spend more in barriers than the producer
saved, and should spin first.
Where retro_procbarrier reports NONE -- the multi-core consoles -- the
object runs the symmetric pair for its whole life, decided once at init,
so a caller never has to know which mode it is in. The park is a condvar
on every backend: cold side, portable, and the correctness argument is
short, since commit_wait re-checks under the lock that notify broadcasts
under. A futex park can replace it once this has been on real SMP.
The test runs the standard consumer loop against a producer publishing
every few hundred cycles, with the consumer genuinely parking each round,
in both modes: the natural resolution and then the symmetric fallback
forced on the same machine. A lost wake is a timeout, not a hang, and
the check bites: making notify never broadcast is caught in both modes.
Its check target forces each barrier tier in turn as the procbarrier
test's does. All of it passes here under ASan, UBSan and TSan -- TSan
because the epoch goes through retro_atomic and is not a data race by
its definition, not because it can see the barrier. On one core the
lost-wake race cannot occur and the test prints the CPU count to say so.
Wired into griffin, Makefile.common, Makefile.test's threads-samples and
console-libc-check, and the samples workflow's RUN_TARGETS -- all four
places the last three breaks taught me to check -- and it compiles clean
under every console shape the lane knows, and C89 -pedantic on Linux and
both Windows targets.
The state singletons belong to the main thread; a worker takes what
it needs through published snapshots, latched fields or arguments
captured when the work was posted. The audit walks the shipping
binary's call graph from every worker entry the source declares -
thread functions, task handlers, the pipeline callback - and reports
each reachable singleton reader, so the doctrine the audio migration
established is checked by a tool rather than remembered. Owner-domain
reads are the sanctioned exception, recorded in
tools/thread_read_allow.list: the video thread reads ra-video's own
pointer-stable state.
Run by hand against the fullest build available:
python3 tools/thread_read_audit.py --binary retroarch
The Linux build stands at 42 worker entries and 39 findings: the
cloud sync, manual scan, core updater, screenshot, save, load, core
backup, cdrom dump and thumbnail download handlers on the threaded
task queue, and the achievement popup's retry thread. Each is a
capture-at-push conversion; the count is the backlog.
The Orbis lane failed to compile retro_procbarrier.c: 'sys/membarrier.h'
file not found. The PS4 toolchain defines __FreeBSD__ -- the PS4 is a
FreeBSD fork -- so the FreeBSD branch fired and reached for the header
that came with membarrier(2) in FreeBSD 14.1. Sony's kernel is a 9-era
fork with none of that and no process-barrier API of its own.
Two things were wrong and both are fixed. The FreeBSD branch now excludes
__ORBIS__, so the PS4 is what it is: a multi-core console with nothing,
resolving to NONE like the rest. And the FreeBSD branch itself now checks
__FreeBSD_version for 1401000 before including the header, so a real
FreeBSD older than 14.1 is not broken either -- it just lacks the
membarrier tier and keeps the x86 page flip.
The console-libc-check lane had an ORBIS entry and did not catch this,
because it set ORBIS alone while the real toolchain sets __FreeBSD__ too.
It cannot simply add -D__FreeBSD__: glibc's own headers then look for
FreeBSD's sys/_types.h. So the lane now pre-includes every libc header
the rthreads sources use and defines __FreeBSD__ after them. The include
guards make the sources' own includes no-ops, and only a header not in
that list -- one the console lacks -- still fails. Run against the code
that broke the PS4 it reports "sys/membarrier.h: No such file or
directory" at line 95, which is the CI failure verbatim; run against this
it passes on all nine platforms.
The samples workflow also refused the test: every built sample must be
named in RUN_TARGETS or NORUN_TARGETS, and retro_procbarrier_test was in
neither. It is in RUN_TARGETS now, running its natural tier; the forced
tiers run from Makefile.test. On the runner's cores the fence check is
load-bearing, which the machine this was written on cannot make it.
C89 on Linux and both Windows targets are warning-free and the four-tier
check passes.
The PSL1GHT lane reported the same sysconf link failure PSP did, from the
same commit. Master was already past it: 311af4c's fix is not
PSP-specific, and the procbarrier compiles clean under -D__PS3__
-D__PSL1GHT__ with sysconf, mmap and sigaction forbidden, referencing
none of them. The CI log's line numbers are fde3566's, from before the
fix landed.
What that report did expose is that the console-libc-check lane did not
list PS3. It does now, setting __PS3__ and __PSL1GHT__ together as
Makefile.psl1ght does; run against fde3566's version it fails on the PS3
entry like the other eight.
The same log carried a warning from rm3u.c, where ppu-gcc says base_dir
may be used uninitialised. The code is correct -- both arms of the
branch write it -- but path_basedir is inlined into rm3u_dump and reads
base_dir's first two bytes, while strlcpy, which fills them on the taken
arm, is not inlined; an older GCC cannot see the write and warns on
exactly those two offsets. They are now written before the branch, where
any analyser can see it, and the strlcpy overwrites them on the path that
runs. The branch itself stays: an M3U with no directory component gets
an empty base so entries are used as written, and path_basedir would
give "./" instead, so it is not a redundant guard around a call.
GCC 13 does not raise the warning, so the fix cannot be shown to silence
it here; it targets the two offsets the diagnostic names and is correct
by construction. The m3u sample test passes unchanged.
The four -Wextra unused-parameter warnings in the same translation unit
are in file_path.c's stub functions, predate this, and are not in the CI
log; they are left alone.
* audio: average the 1.0x flush interval in the fast-forward estimate
The speed estimate divided a sixteen-flush average of the flush interval
by the interval this flush's audio takes at 1.0x. Cores whose batch size
varies from flush to flush - mupen64plus_next alternates 185 and 551
frames - then get a multiplier that swings with the batch size, and the
pitch-preserving stretch runs some 6% short of real time, opening gaps in
every long fast-forward hold.
Average the expected interval the same way, so the ratio compares like
with like. The integer average of the measured interval is kept as it is;
the threaded producer's harness depends on its rounding.
* audio: seed the fast-forward speed estimate at the configured ratio
The first flush of a hold seeded the estimate at 1.0x, and the average
took some fifty flushes to reach the real speed. The frame limiter has the
core at the configured ratio within a frame, so for that long the audio
was compressed too little and ran up to three times real time into a
device the blocking 1x writes had left full. With the non-blocking driver
fast-forward uses, 100-250 ms was dropped at every press, and a 150 ms
hold lost more audio than plain discard does.
Seed at the fast-forward ratio and return its multiplier at once; the
measured speed takes over from there. An uncapped ratio still starts at
1.0x. Measured with 150 ms presses at 3x: 4.0 s dropped in 12.8 s of
fast-forward became 0.06 s on OoT, 5.7 s became 0.05 s on FF7.
* audio: let pitch preservation follow fast-forward speed on its own
Pitch-Preserving Speed Changes only measured the fast-forward speed when
Fast-Forward Audio Speedup was on; with it off the stretch stayed at unity
and the flush fell back to discarding samples, so the setting appeared to
do nothing for fast-forward.
Follow the measured speed when Speedup asks for it or when pitch
preservation owns the transport, on both pipelines. The filter-only
transport keeps the Speedup-or-discard behaviour, since it has no way to
hold pitch.
* audio: widen the WSOLA search to four hops
The similarity search reached half a hop either side of the nominal
position: 64 samples at 48 kHz. A splice can only land on matching phase
if the search reaches a full period, so nothing below about 375 Hz was
being preserved. At 3x a 55 Hz tone came out at 165 Hz with 98% amplitude
modulation, 110 Hz at 143 Hz, 220 Hz at 285 Hz; 440 Hz and up were clean.
Bass lines and low mids in a game's music were transposed and wobbling
while the rest kept its pitch.
Four hops (512 samples at 48 kHz) cover fundamentals down to about 47 Hz
and keep every test tone within 6% of steady amplitude. The search costs
four times as much correlation, under 1% of a desktop core at 3x, and the
ring and its retained lookahead grow with the radius: the harness bounds,
drain scenarios and README figures follow.
* audio: keep the inline stretch active at unity once it has started
The inline transport returned to raw copying whenever the speed came
back to 1.0x. Leaving the engine drains the source lookahead its search
kept, up to twice the radius, as one flush onto a device the hold left
full, and with the VRR runloop's non-blocking frames after a release most
of it is dropped: 2-4 ms per release at the old radius, 10-40 ms at four
hops. The next engage then pauses output while the lookahead fills
again. Leaving the lookahead behind instead shorts the device by as much
and rapid presses run it dry.
Once the engine has started, keep it active at unity. Unity synthesis
lands every splice on its own continuation, or a period of it in a
stationary passage, so the audio passes through all but unchanged; the
cost is the lookahead as holdback, 16 ms at 48 kHz, from the first speed
change until the transport is reset. Measured with 150 ms presses at 3x:
0.5 s dropped per 40 presses became 7 ms.
* audio: end the stream on a ramp at a pause, and bring it back on one
Opening the menu pauses the core, and the audio stream ended wherever
the waveform happened to be: a step at both ends, heard as a click. And
between the last write of the pause and the first of resumed core audio
the runloop hands nothing over, ~33 ms with the menu, so the device
drains through the hole and rate control refills it at half a percent
of real time: PipeWire's pulls came up short for seconds after every
close, 10-48 ms of fabricated silence after a single close and 50-138 ms
per close when tapping the menu key, which is heard as a crackle.
Every frame has already gone to the device by the time the pause is
known, so a plain ramp has only the last sample to decay. Keep a copy of
the last frames of output instead, find the period that best matches the
last few milliseconds, and repeat it under a raised cosine, as a packet-
loss concealer does. The period is scored by distance, since correlation
is blind to level. The tail is written straight to the device under the
state lock, ahead of what the resampler and the stretch still hold:
pause_mute_frames drops the resampler's leftovers, and the inline
stretch is reset so its lookahead does not come out behind the tail.
The ramp back up is owed to the core's first audio after the resume,
not to the menu's silence in between, so every publish says whether it
is the core's own and the ramp is armed on the first that is. The same
flush first fills the device with silence up to write_avail(): silence
is free, the device has been playing it, and the surplus is what rate
control already takes out. It has to be the first flush rather than the
resume itself, since a top-up at the resume is undone by a pull landing
inside the hole.
A core with its own audio thread kept handing frames over after the
tail, since menu_pause_libretro never sets RUNLOOP_FLAG_PAUSED.
audio_driver_pause_fade() publishes core_silenced, which gates the
core's input paths and only those.
Under the threaded pipeline the ring behind the tail still held frames
of the core's audio, and the stretch stage its lookahead. The pause
records the ring's head, and the consumer takes the ring out up to it
and drops what the stage holds; the resume marks where the core's audio
starts, and the consumer never takes a chunk across that point and arms
the ramp on reaching it.
The ramp hangs off retroarch_menu_running() and its finished
counterpart, which the menu hotkey reaches where CMD_EVENT_MENU_TOGGLE
does not, and follows what the runloop will do rather than the setting
alone: under netplay the core keeps running behind the menu. The pause
hotkey, the setting toggled from inside the menu, an audio reinit while
held, the paused rewind step, the BSV checkpoint seek and a netplay
session starting or ending behind the menu go through the same call.
write_raw drivers get the resume ramp and the mute but no tail: they
hand the driver the core's buffer at the input rate, so there is nothing
at the output rate to keep as history.
Measured headless against PipeWire, with 8 toggles of 1.5 s and 12 of
0.3 s on mupen64plus_next and swanstation: the RMS of the 3 ms at each
edge against the 15-25 ms beside it went from 1.0 (a hard cut) to 0.1
(the ramp) at both edges, and the silence fabricated across a run from
64-285 ms to none, on the inline and the threaded pipeline.
* audio: bracket the frontend's state jumps with the pause fade
Loading a state splices the game's own audio, and the frontend does the
splicing on top of a device that may already have run dry: the undo
backup and the core's unserialize both run on the main thread inside the
load task's callback, for a main-loop gap that can exceed the device's
buffer on a large state.
Use the pause tail and the resume ramp the menu already uses, so the
splice and whatever gap the stall leaves both land in silence. The other
three ways the frontend throws the game's audio away mid-waveform get
the same bracket: a core reset re-initialises the machine, an undo pays
the same two serialisations as the load it is undoing, and a load from
the RAM slot is a load. All run on the thread that would otherwise be
feeding the device.
audio_driver_jump_fade_begin/_end name the bracket rather than repeating
its invariant at four call sites - the stream may already be down,
because the jump was made from the menu or while paused, and then the
ramp back up has to stay with whatever took it down.
core_reset() is bracketed at the CMD_EVENT_RESET site rather than inside
it, so netplay's own reset path is untouched.
* runloop: publish the audio snapshot before the frame runs
The snapshot the audio producers read is refreshed at frame end, after
retro_run(), while the runloop flips fast-forward, slow motion and pause
before it. The first frame after every toggle flushed on the previous
state: at a fast-forward press, a full 1x block into a device already
non-blocking and full, 7 ms refused per press.
* audio: wait for device room on the pitch-preserving path
Fast-forward writes the device without blocking, and a write the ring
cannot take is cut short. The discard path means to lose the rest; the
pitch-preserving stream is continuous, so the cut is a gap. At 1x the
blocking writes leave the ring at the brim, and the first stretch output
after a press lands before the device has pulled again: 3-9 ms refused at
every press that follows a second or more at 1x, heard as a click.
Complete a short write with one blocking write for the remainder, on the
inline pitch-preserving path only; the wait is bounded by the remainder.
Measured on the desktop over nine presses: 0 ms lost, 84 ms recovered.
* audio: the fast-forward seed reads the published ratio
The seed of the speed estimate read the fastmotion override and the
fast-forward ratio setting inside the flush - the submit path, which
is the audio thread when the core owns its audio callback, and the
override is rewritten by the environment call at exactly the moments
the seed fires. The publisher resolves the override against the
setting where both belong to the main thread and publishes the result
as a float bit pattern beside the slowmotion ratio; the seed reads
that, and the ratio-seed fixtures publish after setting the ratio,
as fixtures do for everything the audio side consults.
* samples: the pause fade holds its promises under test
The inline half in ff_speedup: the pause tail opens where the stream
stopped and only the join correction can carry that level - the
fixture's history steps 0.3 above its cycles at the end, the way real
content moves - the envelope only decays and reaches silence, keeps
the waveform's pitch, and fits the room a full non-blocking device
has left; the mute drops the resampler's leftovers; the resume ramp
waits for the core's first audio, opens from silence, stands halfway
at its middle and hands over at full level; the jump bracket leaves a
stream someone else took down alone. The test object now depends on
the audio_driver.c it includes: without that an edited driver leaves
a stale binary that silently keeps passing.
The threaded half in pipeline_wakeups, as check-pause: through the
wrapper's parked control transactions the pause lands with the ring
provably holding unplayed source, and the device tap shows none of it
is ever played while the resumed stream reaches the device under the
ramp, opening below 2% of its level and climbing. Sequenced against
the real consumer thread and clean under TSan; the load sweep's
conversion-volume assertion stays with the load sweep.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Three things the first two commits left undone.
The resolution had two states where it needed three. NONE meant both
"not initialised" and "initialised and this platform has nothing", so
retro_procbarrier() on an unsupported system re-ran the whole probe --
sigaction, mmap, a /proc walk -- on every call, because it could not tell
a failed resolution from a fresh one. Measured by forcing every tier to
fail: 1000 calls performed 1000 sigaction and 1000 mmap. There is now a
private RESOLVED_NONE the public API maps back to NONE, and the same
1000 calls perform one of each. Callers should still resolve during
setup and pick a protocol for the life of the object rather than
alternating, but the primitive no longer punishes the lazy path.
Makefile.common did not list retro_procbarrier.o. griffin builds it by
inclusion, so griffin-based targets linked; a non-griffin make would have
failed the moment anything called it. It sits beside retro_eventcount.o
under HAVE_THREADS now.
The test was not in Makefile.test. It gains the same SANITIZER plumbing
and check target the eventcount test has, but its check loops over the
tiers rather than the backends: it runs the natural resolution, then
forces membarrier, the page flip and the signal tier in turn, so every
tier a platform has runs on one machine. Makefile.test runs it under the
concurrency sanitizers beside the eventcount and waitable-spsc checks.
On this box all four runs pass under ASan, UBSan and TSan.
TSan being clean is worth reading precisely: it sees the test's own
release/acquire atomics and finds them consistent. The barrier itself is
a syscall it cannot reason about, so TSan proves the test's handshake is
well-formed, not that the barrier fences. The fence check in the test is
what does that, and only on more than one core.
Also removed a VITA_UP from the single-core console list: no such macro
exists and Vita is four Cortex-A9s. It belongs in the SMP default, which
resolves to NONE there, as intended.
fde3566 broke the PSP build at link: retro_procbarrier_init and
retro_procbarrier both reference sysconf, and PSP's newlib has none.
The cause is a gate on the wrong thing. pb_num_cpus keyed the sysconf path
on _SC_NPROCESSORS_ONLN being defined, and newlib defines the constant
without providing the function -- so the branch compiled on PSP, and only
the link, thirty minutes into that lane, said no. The count now comes
from sysconf only on the platforms this file otherwise handles (Linux,
FreeBSD, Darwin), from GetSystemInfo on Windows, and is 1 by name on the
single-core consoles -- PSP, PS2, GameCube/Wii, DOS -- so the barrier is
correctly free there rather than falling to NONE. Anything else is
assumed SMP, which resolves to NONE, the safe direction.
The same shape of mistake was one line away in the page flip: it was
gated on x86 alone, and an x86 console would have reached for mmap and
mprotect the same way. It is now gated on x86 and one of the OSes that
has them, and its storage with it.
Both are now checked before a console lane runs. Makefile.test gains
console-libc-check: it compiles each rthreads source under each console's
platform define with the libc symbols those ports lack renamed so they
cannot link, mimics newlib's trap by defining _SC_NPROCESSORS_ONLN without
sysconf, and asks nm whether any renamed symbol is still referenced.
Seconds, on the Linux job. Run against fde3566's version it fails on all
eight platforms; against this one it passes. PS2 is left out because its
define pulls in <kernel.h> from the PS2 SDK, which is not on a Linux
runner, so its lane remains its only check.
The host's architecture macros are left defined in that lane -- glibc's
own headers break without them -- which is why the page flip has to be
gated on the OS it needs and never on x86 alone. That constraint is now
in the Makefile beside the check, where the next person adding a tier
will read it.
The three Linux tiers still run and pass; C89 -pedantic on Linux and both
Windows targets are warning-free.
A process-wide barrier does what a fence cannot: when it returns, every
other thread of the process that was running has passed through a full
fence too. That is what lets a handshake be asymmetric -- the side that
runs constantly uses a plain store and a plain load, no locked
instruction, and the side that runs rarely pays for the barrier once
before it sleeps. On a core thread pushing frames to a render thread,
that is a locked RMW on the producer replaced by three ordinary moves,
measured at 6.2 ns against 2.6 ns per publish.
Nothing in user mode can make another core execute a fence, so every
implementation is a request to the kernel to interrupt the others. This
file asks in whichever way the running system supports, resolved once:
membarrier Linux 4.14+, FreeBSD 14.1+. PRIVATE_EXPEDITED
IPIs exactly the CPUs currently running a
thread of this process.
FlushProcessWriteBuffers Windows Vista+. Disassembled from ntoskrnl
10.0.26100: raise IRQL, KiIpiSendRequestEx to
the affinity, spin for acks. Already the bare
IPI; there is nothing to beat on the fast path.
page flip x86 only: change the protection of a locked
page, and the TLB shootdown that forces is an
IPI whose handler is a serialising event.
Reaches back to Windows 2000, Linux 2.6 and
macOS on Intel. Refused on ARM and PowerPC,
where the invalidate is broadcast without an
IPI and the trick fences nothing.
signal Any Linux, any Android, any macOS including
Apple Silicon and PowerPC: signal every thread
currently on another CPU and wait for its
handler to acknowledge. The kernel delivers a
signal to a running thread by IPI on every one
of those -- kick_process on Linux,
cause_ast_check on XNU -- and each was checked
in source before this was written.
uniprocessor Nothing to fence against; a compiler fence.
none The multi-core consoles. retro_procbarrier()
returns 0 and the caller uses a symmetric
primitive instead.
Every tier's syscalls are on Android's seccomp allowlist -- membarrier is
there for ART's JIT, so it cannot be blocked for apps -- and the kernel
version cannot be inferred from the Android version, which is why this is
resolved at runtime rather than by ifdef.
The test resolves a tier, issues barriers with busy and sleeping threads
alive, and then runs the asymmetric protocol for 5000 rounds with the
consumer parked on a real condvar, failing on the first lost wake. That
last check is only load-bearing on SMP: on one core the race cannot
happen, and replacing the barrier with a no-op still passes -- checked,
and the test prints the CPU count so a green run on one core is not
mistaken for a result. RETRO_PROCBARRIER=<tier> forces a tier, so every
tier a platform has can be exercised on one machine; all three Linux
tiers complete here and are clean under ASan and UBSan.
Builds C89 -pedantic on Linux, and on Windows for i686 at
_WIN32_WINNT=0x0500 and x86_64 at 0x0600, warning-free. Darwin is written
from XNU's source -- osfmk/x86_64/pmap.c for the page flip, and
cause_ast_check on x86, arm64 and the xnu-1228 ppc tree for the signal
tier -- and has not been compiled or run here.
Restoring the workflows to the 14th also took out the two test
entries that landed after it with their fixes - the
task_queue_init_failure_test line in the libretro-common suite and
the overlay_visibility_bounds_test job in the tasks suite. They are
back; the workflow delta against the pre-rollback tree is the
action pins, the dependabot hold and the ORBIS checkout step alone.
A rollback to a point in time takes each file's changes commit by
commit, not the file wholesale.
The action pins, the ORBIS checkout step and the dependabot config
go back to their state at 02a1b2c2, before the setup-action bumps:
each lane pins the checkout and setup actions its container's node
can execute, and the bumps moved five of them past that at once.
Version moves for these actions come one lane at a time, together
with a container that carries the runtime they need. The
compile-matrix and budget references keep their current state.
The defaults_target harness stubbed fill_pathname_join() with a
plain strlcpy pair, and the overlap gate's helper table is keyed by
name across the tree - so the stub's unguarded copy marked every
aliased call of the real, memmove-based helper, 126 findings on a
function that is alias-safe.
The stub now carries the real helper's alias-safe shape, and a
helper defined under samples/ applies only to call sites in its own
harness directory: a standalone test's simplified stub of a tree
function describes that binary, not the tree. The selftest holds
the rule with a sample stub whose same-directory aliased call is
flagged while the tree's is not.
config_set_defaults() takes the settings it fills as a parameter, so
config_save_file()'s minimal mode builds its defaults into its own
local struct and config_st is assigned only at init and deinit: a
thread reading config_get_ptr() during a save can no longer see the
defaults or a struct about to be freed. The audio side stopped
reading settings off the main thread already; this closes the same
window for every other reader.
samples/config/defaults_target links the shipping configuration.c
and holds the contract: the live settings never move and never
change while a target fills; the root config directory ignore is
anchored so the sample tree is trackable.
The batch entries, the wide and inline paths, the reverse flush and
the sink estimator take pause, slowmotion, fast-forward, the ratio
and the output rate from the snapshot and the latched state, and the
inline transport's per-submit lowpass hint publishes as
AUDIO_SNAP_STRETCH_LPF - so a core with its own audio callback
submits from the audio thread without reading settings or the
runloop. The follow mode's tempo derives from the snapshot on the
submit path (audio_driver_transport_snapshot_tempo); configure-time
callers keep the live derivation, since they run before the start
publish.
The fixtures publish after every mutation of what those paths
consult, wherever a driven call follows in the same frame; the
wakeups check family and the discrete harness cover every entry,
and the reachability audit from the callback and the batch entries
shows no config_get_ptr() or runloop_get_flags() caller left below
them.
audio_fastpath_s16 publishes as AUDIO_SNAP_FASTPATH_S16 and
audio_driver_mixer_use_s16() consults the snapshot, so the flush's
native lane never reads settings on the pipeline consumer; the hint
is CMD_EVENT_NONE, and a frame-end publish follows a toggle the
same way the live read did. The wakeups check-tempo lane covers the
int16 device lanes this needs.
The discrete harness publishes at its stand-up and after every
mid-fixture toggle of what the consumer consults: up() ends in
snap_pause(), a helper that drives the publish through the stub's
runloop flags, and the fixtures' direct snapshot stores go through
it - or through audio_driver_publish_runloop() where slowmotion or
fast-forward flags are set first - so the snapshot always derives
from the settings the fixture configured, including bits that do
not exist yet.
The futex-backed scond is gated on __ANDROID__, and the comment gives the
reason: Bionic's condvar issues the wake syscall on every signal whether
or not anyone is waiting, so a waiter count in front of it makes the
common empty signal free.
glibc does not have that problem -- its pthread_cond_signal already
returns without a syscall when nobody waits -- so the gate was a fair
reading of where the win was. But the condvar still takes its internal
lock to find that out, and the difference is measurable on one core with
nothing parked: an empty scond_signal costs 3.6 ns through pthread_cond
and 1.5 ns through the futex path, and scond_broadcast the same. Small in
absolute terms; it is the cost a producer pays per publish when the
consumer is awake, which for a core pushing GS packets is the common
case.
So the gate becomes __linux__. The futex path was already written for
glibc's syscall interface -- it goes through syscall(__NR_futex) with the
constants spelled out -- and nothing else in the file keys on Android for
it. retro_eventcount_test and tpool_wait_test pass with the change, as do
the ps2 core's faultstress and spsc suites under TSan with it forced on.
Wake latency, the other half of the comparison, is not measured here: it
needs a second core to mean anything.
The extracted write callback reads the HAL timestamp's rate scalar
and the device-clock members, so the stub AudioTimeStamp, its flag
constants, output_rate and the ct_* fields exist here as in
coreaudio_clock. NULL timestamps still skip the block; the clock
itself stays coreaudio_clock's subject.
The fixture models the main thread, so reset() and the estimation
toggles publish the runloop snapshot after setting
audio_sink_rate_estimation; the refused-frames warning follows the
snapshot bit.
Audio Sync, the fast-forward speedup and the sink-rate estimation
publish as snapshot bits, the slowmotion ratio as its float bit
pattern beside the snapshot, and the output rate setting latches
into the state at driver init. Everything reached from
audio_driver_callback() - the transport and pipeline consumers,
their retry and render, the flush and the sink accounting - reads
those and never config_get_ptr(), so a config_save_file() that
swaps the settings pointer mid-save cannot hand the audio thread
defaults or a freed struct.
The harnesses model the main thread, so their fixtures publish
after mutating what the consumer consults, and pipeline_wakeups'
config_get_ptr() stub counts calls made inside the callback on
whichever thread runs it: a nonzero count is a fixture failure,
in every TRANSPORT and WRAPPER mode.
The overwrite of sig_us for the next signal must not race the audio
thread's read of it for the last one; both ends of the handoff go
through the atomic, so a TSan run of the wrapper modes is clean.
The node24 bump in actions/checkout took out the PS4/ORBIS job, whose
build container cannot exec the runner node binary for that runtime.
ORBIS no longer uses the action, but the rest of the container-based
workflows still do, so stop taking these bumps unattended and move them
by hand once the container side has been checked.
actions/checkout v5 and later declare the node24 runtime. The PS4/ORBIS
job runs inside reallibretroretroarch/libretro-build-orbis:latest, an
Alpine image, so the runner has to exec its musl node build inside the
container; node24 does not come up there and the checkout step dies
before anything is compiled. Pinning the action back to v4 fixes it
today, but the container is old enough that every runtime bump will
break it again, and the same applies to actions/upload-artifact.
A plain run: step executes in the containers own shell and never touches
the injected node, so do the clone by hand instead. The repository is
public, so no token plumbing is needed.
discrete_multichannel_test leaks 466 MB over a run, in 2910
allocations, and cannot be run under LeakSanitizer.
Six of the buffers it installs - output_samples_buf,
output_samples_int16, input_data, input_data_int16, pipe_scratch and
pipe_conv - are slices of arena_int16, arena_float and pipe_arena in
the frontend, so audio_driver_deinit_internal() frees the arenas as
units and only NULLs the named pointers. The test hands the state
plain malloc()s instead, so teardown clears the pointers with nothing
released, and because each stand-up deinits and memsets before
installing its own set, the previous case's blocks are already
unreachable by then.
The test keeps its own handles to those six and frees them at each
stand-up and at the end of main. The buffers deinit really does free -
upmix_buf, upmix_i16, pipe_wide, multi_fold, record_remap - are
deliberately not among them.
audio_driver_extra_free() releases extra.res[] on the float path
through audio_st->resampler, which is a different object: the main
pipeline's resampler, not the one retro_resampler_realloc_hq() picked
for the extras. When it is absent the branch is skipped and every
instance in res[] is dropped, and even when it is present the free
depends on an unrelated pointer being the same driver.
The driver realloc_hq() hands back is kept in the struct beside the
instances it made and the free goes through that, so extra.res[] is
released by whatever allocated it. The int16 path is unchanged: it
frees through resampler_int16_free and leaves res_drv NULL.
discrete_multichannel_test aborts on
double free or corruption (!prev)
after the last transport quality case, every run. fake_device_captured()
handed back g_cap itself, and the device thread grows that same buffer
with realloc from r_releasebuffer(), so the harness was reading the
capture through a pointer the writer had already moved - the decode loop
walks it for the length of the capture, which is ample time. It now
copies into a buffer of its own under the lock and returns that, so the
pointer stays put until the harness asks again; the append side checks
its realloc instead of writing through whatever it got back.
The burst count needed the same correction it was given for the other
end. Capture length does not say how many bursts arrived, because the
device's pump plays on after the content ends and the tail of the
capture is silence, so the count grew with however long the drain
happened to run - a TSan build captures half again what a plain one does
from the same feed and failed on 10 of 16. Bounded by what the encoder
could have produced, it means bursts on any build.
* ozone: slow mouse movement wakes the hover highlight
Any keyboard or pad action, a right-click back, or a tap on the header
or footer turns cursor mode off, and it only came back when the mouse
moved more than 3 px (at scale 1) between two frames. That is a speed
floor of about 240 px/s at 60 Hz and 660 px/s at 165 Hz, so moving
slowly to the next row did nothing until the mouse was flicked.
Measure the movement over 100 ms instead of one frame. The floor is now
about 40 px/s at any refresh rate, and resting drift, or a small bump
followed by a brush seconds later, still does not take the selection
away from the pad.
* menu: kinetic scroll decay runs on elapsed time
The flick deceleration applies its factor once per 16.667 ms of
real time, carrying the remainder between frames, so a flick
travels the same distance at every refresh rate. Step count per
frame is capped, keeping the cost bounded across long gaps.
* materialui: overscroll entry velocity uses the measured frame time
The pointer's per-frame acceleration converts to the px/s the
overscroll spring integrates via the actual frame delta, so the
bounce enters with the same physical velocity at every refresh
rate.
* ozone: drag auto-select settles on a real velocity threshold
The gate that lets the entry under the cursor be selected while
drag-scrolling compares the pointer's acceleration normalised to
px per 16.667 ms, so 'settled' means the same physical speed at
every refresh rate.
* menu: the single-click entry-draw hold is a 500 ms deadline
The playlist single-click hold arms a microsecond deadline
against the menu's frame-sampled clock in all four drivers, so
the hold lasts the same real time at every refresh rate. The
handle field moves to the drivers' 64-bit alignment sections.
* xmb: missing-icon retry runs on a one-second deadline
The retry window after a context reset is a microsecond deadline
against the menu's frame-sampled clock, so an unresolved icon is
retried for the same real time at every refresh rate and a
permanently-missing asset still terminates the loop.
* gfx: timing constants are spelled in their units
The smooth ticker's baseline is a px/s speed with the frame
period derived from it (same float32 value, so behaviour is
unchanged), the stepped ticker intervals carry their us unit in
comments, and the thumbnail stream delays are plain milliseconds
with parenthesised macro bodies. A grep for 60 Hz frame
arithmetic now returns only real suspects.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The networking translation units carry threaded machinery of their own -
the HTTP DNS cache's lock and condition, the task queue's - beside code
that is compiled either way, so a broadcast or a wait written outside
HAVE_THREADS compiles green in every job in the matrix and breaks a
threadless build. A signal added to net_http_resolve(), which is not
itself guarded, was caught by samples/network/webdav_options rather than
here, which is later than it should be and only because that sample
happens to compile net_http.c with threads undefined. net_http.c,
net_socket.c, task_http.c and the cloud sync pair now go through the
same no-threads lane the video and audio files use; it is syntax-only,
so it costs nothing worth counting.
The three report helpers each decided whether to print "ok" from the
script-wide failure flag, so once any lane failed every later lane that
passed went silent and the log stopped distinguishing a lane that
passed from one that never ran. They now track their own outcome and
leave the script-wide flag to the exit status, which is unchanged.
discrete_multichannel_test fails intermittently on
FAIL: only 9 of about 11 bursts reached the device
and the message is wrong about its own subject. The byte count is the
same in a failing run and a passing one - 67584, exactly eleven whole
bursts - so all eleven reached the device. What varied was how many
decoded.
Two things were wrong with the check.
It compared against what was fed, frames / 1536, not against what
arrived. The feed's last partial block is still inside the encoder and
was never a burst at all, so the old comparison asked for a burst that
could not exist.
And it allowed one burst of loss where the capture geometry costs up
to two. The capture does not begin or end on a burst boundary: the
decode loop seeks the first sync word, so a partial head is skipped,
and it needs a whole burst remaining, so a partial tail is left. How
much is lost depends on where the writer was when capture opened - a
plain build decodes ten of eleven here and an ASan build nine, from
byte-identical captures, every time. That is not flakiness between
runs, it is a build-dependent constant that the threshold happened to
straddle.
So the check measures bur_len / IEC61937_AC3_BURST_BYTES and allows
two. Anything worse is a burst that arrived and would not decode,
which is a real failure and still caught.
The count was never this case's subject. What it tests is that a 5.1
core's channels survive the AC-3 encode, and that is the per-channel
tone comparison below it, which runs over whatever did decode and
passed in every run including the failing ones.
Also widened the capture drain, which gave up after fifteen quiet
20 ms laps. Three hundred milliseconds of no progress is a writer
thread that lost its slice on a loaded machine, not a finished stream.
A second now, ten seconds overall, and it still leaves the moment it
has what it asked for.
Three plain runs and three ASan runs pass, where before the ASan build
failed every time and the plain build sat one burst above the
threshold.
Not fixed, and pre-existing: the harness leaks about 30 MB through
up() in producer_spans.h, allocated 120 times and never freed. CI runs
this sample with detect_leaks=0, so it is not what fails, but it is
real.
net_http_wait() has no socket to wait on while a transfer's name is
still being resolved, and a threaded caller has nothing else pacing it,
so it would come straight back into net_http_update() and spend every
pass it is willing to make before the socket exists. net_http_wait_dns()
covers that window: the resolver publishes each entry under the DNS
cache lock and broadcasts on a condition beside it, and the wait sits on
that broadcast until the entry turns valid or the caller's timeout runs
out. It re-reads the cache on each wake rather than holding an entry
across the wait, so net_http_deinit() emptying the list under the same
lock releases a waiter instead of leaving it on a freed entry.
The condition lives with dns_cache_lock behind HAVE_THREADS, and so do
the wait and every broadcast, including the two in net_http_resolve().
Without threads the resolver runs inline and its result is already
published by the time the caller could wait, so net_http_wait() returns
at once.
pipeline_layout and stretch drive their producer and consumer threads
through bare busy loops. On a host with more than one core that is what
the tests want, but on a uniprocessor the spinning thread holds the CPU
for its whole quantum while the thread it waits on never runs, so
progress comes only from preemption and the stress loops take
effectively forever. Both hang on single-core runners and containers:
pipeline_layout never finishes stress(44, false), and stretch stops at
case 38 of 64.
The waits spin a short budget and then yield through retro_sleep(0), as
gfx_thumbnail_status_atomic and cached_frame_hazard already do for the
same reason. On a multi-core host the wait resolves inside the budget
and the yield costs nothing. How long the spin lasts tracks how few
frames the ring holds: pipeline_layout buffers five frames at width 44
against eleven at width 22, so twice the handoffs and twice the quanta
to spend on them.
Both harnesses build at -std=c89 because the pipeline headers have to
stay C89-clean, and under that retro_timers.h cannot see struct timespec
or nanosleep, so their makefiles define _POSIX_C_SOURCE rather than
relax the standard.
The frontend translation units are compiled whether or not a subsystem
is switched on, so each new lane drops one of HAVE_OVERLAY, HAVE_CHEEVOS,
HAVE_NETWORKING and HAVE_RUNAHEAD from a fully featured build, and a
last one drops all four.
Each subsystem's own unit is checked with its gate on and the menu and
widgets off, which is where its widget and menu entry points sit.
drivers_init() and video_driver_build_info() name gfx_display_t,
disp_get_ptr() and gfx_display_init_first_driver(), and gfx/gfx_display.c
is built in every configuration. Include its header directly in
retroarch.c and gfx/video_driver.c so the declarations do not depend on
menu_driver.h or gfx_widgets.h reaching the translation unit.
compile-matrix.sh gets a gate section covering all four combinations of
HAVE_MENU and HAVE_GFX_WIDGETS across retroarch.c, runloop.c,
gfx/video_driver.c and gfx/gfx_display.c.
PSL1GHT's newlib ships no pthread implementation, so rthreads gets a
USE_PS3_THREADS backend built straight on the lv2 primitives: sysThread*
for sthread, lwmutex for slock, and lwcond for scond, bound to its lock
at the first wait the way the Vita backend binds its LwCond. The PS3
SDK's pthreads sit on those same syscalls, so both SDKs take this one
path; the two spellings are reconciled in rthreads.c itself, under
prefixed names of its own so that a unity build sees no collision.
Threads inherit the creating thread's lv2 priority, and an explicit
rthreads priority maps across 2047..512.
The HAVE_CHEEVOS block assigned to ret unconditionally, discarding
both checks above it. On a cheevos build an unparseable frame
argument, or no movie playing at all, still reached
movie_seek_to_frame whenever hardcore mode happened to be inactive.
The argument check was also inoperative: strtoll always writes a
valid end pointer, so the !endptr test never fired. An empty or
non-numeric argument shows up as no characters consumed. arg itself
is now checked too, since this path is reachable from the network
command interface.
task_autodetect.c calls strtoll to read the __total_size and
__newest_mtime fields of the autoconf profile index. strtoll is C99
and the Visual Studio 2005 CRT does not have it, so the MSVC2005 lane
compiled it as an implicit declaration (C4013) and failed at link with
an unresolved _strtoll. A second call in command_seek_replay is only
spared because HAVE_BSV_MOVIE is off in that lane.
Neither call can be demoted to strtol. __newest_mtime is a unix
timestamp and __total_size is a sum of file sizes, both int64_t on the
producing side in input_autoconfigure_index_dir_fingerprint; on the
32-bit lanes strtol would clamp the timestamp at LONG_MAX from 2038
on, the freshness comparison would then never hold, and the index
would be silently discarded on every connect with no diagnostic.
_strtoi64 and _strtoui64 take the same arguments with the same
meaning, so from _MSC_VER 1400 the mapping is exact. Visual C++ 6.0,
2002 and 2003 have neither. The _atoi64 fallback that stood in for
_strtoui64 there is dropped rather than extended to strtoll: it
accepts no base and reports no end pointer, so a caller relying on
either got wrong answers instead of a build failure. compat_strtoll.c
supplies a real implementation for those CRTs, verified against glibc
over 35 inputs by 6 bases for both signednesses on value, end pointer
and ERANGE.
The old strtoull mapping applied to every MSVC version including the
UCRT, which rewrote std::strtoull in C++ translation units as well;
2015 and later now keep the real functions.
Two harnesses fail to compile on audio_driver_st.sink_discarded, which
"Remove eleven write-only state fields" took out. It was not
write-only: the sweep behind that commit looked at the frontend and
not at samples/, and two files there name it.
The two uses are not the same and do not want the same answer.
pipeline_rate_control only printed it in a status line and asserted
nothing on it, so that line drops the count. Nothing is lost - the
number was decoration.
sink_rate asserts on it: CHECK(st->sink_discarded >= 1, "a stalled
window was not left out"), immediately after freezing the device clock
for five seconds. That is a real behaviour with no other observable -
a frozen device must have its window left out of the estimate rather
than folded in - and the counter is the only thing that makes it
testable. So the field and its three assignments come back, with a
comment saying who reads it and why, since a reader looking only at
the frontend will conclude it is dead again.
A field whose only consumer is a test is still a field that is read. A
grep of the whole tree rather than of the frontend is what tells the
two cases apart.
The other ten fields from that commit were checked the same way: none
is named anywhere under samples/ or libretro-common/samples/, so this
is the only one.
Both harnesses build and pass under ASan and UBSan.
A transfer driven from a task thread has nothing to pace it, so
task_http_iterate_transfer() slept a millisecond between passes whether
or not the peer had answered - paid in full even when the bytes were
already there, and a transfer of n passes cost n milliseconds of pure
delay. The FIXME on it asked for a wait with a timeout.
net_http_wait() is that wait. It knows which way the transfer is going,
so it waits for writability while connecting or sending and for
readability once the request is out, and it returns the moment the
transport is ready rather than at the end of a fixed interval. The
unthreaded queue is paced by the frame and waits for nothing, as before.
What makes it safe to wait at all is knowing when there is nothing to
wait for. A pass that stopped on its own drain budget left bytes
buffered and must not block, and the same is true inside the TLS layer,
where decrypted bytes can be held after the socket has gone quiet; only
the receive path's EAGAIN exit has handed over everything it holds, so
only that exit marks the transfer as blocked and every other outcome
returns from the wait immediately.
The stress harness covers a wait that returns on data rather than on the
timeout, a wait with no socket to wait on, and a peer that accepts and
then never answers, which is the case that has to come back on the
timeout rather than hang.
Both drivers asked retroarch_get_rotation() and
video_driver_get_core_aspect() for the plain uniforms and then again a
few lines down to derive OriginalAspectRotated. The values are taken
once into locals that serve both, which is what the GX2 and D3D12
drivers already do.
Each of these is assigned and never read anywhere in the tree, so the
work that feeds them is spent for nothing: content_label_len on the
thumbnail path data, sink_discarded on the audio state,
runahead_save_state_size, task_count and msg_queue_regular_padding_x on
the widgets, pipeline_active on the display draw struct,
resolution_hd_enable on the D3D8 and D3D9 states,
memory_search_initialized on the cheat manager, window_title_len and
video_frame_delay_auto_t::frame_time_avg on the video state, and
input_dialog_kb_type on the menu state. Where a write captured a
strlcpy() or fill_pathname() result the call stays and only the
assignment goes.
Two of them were holding up more than themselves.
gfx_widgets_update_icon_layout() branched on whether icons are loaded
solely to pick a padding value, so that conditional computed nothing
and is gone with the field. memory_search_initialized sat beside
memory_initialized, which is read, leaving two near-identical names for
the next reader to tell apart.
The iteration's measured period and the frame limiter's schedule both want
the instant the iteration ended, and everything between them - the pace
gather, the source decision and the no-window test - is a set of flag tests.
runloop_iterate() takes the reading once into pace_now and the limiter
schedules against it.
video_driver_frame() publishes the microseconds since the previous frame it
handed the driver into video_driver_state_t, taken from the clock reading
that path already makes, and video_driver_get_frame_time_delta_usec() serves
that value. A multi-pass preset asks once per pass, so every pass of one
frame now sees the same delta on every driver and shader backend, and the
chain costs one reading per frame rather than one per pass.
d3d10 and d3d11 gather the remaining per-frame invariants - frame direction,
original fps, rotation and the core aspect - at the top of the shader_preset
block instead of once per pass, as d3d12 and the GX2 driver already do.
runloop_state_t::frame_limit_last_time is written on content load, on the
quit path and once per iteration inside the timer pacing block, and is read
nowhere - the nanosecond anchor carries the schedule on its own. The field
goes, and with it the divide the pacing path did to feed it.
cheat_manager_apply_rumble() takes the timestamp from
cheat_manager_apply_retro_cheats() rather than reading the clock itself. It
runs once per cheat per frame and returns immediately for every entry whose
rumble type is disabled, so one reading now serves the whole pass and every
cheat in a frame is applied at the same instant. The function is file-local
and is marked static.
The handoff statistics - the window's start, a push's accounting, the
window's close - and the display-pacing hold are functions of their
own, out of line, so the push's body carries a call to each rather
than their code. VIDEO_NOINLINE moves to video_driver.h for the
wrapper to share.
The push's reference in the hot-function budget moves from 2374 to
3097 bytes and 88 to 152 of stack: the source-format conversion
staged for the video thread, the perf-counter reads the overlay's
handoff statistics take, and the pacer's drain flag are on the path
now, and the reference was measured before any of them.
The Frames line of the handoff block gains, for the window, how many
pushes replaced a queued frame and how many holds ran a period long
to drain the queue, so the cumulative drop count can be read against
what the pacer is doing now.
A frame handed over more than a period and a half before the vblank
it went out on was queued behind another; its swap waited on that
vblank, not on rendering. The pacer no longer averages such a swap
into render_time, where it grew the reserve to a whole period, ended
the hold and kept the queue full, and it drains the queue by skipping
one content period in the schedule the next time the hold runs, with
a few presents' cooldown so frames already in flight do not drain
again.
samples/gfx/threaded_video gains a lane with a driver that presents
like a vsynced two-deep swapchain: the loop runs unpaced until a frame
queues behind another, the hold comes on, and latency must settle
under a period and a half with the reserve staying well under a
period.
The comment on the producer's notify opened by claiming the consumer is
told even when nothing was written, then spent the rest of the sentence
explaining that the notify is skipped. The code skips it. What makes
that safe is that a write fitting nothing leaves the queue exactly as
the consumer last saw it, and a consumer parks only on a queue with
nothing to read, which a queue too full to accept the write is not.
Replacing an entry's string freed what was there and then assigned the
result of strdup(). With no memory to copy into, the field was left NULL
and the entry marked modified, so a complete entry became an incomplete
one and the next write put that on disk - memory pressure destroying
metadata that was on disk correctly a moment earlier.
playlist_entry_replace_str() makes the copy first and only then releases
what it displaces, so a failure leaves the entry as it was. The ten
places that replaced a string this way go through it: the six fields in
playlist_update(), the four in playlist_update_runtime(), and the base
directory rewrite, which now keeps an entry usable under its old base
rather than losing the path it was found by. Modified is recorded only
where a replacement landed, and an entry's path id is dropped only where
its path actually changed.
playlist_parity_test denies the copy and checks the entry keeps the
values it could not replace, then that the same update lands once the
copy can be made.
The rewind ring derives every buffer dimension from the size a core
reports through retro_serialize_size(): the block rounding, the
per-block compression overhead, the sentinel offsets, and the doubling
that puts both working blocks in one allocation. None of it was checked,
so a size near the top of the range wrapped to an allocation smaller
than the offsets computed from the same numbers, which the harness
reproduces as a segfault. The derivations now go through checked add and
multiply, state_manager_raw_maxsize() reports 0 for a size it cannot
express, and a size that cannot be laid out is refused. The bound in
state_manager_pop() subtracts from the capacity rather than adding to
it, since the value it checks comes out of the buffer.
Two smaller sums get the same treatment. The streamed config parser
totals the retained tail, the incoming packet and a NUL before comparing
against the window capacity, and the message queue totals its node and
its two strings before allocating the block they share. Neither is
reachable from anything RetroArch feeds them, since both lengths come
from real buffers; they are checked because the check costs nothing and
the wrapped case writes past the allocation.
state_rewind_test covers the sizes that wrap, separating those the
compressed-size calculation rejects from the one only the block-pair
doubling catches, and pins that an ordinary size still gets the same
answer as before.
The wrapper's software-framebuffer lend refused any ask carrying
RETRO_MEMORY_ACCESS_READ, so a core that snapshots its frame - prboom
for its wipe - never got a slot and every frame was copied. The slots
are cached host memory; the ask is granted and reported as such
through memory_flags, and asks are counted before any refusal. The
threaded_video harness core now asks with READ as well.
The overlay's handoff block gains a Lend line for the window: how
often the core asked for a framebuffer, how often a slot was lent,
how many loans lapsed because the core pushed from elsewhere, and how
many asks were declined because both slots were taken or the frame
would not fit. Counts only, kept from the moment the overlay comes up.
While the overlay is shown, video_thread_frame() counts the runloop's
side of a push on cpu_features_get_perf_counter(): entry to the worker
signal with the slot wait taken out, the frame memcpy on its own, the
slot wait on its own, the bytes copied, and the window's frames by
kind - copied, zero-copy, hardware, and how many waited for a slot.
The tick rate comes from the span between the two clock reads the push
already makes, so the frame path reads no clock for this; windows of
120 pushed frames, average and worst, shown under VIDEO as
microseconds.
libretro-common's perf counter now reads the CPU's own counter first
where there is one - rdtsc on x86 for GCC and MSVC, cntvct on AArch64,
the ARMv6 cycle register - and only then a platform clock; Windows
without one uses QueryPerformanceCounter. Before, the Windows branch
came first and answered GetSystemTime at millisecond resolution on
every x86 build, and Linux answered clock_gettime ahead of rdtsc.
The index passed to input_overlay_set_visibility() does not come from
the overlay. The overlay LED driver forwards settings->uints.led_map[],
which configuration.c reads out of the config file with
CONFIG_GET_INT_BASE and no range of its own, so "led1_map = 500" reached
the setter as 500 and wrote past overlay_visibility[MAX_VISIBILITY]
during drivers_init(). The same index then went to the video driver's
set_alpha(), which indexes its own per-image storage with it.
The write is now bounded on the same terms
input_overlay_get_visibility() has always used for the read, and the
set_alpha() call is bounded by the images the active overlay loaded, as
input_overlay_set_alpha_mod() bounds its own walk. Clamping led_map at
the config end would be wrong: the other LED drivers use those values as
GPIO numbers and keyboard LED ids, where the overlay array's size means
nothing.
overlay_visibility_bounds_test builds the shipping LED driver to pin
that it forwards config values unaltered, so the setter cannot assume a
caller has checked, and covers the bound itself.
Use the native int16 range to pack exact integer dot products and energies into SSE2 lanes. Correct the exceptional +2^31 pair before widening, and retain scalar normalization, candidate ordering and the non-SSE2 fallback.
Check exact scores against the shared scalar helper across extrema, silence, random values, alignment and every supported hop length.
The Info build overlay line goes, with the clock reads and the state
behind it: no wall clock on the frame path.
video_thread_convert() and video_thread_defer_convert() live outside
the HAVE_VIDEO_FILTER block, as the frame loop calls them whether or
not the software filter is built.
The 0RGB1555 lane in samples/gfx/threaded_video is removed: it never
rebuilt the video driver after changing the core's pixel format, so
no converter existed on either path it checked.
task_queue_init() selects the threaded implementation before running its
init hook, so an init that produced no worker left the queue accepting
tasks nobody would service. slock_new() and scond_new() return NULL when
the allocation or the platform primitive fails and sthread_create()
returns NULL when a thread cannot be started, none of which was checked;
slock_lock() tolerates a NULL lock but scond_signal() does not.
The threaded and GCD hooks now build their synchronisation primitives
through one helper that takes the whole set back if any of it is
missing, and the threaded hook releases them again when the worker
cannot be created. A failed init reports it, and task_queue_init() falls
back to the regular implementation so tasks run on the caller's thread
instead of queueing forever. Callers read the outcome through
task_queue_is_threaded(), which is where the frontend logs it.
task_queue_init_failure_test forces sthread_create() to fail and checks
that the queue reports itself unthreaded, that a task pushed to it still
runs, and that a later init starts a worker again.
Construct native canonical frames in writable SPSC spans, retaining a one-frame wrap fallback and complete-frame publication. Convert only packed active channels with the shared SIMD routines on mixed-format input. Remove the unused canonical staging allocation.
Fuse float unity copying with the existing platform sanitization rules while preserving borrowed input ownership. Cover public producer layout/format/wrap/pressure cases and clamp special values in the existing frontend fixture. Keep ordinary consumer copying after local borrowing experiments failed to establish a consistent pressure/latency benefit.
With a raster block bound, render_msg no longer uploads the atlas for
each string that rasterized a glyph; the renderer keeps merging the
dirty rectangle and flush_block uploads it once before the block's
draw. Vulkan submits one staging command buffer per flush instead of
one per string, D3D11 maps its staging texture once. Without a block
the upload stays with the draw. D3D12 already staged the region and
copied it at draw time.
The runloop-thread cost of assembling video_frame_info_t is measured
around the call and shown under VIDEO as the average over the last 120
frames and the worst frame in that window, in microseconds. Two clock
reads a frame, no logging.
Under the threaded wrapper a frame in a source format the driver does
not take now travels raw: the main thread stages the conversion with
video_thread_defer_convert(), the same way it stages the software
filter, and the worker converts in front of the filter and the driver
call. 0RGB1555 goes through the existing scaler to RGB565, XRGB2101010
is narrowed to XRGB8888 with the existing scratch buffer. The scaler
and scratch are freed only after the worker is idle. A frame being
recorded from the CPU keeps converting on the main thread, since the
recorder reads it there. Without the wrapper nothing changes.
samples/gfx/threaded_video gains a lane: the harness core switches to
0RGB1555 and the driver must receive RGB565 on both the direct and the
threaded path.
Define write_raw returns as accepted input stereo frames, with explicit partial/error, ownership, zero-length and control-failure semantics. Keep ordinary write in bytes and convert raw acceptance to output-frame equivalents for sink accounting.
Validate SDL3 raw requests, propagate failed controls and queue queries, and retain truthful caches on recovery. Count ordinary writes in frames internally while returning bytes. Update the rate-control test driver and add a device-free real-SDL3 contract fixture without expanding CI jobs.
A string drawn without a raster block bound is one backend draw; a
bound block is one draw when it is flushed. The count joins the MENU
UI text line so the effect of per-layer text batching is visible
next to the number of text calls.
dir_list_new() returns NULL for a root it cannot open, so a sync
directory that does not exist - an unset system path, a mount that is
not there - dereferenced NULL while the manifest of local state was
being built. The listing result is now checked, and every caller of the
directory map handles its absence.
Skipping the directory would have been worse than the crash it replaced.
An empty listing is a statement that every file the directory used to
hold is gone, and the diff answers that by removing the server's copies;
a directory that could not be read says nothing at all, so the two have
to be told apart. Where the local state cannot be established, the sync
stops with a failure rather than working from a manifest that is missing
files it never looked for.
The samples/tasks cloudsync test covers that distinction directly, since
the whole fix rests on it.
The font renderer now implements bind_block/flush_block. While a block
is bound, render_msg writes glyph sprites into a CPU array on the font
instead of mapping the sprite ring, and records the count on the
block; atlas staging still happens as glyphs are met. flush_block maps
the ring once, copies the gathered sprites and draws them. The draw
tail is shared with the immediate path, which is unchanged when no
block is bound.
The font renderer now implements bind_block/flush_block. While a block
is bound, render_msg writes glyph sprites into a CPU array on the font
instead of mapping the sprite ring, and records the count on the
block; atlas updates still happen as glyphs are met. flush_block maps
the ring once (DISCARD on wrap), copies the gathered sprites and draws
them. The draw tail is shared with the immediate path, which is
unchanged when no block is bound.
Compose the achieved fast-forward multiplier with rate control and slow motion before driver-side SRC, matching the software paths. Reset inline cadence on release. This also fixes queued filter-only playback through write_raw.
Cover normal, accelerated, combined slow/fast, disabled speed-up and queued native LPF output at the raw driver boundary. Preserve native sample bytes and keep the existing CI matrix.
The font renderer now implements bind_block/flush_block. While a block
is bound, render_msg gathers glyph quads into a CPU array and records
the count on the block; the atlas upload still happens as glyphs are
met. flush_block uploads the gathered quads to the VBO chain once and
draws them with the shared quad index buffer, sliced by its capacity
through the vertex offset. Without a block bound the immediate path is
unchanged.
Prepare the existing bounded native transport when the lowpass option alone is enabled. Keep WSOLA inactive and retain ordinary SRC speed and pitch, including the threaded device-space bound. Freeze this policy at audio reinitialization and retain it through fallback/recovery.
Reuse native filter and stream storage without additional conversion round trips or processing allocations. Update setting help and cover native reference composition, dry playback, multichannel HQ, short writes and recovery in existing frontend tests.
A new notification-visibility bool, statistics_hide_in_menu, sits under
Display Statistics and is shown only while that is on. With it off the
menu frame draws the statistics overlay on top of the menu through the
OSD font, in every driver, so the MENU UI counters can be read where
they are produced. Drivers keep drawing the overlay themselves only
when no menu is up, so the default is unchanged.
NO_OVERWRITE is only valid for regions the GPU is not reading. When
the ring wraps to offset zero the region ahead may still belong to
in-flight draws, so the wrapping Map uses DISCARD, which renames the
buffer and leaves those draws their own copy. Applies to both the
generic display path and the font path.
Each deferred upload is submitted with a fence drawn from a small
recycled pool and released when vkGetFenceStatus reports it signalled,
polled once per frame without blocking. Retirement no longer depends
on the swapchain image count or on the frame-fence cycle, so uploads
made before a swapchain exists, or around a recreation, hold their
command buffer and staging memory until the GPU is done with them.
Link the real state manager into the existing frontend fixture. Exercise six forward frames, three restored states and return to prepared WSOLA playback through native int16 stereo and cached float multichannel callbacks.
Check restored frame identities, callback bindings, native reverse capture and transport reuse. Keep production code and the existing CI job matrix unchanged.
Every flush point now records why the gathered quads had to go out -
text, texture change, blend group, scissor, a non-gathering draw, a
full batch, or an explicit flush from outside - along with quads,
strips, largest strip, and text calls/bytes per menu frame. Counts are
plain increments on the drawing thread and are published atomically
once per menu frame for the overlay, which adds a MENU UI block while
the menu is up.
The deferred command node now carries the staging texture or raw
staging buffer the copy reads from, so a one-shot upload can be
submitted without draining the queue and its resources released when
the consuming frame retires. Static-texture creation and compressed
texture loads use it; the glyph atlas path shares the same entry point.
The dirty-rect atlas copy is recorded into a one-shot staging command
buffer and submitted ahead of the frame in queue order; the trailing
transfer-to-fragment barrier makes it visible to the glyph draws that
follow in the same frame. The command buffer is parked on a per-driver
list and returned to the staging pool once a full swapchain cycle of
submissions has retired it, alongside the deferred-texture tick.
An empty reverse capture still reached submission, unnecessarily resetting prepared inline transport. Return before recording and submission when no valid captured range exists.
Extend the existing lifecycle cases to verify empty playback leaves device/queue output and inline transport untouched. Nonempty native playback is unchanged; no CI cases or jobs added.
New rewind arenas must start empty, and teardown must clear the cursor alongside capacity. Discard captured history when the negotiated source format changes so reverse playback cannot read the other native arena using an old cursor. Repeated format queries retain history.
Cover actual arena initialization and teardown plus both format directions across ordinary, inline and queued configurations in the existing frontend fixture. Sample-processing loops are unchanged.
Match normal, float and multichannel capture: suspended int16 rewind callbacks must not retain speculative audio or advance the reverse cursor. Preserve batch frame accounting and already captured native history.
Extend the existing native and mixed rewind cases with suspended callbacks and byte-exact playback checks, without adding CI cases or jobs.
After float negotiation, reverse playback reads the float arena even when a core sends int16 samples. Capture int16 callbacks directly into that arena with one conversion, preserving interleaved native float samples and avoiding split or stale history.
Cover single-sample, batched and multichannel int16 callbacks interleaved with float, including exact buffer contents and device output. Native int16 and float lanes retain their existing formats.
A menu driver's frame() runs on the video thread under the threaded
wrapper and read the settings while it drew: seventy-one reads across
the four of them, thirty-six distinct settings, every one of them
written on the main thread by the menu itself. A theme changed while a
frame drew was a read of a word being written.
They travel with the frame now, taken in video_driver_frame() where the
menu cannot be changing, in the same block that already takes the
colour theme and the wallpaper opacity for the same reason. The two
that are paths get the frame's own copy in the wrapper, as the widget
paths do, because a path is worse to read mid-write than a word.
The block the menu fills is zeroed when the menu is closed, so the
struct stays deterministic whether or not anything looked at it, and
the two settings that exist only under HAVE_XMB are taken only there.
Renders identically: rgui and materialui to the pixel, ozone and xmb
to an exact match inside their own frame to frame variation.
Cached int16 multichannel callbacks were playing forward during rewind, while inline float on a wide device retained only the front pair. Route both native entries before discrete processing through the bounded stereo fold and existing reverse capture.
Cover native formats, stereo/5.1/7.1, ordinary/inline/queued configurations, split batches, suspension and clipping in the existing frontend fixture. Reserved fold storage does not grow during capture.
Reverse complete frames in native int16 and float batches, matching the single-sample callback. Keep the existing float-to-int16 bridge rounding and reject incomplete-frame room.
Cover split batches, bounded writes and byte-exact HQ playback through rewind bypass and return to inline WSOLA in the existing frontend fixture.
Treat input consumed into bounded inline stretch state as callback progress, even before WSOLA produces a device write. Reset the progress flag per callback so empty, paused and suspended sources still park.
At 32x tempo, both native regression cases previously reported 238 idle passes for 256 source batches. They now report none while retaining the same output and write counts. No sample conversion or allocation is added.
Reject invalid timing and unrepresentable stereo sample counts before converting the menu's frame count to unsigned. Preserve valid menu batch sizes, including sub-frame timing, and keep validation out of core sample paths.
Cover invalid metadata and two valid menu rates in the existing frontend target.
input_phys matching had no test coverage because the test joypad
driver never reported a physical location: input_autoconfigure_connect
was called with a NULL phys, so every profile's input_phys was a
mismatch and the scoring branch was unreachable. That is why #19540
reached a release.
Teach the test driver to carry a phys in the .ratst connect string,
after a "@@" separator, and split it back out of the device name.
Existing step scripts have no "@@" and are unaffected.
P7 pins the ranking itself: two profiles with identical vendor/product
ids and identical input_device, differing only in input_phys, so the
match must beat the mismatch 60 to 40. Run in both file orders,
because the reported symptom looked order-dependent, and both cold and
through the index.
P8 pins the regression. It demotes a profile below its rival before
the index is built, then promotes it back with a size-preserving edit,
which leaves only the modification times to tell the edited directory
from the indexed one - the case a *.cfg count can never catch. The
rival is untouched, so its recorded claim stays honest and passes the
winner re-score, which is what let the wrong profile through.
Both variants are checked: one with a second's delay, and one without,
where the index write and the edit can land in the same whole second.
Verified to fail on the unfixed binary and pass on the fixed one.
The index is built with a device nothing matches, since a connect
scoring 60 takes the scan's early exit and discards the index build -
so the phys device itself cannot be the one that writes it.
Fixes#19540, a regression from the profile index added in 1d1f27a.
The index records each profile's identity tuples, ranks them all with
the real affinity function, then opens only the winner and re-scores
it against the file, accepting it if its real affinity is at least
what the index claimed.
That catches an index which overstates a candidate. It cannot catch
one which understates a candidate: an understated entry never becomes
the winner, so it is never the one re-read. Freshness was keyed on the
directory's *.cfg count alone, which is blind to an in-place edit, so
a profile edited to match a device stayed invisible while a rival
whose own claim was honest verified cleanly and was configured - at
its own lower affinity, without the directory being scanned at all.
The scan's >= 60 early exit never runs in that path, because the index
answers first.
The reporter has two profiles for pads sharing a vendor/product id,
told apart by input_phys. The wrong one is configured at affinity 40
where a scan gives the right one 60.
Key freshness on a fingerprint of the directory's *.cfg entries -
count, total size and newest modification time - taken from the
directory walk itself via retro_dirent_stat, which is free on Windows
and one fstatat per entry on POSIX. No per-file opens, so a hit still
costs the same two file operations. Additions, removals and in-place
edits all change it.
Modification times are whole seconds, so a size-preserving edit made
in the same second as the index write would fingerprint identically.
The index is written after the profiles it describes were read, so in
a fresh directory every profile is strictly older than it; require
that too. The walk picks up the index's own mtime at no extra cost. A
directory that trips this rescans once and rewrites the index with a
later timestamp.
Where the entry metadata is unavailable - a VFS without dirent stat,
or a frontend older than VFS API v5 - no index is written and none is
trusted, so those platforms scan in full as they did before the index
existed.
Index version bumped to 2: v1 files are keyed on the count alone and
cannot be trusted, so they are discarded on sight.
The renamed-or-deleted-winner open failure and the winner re-score
both stay as cheap second lines of defence.
Exercise an empty callback between native transport passes and require an idle result without changing audible continuity. Compare normal callback results with captured device output instead of assuming every source callback writes immediately.
Preserve the two-case matrix and the upstream callback-thread parking contract.
Display refresh changes may adjust the effective audio input rate. Derive both SRC ratios afterward so they use the new rate rather than retaining the previous display timing. Keep the DRC threshold update after the same adjustment.
With the device kept running behind the menu, the wrapper thread stays
in its loop while the core is paused, and on the callback path there
is no device write to pace on: audio_driver_callback() returned at
once and was called again, a hot spin at raised priority against the
main thread for as long as the menu was up. audio_driver_callback()
now reports whether the pass wrote to the device, and the wrapper
parks on its condition for a bounded interval when it did not; a stop
signals the condition, so it cuts the park short.
The handshake sample gains an idle lane: 6.7 million asks in 200 ms
before, 182 after, and a stop landing in the park returns at once.
The inline callback lane expects idle on a paused or suspended pass.
Swept under ASan+UBSan and TSan, clean. Closes#19541.
Run the existing native continuity cases on a persistent callback worker. Hold the source callback open while the main thread ends a frame, then verify the worker retains its pending accumulator and output remains identical to batch playback.
Use bounded condition-variable rendezvous instead of sleeps, retaining the existing case count and CI matrix.
Compare prepared inline transport output from uninterrupted batches with callback delivery interrupted by pause and suspension. Cover float/HQ and native int16 single-sample accumulation, exact output continuity, source gating and retained transport storage in two bounded cases.
Use the existing frontend test target without expanding CI jobs or sanitizer variants.
Every sync opens with OPTIONS, and the response to it is the only thing
that establishes the server speaks WebDAV. MKCOL answers 405 on a
collection that is already there (RFC 4918 9.3.1), and that 405 is
forgiven only for a verified endpoint, so the probe now runs whether or
not a username and password are configured: an endpoint reached
anonymously uploads into directories it created on an earlier sync
instead of failing on the first MKCOL and taking every PUT with it.
A 401 carrying a digest challenge is only worth retrying when there are
credentials to answer it with; without them the retry would meet the
same challenge for as long as the server kept issuing it, so cloud sync
reports the endpoint as needing authentication and stops. The
webdav_options sample drives webdav_sync_begin() through both cases and
asserts the probe is dispatched, with an Authorization header only where
one exists.
x11_display_server_modeline_enum() worked out a mode's line rate as its
dot clock over its horizontal total, in integer arithmetic. A virtual
or headless server lists its one mode with the timing left at zero,
and the division raised SIGFPE - on enumeration, before anything was
asked of the mode.
The mode is real; it is its rate that is unknown. It is kept, with its
rate left at the zero the memset gave it, and the division is only
done when there is something to divide by. A mode with timing is
worked out exactly as before.
Found by running the x11 display server sample under Xvfb, which is
such a server.
Reset released or paused fast-forward timing at inline frame boundaries, including builds without threads. Callback cores reset their own estimate on the audio thread; the main frame leaves it alone. Preserve the interval when an active core batches audio across frames.
Cover silent frames, callback ownership, pause and menu pause permission with a deterministic clock, without adding CI jobs.
cloudsync_savefile_owner_test pins the two properties the startup
rollback fix rests on: that save file ownership tracks the save file
list exactly, from the point content load builds it to the point the
deinit chain frees it, with matching that ignores case; and that
file_list_search() resolves a full portable manifest key to its own
entry rather than to a longer key that shares it as a prefix, which is
what lets the carry-forward recover the hash the last sync recorded.
The predicate reads a file-static list in save.c, so the test keeps a
verbatim copy of it and links the real file_list_search(). The CI Linux
samples/tasks workflow builds and runs it under AddressSanitizer
alongside the other task regression lanes.
Non-destructive mode promises a local copy of anything it displaces, so
task_cloud_sync_backup_file() now reports whether the file actually
reached the backup directory, and the fetch and delete paths act on that:
the local file keeps its place, the sync is marked as having failed, and
both manifests carry forward the hashes they already hold so the next run
sees the same difference and tries again. Destructive deletes check their
own result the same way.
task_cloud_sync_carry_manifests_forward() holds that manifest bookkeeping
in one place, shared with the live-savefile deferral. The backup lookup
also searches the current manifest by the full portable key, which is what
the manifest is keyed on, rather than by the path below the directory
prefix.
A loaded core holds its save RAM in memory and writes it back over the
file on disk when the content closes, so for as long as the content is
up the file is not the authoritative copy. content_savefile_is_live()
answers whether a path belongs to that set, and cloud sync consults it
before replacing or removing a local file: where the running core owns
the file the operation is deferred and both manifests carry forward the
hashes they already hold. The next sync runs once the deinit chain has
written save RAM out, sees the real difference between the two sides and
resolves it, raising a conflict where both of them moved.
This matters most when RetroArch is launched with content on the command
line, because the core loads its save RAM well before the startup sync
has a server manifest to diff against, and a file fetched into place
behind it is overwritten from core memory at unload and then uploaded in
place of the server's copy.
Reset the source timing seed, partial-frame count and published multiplier after a successful device stop. Resume starts at unity instead of treating the stopped interval as a slow source batch. Failed stops retain the running estimate.
Exercise stop/restart with a deterministic clock and cover the prepared native inline stage without extending the CI matrix.
Rebuild native stretch state when a core negotiates float or wide audio after initialization. Flush accumulated source first, reset retained history at the format boundary, and retain the wide arena on stereo return. Failed preparation falls back to ordinary playback until reinit.
Cover native format changes, late wide negotiation, repeated queries, accumulator flushing, allocation failure and teardown with a small frontend regression matrix.
The sibling and descent walks paid a full listing - enumerate, copy,
sort, free - for every folder they passed over, so stepping past the
end of a folder could run that up to once per sibling on the main
thread. Ask first with a stop-at-first-hit walk of the entries, which
allocates nothing, and pay for the listing only on the folder that
answers yes. Same folder chosen as before.
* shaders: step Next/Previous Shader from the loaded preset
The Next/Previous Shader hotkeys still followed the flat shader folder
they were written for. They listed the root of the Video Shaders
directory, or the menu's last browsed folder with "Remember Last Used
Shader Directory" on. With the shader pack layout that root holds no
presets, so the hotkeys did nothing useful.
They now:
- start from the loaded preset, following one #reference level for
RetroArch's own retroarch.* and auto-preset wrappers (a wrapper with
no #reference keeps the old behaviour)
- move to the next or previous sibling folder with presets at either
end of a folder, wrapping around the parent; the walk stays inside the
Video Shaders directory or a shaders_slang/shaders_glsl/shaders_cg
folder, so flatpak and distro installed packs work too
- list presets only, not single shader passes
- keep stepping from the list after a synchronous apply fails, so a
broken preset cannot trap the hotkey
No new settings.
* widgets: show the shader name and load progress in one notification
Shader load notifications showed only the preset's file name, and a
deferred load's progress appeared as a separate bar over it that only
flashed up for most presets. While flipping through presets the centred
name also moved with every change.
Shader messages now show the path relative to the Video Shaders
directory, or the file name for presets outside it, in a fixed layout:
a constant-width box, the name left-aligned at the same position with
long paths shortened from the left, and a reserved right-hand slot
where a dim "Loading N/M" label counts the passes of a deferred load.
The name shows as soon as a load starts and the label stays until the
box fades. Other generic messages keep their centred layout.
The synchronous and failure messages clamp their lengths, so a long
path is truncated instead of overrunning the message buffer.
* shaders: anchor only on the retroarch.* wrapper
The wrapper check matched any preset whose file name begins with
'retroarch', so a user preset such as retroarch-crt.slangp had its
#reference followed instead of being stepped from directly. Require
the extension separator after the name.
* shaders: bound the failed-apply anchor read
The anchor taken after a failed apply indexed the shader list without
checking the selection against its size or the entry for NULL, ahead of
the guard the rest of the function reads the list behind. Check both.
* shaders: honour the message flag on the deferred notification
The widget branch of the deferred load sat outside the message check,
so a caller that asks for a silent apply still got a notification with
widgets on - the shader delay timer after content load and the menu's
cold start are both such callers. Gate it like the queue branch beside
it and the synchronous path below.
* shaders: bring the hotkey path back under the stack frame limit
video_shader_dir_check_shader carried a 4192-byte frame with
PATH_MAX_LENGTH at 2048: the anchor scratch, the auto preset scratch
and the anchor directory all land in it once the static helpers
inline. Fold the anchor resolution into one helper, let the anchor
buffer double as the auto preset scratch, and scope the shader root
to the block that reads it.
Measured with -Wframe-larger-than at -O2: check_shader 4192 -> 3168,
init_sibling 3424 -> 2400.
* shaders: start cycling from the first folder that holds presets
With no preset loaded there is no anchor, so the hotkeys fall back to
listing the Video Shaders root - which holds no presets under the pack
layout, leaving the list empty and the hotkey inert from a cold start.
Walk up to three levels below the root and take the first folder that
lists presets under the context's shader extensions.
* widgets: binary search the fixed message fit
The fixed layout measured the message once per leading character it
dropped, and the name buffer is PATH_MAX_LENGTH, so each measurement
walks what is left of it. Search the cut point instead: a long path
costs a handful of measurements, and the result is the same one the
per-character walk arrived at.
---------
Co-authored-by: LibretroAdmin <[email protected]>
stretch_src_test covers each axis - channel count, native and int16,
ratio, HQ policy, bound capacity - once per combination rather than
every product of them, and the workflow runs the chain once.
Five samples link gfx_widgets.c, gfx_thumbnail.c or font_driver.c
against their own view of the frontend. Those files turn blending on
and off through gfx_display now, and send its batch out before text
and at the end of a frame, and the samples had no stand-in for any of
that: gfx_widgets_sanitize failed to link, and four more with it.
Each gets the ones it links against, in the terms its stub file already
uses - the real types where it includes gfx_display.h, void* where it
deliberately does not - and every one builds and passes as its CI step
runs it.
Prepare canonical native transport storage for multichannel cores and stretch before splitting or downmixing. Reset retained history at layout boundaries, preserve ordinary speed fallback, and reserve extra-channel input capacity for bounded output blocks. Correct the shared 7.1 packer so side channels occupy speaker slots 9 and 10. Verify native channel coherence, pitch, duration, sparse layouts and handoffs.
* task_database: fix memory errors in database scans
DATABASE_SCAN_ITERATE_NEXT handed an exhausted index to
MANUAL_SCAN_ITERATE_CONTENT, which read one entry past the end of the
content list. Since 914f0e26f1 stopped zeroing a grown list's spare
capacity that entry is stale, so a scan could crash on it. Go straight
to the M3U or end state instead.
A folder with no scannable files read entry 0 of the empty list and
ended without reaching MANUAL_SCAN_END. Reject it once the walk
finishes, as the non-database scan already does.
A cue or gdi sheet that names itself freed its own list entry while
pruning its tracks, and the scan then read that path for its serial.
Skip the sheet's own path.
The scan test covers each case under AddressSanitizer, and includes
file/file_path.h for path_mkdir, which GCC 14 rejects as implicit.
* ci: run database_scan_test in the samples/tasks lane
The scan test only builds under 'make check', which no workflow
invokes, so its coverage never runs. Give it its own ASan + UBSan step
next to scan_begin_budget_test: the full-content-list, empty-folder
and self-naming-sheet cases are bounds and lifetime properties a plain
build walks past, and LeakSanitizer takes the scan teardown with them.
---------
Co-authored-by: LibretroAdmin <[email protected]>
A menu driver brackets a run of draws with blend_begin and blend_end and
draws icons inside it. A gathered quad going out in the middle of that
run turned blending on and off around itself, and the blend_end ended
the driver's run as well: every icon after it drew opaque, and came
right only when the highlight redrew it. Quads drawn one at a time did
the same thing, but nothing was ever gathered between two of them.
So the eighty-eight places that turned blending on and off themselves
go through this file now, as the draws already do, and a batch only
turns blending on if nobody else has. Whether it is on is kept as what
it is - a state the driver is in - rather than as a count of how many
callers asked for it: a path that returns between a begin and its end
leaves a count wrong for the rest of the frame.
Ozone renders the same as before it gathered anything, on gl, glcore
and Vulkan, in the main menu and after moving up, down and left.
A menu quad was a draw of its own: ozone asked for twenty-three of them
a frame and got twenty-three viewport sets, twenty-three pairs of
blend_begin and blend_end, and on Vulkan twenty-three uniform buffers
and descriptor sets - for quads that between them used two textures.
They are gathered now, in the strip order the drivers already draw in,
and go out when the texture changes or when anything else draws.
Ozone draws eight times a frame instead of twenty-three, materialui six
instead of fourteen. What is left is the text: it is drawn by the font
driver rather than from here, so a batch has to go out before each of
it, and ozone flushes five times a frame for that.
What was gathered goes out before anything else reaches the driver:
before a slice, a cursor, a scissor change or a line of text, and at
the end of the menu's frame and the widgets' frame so nothing is still
waiting when the frame is over. Every caller outside this file already
reaches the driver through one function, which is what makes that list
a list rather than a hope.
Prepare bounded stereo transport before playback and feed it from both native batch lanes before SRC. Retain borrowed input only within the callback, preserve source cadence for fast-forward, and reset on stop or unsupported-layout/format fallback. Apply the optional speed LPF in native output storage. Keep ordinary playback for multichannel inline sources and preserve existing device-write semantics.
Let inline owners retain read-only native source views while the stream is inactive, preserving partial-consumption and EOF ownership without a dry-path copy. Keep active synthesis in the existing bound output buffer. Verify copied-path equivalence, source lifetime, reset and zero processing allocations.
ec_sleep declares DWORD rc after the timeout conversion block, which is
a mixed declaration. MSVC 2005 compiles C as C89 and rejects it:
retro_eventcount.c(367) : error C2275: 'DWORD' : illegal use of this
type as an expression
retro_eventcount.c(367) : error C2146: syntax error : missing ';'
before identifier 'rc'
retro_eventcount.c(367) : error C2065: 'rc' : undeclared identifier
Declare it alongside ms at the top of the block and assign it where it
was. No behaviour change.
Checked the file and the rest of the recently touched sources with
i686-w64-mingw32-gcc -std=c89 -Wdeclaration-after-statement: this was
the only violation in anything griffin pulls in. The three remaining
hits in the tree are in cores/libretro-net-retropad/net_retropad_core.c,
which griffin does not build, so they are not what stopped this job.
Reserve the bounded wide-source staging capacity during threaded pipeline initialization, avoiding callback-time growth as core batch sizes increase. Keep allocation failure transactional and the stereo pipeline unchanged. Exercise failed reservation, varying native 5.1/7.1 batches and teardown; use the same preparation in the real-wrapper fixture.
Retain native transport storage across unsupported-speed fallback. Resume at a supported speed only after source and pending device output drain, parking the worker for the ownership and metadata handoff. Keep publishing legacy audio while recovery is deferred.
Prepare suspended storage when configured startup encounters an unsupported speed. Cover recovery, pending-output deferral, native format changes and suspended release/teardown in frontend and real-wrapper tests.
Every step in this workflow built its sample serially. The heavier ones
compile 35-odd translation units per sanitizer variant, twice - once for
ASan+UBSan, once for TSan - so pipeline_rate_control alone spends around
70 single-threaded compilations before it runs anything, on a runner
with four cores sitting idle. Across the file that is roughly 875
object compilations that had no reason to be serial.
Not written as "make -j clean all": with -j the clean goal can run
concurrently with all and delete objects as they are produced. Split
into "make clean && make -j$(nproc) ...", which also propagates a clean
failure regardless of whether the step set -e.
Parallel-safe as it stands - no Makefile under samples/audio has two
rules writing the same object, and the sample Makefiles that skip
objects entirely compile each binary straight from its sources.
Verified by building samples/audio/pipeline_rate_control with -j4 from
clean: 35 objects, exit 0, no missing-rule errors.
This does not touch the real-time scenarios. pipeline_rate_control
deliberately runs its device against wall-clock - 28 s for the steady
case plus five shorter ones, then 28 s again under TSan - and that floor
of roughly two minutes is the test doing what it is meant to do, not
overhead to remove.
Capture the shipping native frontend through bounded transport steps and partial device writes. Check pitch, duration, channel coherence and source accounting across six tempos, both native formats, stereo/5.1 and normal/HQ sinc. Register the oracle in the existing plain and sanitizer sample target.
Three problems with this suite, all of them structural rather than any
one test being wrong.
No feedback. stretch_epoch_test and stretch_pipeline_test each sweep a
64-case matrix and print a single summary line at the very end, so the
CI log sits on "./stretch_epoch_test" for minutes with nothing after it.
There is no way to tell a slow run from a wedged one, which is exactly
how the recent ring-size stall presented. Both now print one flushed
line per case with the parameters, so the log streams and a stall names
the case it stalled on.
No containment. check ran the eight binaries as eight recipe lines in
one target, so a single hung test consumed the whole job budget and the
35 steps behind it in this workflow never ran at all. check is now a
per-binary target with a 600s timeout each, overridable via TIMEOUT (set
it empty on hosts without coreutils timeout). A hang now fails in ten
minutes naming the binary, instead of killing the job at 25 with no
attribution.
No parallelism. The eight binaries are independent processes but ran
end to end, and the workflow invoked make without -j, so both the
compiles and the runs were serial. Splitting check into per-binary
targets makes make -j work for both, and the workflow now passes
-j$(nproc).
The runtime itself is not fat to trim: it is linear in channels x frames
and the 11-channel half of the matrix is 255s of stretch_epoch_test's
310s on a single core, with the 2-channel half at 53s. That is real
work, so the fix is to stop running it end to end rather than to cut
coverage.
Register default-off transport and low-pass settings with audio reinitialization. Prepare the automatic native transport before playback, preserving the ordinary pipeline on unsupported configurations or allocation failure. Keep cold startup out of line and preserve existing boolean field offsets.
Cover configured startup and failure in both native lanes and enter the real-wrapper automatic tests through the same settings helper.
concurrent_direct() runs an 11-channel producer/consumer stress over a
256-byte SPSC ring. At 11 channels a native frame is 44 bytes, so the
ring holds five of them; for int16 it holds eleven. The producer writes
one frame at a time and busy-spins when the ring is full, the consumer
takes at most three per call and busy-spins when it is empty, and
neither side yields. Every one of the 200000 frames therefore becomes a
full handoff with both threads spinning on the same head/tail cache
line, with ASan instrumenting each access.
On one core that is merely slow, because the two threads timeshare. On
a multi-core runner they spin in parallel and the cache line ping-pongs
between cores for nearly every frame, which is what has been wedging
the samples/audio job until it hits its timeout - it never got as far as
printing the first result line.
The frame-size dependence is the giveaway: with the 256-byte ring,
concurrent_direct(0) (22-byte frames, eleven slots) took 72s and
concurrent_direct(1) (44-byte frames, five slots) took 158s on the same
box. Halve the slots, double the time.
4096 bytes gives 93 frames of headroom so the producer can run ahead and
the handoffs amortise. Coverage is unchanged: the index wrap is forced
by origin = SIZE_MAX - 127 regardless of ring size, and 200000 frames
through 93 slots still wraps the buffer about 2150 times. Measured on
one core, both passes reporting 0 failures and 0 guarded heap calls
either way:
ring 256: cd0 72s, cd1 158s (230s total)
ring 4096: cd0 5s, cd1 8s ( 13s total)
Not a regression from the recent audio work - 99d852a, before fc91b3e /
8ae61ca / a62b57c / 3ba9724, measures the same throughput. The test has
always been this expensive; the job timeout is what finally surfaced it.
Add explicit opt-in startup for automatic frame-boundary transport controls. Resolve the current runloop state before the first source publish after frame end; retry metadata pressure without publishing source. Unsupported speed parks the worker and releases optional DSP while retaining queued native source for legacy playback. Preserve the mode through native format rebinds and clear it on release. Test producer backpressure, fallback and restart through the real wrapper and register the mode in CI.
The workflow tree has grown to 59 files / 67 jobs, which expand to 84
runner-jobs on a push to master. Wall-clock has been dominated by
queueing rather than by any individual job getting slower. Six
independent changes, squashed:
1. Concurrency groups on every workflow. Not one of the 59 declared
one, so every push to a branch left the previous run's 84 jobs going
to completion, holding slots in the org-wide concurrent-job pool that
the rest of libretro also queues against. Group on the PR number
where there is one and on the ref otherwise, and only cancel outside
master so per-commit status on master stays complete. SourceRelease,
both crowdin workflows and webOS are left alone - they publish
artifacts, and cancelling one midway leaves that half-done.
2. Push triggers restricted to master on the 26 platform workflows that
declared a bare "push:" alongside "pull_request:". Pushing a branch
with a PR open fired both, so every push to a same-repo PR branch ran
37 duplicated jobs. The Linux-* workflows already filter both
triggers to master. Pull requests still build, master still builds,
and the buildbot's repository_dispatch run_build path is untouched.
3. update: true dropped from the MSYS2 setup. setup-msys2 does not
update the installation by default and already caches the base
distribution (its cache option defaults to true). Forcing a full
pacman -Syuu on each of the four matrix legs, before the ~25 qt6-*
pacboy packages are installed, leaves the refreshed package set no
longer matching what was cached, so much of the install is refetched
every run. This is the longest-running job in the tree.
4. ccache for the sample harnesses. The samples deliberately compile
shared sources into the same object paths under gfx/ and
libretro-common/, so each has to be cleaned before the next builds -
hence the "make clean" in samples/Makefile and in the CI steps. The
cost is that the suite recompiles the same translation units over and
over: across the 104 sample Makefiles there are 1519 object compiles
for 384 distinct objects, and 184 of the 206 make invocations in
these workflows are single-threaded. ccache sidesteps the collision
without restructuring any Makefile - clean removes the .o files but
the cache still hits, and the SANITIZER=address,undefined /
SANITIZER=thread variants get their own entries instead of fighting
over one object path. Measured on samples/tasks/database (61
objects): 14s cold, 0s warm after a full clean. The sample Makefiles
resolve the compiler through PATH, so putting /usr/lib/ccache ahead
of it is enough.
5. timeout-minutes on every job. 29 workflows had none, including every
platform build, both Windows lanes and both macOS ones; a job that
wedges there sits for the full six-hour default holding a slot. 30
minutes for the container and Linux builds, 60 for macOS/iOS and the
PPC cross, 90 for MSVC/MSYS2/Windows-ARM64 where toolchain setup
dominates. Backstops, not targets - tighten once there is a
run-duration baseline to tighten against.
6. The dead "Get short SHA" steps removed from eight workflows. They
compute steps.slug.sha8 and nothing references it; there is no
upload-artifact after them, unlike MSVC where the slug is used in the
artifact name. They were also the last users of ::set-output in
those files, deprecated since 2022.
All 59 workflows parse under a duplicate-key-strict loader. actionlint
1.7.7 reports the same two findings before and after (the stale
windows-11-arm label warning in MSYS2.yml and Windows-ARM64.yml).
Prepare native transport and its initial runloop tempo/cutoff in one parked-worker transaction. Validate unsupported speed before allocation and retain the previous session on failure. Reuse the speed resolver and keep existing preparation semantics intact. Cover failure preservation and pre-source control readiness, and use the transaction in the existing real-wrapper runloop startup/restart suite.
Every sample Makefile in the tree was built - 42 under samples/audio,
25 under samples/gfx, 20 under samples/tasks, the twelve smaller
areas, and all 48 under libretro-common/samples. Two failed to link,
and nothing else did.
font_lifecycle did not stub video_driver_thread_wrapper_active, which
font_driver.c asks before routing a font call through the threaded
wrapper. That sample's whole subject is what the wrapper does to a
font's lifetime, so it answers as its neighbouring
video_driver_is_threaded() stub does.
latency_contract linked rthreads.c but not retro_eventcount.c, and the
audio driver it exercises parks on an eventcount now. Its own source
list, so the tree-wide fix did not reach it.
And three built executables were committed: emscripten_mmap_test,
retro_spsc_test and font_lifecycle's lifecycle. Each is its Makefile's
own build target, so any make clean in those directories showed up as
a deletion of a tracked file - which is how they surfaced, during the
sweep above. Removed from the index and from the tree, and ignored,
along with the two other binaries font_lifecycle builds. All three
rebuild from source and the tree stays clean afterwards, which was
checked rather than assumed.
That is the third time a committed binary has turned up this way; the
ignore rules now cover libretro-common/samples as well, which they did
not before.
retro_reset passes RETROK_LAST, the element count, as the byte length
to memset. sizeof the array is what the call wants and is what
retro_deinit already gives it.
IORegistryEntrySearchCFProperty declares its plane parameter as
io_name_t, which is char[128], so a string literal there is a ten-byte
object where the prototype says 128 bytes are read and GCC reports an
overread. The DCP EDID walk hands it a file-scope constant of the
declared width instead; it lands in rodata and the call is unchanged.
Compose slow-motion duration with the producer fast-forward estimate while respecting audio_fastforward_speedup. Publish reciprocal Q16 tempo and optional cutoff as one request, with unity for paused or normal playback. Reject unsupported and invalid ratios without changing queued controls. Keep preparation and fallback explicit; no automatic default-path caller is added. Exercise the resolver and real-wrapper transitions in the existing frontend and CI fixtures.
Derive an optional core-rate cutoff from Q16 speed using bounded integer arithmetic. Keep unity and slow motion dry, and publish the derived cutoff with tempo at one source boundary. Filtering remains independent of WSOLA activation. The existing explicit-Hz API and default playback path are unchanged. Cover the policy, request backpressure and native wrapper transitions, and register wrapper policy checks in CI.
Keep the device at 7.1 while switching stereo, rear/side 5.1 and 7.1 source layouts alongside tempo and filter controls. Check retired metadata and extra-channel positions while the worker is parked, without publishing a reset. Include all four sample-format pairings in the native transport CI job.
ui_qt_widgets.cpp puts the companion's one-line help on its rows with
wrapped_tooltip(), which is defined under HAVE_MENU, from a block that
is not. A build configured --disable-menu with Qt present does not
compile. The headless job configures exactly that and passes because
its runner has no Qt, so the two halves of the break never meet there.
The block is the menu's text, so it goes under the menu's guard.
The vtable check reads the driver structures that are filled in
positionally and compiles each against its interface. It matched names
of letters only, so audio_sdl3, microphone_sdl3, audio_ps3, audio_ps2,
audio_xenon360 and camera_v4l2 were walked past - six drivers, in the
check written because three drivers took a silent shift. Digits count
now, and the display drivers are read too: they fill in a vtable of the
same shape, eighteen of them, and a field was added to it this week.
gfx_display_vk_draw() overwrites vk->vk_vp with each element it draws,
which is right for its own vkCmdSetViewport but leaves vk->vk_vp
holding the last menu icon or widget quad once the menu pass has run.
Both vulkan_check_swapchain() call sites then read it: one at the top
of vulkan_frame(), where it carries over from the previous frame, and
one after the swap, where the menu and widget passes have just run.
That value becomes chain->swapchain_info.vp, which
slang_pass_set_pass_info() feeds to slang_pass_get_output_size() for
GLSLANG_FILTER_CHAIN_SCALE_VIEWPORT passes. Changing resolution,
toggling fullscreen or flipping present mode -- all done from the menu
-- therefore sized every viewport-scaled non-final pass to whatever
quad was drawn last, until slang_pass_build_commands() recomputed from
the live viewport and resized on the next frame. The final pass,
usually the viewport-scaled one, allocates no framebuffer at all,
which is why this stayed quiet.
Keep the presented viewport in its own field, written only by
vulkan_set_viewport() and never by a draw call, and read that
everywhere the filter chain is handed a viewport. The two in-frame
hand-offs move over too: they are correct today only because
vulkan_set_viewport() runs a couple of lines earlier, which is not an
invariant worth depending on.
No functional change to the rendered viewport, and no added work per
frame: the read sites still copy a ready VkViewport, and the two
stores in vulkan_set_viewport() vectorise into one computation.
Exercise 5.1 and 7.1 canonical source lanes against matching device layouts, across all sample format pairings, dry/LPF/stretch modes and live control changes. Verify native extra-channel state after restart and retain conversion-direction assertions. Run the wrapper tempo and wide suites in a separate plain and ASan/UBSan CI job.
slang_chain_flush() calls vkDeviceWaitIdle() and the LUT upload calls
vkQueueSubmit(), both on info->queue, with nothing held. That queue is
not private to the video thread: vulkan_lock_queue/vulkan_unlock_queue
are handed to cores through retro_hw_render_interface_vulkan, and a
Vulkan HW-render core submits to it from retro_run() on the emulation
thread.
With threaded video, a shader change is marshalled as CMD_SET_SHADER
and runs vulkan_filter_chain_free() -> slang_chain_flush() on the video
thread while the core may be inside its own submit. vkQueueSubmit
requires the queue to be externally synchronised and vkDeviceWaitIdle
is specified as vkQueueWaitIdle on every queue, so this is an
unsynchronised concurrent access. vulkan_check_swapchain() already
takes queue_lock around its vkQueueWaitIdle; the filter chain had no
way to, since it is given the queue but never the lock.
Pass the lock down as a handle plus two callbacks so shader_vulkan.h
stays free of rthreads and of HAVE_THREADS. Both may be NULL.
The flush holds the lock across the wait because nothing weaker is
correct; that blocks vkQueuePresentKHR for the duration, which is the
TDR hazard vulkan_common.c documents, but every caller is a chain
teardown or rebuild rather than a per-frame path. The LUT upload drops
it before vkWaitForFences for the same reason -- only the submit needs
the queue synchronised.
* vulkan: keep shader history and feedback across chain rebuilds
Where mailbox is not emulated, toggling fast-forward with vsync on
switches the present mode, which recreates the swapchain and rebuilds
the slang filter chain. The rebuild deleted and cleared the
OriginalHistory and PassFeedback framebuffers, so presets that read
them, such as the handheld LCD response-time shaders and the bloom
CRT shaders, flashed on every fast-forward press and release.
Neither depends on the swapchain: history is sized to the input and
feedback to its pass, and only the final pass, which cannot have
feedback, renders in the swapchain format. The rebuild now keeps
both, clears only buffers it has just created, and leaves a pending
first-frame clear in place.
* vulkan: drop feedback framebuffers a pass no longer needs
slang_pass_build() no longer deletes pass->fb_feedback, which means a
pass that stops reporting a PASS_FEEDBACK semantic across a chain
rebuild keeps its buffer forever. slang_pass_end_frame() would go on
swapping that stale buffer with pass->framebuffer every frame, while
only pass->framebuffer is resized in slang_pass_build_commands().
Reflection is stable across rebuilds of a given chain, so this should
not trigger in practice, but the unconditional delete in
slang_pass_build() made it structurally impossible. Restore that
guarantee cheaply in the one place that knows whether a pass still
wants feedback.
* vulkan: document the invariants the chain-rebuild reuse relies on
Two of the changes here read like unrelated cleanup and will be
reverted by whoever touches this next.
Dropping 'chain->require_clear = false' from slang_chain_init() and
slang_chain_finalize() is load-bearing: a rebuild can land before the
first frame has consumed a pending clear, and resetting the flag then
leaves history and feedback with undefined contents.
The history early-return keys on the buffer count alone, which is only
safe because the buffers depend on chain-lifetime state and are
re-sized per frame in slang_chain_update_history(). Spell both out.
* vulkan: only log feedback framebuffer creation when one is created
The chain is now rebuilt on every fast-forward press and release where
mailbox is not emulated, so this non-debug log fires twice per toggle
per feedback pass while reporting work that no longer happens. Move it
into the branch that actually allocates the buffer.
* samples/audio: link retro_eventcount into device_list_driver
The harness includes audio/audio_driver.c with HAVE_THREADS, so the
threaded pipeline's eventcount waits are part of the build and
libretro-common/rthreads/retro_eventcount.c belongs in the link set.
---------
Co-authored-by: libretro <[email protected]>
Co-authored-by: LibretroAdmin <[email protected]>
The harness includes audio/audio_driver.c with HAVE_THREADS, so the
threaded pipeline's eventcount waits are part of the build and
libretro-common/rthreads/retro_eventcount.c belongs in the link set.
The sink estimate's warn-once flags live in two words, one per owning
thread: sink_warned for the estimator (sink_window, sink_apply) and
sink_warned_flush for the flush (sink_refused). On the threaded
pipeline the estimator runs on the core's thread from submit and the
flush on the audio thread, and |= is a read-modify-write, so each word
is written only by the thread that owns it and no flag lives in both.
samples/audio/sink_rate reads the DROPPED flag from the word that now
holds it.
Swept: samples/audio/pipeline_rate_control clean under TSan and under
ASan+UBSan in all five scenarios; samples/audio/sink_rate clean under
ASan+UBSan.
A third sample fails to link on gfx_display_draw: gfx_thumbnail_anim,
through gfx_thumbnail.c this time rather than gfx_widgets.c.
That is twice now that I fixed the sample named in the failure and
went no further. The first sweep looked for samples linking
gfx_widgets.c, which is the file the first two failures came from, and
missed that gfx_thumbnail.c calls it too.
Done properly this time: every file under gfx/ and menu/ that calls
gfx_display_draw, then every sample Makefile that builds any of them.
Six samples, of which two had the stub, two needed it -
gfx_thumbnail_anim and gfx_thumbnail_preview - and two do not build a
caller at all. All six link now.
The two new stubs take void* for the two pointer parameters rather
than their real types. These files declare their own view of the
frontend, and including gfx_display.h to name the types drags in a
video_driver.h whose declarations conflict with theirs; C linkage does
not carry parameter types, so the symbol matches what gfx_thumbnail.c
calls either way. The comment says so, because a stub whose signature
does not match its header looks like a mistake otherwise.
retro_eventcount.c defines _POSIX_C_SOURCE at the top so that struct
timespec and syscall() are visible under a strict C89 compile. It did
so unconditionally, and that is the cause of the macOS failures rather
than anything in the files that failed.
Darwin does not need it - that surface is visible by default - and
asking for it does harm: _POSIX_C_SOURCE lowers __DARWIN_C_LEVEL,
which hides the BSD names. In a normal build the define would stop at
the end of this file. griffin is one translation unit, so it applies
to every file included after it, and this one is included early.
IFF_UP out of net/if.h and RTLD_DEFAULT out of dlfcn.h are two that
then vanish, in files that never asked for any of this.
rthreads.c, which asks for the same baseline for the same reason,
has guarded it with #ifdef __unix__ all along - and macOS does not
define __unix__ - which is why it never caused this and the new file
did. The same guard is here now.
Demonstrated rather than argued: a file that defines _POSIX_C_SOURCE
and then includes net/if.h cannot see IFF_UP; the same file without
the define reads it as 1. That is the whole failure, reproduced on a
host that is not Apple.
Every other namespace define in a .c file griffin may include was
checked: nbio_linux.c, wayland_common.c and xshm_gfx.c all widen the
namespace rather than restrict it, and none of the three is built on
Apple.
The IFF_UP fallback added a few commits ago stays. It is belt to this
braces: where the platform exposes the symbol the platform's own
definition wins, and if some other file ever restricts the namespace
again, that file keeps building.
The macOS build still fails on IFF_UP, now with -std=gnu99 in the
compile line, so the language standard was not the cause and my
previous change did not fix this. It was worth making on its own terms
- the iOS project has been on gnu99 for a while and the Metal one was
the odd case - but it is not this.
What is true regardless of the mechanism: IFF_UP is 0x1 on every
system that has interface flags, BSD, Linux and Darwin alike, and the
only question is whether the header exposes it. Apple hides it with
the rest of the BSD names when the strict POSIX namespace is in
effect, and griffin is one translation unit, so whether that is in
effect by the time this file is reached depends on what was included
before it. libretro-common/rthreads/rthreads.c and retro_eventcount.c
both define _POSIX_C_SOURCE at their top, and in a unity build a
define does not stop at the end of the file that made it.
So the flag is defined when it is missing rather than chasing that
ordering. Where the platform declares it, the platform's definition
stands and this does nothing: checked by compiling the same guard
under both standards on a host that has the header, where gnu99 takes
the system's and c99 takes the fallback, and both give 1.
That leak is worth a proper look - a file's namespace request reaching
files it was never meant to affect is a hazard beyond this symbol -
but it is a separate change from unbreaking the build.
The macOS build fails on IFF_UP being undeclared in net_ifinfo.c,
twice, with net/if.h included right above it.
Strict C99 is why. Darwin defines _POSIX_C_SOURCE for it and hides the
BSD names behind __DARWIN_C_FULL - IFF_UP among them - and this
project was the only Apple one still asking for c99 while
RetroArch_iOS13 already builds C as gnu99 and gnu17. That is also why
iOS did not fail here and macOS did.
Both configurations in the Metal project now say gnu99, which is what
the rest of the tree builds with. Demonstrated rather than assumed: a
file including net/if.h and printing IFF_UP does not compile under
-std=c99 and does under -std=gnu99, on a plain POSIX host.
This is the same root cause as the RTLD_DEFAULT failure fixed a few
commits ago, from the other side. That one is a local #ifndef and
stays - it defers to the platform where the platform provides the
symbol, so it costs nothing now that the namespace is open, and it
keeps that file building if a target ever tightens again.
No other Apple project asks for c99, checked.
retroarch.c declared mic_st and used it nowhere - one mention in the
whole file, the declaration itself. Removed, and the HAVE_MICROPHONE
guard around it with it, since it held nothing else. Compiles with the
define on and off.
bignum.c's t is not the same case and must not be deleted. It is an
asm operand of the x86 MULADDC_STOP variants in bn_mul.h, and is named
by nothing on the paths whose macros do not use it - arm64 and the
generic C fallback among them - which is why one architecture warns
and the others do not. Removing it would fix a warning on Apple
silicon by breaking every x86 build.
So it is marked used instead, with a note saying which variants need
it. The x86-64 object still compiles here, which is the path that
would have broken.
My first look at this counted 384 occurrences of t in bn_mul.h and
concluded it was used everywhere; they were tab escapes inside asm
strings. The second look asked where t appears as an operand, which is
the question that had an answer.
A threadless build now fails to link on slock_new, scond_new,
slock_free and scond_free, out of retro_eventcount.c. That is my fault
twice over.
The first failure was a threadless build compiling
retro_waitable_spsc.c, which references the eventcount, while the
eventcount was under HAVE_THREADS. I moved the eventcount out of the
guard and wrote that it has no thread dependency of its own - that it
degrades to a spin with no lock or condition variable to hand. That is
false. retro_eventcount.c calls slock_new and scond_new directly, in
twenty-odd places, and rthreads.c is built only under HAVE_THREADS. I
asserted the opposite without reading the file I was moving.
Both go under HAVE_THREADS now, which is the direction that was always
right: a queue whose two ends can sleep is not a thing a build without
threads has any use for. Nothing in the frontend uses it yet, so
gating it costs nothing today.
Checked: both includes are inside the guard in griffin, every
Makefile.* parses that parsed before - the thirteen that do not need
their own toolchains and failed identically beforehand - and
libretro-common's own test suite still builds.
The iOS and tvOS builds fail on every dlsym in this file: RTLD_DEFAULT
undeclared, twenty-odd times.
Apple's dlfcn.h defines it only when the full BSD namespace is visible
- __DARWIN_C_FULL - and this file goes through griffin as C, where
those targets do not get that. Every other dlsym(RTLD_DEFAULT, ...) in
the tree is in an Objective-C file, which does, which is why the
runtime resolution added here was the first thing to hit it.
Defined behind #ifndef, so where the platform provides it the
platform's own wins: checked by compiling the same guard on a host
that defines it, where the fallback stays unused. The value is Apple's
((void*)-2) rather than glibc's 0, and this file builds on Darwin
only.
One other C file has the same pattern - rthreads.c, inside its
__APPLE__ branch - and it is not failing, so something in its include
path is opening the namespace that this one's does not. Left alone on
that basis rather than changed blind; if it ever does fail, it wants
the same three lines.
Compiles clean here in both TARGET_OS_IPHONE configurations against
the stub headers.
The built-in reader was opt-in and almost nothing opted in. Only
Makefile.webos set HAVE_RCHD; the configure default was no, so every
configure-based build took libchdr, and ten platform makefiles enabled
HAVE_CHD without ever mentioning rchd - dingux, dos and its salamander,
lfx000, libnx, miyoo, orbis, retrofw, rg350 and rg350_odbeta.
So the answer to "which platforms still use libchdr" was: all of them
but one.
The configure default is yes now and those ten set it, which leaves no
build in the tree enabling CHD without rchd. Where HAVE_RCHD is set,
Makefile.common compiles formats/chd/rchd.c and none of libchdr at
all - confirmed by configuring here and finding no libchdr object in
config.mk.
What the reader needs is already handled in that branch: deflate and
LZMA unconditionally, FLAC where HAVE_FLAC or HAVE_RFLAC is on, zstd
where HAVE_RZSTD is, and encoding_huffman always, because a version 5
map is Huffman-coded and is needed to open an image whatever its hunks
use.
Worth saying plainly: nothing exercises rchd automatically.
libretro-common/tools/chd carries six rchd_*_test.c programs, and none
of them has a Makefile or a workflow step, so this flip rests on the
reader being right rather than on anything measured. Wiring those up
is the obvious next thing.
Every makefile I touched still parses - clean is reached on all of
them. The thirteen that do not parse here need their own toolchains
and failed identically before this change, which was checked rather
than assumed.
Makefile.common:359 is a missing-separator error, and it is mine: the
comment explaining why retro_eventcount.o sits in the unconditional
list was written between two continued lines of the OBJ assignment. A
comment cannot appear inside a backslash continuation - make sees the
'#' line as a rule and stops - so every build that includes this file
failed to parse, emscripten and miyoo among those reported.
The comment says the same thing above the assignment, where a comment
is legal. The object list is unchanged.
Checked rather than assumed: no comment line anywhere else in this
file sits inside a continuation, and every Makefile.* in the tree now
parses - clean is reached on all of them, including the two that were
reported.
Warnings from the Apple logs, all of them in code this tree carries.
Four comma operators, three in libchdr's CHD map reader and one in its
huffman decoder. Two of them hid something worth seeing: the RLE_LARGE
case made two comma-separated assignments to repcount, so the high
nibble had to be read before the low one and the order was not
apparent from the line. Written out as statements in the same order
they ran in.
features_cpu.c had one of its own: the CRC32 probe tries an older
sysctl key and then a newer spelling, and reset the buffer and its
length inside a short-circuit || with commas. Written as an if/else,
which is what it was already doing.
Three unused variables. glslang_util.c declared a loop index nothing
used. libchdr declared a codec pointer outside the
HAVE_ZLIB/CHD_USE_BUILTIN_DEFLATE guard that is the only thing using
it, so every build without either - the Apple ones - carried it unused;
moved inside, and checked to compile clean both with the define and
without.
libchdr is vendored, and my first answer was that it is upstream's to
fix. That was the wrong call: it is built here, the warnings are in
this tree's builds, and none of these changes alter what the decoder
does. Left alone: the MAX/MIN redefinition warnings out of
libchdr/minmax.h, which are a header conflict rather than a defect and
want a real look at include order.
All four files compile clean, and libretro-common's own test suite
builds.
Same missing stub as gfx_widgets_sanitize, in the sibling sample:
gfx_widgets_draw_icon() reaches the display driver through
gfx_display_draw() rather than one of the helpers, and only the sample
named in the first failure got the stub.
Both build and pass; gfx_widgets_msg_queue_race does not link
gfx_widgets.c at all and needs nothing.
Six copies of the same warning across the iOS, tvOS and macOS logs:
a stored AVSpeechSynthesizer in a Sendable-conforming class.
The conformance is not written in the source. AVSpeechSynthesizerDelegate
is annotated Sendable in recent SDKs, so conforming to it makes this
class Sendable, and AVSpeechSynthesizer is not - which is exactly what
the compiler is reporting.
Declared @unchecked, and the class marked final while the type was
being touched. The alternative was to exempt the one property with
nonisolated(unsafe), which is narrower but says less: what has to hold
here is a property of the whole object, and it needs a newer Swift
than @unchecked does.
The comment says what is being asserted and what is not. One of these
is created per request - a single construction site, checked - kept
alive by one global reference and driven from one place; the
synthesizer is touched by synthesize() and by the two delegate
callbacks, which AVFoundation serialises against each other. It is
NOT safe to share: completion and audioBuffers are plain stored
properties with no lock, so if this ever outlives a request or gains a
second caller the assertion stops being true, and the answer then is
isolation rather than a wider @unchecked.
No runtime behaviour changes and no availability changes: the
@available line is untouched, and @unchecked Sendable is a
compile-time conformance rather than anything the OS sees.
Not compiled here - there is no Swift toolchain in this container - so
the Apple lanes are what confirm it.
The DOS and PSL1GHT builds fail to link, both on
retro_waitable_spsc.c:185 wanting retro_eventcount_cancel_wait.
griffin includes retro_waitable_spsc.c unconditionally and included
retro_eventcount.c only inside #if defined(HAVE_THREADS). Every build
without threads therefore compiled a queue whose parking half was
missing, and found out at link time.
The eventcount has no thread dependency of its own - it names
HAVE_THREADS nowhere and compiles standalone - because with no lock or
condition variable to hand it degrades to a spin, which is exactly
what a build with no threads needs from it. So it goes beside the
queue that uses it rather than behind a guard about something else.
That accounts for two of the three failures in the same report. The
third, the gfx_widgets_sanitize link, was the missing gfx_display_draw
stub and is already fixed.
Both harnesses in that report pass here plain and under ASan and
UBSan with leak checking on, and retro_waitable_spsc.c compiles with
HAVE_THREADS undefined.
Aggregate source frames before ring writes, including dropped source, then publish the speed estimate at frame end. Scanline batching no longer changes the estimate or runs its clock/division work per publish.
Retain sparse audio intervals while fast-forward is active and exclude paused or released gaps. The inline estimator is unchanged. Cover batching, dropped source, sparse delivery and re-entry with a deterministic clock.
Ten places outside this file called dispctx->draw() themselves - six in
xmb, two in materialui, one each in ozone, the widgets and the
thumbnails - so gfx_display, which knows what order the menu is drawn
in, did not see them. They call gfx_display_draw() now, which for the
moment does what each of them did: check the pointer and pass the draw
along.
It is what has to be true before quads can be gathered rather than
drawn one at a time. A gathered quad has to go out before anything
drawn after it, and a caller that reaches past this file is a caller
whose draw arrives in the wrong order. There is no such caller now.
Nothing is gathered yet and nothing about what is drawn changes: the
four menu drivers render byte for byte identically on gl, and on the
drivers whose captures vary with animation the variation is the same
before and after.
Publish tempo, bypass and LPF changes without reset between processing requests. Require source and metadata retirement, resumed writes and an unchanged reset serial in all native and mixed format lanes.
Making the nine-slice one strip assumed every display driver walks
coords->vertices. Three do not. gdi reads a fixed four vertices and
draws their bounding box - its comment says so, and says why: it ends
in a blit, and for a slice's four vertices the bounding box is the
geometry. Handed fifty-two, it drew the first piece and dropped the
other eight. ctr and switch ignore the vertex array altogether.
The driver says which it is, and the slice asks: one strip where a
rasteriser can walk it, a piece at a time where the draw ends in a
blit. Measured both ways on the same build - 23 draws a frame with the
strip, 55 without, which is what the file did before it was one - and
the two render identically.
The flag is what a batcher will need as well: the question of whether
a draw may carry more geometry than one quad is the same question.
Both audits of the SDL conversion left the same row open - real
microphone capture not validated - and the second one noted the mic
side had been reviewed rather than run. SDL's dummy driver supports
capture as well as playback and calls back on a clock, so half of that
row can be closed here rather than waiting for hardware.
The capture lane opens a mic through microphone_sdl's own vtable and
reads a frame's worth every 60th of a second, counting the same things
the playback lane does. What it cannot do is check a pattern: the dummy
capture device delivers silence, so a zero in the buffer means nothing.
sdl_microphone_read()'s return value stands in for it - it reports what
it managed to capture, so a short read is a park that ran out - and the
column is there rather than absent, at zero.
Result, on the same three-runs-each basis as before: identical before
and after the conversion. Measured by building the harness against
8143b50926^'s sdl_audio.c: p50 11.6, 9.7, 7.9 and 1.9 ms across the
sweep before, 11.6, 9.6, 7.8 and 1.8 after, no timeouts and no short
reads either way. The same answer the playback half gave, for the same
reason: a device that keeps calling back covers a lost announcement
within a period, and the quarter-second bound is only reachable once it
stops.
One thing the lane shows that is not about the conversion at all. The
core's read parks on nearly every frame and the park is long - 11.6 ms
of a 16.7 ms frame at the 8 ms setting, because a frame's worth is more
than one of the device's callbacks delivers. That is the
frame-synchronous microphone path doing exactly what the threaded
capture worker was added to stop, and it is worth having a number for.
Still not covered, and not coverable here: any real SDL backend.
PulseAudio, PipeWire, CoreAudio and WASAPI underneath SDL each have
their own callback timing, and the dummy device establishes that the
handshake is right rather than that any of them behave. The header says
so.
Ran: the harness plain and under TSan, clean, and against the
pre-conversion driver for the comparison above.
A publish larger than the source ring can block before its frame-end signal, leaving the consumer waiting for data until timeout. Notify on the full-ring wait path so the consumer can release space.
Exercise 0.25x through 32x native WSOLA with fixed ring and pass budgets.
The documentation half of an audit. Everything else it raised against
its baseline - the three unchecked retro_eventcount_init() calls and
the Windows WAIT_FAILED handling - was already closed by the commit
after it, and I checked that against the tree rather than taking the
table at its word.
What was left is three headers still describing condition variables
that no longer exist, and one of them says the opposite of what the
code now does. rs_wait_writable() explained its bound as being there
because "the error callback's signal can be lost" - which was true, and
was the single strongest argument for converting RSound at all, and is
the exact thing the conversion fixed. Left as it was, the next person
to read it has a documented reason to keep a bound whose documented
reason is gone, and no statement of the one that remains: a server that
stops draining without erroring runs neither callback, so there is
nothing to be notified by. That is now what it says.
ra_asio_wait_writable() and the SDL pair get the same treatment, and
"signals" becomes "notifies" throughout the three files - not a
synonym swap, since after these conversions the two words name
different mechanisms, and a comment in a driver that has both a
callback and an eventcount should not use the condvar word for the
eventcount operation.
The audit also notes what it would not change, and I agree with it:
notifying after every device pull is right for these three, where a
callback is a hardware period, and would be wrong in the generic
pipeline, where a publish is whatever slice a core felt like handing
over. That distinction is the one thing a later reader is most likely
to get backwards, and it is written down in both places.
No code changes.
Ran: sdl_lost_wakeup, rsound.c compiled on this host, and both
sdl_audio.c and asio.c through tools/mingw_syntax_check.sh - asio.c's
c89 arm still reports an empty translation unit, which is what it is on
Linux and what it reported before any of this work.
Three findings from an audit of the SDL conversion, all verified before
changing, and the first one is mine from two commits ago.
retro_eventcount_init()'s return value was dropped on the floor in both
SDL paths and in RSound. It can fail: on the backends that park through
a condition variable - Apple, the BSDs, the consoles, and Windows where
no sleep tier resolves - init allocates a mutex and a cond and either
allocation can come back NULL. What happens then is not a wait that
degrades. slock_lock() tolerates a NULL lock, so the audit's reading was
charitable, but scond_wait() does not tolerate a NULL cond: it hands it
straight to the platform call and dereferences it. So a failed init is
a crash on the first park, on exactly the platforms that take that
backend. All three are checked now and fail their driver's init the way
the allocation failures beside them already do. audio_driver.c and
asio.c already checked theirs.
The Windows wait treated everything that was not WAIT_TIMEOUT as a
wake, so WAIT_FAILED read as one. Since the TLS fallback was fixed that
should not be reachable - a CreateEvent that fails is answered in
ec_win32_park before the block is listed - but if it ever is, the wait
returns immediately, the caller re-checks, parks again, and the loop
becomes a spin on a core with nothing to do. The three cases are
distinguished now. A failure still reports a wake, because the caller
re-checks its own predicate and reporting a timeout instead would be a
lie, but it sleeps a millisecond first, which turns the spin into a
poll. There is nowhere to log it from - this is libretro-common - so
the poll is the whole mitigation, and that branch is not exercised:
no way to induce a failing wait on a valid handle was found.
And sdl_audio_wait_writable() still said it slept on a condition the
speaker thread signals. There is no condition variable in that file any
more.
Ran: the eventcount test on all four Windows tiers under wine
(alert, keyed, event, none), the futex and forced-scond builds on
Linux, sdl_lost_wakeup, rsound.c compiled on this host, and
sdl_audio.c through tools/mingw_syntax_check.sh. Not run: the
allocation-failure paths themselves, on any platform.
The same conversion as sdl_audio, on the two remaining drivers whose
payload is already a lock-free retro_spsc and whose lock therefore
guarded nothing but the wait. Both files already documented the window
it closes, in more detail than SDL did, and in both the window is worse
than SDL's.
RSound. rsound_err_cb sets the error flag and notifies, and librsound
then returns from its worker thread - so that notify is the last one
that will ever be raised. Neither callback held cond_lock, so a signal
raised between the writer's test and its wait reached no waiter, and
losing that particular one left the writer parked with nothing alive to
wake it. Only the bound and the lap count got it out. Registering
before the re-check closes it, and the bound goes back to being what it
is named for: a server that stops draining without erroring.
has_error becomes an atomic with it, and that is not tidying. The
re-check inside the wait window has to see the store that came before
the notify, and volatile orders nothing between threads - so the flag
the window turns on had no ordering of its own. It is published with a
release store before the notify and read with an acquire load
everywhere, including inside the window.
ASIO. asio_cb_buffer_switch is a real-time callback that must not take
a lock, which the file says outright - so the lock could never have
guarded the predicate, and the window was not an oversight but
structural. Three waits converted, all keeping the shutdown and
is_paused tests in the re-check, because those two are also what make
the bound necessary: the callback returns early, before notifying, once
either is set, so a park entered after that has nothing left to end it.
That reasoning is unchanged and the comments keep it.
Not measured, and there is nothing here to measure with. The sdl_audio
harness found the equivalent window cost one device period and moved no
percentile; the reason to expect more from these two is the RSound
error path, which is a park that ends only on a timeout rather than a
period of latency, and no fixture here can reach it - RSound needs a
server and ASIO needs Windows with a driver. So: correctness, argued
from the code and from what the same shape measured elsewhere, not from
a run.
Ran: rsound.c compiles clean on this host; asio.c through
tools/mingw_syntax_check.sh, whose win32 arm is clean - its c89 arm
reports an empty translation unit, which is what asio.c is on Linux and
what it reported before this change too. Not run: either driver against
a device, on any platform. coreaudio.c has the same shape and is not
converted here: nothing in this container can compile it, and a blind
edit to five sites in a driver no one can build is not worth the
tidiness.
Both found by an audit, both verified against the source before
changing, and both are mine: I converted these two protocols and left
the paragraphs above them describing the ones they replaced.
The producer's space wait still said the generation was read under the
lock. There is no lock on that path any more - the generation is an
atomic and the eventcount only parks - and what makes a completed pass
impossible to miss is now the ordering, not a mutex: the generation is
taken, the window is opened on pipe_space, and the generation is read
again inside it, so a pass anywhere across that window is either seen
by the re-check or stops the commit from sleeping. That is the part a
later reader has to get right, and it was not written down.
The consumer's data wait said the same thing about the lock, and also
said the producer signals after every publish. It does not, and the
paragraph on audio_driver_pipeline_consume() twenty lines above says so
in as many words. Two comments in one file contradicting each other
about the same handshake is the thing an earlier commit in this file
called worse than either being wrong alone, and it was reintroduced.
Both rewritten to describe the register-then-recheck shape, which is
what someone modifying either wait needs to preserve.
One real change with them. The eventcount init pair short-circuits:
if (!retro_eventcount_init(&pipe_space) || !retro_eventcount_init(&pipe_data))
so a failure on the second leaves the first up, and the block returned
without freeing it. Whether that leaks depends on whether the caller
goes on to run the audio teardown, which frees both - so it is not a
demonstrated leak and I have not traced every path to claim one. It is
a block that only holds because of what happens elsewhere, which is
worth not relying on. It frees both now; free is safe on an object init
never reached, so no flag is needed to say which one got that far.
Ran: pipeline_wakeups, pipeline_stall and pipeline_clocked.
sdl_audio.c already carried both directions in lock-free retro_spsc
rings, so sdl->lock and mic->lock guarded no data - only the bounded
waits a full or empty ring puts the other side into. Five waits and two
announcements, on both the playback and the capture path, and the two
locks and two condition variables become one retro_eventcount each.
The window this closes is one the driver already documented:
Bounded: sdl_audio_playback_cb signals without holding sdl->lock, so
a signal raised between the avail test and this wait reaches no
waiter. Normally the next callback covers that.
prepare_wait registers before the re-check, so a callback that lands
from that point on either shows up in the re-check or ends the park at
once. "Normally" stops being load-bearing. It also takes a mutex
acquisition off SDL's callback thread, which is the device's own and
runs whether or not anyone is parked.
What it does not do is show up in a measurement, and the harness added
with it says so. The obvious reading of the code is that a lost signal
costs SDL_AUDIO_STALL_TIMEOUT_US - a quarter second - and then dropped
audio, because a timeout is acted on: sdl_audio_write() breaks out and
reports a short write, sdl_audio_wait_writable() returns 0. That
reading is wrong. A device that is still running calls back again a
period later and that callback ends the park, so the window costs one
device period at worst. The bound is only reachable once the device has
stopped calling back altogether, which is the stall it exists for and
not this window.
So samples/audio/sdl_lost_wakeup reports the same numbers before and
after, three runs each at every latency setting: 180 parks per 3 s at 8
and 16 ms, zero timeouts, zero dropped frames, p50 7.2 ms and 1.1 ms -
a device period, which is the honest figure. A single earlier run
appeared to show the tail improving; repeating it showed that was
noise, and it is recorded here rather than quietly dropped because a
one-run difference on a threaded measurement is not a result.
The harness is kept anyway. It establishes that fact about the
handshake rather than leaving it to a reading of the code, and it is
what would catch a later change that made the bound reachable.
It builds audio/drivers/sdl_audio.c and drives it through its own
audio_driver_t vtable against SDL_AUDIODRIVER=dummy, so it needs no
sound card - only libsdl2-dev.
Ran: the harness plain and under TSan, clean, and
tools/mingw_syntax_check.sh on the driver. Not run: the capture half
against a real microphone, and the driver against a real device on any
platform - the dummy device exercises the handshake but not SDL's own
backends.
Exercise int16 and float devices with both source formats. Require matching lanes to avoid frontend conversion, mixed lanes to convert only toward the device, and int16 output to exercise native sinc SRC.
Extend the existing restart matrix without changing production audio code.
Cover both source formats, dry/LPF/WSOLA stages and device pacing regimes. Park, discard, rebuild and restart transport, requiring fresh writes after every restart. Keep lifecycle counts separate from timing.
Advance fragmented source spans correctly and release fixture SRC instances between settings.
Two things, both found by running the check on a machine that had never
run it.
It was a hard failure everywhere without a cross compiler. The i686 arm
was guarded by command -v and said so when it skipped; the x86-64 arm
was not, so on any box without mingw the recipe died on "make:
x86_64-w64-mingw32-gcc: No such file or directory" and the suite
reported BUILD FAILED with no error line under it - the runner greps
for 'error:' and there was none. A missing tool is not a failing check,
and a suite that is red on every machine without one particular cross
compiler teaches people to skim past red. Both arms are guarded now and
both name what did not get proved, which is the part worth saying: a
silent skip on an ABI check is its own hazard. The GUID comparison is
python against the system headers and still runs either way.
With gcc-mingw-w64-x86-64 and gcc-mingw-w64-i686 installed, every arm
passes - including the 32-bit one, which the Makefile calls out as
worth proving rather than asserting and which had apparently been
skipping.
And one assertion added. mingw-w64's ksmedia.h declares
KSPROPERTY_RTAUDIO with a single enumerator in it,
KSPROPERTY_RTAUDIO_GETPOSITIONFUNCTION, so that one value can be
compared on any toolchain while the other ten still need a driver-kit
header and tools/wdmks_wavert_check.sh. It is the one worth having
alone: the driver writes the enumeration out in the kit's order and
takes every value from a name's position in it, so a wrong base moves
all eleven by the same amount, and the kernel then answers a different
property rather than refusing. This pins the base.
Verified by breaking it - setting the base to 1 fails the build on
assert_rt_getposfn - rather than by observing it pass.
audio/drivers/wdmks.c is unchanged; it also goes through
tools/mingw_syntax_check.sh clean, which is the first time that has
been run against it here.
The last hand-rolled condvar handshake in the audio path, and the
smallest: outgoing_samples was already a retro_spsc, so capture_lock
carried no data - the header said as much - and only the two bounded
waits were under it. Five sites.
One eventcount for both directions, not two. A notify releases every
waiter where scond_signal released one, so a single object is only
right if the two sides can never be parked together, and they cannot:
the ring is AUDIO_CHUNK_SIZE_NONBLOCKING * AUDIO_MAX_RATIO = 32768
samples, the worker parks with room for less than one 2048-sample
slice, and the core parks holding less than the frame it asked for,
which is never more than a slice. Both at once needs the ring above
30720 and below 2048. samples/audio/mic_handshake measures it anyway
and reports 0.00% at every latency setting - the argument rests on
three constants a later change could move, and that is where it would
show up.
Both waits keep the re-check that holding the lock across the sleep
used to give: register, read the condition again, cancel if it moved,
park only if it did not. The worker's re-check also takes in
capture_running, so a teardown that lands inside its window does not
cost it the full 20 ms. The core's stays a single bounded wait rather
than a loop - a frame is not held for a device that will not deliver,
and the silence below it is the same answer the synchronous path gives.
Teardown drops its two conditional frees for one retro_eventcount_free,
which is safe on a microphone that never had a worker: the state is
zeroed at open and free tolerates that.
Measured with mic_handshake, 3 s a setting: identical to the commit
before it - no empty device waits, no padding, core read p50 0.7-4.0 ms
against its bounded 10 ms. Also under TSan, clean.
Ran: mic_handshake plain and under TSan, and the audio harnesses that
compile this file - pipeline_wakeups, pipeline_stall, pipeline_clocked,
pipeline_rate_control, discrete_multichannel, ff_speedup,
thread_handshake, mixer_lock, pipeline_format, rate_control_bounds,
sink_rate, extra_capacity. Both touched files through
tools/mingw_syntax_check.sh, including every consumer of
microphone_driver.h; the two failures it reports there,
audio/drivers/sdl3_audio.c and the wasapi sim, fail the same way
without this change - missing SDL3 headers and the sim's own fake
Windows typedefs against a real windows.h. Not run: a real microphone
on any platform, and the alsa-backed harnesses, whose headers this
container does not have.
Before converting it, the same way round as the playback pipeline: the
threaded capture path has a worker owning the device read, the
up-channel and the resampler, and one condition variable between it and
the core's read, used in both directions. Nothing measured it.
This brings a microphone up through microphone_driver_open_mic() rather
than assembling the state by hand, so the ring sizing, the resampler
and the worker's own start-up are the shipping ones, and puts a clocked
capture device under it: samples credited on a real clock, a non-zero
pattern written into every one so a zero in the core's buffer is
padding and can be counted as padding.
The question it was built to answer is whether the conversion needs one
eventcount or two. scond_signal() releases one waiter and an eventcount
notify releases all of them, so a single eventcount is only right if
the two sides are never parked at the same time.
They cannot be. The ring is AUDIO_CHUNK_SIZE_NONBLOCKING *
AUDIO_MAX_RATIO = 32768 samples, the worker parks when it has room for
less than one 2048-sample slice, and the core parks when the ring holds
less than the frame it was asked for, which is never more than a slice.
Both would need the ring above 30720 and below 2048 at once. The
harness samples both conditions on the device clock regardless and
reports 0.00% at every latency setting, because the argument rests on
three constants a later change could move and this is where that would
surface.
Two things about the fixture worth writing down, both found by it
misbehaving first:
The worker asks wait_readable() for a whole 2048-sample slice, which is
larger than a small device's entire buffer - 8 ms at 48 kHz is 384
samples. A driver that takes that request literally can only ever time
out, and the first version of this device did, reading nothing and
handing the core 100% silence at every setting below 64 ms. alsa.c
clamps the request to one period, so this does too. Real drivers are
where the fixture's behaviour has to come from.
And the "both parked" column originally read flags the two sides
raised, which meant it read a flag the worker never set - there is no
hook inside the worker and adding one would have measured the
instrumentation. It samples the two park conditions instead, which are
pure functions of the ring and readable from the device thread.
Baseline, 3 s a setting: no empty device waits and no padding at any
setting, core read p50 0.7-4.0 ms against a bounded 10 ms wait, and the
core's park condition holding on essentially every sample - the core
waits for the worker most frames, which is the existing design rather
than something this changes.
No shipping code is touched.
Query bounded retained engine state before pacing an empty native pass. Keep pending output, EOF and transitions runnable, and require retired processing metadata to be applied before waiting.
Exercise readiness against zero-input processing in both native formats across rates and tempos, plus deferred control events. No sample conversion, allocation or new synchronization is added.
The netplay ping widget asked the menu whether it was open, from
gfx_widget_netplay_ping_iterate(), which the threaded wrapper runs on
the video thread. The menu's flags are a read-modify-write on the main
thread - runloop_check_state() clears two of them in the same
expression - so the read raced the write, on a word the whole menu
keeps its state in.
The frame already carries those flags: video_driver_frame() copies them
where the menu cannot be changing. The worker latches that copy before
it iterates, and the widget reads the latch.
Found with networking on. The threaded video suites had been run
against a build configured without it, so the file this lives in was
never compiled and the widget never ran.
An audit found it and it is real: reproduced, then fixed, then the
reproduction kept as a knob.
ec_win32_resolve() sets ec_g.sleep = 0 when TlsAlloc() cannot give the
per-thread-event tier an index, with a comment saying it falls through
to the condition variable. It did not. retro_eventcount_init() set
lockless = 1 for every RETRO_EC_ADDR_WIN32 build without asking whether
a tier had resolved, so the object got no lock and no cond, and the
wait paths took the win32 branch anyway because that branch was chosen
at compile time. ec_win32_park() then skipped creating the event, since
that is also gated on the EVENT tier, and left w.event NULL. ec_sleep()
fell to its default case and called WaitForSingleObject(NULL, ms),
which returns WAIT_FAILED; the test there is != WAIT_TIMEOUT, so a
failed wait reported itself as a wake. The notifier's SetEvent(NULL)
failed to match.
What that costs is worse than a busy loop. commit_wait_timeout()
returns true for "notified", so a caller with a bound gets no bound at
all - and the audio pipeline's waits are all bounded ones. The
eventcount test does not finish its first lane.
Reproducing it needed the process to run out of TLS indices, which is
why the path had never been run. It has a knob now:
RETRO_EVENTCOUNT_WIN32=none pins the no-tier case, beside the existing
alert/keyed/event. Same variable, same place, and the TlsAlloc() call
answers it too, so the fallback is reachable without exhausting
anything.
The fix is the one the source already promised rather than the one
that fails init cleanly. The generic scond code was already compiled
into the win32 build - it sits after each #if block rather than in an
#else - so the win32 branches only had to stop being unconditional.
Whether an object took the waiter list or the condition variable is
per object and asked as ec->cond: a field that is set at init and never
changes, rather than a global reread on a hot path. init() allocates
the lock and cond when no tier resolved, and notify(), commit_wait()
and commit_wait_timeout() fall through to them.
Failing init instead would have been smaller, and would have taken
RetroArch down over a condition it was written to survive: the audio
pipeline treats a failed eventcount init as fatal and exits.
backend_name() reported "win32 event" for this case, which was the
fallback's only outward sign and was wrong. It now names the tier it
is actually on. spin_iters() returns 0 there - a condition variable has
no flag word to spin on.
Ran, on a mingw cross build under wine64: alert, keyed, event and none,
all four lanes each. Before the fix, none hangs in the first lane and
is killed at 60 s; after it, it passes and reports "scond (no win32
sleep primitive)". The other three are unchanged either way. On Linux:
the futex and forced-scond builds, the futex build under TSan, and the
audio harnesses that link this file. The recipe is in the test's
Makefile - it needs a cross compiler and wine, so it is not wired into
check.
Also the three documentation leftovers from the same audit, all
verified against the source first. notify()'s API comment still
described a fence that was removed when the bump became an SC
read-modify-write, while the header overview twenty lines above
already described the SC pair correctly - two statements in one file
disagreeing about the same operation. prepare_wait()'s comment named
"the two fences" and there are none. And the header's numbered
contract ran 1, 2, 3, 4, 4.
Wait for source before device pacing when a native stage has no retained work, and recheck data predicates after waking. Keep queued passes off the added wait path. Preserve the stall flag when paused discard reports completion through the new source-space eventcount, matching the legacy consumer.
This file says which Qt each screen query is for - QScreen and
QGuiApplication are Qt 5 and later, QDesktopWidget is Qt 4 and Qt 5 and
gone in Qt 6 - and ui_qt_available_geometry_at() does it properly. Two
places did not: the status message asked the primary screen for a
refresh rate, and the companion asked it for the usable area, both with
nothing but a bare QScreen.
The refresh rate is Qt 5 and later and there is no Qt 4 way to ask, so
it takes the second the code already falls back to when there is no
screen. The usable area is asked the way the helper asks it, from
QDesktopWidget below Qt 6.
No screen query in the file is unguarded now.
Second of the two generation-counter protocols, and the one the audit
warned about. pipe_data_gen and pipe_wake become atomics, pipe_data_cond
becomes a retro_eventcount, and with it pipe_lock has nothing left to
guard and is removed from the state. state_lock stays: it protects the
DSP filter and the mixer's voices and is a different thing entirely.
The cadence is the whole risk here and it is unchanged, because no
notify was added. audio_driver_pipeline_signal() is still the only
producer-side announcement and still runs once per frame, from the
frame end and the per-frame producers that have no frame end of their
own. The publish sites were not touched and must not be: the ring
carries the data, so a retro_spsc_write() has nothing to announce. The
header now says so where the field is declared, which is where someone
adding a notify would be looking.
Four waits converted, each keeping the re-check that holding the lock
across the sleep used to provide - register, read the conditions again,
cancel if any of them changed, park only if none did:
- the consumer's first wait, on the ring reaching its target, which
also carries the sticky pipe_wake and the metadata-queue escape
- audio_driver_transport_wait(), on the ring or the layout queue
- the retry path's anti-spin sleep, on pipe_wake alone
- and audio_driver_pipeline_wake() on the other side, which raises
the sticky flag before the generation before the notify, so a
consumer woken by it cannot find the new generation beside a wake
that has not been raised
audio_driver_pipeline_wake() asked whether pipe_lock had been created
to find out whether the pipeline was up. There is no lock to ask about
now, so pipe_park_ready answers it instead - same question, same
answer, set where the eventcounts come up and cleared where they go.
Measured with pipeline_wakeups, which is what it is for: 3.01
wakes/frame flat from 1 publish per frame to 312, unchanged from
before the conversion, 33 us median wake latency, no producer stalls.
The harness still has teeth against the converted code - forced to
notify per publish against a slow core it reads 235 wakes/frame at
N = 312, so the flat column above is the cadence holding rather than
the measurement having gone blind.
Ran: twelve audio harnesses, each cleaned first - pipeline_wakeups,
pipeline_stall, pipeline_clocked, pipeline_rate_control,
discrete_multichannel, ff_speedup, thread_handshake, mixer_lock,
pipeline_format, rate_control_bounds, sink_rate, extra_capacity (which
compiles audio_driver.c without HAVE_THREADS). pipeline_stall and
pipeline_clocked under TSan, clean, and pipeline_clocked's underrun
counts match the run before this change at every latency setting. Not
run: anything on Windows, and the alsa-backed harnesses, whose headers
this container does not have.
First of the threaded pipeline's two generation-counter protocols.
pipe_gen and pipe_stalled become atomics, pipe_cond becomes a
retro_eventcount, and pipe_lock no longer covers any of it - the lock
stays only for the data channel, which is the second protocol and not
this commit.
The shape the audit asked for: retro_spsc already moves the payload,
the generation is the atomic that answers "has a pass completed", and
the eventcount is only how a producer that found the ring full parks.
What it buys is the consumer's side: a pass that completes with nobody
waiting now pays a read-modify-write and a load instead of taking and
releasing a lock, and the consumer completes a pass two to three times
a frame.
The cadence is unchanged and could not have been otherwise here. The
producer waits for a pass to complete, so a pass completing is the only
event there is to report - five sites, all of them the end of a pass,
all mutually exclusive within one. They go through one helper,
audio_driver_pipeline_pass_done(), which takes whether the pass showed
the device draining: the paused-drop path advances the generation so a
waiting producer is not left hanging but has not disproved a stall,
and audio_driver_pipeline_wake() has not either, since a stop or a
reinit is not a pass.
pipeline_wakeups says the conversion costs nothing and buys nothing
measurable on this path, which is the expected answer and the reason
for measuring: 2.2-2.6 wakes/frame with backpressure and 3.01 without,
4.2 ms and 34 us median wake latency, no producer stalls - the same
numbers as the commit before it, within the noise of two runs. The
batching trap lives in the data channel, where the producer publishes
many times per frame and signals once; this channel never had it.
One behaviour change, and it is a fix. The old wait applied
AUDIO_PIPE_WAIT_MAX_US per scond_wait_timeout() call inside a loop on
the generation, so a bound meant to cap one stall at that could hold
the frontend for a multiple of it given enough spurious returns. The
eventcount reports a spurious return as a wake, exactly as a condition
variable does, so the bound is now a deadline taken once and the whole
wait fits inside it.
retro_eventcount.h moves out of audio_driver.h's HAVE_THREADS guard:
the eventcount is a value member of the state rather than a pointer, so
the type has to exist in a build that never calls into it. Eleven
sample harnesses build the fixture by hand and are updated with it;
retro_eventcount.o was already in Makefile.common and griffin.
Ran: the audio samples, each cleaned first - pipeline_stall,
pipeline_clocked, pipeline_rate_control, pipeline_wakeups, ff_speedup,
discrete_multichannel, thread_handshake, mixer_lock, pipeline_format,
rate_control_bounds, sink_rate, extra_capacity (which compiles
audio_driver.c without HAVE_THREADS). pipeline_stall and
pipeline_clocked under TSan, clean. Not run: anything on Windows, and
the alsa-backed harnesses, whose headers this container does not have -
they failed to build before this change as well.
The sweep went into the menu harness job because that is where the
other object checks live. That job turns off OpenGL, OpenGL core,
OpenGL1 and Vulkan - it has to, the harness replaces main() - so gl2.o
and vulkan.o are never built there and the sweep walked past the files
it was written for. It would not have caught one of the twelve kilobyte
frames that prompted it.
It moves to the Linux job, which configures with Qt off and everything
else on, and whose objects include every driver. Checked against that
job's own configure line: the four recorded frames are found and
nothing else is over.
Extend the concurrent wakeup harness with prepared dry, filtered and active WSOLA sessions. Report fixture failures and release owned storage after joining workers. Use absolute spread deadlines so timer rounding cannot accumulate across hundreds of publishes per frame.
Route prepared sessions through device-paced transport passes, draining retained synthesis before waiting for source. Bound startup waiting, let full control queues retire, cancel paused output and recover from underruns through the stage-owned discard. Leave unprepared sessions on the existing consumer.
audio_bitstream_submit() pulled the core out of a DTS frame into a
buffer twice the size of an AC-3 burst, on its own stack: 12424 bytes
where this tree allows four thousand, on the path that submits every
frame of audio, from whichever thread feeds the device.
It moves into the stream the submission is for, which is calloc'd once
and already carries the pause burst for the same reason. The frame is
136 bytes.
The two sizeof() that measured it are named lengths now rather than
left to measure a pointer, which is what makes this kind of move go
wrong quietly.
Measuring the eventcount conversion before doing it. The audit that
suggested converting audio_driver.c's two generation-counter protocols
named the trap - the batching is the point, so a mechanical
scond_signal -> notify at every site would wake a per-scanline core's
consumer hundreds of times a frame - but nothing in the tree can count
a wake, so there was no way to tell whether a conversion had sprung it.
pipeline_clocked and latency_contract both pass either way: the audio
is identical, and the difference is a cost.
So this counts wakes, at the harness's own boundary rather than inside
the driver - returns from audio_driver_pipeline_consume(), calls that
reached the device, frame-end signal to first write, microseconds the
main thread spent inside audio_driver_submit(), and the device's own
short pulls. None of those change meaning when the primitive
underneath changes, which is what makes a before/after comparison
worth anything.
The producer hands one frame's audio over as N publishes, swept from 1
to 312, submitting identical audio at every setting.
What it found is that the trap is real and narrower than the audit's
description, and three things have to hold at once before a per-publish
notify costs anything:
- The device must not be the pacer. With backpressure the consumer
parks inside wait_writable(), not on the data handshake, and extra
notifies arrive at a thread that is not asleep. That is Audio Sync
against a real card, and a wake count taken only there is flat
whatever the handshake does - 2.2 to 2.6 wakes/frame across the
whole sweep, per-publish or not. Both regimes are run for that
reason.
- The publishes must be spread across the frame. Emitted in a burst,
which is how a fast core's retro_run emits them, all 312 are in the
ring before the consumer finishes its first pass and it takes the
lot in one go: 312 notifies, one wake. Still flat.
- And the notify must be per publish.
With all three, 3.0 wakes/frame becomes 211 at N = 312 and the
producer's own cost goes up about sixteenfold. With any one missing the
column does not move. So the failure mode wants a core that batches per
scanline and is slow enough to spread them, against a driver not
applying backpressure - not merely a core that batches.
Both the bad shape (NOTIFY_PER_PUBLISH) and the slow core (SPREAD) are
modes rather than prose, so the column to avoid can be measured instead
of imagined.
No shipping code is touched. The conversion itself is still not done;
this is the measurement it has to be judged by, and it needed to exist
first.
Baselines, 3 s a setting on an otherwise idle machine, in the commit
that adds it: burst/backpressure 2.2-2.6 wakes/frame, 4.2 ms median
wake latency, no producer stalls; burst/no-backpressure 3.01
wakes/frame flat, 34 us median.
gfx_display_sdl2_draw() alloca'd both its vertex and index arrays from
the vertex count the caller passed, so the stack cost was whatever
someone asked to draw: four vertices for a quad, fifty-two for a
nine-slice since that became one strip, and thousands for a pipeline.
The frame measured 8264 bytes and had no ceiling at all. The two
become buffers the driver keeps and grows, freed with it, and a draw
that cannot have them is skipped rather than the stack overrun.
sdl2_raster_font_render_line() built a line into a chunk of glyph
vertices and indices on its frame, 6216 bytes, which is the same thing
the gl drivers and rsx had. It moves into the font's own state.
Neither showed in the sweep that found the others: this file is not
compiled without SDL2, and the configure used then did not have it.
Publish tempo, reset and native filter target together at the producer cursor. Reject missing sessions, invalid tempo and queue pressure without partial control changes; wake the consumer only for a new request. Exercise source ordering and full-queue retry through the frontend.
Give the consumer an exact native source discard that also cancels retained device and transport output, resets DSP history and wakes the producer when source space is released. Oversized requests preserve state. Reuse zero-frame discard after successful stop.
Discard retained WSOLA and LPF output while the consumer is parked, preserving queued source and allocated native storage. Failed stops preserve the session. Cover native formats and stereo/wide restart with retained filtered output.
Five comments describing implementations that no longer exist. All
found by an audit, all verified against the source before changing,
and all documentation only - no behaviour changes here.
retro_eventcount's header said notify costs one RMW, one fence and one
load. There is no separate fence any more: the path is a
sequentially-consistent read-modify-write and a sequentially-
consistent load.
notify() then said "only sound where the fence above is a real
barrier", naming a fence that was removed when the bump became an SC
RMW. What carries the ordering now is that pair, and the comment says
so - and still says why a build whose atomics are not lock-free takes
the lock instead.
notify() also said a registered waiter holds the lock from
prepare_wait until scond_wait releases it. It does not: prepare_wait
registers with atomics alone, and commit_wait takes the lock only for
the final re-check and sleep. That is what makes taking the lock here
safe, so the reason was right and the mechanism described was not.
prepare_wait said "no lock is taken here on any backend", with a
slock_lock four lines below it in the RETRO_EC_LOCKED_BOOKKEEPING
branch. Lock-free backends register without one; that branch does not.
And audio_driver.c's consumer said the producer never signals it and
the ring is the only channel, while audio_driver_pipeline_signal()
plainly increments pipe_data_gen and signals pipe_data_cond. It does
signal - once per frame, deliberately not per publish, which the
producer side already explains twenty lines away. Two comments in one
file saying opposite things about the same handshake is worse than
either being wrong alone.
For a synchronisation primitive this is not cosmetic. A memory-order
comment that names a fence which is not there is the kind of thing a
later reader trusts over the code.
On the larger suggestion in the same audit - converting the threaded
pipeline's two generation-counter protocols to retro_eventcount - I
have not done that here, and would not fold it into a comment fix. It
is a rewrite of a working handshake on the audio path, and the audit
itself names the trap: the batching is the point, so notify must stay
once per frame rather than once per publish. It is worth doing with
the measurements it lists, as its own change.
stretch_test aborts under AddressSanitizer with a 22-byte write past a
16-byte stack buffer, reported inside audio_stretch.c's raw
passthrough.
The library is not at fault. The reset case hands the stream an
output_capacity of exactly one frame and a buffer of eight floats or
eight int16 to put it in. Eight channels is where that is exactly one
frame; the loop above it walks channels up to
AUDIO_STRETCH_MAX_CHANNELS, which is eleven, and a frame there is 22
bytes. The stretcher copied the one frame it was told there was room
for, into a buffer six bytes too small to hold one.
Both buffers are sized by AUDIO_STRETCH_MAX_CHANNELS now, so they
follow the loop rather than a number that was right when it was
written.
Passes plain and under ASan and UBSan with leak checking on: no
sanitizer reports, 0 failures across the stretch, sinc, search-oracle,
canonical and adapter cases.
generic_menu_iterate() holds two things called entry: the one the menu
looks at once a frame, and a const core_updater_list_entry_t * that
core_updater_list_get_filename() fills in. Moving the first off the
stack rewrote every &entry in the function, including the second, where
the address of the pointer is what the callee wants. It builds with the
online updater off, which is why it was not caught here.
Prepare and release optional transport state through the wrapper control transaction. Preserve an existing session on preparation failure, rebuild at an empty native-format boundary, and release state after the worker joins.
Realign the empty source ring while retaining its published controls so float renegotiation cannot force permanent one-frame staging. Exercise owned rendering, allocation failure, queued/pending guards, rebind and teardown.
rgui_update_savestate_thumbnail_path() is the fourth of those and was
missed when the other three were done: a path and a menu_entry_t, 5960
bytes, on a function that runs when the selection moves. It gets the
scratch the others got, and measures 24.
generic_menu_iterate() looks at an entry once a frame and put it on its
own stack, which came to 4312 where this tree allows four thousand. It
moves into the menu state, which is where something looked at once a
frame belongs, and the frame is 440.
Of the menu functions still over the ceiling, these were the last two
that hold a menu_entry_t and run often. What is left is path buffers,
and PATH_MAX_LENGTH is already 512 rather than 2048 on the platforms
whose stacks are small - Xbox, 3DS, PSP, PS2 and GEKKO - so those
frames are a quarter of their size where that matters.
retro_spsc moves the bytes and retro_eventcount parks a thread, and
every subsystem that wants both grows its own mutex, condition
variable and generation counter to say what the queue already knows.
This is the two of them wired together so they need not.
It is a composition rather than a change to retro_spsc: a queue whose
ends never sleep should not carry two eventcounts, so the plain queue
stays plain and this sits on top.
Cancellation is part of it rather than left to the caller. A waiter
sleeps until its bytes arrive or its bound expires and sees nothing
else change in between, so a nudge only sends it round its loop again;
teardown needs a flag the wait itself consults, and the harness is
what established that.
Every write and every read notifies. With nobody parked that is a few
nanoseconds, so the common case is not worth batching for; with
somebody parked it is a wake, and the quiet forms are there for a
caller who knows its own batch.
samples/queues/retro_waitable_spsc_test pushes a known byte stream
through a queue far smaller than it, checking the value of every byte
rather than counting them, so a lost or repeated wake shows up as
wrong data rather than a wrong total. It runs in the concurrency suite
and in the sample sweep.
Each of the three built one with two PATH_MAX_LENGTH buffers and a
menu_entry_t beside them, and each measured 8008 bytes where this tree
allows four thousand. They run when the selection moves rather than
every frame, so one allocation costs less than the frame did: the three
become a scratch the function owns and frees on the way out, including
the early returns. The frames come to 40 bytes.
The sizeof() that measured those two buffers is named rather than left
to measure a pointer, which is what the last of these got wrong.
An acq_rel read-modify-write followed by a standalone seq_cst fence
gives the ordering the wakeup handshake needs and costs a second
locked operation to do it. The pair on each side is sequentially
consistent instead, which puts all four operations in one total order
and says the same thing once. Notify with nobody parked measures
18.61ns before and 9.67ns after.
retro_atomic grows a seq_cst add and load on int for it, native on the
three backends that express that ordering directly. The rest already
make their read-modify-write a full barrier, so the acquire load after
one needs nothing further, or cannot express a barrier at all and
degrade here as they do everywhere else.
The header promises any number of waiters and any number of notifiers.
That was only true where the atomics are lock-free: the volatile
backend spells fetch-add as a load, an add and a store, so two
notifiers could lose an epoch bump and two waiters a count. Those
builds keep the epoch and the waiter count under the mutex they
already own. Nothing is held across the caller's predicate window
either way. The PS2 backend is atomic but deliberately not lock-free,
so it takes the lock as well, on the one core where it is uncontended.
A timeout too long for the millisecond unit of the oldest Windows
sleep is clamped rather than wrapped into a short one, and a negative
bound is defined as a poll rather than left to the backend.
TlsAlloc failing now falls through to the condition variable instead of
handing out an invalid index, the wraparound the epoch key relies on is
written down, and a helper nothing called is gone.
Wait before advancing transport, preserve device retries and SRC reset ownership, and avoid applying tempo twice. Keep the shared renderer inline for the default consumer.
Drain synthesizable old output before changing LPF cutoff so short source epochs remain chunk-independent. Compare the paced pass against manual native rendering with LPF, HQ, backpressure and conflicting speed flags.
xmb_render() declared a menu_entry_t for the entry a pointer press
lands on. It is 3872 bytes, and the frame measured 4312 where this tree
allows four thousand, on a function that runs every frame. It moves
into the handle xmb already carries; the frame comes to 152. One entry
is looked at a time.
No sizeof() measured that entry, which is the mistake the last of these
made: a buffer that becomes a pointer takes its length with it, and the
name of the length has to follow.
The ozone change that moved an entry's buffers into the handle left
five sizeof() behind that had measured those arrays. A pointer's sizeof
is eight, so the tickers it sized truncated their text to seven
characters - the buffers are still the right size, the code just asked
the wrong thing how big they are. The lengths are named now.
It did not show when that change was checked: the menu it was captured
on had no entry long enough to start a ticker, so the path that reads
those lengths never ran. The same submenu compared against the commit
before it is byte for byte identical with the lengths named.
rgui_render() gets the same treatment it prompted: a menu_entry_t and a
sublabel buffer off a frame that measured 5912 bytes, on a function that
runs every frame. The frame is 1280, and the sublabel's length is named
rather than sizeof'd, which is the mistake above, caught this time by
the render differing by 878 pixels.
Publish cutoff with transport controls and apply the native LPF after WSOLA. Filter borrowed source directly into existing output storage, preserve dry ring views, and retain filtered output across partial acknowledgements without processing it twice. Reset filter history at transport discontinuities.
Exercise native formats, fragmentation, wraps, EOF, resets and discard with allocator guards; verify separate-output filtering against the in-place core.
ozone_draw_entries() declared a menu_entry_t in the loop that draws
them, with a sublabel buffer and two label ones beside it. A
menu_entry_t is 3872 bytes - three of its six strings are a kilobyte
each - so the frame measured 9960 bytes, more than twice what this tree
allows, on a function that runs every frame on the thread that draws.
They move into the handle ozone already carries. One entry is drawn at a
time, so one is enough, and the frame comes to 4824. What is left is the
callees the compiler folds in rather than anything declared here.
The struct itself is the larger lever and is not this patch: its label
field is a kilobyte and holds menu identifiers, of which the longest of
the 1696 in msg_hash_lbl_str.h is 58 bytes, and no list appended one
longer than 28 in a menu walk. Shrinking it would take every frame that
holds one of these down at once, which is worth its own change and its
own audit of what writes that field.
Provide caller-owned float and fixed-point int16 filtering with bounded coefficient updates, source-frame fades, priming and bit-preserving dry bypass. Keep sample formats native and leave production playback activation to the transport control integration.
Wire common and Griffin builds and add 60 DSP/transition cases with allocator guards plus Linux sanitizer CI coverage.
The libretro-common samples job fails with:
retro_eventcount_bench is built but appears in neither RUN_TARGETS
nor NORUN_TARGETS. Add it to one of them.
which is the guard working. That check exists because a silent "not
in the run allowlist" skip is indistinguishable from an omission, and
seven self-contained regression tests were once compiled every run and
never executed. A new binary has to be classified deliberately.
This one is a benchmark, not a test, for the same reason
queues/retro_spsc_bench already is: it reports park and commit counts
under contention, and those numbers only mean something on dedicated
physical cores, which a shared runner does not have. So it goes in
BUILD_ONLY_DIRS beside it - which also covers the second binary that
directory builds, the scond variant, without naming either.
Every sample directory with "bench" in its name was checked against
that list rather than just the one that failed: formats/bench is the
only other, and it is a registered run target that passes.
Separate native rendering from source-ring reads so retained WSOLA output can reuse channel splitting, SRC and device-output ownership. Keep channel preparation and flush under one state lock. Verify native queue/WSOLA/frontend chains across layout changes, resets, HQ sinc and partial device writes.
CODING-GUIDELINES puts a ceiling on a function's stack frame and nothing
checked it. The frames this was written for were twelve kilobytes - a
glyph chunk in each of four font drivers, and a batch of overlays in the
Vulkan one - and they had been there for years, because a large local
array compiles, and runs, on a machine with a desktop's stack. It is
only a problem where the stack is smallest, which is the platform
nobody runs the suite on.
The sweep reads the release objects the menu-harness job already builds
and sums the probes a prologue makes, since a frame that size is built a
page at a time. Four functions are over and stay that way, each recorded
with what it is: a CRT mode switch, the shader cache written and read
once a pass, and a preview probe. Anything else over four kilobytes
fails, and a recorded one that grows fails too.
Checked by putting the twelve kilobyte glyph chunk back: the sweep names
it and the run fails.
glslang_read_shader_file_internal() recurses once per level of include
nesting and carried three PATH_MAX_LENGTH arrays, so every level cost
six kilobytes of stack and the frame measured 6312 bytes - the largest
in gfx/ and past what this tree allows. Nothing bounds the nesting, so
a shader that includes itself walks the stack down six kilobytes at a
time.
The three become one allocation the level owns and frees as it unwinds.
The frame is a few hundred bytes; the memory is the same per level and
comes off the heap, where running out says so rather than corrupting
whatever is below the stack.
The sizeof() that sized those arrays goes with them - a pointer's
sizeof is eight, which would have truncated an include's name to seven
characters, so the lengths are named.
An empty source ring may still have a native pass in flight. Apply late format changes through the existing wrapper stop handshake and restore its previous running state. Preserve queued native samples and keep synchronization off the steady-state path.
The same three arrays the gl drivers had, missed when those were moved
because this one spells them float rather than GLfloat: a line of text
was built into vertices, texture coordinates and a colour per vertex for
MAX_MSG_LEN_CHUNK glyphs, all of them locals. Measured with a PowerPC
compiler against PSL1GHT's own headers, rsx_font_render_line() carried a
12688 byte frame where this tree allows four thousand.
They move into the font's own state, which is calloc'd once, and the
frame it is inlined into comes to 512. Nothing about what is drawn
changes.
Forget primary and extra-channel history after source or pending-output discard and after the consumer stops. Add allocation-free nearest and CC int16 resets, preserving backend configuration and native formats. Verify fresh-stream equivalence across all three backends and the scripted frontend.
vulkan_render_overlay() staged each batch of sixteen overlays into
arrays of its own: two of buffer ranges, one of descriptor sets, and a
vk_descriptor_batch, which is sixty-four writes and thirty-two buffer
and image infos - better than four kilobytes on its own. The frame came
to 6712 bytes, the largest in gfx/ and past what this tree allows. The
comment above it said the batch of sixteen was chosen to stay within
stack arrays, which is the constraint this removes.
They move into the overlay state vk_t already carries, so the size of a
batch stops being a question about a stack frame. Overlays are drawn
from the thread that draws, one batch at a time, so one set is enough.
The batch struct is declared above vk_t now, because vk_t holds one.
Own ring reads, ordered controls and partial WSOLA output in one native stage. Borrow inactive spans directly, using one-frame scratch only at physical wrap or unaligned starts. Drain layout tails before changing their tag; explicit discontinuities discard retained history.
Wire the stage into common and Griffin builds. Verify partial acknowledgements, concurrent direct views, reset and allocation failures, and native sinc/HQ chains against direct source-position processing. Frontend activation remains separate.
Each of these built a line of text into three arrays sized for
MAX_MSG_LEN_CHUNK glyphs - vertices, texture coordinates and a colour
per vertex - and every one of them was a local. Twelve kilobytes of
stack frame, measured at 0x3148 in gl2 and 0x3188 in gl1 and gl3, where
this tree allows four. Half of it was the colour array: six colours
copied sixty-four times to say one colour.
They move into the font's own state, which is calloc'd once. The frames
come back at 0x158, 0x1b8 and 0x178 - a few hundred bytes - and the
arrays cost their twelve kilobytes once per font rather than on every
line drawn. A font renders on the thread that draws and one line at a
time, so one chunk is enough.
Nothing about what is drawn changes; the same bytes go to the same
place, from a different address.
gfx_display_draw_texture_slice() drew its nine pieces one at a time,
nine of the eleven places in this file that reach the display driver.
Every one of them set a viewport, began and ended blending, and on
Vulkan allocated a uniform buffer and a descriptor set - for pieces of
a single texture that differ only in where they sit.
They are a three by three grid: four vertical lines and four horizontal
ones, in vertex space and in texture space, with each piece the
rectangle between two of each. Walking that grid emits one triangle
strip, the pieces joined by a repeated vertex at each seam that the
rasteriser drops as zero-area, and one draw carries all nine. A colour
is emitted per vertex now, because the caller supplies four for a
single quad's corners and fifty-two are submitted.
Ozone's menu goes from 54 draws a frame to 21. The comment that said
this would need the coordinates reworked for the strip was right, and
that is what this is. The old file also read past the end of the four
colours it was given; that is gone with it.
Extend layout boundaries with a coherent Q16 tempo, active mode and explicit reset request. Count retired resets so discarding queued audio cannot erase discontinuities. Keep the legacy layout reader free of transport bookkeeping.
Exercise concurrent publication and native WSOLA/sinc chains against direct source-position controls, including wrapped counters, partial output and skipped reset boundaries.
Allow all eleven canonical pipeline slots in the engine, transitions and stream adapter. Preserve the native sample layout and a shared search offset without requiring a compacting pass.
Exercise sparse layouts against compact references, highest-slot search, odd-width native sinc chains, fragmented transitions and allocation-free reset. Existing processing loops are unchanged.
The macOS build fails on both uses of this property: one in
MetalDriver's own implementation through self, one in Overlay through
a MetalDriver* it has cast. Neither could see it, because the
declaration had landed in @interface Context - a class that never
refers to it.
Moved to @interface MetalDriver, beside the other per-frame state it
sits with. Nothing else changes: both call sites were already right
about what they were talking to.
Checked, rather than assumed from the two lines the compiler named:
both uses resolve to a MetalDriver, and no other property in this file
is declared on more than one class. Not compiled here - there is no
clang in this container, so the macOS build is what confirms it.
For canonical 5.1 and 7.1 prefixes, copy native source samples once and clear only absent speaker positions. Keep the general remapper for other layouts. Verify native bit preservation, guard bytes, zero-length calls and non-prefix fallback.
Keep the unwritten tail in existing native output scratch and finish it before consuming new source. Retry without rerunning SRC, mixing or gain, respect device readiness, and abandon retained output at stop, pause, underrun or write failure. Back off zero writes and account for retained frames without reporting them as dropped.
Publish bounded layout boundaries before their audio positions and cap consumer passes at each boundary. Keep producer and consumer ownership separate, retire discarded boundaries and release startup priming under metadata pressure. Recompute availability after underrun discards so empty input cannot replay stale scratch.
Makefile.test's 'all' runs the libcheck suites and then the concurrency
and I/O samples, and what those cost depends on the machine: the
single-producer queue sample paces itself against however many
processors it is given, and the atomic sample skips itself below two.
The suites are bounded work, so they are 'unit-tests', which is what the
job with a fifteen minute budget asks for; 'all' still runs both for
anyone who wants the samples.
video_thread_frame was not the only entry that had drifted from what a
release build produces: video_driver_frame measures 9447 against a
recorded 9225, and audio_driver_sample_batch 394 against 378. Neither
grew past its budget, so nothing failed - but a quarter over a number
the tree left behind is not a quarter over the tree, which is what the
file says these are for.
Both move to what they measure. The stack-frame numbers stay where they
are: two of them are recorded larger than this compiler produces, and
lowering them would tighten the margin against whichever compiler runs
the check - which is the thing the margin exists to absorb.
The last of what gfx_widgets_worker_step() read from the settings while
the video thread ran it. Both are strings, so a read there is worse than
a torn number: the main thread rewrites the buffer the reader is walking.
The frame carries them, and the wrapper points them at the slot's own
copies as it hands the frame over, as it does the statistics text - the
handover is the main thread's, so the copy it makes is not a race.
The copy is skipped when it would write what the slot already holds,
which after the first frame is always: these are paths, and the settings
behind them change when somebody changes them.
video_thread_frame grows 128 bytes for the call and the reference moves
with it. The reference had drifted first - it said 1894 where the tree
measured 2246 before this - so the budget was nearly spent by changes
that did not move it, which is what the file asks for.
Makefile.test's default goal is win32-configs, so the workflow step that
means to build and run every libcheck suite was building the Windows A/W
matrix instead. It asks for 'all' by name.
Running it turns up three source lists that no longer link. test_utils,
test_hash and test_rjpeg each name vfs_implementation.c and file_path.c
twice, which is a duplicate definition of everything in them; test_hash
wants features_cpu.c for the dispatch lrc_hash.c does, and test_rpng
wants encoding_crc32.c for what encoding_deflate.c calls. With those in
place the twelve suites build and pass.
Exercise ownership before release, short cached spans, copy fallback and cursor overflow at every physical ring offset. Stress framed borrow/copy consumption concurrently with publication, and compare contiguous and wrapped native multichannel device output.
Route stereo accumulator and native rewind recording through the bounded layout-aware helper. Direct stereo spans are too short for a surround recorder and can be read beyond their valid samples. Reuse shared conversion staging and remove the obsolete int16 recording and pipeline scratch regions without moving surviving arena offsets.
-setupCameraSession stores a freshly allocated AVCaptureSession and
AVCaptureVideoDataOutput into strong properties, whose setters retain
what they are handed, so each object stood at two references and the
one dropped when the next init assigned over it left the previous
session alive. The manager is a singleton that outlives a driver
instance, and the session holds the capture pipeline, so a core started
with a camera a second time left the first one running. Releasing the
allocation's own reference once the property holds it lets the old
session go where it is replaced.
apple_mrc_ownership_check.py grows a second pass for the shape:
a retaining property handed an expression that is already +1, with
nothing releasing it nearby. It reads the same way as the first pass
and is a report rather than a gate. The line is correct under ARC,
which is what lets it through review of a file built manual-retain-
release.
The power-of-two byte ring cannot hold an integral number of 22- or 44-byte canonical frames. A partial byte write followed by a dropped tail corrupts subsequent channel alignment. Add a producer-side whole-frame write that retains cached cursor ownership, use it for wide audio, and wait when less than one frame is free. Keep stereo on the existing byte write path.
-finishBrowsing: copies the browser's service array so it can enumerate
it without racing the delegate callbacks that mutate the live one, and
-copy hands back a reference the method owns. It runs on the task
worker, which has no autorelease pool, so the snapshot is released
where it is finished with rather than handed to a pool that is not
there. A discovery scan no longer leaves the array behind.
The two allocation failures inside the enumeration leave the loop
instead of the method, so the release sits on the one path out.
The methods behind -TXTdataFromNetplay: hand their NSData straight to a
dictionary literal, which retains it, so each one autoreleases the
reference its allocation carries. Nine of the ten did not, and that
record is rebuilt on every announce, so a session left publishing
accumulated them. -content_crc and -frontend already returned data the
pool owns and are the shape the rest now follow.
The NSNetService and NSNetServiceBrowser stored in the strong service
and browser properties release their own reference once the property
holds them, for the same reason. RARCH_RELEASE and RARCH_AUTORELEASE
carry the ownership rather than bare messages, so the file reads the
same way under ARC, where they compile to nothing.
The three Vulkan context drivers answered GFX_CTX_FLAGS_ADAPTIVE_VSYNC
by walking vk->context.present_modes for FIFO_RELAXED, and that array is
rewritten by vulkan_create_swapchain() - on the video thread, from
vulkan_frame() through set_resize - while the main thread is the one
asking, through menu_shader_get() among others. Sixteen slots read while
they are being filled.
The answer is worked out where the array is filled and published as an
atomic the context drivers load, so nobody walks live swapchain state
from another thread. The walk was reading all sixteen slots rather than
present_mode_count of them anyway, so it could find a mode the surface
no longer offers.
Process staged recording batches in audio-sized chunks so scratch capacity does not grow with an arbitrary core batch. Preserve the direct matching-int16 path and narrow float samples once. Cover exact recording output, bounded callbacks, allocation failure and scratch reuse.
Converting a widened int16 block to float in the same staging buffer overwrites unread input. Build canonical frames in the ring format instead, converting only the two populated slots and preserving native-format copies. Reuse the bounded scratch without extra allocations or a format round trip.
spirv_cache_compute_hash() sums each stage source with encoding_crc32()
and names the entry from the two sums and the two lengths. That folds
with a carry-less multiply on x86 and the CRC32 instructions on ARM, so
naming an entry reads the sources at about 19 GB/s where a SHA-256 of
them ran at 1 GB/s, and the third buffer both stages were copied into
to be digested as one run is gone with the copy: 8 MiB of stage source
is named in 0.87 ms in place of 10.4 ms.
Four fields put 128 bits in the name, so two unrelated shaders land in
one entry only by agreeing on both sums and both lengths at once. Names
from the digest are 64 characters and these are 32, so an entry written
by an older build is never read; those files are inert rather than
wrong, and go when the cache directory is cleared.
Check candidate ranking against independently gathered ring windows across native formats, layouts, masks and wrapped searches. Add an optional warmed stream benchmark with configurable tempo and trial duration so optimization proposals can be measured against the same fixture.
main_flags says of itself that only the main thread writes and reads it,
and one bit of it broke that: video_context_driver_get_refresh_rate()
tests VIDEO_FLAG_CRT_SWITCHING_ACTIVE, and the threaded wrapper calls it
from the video thread through the driver's get_refresh_rate. A
read-modify-write of that word on the main thread - setting the widgets'
rewinding bit, once a frame - races the read.
It becomes a retro_atomic_int_t of its own, written at the two places
that turn CRT switching on and off and read through an acquire
wherever it is asked about, so main_flags is what its comment claims
again.
Which pass a parameter belongs to follows from the order the sources
sit in, so the shader parameters dialog resolves again after moving a
pass rather than shifting the stored index on each parameter by one.
An id both of the swapped passes declare belongs to whichever of them
ends up first, which the shift cannot work out: it hands the parameter
to the pass that moved down, and the dialog then lists it, and resets
it, under the wrong one.
A resolve returns every parameter to its initial value, so the values
in hand are carried across by id around the call and the ones the user
has set survive the move.
Match the stereo suspension contract before splitting or folding native multichannel input. Preserve layout metadata and accepted frame counts while avoiding staging allocations and sample work for discarded frames. Cover native stereo/discrete destinations in the frontend harness.
Fold fallback batches at the same chunk granularity as stereo submission, reusing bounded native staging instead of growing it to the whole batch. Preserve partial-consumption reporting and validate large versus fragmented stereo-device output in both native paths.
gfx_widgets_worker_step() is called from the threaded wrapper's own
loop, so the settings it reads there are read on the video thread while
the main thread writes them. menu_ticker_speed comes from
video_frame_info_t now, as the rest of what a frame is drawn with does.
directory_assets and path_font are still read from the settings a few
lines below, and are not the same problem: gfx_widgets_iterate_frame()
compares path_font against last_font_path every frame to notice a font
change, deliberately from the thread that could rebuild it. Carrying
two PATH_MAX_LENGTH strings per ring slot to fix that would put back
most of the memory the deferred ring just stopped using, so who watches
that path wants deciding rather than latching.
action_ok_close_submenu() and action_ok_cheevos_toggle_hardcore_mode()
are bound only by the achievement entries in the ok-callback table, so
they are compiled under the same guard those entries are. A build with
achievements off has neither, rather than two static functions it warns
about and discards.
Reuse one publication-sized native staging block for large threaded multichannel batches. Keep recording once per source batch and avoid allocation for suspended or gated publishes. Verify canonical slot preservation and bounded staging in the ring harness.
The last of these. metal_set_texture_frame() read menu_linear_filter
from the settings, and the video thread applies it from
thread_update_driver_state() with the main thread free to write it -
the same race the gl drivers had. renderFrame: latches what the frame
carried and the setter reads that.
Not compiled: there is no clang with Foundation and Metal here. What
sank the RS90 one of these was a field read and never written, which
compiles cleanly, so the check that would have caught it is the one
done here - the property is declared once, written once where
video_info is in hand, and read once, and renderFrame: has a single
implementation. A macOS build is still what confirms it.
The expansion capture keeps the pragma-only run of every file, not only
of the leaves. What a file contributes there is its own pragma lines
and those of the files beneath it, which is a few hundred bytes rather
than the flattened source, so holding a second copy of a nested run
costs about what a leaf does - the memory that keeps the full form to
leaves is not in play. A pass that includes a header another pass
already expanded splices it instead of scanning it again, and resolving
the parameters of a twelve-pass pack costs about half what it did.
Split inline multichannel input one stereo flush at a time. Extras are marked consumed by each flush, so splitting the entire batch lost the remaining channels after the first chunk. Bound staging to a chunk and preserve fallback/partial consumption. Add large-versus-fragmented frontend regression coverage.
Report when a consumer can bypass the native stream without abandoning pending output. Keep EOF distinct until reset. Verify one-frame backpressure and reused WSOLA/sinc chains after discarding pending native output and resetting SRC history, without processing allocations.
gl3.c failed pass 2 on every run, naming members declared behind
HAVE_SLANG, and that was taken for a source bug for a long time. It is
not: configure turns HAVE_OPENGL_CORE off when slang is off
(check_enabled SLANG OPENGL_CORE in qb/config.libs.sh), so the file is
never compiled in the configuration the check was compiling it in.
It gets the pairing the real builds have. gl3.c, shader_gl3.c,
slang_process.c and glslang_util.c are checked with HAVE_SLANG,
HAVE_SPIRV_CROSS and HAVE_OPENGL_CORE defined, and all of them pass -
so a genuine failure in one of them is now visible instead of arriving
under a failure everybody had learned to expect.
Allow caller-owned output storage to be filled only up to the current downstream frame budget, including EOF. Preserve pending spans and full-capacity API behavior, and make zero-budget requests non-mutating. Verify direct-call equivalence and the real native SRC chain under varying limits.
glslang_parse_meta() reads lines beginning with '#pragma' and skips
every other one, so the walk that harvests shader parameters asks for
those lines alone. glslang_read_shader_pragmas_cached() follows
'#include' as before and emits only the pragmas, leaving out the
flattened source, the feature defines and the '#line' directives that
the compiler needs and the metadata reader does not: a twelve-pass pack
builds a few hundred lines this way in place of a hundred thousand, and
resolving its parameters costs about half what it did.
The expansion capture in the include cache holds the pragma-only run of
a leaf beside its full one, so the two callers share the cache without
sharing a shape, and a helper that declares no pragmas is captured as an
empty run rather than rescanned per pass.
Exercise real float/int16 WSOLA and sinc/HQ processing with bounded SRC submissions, partial source acknowledgement and short simulated sink accepts. Compare fragmented output against a batch-reference chain across stereo/eight-channel layouts and dynamic ratios. Run the fixture in the existing scalar/SIMD sanitizer job.
Use transition staging only during exit. The engine already retains the overlap needed before a drain gap, so steady active output needs neither a transition-ring copy nor an extra hop of holdback. Verify per-call engine equivalence and preserve complete transition streams.
Two things here, and the first is a fix to what the last one of these
left behind. The RS90 driver read vid->frame_softfilter_type in
set_filtering() and nothing ever wrote it, so its software filter sat at
POINT whatever the setting said. It compiled, which is all that was
checked of it.
The second is what an audit of the other entry points the video thread
runs turned up. thread_update_driver_state() applies set_texture_frame()
and apply_state_changes() from the frame path, with the main thread free
to run, and the drivers answering them read settings the menu writes:
menu_linear_filter in gl1, gl2 and gl3, and the IPU keep-aspect and
integer-scaling pair in the Dingux and RS90 ones. They come from
video_frame_info_t now, latched as each driver draws.
metal.m reads menu_linear_filter the same way and is not touched: it
cannot be compiled here, and the RS90 line above is what shipping one of
these unbuilt looks like.
The include cache keeps a file's bytes, so a preset reads each of its
helpers once; expanding them is what it still pays per pass, and a
25-pass shader pack runs the expander 287 times over 43 files. A file
reached through '#include' produces the same lines wherever it appears
- the '#line' directive returning to the parent's position is written
by the parent, after the recursive call - so the run a leaf include
produces is captured into its cache entry the first time and spliced
in whole on every later one.
Only a file that includes nothing itself is captured. The run of a
file that does include is mostly the runs of the leaves beneath it,
which are held on their own, so keeping a second copy costs several
times the source in memory to save a scan; leaving those out holds the
cache to about a fifth of the bytes it already keeps, and is the
faster of the two. shader_line_buf_reserve() carries the capacity
growth that the batch append and the new block append share.
Bind native output storage and expose explicit partial-consumption acknowledgement. Preserve pending frames under backpressure, latch exit and EOF requests, and reject mixed direct/bound API use. Add no sample storage or copies; validate against direct adapter streams with short downstream accepts.
Own the engine, bounded transition tail and one staging hop under a single consumer. Preserve pending output and drain gaps across backpressure, finish exits before re-entry, and provide EOF/reset semantics without processing allocations. Cover fragmented transitions and direct engine reference output before frontend wiring.
Comments in the slang expander, the gl3 shader backend and the Vulkan
driver that explained a construct by what it replaced now describe what
the construct does. The zeroing initialisers in shader_gl3.c say which
fields start at zero, the hardware render interface in vulkan.c says
where it lives rather than where it does not, the quad descriptor
update says when it allocates, and the pipeline array lists what it
holds.
The line-buffer split in glslang_util.c gets the note the code needs -
growing either array of a combined block splits the pair - in place of
a note to self, and the banner over the helpers it once separated goes.
retro_atomic.h, retro_spsc.c, retro_eventcount.c and tpool.c have no
unit test under test/, and in the standalone libretro-common tree
nothing else compiles them at all, so a break in any of them surfaces
only once RetroArch is built. threads-samples builds and runs the four
under AddressSanitizer and UndefinedBehaviorSanitizer,
threads-samples-tsan repeats them under ThreadSanitizer, and all runs
both.
retro_atomic_test reports a failure when no reader lands inside a
write, which cannot happen on one processor, so it is skipped there
rather than failing a machine that has nothing wrong with it.
gx_gfx.c is the driver the overscan settings were moved into the frame
descriptor for, and the one nothing could compile when they were: it was
consistent by inspection and nothing more. The libogc stub grows to what
it names - GX, VI, CONF, the LWP calls libretro-common's gx_defines.h
maps the OS ones onto, and the matrix helpers - and the lane compiles it.
Signatures and values follow libogc's. Fetching the SDK instead was the
other way and is worse: ctr_gfx.c does not compile against a current
libctru, whose gspSubmitGxCommand() changed shape under it, so a lane
built on someone else's HEAD goes red for their reasons rather than
ours. Pinning the surface here keeps what these lanes check - that this
tree's console code names only what it has arranged to have.
MEM_K0_TO_K1 casts through uintptr_t rather than u32: a pointer is 32
bits on the console and 64 on whatever runs this.
slang_preprocess_parse_parameters_meta() takes the pass its metadata
was harvested from and records it on every parameter it adds, so that
the pass field of a parameter names the first pass declaring that id;
a later pass naming the same id merges into the entry already there
and adds nothing. slang_preprocess_parse_parameters() and its cached
form carry the index down from their callers - the resolver walks the
passes in order, and slang_process() has the pass it is compiling.
This is the field the desktop shader dialog groups parameters by, and
the one its per-pass parameter reset selects on, so both act on the
pass a parameter is listed under.
Retain a fixed native tail and overlap it with incoming audio at explicit boundaries. Copy continuous block middles directly to output, support partial input/output and EOF, and reset without allocation. Validate fragmented streams and actual high-tempo drain gaps before frontend integration.
A PAUSE is worth an order of magnitude more cycles on some processors
than others, so a fixed iteration count is a different amount of time
on every machine. The budget is now three microseconds and the count
comes from a relax timed at resolve time, taken as the best of a few
short passes so a preemption in one does not stretch the answer, and
clamped so a mismeasurement can turn the spin into neither a no-op nor
an unbounded burn.
Three microseconds is where the knee sits: on Windows 11 x64, where a
relax costs 11.8ns, that is 256 iterations, and round trip and
broadcast per waiter go 0.27 and 0.96us at 64 iterations to 0.06 and
0.11 at 256, with nothing above it buying anything.
RETRO_EVENTCOUNT_SPIN_US moves the budget and is honoured whatever the
processor count, so the calibration stays reachable on a machine that
would otherwise skip the spin; RETRO_EVENTCOUNT_SPIN still sets the
count outright. retro_eventcount_spin_iters() reports what was chosen,
and the bench prints it against the relax it measures.
It was the one console driver left out, and the one whose threaded video
is on by default, so a setting that moves into the frame descriptor
lands there first and is seen last. The libnx stub grows to what it
names - a window, a framebuffer, the seven calls it makes of them - and
the lane compiles gfx/drivers/switch_nx_gfx.c against it.
A stub is what the arm lanes already stand on, and carries the same
caveat the file opens with: it does not prove devkitPro's headers match,
it proves this driver names only what it has arranged to have, which is
what broke here before. R_SUCCEEDED stays for mem_stats.c, whose lane
caught its absence when this stub was first rewritten.
Blend caller-owned transition spans with shared channel weights and stable fragmented output. Preserve native lanes and exact in-place operation without allocation or additional engine state. Keep scheduling and history ownership with the caller.
The Win32 backend keeps its own waiter list -- blocks on the waiters'
stacks, the low bit of the head as the list's spin lock -- and
delegates only the sleep, to the best primitive ntdll offers:
NtWaitForAlertByThreadId on 8 and newer, NtWaitForKeyedEvent back to
XP, and a per-thread auto-reset event on anything older. All three are
resolved at runtime, so one binary covers every version and imports
nothing beyond KERNEL32 and msvcrt; the floor is Win98/NT4, set by the
compare-and-swap the list rests on.
A waker takes a block with a fetch_or of WOKEN and issues a wake only
for one that has already set ASLEEP, which is what makes the
keyed-event rendezvous exact. The link comes out of a block before its
claim, and a waiter whose unlink finds the block already taken waits
for that flag before its frame goes away: publishing WOKEN is precisely
what lets the owning thread free the stack the walk is reading.
Waiters spin on the flag word before committing to the kernel, on
multiprocessor only, and that spin is what makes this fast. Measured on
Windows 11 x64 with four waiters on one object, round trip and
broadcast per waiter: 4.21 and 0.94us at no spin, 0.27 and 0.96 at 64
iterations, 0.06 and 0.11 at 256, unchanged at 512 and 4096. 256 is
the default, where a spin that finds nothing costs about what the wake
syscall it avoids would have. RETRO_EVENTCOUNT_SPIN overrides it; an
iteration count does not transfer between processors, so this wants to
become a microsecond budget sharing rthreads' selector.
retro_eventcount_bench measures the cases that separate the backends
and times the relax itself, and the test grows a broadcast-under-load
lane where consumers re-park in a tight loop against a producer that
keeps moving -- the only shape that reaches a waker walking a block
whose owner is on its way out.
video_shader_resolve_parameters() stamps what it resolves from - for
every pass that has a source, the path, its size and its modification
time - and walks those sources only when the stamp no longer describes
what the passes point at. A call that would read the same set again
keeps the parameters it already holds and resets their current values
to the initial ones, which is what a walk leaves behind. Nudging the
pass count in the menu appends or drops a pass carrying no source of
its own, so it costs a stat per pass rather than a read and a parse of
every source.
The stamp is taken ahead of the reads, so a source rewritten while the
walk is in progress is read again on the following call rather than
recorded as parsed. It covers the pass sources, which is the set the
shader file watch follows as well; a file reached through #include is
picked up once the pass source including it moves, or when the preset
is applied again. It is appended to struct video_shader so that every
offset ahead of it stays where it is, and every allocation of that
struct zeroes it, so a shader that has not been resolved yet always
resolves.
Preserve pending synthesis, overlap and unrepresented lookahead on exit without allocation or sample conversion. Report skipped-source boundaries once so the runtime owner can reconcile transitions, and require reset before processing resumes.
Cover fragmented drains, startup and wrapped tails, high-tempo gaps, reset, bounds and allocation guards in both native lanes.
The arm lanes check that one file's headers resolve for a platform, not
that the driver that platform draws with still compiles - so a driver
reading a setting that moved into the frame descriptor, or naming a
struct member that moved, is green here and breaks on the console. That
happened to the Dingux, RS90, Switch and OdroidGo2 drivers, and nothing
in CI saw any of it.
The OdroidGo2 gl2 build wants the GL headers the gl2 arms already use,
and the Dingux and RS90 drivers want SDL 1.2's, so the job installs
libsdl1.2-dev. A lane whose headers are not on the runner says so and
does not fail the run.
The Switch driver is not here: its headers are devkitPro's, and the stub
that carries the arm lanes covers what one small file needs rather than
a video driver. It wants either libnx fetched in the job or a stub grown
to fit, and is worth doing - it is the one of the four whose threaded
video is on by default.
Add a single-owner float/int16 engine with explicit consumption and production counts, fractional hop scheduling and one channel-aligned search offset. Retain only a fixed source neighborhood, overlap and at most one partial output hop; skip source directly at high tempos. Full hops emit into caller storage, with no processing or reset allocations.
Use the strongest eligible overlap channel to avoid stereo cancellation and allow callers to exclude LFE. Reuse the shared correlation kernels and keep integer synthesis fixed point. Add C89 allocation, fragmentation, scheduling, reset, channel-mask and signal tests plus scalar/SIMD sanitizer CI.
Include the engine in shared and Griffin builds without activating it in playback. Runtime transport controls, epoch ownership and entry/exit tail handling remain separate integration work.
set_filtering() and set_aspect_ratio() no longer wait for the video
thread, so the drivers that answer them by reading settings read them on
that thread while the menu writes them: gl2 under HAVE_ODROIDGO2 takes
video_ctx_scaling, the Switch driver video_scale_integer, and the Dingux
and RS90 ones their IPU filter and softfilter types. What had kept those
from racing was the main thread waiting while they ran.
They come from video_frame_info_t now, latched by each driver as it
draws a frame and read from there when the setter runs, as the HDR
values and the recording state already do. video_ctx_scaling and the two
Dingux ones join the descriptor; scale_integer was already in it.
Built for each platform it touches rather than reasoned about: the
Dingux and RS90 drivers against SDL 1.2, the Switch driver against
libnx's own headers, and gl2 with HAVE_ODROIDGO2 defined.
Expose a default-off advanced setting through the shared resampler descriptors and reinitialize audio when it changes. Resolve software HQ activity at initialization for sinc at nominal ratios of at least two, and apply that policy to front and extra-channel resamplers in both sample formats. Preserve it through fallback recreation and clear it on teardown.
Keep driver and unity SRC bypass available when HQ is inactive; active HQ stays in software. Reuse existing kernels and capacities without sample-format round trips. Document the optional CPU, memory and filter-delay costs.
Verify registration and reinit metadata, policy transitions, failure and teardown, and exact multichannel routing with HQ off and on. Keep the test-only settings fixture out of production build lists.
Exercise one through eight extra channels in both native lanes, with independent planar input, per-pair reference histories, format mismatches and dynamic ratios. Check exact routed samples, production counts, output canaries and allocation-free processing. Run the fixture with scalar and SSE selection in CI.
Reject channel counts outside the owned resampler array and reject frame or byte capacities that overflow size_t. Check growth before allocating input storage, including the corresponding output bound, and handle front-capacity growth independently. Keep processing and sample conversions unchanged.
Exercise extreme frame counts, invalid layouts, output-capacity growth, cleanup and recovery in both native lanes.
Keep only the float or int16 pair buffers selected by the extra resampler. Lane changes already rebuild extra state; two extras still use the direct path without pair scratch. Processing and interleaved format-conversion buffers are unchanged.
Exercise native-lane changes, capacity growth, layout transitions and every required allocation failure. Retire freed allocation records in the fixture so repeated lifecycle checks also work without allocator address reuse.
The direct stereo-pair path does not use gather/scatter scratch. Allocate only its native input/output buffers and retain scratch for other layouts. Keep layout-change cleanup and allocation-failure recovery intact.
Test allocation counts, layout transitions, failure of each required allocation, and existing native processing with absent scratch.
Both handlers stepped the pass count under a guard and then called
video_shader_resolve_parameters() whether or not the guard had let the
count move. That call reads every pass's source to find its #pragma
parameter lines, so holding right at GFX_MAX_SHADERS, or left at zero,
was a read of each of them for every keypress, with nothing to find that
was not found the press before.
It goes inside the guard. A count that moved resolves as it did; a count
that could not move does not, and the menu flags and the modified mark
are set either way, as before.
When there are two extra channels, pass their interleaved native input and output buffers directly to the existing resampler. Preserve format conversion, phase state, frame limits and pending bookkeeping.
Verify independent float/int16 reference output across format choices, DRC and short batches, and keep pair scratch untouched on the direct path.
Copy unity-rate extras directly to their native output buffer, or convert once into that buffer when the front path requires another format. Preserve history reset, input/output bounds and pending-frame bookkeeping.
Cover odd channel counts, both formats, special float values, bounds and untouched pair scratch. Existing active-resampling and bypass-history tests remain in place.
Reject non-positive, non-finite and unrepresentable nominal ratios in both native backends before tap sizing or table allocation. Reserve clock headroom for residual phase. Classify non-finite values through integer bits so validation survives fast-math.
Cover invalid inputs, phase-clock boundaries and factory recovery while preserving valid-stream output and unchanged processing loops.
discrete_multichannel_test segfaults, and has been: the AC-3
bit-stream case dereferences a null pointer before printing anything,
which is why the CI log shows a clean build and then exit 139 with no
output to say where.
audio_driver_extra_resample() borrows the front resampler's vtable to
reset the extra channels' own resampler instances. A bit-stream pass
does not resample the front pair, so audio_driver_init_internal()
never makes one, and audio_st->resampler is null - address 0x30 in the
report is the offset of reset in that structure. It is reachable for
any core with more than two channels that is bitstreamed, which is
exactly what that case exercises.
Guarded where it is borrowed. The process() call further down in the
same function is not exposed the same way: it runs only when the front
pair did resample, which is when a resampler exists.
What does not work is guarding the whole function - I tried it, and
three assertions fail. The extras still have to be resampled on the
bit-stream path because they are what the encoder is handed; bailing
out sends it the upmix instead of the core's own channels, which the
test then catches. The narrow guard is the right one and the test
proves the broad one wrong.
Verified: the harness passes plain and under ASan and UBSan, as do the
format, latency-contract and WASAPI pacing ones beside it.
video_shader_resolve_parameters() read each pass source with
filestream_read_file() of its own, so a source that a driver would go on
to ask for was fetched two different ways. It goes through
video_shader_source_read() now, which leaves one call in the tree that
opens a shader source at all.
The path_is_valid() test in front of it goes with it: a source that is
not there fails the read, which is the same skip a step later. With one
door, what a source costs to fetch - read twice for one preset today,
once for the parameters and again to compile - is a question that can be
answered in one place.
Compare scalar and SIMD output across DRC changes and the return to nominal ratios. Extend active HQ coverage through the same return transition and run the AVX fixture on capable Linux CI runners.
retro_eventcount parks a consumer without putting a lock on the
producer's fast path. The consumer opens a wait window with
prepare_wait, re-checks its own predicate, then commits or cancels, so
the predicate stays inlined at the call site and never becomes a
callback. With nobody parked, notify is one read-modify-write, one
seq_cst fence and one load.
The fences on both sides are what close the lost-wakeup window: a
notify cannot read zero waiters while a consumer that has not yet seen
the new epoch is on its way into a park. Where the atomics are not
lock-free that handshake cannot carry the argument, so those builds
take the lock on every notify.
Linux parks on futex, Windows 8 and newer on WaitOnAddress resolved at
runtime so the same binary still starts on 9x and XP, and everything
else on an rthreads condition variable with the lock taken only while a
waiter is registered. RETRO_EVENTCOUNT_FORCE_SCOND selects that last
backend anywhere for testing.
libretro-common/samples/rthreads/retro_eventcount_test covers handoff,
the lost-wakeup window aimed at deliberately, and broadcast, on both
backends, under its own watchdog so a missed wakeup fails rather than
hangs. It runs in the native ASan/UBSan sweep and in a Clang/TSan lane.
retro_atomic_thread_fence_seq_cst() joins the acquire and release
fences on every backend: atomic_thread_fence(seq_cst) on C11 and C++11,
__atomic_thread_fence on GCC, MemoryBarrier() on MSVC, OSMemoryBarrier()
on Apple, __sync_synchronize() on the __sync backend, and the same
compiler barrier the other two fences already degrade to on the PS2 EE
and the volatile fallback.
It orders a store against a later load of a different location, which
an acq_rel read-modify-write does not, so a Dekker-shaped handshake can
be written against this header. retro_eventcount is the caller.
Reading a shader source, resolving an #include against the source that
wrote it, naming a source for the #line directives, and telling a slang
source by its name: all four went through file and path calls here, and
all four go through video_shader_source_*() now. What the preprocessor
holds is a name it was given and hands back, so file/file_path.h,
file/config_file.h and streams/file_stream.h leave this file with
nothing in it that opens or parses a path.
The include cache stays where it is and caches what comes back, so a
preset that expands the same source through several passes still reads
it once. gfx/drivers_shader has no file I/O left outside slang_cache.c,
which is a cache on disk rather than a source to compile.
Add an explicit HQ creation entry point while preserving the legacy default. Apply the option only to sinc; keep backend lookup, replacement and failure semantics unchanged. Processing dispatch and sample formats are untouched.
Test nominal ratio gating, fallback, replacement, independent instances and non-sinc behavior against direct backend initialization. Run the fixture under sanitizers in the Linux audio workflow.
Clear extra-pair histories once when entering unity bypass so they resume alongside the reset front pair. Reuse backend reset callbacks without reallocating or changing sample formats.
Test multiple pairs, repeated transparent bypass, stale-history rejection and allocation reuse.
Reserve enough output scratch for the front pair ratio bound before processing extra channels. Grow pair input scratch with input capacity independently of output capacity.
Test 8x native float/int16 resampling, changing batch sizes, exact agreement with a front resampler and warmed-up buffer reuse.
gl_glsl_load_source_path() read the pass source with
filestream_read_file(), which is the only reason this file knew about
streams or paths at all. It asks video_shader_source_read() for the
bytes of a named source now and owns what comes back, exactly as it
owned what the read gave it, and file/file_path.h and
streams/file_stream.h leave the driver.
Where the bytes live and how a name resolves is decided in
gfx/video_shader_parse.c, which is built wherever this driver is. The
door is one function rather than sixteen scattered reads, so what goes
through it - a cache that would spare the second read of a source the
parameter pass has already been through, or a fetch that does not
happen on the thread that asked - is a question for that one place.
Read only the coefficient row when the subphase is zero. Keep the integer MAC and endpoint rounding unchanged, without a scratch pass or sample conversion.
Extend baseline comparisons to active HQ and start ratio sweeps at the nominal ratio to exercise exact phases before fractional phase changes.
Track allocation and deallocation around both native SRC lanes and float reset. Enable the guard in the existing harness and CI path without adding production instrumentation.
Add explicit initialization for an experimental 384-tap preset at nominal ratios of at least two. Reuse existing float and Q1.30 integer kernels; keep legacy initialization unchanged and leave frontend exposure for separate integration.
Test bypass against baseline, fractional ratios, DRC sweeps, chunking and scalar/SIMD agreement. Include a baseline comparison script and document memory and delay costs.
Keep integer correlation and float scalar/SSE2/NEON kernels unchanged while selecting the float function once per instance. Retain caller-owned search policy and buffering. Add C89 kernel and DSP bypass tests.
Paused, the video thread draws nothing, so the word the frames publish
their answer to goes stale and the frontend would never hear that the
window had gone - which is why this asked the video thread outright and
waited for the reply, once for every iteration of a paused runloop.
It queues the question instead and reads the word, and the handler
publishes what the driver said to that same word, so the answer is there
on the next iteration - a frame later than a driver that drew would have
given it, and with nothing waiting on anything. The video thread is
woken exactly as often as it was before; it simply answers into the word
rather than into a reply the other thread is parked on.
wait_writable's packet wait had grown its own copy of the same
arithmetic - two packets' worth of time, its own floor of two
milliseconds, and no ceiling - while the write path a few hundred
lines away now asks wdmks_watchdog_ms() for exactly that. One question
with two answers, and the previous commit converted two of the three
waits and left this one.
All three ask the same function now. Whatever the policy should be, it
is in one place to change.
Nothing else was pending. Every finding from the audit rounds that I
judged actionable is in, and what remains on those lists is validation
rather than code: the AC-3 path has never met a receiver, and the
eventless, registerless wait has never met a device that would use
it.
It was the one setter held back, on the idea that its resize flag had to
be raised while the video thread was not drawing. It does not: the flag
is read and acted on inside vulkan_frame(), on the video thread, and
CMD_POKE_SET_ASPECT_RATIO runs on that thread whichever way it arrives.
Waiting for it only put the flag up a frame earlier. What the ratio
itself crosses threads through is an atomic word, which is what the
reads in video_driver_update_viewport() needed.
Every poke setter with nothing to say back now travels on the ring.
An audit's remaining finding, and it is right: the packet path still
waited a flat second for a packet to come back. At a 2 ms packet that
is five hundred periods before the wait gives up - a bound in name
only, and increasingly odd beside a driver whose other waits are all
derived from the stream.
There is one rule now: twice the duration of the thing being waited
for, floored at 4 ms and capped at 100. A packet wait gets twice a
packet, the WaveRT event wait twice a ring. Every wait here is a
watchdog rather than a schedule - the event or the completion normally
arrives first - so what it wants is a bound that scales with the
stream, tight enough to notice something is wrong and loose enough not
to trip on ordinary scheduling. The WaveRT side had already grown this
arithmetic by hand; it is one function now and both call it.
Two other things from the same audit.
The ActualBufferSize question they could not verify from what they
retrieved is fixed: the size is used as the driver gave it and a ring
that is not a whole number of frames is declined.
And their remaining concern - that the eventless, registerless wait
now yields for milliseconds where it used to issue ioctls - is fair
and I am leaving it. It is a real cost, it is bounded, and it is
strictly better than the kernel transitions it replaced. Which of the
two matters depends on how many devices have neither a notification
event nor a position register, and I have not seen one yet. That is a
thing to measure on hardware rather than to design around in advance.
Three from an audit. The first is a regression I introduced two
commits ago and had not thought through.
The eventless WaveRT wait yields and re-checks for room each lap. With
a mapped position register that check is a volatile word and polling
it freely is right - finding the room early is worth having. Without
one it is a DeviceIoControl, and a yield that returns at once because
nothing else wants the processor turns a six-millisecond wait into
thousands of kernel transitions. Replacing Sleep(1) with a yield loop
fixed the timing and quietly created that.
The two cases are separated now. Cheap source: poll on every yield.
Expensive source: wait the deadline out and ask once at the end -
which is what the deadline was computed from the cursor and the rate
for in the first place, so that it need not be checked on the way.
Second, and I was wrong about this twice. The WaveRT buffer's size was
being trimmed down to whole frames, which I defended as the same guard
the packet path applies to a transfer. It is not. Trimming a modulus
does not shorten anyone's ring: ours would wrap at N-r while the
hardware kept wrapping at N, the two diverging by r bytes every lap
and never coming back. That is worse than the misalignment it was
meant to answer, and silent with it. The size is used as the driver
gave it, and a driver that returns a ring with a partial frame in it
is told so and declined - there is no arithmetic that makes such a
ring play correctly, so pretending otherwise only moves the damage.
Third, a comment that said the fallback sleeps a millisecond, which
stopped being true when the sleep went. A file whose comments are this
load-bearing cannot afford stale ones.
Not taken: PACKETCOUNT, which the audit itself downgrades now that
presentation position is in - they measure different points in the
pipeline and presentation position is the one this driver wants. And
the counters, which are worth adding when there is a device whose
behaviour they would explain rather than ahead of one.
The last of the three host-clock uses in this driver that was not
essential, and the one an audit picked out.
Where a WaveRT driver offers no notification event there is nothing to
be woken by, so something has to be waited on. What was waited on was
a ring's worth of time: an upper bound, unrelated to how much room was
actually short, yielded through until it expired or room appeared.
It is computed now. The cursor says where the hardware is and the rate
says how fast it moves, so between them they say when the room the
caller could not fit will exist - short by 300 frames at 48 kHz is
6.25 ms, and that is what is waited, not "a while". Both callers pass
what they are short of: the write path what it still has to place, and
wait_writable what the frontend asked to fit.
Floors and ceilings, because the arithmetic can produce silly answers
at both ends: below half a millisecond the wait costs more than the
yield it replaces, and nothing here should outlive one lap of the ring
- past that something has stopped, and the caller's own lap bound is
what should notice rather than this.
Still a host clock at the end, because a wait with no event to wake it
cannot be anything else. What changed is where the interval comes
from: device progress rather than a fixed guess. On a device that
signals an event none of this runs at all, which is most of them.
The other two uses stay, and are not the same kind of thing. The
ambiguity guard needs an independent measure of elapsed time by
construction - a cursor modulo the ring cannot say how many laps went
by, which is why presentation position exists and why that guard is
dead code where it does - and it only ever discards a sample. The fit
measures device rate against host time, which is the quantity, and it
already prefers the timestamp the endpoint takes itself.
Two from an audit, both on the WaveRT cyclic path.
The accumulator adds the modular distance between two readings of the
position cursor, which is right for one wrap and cannot see more. A
process not scheduled for five laps of an 8 ms ring gets back a
position modulo the ring and no way to know five went by. The laps are
lost either way - nothing reconstructs them from a cursor - but the
worse part is what the delta then looks like: an ordinary step, of
ordinary size, entering a ten-second least-squares window as though it
had been measured. One stall moves the fitted rate by as much as the
stall was long, which is the absurd drift figure this driver has shown
before, from a different cause.
So the time between readings is checked against the time the ring
takes to go round. Past that the reading is ambiguous, and the
accumulator re-anchors instead of guessing - the count restarts and
the fit starts again rather than being handed a number that looks
monotonic and is not. Measured with this project's clock, like the
rest of the timing in this file.
And the event watchdog is derived rather than written down. A hundred
milliseconds was a number: at an 8 ms ring that is twelve times round
before it gives up, where the packet path a few lines away already
computes its own from the stream's duration. Two rings now, clamped at
both ends so a tiny buffer does not produce a watchdog that fires on
ordinary scheduling and a huge one does not produce one that never
fires at all.
Not taken from the same audit, with reasons rather than silence: two
of its findings - that there is no presentation-position support and
that the fallback sleeps - describe a tree from before the last two
commits, and both are done. Rejecting a non-frame-aligned
ActualBufferSize instead of trimming it is still, to me, the harsher
failure of the two on a miniport that is already misbehaving. And its
counters are worth having once there is a device whose behaviour they
would explain.
From an audit, and the one thing in it I had not thought of.
The count and the clock fit on a WaveRT pin have been built from the
cyclic cursor: a position that wraps with the buffer, accumulated
between reads, paired with a performance counter this thread reads
afterwards. Both halves are approximations. The count is only right
while the cursor is sampled more often than it laps - a process
descheduled through five laps of an 8 ms buffer loses five and cannot
know it - and the timestamp is taken at a different instant from the
position it is paired with.
KSPROPERTY_RTAUDIO_PRESENTATION_POSITION gives both properly: the
position from the start of the stream, which does not wrap, and the
performance counter the endpoint read at the same moment it read that
position. It is what this driver has been synthesising.
It is asked for once at init and used for the count and for the fit
where it answers; the cursor stays for the devices that have no such
property, and for saying where in the loop it is safe to write, which
is the job it is actually good at.
The structure and both new ordinals are checked against the SDK header
by tools/wdmks_wavert_check.sh, like the rest of the WaveRT interface.
Not claimed: that this changes anything on the HDMI output here, which
answered neither the position register nor - until it is run - this.
The log says which of the three sources a pin gave, so one line
settles it.
Sleep(1) was mine, in both the write path and wait_writable. Nothing
in this driver slept before the WaveRT path was written.
It is wrong here for the reason the ASIO teardown wait already says in
its own comment, a few files away and written the same week: Sleep(1)
is not one millisecond unless something has raised the timer
resolution - it is the scheduler's tick, about fifteen. At an 8 ms
loop that is a wait longer than the whole buffer, so the thing meant
to pause until room appeared was instead guaranteeing the loop ran dry
first. It is the background distortion again, arrived at from a
different direction, and it was in the code the whole time the event
path was being added to fix that.
Both sites go through one wait now: the notification event where the
driver gives one, and a yield otherwise. A yield hands the processor
to another thread that is ready and returns at once when none is, so
the deadline is what stops it spinning - and that deadline is measured
with cpu_features_get_time_usec(), this project's clock, rather than
counted in iterations, so it means one buffer's worth of time whatever
an iteration costs. It exits the moment room appears.
No Sleep call remains in this driver.
playlist_write_file() decides at the top whether to do anything at
all: it returns immediately unless the dirty bit is set, or the
on-disk format differs from the requested one, or the compression
state does. Those three flags are the only things that cause a
playlist to be written.
All three were updated before the temporary was moved into place, and
the dirty bit and the compression state were updated whether or not
the write had even succeeded. So a full disk, a read-only directory
or a failed rename left the playlist marked clean, in the requested
format and at the requested compression - every trigger cleared - and
nothing wrote it again. The entries stayed in memory, the file on
disk kept its old contents, and the next call to this function
returned at the early check. The error was logged; the retry was not
possible.
They move into the branch that runs after the rename returns 0, which
is the point at which the data is actually on disk.
playlist_write_runtime_file() had the narrower version of the same
thing: it guarded its flag clearing with wrote_ok, but still did it
before the commit, so a successful write followed by a failed move
also dropped the dirty bit. That file accumulates play time and
last-played stamps and is rewritten every time content is closed, so
the loss is history that has nowhere else to come from.
Same shape as the config_file save that cleared CONF_FILE_FLG_MODIFIED
after a dump that had written nothing. The commit is the rename, and
nothing that records success belongs before it.
Not fixed here, but found looking: task_cloud_sync_backup_file()
drops the result of the rename that moves a local file into the
backup directory before the server's copy overwrites it. If that
rename fails there is no backup and the overwrite proceeds anyway.
The fix depends on what the sync should do with that file afterwards,
which is a decision rather than an oversight, so it is left alone.
From an audit. The looped path's wait_writable slept a millisecond and
looked again, while the write path a few lines away already waited on
the driver's notification event where one exists. That is the wait the
frontend's audio thread ends up blocked on, so it was waking sixty
times a buffer to ask a question the device would have answered by
signalling.
It waits on the event now, and sleeps only where the driver offers
none. Bounded either way, so a device that has stopped returns nothing
rather than holding the audio thread.
The audit named two other things as blockers on the AC-3 path, and
neither is one in this tree.
A burst the pin takes only part of is already carried: the remainder
is held, resumed before anything new is encoded, and an incomplete
drain returns zero so the frontend offers the same frames again. That
was a real hole when the audit was taken and is not one now.
And the layout the AC-3 path sets is not an overwrite of the
frontend's - it is the answer to it. The layout hook says what this
driver wants handed to it, and on the bit-stream path that is the
encoder's input layout, which is what the frontend must then produce.
A 7.1 request folded to 5.1 is the encoder's limit showing through,
which is what A/52 carries, and it is reported rather than silently
ignored.
video_thread_get_overlay_interface() ran on the main thread and wrote
thr->overlay, which the video thread reads in the overlay commands and,
every frame, in thread_update_driver_state() when an alpha change is
waiting. Nothing ordered the write against those reads. The video thread
takes the table itself now, as it initialises the driver, and the
accessor only says whether there is one; a driver hands back a table of
its own that outlives the call.
With the pointer no longer crossing threads, the overlay's four setters
join the ring. input_overlay_set_vertex_geom() calls one per descriptor,
dozens for a pad, on every load and every rescale, and each of those was
a round trip to the video thread and back.
The layout the frontend is told about is the encoded one. A pin opened
for AC-3 carries two 16-bit channels and the encoder is handed float
frames in the layout it was built for, so the carrier's channel count
is not the answer to what upstream should produce.
Room and buffer size are answered in those same units. 1536 input
frames become one burst, which makes an input frame worth a carrier
frame and the two currencies differ by the width of a float frame:
carrier bytes understate the queue six times over on 5.1, and the fill
the frontend steers by is the one it wrote.
A burst the pin takes only part of is held and finished on the next
call, with nothing new encoded until it is out. A burst is one frame
to the receiver, so handing it the front of one and then the front of
the next loses the stream; a short write upward is the honest answer.
config_file_write() reports failure properly - a write error, a full
disk, a dump that could not allocate its staging buffer - but that is
only worth something where the caller looks. Four call sites did not.
Two of them are the core options save on deinit, and both logged
"Saved core options file to X" immediately *before* the write, then
discarded its result. So a failed save told the user their options
were saved, in the same words a successful one uses, and the changes
were gone at next launch with nothing in the log to say why. The log
line now follows the write and reports what actually happened.
frontend_salamander.c had the same shape: it wrote its config and
then unconditionally logged that the file had been written. A failure
there means the core it just located is not recorded, so the next
boot has to find one again - worth an error rather than a debug line
claiming success.
tasks/task_autodetect.c dropped the result of its controller profile
index write. That one is genuinely not fatal, since a missing index
just means the next connect falls back to the full directory scan,
but rebuilding it on every boot because the write keeps failing
should not be silent either. A warning, not an error.
No behaviour change on the success path in any of the four.
The play offset is accumulated in bytes and converted to frames on
demand. A driver whose position lands mid-frame reports a distance
that is not a whole number of frames, and dividing each distance on its
own drops the remainder every time; with a read on every write_avail
that is a consumed count running slow by up to a frame per read, which
the sink estimate reads as drift.
The WaveRT write gives up after the same number of bounded waits the
room request already uses, and returns what it took. A device that has
stopped playing frees no room however long it is waited on, and a write
that waits on one for good is a frozen frontend.
config_set_string() malloc'd each new entry node individually while
the parser slab-allocated its own, so a config built by setting
values got three allocations per entry and nodes scattered across the
heap, where a parsed one got two and a dense run of them. The
ownership machinery for pooled nodes already existed - teardown
branches on CONF_ENTRY_FLG_POOLED, and the pool's unwind is exact for
the most recent handout, which is precisely what a failed insert
needs to give back.
203701 inserts followed by a sorted dump, x86-64, -O2:
insert 45.9ms -> 41.0ms
sorted dump 21.2ms -> 18.6ms
peak RSS 49.1 MiB -> 46.1 MiB
The memory figure is the solid one: it is allocator metadata for 203k
small nodes, it reproduces to the tenth of a MiB run to run, and it
should be worth more on targets whose allocators are less forgiving
than glibc's. The timings are modest and the dump improvement is
node density showing up where it was predicted to. At 2000 entries -
what an actual retroarch.cfg looks like - none of it is measurable,
which is the expected result and not an argument against it.
config_unset() also stopped remeasuring a key it already has the
length of, on the way past.
Tests: pooled entries either side of a parsed one, across value
replacement, unset, and an append_conf that pilfers them - the mix
that would catch either a free() of a pool-interior pointer or a lost
slot. Invisible in a plain run, immediate under the sanitizers CI
runs this with. 48 checks.
Nothing else from this round. The streaming memmove() was raised
again as the clearest remaining win: it is bounded by total input,
because compaction only runs when the push contained a newline and
the retained tail is newline-free by construction, so everything
after the window's last newline arrived in that same push.
Instrumented, bytes moved over bytes in peaks at 0.49, and the case
put forward - a long line arriving in small packets - performs no
memmove at all. The ownership-chain splices cannot be made O(1) by
changing direction: concatenating two singly-linked lists needs a
tail pointer whichever way round it goes, and adding two is the
opposite of what the last commit was for.
The eight-millisecond minimum was a constant in audio_driver.c,
applied once before any driver saw the value - which is why it showed
up identically on WASAPI, ASIO and WDM-KS, none of which was involved.
Its own comment says why it exists: the setting's range starts at
zero, zero reached the drivers, and each did something different with
it. That is an argument about drivers mishandling zero rather than
about hardware, and it is weakest for the drivers that negotiate a
period with the device directly - those already floor themselves at
whatever their hardware states.
So it is audio_latency_floor now: advanced, one to sixteen
milliseconds, eight by default, which is what it has always been. The
constant stays as an absolute floor of one, because zero is the value
the original trouble was about and "let the driver decide" is not
something this interface can express. It is a frontend floor rather
than a Windows one, so ALSA, CoreAudio and the rest are equally free
of it.
A settings entry in this tree has its strings in one of two places:
generated from the settings_def row, or written by hand in
msg_hash_us.h. settings_def_audio_latency.h is not in a generated
region, so the first version of this showed in the menu as "null"
with a working value beside it - the setting was real and the label
was not. The display name and sublabel are written out beside
AUDIO_LATENCY's now.
Every link was walked rather than assumed - field, binding, default,
menu row, label enum, config key string, display name, sublabel, menu
listing, and the clamp that reads it - because a settings change that
compiles while doing nothing is a thing I have already shipped once
this week, and this one then did it again in a different place.
config_set_string() and config_unset() went through
config_get_entry_internal(), which on a miss walked the entire entry
list to hand back its last node. The insert path needed that node to
append to. It does not any more - conf->tail is authoritative since
the trackers were collapsed - but the walk stayed, and it is
quadratic: 203701 fresh inserts spend 126 seconds in it against 0.17
seconds past it.
That walk is the whole reason
CONF_FILE_FLG_GUARANTEED_NO_DUPLICATES exists. Setting it skips the
lookup, and skipping the lookup skipped the walk. With the walk gone
the flag is worth one hash probe per set - 11ms across those same
203701 inserts - and config_get_entry_internal() is just
config_get_entry(), so it goes.
The flag mattering less is the point, because it was being set where
it is not true. cheat_manager_save(!overwrite) parses the existing
.cht file and then promises the setters that none of the keys it is
about to write are present - when every one of them is. Each set
appended a second entry instead of replacing, the file was written
with both, and since the first of a duplicate pair wins on reload,
the save looked correct in memory while the file on disk kept its old
values and grew by a full duplicate set each time. For a cheat file
from libretro-database that is 203701 lines added per save. The flag
now goes on only in the branch that allocates the config empty, which
is the only branch that can keep the promise; the menu's 'save as a
named cheat file' path (menu_cbs_ok.c) is the one that reaches the
other branch.
The flag's declaration says all of this now, including what it is
worth, since the reason to reach for it has mostly evaporated.
Deprecating it outright is defensible - one hash probe per set
against a footgun that silently corrupts saved files - but that is a
separate decision and both in-tree users are correct as of this
commit.
Tests: setting over keys a parsed config already holds must survive a
dump and reparse with no duplicates, and 40000 inserts without the
flag must not take seconds. 47 checks, clean under ASan+UBSan+LSan.
The eight-millisecond minimum was a constant in audio_driver.c,
applied once before any driver saw the value - which is why it showed
up identically on WASAPI, ASIO and WDM-KS, none of which was involved.
The reasoning behind it, in its own comment, is about drivers
mishandling zero rather than about hardware: the setting's range
starts at zero, zero reached the drivers, and they each did something
different with it. That argument is much weaker for the drivers that
negotiate a period with the device directly, and those already floor
themselves at whatever their hardware states.
So it is audio_latency_floor now: advanced, one to sixteen
milliseconds, eight by default, which is what it has always been. The
constant stays as an absolute floor of one, because zero is the value
the original trouble was about and "let the driver decide" is not
something the interface can express.
It is a frontend floor rather than a Windows one, so ALSA, CoreAudio
and the rest are equally free of it.
Every link of the chain was walked rather than assumed - field,
binding, default, menu row, label enum, label string, menu listing,
and the clamp that reads it - because a settings change that compiles
while doing nothing is a thing I have already shipped once this week.
set_filtering and the five HDR setters have nothing to say back either,
so they go on the ring with show_mouse and grab_mouse_toggle rather than
waiting out a round trip on every settings change.
The overlay's four setters stay where they are:
video_thread_get_overlay_interface() writes thr->overlay from the main
thread and the CMD_OVERLAY_* handlers read it here, and what orders
those two today is the waiting. set_aspect_ratio stays as well - it
raises the driver's resize flag, and a swapchain rebuilt while the main
thread is still handing over frames is what waiting has always
prevented. Both want their own answer before they follow.
conf->tail and conf->last tracked the same node. The parser wrote
only the first, config_set_string() only the second, and the drift
between them dropped entries until the last patch made every path
write both. Writing both is not the fix, it is the same invariant
maintained twice: the next merge or splice added to this file gets to
remember two fields, and forgetting the second one is silent.
So conf->last is gone and conf->tail is authoritative. Nothing
outside config_file.c ever read either. config_unset() tombstones
entries rather than unlinking them, so the list only ever grows at
the end and there is no state in which the two could legitimately
have differed - they were redundant, not complementary. The
no-duplicates fast path in config_set_string() loses a branch and a
fallback along with the field.
While in the struct: includes get a tail pointer. Adding one walked
the whole include list to find its end, so building the list was
quadratic in the include count. That count is always tiny, so this
is not a measurable win and is not claimed as one; it is three lines
and it removes the walk. The list is appended to rather than
prepended because the order is user-visible - config_file_dump()
writes the '#include' lines back in list order - and a test now
covers that through the serialised form, since struct
config_include_list is private to this file.
config_set_string()'s insert path measured each string three times:
strdup() walked it, strlen() walked it again for the length cache,
and config_hash_span(entry->key, strlen(entry->key)) walked the key a
third time. Both strings are now measured once and memcpy'd, and the
hash spans the length already in hand. Note the hash takes key_len
and not entry->key_len - the cache saturates to 0 above 64 KiB, which
would silently hash an empty span. 203701 fresh inserts, x86-64,
-O2: 230 ns/insert before, 208 after.
Tests: conf->tail is checked after each of the five paths that can
extend the list - parse with an #include, append_conf, both set
paths, and the sort fallback - run in sequence through one config,
plus include order through a dump. 45 checks, clean under
ASan+UBSan+LSan.
Not done, from the same review. The streaming parser's per-push
memmove() is fine: compaction only runs when the push contained a
newline, and the retained tail is newline-free by construction, so
everything after the window's last newline arrived in that same push
and the copy can never exceed what the push delivered. Instrumented,
the ratio of bytes moved to bytes in peaks at 0.49, and the case the
review was worried about - a long line arriving in small packets -
does not memmove at all. The RHMAP_FIT call is already behind that
same newline check. The owned-buffer and entry-pool chain splices
walk the child's chain, which is one node for a flat include, and
making them O(1) needs two more tail pointers - the opposite of what
this commit is for. A config_get_entry_span() to save the strlen()
before the hash is public API surface for microseconds across all of
RetroArch's startup.
On a WaveRT pin with no position register - which is what the HDMI
output here turns out to be - every read of the play offset is a
DeviceIoControl. Four things read it: the write loop on every lap,
write_avail, wait_writable on every lap, and frames_consumed, which
then called the clock sampler, which read it a second time.
Two of those are removed and neither costs anything.
The clock sampler takes the count the caller has just read instead of
reading it again. It wanted the same number frames_consumed had
obtained one line earlier, and on this pin that was a second kernel
transition for a value already in hand.
And the write loop asks for room once and spends it, rather than
asking again on every lap. The device only ever frees more room as it
plays, so a figure read a moment ago is a lower bound and writing
against it is safe; it is re-read when it runs out, which is exactly
when the answer would have changed. A frontend write of a video
frame's worth takes several laps of that loop at an 8 ms buffer.
A pin that does have a register was never paying for this - there the
read is a word - so this is the no-register path catching up with it
rather than a saving on both.
Nothing else measurable stands out: the packet path's per-write work
is a memcpy and one ioctl per packet, which is what the interface
costs; enumeration is once per init; and the fit is a handful of flops
against numbers already loaded. Compiles clean in C89 and gnu99 with
-Wall -Wextra, through the mingw gate, ABI still verified.
A layout a pin will not take as PCM is offered as a Dolby Digital bit
stream before it is narrowed, which is what the WASAPI driver does and
for the same reason: an HDMI output to a receiver commonly advertises
stereo PCM and decodes AC-3, so the layout survives as a bit stream
where it would otherwise be folded away. A/52 carries 1.0 to 5.1, so a
7.1 request goes as 5.1.
The carrier is two channels of 16-bit PCM at the encoded rate and the
subtype names the codec. That subtype is declared here like the rest
of this file's interface - no header this build can reach has it, not
the mingw one and not the SDK one - and tools/wdmks_abi_guids.py
checks it by the rule that covers PCM and float: the WAVE format tag,
0x0092, in the family they all share. The extended format structure
that carries the encoded rate and channel count is the same shape
audio/common/mmdevice_common_inline.h has for WASAPI.
What the frontend hands over on this path is float frames in the
encoded layout, not the carrier's samples, so use_float() answers for
the encoder rather than for the pin. Frames are gathered until a whole
A/52 block of 1536 is there, encoded, wrapped in a burst, and written
through the same packet path everything else uses - which is split out
into its own function for that, since what reaches the device is the
carrier's samples either way.
Only on the packet path: a WaveRT pin is not offered it. The looped
path writes into a mapped buffer rather than submitting packets, and a
burst is a packet-shaped thing; whether that combination is even
offered by drivers is a question for hardware rather than one to
assume an answer to.
Not run - there is no receiver here, and no device. It compiles clean
in C89 and gnu99 with -Wall -Wextra, through the mingw gate and the
vtable check, and the ABI it speaks is still verified. What a device
without a receiver behind it does is refuse the format, which is the
ordinary path and leaves PCM as it was.
The log settles what three revisions of this function were guessing
at. On the HDMI output:
WaveRT position: register no, position property yes.
So there is no position register at all, and the property does answer
- which is why preferring the register changed nothing, and why
preferring the property changed nothing either.
What was wrong is the reading. On a WaveRT pin that property's
PlayOffset is a position WITHIN the loop and wraps with it; it is not
the count since the stream started that the same property gives on a
packet pin. Read as though it were, it goes round every 8 ms, and a
count that goes backwards is no count at all - so the sink estimate
had nothing, which is why that line was first wrong, then flickering,
then absent, and why there has never been a Device clock line on this
device.
The accumulation that the register path already did now lives in the
play-offset reader, so it happens whichever source supplies the
offset, and everything on a looped pin - the frame count and the clock
fit - goes through that reader and takes the accumulated count rather
than the raw offset. A pin with a register and a pin without it now
differ only in where the offset came from.
Also from that log, for the record: the FIFO is 193 bytes, which is
not a whole number of frames. It is reported rounded down and that is
fine for a latency line, but it does say the device counts in bytes
that need not align with ours.
Interface still verified against the SDK header; clean in C89 and
gnu99 and through the mingw gate.
The statistics line has been showing 8 ms, which is this driver's own
loop and nothing else - that is why it never looked like WASAPI's
figure, which includes the device's share.
Kernel streaming keeps that share in KSPROPERTY_RTAUDIO_HWLATENCY:
the FIFO the device reads through, plus what the chipset and the codec
hold. FifoSize is in bytes, so it divides by the frame and goes to
audio_driver_set_device_latency() the way the PipeWire and WASAPI
drivers report theirs. The chipset and codec figures are in other
units and are logged rather than added, because a wrong conversion
folded into a latency figure is worse than an absent one - if they
turn out to matter on real hardware, the log line is what says so.
It is a WaveRT property and is asked for only there; the packet path
has no equivalent and would just get a refusal. A pin that does not
answer says so once in the log and the figure stays as it was, which
is the driver's own buffer.
The structure has been declared and asserted against the SDK header
since WaveRT landed; this is the first thing to read it.
The sink estimate still read a million ppm out. The fallback added
last round only fires when KSPROPERTY_AUDIO_POSITION FAILS - and a
WaveRT driver may answer it, refuse it, or answer it with nothing in
it. The third case is the one that bites: a property that succeeds
with a standing zero is believed for ever, and the register sitting
next to it, which was right the whole time, is never read.
So the order is turned round. On a looped pin the register is the
source, for the frame count and for the clock fit alike, and the
position property is what the packet path uses. That is the right way
round on its own terms - the register is what the WaveRT model
exposes - and it is not a fallback any more.
Ordering a fallback by whether a call fails, when the failure mode in
question is a call that succeeds and says nothing, is the mistake
here, and it is the third revision of this one function.
Last round moved frames_consumed onto KSPROPERTY_AUDIO_POSITION for
both streaming families, on the grounds that its PlayOffset does not
wrap where the position register does. On a WaveRT pin that property
is not obliged to answer at all - the driver exposes its position
through the register it mapped - so on an HDMI output the call fails,
frames_consumed returns zero every time, and both reports follow from
that one thing: the sink estimator sees no progress and prints a value
that comes and goes, and the clock fit bails before publishing, so
there is no Device clock line.
I traded a counter that was wrong for one that was absent.
Both sources are used now, in the order that suits each pin: the
position property where it answers, the register otherwise, for the
count and for the clock fit alike. The register still wraps, so what
makes it usable as a count is that every read of the play offset adds
its step - and every write, every write_avail and every lap of the
wait comes through that read, which is many times a frame rather than
once. An init-time line says which of the two the pin answered, so the
next log settles what this paragraph is reasoning about.
Still not solved: audio in the background. The write path waits on the
notification event where a driver offers one and sleeps otherwise, but
either way it only runs when the frontend calls it, and a backgrounded
process is not called on time - so the loop is refilled late and the
hardware replays what is still in it. The poll that keeps the count
honest has the same shortcoming. Both want the same thing, which is a
thread belonging to this driver that fills the loop and reads the
register on its own schedule rather than the frontend's, the way the
WASAPI driver's pump does. That is the next piece of work and it is
not a small one.
Interface still verified against the SDK header. Clean in C89 and
gnu99 with -Wall -Wextra and through the tree's mingw gate. Not run.
The queue the setters put their packets on took thr->lock to say it was
empty, on every pass of the video thread, and the thread copied it to a
frame of its stack to run it. One thread fills it and one empties it, so
it needs neither: head and tail are atomics the two threads read of each
other, the packets run where they lie, and a pass with nothing queued is
one load. The lock is taken only when the ring was empty, where a
sleeping thread has to be woken - so an overlay's burst of geometry
calls takes it once rather than once a call.
The overlay's own setters - enable, tex_geom, vertex_geom and
full_screen - go on the ring too. input_overlay_set_vertex_geom() calls
one per descriptor, dozens for a pad, and each of those was a round trip
to the video thread and back. set_aspect_ratio returns to the ring as
well, now that what it changes crosses threads as an atomic.
Two reports off the first working WaveRT run, one cause each.
The sink estimate read -999043 ppm and the device clock line was
absent. frames_consumed was accumulating the steps of the position
register between reads - and that register is a byte offset into the
loop, which wraps at the end of it. At an 8 ms buffer it wraps every
8 ms; the frontend asks once a frame, 16 ms apart, so every read
missed a wrap, and backgrounded and throttled it missed ten. Counting
that way only works while the reads are closer together than the wrap,
which is a condition nothing here guarantees.
KSPROPERTY_AUDIO_POSITION's PlayOffset does not wrap - it is what the
device has played since the stream started - so it answers this on
both paths now, and the clock sampler runs on both, which is why the
overlay line was missing: it was in the branch the looped path skipped.
The register keeps the job it is good at, saying where in the loop it
is safe to write, which is read far more often than it wraps.
And the audio distorted when the window went to the background. The
write path had no event to wait on, so it polled and slept a
millisecond at a time; backgrounded, the scheduler stops handing that
thread its millisecond, the loop is not refilled in time, and the
hardware plays what is still in it - so the distortion is the loop
repeating, not the samples being wrong.
The buffer request already asked for the notification form, so drivers
that offer an event accept it; the event is registered now and the
write path waits on it, falling back to the sleep where a driver
declines and saying so in the log. Unregistered before the pin closes.
Interface still verified against the SDK header - every size, offset
and ordinal. Compiles clean in C89 and gnu99 with -Wall -Wextra and
through the tree's mingw gate.
Not run: the event path in particular has not been seen working, and
whether backgrounded audio actually holds up is the thing to watch.
The second streaming family, and the one four of six filters on the
test machine need. A WaveRT pin does not take packets: it hands over a
buffer the driver owns, the frontend writes into it in a loop, and a
register the hardware updates says where playback has got to.
Those pins are opened on KSINTERFACE_STANDARD_LOOPED_STREAMING rather
than the standard one - asking for the standard interface is what
returned ERROR_NOT_FOUND for every format on an HDMI or Realtek
output, which is what sent this down the wrong road for several
rounds. The buffer comes from KSPROPERTY_RTAUDIO_BUFFER_WITH_NOTIFICATION,
falling back to KSPROPERTY_RTAUDIO_BUFFER, which every WaveRT driver
has; the cursor from KSPROPERTY_RTAUDIO_POSITIONREGISTER, falling back
to asking the pin, which costs an ioctl but is correct.
Writes fill forward from where this side left off to where the
hardware is reading, keeping one frame back so a full buffer is never
read as an empty one, wrapping in two copies, and cut to whole frames
- a wrap mid-frame rotates every channel after it for good, which is
the same hazard the packet path guards. Where the driver asks for a
memory barrier, the write is made visible before the cursor moves past
it. frames_consumed accumulates the register's steps, so a position
that went backwards is a wrap rather than a rewind.
Teardown closes the pin and drops the pointers without freeing
anything: the buffer is the driver's, and closing the pin is what
releases it.
The interface this is written against was transcribed from a Windows
SDK ksmedia.h and is checked by tools/wdmks_wavert_check.sh - every
size, every field offset read, every property ordinal - which passes
against the 10.0.26100.0 header. That check is why this could be
written at all: the mingw headers declare none of these types and
PortAudio takes them from the SDK, so without it the whole thing would
have been from memory.
NOT RUN. Compiles clean in C89 and gnu99 with -Wall -Wextra, through
the tree's mingw gate and the vtable check, and the ABI is verified -
but no WaveRT device has seen it. The event path is not here either:
the buffer is requested with a notification count so a driver that
offers events accepts the request, but nothing waits on one yet - the
write path polls the cursor. That is the next thing to add, and it
matters for latency rather than for whether sound comes out.
Kernel streaming: the frontend writes to a filter's pin and the
packets reach the device driver without a mixer between. Written
against PortAudio's pa_win_wdmks.c. setupapi.dll and ksuser.dll are
resolved at runtime, so the binary loads anywhere and works on Windows
2000 and up. The interface is declared in this file rather than taken
from ks.h, which ships with the driver kit and is not in every SDK
this tree builds against, and samples/audio/wdmks_abi asserts every
size, field offset, constant, IOCTL code and GUID of it against the
real headers on two word sizes in C89 and gnu99.
Streaming is WaveCyclic and works: a USB DAC at 48 kHz, 16-bit, 8 ms
in four packets, device clock +1 to +5 ppm against a sink estimate of
-5.
THIS REVISION ADDS THE WaveRT INTERFACE, not yet the streaming path.
A filter carrying KSCATEGORY_REALTIME hands over a buffer it owns and
a position register instead of taking packets, and its pins refuse the
standard streaming interface - the ERROR_NOT_FOUND that every format
on an HDMI or Realtek output came back with. Four of six filters on
the test machine are that kind, so it is the difference between one
working device and most of them.
The types for it exist in no header this tree can reach: the mingw
ksmedia.h stops at KSPROPERTY_RTAUDIO_GETPOSITIONFUNCTION and declares
none of the KSRTAUDIO_* structures, and PortAudio does not carry them
either - it takes them from the SDK. Writing them from memory is how
the KSDATAFORMAT alignment and the KSDATAFORMAT_WAVEFORMATEX packing
bugs happened, and the harness that caught those would have had
nothing to check against.
So they are transcribed from a Windows SDK ksmedia.h, and
tools/wdmks_wavert_check.sh cross-checks them against one when it is
to hand - every size, every field offset that gets read, every
property ordinal. Run against the 10.0.26100.0 header: no mismatch.
The script takes the header as an argument rather than the tree
shipping one, which is why this cannot join the CI harness.
The buffer, latency, position-register and notification-event
properties are declared and asserted. The streaming path that uses
them - request the buffer, map it, write into it in a loop against the
position register, register the notification event where the driver
offers one and poll where it does not - is the next piece, and it is
now something that can be written against a verified interface rather
than a remembered one.
Kernel streaming: the frontend writes to a filter's pin and the
packets reach the device driver without a mixer between. Written
against PortAudio's pa_win_wdmks.c. setupapi.dll and ksuser.dll are
resolved at runtime, so the binary loads anywhere and works on Windows
2000 and up; the interface is declared in this file rather than taken
from ks.h, which ships with the driver kit and is not in every SDK
this tree builds against.
WaveCyclic only. A filter carrying KSCATEGORY_REALTIME streams WaveRT
- a mapped buffer and a position register, not packets - and its pins
answer KsCreatePin with ERROR_NOT_FOUND because they do not offer the
streaming interface this driver asks for. Those are recognised and
passed over, and choosing one by name fails saying so rather than
walking on to a neighbour and blaming a format on a device the user
did not pick.
TWO MSVC BUILD FAILURES, both mine.
audio_wdmks's extern had landed inside #ifdef HAVE_ASIO. The 2022
project defines HAVE_WDMKS without HAVE_ASIO, so the declaration
vanished and the driver list referred to an identifier that did not
exist. The two drivers have nothing to do with each other and now
share no flag: the extern sits with the unguarded ones, the list entry
is under HAVE_WDMKS alone, and all four combinations of the two flags
compile. The rest of the audio drivers were swept for the same shape -
an extern reachable only under some other driver's flag - and there
are none.
METHOD_NEITHER and the access bits come from winioctl.h, which some
toolchains' windows.h pulls in and some do not; the griffin build
under MSVC gets neither. They are carried here behind #ifndef, which
is only correct while the values match the system's - so
samples/audio/wdmks_abi includes winioctl.h alongside and asserts
that they do, building both IOCTL codes through the system's own
CTL_CODE to check the arithmetic this file spells out by hand.
That harness already asserts every size, field offset, constant and
GUID of the kernel-streaming interface against the real ks.h, on two
word sizes in C89 and gnu99. It caught three ABI bugs before this ran
on hardware; the ones hardware found - a use-after-free from clearing
a packet's pending flag on an I/O error, and audio at the wrong pitch
from negotiating widths the frontend cannot produce - it could not
have.
Works on a USB DAC: 48 kHz, 16-bit, 8 ms in four packets, device clock
+1 to +5 ppm against a sink estimate of -5.
WaveRT needs KSRTAUDIO_* from a Windows SDK or WDK ksmedia.h. Neither
the mingw headers nor PortAudio carry those types - PortAudio takes
them from the SDK - so they cannot be declared here and checked the
way everything else in this file is. That is the blocker on it, and
IEC 61937 passthrough is the piece that does not depend on it.
The parser side of this file is in good shape; what was left was
correctness around the mutating paths, where an allocation failure or
a stale pointer could destroy entries that had parsed fine.
conf->tail and conf->last are two trackers for the same node and they
had drifted apart: the parse path maintained tail only, and
config_set_string() maintained last only. A config that was parsed
from a file and then flagged CONF_FILE_FLG_GUARANTEED_NO_DUPLICATES
therefore had last == NULL over a populated list, fell back to
conf->entries, and linked the new entry onto the *head* - dropping
every parsed entry behind it. cheat_manager.c does exactly that
sequence on the !overwrite path, so saving cheats over an existing
cheat file silently discarded all but the first entry. Both fields
are now written, and the fallback is conf->tail. They want collapsing
into one field the next time struct config_file's layout can change.
config_file_append_conf() had the same shape one level up. It
prepends the donor list, so an already-populated destination keeps
its own tail - but an empty one has none, and the function left
conf->entries non-NULL with tail and last still NULL. The next
config_file_load_file() through that conf then dereferenced
conf->tail, and the next set spliced onto the head as above.
config_set_string() released the existing value before knowing the
replacement had been allocated. On failure the old value was gone,
entry->value was NULL, and the config was still marked modified - so
the entry was lost and the next dump walked into strlen(NULL). The
replacement is now built first, the way the insert path in the same
function already treats its two strdups. The equal-value early out
stays ahead of the allocation: writing a settings block back mostly
sets values to what they already hold, and that must not start
calling malloc().
config_file_dump() returned void, so a failure to allocate its 4 KiB
staging buffer wrote nothing, left ferror() clean, and let
config_file_write() rename its empty temporary over a good config and
then clear the modified flag, so nothing retried. It returns bool
now and the write path gates on it. (The buffer stays on the heap:
4 KiB is twice the frame budget tools/stack_budget.py enforces.)
config_take_string() leaves its entry keyed but valueless with the
length cache cleared, which is precisely the input the dump loop's
"value_len ? value_len : strlen(value)" fallback could not take.
config_get_config_path() passed conf->path to strlcpy() unchecked,
and every config built from a string has none.
Two things that are not bug fixes but belong with them:
Sorting for a dump no longer merge-sorts the live linked list and
assigns the result back. That was a side effect - saving a config
reordered it in memory - and an expensive one, because the resulting
order had nothing to do with the entry pool's layout, so every later
traversal chased pointers across it. The serialisable entries are
collected into a pointer array and merged there: insertion-built runs
of 16, then a bottom-up merge ping-ponging between the two halves of
one allocation, stable so that duplicate keys - whose order is
meaning, since the topmost wins on reload - keep theirs. Measured
against the old sort, x86-64, -O2, per save:
entries workload old new
300 generated, random keys 0.034ms 0.017ms
3000 generated, random keys 0.55ms 0.50ms
20000 generated, random keys 5.1ms 5.6ms
20001 cheat_manager_save shape 2.9ms 2.2ms
30556 Final Fantasy III (USA).cht 3.7ms 2.6ms
203701 that file saved back through
cheat_manager_save 36.4ms 26.3ms
The one row where the old sort wins is a synthetic config of uniformly
random keys, which is not a shape anything here produces; on real key
sets, which share long prefixes and are written in systematic order,
the array merge is ahead at every size. The last two rows are not a
stress test: cheat_manager_save() writes 20 keys per cheat with
sort=true, nothing caps the cheat count on load, and that cheat file
ships in libretro-database with 10185 cheats in it. It is also the
same path as the entry-dropping bug above.
qsort() was tried first and was a quarter slower than this at 3000 -
an indirect call per comparison and byte-wise element swaps are a
poor trade for 8-byte elements.
The index costs n * 2 pointers, which for that 204k-entry save is
3.3 MiB - against a config that is already 44 MiB of pooled entries,
adopted text and hash map, and measured peak RSS moves by 0.09 MiB
because the index fits in heap the parse just freed. Where it could
still bite is a target whose heap has no room at all, so a failed
index allocation falls back to sorting the list in place rather than
failing the save: bounded memory, the old behaviour's cost (the live
list comes back reordered), and no new way for a save to fail.
That fallback also fixes what the old in-place sort left behind. It
assigned the sorted head to conf->entries and never rebased
conf->tail, so after any sorted write the tail pointed into the
middle of the list and a later config_file_load_file() through the
same conf spliced onto a mid-list node, dropping everything past it.
Both trackers are rebased now. The fallback is not dead code under
test: CONFIG_FILE_TEST_HOOKS (test builds only, compiled out
everywhere else) lets the suite force it and check it produces
byte-identical output to the array merge.
config_file_add_child_list() likewise stops re-walking the merged
list to find a tail the child already tracked.
And the atomic-save comment claimed protection against power loss,
which rename-without-fsync does not provide. It now says what the
code actually guarantees and why the fsync()s are not simply added.
Tests: the append matrix (both destination shapes against both donor
shapes, then a load and a set through the result), the pathless
config path, a dump after config_take_string(), overwrite
consistency, that a sorted dump is stable and leaves the live list
alone, and that the sort fallback agrees with it byte for byte. The setter-versus-#include question the header raises -
config_file.h says a setter will not write to an entry obtained from
an #include, and config_set_string() neither honours entry->readonly
nor leaves it set - is deliberately not changed here; it is pinned by
a characterisation test so that changing it is a visible decision.
The differential fuzzer checked conf->tail only after parsing, which
is why it never saw the set path drift. It now applies an identical
config_set_string() to both configs and re-checks the invariants;
with the tail fix reverted it fails on the empty input.
Kernel streaming: the frontend writes to a filter's pin and the packets
reach the device driver without a mixer between. Written against
PortAudio's pa_win_wdmks.c. setupapi.dll and ksuser.dll are resolved
at runtime, so the binary loads anywhere and works on Windows 2000 and
up; the interface is declared in the file rather than taken from ks.h,
which ships with the driver kit and is not in the 2005 SDK.
WaveCyclic only. A filter carrying KSCATEGORY_REALTIME streams WaveRT
- a mapped buffer and a position register, not packets - and its pins
answer KsCreatePin with ERROR_NOT_FOUND because they do not offer the
streaming interface this driver asks for. Those are recognised and
passed over.
Choosing such a device by name now fails saying so, rather than
walking on to the next one. That last part was a real report: the
audio device setting was changed to an HDMI output mid-session, this
skipped it, the neighbouring device refused a format, and the error
named that device and blamed the format - so the log accused the wrong
device of the wrong thing. A device the user asked for by name is not
something to walk past.
Earlier rounds of testing found a use-after-free that crashed the
process from a core thread (a packet's pending flag cleared on an I/O
error, which is not a completion: the kernel still owned the buffer
and the OVERLAPPED), and audio at the wrong pitch (the negotiation
offered widths the frontend cannot produce and nothing converts). Both
fixed; neither would have been found without running it.
samples/audio/wdmks_abi compiles this driver beside the real ks.h and
asserts every size, field offset, constant and both IOCTL codes on two
word sizes in C89 and gnu99; tools/wdmks_abi_guids.py checks the
GUIDs. That caught three ABI bugs before any of this ran.
Works here on a USB DAC: 48 kHz, 16-bit, 8 ms in four packets, device
clock +1 to +5 ppm against a sink estimate of -5. WaveRT is what the
rest of a typical machine needs and is the next piece.
video_driver_set_aspect_ratio() writes the aspect ratio on the main
thread and video_driver_update_viewport() reads it on whichever thread
draws - under the threaded wrapper, the video thread, in the middle of
a frame. Nothing ordered the two: what kept it quiet was the round trip
the setters took right afterwards, which is not a guarantee and goes
away the moment a setter stops waiting.
It is stored as the bits of the float in a retro_atomic_int_t now,
read through VIDEO_DRIVER_ASPECT_RATIO() and written through
video_driver_aspect_ratio_put(), which carry the bits in a union rather
than through a cast so the aliasing rules hold. The loads and stores
are the same instructions on every target that mattered before.
samples/audio/embedded_alsa/abi_check is an x86-64 ELF executable that
was committed alongside its source, and it is also that Makefile's own
build target - so a make clean in that directory deleted a tracked
file and every build of that sample showed up as a deletion. I hit
that myself running the harnesses.
It is removed from the index and from the tree, and ignored. Nothing
is lost: the Makefile builds it from abi_check.c, and the workflow
that uses it already runs make clean check, so it was rebuilding it
every time and never reading the committed copy. Checked rather than
assumed - that step and the committed binary's date do not even match.
The .gitignore rule is named rather than a pattern, because the rules
above it cover <dir>_test and objects and this binary is neither.
The rest of samples was swept for the same thing: it is the only
tracked file that is an executable, and the only tracked file that is
a Makefile target. Running the sample the way the workflow does, from
a clean tree, passes.
show_mouse, grab_mouse_toggle, set_filtering, set_aspect_ratio and the
five HDR setters sent their packet and waited for the video thread to
answer, though none of them has an answer to give. The packet goes on a
queue instead, and the thread runs it on its next pass, before it draws
the frame it has claimed - so the setting still reaches the driver for
that frame, and the caller carries on rather than waiting out a round
trip on every menu toggle and settings change.
The queued packets go through the same handler the waiting ones do,
with inline_reply taking the reply nobody is waiting for. A queue that
is full, and a call already on the video thread, send the waiting way,
so nothing is ever dropped; the queue is tested with worker_running,
which says the thread still takes commands, and not with alive, which
is the window's answer and goes false while it still runs.
Same cause as the include fix, second half of it. audio_init_thread()
calls cpu_features_get_time_usec(); the header gave the declaration,
and this harness links a source list of its own that did not include
features_cpu.c, so it compiled and failed at the link.
Every other harness under samples/audio and samples/gfx was checked
for the same gap - a linked source that calls it without the
definition on the list - and there are none. All of them build.
Clean under ASan, UBSan and TSan.
cpu_features_get_time_usec() is called here and features/features_cpu.h
was never included, so the call was an implicit declaration. GCC warns;
clang, which the Apple builds use, makes it an error under C99 and the
build stops there.
Nothing else in audio or gfx is exposed the same way: video_driver.c
calls it too and reaches the declaration through retro_timers.h, which
is checked rather than assumed.
audio_init_thread() waited for the handshake with no way out: a driver
whose init() does not return, or a thread that died inside it, left
drivers_init() parked there for good, warning every couple of seconds
and going no further. It gives up after thirty seconds now - far longer
than a device that is merely slow to open - and hands the thread its own
state as it goes, because freeing it here would pull it out from under a
thread still inside init(). The thread sees that when init() returns,
frees the driver and itself, and exits. Nothing is freed twice and the
frontend goes on without audio, which is what drivers_init() already
does with a failed audio driver.
ThreadSanitizer reported the harness writing thr->poke under the
wrapper's lock while video_thread_async_run read it without one.
The read is the side that is wrong. The list of pending uploads comes
out under thr->lock two lines earlier, and the driver to upload
through was then read per node, outside it - so a writer holding the
lock, which is the correct thing for a writer to do, still raced.
Taking the pair in the same critical section the list comes out of
closes it for nothing: it is two reads inside a lock that was already
being taken.
The upload itself stays outside the lock, which is the point of
snapshotting rather than holding: load_texture is the driver talking
to the GPU and can take as long as it likes, and the frame path takes
this lock.
Whether the pair can change under a running video thread in normal
use is a fair question - a reinit builds a new wrapper rather than
swapping into the old one - but the harness does exactly that when it
puts a counting poke in around a batch of uploads, and code that is
correct only because nothing currently exercises it is worth two
reads.
Verified by building the tree under both sanitizers and running the
lane that reported this: sixteen lanes pass under TSan with no
warnings, and sixteen under ASan and UBSan with none. The tree needed
--disable-x11 here for a missing library, which is local and not part
of the change.
win32_meta_key_to_name() and the menu's bind buffer read and write
the key through the accessors, which is every remaining site: a
type-aware sweep over the names declared as retro_keybind finds none
left.
The keybind accessors reach the remaining call sites: the input
drivers for Android, Cocoa, DirectInput, DOS, PS3, QNX, RWebInput,
SDL, SDL3, udev, UWP, Wayland, WinRaw and X11, and the Qt user-binds
page. A build that enables any of those reads enum_idx, key and valid
through the accessors like the rest of the tree.
msg_hash_enums needs 16 bits, retro_key tops out at RETROK_LAST so 15
is generous, and valid is a flag, so the three share one word and
struct retro_keybind packs to 20 bytes with no padding. Two full bind
sets exist per user, which is thousands of instances, and whole sets
are copied on every remap and autoconfig apply.
The default tables hand the three to RETRO_KEYBIND_PACK() rather than
listing them positionally; everywhere else reads and writes them
through the accessors.
directory_ptr is an index into the list the item belongs to, so 32
bits covers any list that fits in memory. Moved below entry_idx it
pairs with 'type' in space the padding was already taking, and
struct item_file goes from 64 bytes to 56. One of these exists per
menu row, and a playlist view has one row per entry.
Nothing takes the field's address or prints it, so every existing
call site converts as it always did.
Unloading a texture drained the frame ring through
video_thread_wait_idle() and then sent the driver's delete to the video
thread and waited for that too - twice blocking the main thread, on
every menu texture the frontend lets go of. The id goes on a list
instead: the next frame handed over carries it, and the video thread
frees it once it has drawn that frame. Frames are drawn in the order
they are handed over, so every frame that could still name the texture
- the in-flight one the AI service overlay draws from, and any behind
it - is drawn by then, which is what the drain was for. Neither thread
waits.
The driver's own call still runs here for a texture the main thread
never shared, from the video thread itself, and if the node cannot be
allocated. What is left when the wrapper is freed is freed with it; the
textures went with the driver.
ThreadSanitizer reported the pump reading q_cb unlocked while
obj_destroy cleared it under the lock. The unlocked read is the
symptom; what makes it matter is behind it - the pump could read the
callback, the player be destroyed, and the pump then call into a
context the driver had already freed. The comment beside obj_destroy
says the mock must not call back after the player is gone, and the
mock was not holding to it.
The callback cannot be held under q_lock, because the driver's
callback enqueues and that takes the same lock. So the pump takes the
lock, snapshots the callback and marks itself inside one, calls it
outside the lock, and clears the mark on the way out; a destroy clears
the callback and then waits for the pump to leave before returning.
That is what the real implementation guarantees by not calling back
once the player is destroyed.
This is the harness, not the driver. audio/drivers/opensl.c is
unchanged.
TSan is quiet across three runs where it reported on the first before
- the race needed the destroy to land inside the pump's sleep, so one
clean run would not have been worth much. ASan and UBSan clean,
leak-checking on, and the plain build passes.
The three ride in one word: the slot index is a subsystem content
slot, the runtime status has three values, and the thumbnail flags
are five bits. struct playlist_entry goes from 120 bytes to 112, so
a large playlist sheds another eighty kilobytes on top of what the
timestamps returned.
The JSON reader gains one more name for the slot, since a packed
field has no address to hand it.
The nine values ride in three words instead of nine: hours get 20
bits, which is 119 years of play, and the rest are what a calendar
and a clock allow. struct playlist_entry goes from 152 bytes to 120,
and a playlist holds one per item, so a large one sheds hundreds of
kilobytes.
The JSON reader named these fields by taking their address, which a
packed field does not have, so it now names which one it is filling
and goes through the setters.
The three share one word. An entry taller than 65535 pixels is not an
entry and a sublabel past 255 lines is not a sublabel, so 16 bits and
8 bits cover them with the wrap flag in a ninth, and ozone_node_t goes
from 48 bytes to 40. A list holds one node per entry, so that is eight
bytes times the length of a playlist.
Running the test cannot tell whether the seqlock's fences are there:
x86_64 supplies the ordering they ask for, and qemu-aarch64 does not
model reordering either, so a build with the release fence deleted
passes under both. The emitted code does change, so check-barriers.sh
cross compiles for aarch64 and asserts each half of the pattern --
the fences as dmb, the stamps as stlr and ldar, the relaxed fields as
plain str and ldr. Deleting either fence fails it. The script skips
when no aarch64 toolchain is installed, and make check runs it after
the test.
frame_surface is allocated once, to the geometry the core declared at
init, so a core that later hands over a larger frame writes past it.
oga_get_current_software_framebuffer() refuses a request that does not
fit rather than lending a buffer the core would render off the end of,
and oga_frame() takes the rows the surface has and the bytes one of
its rows holds, so a wider source pitch cannot run a row long. The
blit that follows reads the same bounds.
frame.lent_width and frame.lent_height are written when a slot is
lent to the core and read nowhere: the push that claims the loan
matches on the buffer pointer, and the rows that fit are computed
from the slot's own size.
The producer waits for a reader to take the frame it published before
retiring it, so the drain and re-validate lanes are reached on every
run rather than when the scheduler happens to oblige: a publish is a
handful of stores, and a run with no overlap asserts nothing. Each
turn of the wait yields, since on one core spinning is what stops the
reader being waited for from running at all.
Width and height share a 32-bit slot, 16 bits each, so both travel in
one relaxed store and arrive from one relaxed load; each is clamped on
the way in, past what any driver here allocates. The statistics
overlay reads them through the same accessor as the rest of the file
rather than touching the storage, which under a C11 atomics backend
was a sequentially consistent load per field.
The tuple lock the non-hazard path uses is gated on HAVE_THREADS, and
the hazard slot index is declared only where the hazard path is
compiled in, so a build without threads compiles again. That is the
emscripten configuration and the "no threads" arms of the compile
matrix.
The gl1, gl2, gl3, Vulkan, GX and PS2 drivers read the HDR nits and
expand gamut, the menu's linear filter, integer scaling, smoothing, the
swapchain bit depth and the overscan correction out of the frontend's
settings from inside frame(), which the threaded wrapper runs on the
video thread while the menu writes them. They come from
video_frame_info_t now, filled once a frame by
video_driver_build_info() on the thread that owns them, as the aspect
ratio index and the rest of that descriptor already do.
The three GL drivers also read the HDR settings from the scRGB encode
helpers their frame calls, so each latches the frame's values into its
scrgb state and those helpers read the latched copy. The settings
pointer's NULL tests go with it: the values reach every one of those
paths from the frame.
video_thread_frame() takes its direct path when it finds itself on the
video thread, rather than when a flag says a read_viewport is in
progress. Two callers arrive that way: a driver's read_viewport(),
which renders a cached frame to get the back buffer it reads, and the
Win32 modal size/move loop, which pumps on the thread that owns the
window -- the video thread, since that is where the driver's init()
created it. The runloop thread can be inside video_thread_frame() at
the same time, parked on the ring while the video thread sits in the
modal loop, and a flag one thread sets is a flag the other can read;
thread identity cannot be misread.
win32_sizemove_tick() therefore runs under threaded video too, so
dragging or resizing the window keeps presenting the pause picture
instead of leaving stale content until the modal loop exits.
The threaded video harness gains a lane that presents a cached frame
from alive(), which is where the modal loop's tick lands, while the
runloop thread pushes frames of its own, and asserts the driver's
frame() only ever runs on the video thread.
The (data, width, height, pitch) tuple is stamped odd, written, then
stamped even, and a reader that sees the stamp move across its read
starts over. Readers hold nothing a publish waits for, so
video_driver_frame() pays two release stores and a fence. The fields
are relaxed-access atomics: the stamp and the fences carry the
ordering, and spelling the accesses that way keeps the pattern free
of formal data races so a real one in this file still stands out
under ThreadSanitizer.
A publish whose tuple matches what is already there returns without
stamping, which covers a core reusing its framebuffer and leaves the
producer side a single writer: the replay path re-enters it from
inside a read with the tuple it just took. That is what lets
video_driver_cached_frame() take the tuple whole, so the driver can
no longer be handed one frame's pointer with another's dimensions.
The viewport and aspect-ratio consumers read through the same
accessors. Targets without a lock-free pointer CAS keep the tuple
lock.
cached_frame_hazard mirrors the new publication path, and its
producer now waits for a reader to take the frame before retiring it:
a publish is a handful of stores, so back-to-back publish and retire
left the overlap the drain and re-validate lanes need to chance.
retro_atomic_load_relaxed_* and retro_atomic_store_relaxed_* are
offered on int, size_t, pointer and int64_t, each gated exactly as
that width's other operations are. Two callers want them: a thread
reading a counter only it writes, and a seqlock, whose data fields
take their ordering from the stamp and the fences around it rather
than from the accesses themselves. Spelling those accesses as
relaxed atomics costs nothing -- every backend emits the instruction
a plain access would -- and it keeps a seqlock free of formal data
races, so ThreadSanitizer stays quiet and a real race in the same
code remains visible.
All eight backends implement them: relaxed orderings on C11, C++11
and GCC __atomic_*, and aligned volatile accesses on MSVC, Apple
OSAtomic, GCC __sync_*, the PS2 EE and the volatile fallback. Where a
width is not indivisible in one instruction -- int64_t on any 32-bit
target, and on the EE, whose ABI moves a 64-bit integer as a register
pair -- the relaxed form rides the same primitive as the ordered one
and is relaxed in ordering only, so a caller never has to branch on
width.
retro_atomic_test gains a round-trip check across every width and a
threaded seqlock lane: a writer publishing three fields that sum to
zero, four readers snapshotting them and checking the sum. The file
header records which mutations that lane catches and which it does
not, since on x86_64 a removed fence is hidden by TSO.
The blank-thumbnail regression was a bug class, not a one-off: a
decoder that turns the requested resident-byte frontier into a pointer
at set_avail() time has nothing to clamp against when task_image
raises it to (size_t)-1 before the first process() call.
This file only ever exercised TGA and BMP, the two that had the bug.
It now drives PNG and JPEG through the same ordering, via the
image_transfer facade and with the start/iterate phase those two need,
so all four avail-aware still decoders are checked against the order
their caller actually uses.
They pass, and the reason is worth recording: PNG, JPEG, WEBM and MP4
clamp the request in the size domain first, because their
set_buf_ptr() is handed the buffer length. rtga_set_buf_ptr() and
rbmp_set_buf_ptr() are not - that asymmetry in the existing API is
precisely why those two were the only ones able to get it wrong. A
sixth avail-aware decoder will inherit whichever shape it copies, so
the check belongs in the suite rather than in anyone's memory.
Fixtures stay generated: the PNG is built with zlib in the test, the
JPEG is a 678-byte baseline image embedded as an array. Reverting the
two decoders to their pre-fix state fails 44 checks; 285 pass on
master, clean under ASan/UBSan/LSan.
The frame descriptor carried the overlay's text in a 1024 byte array,
three quarters of its 1344 bytes, and the threaded wrapper copied all of
it into a ring slot every frame inside the lock the video thread waits
on - text a driver reads only while the overlay is shown, and stack
left over from the frame before while it is not. stat_text is a pointer
now: the text is built in video_driver_state_t, which the main thread
owns, and the wrapper copies it into the frame's own buffer when there
is any, as the widgets' status text already travels. The descriptor is
328 bytes, the handoff copies that instead of 1344, and
video_thread_frame spends 131 fewer instructions a frame.
The buffer is emptied as the descriptor is built, so a driver that tests
the text finds none rather than the previous frame's.
Fixes the --disable-menu build, which a826781c51 broke:
gfx_thumbnail_animate() called menu_driver_get_current_time(), which
does not exist without HAVE_MENU - implicit declaration at compile,
undefined reference at link. Linux-Headless, Linux-asan-ubsan,
Linux-autoconf-tests and Linux-tsan-threaded-video all configure with
--disable-menu.
Guarding the call with an ifdef would have kept the real mistake: a
gfx-layer component has no business reaching into the menu driver, and
none reading a clock of its own either. gfx_thumbnail_animate() now
takes the frame's monotonic timestamp as a parameter, exactly as
gfx_animation_update() already does with the value runloop_iterate
samples once per iteration. The three menu drivers pass it (a field
read, not a clock read), so there is no menu dependency in the pacing
path, no ifdef, and every thumbnail advanced in a frame still shares
one coherent now.
It also makes the pacing testable with synthetic time, the way
samples/gfx/gfx_animation_sanitize already drives gfx_animation_update
- which a private clock read made impossible.
f5388788f7 fixed the blank BMP/TGA thumbnails, but the test added
alongside it still asks the decoders questions in an order this file
invented rather than the one the caller uses - which is precisely how
that regression reached master with a passing test suite.
Every existing check here raises the resident-byte frontier only AFTER
the first process() call. task_image does the opposite: it calls
set_buffer_ptr, then raises the frontier BEFORE each process(), and
the moment the read completes it raises it to (size_t)-1 - which for a
small file, i.e. every file-browser icon, happens before any process()
has run at all. That last case was the bug, and nothing here went
near it.
So replay the caller's sequence instead of imagining one, at four read
granularities: the whole file in one chunk (the completion raise lands
first), and one-byte, 64-byte and 4 KiB reads (many partial raises,
then the completion raise). Each combination must run to
IMAGE_PROCESS_END and be byte-exact against the untouched whole-buffer
decode, for raw and RLE TGA and for 24bpp and paletted BMP.
Checked against the decoders as they were before f5388788f7: 40
failures, where the test that shipped with the fix finds 8 and the
suite before it found none.
Six reader threads race a producer that frees every buffer behind a
retire, so any hole in the protocol surfaces as a heap-use-after-free
under ASan. The plain build asserts that a retire waited on a live
reader; TORTURE=1 widens the window after the tuple snapshot and
asserts that the re-validate branch fired.
A reader of the cached frame samples a generation counter, snapshots
the (data, dims) tuple, arms a hazard slot with the pointer it is
about to touch, then re-validates the generation before running its
callback outside the tuple lock. Teardown paths call
video_driver_cached_frame_retire(), which bumps the generation and
waits until no hazard slot names memory it is about to release, so
publishing a frame never waits on a reader. Targets without a
lock-free pointer CAS hold the tuple lock across the callback
instead.
Swapchain texture deinit and recreation, the d3d12 software
framebuffer paths, oga_free() and driver_uninit() all retire, the
last of them before video_driver_free_internal() releases memory a
lent software framebuffer may still be cached from. Core unload,
core deinit and core reset retire as well; sites that release
nothing keep video_driver_cached_frame_invalidate(). Retiring is
correct whoever owns the buffer, so the per-driver
invalidate_if(predicate) hook is dropped.
Regression from the partial-buffer work: BMP and TGA thumbnails
stopped appearing in the file browser entirely.
task_image raises the decoder's frontier before the first process()
call, and when a file finishes reading before any decode has run it
raises it to (size_t)-1 outright ("the frontier is now the whole
buffer"). set_avail turned that into a pointer immediately -
buff_data + SIZE_MAX - which wraps. true_end was still NULL at that
point, so the clamp meant to catch exactly this was skipped, and every
later comparison against the wrapped end was nonsense: the header
check saw a negative length, returned IMAGE_PROCESS_WAIT for bytes
that had already arrived, and the image never decoded. Small files -
which is every file-browser icon - take that path every time.
set_avail now records the requested frontier and resolves it against
the buffer once process() learns the length, so the call is safe in
any order and (size_t)-1 simply means "all of it".
samples/formats/image_avail covers the ordering: for four fixtures it
raises the frontier before the first process() call, with both
(size_t)-1 and the true length, and requires the decode to complete
and be byte-exact against the untouched whole-buffer path. Reverting
just the two decoders makes it fail 8 checks - which is what it should
have been doing before this shipped.
a826781c51 was meant to carry only the pacing fix, but the
frame-collapsing loop was left in the same file when the two were
split apart, so master calls image_transfer_anim_stream_skip() and
_render() - which are not declared anywhere, because the facade half
of that work was deliberately held back for being unproven.
The frame acquisition goes back to image_transfer_anim_stream_next().
The pacing fix - the 1 ms duration guard in place of the 16 ms
playback-speed bug, and taking the runloop's per-iteration time sample
instead of re-reading the counter - is untouched and is what that
commit was for.
GFX_THUMB_ANIM_DUR_MIN was 16 ms, a hardcoded 62.5 Hz assumption, and
it was applied to each frame's duration BEFORE that duration was
accumulated into the next-frame deadline. So it was not a frame-rate
cap, it was a playback-speed bug: a 120 fps source's 8.33 ms frames
each became 16 ms and the clip played at half speed. Measured over
600 frames of a 120 fps source, a 4.80 s clip took 9.60 s; a 30 fps
source was unaffected, which is why this went unnoticed.
The floor is now 1 ms and does only what its comment always said it
was for - stopping a hostile file from asking for thousands of
decodes per second. Frames arriving faster than the presenter are
dropped by the deadline comparison, which already adapts to whatever
cadence the caller runs at, so no refresh-rate query is needed or
wanted here.
gfx_thumbnail_animate also re-read the monotonic counter itself. The
runloop already samples it once per iteration and threads it down
(runloop_iterate -> menu_st->current_time_us); gfx_animation and the
menu's own timers pace off that value, and every caller of this
function is a menu driver inside that path. Re-sampling cost a read
per animated thumbnail per frame and gave two thumbnails advancing in
the same frame two different "now"s, so deadlines that should be
coherent within a frame drifted apart from each other and from the
animation tick. It takes the frame's sample now, falling back to a
read only if a caller outside the menu iterate ever appears.
gl3_frame() latches video_frame_info_t::gpu_recording into
GL3_FLAG_GPU_RECORDING, and the PBO ring's teardown there and its lazy
init in gl3_read_viewport() read that instead of the frontend's
recording state. The video thread was reading it under the threaded
wrapper while the recording toggle wrote it, which a start/stop stress
reported on every run and reports on none now; gl2 and the Vulkan
driver take the flag the same way.
gl3_init() still reads the recording state directly; it runs on the
video thread while the main thread waits for the init reply.
Both decoders became avail-aware, and the task layer raises their
resident-byte frontier each tick, but nothing ever started them: the
gate that decides a decode can begin early had cases only for the
video types, WEBP, PNG and JPEG, so TGA and BMP waited for the whole
transfer and the wall never engaged.
rtga_header_ready() and rbmp_header_ready() fill that in, mirroring
rpng_header_ready(). BMP's answer is bfOffBits - the DIB header, any
bitfield masks and the palette all lie below it, and pixel data starts
there. TGA has no magic, so the check is that the image type byte is
one this decoder slices and that the id field and colour map, both
sized by the header, are resident; an unsupported type declines and
loads whole exactly as before.
samples/formats/image_avail checks the new gate against the decoder
rather than against itself: for each fixture it walks the prefix
length one byte at a time and requires the answer to be monotonic, to
turn true before the whole file has arrived (otherwise partial
decoding is silently off and every other test here still passes), and
- at exactly the frontier it first accepted - to be a prefix the
decoder really can open on. 217 checks, clean under ASan/UBSan/LSan,
image_decode_fuzz unaffected.
The lane skipped HAVE_SHADERPIPELINE over the only thing it reported:
the GLSL sources built into gfx/drivers/gl_shaders are longer than the
509 characters C90 guarantees a compiler will take in a string literal,
which nothing in our declarations can change. -Wno-overlength-strings
covers that one diagnostic, as tools/mingw_syntax_check.sh already does,
and the lane keeps checking what it is for: declarations after
statements, C++ comments and the rest of C99. The pipeline's translation
units build and run clean under it.
HAVE_OPENGL_CORE and HAVE_VULKAN stay out. Both drivers reach the slang
filter chain in every path that draws, and it is built from glslang and
SPIRV-Cross, which are C++: gl3 compiles without them once its
scRGB and pipeline state leaves the HAVE_SLANG guard, and then segfaults
on the first frame for want of the programs that guard was hiding.
BMP joins TGA, PNG, JPEG and the two video types as an avail-aware
still decoder: rbmp_set_avail() raises the resident-byte frontier and
a slice that cannot complete a row below it returns
IMAGE_PROCESS_WAIT. A caller that never calls set_avail decodes
exactly as before.
As with TGA the wall is the cursor's existing img_buffer_end; what
changed is the places that treat 'ran out of bytes' as 'file ended':
- neither row loop had a frontier check, and past the end rbmp_get8
returns zeros - a valid palette index and a valid pixel, so a stall
decoded zeros as image data rather than failing visibly. Both now
stop when fewer than one padded row is resident.
- rbmp_begin validated the header's geometry against the whole pixel
payload using img_buffer_end, which with a frontier is a prefix:
every partial file was rejected outright. It measures against the
true end of the data instead, so the check keeps meaning what it
meant (does the file hold what its header claims) rather than
becoming a residency test.
- begin also walks the DIB header, bitfield masks and palette, all
located by the file itself; rather than bound each read it waits
until bfOffBits, the point where pixel data starts, is resident.
samples/formats/image_avail gains six BMP fixtures - 32/24/16bpp,
top-down as well as bottom-up, and 4bpp and 8bpp paletted - each
decoded whole and again with the frontier raised 1, 3, 7 and 1024
bytes at a time, surfaces required bit-identical. Bottom-up storage
makes this a real test of resumption: a prefix is the bottom of the
image, so a row cursor resumed wrongly displaces the picture instead
of producing noise. 193 checks, clean under ASan/UBSan/LSan, and the
image_decode_fuzz sweep still passes.
* iOS: fix hardware keyboard keys getting stuck
- Handle pressesCancelled as key release; UIKit delivers it instead of
pressesEnded when the system interrupts a press, leaving the key
latched in apple_key_state (#16212)
- De-dup key events per (timestamp, keycode, direction) instead of
timestamp alone, which dropped the second of two keys pressed or
released in the same frame
- Reset keyboard state when the app resigns active, matching the macOS
port
* iOS: record the keyboard event de-dup per key and direction
UIKit hands each key event to -handleKeyUIEvent: and
-_keyCommandForEvent: twice, so both paths accept one event per
(key, direction) pair per timestamp and pass everything else straight
to super. The record is a bitmap indexed by keycode and cleared
whenever the timestamp moves on, so any number of keys can change
state within a single frame and each of them is delivered exactly
once regardless of the order UIKit chose to interleave them in.
Events without a _hidEvent, and events whose keycode falls outside the
range apple_key_state covers, go to super untouched: the keyboard
layer has nothing to do with either.
* Apple: publish the release when keyboard state is dropped
apple_input_keyboard_reset() walks the keys it finds held, sends each
one a key-up through input_keyboard_event() and then clears the local
state, so a core's keyboard callback and the menu's wait-for-release
see the release that focus loss, the companion window taking the
keyboard or a closing OSK dialog swallowed. The small-keyboard layer
latches on a held modifier and is dropped along with the keys it was
tracking.
The tvOS view releases cancelled presses too: UIKit sends
pressesCancelled: instead of pressesEnded: when it takes a press away,
and it is the last event that press produces, so it takes the same
path as an ended one.
---------
Co-authored-by: Radoslav Bacik <[email protected]>
Co-authored-by: libretroadmin <[email protected]>
TGA joins PNG, JPEG and the two video types as an avail-aware still
decoder: rtga_set_avail() raises the resident-byte frontier and a
slice that cannot make progress below it returns IMAGE_PROCESS_WAIT
instead of treating the wall as end-of-file. A caller that never
calls set_avail decodes exactly as before.
The wall itself is the cursor's existing img_buffer_end, so every
bounds check in the decoder already respects it; what had to change is
everywhere the decoder treats 'ran out of bytes' as 'file ended':
- rtga_fast_rows padded the remaining rows and declared the image
finished. Correct for a truncated file, wrong for a frontier: it
now returns with the row counter where the loop stopped.
- the generic (RLE/truecolour) loop had no bounds check at all - past
the end rtga_get8 hands back zeros, so a stall decoded zeros as
pixels. It stops when fewer than five bytes are resident, the
conservative bound covering a packet header plus four components.
- the indexed loop had the same hole, and zero is a valid palette
index there, so the wall has to be tested rather than detected.
- rtga_begin reads the id field and colour map whose sizes only the
header knows, so the frontier check sizes them from the header
before deciding it has enough.
Verified by samples/formats/image_avail, which decodes each fixture
whole and again with the frontier raised 1, 3, 7 and 1024 bytes at a
time and requires the surfaces to be bit-identical: raw 24/32bpp,
bottom-up and top-down, RLE 24/32bpp with runs straddling the wall,
and indexed with the wall inside the palette. 97 checks, clean under
ASan/UBSan/LSan, and the existing image_decode_fuzz sweep still
passes. It found every bug listed above.
The harness stubs filestream_copy_begin/step/close as an observable
copy - it records every destination it is asked to open, can refuse
the open so the temporary-name walk stays visible, and hands its
bytes out one bounded step at a time - and links the real shared
per-frame I/O window from task_nbio_slice.c rather than a stand-in.
That lets the cases pin the handler's pass split: the first pass
opens the destination and moves nothing, the copy advances in steps
no larger than the step bound after it, and a copy abandoned by a
generation bump is closed and its temp file deleted.
-Werror=implicit-function-declaration makes a fragment helper the
harness has not kept up with fail at the call site.
runahead copied a core with a 1MB heap buffer and a read/write loop
that ran to completion inside the task handler: a 40MB core (mame,
dolphin, ppsspp) held the handler for the whole file, and on a
non-threaded task queue that is the frame.
It is now a resumable handler over the VFS copy. Pass one opens the
copy (no bytes move); each later pass advances it by one share of the
shared per-frame I/O window - the same window task_file_transfer.c and
the content scanner use, so a copy running alongside them costs the
frame the window once rather than a slice each - and returns. Nothing
sleeps and nothing runs to completion inside a pass. A cancelled task
closes the open copy, which removes the partial destination.
The transfer buffer is gone on every desktop platform: the VFS uses
copy_file_range on Linux, CopyFileW on Windows and clonefile on APFS
(constant time, no bytes move at all), none of which brings bytes into
user space. Where there is no kernel primitive the VFS uses one
bounded buffer of its own, sized from the file and capped at 128KB.
copy_file_with_random_name becomes copy_begin_with_random_name: it
picks a free destination and opens the copy, so the fallback path
cannot block either.
task_content.c's UWP branch calls filestream_copy() rather than
CopyFileFromAppW directly - the same kernel copy through the UWP VFS
backend, without the local UTF-16 conversion, and it creates the
destination directory.
* libretro.h: VFS API v5 - read-only state, mtime, copy, dirent_stat
Adds RETRO_VFS_STAT_IS_READONLY, RETRO_VFS_COPY_OVERWRITE and five
interface entries appended after stat_64: set_readonly, get_mtime,
set_mtime, copy, dirent_stat. Existing offsets are unchanged; cores
requesting <= 4 see no difference.
Motivated by melonDS DS (host <-> emulated FAT sync mirrors read-only
state and modification times), Craig (a file copy through the VFS,
cp/std::filesystem::copy_file semantics) and Psyraven (entry size
during enumeration without opening the file).
For more information, see this issue
https://github.com/libretro/RetroArch/pull/19523
SDL3 has no device clock - there is no position and no timestamp to be
had from its audio API, which is a stream abstraction over the backend
- so there is nothing here to fit a drift estimate from, and this
driver gets none.
What it was missing is more basic. frames_consumed was NULL, so the
sink rate estimate that drives rate control had nothing from the
driver at all on SDL3 and ran on the frontend's own accounting alone.
SDL does say how much of what was put in is still queued, and what
went in less what is still waiting is what the device has taken. That
is the shape ALSA uses, and it is worth more here than a clock this
API cannot provide.
Counted in device frames at the one point both write paths funnel
through, so the write_raw path - int16 stereo at the core's rate, SDL
resampling - is converted by the rate ratio on the way in, the same
conversion write_avail() already applies to the queue depth. The
priming silence is counted too: it is queued like anything else, and
leaving it out would hold the count at zero until it drained and
short by that much for ever after.
Stated plainly in the comment: this does not account for the frequency
ratio rate control sets on the stream, which shifts the input-to-output
relationship by its own correction, a fraction of a percent. So it is
an estimate of the device's consumption rather than a reading of it -
the same standing as the ALSA position, and unlike a real device
clock. There is no device_clock_ppm() beside it for that reason.
Verification is thinner than I would like and worth saying so. SDL3's
headers are not present where I can build, so the file as a whole has
not been compiled - only the new function, against a minimal stub,
driven through the device-format case, the resampled raw-path case
and an underfilled one to check the subtraction cannot wrap. Those
read 46976, 46886 and 0 frames as they should. The CI that builds
SDL3 is what will catch anything else.
Of the three drivers asked about, this is the one that has a real
device clock to read.
ALC_SOFT_device_clock reports the device's own clock in nanoseconds.
Paired against the frames this driver has seen consumed, its slope is
the rate the hardware is really running at - the same fit the other
six drivers now carry.
It is an OpenAL Soft extension, not core OpenAL, and Apple's framework
does not carry it, so it is asked for at open through
alcIsExtensionPresent and alcGetProcAddress and is simply absent where
it is not offered. The constant and the function type are declared in
this file rather than taken from alext.h, which is the pattern the
AL_SOFT_events block above already follows and for the same reason -
that header is not shipped everywhere this builds.
Two things checked against the real library rather than assumed. The
extension resolves before a context exists, which is where this probes
for it. And the clock reads zero and does not advance until a context
is playing, then ticks with the mixer - 280 ms across a 300 ms sleep
on the null backend - so the guard that skips a non-positive reading
is doing real work rather than covering a case that cannot happen.
The frame count is quantised to a queued buffer, since that is when a
buffer is reported processed. That shifts the line and leaves its
slope alone, which is what the fit reads, so it costs nothing here -
the same reason ALSA can fit a delay-corrected position.
Logged at teardown and offered to the overlay, like the others.
Nothing acts on it.
Not driven end to end: the driver compiles and the extension was
exercised directly, but there is no harness running this driver the
way alsa_clock and the WASAPI pacing test run theirs.
audio_driver_set_core_float() takes the format when the negotiation
lands after init on a ring with nothing in it, which it must - the
core can hand the float entry over at any point. But it recomputed the
stride for a stereo frame, while a core that had already taken the
multi-channel entry has a ring built for the canonical wide frame. The
two then disagreed: a stride of 8 bytes against a frame eleven slots
across, where it should be 44.
Every count on the pipe is in frames and only the ring's edge speaks
bytes, so what the disagreement produces is a ring read at a fraction
of the frames it holds. The other place that sets these three derives
the stride from pipe_channels; this one had the 2 written in.
The order that reaches it is multi negotiated first, the pipe built
wide, float negotiated after on an empty ring. Whether a core and the
runloop produce that order I have not traced, so I am not claiming
this bites anyone today - but it is unconditionally wrong when the
path runs wide, and the fix is smaller than tracing it would be.
samples/audio/pipeline_format grew that case, which fails before the
change with the numbers above and passes after. It sits beside the
36 existing format cases, which still pass, as do the clocked,
latency-contract and discrete-multichannel harnesses.
This is the concrete instance of the thing an audit note asked for
architecturally - that the pipe's format fields be one object rather
than several mutable ones. That change is not made here. The stride
is derived from the width and the format, and storing derived state
separately is what allows one of them to be set alone; if a fifth
field ever joins them, that is the moment to make them a single
constructed object rather than now.
I missed this driver when I did the other five, which is the wrong way
round: WASAPI reads the best clock of the lot and was the closest of
all of them to having this.
Both IAudioClock::GetPosition and IAudioClock2::GetDevicePosition hand
back a QPC timestamp beside the position - the instant the position
was taken at, in hundreds of nanoseconds, which the interface
specifies so no counter frequency is needed. This driver read it into
a variable at both call sites and dropped it. Position against that
timestamp is the rate the hardware is really running at, for no call
that was not already being made.
It is not the Clock-vs-events line, which is the clock against this
driver's own event count and says what the approximation costs. This
is the clock against nominal and says what the hardware is doing.
Both are shown now.
And unlike the ASIO, CoreAudio, PipeWire and JACK ones, this is
tested against the shipping driver rather than a copy of its
arithmetic: the pacing harness compiles audio/drivers/wasapi.c, so the
fake endpoint grew a real clock - its QPC timestamp derived from the
position at a rate that is deliberately not the nominal one - and the
new case runs a device at a known offset and reads the driver's answer
back through the vtable. A device at 0, +50 and -80 ppm reads +0, +50
and -79. The fake had been returning the position as its own
timestamp, which is why nothing fitted before.
Logged at teardown and shown in the overlay, like the others. Still
nothing acts on it.
Clean under ASan, UBSan and TSan; through the mingw check; driver
sweep passes; every audio_driver_t initialiser still checked with
-Werror=incompatible-pointer-types.
Five drivers now fit an estimate of what the device's clock is really
doing, and all five printed it once, at teardown. To compare a figure
against the sink estimate that drives rate control you have to watch
it settle, which means watching it - so it goes in the overlay beside
the sink line, live.
A vtable entry carries it: device_clock_ppm(), returning false until
there is an estimate and NULL for the drivers that have no clock to
read, which is most of them. ALSA gives its audio timestamp where the
card counts one and the fitted position otherwise; CoreAudio gives
mRateScalar where the HAL fills it in and the fit otherwise; PipeWire,
JACK and ASIO give their fits. The threaded wrapper forwards it, which
is the part that broke the Android build the last time I added to this
vtable.
The entry is at the END of the struct, with a comment saying why it
must stay there. The same mistake - an entry inserted in the middle,
shifting every driver's initialiser by one - is what shifted underruns
into layout across three drivers a fortnight ago, and the compiler
only complains when the types happen to differ.
Still nothing acts on any of it. The overlay line exists so that the
question these were built to answer can be answered: whether the
device's own clock agrees with the estimate already driving DRC. That
is a reading off real hardware, and none of the five has had one.
Checked: every file declaring an audio_driver_t compiles with
-Werror=incompatible-pointer-types, which is what catches a shifted
initialiser; the driver sweep passes; asio through the mingw check;
coreaudio in all four combinations of TARGET_OS_IPHONE and
HAVE_MICROPHONE; and both clock harnesses still pass.
vulkan_frame() latches video_frame_info_t::gpu_recording into
VK_FLAG_GPU_RECORDING, and the streamed readback's teardown there, its
lazy init in vulkan_read_viewport() and vulkan_init_readback() itself
read that instead of the frontend's recording state. The video thread
was reading it under the threaded wrapper while the recording toggle
wrote it, which a start/stop stress reported on every run and reports
on none now; gl2 takes the flag the same way.
vulkan_init_readback() takes the state as an argument because both of
its recording callers reach it from the video thread. Only vulkan_init()
passes a direct read, on the thread that writes it, before the video
thread runs.
The last two of the five. Both servers already hand over the pair this
needs and both drivers were using half of it.
PipeWire costs nothing at all: pw_time carries now, ticks and delay,
this driver fetched the struct for the latency line and read only
delay. ticks against now is the graph's clock against the wall, and
since ticks are in units of pw_time.rate - 1/samplerate for an audio
stream - both axes are seconds and the slope is the ratio, needing no
sample rate to interpret. It goes in beside the existing read, under
the lock that was already held, so there is no new call and nothing
added to the data thread.
JACK costs one call: jack_get_cycle_times() returns the frame time at
the start of the cycle with the microsecond time of the same instant,
and is meant to be called from the process callback, which is where
the frame count is already kept.
The catch there is that current_frames is 32 bits and wraps in about a
day at 48 kHz - sooner if the server was up before RetroArch started,
the counter being the server's and not ours. The difference is taken
in 32 bits, where the wrap cancels, and accumulated into 64. Widening
first turns one wrap into a four-billion-frame jump and a wildly wrong
clock for the rest of the session, which is silent rather than
obvious, so the harness starts a run two seconds short of the wrap and
requires the same answer.
Logged at teardown, not acted on, as with the other three. Both are
called a graph or server clock rather than a device clock in the log,
because that is what they are: with a device driving the graph it is
that device, and with something else driving it, it is that.
Neither a JACK server nor a PipeWire daemon can be started where I can
run things, so this harness mirrors the two conversions rather than
driving the drivers - a weaker check than the ALSA one, which drives
the shipping code on a real PCM. Both drivers compile against the real
system headers. What is not covered is anything only a running server
would show.
Third of these, and the first where the harness drives the shipping
driver rather than a copy of its arithmetic.
frames_consumed() is this driver's own write count less what the
device still holds. That is close, a device being what consumes the
frames, but it is an inference, and it says nothing about when. ALSA
will say when: snd_pcm_status() hands over the delay and a timestamp
for it in one snapshot, and where the driver counts an audio timestamp
as well - the DMA or link clock - it hands over the hardware counting
itself.
So two estimates. The audio timestamp against the system timestamp is
two clocks compared directly, needing neither a rate nor a delay, and
it is the better number where it exists. Plenty of devices leave it at
zero - the null PCM does - so the position is fitted against its
timestamp as well, which every device can answer.
That position is inferred, and fitting it is what makes that
tolerable: an error in the delay shifts the whole line and leaves its
slope alone. The harness makes that concrete by putting an absurd but
constant 40000-frame delay in and requiring the same answer, which it
gets.
Both are logged at teardown and neither feeds rate control, as with
the ASIO and CoreAudio ones: these are measurements until they have
been read off a range of hardware.
The harness opens the null PCM for real and interposes the four
libasound calls the estimate is built from, so a device can be made to
run at a chosen offset and the driver's own answer read back out of
what it logged - the line a user would see, not a field reached into
behind the driver's back. Clean under ASan, UBSan and TSan.
This is the one of the three you can check yourself without borrowing
hardware: it should print a ppm figure against any card you have, and
whether it agrees with the generic estimator is the thing worth
knowing before any of these feed rate control.
Whether a row may invent a conversion English does not have turns on
whether anything formats it, so tools/msg_hash_format_check.py measures
that instead of reading it off the MSG_ prefix: format_site_keys()
walks the sources for printf-family calls whose format argument
resolves to msg_hash_to_str(), directly or through a local it was
assigned to, and the MSG_ namespace is unioned in so a site the scan
cannot see keeps the cover it had. Ninety rows reach a format argument
today and twenty-nine of them sit outside that namespace; each already
carries a conversion in English, so the ordering rule holds them and
the tree is unchanged by the widening.
The scan is heuristic in one direction only. A site it misses leaves
the row where it was, and a site it invents costs a rule applied to a
row that did not need it, so it is safe to be generous. Its own
fixtures are in the self-test: a format argument taken from the table
straight and through a local must be seen, and a string that is
copied, passed as a value, or split on a placeholder must not be.
A row that reaches snprintf() is formatted with the arguments its call
site supplies, whatever language is loaded - cheevos.c passes
MSG_CHEEVOS_HARDCORE_PAUSED_SYSTEM_NOT_FOR_CORE two char*, connmanctl.c
passes MSG_LOCALAP_STARTING two more - so a translation whose
conversions differ from the English row's makes snprintf read an
argument list nobody built for it. Three rows do: Simplified Chinese
reorders the achievement message with positional %n$, which is POSIX
and which the MSVC runtime does not implement, and Polish loses a %s
from the access point message and a %u from the runahead one. They come
out of the tables, so the lookup falls back to the English row, and
intl/json2h.py drops such a row instead of emitting it - a bad Crowdin
sync degrades to untranslated rather than to a bad vararg read.
tools/msg_hash_format_check.py is the rule in full: the same
conversions in the same order wherever English has any, no positional
conversion anywhere, and no conversion invented in an MSG_* row that
has none in English. Outside that namespace a row with no English
conversion is displayed rather than formatted - menu_driver.c strlcpy()s
sublabels and help resolves through msg_hash_get_help_us_enum() - so a
percent sign in prose stays legal there, as does the space flag, which
C defines for signed conversions only and which no English row uses
deliberately. It runs beside the pairing check in the same lane; json2h
is importable now so both it and the check read English from one
expansion of msg_hash_us.h.
The same measurement the ASIO driver just gained, and CoreAudio was
closer to it than ASIO was: frames_consumed() already asks the HAL for
the device's sample position, so half the answer was in hand. What was
missing is the other half - the render callback is handed an
AudioTimeStamp on every call and threw it away, and that timestamp
carries mHostTime beside mSampleTime. The slope of one against the
other is the rate the hardware is really running at, for a handful of
flops per period and no call of any kind.
Beside it, mRateScalar: the HAL's own figure for the same quantity,
computed by something with better information than this fit has, and
read here where the HAL flags it valid. The two should agree, and a
disagreement is the interesting result.
Both are logged at teardown and neither feeds rate control, for the
same reason as the ASIO one: until these have been read off real
hardware they are measurements, and a clock estimate that is wrong is
worse than one that is absent.
The fit is a least squares over every callback rather than two points,
for the reason the ASIO harness demonstrated: noise on a single anchor
is divided by the window and reads as drift, fifty parts per million
for half a millisecond at ten seconds.
What is genuinely different here is the time axis, and it is the part
worth testing. mHostTime is in mach absolute units, not nanoseconds.
mach_timebase_info gives the ratio: one to one on Intel, where the
distinction cannot be seen, and 125 to 3 on Apple silicon, where
ignoring it reads a 48 kHz device as 1152 Hz. Code that only ever ran
on an Intel Mac would look perfect and be wrong on every Mac sold
since 2020, so the harness drives the same clock through both
timebases and a third that ships nowhere.
This one is not behind the desktop guard: the callback's timestamp
arrives on iOS and tvOS too, and so does mach_timebase_info. Checked
in all four combinations of TARGET_OS_IPHONE and HAVE_MICROPHONE
against the stub headers, warnings as errors, and against the
declared-but-never-defined check - given how many times I have put
CoreAudio code on the wrong side of a guard this month.
The gfx sources that use retro_atomic.h are compiled under each
RETRO_ATOMIC_FORCE_* backend with -Werror, so an ungated extended
operation or a counter that warns as a statement is caught here rather
than on the console lane that happens to select that backend.
The read-back frame is published and taken with one atomic exchange
each, and retro_atomic.h offers the extended operations only on the
backends that can express them, asking callers to gate on the
primitive. video_thread_record_take() returns -2 where there is none,
the code the recorder already takes when a driver has no read-back, so
recording falls to the frontend's own frame copy there.
An expression that recovers the old value arithmetically evaluates its
operand twice and draws -Wunused-value wherever the result is
discarded, which is most of the tree's counters. A helper is what OR
and AND already need, being non-invertible, and it is the form MSVC can
take, having no statement expressions.
ARMv5 has no ldrex/strex, so the compiler will not inline a
read-modify-write and leaves __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4
undefined, but libgcc builds config/arm/linux-atomic.c for every
arm*-*-linux* target and exports __sync_fetch_and_add_4 and the rest on
top of the kernel's __kuser_cmpxchg. The builtins compile to calls
those helpers resolve, which is what separates this case from a target
where nothing supplies the call, so the __sync backend is available:
the ARMv5 handhelds get atomics against preemption on their task
counters and thread state words, and the extended operations with
them.
From the same audit. This driver answers kAsioSupportsTimeInfo with
yes and then hands the ASIOTime straight past without looking at it,
which is a claim it does not act on. The information is worth having:
a callback's arrival time carries the OS's scheduling jitter, and the
sample position and system time a driver reports do not, so the ratio
between them is a cleaner reading of what the hardware clock is
actually doing than anything timed from the outside.
It is measured and logged at teardown, and nothing else. It does not
feed rate control, because I have no interface to read it off and a
clock estimate that is wrong is worse than one that is absent. What
this lands is the plumbing and the arithmetic, so that a number from
real hardware can be compared against the estimator already driving
rate control before either is preferred.
Two things needed care.
The layout is an ABI. The position and the timestamp are 64-bit
quantities the SDK carries as two 32-bit halves on Windows, because
asiosys.h leaves NATIVE_INT64 at 0 there. Getting that wrong does not
fail to build - it reads a driver's numbers as garbage inside a
real-time callback - so the offsets are asserted at build time, where
the compiler that sees the real Windows ABI checks them, and again in
the harness against the SDK's published ones. Swapping two fields
fails the build; I checked that it does rather than assuming.
And the estimate was wrong when I first wrote it. Anchoring on one
observation and measuring to the next divides that anchor's own noise
by the window and reports it as drift: half a millisecond on the
anchor is fifty parts per million at a ten-second window, which is
larger than the drift being looked for. The harness caught it - it
read +450 ppm on a clock running at +50 - and it is a least-squares
fit over every callback now, which has no privileged point for the
noise to hide in. Same test now reads +49 ppm through two
milliseconds of jitter, far worse jitter than a real system.
The fit is kept in seconds and samples relative to the anchor rather
than raw nanoseconds, because a fit on numbers of order 1e10 loses
its answer to cancellation when the squares are subtracted.
Runs on Linux in the samples-audio workflow beside asio_ring, clean
under ASan and UBSan. The driver compiles through
tools/mingw_syntax_check.sh.
The PS2 lane compiles the backend; this runs it. ps2stub/ stands in for
ps2sdk's <kernel.h> with an interrupt mask the test can inspect, so the
sample checks the capability flags, every operation's return value and
stored result, that each operation opens and closes exactly one masked
section, and that an operation nested inside a section the caller
opened leaves the mask alone on the way out - the property DIntr()'s
return value exists for, and one that would otherwise show up on
hardware as rare corruption rather than as a failing test.
The R5900 has no LL/SC - GCC's ISA_HAS_LL_SC excludes TARGET_MIPS5900 -
so the builtins lower to __atomic_* libcalls that the ps2sdk toolchain
has no libatomic to resolve, and the header takes the interrupt mask
instead: the EE is one in-order core whose kernel reschedules only out
of an interrupt, so DIntr()/EIntr() around a read-modify-write is the
atomic. That covers the whole surface, including exchange, CAS, the
pointer ops and the 64-bit ops, so RETRO_ATOMIC_HAS_CAS, _HAS_PTR and
_HAS_64 hold on PS2 and rthreads.c's fetch_or on the thread state word
is indivisible there.
RETRO_ATOMIC_LOCK_FREE stays unset on this backend: the ops are atomic
but masked, and a spin loop on one core never lets the thread holding
the word run again. font_driver.c's font-file lock gates on that as
well as on the primitive, so it keeps its unlocked path here.
intl/msg_hash_<lang>.h pairs its blob against its ids[] by position and
by nothing else: msg_hash_strtab_index_build() walks the blob once and
gives row N to ids[N]. Two rows sitting in twenty-nine blobs with no
ids[] entry of their own shift every row that follows them, so each of
those languages shows real translated strings belonging to other menu
entries from the audio device row onward. The rows go, and the
contiguity constants come down with them.
tools/msg_hash_packed_check.py holds the invariant from here on. Member
names are the djb2 hash of the key their row was emitted for, so it
re-derives the pairing and compares it against ids[] row for row under
the guards, together with each member's width, embedded NULs, and the
sizeof() constant that tells a padding compiler to fail the build.
Reading the headers rather than building them means one run covers
every language and every #ifdef configuration, which no build job can
do and which an English-only test cannot see at all, msg_hash_us.c
being a switch rather than a packed table. intl/json2h.py runs the
check on its own output before writing, and Msg-Hash-Packed-Check.yml
runs it on the tree.
video_frame_info_t carries gpu_recording, set once a frame by
video_driver_build_info() on the thread that writes the recording state,
and gl2_frame() latches it into GL2_FLAG_GPU_RECORDING. The PBO ring's
init and teardown read that flag rather than the frontend's recording
state, which the video thread was reading under the threaded wrapper
while the recording toggle wrote it: a start/stop stress reported the
race on every run and reports none now. It arrives the way the driver's
other per-frame state does, so it costs no lock and no round trip.
gl2_init() still reads the recording state directly; it runs on the
video thread while the main thread waits for the init reply.
I have been carrying a note that the CoreAudio ring corrupts 5.1: the
capacity is the sample count rounded up to a power of two, six
channels divide no power of two - 4096 samples is 682 frames and four
over - so the cursors wrap four samples into a frame, and I had that
down as the channels rotating from there on. Stereo, quad and 7.1
divide a power of two and are fine, which fitted the absence of any
report.
This is the harness I wrote to prove it, and it disproves it. The ring
is a FIFO. A read returns samples in the order they were written,
whatever position in the buffer they sit at, so where the wrap falls
never reaches the device. Case 3 runs the traffic for every layout and
case 4 forces every single transfer across the wrap on purpose; six
channels come back exactly as intact as two. I had reasoned about the
buffer's geometry and not about what a FIFO does.
So there is no driver change here. What the harness is worth keeping
for is the hazard that is real and next door to it: a transfer whose
COUNT is not a whole number of frames leaves the read cursor a sample
behind the write cursor, and every frame after it is skewed for good.
That is why the callback trims a short read to whole frames. Case 5
does the untrimmed read deliberately and fails if it goes unnoticed,
so that line has a guard rather than a comment.
Arithmetic only, so it runs on Linux in the samples-audio workflow
beside asio_ring. Clean under ASan and UBSan.
The note is retired. Three of the four pending CoreAudio items are
still open and still need a Mac.
An undersized output buffer could leave a partial frame in the ring cursor and uninitialized trailing bytes in the output. Clamp reads to whole frames, silence the incomplete tail, and count invalid storage. Treat a null output pointer as an invalid buffer without consuming queued samples.
Round writable byte demands upward before frame alignment so a fractional sample cannot make insufficient free space count as success. Keep the existing oversized-request policy and wait mechanism.
Extend the deterministic source-level tests for stereo and six-channel short buffers, subsequent channel alignment, null output, and byte rounding. All 92 checks pass; the existing 36 pipeline-format cases also pass. Apple hardware playback remains unverified.
From an audit of the backend. Two findings, one of them a hole the
rate-change callback was meant to close and does not.
ad->sample_rate is only ever what this driver last asked the hardware
for. A reset raised from the driver's own control panel may have
moved the rate since - which is precisely what
sampleRateDidChange() reports, and why it asks the frontend to
reinitialise - but nothing read the new value back. So on the reclaim
path the comparison was the core's request against a stale number: a
driver moved to 44100 while the core still wanted 48000 compared
48000 with 48000, found them equal, set nothing, and then built the
buffers, the period, the latency conversion and the ring from a rate
the hardware had stopped using.
The driver is asked what it is at now, before any of that. Then the
core's request is reconciled against the answer, and a rate that
moves joins the reasons to rebuild the buffers rather than being
applied after they were made - which is where it was, and which would
have left them sized for the old rate even when the set succeeded.
And consumed is atomic. It is written by the callback thread and read
by the writer, and the comment defended that with the word being
aligned - but an aligned word is what the machine will not tear,
which is not the same as the language permitting the unsynchronised
pair. The underrun counter in the same structure was already atomic
for exactly this reason.
Compiled with tools/mingw_syntax_check.sh, which is a real compile of
this file rather than a reading of it.
Two more things the audit raises are left alone deliberately: the
g_asio pointer has the same language-level question as consumed did,
but retro_atomic has no pointer type and inventing one under an audio
driver is the wrong place to start; and feeding ASIO's own time
information to the sink estimator is worth doing and needs an
interface to land on first.
Read completed GPU frames on the video thread and publish the latest
CPU buffer through a three-buffer exchange. Keep recording state private
so shared thread and poke-interface layouts remain unchanged.
Use dedicated GL2 PBO and Vulkan staging readers without cached-frame
redraws. Track readback validity and completion, retain in-flight storage,
and scale changed viewports into the recording's fixed output dimensions.
Validated with Windows LRPS2/paraLLEl-GS running Tekken Tag Tournament:
startup succeeds and recording survives the boot-time viewport change.
The recording decoded to 890 frames. Also checked GL2/Vulkan sample cores,
buffer-exchange stress, resize/letterbox behavior, and Vulkan error paths.
Keep the useful writer change from coreaudio-debug commit 09b66da: successful writes reset the retry budget, and waits request room for a whole frame. Healthy batches can span more than eight device callbacks without losing their tail.
No clock reads or new timing mechanism are added. The existing semaphore timeout and stalled-device checks are retained; wait_writable is unchanged.
Add deterministic source-level tests: 55 checks pass, versus 24 failures on the base revision. Apple hardware playback is not exercised.
The sink estimator reset cleared pipe_float after the ring had been configured, leaving float-sized frames interpreted as int16. Split consumer passes skipped samples and read beyond the converted batch. Preserve the negotiated format and stride.
Add a deterministic regression test through real initialization and both batch paths: 36 float/int16 and threaded/inline cases. Float threaded output was corrupted below the whole-batch threshold; all cases now preserve the input samples.
The 2005 toolchain fails on KSDATAFORMAT_SUBTYPE_IEEE_FLOAT and
KSDATAFORMAT_SUBTYPE_PCM as undeclared identifiers: they live in
ksmedia.h, which that SDK does not reach at _WIN32_WINNT=0x0400, and
the assignment then tries to convert an int to a GUID.
They are spelled out by value instead, which is what
audio/common/mmdevice_common_inline.h already does for the WASAPI
side - there for a different reason, that a newer SDK declares them
without an instance to link, but the same answer. Their values are
fixed by the WAVE format: 00000001 for integer PCM and 00000003 for
float, both in the 0000-0010-8000-00AA00389B71 family. Local to this
file rather than shared, because the header that has them is the
MMDevice one and dsound has no business including it.
Both uses are converted - the format the driver asks for, and the
name it logs a format by.
The other two drivers that name these, wasapi and xaudio, include
that header and are not exposed; checked rather than assumed, so this
does not come back as a second report from the same build.
Not built here. The remaining diagnostic in that log, a volatile
qualifier warning in rthreads at W3, is not from this and is not an
error.
GetTickCount was the wrong reach, and GetTickCount64 before it was
the wrong reach that also did not link.
cpu_features_get_time_usec() is what the rest of the tree uses, and
on Windows it is QueryPerformanceCounter: microseconds, monotonic, no
wrap to reason about, present everywhere this builds. It is also the
call the WASAPI pump already measures its own lateness with, a few
files away, which is what makes reaching for a Win32 tick counter in
a Win32 file the mistake it was rather than a judgement call.
On the resolution, since it was asked: GetTickCount resolves to the
scheduler tick, some fifteen milliseconds. For a 200 ms safety bound
that is harmless, and it would have stayed harmless - but it is most
of an audio period, so the habit of using it near this code is worth
not forming. The wrap arithmetic it needed is gone with it.
Two CI failures, both from my own recent work.
The 32-bit mingw link fails on GetTickCount64, which the ASIO
teardown wait uses for its deadline. That call is Vista and this
binary still builds for older Windows, where the import library has
no such symbol - the compiler said so as an implicit declaration and
the linker then said it plainly. It uses GetTickCount now, with the
deadline tested on the unsigned difference, which is correct across
the 49-day wrap.
And the pacing harness under ThreadSanitizer reported five races,
three of them one bug: the pump thread reads w->rate to work out the
period it is due at, and w->rate was assigned after wasapi_pump_start
had already created that thread. It is set before the pump exists
now.
The other two are the harness racing itself, from the withheld-events
case I added: the switch the test flips is read by the engine thread
every period, so it is an atomic store and load; and the fake's
device clock read c->stats.frames_consumed without the lock the
engine writes it under, which a real device clock would not do.
TSan reports nothing now where it reported five, ASan is clean, and
the driver sweep passes.
Three build failures across four CI jobs, all from code of mine
sitting under the wrong #ifdef.
iOS and tvOS: the property shim reaches for
kAudioDevicePropertyScopeInput and kAudioHardwareUnspecifiedError,
which belong to the desktop HAL's device and stream objects. The
phone platforms do not carry them - they drive the unit directly and
never ask an AudioObject for anything - so the shim goes behind
!TARGET_OS_IPHONE, where the rest of the HAL code in this file
already sits.
PowerPC 10.4 and 10.5: the default-output listener - both callbacks
and the function that registers them - sits inside
#ifdef HAVE_MICROPHONE while being declared and called outside it, so
a build without the microphone links with it missing. The block moves
out, keeping its own !TARGET_OS_IPHONE guard.
And ui/companion/companion_dock.c calls strlcpy without including
compat/strl.h; it relied on string/stdstring.h pulling it in, which
it does on the platforms I build but not where the header check
looks.
The check that finds this class is not a compile - the macOS
configuration is happy either way - but asking, for each combination
of TARGET_OS_IPHONE and HAVE_MICROPHONE, whether anything declared
static in the surviving text is never defined there. All four
combinations report none now; before this, the macOS-without-
microphone one named the listener.
Verified: the driver compiles clean in all four combinations against
stub Apple headers with -Wall -Wextra and warnings as errors,
companion_dock.c compiles, and the driver-vtable sweep passes.
video_thread_read_viewport() lets the worker draw every frame pushed
before it sends CMD_READ_VIEWPORT, so the read gets what the last frame
left, as the path without the wrapper does. The worker takes a command
ahead of a pending frame; gl2, which reads back through its PBO ring,
would then find the slot the previous read consumed and fail, and GPU
recording would lose that frame whenever the main thread ran ahead of
the worker - about one frame in a hundred uncapped. With the wait every
frame is recorded, each one core frame after the last.
init() started the unit as its last act and start() started it on the
spot, both with the ring empty. The device begins pulling immediately;
the frontend has not written yet, and on the threaded pipeline it will
not for a while, since the consumer's first pass deliberately waits for
the pipe's target and the device's buffer before it runs. Every pull in
that window is a full underrun.
samples/audio/pipeline_clocked measures it: 32 to 80 ms of unbroken
silence on every init, at every latency setting, with no dependence on
the setting at all. An init happens on every Audio Latency change, every
vsync toggle, every device change and every unpause - which is the burst
of noise heard on each of them, and it is not the same fault as the
steady-state one.
start() and stop() now set an intent, and coreaudio_run() starts the
unit once the ring holds a period - or at once when the writer has more
than fits, or when wait_writable() is about to sleep on room that only
the device can make. The harness goes from 16 short pulls and 32 ms of
silence at startup to none, at every setting.
The writer's stopped-unit bail-out had to learn the difference between
a unit that has stopped and one that has not been started yet: the
IsRunning check is only asked of a unit that was started, since a unit
about to be started reads exactly like an interrupted one and the writer
would have given up on the first write of every session.
Every pipeline harness here answers a contract question against a
scripted device: it takes what it is given, when it is asked. None of
them can show an underrun, because nothing in them runs to a clock -
which is why the CoreAudio buzz has been argued about from source for
a week rather than measured.
This one has a clock. The device drains its ring in real time, a period
at a time, and counts the pulls that found less than a period there.
The fixture is the reported configuration on a MacBook Pro: 48 kHz
stereo float out, a ring of the latency setting, a HAL period of a
quarter of it, a core publishing one video frame of int16 at a time,
Audio Sync and rate control on. The driver's write, wait_writable and
render callback follow coreaudio.c rather than a simplification - whole
frames, a semaphore the callback signals, a bounded wait. The shipping
producer and consumer run, by including audio/audio_driver.c.
It sweeps the latency setting and separates the startup transient from
steady state, because they turn out to be two different faults:
8 ms startup: 16 short, 32.0 ms silence | steady: 0.30%
16 ms startup: 8 short, 32.0 ms silence | steady: 0.20%
24 ms startup: 8 short, 48.0 ms silence | steady: 0.15%
32 ms startup: 10 short, 70.7 ms silence | steady: 0.00%
64 ms startup: 5 short, 80.0 ms silence | steady: 0.00%
The startup column is the same fault at every setting and is fixed in
the next commit. The steady column has a cliff between 24 and 32 ms,
on a machine with an idealised scheduler and a producer with no jitter
at all - which is the floor reported by ear on hardware, at 35 ms. That
one is not fixed here and should not be guessed at again; this is the
rig to find it on.
EAGER_START in the environment restores the pre-fix start behaviour, so
the two can be compared in one sitting.
Your log measures this rather than argues it. The driver negotiates
correctly at every setting - period always exactly what was asked for,
pulls always exactly what was declared, never one over - and at each
setting the shortfall is the whole pull against nothing found. The ring
is not short, it is empty. What changes with the Audio Latency setting
is only how long the ring lasts, and the threshold you found by ear,
35 ms at 60 Hz, is the arithmetic below.
The core delivers a frame's worth of audio in one publish and nothing
holds a reserve between publishes. The consumer moves what was published
at the device's pace and then waits for the next one, so whatever the
device still holds when the pipe empties is the whole margin. That is
the buffer less one chunk, since a pass will not begin until a chunk of
room has opened: half the setting before the previous commit, three
quarters of it now. Under a frame period of that, the device runs dry
between publishes every frame, and rate control cannot help - the audio
is not late, it has not been produced yet. Half of 35 ms is 17.5, and a
60 Hz frame is 16.7.
The inline path has no such requirement, which is exactly why it stays
clean where this does not, and the difference is not thread scheduling.
Its write blocks inside the driver during the frame that produced the
audio, so the publisher tops the device up continuously and there is no
interval to bridge at all.
So the pipeline is refused below that floor and the inline path is used,
in the manner already established for a driver with no wait_writable().
The log line gives the three numbers - what the buffer leaves, what a
frame costs, what setting would enable it - rather than only saying no.
This also corrects the claim in config.def.h that the threaded pipeline
costs no latency at any Audio Latency setting. It costs none, but it has
a floor under the setting, and below that floor the inline path is not a
fallback, it is the only one of the two that can work.
It has been compiled out since it was switched off, so this changes
nothing that runs - but a dead path that still looks like a path is what
let the live one go untested in the first place. coreaudio_write() had a
frame-alignment bug for as long as write_raw() existed, because with the
fast path on, most cores never reached it.
The converter, its input callback, the rebuild helper, the conv_buffer
and the four struct members that only it used all go. What stays is the
note: why the path was wrong (the rate adjustment fell inside its own
half-percent dead zone, which is the width of the frontend's entire rate
control range), what a second attempt needs before it starts (a
converter whose ratio changes in place, or a way to tell the frontend to
keep its own resampler), and the one plausible-sounding change it must
not make (kConverterPrimeMethod_None adds through latency rather than
saving it; Normal, the default, is the zero-latency one).
write_raw stays NULL in the descriptor.
Your log says what the fault is not. The device declared a largest pull
of 96 frames, made a largest pull of 96, and never once went over: the
variable-buffer-frame-size case does not arise on this hardware, and the
ring at 768 samples is eight periods rather than the four the floor
requires. Sizing is not the problem.
What it says instead is that at worst the callback wanted 192 samples
more than the 0 it found. 192 samples is the whole pull. The ring was
not short, it was empty - so this is a late refill, not a small buffer,
and the two need different fixes.
Where the lateness comes from is the fraction. A pass waits for room for
a whole chunk before it takes one, so the device's fill runs between the
buffer less a chunk and the buffer. The chunk is therefore not just the
size of the write, it is the margin the consumer thread has to wake and
finish in. At half the buffer and an 8 ms setting that margin is 4 ms,
required 250 times a second, with a semaphore wake and a resample inside
it. The inline path has no band at all - its write blocks in the driver
and holds the device near full - which is why it stays clean on hardware
where this does not.
A quarter puts the floor at three quarters of the buffer, 6 ms rather
than 4, and makes a pass one device period on any driver that sets its
IO period from the buffer, which CoreAudio does: the log's 96-frame
period is a quarter of its 384-frame ring. Twice as many passes, at the
rate the device's own callback already runs at.
The fraction had a second reader that did not know it. Rate control's
free space is corrected by "a quarter of the buffer so its half-to-full
band reads as no error" - a constant that is the right correction for a
chunk of half the buffer and for no other, written out in a different
function from the cap it depended on. It is computed from the chunk now,
so the two cannot drift apart again.
And the cap is applied rather than skipped when it comes out under 32
frames. Skipping it left have at a whole publish, which asks the driver
to wait for room for more than its whole ring - a wait that can only end
with the device empty, which is the state this commit exists to stop.
Three things were being treated as one number.
The period that was asked for. The period the device settled on. The
largest number_frames a render callback may actually be handed. The
ring's floor was four of the first two conflated, and the third was
never asked about at all.
The readback first. Where AudioUnitGetProperty() came back with nothing,
period_frames took the value that had been requested - the one number
that cannot be the answer, since the reason for reading back is that the
request is only advisory. The device object is asked when the unit will
not say, and if neither says, this stays zero and is treated as unknown.
The log line that already prints "got 0 frames" no longer sits beside
internal state claiming otherwise.
Then the pull. kAudioDevicePropertyBufferFrameSize is the device's
normal size; a device that carries
kAudioDevicePropertyUsesVariableBufferFrameSizes says with it the
largest buffer the IOProc may be passed, and the frame size is then only
the smallest. Nothing here queried it. A device advertising 256 nominal
and 1024 maximum got a ring floor of four times 256, rate control held
that around half full, and a legal 1024-frame pull then found under half
of what it asked for and padded the rest with silence - every pull, at
the pull rate, over a signal that is mostly gaps. Where the device says
nothing either way, the unit's MaximumFramesPerSlice is the ceiling it
has been prepared for; that is used only as the fallback, since it is a
capability rather than a pull and sizing against it unasked would throw
the latency setting away on every machine that has one.
The floor is now the higher of four periods and twice the largest pull.
Twice is the binding one: rate control holds the ring around half, so a
ring under twice the largest pull cannot fill a pull at the setpoint.
And it is measured rather than left to the properties. The callback
records the largest number_frames it was handed and counts any above
what the device declared - two atomic operations on the render thread,
nothing logged from there - and free() prints declared, observed and
over-declared together. That line is what says whether the reasoning
above matched the hardware, on a machine none of us has.
Two smaller things in the same file, both about the same fail-open
habit:
A buffer list that is not the single interleaved buffer the stream
format asked for returned noErr with the buffers untouched, which hands
the device whatever was in them. Every buffer is zeroed, the silence
flag set and a counter incremented instead.
worst_short and worst_short_avail were declared under !TARGET_OS_IPHONE
but read unguarded by the callback and by free(), so coreaudio.c has not
compiled for iOS or tvOS since they were added. They are not the HAL's
and do not belong beside period_frames; they move out of the block.
The contract is: do not come back until len bytes can be accepted. This
answered a different question. Anything over half the ring was silently
reduced to half, and the yes that came back was a yes to that, not to
what was asked.
The caller acts on the answer by taking the chunk out of its own queue
and writing it, so a yes that meant half left the write to finish the
job - blocking inside itself against the render callback, or, with the
non-blocking flag set, dropping the tail of a chunk the pipe had already
let go of. And len was over half as a matter of course rather than by
accident: the frontend capped a pass at half the buffer and then asked
for the resampler's bound on it, which is that plus a sixteen-frame
margin. The cap is the inverse of the bound now, so the request fits
without the clamp, and the clamp goes.
What replaces it is the one ceiling that is not a reinterpretation: the
ring empty. Above that there is no wait that could end in a yes. Below
it the wait is for the whole of what was asked, rounded up to a whole
device frame, since the writer only moves whole frames into the ring and
room for part of one is not room the write can use.
audio_driver_t grew frames_consumed_fallback(), the driver's own count
of frames the device took, kept beside the device clock so the sink rate
estimate has something to check itself against. The wrapper's descriptor
was never extended for it: the initialiser ends at audio_thread_layout,
so the field is NULL and the call goes nowhere.
The effect is only lost telemetry, but it is lost on precisely the
configuration where the clock is hardest to reason about - the threaded
one, where the frames the device consumed and the frames the frontend
offered are counted on different threads.
audio_driver_pipeline_consume() capped a pass at half the device buffer
divided by the resampling ratio, then asked the driver to make room for
audio_driver_output_bound() of that same chunk. Those are not the same
number. The bound is (input + 16) * ratio + 1: it carries a sixteen-frame
resampler margin and a rounding frame that the cap's plain division
leaves outside itself. So out_bytes came out a little over half the
buffer on every pass, not occasionally.
What that costs depends on the driver. One that honours the request
waits for more than half its buffer to drain, which is a wait the pass
did not intend and the caller's pacing did not budget for. One that
quietly tops the request out - CoreAudio clamped it to half - answers
yes to a question it did not answer, and the write that follows either
blocks inside itself or, non-blocking, drops the tail of the chunk. The
pass has already taken that chunk out of the pipe by then, so what is
dropped is gone.
audio_driver_input_bound() is the inverse: the largest input whose bound
still fits in the room asked for. It solves for it and then walks the
answer down until audio_driver_output_bound() agrees, so the two cannot
part company over a rounding step, and the cap now means what the wait
below is going to ask for.
The ratio itself is hoisted while doing this, since the cap and the
bound have to be computed from the same one. On the threaded path the
fast-forward multiplier is a value the producer published rather than
one derived from the frame count passed in, so taking it before the cap
rather than after is the same number.
The DSP filter set gains a bitcrusher, an overdrive, an early-reflection
reverb and a WSOLA pitch shifter with their presets, and every filter
processes float and int16 natively, so an all-float or all-int16 chain
never converts samples. The four are registered in the builtin table in
dsp_filter.c, in griffin and in the static audio filter objects of
Makefile.common alongside the existing eleven.
Every filter builds in the strict C89 lane whether or not the C99 math
names are declared, which compile-matrix.sh checks for each filter as
consoles build it; eq's float output grows with the input instead of
overrunning a fixed buffer; and retro_dsp_filter_free() releases the plug
table in builtin-only builds as well.
audio_driver_pipe_target_frames() returned zero whenever Audio Sync was
on. That number is not only how much the pipe should buffer; it is the
setpoint rate control measures the fill against. The producer computes
the controller's free space as the device's, plus a quarter of the
device buffer, plus this target, less what the pipe is holding - so a
target of zero means every byte in the pipe reads as overfill.
The pipe cannot hold nothing. The core publishes a frame's worth in one
go and the consumer moves it at the device's pace, so a publish sits in
the ring for most of every frame no matter what the writer does. Several
times the device buffer of it, against a setpoint of zero: the free
space clamped to zero and stayed there, and the controller corrected
downward at its full rate_control_delta for as long as it did. That
pulls the consumer ahead of the producer, the pipe drains, and once it
is empty the fill reads as no error and the loop turns back around.
At the drained end of that cycle the consumer is waiting on the next
publish with only the device's own buffer behind it. Where that buffer
is shorter than a video frame - CoreAudio at an 8 ms latency setting is
384 frames against a 16.7 ms frame - the device runs dry before the
publish arrives, and the dropout lands once per frame. That is the 60 Hz
artefact on macOS, and it is why turning the threaded pipeline off made
it go away: the inline path reads the device's own write_avail() and
never sees the pipe at all.
The one-publish floor that follows was already there and already had the
right comment on it - "a pipe holding less than that between publishes
is a device that runs dry between them whatever its own buffer holds" -
but the early return jumped over it. Audio Sync now takes zero as its
base and falls through to the floor, so both writers centre on one
publish; the non-blocking side keeps the extra buffer's worth on top.
No change to the drop-late-audio path, which is gated on Audio Sync
being off, and none to the inline pipeline.
video_thread_texture_handle() hands func() to the video thread while
the worker still takes commands - thr->worker_running, set before it
starts and cleared as it handles CMD_FREE - and runs it on the calling
thread only after that. thr->alive, the driver's answer for the window,
goes false on a window close or a quit signal while the worker still
runs and holds the GL context current; a texture unloaded in the
shutdown that follows (the menu's white texture, for one) would
otherwise make that context current on the main thread, which X refuses
with BadAccess on GLXMakeContextCurrent, and Xlib's default handler
exits.
This is the buzz.
coreaudio_write() took min(free space, what it was given) and wrote
that. The ring is counted in samples and its free space is whatever
the render callback happened to leave, so that count is arbitrary -
and a write ending half way through a frame leaves the ring's read
cursor a sample out from its write cursor, permanently. Every frame
after it straddles two of the source's, with left in right's place
for the rest of the session. On correlated stereo the two then partly
cancel, so it is heard as a buzz over something much quieter than it
should be, which is the report exactly.
The same on the way out: a short read on an underrun ended mid-frame
and moved the cursor by the same sample.
Both take whole frames now, which is what ra_asio_write() was fixed
to do a few days ago and what the JACK ring already does.
Why now: this was reachable for as long as the write path existed,
but the int16 fast path carried every int16 core past it, so only
float cores went through here. Switching that path off pointed all of
them at a function with this in it. My previous two attempts at this
report - the buffer-size clamp and then the ring floor - were both
looking at how much audio was available rather than at whether what
was there was aligned, and neither was the cause.
Related and not fixed here, because it wants a restructure rather
than a line: the ring's capacity is rounded up to a power of two and
the cursors wrap on that, so for any layout whose channel count is
not itself a power of two - 5.1 above all - the wrap lands
mid-frame and reintroduces exactly this skew. Stereo, quad and 7.1
are unaffected because their counts divide a power of two. Worth its
own change, with the cursors kept in frames.
I broke this. Asking the device for its buffer-frame-size range and
clamping to it replaced a hard 64-512 clamp, and the ring is sized
from the latency setting alone - which was sufficient only because
the period could never exceed 512. A device whose minimum is larger
than the quarter-ring target gets clamped up to that minimum, and
some aggregate and HDMI devices ask for thousands of frames. The ring
then holds one or two periods instead of four.
Rate control keeps that ring about half full, so every callback finds
less than a period in it, takes what is there and pads the rest with
silence. That is heard as a buzz at the period rate over a signal
that is mostly gaps, and as low volume because most of each period is
silence. Which is the report.
So the period the device settled on is kept, and the ring is never
smaller than four of them. Where that is larger than the latency
setting asked for it says so in the log, because it is the setting
not being honoured and the reason is the device.
The two other things that changed in this path are not this. The
render callback's frame count moved from the buffer's byte size to
number_frames * channels, and for the interleaved float format this
driver sets - mBytesPerFrame = channels * 4 - those are the same
number, so it is arithmetically unchanged. The silence flag is now
set only when there is nothing at all rather than on every partial
underrun, which can only reduce silence.
Still not built here. If this is not the whole of it, the line to
look at is the one the previous commit added: "[CoreAudio] IO buffer:
asked N, device takes MIN to MAX, got G frames" - G against the ring
the setting gives is exactly the ratio this is about.
From an audit of the driver. Five things, all in the same area: what
happens when something other than steady playback occurs.
Pause did not pause. stop() set a flag the process callback never
read, so the callback went on reading the ring every period - the
queued audio drained, and only then did silence start. alive()
reported paused while the pipeline was not. The callback reads the
flag now and writes silence without advancing the ring, which keeps
the client live and connected and makes resuming the next period
rather than a reconnection.
The server's rate and period changing were not handled at all. A
rate change is the serious one: 48000 against 44100 is nearly nine
percent where rate control moves half of one, so nothing downstream
can recover it and the frontend simply plays at the wrong rate. Both
callbacks are registered now, before activation as JACK requires, and
both ask the frontend to build this driver again rather than
attempting repair from the server's own thread.
alive() said a client whose server had died was alive, because it
only reported the pause flag; start() returned success on the same
client. Both account for the shutdown flag now.
That flag was a volatile bool written by the server's thread and read
by the frontend's, which orders nothing between threads; it and the
pause flag are atomics. The pause flag is initialised to what the
plain bool it replaced was zero-initialised to, so a path that does
not call start() is not left silent.
And two counters where there were none. The callback already knew
when it could not fill a period; that is the underruns hook, which
was NULL. The server missing its own deadline is a different event
with a different answer - a bigger ring here fixes the first and does
nothing for the second - so the XRUN callback counts those apart, and
they are logged at teardown rather than reported as this driver's.
The ring is also locked into memory, which is what
jack_ringbuffer_mlock() is for: the callback reads it on a realtime
thread, where a page fault is a missed period. A system that refuses
says so and audio carries on.
Compiles clean here with -Wall -Wextra, -Werror on mismatched
pointers and unused functions, and under gnu89. Two dead things went
with it: an unused variable in free() and a declaration after a
statement in the callback, which a C89 build would have refused.
The Android build broke on audio/audio_thread_wrapper.c, which fills
in an audio_driver_t and does not live under audio/drivers - so the
job written to catch exactly that class of break did not compile it.
A list of directories is a guess about where a structure gets filled
in, and this is the guess being wrong.
It looks for them now: every .c outside samples that assigns an
audio, camera, record or microphone driver structure, found by
grepping for the assignment. Thirty-six files where the directory
list found thirty-three, and the three it adds include the wrapper
that broke.
The list it prints is part of the output, so a file that stops being
found is visible rather than silently dropped.
Verified by putting the offending entry back in the middle of
audio_driver_t and watching the job fail - on the wrapper among
others, which is what the old version could not do - then taking it
out again: 33 compiled, 3 skipped for SDKs they need, none
mismatched.
The break itself was fixed 22 commits ago in 561172e4f7; the
wrapper compiles clean on master today. This is so the next one does
not need a failed Android build to find it.
The d3d9 Cg and HLSL drivers assign their shader constant vectors, the
rotated-quad corners and the texcoord vertex element after the
declarations instead of initialising them from runtime values, which C89
does not allow; every value and the order it is set in are unchanged.
The bundled d3dcompiler.h stands in for the Windows SDK header and is
marked a system header for GCC, so its C++ comments are not held to the
project's pedantic C89 checks.
tools/mingw_syntax_check.sh's C89 pass defines HAVE_SCREENSHOTS, as the
linux-c89 build does, and gives Windows-only translation units the Win32
feature set of the first pass, the only one they are built with: without
it the pass found wnd_proc_d3d_common undeclared in every d3d driver and
the screenshot settings missing in task_screenshot.c. Over the whole
tree eleven translation units go from failing to clean and none from
clean to failing.
Asked to look into setting the converter's prime method to
kConverterPrimeMethod_None for the int16 fast path, on the grounds
that the default priming is a hidden latency source and that Apple
recommends None for live real-time work. The first half is backwards
and the second is half of a sentence.
Apple's own header gives all three and what each costs:
Pre primes with leading and trailing input frames
Normal primes with trailing only, leading assumed silence;
"requires no pre-seeking of the input stream and
generates no latency at the output"
None "acts in 'latency' mode": both assumed silence, and
trailingFrames of through latency appear at the start of
the converter's output
So Normal - the default, and what this code already left in place -
is the zero-latency one. None is named latency mode in the header.
It exists for a source that cannot be read ahead of: live input with
nothing before its first frame, where the read-ahead Normal wants
cannot be supplied. It buys that by accepting the latency Normal
avoids. This path feeds from a queue of the core's audio, where
reading ahead is precisely what the input proc can do, so Normal is
right here for the reason it is the default.
Setting it would have cost latency in a driver being worked on to
save latency, and it would have looked like an optimisation in the
diff.
What is worth doing instead is asking rather than guessing:
kAudioConverterPrimeInfo reports leadingFrames and trailingFrames
for the converter as configured, so the read-ahead can be measured.
That is written into the note above the disabled fast path, with the
rest of what the next attempt needs to know.
Comment only; no code changed, and the code it describes is disabled.
Choosing the output device compared the setting against the device's
name and nothing else. A name is what a user reads, and it is also
what two devices can share - plug in a second "USB Audio Device" and
which one a saved setting means is whichever the HAL enumerates
first - and it changes when the device is renamed. The UID does
neither.
So the setting is matched against the UID first, then the CFString
name, then the older name property that was being used. A setting
saved before this carries a name, which is why the names are still
tried; anything carrying a UID now resolves to the right device
whatever it is called. The microphone half already looks its device
up this way, so this is that design on the output side.
It also says which way it matched, and warns when nothing did rather
than falling back to the default in silence - which is what happened
before, and is hard to tell from the device simply not working.
The CFString reader is its own here rather than the microphone's:
that one sits behind HAVE_MICROPHONE and this must work in a build
without it.
What this does not do is make the menu store UIDs. The audio device
dropdown saves the string it displays, so storing a UID there would
show one to the user; the microphone list carries its UID beside the
name in the element's attribute instead, and the equivalent for
output is a frontend change affecting every audio driver, not a
driver one. Until then this reads a UID where a config has one, which
is the half that belongs here.
Not built here; macOS only. Checked: live text balances, names
nothing from after 10.4, no static that nothing calls.
The microphone half of this file listens for the default input device
changing and reconnects; the output half had no equivalent, so a
default that moved while content ran left the frontend playing to
hardware the user had just stopped using, until something else
happened to reinitialise audio.
It listens now, with two differences from the microphone's design,
both deliberate.
It only listens when it is following the default at all - a device
the user named is not disturbed by the default moving, and the empty
device string is what says which case this is.
And it does not rebind in place. The microphone can, because its
reconnect runs on the reader's thread and keeps the format it opened
with. An output cannot: the unit cannot be rebound from a HAL thread
with the render callback live, and rebinding at all means
renegotiating the rate, the period, the layout and the latency
against different hardware. So it does what the WASAPI device-change
path does - asks the frontend to reinitialise audio, on its own
thread, in its own time - and this driver is built again against
whatever the default has become.
The listener comes off in free(), first thing, since the HAL calls it
on a thread of its own and hands it this pointer.
It goes through the listener shim, so it registers on 10.4 through
AudioHardwareAddPropertyListener() and on anything newer through
AudioObjectAddPropertyListener(), with the two callback shapes that
needs.
Not built here; macOS only. Checked: the live text balances, names
nothing from after 10.4, and has no static that nothing calls - the
last of which is the check that caught the missing call site in the
WASAPI pump two days ago.
The HAL's buffer frame size was held between 64 and 512, which is
wrong in both directions. A device that will run at 32 frames was
kept at 64 and lost half the latency it was offering; a device whose
minimum is 128 was asked for 64, refused, and left at whatever it
had - the property is advisory and the return value was not read.
kAudioDevicePropertyBufferFrameSizeRange is what the device says it
takes, so it is asked and the quarter-of-the-ring target clamped to
that. What was actually accepted is then read back and logged beside
what was wanted and what the range was, so a device that ignores the
request says so in the log rather than silently:
[CoreAudio] IO buffer: asked 96, device takes 32 to 4096,
got 96 frames (2.00 ms).
A device that names no range keeps the old 64-512 bounds, since
without a range there is nothing better to go on.
The query goes through the property shim, so it reads the range on
10.4 through AudioDeviceGetProperty() and on anything newer through
AudioObjectGetPropertyData(), the same as every other read here. The
selector itself is 10.0-era, and AudioValueRange older still.
Not built here; macOS only. Checked: the live text still names
nothing from after 10.4 and balances, and the driver sweep passes for
what does build here.
Resolving the calls at runtime was only half of it. The constants
that came with the AudioObject API - kAudioObjectUnknown,
kAudioObjectPropertyScopeGlobal, kAudioObjectSystemObject - are not
declared by a 10.4 SDK either, and they are enumerators rather than
macros, so there is nothing to test with #ifndef. The only way to
name them on every SDK is to name them here, which costs a define
apiece: their values are fixed by the ABI the HAL speaks, and that is
what both sets of calls see.
kAudioStreamPropertyLatency is the same case with a cheaper answer.
Its selector is the four characters 'ltnc', which is also
kAudioDevicePropertyLatency - a spelling every SDK back to 10.0
declares - and the HAL reads the number rather than the name, so the
stream's latency is asked for through that one and a name a 10.4 SDK
may lack goes away.
The rest of the selectors this file names were checked against their
own introductions and are 10.2 or earlier: device streams, device
UID, the CFString name, nominal rate, buffer frame size, safety
offset, the scopes, and the error code.
With this the macOS half names nothing from after 10.4 in code that
compiles - the newer calls exist only as strings handed to dlsym.
Verified by sweeping the live text, the disabled fast path stripped
out, for any kAudioObject* name at all: none.
video_driver_update_viewport() runs on the threaded video worker under
the wrapper, while the main thread frees and replaces overlays - on menu
open and close, on load and on OVERLAY_NEXT. It reads the active
overlay's viewport override from video_driver_state_t instead of
through input_st->overlay_ptr: video_driver_set_overlay_viewport()
publishes the flags and the rectangle there each time the active
overlay changes. overlay_vp_flags, an atomic, alone says whether there
is an override, so without one the check is a single load and no lock;
with one the rectangle is copied under display_lock.
The override is logged when it is published, once per overlay shown, so
the video side writes nothing into the overlay and
viewport_override_logged goes.
The HAL property calls this file used - AudioObjectGetPropertyData()
and its neighbours - arrived in 10.5. Before that there was a call
per kind of object: AudioHardwareGetProperty() for the system,
AudioDeviceGetProperty() for a device, AudioStreamGetProperty() for a
stream, and AudioHardwareAddPropertyListener() for a listener. Those
are all still present, deprecated, in every SDK since.
So both sets are looked up by name when the process starts, beside
the Component Manager fallback that was already doing exactly this
for the AudioComponent calls, and the newer set is used where it is
found. One binary: a build on a current SDK still runs on 10.4, and a
build on a 10.4 SDK - where AudioObjectGetPropertyData is not even
declared - still uses the newer calls on a system that has them.
Nothing is decided at build time, which is the point.
Three things the mapping had to get right. The old calls take an
element and an is-input flag where the new ones take an address, so
the scope becomes that flag - every read here is on an output or a
global scope except the microphone half, which asks for input. The
old HAL has no has-property question, so asking for the size is the
question. And the listener comes in two shapes, the newer handed the
object and what changed on it and the older only the selector, so
there are two callbacks doing the same thing.
The address type and the system object's id are declared here rather
than taken from the SDK, since a 10.4 SDK declares neither, and the
symbols are looked up as strings for the same reason.
Not built here; macOS only. What was checked: no 10.5-only symbol is
called directly anywhere in the live text - every one now goes
through the shim or is a string handed to dlsym - and with the
disabled fast path stripped out the remainder still balances.
Requested by cscd98 on #19389. Holding it back until the PR landed was
deliberate: with the OSK gamepad block fully behind the native-keyboard
guard, defaulting this on would have shipped webOS users straight into
a dialog they could not dismiss whenever the panel failed to emit a
Return.
That escape is in now, so the default can flip. webOS is the case the
opt-in exists for - its own panel is what the remote is built to drive,
and the built-in OSK is the awkward option there.
Asked whether the recent work here stays good back to 10.4, so it was
checked rather than assumed.
The clock call added for frames_consumed() is fine:
AudioDeviceGetCurrentTime() is one of the original HAL calls, marked
available since 10.0 in the 10.2.8 SDK's own header, and it is not
among the ones deprecated in 10.5 or 10.6 - those are the property
accessors and the IOProc registration, a different family. It adds
nothing to what this file already requires, and the comment beside
it now says so.
What it already requires is the part worth writing down. The
AudioComponent calls are resolved at runtime with a Component
Manager fallback behind them, which does reach back before 10.6 - but
the property reads are AudioObjectGetPropertyData() and friends,
which Apple's TN2223 dates to 10.5, and there are eleven of them
across device enumeration, the hardware rate, the latency sum and the
microphone half. So the macOS path needs a 10.5 SDK as it stands.
That is not something the recent work changed in either direction,
and reaching 10.4 would mean a second path through
AudioDeviceGetProperty() for all eleven - a real piece of work rather
than a guard, and not one to start without saying so. The floor is
stated at the top of the file so the next person asking gets the
answer from the source rather than from a search.
Comment only; no code changed.
The call is specified to return frames the device has consumed on its
own clock. This driver summed the frames each render callback asked
for - which is close, a render callback being device-paced, and much
better than counting writes - but it is still this driver's
inference rather than the device's answer, and the HAL will give the
answer outright.
So on macOS it asks: the device the unit is bound to is kept from the
latency block that already reads it, and AudioDeviceGetCurrentTime()
gives that device's sample position. The call fails while the device
is not running, which is every call before the first render, so the
counter answers until it succeeds and the position is then taken
relative to where the clock stood at that first success. iOS keeps
the counter, the HAL device APIs not being there.
And the counter becomes frames_consumed_fallback(), which the
frontend compares against the clock and reports in parts per million
- the same treatment WASAPI got. The two should agree closely here,
unlike WASAPI's service events, so a difference is worth seeing
rather than assumed absent. Nothing is decided from it.
Not built here; macOS only. What was checked: the vtable's new entry
is last, where the interface puts it and where it cannot move
anything; with every gated block removed, the live text names
nothing from the disabled fast path and balances; and the driver
sweep still passes for the drivers that do build here.
The driver is handed a rate adjustment on every write and is
responsible for applying it. This path rebuilt its converter only
when that adjustment moved by more than half a percent - and the
frontend's own rate control range is half a percent, so in ordinary
use every correction it makes falls inside the dead zone and none is
applied at all. The one that eventually does not arrives as a step,
with the converter destroyed, remade, and its filter history lost.
So a path that exists to save the frontend's resampler has been
quietly defeating the frontend's rate control instead, which is a
poor trade at any CPU price. It is off: the vtable offers no
write_raw and the frontend resamples, which costs more and is
correct.
Switched off rather than deleted, because what it should do is worth
having and what is wrong with it is specific. Lowering the threshold
is not the fix - that trades the dead zone for a converter rebuilt
whenever the buffer moves, which is worse. What it needs is a
converter whose ratio changes in place and keeps its state, or a way
for a driver to say it cannot follow a rate adjustment so the
frontend keeps its own resampler; and then its output compared
against the sinc resampler's sample for sample rather than assumed
equivalent. The note above the gated code says so, so the next
attempt starts from there.
The converter reset in start() goes with it - it was there for a
converter this path left stuck at end-of-stream - and the comments
calling AudioConverter "hardware-accelerated" now say system
sample-rate conversion, which is what it is.
Not built here; macOS only. What was checked is that with every
gated block removed, nothing that remains names the converter, the
buffer, the threshold or either function, and that the remaining
text balances. That check found the first attempt closing a gate in
the middle of a function, on an early return - the whole-file brace
count did not, because text inside #if 0 still balances.
Measured on the reporter's machine: handing the device thread to the
multimedia class scheduler made the worst frame time worse, not
better. That is the number that matters at a period of a few
milliseconds - a mean that improves while the tail gets longer is not
an improvement - and it is the opposite of what I expected when the
change went in, which is why the lateness telemetry went in beside
it.
So it is a setting rather than the behaviour, and off: the driver
raises the thread's priority directly as it always did unless asked
otherwise. "Prefer Pro Audio Scheduling", advanced, and shown only
where Elevate Audio Thread Priority is on, since it says which way
that request is made rather than whether one is made. Windows only,
like the rest of the WASAPI group. It also declines where the
priority setting is off, so the two cannot disagree.
The environment override it replaces is gone; a setting says the
same thing and survives a restart. The summary at teardown still
names which scheduling ran and how late it woke, which is what makes
the two comparable on a given machine - and what turned my
expectation into a measurement that contradicted it.
Two things in the render callback, from an audit of the driver.
The silence flag was set on every underrun, including the ones where
most of the period was real audio. kAudioUnitRenderAction_OutputIsSilence
says the buffer holds nothing but silence, and a unit is entitled to
skip a buffer that says so - so the samples read into it a line
earlier were liable to be thrown away, and a period that should have
been mostly audio became one of silence entirely. A ring that ran a
little short cost a whole period rather than the part it was short
by. The flag is set only where there was nothing at all, and ored in
rather than assigned: the flags are the unit's and it may have set
some of its own on the way in.
And the frame count comes from number_frames, which is what the unit
asked for, rather than from the byte capacity of the buffer it
handed over. Those agree for the interleaved format this driver sets
up, but they are different questions - a buffer is allowed to be
larger than the request - and the byte size is now the sanity check
it should always have been rather than the source.
Not built here; this driver is macOS and this machine is not. What
was checked is the flag's contract against Apple's definition of it,
that number_frames is frames of every channel where frames_needed is
samples, that the byte capacity still bounds the write, and that the
file balances.
wasapi_pump_mmcss_end() was written and never called. The pump took
the Pro Audio class and kept it: the characteristic was never
reverted and avrt kept a reference for every audio init the session
made, which on a driver that reinitialises per content load is every
one of them. The call goes where the thread ends, which is where it
was meant to be - it was in the first draft and did not survive the
change being taken out of the tree and put back.
The compiler said so on the first build, in the only way it can: the
helper that returns the class was defined but not used. I did not
read it, for the same reason I did not read the multistatement-macro
warning yesterday - my build checks grep for ": error:" and throw
away the rest, which discards exactly the diagnostics that catch a
missing call site.
So the driver-vtable job takes -Werror=unused-function as well. A
static nothing calls is the same family as a shifted vtable entry:
both are the compiler describing a structure that does not hold
together, and both are only errors if asked. Clean across all 35
drivers it compiles today, and it fails on this break - verified by
taking the call back out and watching it.
The comment said "no build target gains an avrt import for a library
that may not be there", which is vague enough to read as a build
convenience. It is not one.
avrt.dll arrived with Vista and this binary still runs on Windows
versions that predate it. A static import is resolved before any code
runs, so one would stop the whole program starting on those -
including for a user who never chooses WASAPI and ends up on
DirectSound. Loading it when the pump starts makes a system without
it a "no" at that point rather than a program that will not load.
And while the question was asked: there is nothing to be saved by
reaching past the library either. The function reads the task's
profile out of the registry and hands the thread to the MMCSS kernel
driver over a private interface, so there is no arithmetic to inline
- only an undocumented handshake a Windows update may change without
saying so. It runs once, on the pump thread, before its loop.
Comment only; no code changed.
Two halves of one question, which is why they are together: whether
the multimedia class scheduler gets the pump to its period more
reliably than a time-critical priority does. Shipping the change
without the measurement would leave that unanswerable, which is what
the audit this comes from asked not to do.
The scheduler first. AvSetMmThreadCharacteristicsW(L"Pro Audio") is
what Windows offers a thread that must run inside a device period,
and what Microsoft's own low-latency event-driven WASAPI sample uses.
It is loaded by name, so no build target gains an avrt import for a
library that may not be there, and a system without the class falls
back to the priority this thread has always run at - so the worst
case is what it did before. The two are not combined: the class
carries its own priority.
Then the measurement. Every wake is compared against the period it
was due at, and the pump keeps four numbers - wakes, mean lateness,
worst, and how many arrived a whole period late or more. Atomic adds
on a path that already takes a lock; nothing is logged from the pump
itself, since a line inside a three-millisecond period costs a
fraction of its margin. The summary goes out at teardown, with the
scheduling it ran under on the same line:
[WASAPI] Pump (Pro Audio): 12043 wakes, 0.02 ms late on average,
0.31 ms at worst, 0 a period or more late.
RETROARCH_WASAPI_NO_MMCSS in the environment forces the fallback, so
one binary produces both rows and they can be put side by side. That
is the whole point; on a 3 ms exclusive period the difference, if
there is one, is in the tail rather than the mean.
No interface changed. The numbers are the driver's own and are
logged where it tears down, rather than through a new vtable entry -
the last optional entry I added moved three drivers' function
pointers and broke the build.
Checked: the pacing lane builds and passes plain and under ASan, and
prints the summary from its own runs (374 wakes, 0.01 ms mean,
0.25 ms worst on this machine's scripted device); every driver still
compiles against the interface; and the generated driver text has
the same five pre-existing C89 complaints from the harness's stub
settings as it did before, and no new ones.
audio/drivers/alsa.c carries both drivers, each behind its own
define, and the samples job already built it once through configure
with alsa-lib on - because a distribution takes HAVE_ALSA from
config.h rather than from the command line, and the file gates its
includes on it.
configure puts HAVE_TINYALSA in config.h too, so the embedded driver
has the same question to answer and nothing was asking it. The two
other arrangements a distribution can ask for now build there as
well: the embedded driver with no alsa-lib anywhere, where config.h
carries no HAVE_ALSA at all, and both drivers in the one object.
All three verified here before committing - configured, built, and
the config.mk checked to say what was actually asked for rather than
what was intended.
VIDEO_DRIVER_OUTPUT_SIZE(video_st) is one load of output_size_packed,
and VIDEO_DRIVER_OUTPUT_WIDTH()/HEIGHT() take the halves of that value,
so a caller holding video_st reads a matching pair without a call and
without a pair of locals whose addresses it hands out. The callers the
packing converted use them, one load per pair, and
video_driver_get_output_size() is built on them for the drivers and
menus that keep calling it.
A driver fills in a vtable of function pointers, and almost every one
of them does it positionally and names nothing. So an entry added to
the middle of one of those structures silently moves every entry
after it in every driver that did not name its fields, and what the
compiler then says is that a layout function is being installed as an
underrun counter - which is a warning, and an error only if you ask
for it to be.
That happened yesterday: frames_consumed_fallback went in between
frames_consumed and underruns, three Windows drivers took the shift,
and it reached master because what I ran before committing was a
syntax check of one file rather than of all of them. The check that
would have caught it takes a few seconds and I had it available.
So it runs in CI now, over audio, camera and record drivers, with
-Werror=incompatible-pointer-types - the diagnostic that named the
three. A driver whose platform SDK is not on the runner does not
compile at all and is skipped, since that is not what this is about;
only a mismatched function pointer fails the job, and the message
says where an optional entry belongs.
Checked by putting the entry back in the middle and watching the job
fail on five drivers, then taking it out again: 35 compiled, 3
skipped for their SDKs, none mismatched.
I put it between frames_consumed() and underruns(). Almost every
audio driver initialises audio_driver_t positionally and names
nothing, so an entry added anywhere but the end moves every entry
after it: asio, dsound and xaudio ended up with their layout
function installed as their underrun counter, and the build stopped
on all three.
It goes last now, and the comment says why, so the next optional
entry does not repeat it. wasapi's initialiser follows.
Checked this time rather than assumed: every file under
audio/drivers built with -Werror=incompatible-pointer-types, which
is the diagnostic that named the three, and the tails of the ones
that cannot be built here read by hand - none of them initialises
past layout, so the new slot is beyond all of them.
The MMCSS work that was in the tree beside this is taken back out.
It is a separate concern, I had not validated it past a compile of
its fallback path, and it had no business riding along with a fix
for a build I broke.
frames_consumed() now reads a device clock where there is one, and
the number it used to return - service events counted and multiplied
by a period - is still there. What nobody can say is what the
approximation costs on real hardware: whether a device's events
really do arrive one per period, and where they do not, by how much.
So a driver may offer both. frames_consumed_fallback() is the count
it would have produced without a clock, on the same terms, and the
frontend compares the two over the window it already measures and
reports the difference in parts per million. Nothing is decided from
it - the estimate the bias comes from is frames_consumed() alone -
and the line only appears where a driver has a second number to
disagree with, which today is WASAPI and nothing else.
Sink/Src: -12/-3 ppm (bias +9)
Clock vs events: +41 ppm
That is the measurement the audit asked for before anyone retunes
the estimator's four-second window and thirty-second baseline, and
it is worth having now rather than later: the reporter's endpoint
offers IAudioClock2, so the comparison has something to say on their
machine from the first window.
Also loosened a bound in the AC-3 bitstream case, which demanded no
unanswered period at all: it asked for one percent over ninety-odd
periods, and one percent of ninety is zero. That is a statement
about the machine, not the driver - the pump here is an ordinary
thread a loaded box can hold past a period, where on Windows it runs
time-critical. One percent and one period besides, which is the
allowance the pacing scenarios already make.
The clock work went in with its own coverage named as a gap and left
open, and three defects shipped through that gap in as many commits.
This closes it with the case the whole change exists for.
The fake endpoint can now withhold its event signal while its engine
keeps playing - which is what a pump held past its period looks like
from the driver's side: the frames went out, the events that would
have counted them did not arrive. The case runs a stream normally,
withholds the signal for a stretch, and compares what
frames_consumed() reports against what the engine actually played.
Reading the clock: the device played 384 frames and the driver
reports 384. Counting events, which is what the fallback does and
what the driver did before: the device played 384 and the driver
reports none.
The tolerance had to be tightened to mean anything. It started at a
few thousand frames, which the old behaviour passes - reporting
nothing at all is within a few thousand of anything over a short
window. An eighth of what the device played does not: it admits the
period between the engine's last step and the clock being read, and
nothing like a window's worth of missed counting. Verified both
ways, by taking the clock out of the read and watching the case fail
with the message it was written to print.
RELEASE(x) expanded to two statements - the test-and-release, then
the assignment - with the assignment outside whatever guarded the
macro. Written under an unbraced if, the interface was released
conditionally and the pointer cleared unconditionally.
That is what the device-clock code did:
if (!w->clock_frequency)
RELEASE(w->clock);
...
else
RELEASE(w->clock2);
so both pointers were cleared on every path and frames_consumed()
could never reach either clock. Not a crash - a feature silently
switched off, which is the worse of the two, and the only thing that
said so was -Wmultistatement-macros in the build log.
The macro is a do/while now, in both its C and C++ branches, and the
same two-statement shape in Cocoa's RELEASE is closed with it - the
one use there is braced and was safe, but the trap is identical and
the next one need not be.
The pacing harness runs with the fake endpoint offering both clocks,
so this is a path the lane actually takes: the driver reaches the
clock it acquired instead of falling through to the event count. The
build is clean under -Wmultistatement-macros.
Worth saying plainly: I wrote that if and that else, and the compiler
had already printed the warning that named them.
The clock work crashed at startup: a null dereference inside
wasapi_frames_consumed(), with the interface pointer as the argument
to a virtual call.
The cause was a pair of bools, clock_valid and clock2_valid, kept
beside the interfaces they described. RELEASE() nulls the pointer and
leaves the bool saying the clock is there, so any path that released
the client and did not start again - and there are three, the AC-3
encoder's failure path, the close, and the driver's own teardown -
left frames_consumed() calling GetPosition on nothing. The menu's
sink update calls it every frame, so it did not take long.
A flag that describes a pointer can outlive it; the pointer cannot
outlive itself. The bools are gone and the pointer is the only thing
consulted - there is a clock if and only if there is an interface -
and the clocks are released everywhere the client is, which they were
not before.
The pacing harness now runs with the fake endpoint offering both
clocks, so what the lane exercises is the clock path rather than the
fallback. That was the gap named in the commit that introduced this,
and it is the gap that let it ship: with the clocks off, the driver
never entered the code that crashed.
The audio driver interface says frames_consumed() returns frames the
device has consumed on its own clock, and the frontend's sink-rate
estimator takes it at its word. WASAPI returned service events
counted and multiplied by a period, which is the same number only
while every event is seen and every event is one period. A pump
descheduled past two periods wakes once and counts one; the
measurement window it lands in reads slow by the difference, and the
estimator either rejects the window or believes it.
WASAPI has the interfaces for the question that was being
approximated. IAudioClock2 reports the hardware's position in
frames, and is shared-mode only. IAudioClock reports the stream's
position in units of its own frequency - which the documentation is
explicit are not to be assumed to be frames - so it is converted
through that frequency and the stream's rate. Both are acquired
where the endpoint offers them, and neither is required of a device,
so the event count stays as the fallback. Which one is in use is
logged at init.
The IIDs go beside IAudioClient3's in mmdevice_common_inline.h, for
the same reason that one is there: mingw's libuuid does not export
them.
The fake endpoint in the pacing harness gains both interfaces, with
its position taken from the frames its scripted engine has actually
played - independent of whether the driver saw the events that
played them, which is the discrepancy this is about. Its IAudioClock
deliberately reports a frequency that is not the sample rate, so a
driver that assumed the position was already in frames would read
wrong.
What is not here: the harness still runs with the clock off, so the
new path builds and the fallback is what the lane exercises. A case
that drops service events and watches the event count fall behind a
clock that does not is the next step, and the fake is now able to
stage it.
Scenario 8 of that harness fails intermittently on this machine both
with this change and without it, which is worth someone's attention
but is not this.
video_driver_state_t's width and height become output_size_packed,
width in the high 16 bits and height in the low 16, stored whole by
video_driver_set_output_size() and loaded whole by
video_driver_get_output_size(), so a reader always gets a matching pair
without display_lock. Under the threaded video wrapper that lock was
taken on every call: the drivers set the size on the video thread each
frame while the main thread, the menu and tasks read it. Every direct
reader of the two fields goes through the getter, one load per pair -
including the screenshot task, which can run on a task thread - and
video_driver_build_info() keeps display_lock under the wrapper only
around its one read of 'flags', whose driver bits the video thread
still changes under it.
Each dimension is clamped to 65535.
The binaural stage renders a 5.1 to two ears, and that 5.1 was always
the upmix's guess at one: the stereo mix widened, its rears a copy of
its fronts at -3 dB. A core that delivers its own rears through the
multi-channel batch entry had them folded into that stereo and then
invented again from it - so a sound that was only ever behind the
listener arrived in front, was placed in front, and a copy of it was
placed behind.
The entry may take the discrete path into the virtual 5.1 now, as it
does into a device that has the speakers: the front pair goes through
the pipeline and the core's other channels beside it, and where they
arrive they replace the invented ones before the binaural stage sees
the frame. What reaches the ears is then the core's rear placed
behind the listener rather than its front placed behind them.
The merge takes the layout it is merging into, since the virtual 5.1
is not the device's stereo; the device-frame caller keeps its old
shape.
The harness measures the virtual 5.1 rather than the ears, which is
the second thing it tried. A rear-only sound reaches the ears under
both behaviours - the fold puts it into the stereo - so its presence
there proves nothing, and the first version of this case passed
against the old code. What differs is where it sits in the frame the
binaural stage is handed: carried, the virtual fronts stay silent
because the core's fronts were; invented, they hold the folded rear
at 0.0800 with a copy behind at 0.0400. The case fails against the
old gate and passes against the new one.
Some drivers - ASIO4ALL among them - still have a buffer-switch
callback in flight when ASIOStop returns, and the buffers it is
writing into are about to be disposed. The teardown handled that with
Sleep(20), which is a guess in both directions: too long where the
callback had already left, and on a driver or a machine where it had
not, not a wait at all but a gap before the same use-after-free.
A count of callbacks that have entered and not left answers the
question instead. It lives outside the instance, because its whole
purpose is to be readable after the instance is gone, and it is
raised before g_asio is read rather than after: a callback that
observes a live pointer has already counted itself, so a teardown
that clears the pointer and then waits for the count to reach zero
cannot be running while one is inside. Raised after the read, a
callback preempted between the two would be invisible to that wait -
which is the hole a naive counter leaves.
The wait yields before it sleeps. A callback in flight is part-way
through one period and finishes in well under one, so yielding to it
is all that is usually needed - and Sleep(1) is not a millisecond
unless something has raised the timer resolution, it is the
scheduler's tick, about fifteen. Sleeping first would have made the
common case cost more than the Sleep(20) it replaces rather than
less. The deadline is measured in time rather than counted in
iterations, so it means 200 ms whatever either of those costs, and
it says so if it expires: a driver that never returns from its
callback must not hang the process on exit.
What this costs at run time is two atomic increments per buffer
switch - one cache line, written by the callback thread alone,
against a callback that is already deinterleaving and converting a
period's worth of samples. Teardown gets faster in the ordinary case,
where the old code slept 20 ms for nothing.
The reclaim path waits too. Parking frees nothing, so a stray
callback from before it was harmless; reclaiming clears and resizes
the ring under it, which is not. That path argued its safety from
ASIOStop and a cleared pointer; it checks now.
Not built here - this file is Windows-only and the machine is not.
What was checked is that every exit from the callback decrements,
that the increment precedes the read it guards, and that the file
still balances.
Three correctness items, all of them things the move from a fixed
stereo pair to an arbitrary speaker layout left behind.
The write cut its length back to eight bytes - a stereo float frame
- whatever the layout. The callback reads the ring a frame at a
time, so with 5.1's 24 bytes or 7.1's 32 a write could end part-way
through one and the callback would take the remainder as the head of
the next: every channel rotated by a sample, for as long as the
stream ran. Quad and 7.1 divide by eight and were right by accident;
5.1 is the one that breaks. The rule moves into asio_ring.h beside
the other arithmetic the driver cannot test on the machine it is
written on, and asio_ring_test checks it for every layout and every
length up to a period.
Every opened output's sample type is asked for now, not just the
first. ASIO describes the type per channel and this driver converts
with one for all of them, which held while it opened two adjacent
outputs; opening a run of six or eight, a device whose outputs
differ would have had the wrong format written to all but the first.
Mismatches are refused with which output differs and how, rather
than played.
And sampleRateDidChange() reconfigures rather than logging. The
device's rate is what the ring's frames-to-time conversion, the
reported latency, the resampler's output rate and the sink estimator
were all sized and timed from; a change to it invalidated every one
of them and the pipeline carried on at a rate the hardware had
stopped using. It now does what a reset request does - asks the
frontend to reinitialise audio on the main thread, in its own time.
From an external audit of the backend. The rest of what it raised -
moving conversion off the callback, zero-copy spans out of the ring,
ASIO's own sample-position data for clock estimation, the teardown's
fixed 20 ms wait, the host window - is worth doing and is not this:
each needs measurement on real hardware, which none of these three
did.
The camera driver had no test, because testing it meant owning a
webcam. It does not: the driver reads from a libavdevice input
format, and built with that format set to libavfilter a filter graph
is the device. The whole pipeline then runs with nothing attached -
avformat opens the source, the poll thread reads packets, the
decoder decodes them, swscale converts into the target buffer and
poll() takes the frame out under the buffer's lock. Two seams make
that possible, and both are worth having on their own account:
- FFMPEG_CAMERA_DEFAULT_BACKEND respects a definition the build has
already made, rather than choosing by platform regardless.
- The device the frontend names is used. It was taken as an argument
and never read: a user who picked a camera got whichever one
enumerated first. A named device is now the source url and
enumeration is skipped.
ffmpeg_live_test drives what the threading has to get right and what
no eyeball on a webcam would show: open, run and close; stop and
start three times over; stop while the poll thread is mid-frame,
five times with the stop at a different point each time - which is
the ordering fixed a moment ago, where the flush used to run against
a live decoding thread; free without stopping first, as a host
dropping content does; poll before starting; and a source that does
not exist. Every frame is read pixel by pixel, so a buffer that was
freed or is the wrong size shows under ASan rather than as a
wrong-looking picture.
And one in libretro-common, which UBSan found on the first run:
memalign_alloc keeps the pointer to free in the word before the
address it returns, so that address must be aligned for a pointer
whatever the caller asked for. The camera asks for 4, which on a
64-bit host put the header on a four-byte boundary - a misaligned
store of a pointer, undefined everywhere and a fault on a target
that does not allow it. A boundary is a minimum, so it is raised to
a pointer's.
Plain, ASan/UBSan and TSan all clean; in the cores samples CI.
unix_sighandler_quit counts SIGINT/SIGTERM. The handler writes it and
the main thread reads it - and so does the threaded video worker,
through x11_alive() and frontend_driver_get_signal_handler_state(),
which a volatile sig_atomic_t, safe only between a handler and the
thread it interrupts, does not cover. It is a retro_atomic_int_t: the
handler counts with a fetch_add, lock-free for an int and so safe
there, and the readers and setters load and store it.
Auditing the two other ffmpeg users the way the core was audited.
camera/drivers/ffmpeg.c had two things:
- ffmpeg_camera_stop() flushed the decoder and then joined the poll
thread. The poll thread is the decoder's other user - it sends
every packet it reads and receives every frame - and an
AVCodecContext takes one user at a time. So the flush ran against a
live decoding thread. The join comes first now, and the flush
after, where the context has one user again.
- 'done', the poll thread's loop condition, was a volatile bool: set
by the main thread in stop(), read by the thread every turn, with
nothing held on either side - target_buffer_lock covers the frame,
not this. volatile is not a synchronisation primitive; it stops the
compiler caching the load and orders nothing, so the thread had no
guarantee of seeing what the main thread wrote before it. An
atomic, as the ffmpeg core's own flags became for the same reason.
record/drivers/record_ffmpeg.c came out clean. Its queues are
already lock-free retro_spsc rings and 'alive' is already an atomic.
'can_sleep' carried a volatile and does not need one: every access is
under cond_lock, or - the one in init_thread() - before the thread
that reads it exists. The volatile said the opposite, that here was a
flag crossing threads without a lock, so it is gone and the comment
says what actually protects it. No behaviour change there.
The record pipeline harness passes, plain and under TSan.
Two orderings around the video buffer, both found by running the
sweep under TSan.
retro_unload_game() waits for the scaling pool and then tells the
decode thread to stop - in that order, so the thread queues more
scale work between the wait and its exit, and nothing waits for the
pool again. A worker is therefore still inside
video_buffer_finish_slot(), holding the buffer's lock, when
retro_deinit() frees that lock: a use-after-free on every unload,
which TSan reports as a race on the mutex and a user meets as a
crash on closing content. The pool is waited on again after the
join, where nothing can queue to it any more, and retro_deinit()
destroys the pool before the buffer its workers write into rather
than after.
And the buffer itself was created on the decode thread and read by
the main thread through a plain store - no synchronisation either
way. That is what made the NULL dereference fixed a moment ago
possible at all: a clip with no audio has nothing to pace the main
thread, so it reached the buffer before the decode thread had made
one. Both the buffer and the pool are now made in retro_load_game()
before the thread starts, so the thread's own creation orders them
and there is nothing to publish. The NULL check stays, since a
failure to create is still a thing to survive.
The sweep's TSan lane reported nine races before these two changes
and the spin fix, seven of them on the freed mutex; it reports none
through the portion that ran here. The plain lane logs no "deadlock
detected" at all now, where it logged sixteen before today.
runloop_check_state() calls gfx_widgets_iterate() without
RUNLOOP_MSG_QUEUE_LOCK. The iterate reads nothing that lock guards, and
the only widget pushes other threads make - runloop_msg_queue_push()
without a task - touch just the pending ring, under its own lock, and
the message they allocate; task messages come from the task queue's
progress callback on the main thread.
VIDEO_DRIVER_IS_THREADED_INTERNAL() tests the Threaded Video setting
first, so without it the answer is known before
video_driver_is_hw_context() takes context_lock. With both, a frame
without threaded video takes two lock and unlock pairs fewer.
decode_video stops while the video buffer is full, and the way it
stopped was to ask video_buffer_has_open_slot() in a loop - taking
and dropping the buffer's lock as fast as the thread could go. A
core burned for as long as the buffer stays full, and full is the
normal state whenever the decoder outruns the display, which is most
of playback.
The condition a released slot signals has been there all along, and
nothing waited on it: video_buffer_open_slot() signals open_cond and
only the destructor ever touched it. The loop now waits on it, with a
five-millisecond bound so the escape below is still reached - the
main thread going to sleep with the buffer full is what that calls a
deadlock, and nothing signals that.
The spin was not only wasteful. Sweeping the core across eight clips
logged sixteen "Video deadlock detected" errors, each of which
throws away the decoded frames in the buffer: the spinning thread
kept observing a main thread that was briefly asleep and took it for
a deadlock. Waiting instead, the same sweep logs one and clears the
buffer not at all. Frames a user would have lost.
Found by ffmpeg_sweep_test, which reports the count, so a regression
here shows as the errors coming back.
gfx/common/vulkan_common.c has been failing the linux-c89 flags on
master: a declaration after a statement at the top of
vulkan_destroy_swapchain(), where the Win32 exclusive-mode release
sits above `unsigned i`, and two VkSurfaceFullScreenExclusive*
structs initialised from settings read earlier in the same function,
which C89 will not compute at load time. The declaration moves above
the #ifdef and the two structs are assigned rather than initialised;
no behaviour changes on any path.
Neither would have been caught by tools/mingw_syntax_check.sh even
when run over the whole tree, because the C89 pass did not define
HAVE_VULKAN: the file's settings members did not exist, so it failed
on those instead and the real violations never got that far. The
define joins that pass, which is what made both visible.
The decl gate added last commit then found six more files at the top
level using strlcpy through a transitive declaration - the core
option manager, the core updater list, both disk-control files, the
manual scanner and runahead - and they now include <compat/strl.h>
themselves.
video_driver_state_t::widgets_flags becomes main_flags, the display
state only the main thread writes and reads, and VIDEO_FLAG_ACTIVE and
VIDEO_FLAG_CRT_SWITCHING_ACTIVE move into it from 'flags', which the
video thread's drivers also change. video_driver_frame() sets both
every frame - ACTIVE from the driver's frame() result, the CRT bit while
switching is off - and run-ahead flips ACTIVE around every hidden frame;
all of that is plain writes rather than video_driver_modify_disp_flags()
under display_lock. The frame's snapshot still carries both words.
That is two lock and unlock pairs fewer per frame, with threaded video
or without, and more with run-ahead.
ffmpeg_audio_pipe_test drives one clip carefully. This is the other
half: eight clips across containers, video and audio codecs, rates
and channel counts, plus the two one-sided cases, each loaded, run,
seeked both ways, reset and unloaded in one process - the core keeps
its state in globals, so the second load is what finds what the
first unload left behind. Then files that are truncated, stubbed,
noise, empty or absent, and teardown while the decode thread is
still working.
It found two on its first run, both on the clip with no audio track:
- retro_run declares int16_t audio_buffer[sample_rate / 20]. A clip
with no audio has a sample rate of zero, so that is a zero-length
array, which is undefined at the declaration, before anything
reads it. UBSan said so. One element more costs nothing and is
never read, since the frame count comes out of the same rate.
- The decode thread creates the video buffer; the main thread waits
on it in retro_run without checking that it exists. A clip with
audio is paced by its audio and the buffer is there by the time
the main thread arrives, which is why this had never shown. A clip
with no audio has nothing to pace it, the main thread gets there
first, and video_buffer_wait_for_finished_slot dereferences NULL -
a segmentation fault on the first frame, every time. No buffer yet
is no frame ready, which is what the duplicate-frame path already
handles.
Both are the kind a matrix finds and a single careful clip does not:
the one-sided cases are where the assumptions that hold for ordinary
media stop holding.
The sweep is in the cores samples CI, plain and under both sanitizer
sets, sharing the audio-pipe lane's libav* suppressions.
modeline_edid.c called strlcpy with no <compat/strl.h> and reached
master: glibc 2.38 declares strlcpy and strlcat itself, so it built
clean here and passed both syntax-check passes, while MXE, clang and
the webOS toolchain - which have only libretro-common's declaration -
did not. Neither pass could have caught it, since both ask the host's
headers what exists.
A grep can, and does not care what the host provides: a .c that uses
strlcpy, strlcat, strlcpy_lit, strlcat_lit or strcasestr must include
the compat header itself rather than lean on whatever a project
header dragged in, which is the standing rule anyway. It runs as pass
0 of tools/mingw_syntax_check.sh, so it fires on the files being
checked before delivery and costs nothing.
Fourteen files were leaning on the transitive declaration and now
include it directly: the font driver and its freetype renderer, the
widgets, vulkan_common, the GX2, PS2 and OGA drivers, the Android
display server, the screensaver and cancel callbacks, the audio
driver, the test input and joypad drivers, and the webOS Wayland
helper. No behaviour changes; the ones this host can build were
checked, and the failures that remain there are the ones master
already had.
modeline_edid_gamut_str() calls strlcpy and the file never included
<compat/strl.h>. glibc 2.38 declares strlcpy itself, so it built here
and on both syntax-check passes; MXE, clang and the webOS toolchain
have only libretro-common's declaration and fail.
VIDEO_FLAG_WINDOW_TITLE_UPDATE becomes
video_driver_state_t::window_title_update, an atomic raised with a new
window_title under display_lock and taken with it by
video_driver_get_window_title(), which tests it first and returns
without the lock while nothing is waiting - on nearly every frame, on
whichever thread draws. video_driver_update_title() takes its copy
through video_driver_get_window_title() as well, rather than reading
window_title and the flag without the lock while the main thread
rewrites them.
That is one lock and unlock pair fewer per frame, with threaded video
or without.
modeline_list.c calls modeline_edid_parse() and
modeline_edid_fill_ranges() since the EDID preset went in, and the
two live display-server samples build modeline_list.c without
modeline_edid.c, so both stopped at the link. Added to each; the x11
one builds and links here, the win32 one is the same edit to the
same list.
And the ffmpeg audio-pipe sample's TSan lane, which reports a race
between a frame-threading worker allocating an AVBufferRef and the
decode thread releasing it. That ordering is the buffer's own
reference count, an atomic compiled into libavutil - a stock shared
library with no instrumentation in it - so TSan sees both writes and
no relation between them. The lane now runs with a suppression list
naming those libraries, and only those: a race in the core's own
code is still reported, and checking the libraries themselves would
mean building ffmpeg with TSan, which is not what this sample is
for.
Said plainly: the race does not reproduce here, so what confirms the
suppression is the lane itself. What is verified is that the lane
passes with the list and that nothing else in it was suppressed - a
run without the list reports nothing on this machine.
Also dropped last_phase_err from that sample, which nothing had read
since the phase check was rewritten and which the build warned about
on every compile.
The chromaticity row gives eight numbers and leaves the reader to
recognise them. The primaries are the one piece of colour data that
is measured rather than enumerated, so they can answer the question
the CTA colorimetry block was going to: which gamut is this.
modeline_edid_gamut_str() matches the block's primaries against
sRGB/BT.709, Display P3, DCI-P3, Adobe RGB, BT.2020 and SMPTE C
within 0.015 - loose enough for a binned panel and a ten-bit
encoding, far tighter than the distance between any two of those -
names the white point where it lands on a standard illuminant, and
gives the area the primaries enclose against sRGB's. Primaries that
match nothing are "custom", which is a fact about the display rather
than a failure to identify it. Only Display P3 and DCI-P3 are told
apart by white point, since they share primaries; everything else is
named from the primaries alone, so a panel calibrated to D50 still
gets its gamut named.
This works wherever chromaticity does: a real EDID read over DDC on
any platform, and the Apple Silicon panel's block, whose primaries
come from the display's ICC profile. The M1 13" panel reads
"Display P3, D65, 136%% of sRGB area", which is what that panel is.
retro_task_threaded_gather() runs every frame. The sizes of
tasks_running and tasks_finished are also kept in two atomics, changed
where a task enters or leaves those queues under the queue's lock, and
the gather returns before taking any lock while both are zero - no task
running and none to retire, as on nearly every frame. A finishing task
counts as finished before it stops counting as running, so both are
never zero while one is in flight; a task that finishes just after the
test retires on the next call.
That is three lock and unlock pairs fewer per frame for the threaded
task queue, with threaded video or without.
Three of the rows the menu draws were empty for the Apple Silicon
panel, and two of them need not have been.
Chromaticity read 0.000 across the board. The display's colour
profile has the primaries: CGDisplayCopyColorSpace gives the profile,
its rXYZ/gXYZ/bXYZ/wtpt tags give the colorants. ICC stores those
adapted to D50 with the adaptation it used in the 'chad' tag, so
reporting them as-is would put the white point at D50 rather than
where the panel actually sits; the adaptation is inverted first, and
a profile with no 'chad' reports nothing rather than the D50 numbers.
Gamma comes from the same place, and the base block now carries both.
The colour data the DCP lists per timing - ColorModes' Depth,
PixelEncoding, Colorimetry and EOTF, plus the panel's BLNitsCap as a
peak luminance - is a colorimetry data block and HDR static metadata,
so the block now has a CTA-861 extension carrying exactly those and
nothing else. No video data block: a native panel timing has no VIC
to name. No audio data block and no basic-audio flag: the panel makes
no sound. No detailed timing in the extension either, since the base
block already carries the panel's one mode. What the extension says
is what the DCP said.
Established and standard timings stay absent, which is correct: they
are legacy VGA mode bitmaps, the panel runs one timing, and a guessed
DMT list would be a claim nothing made.
The drivers update their own bits of the display flags - the HDR
support trio, USE_RGBA, GPU_DEVICE_LOST - with
video_driver_modify_disp_flags(), one read-modify-write under
display_lock, as the rest of the tree does. Under the threaded video
wrapper those updates run on the video thread (driver init, swapchain
recreation, HDR probes, device loss) while the main thread changes other
bits of the same word, and the one locked step keeps either side's
change from being lost between a separate read and write.
video_driver_set_disp_flags() goes; the Vulkan and Metal HDR probes
gather their bits and apply them in the one call.
The VIDEO_FLAG_WIDGETS_* bits - paused, rewinding, fast and slow
motion - are set by the runloop on the main thread every frame and read
through the frame's snapshot (video_frame_info_t::video_st_flags). They
live in video_driver_state_t::widgets_flags, which the runloop writes
directly, instead of in 'flags', where each write went through
video_driver_modify_disp_flags() and display_lock.
video_driver_build_info() puts both words in the snapshot.
That is three lock and unlock pairs fewer per frame, with threaded video
or without.
The internal panel has no EDID and never will: it is driven over an
internal bus with no DDC behind it, so IOAVServiceCopyEDID fails and
nothing ever negotiated a block. The DCP publishes the facts a block
would carry anyway, on its AppleCLCD2 node, as ordinary registry
properties needing no private API - TimingElements gives every mode's
Total, Active, FrontPorch, SyncWidth, SyncPolarity and PreciseSyncRate
(16.16 fixed point, kHz horizontally and Hz vertically) along with
IsInterlaced and IsPreferred; IOMFBDisplayRefresh gives the refresh
bounds in mach ticks; IOMFBMaxSrcPixels gives the pipe's maximum dot
clock. That is a detailed timing and a range-limits descriptor, and
CoreGraphics supplies the identity and the physical size.
modeline_edid_synthesize() assembles those into an EDID 1.4 base
block in memory, so the menu decodes it with the same parser as a
block read off a wire and every row means what it says. Nothing is
written to disk and nothing is sent to a display.
What was not reported is left at the encodings the spec has for
"undefined" rather than filled with plausible values: no chromaticity,
no gamma, no established or standard timings, no power-management
claims, no sRGB claim. The range descriptor is type 0x01, limits only,
since no GTF or CVT formula was implied. The unspecified-text
descriptor reads "DCP timings", so the origin is visible in the menu
itself, and the monitor name is "Built-in".
Apple Silicon only, on __aarch64__: Intel and PowerPC Macs read a
real EDID through IODisplayConnect and keep showing that.
The synthesizer lives in the engine rather than the Cocoa file, so it
is C89 and testable: the parse sample round-trips a block built from
a MacBook Pro 13" M1's published numbers (2560x1600, htotal 2642,
vtotal 1682, 6613877/3932151) and checks the parser reads back
2560x1600p 60.00 Hz, 266.63 MHz, 100.92 kHz with the porches, the
polarities and the 30-60 Hz range intact, plus the two-timing slot
shuffle and the refusals. The Cocoa half was checked against stub
CoreFoundation and IOKit declarations; the MacOS lane is its gate.
"Read: N/A" on its own does not distinguish a display server with no
way to read an EDID from a display that carries none, and the second
is what a MacBook shows: its panel is wired to the DCP as an Embedded
service with no DDC behind it, so IOAVServiceCopyEDID fails and there
is nothing to report. A line under the N/A now says a built-in laptop
or all-in-one panel has no EDID and an external display over DDC
does, which is the answer to the question the empty list raises.
get_edid returned early when IOServiceGetMatchingServices() failed for
IODisplayConnect, which is exactly what Apple Silicon does - the class
is not there - so the DCP fallback added in c851c332 never ran and the
menu showed "Read: N/A" on an M1. A failed match now just leaves the
iterator empty and the walk falls through to the DCP, which was the
intent.
Two other things that would have hidden the same symptom: the
IOAVService symbols are now also looked for in IOKit.framework by path
when dlsym(RTLD_DEFAULT) comes up empty, since RTLD_DEFAULT only
searches images already loaded with global scope; and when nothing is
found the log says how many DCPAVServiceProxy services were seen, how
many were External, how many answered with an EDID, and the last
IOAVServiceCopyEDID result. A machine showing only its built-in panel
has no External proxy and no EDID to read - the log now says so
rather than leaving "N/A" unexplained.
Darwin has forced HAVE_MODELINE off since 3cad8ac8 (2023), when
HAVE_CRTSWITCHRES still meant the external SwitchRes library and only
XRandR and Win32 could apply a timing: "bake out Switchres
(Linux/windows only)". The engine is in-tree now, C89 with no
libraries of its own, and it carries the EDID reader as well as the
mode generator - so the disable also compiled out System Information
> Display Information > EDID, which is why the submenu did not appear
on an M1 even with the DCP reader in place.
The qb force-off goes, and -DHAVE_MODELINE joins the macOS Xcode
builds: Metal.xcconfig's macosx line and the Steam, AppStore and
QTConfig layers that the Metal project's other configurations read,
plus the inline flag lists of the three macOS-only projects
(RetroArch, OSX107, PPC). The iOS and tvOS projects are left alone;
BaseConfig is shared with RetroArch_iOS13, so the define is placed in
the macOS-only layers rather than there.
macOS still cannot program a timing - dispserv_apple leaves the
modeline ops NULL, so the CRT switcher generates and finds no way to
apply, exactly as on any other server without them. What the build
gains is the EDID reader and its menu.
alive, focus, presentable, has_windowed and suppress_screensaver are
single values the worker publishes at the end of each frame and the
main thread reads, most of them every frame. They are atomics, stored
with release inside the worker's existing critical section and loaded
with acquire by readers that no longer take the lock. The content scale
for the statistics is one packed atomic, so width and height are read
together, and video_thread_async_deliver() takes the lock only when the
worker has left uploads to deliver.
That leaves the frame handoff's two acquisitions as the main thread's
only wrapper locks per frame, where there were eight.
get_edid walked IODisplayConnect, which does not exist on Apple
Silicon: the display stack moved to the DCP, a coprocessor with its
own firmware that owns the link, so the op reported -1 there and the
menu said "not available".
The DCP does publish one DCPAVServiceProxy per connected display, and
IOKit carries two unpublished functions to get at it -
IOAVServiceCreateWithService() to bind a proxy and
IOAVServiceCopyEDID() to ask the DCP for the block it read over DDC.
That is the route BetterDisplay and Lunar take; there is no public
equivalent. Used here as the fallback when the IODisplayConnect walk
finds nothing, so PowerPC and Intel keep the published path and only
Apple Silicon reaches this one.
Unpublished means the symbols can be withdrawn, so they are resolved
with dlsym rather than linked: a macOS that drops them leaves the
pointers NULL, logs once and reports -1, instead of a binary that
will not launch. It is plain C - CoreFoundation and IOKit only, no
Objective-C - and every CF object it takes is released, including
on the paths that find nothing.
The proxies carry no CoreGraphics display ID, so the one for the
display in use is found by reading each external candidate's EDID and
matching the identity bytes (manufacturer 8-9, product 10-11, serial
12-15) against what CoreGraphics reports for that display: the same
three-way match the IODisplayConnect path makes, done on the block
itself. Embedded proxies are skipped, since the built-in panel has no
EDID to read. When nothing matches but exactly one external display
answered, that block is taken - with two, the guess would be unsafe
and is not made.
No Metal or Apple Silicon toolchain here: the function body was
checked against stub CoreFoundation and IOKit declarations, and the
MacOS lane is the compile gate.
The audio path carries PCM. AC-3 to a television works within that -
the frontend has the samples, the driver encodes them and sends the
result - but a stream that arrives already coded has nothing to
encode: a DTS track, a core handing over a bitstream. It reaches the
device as it is or not at all, and there was nowhere for it to go.
audio/audio_bitstream.c is that place. It takes complete coded
frames, wraps each in the IEC 61937 burst its format asks for, and
hands out bytes a device opened for a bitstream expects - which are
shaped exactly like two channels of 16-bit PCM, because that is what
a burst is on the wire. So no audio driver needs a new entry point;
what a driver must do is open the device in the right subtype and
stop treating what it is given as samples.
Three things it settles that a caller would otherwise get wrong:
- Pacing. A burst stands for a fixed number of PCM frames, and the
device consumes it in exactly that much device time, so a bitstream
is paced by the device like any other stream. The frames each read
accounts for are reported, which is what rate control and the clock
need. The queue holds four bursts, so the device rather than the
source decides how far ahead it runs.
- Gaps. A receiver locked to a burst stream must not be handed
silence when the source is late - it drops the lock and mutes for a
second or more. A starved read produces the pause burst the
standard has for that, and exactly one: a caller with room for
twenty has room, not twenty periods of gap to fill, and the device
asks again on its next pass. A pause burst accounts for its own
length in frames and no more, so a gap is not claimed as the time
the missing audio would have taken.
- Alignment. A burst is all-or-nothing; a device given half of one
loses the rest of the stream. Reads come out on burst boundaries
and a request smaller than a burst produces nothing rather than a
truncated one.
DTS settles its burst size on the first frame, since 512, 1024 and
2048 PCM frames are all possible and a stream does not change which;
frames in any of the four packings are repacked to what a receiver
takes. AC-3 and E-AC-3 are carried as they arrive.
bitstream_test drives it with real ffmpeg streams: a frame in and the
burst a receiver would read back out of it, the queue refusing a
fifth frame, two seconds of DTS accounting for 95744 frames of device
time with no gaps, a starved read probing as a pause burst with its
padding zeroed, and noise refused rather than wrapped.
What a caller must still arrange: the mixer cannot be mixed into a
bitstream, so menu sounds are silent while one plays, and only a
device opened for the subtype will take it. Wiring a driver to open
that way, and choosing where a bitstream source is selected, is the
next step.
Two things a DTS path without a decoder needs.
The primary audio coding header, which sits behind the frame header:
the subframe count, the primary channels, the subbands, where the
high-frequency vector quantiser starts and whether joint intensity
coding is on. What it is really for is
rdts_decodable_from_spec(). ETSI TS 102 114's Annex D leaves out
both large vector codebooks - D.10.1's 4096 ADPCM coefficient
vectors and D.10.2's 1024 high-frequency vectors, each with the
words "Due to its extensive size, this table is not included here" -
so a frame using either cannot be decoded by anyone working from the
published standard, only carried. The coding header is what says
which frames those are.
That matters more than it looks. ffmpeg's own encoder emits neither,
so its streams read as decodable-from-spec - which is exactly why a
decoder written from the standard and tested against ffmpeg would
look correct and then produce noise on the first disc anyone tried.
The harness prints the verdict per stream so the distinction is on
the record rather than in a commit message.
And rdts_burst_type(), which gives the IEC 61937 burst a frame
belongs in and the period it needs: 512 PCM frames for type I, 1024
for II, 2048 for III, refused for a sample count none of them covers
and for a frame too long for its period. Nothing in rdts depends on
iec61937 and nothing there depends on rdts; these numbers are the
contract between them. A caller sending DTS to a receiver parses,
repacks with rdts_to_core() where the stream is 14-bit or
byte-swapped, and wraps - which the harness now does for eight
frames and reads back out of the bursts byte for byte.
The driver side of that route is a separate piece of design: the
audio path carries PCM, and a bitstream that is not encoded here has
no source in it yet - unlike AC-3, which is encoded from the PCM the
pipeline already has.
The rest of the wiring for AUDIO_TYPE_LPCM: an AUDIO_MIXER_TYPE_LPCM
sound and audio_mixer_load_lpcm(), the nine voice dispatch sites
beside the AC-3 ones (set_avail, tell, destroy, play in both formats,
release, mix in both formats), the loader dispatch in
audio_driver_mixer_add_stream, the task's two push paths and its
NBIO type, FILE_TYPE_LPCM, the .lpcm and .pcm extensions in the
browser's filter and in what retroarch.c treats as a playable sound.
The one decision the mixer forced: a sound is a file someone dropped
in a directory, and raw linear PCM says nothing about itself, so
there is nobody to ask what shape it is. A buffer that opens with a
DVD or Blu-ray header keeps what the header says - the arm reads
that first. A buffer that does not is taken as 16-bit stereo at
48 kHz, little-endian, which is what a headerless .pcm file usually
is; a caller that knows better says so through
audio_transfer_lpcm_format() before starting, which is what a core
or a disc reader would do.
audio_mixer_loop_test gains the two contracts every other codec
there has - a stream that replays after a seek to 0, and a repeating
voice that keeps playing past its loop point, in float and in int16.
Both switches over the two enums were swept with -Wswitch across
every file that names them, which is how the AC-3 loader case came
to be missing earlier this week.
DTS still has no arm: rdts is the frame layer, and playback needs a
decoder.
The arm for AUDIO_TYPE_LPCM, beside the AC-3 one: allocate, take a
buffer, start, report, read to float or to int16, seek, tell, free.
Raw samples say nothing about themselves, so where the shape comes
from is the thing this had to settle. A buffer that opens with a
Blu-ray access unit header is recognised by it - its four bytes name
a payload length, and a length that agrees with the buffer is what
tells that header apart from a DVD one, which carries no length. A
buffer that is neither is the caller's to describe, through
audio_transfer_lpcm_format(), before starting: a track ripped from a
disc, a core's own samples.
What LPCM gives that a compressed arm cannot is an exact seek. Every
frame is the same size, so the offset is arithmetic - nothing to
decode up to, no state to carry across - and the frames a buffer
holds are known before a byte is read. At twenty bits the frame is
not a whole number of bytes (two samples to five, and an odd channel
count straddles the pair), so the arm counts in bits and converts
once, which is also why the frames-left helper exists rather than a
division at each site.
The harness drives it against the codec: chunked reads that have to
match rlpcm's own output sample for sample, the compressed frontier
landing on the end of the buffer, a seek to frame 1000 that produces
the samples at frame 1000, and a seek past the end that is refused.
Still to come: the mixer type and its voice dispatch,
task_audio_mixer, the browser's extensions, and what the playlist
preview reports. DTS has no arm and will not have one until it has a
decoder; rdts is the frame layer.
Two formats the audio side had no reader for.
rlpcm is a codec: samples in the order they were taken, which leaves
the shape to get right - which way round the bytes go, how many bits
a sample has and how they sit in the bytes it takes, and which
speaker each belongs to. It reads and writes 16, 20 and 24 bits
either way round, and takes the shape from a DVD-Video private
stream header, an HDMV access unit header, or from a caller that
has neither. 20-bit is the awkward one: both disc formats carry it
as pairs of samples in five bytes, the fifth holding both low
nibbles, so the pair rather than the sample is the unit and a frame
of an odd channel count is not a whole number of bytes. It is
bit-exact against ffmpeg's own converters at 16 and 24 bits in both
orders, and what it writes back is the bytes it read.
rdts is the frame layer, and only that. It finds frames, reads the
rate, the arrangement, the sample count, the nominal rate and the
extensions a frame names, in all four packings - 16-bit either way
round, and the two 14-bit forms a DTS audio CD carries, whose words
hold fourteen bits of the stream sign-extended to sixteen - and
repacks any of them into the plain form a decoder or a receiver
takes. That is what carrying a DTS stream to a decoder that is not
this one needs, and iec61937 already frames the bursts. It does not
decode: the subband synthesis, the ADPCM predictors and the vector
codebooks are a far larger piece of work than the frame layer, and
nothing here stands in for them.
dts_lpcm_test drives both against ffmpeg - the frames where its
encoder put them, the rate and the arrangement against ffprobe's
reading of the same file, the samples through its converters and
back - and is in the formats samples CI.
Building the frontend on these is the next step and is not here: the
audio_transfer arm, the mixer type, the file extensions and what the
playlist preview says are all still to do.
A filter can output another pixel format than the core's: ntsc_crt
always gives XRGB8888. The driver is set up for the filter's format only
while the filter is enabled; with video_filter_enable off a loaded
filter stays in place for the toggle and the driver takes the core's
format. VIDEO_FILTER_TOGGLE sets the driver up again when the loaded
filter changes format (video_driver_filter_changes_format()) and stays
instant when it does not, and removing the filter from the menu
reinitialises video as resetting it does.
Unfiltered frames so never reach a driver set up for another format,
which read an RGB565 core's rows at four bytes a pixel, past the end of
the frame, in the threaded video wrapper's copy and in the driver's
upload.
Under the threaded video wrapper video_driver_frame() hands the core's
frame to the worker unfiltered, marked with video_thread_defer_filter();
video_thread_frame() copies it into the ring slot at the core's pixel
size, and the worker runs the filter into state_buffer just before the
driver's frame, as it does on the read_viewport path. The main thread no
longer filters, and copies the smaller unfiltered frame into the slot.
While the filtered frame is being recorded (post_filter_record) the
filter stays on the main thread, where the recording is.
video_driver_init_filter() and video_driver_filter_free() wait for the
worker to go idle before they replace or free what it filters with.
The Metal driver's semantic table stopped at OriginalAspectRotated, so
a slang shader reading SwapCount, TotalSubFrames or CurrentSubFrame got
the zero default slang_process leaves for an unwired semantic - on
Metal SwapCount was 0 threaded or not, which is the report behind
PR #19530 (whose change is on the main-thread copy the video thread
overwrites anyway).
Each pass gets the three fields, the table names them at their slang
indices (the HDR and sensor slots in between are named as NULL, since
the table is positional), and the frame view takes the values from the
frame info on every render: SwapCount advanced by the sub-frame index,
since each sub-frame is one more present, and CurrentSubFrame 1-based
as the other drivers report it. metal_frame's sub-frame loop now sets
video_info->current_subframe before each re-render, as d3d11 does,
so the per-sub-frame values are right rather than all reading as the
first.
Written without a Metal toolchain to hand; the MacOS lane is the
compile check.
current_msgs[] is touched only by the thread that owns the widgets -
the threaded video worker when it draws them, the main thread
otherwise - and at init and deinit, so current_msgs_lock guarded nothing
and goes. The pending ring's count is written under msg_queue_lock as
before and read without it by the consumer, so a frame with nothing
pending takes no lock; a push that lands after the test is taken on the
next frame.
That is two lock and unlock pairs fewer per frame with nothing pending,
with threaded video or without, and one more while messages are shown.
Since afacc8c neither guards a fifo: the ring is lock-free and the
flush runs outside them. All they do is carry the two bounded waits
- the worker's when the ring is full, the core's when it is short -
and the signals that end them. Name them for that. Rename only; no
change in what is locked where.
The queue from the capture worker to the core's retro_microphone_read()
was a fifo_buffer_t under fifo_lock, and the worker held that lock
across the whole of microphone_driver_flush() - the device read, the
dual-mono up-channel, the resampler and the write - so the core's
read, which needs the same lock to take its frames, queued behind a
slice's resampling every time the two met. On the frame-synchronous
path (no worker) the same thread flushes and reads, so the lock was
never the issue there.
One producer, one consumer, either way: replace it with a
retro_spsc_t. The worker's flush runs without the lock and its room
check is a producer-side query; the core's read takes what is there
without the lock. fifo_lock and fifo_cond stay for the two bounded
waits and the signals, as they were. retro_spsc rounds capacity up
to a power of two; outgoing_size keeps the size asked for, the
flush's room is measured against it, and the "asked for more than
the queue holds" check compares to it.
audio_driver.o compiles through the real build rule; the eight
audio samples that #include audio_driver.c build and pass under
ASan+UBSan. There is no microphone harness and no capture device
here, so the mic path itself is not run.
The audit of menu/cbs and menu/drivers turned up little: the menu
drivers and the title, sublabel and ok callbacks draw everything
through msg_hash already, and what remained was a handful of value
strings in the setting string-representation functions. "Auto" (four
settings), "Always", the XMB layout's "Console" and "Handheld", the
WASAPI shared-buffer mode names, the SMB2 "KRB if available, NTLM
if not" fallback, the CPU frequency "Min." / "Max." and the
"Keyboard <key>" bind description now come from msg_hash.
MENU_ENUM_LABEL_VALUE_AUTO existed as an enum with no string behind
it; it has one now.
Left as they are: hardware and protocol names (iPega, SNES30,
NTLMSSP, Kerberos), the animation easing names, the developer
counters' "ticks, runs" and "Prev/Curr" cheat search values, and
the Switch overclock messages. The localize markers on RARCH_LOG
lines go: logs are not translated.
wiiusb_hid_process_control_message() retried a failed USB write until
it succeeded. A pad unplugged with a message pending never succeeds,
and the poll thread - the one thread that services every pad's reads
and honours poll_thread_quit - was stuck in that loop for good: every
other pad went dead and the driver could not be shut down. Three
attempts, then the message is dropped with a warning.
wiiusb_hid_device_send_control() copied the next message into the
adapter's single mailbox unconditionally. The poll thread sends from
that mailbox with a blocking USB write, which on LWP is exactly when
this thread gets to run, so the next rumble or LED message could be
written under a transfer in progress and the device handed a torn
one. The mailbox now takes a message only when the previous one has
been sent (send_control_type back to NONE); a message that arrives
before then is dropped, and for rumble and LED updates the next one
wins. The length is checked too: len of 0 underflowed the payload
size, and anything over the mailbox's 128 bytes overran it.
No devkitPPC here: syntax-checked against stubbed libogc headers,
not built or run.
Six things in one driver, found reading it for thread and lifetime
bugs; none needs the others but they all touch the same few
functions.
The poll thread called libusb_handle_events_timeout_completed() with
a zero timeout, so it returned at once and spun a core for as long as
the driver was loaded. It also passed NULL for the context - the
default context - while the driver's devices and hotplug callback
live on hid->ctx, so hotplug events on that context were never
dispatched by this thread at all. Block on hid->ctx for up to 100 ms
per lap; hid->quit is the completed flag, so shutdown does not wait
out the timeout.
remove_adapter() looked only at the head of the adapter list, so
unplugging any pad but the most recently plugged one was ignored:
its thread kept issuing transfers to a device that was gone, its
slot stayed connected, its handle leaked. And for the head it read
slot, send_control_lock and send_control_buffer from the list's
sentinel instead of from the adapter - slot 0, NULL, NULL - so it
disconnected and deinitialised slot 0 whatever pad had left, freed
neither the lock nor the fifo, and left the real slot connected.
Walk the list and tear down the adapter that matched.
libusb_hid_init() enumerated devices with add_adapter() and then
registered the hotplug callback with LIBUSB_HOTPLUG_ENUMERATE, which
delivers an ARRIVED for every attached device during registration:
every pad was added twice, the second opening another handle and
failing to claim the interface. With hotplug the enumeration is the
callback's; the manual scan is kept only where there is no hotplug.
adapter_thread held send_control_lock across the outbound
libusb_interrupt_transfer(), a call with a one-second timeout, so a
send_control() from the frontend - rumble, LEDs - could block for
that long. Copy the command out under the lock and send with it
released. Its payload check compared the bytes left against
sizeof(len) rather than len (harmless, since length and payload are
written together, but wrong); it now checks len and the buffer it
copies into.
add_adapter()'s error path left an opened handle and a claimed
interface behind, and would have joined the thread without asking it
to quit; it closes and releases now. libusb_hid_free() removed the
adapters while the poll thread - which runs the hotplug callbacks
that mutate the same list - was still alive, and looped forever if
a removal failed; it joins the poll thread first and gives up on a
failed removal.
Compiled through the real build rule with HAVE_LIBUSB. No USB pad on
this box: not run.
The Disc Information list (Title, System, Serial, Version, Release
Date, Genuine Disc, Number of tracks, and each track's Mode, Size and
Length), the subsystem "Load <name>" / "Start <name>" / "Current
Content:" entries and the "Keyboard <key>" bind description were
literal English. Format strings for each; Disc Information's five
text fields go through one table instead of five copies of the same
block, and the fixed-size per-line buffers become a single entry
buffer. Text as displayed is unchanged. The stale localize marker on
the no-disc message, which already used MSG_NO_DISC_INSERTED, goes.
decode_thread_dead, main_sleeping and do_seek are the two threads'
flags to each other. All three are written under fifo_lock, but
each is read somewhere without it: decode_thread's loop condition
and the video path's waits read decode_thread_dead; decode_video's
deadlock check reads main_sleeping and do_seek. They were a
volatile bool and two plain bools, and TSan reported all three
against ffmpeg_audio_pipe_test - the races it found on the fifo
version too, before the ring conversion.
Make them retro_atomic_int_t, read with acquire and written with
release through the existing *_STR macros and new *_SET() ones.
The lock discipline around them is unchanged; this only makes the
unlocked reads well-defined.
ffmpeg_audio_pipe_test is now clean under TSan across repeated
runs, so its workflow runs TSan with halt_on_error=1 rather than
reporting and continuing.
cores/libretro-ffmpeg had no harness. This one links ffmpeg_core.c
as a plain libretro core (no CORE_PREFIX) and drives it as a host
would: generates a clip with the ffmpeg CLI (a 440 Hz tone under a
test pattern), loads it, runs 5 s of frames and checks the audio
frame total against the core's reported rate and that every batch
carries the tone, then presses R1 (+30 s) and L1 (-30 s) - the seek
handshake between the decode and main threads around the audio ring
- and checks audio resumes after each, then two seeks back to back
and a mid-stream teardown. Every retro_run() runs on a watchdog so
a hang in the handshake fails rather than hangs.
A workflow builds and runs it under ASan+UBSan (leak detection off:
libavutil's HW-device probing keeps a few objects) and TSan on
changes to the core, the queue sources, or the sample.
The fifo carrying decoded audio from the decode thread to retro_run()
was a fifo_buffer_t under fifo_lock, and fifo_lock is the core's
whole audio handshake: the waits in both directions, decode_thread_
dead, main_sleeping, the seek request and decode_last_audio_time.
The ring's own read and write were done under it too, so the two
copies could not overlap.
The queue is single-producer/single-consumer. Replace it with a
retro_spsc_t and take the read and the write themselves out from
under the lock; everything else the lock covers stays under it, the
room and availability are still counted under the lock before each
wait, and the condition signals are unchanged.
The three places that emptied the fifo needed thought, because a
retro_spsc can only be reset whole with both sides quiesced:
- the main thread's seek request (consumer side, decode thread
possibly mid-write) skips what the ring holds - consumer-safe;
- the decode thread's post-seek reset runs under fifo_lock while
the main thread is parked in its seek wait on that lock, so the
consumer is quiesced and a whole reset is legal;
- the decode thread's "audio deadlock" reset likewise runs under
the lock only when main_sleeping says the main thread is in
scond_wait on it.
The standalone core build (Makefile.common) links retro_spsc.c in
place of fifo_queue.c; the RetroArch build already had it.
Verified by samples/cores/ffmpeg_audio_pipe (next commit): the core
driven as a host would, 5 s steady with the audio frame count exact
and no silent batch, seeks forward and back with audio resuming, two
seeks back to back and a mid-stream teardown, on a watchdog; clean
under ASan+UBSan. TSan reports the same pre-existing flag races as
on the fifo version (main_sleeping read outside the lock in
decode_video, the volatile decode_thread_dead) and nothing on the
ring. ffmpeg_core.o compiles through the real build rule.
The fifo between ps3_audio_write() (the frontend) and ps3_event_loop()
(the output thread) was a fifo_buffer_t under an lwmutex both sides
took: the writer around its write and its room checks, the output
thread around its pull. Single-producer/single-consumer, so replace
it with a retro_spsc_t and drop that mutex; cond_lock/cond and the
bounded waits from 790438f are unchanged, the output thread signals
after each block as before.
retro_spsc rounds capacity up to a power of two; ring_size keeps the
size asked for, write_avail() measures room against it and
buffer_size() reports it, so rate control sees what it saw.
Same shape as the sdl_audio, rsound, coreaudio and audioworklet
conversions. No PS3 toolchain here: syntax-checked against stubbed
PSL1GHT headers, not built or run.
A message pushed without a task goes onto the pending ring as it
arrives: the producer fills the widget with the raw text and pushes it
under msg_queue_lock alone, without the widget state lock, and
gfx_widgets_iterate_frame() measures and wraps it with the message font
when it takes it off the ring - on the threaded video worker when that
draws the widgets, on the main thread otherwise. Task messages are
measured at push as before, where their updates happen.
The write-only gfx_widgets_frame_count goes.
ac368f4 taught intfstream_crc_step() to fold the checksum straight
out of the VFS's mapping when the file was opened with
RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS, and bade0b7 moved the
content scanner onto it. The other places that open a file only to
hash all of it were still HINT_NONE: the core updater's sliced CRC
of the installed core, the core backup and restore CRCs of the
backup file, and netplay's CRC of the content at session start.
Same hint at all four; where the VFS cannot map (a URL scheme, a
huge file on a 32-bit host) it is a hint and the read path is what
it was.
Compiled through the real build rule; core_crc_slice_test and
core_backup_io_test pass under ASan.
intfstream_file_get_crc_and_size() opened the file HINT_NONE and
hashed it through a 256 KB buffer; the bounded-span path (a CD data
track, hundreds of MB) had its own copy of that loop with its own
malloc. Open FREQUENT_ACCESS instead, so the VFS maps the file
where it can and intfstream_crc_step() folds the CRC straight out
of the mapping - no read, no copy - and route the bounded span
through crc_step() as well, which keeps one scratch buffer per
stream on hosts that cannot map. Where the hint cannot be honoured
(a URL scheme, a huge image on a 32-bit host) the read path is what
it was.
A short step inside the span fails the call, as a short read did.
intfstream_crc_step() malloc'd 256 KB, read into it, hashed it and
freed it - per step. 256 KB is above glibc's mmap threshold, so
every quarter megabyte of a scanned disc image was a fresh mmap(),
64 zero-filled page faults, a read() copy, and a munmap(): the
allocator and the fault handler doing more work than the CRC.
Two changes. A file the VFS has mapped whole - opened with
RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS, borrowed through
filestream_get_mapped_ptr() - is now hashed straight out of the
mapping, position advanced by seek: no scratch, no copy. Whole file
or nothing, so file offsets are map offsets. Everything else reads
into a scratch buffer allocated on the first step and kept with the
stream (freed in intfstream_close()), so a multi-gigabyte image
costs one allocation rather than thousands.
Checked against a reference CRC over a 3 MB pattern: whole file and
a bounded span, mapped and buffered, all agree; intfstream_get_crc()
too. core_crc_slice_test and core_backup_io_test pass under ASan.
file_archive_parse_file_init() opened the archive through the VFS
and then, under HAVE_MMAP, opened the same path a second time with
open() and mmap()ed it itself, keeping its own fd, to give the zip,
7z and zstd decoders a whole-file pointer to index by offset. That
was POSIX only - the Windows build streamed every archive through
filestream_read() - and it had to special-case what the VFS already
knows (URL schemes, failed maps).
The VFS maps files itself now, on POSIX and on Win32, when opened
with RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS, and lends the
mapping out through filestream_get_mapped_ptr(). Open with that
hint - under the same 256 MiB ceiling as before, sized with
path_get_size() up front, so a huge archive is still streamed rather
than mapped whole - and borrow the pointer; whole file or nothing,
since the decoders index it by archive offset. One open instead of
two, no private fd, no munmap: filestream_close() takes the mapping
with it. archive_mmap_fd goes; archive_mmap_data stays, now gated
on VFS_HAVE_FILE_MAPPING (vfs.h's umbrella for HAVE_MMAP or Win32)
rather than HAVE_MMAP, in the header and at the eleven decoder sites,
so the Windows build takes the zero-copy decode paths as well.
vfs.h uses FILE but did not include <stdio.h>; it does now, so
archive_file.h can include it. archive_zip_test checked the old fd
field; it checks the pointer.
Verified: the four archive objects compile through the real build
rule and, with mingw-w64, for Windows; archive_zip_test, archive_
zstd_test, r7z and rzip samples pass under ASan+UBSan, and strace
shows the zip test's archives being mapped MAP_SHARED through the
VFS.
A super resolution is a wide mode at the CRT's line rate, and its
pixel clock is width over the active fraction of the line times
hfreq: 3840 wide at 15.7 kHz is about 80 MHz, at 31 kHz about
160 MHz. A display, or the converter in front of it, that states a
lower maximum in its range limits descriptor will refuse such a mode
outright. With the edid preset the block is in hand, so
modeline_edid_super_width() steps a chosen width down through
3840 / 2560 / 1920 until the clock at the top of the display's
horizontal band fits the stated maximum, and the switcher applies
the result with a log line saying why. An unstated maximum, Native,
Dynamic, or a width that already fits leaves the choice alone; 1920
is the floor since the engine lists nothing narrower.
The Win32 EDID reader picked the adapter under the RetroArch window.
With the CRT switcher running that is the wrong head on a two-head
box: the window sits on the desktop monitor while the CRT is the
adapter the modeline path opened (by monitor index, name or "auto")
and stored in ml.device_name. Prefer that adapter while the path is
open, so the EDID the menu shows and the "edid" preset reads is the
display being driven; without an open path the window's adapter, then
the primary, as before.
string_list_capacity() is public and took any cap. A cap below
size realloc'd the element array out from under the live entries
while size still counted them, so the next walk or string_list_free()
ran off the end; a cap of zero is realloc(p, 0), which on glibc frees
p and returns NULL, leaving elems dangling behind a "failure". Refuse
both, and a NULL list. Every in-tree caller grows, so nothing
changes for them.
string_list_append_n() takes any size_t len and did malloc(len + 1):
at SIZE_MAX that wraps to a zero-byte allocation the memcpy then
runs len bytes into. Not reachable from a real string; refused
anyway, since the entry point is public.
Checked under ASan: below-size, zero and NULL refused, exact and
larger accepted, SIZE_MAX append refused, string_split unaffected.
tools/settings_check.py table --check has been failing on master
since audio_block_frames was removed and audio_headphone_virtual_surround,
audio_output_layout and save_compression_codec were added without
the reference being regenerated; tools/gen_lbl_str.py likewise had
three MENU_ENUM_LABEL_*_STR macros to emit for those rows. Both
regenerated, nothing else touched: the reference reflects the
settings definitions as they are, and the three label strings now
exist for the comparison chains that want them.
dispgfx_widget_t::msg_queue held disp_widget_msg_t pointers - up to
MSG_QUEUE_PENDING_MAX of them, pushed from any thread, popped by
gfx_widgets_iterate() on the main thread - as sizeof(pointer)-byte
records in a fifo_buffer_t: a heap buffer allocated at init, byte
arithmetic at every site, and a fifo_write() that silently wraps on
overflow, which is why the push had to pair its own full check with
the write under the lock and comment on it.
It does not need to be one. Replace it with an inline array of
MSG_QUEUE_PENDING_MAX pointers plus head/count, and two static
helpers, gfx_widgets_pending_push() (fails when full, no wrap) and
gfx_widgets_pending_pop(), called at the five sites - push, the
per-frame pop in iterate, the discard path and the free path - each
under msg_queue_lock exactly as before. Several producers, so this
stays a locked queue; it is not an SPSC ring. fifo_queue.h leaves
the header. Behaviour is unchanged: same capacity, same FIFO order,
same reject-when-full.
The two samples that build gfx_widgets.c (gfx_widgets_sanitize,
gfx_widgets_task_bar_stack) no longer link fifo_queue.c;
gfx_widgets_msg_queue_race models the old protocol with its own
fifo_buffer_t and is left as it is. All three pass under ASan+UBSan,
the race test under TSan too; gfx_widgets.o compiles through the
real build rule.
CRT SwitchRes gains a fifth mode, EDID, next to 15 KHz / 31 KHz /
31 KHz 120 Hz / INI. It hands the modeline engine an "edid" monitor
preset whose ranges come from the range limits descriptor of the
display in use, read through the display server's get_edid once the
server is bound at engine init. switchres.ini can ask for the same
with "monitor edid".
modeline_edid_fill_ranges() splits the EDID's horizontal band at the
15 / 25 / 31 kHz arcade boundaries and gives each piece the blanking
of the matching arcade preset, with VESA GTF blanking above 40 kHz
sized for the tallest picture the top rate scans; the vertical band
is the EDID's, clamped to the engine's 40-200 Hz. The descriptor
holds whole kHz and Hz rounded by the maker, so each maximum extends
to the next whole unit: a 15.734 kHz set that says 15-15 still admits
15.734. A block without a usable descriptor, or none the server could
read, falls back to generic_15 with a warning rather than a dead
preset.
The gen keeps the raw block so the preset can be re-applied from the
engine alone, which keeps the gold harness and the engine free of the
display server. The parse sample covers a 15 kHz consumer set, a
tri-sync chassis, a VGA multisync, the clamps, the cap and a
range-less block.
The setting's maximum moves to CRT_SWITCH_EDID; the settings table
records name, type and flags, so tools/settings_table.reference is
unchanged by this.
audio_thread_wrapper.c and psp_audio.c include queues/fifo_queue.h
but neither touches a fifo_buffer_t: the wrapper never did, and
psp_audio keeps its own ring (write_pos/read_pos over a uint32_t
array; the "fifo_lock" name is all that is left). With the audio
drivers that did use it now on retro_spsc, the remaining fifo_queue
users in the tree are wasapi, ps3_audio, gfx_widgets, libusb_hid and
the ffmpeg core - each with a real reason - so a stray include no
longer has cover.
record/drivers/record_ffmpeg.c had no harness. This one opens the
driver on the lossless preset (libx264rgb + flac), pushes 240 frames
whose pixels encode the frame number - one colour in the top half,
inverted in the bottom, so a torn or reordered frame is caught by
any pixel - plus audio, from the test thread while the driver's
encoder thread drains, finalizes, decodes the file back with
libavcodec and checks every frame in order and pixel-exact.
Covers the attr/video/audio queues under contention, the blocking
wait on a full queue, and the drain after the encoder thread is
joined. A workflow runs it under ASan+UBSan and TSan on changes to
record/, the queue sources, or the sample.
The three fifos between the pushing thread (ffmpeg_push_video /
ffmpeg_push_audio, on the frontend's audio and video paths) and the
encoder thread were fifo_buffer_t under one lock, and both sides held
that lock for their whole frame copy: the push wrote a frame - up to
fb_width x fb_height x pix_size, row by row - under it, and the encoder
read one out under it. At 1080p XRGB that is an 8 MB memcpy on each
side that the other could not overlap with.
Each queue is single-producer/single-consumer, so replace them with
retro_spsc_t and drop the lock: the two copies now run concurrently.
cond/cond_lock and the can_sleep handshake are unchanged.
Ordering: the frame's rows are written before its attr, and the
encoder takes the attr as the signal that a whole frame is behind it.
The attr ring's release/acquire orders the video ring's publication
before it, so the encoder's read of the frame sees all of it - which
the old code got from the lock instead.
The push's room check asked only the attr fifo, relying on the two
being sized in step; the rings round up to powers of two
independently, so it now asks both. fifo_new() was never checked;
a ring that fails to init fails the recorder. The volatile alive
flag becomes a retro_atomic_int_t: TSan flagged the encoder loop's
read against deinit_thread()'s clear.
Verified by samples/record/ffmpeg_pipeline (next commit): 240 frames
whose pixels encode their number, pushed with the encoder draining
concurrently, decoded back in order and pixel-exact; clean under
ASan+UBSan and TSan. The same harness passes against the fifo
version, with the alive race as TSan's one finding there.
System Information listed Pacing, Display Width (mm), Display Height
(mm) and Display DPI between the video context driver and the
feature table, and Display Information showed the same metrics again
as a combined size line. They are properties of the display in use,
so they now live only under System Information > Display Information,
with their original labels (the existing translations carry over) and
Pacing alongside them. The two combined-line strings that Display
Information had introduced go away unused.
When the chunk to fill was the one queued for DMA or the one being
played, gx_audio_write() spun on the two volatile indices with no
sleep and no exit: a pure busy-wait on a single-core machine, and an
endless one once the DMA stopped calling back - stop() and pause set
is_paused, free() sets stop_audio, and a blocking write in flight
then never returned. In non-blocking mode it did not wait, but it
did not stop either: it went on and wrote the chunk over the one the
DMA was playing, which the old FIXME above it admitted.
Wait the way gx_audio_wait_writable() already does: sleep on the DMA
callback's thread queue, one chunk at a time, and leave the loop -
returning the bytes that went in - when non-blocking, stopped or
paused. A callback landing between the check and the sleep costs one
extra chunk of waiting, as in wait_writable().
No devkitPPC here: syntax-checked against stubbed libogc headers,
not built or run.
The ring is counted in uint32_t frames - write_pos, read_pos and
AUDIO_BUFFER_SIZE all are - but both room checks in psp_audio_write()
compared that frame count against len, which is bytes, so they
demanded four times the room the write needed. In non-blocking mode
(fast-forward) a write was refused and its audio dropped while there
was plenty of space; in blocking mode it waited for space rate control
was not trying to free. psp_write_avail() and psp_wait_writable()
already convert between the two; compare frames to frames here too,
using the sample_count the copy below already uses.
No PSP/Vita toolchain here: syntax-checked against stubbed headers,
not built or run.
sysLwCondWait() requires the calling thread to own the lwmutex the
lwcond was created with - cond_lock here - and fails at once (EPERM)
otherwise. Neither wait in this driver ever took it, so on a full
fifo the "bounded wait" in ps3_audio_write() and ps3_audio_wait_
writable() was PS3_AUDIO_WAIT_LAPS immediate failures back to back,
a few microseconds in total, after which the write returned 0 and the
audio was dropped. Blocking mode never actually blocked.
Wrap the wait in ps3_audio_wait_block(), which locks cond_lock around
it. The output thread's sysLwCondSignal() does not need the mutex
and is unchanged.
Also take the fifo lock for the room checks in ps3_audio_write() (via
ps3_audio_write_avail(), which already does) instead of reading the
fifo's cursors bare while the output thread moves one of them, and
check fifo_new() in init, which was never checked.
No PS3 toolchain here: syntax-checked against stubbed PSL1GHT
headers, not built or run.
The macOS display server left get_edid NULL. IODisplayCreateInfoDictionary
has published the raw block under kIODisplayEDIDKey since 10.0, and
IOKit is already linked for the power-source query, so read it: walk
the IODisplayConnect services for the one whose kDisplayVendorID,
kDisplayProductID and kDisplaySerialNumber match what CoreGraphics
reports for the display the RetroArch window is on (its NSScreen's
NSScreenNumber, the main display without a window), and copy the
block in whole 128-byte units. Matching on the identifiers rather
than CGDisplayIOServicePort keeps this off an API deprecated in 10.9,
and the serial only breaks a tie between two identical monitors when
both sides report one.
That covers every PowerPC and Intel Mac. On Apple Silicon the display
stack moved to DCP, the IODisplayConnect services do not exist and
Apple published no replacement, so the walk finds nothing and the op
reports -1: the menu shows "Not available" there, as before.
Two new information lists. Display Information shows the display in
use as the display server sees it: server, each output with its size
and placement, desktop resolution, refresh rate, physical size, DPI
and orientation. Under it, EDID decodes the block the server read
off the display: version, manufacturer, product, serial, date,
monitor name, video input, screen size, gamma, features,
chromaticity, range limits, established and standard timings, every
detailed timing as a summary line and an xorg modeline, the base
checksum, each extension block with its CTA-861 capabilities, video
and audio formats, HDMI, HDR static metadata and colorimetry or its
DisplayID sections, and finally the raw bytes sixteen to a line for
pasting into edid-decode. "Not available" when the server cannot read
one.
Both lists sample the hardware, so they join the information lists
the displaylist fingerprint harness skips.
A new optional op on video_display_server_t, get_edid(data, out, max),
copies the EDID of the display the RetroArch window is on into the
caller's buffer in whole 128-byte blocks and returns the length, or
-1 when the server has no way to read one. video_display_server_get_edid()
wraps it on the current server.
KMS reads the connector's EDID property blob through libdrm, and
falls back to the kernel's sysfs copy when the context is down.
X11 reads the XRandR "EDID" property of the output under the window
(the first connected one when there is no window yet), and does not
fall through to another head's block when that output has none: the
EDID shown must be the display in use. The DDX only relays the
property when it chooses to, so sysfs stands in there too, and it
is the whole route without XRandR. Wayland has no protocol for the
EDID at all; the server now binds wl_output at v4 when the
compositor offers it to learn the connector name, and reads the
sysfs node for it. Win32 asks EnumDisplayDevices for the monitor's
device interface name and reads the PnP monitor's registry EDID
value, which monitor.sys has kept at the same place since Windows
2000; the value is sized first because the registry API refuses a
partial read. The sysfs reader is a header-only helper shared by the
three Linux servers. Apple, Android, UWP and the SDL servers leave
the op NULL.
The KMS sample stubs the six libdrm property calls and checks that
get_edid takes the blob named EDID (not the other blob), copies whole
blocks into a smaller buffer, refuses one too small for a block, and
frees every object it fetched. The X11 sample fabricates the output
property and checks the same, plus the no-property and error cases.
The other direction of the CRT-preset EDID writer in
gfx/modeline/modeline_edid.c: decode a block a display server pulled
off a real monitor into what the menu can show.
modeline_edid_parse() covers the EDID 1.0-1.4 base block - vendor,
product, serial, manufacture date (1.4 model year), video input with
the 1.4 bit depth and interface codes, screen size, gamma, feature
flags, chromaticity, established and standard timings, and the four
descriptors: monitor name, serial string, unspecified text, range
limits with the 1.4 +255 offsets, additional standard timings, and
detailed timings - plus the extension blocks a modern display
appends. CTA-861 yields the capability flags, short video and audio
descriptors, the HDMI and HDMI Forum vendor blocks (physical address,
max TMDS), HDR static metadata, colorimetry, 4:2:0 presence and its
DTDs; DisplayID 1.x/2.0 yields the section tags and type I / type VII
timings. Formatters give the menu a timing summary, an xorg-style
modeline, VIC names for CTA-861-H codes 1-107, established timing
names and extension / section names.
Every length and offset in the data is treated as hostile: data block
lengths, DTD offsets and DisplayID section sizes are clamped to their
block, extension walking stops at the declared count and the data's
end, and a truncated or corrupt block returns whatever could be read
with the header, checksum and truncation flags set rather than a
parse failure.
samples/gfx/modeline_edid_parse covers the writer-to-reader round
trip, a hand-built EDID 1.4 + CTA-861 + DisplayID 2.0 block field by
field, the defect cases and a deterministic byte-noise sweep, and
runs under ASan/UBSan in the Linux-samples-gfx lane. The gold
harness includes this file and its reference is unchanged.
The queue between audioworklet_write() (main thread) and the worklet's
process callback was a fifo_buffer_t under an emscripten_lock_t that
both sides took: the worklet with a bounded busy-spin, since it cannot
Atomics.wait, and it played a period of silence ("could not acquire
lock") whenever it lost the spin to a write in progress; the writer
with a timed wait that dropped the rest of the frame when it lost.
Alongside it a separate write_avail_bytes atomic mirrored the fifo's
room for the lock-free readers (write_avail(), the external-block
waits, the busy-wait spin).
The queue is single-producer/single-consumer. Replace it with a
retro_spsc_t: the writer no longer takes buffer_lock at all, and the
worklet's pull can no longer collide with a write. The mirror atomic
goes away too - retro_spsc_write_avail() is exact from the producer
side, and every reader of the mirror was on the main thread - and
audioworklet_room() measures it against the size asked for (visible
buffer plus the external-block reserve), since retro_spsc rounds up
to a power of two. Every consumer of the old value keeps its
semantics: write_avail() still subtracts write_avail_diff on the
external-write-block build, the fake-block threshold and the async
and busy-wait waits compare the same quantity they did.
buffer_lock stays, for lifetime only: the worklet takes it around its
pull and audioworklet_free() around tearing the ring down, so a pull
can never read a freed ring. Contended only at free, so the
worklet's spin effectively never loses now. The PROXY_TO_PTHREAD
condvar signal after each pull is unchanged.
fifo_new() was never checked; a ring that fails to init now fails
the driver - the reuse path returns NULL, the first-init path sets
init_error so both the waiting and the external-block init paths take
their existing error exits.
Compiled with emsdk under every audioworklet configuration
Makefile.emscripten produces: PROXY_TO_PTHREAD, the default
(EXTERNAL_BLOCK + FAKE_BLOCK + EXTERNAL_WRITE_BLOCK), ALLOW_AUDIO_
BUSYWAIT, ASYNC=1 (full asyncify) and MIN_ASYNC=1 (ASYNC_BLOCK).
retro_atomic.h selects its __atomic backend under emcc, so the ring
is lock-free there. Not run in a browser.
RETRO_ENVIRONMENT_SET_MESSAGE gives its duration in the core's frames,
so the widget path converts it at the core's frame rate, as
SET_MESSAGE_EXT does, falling back to 60 when none is set.
Under the threaded video wrapper, video_driver_frame() stages the
panels' text (FPS, frame count, memory, core status, time) with
video_thread_status_text(), video_thread_frame() copies it into the ring
slot of the frame it belongs to, and gfx_widgets_worker_step() takes it
from there, so the main thread neither writes widget state nor takes
the widget lock for it each frame. Without the wrapper the frame path
is unchanged.
The Threaded Video help text describes what the setting does now, and
the samples that link gfx_widgets.c stub the handoff.
da4e4ca moved sdl_audio's queues onto retro_spsc but this sample's
Makefile still listed fifo_queue.c as the ring behind the driver, so
the link failed on every retro_spsc_* symbol. Swap the source; none
of the other drivers this sample builds (alsa, openal) use
fifo_queue.
Built and run with SANITIZER=address,undefined; passes.
audio_driver.c calls command_event(CMD_EVENT_DSP_FILTER_INIT) in
audio_driver_init_internal() but never included command.h itself; it
reached it through input_driver.h, which only includes command.h
under HAVE_COMMAND. Makefile.common defines HAVE_COMMAND only when
configure enables the network command interface, so a build with
--disable-networking failed on
audio_driver.c: implicit declaration of function 'command_event'
audio_driver.c: 'CMD_EVENT_DSP_FILTER_INIT' undeclared
command_event() itself is unconditional (retroarch.c); only the
declaration was missing. Include the header the way every other
caller does. Compiled with and without HAVE_COMMAND.
Two places on the pipe consumer drop audio rather than play it: the
underrun trim, which pulls the ring back down to its target after
the driver reports an underrun, and the paused path, which keeps the
ring flowing for the producer while nothing goes to the device. Both
did it with retro_spsc_read() into pipe_scratch/pipe_wide, copying
up to a pass of frames out purely to throw them away.
retro_spsc_skip() advances the consumer's cursor without touching
the bytes; use it. Same publication (release-store on tail), no
copy.
Every write() acquire-loaded the consumer's tail and every read()
the producer's head: one pull of the other core's cache line per
operation. Each side now keeps a private copy of the remote cursor
on its own line (cached_tail next to head, cached_head next to tail;
the pads shrink to match, sizeof is unchanged) and computes room /
availability from that, re-reading the real cursor only when the copy
says there is not enough for the request (write, read, peek, skip)
or none at all (write_begin, read_begin). The remote line is then
touched once per fill or drain instead of once per op.
The copies are always behind the real cursor - tail and head only
grow, and the producer never advances head past cached_tail +
capacity, the consumer never advances tail past cached_head - so
room and availability are only ever under-stated, never over, and
the acquire/release pairing on the re-read is the one that was there
before. write_avail()/read_avail() keep reading both real cursors:
they are the exact queries, and audio_driver.c calls read_avail()
from the producer side.
Measured with samples/queues/retro_spsc_bench on a Windows desktop
(two threads on separate cores), ns per write+read:
payload before cached
8 8.5 2.7
32 8.5 2.8
128 16.6 8.2
1024 32.0 29.6
4096 115.2 114.3
16384 419.9 407.5
3x at message sizes, noise once the memcpy dominates.
Compile-checked under RETRO_ATOMIC_FORCE_{C11,CXX11,GCC_NEW,SYNC,
VOLATILE}, FORCE_MSVC via mingw-w64, and gnu89. retro_spsc_test
passes under TSan and ASan+UBSan.
Every retro_spsc_write() loads the consumer's tail and every read()
loads the producer's head, so each operation pulls the other core's
cache line across. The usual next step for an SPSC ring is for each
side to keep a private copy of the remote index and only reload it
when the copy says the ring is full (producer) or empty (consumer);
the remote line is then touched once per fill/drain rather than once
per operation.
Whether that is worth carrying in retro_spsc depends on the hardware
- the cost is coherence traffic, so it is invisible on one core and
varies with the interconnect - so before changing the ring this adds
a harness that measures it: the library's retro_spsc against an
in-file copy with cached indices, one producer thread and one
consumer thread, payloads from 8 bytes to 16 KiB, reporting ns per
write+read and MB/s. Run it on a machine with two physical cores
and compare the rows; if "cached" is not clearly faster on the small
payloads the change is not worth making.
Build-only in the sample CI: a runner is not two dedicated cores and
the numbers there would only mislead. Builds with gcc and
mingw-w64.
The capture callback ran on the audio unit's IO thread and wrote into
a fifo_buffer_t under a mutex it would only slock_try_lock(): if the
core thread happened to hold the lock in coreaudio_mic_read() or
coreaudio_mic_wait_readable(), the whole slice was dropped - a hole
in the captured audio every time the reader and the callback
coincided.
The queue is single-producer/single-consumer, so replace it with a
retro_spsc_t. The callback now never waits and never drops for
contention; it only skips a slice when the ring is genuinely full,
as before. The reader no longer holds the lock across the read
either. lock/cond remain solely for the reader's timed wait, and the
callback signals the cond without the lock as it did before.
The three fifo_clear() sites (start, stop, default-device reconnect)
become coreaudio_mic_drain(), which skips read_avail() bytes from the
consumer side. That is safe against a live producer, unlike
retro_spsc_clear(), so it does not depend on AudioOutputUnitStop()
having joined the IO thread before returning.
retro_spsc rounds capacity up to a power of two; ring_size keeps the
latency-derived size and the callback measures room against it, so
the buffered depth is what it was.
No macOS toolchain on this box: the file was syntax-checked with the
Apple headers stubbed out and none of the remaining errors touch the
changed code, but it has not been built or run for real.
The fifo between rs_write() (core thread) and rsound_audio_cb()
(librsound's worker) was a fifo_buffer_t guarded by librsound's
rsd_callback_lock(), so every write from the core serialised against
the server's pull. It is single-producer/single-consumer; replace it
with retro_spsc_t and drop the callback lock around every queue
access, as sdl_audio, dsound and asio already do. cond_lock and the
bounded cond waits are unchanged.
retro_spsc rounds capacity up to a power of two; fifo_size (what
buffer_size() reports) is kept and rs_ring_room() measures room
against it, so rate control sees the same occupancy as before.
rs_init()'s error path freed only librsound's handle: a failed
rsd_start() leaked the condition pair (and the old code never checked
fifo_new() at all). The label now frees the ring, cond, cond_lock,
the handle and the struct.
Compile-checked against the in-tree rsound.h; librsound is not on
this box so not run.
Both queues in this driver are single-producer/single-consumer: the
core thread writes samples in sdl_audio_write() and SDL's playback
thread drains them in sdl_audio_playback_cb(); SDL's capture thread
pushes in sdl_audio_record_cb() and the core thread pulls in
sdl_microphone_read(). They were fifo_buffer_t rings guarded by
SDL_LockAudioDevice(), which means every write from the core stalled
the device callback (and vice versa), the same shape dsound and asio
had before they moved to retro_spsc.
Replace both with retro_spsc_t and drop the SDL_LockAudioDevice()
around every queue access. The condition variables and their bounded
waits are unchanged: the writer still sleeps on sdl->cond when the
ring is full and the callback still signals after each pull.
retro_spsc rounds capacity up to a power of two, so the requested
size is kept alongside the ring (speaker_ring_size / ring_size) and
sdl_ring_room() reports room against that, not the physical
capacity. sdl_audio_buffer_size() returns the requested size, and
the producer never fills past it, so rate control sees exactly the
occupancy it saw with the fifo. The pre-fill of one buffer of
silence at init is kept.
SDL1 (HAVE_SDL) builds are unchanged in behaviour: the SDL1 shim
macros are kept as they were and the non-SDL2 speaker path uses the
same ring.
Compile-checked: SDL2 with and without HAVE_THREADS/HAVE_MICROPHONE,
SDL1 with and without HAVE_THREADS. Not run against a device here.
Follow-up to 949c196, which only touched *_end(). The same argument
applies to the other side-bound operations: retro_spsc_write() and
write_begin() run on the producer and can read head relaxed; read(),
read_begin(), peek() and skip() run on the consumer and can read tail
relaxed. The acquire on the *remote* counter - tail for the producer,
head for the consumer - is untouched, so the release/acquire pairing
that orders the data copies is intact.
On the C11/GCC backends this drops a fence on weak-memory targets; on
MSVC/Apple/__sync it removes a locked RMW (InterlockedCompareExchange
or barrier'd fetch_add(0)) from every write, read and span begin.
retro_spsc_write_avail()/read_avail() deliberately keep both loads
acquire. The header documents them as producer-only/consumer-only,
but audio_driver.c calls read_avail() from the producer side as a
fill-level query (audio_driver_sink_pipe_frames, rate control, the
empty-ring format switch), so a relaxed own-counter load there would
be relaxed on the wrong thread. They are not hot; leave them. Also
drops the now-stale "retro_atomic.h does not expose relaxed loads"
comment.
Compile-checked under RETRO_ATOMIC_FORCE_{C11,CXX11,GCC_NEW,SYNC,
VOLATILE} and FORCE_MSVC via mingw-w64; samples/queues/retro_spsc_test
passes under TSan.
retro_spsc_write_end() published the new head by acquire-loading head
and adding to it; retro_spsc_read_end() did the same for tail. Both
counters are single-writer: head belongs to the producer and tail to
the consumer, so each *_end() is re-reading a value its own thread was
the last to store. No ordering is needed for that read.
This matters more than it looks: on the MSVC backend the acquire load
is InterlockedCompareExchange (a lock-prefixed RMW), and on the Apple
OSAtomic and __sync backends it is a barrier'd fetch_add(0). So every
span commit was paying for a locked instruction on its own cache line.
Add retro_atomic_load_relaxed_size() to every backend (C11/C++11/GCC
__atomic: memory_order_relaxed; MSVC/Apple/__sync/volatile: a plain
read of the volatile native-width typedef, which is guaranteed
tear-free on all of those targets) and use it in both *_end() paths.
The release store that actually publishes the counter is unchanged,
so the acquire/release pairing with the other side is intact.
Compile-checked with RETRO_ATOMIC_FORCE_{C11,CXX11,GCC_NEW,SYNC,
VOLATILE} and, via mingw-w64, FORCE_MSVC. samples/queues/
retro_spsc_test passes under TSan and ASan+UBSan.
linked_list: linked_list_add(), linked_list_insert() and
linked_list_iterator() dereferenced malloc() results without a NULL
check. Bail out on OOM instead (list unchanged / NULL iterator),
matching the treatment generic_queue already received.
string_list: string_list_capacity() zeroed the entire newly grown half
of the element array on every doubling. Slots in [size, cap) are never
read - string_list_free() walks [0, size) and the append paths fully
initialise a slot when it becomes live - so the memset was pure
bandwidth (hundreds of thousands of elements cleared to add one entry
on large lists). Remove it and have string_list_append()/_append_n()
set userdata = NULL explicitly so string_list_free() cannot free
garbage. string_list_clone() already calloc()s its own array and every
external writer of elems[].userdata targets size - 1, so nothing relied
on the pre-zeroing.
Also guard the cap * sizeof(elem) multiplication in
string_list_capacity() and the cap * 2 doubling in the append paths
against size_t overflow.
gfx_widgets_sanitize and gfx_widgets_task_bar_stack link the real
gfx_widgets.c, which calls the widget variants of the animation API and,
for gfx_widgets_worker_step(), config_get_ptr() and
cpu_features_get_time_usec(). Their stubs provide those in place of the
plain animation calls the file no longer makes.
gfx_widgets_task_bar_stack runs in the samples workflow under ASan and
UBSan and under TSan, like gfx_widgets_sanitize, so a change to what
gfx_widgets.c links against breaks CI instead of the sample alone.
Under the threaded video wrapper the worker owns the widgets' per-frame
work, as the audio thread owns the threaded audio pipeline: before each
driver frame it ticks the widget tweens, kept in an animation instance
of their own, and runs the widgets' iterate() and message queue. The
main thread keeps the layout, which builds and retires fonts, and the
widget setters, both under the widget state lock, and no longer ticks
widget tweens. The tweens change hands at widget init and deinit, so
they carry on across a runtime toggle; without the wrapper they stay in
the main list and tick with it, and the frame path is as before.
gfx_animation takes an instance internally, keeps its update
bookkeeping per instance and its public API unchanged, and gains widget
variants of push, kill, timer and ticker. The achievement and
leaderboard entry points take the widget state lock like the other
setters, and gfx_animation_sanitize gains a lane for the hand-over.
The CRT consumer captured the display server instance pointer at
engine init. A content load or close goes through the main deinit,
which clears the runloop inited flag, so the next video init destroys
that instance and builds a new one; the win32 modeline_close puts the
desktop timing back on the wire and the consumer's ops table points
at freed memory. Every later set read ml->opened from the fresh
zeroed struct, returned false before logging, and the screen stayed
on the desktop mode (480i on CRT Emudriver) while the log showed the
modeline calculated and "Error switching" with no CDS call.
video_display_server_destroy now tells the consumer the instance is
going away: the ops table is dropped, current is cleared and the
change detection is reset. On the next switch the engine rebinds to
the server that is up and applies the mode again through a real set.
Fixes#19528
retro_task_threaded_wait() and retro_task_gcd_wait() looped on
gather() with nothing in between until the task they were waiting on
retired. That pinned a core for the whole wait: profiling a CLI
--scan showed the main thread at 50% of the process, all of it taking
and releasing running_lock and finished_lock in the loop. The same
loop serves the blocking save/load-state paths in task_save.c.
Add finished_cond, signalled by both workers when a task moves onto
the finished queue, and park the waiter on it between passes. The
wait is bounded at 1 ms so the states the signal does not cover - the
caller's own condition flipping, a task that keeps yielding without
finishing - still resolve promptly; a task already sitting on the
finished queue is not waited for at all, gather() retires it on the
next pass.
--scan of the PC Engine set against all 147 rdbs: 78 ms task-clock
for 79 ms elapsed (0.99 CPUs); the second core is idle. Wall time is
unchanged. Playlist output byte-identical. TSan clean on the scan,
ASan+UBSan+LSan clean.
Same change belongs in the standalone libretro-common repository.
With the threaded video wrapper the worker draws the widgets during
content while the main thread animates, iterates and sets them. A
widget state lock covers both sides: the worker holds it across
gfx_widgets_frame(), the main thread across gfx_animation_update(),
gfx_widgets_iterate(), the status text copy in video_driver_frame() and
every public widget setter. It is recursive for its owner and does
nothing without the wrapper.
A writer that loads or frees a texture waits on the worker, which may
be blocked on the lock mid-draw, so the wrapper releases the lock for
every wait on the worker - the user command slot and
video_thread_wait_idle() - and retakes it afterwards. The compile
matrix gains a threaded widget lane and a threadless one for the macro
stand-ins.
Every hit answered from the crc or serial index opened the database
twice: libretrodb_open() to read the header, then libretrodb_read_at()
opened a second, buffered stream with a 64 KB window to fetch the one
record at the known offset, and both were closed again. On the
Master System set that was 1214 opens and 36 MB of reads against a
300 KB database for 607 matches.
Keep one buffered stream (16 KB window - the records read back are a
few hundred bytes) on the index, opened on the first hit and closed
with the index. Opening lazily means only databases that actually
match content hold a file handle; a scan can index every claimed
database and console fd tables are small. The lookups now take a
non-const index for that reason.
Measured on the Master System set against all 147 rdbs: opens of the
SMS database 1216 -> 3, read() calls 10559 -> 7687, playlist output
byte-identical. The wall time was already dominated elsewhere after
fbe3d2b246, so this is fewer syscalls rather than a visible speedup.
database_index_test (14 checks), database_scan_test (10) and
scan_begin_budget_test pass; ASan+UBSan+LSan clean.
task_database_iterate_crc_lookup() consulted the size gate before the
core-info gate. The size gate is not free: the first time a database
is consulted, task_database_fill_db_min_max() walks the whole .rdb to
build its crc index and derive the size range. The core-info gate is
an in-memory string comparison. Doing them in that order meant every
database in the directory was read end to end once per scan, whether
or not any installed core claims it - 201 MB of .rdb reads to scan a
71 MB Master System set against the shipped 147-file database
directory, most of it the 47 MB J2ME database that can never match a
.zip of SMS ROMs.
Run the cheap gate first. Databases no core claims are now skipped
without being opened.
Measured, --scan of the reporter's sets against all 147 rdbs, release
build, same machine:
before after
SMS (620) 6.76 s 0.19 s
PCE (412) 6.70 s 0.16 s
Lynx (95) 1.28 s 0.05 s
Playlist output is byte-identical to the pre-change build. The gain
scales with how many databases the installed cores do not claim; a
setup with every core installed still walks each claimed database
once, as before. ASan+UBSan+LSan clean.
Tatsuya79, on the Qt settings dialog after 9eaa3750bd: "that's a
concise tip" - the Threaded Video sublabel rendered as one line the
width of a 2560 px desktop.
Qt shows a plain-text tooltip on a single line whatever its length
and only wraps rich text, so every tip now goes in as escaped rich
text (<qt>...</qt>), which Qt wraps to its usual tooltip width. The
core-option tips, which word_wrap() already breaks at 50 columns,
keep their breaks inside <pre>.
The Win32 list-view info tip has the same habit until its tooltip
control is given TTM_SETMAXTIPWIDTH; both the settings and core
options lists now set one (420 px at 96 DPI, scaled). Cocoa wraps
tooltips on its own.
Checked: Qt6 build, tools/companion_qt_persist_test.sh,
tools/companion_win32_test.sh under Wine, mingw_syntax_check.
The windowed session kept a fixed 8 MiB ahead of and 8 MiB behind the
decoder on top of the 4 MiB resident head - 20 MiB per playing
preview for any file longer than that, regardless of what it holds.
For a 4 Mbit/s trailer that is 16 s of video each way, resident for
nothing: the feed refills every animate() tick with a 512 KiB budget,
so a lookahead of a couple of seconds is many ticks of slack, and
the bytes behind the cursor are never re-read (a loop's rewind drops
the frontier back to the head, which is resident anyway).
The video types now name their duration (rwebm_video_stream_/
rmp4_video_stream_duration_ns, image_transfer_anim_stream_
duration_ns; 0 for the frame-indexed types, which carry per-frame
delays only). At open the session sizes the moving window to the
bitrate: 2 s ahead, 500 ms behind, floors of 1 MiB / 256 KiB for a
low-bitrate file, the old constants as ceilings. The open primes the
same lookahead instead of the fixed one. WEBP and APNG keep the
ceilings. Admission still charges the fixed span, as an upper bound.
Measured, samples/gfx/gfx_thumbnail_preview on a new fixture
(long_video.mp4: 60 s, 640x360, 4 Mbit/s, 29 MB, generated by
mkfixtures.py), all 1800 frames through the session in both cases:
peak resident, session count and mincore over the reservation
fixed 4+8+8: 20.00 MiB
bitrate-sized: 5.25 MiB (head 4 + ahead 1 + back 0.25)
decode+feed, -O2, three runs each
fixed: 1.09-1.11 ms/frame
bitrate-sized: 1.08-1.12 ms/frame
No stall either way; the shorter window costs no throughput. The
frame-indexed fixtures (W1-W6, A1-A2, S1) are unchanged.
data_transfer_window_resident() is added for the number: the head
plus the moving window as a count, deliberately not a position, so it
gives a second consumer nothing to read by. gfx_anim_preview_
resident_bytes() exposes it per session; the preview harness prints
it, and preview_anim_window_test gains V1, which plays the new fixture
end to end and requires the resident bytes to stay inside head +
window and below the fixed span. The window test now links the MP4
decoder to do so.
Also: the "(WEBP)" note on gfx_thumbnail.c's upload swizzle was
stale - every stream honours the order request, the swizzle is a
fallback none reaches.
Checked: mingw_syntax_check on every touched .c; preview harness
(video and window tests), gfx_thumbnail_anim harness; full Qt build
links. The threaded_video harness was not re-run (it needs the
non-Qt build); nothing in the wrapper changed.
Reported by Tatsuya79: the automatic directory scan stopped on the
first or second file of every set since early August, while the same
folder scanned to the end from the CLI on another machine and via
Manual Scan. His log shows the loop:
[Scanner] Type 6, ".../20 em 1 (Brazil).zip" against ".../FB Alpha - Arcade Games.rdb".
[Scanner] Size: min -1 actual 262144 max -1.
[Scanner] Scan verdict is 0 for .../20 em 1 (Brazil).zip
repeating every tick, forever.
Two defects combine:
1. libretrodb_open() rejects the 2015-2016 era .rdb files. Those
writers never rewrote the header, so metadata_offset is 0. The
old rmsgpack_dom_read_into() silently returned 0 when the value
at offset 0 (the header's own bytes) was not a map, leaving
md.count uninitialised but the open "successful"; the hardened
reader correctly returns -1, so the open now fails. Users whose
database directory predates upstream renames still have such
files (the online updater never deletes removed ones; "FB Alpha -
Arcade Games.rdb" was renamed to FBNeo in 2019), and their
records are perfectly readable - the cursor walks from the header
to the RDT_NULL sentinel and never uses the count. Accept a zero
metadata_offset as "no metadata, no indexes" instead of failing.
2. task_database_iterate_crc_lookup() and the serial lookup ran the
query at entry_index == 0 and then, with a NULL info, fell through
to entry_index++ and SCAN_VERDICT_CONTINUE without ever touching
list_index. The next tick re-entered with entry_index != 0,
skipped the query, and returned CONTINUE again. Any database the
scanner cannot open or query therefore hung the scan on the first
content file that reached it - and only if a core claimed that
database, which is why one machine hung and another did not.
Treat an unanswerable database as one with no match and advance.
The dispatcher's SCAN_VERDICT_ERROR case had the same shape
(log, then re-run the identical step next tick); it now moves to
the next content file.
Also guard the DEBUG-only size log in task_database_iterate(): it
read min_sizes[list_index] on the "against (null)" tick where
list_index == list->size, one past the arrays.
Verified on the reporter's three no-intro sets (Lynx 95, PCE 412,
SMS 620 members) with the 2016 FB Alpha rdb in the database list and
a core claiming it: reproduced the spin on master (3M ticks in 30 s
on the first file), and with this change the scan completes with the
same playlist output as the last-good build, from --scan and from
the menu, with the task queue threaded and unthreaded, under
ASan+UBSan+LSan.
rcheevos_get_badge_texture() read the PNG, decoded it and uploaded it
inline on the thread that asked - a file read plus a decode, and under
threaded video a present's worth of waiting for the upload - at the
moment an achievement unlocked during gameplay. Every caller already
copes with a 0 answer, because badges can still be downloading: the
menu re-asks after MENU_BADGE_RETRY_RELOAD_FRAMES frames, the popup,
challenge and progress widgets every 250 ms. So the load moves off
the caller's thread with no change at the call sites.
New cheevos/cheevos_badge.c: a 16-slot round-robin table keyed by
badge file name. The first call for a badge on disk posts an image
task and returns 0; the decode's completion posts the upload through
video_driver_texture_load_async() (synchronous when there is no
wrapper); the first call after the upload lands takes the handle, and
the handle is the caller's to unload, as before. A missing file
returns 0 and requests the download only with download_if_missing, as
before. A failed decode or upload frees the slot so the next ask
starts over. rcheevos_badge_cache_reset(), called from
rcheevos_menu_reset_badges(), unloads ready handles and bumps the
sequence of in-flight ones so a late delivery is unloaded rather than
handed out. Off the main thread the answer is 0 with no side effects
(the table and the task callbacks are main-thread only); the old code
returned 0 there too unless video was threaded.
The three download-by-kind helpers stay in cheevos_menu.c behind one
exported rcheevos_badge_request_download(), so the cache has no
rc_client dependency and can be linked alone.
samples/cheevos/badge_cache: the real cheevos_badge.c against
steppable stubs for the image task and the upload; eleven scenarios
including hand-over-once, separate lock key, failed decode, failed
upload, reset with a load in flight, off-thread, and a refused async
upload. Wired into Linux-samples-gfx.yml under ASan+UBSan with leak
detection. Full Qt build links; mingw_syntax_check clean on both
files; -Werror=implicit-function-declaration clean.
Badge appears up to one retry interval after the unlock instead of
stalling the frame it unlocks on; with the popup's slide-in that is
not visible. To confirm on hardware: unlock an achievement with
threaded video on and watch the frame-time graph.
Follow-up to 9eaa3750bd. Qt's Core Options dialog already shows each
option's info text from companion_core_option_info(); the Win32 and
Cocoa dialogs did not. Same mechanism as the settings rows: the Win32
options list gets LVS_EX_INFOTIP with an LVN_GETINFOTIP handler, and
the Cocoa options table is covered by the same toolTipForCell:
delegate.
Checked: tools/mingw_syntax_check.sh, tools/companion_win32_test.sh
under Wine, tools/companion_cocoa_test.sh under GNUstep.
Tatsuya79: tooltips in the Qt settings dialog stopped working; only
Video > Video Filters and a few achievement rows still had one.
Cause: d636ba135c / f5c0413682 turned most sublabels into a table.
menu_cbs_init_bind_sublabel() now stores the message in
cbs->sublabel_enum and binds action_bind_sublabel_from_enum(), which
reads it back through file_list_get_actiondata_at_offset(list, i). The
Qt dialog has no list - it always called action_sublabel() with NULL -
so every table-driven sublabel came back empty, and only the entries
that still have a function (video filter, the cheevos rows) kept
their tooltip.
menu_cbs_sublabel_for_enum() is the list-free entry point: it binds,
reads sublabel_enum when the sublabel is data, and calls the function
when it is code. add_sublabel_and_whats_this() uses it; every setting
row in the Qt dialog has its tooltip again.
The View > Options rows (the small desktop-menu settings the Qt, Win32
and Cocoa companions all render) never had help text at all. Thirteen
MENU_ENUM_SUBLABEL_QT_MENU_VIEW_OPTIONS_* messages are added, served
through companion_core_setting_sublabel(), and shown as tooltips on
all three: Qt sets them on each control and its form label, the Win32
options list gets LVS_EX_INFOTIP with an LVN_GETINFOTIP handler, and
the Cocoa options table answers tableView:toolTipForCell:.
Checked: Qt6 build of the tree, tools/companion_qt_persist_test.sh,
tools/companion_win32_test.sh under Wine, tools/companion_cocoa_test.sh
under GNUstep, tools/companion_core_test.sh (new: every row has its
own help message), mingw_syntax_check on the C files. Visual check of
the tooltips themselves on Windows and macOS still needed.
The bench lane measured wall-clock to a consumed result through the
shared per-frame I/O window. That window admits 4ms of work per
16.67ms period and then one item per task_queue_check() until the
period rolls over, so the deferred path's wall-clock is quantised to
whole periods (~34ms for the 40k fixture on every machine tried),
while the blocking reference scales with the machine. The ratio was
therefore grading runner speed against the period length: 27ms
blocking locally passed at 1.27, 21.8ms on the CI runner failed at
1.57 - a faster box fails harder.
Split the clock the test provides into the instrument the mechanism
reads and a real stopwatch the test measures with. The pacing rounds
keep the real clock and the stall bound. The bench rounds hold the
instrument frozen so no budget check ever sees the window exhaust,
and the deferred machinery (iterator, task hand-off, resumable
mergesort, main-thread callback) runs in as few invocations as the
queue allows. That number is the mechanism's cost - it measures
1.07-1.16x blocking here, the 10-15% the factor was written for -
and it no longer depends on where the runner's speed lands the job
relative to a period boundary.
The main thread bumps the font generation while the threaded video
worker reads it to decide whether widget metrics are stale, so the
counter is a retro_atomic_int_t, incremented with
retro_atomic_fetch_add_int and read with an acquire load.
x11_input probes for the XInput extension before querying devices.
Without an XInput 2 device list it reads the core pointer as a single
mouse, with buttons 4 and 5 taken from the button events, the same as
a build without XI2, so the mouse works on servers such as Xvnc.
The core pointer query is shared by both builds and stores each button
as a truth value, so clicks register where bool is an unsigned char.
Three things from the last run.
The mock's Destroy did not stop the player's callbacks, so its pump
kept calling into the handle the driver had already freed and ASan
reported a use-after-free at opensl_callback. The driver is right:
it stops the player, destroys the player object, and only then frees
its own state, which is the order OpenSL's contract asks for -
Destroy does not return until the object's callbacks have ceased.
The mock now models that, since a fake that outlives its contract
reports the driver for its own fault.
emscripten_mmap's check-wasm still ran whatever node the PATH had,
because reading NODE_JS out of emcc's config did not match its
format on the runner. It no longer parses anything: it tries the
node the caller named, the one emsdk exports, the one emcc records,
then node and nodejs, and asks each for its version before using it -
a name on the PATH is not enough when the entry is there and not
executable. With none that runs it says so and skips, rather than
failing a build whose wasm compiled fine; the sample's native check
covers the same code either way.
And the block_frames removal left a blank continuation line where
the parameter had been, in seventeen files. Tidied.
Not from this session: samples/menu/dirwalk reports its deferred
walk at 1.57x the blocking reference against a 1.35x limit. That is
the menu's own bench and its own lane.
The EINTR retry declared its poll result after a statement, which
the C89 build refuses. Both loops declare it at the top of their
block now. Checked with -std=gnu89 -Werror=declaration-after-
statement, which is what that build uses and what the gnu99 check
this went in under does not.
emscripten_mmap's check-wasm ran a bare "node". On a runner that is
whatever the PATH has, and there it is an entry with no execute bit:
the build succeeds and the run dies with "Permission denied". It
takes the node emcc itself uses now, read from --em-config, with a
PATH node as the fallback for a machine without the config, and says
which node it could not use rather than letting make report a bare
127.
The video thread reads thr->poke for every queued upload, in
video_thread_async_run. The asynchronous-upload lane swapped
thr->driver and thr->poke from the main thread while that thread was
running, which TSan reports as a race between the two - and it is
one, whoever writes it: the header says both lists and the state the
thread walks are guarded by thr->lock, and the lane is reaching into
the wrapper's own state without it. Every other place in this test
that touches those fields already takes the lock; this one did not.
Not run here - the harness links a whole tree build - so it is the
lane's own TSan run in CI that confirms it.
tinyalsa_write_test includes audio/drivers/tinyalsa.c, which the
consolidation into one alsa.c removed, so the sample stopped building
and the job stopped at it. What it checked is worth keeping: the
write and wait loops in the states no real card gives on demand - a
full device refusing in both modes, a device that never signals, a
poll cut short by a signal, and the first write reporting what it
took. The loops it checked are gone, but the embedded driver that
replaced them has loops of its own with the same properties to break,
so the cases move to ealsa_test, which drives them through the
syscall seam rather than through -Wl,--wrap. Its mock gains a device
that takes nothing and never signals, a poll that returns EINTR, and
a count of polls, which is what says a signal was retried past rather
than taken for an answer.
Moving them found one: both loops treated any poll that did not
return a descriptor as the device saying no, so a poll cut short by a
signal ended the write short. RetroArch takes signals for its timers,
so that is a short write for no reason, on a device with room. EINTR
is retried in both loops now, and the case asserts the loop polled
past three of them rather than stopping at the first.
menu_shader_manager_save_preset_internal() hands preset_path to
menu_shader_manager_set_preset() after the directory loop, and for a
relative preset name preset_path points at the joined path buffer, so
that buffer is declared at function scope rather than inside the loop's
block. The frame size is unchanged.
The Qt shader parameter dialog's checkbox handler marks the video shader
as modified only when there is one, as the slider and spin box handlers
already do; during a context reset the dialog has no video shader.
On an X server without GLX the GL drivers bring up the egl_x context,
whose get_proc_address resolves desktop OpenGL entry points through EGL
the same way it does for GLES, so gl and glcore get a complete symbol
table. Fixes#16855.
x11_input treats a NULL device list from XIQueryDevice() as a server
without XInput 2 and skips XIFreeDeviceInfo() on teardown, and
x_input_poll releases the button mask XIQueryPointer() allocates for
the caller on every poll.
Nothing reads it any more. The number it held is the device's
transfer granularity, which the platform reports and
audio_driver_device_block_frames() hands to the driver that wants
it, so the setting had nothing left to do: the field, the menu row,
the configuration row, the label and the thirty-nine translation
entries that named it are gone.
With the setting gone the block_frames parameter in
audio_driver_t::init() has no source, so it goes too - out of the
interface, out of the threaded wrapper that carried it through, and
out of all thirty drivers, twenty-seven of which had only ever cast
it to void. That is the uniformity: every driver now takes a device,
a rate, a latency in milliseconds and a rate to report back, sizes
its buffer from the latency and what the device will do, and says
what it got through buffer_size().
g_defaults.settings_out_block_frames stays. It is where the platform
puts the fact - Android from OUTPUT_FRAMES_PER_BUFFER, or from the
command line - and it is what the accessor reads. A user who needs
to override the burst still can, at the platform's edge, where a
device fact belongs.
The harnesses follow: the driver fakes lose the parameter, and the
ones that link audio_driver.c gain the g_defaults the accessor reads.
Every audio sample builds and passes.
open_devices() and udev_open_joystick() report the errno of a failed
open, so a device node that exists but rejects the open is visible in
the log instead of silently dropping out of the enumeration. The input
driver counts the nodes that came back EACCES and, when nothing at all
opened, follows the existing warning with one naming the cause: seat
ACLs do not cover keyboards and mice on most distributions, so read
access to them has to come from elsewhere, typically the "input" group.
Diagnostics only - which devices are opened, and what the driver does
when none are, is unchanged.
Every audio driver but three takes a latency in milliseconds, sizes
its buffer from it, and reports what it got. The three took a
block_frames as well, and on Android that number came from a user
setting whose value the platform had seeded from
android.media.property.OUTPUT_FRAMES_PER_BUFFER - the fast mixer's
burst. That is a property of the sink, in the same class as the
device period WASAPI reports or the period grid an ALSA card refines
to, and both of those are asked of the device rather than of the
user. Android was the only place a device fact reached a driver
through a preference.
audio_driver_device_block_frames() hands it over directly, and the
OpenSL driver treats it as what it is: a granularity, not a block
size. The block is one burst where the platform reports one, and the
latency decides how many - so a phone reporting 192 frames opens
64 ms as sixteen blocks, 16 ms as four and 8 ms as two, each a whole
burst. A queue whose blocks are not a multiple of the burst leaves
the fast mixer, which costs latency rather than saving it, which is
why the setting could not deliver what it was for: the only thing it
could do that the device fact cannot is ask for a block the device
does not use. Where nothing is reported the block still follows the
latency, as it did.
So the setting loses its descriptor row and is no longer offered in
the menu. Its configuration and string rows stay, so a config
carrying audio_block_frames still loads and still round trips rather
than being dropped on the next save, and the platform override that
seeds g_defaults is untouched.
Two things fell out of reading the other two callers:
- alsa_qsa sized its block as block_frames * 4 - a number a user set
for an Android device's burst, applied on QNX, which reports no
granularity at all. It follows the latency now.
- switch_audio assigned block_frames to swa->blocking, a bool: any
block size at all made the driver blocking until the frontend's
next set_nonblock_state overwrote it. It starts blocking and is
set by that call alone.
opensl_test takes the burst through the accessor and checks the
buffer against it: every block a whole burst, never fewer than two,
and the latency reachable down to two of them.
Issue #6405 reports that certain audio-latency values hang RetroArch
on Android after seconds to minutes, and works out why: at a fixed
1024-frame block, a latency of 34 ms or less made the block count
round to one, and a queue of one block cannot be written to at all -
the block being filled is the block the device is playing.
The driver has since gained the floor the reporter asked for, a
block size that follows the latency where the device does not name
one, a bounded wait on the buffer-done callback, and an enqueue
failure that is reported rather than counted as written. None of it
had a test, and a phone is the only place it runs.
opensl_test builds the driver against stand-in <SLES/*.h> - the
object and interface shapes it uses - over a scripted player whose
queue a thread drains at the rate the format asks for, and covers
what the issue is about: the block arithmetic at the reporter's own
44.1 kHz and 1024-frame block, at 35 ms and at the 34 that used to
wedge and at 16, 8 and 1; the block the driver picks for itself,
which follows the latency (64 ms opens 64.0, 8 ms opens 8.0); the
queue accounting, with no block ever pushed at a full queue; float
where the device takes it and 16-bit where it does not; a full queue
reporting no room rather than an unsigned underflow; and a device
that stops consuming, where the write comes back short and
wait_writable returns nothing rather than parking the core's thread.
Plain, ASan/UBSan and TSan builds all pass. In the audio samples CI.
image_decode_slice_test stubs the video driver, and the icon path
now hands its texture to video_driver_texture_load_async, which the
sample had no stub for: it built until the link and then failed on
the undefined symbol. Stubbed the way the contract has it - the
handle back through done(), the image through release(), both at
once, which is what the synchronous case does - so the task is
exercised handing its image and tag over exactly once.
Two signatures in the discrete harness that the compiler had been
warning about: its wait_writable returned a bool where the contract
returns the bytes the device can take, and its consumer thread
returned void* where sthread_create takes a function returning void.
Both compiled and neither was right.
Runs samples/memmap/emscripten_mmap under node and compiles
vfs/vfs_implementation.c with HAVE_MMAP, the in-tree translation unit
that includes memmap.h and <sys/mman.h> together. The native sample
sweep builds the same sources with every Emscripten branch
preprocessed away, so this lane is what a regression there would trip.
Both guards sit beside comments that gate on what the headers provide,
so record why this one target is spelled out instead. The sample checks
memprotect() as well, since the header selection decides which
implementation it gets.
Use sys/mman.h so mmap callers agree with the libc off_t ABI. Keep address-space reservation and instruction-cache synchronization on their existing Emscripten fallback paths.
An audit of this tree proposed decomposing runloop_iterate() for
I-cache footprint and register spills. Measured on the release build,
it has an 88-byte stack frame and eleven stack references: there are no
spills to decompose, and the compiler has already sunk the cold work
into the branches. What the same measurement does show is
video_driver_frame() at 9 KB of code and a 4 KB frame - the frame is
video_frame_info_t by value, a kilobyte of which is the statistics
text - and that is worth knowing before someone doubles it.
So the numbers become a check rather than a refactor. The script reads
the release objects, compares four functions against a reference
checked in beside it, and fails a quarter past today's figures: loose
on purpose, since compilers differ and the point is a step change, not
a byte count. A deliberate growth moves the reference in the same
commit, which is the moment to say why. The menu-harness job already
builds a release tree, so it runs there.
The AC-3 bitstream path had been proven from a discrete 5.1 float
buffer handed straight to the driver, and the frontend's merge had
been proven to build a discrete device frame; nobody had run the two
together, so that a core's channels reach a television's decoder was
an inference and not a measurement.
The discrete harness now runs the shipping WASAPI driver over the
fake device from wasapi_exclusive_pacing - the driver's own text,
its three Windows includes swapped, as that harness already does -
configured the way the reporter's television is: no PCM wider than
stereo, Dolby Digital accepted. A 5.1 core feeds the multi-channel
batch entry with a different tone on every channel; the bursts the
device is given are decoded back with rac3 and each channel has to
carry its own tone and none of the others'. It does: every channel
at full energy with the loudest other 40 dB down, where the fold and
the upmix would have put the fronts in the rears at -3 dB.
Two things the case needed to be dependable rather than lucky. The
drain after the feed waits for the bursts the feed produced, or for
the device to stop taking, or for a deadline: a fixed wait ends a
sanitized run after a burst or two, and the pump never goes quiet
for good, so waiting for quiet alone does not end at all. And the
burst walk steps over silence a frame at a time instead of taking
the first gap for the end of the stream - a slow run leaves gaps
between bursts, and under TSan that had ten of eleven bursts
disappear.
The threaded cases are made deterministic while here. The ring's
layout is published per batch and read once a pass, so the pass that
straddles the first 5.1 batch takes its extra slots from the stereo
side and the upmix fills them - one pass, documented, and not what
those cases are about. They judged the last half of the capture,
which held that pass or not depending on the consumer's phase; they
now drop what was captured before the ring settled and judge
everything after. The capture goes under a mutex, since the consumer
appends to it while the test resets it, which TSan found. DM_ONLY
runs a single case when one is being worked on.
Plain, ASan/UBSan and TSan builds all pass.
switch_audio.c is the only consumer of the libnx/libtransistor compatibility layer, so the definitions live directly in the driver and the header is gone.
The file gates its alsa-lib includes on HAVE_ALSA, and a qb build
carries HAVE_ALSA in config.h rather than on the command line, so
config.h is included ahead of the first test and <alsa/asoundlib.h>
is in scope for the driver and for audio/common/alsa.h.
The samples/audio job builds audio/drivers/alsa.o and
audio/common/alsa.o once through configure --enable-alsa, so the
object a distribution ships is covered alongside the ones the
harnesses compile with -DHAVE_ALSA.
The embedded driver talks to the kernel's PCM ABI through one seam -
open, ioctl, close, poll - so a scripted device that answers the way
a card does runs it on a machine with no card, and, unlike a card,
can be told to refuse a rate, a format, a channel count or a period
grid. ealsa_test does that: a cooperative card, a card with one
rate, a card without float, a card whose periods come in twos and
whose period size is a multiple of 512, a stereo-only card with 5.1
asked for, a 5.1 card, a card that is not there, a full device, an
underrun, a card that cannot pause, and the device listing. Built
with HAVE_TINYALSA alone, so the build never sees an alsa-lib header
either. In the audio samples CI.
Two defects in the hardware-parameter negotiation, both of which
would have needed the wrong card to find:
- The period size and their count were asked for as exact values,
768 frames and four of them. A refine narrows a parameter set to
what the card can do and refuses when the result is empty, so a
card whose periods come in twos, or whose period size is a
multiple of 512, refused the set outright and the device would not
open at all. They are now asked for as "no smaller than this".
- A range alone is not enough either: the kernel settles every
interval at its minimum, so the first fix opened the smallest
buffer the card had - eight milliseconds where sixty-four were
asked for, which underruns on every frame. The range is refined
first to see what the card can do, the smallest period at or above
what was wanted is taken from the result and committed exactly,
and the buffer is then asked for as a whole number of that period
so the card picks the count that makes it. On the cooperative card
a 64 ms request now opens 64.0 ms; on the card with its own grid
it opens 1024 x 2 frames, 43 ms, instead of failing.
The harness checks the buffer against the latency that was asked
for, which is what neither defect would have shown on a card that
happens to like round numbers.
The saturation, deviation and water-mark percentages the statistics
overlay shows had no coverage, so a change to what the producer
publishes for rate control could move them and nothing would say so.
The harness now computes them at the end of a run and requires a run
that held its target to report a middling saturation and few marks.
On the threaded pipeline they are computed from the fill the
producer publishes for the controller, which is the pipe and the
device together folded into the device's own range: on target is
half free, and the marks are a quarter of a device buffer either
side. That is centred correctly - this run reports 46% saturation -
and it is why a ring too small to hold a publish reads as time at
the marks: at 21 ms the same run reports 24% near underrun and at
10 ms half its samples, which is the ring, not the measure.
audio/drivers/alsa.c now holds both: "alsa", the alsa-lib driver,
unchanged and behind HAVE_ALSA; and "tinyalsa", the embedded one,
behind HAVE_TINYALSA with its own includes, so a build with only the
embedded driver never sees an alsa-lib header. Both idents, both
vtables and both build switches are what they were, so no
configuration and no platform makefile changes meaning.
audio/drivers/tinyalsa.c is gone, and with it the vendored copy of
Android's tinyalsa - 2080 of its 2651 lines were that library, under
its own licence, which is why the two files could not simply be
joined. The embedded driver is now RetroArch's own, written against
<sound/asound.h>: the PCM character device and its ioctls, the
mask-and-interval negotiation refined and then committed, RW
interleaved transfers, DELAY for the telemetry, poll for the wait.
It is an independent implementation from the kernel's UAPI, not a
clean-room reproduction - the file it replaces was read in the
course of deciding what to do about it, and the distinction belongs
in the record.
The ABI is spelled out rather than included, for two reasons: an
embedded toolchain need not carry that header, which is this
driver's whole point; and alsa-lib's headers define the same names
differently and cannot share a translation unit with it. A copy of
an ABI is safe in a way a copy of an interface is not, and
abi_check keeps it honest - it extracts the block from the driver
itself and compares every struct size and field offset, every
constant and every ioctl number against <sound/asound.h>.
What the driver gains by being ours: float output where the device
takes it (the pipeline is float, and the vendored one was s16 only),
the channel layout the frontend asked for, frames_consumed for the
sink rate estimate, wait_writable, and a stop that falls back to
drop and prepare where the device cannot pause. The old one had
none of the last four.
embedded_alsa_test runs it against a mock kernel device that refines
parameters the way the kernel does: a missing device, the negotiated
rate reported back, a rate outside the device's range pulled into
it, an s16-only device, writes reaching the device and the stream
started once it has audio, frames_consumed against what the device
holds, a full device refusing a non-blocking write and taking a
blocking one, underrun recovery, pause and the drop fallback, and
the device listing. It found an overflow in my own sw_params
boundary loop - the doubled value wraps at the top and the loop
never ends - before this went anywhere near a device.
Every frame with autosave on, the runloop took each autosave handle's
mutex before retro_run() and released it after the post-core work: a
lock held across core execution, whose cache line moved between the
emulation thread and the storage worker sixty times a second, to keep
the worker from snapshotting SRAM mid-frame.
The boundary is published now instead. The main thread makes a
sequence odd before the core runs and even after - two plain release
stores per frame, no mutex, no read-modify-write - and the worker
reads it either side of its snapshot: odd before, or a different value
after, means the copy may straddle a frame, so it is dropped, the
buffer is left dirty, and the next interval retries. The file is only
ever written from a snapshot known whole, which is what the lock was
there to guarantee. The worker's own mutex stays where it was, around
its buffer and flags, and it never spanned the file write.
The dirty bit is now set on the main thread without that mutex. It is
one bit, the worker only clears it, and a lost update costs one extra
comparison rather than a missed save: what drives the write is the
comparison, not the bit.
The QNX driver shares a name with the ALSA one and nothing else -
QSA is the ALSA 0.5 API, fragments and channel params where alsa.c
has periods and hw_params - so the two cannot be one file. What it
did share with every other driver was the audio_driver_t contract,
and it was not keeping it. It has no build in CI and no hardware
here, so it is now built against a stand-in <sys/asoundlib.h> over a
scripted device - the twelve entry points it uses, a fragment size
the device chooses, a full device, an underrun - and run. Not a
substitute for a QNX build; it is the coverage the driver had none
of, and it found:
- Init returned (void*)-1 on every failure. The frontend tests that
pointer for NULL, so a device that would not open was taken for a
live driver and the first write dereferenced -1. The device and
the allocation leaked with it, one per attempt.
- The rate was fixed at 48000 whatever the frontend asked for, and
new_rate was never written, so nothing was told. A core at 44.1
kHz played a tenth sharp for the session; the rate is now
requested and reported the way every other driver reports it.
- A non-blocking write on a full device advanced the block index and
cleared the pointer before checking the result, so the block was
thrown away and counted as written: silent dropouts, and in
blocking mode a spin that lost a block per turn. The block now
stays put and the caller is told what was taken.
- write_avail was computed from buffered_blocks, a counter for a
writer thread that does not exist and that nothing ever assigned;
it reported a constant. It is now the room in the staging block,
which is what the driver can take without waiting.
- A blocking write on a device that had stopped draining - stopped,
or gone - looped for ever. It gives up after a bounded number of
retries and returns what went.
Also: buf_count is at least two (zero made the allocations empty and
every index a read past them), voices comes from CHANNELS rather
than a literal, and the file includes verbosity.h, which it had been
getting by accident.
Two of the standing verification items - PulseAudio with the
telemetry cache, ALSA on a device that cannot pause - turn out not
to need hardware after all: a PulseAudio daemon with a null sink
and ALSA's "null" plugin are the real libraries and the real
drivers, only without a speaker. live_drivers_test drives both
through init, blocking writes with the telemetry sampled at every
one, stop and start, and non-blocking writes to refusal; ALSA is
run twice, once as the plugin reports itself and once with
can_pause cleared, which is what a device that cannot pause looks
like to the driver - stop must fall back from snd_pcm_pause to
snd_pcm_drop and start must recover. In the audio samples CI, with
the server started in the step.
The Pulse cache behaves: write_avail follows the server's requests,
and a blocking write ends at its bound rather than hanging when the
server raises no request - which the null sink does for the two
seconds before prebuf fills, and which is exactly the case that
bound was added for. ALSA's pauseless stop drops and restarts, and
writes go on after.
Three defects the build of the harness surfaced, all real:
- pulse_buffer_attr_cb assigned pa->minreq outside the if that
guards the server's attributes, so a stream with no attributes
read them from a null pointer; -Wmisleading-indentation had been
pointing at it.
- The same shape in pulse_start's fallback: buffer_size in the else,
minreq unconditionally after it, so a served attribute set was
immediately overwritten with tlength/4. minreq bounds the wait in
pulse_write_avail, so it decided how long a starved write waited.
- alsa_init_pcm tested snd_pcm_frames_to_bytes for failure after
storing it in an unsigned field, where the test can never be true;
a failure became an enormous period or buffer size. Both sites now
test the return value, and report it rather than the frame count.
The emulation frame path makes no general-purpose heap allocations -
measured, not assumed: zero over three hundred frames, unthreaded and
under the wrapper, with a counting allocator in front of glibc. That
is worth keeping. An allocation on that path is a lock, a possible
syscall and a fragmentation source on the one path that must not have
any, and it arrives silently: nothing else in the suite would notice.
The lane counts them in the harness itself, by defining
malloc/calloc/realloc over glibc's and forwarding to __libc_*, and
fails if the count is not zero. It runs where that interposition is
the only one - not under ASan or TSan, which replace those symbols
themselves and would have their own bookkeeping counted - and says it
skipped otherwise; the menu-harness job builds without a sanitizer, so
CI runs it there. Verified against an injected allocation in
video_driver_frame: 300 calls over 300 frames, both configurations.
Two per-delivery costs in the icon view, both paid for every
thumbnail the engine hands back, two thirds of which are prefetch rows
off screen:
LVM_REDRAWITEMS asks the control for the item's rect, which in icon
view on a virtual list is a layout pass, and it does so for off-screen
rows too. cw_cell_invalidate() computes the cell from the same grid
arithmetic cw_visible_rows() uses (origin, spacing, column count),
returns without touching the control when the cell is off screen, and
InvalidateRect()s it with an 8 px margin when it is not.
LVM_SETICONSPACING re-lays-out every item and was sent on every list
change from cw_thumbs_reset(); it now goes only when the computed
spacing differs from what the control already has (zoom or DPI
changed).
cw_thumb_tick() itself is left alone: after 7a62e02282 its per-frame
work is three constant-time control queries and one lock check, and
gating those behind a scroll compare would cost the same query.
Checked: tools/mingw_syntax_check.sh, tools/companion_win32_test.sh
under Wine.
The flag let a host clock driving runloop_iterate() from a modal loop
skip the frame limiter's sleep, the frame delay and the no-window
wait. Its only setter was the Win32 size/move pump from 91920289,
removed in 7fbc868499; the replacement (6c0cedc945) never runs the run
loop from the modal loop, so nothing has set pace_external since. The
bit (1 << 5) is left vacant with a note; PACE_FACT_EXTERNAL goes with
it and the three guarded sites lose their dead branch.
Checked: host build of runloop.o, tools/mingw_syntax_check.sh,
samples/runloop/pacing.
Phase 5's recording: a core delivering 5.1 was recorded as the
fold. The recorder now opens in the core's layout when the core had
one at the start of the recording and the container has a default
layout for the count - quad, 5.1 with the pair at the back, 7.1,
the layouts in the order the recorder takes frames - and every push
is brought to that layout by one function: a batch in the layout
goes as it is, a stereo batch on a 5.1 recording lands in the
fronts with the rest silent, a batch of another layout is remapped
(positions both have copied, positions only the recording has zero,
positions only the batch has folded into the fronts at the BS.775
gains, LFE dropped), a float batch narrowed once. The classic
entries, the wide entry on the threaded ring and the non-threaded
path all push through it. A stereo core records as it always did.
The FFmpeg recorder takes the container's default layout for the
count instead of stereo, and resamples a wider frame through one
instance of its float resampler per pair at one ratio, so a
recording sample rate still applies. The AVFoundation recorder
names a wider layout by its channel bitmap, since the frames are
in the bitmap's order and the MPEG 7.1 tags put the pairs the
other way about; not compiled here.
The upmix harness checks the remap's cases; the discrete harness
records a 5.1 batch and a stereo batch through the real push with
a capturing recorder opened in 5.1 and then in stereo, and finds
the core's rears in the recording's, the stereo batch in the fronts
alone, and the fold at -3 dB.
Follow-up to 2a9b96830c. Under threaded video every thumbnail upload
was a round trip the main thread waited out, up to one present each:
one per row while scrolling a playlist with boxart, and one per frame
for a playing video/animated preview. Both now go through
video_driver_texture_load_async(); nothing changes without the wrapper.
The still: gfx_thumbnail_handle_upload() hands the task's image over
and leaves the status PENDING; width/height are set at once so layout
does not wait. The handle comes back through
gfx_thumbnail_upload_done(), which sets AVAILABLE (or MISSING on a
failed upload) and starts the fade, the two things the synchronous
path did inline. The animation is still opened from the task, since
the stream handover needs it; if its first frame lands before the
still, the still yields to it - the frame is newer.
Animation frames: the decoder's buffer is reused for the next frame,
so the wrapper gets a copy (a few hundred microseconds at most against
the present it replaces). One frame in flight per thumbnail; a frame
decoded while one is travelling is skipped, which is what a slow
present would have shown anyway, so a stalled presenter never grows a
queue of frames.
Both deliveries are validated against the list generation and a new
per-thumbnail upload_seq that gfx_thumbnail_reset() bumps: a result for
a reset or superseded thumbnail is unloaded on arrival, never installed.
Both fields are zeroed by gfx_thumbnail_init_blank().
samples/gfx/gfx_thumbnail_anim: the stubs gain the async API with a
flush, and a third case drives it - one frame posted and in flight
while the animation keeps running, nothing at the driver until flush,
texture installed and the next frame posted after it, and a reset
with a frame in flight discards it on delivery. Passes plain and under
ASan/UBSan with leak detection. samples/gfx/threaded_video still green.
Not converted: the menu wallpaper (one upload at init) and the
gfx_display_reset_textures_list* sync helpers used for icon sets,
badges and widgets - those callers write into arrays without a
generation guard, so the async delivery needs one before they follow.
The threaded pipeline's ring carried stereo, so a multi-channel core
on it folded. Now a core that negotiated the multi-channel batch
entry before the driver came up gets a ring of the canonical wide
frame: a slot per speaker position bit, FL and FR first, eleven
wide, the slots a batch lacks zero. The width is fixed for the
driver's life, so the ring is never switched under the consumer -
a first attempt switched it at an empty ring and TSan found the
producer reading the width mid-switch - and the frames' layout is
published beside them per batch, read once a pass: the consumer
takes the fronts to the scratch the flush has always taken and the
slots of the layout to the extras, which then go through their own
resamplers and to their speakers as on the non-threaded path. A
stereo publish from the classic entries on a wide ring is widened
into the frame with the rest zero, and the upmix fills the device
as for a stereo core. The wide entry does what the classic one does
before it publishes: the suspend and netplay gates, the recorder
fed the fold, the reverse-play frames to the classic entry's fold.
A negotiation after the driver is up folds until the next init;
the flag is cleared at unload.
The harness gains the threaded ring in both formats - each device
channel its own tone through the consumer thread - and a stereo
batch on the wide ring, the rears at the upmix's -3 dB. TSan clean;
the pipeline rate-control, stall and handshake harnesses unchanged.
Under threaded video every menu icon went through
video_driver_texture_load() -> video_thread_texture_handle(), a
command the main thread posts and then waits on. The video thread only
services commands between frames, so each icon cost the main thread up
to one present - at swap interval 3 on 165 Hz, ~18 ms - and the menu
loads dozens at startup and on every driver switch. The first trace of
the companion deadlock (cb_task_icon_load -> vulkan_load_texture ->
video_thread_wait_reply) was this stall.
video_thread_texture_load_async() takes the image, queues it under
thr->lock and wakes the worker; the worker uploads with the driver
directly (it is the driver's thread), releases the image and parks the
handle; video_thread_frame() delivers done(user, handle) on the main
thread before the frame that may draw with it. Post order is delivery
order. Teardown releases what was never uploaded and answers every
outstanding load with 0 after the join, so no handle from a dead
context is ever delivered. video_driver_texture_load_async() wraps it
with the same CPU-side fixups as the synchronous load and falls back
to the synchronous path when there is no wrapper, on the video thread,
or for compressed images.
cb_task_icon_load() posts instead of waiting. The generation check
moves to delivery, where it belongs now that the target can be freed
while the upload is in flight; a stale result is unloaded, and a
target that already got a newer handle has the old one unloaded before
the write, so nothing leaks.
Harness: samples/gfx/threaded_video gains an async lane - posting six
loads returns in under 5 ms, uploads run on the worker, done() runs on
the main thread in post order with the driver's handles, every image
is released once, teardown with loads in flight answers all six with
0, and the no-wrapper fallback completes before returning. Suite green
three runs in a row on the release build.
Checked: mingw_syntax_check on the three .c files (consumer failures
identical to master), host build of the objects, full non-Qt build and
harness run.
The shifted-pair fix let every window left out since the last kept
one into the sums with it, the device steady through them being
the test. That is the wrong test: a core pausing half a second
every ten, a stall the source never made up, a slow start - all
have the device steady and the source short, and merging them
summed the shortfall as a slow source. The sink-rate harness has
those very scenarios, and four of them have failed on master since.
Now the windows left out come in only when, taken with the window
that follows them, they read as the clock - the source within the
band a bias could correct over the span, and the device with it.
A stall's short window and its long neighbour do; a pause and its
normal neighbour do not. And the ring of recent windows that the
sum test runs over takes only windows within their own band: a
window outside it - a pause, a dry spell - is not a clock, and in
the ring it failed every sum for the next few windows whatever
they measured, which is why that scenario never applied.
All nineteen sink-rate scenarios pass again; the pipeline harness's
stall case still applies about zero.
The multi-channel batch entry now sends a core's channels to their
speakers when the device has them, instead of folding to stereo and
letting the upmix guess them back. The front pair goes through the
pipeline as every stereo core's does - the filters, the resampler,
the mixer, rate control - and the other channels go beside it
through their own instances of the same resampler, in pairs, at the
same ratio on the same input count, just after the fronts went
through; the same kind of resampler on the same count at the same
ratio produces the same number of frames, which is what keeps the
channels aligned. At the write the upmix fills every device slot
from the fronts as before, and the core's channels then take the
slots of their positions: a back pair to a side-only device and the
reverse, a position the device lacks folded into the fronts at -3
dB, LFE dropped. The int16 and float forms each stay in their
format; a core in the other format converts once, as the fronts do.
The discrete path is taken on the non-threaded pipeline to a device
wider than stereo with the front pair in the layout; the threaded
pipeline's ring carries stereo and folds until it carries more,
which is the next step, and headphone virtualisation folds too.
discrete_multichannel_test includes the shipping audio_driver.c: a
5.1 core with a different tone on every channel, through the entry
to a 5.1 device at a ratio off unity, in all four format pairings -
each device channel carries its core channel's tone at full energy
and the loudest other tone is 40 dB down, where the fold-and-upmix
put the fronts in the rears at -3 dB; and to a stereo device, the
fold at the BS.775 gains with the LFE dropped. In the audio samples
CI.
An int16 core into an int16 device opened wider than stereo was
widened to float for the upmix and narrowed again for the device:
two conversions and a rounding on the one path the integer fast
path exists to keep exact. The upmix now has a fixed-point form -
the gains in Q15, the LFE low-pass on a Q30 state with a Q30
coefficient - and an int16 mix into an int16 device goes through
it, so the integer pipeline is integer to the device. The float
pipeline into a float device was already float end to end; the
mixed cases (int16 core, float device, and the reverse) make the
one conversion they must.
The harness runs both forms on a stereo tone set through 5.1: the
fronts, centre and rears agree to the LSB, and the LFE against a
double-precision run of the same filter is 0 LSB in the int16 form
and 1 in the float form - a Q15 coefficient was 5 LSB off on a 60
Hz tone, the coefficient being a hundredth or so and its rounding a
tenth of a percent of the cutoff, which is why it is Q30.
The remaining conversions in the pipeline, audited: the headphone
virtualisation (float filters, opt-in), the AC-3 encoder (float in,
one conversion for an int16 core), a float mix into an int16 device
and an int16 mix into a float device - each one crossing, none a
round trip. The multi-channel batch entry folds int16 in int16 and
float in float.
Phase 4 of the multi-channel plan, the API. A core whose source has
more channels than two - a console with discrete surround, a media
player - has had to fold them at the libretro boundary. This call
hands it a pair of batch entries taking a frame of any layout of
the eleven WAVEFORMATEXTENSIBLE positions, interleaved in ascending
bit order, int16 always and float when the float call was granted;
the layout may change between calls, the sample format may not,
and a layout the frontend does not know returns 0 frames. ABI-safe
in the manner of the float call: an environment command (94,
experimental), no signature changes, a frontend that does not know
it returns false and the core keeps folding. RETRO_AUDIO_SPEAKER_
bits and RETRO_AUDIO_LAYOUT_ constants name the positions and the
layouts a core is expected to use. The wording is written for review.
The frontend's first implementation folds at the boundary: the
core's frame goes to stereo at the ITU-R BS.775 gains the mixer uses
(centre and surrounds at -3 dB, a back centre at -6 dB, LFE dropped,
mono to both sides, int16 saturated), into the classic entry of the
same format, so the filters, the resampler, the mixer, recording,
rewind and netplay's gate see what they always have, and a device
opened wider gets the upmix's widening as for a stereo core. The
overlay says what the stereo was folded from. Any core adopting the
call works on every device today; the discrete path - the core's
own rears kept apart to a device that has them - is the next step
and takes over at this entry when it exists.
audio_downmix_f32/s16 and audio_layout_known live with the upmix,
and the upmix harness checks the gains, the int16 form, the back
centre, mono, saturation and the edges.
The push read the clock four times for one instant - the core-time
sample, the slot's push time, the hold's start, and the wait loop's
own - on the main thread, on the paced path, with nothing between
them a timestamp needed to bracket. On x86 that is a few tens of
nanoseconds; on the consoles where the clock is a syscall it is not.
One read at entry serves the first three, refreshed once if the ring
wait ran; the wait loop keeps reading, since it brackets a wait.
The frame limiter's spin to the deadline - the remainder the sleep
does not cover, sized by the sleep's measured overshoot - now gives
the core a pause hint each turn: PAUSE on x86, YIELD on ARM,
nothing elsewhere. It changes when the loop notices the deadline by
nothing and lets a hyperthread sibling run, and the spinning core
draw less, for the microseconds the spin lasts.
Levels 1 to 3 - what RZIP, the savestate writer and the replay
checkpoints run on the frame's time - are the encoder exactly as it
was: byte-identical output, the same speed. What is new is what
level 5 and up buy, and a bug fix.
The ratio gap to deflate-6 measured in session 4 was taken apart
with the decoder tapped for every block's literal and sequence bytes
and every sequence's lengths and offset, ours against the
reference's. The reference's extra matches were nearly all to the
same offset in a page a megabyte earlier: across the block boundary,
which the format allows (single segment, the window is the content)
and the search forbade. Taken, they are worth a sixth of the output
- but every candidate that far back is a cache miss to read, and it
is three times slower, and no prefetch, key check or distance cap
recovered any of that: the matches are where they are. So it is a
level's choice, not a default's.
With the far matches in, two more things pay: the block's own
sequence tables where their ideal cost beats the predefined ones -
the predefined tables charge the codes a memory image produces four
to six bits each, most of a sequence - and a weighing of short
matches against the offset's extra bits, so a four-byte match a
quarter of a megabyte back is left as literals. Both are worth two
percent alone at a tenth of the speed, which is no trade, so they
come with the window. A fixed-point log2 does the cost comparison;
the encoder needs no libm.
On the session-4 buffer, 16 MiB, x86-64 -O2: level 3 is 9.22 MB at
125 MB/s as before; level 5 is 7.78 MB at 40 MB/s - smaller than the
reference's level 4 (7.86) and 9% smaller than deflate-6 (8.56), with
decoding unchanged at 900 MB/s. Nothing in the frontend selects level
5 yet: it is for a caller that compresses off the frame's time.
The fix: a candidate's byte at the best length was read one past the
block when the best already reached its end. ASan caught it on the
multiblock test on master as well; the comparison is skipped where
no candidate can beat the best.
process_events() was skipped whenever video was threaded. With threaded
video the RetroArch window is on the video thread and pumps itself from
the driver's alive(), but the Win32 companion windows are created on
the run loop's thread, and PeekMessage(NULL) only returns the calling
thread's messages, so nothing pumped them at all: the companion went
"Not Responding" on F5 while the game kept rendering, in list and icon
view alike. Five gdb samples of the main thread showed it in a normal
runloop_iterate() (core_run, video_thread_frame) every time - it was
never stuck, the queue just had no consumer.
The gate goes. Under non-threaded video this thread already pumped
here and in win32_check_window(); either dispatches first, no message
is seen twice. Cocoa's process_events() is a no-op, so only Win32
changes.
Checked: tools/mingw_syntax_check.sh runloop.c. Windows: threaded video
on, lrps2 on Vulkan, F5 - companion responds; drag it, scroll the 42k
list, switch Icons/List.
Companion "Not Responding" on a 42k-entry All Playlists in icon view
while the main window kept rendering. gdb: the main thread was inside
comctl32's list view under SendMessage, in DrawTextW. In icon view a
virtual (LVS_OWNERDATA) list has no stored item positions, so every
LVM_GETITEMRECT for row r lays out rows 0..r again and measures each
label with DrawText through LVN_GETDISPINFO. cw_visible_rows() did a
binary search with it - sixteen O(n) layouts per frame over 42116 rows
- and the thread never got back to the message pump.
The visible range is now arithmetic from what the control was told:
LVM_GETORIGIN for the scroll offset, LVM_GETITEMSPACING for the cell
size, client width for the column count. O(1), no DrawText.
Checked: tools/mingw_syntax_check.sh, tools/companion_win32_test.sh
under Wine. Windows: open a large playlist in icon view with threaded
video on; scroll; thumbnails should still fill in for the rows on
screen.
The one switch on the mixer sound type the AC-3 commit missed: the
stream added from the task fell through with no sound, and the
compiler said so. Every other switch over the enum was checked
with -Wswitch under the mixer's codecs on.
Deadlock with threaded video + Vulkan + a core that logs from its own
threads (pcsx2): pressing F5 hung the new window. gdb: the main thread
sat in video_thread_wait_reply() under vulkan_load_texture() from
cb_task_icon_load(), waiting for the video thread; the video thread sat
in SendMessageW(). The companion's log hook, reached from RARCH_LOG on
whatever thread logs, did SendMessageA(EM_REPLACESEL) to the log edit,
which belongs to the main thread. SendMessage from another thread does
not return until the target's thread pumps, and the main thread was
not pumping - it was waiting for the very thread that was calling it.
The log line is now converted, copied to the heap and PostMessage'd to
the companion window; the wndproc commits it on the owning thread and
frees it. PostMessage never blocks; a full queue drops the line. The
status bar hook had the same shape (runloop_msg_queue_push() runs on
core and task threads, under RUNLOOP_MSG_QUEUE_LOCK) and gets the same
treatment. Both message ids are drained before DestroyWindow so posted
lines do not leak.
Checked: tools/mingw_syntax_check.sh, tools/companion_win32_test.sh
under Wine. Repro to confirm on Windows: threaded video on, lrps2 on
Vulkan, F5.
The same decoder takes an E-AC-3 frame: the Annex E bsi, walked
through its mixing and informational metadata to the audio frame
header, and the header's strategies applied to one to six blocks -
which blocks carry a coupling strategy and use coupling, the
exponent strategies per block or from Table E2.10's frame
combinations, block switch and dither flags sent or defaulted, the
allocation parameters sent or at Annex E's defaults (floor code 7),
the three SNR offset strategies, fast gains sent or 4, the first
coupling coordinates and leak of a frame without their flags, the
default coupling banding of Table E2.12 by absolute sub-band, and
delta allocation and skip fields only when the frame says. Table
E2.10 is transcribed. Spectral extension, enhanced coupling and the
adaptive hybrid transform are read from the header and refused
where a stream uses them; the halved sample rates too.
rac3_decode_test gains six E-AC-3 streams from ffmpeg's encoder,
tones and pink noise, mono to 5.1: 77 to 107 dB against ffmpeg's
decode, the dither-limited pink stereo at 15 dB with the same gain
as its AC-3 twin. The audio_transfer arm takes .eac3 and .ec3, an
independent stream or an AC-3 conversion, frames of any block
count, with a 5.1 E-AC-3 case in its oracle; the mixer and the file
browser take the extensions. The decoder is C89 again: three ldexpf
calls are ldexp.
The frontend's requested layout is asked of the device as its channel
count for quad, 5.1 and 7.1; a device that answers with any other
count is reopened stereo. OSS's channel order is its own - FL FR BL
BR FC LFE SL SR - not the mask's, so a wider frame is permuted into
it on the way through write(), and the layout reported names the
back pair, which is the pair OSS's order has. The frame is the
channel count of int16 everywhere it was four bytes.
Compiled against the OSS headers; no device here to hear it on.
The frontend's requested layout is one output port a position, named
for it - front-left, front-right, front-center, lfe, rear-left,
rear-right, side-left, side-right - in the mask's ascending order,
which is the order the ring holds the interleaved frame in; stereo
keeps its left and right as always. The first two ports connect as
they always did, to the setting's names or the server's first
physical inputs; the rest go to the physical inputs in order, and a
position with nowhere to go is left for the patchbay and said so
rather than failing the open. The layout hook reports the mask as
asked: JACK routes by port, so the positions are what the ports say.
Compiled against the JACK headers; no server here to run it on. The
unused variable warning in ja_free predates this.
AL_EXT_MCFORMATS gives quad, 5.1 and 7.1 formats, int16 and float,
in the WAV order - FL FR FC LFE BL BR SL SR - which is the frontend's
mask order with the rear pair at the back; the requested layout is
opened as the one of those that fits its channel count and reported
naming the back pair, and a library without the extension, or a
count none of them fit, opens stereo as before. The buffer queued to
the source was a fixed 1024 bytes; it is that made a multiple of the
frame now (3072 for six channels), since a buffer must be whole
frames, and the buffers finished are counted in frames of the opened
format for the sink rate estimate.
Compiled against the OpenAL headers; no device to hear it on.
The frontend's requested layout is opened as its channel count when
the device itself has that many channels - SDL would otherwise fold
the channels back down on its own, and the frontend's stereo mix is
the better source for a stereo device than an upmix of it folded
again. SDL's channel order is the WAV order (FL FR FC LFE BL BR SL
SR), the frontend's mask order with the rear pair at the back, so the
layout reported names the back pair for 4, 6 and 8 channels and the
frontend upmixes to that. A count the device refuses at open falls
back to stereo, and the layout hook says which.
Checked against a stub of the SDL3 audio API; no SDL3 here to build
against and no device to hear it on.
The driver opens the frontend's requested layout: a buffer wider than
stereo is a WAVEFORMATEXTENSIBLE with the frontend's mask as the
channel mask - the same bits DirectSound and WASAPI use - so each
channel goes to its speaker rather than the driver's guess for the
count. The formats are given up in order: the layout in float, the
layout in int16, then stereo in each, so a refused layout costs the
layout before it costs the sample format; the layout hook reports
what was opened and the frontend upmixes to it.
The mixer thread's lock unit was 256 bytes; it is now that made a
multiple of the frame - 768 for six channels of float or int16 - so
no lock and no ring boundary splits a frame, and the ring's size,
a multiple of it, is the multiple of the block align DirectSound
wants. The play cursor's advance is counted in frames of the opened
format for the sink rate estimate, where it assumed stereo.
Untested on hardware: the harness set has no DirectSound fake. The
XAudio2 driver's identical format and mask are what a 5.1 endpoint
took there.
AUDIO_MIXER_TYPE_AC3 and audio_mixer_load_ac3(): a .ac3 file of
syncframes is a stream voice like an M4A, through the same dozen
dispatch sites (play, mix, tell, set_avail, release, destroy, in
both the float and the s16 paths). task_audio_mixer takes the
extension on both push paths with its own NBIO type and upload
handlers, the file browser lists .ac3 as music where the mixer has
the codec, and FILE_TYPE_AC3 maps the extension. Full load, not the
windowed path: the arm counts the stream's frames at start by
walking every header.
The mixer folds by channel count in the WAV order, and A/52 has
configurations the count does not tell apart - 3/1, 2/2, 3/0 with
LFE and 2/1 with LFE are all four channels - so the arm now hands a
frame over in the WAV shape for its count: the LFE kept only where
WAV's count puts one (5.1), a lone back centre split into a phantom
pair at -3 dB each, the surround pair in the pair's place. The
oracle checks the shapes on ffmpeg's 4.0 (3/1 -> five), 3.1 (3/0
with LFE -> three) and 2.1 (-> two) alongside the stereo and 5.1
decodes.
audio_mixer_loop_test gains the AC-3 arm and sound type on a 5.1
fixture the encoder makes at run time: the seek-to-0 replay contract
and the repeating voice, float and int16.
AUDIO_TYPE_AC3: a buffer of syncframes (.ac3) behind the same four
calls as every other codec. The arm walks the buffer with the frame
layer, decodes a frame at a time into a pending block the reads
drain, hands the channels over in the speaker-mask order - which is
the WAV order the mixer's fold already knows, so 5.1 folds right -
and reports channels, rate and the frame count from the headers. A
seek decodes the frame before the target and discards it, so the
target comes out as a playthrough would have left it; a seek to 0 is
the loop's rewind. set_avail is honoured as a frontier: a read there
starves rather than ends. s16 is the f32 saturated.
audio_transfer_ac3_test: ffmpeg encodes a stereo and a 5.1 tone set;
the arm's decode matches ffmpeg's own to 85 and 77 dB over the whole
file, a seek into the middle of a frame lands within 0.1 dB of the
playthrough, the rewind matches the first read, s16 is the f32 to
the bit, and the frontier starves then resumes. In the samples CI.
The codec and the IEC 61937 framing are now compiled in on every
Makefile.common build and in griffin (HAVE_RAC3, no dependencies),
not only with the WASAPI driver; the MSVC projects define it, and
Makefile.griffin does wherever the mixer is.
Two harness faults CI's TSan run found, neither the driver's. The
fake's capture tap was switched on and read by the main thread while
the device thread appended to it; it is under a mutex now. And the
walk over the captured bursts stopped at the first period of silence
between them, which a writer that cannot keep up in real time - the
encode under TSan is slower than a burst - leaves the pump to fill;
the bursts are whole and nothing is lost, so the walk steps over
silence to the next preamble and ends only on the trailing silence.
The bound on where the first burst begins is loosened for the same
reason, from three periods to the fifo.
The pump counted every silent period as an underrun from the moment
the stream started, including the ones before anything had been
written. On the PCM path that is a period or two. On the AC-3 path
the first burst is 32 ms of input and an encode away, and the
frontend, told of underruns meanwhile, did what it does on a dry
spell: discarded what the pipe held past its target - the audio it
had just primed the pipe with to fill the device at start. The fifo
then filled at rate control's pace, half a percent of the rate, and
sat below a quarter for the first fifteen seconds of every session
(the reporter's overlay: underrun 100% at start, falling to zero over
the next ten thousand samples, fifty-odd dropouts all at the start).
Silence is an underrun once the fifo has been fed. Same overlay
count on the PCM path within a period; the AC-3 path primes.
write_avail() on the AC-3 path subtracted the encoder's count of
frames not yet in a burst from the fifo's room, read as two figures:
the room under fifo_lock, the count without. The threaded pipeline
samples write_avail() on the producer while the consumer writes, so
the two could come from either side of a burst's push - the room from
before it and the count from after, or the other way - and the
occupancy rate control saw jumped by a burst, half the setpoint,
whenever the sample landed there. The reporter's overlay: saturation
31%, deviation 15%, underrun 26% and falling, a dropout a second.
The driver now keeps one figure under fifo_lock - the frames the
frontend has written that the device has not taken, the fifo's and
the encoder's together - added to as input is accepted and taken
from as the pump drains, and write_avail() is the buffer less that.
A burst's push moves frames from one side of the sum to the other
and changes nothing.
The last two stereo frames in the threaded pipeline's consumer. The
cap on a pass - never more than half the device buffer, so the room
waited for exists at any fill - and the room waited for before the
chunk leaves the ring were both figured at 8 bytes a frame; on the
5.1 AC-3 path the cap was three times half the buffer and the wait
was for a third of what the write needed, so chunks reached a fifo
without room for them, and in non-blocking mode the driver wrote what
fit: part of a burst, the receiver out of sync until the next
preamble, silence in between. The reporter's overlay after the
previous fix: underrun 100%, 57 dropouts, rate control saturated.
And the driver no longer lets that happen at all: without blocking,
the frames that would complete a burst are not taken until the fifo
has room for the whole burst, and the caller offers them again; a
burst cut short by a blocking write that gave up is counted with the
underruns, since the receiver will resync at the next preamble.
Two more places took a device frame as stereo whatever the layout.
The threaded pipeline's occupancy for rate control put the pipe's
frames and the pipe target into device bytes at 8 bytes a frame
against write_avail() and buffer_size in the device's 24: the target
read as a third of what it was, an occupancy the fifo could never
reach, and rate control sat at its bound - the reporter's overlay on
the 5.1 AC-3 path: underrun 100%, saturation, dropouts, pitch up 2%.
And audio_driver_get_buffer_ms(), the overlay's "Buffer:" figure, read
384 ms for a 128 ms fifo. Both use the device frame now, as the sink
estimate and the sink apply already did.
The frontend reads a driver's layout through the driver table it
holds, and with the threaded audio driver that table is the wrapper's.
The wrapper forwarded use_float, frames_consumed and underruns but not
layout, so a device opened wider than stereo under it was never seen
as such: no upmix stage, and the stereo mix written as 8-byte frames
into 24-byte ones. The reporter's log has it: the WASAPI driver
opened the TV as 5.1 over AC-3, the frontend logged a 384 ms buffer
for a 128 ms one, and no "Device opened with 6 channels" line.
The wrapper reads the layout on its thread right after init, as it
reads use_float, and hands it back from the hook. The handshake
harness's fake driver reports 5.1 and the case checks the wrapper
says so - 0x000 before this.
The line reporting a driver's buffer against the latency setting
divided by a stereo frame whatever the layout: the WASAPI AC-3 path
on the reporter's TV, 6-channel float in a 6144-frame fifo, read as
384 ms where it is 128. Rate control itself was right - it uses the
device frame - so only the line lied. It now names the channel
count as well.
The reporter's LG TV, on HDMI, advertises two PCM channels and
nothing wider - and decodes Dolby Digital, as TVs and receivers do.
A wider layout the device refuses as exclusive PCM is now tried as
AC-3 in IEC 61937 bursts before the driver falls back to stereo: an
exclusive stream opened with the WAVEFORMATEXTENSIBLE_IEC61937 form
and the Dolby Digital subtype (both defined in mmdevice_common_inline.h
under our own names - MinGW lacks the struct and neither toolchain
links an instance of the GUID), a 2-channel 16-bit carrier at the
encoder's rate, 48 kHz unless the frontend's is 44.1 or 32. A/52
takes 1.0 to 5.1; a 7.1 request goes as 5.1, its back pair folded
onto the surround pair by the upmix to the layout the driver
reports. 640 kbit/s for five channels and up, 448 for three or four.
write() takes float frames of the layout as before; 1536 at a time
they are encoded on the writer's thread, wrapped, and the 6144-byte
burst goes to the fifo through the raw path, which blocks or drops as
the mode says. A burst is 1536 frames of the carrier for 1536 frames
of the layout, so a frontend frame and a device frame are the same
count and only the bytes per frame differ: write_avail() and
buffer_size() report in the frontend's, use_float() says float. The
fifo holds at least four bursts, so a burst's encode never finds it
empty and rate control sees the fill in steps of a burst against a
span of four; the TV's own decode latency comes on top of that in
any case. The exclusive Initialize with its period policy and
retries is factored out of the PCM path and shared.
The pacing harness's fake gains a PCM channel gate, the Dolby Digital
subtype and a capture tap, and a case: a 5.1 request against a
device that takes two PCM channels and AC-3. The driver must open it
as AC-3, report the 5.1 layout and take float, feed the device every
period, and every byte the device receives is captured: the bursts
begin on a period boundary, abut, and decode back to the tones that
were written at 44 to 60 dB per channel, LFE included - 46 bursts of
the 47 written, the last still in the encoder. The AC-3 sources are
compiled in wherever HAVE_WASAPI is, in the Makefile and griffin.
Profiled on a 5.1 frame: the forward transform's direct sum was
walking a phase table entry by entry, and the bit writer put one bit
per call, 800 000 calls a frame. The transform is now a dot product
per output against a full 256 x 512 cosine table built once (512 KiB
the process pays once), with four partial sums written out so the
compiler is free to vectorise a float sum; the writer takes a field
a byte at a time. 5.8 ms to 3.0 ms per 5.1 frame of 32 ms.
rac3_layout_acmod() answers which A/52 configuration a layout mask
is, or -1: the driver that will bit-stream a layout needs to know
before it opens the device, and the probe no longer puts a whole
encoder on the stack to find out. The two allocation helpers the
encoder shares by name with the decoder are renamed, so both compile
into one translation unit.
The estimate's windows close on the producer, so a publish a stall
pushes across a window boundary is short in one window and long in
the next; only the pair sums to the clock. The band test on the last
few windows was meant to cover that, but only kept windows were in
it and only kept windows went into the sums: when the short half
failed the band and was left out, the long half then passed against
windows that did not include its partner, and the kept sums carried
the shift as a bias. The application refused it as implausible -
CI's float run: the device took 100.1%, the source showed -989 ppm,
four seconds summed, nothing applied - or, at a smaller shift,
applied it: a 4 ms stall every five seconds against a 64 ms device
biased resampling by -380 ppm on a run whose clocks matched.
Now the last few windows the device was steady through tile the time
before the current one whether they were kept or not, and the
windows left out since the last kept one, the device steady through
them, come into the sums with it: the same span of both clocks. A
dry spell still breaks the span: nothing before it is carried past
it, and what follows is shown until the next kept window, never
merged. The same stall now applies -21 ppm.
The pipeline harness gains a stall mode - one publish in every 300
held 4 ms - run in CI against the 64 ms device, where it costs
nothing but the boundary crossing and the estimate must settle and
apply about zero.
The 10.4 SDK the PPC build uses has AudioChannelLayout.mChannelBitmap
as a UInt32 and no AudioChannelBitmap name for it; the name came
later. Assign the frontend's mask as the field's own type.
How a compressed frame reaches a receiver or TV over S/PDIF and
HDMI: as if it were 2-channel 16-bit PCM, a burst of four preamble
words (Pa 0xF872, Pb 0x4E1F, Pc the data type, Pd the length), the
frame's bytes in 16-bit words most significant byte first, and zeros
to the repetition period - the PCM frames the compressed frame
stands for, times four bytes. formats/iec61937.c wraps an AC-3
syncframe (1536 frames, 6144 bytes, bsmod in Pc's upper bits), an
E-AC-3 access unit (four times that, at four times the rate, Pd in
bytes), a DTS core frame (type I/II/III by its 512/1024/2048
frames), and the pause burst that keeps a receiver locked instead of
falling back to PCM; iec61937_probe() reads a burst's type and
length back. rac3_frame_info gains bsmod for the AC-3 wrap.
iec61937_test: our AC-3 encoder's frames wrapped here decode through
ffmpeg's spdif demuxer back to the source (50-65 dB), and an AC-3
stream ffmpeg wraps with its own spdif muxer is, all 32 bursts, byte
for byte what we produce for the same frames. The preamble, byte
swap, odd-byte rule, padding, DTS types and pause burst are checked
by construction. In the samples CI run list; ASan/UBSan and C89
clean.
The counterpart of the decoder, and the standard's "basic encoder":
the long transform only, no coupling, no rematrixing, no block
switching, new D15 exponents in block 0 reused for the frame, the
core bit allocation with 8.2.12's nominal parameters and the SNR
offset found by bisection so the frame is filled, 7.3's quantisers
with 7.3.5's grouping, dialnorm -31, both CRCs. It is what the
frontend will bitstream to a receiver or TV that takes AC-3 but not
multichannel PCM, which is what the user's LG TV turned out to be.
rac3_encoder_new(rate, layout, kbps) takes 48/44.1/32 kHz, any of
the standard's channel configurations as the frontend's layout mask
(1.0, 2.0, 3.0, 2.1 as L R BC, quad, 5.0, 5.1, each with or without
LFE) and any of the nineteen rates; rac3_encode_frame() takes 1536
interleaved float frames and writes one syncframe. 44.1 kHz
alternates the two frame sizes so the rate is exact.
Two details worth recording. A grouped codeword (bap 1, 2 and 4)
sits in the stream where the FIRST mantissa of its group is read, and
the groups run across channels within a block: so a block's
mantissas are quantised into a queue first and written after, each
codeword looking ahead for the rest of its group. Writing the
codeword at the third mantissa instead decodes channel 0 perfectly
and every later channel as noise. And crc1 covers the field that
holds it, so it is the X with CRC(X || rest) = 0: CRC(X || 0^n) is
linear in X, its 16 columns come from the unit vectors, and a GF(2)
elimination solves it per frame.
rac3_encode_test is the oracle: tones and pink noise at all three
rates and mono, stereo, 2.1 and 5.1 are encoded here and decoded by
ffmpeg, which must give the source back one transform block later at
unity gain - 44 to 63 dB on tones, and on pink noise the 10 dB that
ffmpeg's own encoder gets on the same signal at the same rate,
which is what a perceptual codec's waveform SNR looks like. ffprobe
must count every frame, and the decoder here must agree with
ffmpeg's decode to better than 60 dB (it does to 104-108: no
dither, so the two have no random part). Under ASan and UBSan, C89,
and no stack frame over 2 KiB: the windowed block lives in the
encoder.
The decoder, written from sections 5, 6 and 7 of the standard's text
and nothing else: the audio block syntax, exponent decoding (7.1.3),
the parametric bit allocation in its seven steps (7.2.2) on the
tables transcribed before, the symmetric and asymmetric mantissas
with their grouping and dither (7.3), coupling with coordinates,
band structure and phase flags (7.4), rematrixing (7.5), dynamic
range control (7.7.1), and the 512- and 256-point transforms with
the window and overlap-add (7.9.4). Output is interleaved float in
the frontend's speaker order for the stream's configuration, the
LFE after the centre.
Against ffmpeg's decoder on ffmpeg's streams - tones and pink noise,
mono, stereo and 5.1, 48 and 44.1 kHz, 64 to 640 kbps - the two
agree to 76 to 107 dB SNR at unity gain where the streams leave no
zero-bit mantissas, and to the noise those get dithered with where
they do, each decoder's dither being its own. Scratch lives in the
decoder, not the stack; ASan and UBSan clean.
The tables of ATSC A/52:2018 the codec's allocation and transform are
built on - the bit allocation parameters (7.6 to 7.11), the banding
structure (7.12), bin to band (7.13), the log-addition table (7.14),
the hearing thresholds at each rate (7.15), the allocation pointers
(7.16), the mantissa bits per pointer (7.17) and the transform
window (7.33) - transcribed from the standard's text by table number
and nothing else. A harness checks them against each other: the
bands tile the bins and masktab is the band each bin falls in, latab
falls from 0x40 to 0, baptab climbs to 15, the thresholds are in
range, and the window is monotone and satisfies Princen-Bradley,
which the overlap-add depends on.
My memory of the banding structure had the wrong widths in three
places; the standard's text is the source now, not memory.
The first of the codec's layers, written from the standard and
nothing else: the sync information and the bit stream information as
far as a frame's size and what it carries, for both syntaxes - bsid
up to 8 for AC-3 and 11 to 16 for E-AC-3 - the frame-size rule at
each rate including 44.1 kHz's alternating pair of sizes, the
channel configuration as the frontend's speaker mask with A/52's Ls
and Rs at the side pair and a lone S at the back centre, E-AC-3's
stream type, substream, block count and reduced rates, a sync
search that requires two consecutive headers to agree, and the CRC:
crc1 over the first five eighths and crc2 over the rest for AC-3,
one CRC for E-AC-3.
The harness has ffmpeg encode tones across the rates, bit rates and
configurations in both syntaxes and ffprobe list their packets; the
walk must land on every packet with its size, rate, channel count
and layout, and every CRC must hold. Two things my memory of the
standard had wrong the oracle corrected before this landed: the
44.1 kHz sizes, and the five-eighths point crc1 covers. The decoder
and the encoder follow on this layer.
XAudio2: a wider layout's source voice is a WAVEFORMATEXTENSIBLE
with the speaker mask - the frontend's bits are Windows' own - so
the engine knows which channel is which and routes each to the
mastering voice's speaker of that position, mixing where the device
lacks one; without the mask it would assume a default layout for the
count, and six channels' rear pair would be the back pair whatever
was meant. Stereo is the plain format it always was. The buffer is
sized at the layout's width, and the layout is reported as carried.
CoreAudio: the output unit's input bus takes the stream format at
the layout's width and an AudioChannelLayout of the positions as a
channel bitmap - Core Audio's bits are the frontend's, Left 1 to
RightSurroundDirect 0x400 in the same ascending order, the surround
pair the back pair and the direct pair the sides - so the two 5.1s
go across as themselves. The converter, the raw path and the ring
count frames at the layout's width. A unit that will not take the
stream at that width gets stereo, said so. Parsed against a stub of
the frameworks here; the frameworks themselves are for a Mac.
The frontend's raw int16 fast path hands the driver stereo as it
is, past the upmix and the headphone render; it is taken only when
the output is plain stereo.
A Speakers line under AUDIO: the layout the device opened with and
how it is filled - stereo; a wider layout upmixed from stereo, the
two 5.1s told apart; or stereo through a virtual 5.1 to headphones -
so what a device took and what the mix went through can be read off
the screen rather than the log.
On a stereo device with the setting on, the stereo mix is widened by
the upmix to a virtual 5.1 and rendered back to two ears as a head
would hear those speakers: each speaker at its ITU direction, the
near ear as it is, the far ear later by the head's width at the
speed of sound, quieter and low-passed as the head shadows it, and
the rear pair low-passed a little to both ears as the pinna does to
sound from behind - which is most of the cue that it is behind. The
centre and the LFE go to both ears alike. No HRTF set, so no data
and no dependency; a stated model, mild and the same for every
listener, a few multiplies per speaker per frame. The render is
scaled so a source in the middle of the stereo leaves at the level
it arrived and a full-scale one leaves at full scale.
Settings > Audio > Output: Headphone Virtual Surround, off by
default; it is for headphones and narrows the stereo on speakers,
and does nothing when the device is opened wider than stereo.
samples/audio/binaural checks a left speaker arrives left first,
louder and brighter, the rears duller than the fronts, centre and
LFE alike, the level rule, the two 5.1s rendered apart, and stereo
as a mild crossfeed.
The driver it compiles now asks the frontend for a layout, sizes by
the layout's channels, and reads the endpoint's mix format on a
refusal; the harness's stubs answer stereo, the fake client's
existing GetMixFormat gets the wrapper the driver calls it by, and
the upmix stage is linked as in the other audio harnesses.
Both servers take a stream with named positions and route each to
the sink's channel of that position, or remix where the sink lacks
one, so what the frontend asks is what is carried: PulseAudio as a
pa_channel_map, PipeWire as spa_audio_info_raw's position array,
each built from the layout mask in its ascending-bit order. The two
six-channel layouts differ in the rear pair and each goes as itself;
PipeWire's table keyed by channel count, which named six channels'
rear pair the back pair whatever was meant, is gone. Frames written
are counted at the layout's width.
MSVC's headers declare the GUID without an instance to link, as
with the float one this file already defines; the wide integer
formats that use it did not link on the MSVC builds.
Exclusive mode opens nothing wider than the endpoint's speaker setup
in Windows, which the mix format shows; a stereo one means every
six-channel format is refused whatever its mask or sample format,
and the log now says that the setup is the thing to change - and
that an HDMI display often does not offer it for PCM.
A refused six-channel request now logs the endpoint's mix format
and, for 5.1 under each mask and for 7.1, whether the mode takes
16-bit, 24 bits in a 32-bit container, and float at 48 kHz. The
frontend writes 16-bit or float; an HDMI endpoint that takes
multichannel only as 24-in-32 shows here, which is the fact a
conversion in the driver would be built on.
A 5.1 endpoint on Windows is usually KSAUDIO_SPEAKER_5POINT1_SURROUND
- the rear pair at the sides - and exclusive mode takes exactly the
mask or nothing, so a request for 5.1 with the pair at the back was
refused outright, the driver's own search retried only that mask at
other rates, shared mode was tried next, Windows offered stereo for
it, and the driver took that: a shared stereo stream at 42 ms in
place of an exclusive one at 8. Now the search tries the sibling
mask - the same speakers under the other pair's bits - and reports
whichever the device took, never the request; a suggestion with
fewer channels than asked is the layout refused, not the request
granted; and the order is the preferred mode with the layout, the
preferred mode stereo, then the other mode with either, so the
latency the mode is for is not given up for a layout the device
would not take.
WASAPI: a wider layout is a WAVEFORMATEXTENSIBLE with the mask - the
frontend's bits are Windows' own - PCM or float; the format Windows
takes, or suggests, or the driver finds, is suitable only if its
mask is one the frontend can fill and its count matches its mask, so
a six-channel format without positions is not one; the layout the
stream opened with - for a shared stream the engine's mix format's
mask, which is what the device actually drives - is reported to the
frontend, and a device that will not open with the layout gets
stereo, said so. The frame size is the channels at the sample size,
up to 32 bytes, and the write path's byte-to-frame conversions
divide by it; a shift was possible only while every frame size was a
power of two.
ASIO: a device lists its outputs as numbered mono channels with no
positions of their own, so the positions are the user's wiring and
the setting's word for it: outputs first+0.. carry the layout's
positions in ascending bit order, said so in the log; a device short
of outputs for the layout plays stereo. Buffers, ring, scratch and
the deinterleaving converter are channels wide - the converter takes
the channel count and an array of output buffers, and its harness
adds a six-channel case beside the stereo ones for every sample
type.
A speaker layout is a mask of positions - WAVEFORMATEXTENSIBLE's
bits, interleaved in ascending bit order, which is also SDL's fixed
order and PipeWire's and PulseAudio's default maps - and never a
count: six channels with the rear pair at the back and six with it
at the sides are two layouts, AUDIO_LAYOUT_5POINT1 and
AUDIO_LAYOUT_5POINT1_SURROUND, and neither is read as the other. A
driver that can open wider than stereo asks the frontend for the
layout it wants, opens what the device grants, and reports the
positions it actually opened - ALSA from the device's own channel
map, SDL the request when the count matched since SDL maps the
frames itself. The frontend widens its stereo mix to that layout at
the last step before write(), filling positions by role: fronts as
they are, the centre the sum at -6 dB, the rear pair the stereo at
-3 dB in whichever slots the layout has, the LFE the bass through a
120 Hz low-pass. The pipeline - core, filters, resampler, mixer,
rate control - stays stereo and is byte-for-byte what it was on a
stereo device, which the pipeline harness confirms.
Settings > Audio > Output: Output Speaker Layout - Stereo, 4.0, 5.1,
5.1 Surround, 7.1; the device is reopened on change. Drivers without
the hook stay stereo. samples/audio/upmix checks the slots of each
layout, the two 5.1s apart, every role's level, the LFE's bass and
its lack of treble, and the refusals; the audio harnesses that link
the driver link the stage.
The last patch unified the predicate but found the sites by the guard
text at a handful of line numbers, and gl3 has more: the end of
gl3_init itself, the tail of set_nonblock_state - which the wrapper
calls on the video thread - and the shader loader's begin and step.
Any one of them taking the core's context on the video thread makes
the main thread's bind fail on WGL, which is what boom3 still hit.
The sites are found now by what follows each bind: a bind-true that
is answered by a bind-false before the function ends is a pair around
work on the core's objects and stays as it was; one that is not asks
the predicate. Eleven more sites across the two drivers.
And two failures that were silent say so: WGL's bind logs a failed
wglMakeCurrent with the error code, and the ring logs when the core's
context could not be taken on the main thread. Neither is on a paced
path; both fire once, at a bind.
boom3 on glcore under threaded video died at its first GL function
lookup: wglGetProcAddress on the main thread, with no context current
there. The ring binds the core's context on the main thread before
context_reset; that bind fails on WGL when the context is current on
another thread, and it was - the video thread had taken it during
init and kept it. Two predicates guarded the places this thread takes
that context for itself: "the ring has it" (a flag, set when the main
thread binds) and "the ring will have it" (decided from init on). The
init-tail binds asked the second; the frame-end binds asked the first;
gl3_set_shader's tail asked the first alone, and gl3 loads the stock
shader during init, before the flag exists. So that bind took the
context on the video thread, and the main thread's own bind, moments
later, failed silently. gl2's init does not go through set_shader,
which is why gl cores on it worked and glcore ones did not.
One predicate now, in both drivers: the core's context is the main
thread's if the ring has it or will have it, and every site that would
leave this thread holding it asks that and nothing else. gl2's
bind-true/bind-false pair around building the core's objects keeps
asking the flag alone, on purpose: before the ring has taken the
context this thread must build them in it and give it back.
Two breaks from the last two patches, both mine.
The bounded fence wait compares against GL_TIMEOUT_EXPIRED, and the
PowerPC macOS headers do not have it, as they did not have
GL_TIMEOUT_IGNORED before it. gl2.c already keeps a block of ARB_sync
names defined for headers that lack them; the ring's are all in that
block now, and gl3.c gets the same block, instead of one name at a
time as each old header turns one up.
The resize lane's stage flag was a volatile int written on the main
thread and read on the video thread, and the TSan harness job called
it what it is. It is an atomic now. The lane was delivered after an
ASan run only; the TSan job exists to catch exactly that, and did.
The RZIP write codec is a process-wide choice, so every compressed file
the frontend writes - save files, save states, playlists, core backups
and the core info cache - uses whichever codec this option selects. Say
so in the sublabel, so the option under Saving is not read as applying
to saves alone.
The resize fix went out on reasoning alone: no lane failed without
it. This one does. Frame K is held on the video thread until the main
thread has pushed K+1, built with the size known then; K reports the
resize as a context driver's check_window would; K+1 is drawn, and the
size it is drawn at must be the reported one. Without the fix it is
the size K+1 was pushed with - seen as 0x0 against a reported 320x200
in the harness, the same stale copy that laid hunterk's menu out for
the wrong window. The ordering is forced rather than left to timing,
since three timing-based attempts never reproduced it: the menu-frame
push drains the video thread, and a null driver draws instantly. The
recording keys on a flag, not the value, because the pre-resize size
in the harness is zero.
The bounded wait with the main-thread pump between slices exists for
Cocoa, where the video thread may need the main thread before it can
signal. Everywhere else there is no trampoline to pump and the slices
were a wake-up per two milliseconds of wait for nothing. The ring now
waits in slices only on Apple; every other platform makes the single
unbounded call it made before, through HW_RING_WAIT_FOREVER, which
each driver maps to its own unbounded timeout.
Three of the four reports against threaded video forced on for macOS
(#19522), traced to their causes; the fourth, that the setting is
hidden, is known and not this patch.
melonDS and SuperModel on glcore locked up and crashed. The ring gives
an OpenGL core a context of its own that must share objects with the
driver's - textures, framebuffer attachments, sync objects - and the
Cocoa GL context driver creates its hardware context with no share
group, so a core on the ring there rendered into objects the video
thread could not see. Until that driver shares the two, OpenGL cores
decline the ring on Apple and run unthreaded, rather than being
installed with a ring that cannot work.
LRPS2 on Vulkan locked up with "Main-thread trampoline stalled". The
ring's fence waits on the core's thread - the main thread - were
unbounded, and on Cocoa the video thread may need the main thread to
run a job before it can signal: a swapchain rebuild marshalled through
the trampoline. Each waited for the other. The fence-wait poke is
bounded now, in all four drivers, and the ring waits in slices with
the main-thread pump between them; off Apple the pump is nothing and
the slices cost a wake-up per two milliseconds.
Pause on focus loss was ignored. cocoa_has_focus() publishes the live
state only when asked on the main thread, and under threaded video
nothing there asks, so the worker read "never published", which reads
as focused - a limitation the function's own comment recorded. The
application delegate's activation notifications publish it now.
A frame's info is built on the main thread when the frame is pushed,
with the output size known then. A window resize is noticed on the
video thread, in the driver's alive() between frames, which records
the new size; a frame pushed before that carries the old one. The
Vulkan driver sizes its swapchain rebuild and its viewport from the
frame info, so on a resize it rebuilt them at the old size, cleared
its resize flag, and drew the menu into a corner of the window while
the content sat at another scale beside it - reported on Windows,
Vulkan, windowed, going back into the menu; fullscreen, which never
resizes, was fine. Before the frame call the video thread now writes
the size the driver itself last reported into the frame info it is
about to draw with. Only that: the size the driver already knows.
Unverified against the report: the harness's null driver sizes nothing
from the frame info, and the menu-frame push drains the video thread
before each push, so the ordering did not reproduce there. The change
can only ever substitute a size the driver has already recorded for a
stale copy of the same field.
The legacy Cocoa companion driver re-dispatches every command it is
handed through command_event(); CMD_EVENT_QUIT and
CMD_EVENT_MENU_SAVE_CURRENT_CONFIG are sent to it as layout-snapshot
notifications, so echoing them back re-enters the sender and
"Save Current Configuration" writes retroarch.cfg forever (#19521).
Match Win32/Qt and drop them.
Tatsuya79 on 6c0cedc945 + 8d67323a80: with vsync on, D3D12 and Vulkan
dragged with a lag and refreshed late for both the main window and the
companion; GL was smooth, and Vulkan was smooth with vsync off. That is
the present in every 16 ms tick blocking for a refresh and starving
DefWindowProc's loop.
A move never invalidates the client area - the compositor keeps the
last buffer - so a plain title-bar drag now presents nothing. The tick
presents only when WM_SIZE changed the routed window's size since the
last present, which is the one case a present is needed for. A
companion drag or resize never dirties the main window, so it presents
nothing either.
Checked: tools/mingw_syntax_check.sh gfx/common/win32_common.c. Needs
the D3D12/Vulkan vsync-on drag and border-resize rows from Tatsuya79.
Follow-up to 6c0cedc945. A floating pane is its own top-level window
with its own wndproc, so dragging or sizing one ran the same modal loop
with none of the handling: video froze stale and audio underran for the
length of the drag. Route WM_ENTER/EXITSIZEMOVE, WM_ENTER/EXITMENULOOP,
WM_TIMER, WM_CLOSE and WM_DESTROY to the same four win32_sizemove_*
functions. On WM_EXITSIZEMOVE the pause is ended before the drop is
applied, since docking the pane can destroy the floating window.
Checked: tools/mingw_syntax_check.sh, tools/companion_win32_test.sh
under Wine. Windows hardware: drag a floating pane over the companion
and drop it docked, and drop it floating; both with the main window's
video visible.
Second attempt at the problem 91920289 tried to solve, informed by its
revert (7fbc868499). Dragging the title bar, resizing by the border or
browsing a menu bar runs a modal loop inside DefWindowProc; with
non-threaded video the run loop is on that thread and stops. Before
this the client area went stale or black and the audio sink underran
and popped on release.
Now: on WM_ENTERSIZEMOVE / WM_ENTERMENULOOP the audio driver is stopped
(unless RUNLOOP_FLAG_PAUSED is set - a user who paused keeps it paused)
and a 16 ms timer re-presents the last frame at the current window
size. Each tick makes the two calls the run loop makes per frame and
nothing else: the driver's alive(), which is where win32_check_window()
consumes WIN32_CMN_FLAG_RESIZED and arms the swapchain resize, then
video_driver_cached_frame(). Nothing runs runloop_iterate(), the menu,
input or the task queue from inside the modal loop; content pauses for
the length of the drag, as in every other windowed game. Threaded video
returns early at enter - the window is on the video thread and the run
loop never blocked.
One timer for the process with a stored owner HWND, so the companion
and the main window cannot kill each other's; a depth counter for
nested loops (system menu opened while sizing); abort on
WM_CLOSE/WM_DESTROY/WM_QUIT with no audio restart. The companion's main
wndproc routes the same messages; its floating dock panes are not wired
in this change. No changes to runloop.c, ui_win32.c or the pace flags.
Checked: tools/mingw_syntax_check.sh on the three .c files and
--consumers-of win32_common.h (sdl3_gfx/uwp_egl_ctx fail identically
on master); tools/companion_win32_test.sh under Wine. NOT yet run on
Windows hardware: needs the row table (drag/resize/system menu/alt-tab/
companion menu bar/quit mid-drag, D3D11 and GL, vsync on/off, threaded
on/off, paused) green from Tatsuya79 and sonninnos before merge.
The facts the decision reads were a struct of eighteen bools: eighteen
bytes gathered, passed by pointer, and loaded one at a time to test.
They are one word now, PACE_FACT_* bits in a runloop_pace_facts_t,
which the tree already speaks everywhere else: the gather ORs bits, the
decision masks them, the sample's table builds its rows with the same
constants. Same decision, same rows; the function reads as the flags
it tests rather than as field names.
Display pacing's hold used the display's period whenever the menu
texture was up. That is right for a stopped core - the frames are the
menu's - and wrong for a core running under the menu with Pause
Content When Menu Is Active off: still content, and at the display's
period it ran at twice its speed on a 120 Hz panel. The frame info now
says whether the core ran this iteration - not paused, and not stopped
by menu_pause_libretro - and the hold keeps the content's period while
it does. The harness measures both at 120 Hz: sixty frames in a second
with the core running under the menu, in half a second with it paused.
The pure decision shadowed the paths through a debug assertion, on
this tree and then in CI, and never disagreed. The paths now call it.
The pace block gathers this iteration's facts once, takes the sources
from runloop_pace_sources(), applies the no-window wait and the gap
limiter - the two steps with an effect beyond the bits, the wait
ending the iteration and the gap limiter setting the timer's period -
through the same predicates the pure function uses; the menu path's
early return decides through the function too. The per-bit conditions
that used to sit in the block, testing fast-forward four times under
two spellings and the menu three times, are gone: a hundred and
twenty-three lines out, fifty in.
No behaviour changes. The menu table pins every row as it was, and a
row that should change is now one line in the function beside its
line in the table, not an edit to a conditional somewhere in the
runloop. The debug job keeps building with assertions live and running
the harness; its comment says what it is for now that the shadow it
was made for has done its work.
Linux-tsan-threaded-video.yml builds with TSan and runs RetroArch
headless with the wrapper on: one configuration. The harness has a
lane for each thing the wrapper does and runs them against the
shipping objects, and under TSan on a developer's machine it found
the send/wait race in the call on the waiting thread. Nothing in CI
ran it that way. This job does: a TSan DEBUG build, every lane at
CYCLES=2, null drivers so no display and no suppressions, and the
harness's own halt_on_error so any report fails the job.
The shadow assertion exists only under DEBUG=1, and every job that ran
the harness linked a release build, so the check that decides when the
paths can defer to runloop_pace_decide() ran on one machine and no CI.
This job builds with DEBUG=1 under ASan and UBSan, runs every harness
lane with the assertion live, and runs the pacing sample's menu table.
A failure names the iteration on which the pure function and the paths
disagree.
What holds the loop each iteration is decided across the menu path,
its early return, the pause path and the pace block, each re-deriving
the same facts - fast-forward four times under two spellings, the menu
three times - and the overlap and the mislabelled early return came
from those decisions not being one. runloop_pace_decide() is that one
decision: a pure function of a struct of this iteration's facts,
returning the RUNLOOP_PACE_* bits. It reproduces the shipping result
bit for bit, the early return included, and says so in an assertion:
debug builds gather the facts once, run both, and assert they agree at
each point the paths decide - the early return, the no-window wait, and
after the gap limiter. The harness runs every lane under it with no
disagreement.
The paths still decide for themselves in this patch; the function only
shadows. Once it has run for a while without the assertion firing, the
paths defer to it and the scattered logic goes. The pacing sample gains
the menu table - thirteen combinations of the sync settings over a
paused core, each row the fact the function returns, so a row that
changes is a behaviour change made on purpose. One row corrects the
table given by hand: Scanline in the menu is Scanline+Timer, since the
menu's refresh-rate timer stands aside for vsync, focus and display
pacing and not for scanline.
The menu and pause clauses put the timer on the loop where nothing
else holds it. Under display pacing the handover holds it there too,
at the display's rate, so with the core paced by Display alone the
menu read Display+Timer - two clocks anchored on the same frame, the
timer's by a sleep and a spin to the deadline. Those two clauses now
stand aside when display pacing is the clock, and the menu is paced
by the same clock as the content: the line reads Display in both.
Fast-forward and VRR keep the timer; neither holds the handover.
The overlay does not draw over the menu, and what paces the menu is
exactly the question that needed answering. The pace string - the
sources that held the loop on the last iteration and the measured
rate beside them - moves into runloop_pace_string(), which the overlay
and System Information both use, so the two can never disagree. The
System Information line is read when the list is built; back out and
in again to refresh it.
The default went on with the patch that took the last two per-API
switches away. The instruction was to make threaded video the default
for hardware-rendered cores - which those switches had gated, and
which now follow Threaded Video on every API with a ring - not to turn
Threaded Video itself on for everyone. This puts the default back where
it was: off, with the Switch keeping its own. Nothing else changes; a
config that sets video_threaded explicitly is unaffected either way.
The menu measurement ran with the core still running under the menu.
The quick menu over a paused core is the common case and a different
path through the runloop - the cached frame, with no core run before
it, and for a hardware core a null frame - so the lane measures it as
well: sixty frames in half a second on a 120 Hz display, the same as
with the core running.
With display pacing as the only clock - audio sync, vsync and the
content-rate sync all off - content paced perfectly and the menu ran
unthrottled: the hold stood down under the menu, and the runloop's gap
limiter had been told to stand aside for display pacing, so between
them nothing paced the menu at all. The hold now holds in the menu as
well, to the display's period rather than the content's, since a menu
is not content and runs at the display's rate. The harness lane
measures it: sixty menu frames in half a second on a 120 Hz display,
beside the content's sixty in a second.
And the overlay's worst latency was the worst since launch, so a hitch
at content load or one menu trip stood on the line for the session and
said nothing about the pacing now. It is the worst over the last two
seconds, and the line says so.
The dock restore and the core picker asked which screen a point is on
through QScreen and QGuiApplication, which are Qt 5, and through
screenAt, which is Qt 5.10; the Linux Qt job builds against older Qt,
and the tree's floor is older still. One helper now answers "the
usable area of the screen this point is on": QDesktopWidget below
Qt 6, which Qt 4 and Qt 5 both have, and QScreen from Qt 6 on, where
QDesktopWidget is gone. The three callers take a QRect from it and
never see a screen object, so the version split lives in one function
and nowhere else.
win32_common.c included dwmapi.h for DWM_TIMING_INFO, and the oldest
MSVC job builds against an SDK that has no such header: the VS2005
build broke on the include. The entry point was already resolved at
runtime with dwmapi never linked, and the header was the last thing
tying the file to a newer SDK than the tree requires. It is gone. The
structure is declared locally, field for field as the SDK lays it out
and byte-packed as the SDK declares it, which changes its size and
without which the call rejects it; on MinGW, where the SDK header
exists, its size is checked against that header at compile time, so a
transcription error is a build error rather than a wrong vblank.
ui_qt.cpp used QGuiApplication::screenAt, which is Qt 5.10, and the
Linux Qt job builds against an older Qt 5. A local walk over
QGuiApplication::screens() does the same on any Qt 5 and replaces the
three uses.
The hold released the core a display period after the presenter's next
vblank. That is the content's cadence only when the display runs at
the content's rate; on a 120 Hz panel a 60 fps core was released every
vblank and ran at four times speed the moment the gap limiter stood
aside, which the last patch made it do - and the overlay said
"Display (247.6 fps)".
The hold now keeps a due time that accumulates in the content's own
period, from the core's declared rate, so the cadence is the content's
over any stretch; each frame goes out on the first vblank at or after
its due time, on the display's grid, and the target is measured back
from that vblank. On 120 Hz a 60 fps core is due every other vblank,
with the odd third one keeping 59.94 exact; on a display at the
content's rate nothing changes. After a stall the schedule restarts
from the presenter's next vblank rather than carrying a backlog.
The harness's display-pacing lane measures at a 60 Hz display and at
120 Hz, sixty frames in a second at both; at 120 Hz it would have
measured a quarter of that before.
Display pacing did nothing on a machine with vsync off, and the
overlay said so in two ways once it could: "Pacing: Timer" with the
setting on, and a "core" reserve of a whole period beside a core that
ran in three milliseconds. Two causes, both real.
With vsync off the runloop's pace came out as none, so its gap limiter
engaged and held the loop to the content frame time on a schedule
anchored on the last frame. That timer fires before the handover's
hold gets its turn; the core runs on the timer's cadence, the push
measures a full period as core time, reserves all of it, and holds for
nothing. The runloop gains RUNLOOP_PACE_DISPLAY, set when the wrapper
is up with display pacing on and fast-forward off, which keeps the gap
limiter out and reads "Pacing: Display".
And the hold stood down on the driver's nonblock state, which
fast-forward sets - and so does vsync off. Pacing a core to the
display's vblank with a present that does not block is the point, not
the exception; the hold now stands down on the fast-forward flag the
frame info carries, and on nothing else.
The harness's display-pacing lane had passed on the gap limiter's
pacing all along, with the hold never running: the harness's menu is
alive over the core, and the hold rightly stands down under the menu.
The lane closes the menu for its measurement and checks the hold on
its own, sixty frames in a second; the pacing sample checks that the
display bit keeps the gap limiter out.
The histogram, the code, and the Huffman build's queues were locals
- six kilobytes once the inliner merged them into one frame, which
is not a frame for a platform with eight-kilobyte threads. They are
a struct allocated once with the encoder's other scratch, and the
file's largest frame is back under the budget. Output and speed
unchanged; every frame still decodes with the reference decoder.
The hardware ring binds the core's context on the main thread and, at
teardown, left it current there for the context driver's destroy to
cope with - which GLX and WGL define, but is not clean. Context drivers
gain release_current(), which makes no context current on the calling
thread; GLX, WGL and the shared EGL path implement it, and the ring's
context_free calls it. That free runs on the video thread inside the
free command the main thread is waiting on, so the ring marshals it to
the waiting thread through video_thread_call_on_waiter(): the context
is released where it is current. bind_hw_render(false) could not do
this, since it binds the driver's own context, the video thread's, on
whatever thread calls it.
gl3 had eight non-constant aggregate initialisers that C89 rejects,
tolerated because glcore is not in the C89 job's configuration; they
are assignments after declaration now, and the compile matrix gains a
C89 lane for gl3 with the flags the Makefile's lane uses, _GNU_SOURCE
included, since -ansi otherwise hides the C99 math names.
The first readout measured to the wrong end. Without a display report
it stopped at the clock after the present call, and a present that
does not block returns in well under a millisecond: DMC3 on the PS2
core showed 0.2 to 3 ms "to present", a number about the CPU's return
from vkQueuePresentKHR and nothing about when the frame is seen. With
a report it stopped at a vblank that had already happened, before the
present, which is earlier still.
The frame just queued goes out on the first vblank after the present.
That is the next_present the presenter already computes: on the
display's own grid when the driver reports one, else on a grid laid
from the clock, which is an estimate and is now labelled as one. The
line reads "to vblank" or "to vblank (est.)".
The bit writer is called once per literal and flushed a byte at a
time in a loop, a sixth of a block's time; it now moves four bytes
out at once when it has them, and close() drains whatever is left.
The literal- and match-length code lookup walked the base table
from the top, where nearly every value lives at the bottom; it is a
binary search. 72 to 90 MB/s at level 3 on the savestate-shaped
buffer, output unchanged. Every frame decodes with the reference
decoder; ASan and UBSan clean.
A savestate's literals are most of its output, and they are bytes
with a few bits of entropy each; raw, they cost eight. The encoder
now builds a length-limited Huffman code over a block's literals,
sends it as Zstandard weights - FSE-coded as two interleaved states,
or two to a byte where the alphabet allows - and writes the literals
as four streams, or one for a small block, in the layout the decoder
reads. The code assignment mirrors the decoder's table: slots by
ascending weight, symbols in order within a weight. A tree as
written is read back through the decoder before it is trusted, and
one that does not read back is not sent. Raw stays where the code
would not pay.
On the 16 MiB savestate-shaped buffer, 128 KB blocks: 9135 to 8718
KiB, which is the reference encoder's own size at level 3 (8771) on
the same input; deflate at level 6 gets 8096 there, a gap that is
the format's fast levels against deflate's per-block tables, not
this encoder against the reference. Speed 72 MB/s at level 3, the
reference's 220 on this box. Every frame decodes with the reference
decoder across the three thousand cases of the encode test and the
multiblock test; ASan and UBSan clean.
The reference library went from the tree; the project still listed
its decoder's assembly file in the four targets' sources and its
include path, and the build stopped at the file it could not find.
Matches are measured eight bytes at a time; a candidate is measured
only when its first four bytes are the position's and its byte at
the best length so far differs from nothing; the repeat check that
runs at every position compares four bytes before it measures; and
on data that yields nothing the search steps further the longer it
has found nothing, and resets on a match. The hash table is one
bucket per position of a full block instead of four to a bucket.
The level means something: the depth of the bucket walk, and lazy
matching from level 3.
On a 16 MiB savestate-shaped buffer, 128 KB blocks, x86-64 -O2: 62
to 95 MB/s at level 1, 79 at level 3, the output within a percent of
before; in the RZIP container, the write 257 to 122 ms against
deflate's 319. Every frame still decodes with the reference decoder
across the three thousand cases of the encode test and the
multiblock test, whose build note pointed at the reference sources
the tree no longer carries.
The output is still a seventh larger than deflate's: literals go out
raw, and on a savestate they are most of the output. Huffman literals
are the ratio, and the next step.
The codec compressed save files and save states are written with:
Deflate or Zstandard, save_compression_codec, Zstandard by default
where the built-in codec is compiled in. The row appears when both
codecs are in the build; applied when the config loads and when the
setting changes. Files written with either codec read back; only new
files change.
The RZIP container's version byte names its codec: 1 is chunks of
deflate, as ever; 2 is a Zstandard frame per chunk, through a
trans_stream backend over the built-in codec, one frame per flushed
transform, which is how the container drives every backend. A reader
takes either version where the codec is compiled in; a writer's
version is the codec chosen through rzipstream_set_write_codec(),
Zstandard by default where the codec exists and deflate otherwise.
The parallel chunk writer uses the stream's backend.
What it buys, on a 16 MiB state-shaped buffer, single-threaded: the
read 63 ms to 9 ms - the built-in decoder is seven times inflate -
and the write 330 ms to 257 ms, for a file a seventh larger; the
built-in encoder runs at one speed whatever the level. A load is
what a user waits for.
samples/streams/rzip/rzip_codec_test writes and reads both versions
byte-exact, checks the header byte, reads each with the other codec
set for writing, and prints the numbers.
Threaded video is on by default on a claim - that with display pacing
it reaches the latency Frame Delay does on the direct path - that the
overlay could not show. It shows it now: each slot records when the
core handed its frame over, and after the frame goes out the wrapper
measures to the present's end, which is the display's own report where
the driver gives one and the clock after the frame call where it does
not, and says which. A moving average and the session's worst, in a
Latency: line beside Core Start:. Repeats present in their own branch
and add nothing: a repeat has no handover to measure from.
With every platform build on HAVE_RZSTD, HAVE_ZSTD had no user: the
configure option, the deps/zstd tree it built, griffin's inclusion
of it, the Makefile.common objects, the UWP and Android include
paths, and the other branch of every site that could take either
backend - the input replays, the RVZ hashing, the .zst archive and
zip-member readers, the CHD hunk decoder - are gone, and each site
is the one path. Every gate that read 'either backend' reads
HAVE_RZSTD.
The configure builds, Android, Apple, UWP and webOS were on the
built-in codec, HAVE_RZSTD; nobody set HAVE_ZSTD, the reference
library, any more; and the console and handheld makefiles - 3DS,
PSP, PS2, Wii U, GameCube, Wii, Vita, Switch, Orbis, Emscripten, DOS,
the Dingux family, the Pandora, the Win32 griffin build - had
neither, so .zst archives, Zstandard CHD hunks, RVZ hashing and
compressed input replays were absent there. HAVE_RZSTD everywhere:
one C file with no dependency, compiled into griffin builds through
griffin.c and into the others through Makefile.common.
The gl driver rebuilds its HW-render framebuffers when a shader change
needs more history textures than it has. Framebuffers are not shared
between contexts, so they must be made in the core's, and under the
hardware ring that context is current on the core's thread, not the
video thread the shader command runs on. The wrapper gains what that
needs: from the video thread, while it is answering a synchronous
command, video_thread_call_on_waiter() runs a call on the thread
waiting for the reply and returns when it has run. That thread is the
core's, which is inside its wait and can service it. With no command
in flight, or off the video thread, the call runs on the caller as it
always did; threadless builds get a macro stand-in.
gl2's shader change marshals the teardown and rebuild of the HW-render
objects through it, with a flush between so the new textures are
visible to the other context before its framebuffers attach them; the
binds inside those paths are skipped under the ring, where the context
is already current where the work runs. gl3's shader change never
touched its HW target, so it needed only the bind at its end gated.
The harness models it: a driver whose set_shader asks for the call,
checked to run on the waiting thread, before the reply, once. Its
first run caught a race: the video thread can take a command and ask
before the sender has reached its wait, so waiters are counted from
the send. And a subframe read of swap_count on the main thread, the
one site of three without the wrapper guard, gets it.
A delta's typical dirty run is a few words, where a call costs more
than the loop that copies them; a redrawn framebuffer or tilemap is
one run of thousands. From thirty-two words a run is a memcpy, on
both the encode and the decode. The format is unchanged and the
harness's round-trips cover both paths. On a 4 MiB state with a 64
KiB region redrawn and scattered few-word changes, x86-64 -O2: the
encode 0.47 to 0.38 ms; the decode's share is a few hundredths of a
millisecond either way, under the block copy that precedes it.
A secondary instance is a second copy of the core's binary loaded
beside the first, so it exists only where the core is a dynamic
library. A build that can load libraries but links its core
statically - HAVE_DYLIB without HAVE_DYNAMIC - has no binary to copy,
yet compiled the secondary's creation and teardown while the
deserialize and the run loop's use of it were compiled out. Every
gate is now the same test, and that build compiles the whole path
out together, with the entry points its callers link against
answering 'unavailable' so run-ahead there is the single-instance
method throughout. The compile matrix builds runahead.c with
neither macro, each alone, and both.
The last value each input tuple returned, compared on every core
poll, was found by walking a heap list up to three times per poll.
A plain joypad's sixteen buttons and its mask, and an analog
device's two sticks and sixteen buttons, now live in a fixed table
indexed by (port, index, id) - a load - allocated once when the hook
is installed; a subclassed device, a keyboard, a pointer, any other
tuple, goes to the list as before, so nothing valid is dropped and a
tuple never set reads 0 from either.
The preempt frame buffers are one allocation at state_size strides,
the product checked before it is asked for, freed once.
retro_game_info.data stays aliased: the copy task copies the core
binary, the data is read only at the secondary's retro_load_game on
the main thread after the task reports ready, and unloading destroys
the secondary - invalidating any in-flight result through the
generation - before the content is freed. The audit is the comment.
The harness extracts the cache's slot rule and the slab and checks
every common tuple has its own slot, the refusals, a set that reads
back, the slab's strides and its refusal of an impossible product.
A patch is applied in two passes: the first walks every token and
checks that each skip and each run of changed words stays inside the
block and that the record ends inside its own bytes; the second
writes. A record that fails the first pass changes nothing and is
refused, so a corrupted ring cannot write past the block or leave it
half-applied. pop hands the decoder the record's extent - from the
record's start to the size_t that pointed to it - and a refused
record is the end of the ring's usable history: the head stays, and
the rewind stops. The format is unchanged.
The harness applies a patch whose skip runs past the block, a patch
with a late token forged, a record cut before its terminator, and
three hundred seeded single-byte mutations; a refused record leaves
every byte as it was.
state_manager_event_init returns when no manager could be made
instead of pushing to a null one. The ring's entry count drops with
the record dropped when the head folds to the start of the ring,
as it does for one dropped for room anywhere else. The codec's
comment says what it is: a reverse delta, the patch from (old, new)
applied to new restoring old.
preempt_input_state returns no input for a port past MAX_USERS, and
an analog (index, id) pair with no bit in the mask - an index past
the buttons, an id past the two axes or sixteen buttons - is
rejected rather than shifted by 32 or folded onto a bit that exists;
the rule is a function the harness extracts and checks against the
reader's layout. The temporary-name generator keeps its state in a
uint32_t and advances it for every candidate, so thirty attempts try
thirty names. The harness checks for the names and the entry count
are on.
samples/state_rewind includes state_manager.c and drives the codec
and the ring directly: the reverse delta round-trips - compress(old,
new) applied to new restores old - at one dirty word, past a
UINT16_MAX-word run, with the tail dirty into the scan pad, and over
two hundred seeded pairs; the scanner stops at the sentinel; the
ring pops what it pushed through wraps and drops. samples/runahead
extracts the core-copy fragment runahead.c marks out and runs it on
a stubbed file system with the real, unthreaded task queue: pending
then ready, one task at a time, a reset in flight discards, a stale
core's result is dropped. Two cases report what later fixes lock -
the temporary names not advancing and the ring's entry count
drifting on a fold - without failing until they land.
Linux-state-runahead-tests.yml runs both, plain and under ASan and
UBSan, on the paths that matter.
The same shape as gl2's ring, on the other OpenGL driver. gl3 keeps one
HW-render target rather than one per texture index, so the ring makes
two more beside it in the core's context - same format, sharing the
depth renderbuffer, which only the core's rendering touches and never
two slots at once - and slot 0 is the driver's own. The framebuffer
the core is handed is the slot's; the frame reads the slot's texture as
its HW-render texture after waiting the core's fence on the server; the
frame's own fence marks the slot readable again. glcore is a core
profile, so sync objects are always there and there is no finish
fallback. The four places the driver would take the core's context for
itself are gated as in gl2, and hardware cores on glcore now follow
Threaded Video like those on gl. The compile matrix gains gl3 under
desktop and GLES3.
Each hardware ring shipped behind a switch until hardware said it was
right: Vulkan and Direct3D 12 lost theirs when they were, and now
Direct3D 11 - swanstation, through the proxy context - and OpenGL have
too. A ring gives a core exactly what the unthreaded path gave it, a
context or interface of its own and a frame it can ask about, so there
is no trade-off left for a setting to express; the two switches go,
and a hardware core follows Threaded Video on every API that has a
ring, which is now all of them the wrapper can drive.
With that, Threaded Video is on by default. What it does today is not
what its sublabel described: the display keeps its cadence when the
core stutters, the core overlaps with the shader chain and the present,
and with display pacing it starts as late as the next refresh allows,
which is the latency Frame Delay reaches on the direct path. The
sublabel says that instead, and the Switch-only default becomes the
default everywhere.
A dry spell discards audio and the estimate's sum starts over, by
design; whether it reaches the baseline again before the run ends is
the run's length, not the estimate. Settling is required of a clean
run, which is what measures it.
A device that runs dry with audio waiting in the pipe had a consumer
that was not running: the machine held the thread, as a sanitizer's
instrumentation does to one waking every few milliseconds. Dry with
the pipe empty is the producer's, which its own lateness count
covers. The 8 ms device's steady-core checks are not evaluated when
either side was held more than twice between them.
The deferred context the ring gives a Direct3D 11 core has a rule the
cores were not written for: the first map of a dynamic resource in
each command list must discard. The cores' stream buffers discard only
when they wrap and map no-overwrite otherwise, so once the ring closed
a command list each frame the next map failed, returned null, and the
core wrote its vertices through it - a crash on the second frame, seen
with swanstation.
The core now drives a proxy ID3D11DeviceContext in front of the
deferred one. Every method forwards; Map rewrites the first
no-overwrite map of a resource in a list to a discard, and
FinishCommandList starts the next list's bookkeeping. The core keeps
writing at its own cursor and drawing from its own offsets, none of
which reach back into the previous list, so the renamed buffer holds
everything the list reads. QueryInterface resolves the base interfaces
to the proxy and refuses the 11.1+ context interfaces, which would
bypass the rewrite; a core that asks for them falls back to what it
can do through this one. The proxy is generated from the vtable in the
tree's own d3d11.h by tools/gen_d3d11_deferred_proxy.py, so its 115
signatures cannot drift from what the driver compiles against.
Still behind Threaded Video For Direct3D 11 Cores, off and now marked
experimental: the map rule is handled, but a deferred context cannot
map for reading or read a query, and the 11.1 refusal is a limit some
core may run into. The sublabel says so.
The OpenGL ring's sync objects compiled under a guard the desktop build
satisfied and the GLES builds did not, and Android, Emscripten, Vita
and PS4 went red together on the first build after merge; before that
a threadless build had been broken for longer than anyone noticed, on
wrapper calls that compile against a prototype and fail only at link.
The matrix script exists for exactly this - a file that compiles for
the configuration one job builds and not for a console's - so it gains
gl2 under GLES2, GLES3 and desktop, the hardware ring under each API
alone and none, and the threadless syntax lanes for the files that had
crossed into the wrapper. Reverting the GLES guard fails the first of
them.
Locked by the caller and not lock-free, with retro_spsc_t named for
the unlocked single-producer case; one byte larger than its capacity,
so size is never a power of two, and neither rounding the allocation
nor masking the wrap is a fix.
The node and its strings are allocated as one block and freed as
one: a push is one allocation with one failure point, an expired
notice one free. pull() keeps the removed node whole as the queue's
temporary, so the message it returned stands until the next pull.
The node type moves into message_queue.c - only the queue touches a
node, and only RetroArch uses the queue - so nothing outside can free
a string that lives inside a block. The harness includes the source
for its invariant checks and covers the block's layout and lifetime.
len is under the ring's size, so an index crosses the end at most
once: a compare and a subtract, not a division by a size that is
never a power of two. The availability macros keep their names and
resolve to inlines that load first, end and size once each, so the
argument is evaluated once and a reader sees one snapshot.
On x86-64, the write step from the availability to the index, with
the copy: 64 B 4.4 to 2.7 ns, 256 B 5.1 to 3.9 ns, 1024 B and above
within the copy's noise. ARM, where a variable division costs more,
is not measured here.
message_queue: randomised push, pull and extract at every capacity
from 1 to 8 against a reference multiset, the heap property, node
uniqueness and cleared slots checked after each step; pull's
lifetime; allocation failure at each of a push's three allocations
through wrapped malloc and strdup; construction at SIZE_MAX.
fifo_queue: the unchecked and the checked calls against a reference
deque at capacities 1, 7, 64 and 4096; the lengths a checked call
must survive, to SIZE_MAX with the index at the end of the ring;
construction at 0, 1 and SIZE_MAX. In the libretro-common samples
job under ASan and UBSan.
The node, the message and the title are allocated before the heap
is touched; a failure at any of them frees the rest and leaves the
queue as it was. msg_queue_try_push() reports that and a full queue;
msg_queue_push() keeps its signature as the silent wrapper. A
duration of zero is one pull: nothing in the tree asks for zero, a
core's message shorter than a frame rounds to it, and shown once is
what that means.
fifo_write_checked() and fifo_read_checked() clamp to what is
available and return how much moved, so a length past it is a short
copy and never one past the ring. fifo_write() and fifo_read() keep
their names and their contract - the caller has established the
length under its own synchronisation, as the audio drivers do - and
the header says so. The tail test is written so it cannot wrap
size_t, and SIZE_MAX is refused before len + 1 does.
pull and extract share one removal: the last node moves into the
hole and sifts down comparing priorities, over the nodes [1, ptr)
with ptr the next free index, and the vacated slot is cleared. The
heap is len + 1 slots and SIZE_MAX is refused before it wraps.
The ring's fences used glFenceSync, glWaitSync, glClientWaitSync and
GL_TIMEOUT_IGNORED under HAVE_GL_SYNC alone, but the driver defines
that unconditionally and guards its own fence path with
!HAVE_OPENGLES on top: on the GLES2 builds - Android, Emscripten, Vita,
PS4 - those names do not exist, and the PowerPC macOS headers lack
GL_TIMEOUT_IGNORED. The ring's sync code now compiles under exactly the
driver's condition, defines GL_TIMEOUT_IGNORED where the headers do
not, and the GLES builds finish instead, which the fallback already
did. A GLES-only build also left the ring's interface setup with an
unused local, which builds with warnings as errors turn into one.
Two C89 breaks in the same lane, neither from this series but both in
front of it: runloop_event_deinit_core() and x11_input_state() each
had a statement before their declarations.
An OpenGL core renders with the context that is current on its thread,
into a framebuffer the driver hands it. Unthreaded that is the driver's
own context; under the wrapper that context is the video thread's, and
a context is current on one thread at a time. The gl driver already
has a second context shared with its own - the shared-context mode,
where a core's state is kept apart from the driver's on one thread -
and its HW-render framebuffers are already made in that context. The
ring uses it across two threads: forced on when the ring will drive
the core, taken by the main thread before the core's context_reset,
and never taken by the video thread again. The HW-render texture count
becomes the ring's slot count, so the core renders into one texture
while the frame reads another, and the framebuffer the core is handed
is the slot's rather than one chosen from video-thread state.
Synchronisation is sync objects, shared between the pair: the push
fences the core's rendering into the slot and flushes, the frame waits
that fence on the server before reading the slot's texture, the frame's
own fence marks the slot readable again and the core's thread waits it
client-side before rendering there. Without sync objects, finish.
Behind Threaded Video For OpenGL Cores, off by default and experimental:
gl only for now, and a shader change while such a core runs recreates
the framebuffers from the video thread, which cannot take the core's
context - not yet supported, said so in the sublabel. The wrapper's poke
table gains proc-address forwarding, which OpenGL cores need and which
was never there because they were never allowed.
A renderer's is_threaded argument means "you are not on the context
thread, bind it yourself"; the GL renderers answer it with
make_current(), and on free that unbinds the context from the calling
thread. Under the wrapper both calls are marshalled to the video thread,
so they now arrive there with is_threaded false: the worker's context is
already current and stays that way across a deferred font release.
The threaded_video harness gains a lane that drives font init, the
deferred free and the immediate free through a renderer that keeps a
context flag the way GLX does, and the harness now runs in the menu
harness workflow.
Sanitizer sweep: ASan+UBSan and TSan clean on the harness.
The stream's state there lives in the VFPU - c720 the accumulating
output pair, c730 the fractional position - and reset zeroes them as
init does, leaving the constants; it takes no handle, as init hands
out none.
A Direct3D 11 hardware core is handed the driver's immediate context
and leaves its output bound at pixel shader slot 0 on it; the driver's
frame reads it from there. The immediate context is not safe to share
between threads, and under the wrapper it is the video thread's, so
the core gets a deferred context of its own instead: it records there
on the main thread, and at each push the wrapper closes the recording
into a command list and takes the texture bound at slot 0 - the same
contract, read from the core's context - into the slot. The video
thread replays the list on the immediate context before its frame and
reads the slot's texture in place of slot 0. Every use of the core's
texture executes on the immediate context in order, so no GPU fence is
needed; the ring's fences are events that mark a slot replayed.
Behind Threaded Video For Direct3D 11 Cores, off by default, because a
deferred context cannot map a resource for reading or read a query, and
a core that does either - the readback paths some of these cores have -
misbehaves on one. That is a trade-off the core decides, not the ring,
so unlike Vulkan and Direct3D 12 this keeps its switch. The sublabels
say which cores run threaded and when.
The reverse buffer for a float core holds its frames as it gave them
and plays them back as float: no conversion in, none out, and the
rewound audio is not quantised. The recorder still takes int16, from
the same staging as the forward path, converted only while it
records. Netplay intercepts the int16 callbacks by swapping them; the
float batch entry the core holds by pointer cannot be swapped, so it
asks a gate netplay sets - the same skip as the int16 batch, for a
replayed frame during rollback or a stall - and drops the audio the
same way.
A core that negotiated float output publishes float frames into the
ring and the consumer flushes them as float; an int16 core's ring is
int16 as before. The runloop records the negotiation when it hands
the float batch entry out and clears it when the core goes; the ring
takes its format from that at pipe init, or on an empty ring when the
negotiation lands later. A publish in the other format - the menu,
rewind, the accumulator on a float ring - is converted into the
producer's staging a pass at a time. The recorder keeps its int16
tap, converted only while it records. The consumer's bounce and the
producer's staging move to an arena sized for a float pass.
The pipeline harness gains a float mode: a float core on a float ring
into a float device, publishing a signal a tenth of an int16 LSB
high, which must arrive as published.
Every count on the pipe - the pass bound, the target, the priming,
the discard, the consumer's read, the estimate's source count - is in
stereo frames, and bytes appear only at the ring's edge through
pipe_frame_bytes, the ring's unit. int16 today; the unit is what a
float core will change. Same behaviour, same bytes on the device.
The writable size comes from the write callback's argument and is
refreshed after each write; the sink's own latency from the latency
update callback. Both are atomics the frontend reads every frame with
no lock - stale by at most one server period, and on the full side,
since only a callback raises the writable size and every write lowers
it at once. frames_consumed(), read once a window, still takes the
lock for the latency query.
A Direct3D 12 hardware core hands the driver a whole texture each frame
through set_texture, which the driver copies into its own frame texture
at frame time; the core keeps rendering into the same texture. That
holds up unthreaded because the copy is queued before the core's next
frame on one in-order queue. Under the wrapper the frame runs later, on
the video thread, so the core could overwrite the texture before the
copy was even queued.
So the ring takes the copy itself, on the core's thread, at the push:
the driver copies the core's texture into a texture of its own for the
slot and submits that copy right there, ordered on the queue before
whatever the core submits next, and the video thread's frame reads the
slot's copy exactly as it would have read the core's. The ring code
becomes API-agnostic in its bookkeeping - slots, index, fences - with a
per-API payload and interposed interface: Vulkan's image, semaphores and
command buffers as before; Direct3D 12's texture and format. Two
D3D12-shaped pokes carry it - capture, on the calling thread, and
present_slot, on the video thread - beside the Vulkan-shaped install.
The fences are ID3D12Fence with an event, signalled queue-side.
The core's texture size is read through ID3D12Resource::GetDesc, which
the C vtable declares struct-by-value; the call goes through the
hidden-return-pointer signature mingw actually emits, as the driver
already does for GetCPUDescriptorHandleForHeapStart.
retro_resampler_t gains an optional reset(): the stream's history -
the rings, the phase, the fractional position - is forgotten, the
configuration stands, and no allocator is touched. It is last in the
table so an implementation that does not set it is NULL there and the
caller re-creates the state as before. sinc, nearest and CC have it.
The frontend resets the resampler when the unity bypass stops being
taken, on the thread that resamples, instead of freeing and
re-creating it there.
samples/audio/resampler_reset runs each backend at the ratios the
frontend meets and requires a reset stream to match a fresh one to
the sample, with allocation counted through wrapped malloc, calloc
and memalign_alloc.
audio_driver_effective_ratio() composes the current ratio - rate
control's adjustment and the sink bias in it - with slow motion and
fast-forward's speedup; audio_driver_output_bound() is the resampler's
overshoot margin on top. Both flush paths and the consumer's
preflight use them, so the room the consumer waits for is the room
the write needs: with slow motion on the reservation was a third of
the output. A harness may define AUDIO_OUTPUT_BOUND_CHECK to count a
flush that produced more than its bound; the shipping build has no
check on the audio thread. The pipeline harness counts, in every
mode, and requires none.
The device usually has the room; a wait before every write was a
syscall and a scheduler round trip for nothing. The write is
attempted first, a wait follows only when the device says EAGAIN,
and the wait is bounded at one buffer's worth; a device that reports
no space twice running gets a short write, not the audio thread. XRUN
and suspend go through the recovery path as before, and the
non-blocking write is unchanged.
The lifecycle harness runs the write loop against a device that takes
the write whole, one that says EAGAIN once, one that underruns, one
that never accepts, and the non-blocking case.
alive() is false after stop() and true after start() whether or not
the hardware can pause. A device that can pause holds its buffer
across the stop; one that cannot, or whose pause fails when asked -
a USB gadget, the Pulse and PipeWire plugins - has its buffer dropped
and the stream prepared again on start, the first write starting it.
The frontend's ring is the source of truth for what was queued.
samples/audio/alsa_lifecycle links the shipping driver against the
null PCM with the lifecycle calls scripted, and runs the three kinds
of device and a hundred stop/start rounds; its blocking-write
scenarios are for the write loop that follows.
The ring shipped behind a setting, off, marked experimental, because
the GPU side could only be proven on hardware. It has been: a Vulkan
hardware core runs on it correctly. The ring gives such a core exactly
what the swapchain gave it unthreaded - a sync index it can ask about
and an image that is free when it says so - so a core that honoured the
interface there honours it here, and there is no trade-off left for a
setting to express. Vulkan hardware cores now follow Threaded Video
like software cores do; other hardware contexts have no ring yet and
stay unthreaded.
Three sublabels caught up with the code while here: Threaded Video's
no longer says hardware cores always run unthreaded, Frame Repeat's no
longer lists which backends can repeat (the statistics line names what
a session got), and Vertical Refresh Rate's no longer says threaded
video ignores it, which has not been true since the wrapper paced on it.
A hardware core renders with the video driver's device, and on the
unthreaded path the swapchain is its clock: get_sync_index is the slot
the driver is on, and the driver's swap has already waited that slot's
fence before retro_run runs. Under the wrapper the video thread moves
that slot during acquire and present, so a core on the main thread
reading it would render into an image the GPU may still be presenting -
which is why hardware cores have run unthreaded whatever Threaded Video
was set to.
The wrapper now gives such a core a sync index space of its own: a
three-slot ring, each slot a copy of what the core hands over for a
frame - image, semaphores, command buffers - and a fence. The interface
the core receives is the driver's own with the bookkeeping calls
redirected to the ring; queue locking and the instance proc address
forward to the driver, which already serialises queue use because cores
may submit from other threads. get_sync_index is the ring index,
wait_sync_index waits the slot's fence, and the frame push publishes the
slot the core just filled and waits the one it fills next, so a core
that never calls wait_sync_index (it was a no-op) is still safe. The
video thread installs a slot's state into the driver through a poke
before the frame call and fence-signals after it, with an empty queue
submission under the queue lock, which completes after the frame's own.
The ring is freed before the device.
Vulkan only, and only where the driver provides the hw_ring pokes; the
vulkan driver does. Behind Threaded Video For Vulkan Cores, off by
default and marked experimental: a core that reuses an image without
asking whether it is free shows corruption here, which is the failure
the setting exists to keep away from everyone else. The ring lives in
its own translation unit so the wrapper stays free of API types, and
compiles to stubs without HAVE_VULKAN.
Three breaks, found by building and linking with --disable-threads
rather than compiling single files: the wrapper header declared
video_thread_swap_count() as a function while video_driver.h's
threadless branch defines it as a macro, so any includer of both failed
to parse - the header's body is now under HAVE_THREADS, since the object
it declares only exists then, and callers keep the macro stand-ins; the
microphone capture path read worker_sample_size, a field that only
exists with a worker; and the Linux illuminance sensor created and
joined its polling thread unconditionally, which predates all of this.
The picker's last column tracks the table's width through AppKit's
own tiling (NSTableViewLastColumnOnlyAutoresizingStyle, a 10.4 API
GNUstep has too), whatever the timing of the window's resize against
the layout in coresLayout; the width computed there from the clip view
is then a starting point rather than the only thing keeping the
columns inside the table.
ARC infers NSView** as an autoreleasing out-parameter, which cannot be
fed from a local NSView* array. The views are the controller's own
strong ivars and the array is scratch for one call, so mark both the
parameter and the callers' arrays RARCH_UNSAFE_UNRETAINED; under MRC
the qualifier is empty. Drop the two declared-but-never-defined
methods that were left in the interface.
gl2 and gl3 stamp the shader swap counter through
video_thread_swap_count(), which is declared in video_thread_wrapper.h
and only reachable under HAVE_THREADS. Give video_driver.h's threadless
fallback block a macro for it that reads video_st->swap_count directly,
next to the existing video_driver_thread_wrapper_active() fallback: with
no wrapper there is no other thread to read it under a lock.
wgl_ctx.c and w_vk_ctx.c report the last present time through
win32_dwm_last_vblank_time(), which lives in win32_common.c, and that
file is not part of the UWP build. Add it to the UWP win32_* set: DWM
composition timing is not reachable from an app container, so it
reports no timestamp and the presenter paces on its own clock.
video_driver_frame() read video_driver_state_t::thread_wrapper_active
directly, but that member only exists under HAVE_THREADS; go through
video_driver_thread_wrapper_active(), which is a constant false without
threads. audio_driver_pipe_target_frames() is only called from the
HAVE_THREADS pipe path, so it now lives under the same guard.
The Help menu of every companion keeps About and Documentation and
loses the two items that carried no RetroArch information of their
own: the Contributors window (and the button for it in the About
dialog), whose text - AUTHORS.h, its generated C copy AUTHORS_c.h and
the two scripts that produced and checked them - goes with it, and
Qt's About Qt dialog. The QT_MENU_HELP_ABOUT_CONTRIBUTORS label leaves
msg_hash.h and every translation header, the Cocoa harness and the
parity script no longer look for the window, and CI no longer checks
the generated copy against its source.
The Load Core picker sizes its Version column from the clip view the
table sits in rather than the table's own frame, which is not always
re-tiled by the time the window's frame has been set (CI's GNUstep
left the columns 6 px wider than the table), and sizes the last column
to fit after that. The Cocoa harness's saved-geometry checks expect
the height the driver actually restores: the saved 700 where the
screen's visible frame holds it, the clamped height where it does not
(CI's Xvfb is shorter than a desktop's).
A core that asks for a framebuffer through
RETRO_ENVIRONMENT_GET_CURRENT_SOFTWARE_FRAMEBUFFER was always told no
under the wrapper, so it rendered into its own buffer and the push
copied that into a ring slot, on the main thread, every frame. The
wrapper now lends such a core a slot that neither side holds - free by
the same test the push uses - and a push whose data is that slot's
buffer publishes it as it stands, with no copy. Any other push lets the
loan lapse and copies as before; dupes do exactly that. The loan is
declined when no slot is free, when the geometry does not fit, and when
the core asks for read access: the lent slot last held the frame before
the previous one, not the previous one, so a core that reads back would
see the wrong image, and the copy path is what it expects. A core that
never asks is unaffected, so there is nothing to set.
The harness core now uses the framebuffer interface and declares its
pixel format, which it never had - the frontend was treating its 16-bit
frames as XRGB8888 - and a lane checks that its asks are granted and
those frames publish zero-copy, with the dupes lapsing the loan. The
teardown statistics count zero-copy frames.
With both native companions rendering the shared dock model, nothing
reads the rows as the fixed grid any more: companion_dock_grid_read /
write and their test go, and the rows have one reader and one writer
on every backend.
The same for the Cocoa companion, within the 10.4 baseline. The
panes' views and the content view live in RACompanionDockView, a
flipped view under the content view, so the model's top-down
coordinates are the view's own: layoutViews frames each pane's views
in the rectangles the model lays out, hides a hidden pane's, and
moves a floating pane's into a RACompanionFloatWindow of its own. The
dock view draws the strips, tab bars and drop marker and takes the
mouse on them (strip drag, float and close glyphs, double-click to
float, tab to raise, gap to resize with cursor rects); the float
window's title bar is handled in sendEvent: - a double-click docks the
pane back, a drag is tracked by the controller itself so the window
follows the pointer and a release over the docks drops the pane where
a strip drag would. The caption labels, the segmented thumbnail
control and the single boxart view go; four RACompanionBoxart views
take their place, each installing a dropped image as its own type.
The Companion menu's three toggles become Qt's Closed Docks submenu,
filled as it opens (guarded against GNUstep re-asking the delegate
from inside the item edits).
The GNUstep harness drives it the same way the Win32 one does, through
the controller's mouse methods on the dock view and the float drag
hooks.
The Win32 companion's panes become docks in the sense the Qt
companion's are. The shared model owns where every pane is; the
driver moves each pane's controls into the rectangles the model lays
out, hides a hidden pane's controls, and re-parents a floating pane's
into a RetroArchCompanionDock window of its own (the controls'
notifications are forwarded to the companion's window procedure,
closing the window hides the pane, a caption double-click docks it
back, and dragging it over the companion window docks it where the
pointer is: WM_MOVING tracks the pointer against the dock geometry
and shows the marker, WM_EXITSIZEMOVE applies the drop). The title
strips, the tab bars of tab groups and the drop marker are painted in
WM_PAINT, since no control draws there; a strip drags its pane on
mouse capture the way the splitters did (its float and close glyphs
are DrawFrameControl caption glyphs, a double-click floats it), a tab
raises its pane, a gap resizes with the size cursors. The fixed grid,
the splitter code, the caption STATICs and the one thumbnail pane with
its own tabs go: there are four thumbnail panes now, each requesting
its own type (the pane index rides in bits 29-30 of the request tag,
browser previews moved to bit 28), so a layout with Screenshots above
Title Screen renders as it does in Qt. View gets Qt's Closed Docks
submenu, filled with the hidden panes as it opens.
Three frames that were over the 4 KiB ceiling come down with it:
cw_core_combo_fill's launch-option scratch lives in the driver struct,
the New / Delete Playlist bodies are functions of their own rather
than buffers in the window procedure, and cw_add_files_dialog keeps
its join buffer on the heap. The harness drives the model through
the real messages: default rows, a saved layout restored, gap drags,
a strip dragged into a tab group and out to a float, the float
dragged back over a pane, the caption double-click, the close glyph
and Closed Docks.
The Qt companion's panes are QDockWidgets: they can be dragged onto
any side of the window, stacked into tab groups, floated and closed,
and the layout persists through the dock rows. The Win32 and Cocoa
companions had a fixed grid that the rows were mapped onto, so a
layout arranged in Qt (Tatsuya79's Screenshots above Title Screen)
opened as something else natively and nothing could be rearranged.
companion_dock.c gives all three the same model in plain C89: four
sides of ordered slots, each slot a pane or a tab group with a raised
member, floating rectangles and a shown flag per pane. It reads and
writes the shared rows, applies the edits (place, tabify, float, show,
raise), lays the model out inside a client rectangle (pane, strip,
tab-bar and gap rectangles plus the content rectangle, the sizes
used written back so what was on screen persists), hit-tests strips,
tabs and gaps, moves a gap, and turns a drag position into a drop
target the way QMainWindow does: over a pane's first or last third a
slot before or after it, over its middle its tab group, along an
edge of the client the end of that side, elsewhere floating.
The native backends become renderers of this model in the patches
that follow. The core test covers the default layout, the geometry,
hit-testing and drags, the drop rules, the edits keeping a group's
raised member valid, the rows round-tripping as a fixed point, and
the rows the Qt companion itself wrote for the reported layout
reading into it.
Threaded video paced the runloop on a timer - hand a frame off, then
release the main thread a period after the last release - which drifts
against the display and lets a frame of slack build up between the core
finishing and its frame reaching the screen. That slack is the latency
people notice with threaded video, and it is not the copy or the thread
switch. With Threaded Video Display Pacing on, the main thread is held
after each handoff until the next core frame's display slot minus what
that frame will need: the render time the video thread measures for
driver->frame(), the core-plus-runloop time measured between handoffs,
and a margin. The slot comes from the presenter's next_present, so it
is display-phased wherever a driver reports and a period past the last
present otherwise. A frame that runs long past its slot is repeated by
the presenter instead of missed, which is the guarantee frame delay on
unthreaded video does not have. Off by default; the timer paces as
before. Skipped in fast-forward and under the menu.
The presenter's per-frame inputs were written by the video thread
before it took the publish lock and now the main thread reads them
under it, so they move into the publish block. The statistics overlay
gains a Core Start line with the two reserved times, and the harness a
lane checking that the pacer holds sixty frames to about sixty periods
with none dropped.
The sublabel described what the measurement no longer carried in a way
that read as a pacing improvement. It is a readout; frames are presented
exactly as before.
It was listed in Frame Time Counter beside the other sampling control,
three menus from 'Estimated Screen Refresh Rate' in Video > Output,
which is the only thing it changes. It is in Output now, directly under
that readout, and in one place only.
The FrameTime line now says whether the value behind the refresh-rate
estimate was measured from the display's reported present times or
from the frame loop, so the Sample Frame Time From The Display setting
can be seen taking effect - or not, on a session with nothing to report
or under the threaded wrapper - the way the Repeat line already names
the presenter's cadence source.
VK_GOOGLE_display_timing is the device's own report of when a present
reached the display, and NVIDIA's Windows driver does not expose it, so
a Vulkan session there had no reporter at all and stayed on the clock.
On current Windows a Vulkan swapchain presents through DXGI under the
compositor regardless, so DWM's vertical-blank timestamp is the same
signal the D3D drivers already read from their swapchains. The DWM
reporter moves from the GL context into win32_common, the Win32 Vulkan
context exposes it through the context slot, and the Vulkan driver's
poke asks the device first and the context second. dwmapi stays
resolved at runtime and unlinked.
The two settings' descriptions drop their lists of which backends
report; that list changes as reporters are added and belongs in the
statistics line, which names what a session actually got.
'Estimated Screen Refresh Rate' is measured from the frontend's frame
loop, so it carries the core's cadence and the loop's own jitter along
with the display's period. The drivers that can report when a present
reached the display now have a second use for it: with Sample Frame Time
From The Display on, consecutive reports give the period directly. Off
by default and off wherever nothing can report, so the estimate keeps
measuring exactly what it always has unless asked otherwise - the
existing sampling modes, their reset toggles and everything downstream
of the estimate are untouched in that case.
Not applied under the threaded wrapper: the video thread owns the query
there and this thread's frame boundaries are not presents. A report that
has not moved since the last frame - compositor bypassed, driver not
ready - leaves the loop measurement in place for that frame.
Qt's picker has no Load / Cancel buttons - a double-click or Return
loads the row, Escape closes - and instead a "Load Custom Core..."
button at the bottom left and "<version> - <core>" in a status bar.
The native pickers had Load / Cancel and nothing else. Both now match:
the buttons are gone, "Load Custom Core..." opens the file picker and
loads the chosen library (the Load Core window goes away on success;
Cocoa also gains the entry in its Companion menu, which Win32 and Qt
already had in File), the status bar shows the version and running
core, Return and Escape still load and close (Win32: NM_RETURN and
LVN_KEYDOWN on the list; Cocoa: two zero-size buttons carrying the key
equivalents, the window having no responder of ours to take
cancelOperation:). The picker's layout accounts for the status bar and
Win32's list and button take the companion's font.
Also fixed on Win32: every row but the first showed no version. The
list sorts on insert (LVS_SORTASCENDING), so the version written to
row i landed on whichever core had sorted there; it now goes on the
row the insert reports.
Tests: both harnesses assert no Load / Cancel, the Load Custom Core
button, the status text, and (Win32) the last row's version. Verified:
companion_win32_test (mingw + Wine) OK, companion_cocoa_test (GNUstep)
plain and asan OK, parity green, Win32 TU in the C89 lane with and
without HAVE_THREADS.
The presenter asks the driver when its last present reached the display
and paces the next repeat from that; where nothing can say, it uses the
frontend clock. Which of the two is in use was decided entirely by what
the platform happened to support, with no way to choose and no reason
given. It is a setting now - Pace Repeats From The Display, on by
default, listed under Frame Repeat and only where a repeat can happen -
and the video thread carries it across with the frame, so turning it off
puts every driver on the clock. The statistics line already names which
one a session ended up on.
The Win32 GL context gains a reporter too, from the compositor's
qpcVBlank via DwmGetCompositionTimingInfo: a reported timestamp on the
same clock as the DXGI and GLX paths, not a scanline estimate. dwmapi is
resolved at runtime and not linked, as the D3DKMT entry points beside it
are, so nothing changes for targets without it. A context that bypasses
the compositor reports a timestamp that stops advancing, which the
presenter already treats as absent.
Qt sized the picker to half the companion's width and its full height,
then only the width to the columns: with a large or maximised companion
it ran off the bottom of the screen, and the width left out the row
header and the scroller so both scrollbars appeared. Win32 opened it at
a fixed 420x400 however long the list, and Cocoa the same. One rule
now, companion_place_window() in the shared core: the window takes the
size that shows every row and both columns without a scrollbar, never
below a 420x400 floor, never above the work area of the screen the
companion is on less a margin, centred over the companion window and
then moved inside that area. Each backend measures its own need (row
and header heights, the widest name, a scroller's worth for when the
height is clamped, its button strip, its frame) and applies the result;
the Version column takes any spare width so no horizontal scrollbar is
needed, and a list taller than the screen scrolls inside a window that
fits it. Win32's picker layout scales with the DPI as the rest of it
does.
Tests: companion_core_test covers the rule (fits, taller than the
screen, below the minimum, owner off the edge, an offset work area, a
screen smaller than the minimum). The Win32 and Cocoa harnesses gain a
fixture core list (stub_core_count) and open the picker over twelve
cores - no smaller than the floor, inside the work area, every row and
both columns visible with no scrollbar, centred - and over eighty:
clamped to the work area with the list scrolling. Verified: both
harnesses OK (Cocoa also asan); the Qt build run offscreen with 10
short cores opens the picker at the 420x400 floor with no scrollbar,
with 30 long-named cores at 468x768 inside an 800x800 screen with only
the vertical scrollbar; persist test OK; C89 lanes clean.
The legacy GLSL chain builds its own preprocessor defines rather than
going through glslang_util, so it never advertised _HAS_SWAPCOUNT_UNIFORM
and never looked the uniform up: presets took their fallback path there,
which was at least honest, but the count is as meaningful for a GLSL
preset as for a slang one. video_shader_ctx_params_t carries it now, gl2
and gl3 fill it at each of their pass setups - from video_info where it
reaches, and from the shared counter in the render-chain passes, which
are intermediate passes of the same present - and the chain uploads it
unmodulated, since a pass's frame_count_mod has nothing to say about
what the display was shown.
The semantic shipped wired for the Vulkan and GL3 chains only, so a
preset reading SwapCount on a Direct3D backend got the slot's zero
default rather than the presents the display had seen - a shader that
looks broken rather than one that reports the feature missing. Each
chain now carries it per pass beside CurrentSubFrame, from the counter
video_driver_frame() and the video thread keep, with the sub-frame
offset added in the sub-frame loop as the other per-present semantics
do here.
The last item of the Core combo did nothing when picked on the native
companions - the selection handlers only refreshed the Core Info pane -
while in Qt it is an action: picking it opens the Load Core window, and
the combo goes back to its previous pick when that window closes
(last_launch_with_index). Both natives now do the same. Every real pick
is remembered; picking "Load Core..." opens the window instead, and each
way out of it (Load, Cancel, the close box: Win32 sees them all as the
window being hidden in WM_SHOWWINDOW, Cocoa routes them through
-coresDismiss, the window's delegate included) puts the remembered pick
back and refreshes the info pane, so the combo never sits on the action
item.
Tests: both harnesses pick the item, assert the Load Core window is up,
cancel it, and assert the combo is back on the previous pick. Verified:
companion_win32_test (mingw + Wine) OK, companion_cocoa_test (GNUstep)
plain and asan OK, Win32 TU in the C89 lane with and without
HAVE_THREADS.
config_save_credentials() now takes the loaded retroarch.cfg and copies
any sensitive key the current build has no setting for into
retroarch-keychain.cfg, and config_save_file() strips the full key list
from retroarch.cfg once the keychain is written. A build without a
feature (SMB client, cloud sync, cheevos) previously carried that
feature's secrets over as unknown entries on every save.
The native companions laid their panes out on fixed numbers - 280 px
columns, Core Info always above the thumbnails at half the column, the
log at 120 - and neither read nor wrote the desktop_menu_dock_* rows or
desktop_menu_window_*, so a layout arranged in one companion was not
the layout the next one opened with. Both now read the rows through
companion_dock_grid_read() at init: the left and right column widths,
which of Core Info / the thumbnail pane / the log is shown, which of
the two right panes is on top and the ratio of their heights, the log
height and the raised thumbnail tab (a thumbnail dock split out of the
tab group in Qt is the tab shown here when it is the topmost one). The
layout on screen goes back through companion_dock_grid_write() from
every relayout while the window is up and from CMD_EVENT_QUIT /
CMD_EVENT_MENU_SAVE_CURRENT_CONFIG, which reach the backend before
retroarch.cfg is written - the event_command hooks were empty. The
window frame is restored from desktop_menu_window_* under "Remember
Window Geometry", clamped to the work area, and saved on move and
resize; Win32 scales the logical values by the monitor DPI as the Qt
companion does, Cocoa keeps the row's top-left origin.
Win32 gains three splitters next to the existing left one: the gap
before the right column, the gap between Core Info and the thumbnails,
and the gap above the log, so the sizes the rows carry can be set here
and not only in Qt; the cursor follows the gap's direction. Cocoa takes
its sizes from the rows (it had no splitters and still has none).
Tests: both harnesses write the default layout back as rows Qt reads,
open a second driver from the reported Qt layout (Screenshots above
Title Screen, Core Info hidden, wider left column, log shown, a saved
frame) and assert the tab, the visibility, the column widths, the log
height, the frame, the re-saved rows, and Core Info coming back below
the thumbnails when toggled on; the Win32 one also drags two of the
splitters. Verified: companion_win32_test (mingw + Wine) OK,
companion_cocoa_test (GNUstep) plain and asan OK, companion_parity
green, Win32 TU in the C89 lane with and without HAVE_THREADS, frames
under 250 bytes.
Two docks stacked on one side came back in the built-in dock order
whatever the user had arranged (Tatsuya79: Title Screen back on top of
Screenshots after every restart, and Screenshots the smaller of the
two), because a row said which side a dock is on but not where along
it, and restore re-added docks to their side in list order. Each row
now ends with the dock's slot on its side, counted from the top or the
left, tabs sharing their group's slot:
<area>,<shown>,<width>,<height>,<tabbed_with>,<raised>,<order>[,<x>,<y>]
The field count tells old rows apart (six or eight fields: order 0, a
floating row's x,y where they were), so rows written by earlier builds
still parse. On save the slot comes from the laid-out docks' geometry
along the side - only a shown, raised or standalone dock has one, a
tab that is not raised keeps a stale off-screen rectangle - and on
restore docks are re-added sorted by side and slot, which with
addDockWidget() appending to a side rebuilds the stacking exactly. The
size of a tab group is written by its raised tab rather than the
earliest dock in the list (whose geometry is stale when it is not the
one raised), a size of 0 follows the slot on restore, and a hidden
dock keeps the size its row already had so it does not lose it while
hidden.
The dock list, its keys and the settings_t rows move into the shared
core (companion_dock_id, companion_dock_key, companion_dock_row) so
every backend uses one table; companion_dock_grid_read/write map the
rows onto the fixed pane grid the native companions lay out, for the
Win32 and Cocoa change that follows.
Tests: companion_core_test covers the order field round trip (docked
and floating), the six/eight-field rows, out-of-range order, the dock
keys, and the grid mapping of the reported layout (Screenshots the
topmost split-out dock becomes the raised tab at the taller height,
Core Info hidden and below, a write/read fixed point).
companion_qt_persist_test seeds Screenshots 420 above Title Screen
210, both standalone, and asserts the slots and the taller dock after
a restore. Verified: persist test OK, a full nine-row config re-saves
identically across three launches at the 2:1 ratio; core test plain,
asan+ubsan+lsan strict, tsan; C89 lane with and without HAVE_THREADS;
ui_qt.cpp -Wall -Wextra clean under Qt 6.4.
The wrapper's command mailbox is a single slot (cmd_data) and
video_thread_wait_reply() wakes on the command type alone, so two
user-side threads posting at the same instant can both leave with the
first reply. The video thread then runs the second packet after its
poster has returned, with a payload pointing into a dead stack frame.
Seen on a handheld as a segfault in video_texture_load_wrap_gl3_mipmap
(ldr x0, [x0] with x0 = 0xffff00000002) whenever a RetroAchievements
unlock uploaded its badge (main thread, CMD_CUSTOM_COMMAND through
video_thread_texture_handle) while the automatic screenshot the same
unlock asked for ran on a task thread and reached the video thread for
the cached frame -- also CMD_CUSTOM_COMMAND, so the waiters could not
be told apart. Reproduced on three unlocks across two cores; the core
dump shows the texture command's gl3_texture_cmd_t on a returned
frame.
Hold a poster slot from send to reply: cond_user with an owner and a
depth. It is re-entrant on its owner, because the cocoa trampoline that
video_thread_pump_wait() drains can post again from the waiting thread,
and its wait pumps like the reply wait so a main-thread waiter keeps
running work the worker marshals to it while another thread's command
is outstanding. All user-side posters go through
video_thread_send_and_wait_user_to_thread() or
video_thread_texture_handle(), which now take and release the slot.
Compile-tested on Linux (release and DEBUG, and -std=c89); a backport
of the same fix runs on ROCKNIX's 1.22.2-based RetroArch.
Co-authored-by: Max Engel <[email protected]>
Validation errors go through RARCH_ERR and warnings through RARCH_WARN,
so they stay visible regardless of the frontend's log verbosity;
info/verbose messages remain on RARCH_LOG.
pEnabledFeatures is an opt-in list. Requesting every feature the GPU
advertises turns on robustBufferAccess, which makes slang parameter
UBO reads return 0.0 on Intel/hasvk (#19519). Enable features
individually where a code path needs them.
The Linux-qt-companion job crashed at startup (exit 139) once it linked.
Not the shader stack this time: the job also passes --disable-networking,
and the Qt View Options dialog builds its Network category
unconditionally. NetplayPage::widget() constructs CheckBoxes directly
from menu_setting_find_enum(NETPLAY_REQUEST_DEVICE_n), which is NULL
when netplay is compiled out, and the CheckBox constructor
dereferences the setting. The Achievements category was already gated
on HAVE_CHEEVOS the same way; Network now is on HAVE_NETWORKING (both
its pages, netplay and updater, are networking).
Reproduced here with the job's exact configure line (no shader stack,
no networking): gdb put the fault in CheckBox::CheckBox <-
NetplayPage::widget <- ViewOptionsDialog; with the gate,
tools/companion_qt_persist_test.sh passes twice. The test script now
prints the tail of run.log when RetroArch exits non-zero, so a crash
is diagnosable from the CI log instead of "see run.log".
The producer notes each publish the runner let go past half the
device. More than two on an 8 ms device is a machine that could not
have kept it fed whatever the pipeline did; the steady-core checks
are not a measurement of the code there and the run says so instead
of failing on it. A hiccup's silence is budgeted at the runner's
hold, 30 ms. The 64 ms device has the margin and is checked
regardless.
8cb5f8a compiled the companion core's shader accessors out when no
shader stack is defined and added -DHAVE_GLSL to the three companion
test scripts whose fixtures stub menu_shader_get() - but missed the
macOS variant of the Cocoa script, so the MacOS job's Shader Parameters
window listed 0 parameters (4 failures). Same one-word fix.
audio_driver_mixer_add_stream takes ownership of buf_owner on the call
in every outcome and runs buf_owner_free itself on each failure return,
so audio_begin now returns after it; releasing the window a second time
freed the RFILE twice and crashed in fclose for any WEBM whose audio
track failed to open.
Sanitizers: gfx_thumbnail_preview harness (ASan+UBSan) PASS, 0 failures.
The new Linux-qt-companion job failed to link: the runner has no
OpenGL/Vulkan dev packages, so with --disable-opengl/--disable-vulkan
the build had no shader stack at all (HAVE_GLSL, HAVE_SLANG, HAVE_CG,
HAVE_HLSL all off). menu_shader_get() is only compiled under one of
those, but two places gated on HAVE_MENU alone:
- ui/companion/companion_core.c called menu_shader_get() from its
shader-parameter accessors -> undefined reference. Now under
HAVE_MENU + a shader stack (COMPANION_HAVE_SHADERS); without one the
accessors report no shader, which is the truth.
- ui/drivers/ui_qt_widgets.h declared ShaderPass / ShaderParamsDialog
under HAVE_MENU while ui_qt_widgets.cpp defines them under HAVE_MENU +
shader stack, so moc emitted metaobject code for 40 methods nobody
defined. The declaration now uses the same test; all use sites in
ui_qt.cpp were already gated.
Reproduced and re-checked at translation-unit level with the shader
defines removed (config.h and -D): before, nm showed the 40 dangling
ShaderParamsDialog references and menu_shader_get; after, none. The CI
job now passes --disable-glsl --disable-slang --disable-cg
--disable-glslang --disable-spirv_cross explicitly so it builds this
configuration regardless of what the runner has installed - it is the
one configuration no other job covers.
The companion test scripts (core, win32, cocoa) stub menu_shader_get()
and test the shader accessors, i.e. they emulate a build WITH a shader
stack; they now say so with -DHAVE_GLSL. companion_core_test plain +
asan OK; companion_win32_test OK under wine.
The last gap in the per-dock rows: a dock the user had floated came
back floating at Qt's default spot. A floating dock is a window of its
own, so its row now carries its screen position as two trailing fields,
written only for area float:
<area>,<shown>,<width>,<height>,<tabbed_with>,<raised>[,<x>,<y>]
e.g. desktop_menu_dock_core_info = "float,1,300,200,-,0,100,60"
Docked rows are unchanged (six fields); a floating row from before this
parses with x = y = 0, i.e. the old behaviour. Negative or absurd
positions parse as 0. On restore the dock is floated and placed with
setGeometry(), clamped into the available geometry of the screen under
the saved corner (primary screen if that monitor is gone) so it is
always reachable. The saved position is the client-area corner to match
setGeometry(): saving pos(), the frame corner, walked the dock by one
frame width per launch - caught by the round-trip test. Row buffers go
from 48 to 64 bytes for the two extra fields.
Tests: companion_core_test covers the floating row (x,y round trip,
docked rows never grow the fields, old floating rows parse, off-screen
positions become 0); companion_qt_persist_test seeds a floating Core
Info dock and asserts it is re-saved at the same size and position
after a restore.
Verified: companion_qt_persist_test OK twice; companion_core_test plain
+ asan OK; mingw/c89 syntax check on configuration.c and
companion_core.c clean; settings table matches; parity green; Qt 6
RetroArch builds.
config_load_file() runs once before file logging is initialised; the
retroarch-keychain.cfg merge logged unconditionally, so that line went
to the console. Gate it on !first_load like the neighbouring append
blocks. Reported by sonninnos.
dinput_poll() reads the mouse in two steps: GetDeviceState() on the
DirectInput mouse, then GetCursorPos() + ScreenToClient() to derive
absolute coordinates for the menu and RETRO_DEVICE_POINTER.
The DirectInput device is opened with DISCL_FOREGROUND, so the first
step correctly fails whenever the window is not in the foreground and
buttons/deltas are zeroed. The second step ran unconditionally, and
GetCursorPos() does not care about focus, so mouse_x/mouse_y kept
tracking the desktop cursor through a minimized window or with the
Qt desktop menu on top. The menu saw a moving hover position and kept
playing the focus sound while keyboard and buttons were blocked.
Only update the absolute position while the DirectInput mouse is
acquired. The position freezes at its last in-focus value and resumes
on the first poll after focus returns.
Reported by sonninnos.
Follow-ups to the "Save Dock Positions" / "Remember Window Geometry"
reports on #19505 (Tatsuya79):
"everything is reset on next launch", "half working if you close the
qtui window but not RA and then quit"
RetroArch writes retroarch.cfg from retroarch_main_quit(), inside the
run loop, before the companion driver is deinitialised - so a
persist done in closeEvent / deinit lands after the file is written
and only the X-button path ever worked. Three things now make it
independent of the window being closed:
- live persistence: every move/resize/show/hide of the window or of
a dock arms a 250 ms debounce that re-snapshots geometry and dock
rows into settings_t (MainWindow::schedulePersistSettings, an
event filter on every QDockWidget);
- a synchronous snapshot right before the config is written:
retroarch_main_quit() and CMD_EVENT_MENU_SAVE_CURRENT_CONFIG send
the companion CMD_EVENT_QUIT / _SAVE_CURRENT_CONFIG through
ui_companion_event_command() and the Qt backend answers with
persistSettings() (Win32/Cocoa ignore it, as before);
- nothing is snapshotted while the window is not on screen: a
hidden (F5) window reports every dock hidden, which used to come
back as an empty window; geometry is kept only once the window
has been shown (m_hasBeenShown) and never while minimised.
"the way you arranged them is not remembered (like splitting
screenshots and covers in 2 docks)"
One row per dock now, nine rows, in a format shared with the native
backends (ui/companion/companion_core.c companion_dock_row_format /
_parse, C89):
<area>,<shown>,<width>,<height>,<tabbed_with>,<raised>
e.g. desktop_menu_dock_screenshot = "right,1,186,180,-,0"
desktop_menu_dock_title = "right,1,0,0,boxart,0"
tabbed_with names the earlier dock a tab is attached to ("-" for a
standalone dock), so pulling Screenshots out of the thumbnail tab
group into its own stacked dock is exactly what the row says. The
first shipped three/four-field rows still parse. Restore: every dock
is re-added to its saved side in row order (that is what takes a
dock out of the default tab group), then tabs are re-attached, then
visibility and sizes, then the saved tab of each group is raised.
Two things that made the layout drift on every launch are handled:
a tab that is not raised is not laid out (its size is stale) so it
stores 0 and follows its leader; and the leader's height includes
the group's tab bar, because resizeDocks() sizes the whole slot
while height() is the dock alone (the thumbnail pane shrank by one
tab bar per launch). The View Options dialog's own geometry rides in
desktop_menu_options_window ("x,y,w,h") under the same checkbox.
Regression tests:
- ui/companion/test/companion_core_test.c: dock row format/parse
round trips (tab partner, raised, hidden, floating), bogus sizes
become 0, the old short rows still parse, garbage is rejected.
- tools/companion_qt_persist_test.sh (new; new CI job
Linux-qt-companion.yml builds a null-driver Qt RetroArch for it):
runs RetroArch headless (QT_QPA_PLATFORM=offscreen) with the
companion shown on boot and quits via --max-frames, i.e. through
retroarch_main_quit() with the window still open. Asserts the
window size was saved, all nine dock rows are present in the
six-field format, a seeded layout (log shown at the bottom,
Screenshots split out of the tabs) was honoured, and a second
launch restores and re-saves byte-identical rows.
Verified here: companion_qt_persist_test OK three runs in a row;
companion_core_test plain + asan OK; mingw/c89 syntax check on
configuration.c, retroarch.c, companion_core.c clean; settings table
matches reference; companion parity green; full Qt 6 RetroArch builds
and links.
* Separate sensitive credentials into credentials.cfg
* Sensitive settings: retroarch-keychain.cfg, 0600 permissions, merge before appendconfig, reuse settings tables
- Rename the credentials file from credentials.cfg to
retroarch-keychain.cfg so the filename says which program owns it and
what it holds, and does not collide with other tools' credentials.cfg.
- chmod retroarch-keychain.cfg to 0600 on POSIX desktops after a
successful write; config_file_write() creates it with the process
umask, which is usually world-readable, so the split alone did not
protect the secrets from other users on the machine.
- Merge retroarch-keychain.cfg before --appendconfig files are applied,
so an explicit command-line override of a credential is no longer
silently clobbered by the keychain file.
- config_save_credentials() now takes the array/path settings tables
config_save_file() has already built instead of populating them a
second time, and becomes static; it had no external callers.
* Move keychain permission handling into libretro-common: path_set_private()
configuration.c no longer carries platform ifdefs or <sys/stat.h>.
path_set_private() (file_path.h) wraps retro_vfs_restrict_permissions_impl()
in the VFS implementation: chmod 0600 on POSIX, no-op success on Windows,
consoles, mobile sandboxes and Android SAF where the location is already
private or there is no per-user permission model. Frontend-only, like the
other _impl helpers; not added to the core-facing retro_vfs_interface.
* vfs_implementation_uwp: add retro_vfs_restrict_permissions_impl
UWP has its own VFS backend and every _impl symbol must be defined in
both, or griffin fails to link (LNK2001). UWP app data is already
private to the package, so this is a no-op that reports success.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The ratio carries rate control's adjustment - a steady one migrates
into the bias - but a window in which rate control sat at its bound,
filling a buffer or a pipe, is not a clock, and summed for the
session it left the ratio implausible for good. A window is kept
only when the ratio over the horizon is within the band a bias could
correct. A device that is off is still measured and said so.
The checkbox went away with retroarch_qt.cfg because its value was a
QMainWindow::saveState() byte-array blob - Qt-only, opaque, and
invalidated whenever a dock is added or renamed. Users noticed. It is
back, but the layout is now six readable rows in retroarch.cfg that
the native companions can honour too:
desktop_menu_dock_search = "left,1,0"
desktop_menu_dock_playlists = "left,1,280"
desktop_menu_dock_core = "left,1,0"
desktop_menu_dock_thumbnails = "right,1,320,2"
desktop_menu_dock_core_info = "right,1,320"
desktop_menu_dock_log = "bottom,0,160"
<area>,<shown>,<size>[,<tab>]: area is left/right/top/bottom/float,
shown 1/0, size the width (left/right) or height (top/bottom) in
logical px with 0 = default, tab the raised tab of the thumbnail
group (its four docks are tabbed together and share one row).
desktop_menu_save_dock_positions gates both save and restore; the
checkbox is in View Options next to "Remember Window Geometry", and the
retroarch_qt.cfg import carries the old value over. Saving happens in
MainWindow::persistSettings() (close, F5-hide, quit) via saveDockLayout();
restoreDockLayout() runs from qt_companion_restore_settings() after every
dock is built: a dock whose side is unchanged is left where the default
layout put it (that keeps the playlists/core split and the thumbnail
tabs), a moved one is re-added to its saved side, the thumbnail group
moves and re-tabs as one, and sizes are applied with one resizeDocks()
call per orientation. This cannot express everything the old blob could
(nested splits, a floating dock's screen rect - a floating dock comes
back floating at Qt's default spot), but it covers what people do: hide
the log, widen the thumbnail pane, move a dock to the other side.
The settings are hand-declared (SETTING_BOOL / SETTING_ARRAY under
HAVE_COMPANION_WIMP) like ui_companion_toggle: they are companion
presentation state with no RetroArch-menu entry. Row parsing avoids
strtok_r (not on MSVC).
Verified: ui_qt.cpp, ui_qt_widgets.cpp and both moc TUs compile against
Qt 6 (-Wall clean); configuration.c and companion_core.c pass
tools/mingw_syntax_check.sh; settings_check.py table (reference
regenerated: one new bool row) and unity clean; settings_migrate_selftest
OK; companion_core_test OK; companion_parity green. Not run on a desktop
here - the restore needs a manual pass (move/hide/resize docks, restart).
Two regressions reported against the companion merge (#19505):
- Thumbnails had a white background in the Qt companion's dark theme.
The shared engine composites every thumbnail onto an opaque @bg and
fills the letterbox with it, and the Qt backend passed opaque white
(0xffffffff) for every request - so a transparent PNG became a white
box, and a letterboxed one got white bars, regardless of the theme.
The pre-merge Qt code drew the QPixmap with its alpha over the cell.
The engine now treats a @bg with alpha 0 as "keep transparency": the
letterbox is transparent and image pixels keep their own straight
alpha; the Qt backend (grid, boxart pane, animation frames) and the
Cocoa backend (grid, boxart pane) pass 0 so the widget's own themed
background shows through, as Qt's Format_ARGB32 / Cocoa's
hasAlpha:YES reps already carry. Win32 keeps its opaque
COLOR_WINDOW / COLOR_BTNFACE (a DIB image list has no alpha).
companion_thumbs_test covers the transparent path (clear letterbox,
opaque image pixel kept, translucent pixel returned as-is, opaque bg
still composites).
- "Remember Window Geometry" did nothing for most people. The geometry
was only captured in MainWindow::closeEvent, i.e. only when the window
was closed with its X button; hiding it with F5 or quitting RetroArch
with it open never wrote desktop_menu_window_* into settings_t, so
retroarch.cfg kept the zeros and the restore path (which needs a
non-zero size) was skipped. The old Qt code had the same closeEvent-only
capture but QSettings persisted on its own; now the values must be in
settings_t before main_exit() writes retroarch.cfg. closeEvent's
persistence is split into MainWindow::persistSettings() and
ui_companion_qt_deinit() calls it before deleting the window - deinit
runs from the run loop's exit, ahead of main_exit(), and a hidden
window still reports its last geometry. A minimised window is skipped
so a bogus off-screen rect is never saved.
Verified: tools/companion_thumbs_test.sh (plain and asan) OK;
tools/mingw_syntax_check.sh ui/companion/companion_thumbs.c clean;
ui_qt.cpp, ui_qt_widgets.cpp and moc_ui_qt.cpp compile against Qt 6.
Not run on a desktop here; the Cocoa change is not compiled on this box.
Reverts the effect of 91920289e3. Running runloop_iterate() from a
tick posted into DefWindowProc's size/move and menu loops re-enters
the menu and input drivers while the modal loop has the mouse
captured: window drag stalls, focus sticks, the menu bar stops taking
clicks after the first, and menu hover sounds keep firing on a hidden
window. Bisected by Tatsuya79 to Sept 4..5; sonninnos confirmed the
symptoms go away once the pump is disabled.
The win32 side is removed outright: the clock thread, the WM_TIMER
fallback, the tick handler, the message routing in every wndproc and
the companion window, and the mouse-message filtering in the ui
driver's pump. RUNLOOP_PACE_EXTERNAL stays in runloop.c since the
pacing rework in e0acbe2639 is built on it; nothing sets it now.
A steady core on a shared runner can still see the consumer held past
an 8 ms device once or twice in a run; each is a few milliseconds of
silence and a refill at the controller's bound. The steady-core
checks allow for up to two, and no more.
LOAD_CONTENT, DRIVERS_REINIT and their relatives reinitialise the input
driver synchronously, and the input driver owns the stdin, network and
UDS command objects - so the command frees the very object whose poll
is dispatching it, and control returns into freed memory: the stdin poll
wrote back into its buffer, the network poll read its socket again, the
parser went on to the next line with a dead handle. The input driver now
bumps a generation counter whenever it tears the interfaces down; the
message parser stops after any action that changed it, the network and
UDS polls stop reading for that pass, and the stdin poll takes the line
off its buffer first and parses from a private copy, touching nothing
of its object afterwards. Found by ASan when driving a content load
over stdin; the same load and a following session now run clean.
It was released inside the HDR-support block of vulkan_free, so a
context without HDR leaked the image and its memory each session and
destroyed the device with them live. Released unconditionally now,
before the device goes.
A Repeat line under LATENCY while threaded video has a frame retained:
how many repeats the presenter has made this session, and whether their
deadline is phase-locked to a display timestamp the driver reported or
runs on the timer. That is the only way to see from the overlay which
of the per-backend timing sources is actually in use. The wrapper
records the phase source when it schedules, under its lock, and
publishes both through video_thread_presenter_stats().
The context driver gains an optional last_present_time() slot, placed
last like presentable so positional initialisers leave it NULL. The X11
context answers from glXGetSyncValuesOML when GLX_OML_sync_control is
present: UST is the time of the last vertical retrace, in microseconds
on the monotonic clock under Mesa, which is the clock the frontend keeps
there. gl2's get_last_present_time() poke forwards to the context it
runs on, so a context without the slot leaves the presenter on the timer
as before.
The extension is requested as optional; where the device has it, every
present carries an ID and vulkan_last_present_time() asks the driver
when its most recent timed present actually reached the display, on the
platform's monotonic clock in microseconds - the clock the frontend
keeps on the platforms that expose this extension. The video driver's
get_last_present_time() poke answers from that, recovering the owning
context data from the vulkan_context_t it holds, since every context
embeds it the same way. Without the extension the poke answers 0 and the
presenter paces on the timer as before.
The overlay's VIDEO line read current_video->ident, which under threaded
video is the wrapper's "Thread wrapper". It now goes through
video_driver_get_ident(), which already answers with the wrapped
driver's name, as the AUDIO line does through audio_driver_get_ident().
The D3D12 side of the presenter. A frame() that runs with retain_output
records, into its own command list after the backbuffer reaches PRESENT,
a copy into a driver-owned committed texture that matches the swapchain's
size and format - taken from GetDesc1 on the swapchain, since the C
vtable declares ID3D12Resource::GetDesc struct-by-value - and notes the
group it puts on screen. poke->present_last() replays that group with
the list dx12_inject_black_frame() records, a copy where the clear is
for each light present and the clear itself for each dark one, each
behind the queue fence and the frame latency waitable object as the
driver's own presents are. get_last_present_time() answers from DXGI
frame statistics on the QPC clock. Nothing is created or copied unless
retain_output asks; the texture goes with the swapchain objects.
The last part of the High profiles rh264 refused: bit_depth_luma_minus8
above 0. Above 8 bits the picture planes hold uint16_t samples and
the sample kernels run as the 16-bit instantiation of rh264_bd.inc
(the C code; the SIMD paths stay 8-bit), with everything above them -
parsing, prediction control, the transforms on coefficients, the
deblocking strengths - shared and made depth-aware where the spec is:
- Frame: bd / bsh (bytes per sample as a shift) / qpbd on the frame;
planes, whole-plane clears and copies, the bi-prediction scratch and
the plane accessor in bytes. Every reader-side plane pointer goes
through RH264_OFF (found with a void* probe build for the base
forms; the derived and assignment forms by grep - two of those,
the CABAC P chroma residual's plane bases and one I_16x16 block
pointer, were the last to fall). The kernel wrappers take the
frame and dispatch on its sample width.
- The QpBdOffset domain (7.4.3, 8.5.8, 8.7.2.2): SliceQPY down to
-QpBdOffsetY, pic_init_qp down to -36, mb_qp_delta over the
extended range and wrap, QP'Y = QPY + QpBdOffsetY for scaling,
QP'C through the clipped qPI, the per-macroblock QP grid storing
QPY + offset so a byte holds the negative range, alpha / beta / tC0
and the weighted-prediction offsets scaled by 1 << (bd - 8).
Intra DC defaults and every clip take the depth from the kernel's
bd argument; I_PCM reads bit_depth-bit samples in both entropy
modes.
- The luma interpolation's unshifted six-tap sums are 32-bit above 8
bits (52 x 1023 overflowed the int16_t that fits 52 x 255; it
showed as one quarter-sample position in one block, off by 32).
- The 16-bit interpolation's reference patch and taps, and the
transform-bypass deblocking save area (doubled for 16-bit
samples), live in the working frame's scratch arena rather than on
the stack: rh264_mc_luma_16 and the P-slice deblocker had crossed
the 2 KiB frame budget for the console thread stacks.
- rh264_video_bit_depth() reports the depth; rmp4_video delivers
10-bit 4:2:0 through the shared HDR blits (XRGB2101010 when the
caller asked for it, tone-mapped 8-bit otherwise - the rh265
Main10 / VP9 Profile 2 route, so a PQ / HLG transfer is honoured)
and any other depth or subsampling through a scalar scale-down to
the 8-bit conversion.
Also corrects the CABAC I_PCM start rule from the earlier fix: the
samples begin at the byte boundary at or after the arithmetic
decoder's read position - its nine-bit window already covers the
encoder's flush and stop bit - i.e. give back the whole unconsumed
prefetched bytes (buf -= bitcnt >> 3), not one bit further. The two
rules only differ when the prefetch holds a whole number of bytes,
which the 8-bit test streams never hit; the 10-bit 4:4:4 lossless
stream did, and started its samples a byte late. Every earlier I_PCM
stream stays exact.
Byte-exact against ffmpeg on 10-bit streams from x264: 4:2:0, 4:2:2
and 4:4:4, CAVLC and CABAC, I/P/B with a B-pyramid and explicit
weighting, 4x4 and 8x8, deblocking on and off, custom matrices, low
and negative QP, lossless and ten-bit I_PCM - 34 streams, plus the 24
8-bit 4:4:4 / constrained-intra / I_PCM streams; six join
rh264_lossless_test (whose comparison is now sample-width aware). The
22-stream 8-bit regression corpus is hash-identical, on the SIMD build
and on a SIMD-less one. Stack budget, C89 and mingw gates clean.
Not carried: luma and chroma at different depths, and monochrome.
Groundwork for High 10 / High 4:2:2 / High 4:4:4 Predictive at 9 to 14
bits, the last part of the profile rh264 refuses besides separate
colour planes. Same shape as rh265: everything that touches samples
- intra prediction (4x4, 8x8, 16x16, chroma), residual add and bypass
add, the luma six-tap and chroma bilinear interpolation, weighted and
bi-predictive combination, the deblocking edge filters, the bypass
save/restore copy - moves verbatim into rh264_bd.inc, which rh264.c
includes once per sample type with RH264_PEL / RH264_BD / RH264_FN
naming the instantiation; the SIMD paths are guarded to the 8-bit one.
The parsing, prediction control, coefficient transforms and strength
derivations stay in rh264.c once.
Only the 8-bit instantiation exists yet, and the kernels keep their
names through aliases, so no call site changes; the two averaging
loops of the B block predictor become a kernel (avg_block) so that no
sample SIMD remains in rh264.c. The 22-stream corpus is hash-identical
with and without SIMD, the 4:4:4 / lossless / I_PCM streams stay
byte-exact, the 4K 60-frame decode is unchanged (2.0-2.2 s either
way), stack budget and the C89 gate are clean.
The video thread copied cmd_data and dispatched it on every wake. That
was sound while the only wakes came from the main thread, which cannot
send a command and push a frame at once; with the presenter the thread
also wakes on its own for a repeat, and if that lands after a command's
reply has been written but before the sender has consumed it, the copy
still names that command and it runs again. A texture unload run twice
is a double free - the heap corruption seen on D3D11 during content
load with Frame Repeat on, where texture teardown is immediate; Vulkan's
is deferred and only happened to survive. Whether a command is pending
is now decided under the lock, with the copy, and the copy is dispatched
only then. The harness sends commands into a repeating presenter and
checks each runs exactly once; without the fix every one ran twice.
A new poke, get_last_present_time(), reports when the driver's most
recent present reached the display, on the clock the frontend keeps,
without blocking. The presenter now sets the next repeat's deadline a
period after that timestamp when the driver has one, and after the
clock otherwise, always advancing in whole periods past the present's
completion so a late report cannot fire several repeats at once. The
worker keeps waiting on its own condition variable, so a new frame on
the ring still wakes it at once - no driver-side block, no interrupt
event. D3D11 answers from DXGI frame statistics on the QPC clock;
Vulkan and GL leave the slot empty until their timing queries are wired
and fall back to the timer. A harness lane feeds the presenter a report
three periods stale and checks the cadence holds.
What the device's buffer holds was offered but is not yet consumed,
like what the pipe holds; the change in it over a window comes out of
the source's count, as the pipe's does. Filled from empty at start, a
64 ms buffer otherwise read as a source 2000 ppm slow over the
baseline and the bias was refused.
MENU_ENUM_LABEL_CHEEVOS_MENU_SUBMENU_STR is not a settings row, so
tools/gen_lbl_str.py has no source for it and rebuilding the generated
region dropped it, breaking msg_hash_us.c for whoever next ran the
generator. It now sits above the region with the other hand-written
defines, and the region itself is exactly what the generator emits, the
video_threaded_present_repeat row included in sorted order.
A hardware-rendered core, or a platform whose render context has to stay
on the main thread, runs unthreaded whatever video_threaded says. That
is now logged once at video init with the reason, the Threaded Video
sublabel says the same, and the Frame Repeat row is listed only where
threaded video is on and can be honoured.
The GL side of the presenter, on nothing newer than GL 1.1 / GLES2 so
every context the driver runs on has it: a frame() that runs with
retain_output copies the backbuffer into a driver-owned texture with
glCopyTexSubImage2D before the swap and records the group it puts on
screen. poke->present_last() replays that group - the retained texture
drawn over the whole window with the stock shader for each light
present, a clear for each dark one, a swap after each - so BFI keeps its
strobe pattern through a repeat. Nothing is created or copied unless
retain_output asks; the texture follows the window size and goes with
the menu texture.
video_threaded_present_repeat moves from a hand-written config key to a
descriptor row in settings_def_video_sync.h, which supplies the key, the
menu row, its label and sublabel; the row is listed after Waitable
Swapchains and shown while Threaded Video is on. The label string and
the sublabel dispatch entry are added by hand as the block's other rows
are, and the settings table reference is regenerated for the new row.
The D3D11 side of the presenter: a frame() that runs with retain_output
copies the backbuffer into a driver-owned texture before the present,
since a flip-model backbuffer is undefined afterwards, and records the
group it puts on screen. poke->present_last() replays that group: each
light present copies the retained texture into the next backbuffer, each
dark one clears it, and each waits on the frame latency waitable object
first when the swapchain has one, so the cadence comes from the
swapchain rather than the wrapper's timer where it can. Nothing is
created or copied unless retain_output asks; the texture follows the
swapchain's size and goes with the other swapchain objects.
A wrapper entry point reached from inside driver->frame() runs on the
video thread - the menu drivers call current_video->set_viewport() while
they draw - and since set_viewport and set_nonblock_state became
commands, that call sent a command to the only thread that could answer
it and waited on itself, with the main thread then stuck in the paced
ring wait: threaded video hung as soon as the menu drew. send-and-wait
now recognises its own thread and runs the handler in place, replying
through the caller's packet rather than the mailbox so a command the
main thread sent meanwhile is neither answered nor overwritten. The
harness gains a driver whose frame() re-enters the wrapper through the
frontend's current driver, with the main thread sending its own commands
at the same time.
present_last() now returns the number of swaps it made and, on Vulkan,
replays the group the retaining frame put on screen - its light presents
then its dark ones - so BFI keeps its strobe pattern through a repeat.
The wrapper no longer opts BFI out, paces a repeat a group's worth of
periods after the last present, and advances swap_count by what the
driver reports; the harness checks a group repeat moves it by two.
A window closes at a publish, so its length carries that publish's
lateness - a few milliseconds of the core's run time - which the next
window carries back. The window on its own is held to a wider band,
enough to leave a stall out, and the band a bias could correct is
judged over the last few windows together, where the lateness has
cancelled. And the source's count is less the change in what the pipe
holds: what it holds was offered but is still ahead of the device,
and primed to its target it would read as a slow source for the
whole baseline.
The sink suite runs a closing publish late by 6 ms; the pipeline
harness sizes its window band to its window and runs longer under a
sanitizer.
Found by widening the ffmpeg oracle sweep (multiple slices, intra
refresh, field coding, weighted prediction, custom matrices, temporal
direct, open GOP all came out exact; constrained_intra_pred did not).
With constrained_intra_pred_flag set, an inter-coded neighbour is not
available to intra prediction (8.3.1.2) while it still counts for the
CABAC context derivations (9.3.3.1.1). The CABAC intra macroblock
reader applied that to the most-probable-mode derivation only and
predicted from the inter neighbours' samples regardless; the CAVLC
reader excluded the up and left neighbours but not the up-right and
up-left ones, which Intra_4x4 / 8x8 read too. An intra macroblock
amid inter ones in a P or B picture came out wrong and, through the
deblocking filter, so did the rows of its neighbours.
Fix: the CABAC reader keeps a separate set of prediction-availability
flags (up, left, up-right, up-left), cleared for inter neighbours
under constrained_intra_pred, used by every prediction call including
the 4:4:4 plane reconstruction, while the have_* flags the contexts use
are untouched; the CAVLC reader clears its up-right / up-left flags
as well.
Byte-exact against ffmpeg on constrained-intra streams in both entropy
modes, 4:2:0 and 4:4:4, with and without B frames and the 8x8
transform. Two cases join rh264_lossless_test; the CABAC one fails
on master. The 22-stream regression corpus is hash-identical; the
stack budget and the C89 gate are clean.
Command replies wait on cond_reply, signalled once per command, which is
the one place a single waiter is inherent: cmd_data is a single slot.
Ring progress waits on cond_ring, broadcast by the video thread when it
claims a slot and when it completes one, so the paced wait, the menu
drain and video_thread_wait_idle() can all block at once from any thread
and each wakes on its own predicate. The claim broadcast also wakes the
paced wait a render earlier than completion did. The debug waiter
accounting now covers only cond_reply. The harness runs a second thread
through wait_idle() while the main thread paces and drains, and its
check target reports the run's exit status.
Two CI failures from the last run:
1. tools/stack_budget.py: rh264_cabac_decode_bslice.constprop grew
from 1616 to 2112 bytes, over the 2048 budget for the 8 KiB console
thread stacks. The 4:4:4 commit had widened rh264_cbf - the
per-macroblock coded_block_flag state - by 34 ints (Cb / Cr flags,
their DC flags) plus an I_PCM marker, and the B slice reader keeps
four of those on its stack (current, left pair, dummy). The
per-block flags are now bytes (luma, chroma AC, Cb, Cr) - they only
ever hold 0 or 1 - which brings the readers under where they were
before 4:4:4: bslice 2112 -> 1360, pslice 1776 -> 1008. The
bi-prediction temporaries (two lists x three planes, 1.5 KB since
the chroma ones went luma-sized) also move off the stack into the
working frame's scratch arena: rh264_b_pred_block 1808 -> 240.
Decoded output is unchanged: the 22-stream 4:2:0 / 4:2:2 corpus
is hash-identical and every 4:4:4, lossless and I_PCM stream stays
byte-exact against ffmpeg.
2. The PS4 / 3DS / PSP / Vita / Wii / GameCube / Switch / PS2 / PS3 /
DOS / Emscripten / MXE / RS90 / RetroFW / Miyoo / PPC-cross and the
Linux container jobs failed at checkout: actions/checkout@v5 runs
on Node.js 24, whose binary does not load on the musl of those
build images (pthread_getname_np missing). They already opt into
Node 20 with ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION; they go back
to checkout@v4 (Node 20, the newest that runs there). The
non-container workflows keep @v5. Note that the runners drop Node
20 on September 16th: those images will need a newer base by then
regardless of the action version.
The expand-gamut command now replies and stops where it should instead
of running on into scanlines with a value the packet never carried.
set_nonblock_state crosses to the video thread with all three arguments,
so fast-forward, vsync-off and swap interval reach the swapchain rather
than only the wrapper's paced wait; set_viewport goes through its own
command instead of calling the driver from the caller's thread. The poke
interface gains get_refresh_rate, published by the video thread after
each frame and read under the lock, which the presenter now paces on in
preference to the setting; and present_last, which asks the video thread
for one repeat rather than presenting from the caller. A frame larger
than the declared geometry is logged once when cropped. Recording start
and stop wait for the in-flight frame before touching the readback path.
The harness walks every marshalled entry point with the wrapper up and
checks the worker keeps consuming afterwards.
Three things, all in the last two runs:
1. companion_thumbs_test (test_video_hover / test_apng_hover) failed
on the runners with the animation's first frame being frame 0
again. The "blocker" the test queued ahead of the video's still
to keep that still pending until animate() arrived did no such
thing: both were prefetch requests, and prefetch is popped newest
first, so the video's still ran FIRST and could finish before
animate() - at which point the engine correctly plays from frame
0, as it does for a cached still. The blocker is now queued
urgent and the video's still prefetch: urgent is served before any
prefetch job by the documented policy, independent of the order
within a ring. The premise is now asserted (the blocker's result
arrives before the still's); with the old arrangement that check
fails, so the ordering can no longer silently slip.
2. Linux-samples-gfx was cancelled again in the TSan pass of
gfx_thumbnail_preview_test. In the harness's [noreserve] pass
data_transfer_reserve_supported() still answered "yes" (the test
hook only made memreserve() refuse), so gfx_anim_preview_open
skipped its whole-file size gate and data_transfer_open_window
fell back to malloc(7.8 GB) for the sparse fixtures. ASan's
allocator_may_return_null turned that into a refusal; TSan's
allocator, given no such option, refused it fatally here and on
the runner apparently granted and paged it. The hook now answers
the question it exists to answer: reserve_supported() is false
under MEMMAP_NO_RESERVE, the gate refuses the file before any
allocation, and the TSan pass runs in 86 s here (was unbounded);
allocator_may_return_null is set for it as well.
3. actions/checkout@v3 / @v4 run on Node.js 20, which the runners
drop on September 16th; every workflow moves to @v5.
With video_threaded_present_repeat on, the video thread's wait for the
next frame has a deadline - one display period after the last present -
and passing it with nothing new presents the retained frame again through
poke->present_last(), so the display keeps its cadence while the core
stalls. Frames render with retain_output set when the driver can repeat
them; BFI and shader sub-frames opt out since a repeat would show only
the last of their several images. A repeat is not a rendered frame: it
touches no menu texture and does not hold up video_thread_wait_idle().
Every swap_count advance now happens on the video thread under the
wrapper's lock, read by the main thread through video_thread_swap_count();
the null driver gains a present_last so the harness drives the presenter,
and a lane checks repeats during a core stall and none with the setting
off. Setting is a config key for now; the menu entry follows.
The still of an animated WEBP or an APNG went through the still image
loader: read the whole file, parse every chunk, decode the first frame
- then the animation opened its windowed session and decoded that
same frame again. Videos have taken the anim-first route (the
session's first frame IS the still) since the windowed open landed;
this extends it to the two frame-indexed types, decided from the
file's head:
- gfx_anim_preview_probe(): 32 bytes of a WEBP (VP8X's animation
flag, rwebp_is_animated_ex) or 4 KiB of a PNG (acTL before the
first IDAT, rpng_is_apng_ex); 1 animated, 0 still, -1 unreadable.
- The raster menu (gfx_thumbnail_try_video_open) routes an animated
WEBP anim-first. PNG keeps the still task there: a playlist's
stills are PNGs and the task supplies the APNG verdict from bytes it
reads anyway, where a probe would be an extra open per thumbnail.
- The desktop companions route both through the session, and the
still / animation hand-off covers them too. The probe is done once
per job on the worker (also deciding the wait), and the in-flight
slot is now published in the same critical section that pops the
job - a job between the ring and its slot read as "not pending" and
the animation opened a second session (seen under TSan).
Companion hover, warm cache, 5 runs each:
40 MB animated WEBP 30 MB APNG
before after before after
time to still 65 ms 27 ms 40 ms 16 ms
CPU 105 ms 80 ms 61 ms 41 ms
peak RSS growth 62 MiB 19 MiB 49 MiB 19 MiB
Cold cache the still was 625 ms / 598 ms: the whole-file read.
Tests: gfx_thumbnail_preview_test now also runs anim_lossless.webp
through the request route (R4 "no whole-file still load" fails on
master); companion_thumbs_test gains test_apng_hover (still is frame
0, the animation continues from frame 1; fails on master) and the two
hover tests hold the still behind a large decode on a single worker so
the hand-off is exercised regardless of timing. Plain, TSan and ASan
pass; the syntax gate is clean on the four edited TUs.
The second half of High 4:4:4 Predictive after the transform bypass:
streams with chroma_format_idc 3 (separate_colour_plane_flag 0), which
rh264 refused at the SPS. In this format Cb and Cr are not chroma in
the 4:2:x sense at all - each is a luma-sized plane coded exactly as
luma is (8.3.4.5, 7.3.5.3 residual_luma for Cb and Cr after Y) - so
the change routes the two planes through the luma machinery rather
than widening the chroma one:
- SPS / PPS: twelve scaling lists (six 8x8: Y, Cb, Cr x intra/inter,
with the rule A / B fall-backs to the previous list of the kind);
the PPS list count follows the SPS's chroma format. Table 9-4's
ChromaArrayType 0/3 column for coded_block_pattern, no intra_chroma_
pred_mode, no chroma cbp bins in CABAC.
- Geometry: luma-sized chroma planes, luma-shaped coefficient-count
grids for Cb and Cr, I_PCM of 768 samples, full-width chroma from
rh264_video_plane.
- Intra, CAVLC and CABAC: per-plane reconstruction with the luma
prediction modes (resolved by the luma pass), the luma residual
syntax against the plane's own counts / coded_block_flags, the luma
transforms at the chroma qP with the chroma weight matrices, the
bypass DPCM included. CABAC: ctxBlockCat 6..13 on their own context
ranges (Table 9-34), and the 8x8 coded_block_flag that only 4:4:4
codes (cat 5/9/13, the 9.3.3.1.1.9 neighbour rule). The context
init tables now hold all 1024 contexts: 460..1023 taken from
libavcodec's cabac_context_init tables, whose entries 0..459 agree
with the libopenh264-derived ones in every position, both slice
kinds, all three cabac_init_idc.
- Inter: Cb and Cr through the luma quarter-sample interpolation with
the luma vector (8.4.2.2), bi-prediction and explicit / implicit
weighting per component at full size, luma-style residual in both
entropy modes (inter chroma qP and matrices).
- Deblocking: the chroma planes go through the luma edge filter at
every luma edge (chromaStyleFilteringFlag is 0 for ChromaArrayType
3), 8x8-transform edge rules included, with the chroma qP average
per plane; bS is the luma-block derivation, unchanged (a variant
that also counted Cb / Cr coefficients disagreed with ffmpeg). The
transform-bypass save / restore covers the wider planes.
- rmp4_video: a 4:4:4 blit row (one chroma sample per luma sample;
the SIMD rows pair one with two).
Byte-exact against ffmpeg on 4:4:4 streams from x264 covering intra
and inter, CAVLC and CABAC, 4x4 and 8x8, weighted bi-prediction with a
B-pyramid, I_PCM, deblocking on, lossy and lossless - seven of them
now in rh264_lossless_test (which builds them with ffmpeg at run time
and fails on master with "decode refused"). Through the preview
session the rendered ARGB matches ffmpeg's RGB conversion sample for
sample at the points checked. The 22-stream 4:2:0 / 4:2:2 regression
corpus (MBAFF, weighted prediction, B-pyramid, lossless, the mixed
bypass stream) is hash-identical before and after.
Not covered: separate_colour_plane_flag 1 (three monochrome pictures)
and high bit depth, both still refused. The preview memory admission
still estimates a 4:2:0 decoded-picture buffer; a 4:4:4 stream's DPB
is twice that.
On mb_type I_PCM the CABAC path restarted at the byte the engine had
fetched up to (cb->buf, bitcnt dropped). The engine prefetches up to
three bytes, so that is past where the samples begin: the spec
position is the byte boundary after the bit that follows the arithmetic
decoder's read position (the encoder's flush, 9.3.4.5, wrote its low
register and a stop bit; the decoder's nine-bit window ends one bit
short of that stop bit; then pcm_alignment_zero_bits, 7.3.5). With
the sample copy starting one or two bytes late every I_PCM macroblock
was wrong and everything after it desynchronised.
Unreachable before the transform-bypass commit - x264 only emits I_PCM
in its lossless mode, whose SPS was refused - so this is that commit's
exposure. The 4:2:0 lossless corpus never hit it because its content
was cheaper coded than raw.
Fix: step buf back by the unconsumed prefetched bits less the one
stop bit (forward one byte when nothing is prefetched), then restart
byte aligned. Verified byte-exact against ffmpeg on a lossless CABAC
stream whose every macroblock is I_PCM (random noise), added to
rh264_lossless_test as pcm_cabac; fails on master. The 22-stream
regression corpus is hash-identical.
Also fixes the test's drain loop, which tested rh264_video_drain for
1 where the contract is 0 (picture) / -1 (nothing pending): the
comparison stopped short of the pictures held for reordering.
A frame() that runs with video_frame_info_t::retain_output set records a
copy of the backbuffer into a driver-owned image after it reaches
PRESENT_SRC_KHR, and poke->present_last() then puts that image on screen
as one more swap: a copy into the swapchain image the previous
swap_buffers() acquired and a present, no shader chain, no menu or OSD.
The image is created to match the swapchain and recreated when that
changes, and freed with the other HDR-path buffers. Nothing is allocated
or copied unless retain_output is asked for, so the plain path is
unchanged. Drivers that cannot repeat a frame cheaply leave the hook
NULL; this is the Vulkan implementation the threaded presenter builds on.
A uint semantic that advances on every swap the display sees - core
frame, BFI dark frame, shader sub-frame, and repeat - so a shader can
key an effect to the monitor's cadence rather than FrameCount, which
only moves with the core. Presets test _HAS_SWAPCOUNT_UNIFORM. One
counter lives in video_st, carried per present in video_frame_info_t
and advanced by whichever thread presents: video_driver_frame() on the
direct path, the video thread under the wrapper. Wired for the Vulkan
and GL3 chains, which add the sub-frame offset themselves; D3D and the
legacy GLSL chain leave the slot unsourced, which slang_process now
treats as "not provided" instead of uploading from NULL.
Links the shipping objects with only main() replaced, boots the frontend
on the null drivers with a no-content core (so the menu can be closed
over it), and drives what the user does: flip video_threaded through the
setting's own write handler with the core running, leave the menu, run
frames, come back, flip it again, plus VIDEO_REINIT and full driver
reinit with the wrapper up, and an orderly exit with it installed. Every
step checks the wrapper is installed or absent as expected, that its
worker consumes frames, and that the runloop-thread readers (presentable,
focus, has_windowed, viewport) work in both states. SAN=address|thread
selects which tree build to link; CYCLES= sets the loop count.
The job was cancelled at its 20-minute limit mid-way through the TSan
pass of the gfx_thumbnail_preview step: preview_anim_window_test had
added ~110 MB of lossless fixtures, generated twice (once per
sanitizer pass) and decoded three times each under ASan.
- One WebP fixture instead of two: the 12 MB padded frame that pins
next_span now sits inside anim_lossless.webp (base shortened to 17
s), so the 46 MB anim_bigframe.webp is gone. Fixture bytes decoded
under ASan: 111 MB -> 72 MB, 91 s -> 54 s here.
- mkfixtures.py reuses an existing complete set instead of encoding
it again for the TSan pass (the outputs are deterministic; the CI
ffmpeg is the same 6.1.1 build as here and produced byte-identical
sizes). Generation: twice ~30 s -> once 27 s.
- The job timeout goes to 30 minutes for headroom; the step itself
is ~5 min.
Every check is unchanged: residency, cursor, both laps byte-exact, the
oversized frame, the lowered bound, the channel-order asks, the
DISPOSE_PREVIOUS fixture, the still.
rh264 refused any SPS with qpprime_y_zero_transform_bypass_flag set,
which is every x264 lossless encode (-qp 0): the file browser preview
and the companions showed nothing for them. With a yuv420p source such
a stream is plain 8-bit 4:2:0 apart from the bypass, so this is the
half of the profile the existing pipeline can carry:
- TransformBypassModeFlag (7.4.5) is the stream flag and the
macroblock's QP'Y == 0 (8-bit: QPY). In that mode the residual IS
the inverse-scanned coefficient array: no scaling, no 4x4/8x8
transform, no Intra16x16 / chroma DC Hadamard (8.5.10-8.5.12), added
to the prediction as samples (no (x+32)>>6). All twelve
reconstruction sites take the branch: CAVLC and CABAC, intra 4x4 /
8x8 / 16x16 and chroma, inter luma and chroma.
- 8.5.15: with vertical or horizontal intra prediction the bypass
residual is the difference to the previous sample in that direction,
accumulated down the columns or along the rows of the whole
predicted block - 4x4, 8x8, the 16x16 macroblock, the chroma
macroblock (intra_chroma_pred_mode numbers the two the other way
round from the luma modes).
- 8.7.2.1: the deblocking filter never modifies the samples of a
bypass macroblock, on either side of an edge, while the other side
is filtered from the unfiltered values. Rather than thread a
per-side flag through every edge kernel, a bypass macroblock (and
the three columns / rows of a bypass left / top neighbour) is copied
aside before its edges are processed and restored after.
Verified byte-exact against ffmpeg on a six-stream lossless corpus
built with x264: CAVLC and CABAC, 4x4 and 8x8 transforms, I/P/B and
all-intra, two sources. Ordinary lossy streams are unchanged (same
oracle, and the preview harness).
The deblocking rule has no encoder to produce it (x264 lossless codes
every macroblock at QP 0 and turns the filter off), so the test builds
the stream: two slices per picture, the first from a lossless encode,
the second from a CQP encode of the same frames, the second's header
rewritten to its real QP with the filter enabled and offsets +3/+3.
ffmpeg filters the bypass side too (measured: 17-24 luma samples of
the bypass slice change per frame), so on that stream the oracle is
split: the bypass slice must be the untouched source, the lossy slice
must match ffmpeg (its q-side filtering reads the unfiltered p samples
either way). Without the rule, 104 samples differ.
libretro-common/samples/formats/h264/rh264_lossless_test builds all of
that with ffmpeg at run time (the samples job installs it) and fails
on master with "decode refused".
Still not decoded: 4:4:4 chroma (chroma_format_idc 3), which needs the
Cb/Cr planes coded as luma (CAVLC nC and the CABAC ctxBlockCat 6-13
contexts, luma intra modes and quarter-pel MC for chroma, luma-style
chroma deblocking) and a full-width chroma path in the YUV blit.
One hover on a video is, on every backend, a request() for the still
followed by an animate() for the same path. The still is the first
frame through a preview session on a decode worker; the animation
thread opened a SECOND session and decoded that same first frame again,
concurrently: two sliding windows, two decoders (a 4K H.264 DPB each),
one picture decoded twice, both threads contending for the same
cores.
Now the worker that decodes a video still parks its session when the
animation wants that path (or is already waiting for it), and the
animation thread takes the parked session and carries on from the
second frame - the still on screen IS the first. While the still is
queued or in flight the animation thread waits for it (both rings and
the per-worker inflight slots, under the lock) rather than racing it;
a failed or aborted still parks nothing and the wait ends with the
job, after which the thread opens its own session as before. A
cached still (no request) also takes the old path. A parked session
nobody takes is dropped on the next animate()/stop()/free().
One hover on the 12 MB 4K fixture (trailing_large.mp4), 2 workers,
RSS sampled every 0.5 ms:
before after
time to still 153 ms 80 ms
CPU 223 ms 160 ms
peak RSS growth 224 MiB 141 MiB
Test (companion_thumbs_test, test_video_hover): an 8-frame Baseline
MP4 of distinct solid colours, built by the script with ffmpeg (now a
hard requirement; the libretro-common-tests job installs it). The
reference is the session API driven directly with the same scaler;
the delivery must be still == frame 0, then frames 1, 2, 3 - on
master the animation's first delivery is frame 0 again. Also checks
that with the still cached the animation starts at frame 0 on its own
session. Plain, TSan and ASan pass.
rpng_apng_stream_set_argb answered false to any ask after the first
frame - including a repeat ask for the order it was already emitting.
The raster menu's decode worker (gfx_thumbnail_anim_job_step) asks
per frame and runs its full-canvas R/B swizzle whenever the answer is
false, so every APNG frame but the first was uploaded with R and B
swapped, after a pass over the canvas it did not need.
Now the order already being emitted is trivially honoured, and a
genuine switch converts the persistent canvas in place once at the
frame boundary, as rwebp does - together with the region saved for a
pending DISPOSE_PREVIOUS, which is copied back onto the canvas before
the next frame and would otherwise stamp the old order back in.
Compositing is order-agnostic (alpha is the top byte either way), so
that is the whole of it.
Measured on the 300-frame 720p APNG fixture (job-step shape: ask,
next, memcpy on true / swizzle on false): 0.63 ms/frame of swizzle
becomes 0.25 ms/frame of memcpy, and the frames come out right.
Test (preview_anim_window_test): A1 the repeat ask is honoured and
frames 1-2 match the ARGB reference; A2 a switch after frame 1 yields
frame 2 byte-exact against a decode in the new order - on the 720p
APNG (fails on master both ways) and on a hand-built 8x8 APNG whose
frame 1 disposes to PREVIOUS through a blended sub-frame, so the saved
region is live across the switch (A2 fails if that region is not
converted). WEBP, WEBM and MP4 already passed both.
The W2 check measures the file mapping's resident pages by mincore
and bounds them at head + lookahead + margin + one feed budget + 2 MB.
On the GitHub runners THP is "always", so a touch at a window edge
faults in a whole 2 MiB page and residency runs up to two huge pages
past what the window commits: 22.0-23.2 MiB against a 22.5 MiB budget
(Linux-samples-gfx, ASan pass). Locally, with THP in madvise mode, the
same run reports exactly 20.0 MiB and the check passed.
The test now asks for huge pages on the reservation itself
(MADV_HUGEPAGE), so every host reproduces the runner's rounding, and
the budget carries that allowance explicitly: two huge pages, 26.5 MiB
on the 30-35 MB fixtures, 38.5 MiB on the 46 MB one. A whole-file load
still reports the file length, so the check keeps its teeth.
video_driver_get_ptr() unwraps the thread wrapper and returns the concrete
driver's private data, so casting its result to thread_video_t and locking
thr->lock enters whatever pointer sits at that offset in the driver struct
(STATUS_INVALID_HANDLE in RtlEnterCriticalSection on Windows) as soon as
the runloop consults presentable with threaded video installed. Resolve
the wrapper from video_st->data as the other entry points do, answer from
the context directly when called on the video thread, and seed presentable
true at init so the runloop is not told there is nothing to present to
before the first frame completes.
gfx_anim_preview_open ends with a whole-buffer fallback for "types
without a progressive open (animated WEBP)": data_transfer_window_extend
to the file length, then the whole-buffer stream. What that cost:
- every animated WEBP was read in full into the reservation, then
played from a fully resident buffer with p->windowed still true -
so memory admission (gfx_thumbnail_anim_install, mem_ok) charged
KEEP + AHEAD + BACK for a file of any length, and the feeder ticked
against a cursor that returned 0;
- every STILL WEBP thumbnail took the same fallback on the animation
probe: the still decode's whole-file read, then a second whole-file
read into the reservation, to learn from the last chunk that there
were no frames;
- APNG had the progressive open but no cursor, so after the 12 MB
primed at open the feeder never moved and a longer APNG looped there
for good.
With rwebp and rpng_apng now carrying the cursor (previous commits):
- image_transfer dispatches new_avail / set_avail / media_floor /
consumed for WEBP and APNG, and gains anim_stream_next_span: the
byte range of the next frame for the frame-indexing types (0/0 for
the video types).
- the fallback is gone. NULL with need_more clear is conclusive for
every streaming type; a still WEBP is refused from its first chunk.
- the feed covers a frame larger than its lookahead: when next_span
reaches past anchor + AHEAD, that one extend is unpaced (budget 0),
the same burst a lap or an open already pays. Without it the frame
never fits below the wall, consumed never moves, and the paced feed
extends to the same short target every tick.
Regression test: samples/gfx/gfx_thumbnail_preview/preview_anim_window_test
drives the shared session (the object both the raster menu and the
desktop companions play through) over ffmpeg-built lossless fixtures -
a 35 MB animated WEBP, the same with a 12 MB second frame, a 30 MB
APNG, a still WEBP - and asserts, against a whole-buffer decode of the
same file: every frame plays byte-exact across two laps; the file
mapping's resident pages (mincore over the reservation, so the bound
holds under ASan) stay inside head + lookahead + margin, not the file
length; media_floor/consumed are non-zero; a lowered bound refuses the
next frame and a raised one yields the right frame; the still is
refused from a 64-byte prefix. On master: WEBP mapping fully resident
(34.6 / 46.4 MB), cursor 0, APNG stops at 118 of 300 frames, all three
streams ignore a lowered bound - 13 failures. Wired into
Linux-samples-gfx under ASan+UBSan (single-threaded harness; TSan adds
nothing and takes it past the job budget).
The APNG stream had a progressive open but no byte cursor: media_floor
and consumed were not implemented (0 through image_transfer), so a
feeder anchored at 0 and never moved - an APNG longer than the window
primed at open played that much and looped there forever. And its
set_avail refused to lower ("monotonic"), which is the growing-read
model; a windowing feeder takes pages back behind the decoder and
rewinds the window at every lap, so a bound that will not fall admits
decodes off pages that are gone.
- Each indexed frame records the byte span of its data chunks (first
chunk header to last chunk's CRC end).
- media_floor: the first frame's data start. consumed: the next
frame's data start; the scan position while the index is still
arriving; len once the file is exhausted. rewind() drops it to the
floor. next_span: the next frame's [lo, hi).
- set_avail is an exact store; a raise resumes the chunk walk as
before. next() additionally checks the frame's data against the
bound and refuses it (NULL, nothing consumed) when it lies past.
The animation stream could only be opened over a whole file: open()
walked every chunk to the end, so a caller with a sliding window had
nothing to call (image_transfer_anim_stream_new_avail returned NULL,
need_more clear), no bound to move, and no cursor to feed against.
Now:
- rwebp_anim_stream_open_avail(buf, len, avail, need_more) walks chunk
HEADERS only, never at or past 'avail'. VP8X's animation flag
(0x02) decides "animated": a file whose first chunk is not VP8X, or
whose VP8X lacks the flag, is a still, conclusively, from the first
chunk. The open succeeds once two ANMF headers are indexed (the
first frame then lies wholly below the wall, and the file is known
to have a second) or one when the whole file was scanned, so a
single-frame file gets the verdict it always did. Otherwise NULL
with need_more set: retry with a larger prefix.
- The index is lazy: set_avail and next() resume the header walk from
where it stopped. next() refuses a frame whose payload straddles
the wall (NULL, nothing consumed), so a caller that lowered the
bound never decodes off pages it took back.
- set_avail is an exact store, as rmp4's is: a windowing caller's
bytes un-arrive when its feeder decommits behind the decoder and
rewinds at a loop.
- media_floor (first ANMF header), consumed (header of the next frame
to decode; the scan position once the index is exhausted; len once
the file is), and next_span (the next frame's [lo, hi)) give a
feeder what it needs. next_span is what lets a frame larger than
the feeder's lookahead be made resident before it is asked for -
a lossless 4K frame is tens of MB, and a fixed lookahead would
never cover it.
- num_frames in get_info is the indexed count: exact once the whole
file has been scanned, a lower bound before.
rwebp_anim_stream_open(buf, len) is open_avail with avail == len and
produces the same index as before.
gfx_thumbnail_animate called gfx_anim_preview_feed on the poll thread
every vsync while the worker (gfx_thumbnail_anim_job_step) was decoding
the same stream. The feed reads the demuxer's cursor (consumed /
media_floor) and stores its bound (set_avail); the decoder writes the
cursor and reads the bound on every packet. Those are plain, unlocked
accesses to the same fields from two threads - a data race for every
windowed type, and the window's own contract ("SINGLE OWNER, ONE
THREAD") already forbids it. TSan on samples/gfx/gfx_thumbnail_anim
reports it the moment a WEBP stream has a cursor for the feed to read.
The feed now runs where the decode runs: in the worker's job step
(the job borrows the session), and in the synchronous path right
before next(). The preview AUDIO feed stays on the poll: its
transfer is separate and its consumer is the mixer, under the mixer's
own lock. Jobs are drained before the session is released
(gfx_thumbnail_anim_close), so the borrowed pointer never outlives it.
Both request paths (gfx_thumbnail_request and gfx_thumbnail_request_file)
did
if (task_push_image_load(..., gfx_thumbnail_handle_upload, thumbnail_tag))
thumbnail->list_id = thumbnail_tag->list_id;
but from the push on the tag belongs to the task: gfx_thumbnail_handle_upload
frees it when the load completes, on the task thread, so a load that finishes
before the poll thread gets back to that line reads freed memory. The
preview harness's task stub completes synchronously and ASan reported
exactly that (samples/gfx/gfx_thumbnail_preview, Linux-samples-gfx job).
The value wanted is the generation the tag was stamped with, which is
p_gfx_thumb->list_id, still in scope and never freed - read that instead.
Core frames now cross to the video thread through two slots instead of
one, so the main thread's copy of frame N+1 overlaps the worker's upload
and render of frame N, and the paced wait blocks on the worker claiming
the previous frame rather than finishing it. When both slots are taken
the newest unclaimed frame is replaced, so a single slow render no longer
costs a frame. The slot carries the video_frame_info_t and is handed to
the driver by pointer, dropping the copy from the worker's stack (1648 to
512 bytes), and rows with no padding are copied in one memcpy.
vulkan_create_swapchain() treated any swap_interval that differed
from the one the current swapchain was created with as grounds for
a full replacement, but the interval reaches the swapchain only
through the present mode it resolves to. On a surface that offers
FIFO alone - which is what VK_KHR_display on PowerVR Rogue reports -
every interval resolves to FIFO, and intervals above 1 are emulated
by frame duplication in the video driver, so the replacement built
an identical swapchain. That is wasted work on every driver and a
failure on some: PowerVR's display WSI refuses to create the
replacement while the old swapchain still holds the plane, and
because vkCreateSwapchainKHR retires oldSwapchain even when it
fails, the context was left presenting to a retired handle. On the
Trimui Smart Pro that was a black screen at launch under 1.21, where
the flag set by surface creation forced exactly this rebuild on the
first frame.
Resolve the present mode before the redundancy check and skip the
rebuild when the resolved mode and the extent both match, taking the
new interval into the context so frame duplication and the mailbox
emulation still see it. When a replacement create does fail with an
oldSwapchain, destroy the old one - it is retired regardless - and
try once more without the handoff; if that fails too, leave the
context in the same no-swapchain state the zero-extent path uses so
the acquire path retries on a later frame instead of presenting to a
handle that is no longer valid. The context records the mode it was
built with in a new swapchain_present_mode field.
Fixes#18206
Every netplay state buffer is state_size bytes, budgeted for the
largest container this side can build, but the container written
into it is usually shorter: a client builds without the ACHV block,
and a peer's state can be a different size again. The block walk in
netplay_process_savestate1 and netplay_get_savestate_coremem ran to
the end of the buffer instead of the END block, so once a slot had
held a server state (rcheevos data) or been grown by realloc, the
walk read the stale tail as block headers. A header byte with the
top bit set went through 'input[7] << 24' as int, sign-extended
into the size_t step, and wrapped the cursor to a wild address;
that is the client-side SIGSEGV in netplay_sync_input_post_frame
reported since 1.21, with the faulting address of the form
0xf963241f.
Read the size field through uint32_t, stop at END, reject a block
whose padded size does not fit before stepping over it, and return
NULL from the coremem lookup for a malformed container so the CRC
path reports a mismatch (which requests a fresh state) instead of
hashing past the buffer. The ACHV reader stepped the cursor an extra
8 bytes past the flags word, and the writer stepped by the unpadded
cheevos size, so the END header did not sit where a walk that stops
at END looks for it; both now use the padded block size. Grown
buffers get their new tail zeroed.
Fixes#17858
The companion-UI merge moved gfx_thumbnail's animated-preview open into
the shared gfx_anim_preview. The move was faithful except for one line:
gfx_thumbnail_anim_open_probed re-installs the opened session and passed
a hardcoded windowed = 1, discarding the flag gfx_anim_preview_open had
computed (data_transfer_window_is_reserved - whether the mapping is a
sliding window or a whole-file read).
Where address-space reservation is unavailable (a platform without
mmap / VirtualAlloc, or reservation refused under memory pressure) the
open degrades to a whole-file mapping and the flag is false. Installed
as true anyway, two things broke on large videos, which is what was
seen on 2-3 GB files:
- memory admission charged one small window (~20 MiB) instead of the
file, so a multi-GB video was admitted as if it were cheap;
- the per-frame feeder ran the sliding-window advance against a
buffer that was already fully resident, chasing a frontier that
was not there - slow to first frame, or no frame.
gfx_anim_preview_windowed() exposes the flag; open_probed carries it
into install. Everything else in the moved path is unchanged.
Regression test: gfx_thumbnail_preview gains a second pass over the
fixtures with reservation forced off (a MEMMAP_TEST_NO_RESERVE build
hook makes memreserve() refuse, the real no-reserve path). It checks
the windowed flag matches the mapping, that it is false without a
reservation, and that RSS never grows toward the file size. On the
hardcoded-1 code the trailing_large fixture reports windowed with RSS
climbing as it decodes against the whole-file buffer (the fixed code
keeps the flag honest and the huge sparse fixtures fall back to a still
under admission, RSS flat). Red on the bug, green on the fix.
Verified: gfx_thumbnail_preview and gfx_thumbnail_anim suites pass
(anim under ASan/UBSan/TSan sweep); gfx_anim_preview.c, gfx_thumbnail.c
and memmap.c pass tools/mingw_syntax_check.sh; companion_core_test.sh
and companion_cocoa_test.sh pass; full build; overlap_copy_check.py
clean.
RetroActivityCommon.onCreate ran cleanupSymlinks() and updateSymlinks()
synchronously on the UI thread. cleanupSymlinks() lists the cores
directory and unlinks every symlink in it; on a Play Store build
updateSymlinks() then walks the whole app data and native library
trees looking for Dynamic Feature Module cores to relink. Both are
disk I/O, and on a slow flash device with many installed modules the
walk is long enough to stall the first frame and count towards an ANR.
Run the pass on a worker thread instead and fold it into the startup
gate the native thread already waits on: the gate is now released by
whichever of the storage permission outcome and the symlink pass lands
last, so cores are still linked before native core enumeration runs,
and the native upcall only ever fires after super.onCreate has loaded
the library. The pass is serialized against the one PlayCoreManager
runs after a module install so two walks never delete and recreate
the same links at once, and cleanupSymlinks() no longer dereferences a
null listFiles() result when the cores directory cannot be read.
Fixes#18332
A keep-alive connection taken from the pool can have been closed by
the server while it sat idle; the close is only certain to be seen
on the first send or the first recv of the next request. When that
happens before any byte of the response has arrived, open a fresh
connection and send the request again, once. A fresh connection, a
second failure, or any failure after response bytes have been read
still ends the transfer with status -1.
Fixes#19457
The last audit gap: Qt's playlist context menu can hide a playlist,
put it back from a "Hidden Playlists" submenu, create one and delete
one; the natives had none of it.
Hiding is now a property of the listing rather than of a toolkit's
widget. The core's index map skips hidden playlists, so
companion_core_playlist_count / _name / _path agree everywhere and the
indices stay consistent - Qt hid rows in its own list widget, which the
natives have no equivalent of. The hidden ones are enumerated
separately (companion_core_hidden_count / _name / _path) for the menu
that restores them, and specials can be hidden too, as in Qt.
companion_core_playlist_new writes an empty .lpl in the playlists
directory (refused if the name is taken or has a separator) and
_delete removes one from that directory (specials refused); both
refresh the listing.
Win32: New / Rename / Delete, a Hide item and a Hidden Playlists
submenu whose entries unhide (an id range, so it is handled before the
command switch). New Playlist takes a free default name and opens the
label for editing - the in-place rename the menu already had, so there
is no dialog. Delete asks first.
Cocoa: the same items; New and Delete are sheets, and the submenu is
rebuilt whenever the listing reloads rather than from a menu delegate
(the hidden set only changes then - and GNUstep faulted inside its own
NSMenu delegate machinery, which is one more reason not to need it).
Qt keeps its own list widget and dialogs; it already asks the core
whether a playlist is hidden. Waived in companion_parity.py with that
reason.
Tests: companion_core_test covers hide / unhide (the row leaves the
listing and comes back in its sorted place, a special too), the hidden
enumeration, new (refusals included), the empty playlist parsing, and
delete (a special outside the directory refused) - plain and ASan. Both
GUI harnesses drive the real menus: hide, unhide through the submenu
entry, create, delete, checking the row counts and the restored
position. companion_parity.py is at 38 features and the gap audit at
0 for both natives.
Verified: companion_core_test.sh plain / ASan, companion_thumbs_test.sh,
companion_cocoa_test.sh plain / ASan, companion_win32_test.sh pass;
companion_core.c and ui_win32_companion.c and the consumers of
companion_core.h pass tools/mingw_syntax_check.sh; full Qt 6 build and
headless start; overlap_copy_check.py clean; MRC lint unchanged.
Initialize the packet pointer/length in audio_transfer_aac_fill() and
pulse.number in raac_decode_ics(); both are only read on paths the
return codes already guard, so the values are inert.
Qt::SkipEmptyParts is 5.14 and later - before that the same flag is
QString::SkipEmptyParts - so the hidden-playlist split broke every
older Qt. Rather than version-guard it, the list moves into the shared
core in C: companion_core_playlist_is_hidden() /
_set_hidden(), over string_split() and string_list_join_concat(), which
behave the same on every Qt back to 4 (string_split drops empty parts,
so a stray comma cannot survive a round trip). Qt's helpers become two
calls; its context menu and playlist fill are unchanged otherwise.
companion_core_test covers it: the file name only is stored, hiding
twice adds one entry, unhiding keeps the others, an absent name is a
no-op, an empty name matches nothing, and empty parts are dropped on
the round trip.
While checking parity for the new core call, tools/companion_gap_audit.py
turned out to be reading ui_qt.cpp alone - but half the Qt companion is
in ui_qt_widgets.cpp, so every Qt column was tested against the wrong
file, and its "0 gaps" was worth less than it looked. It now reads the
window, the widgets and the header. With that fixed it reports the one
real gap: playlist hide/show exists only in Qt - the natives need a row
map in their playlist lists before they can hide rows - which is
recorded as an explicit parity waiver rather than left silent.
Verified: full Qt 6 build and headless start; companion_core_test.sh
plain and ASan, companion_thumbs_test.sh, companion_cocoa_test.sh,
companion_win32_test.sh pass; companion_core.c and
ui_win32_companion.c pass tools/mingw_syntax_check.sh;
companion_parity.py green (35 features).
MSVC 2005 (which builds these through griffin) rejected
ui_win32_companion.c: four declarations after a statement in the
footer-layout block, plus C4293 on the thumbnail tag macros, whose
generation field shifts by 32. A sizeof() ternary does not spare a
32-bit build - both arms are still compiled - so the choice moves to
the preprocessor: with 64-bit pointers the generation rides in the top
half as before, with 32-bit there is no generation and the row and
size checks alone reject a stale delivery. The Cocoa companion had the
same pattern (it matters for the 10.4 PPC / i386 cross), fixed the same
way.
The gate should have caught this and could not: its C89 pass runs the
host gcc, which has no <windows.h>, so Windows-only units were skipped
by name - and the missing-header filter hid the failure for anything
else that included it. It now runs that pass with the 32-bit mingw
compiler (same width as the MSVC 2005 build, so an undefined shift
shows up too), and says "skip" out loud when that compiler is not
installed instead of passing silently. On the pre-fix file it reports
the mixed declaration at line 1179; on this one it is clean. A CI step
runs it over the companion sources.
Verified: the gate passes on companion_core.c, companion_thumbs.c,
gfx_anim_preview.c and ui_win32_companion.c, and fails on the latter
before this change; companion_core_test.sh, companion_thumbs_test.sh,
companion_cocoa_test.sh and companion_win32_test.sh pass; full Qt 6
build; companion_parity.py green.
* ui/companion: add shared toolkit-agnostic companion core (C89)
First step of the companion UI native-backend refactor: a pure-C89
model/command layer with no toolkit types, to be shared by the Qt,
Win32 and Cocoa desktop companions.
- opaque companion_core_t + callback table for model changes, status,
log and refresh notifications
- playlist file enumeration and accessors
- selected-playlist loading driven through playlist_parse_step() under
a microsecond budget from companion_core_iterate(), so parsing a
large playlist never stalls the runloop
- load requests routed through the existing
task_push_load_content_*_from_companion_ui paths
Passes tools/mingw_syntax_check.sh (mingw gnu99 + linux c89 pedantic).
Not yet wired into any build; the next commits add the consumers.
* ui: make the desktop companion a selectable driver (Settings -> Drivers -> Companion UI)
The companion layer now has two slots. The platform driver (win32 /
cocoa / cocoatouch) is unchanged and always active: it keeps owning the
message pump, message boxes, file browser and window glue, so the Qt
companion on Windows and macOS behaves exactly as before. The desktop
("WIMP") companion - the playlist / content browser window that used to
be hard-wired to Qt via qt_data - is now chosen from a driver table by
the new ui_companion_driver string setting, exposed as
Settings -> Drivers -> Companion UI. When a build carries more than one
desktop companion the user can switch between them.
- uico_driver_state_t: qt_data -> wimp + wimp_data,
UICO_ST_FLAG_QT_IS_INITED -> UICO_ST_FLAG_WIMP_IS_INITED
- HAVE_COMPANION_WIMP (ui_companion_driver.h): any desktop companion is
available (Qt, Cocoa, or desktop Win32); replaces the HAVE_QT /
HAVE_QT||HAVE_COCOA gates in retroarch.c, menu_displaylist.c,
menu_setting.c, menu_cbs_sublabel.c and settings_def_desktop_menu.h
- new iterate slot in ui_companion_driver_t (all initialiser tables
updated) and ui_companion_driver_wimp_iterate(), called from the main
loop in retroarch.c and from rarch_main in ui_cocoa.m, so native
companions that ride the platform pump still get per-frame time
- ui_companion_wimp_find_driver/_find_ident,
config_get_ui_companion_driver_options, config_get_default_ui_companion,
STRING_LIST_UI_COMPANION_DRIVERS, find_driver_nonempty case
- companion core wired into Makefile.common (once, for Qt / Win32 /
Cocoa) and griffin.c
- tools/settings_table.reference regenerated (no row changes; verified
with --check, unity and settings_migrate_selftest)
Verified: tools/mingw_syntax_check.sh --consumers-of
ui/ui_companion_driver.h ui/companion/companion_core.h (16 TUs clean).
* ui/drivers/qt: route status and log messages through the shared companion core
Phase 1 plumbing proof for the companion refactor: ui_companion_qt_t
now owns a companion_core_t; msg_queue_push and log_msg go
core -> callback -> MainWindow::showStatusMessage / appendLogMessage.
No behaviour change for Qt users; further subsystems (playlists, scan,
load requests, thumbnails) follow the same route one at a time.
* ui/drivers: add native Win32 desktop companion ("win32" companion UI driver)
Pure C89 (MSVC 2003 / Win9x baseline) desktop companion built on the
shared companion core and classic common controls only: LISTBOX for
playlists, SysListView32 report view for entries, msctls_statusbar32,
ANSI entry points, no static imports of post-9x APIs. Load Core / Load
Content reuse the platform driver's existing dialog flow through
win32_menu_loop(); Run / double-click loads through the core's task
path. The window is dispatched by ui_win32.c's existing thread pump;
the only per-frame work is a 2 ms-budgeted companion_core_iterate()
from the iterate slot, so the runloop is never blocked.
When this driver is selected the companion is Win32-only: Qt is not
touched. Wired into Makefile.common (Win32 targets) and griffin.c
(MSVC projects). Playlist browser / load only for now; scan, core
association, thumbnails and the parity checklist follow.
Verified with x86_64-w64-mingw32-gcc -fsyntax-only (gnu99, -Wall
-Wextra clean); no MSVC 2003 toolchain on this box.
* ui/drivers: add native Cocoa desktop companion ("cocoa" companion UI driver)
OS X 10.4-baseline AppKit shell over the shared companion core:
NSWindow, NSSplitView, two NSTableViews using the classic dataSource /
delegate protocols (no blocks, no view-based cells, no properties),
NSTextField status line and a "Companion" menu appended to the main
menu bar. Load Core / Load Content forward to the app delegate's
existing openCore: / openDocument: actions; Run / double-click loads
through the core's task path. Pumped by ui_cocoa.m's existing run loop;
per-frame work is a 2 ms-budgeted companion_core_iterate().
Objective-C is confined to the RACompanionController; all AppKit
objects are its ivars so both the MRC per-file Makefile build and the
ARC griffin_objc.m Xcode build own them correctly. The controller is
held from the C driver struct through CFBridgingRetain under ARC and
plain +1 under MRC. Uses the NSWindowStyleMask* compat names from
defines/cocoa_defines.h and a 10.4-SDK NSInteger fallback.
When this driver is selected the companion is Cocoa-only: Qt is not
touched. Wired into Makefile.common (HAVE_COCOA_COMMON) and
griffin_objc.m (TARGET_OS_OSX).
Not compiled here (no Apple toolchain on this box);
tools/apple_mrc_ownership_check.py reports nothing for the new file.
* ui: fix Win32 link (comctl32) and Xcode Metal link (HAVE_COCOA_METAL) of the native companions
Two CI link failures on the uicompanion branch:
- MSYS2 (CLANG64/MINGW32/MINGW64): undefined InitCommonControls. The
native Win32 companion uses comctl32 (list view, status bar) and
the Makefile never linked it - ui_win32.c only carried the MSVC
#pragma comment. Add -lcomctl32 to LIBS in the Companion UI block,
and the same pragma to ui_win32_companion.c for MSVC per-file builds.
- Xcode RetroArch_Metal (arm64): companion_core_* undefined. Xcode
defines HAVE_COCOA_METAL, not HAVE_COCOA, so HAVE_COMPANION_WIMP was
never set for griffin.c and companion_core.c was not compiled, while
griffin_objc.m (TARGET_OS_OSX) pulled in ui_cocoa_companion.m and its
references. HAVE_COMPANION_WIMP, the wimp driver table and the
desktop-menu settings_def guard now all include HAVE_COCOA_METAL, so
the three stay in sync as the header comment requires.
tools/settings_table.reference regenerated (unchanged).
* ui/companion: move load-request and core lifecycle logic out of the Qt companion
Phase 2, first subsystem. The tail of MainWindow::loadContent (core path
sanitising via core_info_find, caching it across CMD_EVENT_UNLOAD_CORE,
.lpl suffixing of db_name, the load task push and the pending quick
menu), LoadCoreWindow::loadCore's core-load sequence, the unload-core
paths and getPlaylistDefaultCore now live in companion_core:
- companion_core_request_load_content()
- companion_core_load_core() (HAVE_DYNAMIC)
- companion_core_unload_core()
- companion_core_current_core_path()
- companion_core_playlist_default_core()
companion_core_request_load_entry() now routes through
request_load_content too, so the native Win32/Cocoa companions get the
same core-path sanitising and quick-menu behaviour as Qt. Qt keeps only
the widget logic and the failure message boxes; it reaches the core via
ui_companion_qt_core() (ui_qt_widgets.cpp is a separate TU).
Behaviour is unchanged for Qt users. playlist_default_core still parses
the playlist synchronously when it is not the menu's cached one, exactly
as the Qt code did; moving that onto the budgeted parse is a follow-up.
Verified: ui_qt.cpp / ui_qt_widgets.cpp compile against Qt 6;
tools/mingw_syntax_check.sh on companion_core.c and the consumers of
companion_core.h (mingw gnu99 + linux c89 pedantic clean).
* ui/companion: move playlist editing out of the Qt companion
Phase 2, second subsystem. Every Qt playlist edit repeated the same
sequence: borrow the menu's cached playlist when it is the same file,
otherwise parse; mutate; write; free if owned. That now lives once in
the shared core:
- companion_core_playlist_open() / _release() (cache borrow, parse,
write, free)
- companion_core_playlist_update_entry()
- companion_core_playlist_delete_entry()
- companion_core_playlist_set_default_core() (core_info lookup,
"DETECT" fallback)
Qt callers reduced to string marshalling + one call:
MainWindow::updateCurrentPlaylistEntry, deleteCurrentPlaylistItem, the
"Associate core" context-menu action, and PlaylistModel::getPlaylistItems
(read path, still populating the Qt model itself).
companion_core_playlist_default_core() uses the same open/release.
The per-call playlist_config_t setup disappears from four Qt functions;
the config is built in one place in the core.
Behaviour unchanged for Qt users; the native Win32/Cocoa companions can
now offer entry editing and core association through the same calls.
Verified: ui_qt.cpp / ui_qt_widgets.cpp compile against Qt 6 (only the
two pre-existing warnings); companion_core.c and the consumers of
companion_core.h pass tools/mingw_syntax_check.sh.
* ui/drivers: playlist editing in the native Win32 and Cocoa companions
Wire the shared core's editing surface into both native companions so
they match the Qt companion's context menus:
- entries: right-click -> Run / Delete Entry (with confirmation); the
playlist is re-selected through the core's budgeted parse afterwards
- playlists: right-click -> Associate Core submenu (<Detect> plus every
installed core, rebuilt on open) / Refresh Playlists
companion_core gains read-only accessors for the installed core list
(companion_core_installed_core_count/_path/_name) so backends do not
include core_info.h.
Win32: WM_CONTEXTMENU on both controls, LB_ITEMFROMPOINT so the action
applies to the right-clicked playlist (a list box does not move its
selection on right-click), menu IDs 51000+ reserved for core entries.
Cocoa: per-table -setMenu:, NSMenuDelegate -menuNeedsUpdate: fills the
core submenu, actions use -clickedRow; NSAlert confirm (10.3+).
Verified: mingw gnu99 -Wall -Wextra and linux c89 pedantic clean on the
Win32 file and the core; tools/apple_mrc_ownership_check.py reports
nothing for the Cocoa file (no Apple toolchain here).
* ui/drivers/qt: populate the playlist model through the companion core's budgeted parse
Phase 2, third subsystem, and the first non-blocking win for Qt users:
PlaylistModel::getPlaylistItems parsed the whole playlist on the UI
thread (the "first touch of a large playlist is the grid view's hitch"
the code itself documented). It now asks the core to
companion_core_select_playlist_path(); the parse advances in 2 ms slices
from the runloop via the new ui_companion_qt_iterate() hook, and
on_playlist_changed lands in PlaylistModel::onCorePlaylistChanged(),
which deep-copies the entries and starts the next pending file. The
"all playlists" view chains through its list one completion at a time.
Entries are committed to the model in a single reset when the last
playlist is in, then playlistsLoaded() fires.
MainWindow::initContentTableWidget() no longer sorts / counts / scrolls
right after asking for the data; that moved to the new
onPlaylistModelLoaded() slot, which also refreshes the launch-with
actions (they read the current entry, which did not exist yet).
Restarting a load supersedes the one in flight: the core aborts the
pending parse on the new selection and partial entries are dropped.
Core additions: companion_core_select_playlist_path() (any .lpl path,
not just the playlist directory) and
companion_core_selected_playlist_path().
Verified: ui_qt.cpp, ui_qt_widgets.cpp and both moc TUs compile against
Qt 6 (only the two pre-existing warnings); companion_core.c and the
consumers of companion_core.h pass tools/mingw_syntax_check.sh. Not run
here (no display); the grid, list, history and all-playlists paths need
a manual pass on a desktop.
* ui/companion: move bulk playlist add out of the Qt companion
Phase 2, fourth subsystem. MainWindow::addFilesToPlaylist was the last
Qt code constructing a playlist_config_t and calling playlist_init /
playlist_push / playlist_write_file directly. The core now provides the
bulk-add surface:
- companion_core_playlist_open_private(): a fresh instance, never the
menu's cached playlist, so a cancelled add is discarded unwritten via
companion_core_playlist_release(core, pl, true, false) and committed
with (…, true, true)
- companion_core_resolve_content_path(): the "archive with exactly one
file -> archive#file" rule, previously duplicated in the add and the
update-entry paths
- companion_core_playlist_push(): DETECT defaults for core path / name,
the unknown-crc marker
Qt keeps the dialog, the file/directory gathering, the progress dialog
and the extension filter (which now applies to the resolved path from
the core). updateCurrentPlaylistEntry uses the same resolver. After
this no Qt source touches playlist_config_t, playlist_init(),
playlist_get_cached() or playlist_push(); all playlist model logic the
plan lists for extraction goes through ui/companion.
Verified: ui_qt_widgets.cpp compiles against Qt 6 (only the pre-existing
warning); companion_core.c and the consumers of companion_core.h pass
tools/mingw_syntax_check.sh.
* ui/companion: move directory scan out of the Qt companion; add it to the native companions
Phase 2, fifth subsystem. The scan was already task-based; the request
plumbing and the completion handling move into the core:
- companion_core_request_scan(path, directory, show_hidden): pushes
task_push_dbscan against the configured playlist directory and content
database; false when the build has no libretrodb or the push failed
- on_scan_finished callback: fired after the menu's horizontal list has
been reset (that reset was Qt's scan_finished_handler; it belongs to
RetroArch state, so it is the core's now). task_push_dbscan carries
no user data, so the requesting core is remembered in a static and
cleared in companion_core_free (one desktop companion is active at
a time).
Qt: onFileBrowserTreeContextMenuRequested calls the core; the
"scan finished" confirmation dialog stays in Qt as the callback.
Native companions gain File -> Scan Directory...: Win32 through
SHBrowseForFolderA (BIF_RETURNONLYFSDIRS, no new-style flag; shell32 /
ole32 added to LIBS and the MSVC pragmas), Cocoa through NSOpenPanel
using the same URL-or-objc_msgSend split as ui_cocoa.m so the 10.4
selectors do not appear as declarations to a modern SDK. Both refresh
the playlist list on completion.
Verified: ui_qt.cpp compiles against Qt 6; companion_core.c with and
without HAVE_LIBRETRODB, ui_win32_companion.c and the consumers of
companion_core.h pass tools/mingw_syntax_check.sh (mingw -Wall -Wextra
clean). apple_mrc_ownership_check.py reports the local
[NSOpenPanel openPanel], used within the pool exactly as ui_cocoa.m
does - a read-and-dismiss hit.
* ui/companion: move thumbnail path resolution out of the Qt companion
Phase 2, sixth subsystem. The thumbnail repository layout, the label
sanitising rule (& * / : ` < > ? \ | -> '_'), the extension probe order
(.png .jpg .jpeg .bmp .tga, .png as the save target) and the
"image content is its own thumbnail" rule were all Qt-side. They are
now C89 in the core:
- COMPANION_THUMB_BOXART / _SCREENSHOT / _TITLE / _LOGO subdirectory
names (ui_qt.h's THUMBNAIL_* macros alias them)
- companion_core_thumbnail_dir(db_name, subdir)
- companion_core_thumbnail_path(db_name, subdir, label, content_path)
PlaylistModel keeps three thin QString wrappers (getPlaylistThumbnailsDir,
getThumbnailPath, getRepositoryThumbnailPath - the last for the save
target and the sidebar images, where the content path must not short-
circuit the lookup); getSanitizedThumbnailName and the sanitiser
QRegularExpression are gone. changeThumbnail and the current-item
thumbnail loader use the wrappers.
The same calls are what the native companions' grid / list thumbnails
will resolve through.
Verified: ui_qt.cpp, ui_qt_widgets.cpp and moc_ui_qt compile against
Qt 6 (only the two pre-existing warnings); companion_core.c and the
consumers of companion_core.h pass tools/mingw_syntax_check.sh.
* ui/drivers/qt: define MainWindow::playlistModel()
The getter was declared in ui_qt.h but never defined - nothing used it
until the companion core's on_playlist_changed callback did, so the
MSYS2 Qt builds failed to link:
undefined reference to `MainWindow::playlistModel()'
Define it alongside the other trivial accessors. This box only ever
compiled ui_qt.o, never linked a Qt retroarch, which is how it slipped.
* ui/companion: move "launch with" core selection out of the Qt companion
Phase 2, seventh subsystem. MainWindow::setCoreActions built the
"Launch with" list itself: the running core, the entry's own core, the
playlist's default core, each de-duplicated against the others by path,
core name, display name and core_file_id prefix, spread over ~150 lines
of QVariantMap juggling (including a per-row loop that re-resolved the
same default core for the all-playlists view). The selection logic is
now C89:
- companion_core_launch_options(entry core, playlist name,
suggest_loaded_first, out[], max): the de-duplicated candidates in
menu order, each tagged companion_launch_selection
- companion_core_current_core_name() / _version() /
_supports_no_content(): the runloop system info Qt read directly
Qt's core_selection enum now aliases companion_launch_selection so the
combo box stores what the core hands back; setCoreActions fills the
combo from the array and appends its Ask / Load Core... entries;
setCurrentCoreLabel uses the accessors instead of runloop_state.
Verified: full Qt 6 retroarch links on Linux (the miss behind the
playlistModel() CI failure is now covered on this box) and starts
headless with desktop_menu_enable + the qt companion under
QT_QPA_PLATFORM=offscreen without faulting; companion_core.c and the
consumers of companion_core.h pass tools/mingw_syntax_check.sh.
* ui/companion: move thumbnail downloads out of the Qt companion
Phase 2, eighth subsystem. The single-thumbnail and thumbnail-pack
downloads - server URLs and per-component percent-encoding, the label
scrub the server expects, the .partial write + rename, the pack's
"delete or rename aside with .tmp" replacement pass, the decompress
task and its source cleanup, plus three retro_task callbacks and a
userdata struct - were ~430 lines of Qt. They are C89 in the core now:
- companion_core_thumbnail_download(db_name, label, subdir)
- companion_core_thumbnail_pack_download(db_name) (download + extract)
- companion_core_download_cancel() / _active()
- on_thumbnail_downloaded / on_thumbnail_pack_finished callbacks, the
latter carrying a companion_download_result the UI maps to its
messages (network / rename / delete / empty archive / extract)
One transfer is in flight at a time, as before. The task user data is
owned by the task chain; the core only points at it and clears the
back-pointer when the transfer is cancelled or the core is freed, so a
late callback frees the data and notifies nobody.
Qt keeps its queues (per-type list for one entry, per-playlist list),
counters and the three progress dialogs; downloadThumbnail,
downloadAllThumbnails and downloadNextPlaylistThumbnail are now a core
call plus the dialog. onExtractArchive, the cb_* callbacks,
qt_download_userdata_t, m_currentHttpTask, the extraction progress
dialog and scrub_qstring are gone. A cancelled transfer no longer
touches the UI at all.
Verified: full Qt 6 retroarch links on Linux and starts headless with
the qt companion; companion_core.c with and without HAVE_NETWORKING and
the consumers of companion_core.h pass tools/mingw_syntax_check.sh. The
actual transfers are not exercised here (no thumbnail server access
from this box).
* ui/drivers: Win32 companion keeps RetroArch running while dragged, gets a splitter; Qt gains Scan Directory / Quit
Win32 companion:
- Dragging or sizing the companion window (or browsing its menu bar)
froze RetroArch's video: DefWindowProc runs those loops modally on
the main thread, and only the main window's WndProc clocked the run
loop through them. win32_common.c's modal clock (win32_modal_enter /
_exit / _tick, the WM_RA_MODAL_TICK message and WIN32_MODAL_TIMER_ID)
is now exported, and the companion window routes WM_ENTERSIZEMOVE /
WM_ENTERMENULOOP / WM_EXITSIZEMOVE / WM_EXITMENULOOP and the ticks to
it exactly as the main window does; win32_modal_window_destroyed()
ends a session if the clocked window goes away mid-drag.
- The playlist pane was fixed at 200 px. A 5 px splitter between the
pane and the entry list now drags (IDC_SIZEWE, capture, clamped to
100 px from either edge, re-clamped on window resize).
Qt companion: File menu gains "Scan Directory" (QFileDialog directory
picker -> companion_core_request_scan) and "Quit RetroArch"
(CMD_EVENT_QUIT through the core), matching the native companions'
menus. Both use existing menu strings.
The two native callback tables spell out the new thumbnail callbacks as
NULL (they were relying on zero-initialisation; -Wextra flagged it).
Verified: ui_win32_companion.c and win32_common.c pass
tools/mingw_syntax_check.sh (the one consumer of win32_common.h that
fails, uwp_egl_ctx.c, fails identically on the branch tip); full Qt 6
retroarch links and starts headless with the qt companion.
* verbosity: mirror the log into any desktop companion, not just Qt
The companion log path in verbosity.c was Qt-specific twice over: it
was gated on HAVE_QT, and it replaced the platform sink - on a Qt build
the line was formatted into a buffer, written to the log file from
there and handed to ui_companion_driver_log_msg(), while the macOS
terminal/asl path and the plain vfprintf path were skipped entirely.
Now:
- verbosity.c includes ui_companion_driver.h (RARCH_INTERNAL) and gates
on HAVE_COMPANION_WIMP, so the native Win32 and Cocoa companions get
the log too
- the companion copy is orthogonal to the platform sink: it is formatted
from a va_copy of the arguments and the normal output below still
happens unchanged (macOS Qt builds regain their Terminal/asl output).
A va_copy fallback (__va_copy, or plain assignment) keeps the file
C89-clean; the linux-c89 job compiles it
- nothing is formatted unless someone is listening:
ui_companion_driver_log_active() (setting + inited + is_active +
log_msg present) is asked once per line, and
ui_companion_driver_log_msg() then delivers without repeating the
setting lookup and is_active() round trip - the previous code paid
both on every RARCH_LOG whenever Qt was compiled in, even with the
desktop menu disabled
- the __WINRT__ buffered path is unchanged, just no longer shared with Qt
The native companions gain the log view the plan asks for (View > Log
on Win32: a read-only multiline EDIT appended via EM_REPLACESEL at the
end, trimmed by half in one cut past 256 KiB so appends stay O(line);
Companion > Log on Cocoa: an NSTextView added under the split while
shown, same trim rule). The 10.4 NSInteger fallback grows NSUInteger.
Verified: verbosity.c and ui_companion_driver.c compile with the
linux-c89 flags; all 15 consumers of ui_companion_driver.h pass
tools/mingw_syntax_check.sh; ui_win32_companion.c is mingw -Wall
-Wextra clean; full Qt 6 retroarch links and runs headless with
--verbose (the mirror path exercised into Qt's log widget).
* ui/companion: move the core picker filter and the core-information rows out of the Qt companion
Phase 2, ninth subsystem: everything the Qt companion still computed
from core_info itself.
- LoadCoreWindow::initCoreList filtered the core list by gathering the
content's extension plus every archive member's extension into a
QStringList and matching it against each core's supported_extensions
string. That was a Qt re-implementation of core_info's own matcher;
companion_core_installed_cores_supporting(content_path) now calls
core_info_list_get_supported_cores() - the exact rule the menu uses,
directories included - and reports how many cores lead the list. The
table is built from the installed-core accessors (a _version accessor
is added; _name falls back to the core file's base name, as the table
did) and hides the rows past that count. MainWindow::loadContent's
CORE_SELECTION_ASK path no longer walks archives; it passes the path.
- The core-information collector (~240 lines: name / version / system /
authors / permissions / licences / extensions / firmware status with
per-row colouring / notes) was already toolkit-free C living in
ui_qt.cpp under a qt_ prefix. It moves verbatim into the core as
companion_core_core_info_rows() with companion_core_info_row_status;
MainWindow::getCoreInfo keeps only the HTML/CSS decoration of the
firmware rows.
Both are what the native companions' Load Core dialog and core-info
panel will consume.
Verified: full Qt 6 retroarch links and starts headless with the qt
companion; companion_core.c and the consumers of companion_core.h pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean).
* ui/companion: one window hand-off and one start-core path for all three companions
Two pieces of RetroArch-facing logic were still written three times,
once per backend, and once in Qt only:
- Raising the companion window meant "release the mouse grab, show the
cursor, leave fullscreen" in ui_qt.cpp, ui_win32_companion.c and
ui_cocoa_companion.m, each reading input_state / video_state /
settings itself. companion_core_prepare_show_window() is that
sequence; companion_core_video_started_fullscreen() answers Qt's
"stay behind the video window" question. The two native files drop
their video_driver.h / input_driver.h includes.
- Starting a contentless core: Qt cleared RARCH_PATH_BASENAME and
pushed task_push_start_current_core; the native companions had been
wired to companion_core_request_load(NULL, NULL), which is a different
task. companion_core_start_core() is the Qt sequence, and all three
use it (native ones report failure in the status bar).
getSelectedCorePath's CORE_SELECTION_CURRENT reads the core path from
the core accessor; the Win32 Quit item goes through
companion_core_event_command like the Qt one. ui_qt.cpp no longer
calls command_event, input_state_get_ptr or video_state_get_ptr.
Verified: full Qt 6 retroarch links and starts headless with the qt
companion; companion_core.c, ui_win32_companion.c (mingw -Wall -Wextra)
and the consumers of companion_core.h pass tools/mingw_syntax_check.sh.
* ui/drivers: Load Core window for the native Win32 and Cocoa companions
Parity with Qt's LoadCoreWindow: File -> Load Core... in both native
companions now opens the companion's own picker - a Name / Version list
of the installed cores from companion_core_installed_core_* - with
Load / Cancel buttons and double-click, and loads through
companion_core_load_core(). Previously the Win32 item forwarded to the
main window's file-open dialog and the Cocoa item to the app delegate's
openCore:, i.e. a raw .dll / .dylib chooser.
Win32: a second top-level window owned by the companion (stays above it,
hides with it, WS_EX_TOOLWINDOW), SysListView32 report view sorted by
name with the core index in lParam so a sorted row still maps back, and
the same modal-loop clocking as the companion window so dragging it
does not stall RetroArch. Cocoa: an NSWindow with an NSTableView fed by
the same controller (keyed on the table object), created on first use.
Load Content keeps using the platform driver's dialog.
Verified: ui_win32_companion.c passes tools/mingw_syntax_check.sh
(mingw -Wall -Wextra clean); apple_mrc_ownership_check.py reports only
the known local NSOpenPanel hit. Not run here (no Windows / macOS).
* ui/drivers: core-information pane for the native Win32 and Cocoa companions
Parity with Qt's core-info dock: View -> Core Information (Win32) /
Companion -> Core Information (Cocoa) shows the rows
companion_core_core_info_rows() produces for the running core - name,
version, system, authors, permissions, licences, extensions, firmware
status, notes - as a two-column key / value list beside the entries.
Win32: a SysListView32 pane on the right of the entry list; it yields
when the window is too narrow for three panes. Cocoa: a third pane that
joins the NSSplitView while shown, rows cached in two string_lists.
Only the shader commands are forwarded to companions, so a core load or
unload is not an event a companion sees; both panes instead follow the
running core from the per-frame iterate hook with one short strcmp
while shown, refilling only when the path changed.
Verified: ui_win32_companion.c passes tools/mingw_syntax_check.sh (mingw
-Wall -Wextra clean); apple_mrc_ownership_check.py reports only the
known local NSOpenPanel hit. Not run here (no Windows / macOS).
* ui/drivers: icon view with thumbnails for the native Win32 companion
Parity with Qt's grid view. View -> Icons switches the entry list to
LVS_ICON with 128 px boxart thumbnails; View -> List returns to the
report view. Thumbnails resolve through companion_core_thumbnail_path()
(the same rule as Qt: repository layout, sanitised label, extension
probe, image content as its own thumbnail) and decode through
image_texture_load(), whose ARGB8888 output is exactly the 32-bit BGRA
DIB order Windows wants, so a thumbnail is a nearest-neighbour
letterbox into a CreateDIBSection and an ImageList_Add.
Never blocks: the image list starts with a placeholder at index 0 for
every item, and the iterate hook decodes one pending thumbnail per frame
while the icon view is showing - one file decode per frame however long
the playlist. Decoding is sequential from the top for now; prioritising
the visible range is a follow-up.
Everything used exists on Windows 95 with comctl32 4.70 (ImageList_*,
LVS_ICON, LVM_SETICONSPACING); ILC_COLOR32 degrades to what the
installed comctl32 supports.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean, linux
c89 pass skipped for the Win32 path as the script does). Not run here.
* ui/companion: destroy the desktop companion window at quit, before the drivers go away
Quitting from RetroArch's own menu with the native Win32 companion open
left the process hung ("Not Responding") with an undestroyed window:
the companion window is a top-level window on the main thread, and the
only thing that destroyed it was ui_companion_driver_deinit(), which
main_exit() runs after driver_uninit(DRIVERS_CMD_ALL) has already torn
down the video driver - i.e. after the message pump that services the
window (ui_application_win32_process_events on the main thread, or the
video thread under threaded video) is gone. DestroyWindow() then has no
pump to run the window's WM_DESTROY through, and shutdown stalls.
Fix: close the companion window while its pump and the drivers are
still alive. ui_companion_driver_wimp_deinit() is split out of
ui_companion_driver_deinit() (which now calls it), made idempotent, and
invoked right after the run loop breaks - in retroarch.c's
retroarch_main_loop and in ui_cocoa.m's rarch_main - before main_exit().
main_exit()'s existing ui_companion_driver_deinit() still runs and is
now a harmless second call for the WIMP part while still tearing down
the platform driver.
This is the Qt companion's behaviour too (it lived inside the same
deinit), so no change there; it fixes the native Win32 and Cocoa
companions, whose windows previously outlived the pump.
Verified: retroarch.c, ui/ui_companion_driver.c and all 15 consumers of
ui_companion_driver.h pass tools/mingw_syntax_check.sh; full Linux
build links. The hang itself is Windows-only and not reproducible here.
* ui/drivers: icon (grid) view for the native Cocoa companion
Parity with Qt's grid and the Win32 icon view. Companion -> Icons swaps
the entries table in its scroll view for a thumbnail grid; Companion ->
List swaps it back.
NSCollectionView is 10.5+ and the baseline is 10.4, so the grid is a
hand-laid flipped NSView (RACompanionGrid): fixed 128 px cells flow-laid
to the width, each drawing a letterboxed boxart above a truncated label,
with mouse and arrow / Return selection and double-click to load. The
scroll view is reused, so no second data path.
Thumbnails resolve through companion_core_thumbnail_path() - the same
rule as everywhere else - and load as NSImage; like the Win32 icon
view, one is decoded per frame from the iterate hook while the grid is
showing, so a playlist of any size never costs more than one decode per
frame. One NSImage per row is cached and drawn as it arrives.
Verified: apple_mrc_ownership_check.py reports only local autoreleased
objects used within their pool (the drawRect: attributes dictionary and
the existing NSOpenPanel), neither stored past the pool. Not compiled
here (no Apple toolchain).
* ui: pick a core on Run when a playlist entry has none (native companions)
When a playlist entry's core is empty or "DETECT" and no core is
loaded, Run silently failed on the native companions (Qt shows its
core-selection combo / Load Core window). companion_core_entry_needs_core()
reports that case and hands back the content path; both native companions
then open their Load Core window filtered to the cores that can run that
content, and the pick launches the content with the chosen core through
companion_core_request_load_content().
Win32: cw_run_selected checks first and calls cw_cores_show(content); the
Load Core window carries the pending content, filters its list with
companion_core_installed_cores_supporting(), and its OK path runs the
content instead of just loading the core. Cocoa: runSelected and gridRun
do the same via -showCoresForContent:; the window build is split into
-buildCoresWindow so it is populated and raised once. Plain File -> Load
Core is unchanged (no content, full list, loads the core only).
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); the consumers of
companion_core.h pass; apple_mrc_ownership_check.py reports only the two
known local-autorelease hits. Not run here (no Windows / macOS).
* ui: file-system browser for the native Win32 and Cocoa companions
Parity with Qt's File Browser tab: File -> Browse Files (Win32) /
Companion -> Browse Files (Cocoa) turns the entry list into a filesystem
browser - directories first with a ".." parent link, a directory
descends, a file loads through the current core (or, when the file has
no usable core, opens the Load Core picker filtered to it). Picking a
playlist returns to playlist mode.
The listing and navigation are in the core, toolkit-free:
companion_core_browse_open() (dir_list_new + dir-first sort, defaulting
to the content directory), _browse_count/_name/_path/_is_dir with a
synthesised ".." entry, and _browse_activate() which descends, loads, or
reports needs-core with the content path for the picker - reusing
companion_core_request_load_content(), so the same core-path sanitising
and quick-menu behaviour as everywhere else.
Backends only present it: Win32 reuses the entries list view (trailing
slash marks directories), Cocoa the entries table; both branch their
rebuild / run on a browse_mode flag and leave it on playlist select.
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean, linux c89 pass);
the consumers of companion_core.h pass; apple_mrc_ownership_check.py
reports only the two known local-autorelease hits. Not run here.
* tools: companion backend feature-parity check (Phase 6)
The refactor's final phase requires the three desktop companions - Qt,
native Win32, native Cocoa - to stay interchangeable: "same actions
produce the same results". Nothing enforced that. A backend can omit a
whole feature (file browser, scan, core picker, core-info panel) and
still compile and run; the gap only surfaces when a user switches the
Companion UI driver and finds a menu entry gone.
companion_parity.py defines the feature set by the companion_core_*
entry point each feature needs and checks every backend references it -
the same source-level, zero-cost, Linux-only shape as
vfs_backend_parity.py. Qt's two presentation TUs are checked together;
where Qt implements a feature a toolkit-native way (playlist listing and
the file browser through its own QDir / QFileSystemModel, Run and the
core picker through the launch-with combo) the equivalent call is
recorded in QT_EQUIVALENT with a reason. A backend that genuinely should
not carry a feature goes in WAIVERS, so dropping one is a reviewed
choice, not an oversight. --list prints the matrix.
Wired into Linux-libretro-common-tests.yml beside the VFS parity check.
Confirmed it passes on the current three backends (19 features) and
fails when a feature call is removed from one.
This is Phase 6's parity gate as an enforced check rather than a static
checklist that would drift; the remaining visual / behavioural parity
pass stays manual.
* ui/drivers: Win32 companion opens at the Qt companion's startup size
The Qt companion opens at 1280x720 centred on the primary monitor,
clamped to it; the native Win32 companion opened at CW_USEDEFAULT with a
fixed 800x520 fallback - a much smaller window in a different place, so
the two look nothing alike on first F5. Match Qt: 1280x720, centred on
the work area (screen minus taskbar), clamped to it.
Made scalable for small displays, which the 9x baseline can hit (640x480
/ 800x600 CRTs): the size is clamped to the work area and only floored to
the window's own minimum when the work area is large enough for it, so it
never opens larger than the screen. SPI_GETWORKAREA falls back to the
full screen metrics when it fails, as it can on 9x.
The default view stays the report list, which matches the Qt companion's
own default (VIEW_TYPE_LIST); the grid in the reference screenshot was a
persisted user choice, not the startup state.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean). Not
run here (no Windows).
* ui/drivers: search box and Edit menu in the native Win32 companion
Visual/behavioural parity with the Qt companion, which has a Search box
and an Edit -> Search action. The Win32 companion gains an EDIT strip
above the entry list and an Edit menu with Search (Ctrl+F focuses the
box, matching Qt's shortcut). Typing filters the current content view -
playlist entries or the file browser - by case-insensitive substring,
the same field Qt's proxy model filters on.
Filtering makes list-view rows non-contiguous with entry indices, so the
entry index now rides in each row's lParam and the readers use it: a new
cw_selected_entry() resolves the selection through lParam (browse rows
are never filtered, so they stay row == index), and the lazy thumbnail
decode walks visible rows and reads each row's lParam. The image list is
sized to the full count and installed before items reference it; the
per-frame decode budget is unchanged.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean, linux
c89 pass for the shared paths). Not run here (no Windows).
* ui/drivers: boxart preview pane for the native Win32 and Cocoa companions
Parity with the Qt companion's Boxart dock. View -> Boxart (Win32) /
Companion -> Boxart (Cocoa) shows the selected playlist entry's boxart,
updated as the selection moves. It resolves through the same
companion_core_thumbnail_path() rule as the grid, so image content and
the repository layout behave identically.
Win32: a STATIC (SS_BITMAP) in the right column below the core-info pane
- the column appears when either is shown and splits between them. The
image is decoded on LVN_ITEMCHANGED, scaled to the pane preserving
aspect into a 32-bit DIB (image_texture_load's ARGB8888 is Windows' BGRA
order), and the previous HBITMAP is freed. Cocoa: an NSImageView on the
right edge (proportional scaling), refreshed from the entries table's
selection-changed delegate.
Nothing decodes while the pane is hidden or in file-browser mode, and
only the selected entry is decoded, so there is no per-frame cost.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean);
companion_parity.py green; apple_mrc_ownership_check.py reports only the
two known local-autorelease hits. Not run here (no Windows / macOS).
* ui/companion: lead the playlist list with the special playlists
The Qt companion's left pane starts with Favorites, History, Images,
Music and Videos above the system playlists; the native companions,
which build their list from companion_core_playlist_count/_name/_path,
only saw the .lpl files in the playlist directory - so History and
Favorites were unreachable there.
The core now returns those five as leading playlist entries, in the Qt
order, each a real .lpl at its configured path
(path_content_favorites / _history / _image_history / _music_history /
_video_history); a path left empty in the config is skipped, so the
lead count is not always five. Names come from the existing *_TAB label
strings. Selection, delete and core-association already key off the
public index / path, which now map through the specials, so they follow
without change; select_playlist_path matches against the public paths
too.
Qt is unaffected: it builds its own special-playlist items (including
the synthetic "All Playlists", which is not a file and stays a
presentation concern) and does not enumerate through these accessors.
Verified: companion_core.c and the consumers of companion_core.h pass
tools/mingw_syntax_check.sh; companion_parity.py green.
* ui/drivers: Win32 companion puts the search box in the left column
The Qt companion's search field sits at the top of the left content
column, above the playlist list; the native Win32 companion had it above
the entry list on the right. Move it to match: the left column is now
the search box on top and the playlist list below it, and the entry list
spans the full height on the right, so the two windows line up.
Layout only - the EDIT still filters the current content view the same
way, and the splitter, browse, info and boxart panes are unchanged.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean);
companion_parity.py green. Not run here (no Windows).
* ui/drivers: lay the Win32 companion out like the Qt companion
The Qt companion is the reference look; the native Win32 window had the
same features behind a different shape. This makes its structure match
the Qt startup layout, using classic controls only (all on Windows 95
with comctl32 4.70) and the same msg_hash strings Qt shows:
Left column, as Qt's docks:
Search [edit][Clear]
Content Browser
[Playlists | File Browser] (a tab control; the second tab enters
the file browser, the first returns)
[playlist list] (a list view with the shell folder icon
per row, like Qt's, replacing the LISTBOX)
Core
[launch-with combo][Info][Run]
The Core section is Qt's launch-with combo: companion_core_launch_options
for the selected entry (running / entry's own / playlist default core),
plus "Ask" and "Load Core..."; Run launches with the chosen core, Ask and
Load Core open the picker. This was the last Qt behaviour the natives
lacked - they previously always asked.
Centre: the entry list over Qt's footer ("N items" from
QT_ITEMS_COUNT on the left, a View: List / Icons combo on the right).
Right column: "Core Info" and "Boxart" captions over their panes, both
shown by default as Qt does. Menubar: File / Edit / View / Help with
Qt's titles; Help gains Documentation (docs.libretro.com) and About.
Status bar defaults to "<version> - <core or No Core>" and follows the
running core.
Every control is set to DEFAULT_GUI_FONT - the bold bitmapped System
font the controls default to was most of why the window looked nothing
like Qt.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean);
companion_parity.py green. Not run here (no Windows) - the pixel pass
against the Qt screenshot needs a real box.
* ui/drivers: Win32 companion scales with the display DPI like Qt
The side-by-side on a 200% display showed the Win32 companion at half
the Qt companion's size with captions overlapping their controls and
text truncated everywhere. Cause: Qt sizes in logical (96-dpi) pixels
and renders them at the monitor's DPI - its 1280x720 fills a 2560x1440
screen - while the native window used the same numbers as raw pixels,
and its DEFAULT_GUI_FONT is a fixed 8pt bitmap that does not scale, so
the rows sized for it could not hold the text the display wanted.
Now everything scales the way Qt does:
- LOGPIXELSY is read once and every logical size (startup 1280x720,
minimums, pane and column widths, splitter, boxart height, icon size,
icon-view spacing, padding) goes through MulDiv against 96
- the font is the system message font from NONCLIENTMETRICS, which is
the DPI-scaled UI font (Segoe UI at the display's size on modern
Windows, MS Sans Serif on 9x), created once and freed on deinit; the
stock font remains the fallback
- row heights derive from that font's TEXTMETRIC (caption = one line,
control = line + chrome) instead of fixed 18/24
- list columns are sized to their panes on every layout (playlist
column to the pane, Name/Core 2:1 across the entry list, info key
column narrow with the value taking the rest), so nothing truncates;
the info list drops its column header as Qt has none
- the left pane defaults to 280 logical px, Qt's dock width
All APIs used date from Windows 95 (GetDeviceCaps, CreateFontIndirect,
SPI_GETNONCLIENTMETRICS, MulDiv).
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean);
companion_parity.py green. Not run here (no Windows).
* ui/drivers: fix the zero-size Win32 companion window and the Xcode build
Win32: the DPI-scaling commit read w->dpi before it was set. The DPI /
font block sat after the main window's size was computed and the window
created, so CW_S() ran MulDiv(x, 0, 96) = 0 for the startup size and the
WM_GETMINMAXINFO minimums - a zero-size window, the near-invisible strip
in the report. Move the DPI / font setup to the top of cw_create_window,
before anything is sized.
Cocoa: the Xcode (ARC) build failed on the core-info pane's construction
- "passing address of non-local object to __autoreleasing parameter for
write-back": an ivar (infoScroll) was used as the scroll-view
out-parameter of -makeTable:scroll:twoColumns:. Under ARC such a
parameter is __autoreleasing and must be a local; go through one and
assign it to the ivar (ARC retains the strong ivar, KEEP_IVAR adds the
retain under MRC). The other three callers already used locals.
Also declare the controller's protocols (NSTableViewDataSource /
Delegate, NSWindowDelegate, NSMenuDelegate) when the SDK is 10.6+, so
the four setDataSource: / setDelegate:self "incompatible type" warnings
in that build go away; older SDKs use the informal protocols and keep
the plain declaration.
Verified: ui_win32_companion.c passes tools/mingw_syntax_check.sh (mingw
-Wall -Wextra clean); companion_parity.py green;
apple_mrc_ownership_check.py unchanged (the two known local hits).
* ui: playlist icons, single-column core info, 50/50 right column on Win32 (Qt parity)
Second side-by-side pass at 200% DPI, Qt leading. The window now opens
at the same size with legible controls; the remaining differences were:
- Playlist icons. Qt shows the XMB dot-art asset for each system
(<assets>/xmb/dot-art/png/<playlist>.png, folder.png as fallback) at a
large row height; Win32 showed a 16 px shell folder on every row. The
asset rule moves into the core (companion_core_playlist_icon_path, so
Cocoa gets the same icons) and the Win32 list decodes each icon into a
32 px image list through the existing scaler, which also gives the
rows Qt's height. The shell folder stays at index 0 as the fallback
when the asset set is not installed. Icons are rebuilt with the list,
i.e. on playlist-directory changes only, not per frame.
- Core Info. It was a two-column key / value list, so the no-core text
(a key with no value) truncated at the key column, and long firmware
lines were cut. Qt renders "Key: value" as one text line per row; do
the same in a single wide column that scrolls horizontally instead.
- The right column splits 50/50 between Core Info and Boxart, as Qt's
two docks do, rather than a fixed boxart height.
- Entry columns titled "Name" / "Core" (QT_NAME / QT_CORE) rather than
the entry-dialog labels with their colons; the footer drops the
separate "View" caption Qt does not have.
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); consumers of
companion_core.h pass; companion_parity.py green. Not run here.
* ui: the main loops and the Win32 window stop hard-coding Qt as "the" companion
Sweep of HAVE_QT / HAVE_QT6 outside the Qt driver itself. Most sites are
legitimately Qt-specific (griffin_cpp's TU list, the feature-list
strings, the griffin HAVE_MAIN hack, config.features.h, Qt's own entry
in the companion driver table, and ui_cocoa.m choosing rarch_main over
the draw observer because of Qt's event loop) and stay. Five were
companion-generic behaviour wired to Qt alone:
- retroarch.c main loop: `ui_companion_qt.application->exiting` and
`->quit()` under HAVE_QT. That is "the active desktop companion asked
to exit / tell it to quit", whichever driver is selected. New
ui_companion_driver_wimp_exiting() / _wimp_quit() route it through
uico_st->wimp; ui_cocoa.m's rarch_main and cocoa_common.m's draw
observer use the same two calls (the observer path also gains the
wimp deinit before main_exit, the ordering fix the Win32 quit hang
needed, which it had missed).
- ui_win32.c: the main window's "Toggle Desktop Menu" item (and its
label lookup) existed only #ifdef HAVE_QT, so a native-companion
build had no menu entry to open its companion. Now HAVE_COMPANION_WIMP.
- configuration.c: the ui_companion_toggle SETTING_BOOL was gated on
HAVE_QT || HAVE_COCOA - the pre-refactor condition, missing desktop
Win32 and HAVE_COCOA_METAL. Now HAVE_COMPANION_WIMP.
- settings_def_menu_main_lists_2.h: the SHOW_WIMP ("Show Desktop Menu")
action row was HAVE_QT only, so the main menu had no way to open a
native companion. Guard widened to the HAVE_COMPANION_WIMP condition,
spelled out as settings_def cannot include the header.
tools/settings_table.reference regenerated (unchanged - the desktop
profile already defines HAVE_QT).
Behaviour for Qt builds is identical (same calls, through the driver
slot). Verified: settings table --check, unity (3 profiles) and
migrate selftest pass; all 15 consumers of ui_companion_driver.h,
configuration.c, ui_win32.c and retroarch.c pass
tools/mingw_syntax_check.sh; full Qt 6 retroarch links and starts
headless; companion_parity.py green.
* ui/companion: native companions read and write retroarch_qt.cfg
The Qt companion keeps its own settings in retroarch_qt.cfg beside
retroarch.cfg (QSettings, INI form); the native Win32 and Cocoa
companions ignored it, so a view type, thumbnail type, start playlist or
"suggest loaded core first" set in one did not carry to the others. All
three now share that file.
RetroArch's config_file cannot read it: its key scan takes every graph
character as part of the key, so QSettings' unspaced `key=value` becomes
a key with no '=' and the line is dropped - confirmed with a harness, it
parsed zero entries from a real retroarch_qt.cfg. Rather than touch that
hot parser, the core gets a small dedicated reader/writer for the
dialect: "[section]" and comment lines skipped, split at the first '=',
values verbatim, written back in QSettings' own shape so Qt reads it.
Unknown keys (Qt's @ByteArray geometry, theme...) survive a rewrite.
Verified with the harness: every key read, a set-and-reread round trip,
and the geometry blob preserved.
companion_core_setting_get / _get_bool / _get_int / _set / _set_bool
Both native companions honour the keys Qt writes:
view_type list / icons at startup; persisted on switch
icon_view_thumbnail_type boxart / screenshot / title / logo for the
icon view and the boxart pane
initial_playlist opened at startup, History as the fallback
(Qt's default), applied once
suggest_loaded_core_first launch-with combo ordering
show_hidden_files directory scan
save_last_tab / last_tab restore the Playlists / File Browser tab;
persisted on change when save_last_tab is on
Reads are a file load per call: only at startup and on user actions,
never per frame.
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); consumers of
companion_core.h pass; full Qt 6 retroarch links; companion_parity.py
green; apple_mrc_ownership_check.py unchanged. Not run here.
* ui: companion settings move from retroarch_qt.cfg into retroarch.cfg; Qt-isms dropped
The Qt companion kept its presentation settings in a private QSettings
INI, retroarch_qt.cfg, and the previous commit taught the natives to
read that file. One config is cleaner: the values become real RetroArch
settings in retroarch.cfg, saved with everything else, visible in the
Settings menu, and read by all three companions through settings_t. The
retroarch_qt.cfg reader/writer added last commit is removed again.
New settings (settings_def_desktop_menu.h, same HAVE_COMPANION_WIMP
guard; menu placement beside desktop_menu_enable; tools/
settings_table.reference regenerated, 19 rows):
desktop_menu_view_type uint 0 list / 1 icons
desktop_menu_thumbnail_type uint boxart/screenshot/title/logo
desktop_menu_suggest_loaded_core_first bool
desktop_menu_save_last_tab / _last_tab bool / uint
desktop_menu_save_geometry bool
desktop_menu_window_x/_y/_width/_height uint - the window geometry as
plain ints, replacing Qt's
QByteArray saveGeometry() blob
desktop_menu_show_welcome_screen bool
desktop_menu_scan_finish_confirm bool
desktop_menu_thumbnail_cache_limit / _max_size / _quality uint
desktop_menu_icon_view_zoom uint
desktop_menu_all_playlists_list_max_count / _grid_max_count uint
desktop_menu_theme uint system/dark/custom
desktop_menu_initial_playlist, desktop_menu_custom_theme (SETTING_PATH)
desktop_menu_hidden_playlists (comma list), desktop_menu_highlight_color
("#rrggbb") (SETTING_ARRAY)
Qt's private copy of show_hidden_files is dropped for RetroArch's own
show_hidden_files.
Qt-isms removed rather than carried: the QByteArray blobs for dock
positions (and its View Options checkbox), the file-browser table
header state and the options-dialog geometry are gone; window geometry
restores from the four ints. QSettings is no longer used anywhere: the
member, its construction, MainWindow::settings() and the two widget
copies are deleted, and every read/write goes through settings_t (the
View Options dialog included). Writes take effect immediately and reach
disk when RetroArch saves its config.
The companion core exposes the shared values to the backends as typed
accessors (companion_core_pref_*), including the thumbnail-type ->
repository-subdir mapping in one place; both native companions use them.
Verified: settings table --check, unity (3 profiles) and migrate
selftest pass; label strings regenerated (gen_lbl_str.py); consumers of
companion_core.h, ui_win32_companion.c, configuration.c,
menu_displaylist.c and menu_setting.c pass tools/mingw_syntax_check.sh
(mingw -Wall -Wextra clean); full Qt 6 retroarch links and starts
headless; companion_parity.py green; apple_mrc_ownership_check.py
unchanged.
* ui/companion: retroarch_qt.cfg is gone - one-time import into retroarch.cfg, then deleted
The previous commit moved the companion settings into retroarch.cfg; this
one makes sure the old file actually disappears. On the first companion
start that still finds retroarch_qt.cfg beside retroarch.cfg, the core
imports the values that have a home into settings_t and deletes the
file, so it does not linger as dead configuration after an upgrade.
Runs once per process at most.
Imported (with the same value mapping the Qt companion used):
view_type, icon_view_thumbnail_type, initial_playlist,
suggest_loaded_core_first, show_hidden_files, save_last_tab / last_tab,
save_geometry, show_welcome_screen, scan_finish_confirm,
thumbnail_cache_limit / _max_size / _quality, icon_view_zoom,
all_playlists_list/grid_max_count, theme, custom_theme, highlight_color,
hidden_playlists (QStringList's ", " separators normalised to ",").
Dropped with the file: the Qt blobs (geometry, dock_positions,
options_dialog_geometry, file_browser_table_headers).
Stale comments naming retroarch_qt.cfg in the natives, config.def.h,
settings_def_desktop_menu.h and menu_displaylist.c are updated; no
source mentions the file any more except this migration.
Verified with a harness against a real QSettings-shaped file: every key
lands as the right typed value, blobs are skipped, the file is removed
(a whole-process run confirmed the import log line and the deletion;
the exit-time config save could not be exercised headless).
companion_core.c passes tools/mingw_syntax_check.sh; consumers of
companion_core.h pass; full Qt 6 retroarch links; companion_parity.py
green.
* ui/drivers: Win32 companion - core combo and Core Info work, icon alpha, zoom, Boxart tabs
Five defects from the side-by-side, all in the Win32 companion:
- The Core combo dropped down to nothing. A COMBOBOX's window height is
the height of its *dropped list*, and the layout sized both combos to
one closed row, so the list had no room. They are now given list room
(the closed control stays one row tall regardless). The combo was in
fact populated; it just could not show its items.
- Core Info always read "No Core Information Available". The pane
followed the running core; Qt shows the core picked in the Core combo
(the entry's own or the playlist default). It now does the same:
cw_combo_core_path() resolves the combo's selection and the pane
refreshes on entry selection, on combo change, and when that core
changes.
- Playlist icons sat on black. The scaler forced alpha to opaque, so
transparent XMB PNGs kept whatever was under them. It now composites
the source alpha over the control colour, giving an opaque result
every comctl32 draws identically (v6 alpha and 9x mask-less alike).
- No thumbnail zoom and a fixed thumbnail size. The icon-view edge now
comes from desktop_menu_icon_view_zoom (0..100 -> 64..320 logical px,
DPI-scaled), with Qt's Zoom slider (msctls_trackbar32) and Thumbnail
type combo in the footer; a change rebuilds the image list and
redecodes. New companion_core_pref_icon_view_zoom / _thumbnail_type
accessors (and setters) back both.
- No way to switch the boxart pane's type. It gets Qt's four tabs
(Boxart / Title Screen / Screenshot / Logo) under its caption; the pane
keeps its own type, independent of the grid's Thumbnail combo, as Qt's
dock does.
Also: the status bar returns to "<version> - <core>" once a playlist has
loaded instead of staying on "Loading playlist...".
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); consumers of
companion_core.h pass; full Qt 6 retroarch links; companion_parity.py
green. Not run here (no Windows).
* ui/drivers: fix the Win32 icon view's cell spacing (grid came up scrambled)
After the zoom change the icon view laid items out in cells a few pixels
wide: labels wrapped to three characters and items piled on top of each
other. The cell spacing was computed from w->thumb_px, which is only set
when the thumbnail image list is (re)built - but at startup the view
type is applied before the first playlist builds it, so the spacing was
0 + 24 by 0 + 40 px. A zoom or thumbnail-type rebuild then resized the
image list without ever reapplying the spacing either.
The spacing now comes from the live thumbnail edge whenever it is
applied (cw_icon_spacing_apply), is reapplied every time the image list
is rebuilt at a new size, and the grid is re-arranged (LVM_ARRANGE)
after a rebuild or a view switch so existing items take the new cells.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean);
companion_parity.py green. Not run here (no Windows).
* ui: "All Playlists" for every companion, and it is where all three start
Qt's playlist list begins with All Playlists - the union of every
playlist - and that is the view it opens on. The native companions had
neither, so the two windows came up on different pages and the natives
could not show the combined library at all.
The aggregate is in the core so every backend shares it. Public playlist
index 0 is now All Playlists (name from QT_ALL_PLAYLISTS, path the token
"|||ALL|||" Qt has always used, so desktop_menu_initial_playlist can
name it and the setting means the same thing everywhere); the specials
and files follow. Selecting it parses every playlist file in turn under
the same per-frame budget as a single playlist - one file's parse
finishes, the next begins, the view updates once at the end - and the
entries are referenced through a (list, index) table rather than copied,
then sorted by label (strcasecmp) as Qt sorts, and capped by the larger
of desktop_menu_all_playlists_list/grid_max_count (0 = no cap).
Edits under the aggregate go to the entry's own file: new
companion_core_entry_playlist_path() / _entry_index_in_playlist()
resolve an aggregate row to the file it came from and its index there;
the Win32 and Cocoa delete actions use them, and playlist_open() rejects
the token so nothing can ever try to write "|||ALL|||.lpl".
Startup: with desktop_menu_initial_playlist empty, Qt, Win32 and Cocoa
all open All Playlists (Qt's History fallback goes too, so the three
agree). The All slot shows the folder icon like the specials.
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); consumers of
companion_core.h pass; full Qt 6 retroarch links; companion_parity.py
green; apple_mrc_ownership_check.py unchanged. The aggregate itself
needs a run with real playlists (it cannot be driven headless here).
* ui: select the first entry on load, and All Playlists is uncapped by default
Two more from the side-by-side, Qt leading:
- Qt selects the first entry of a freshly loaded playlist, so the boxart
pane and the Core section have content immediately; the natives left
nothing selected and the panes empty. Win32 now selects row 0 after the
entry list is rebuilt (LVN_ITEMCHANGED drives boxart, core combo and
core info from there); Cocoa selects row 0 in the table and the grid.
- All Playlists showed "1000 items" against Qt's 42116. The cap I gave
it took the larger of the list / grid max counts, and I had invented a
grid default of 1000. Qt in fact never applies either count (the
setters are set-only and their spin boxes are commented out), which is
why it shows everything. The grid default is now 0, and a cap applies
only when both counts are set (0 = no cap on that side), then the
larger - so the default is uncapped like Qt, while a user who sets
both still gets a bound. tools/settings_table.reference updated for
the default.
Verified: settings table --check, unity (3 profiles) and migrate
selftest pass; companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); full Qt 6
retroarch links; companion_parity.py green; apple_mrc_ownership_check.py
unchanged.
* ui: align companion guards with master's HAVE_COCOA; unretained grid back-pointer
* ui/drivers: bring the Cocoa companion up to the Qt / Win32 layout and feature set
The native Cocoa companion had fallen well behind: two bare tables in a
split view, no launch-with core, no zoom, no thumbnail or boxart type
switches, no search, no playlist icons, Core Info following the running
core, and its own status text. This puts it on the same footing as the
Win32 companion, Qt leading:
Layout (explicit frames recomputed in -layoutViews on resize; 10.4-safe,
no autolayout; points, so the display scales it as it does Qt):
Left column, as Qt's docks: Search [field][Clear]; Content Browser
with a Playlists | File Browser NSTabView (the second tab enters the
file browser, the first returns); the playlist table with an icon
column (XMB dot-art per system via companion_core_playlist_icon_path,
the Finder folder as fallback) at Qt's row height; Core: launch-with
NSPopUpButton + Info + Run.
Centre: the entries table or grid over Qt's footer - "N items", Zoom
slider, Thumbnail type popup, View (List / Icons) popup.
Right column: Core Info caption + single-column "Key: value" list,
Boxart caption + NSSegmentedControl for the four types + image view;
both shown by default, 50/50, like Qt's docks.
Window opens at 1280x720 centred, clamped to the screen; status bar
reads "<version> - <core or No Core>" and follows the core.
Behaviour, all through the shared core and its pref accessors:
- launch-with popup from companion_core_launch_options plus Ask and
Load Core...; Run launches with the choice, Ask / Load Core open the
picker; Core Info follows the popup's core as Qt's does, refreshed on
entry / grid selection and popup change
- zoom slider drives the grid's thumbnail edge (64..320 pt) and
persists desktop_menu_icon_view_zoom; thumbnail popup switches the
grid's type and redecodes; boxart segments switch the pane's own type
- search field filters the entry table (case-insensitive substring)
through a row -> entry map, so run / delete / boxart / popup resolve
the right entry under a filter; Clear resets it; Cmd-F focuses it
- view popup and menu items stay in sync; last tab persisted when
desktop_menu_save_last_tab is on; shared settings applied at start
- grid selection now drives boxart / popup / info like table selection
- Companion menu uses Qt's strings and gains Documentation
The split view and the toggles that added panes to it are gone; Core Info
/ Boxart toggles show or hide within the fixed layout.
Verified: apple_mrc_ownership_check.py unchanged (the two known local
autoreleased objects); bracket / brace / paren balance checked by a
character-level scan; every declared selector has an implementation;
companion_parity.py green. Not compiled here (no Apple toolchain) - the
Xcode build and a run are the real test for this one.
* ui/drivers: fix the Cocoa companion build (undeclared -wimp, Carbon icon constant)
CI on the Cocoa layout commit (95dee5f) fails the three Apple jobs -
macOS (Xcode, clang/ARC) and macOS PowerPC cross 10.4 / 10.5 (GCC 14,
10.5 SDK, MRC) - while every Windows, MSVC, MXE, c89 and Linux job is
green, so the breakage is confined to ui_cocoa_companion.m. Two errors,
found by auditing the file against those toolchains (the job logs are
not readable without a login from here):
- RACompanionGrid's -drawRect: calls [owner wimp] to reach the core,
but the controller never declared or defined a -wimp accessor. GCC
rejects the unknown selector on a typed receiver; clang under ARC
rejects the resulting id -> ui_companion_cocoa_wimp_t* conversion
outright ("implicit conversion of an Objective-C pointer to a C
pointer is disallowed with ARC"). Declared and defined.
- The playlist folder fallback used kGenericFolderIcon, a Carbon
constant from Icons.h that AppKit alone does not declare. The generic
folder's HFS type is 'fldr'; pass the literal to
NSFileTypeForHFSTypeCode (10.0), which the 10.4u import audit accepts.
Re-checked: every selector sent to self / owner is now declared and
implemented; all top-level C functions are static (the build carries
-Werror=missing-declarations); bracket balance by character scan;
apple_mrc_ownership_check.py unchanged; companion_parity.py green.
* ui/drivers: fix the Cocoa companion build (Xcode log: shadowed grid edge, BOOL receiver)
From the CI Xcode log on c2e0e39:
- error: passing 'NSRect' to parameter of incompatible type 'CGFloat'
(and five "invalid operands" errors following it) in
RACompanionGrid -drawRect:. When the grid's fixed CC_GRID_THUMB
became the zoomable `thumb` ivar, the rename collided with the
method's existing local `NSRect thumb` (the thumbnail's box), so the
edge and the box became one identifier. The box is now `trect`; the
edge stays `thumb`.
- error: bad receiver type 'BOOL': [[window makeFirstResponder:grid]
self] messaged the BOOL that -makeFirstResponder: returns. Dropped the
outer message.
Also cleared the warnings the log shows in the same method:
-setFlipped: (deprecated 10.6) is now only used below the 10.6 SDK; on
10.6+ the draw call takes respectFlipped:YES instead, which is the
replacement Apple's notes describe. NSCompositeSourceOver goes through
the compat name the defines header already provides, and the two
NSTextAlignment* names are mapped to their pre-10.12 spellings under an
SDK check so the 10.5-SDK PowerPC build sees them too.
Re-checked: bracket balance by character scan; every selector sent to
self / owner declared; apple_mrc_ownership_check.py unchanged.
* ui/drivers: Win32 companion entry list goes virtual - grid builds instantly
Building the icon view took minutes on a large library where Qt is
instant. Three costs, all structural: the entry list was a retained
list view, so populating it was one LVM_INSERTITEM (plus a sub-item
set) per entry - 42k round trips through the control; LVM_ARRANGE then
laid out 42k icon cells; and thumbnails were decoded top-to-bottom at
one per frame regardless of what was on screen, so the visible screen
only filled in once everything above it had (42k entries: twelve
minutes). Qt's view is virtual: its model reports a count, cells are
drawn on demand, thumbnails load for the cells actually shown.
The entry list is now the Win32 equivalent, LVS_OWNERDATA (comctl32
4.70, Windows 95):
- populating is a filtered row map (rows[]: visible row -> entry index)
and one LVM_SETITEMCOUNT, whatever the size; no inserts, no arrange.
The search filter builds the same map, so run / delete / boxart / core
combo resolve through rows[] instead of a per-item lParam.
- the control asks for each row's text and image as it draws
(LVN_GETDISPINFOA - the ANSI notification, matching the ANSI control
even under a UNICODE build); the browse view is served the same way,
with the trailing backslash on directories.
- thumbnails are decoded only for rows the control has asked to draw:
GETDISPINFO queues a row that has no image yet, and the iterate hook
drains the queue newest-first (the rows most likely still on screen)
within a 4 ms budget, redrawing each row as its image lands. Nothing
is decoded for rows never scrolled to.
- memory is bounded: decoded thumbnails live in a ring of 512
image-list slots, recycled round-robin; an evicted row falls back to
the placeholder and re-queues if it comes into view again. Previously
the image list grew by one bitmap per entry (42k x 128 x 128 x 4 =
2.7 GB for a full pass).
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean), plus
a -DUNICODE syntax pass; companion_parity.py green. Not run here.
* ui/drivers: Win32 companion decodes thumbnails on a thread pool
Still slower than Qt after the virtual list. The remaining gap is where
the decode runs: Qt's ThumbnailLoader is a dedicated thread that runs
image_texture_load (the same decoder) with no per-frame budget, and its
UI thread only receives finished images. The Win32 companion decoded on
the UI thread under a 4 ms/frame budget - a boxart PNG takes 5-20 ms,
so about one thumbnail per frame, sharing that frame with emulation.
Now the decode is off the UI thread, and arranged to come out ahead of
Qt rather than level with it:
- a pool of decode threads (cores - 1, at least one, at most four;
libretro-common rthreads, so it builds wherever HAVE_THREADS does,
9x included) takes jobs newest-first - the rows most likely still on
screen - and decodes in parallel. Qt has one thread.
- each worker scales to thumbnail size itself and hands the UI thread a
T x T ARGB buffer; the UI thread's per-frame work is a DIB and an
image-list replace per finished thumbnail (~0.2 ms), never a decode.
Qt caches the full-size pixmap and rescales it on every paint.
- the request path resolves the file on the UI thread (the companion
core is single-threaded), so a job needs nothing from the core.
- a generation counter, bumped when the list, filter, zoom or thumbnail
type changes, makes in-flight results stale; they are dropped when
they land rather than blitted into the wrong row or size.
- all cross-thread state (queues, row map, thumb_idx, thumb_px) is read
and written under one lock; workers park on a condition and are woken
by requests and by ring space; shutdown joins them.
- without HAVE_THREADS the UI thread decodes under the old budget, so
the driver still builds and works.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean) with
and without HAVE_THREADS; companion_parity.py green. The pipeline
itself (queue, ready ring, worker loop, generation handling, shutdown)
was extracted into a Linux harness against the real rthreads and run
under ThreadSanitizer: no races, every decode installed exactly once,
stale results dropped, nothing leaked. The decode / GDI halves need a
Windows run.
* ui/drivers: Win32 companion - fix the quit hang; thumbnails cached across playlists
Two reports after the thread-pool commit: RetroArch hangs at close, and
the grid, while faster, is still not "instant" the way Qt's is.
The hang. ui_companion_win32_wimp_deinit freed the row map, thumb_idx
and the image list and only then called DestroyWindow. A virtual
(LVS_OWNERDATA) list view can ask for rows (LVN_GETDISPINFO) while it
is being torn down, so the handler read freed memory and a destroyed
image list during DestroyWindow - a fault inside the modal-clocked
teardown, which presents as a hang. Deinit now empties the list
(LVM_SETITEMCOUNT 0, image list detached) and destroys the windows
first, and frees the data after; the hwnd/entries pointers are cleared
so a late iterate or log call is a no-op. The pool shutdown is also
made hang-proof on its own: workers wait with a 100 ms timeout so a
lost wake-up on the Windows condition variable can never keep one
parked past quit, and stop broadcasts rather than signalling per
worker. (A TSan harness of the real pipeline code with four workers,
mid-decode and ring-blocked at quit, stops in 0 ms with no races.)
The "instant". The Qt companion keeps decoded pixmaps in a QCache keyed
by file path that survives switching playlists, so revisiting one draws
thumbnails with no decode at all. The Win32 slots were per row and
reset on every rebuild. They are now keyed by thumbnail path in an
open-addressed hash table and kept across list changes: only a new
thumbnail size or type drops them. A row whose file is already a slot
gets it on the spot, no decode. The slot budget follows the size to
stay near 64 MiB (a few hundred at 320 px, thousands at 64 px). Rows
whose thumbnail file does not exist are marked once (-2) so the
filesystem is not probed for them on every paint.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean) with
and without HAVE_THREADS; companion_parity.py green; the pipeline
harness under ThreadSanitizer (shutdown stress, 5 trials). GDI / window
teardown needs the Windows run.
* ui: Win32 File Browser laid out like Qt's; drive list on Windows
The File Browser tab on Win32 came up empty and did not work the way
Qt's does.
Empty: with no content directory configured, the core refused to open
a top level on Windows (I had ruled out a "bogus root"), so there was
nothing to list. On Windows the top level is the drive list: the core
now lists GetLogicalDrives() as directory entries ("C:\") for an empty
browse_dir, a drive root's ".." goes back to it, and the status bar
calls that level "Computer".
Layout: Qt's File Browser keeps the folder tree in the left pane and
shows the selected folder's files in the content view. Win32 had turned
the content view into a combined folder + file listing and left the
playlists in the left pane. Now, in the browser: the left list shows
the current folder's subfolders (".." first, folder icon), the content
view - list or grid - shows its files, a folder descends on double-
click / Enter, and a file loads (or opens the core picker) as before.
Leaving the tab, or picking a playlist, puts the playlists back in the
left pane; a playlist refresh that lands while browsing is deferred to
then. The core exposes the split (companion_core_browse_dir_count: the
listing is sorted folders first, so [0, dc) are folders and [dc, n)
files) for any backend that wants the same two-pane shape.
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); consumers of
companion_core.h pass; companion_core.o builds; companion_parity.py
green. Not run here (no Windows).
* ui: Win32 file browser - drive names shown, folder icon opaque
Two defects in the drive list the previous commit added:
- No names. companion_core_browse_name() took path_basename() of each
entry; a drive root "C:\" has no basename, so every row was blank.
When the basename is empty the name is the path itself.
- Black squares behind the folder icons. The fallback icon went into
the playlist image list raw (ImageList_AddIcon) and the list was
created with ILC_MASK; a masked list draws an alpha icon's transparent
pixels as black. The list is now unmasked and every image in it is an
opaque bitmap composited on the window colour: index 0 is the XMB
folder.png asset (what Qt shows) through the same compositing scaler
the playlist icons use, and only if the asset set is missing is the
shell folder icon used - drawn with DrawIconEx onto a window-coloured
bitmap, so it too is opaque. companion_core_folder_icon_path() gives
backends the asset's path.
Verified: companion_core.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean); consumers of
companion_core.h pass; companion_parity.py green. Not run here.
* ui/companion: shared thumbnail engine for every companion, with a regression test
Thumbnails in the Win32 grid only appeared for entries that were
clicked. Reading the Qt companion's pipeline end to end
(ThumbnailLoader, PlaylistModel::loadThumbnail, MainWindow::
updateVisibleItems, GridView::visibleIndexes) gives the model and, by
contrast, the bug:
Qt requests decodes for the rows its grid reports as visible, on a
50 ms scroll debounce, and never for anything else. Win32 requested
from LVN_GETDISPINFO - and comctl32 sends that for every item of a
virtual icon view while laying it out, so a 42k-entry list flooded
the 512-job queue, the oldest (the visible top) were dropped, the
workers decoded the bottom of the list, and a visible row was only
requested again when something repainted it: a click.
Rather than patch that in place, the whole pipeline moves into the
core as one engine every companion uses, ui/companion/companion_thumbs:
- cache of decoded, already-scaled thumbnails keyed by (path, edge),
byte budget (64 MiB default) with LRU eviction; survives switching
playlists, so a revisit draws with no decode (Qt's QCache behaviour)
- decode thread pool (cores - 1, 1..4; Qt has one), scaling on the
worker so the UI thread only ever copies finished pixels; a budgeted
UI-thread fallback without HAVE_THREADS
- two-lane request queue: urgent jobs newest-first (the rows on
screen, topmost first), then a prefetch lane oldest-first; keys are
de-duplicated against the cache and the queues
- cancel() empties the queues and keeps the cache; results in flight
still land and are cached; delivery carries the backend's tag, which
encodes its own row generation, so a stale result is dropped by the
backend, never blitted into the wrong list
- pixels are opaque ARGB8888: Windows' DIB order, a byte swap for Cocoa
Both natives now drive it the way Qt drives its loader:
Win32: each frame, the rows on screen are found from the list view's
own item rects (monotonic in row order, so a binary search), requested
urgently topmost-first, a screen either side prefetched; finished
pixels go into a recycled image-list slot ring; LVN_GETDISPINFO only
reads a row's slot. The local thread pool, hash table and rings are
gone.
Cocoa: the grid reports its visible rows from its visibleRect; the
same request / prefetch / poll cycle runs from iconTick; finished
pixels become an NSImage through an NSBitmapImageRep. The per-frame
synchronous NSImage load is gone.
Regression test: ui/companion/test/companion_thumbs_test.c, run by
tools/companion_thumbs_test.sh plain, under ThreadSanitizer and under
ASan+UBSan, against the real decoder (image_texture_load) with TGA
fixtures it writes itself. It checks a decode lands with the right
scaled / letterboxed pixels, cache hits without decode, LRU eviction
under budget, cancel keeping the cache and allowing re-requests, an
undecodable file delivered as NULL and forgotten, 300 keys over four
threads delivered exactly once, and free() with work in flight. The
engine is also compiled once as C89 (-ansi -pedantic, no threads).
Wired into Linux-libretro-common-tests.yml beside the parity check.
The test found one engine bug before it ever reached a backend
(re-request of a key whose decode was in flight across cancel()).
Verified: the test passes plain, TSan and ASan; companion_thumbs.c,
ui_win32_companion.c and the consumers of companion_thumbs.h pass
tools/mingw_syntax_check.sh (also with HAVE_THREADS off); full Qt 6
retroarch builds with the engine in and starts headless;
companion_parity.py green; apple_mrc_ownership_check.py unchanged.
Not run on Windows / macOS here.
* ui/companion: boxart panes on the engine; Cocoa two-pane browser; engine takes w x h
Follow-ups to the shared thumbnail engine, so both natives are on it
for everything and nothing decodes on a UI thread any more:
- The engine keys and scales by (path, w, h) rather than a square
edge, so the boxart pane can ask for its own rectangle and get an
aspect-fit, letterboxed image instead of a square. Grid cells pass
edge x edge. The test gains a rectangular case (20x80 into 60x30:
centre column image, corners letterbox).
- Boxart panes (Win32 and Cocoa) come from the engine: cached at once,
otherwise an urgent request tagged with the entry and a boxart bit,
delivered through the same per-frame poll, which now also runs in
list view so the pane fills in there too. The pane is cleared while
it waits, so a stale cover never sits under a new selection. Both
panes decoded synchronously per click before; Cocoa's per-frame
NSImage loader (cc_thumb_image) is gone.
- Cocoa File Browser is now two-pane like Qt's and Win32's: the left
table shows the folders (companion_core_browse_dir_count, ".." first,
XMB folder.png or the Finder folder) and the content table the files;
double-click a folder to descend, a file loads; leaving the tab or
picking a playlist restores the playlist pane. The row map serves the
browser too, so run / boxart / popup resolve files correctly.
- companion_parity.py requires the engine's request and poll calls and
the folder/file split on every backend (Qt waived: its own
ThumbnailLoader / QFileSystemModel), so a backend cannot quietly go
back to synchronous loading. 22 features, all three agree.
Verified: companion_thumbs_test.sh plain, TSan and ASan pass (ASan
caught a test-side index bug in the new rectangular case, fixed);
companion_thumbs.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (also with HAVE_THREADS off); consumers of
companion_thumbs.h pass; full Qt 6 retroarch builds and starts
headless; apple_mrc_ownership_check.py unchanged. Not run on Windows /
macOS here.
* ui: Qt on the shared thumbnail engine; Win32 / Cocoa file browser like Qt's
Qt joins the engine, so all three companions decode through one path:
- PlaylistModel drops ThumbnailLoader (its own QThread) and the QCache
of full-size pixmaps; it owns a companion_thumbs instance instead.
loadThumbnail() requests at the grid's cell size (fed from the zoom
slider), data(THUMBNAIL) converts the engine's pixels to a QPixmap
once per (path, size) into a small conversion cache, and a 16 ms
timer polls while anything is pending; thumbnailReady(path) is
emitted for the row's dataChanged and for other consumers. The
thumbnail-cache-limit setting becomes the engine's byte budget.
- The sidebar's four thumbnail widgets and the file-browser image
preview go through the same engine (imageAt / requestImage at the
widget's size), replacing the second loader thread and two
synchronous full-size decodes on the UI thread. ThumbnailLoader and
pixmapFromPathRA are gone; the previewLoader / m_thumbnailPixmaps
state with them.
- The engine gains set_budget(), forget(path) (a downloaded thumbnail
replaces its cached sizes) and pending() (queued + decoding + not
yet polled, so a poll timer never stops early). All three are in the
regression test.
The File Browser on Win32 now matches Qt's (screenshots side by side):
drive entries named "<label> (C:)" from GetVolumeInformation (the core
does it, so Cocoa would too); Up / Start Directory / Downloads buttons
under the tabs (Qt's toolbar); the content view is the folder's whole
listing as a table with Name / Size / Type / Date Modified, drawn with
the shell's own small image list so drives, folders and file types
have Explorer's icons (system image list: nothing of ours to
composite); size and date from FindFirstFile (Windows 95); the browser
is always a table like Qt's, and the playlist view type returns on
leaving. The boxart pane is cleared in the browser - Qt shows none for
a file selection - which the screenshot showed still holding the
previous playlist entry's cover. companion_core_browse_up() is the
"Up" action for any backend.
Cocoa gets the same: the three buttons, the whole listing with a Type
column, boxart cleared, Up / Start / Downloads.
companion_parity.py: Qt's engine waiver is gone - all three backends
reference the engine's request and poll for real.
Verified: companion_thumbs_test.sh plain, TSan, ASan pass with the
new cases; companion_core.c, ui_win32_companion.c and the consumers of
companion_core.h / companion_thumbs.h pass tools/mingw_syntax_check.sh;
full Qt 6 retroarch builds with the model on the engine and starts
headless with the qt companion; companion_parity.py green;
apple_mrc_ownership_check.py unchanged. Not run on Windows / macOS.
* ui/companion: regression test for the companion core; two bugs it found
A second regression test, this one for the toolkit-free model every
desktop companion drives (companion_core.c). It links the real core
and playlist.c against libretro-common with RetroArch's state replaced
by fixtures (companion_core_stubs.c: settings, runloop, path slots, an
empty core-info world, recording stand-ins for the load / scan tasks),
writes real .lpl files, and checks:
- the playlist listing: All Playlists first with the token path, the
configured specials next, then the files in order, with their paths
- selecting a playlist through the budgeted iterate and reading its
entries; each row's own file and index within it
- All Playlists: every file merged, sorted by label case-
insensitively, rows mapped back to their file and index
- the thumbnail path rule: the repository layout, label sanitising
(& * / : ` < > ? \ | -> _), the extension probe, the default path
for a missing file, image content as its own thumbnail
- the file browser: folders first with "..", the folder / file split,
descending, going up to the root, up refused at the root
- Run: a DETECT entry with nothing loaded asks for a core with its
path; an entry with a core pushes the load task with that core
- launch options: the entry's own core is offered, nothing duplicated
Run by tools/companion_core_test.sh, plain and under ASan+UBSan; wired
into Linux-libretro-common-tests.yml. On its first run it found two
bugs in the browser that no screenshot had shown:
- a use-after-free when descending into a folder: browse_open() was
handed a pointer into the current listing, freed that listing, then
copied the path from it. The path is copied before anything is
freed.
- "Up" could never reach the root on Unix: path_parent_dir() turns "/"
into "", which browse_open() took as "use the default directory", so
going up from the content directory looped back into it. The parent
step now maps "" to "/" and keeps browse_dir canonical (no trailing
separator below the root), so it compares and terminates.
tools/mingw_syntax_check.sh --consumers-of skips */test/ harnesses:
they are Linux-only by design and have their own scripts.
Verified: companion_core_test.sh plain and ASan pass;
companion_thumbs_test.sh passes; companion_core.c passes
tools/mingw_syntax_check.sh (mingw -Wall -Wextra + linux c89 pedantic)
and its consumers pass; companion_parity.py green.
* ui: abortable decodes (no exit hang), browser previews, no disk I/O on paint
Three reports: a thumbnail preview in the File Browser can take a very
long time to show, RetroArch can still hang on exit, and the Win32 /
Cocoa companions never preview in the browser at all. Traced, and the
traversal audited while at it:
Directory traversal is not the problem: dir_list_new gets each entry's
type from the enumeration itself (FindFirstFile attributes on Windows,
d_type elsewhere) - no per-file stat. The bottlenecks were these:
1. A decode could not be interrupted. image_texture_load runs both of
its stages incrementally (one chunk / one pass per step) but never
looked up between steps, so a worker decoding a large image ran to
completion whatever happened. On exit, free() joined that worker:
the hang. In the browser, clicking through large images queued
full decodes behind each other on the worker.
libretro-common gains image_texture_load_ex() with an abort hook
asked between chunks and between passes (image_texture_load is now
that with no hook). The engine's workers decode through it and
abort when the engine is shutting down or when their job's epoch is
stale - i.e. a cancel() happened since it was queued. Aborted jobs
are not delivered and their entries are dropped, so a later request
decodes again. Two new tests: cancel() during 8 x 16 MiB decodes
lets the re-requested key land first; free() with the same in
flight returns in well under a second.
2. Win32 did disk I/O while painting the browser table: FindFirstFile
per row per sub-item on every repaint, and SHGetFileInfo hitting
the shell and disk for every directory. Metadata (size, date, icon)
is now resolved once per row - one FindFirstFile, and the icon by
attributes alone (SHGFI_USEFILEATTRIBUTES; a drive keeps its real
icon) - and paints read the cache.
3. Qt's QFileSystemModel asks the shell for every folder's own icon
(a disk lookup per folder) and starts a QFileSystemWatcher on every
directory it has shown. Both are off now
(DontUseCustomDirectoryIcons, DontWatchForChanges): the generic
folder icon is what the natives show too, and the table is rebuilt
on navigation. Selecting an image in Qt's browser abandons the
decode of the previous one (abandonPending -> engine cancel).
4. Previews in the natives: Qt previews an image file selected in the
browser in its sidebar; Win32 and Cocoa cleared the pane in browse
mode. Both now show the selected image file through the engine
(cw_boxart_update_path / refreshBoxart's browse branch), cleared for
non-images. Since abandoned decodes leave grid rows marked
"requested", both natives re-ask for empty on-screen cells when the
engine goes idle.
Verified: companion_thumbs_test.sh plain, TSan and ASan pass with the
abort tests; companion_core_test.sh passes; image_texture.c and its
consumers, ui_win32_companion.c pass tools/mingw_syntax_check.sh;
full Qt 6 retroarch builds and starts headless; companion_parity.py
green; apple_mrc_ownership_check.py unchanged. Not run on Windows /
macOS.
* ui/companion: directory listings enumerate off the UI thread, for every backend
Going up into a large directory froze the Qt companion's File Browser
for seconds; the natives enumerated on the UI thread too. Nothing in
the browser may block, so the listing moves into the core as one
asynchronous, shared implementation, and each backend only paints it:
- companion_core_browse_open() now enumerates on a worker (rthreads;
synchronous fallback without HAVE_THREADS): dir_list_new, the
folders-first sort, and one stat per entry for size and mtime - all
off the UI thread. The result lands in companion_core_iterate() and
the new on_browse_changed callback fires (added last in the callback
table, so existing positional tables stay valid). The previous
listing stays readable until then. A newer open supersedes the one
running: it stops between entries, its result is discarded, and no
second thread is ever spawned alongside it. browse_busy() says when
"Loading..." applies.
- Size / Type / Date Modified are formatted by the core
(browse_size_str / _type_str / _date_str) from what it gathered, so
the three backends show the same strings and none of them touches
the disk to paint. The Win32 FindFirstFile-per-row cache is gone;
only the shell icon index stays local (by attributes, no lookup).
Qt: the file table drops QFileSystemModel + the sort/filter proxy.
That pair was the stall: the model kept every visited directory
loaded, the gatherer inserted rows in batches with the proxy
re-filtering and re-sorting on each one (O(n^2/100) on the UI
thread), and header sorting re-sorted the whole directory on every
navigation. BrowseTableModel paints the core's listing (Name / Size /
Type / Date, three stock icons resolved once, the search filter as a
row map) and resets when the listing lands. The directory tree keeps
QFileSystemModel (folders only) with its watcher and custom icons off.
Win32 / Cocoa: open / up / descend show "Loading..." and rebuild when
the listing lands, instead of right after open() (which would show the
old listing now).
companion_core_test covers the async path: the listing lands through
iterate(), busy() clears, size / type / date strings, supersession
(open A then B: only B delivers, once), Up to the root; run plain,
ASan and now TSan. companion_parity.py requires on_browse_changed on
every backend (Qt's two-pane waiver becomes its BrowseTableModel).
Verified: companion_core_test.sh plain / ASan / TSan and
companion_thumbs_test.sh pass; companion_core.c, ui_win32_companion.c
and the consumers of companion_core.h pass tools/mingw_syntax_check.sh;
full Qt 6 retroarch builds and starts headless; companion_parity.py
green; apple_mrc_ownership_check.py unchanged. Not run on Windows /
macOS.
* ui/drivers: clear the boxart pane the moment the File Browser is entered
Switching to the File Browser tab still left the previous playlist
entry's cover in the boxart pane on Win32. Two reasons:
- the clear had moved to the listing's landing (cw_browse_rebuild),
which is now asynchronous; until then the old cover stayed
- a static control does not repaint on STM_SETIMAGE with NULL: it
keeps drawing the image it had until something invalidates it, so
even once the clear ran, the cover remained on screen
Now cw_browse_enter clears the pane at once (and resets the core popup
and Core Info to the no-entry state, as Qt shows for the browser), and
the clear invalidates the control. Cocoa does the same on entry
(refreshBoxart sets nil, popup and info reset) so both natives match.
Verified: ui_win32_companion.c passes tools/mingw_syntax_check.sh
(mingw -Wall -Wextra clean); companion_parity.py green;
apple_mrc_ownership_check.py unchanged. Not run on Windows / macOS.
* ui/companion: header-click sorting in the file browser, once, for every backend
Clicking Name / Size / Type / Date Modified in the browser did nothing
(Win32, Cocoa) or nothing useful (Qt: the core-backed table had sorting
off). The sort lives in the core, so the three companions order the
listing identically:
- companion_core_browse_sort(column, ascending): folders always first
(".." first of all), then the files by name (case-insensitive), size,
type (extension) or modification time, ties broken by name. One qsort
over the already gathered arrays - no I/O. It applies to the current
listing at once (on_browse_changed fires) and is remembered, so every
listing that lands afterwards comes in the chosen order.
- Qt: BrowseTableModel::sort() forwards the header click; sorting is
on again, with the indicator on Name.
- Win32: LVN_COLUMNCLICK forwards it; the same column again flips the
direction, a new one starts ascending, as Qt's header does.
- Cocoa: the entries table gets sort descriptor prototypes (Name / Type
/ Size / Date; the Size and Date columns show only under the
browser) and sortDescriptorsDidChange: forwards them.
companion_core_test covers each column both ways, the folders-first
invariant, tie-breaking by name, and the order persisting across a
re-enumeration. companion_parity.py requires the sort call on every
backend (24 features).
Verified: companion_core_test.sh plain and ASan pass; companion_core.c,
ui_win32_companion.c and the consumers of companion_core.h pass
tools/mingw_syntax_check.sh; full Qt 6 retroarch builds and starts
headless; companion_parity.py green; apple_mrc_ownership_check.py
unchanged. Not run on Windows / macOS.
* ui/companion: fix the crash on the drive list after a sort; scenario regression tests
Clicking a drive in the File Browser crashed on Windows once the
browser had been sorted by Date (or Size). The drive-list branch of the
listing worker returned without allocating the size / mtime arrays
(only the Windows path did that), and the sort comparator indexed
mtime[] for the Date column unconditionally: NULL dereference in
companion_core_browse_cmp the moment the drive list landed and the
remembered sort was applied to it.
Both ends fixed: the drive list now goes through the same metadata
allocation as every other listing (a drive root is simply not stat'ed),
so there is one job path; and the comparator never indexes an array it
does not have.
Tests, in the shape that catches this class before it reaches a
Windows box:
- test_sort_without_metadata builds the drive-list shape directly (a
folders-only listing with no metadata arrays, via a COMPANION_CORE_
TESTING hook) and sorts it by every column both ways. On the old
comparator this SEGVs under ASan at the exact line; on the fix it
passes - verified both ways.
- test_sort_then_folders_only runs the real job path: every sort
setting, then open a folders-only directory, then sort the landed
listing by every column again.
- test_backend_scenario replays the click sequences the backends send
to the core in a user's order: enter the browser, click every header
both ways, select every row (name / path / size / type / date /
is-dir, as the panes and tables resolve them), descend, activate a
file, Up, Start Directory, rapid navigation without waiting, sort
again, back to a playlist. Any fault on the core side of any
companion trips it. Runs plain, ASan and TSan.
Verified: companion_core_test.sh plain / ASan / TSan pass;
companion_core.c passes tools/mingw_syntax_check.sh (mingw -Wall
-Wextra + linux c89 pedantic) and builds; companion_parity.py green.
* ui/companion: stop the reset -> sort -> changed -> reset recursion (Qt stack overflow)
Clicking a drive in the Qt File Browser blew the stack: the backtrace
is ui_companion_qt_core_on_browse_changed -> Qt -> the same, twelve
thousand frames deep. The loop:
the listing lands -> onBrowseChanged -> the model resets ->
QTableView re-applies its header sort on a reset -> model sort() ->
companion_core_browse_sort() -> on_browse_changed -> ...
companion_core_browse_sort() fired the callback even when the requested
order was the one already in place, so a view that re-applies its sort
on "changed" - which Qt's table does on every model reset - never
converged.
Fixed once in the core: a sort that changes nothing does nothing and
fires nothing. The Qt model's sort() is idempotent too, and
onBrowseChanged has a re-entrancy guard. companion_core_set_ud() is
added (a backend can create the core before the object its callbacks
act on).
Regression test: test_sort_callback_reentrancy registers a listener
that does what Qt's table does - re-applies the current sort on every
"changed" - with a depth counter that stops it at 3, so a loop is
reported rather than crashed in. On the previous core it reports depth
4 in every case; on this one, one callback per real change and none
for a repeat. Verified both ways.
Verified: companion_core_test.sh plain / ASan / TSan pass;
companion_core.c and the consumers of companion_core.h pass
tools/mingw_syntax_check.sh; full Qt 6 retroarch builds and starts
headless; companion_parity.py green.
* ui: browser header clicks sort, and stay sorted, on Qt / Win32; Date and Type proven
Header clicks in the Qt browser appeared to do nothing: every column
sorted the same. The core sorted correctly; the Qt window then undid
it. onBrowseChanged() restored the header state saved before the last
navigation - and that state carries a sort indicator (Name ascending),
so after each click's reload the view snapped its indicator back to
Name and re-sorted by it. Now the saved state (column widths) is
restored once, after a navigation, and the indicator is always set from
the core's actual order (signals blocked, so it never re-sorts by
itself).
Win32: the clicked column shows a sort arrow (HDF_SORTUP / SORTDOWN;
ignored by older comctl32), and a rebuild no longer resets the column
widths the user set (the browser rebuilds on every landing and every
sort). Cocoa's descriptor path re-read and left as is: it already
forwards the clicked column and direction.
The core test now proves each column orders differently, not just
Name and Size: Date with explicit, distinct mtimes (newest first
descending, oldest first ascending), Type across six extensions
(grouped and ordered by extension, names within a group, reversed when
descending), and an explicit check that the four sorts produce four
different first files.
Verified: companion_core_test.sh plain / ASan / TSan pass;
ui_win32_companion.c passes tools/mingw_syntax_check.sh; full Qt 6
retroarch builds and starts headless; companion_parity.py green;
apple_mrc_ownership_check.py unchanged. Not clicked on Windows / macOS.
* ui/drivers: browser header shows the sort direction on Win32 without comctl32 v6
The Win32 companion showed no arrow on the sorted column: it used
HDF_SORTUP / HDF_SORTDOWN, which only a themed (v6) comctl32 draws, and
this build carries no common-controls v6 manifest - the header is the
classic one, which ignores those bits.
The classic header shows a picture instead: the sorted column now gets
HDF_IMAGE with an image on the right of its caption, from a two-entry
image list (up, down) drawn with GDI on the button colour at creation.
The v6 bits are still set for a themed header, so either kind shows
the direction; leaving the browser clears the arrow off the playlist
columns. Qt explicitly turns its header's indicator on.
Verified: ui_win32_companion.c passes tools/mingw_syntax_check.sh
(mingw -Wall -Wextra clean); full Qt 6 retroarch builds;
companion_parity.py green. Not run on Windows.
* ui/drivers: Qt and Cocoa browser headers follow the core's sort like Win32's
One rule for the browser header on all three companions: a new column
starts ascending, the same column again flips, and the header's
highlighted column and arrow always show the order the core's listing
is actually in - whether it came from a click on this backend or was
remembered from before (a re-opened directory lands already sorted).
Cocoa: after every reload the table's sort descriptors, highlighted
column and indicator image are set from the core (syncSortIndicator;
the programmatic set is flagged so sortDescriptorsDidChange: does not
treat it as a click). Leaving the browser clears the highlight and
arrows off the playlist table. Qt: the initial indicator is taken from
the core rather than hard-coded to Name ascending; the per-landing sync
was already in place. Win32 already did both.
Verified: full Qt 6 retroarch builds; companion_parity.py green;
apple_mrc_ownership_check.py reports one new local autoreleased object
(the descriptor, retained by setSortDescriptors: within the call).
Not run on macOS.
* ui/drivers: Win32 browser header showed a document, not the sort arrow
A report-view list view hands its small image list to its header when
LVM_SETIMAGELIST(LVSIL_SMALL) is sent, replacing whatever the header
had. cw_browse_rebuild set the header's arrow image list first and the
rows' shell image list after, so the header's image 0 / 1 became the
shell's first two icons - a document. The arrow image list is now set
after the rows' one, every rebuild.
Verified: tools/mingw_syntax_check.sh (mingw -Wall -Wextra clean). Not
run on Windows.
* ui/companion: thumbnail engine use-after-free on a doubly-recorded key; test races
CI's TSan run of companion_thumbs_test failed at "re-requested decoded"
while the plain run passed. Two test defects and one engine bug behind
it:
- The waits were counting deliveries, not the delivery they meant. A
job popped before a cancel() still lands with its old epoch; with one
worker that stale delivery satisfied drain(1) before the re-requested
key had landed, and the following get() saw nothing. TSan's
scheduling made it hit. The tests now wait for the specific tag
(drain_tag) wherever a re-request is checked.
- strdup was implicitly declared under the script's -ansi pass (the
warning in the log); on a 64-bit host an implicit int return would
truncate the pointer. Replaced with strldup(path, strlen(path) + 1) -
and note the + 1: strldup(s, n) copies n - 1 characters, strlcpy
style. My first attempt passed strlen(path) and silently truncated
every path by one character, which made every decode fail and, with
the next point, exposed the engine bug.
- Engine: an entry could be freed while another finished record still
pointed at it. A failed decode's record freed the entry in poll()
("undecodable: forget it"); a second record for the same key - a job
taken before a cancel() and a re-request after it - then dereferenced
it (ct_find walked a freed chain link). Entries are now reference-
counted by the jobs and records that point at them (request +1, ring
drop / cancel pop / polled record -1) and freed only at zero.
Regression test: test_double_failure_no_uaf builds that shape twenty
times (request an undecodable file, cancel, request again, poll, look
it up). On the branch-tip engine it is a heap-use-after-free under
ASan five runs out of five; on this engine it passes, plain, TSan and
ASan, repeatedly.
Verified: companion_thumbs_test.sh plain / TSan / ASan pass (ASan x5);
companion_core_test.sh passes; companion_thumbs.c passes
tools/mingw_syntax_check.sh (mingw -Wall -Wextra + c89 pedantic) and
builds.
* ui/companion: animated previews (APNG, WEBP, WEBM, MP4) in every companion's pane
The File Browser preview showed only the first frame of an animated
file. RetroArch's own File Browser (gfx_thumbnail) plays the selected
thumbnail through libretro-common's animation stream API -
image_transfer_anim_stream_* over APNG (rpng_apng), animated WEBP, WEBM
and MP4, frames on the container's clock, no audio. The companions now
do the same, through the shared engine, once for all three:
- companion_thumbs_animate(path, w, h, tag, bg): one animation at a
time (the selected file). Its own thread reads the file, probes it
the way gfx_thumbnail does (a cheap type gate, then rpng_is_apng_ex
over the head of the buffer for PNG - an ordinary PNG costs one
4 KiB probe and nothing else), opens the stream, and loops: next
frame (honouring the loop count, rewinding at the end of a pass),
ARGB from the stream where it can supply it or a swizzle where it
cannot, scaled and letterboxed exactly like a still, pushed as a
delivery for (path, w x h, tag), then sleeps the frame's duration.
A still produces nothing. animate_stop(), a newer animate(), or
free() ends it within one frame duration; frames of a superseded
animation are dropped at poll. Files over 256 MiB are not animated.
Frames land through companion_thumbs_poll like any other delivery,
so a backend's existing "blit these bits into the pane" path draws
them; pending() and animating() keep its polling alive.
- Win32 / Cocoa: the boxart pane starts the animation for the selected
file after showing its still, and stops it when the selection moves.
Qt: the sidebar's four widgets play frames for a browser selection
(frameReady from the model, QT_TAG_ANIM_FRAME deliveries), stopped on
the next selection. Any previous animation stops first.
Regression test: test_animation writes a two-frame APNG by hand (IHDR,
acTL, fcTL, IDAT, fdAT - zlib + CRC), checks it decodes as a still
(frame 0, red), then that animate() delivers at least six frames in
four seconds with both colours seen, that animating() reports it, that
stop() ends it, that a still animates nothing, and that free() with an
animation running returns. Plain, TSan and ASan. The script's C89 pass
gains _GNU_SOURCE (as the real C89 gate has) for the POSIX types
libretro-common's headers use, and HAVE_RPNG for the APNG probe.
companion_parity.py requires the animate call on every backend (25
features).
Verified: companion_thumbs_test.sh plain / TSan / ASan and
companion_core_test.sh pass; companion_thumbs.c, ui_win32_companion.c
and the consumers of companion_thumbs.h pass tools/mingw_syntax_check.sh;
full Qt 6 retroarch builds and starts headless; companion_parity.py
green; apple_mrc_ownership_check.py unchanged. Not run on Windows /
macOS.
* ui/drivers: remove stale companion_preview references from the Win32 companion
The animated-previews commit left the Win32 companion with an include
of, and calls into, a companion_preview module from a direction I had
abandoned in the same session (the Qt and Cocoa files had been reset;
this one had not). The header does not exist: the Windows build failed
at the include. All of it is gone; the pane's animation goes through
companion_thumbs_animate like the other two backends.
The syntax gate should have caught it and did not: it forgave every
"No such file" error so that optional platform headers this box lacks
(d3dkmthk.h) do not fail unrelated files - which also forgave a missing
project header. It now forgives only a missing header named without a
path; one with a path component (../companion/x.h) fails the gate.
Probed both ways.
Verified: ui_win32_companion.c compiles to an object with mingw
(-Wall -Wextra, no warnings), and passes the fixed gate along with
companion_thumbs.c, companion_core.c and the 15 consumers of
ui_companion_driver.h; companion_parity.py green.
* gfx: share the File Browser's animated preview with the companions (gfx_anim_preview)
The companions' previews of an animated file were slow to start and
silent, while RetroArch's own File Browser thumbnail plays the same
file at once and with sound. The reason was that the companion engine
had its own playback path: read the whole file, open the stream, no
audio. The menu's path (gfx_thumbnail.c) does it right - a sliding-
window progressive open so frames start from the first resident bytes
and a tail-moov MP4 opens from a few MiB, memory admission that scales
with the heap, the container's audio track through the mixer over its
own window - so that is what everyone uses now.
gfx/gfx_anim_preview.{c,h}: the session that machinery becomes, moved
out of gfx_thumbnail.c with its reasoning comments - open (type gate,
APNG head probe, windowed progressive open with the moov-island jump
and the media-floor prime, admission), feed (keep the window
straddling the decoder's frontier under the per-tick I/O budget and
raise the demuxer's bound to what is resident), next / rewind with
the container's duration rules, and the preview audio (the second
window, moov head sizing, mixer stream, its feeder, stop). Nothing in
it knows about a thumbnail, a video driver or a toolkit.
gfx_thumbnail.c keeps its threaded upload path and now drives the
open, both feeders and the audio through a session it wraps around the
stream and mapping it owns (gfx_anim_preview_wrap / _release, non-
owning); ~430 lines of inline copies are gone. Both of its sample
suites pass on the result: gfx_thumbnail_anim (plain, ASan+UBSan,
TSan sweep) and gfx_thumbnail_preview (make check: windowed opens over
sparse tail-moov fixtures, audio-window islands, decode budgets).
The companion engine's animation thread is that session too: open,
feed, next, rewind, close; the UI thread starts the preview audio once
the session is published and feeds its window every poll (mixer calls
stay on the UI thread), stop() silences at once. A video's still is
now its first frame through the same windowed open instead of
image_texture_load reading the whole file - the other half of the
startup latency. So the companions get the menu's startup time and its
sound (honouring menu_thumbnail_preview_audio) from the same code.
One bug found by the engine test while doing this, in the shared code:
image_transfer_anim_stream_set_argb is honoured only before the first
frame, and APNG reports false to a repeat request afterwards although
the order stands. Asking per frame and trusting the answer swapped R
and B from the second frame on (a blue-tinted preview after the first
loop). The session decides the order once at open and remembers it.
The animation test now checks every frame is one of the fixture's two
colours over ten frames, which is what caught it.
Verified: companion_thumbs_test.sh plain / TSan / ASan pass (the module
linked in); companion_core_test.sh passes; both gfx_thumbnail sample
suites pass; gfx_anim_preview.c, gfx_thumbnail.c, companion_thumbs.c,
ui_win32_companion.c and the consumers of gfx_thumbnail.h pass
tools/mingw_syntax_check.sh; full Qt 6 retroarch builds and starts
headless; companion_parity.py green; apple_mrc_ownership_check.py
unchanged. Xcode and MSVC build through griffin, which includes the
new file. Not run on Windows / macOS.
* ui/companion: per-frame preview cost - fold the byte swap into the sampler, 4-tap downscale, no GDI churn
Measured per frame at pane size (320 x 400), -O2:
nearest full-canvas swizzle folded swizzle 4-tap (+swizzle)
3840x2160 source 0.35 ms 11.8 ms 0.22 ms 0.98 ms
1920x1080 0.13 ms 1.5 ms 0.16 ms 0.91 ms
640x480 0.09 ms 0.29 ms 0.12 ms 0.93 ms
So the scale to pane size was never the cost. What was: for a stream
that will not emit ARGB words (animated WEBP), the whole canvas was
byte-swapped into a fresh malloc before sampling - 12 ms a frame at
4K, a third of a 30 fps budget - to feed a sampler that reads a few
hundred thousand of those pixels. companion_thumbs_scale_ex takes the
source order as a flag and swaps the sampled pixels only: 0.2 ms, no
allocation. The animation thread and the video-still path use it.
(Handing the raw canvas to the toolkit and letting it scale would be
worse, not better: every toolkit here would copy the full canvas into
its own image per frame - 33 MB at 4K - and scale on the CPU anyway.)
Quality, the actual weakness: nearest-neighbour downscaling a video to
a pane aliases and shimmers on motion. When shrinking by two or more
in both directions the sampler now averages four taps per output
pixel, placed a third in from each edge of the source cell so they
always fall on rows and columns of different parity (a 1-px
checkerboard, the worst case, averages to grey instead of picking one
colour - the test caught the quarter-point placement getting that
wrong). Under 1 ms at any source size. Enlarging stays nearest.
UI-thread side, per frame: Win32 created and destroyed a DIB section
and re-set the static control's image every frame; it now keeps one
DIB for the pane, copies the frame into its pixels (after GdiFlush)
and invalidates. Cocoa allocated an NSBitmapImageRep and an NSImage
per frame; it keeps one rep and writes into its bitmapData. Qt did
img.copy() before QPixmap::fromImage(), which copies again; the copy
is gone.
test_scaler covers the R,G,B,A-order path (red in, ARGB red out; the
same words read as ARGB are blue), the 4-tap average on a checkerboard
and nearest on enlargement.
Verified: companion_thumbs_test.sh plain / TSan / ASan pass;
companion_thumbs.c and ui_win32_companion.c pass
tools/mingw_syntax_check.sh (the Win32 TU also compiled to an object
with mingw, no warnings); consumers of companion_thumbs.h pass; full
Qt 6 retroarch builds and starts headless; companion_parity.py green;
apple_mrc_ownership_check.py unchanged. Not run on Windows / macOS.
* ui/drivers: Cocoa - folders pane under the File Browser tab, grid layout on attach, focus back on close
Three reports from running the Cocoa companion:
- The File Browser was empty. The playlists table was tab item 0's
view, and tab item 1 (File Browser) had no view at all, so selecting
that tab showed an empty pane and hid the very table that lists the
folders in browse mode. The tab view is now only the strip; the table
sits under it in both modes and layoutViews places it.
- The grid view showed nothing. RACompanionGrid computed its column
count from its own bounds, which are zero until its first relayout
inside the scroll view, and it only relaid out when explicitly told
to; a zero-width layout stayed. It now takes the width from the clip
view when it has one and relays out on viewDidMoveToSuperview, so
becoming the document view lays it out on its own.
- Closing the companion (or loading content from it) left the keyboard
with no window: RetroArch's window is now made key when the companion
hides, on every path that hides it (close button, entry run, load
with the popup's core, load through the core picker).
Verified: static checks (bracket balance by character scan, every
selector sent to self declared and implemented,
apple_mrc_ownership_check.py unchanged); companion_parity.py green.
Not compiled here (no Apple toolchain): the Xcode CI build is the gate.
* ui/drivers: Cocoa - keyboard back to RetroArch's real window; grid paints visible rows; diagnostics
Focus. Closing the companion still left RetroArch without keyboard
control. hideAndFocusRetroArch made key the window of [CocoaView get]
- but on a Metal build the render view is not a CocoaView and that
object sits in no window, so the call went to nil. It now asks the
platform: the window hosting the render view (RetroArch_OSX
-hostWindow), activates the app, makes that window key and makes the
render view its first responder - without which key events reach the
window and nothing in it.
Grid. drawRect walked every entry of the list on every paint (42k
rectForRow calls per repaint); it now paints only the rows the dirty
rect covers. And, since the grid still shows nothing on a real Mac and
I have no Mac, it logs its first paint - count, frame, dirty rect,
thumbnail edge, columns, owner and wimp pointers - so a --verbose run
says which of those is wrong.
Folder pane. Names are missing while icons show; the data-source path
reads correctly, so it logs the first name it hands the table and
whether it is valid UTF-8 (a non-UTF-8 name makes stringWithUTF8String
return nil, i.e. an empty cell).
Verified: static checks (bracket balance, MRC lint unchanged);
companion_parity.py green. Not compiled here: the Xcode CI build is
the gate, and the two log lines are what I need from the next run.
* ui/companion: Cocoa regression harness on Linux (GNUstep AppKit, headless); focus hand-back after AppKit's close
Until now the Cocoa companion had never been compiled, let alone run,
anywhere I could observe; every change to it was made blind and judged
by static checks. GNUstep provides a real Foundation + AppKit on
Linux, so it can be:
tools/companion_cocoa_test.sh compiles the real
ui/drivers/ui_cocoa_companion.m against GNUstep's AppKit (Apple-only
headers - AvailabilityMacros.h, TargetConditionals.h, QuartzCore.h -
shimmed in GNUstep's own version scale), links it with the real
companion core, the core test's stubs and fixtures and a stub platform
(a window with a render view that takes first responder, as CocoaView
does), and runs it under Xvfb. companion_cocoa_test.m drives the
driver the way RetroArch and a user do - init, iterate until the
playlist lands, select Icons in the view popup, click the File Browser
tab, close the window through the delegate and through AppKit's own
performClose - and asserts on the AppKit objects: the grid is the
scroll view's document with a frame and the playlist's count and
paints; the folder table stays visible under the File Browser tab and
hands back names; the content table has the listing's rows; after
closing, RetroArch's window is key and main with the render view first
responder. Wired into CI beside the other two suites (apt-get in the
job).
What the harness found while being brought up: the core-test stub's
msg_hash returned one placeholder for every message, and a popup drops
a second item with the same title - the stub now returns a distinct
string per message; the driver's 10.4 open-panel branch calls
objc_msgSend, which the GNU runtime does not have (guarded, since that
runtime never lacks -URL).
What it did not reproduce, stated plainly: the grid and the focus
hand-back both pass under GNUstep on the previous driver too, so the
empty grid and the lost keyboard seen on a real Mac are AppKit-on-Mac
behaviour this harness does not model. For focus, the difference most
likely to matter is that AppKit re-arbitrates the key window as part of
closing the one that was key, after windowShouldClose: has returned -
so the hand-back (key, main, first responder on the render view) is
now repeated from windowWillClose: and once more on the next run-loop
pass, to win over that arbitration. The grid's first-paint log from
the previous patch remains the way to pin the grid.
Verified: companion_cocoa_test.sh passes; companion_core_test.sh
passes; companion_parity.py green; apple_mrc_ownership_check.py
unchanged. The Xcode CI build remains the Mac compile gate.
* tools: companion gap audit vs Qt; Cocoa harness action audit mode
Two audits, to answer "what does the Cocoa (and Win32) companion still
lack against Qt's":
- COMPANION_AUDIT=1 tools/companion_cocoa_test.sh: after the regression
checks, the harness fires every user action the Cocoa controller
implements (view switches, zoom, thumbnail and boxart types, the
three dock toggles, search focus / edit / clear, the browser's tab
and three buttons, folder double-click, core popup, picker Load /
Cancel) on the real AppKit objects under GNUstep and reports each as
implemented / missing and whether it ran. Actions that launch
content, open modal panels or delete are listed but not fired.
Result on this tip: every action implemented, none faults.
- tools/companion_gap_audit.py: a feature-by-feature table of what
Qt's window offers against evidence in the Cocoa and Win32 sources,
and the resulting gap lists.
Verified: companion_cocoa_test.sh passes (with and without the audit).
* ui/drivers: Qt parity wiring - Stop, Unload Core, Exit, About, Contributors, Load Custom Core, search Enter
From the gap audit, the items that are pure wiring of native widgets
to core calls Qt already uses:
Cocoa: a Stop button beside Run (companion_core_unload_core, like
Qt's), File > Unload Core and File > Exit RetroArch (CMD_EVENT_QUIT),
Help > About (version alert) and Help > About Contributors (the
AUTHORS list in a scrolling text window).
Win32: the same Stop button and Unload Core item; Help > About
Contributors (the list in a read-only edit); File > Load Custom Core
(a file picker for a core library, into companion_core_load_core);
and Enter in the search box runs the focused entry as Qt's
onSearchEnterPressed does (the EDIT is subclassed for the key).
AUTHORS.h is a C++11 raw string, which Qt can read and C / Objective-C
cannot: tools/gen_authors_c.py writes AUTHORS_c.h, the same list as a
plain C literal, and CI checks the two agree.
companion_parity.py now requires unload core, exit and the
contributors list on every backend (28 features). The gap audit's
Win32 patterns are corrected (Load Core, Exit and About were already
there; they were regex misses) - Cocoa's remaining gaps are 10 and
Win32's 8, all dialogs, drag-and-drop, rename, progress and theme.
The Cocoa harness checks the new wiring on a real AppKit: Stop reaches
the core (command_event observed by the stub), the contributors
window opens with the text, and the remaining selectors exist; its
audit mode lists them.
Verified: companion_cocoa_test.sh passes; ui_win32_companion.c
compiles to an object with mingw (no warnings) and passes
tools/mingw_syntax_check.sh; full Qt 6 retroarch builds;
companion_parity.py green; apple_mrc_ownership_check.py unchanged.
Not run on Windows / macOS.
* ui/companion: playlist rename, add files / file drop, thumbnail drop - in the core, on every backend
Three more of Qt's features, implemented once in the C89 core and bound
on both natives (Qt's rename now goes through the core too):
- companion_core_playlist_rename(): Qt's rules - only inside the
playlists directory (history / favorites refused), no path
separators, never over an existing name - then the move and a list
refresh.
- companion_core_playlist_add_files(): Qt's drop / "Add Files" for
paths - a file is pushed with label = name without extension and
db_name = the playlist, a directory is walked recursively, DETECT or
a given core; All Playlists refused; the shown playlist reloads.
- companion_core_thumbnail_install(): Qt's changeThumbnail for a
dropped image file - decoded, downscaled to
desktop_menu_thumbnail_max_size, written as PNG at the repository
path with the directory created.
Win32: Rename (in-place label edit on the playlist list, applied on
LVN_ENDLABELEDIT through the core), Add Files (multi-select picker),
WM_DROPFILES - an image over the boxart pane installs the thumbnail
of the pane's type, anything else goes into the selected playlist.
Cocoa: Rename (a sheet with a field), Add Files (open panel), file
drops on the entries table (data-source validate / accept) and an
image drop on the boxart pane (RACompanionBoxart). Both forget the
changed thumbnail in the engine and show it again.
Qt: renamePlaylistItem is the core's rename (72 lines -> 30); its Add
Files dialog (extension filter, names, core choice, progress) and its
QImage-based thumbnail drop stay, as waived in companion_parity.py.
Tests: companion_core_test covers the three (refusals, the move, the
list refresh; a file plus a walked directory with the label / db_name
/ DETECT of each entry; a PNG at the repository path downscaled to the
setting), plain and ASan. The Cocoa harness drives the controller: a
dropped file and directory add two entries, a dropped image installs
at the repository path, a rename shows in the reloaded table.
companion_parity.py: 31 features. The gap audit drops "status bar
progress": Qt's QProgressBar there is created, hidden and never driven.
Remaining gaps: the Core Options, Shader Params and Options dialogs,
and the dark theme on Cocoa.
Verified: companion_core_test.sh plain / ASan pass;
companion_cocoa_test.sh passes; ui_win32_companion.c compiles to an
object with mingw (no warnings) and passes tools/mingw_syntax_check.sh
with companion_core.c and the consumers of companion_core.h; full Qt 6
retroarch builds and starts headless; companion_parity.py green;
apple_mrc_ownership_check.py: one new local autoreleased object (the
rename field). Not run on Windows / macOS.
* ui/companion: Core Options, Shader Parameters and Options dialogs on both natives; Cocoa dark theme
The last of Qt's features the natives lacked, each as a flat table the
core exposes toolkit-free and a table window per backend:
- Core Options (companion_core_option_*): the running core's options
with description, info, value labels, current value; set, reset,
reset all - over runloop's core_option_manager, written when the
core flushes as the menu's are. Win32 / Cocoa: option | value,
double-click cycles, Reset / Reset All.
- Shader Parameters (companion_core_shader_param_*): the menu shader's
parameters with description (id when there is none), range, step,
current; set (clamped), reset to initial, apply
(CMD_EVENT_SHADERS_APPLY_CHANGES). Win32: parameter | value | range
with a field and Apply / Reset; Cocoa: the same with the Value column
editable in place.
- Options (companion_core_setting_*): the desktop-menu settings Qt's
View > Options edits, as typed rows (bool / uint / string / choice)
read and set from text with type checks. Win32: setting | value,
double-click toggles a bool or cycles a choice, a field and Apply
for the rest; Cocoa: editable Value column, double-click likewise.
- Cocoa dark theme: the Theme setting is honoured on 10.14+ by giving
the companion window the dark appearance (system default follows the
system).
Tests: companion_core_test builds a real core_option_manager from v2
definitions and a stub menu shader and checks every accessor, set /
reset / clamp / out-of-range, apply, and the settings table's read-back
and type refusals - plain and ASan. The Cocoa harness opens all three
windows on a real AppKit and drives them: rows and values, cycling and
reset, editing a shader value in the cell and applying, toggling a bool
and editing a number and the theme in the Options table.
companion_parity.py: 34 features (Qt's own three dialogs, which predate
the core, are waived with reasons). tools/companion_gap_audit.py now
reports zero gaps vs Qt for both Cocoa and Win32.
Verified: companion_core_test.sh plain / ASan pass;
companion_cocoa_test.sh passes; ui_win32_companion.c compiles to an
object with mingw (no warnings) and passes tools/mingw_syntax_check.sh
with companion_core.c and the consumers of companion_core.h; full Qt 6
retroarch builds and starts headless; companion_parity.py green;
apple_mrc_ownership_check.py: one more local autoreleased object. Not
run on Windows / macOS.
* ui/companion: Win32 regression harness on Linux (mingw + Wine, headless)
The Win32 companion gets the same net the Cocoa one has: the real
ui/drivers/ui_win32_companion.c, built with mingw and linked with the
real companion core, the core test's stubs and fixtures and a stub for
RetroArch's main window and modal helpers, runs under Wine on Xvfb
(tools/companion_win32_test.sh). Wine's comctl32 is not Microsoft's,
but the object graph, the messages and their side effects are the same
- which is where every Win32 bug so far has been.
companion_win32_test.c drives the driver as RetroArch and a user do -
init, iterate until the playlist lands, then the messages the widgets
send (the View combo's CBN_SELCHANGE and the menu's WM_COMMAND ids, the
tab's TCN_SELCHANGE, the header's LVN_COLUMNCLICK, list selections,
WM_CLOSE) - and checks the real controls: the playlist list's rows and
text; the entries list's count, its view style after switching to
Icons and back; the search filter and Clear; the File Browser - the
async listing landing, the folder pane's rows and "..", the content
list's rows and its four columns and the Type text, the empty boxart
pane, a header click sorting by Type and the same click flipping it,
Up and Start Directory; Stop reaching the core; the Core Options,
Shader Parameters and Options windows opening with the right rows,
value labels, a double-click cycling an option; the window hiding on
close. 33 checks, all green; with the browse-landed rebuild removed it
reports the four panes that go stale, so it catches the class of bug
it exists for.
The driver's control and menu ids move to ui_win32_companion_ids.h so
the harness sends exactly the commands the widgets send (its first
draft carried a copy of the enum, which had drifted). The core test's
fixtures gain the two Windows spellings they needed (mkdir, utime,
sleeping). Wired into CI beside the other suites.
Verified: companion_win32_test.sh passes; companion_cocoa_test.sh and
companion_core_test.sh pass; ui_win32_companion.c and the consumers of
the new header pass tools/mingw_syntax_check.sh; companion_parity.py
green.
* ui/drivers: fix the Xcode (ARC) build and the overlap-copy finding
Xcode, which builds the Cocoa companion with -fobjc-arc: the grid's
first-paint log cast an object pointer to void* without a bridge
(BRIDGE, as the rest of the file does), and the three table windows
passed the address of an ivar as the makeTableWindow out-parameter,
which ARC treats as an __autoreleasing write-back and refuses for a
non-local; each now goes through a local and assigns the ivar.
tools/overlap_copy_check.py: the Win32 Options list read buf[0] in the
source expression of a strlcpy into buf; the value is now read into a
bool before either literal is copied.
Verified: overlap_copy_check.py clean; companion_cocoa_test.sh and
companion_win32_test.sh pass; ui_win32_companion.c passes
tools/mingw_syntax_check.sh; apple_mrc_ownership_check.py unchanged.
The deprecated NSTitledWindowMask names in the log are the pre-10.12
spellings the file uses throughout for the 10.4 SDK; warnings only.
* ui/drivers: the Mac main loop never iterated the companion - "Loading playlist..." for ever
The screenshot says it: "Loading playlist..." in the status bar, the
entries pane empty in both views, the File Browser empty. Every one of
those is an asynchronous result that lands through
ui_companion_driver_wimp_iterate() - the playlist parse, the browser's
listing, decoded thumbnails, animation frames, status - and on a real
Mac that hook was never called.
The only call was in the Qt-era -rarch_main loop in ui_cocoa.m, which
is under #ifdef HAVE_QT. The Xcode build has no Qt: its whole main loop
is the CFRunLoop observer rarch_draw_observer in cocoa_common.m, which
called runloop_iterate() and task_queue_check() and nothing else. So
the Cocoa companion built its window, kicked off its work, and nothing
ever arrived. The observer now calls the hook each frame, as the
generic loop in retroarch.c does for the other platforms.
Why the harness did not catch it: it calls the driver's iterate itself,
standing in for the platform loop. companion_parity.py now also checks
that the platforms' real main loops call ui_companion_driver_wimp_
iterate() (rarch_draw_observer today; the generic loop covers Win32 and
Qt) - it fails on the previous cocoa_common.m and passes on this one.
This is also the most likely cause of the earlier "grid shows nothing"
and "browser is empty" reports, which the GNUstep harness could never
reproduce: it iterates.
Verified: companion_parity.py green (with the new check);
cocoa_common.m compiles under GNUstep's syntax check. Not run on macOS:
the Xcode CI build and your next run are the gate.
* ui/drivers: Cocoa - hand the keyboard back in every run-loop mode, and when the window resigns key
After closing the companion the mouse controls RetroArch again but the
keyboard does not. On macOS the mouse goes to the window under it,
while key events go to the key window through RAWindow -sendEvent:, so
this is the picture of RetroArch's window being main but not key after
the close.
Two things about the previous hand-back could let that happen. Its
deferred re-assert was a default-mode timer, and the close button's
click is dispatched in the event-tracking mode, so the re-assert could
wait until the mode changed; and it ran only from the delegate's close
methods, whereas AppKit's own choice of the next key window - made
while the close is in progress - is what has to be beaten.
Now: the re-assert is scheduled in the default, event-tracking and
modal-panel modes, three times over 40 ms (the arbitration can run
late); it also runs when the companion window resigns key while not
visible (the moment the close takes the key status away); and it does
nothing if the window has been shown again meanwhile. Each attempt
logs whether the host ended up key and main and what NSApp's key
window is, so a --verbose run says exactly where it lands if this is
still not it.
Verified: GNUstep compile; companion_cocoa_test.sh passes;
companion_parity.py green; MRC lint unchanged. Not run on macOS.
* ui/drivers: Cocoa - let the companion window actually close; the keyboard comes back
The keyboard stayed with a window that no longer existed on screen.
windowShouldClose: hid the companion itself and returned NO - so AppKit
was told the window was not closing. It stayed open in AppKit's eyes,
hidden but still the key window, and key events kept going to it while
mouse events went to the window under the pointer: exactly "mouse works,
keyboard does not". And since no close ever happened, windowWillClose:
and windowDidResignKey: - where every hand-back attempt of the last
three patches lived - never fired.
windowShouldClose: now returns YES. AppKit closes the window (kept
alive by setReleasedWhenClosed:NO; the driver's toggle orders it back),
resigns its key status, and the hand-back runs from the delegate
methods that now fire. The programmatic hide on loading content keeps
its own hand-back.
The harness asserts the delegate returns YES and closes the window
through AppKit, then shows it again through the driver's toggle and
checks it still lists the playlists.
Verified: companion_cocoa_test.sh passes; MRC lint unchanged. Not run
on macOS.
* ui/companion: run the Cocoa harness on Apple's AppKit in CI; assert a keystroke after the close
The keyboard still does not return to RetroArch after the companion
closes on a real Mac, and the GNUstep harness keeps passing: whatever
this is, it is in Apple's AppKit - key-window arbitration, activation -
which GNUstep only approximates. There is no Mac here to debug on; the
macOS CI runner has a window server and can run AppKit, so the harness
runs there now, built with Apple's clang against the real Cocoa
framework (tools/companion_cocoa_test_macos.sh, no shims), as a step of
the macOS workflow ahead of the Xcode build.
The harness also asks the actual question now. RetroArch reads the
keyboard in RAWindow -sendEvent:, which AppKit calls on the key window
for key events; the harness's stand-in for RetroArch's window counts
the key events reaching its -sendEvent:. After closing the companion
through AppKit (performClose:), it posts a keystroke through NSApp and
asserts it arrived - and prints which window NSApp holds as key if not.
The harness also becomes a regular, activatable app first, since
Apple's AppKit gives no key window to a background process.
Under GNUstep the new checks pass (NSApp's key window is RetroArch's;
the posted keystroke reaches -sendEvent:). The macOS job's log is the
answer for Apple's AppKit.
* ui/companion: Cocoa harness - report faults and exceptions with a backtrace
The first run on Apple's AppKit got through init, the playlist, the
grid (a real frame, painted), the File Browser, Stop, Contributors and
the menu items, then died with "Trace/BPT trap: 5" right after
"addPaths: implemented" - a SIGTRAP with no message. The harness now
installs handlers for SIGTRAP / SIGSEGV / SIGBUS / SIGABRT / SIGILL that
print a backtrace, and an uncaught-exception handler that prints the
exception's name, reason and call stack, so the next log names the
place. Methods returning size_t / BOOL are invoked through NSInvocation
rather than performSelector: (which is for id returns and undefined for
these on Apple's runtime), in case that was the trap.
Under GNUstep the harness still passes.
* ui/companion: overlapping strlcpy on a playlist reload (trapped on macOS); test builds trap it too
The harness on Apple's AppKit got through everything up to adding files
to the shown playlist, then trapped in __strlcpy_chk -> __chk_fail_
overlap inside companion_core_begin_playlist. The path: a backend adds
files to the playlist it is showing and hands the core the pointer it
got from companion_core_selected_playlist_path() - the core's own
selected_path buffer. add_files reloads that playlist through
select_playlist_path -> begin_playlist, which strlcpy'd the path into
selected_path: source and destination the same buffer. Undefined, and
macOS's fortified libc aborts on it; Linux's strlcpy (libretro-common's
own) silently did the copy, which is why every Linux suite passed.
begin_playlist now takes a copy of @path before it clears state or
writes selected_path.
So the Linux suites catch this class from now on: under
LIBRETRO_STRL_CHECK_OVERLAP, libretro-common's strlcpy aborts on an
overlapping copy exactly as Apple's does, and the four companion test
scripts build with it. companion_core_test adds the aliasing call the
backends make (add files through the selected-path pointer, reload,
count) - on the previous core it aborts with "strlcpy: overlapping copy";
on this one it passes. All four suites pass with the check on.
Verified: companion_core_test.sh plain / ASan, companion_thumbs_test.sh,
companion_cocoa_test.sh, companion_win32_test.sh pass;
companion_core.c and compat_strl.c pass tools/mingw_syntax_check.sh
and build; overlap_copy_check.py clean.
* ui/drivers: Cocoa - forget held keys when another window takes the keyboard (the dead keyboard)
The harness on Apple's AppKit settled the key-window question: after
the companion closes, RetroArch's window is key and main and a posted
keystroke reaches RAWindow -sendEvent:. So the keyboard was arriving,
and RetroArch was ignoring it.
The mechanism is RetroArch's own held-key state. The companion is
opened with a key (its hotkey, or Enter in the menu). The moment its
window becomes key, that key is still down; its release is delivered to
the companion window, so RetroArch's window never sees the key-up and
apple_key_state keeps the key "held". The menu's flush-and-wait-for-
release (menu_st->input_driver_flushing_input ->
INP_FLAG_WAIT_INPUT_RELEASE) then waits for that release - for ever.
Keyboard dead, mouse (which does not go through that gate) fine, and
closing the companion changes nothing because the key is still "down".
Deactivating the app clears it, which is why RetroArch already calls
apple_input_keyboard_reset() from applicationWillResignActive.
Same treatment for losing the keyboard to another window of our own:
RetroArch_OSX (RAWindow's delegate) resets the keyboard state in
windowDidResignKey:, and the companion's toggle resets it right after
taking the keyboard, covering a show from a non-key state. The harness
stubs apple_input_keyboard_reset and asserts the companion's show calls
it.
Verified: companion_cocoa_test.sh passes under GNUstep; MRC lint
unchanged. Apple's AppKit run of the harness (previous commit) is what
established the key-window half. Not run in the real app on macOS.
* ui/companion: keep the playlist config off the stack; guard the selected_path self-copy by pointer
companion_core_begin_playlist() no longer takes a PATH_MAX_LENGTH copy
of @path to survive being handed core->selected_path: the copy is
skipped when the pointer already is that buffer, the way path_set()
does it, and the buffer already holds the path. The playlist_config_t
that begin_playlist, all_next, playlist_open and playlist_open_private
each built on the stack now lives in companion_core_t; playlist_init()
and playlist_parse_begin() copy it, and all four run on the UI thread,
so one is enough. begin_playlist drops from 5184 to 32 bytes of frame
and the other three from ~3 KiB to under 64 bytes, within the 4 KiB
ceiling CODING-GUIDELINES sets.
* ui/drivers: Cocoa - the icon grid's prefetch read past the thumbnail marks (the crash)
Guard Malloc named it exactly: a byte read one past the end of a
MALLOC_TINY block in -[RACompanionController thumbWant:urgent:]. (The
earlier crash in MoltenVK was the same corruption, noticed later by an
unrelated allocation - the usual way an overrun surfaces.)
iconTick prefetches a screen either side of the visible rows, so it
calls thumbWant: with rows that do not exist. hasImageForRow: answers
NO for those - it means "no image", not "no row" - so thumbWant: went
on to read thumbNone[row], and thumbNone is only as long as the list.
Reading one byte past a heap block: harmless on glibc, fatal under
Apple's allocator.
thumbNone now carries its length (thumbNoneCount) and every access is
bounded by it: thumbWant: returns for a row outside the list, the idle
re-scan clamps, the delivery callback checks the array's length as well
as the current entry count (a delivery for the previous list), and the
two memsets use the array's own length rather than a count that may
have moved on since it was allocated.
Regression test, in the harness: drive iconTick repeatedly and call
thumbWant: with rows either side of the range, and run the whole
harness under ASan (tools/companion_cocoa_test.sh asan, wired into CI
beside the plain run). On the previous driver ASan reports
heap-buffer-overflow at ui_cocoa_companion.m:976 in thumbWant: and the
script exits 5; on this one both passes are clean. The script also
propagates the run's exit status now - the pipe through the log filter
was swallowing it, so a sanitizer abort would have looked like a pass.
Verified: companion_cocoa_test.sh plain and ASan pass (and fail as
described on the previous driver); companion_core_test.sh,
companion_thumbs_test.sh, companion_win32_test.sh pass;
companion_parity.py green; MRC lint unchanged.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Apple's fortified strlcpy() aborts when the source and destination
overlap, so an in-place copy is a crash on macOS rather than a no-op.
The $CONTENT-DIR$ and $PRESET-DIR$ wildcard expansions copied the
directory name onto itself through path_basename_nocompression(),
which returns its input unchanged now that fill_pathname_parent_dir_name()
produces a bare name; the copy is dropped. XMB's content-list entry
stripping now sources fill_pathname() from entry.path instead of
shifting the scratch copy in place.
A device within a crystal's tolerance against a source well outside it
is the source's clock - with audio sync off, the frame timer or the
display - which rate control absorbs and a bias should not; the other
way round is a driver not counting device time; both off is the two
counts not measuring the same thing. The warning names the case, and
the driver, not the thread wrapper.
filestream_rename() now replaces an existing destination on every
target, keeping the file being replaced on disk until its replacement
is in place, so the move-aside helpers each of these carried on top of
it duplicate what the VFS already guarantees. Drop them and rename
the completed temporary over its destination in one call.
MoveFileEx() is unavailable there, so the local-encoding path replaced
a destination by removing it and renaming after, leaving nothing on
disk between the two calls. Move the destination aside instead, rename
into its place, and remove the moved-aside copy only then - or put it
back if the rename fails - so the file being replaced stays on disk for
the whole operation, as on every other target.
The Xbox builds fall outside the Win32 MoveFileEx() path, and the XDK
has no MoveFileEx(), so rename() there is the CRT's and refuses a
destination that already exists; the temporary-then-rename saves -
config file, playlists, core info - fail whenever the target is
already on disk.
Add an Xbox branch on MoveFile(): a destination in the way is moved
aside, the source moved into place, and the moved-aside copy deleted
only then - or put back if the second move fails.
PSL1GHT's rename() is LV2's sys_fs_rename, the same syscall behind
cellFsRename, and it refuses a destination that already exists, so the
temporary-then-rename saves - config file, playlists, core info,
updater downloads - fail whenever the target is already on disk.
Fold PS3 into the console replace path: a destination in the way is
moved aside, the temporary renamed into place, and the moved-aside copy
removed only then - or put back if the rename fails.
libfat's rename refuses a destination that already exists, so the
temporary-then-rename saves - config file, playlists, core info,
updater downloads - fail on SD and USB whenever the target is already
on disk.
Fold GEKKO into the console replace path: a destination in the way is
moved aside, the temporary renamed into place, and the moved-aside copy
removed only then - or put back if the rename fails.
pspsdk's rename() is a bare sceIoRename(), and the kernel refuses a
destination that already exists, so the temporary-then-rename saves -
config file, playlists, core info, updater downloads - fail on ms0:
whenever the target is already on disk.
Fold PSP into the Vita replace path, which drives sceIoRename() itself
and moves a destination in the way aside rather than deleting it.
sd_fat_devoptab's rename is a bare FSRename(), and coreinit defines
FS_STATUS_EXISTS as one of its results, so the temporary-then-rename
saves cannot rely on it replacing a destination that is already there.
Fold WIIU into the console replace path. A destination in the way is
moved aside before rename() is called, so this holds whichever way
FSRename() treats an existing target, at the cost of one stat() per
overwrite.
bdmfs_fatfs, the driver behind mass: and mx4sio:, is FatFs, and its
f_rename() refuses a destination that already exists; fileXioRename
passes that through, so the temporary-then-rename saves - config file,
playlists, core info, updater downloads - fail on USB and SD storage
whenever the target is already on disk.
Fold PS2 into the console replace path: the file in the way is moved
aside, the temporary renamed into place, and the moved-aside copy
removed only then - or put back if the rename fails. PFS on hdd:
already replaces on rename and is unaffected.
3DS FS refuses to rename onto an existing entry, so the
temporary-then-rename saves - config file, playlists, core info,
updater downloads - fail outright on libctru 1.x and 2.0, and on
libctru 2.1+ go through its own delete-then-rename, which leaves
nothing on disk until the second call lands.
Fold 3DS into the Switch replace path and check for a destination
before calling rename() at all, so libctru's delete step is never
reached: the file in the way is moved aside, the temporary renamed
into place, and the moved-aside copy removed only then - or put back
if the rename fails.
sthread_raise_current_priority() sets the time-constraint policy on
Darwin, which is what makes a thread real-time there;
pthread_setschedparam() only moves it within the time-shared band. A
period close to a small audio IO cycle with a fraction of it as the
budget, as the HAL's own IO thread runs. Needs no privilege.
libnx's rename() is a single fsFsRenameFile() call, and FS refuses a
destination that already exists, so the temporary-then-rename saves -
config file, playlists, core info, updater downloads - all failed
whenever the target was already on disk.
Give SWITCH/HAVE_LIBNX the same replace path as Vita: when a plain
rename() is refused and a destination is in the way, move it aside,
rename the temporary into place, then remove the moved-aside copy -
restoring it if the second rename fails. The file being replaced is
on disk for the whole operation.
Fixes#19507
* add initial guidelines for agentic contributions
* Revise AGENTS.md for clarity and additional guidelines
Updated AGENTS.md to include new sections and clarify existing guidelines on project structure, comments, commit messages, menu changes, shader functionality, and scope of changes.
---------
Co-authored-by: hunterk <[email protected]>
write_avail() runs on the frontend's thread and brings the wave buffer
states up to date with audrvUpdate(); the writer, on the audio thread,
adds and starts. Every call into the renderer, and the reads and
writes of the wave buffer states and the buffer being filled, are
under the one lock, so the voice's buffer list is never rewritten
under an add - which lost buffers from the voice with their audio,
left them counted free, and took the backpressure with them.
The driver is blocking until the frontend says otherwise; it sets
non-blocking at init only when audio sync is off.
Fixes#19508.
The HAL holds a whole pull before it feeds any of it on, so the IO
buffer is a stage of its own; a quarter of the ring keeps it small
against the setting, and room comes back to the writer in smaller
pieces. 96 frames at the 8 ms minimum, within the 64 to 512 a device
takes.
The ring's power-of-two capacity is only its container: free space,
buffer_size() and the writer's wait are counted against the setting
itself, so an 8 ms setting reports 8 ms rather than the 10.7 the
container rounds to. The output unit asks the HAL for an IO buffer of
half the ring, within 64 to 512 frames, so a pull always fits what
the writer waits for, and the device stage the overlay reports shrinks
with the setting rather than staying at the HAL's 512-frame default.
macOS only; RemoteIO keeps its own.
With audio sync off, a fifo under one frame of core audio drops the
remainder of each frame only when the writer is the frame-synchronous
one. The threaded pipeline's writer hands over at most half the
buffer a pass and keeps the rest in its ring, so the warning is not
given under it.
A writer that blocks inside retro_run() - audio sync on, the
frame-synchronous pipeline - keeps a floor of a frame plus an engine
period, never under the engine buffer. A writer that cannot stall a
frame - the threaded pipeline, or audio sync off - is floored at the
frame alone: one at 50 fps plus the 2% rate control can stretch it,
plus a frame of slack, never under two periods. On the legacy 10 ms
engine that is 42.5 ms in all at the minimum setting. The pacing
harness restates the rule its writer exercises.
The left action of a float setting now snaps a value within half a
step of an enforced minimum to it, as the right action does at the
maximum. Five 0.001 steps down from 0.005 land at -7e-10, which
prints as -0.000 and is stored as such; they land at 0 now.
A float setting stepped down to its minimum lands a few ulps under it
- five 0.001 steps down from 0.005 give -7e-10 - which prints as
-0.000 and is stored as such. The left and right actions now snap a
value within half a step of an enforced minimum or maximum to that
end exactly, as they already wrap one that is past it.
Elevate Audio Thread Priority defaults to on wherever there is an
audio thread to raise: it is best effort, a system that refuses keeps
the default priority, and the lower latency settings the threaded
pipeline allows depend on it.
Each of these knows what sits behind the ring it hands the frontend,
and now tells the statistics overlay, as ASIO and WASAPI do.
JACK: the playback port's latency range, which the ring size already
comes off. PulseAudio: the sink's own latency from the stream's timing
info, the part of pa_stream_get_latency() that is not the server's
queue; read with the buffer size in write_avail(), since timing data
arrives after connect and the sink can change. PipeWire: pw_time.delay,
the frames between the stream and the device, read the same way once
the graph is running. CoreAudio: the HAL's IO buffer, device latency,
safety offset and the output stream's latency, summed at init; macOS
only, the RemoteIO path on iOS reports none.
Exclusive: buffer_size() is the fifo, and the endpoint buffer behind
it - one period, event-driven - is the device's own, so that period
is the stage. Shared: buffer_size() already spans fifo and engine
buffer, so the stage is the engine's own latency between that buffer
and the endpoint, which GetStreamLatency gives. Either way the
overlay's Buffer line reads ring+device, as it does for ASIO.
The ring in front of the ASIO device is floored against main-thread
scheduling jitter, a span of time; a floor of four periods scales with
the period instead - 5 ms on ASIO4ALL at 64 frames, where it was
tuned, and 44 ms on a wrapper that locks its period at 528, four
times an 8 ms setting on a device whose own stage is 11 ms. The floor
is now four periods where that is under 6 ms, 6 ms where it is over,
and two periods at the least, since the callback takes a whole period
at once and the writer's next burst lands whenever it lands. ASIO4ALL
at 64 frames keeps its 256; a 128-frame period gets 288 for 512; the
528-frame wrapper gets 1056 for 2112. The arithmetic moves to
asio_ring.h so it can be checked off Windows; samples/audio/asio_ring
pins it against those devices and the points where the rules meet,
and CI runs it.
The device stage behind the driver's buffer - what ASIOGetLatencies
reports, 1.5 ms on a USB DAC and 12.8 on ASIO4ALL over HDMI - was in
the log only. Drivers report it through
audio_driver_set_device_latency(); ASIO does, at init and when the
device changes it. The statistics overlay shows it on the Buffer line
as ring+device, so the sum is what leaves RetroArch.
The overlay's Sink and Source lines are one line of ppm against the
output rate, which the SampleRate line above already gives in Hz.
The MMDevice API needs COM initialized on the calling thread, and the
audio driver runs on whichever thread the frontend gives it - with the
threaded audio pipeline that is a fresh worker thread with no
apartment, so the enumerator fails with CO_E_NOTINITIALIZED and the
driver cannot open at startup.
mmdevice_com_init()/mmdevice_com_uninit() own that: wasapi_init and
open_mic bring COM up and free/close_mic release it after the last COM
object, so init and free stay balanced per instance. mmdevice_list_new
and mmdevice_get_samplerate are self-contained, since the menu asks for
the device list before any driver is up. The HRESULT name table now
covers the COM-level errors so the log says which one it was.
The pacing harness counts the references and fails if any are left
after free.
Every triangle-strip pipeline - the alpha-blend, menu display, HDR
and shader-chain pass pipelines - now sets primitiveRestartEnable.
Restart only acts on indexed draws that contain the 0xFFFF index;
the shader chain draws unindexed quads and the shared quad IBO never
holds that index (vulkan_init_quad_ibo now caps its size below it),
so the rendered output is unchanged. Metal has no way to turn restart
off for strip topologies, so MoltenVK reported
VK_ERROR_FEATURE_NOT_PRESENT for every one of these pipelines, once
per pass on every shader preset load.
ARC requires an explicitly declared ivar behind an 'assign' object
property to be __unsafe_unretained; the Xcode build stopped on it.
RARCH_UNSAFE_UNRETAINED in cocoa_defines.h spells the qualifier under
ARC and nothing under MRC, so the declaration - which the fragile
32-bit runtime needs written out - reads the same for both.
-[NSView convertRectToBacking:] is sent as a plain message; on SDKs
that predate it (10.5, 10.6) a category declaration in cocoa_defines.h
supplies the signature, and the runtime check that already guards the
call decides whether it is sent. objc_msgSend_stret does not exist on
arm64, where structs return in registers, so the Apple Silicon build
could not compile the previous spelling.
The two pasteboard types the drop handler reads are spelled out as
the strings AppKit delivers on every release (the values of the 10.0
constants the 10.14 SDK deprecates), so no SDK has an opinion about
them and the -Wdeprecated-declarations pragmas go.
A Linux job builds the Cocoa + OpenGL frontend for 32-bit PowerPC with
a GCC 14.2 powerpc-apple-darwin8 cross compiler against the 10.5 SDK,
once each for the 10.4 and 10.5 deployment targets. It is the
compiler-compatibility leg for the Apple code: no clang, ARC, blocks
or GCD, and nothing newer than Leopard in the SDK, so anything the
Xcode 3 / PowerPC project would reject fails here first. The 10.4
build is also audited: every symbol it imports must be exported by
the 10.4u SDK, which is the condition for launching on Tiger.
A PowerPC build against the 10.5 SDK with a 10.4 deployment target
now imports nothing the 10.4u SDK does not export, so the binary
loads on Tiger. The 10.5 APIs the macOS path uses are reached at run
time: pthread_setname_np in rthreads and CFLocaleCopyPreferredLanguages
in platform_darwin go through dlsym, and RetroArch_OSX writes out its
window accessors so no objc_setProperty / objc_getProperty is
referenced. rcheevos no longer requests _XOPEN_SOURCE 500 on Darwin,
whose headers would otherwise select the 10.5-only UNIX03 pthread
entry points; the recursive mutex attributes it wants are in the
default Darwin environment. qb leaves the 10.5 IOHIDManager joypad
driver out of pre-10.5 targets unless asked, and the make build's
ARCH=ppc branch takes its deployment target from
MACOSX_DEPLOYMENT_TARGET.
Verified with GCC 14.2 for powerpc-apple-darwin8: full builds at 10.4
and 10.5 deployment targets against the 10.5 SDK, and a symbol audit
of the 10.4 binary against the 10.4u SDK stubs.
cocoa_main_thread_sync() carries its job to the main thread with
-performSelectorOnMainThread:withObject:waitUntilDone:modes: and waits
on an rthreads condition, so the macOS glue no longer needs blocks or
Grand Central Dispatch: it builds with GCC as well as clang and runs
on releases before 10.6. The dispatch header is included for iOS/tvOS
only, where the remaining GCD users live.
The GL context sends -convertRectToBacking: through objc_msgSend_stret
so SDKs that predate it still see the NSRect result; the scan-file
action in the open panel is gated on HAVE_LIBRETRODB like its
declaration; the two forward-referenced RetroArch_OSX methods are
declared. The make build only passes -fobjc-arc to clang, and links
IOKit for every macOS build since platform_darwin.m queries the power
sources unconditionally. qb takes --host=*-apple-darwin* as a Darwin
target and leaves the ARC-and-blocks CoreLocation driver out of
pre-10.7 targets unless asked, as it already does for AVFoundation.
Verified with a GCC 14.2 powerpc-apple-darwin8 cross build against the
10.6 SDK: every configured object compiles and links into a ppc
Mach-O executable.
qb, Makefile.common and the Xcode configurations set HAVE_COCOA for
macOS and HAVE_COCOATOUCH for iOS/tvOS; which video drivers are built
is decided by HAVE_METAL, HAVE_VULKAN and HAVE_OPENGL alone. The Xcode
macOS build now carries HAVE_COCOA like the make build, so the desktop
companion settings appear there too. A Darwin Vulkan build links the
Metal frameworks its CAMetalLayer render view needs, whether or not the
Metal driver is included.
The remember-window-position setting and the SDL exclusion on macOS
key off HAVE_COCOA, the one define every macOS build carries; iOS and
tvOS are told apart by HAVE_COCOATOUCH as before. The settings
migration tool no longer pairs HAVE_COCOATOUCH with a second define.
The ApplePlatform protocol - a render view the application delegate
owns and swaps per video driver through -setViewType:, and window and
full-screen handling behind -setVideoMode: - is now the only host
architecture on macOS, iOS and tvOS. The GL and Vulkan context drivers
and the Metal driver all reach the application the same way, and every
context driver implements set_resize.
What the system in front of the binary can do is asked at runtime, not
decided by the build SDK: native full-screen (10.7) is used where the
window responds to -toggleFullScreen:, and otherwise the delegate hosts
the render view in a borderless RAWindow above the menu bar, so key and
mouse events keep flowing through -[RAWindow sendEvent:]. The alert
sheet uses the block completion form where the system has it (10.9)
and the delegate form elsewhere. Drag types are the classic names the
drop handler already reads. Everything AppKit-facing is written in the
spelling GCC 4.0 on the 10.5 SDK accepts (bracket syntax, alloc/init
arrays, RARCH_* ownership macros, delegate protocols adopted only on
SDKs that declare them) and compiles unchanged under ARC.
Metal and Vulkan view types are compiled in by HAVE_METAL and
HAVE_VULKAN respectively; nothing about the view architecture depends
on which renderers are present. On iOS the render view is pinned with
layout anchors where the view has them (iOS 9) and with an autoresizing
mask otherwise, and -setVideoMode: only sets a drawable size on an
MTKView. gl2's second macOS set_video_mode call goes with the view path
that needed it.
RARCH_NSWINDOWSTYLEMASK_FULLSCREEN spells out the native full-screen
style bit so callers can test -[NSWindow styleMask] on any SDK;
RARCH_PROTO_NSAPPLICATIONDELEGATE / RARCH_PROTO_NSWINDOWDELEGATE
adopt the formal delegate protocols only on SDKs that declare them;
RARCH_AUTORELEASEPOOL_BEGIN / _END open an autorelease pool scope
in whichever spelling the current memory model accepts. These let
one AppKit code path build against the 10.5 SDK with GCC 4.0 and
against a current SDK under ARC alike.
The pause-time flush writes the config while a loaded core may still
have video_driver overwritten with the renderer it forced, with the
user's choice parked in cached_driver_id. Restore the cached name for
the duration of the save, as main_exit() does, and put the live value
back afterwards so a resumed activity keeps running on the renderer it
was using.
c6b66c9 widened green in the RGB565 converters from 5 bits to 6, which
was correct, but left the packing shift at 6. A 6-bit green belongs at
bit 5; at bit 6 it spans bits 6..11, and bit 11 is the LSB of red.
Green's top bit therefore landed on red, and the green field became
(g & 31) << 1 - so green wrapped hard at half scale. Mid grey rendered
as (139, 0, 131) instead of (131, 129, 131), and orange as pure red.
Half of all green inputs also nudged red up a step.
Affects sdl_dingux, sdl_rs90 and xvideo, in both the plain and the
dithered path, so menu chrome, wallpaper and thumbnails alike. Every
other converter is unaffected: their channel widths are uniform, so
nothing outgrew its shift.
Nothing in CI could have caught this. The formats are per-platform and
no runner boots a 565 menu, so the bug was invisible to every build
that does run, and it is not a build break either - a width that
outgrows its shift just ORs into the neighbouring field, silently. It
took a hand bisect on an Anbernic RG300X to find.
So add tools/rgui_pixfmt_check.py, wired up as its own workflow. It
lifts the marked pixel format conversion block out of rgui.c and
compiles it standalone against two stubs, so it exercises the real
shipped converters and the real dispatch table rather than a copy that
can drift. Per driver ident, and for the unknown-ident fallback, it
measures which output bits each input channel can change and requires
them to be disjoint and contiguous, the endpoints exact, the field
monotonic in the input, and the dithered twin to cover the same bits
within one level of the plain converter. Field layouts are measured
from the converters, not hardcoded, so a new format is covered as soon
as its row joins rgui_pixel_format_map.
--selftest checks the checker: the corrected green passes, this bug is
caught as a red/green overlap, and a row wired to another format's
dithered twin is caught as a reach mismatch. Both green fixtures are
built by forcing the packing shift either way, so the selftest reports
the same thing regardless of the state of the tree.
Both steps together run in under half a second.
Fixes#19469.
* gfx: video modeline engine in gfx/modeline/, with goldens against SwitchRes 2.2.1
gfx/modeline/ is a C89 modeline generator: given a source width, height,
field rate and rotation plus a description of what the display syncs to,
it produces the full timing (pixel clock, the eight sync counts,
polarities, interlace/doublescan) as a video_modeline_t POD, together
with the scale factors and compromise bits a consumer needs for aspect.
modeline_core.c holds create/compare/adjust, VESA GTF and user-modeline
parsing; modeline_monitor.c the preset table (15 kHz consumer and
arcade sets, multi-sync chassis, PC CRTs, LCD, VESA bands, custom
ranges) with range validation; modeline_list.c the fixed 256-entry mode
list, option setter, filtering, LCD auto specs and the get/flush/
restore/set cycle against an abstract backend ops table; modeline_ini.c
the "key value" ini loader with the working-directory search order.
Policy inputs (super width, dotclock minimum, geometry sliders, aspect,
pixel precision) live on video_modeline_gen_t, not inside the mode.
Backend capabilities are cached on the generator at list init so option
reloads keep the desktop's editability. When the list is full the oldest
generated mode that is not on the wire is dropped so long sessions keep
switching.
tools/modeline_gold.c drives a fixed matrix of presets, options and
sources through the engine and, built with -DMODELINE_GOLD_REFERENCE as
C++, through the 2.2.1 library it replaces; tools/modeline_gold.ref is
the reference output. The arithmetic is kept bit-for-bit, including the
integer line-rate division in user modelines and the float intermediate
in the scoring tie-break, and the 1680-line matrix matches across the
dummy, update-only and add-capable display shapes.
* video: apply modelines through the display server and retire deps/switchres
The display server gains modeline_* ops: list_outputs, open, close,
caps, enum, add, update, delete, set and flush, all taking the full
video_modeline_t. KMS generates freely and hands the timing to the DRM
context through the CRT consumer's drmModeModeInfo mirror with a plain
mode set, never a driver REINIT. X11 applies through XRandR on the shared
Display: add/update/delete/set with a saved crtc layout for restore,
plus the screen_compositing and screen_reordering options, and the old
vmax-table heuristic generator is gone. Win32 picks PowerStrip when asked
and otherwise ATI legacy or ADL from the PCI id; the vendor paths live in
win32_modeline_{adl,ati,pstrip,resync}.c, ADL stages overrides per mode
and does its list refresh and monitor resync once at flush, the resync
window runs on an rthreads thread behind HAVE_THREADS with a bounded
condition-variable wait, and CDS switches only between listed modes.
Servers without a path leave the ops NULL and the consumer generates
only. DISPSERV_CTX_MODELINE replaces the CRT-named bit with an alias.
gfx/video_crt_switch.c stays the CRT policy consumer on top of the
engine: menu mode to preset, super width, switchres.ini and the
core/directory/game overlays (reloading the base set in init order when
overrides are dropped), the Windows dummy geometry set, and after set the
double field rate is published while the scanline calibration and auto
frame-delay target are reset for the new vtotal. HAVE_MODELINE is the
build switch, with HAVE_CRTSWITCHRES as a one-cycle alias in qb, the
Makefile and video_defines.h; the engine is C89 so the switch no longer
needs C++11 and -lstdc++ leaves the link line. deps/switchres is removed.
* video: the null display server carries the modeline ops; CI runs the modeline goldens and C89 lane
dispserv_null in video_driver.c is the fallback vtable and needs the
same nine modeline_* slots as the real servers, otherwise its ident
string lands in modeline_list_outputs. The gfx samples workflow now
builds tools/modeline_gold.c under the C89 lane flags and under
ASan/UBSan and diffs both against tools/modeline_gold.ref, then
syntax-checks the engine, the CRT consumer and the X11 apply path
under the same dialect rules, so scoring drift or a mixed declaration
fails on the pull request rather than on a CRT.
* video: SDL display server for listed-mode switching, selected by a Video setting
gfx/display_servers/dispserv_sdl.c is a thin layer over the SDL2 window:
it lists the SDL displays and their modes, switches the window to a
listed WxH@R with SDL_SetWindowDisplayMode in exclusive fullscreen, and
puts the desktop mode back on close. Its modeline caps are 0, so the
engine can only pick a listed mode through it; SDL cannot create a
custom timing, and this path is never the custom-timing backend.
It never replaces the native server wholesale. video_driver.c keeps the
native server for metrics, orientation and decorations and routes only
the mode-switching calls (resolution list and set, refresh rate, output
size, flags and the modeline ops) through video_display_server_modes(),
which picks SDL according to the new video_sdl_display_server setting:
Off never uses it, Auto (default) uses it only when the native server
cannot switch modes at all - Wayland, or an SDL window on a platform
without a native server - and Always overrides the native server, which
is the choice for a listed-mode switch on X11 or Win32 at the cost of
custom CRT timings. The setting sits in Settings > Video under the CRT
entry and is listed only while an SDL2 video driver or context owns the
window (video_display_server_sdl_available()). It is a single-source
descriptor row (settings_def_video_sdl_display_server.h) wired through
msg_hash, configuration and the video settings group; the settings table
reference gains its three polarity rows.
* ci: the modeline golden step builds and runs in a scratch directory
The four files the previous commit left at the repository root were
the golden binaries and their outputs from running the step in a
checkout; the step now writes to mktemp -d and removes it, so a local
run leaves nothing to stage.
* samples/gfx: the display_servers test covers the XRandR modeline path
The test links the real dispserv_x11.c against stubbed Xlib and RandR
entry points, and the modeline path added XSetErrorHandler, XCreateGC,
XFreeGC, XFillRectangle, XClearWindow, XRRQueryVersion,
XRRConfigCurrentConfiguration and XRRGetScreenSizeRange to the set the
driver reaches, plus the RARCH_DBG/WARN/ERR logging entry points; the
stubs now supply them and the create/destroy/attach/detach calls are
counted. The set_resolution case went with the heuristic generator it
exercised; in its place one case drives modeline open, enum, add, set,
delete and close through the stub server and checks every RandR call
it should produce, and one drives open through each failing query
(no display, no resources, no output info, disconnected output) and
checks that it refuses, that later ops refuse too, and that nothing
leaks. The realistic layout where the output's crtc is listed among
the screen's crtcs is opt-in per case, since the orientation cases
keep the two disjoint on purpose.
* samples/gfx: live mode switching through dispserv_x11 and dispserv_sdl on an Xorg dummy server
Two samples run against a real X server rather than stubs.
display_servers_x11_live links the modeline engine and dispserv_x11
against libX11/libXrandr, generates a 15 kHz timing for 320x240@60,
puts it on the wire, reads the crtc back and compares every field of
the mode the server is scanning out with the mode the engine
produced, switches again to [email protected], then restores and closes
and checks the desktop mode is back and both generated modes are
gone. display_servers_sdl_live links dispserv_sdl against SDL2, opens
a window, lists the outputs and the listed modes, switches the display
to a listed non-desktop mode and back through modeline_set, drives the
menu picker through get_resolution_list/set_resolution, and checks the
desktop mode returns on close. Both skip without a DISPLAY.
Xorg's dummy driver (xserver-xorg-video-dummy) accepts XRRCreateMode,
XRRAddOutputMode and XRRSetCrtcConfig with arbitrary timings, which is
what makes the switch real; Xvfb's RandR shim returns success and
lists nothing, so it is not a target. The gfx samples workflow starts
that server from the committed xorg-dummy.conf and runs both samples
under ASan/UBSan, then checks no generated mode was left behind.
dispserv_sdl's enum flags the current mode as the desktop when the
backend has no desktop mode to report.
* samples/gfx: the Win32 display server's listed-mode path runs under Wine on the dummy X server
display_servers_win32_live cross-builds dispserv_win32.c, the ADL, ATI
legacy and PowerStrip paths, the resync helper and the modeline engine
with mingw-w64 and runs the result under Wine on the same Xorg dummy
server the X11 and SDL samples use. Wine backs EnumDisplaySettingsExA
and ChangeDisplaySettingsExA with the server's RandR modes, so the
switch is real: open binds \\.\DISPLAY1 and parses its PCI id, the
vendor paths decline (no AMD driver, no PowerStrip) and caps come back
0, enum lists the driver's modes as system modes with the desktop
flagged, the engine - with lock_system_modes 0 and a range wide
enough for any listed 60 Hz mode, the ini a CDS-only user sets - picks
a smaller listed mode, set switches through CDS, the current settings
read back as that mode, and restore + close bring the desktop back.
The three live samples now compile into a local obj/ directory: they
share source files with each other and with display_servers_test but
build them with different flags, and an object left next to the source
by one sample was being linked by the next. The gfx samples workflow
gains gcc-mingw-w64-x86-64 and wine64 and runs the Win32 sample after
the X11 and SDL ones.
* samples/gfx: the three Windows vendor timing paths run under Wine against stand-ins; resync arms before the call
display_servers_win32_vendor cross-builds the ADL, ATI legacy and
PowerStrip paths and the resync helper with mingw-w64 and runs them
under Wine on the dummy X server. A mock atiadlxx.dll beside the exe
provides the seven ADL2 entry points over one adapter and one display
and keeps the timing-override table; a forced Set re-plugs the monitor
the way the driver does, by sending the two device notifications to
the resync window. The backend loads it, maps \\.\DISPLAY1, stages an
update, and flush issues exactly one forced set, after which the table
carries the generated timing with the polarity inversion the driver
version rule requires; get_timing reads it back through the query and
the refreshed cache, and delete removes it through another forced set.
A scratch HKLM key stands in for the adapter's registry key: update
writes DALDTMCRTBCD<w>x<h>x0x<r>, every BCD field, the flags word and
the checksum are checked against the modeline, and get_timing decodes
the 68 bytes back to the same timing. A TPShidden window answering
UM_GETTIMING and UM_SETCUSTOMTIMING through global atoms stands in for
PowerStrip: create finds it and backs the timing up, update sends the
generated timing as porch widths after the dotclock probe, close puts
the backup back.
Writing the mock showed a hole in the resync helper: the wait cleared
the notification flags on entry, so a re-plug that arrived before the
wait began - which is what a synchronous driver does - was lost and
the wait ran to its bound. The helper now has arm(), called before the
forced set, and wait() consumes what arrived since; the ADL flush uses
the pair. The helper also reports a failed window creation.
* crt: the SDL display server set to Always unlocks listed modes without known timings
The engine locks system modes by default - listed WxH@R entries whose
timing nobody can read - so a 15 kHz CRT is never sent a stock
driver's VESA timing by label. Through the SDL display server every
listed mode is a system mode and nothing can be added or rewritten,
so with that lock the CRT consumer could only ever land on the desktop
mode. A user who sets the SDL display server to Always has chosen
listed-mode switching over the native server; the consumer now treats
that choice as the unlock, clearing lock_system_modes on the generator
after the base ini and again after each overlay set so an ini cannot
quietly relock it. Every other path keeps the ini-controlled default.
The setting's sublabel says so.
* display servers: the modeline path binds the head the window is on, and the monitor index setting names it
The CRT consumer hands the display server a screen of "auto" or the
monitor index minus one. On X11 that index used to count connected
RandR outputs while the context driver places the window on a
Xinerama screen of the same number, and the two orders are not the
same thing; "auto" took the first connected output regardless of
where the window was. dispserv_x11 now binds by geometry: "auto" is
the output whose crtc holds the centre of the RetroArch window, an
index is the output under the Xinerama screen the context driver maps
that index to, a connector name still binds directly, and the output
count is the fallback only when neither is known. dispserv_win32's
"auto" is the monitor MonitorFromWindow reports for the RetroArch
window, the primary device without a window. The SDL server already
used the window's display.
The Video > Monitor Index setting now shows the connector name and
size of the head an index lands on, from modeline_list_outputs through
the new video_display_server_list_outputs(), so the number can be read
against what is plugged in. The stubbed X11 sample supplies the two
Xlib calls the geometry lookup adds; the live X11 sample maps a window
on the dummy server and checks "auto", a Xinerama index and a
connector name each bind the head under it.
* modeline: EDID generation for a CRT preset, written to the config directory on request
gfx/modeline/modeline_edid.c builds an EDID 1.3 base block for a
monitor that syncs a given range, with a modeline as its preferred
detailed timing and the preset name as the monitor name, checksummed,
no extension blocks. A 15 kHz CRT on a VGA or SCART adapter has no DDC
to tell the kernel or the display driver any of that; loaded as a
firmware EDID on Linux (drm.edid_firmware=<connector>:edid/<file>) or
as an EDID override on Windows, the block makes the connector report
as connected, gives the driver the sync limits so CRT timings are not
pruned, and puts the console and desktop on a rate the tube can show
before RetroArch runs. The block for a generator's preset uses the
preset's own 320x240@60 and its first live range.
The CRT SwitchRes menu gains "Write EDID for CRT Preset", an action
that builds the block for the configured preset - menu mode, or the
ini set for mode 4, loaded in the same order the switching path uses
- and writes <config>/edid/<preset>.bin. It installs nothing: an EDID
override is persistent, system-wide and takes effect before any UI,
so a wrong preset on the wrong tube would follow the user to the boot
console and be awkward to undo. The notification names the file and
the log names the connector and the kernel parameter to use, and how
to apply it on Windows.
The golden matrix now carries the block's detailed timing, range
limits and name bytes for the 320x240 source on every preset, matched
against the 2.2.1 generator; the two differ, by design, in the vendor
and serial bytes, in the feature byte where 2.2.1 put the hsync
polarity in the vsync bit, and in padding the name with spaces after
its terminator as the standard says rather than with zeros.
* samples/gfx: the KMS display server's modeline ops and the DRM timing mirror
display_servers_kms_test builds dispserv_kms.c with HAVE_MODELINE and
supplies the two symbols the modeline path reaches, video_state_get_ptr
and video_driver_set_video_mode. The new case checks the KMS shape of
the ops - list_outputs reports the live connector, caps are ADD only,
enum lists nothing because the engine generates, add/update/delete/
flush succeed without touching the driver - and then set: the CRT
consumer's drmModeModeInfo mirror in the video state carries the
timing as the DRM context reads it (clock in kHz, the sync counts,
interlace, doublescan and polarities as separate ints, vrefresh from
the label), exactly one mode set of the mode's size is requested, and
a second set rewrites the mirror rather than reinitialising anything.
* video: dispserv_sdl2 and dispserv_sdl3, one per SDL major
The SDL display server was SDL2-only under a generic name; it is now
gfx/display_servers/dispserv_sdl2.c, and dispserv_sdl3.c is its SDL3
counterpart: displays are ids from SDL_GetDisplays rather than
indices, the mode list is the SDL-owned array from
SDL_GetFullscreenDisplayModes with float refresh rates rounded to the
integer label the engine scores on, and a listed mode goes on the wire
as the window's fullscreen mode followed by fullscreen, with a NULL
mode for desktop fullscreen on close. Both report caps 0 and system
modes only; neither is a custom-timing path. video_driver.c binds
dispserv_sdl to whichever the build has, so the video_sdl_display_server
setting and the CRT consumer's Always unlock apply to either, and the
setting is listed for an sdl3 driver or context too.
The live SDL sample is renamed for SDL2 and gains an SDL3 twin; the
gfx samples workflow builds SDL 3.2.20 from its release tarball, the
version the tree requires and one the runner image does not ship, and
runs both. SDL3's X11 backend makes a window fullscreen through the
window manager and reverts the switch when none answers, so the step
starts openbox before the SDL samples.
* qb: the modeline engine defaults to yes, not auto
The engine is C89 over libretro-common and its apply paths are gated on
their own switches, so there is nothing to probe for: auto was a
leftover from the C++11 check the outgoing library needed. Both
--disable-modeline and the old --disable-crtswitchres turn it off, and
the alias always follows the real switch.
---------
Co-authored-by: libretroadmin <[email protected]>
* menu: the mouse wheel moves the list, not the selection
Ozone and GLUI already scroll from pointer.y_accel when you drag. The
wheel was MENU_ACTION_UP/DOWN in menu_driver.c for every driver, so a
notch stepped the highlight the way a d-pad does. A wheel_scroll hook
lets a driver take the notch as list movement; XMB and RGUI leave it
NULL and keep the old step.
Ozone queues notches and spends them in the same clamp as drag. Three
rows per notch. It also turns cursor mode on, because a wheel moves no
pointer, and without that the hover code never runs. GLUI uses the
animated scroll it already had for swipes.
Going back to the Ozone sidebar hides the thumbnail bar. That rewraps
every entry and used to re-center on the selection, which a pointer
leaves wherever it was resting. preserve_scroll_on_compute keeps the
same top entry in view, the way Material UI already does when geometry
changes without the list changing.
* rgui: Add mouse wheel scroll support
* ozone: wheel keeps flipping fullscreen thumbnails; preserve view only for pointer transitions
With a fullscreen thumbnail up the wheel falls back to MENU_ACTION_UP/DOWN
so it keeps flipping entries behind the overlay instead of scrolling the
hidden list.
preserve_scroll_on_compute is now set only where the pointer drives the
sidebar transition (mouse follow and press-and-hold in ozone_render) and
when a hover changes the selection under current-selection-only
sublabels, so the list no longer snaps back to the hovered entry after
a notch. Pad and keyboard transitions through ozone_go_to_sidebar and
ozone_leave_sidebar re-derive the scroll from the selection as before,
and any explicit entry action clears the flag so it cannot outlive the
compute it was raised for.
The sidebar wheel step is measured in sidebar rows, and the compute
path clamps to the same bottom boundary ozone_render uses for drag and
wheel.
* rgui: make mouse scroll on short lists do nothing
* rgui: widen scroll_y to int32_t
A playlist of a few thousand entries at a 10px stride puts the list
offset past 32767, so drag and wheel scrolling both wrapped. The
field moves next to the other 32-bit members to keep the struct
sorted by alignment.
---------
Co-authored-by: Rob Loach <[email protected]>
Co-authored-by: LibretroAdmin <[email protected]>
* ozone: the mouse highlight goes where the mouse is
The sidebar cached a hovered category for the next click without
showing it, so the cursor sat on the active tab while the pointer was
somewhere else. It now follows the pointer down the sidebar; the tab
in use is still legible through its highlighted label, which is how it
already reads when focus is in the entry list.
Both lists also drew a highlight while the pointer rested between rows,
over the header, the footer or the thumbnail bar, pointing at something
the next click would not reach. Nothing is drawn there now. And because
the entry cursor comes from ozone->selection, which the hover auto
select only reaches on the following frame, the box could sit a frame
behind the pointer, or several while the accel guard held it back. It
stays hidden until it would land under the pointer, and the crossfade
from the previous entry is skipped when a mouse is driving, since that
row has already been left.
A keypress clears cursor mode, and every highlight returns with it.
* ozone: let the pointer move the highlight instead of hiding it
Follow-up to the sidebar hover change. Two problems with hiding the
cursor whenever the pointer is not on a row.
Entry labels have no selected colour - every one of them draws with
theme->text_rgba, and entry_selected only feeds the ticker - so the box
is the only thing saying which row is live. Bump the mouse while it
rests over the footer, the thumbnail bar or a gap and cursor mode comes
on, nothing is under the pointer, and the highlight goes out while OK
still activates the row it was on. The sidebar can afford this because
the active tab keeps text_selected_rgba; the entry list cannot.
The hiding also came a frame early. ozone->selection is synced from
menu_st->selection_ptr near the top of ozone_render, before the loop
that auto-selects under the pointer, so on the frame the pointer
crosses into a new row i != ozone->selection and no box was drawn at
all. Sweeping the list strobed the highlight once per row - louder than
the one-frame lag it was replacing.
So do for the entry list what the sidebar change did for categories:
cache the entry under the pointer and draw the box there. A tap already
acts on that same index, not on ozone->selection, so the box is honest
during the accel guard too, and the lag is gone without a blank frame.
With nothing under the pointer the box stays on the selection.
OZONE_FLAG2_POINTER_OFF_ITEM was also one bit answering for two lists.
It was cleared only inside the entry or category loop but suppressed
both panes' cursors, so a mouse over the empty side of an empty
playlist blanked the sidebar highlight - the one place the cursor is
pinned in the sidebar and cannot follow - and the sidebar-to-entries
handoff blinked, both boxes gone on the frame the pointer landed off
the current selection. Replaced with per-pane state, and the outgoing
crossfade is no longer suppressed by either: it is an animation that
has already left, not a claim about where the pointer is.
Last, clicking a sidebar tab restarted the crossfade from zero on the
row that already had the box, and faded a ghost in on the outgoing tab
that had not been drawn since the pointer arrived. Keep the cursor
solid when the goto lands on the row the pointer was leading.
---------
Co-authored-by: LibretroAdmin <[email protected]>
* cocoa_gl: don't release a still-current GL context before reassignment
EXC_BAD_ACCESS in SLSFlushSurfaceWithOptionsAndIndex (SkyLight, called
from AppleMetalOpenGLRenderer's gldPresentFramebufferData, called from
our own cocoa_gl_gfx_ctx_swap_buffers -flushBuffer) on the very first
frame after entering fullscreen. Reproduced 100% of the time.
Root-caused via git bisect (~5000 commits against v1.22.2, which never
hits this) down to bd450862 ("Normalise MRR safety in cocoa_gl_ctx.m").
That commit added a defensive RELEASE(g_ctx)/RELEASE(g_hw_ctx) in
-set_video_mode's mainthread work, reasoning that "the normal reinit
flow has -destroy running first so both statics are already nil here."
That invariant does not hold on macOS: gl2_init() calls -set_video_mode
twice back-to-back as a pre-existing, separate workaround (see the
"very annoying issue" comment a few lines up in gl2.c). On the first
call, a real NSOpenGLContext is created, -setView:'d onto the actual
on-screen window, and made the current context. On the second call
(milliseconds later, same synchronous call chain), the new defensive
RELEASE drops that context's last reference - while it is still the
current context and still attached to the live, on-screen view -
before immediately creating a replacement and re-attaching it to the
same view. Releasing a context that way (a plain release, not the
-clearCurrentContext / -clearDrawable sequence -destroy itself always
uses) leaves WindowServer's surface bookkeeping for that view briefly
inconsistent; the replacement context's first real frame lands on that
and crashes.
Confirmed genuinely deterministic, not a timing race: pausing at a
breakpoint anywhere in this path does not change the outcome, and
adding an outright 2-second delay before the first frame (as an earlier,
wrong theory of mine) had no effect either - only fixing the actual
release does.
Fix: before either defensive RELEASE, clear the context if it's still
current and clear its drawable, mirroring what -destroy already does on
the normal teardown path - so a context is never released while still
current and attached, whether this runs once or (as on macOS) twice.
Trigger conditions (why this went unnoticed): the official/CI macOS
build defines HAVE_COCOA_METAL, which routes fullscreen through
-[apple_platform setVideoMode:] and native -[NSWindow toggleFullScreen:]
- this code path is never reached there. It only reproduces when: (a)
built without HAVE_COCOA_METAL (e.g. `xcodebuild -scheme RetroArch`
against pkg/apple/RetroArch.xcodeproj without selecting the Metal
xcconfig - the default "RetroArch" scheme, not "RetroArch Cg"), (b)
video_driver is the legacy "gl" driver rather than metal/vulkan/glcore,
(c) video_fullscreen is enabled, and (d) running on Apple Silicon -
the null-surface crash is specific to AppleMetalOpenGLRenderer's
OpenGL-over-Metal translation layer, which older Intel Macs with native
OpenGL never go through. That's a narrow enough combination that it
plausibly never came up before.
Verified with 3 consecutive clean launches after the fix (previously
100% reproducible crash on every launch: direct double-click, under
lldb, and launched by a frontend).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
* cocoa_gl: detach the previous GL context on the correct threads before release
On non-Metal macOS gl2_init() calls -set_video_mode twice, so the
main-thread body can run with a live g_ctx/g_hw_ctx from the first
call. Clear the render thread's current context in
cocoa_gl_gfx_ctx_set_video_mode before marshaling to the main thread
(current-context state is per-thread, so a +currentContext check
inside the main-thread body never matches under threaded video), then
clearDrawable and RELEASE both contexts on the main thread, mirroring
-destroy. The detach is hoisted above the HW/non-HW branch so a stale
g_hw_ctx is dropped on either path.
Affects the GitLab OpenGL x64 build and Makefile GL builds, which
compile this file without HAVE_COCOA_METAL and without ARC; under ARC
RELEASE() still releases via nil-assignment, so the comment no longer
claims otherwise.
---------
Co-authored-by: chulq <[email protected]>
Co-authored-by: Claude Sonnet 5 <[email protected]>
Co-authored-by: LibretroAdmin <[email protected]>
The test links the real dispserv_x11.c against stubbed Xlib and RandR
entry points, and the modeline path added XSetErrorHandler, XCreateGC,
XFreeGC, XFillRectangle, XClearWindow, XRRQueryVersion,
XRRConfigCurrentConfiguration and XRRGetScreenSizeRange to the set the
driver reaches, plus the RARCH_DBG/WARN/ERR logging entry points; the
stubs now supply them and the create/destroy/attach/detach calls are
counted. The set_resolution case went with the heuristic generator it
exercised; in its place one case drives modeline open, enum, add, set,
delete and close through the stub server and checks every RandR call
it should produce, and one drives open through each failing query
(no display, no resources, no output info, disconnected output) and
checks that it refuses, that later ops refuse too, and that nothing
leaks. The realistic layout where the output's crtc is listed among
the screen's crtcs is opt-in per case, since the orientation cases
keep the two disjoint on purpose.
gfx/drivers/metal.h carries everything the Metal driver shares with its
neighbours: the vertex, uniform and HDR structs that metal.m and the shaders
must agree on, plus the MetalView @interface that ui_cocoa.m and
ui_cocoatouch.m need in order to instantiate the render view. The
Objective-C half sits behind __OBJC__ so the Metal compiler sees only the
struct layouts, and the include guard keeps a single @interface in the
griffin unity build, where metal.m and the Cocoa UI driver share one
translation unit.
Shaders.metal moves alongside it into gfx/drivers/metal_shaders/, matching
the gl_shaders, d3d_shaders and vulkan_shaders layout. The commandline
metallib rules, the clean rule and the three Xcode projects that reference
either file follow the new paths.
webOS hands the screen back to Home as soon as the app's only wl_surface is destroyed, and every content load tears the Wayland context down. Stash the connection and shell surfaces, re-point listener user_data on adopt, and cancel any in-flight frame callback so the compositor never sees a zero-surface app.
CI's wasapi_exclusive_pacing job failed three scenarios after dc62edb
changed the shared fifo floor to a frame at 50 fps plus a period. Two
of the failures are the test being out of date and one is a bug in
that commit, and they are fixed together because the test is what
tells them apart.
The bug: dc62edb's comment says the floor uses the period the engine
actually runs at, "the one its events follow", and notes it may be
below the default under IAudioClient3. The code then asked
GetDevicePeriod(), which only knows the default. So a stream opened at
the 3 ms engine period got a floor sized for 10 ms - a 30 ms fifo where
23 was the point of asking for the small period. The period is already
recorded in wasapi_sh_engine_period by the IAudioClient3 path before
the fifo is sized; the floor consults it now and falls back to
GetDevicePeriod() only when that path was not taken. The 3 ms scenario
reports 32 ms against a 32 ms setting, as the test always expected.
The test: its floor was still "20 ms fifo plus the engine buffer", the
rule before dc62edb. It now restates the driver's rule - a frame at
50 fps plus one engine period, never below two periods, never below the
engine buffer - in its own arithmetic, so a change to either side fails
it rather than passing by coincidence. The legacy-period scenarios at
32 and 24 ms therefore expect 52 ms now, which is what the new rule
says a 22 ms engine buffer costs: the setting cannot be honoured below
the floor, and the floor is the driver's to set, for the reason in its
comment - a fifo of exactly one frame made PAL wait a period on every
write.
All twelve scenarios pass under ASan; win32 syntax is clean.
The statistics overlay prints the Sink and Source lines whenever a
sink rate is known. Turning the option off cleared the bias but kept the
last measured rates, so the lines stayed on screen until an audio reinit
- toggling audio sync or the threaded pipeline - happened to zero them.
Off now zeroes both figures with the bias, and the overlay drops the
lines at once.
Sink-rate harness passes under ASan+UBSan and TSan; audio_driver.c
syntax-checked as C89.
The floor sizes the fifo whenever the latency setting is below the
engine buffer plus the floor, so it is the fifo at the minimum setting.
It was rate/50 - one frame at 50 fps exactly - and the fifo cannot hold
quite that: fifo_new() keeps a byte, and rate control resamples the
chunk a little long whenever the device runs ahead. At 48 kHz a PAL
core offered 960 frames to 959 of room, so a blocking write waited a
period for the pump on every frame, inside retro_run(): every PAL core
stuttered at the minimum setting with the inline pipeline, under any
video sync. 60 fps content fit and did not stutter, 44.1 kHz fit as
well.
The floor is now one frame at 50 fps plus one period, which is what the
pump can free while a frame is being written, and never below the engine
buffer. On the legacy 10 ms engine that is 1440 frames at 48 kHz, 30 ms,
in front of the 22 ms engine buffer; under IAudioClient3 it is a frame
and a 3 ms period. The exclusive path sizes its fifo from the setting
and the device period and is not changed.
Syntax-checked with tools/mingw_syntax_check.sh (win32 gnu99 and linux
c89 pedantic).
With the option off, audio_driver_sink_update() returned before doing
anything, which left sink_bias at whatever the estimator last set while
the option was on. compute_rate_adjust() multiplies every adjustment by
it, and without rate control the ratio itself carries it, so a bias set
in one part of a session stayed in force after the user switched the
option off - the toggle looked like it did nothing.
Off now means a bias of exactly 1.0 and, without rate control, the
nominal ratio; the baseline is dropped as well so that turning the
option back on measures from scratch rather than resuming a window.
Sink-rate harness (samples/audio/sink_rate) passes clean under
ASan+UBSan and under TSan; audio_driver.c syntax-checked as C89.
Asked whether MIDI is on the threaded pipeline. It is, by inheritance:
the synth renders inside audio_driver_flush(), which the pipeline
consumer runs on the audio thread. But only the rendering moved.
fmsynth_midi_write() still applies a message straight into the voice
table from whichever thread the core calls it on - the frame - while
fmsynth_render() walks that same table on the audio thread, and there
is no synchronisation anywhere in the file.
Measured rather than argued: a core emitting a note on and a note off
every frame, under ThreadSanitizer, reports twenty-four distinct races
with the threaded pipeline on and none with it off. Every voice field,
the channel state, the allocator. Turning the pipeline on by default
turned that on for every core that uses MIDI.
Writes queue now and the renderer applies them at the top of the block
it is about to synthesise. The lock is held for the copy in and the
copy out and never across the synthesis, so a core writing a note is
never held for a render. Applying at a block boundary changes nothing
audible: the renderer already produced a whole block per call, so a
message arriving mid-block took effect at the next one either way.
A full queue drops the oldest rather than the newest, and says so once
per block: a dropped note-off leaves a voice sounding until its
envelope ends, a dropped note-on is silence nobody notices. 256
messages is a whole GM reset across sixteen channels and then some;
the test core never filled it.
Without HAVE_THREADS there is no lock and write() applies directly, as
before - correct on a build where the renderer runs on the caller's
thread.
Zero reports over four runs after, twenty-four before, and the notes
still arrive: 400 note-ons over 400 frames with the pipeline on and
off alike, no drops. ASan and LSan clean both ways, win32 syntax clean.
ctr_dsp_thread, switch_thread and switch_audren_thread are the same
shape alsathread was: a fifo in front, a worker behind, doing the
blocking device write while the frontend writes into the fifo. The
threaded pipeline does that from the audio thread and takes the
resampler off the frame as well, and all three plain counterparts -
ctr_dsp, switch, switch_audren - have wait_writable(), so they can host
it. Two mechanisms where one does more.
They differ from alsathread in one way worth stating: these three also
have wait_writable(), so they never disabled the pipeline the way
alsathread silently did. They doubled up with it instead - two workers
and two buffers - which is a smaller fault, and the reason this is a
removal on the strength of the structure rather than of a measured
regression.
Checked for anything worth keeping first. The libnx one pins its worker
to CPU 2 at priority-1, but the frontend's audio thread already takes
both settings (audio_thread_priority, thread_prefer_fast_cores) and
applies them wherever it runs, so that is covered generically rather
than per driver. Its detailed latency log line is in the plain audren
driver too. Nothing else was unique; the rest is fifo and thread
machinery.
The Switch default moves with them. AUDIO_SWITCH resolved to the ident
"switch_audren_thread" on libnx builds, so the threaded driver was what
a Switch user got; it is "switch_audren" now, which is the plain
driver's actual ident. The menu label string moves the same way, or the
entry would name a driver that no longer exists.
Also removed: the 3DS platform hook that nulled audio_ctr_dsp_thread
when ndspInit() failed, the griffin include, and the build list
entries.
Not compiled - there is no devkitARM or devkitA64 here, and I said so
before doing it rather than after. What could be checked was: no
reference to any of the three survives anywhere in the tree, every
remaining audio vtable still fits audio_driver_t, the ident the default
and the label now name is the one the plain driver declares, and the
Linux build, ASan run, win32 syntax and settings table are clean. A
3DS and a Switch build are the real test.
ALSA got it with the threaded capture path; these are the other five,
each in the shape its API offers rather than one imposed on all.
PipeWire and CoreAudio already had the wait inside their read - a
bounded loop on the thread loop for one, a bounded condition wait on
the capture fifo for the other - so wait_readable is that loop with the
copy removed. WASAPI has an event: wait on it with WASAPI_TIMEOUT, then
drain what arrived into the fifo, which is exactly the first half of
its read and bounded for the reason written there, an unbounded wait
parking the caller with no way out if the device stops signalling.
SDL2 has a condition its capture callback signals, and the same
SDL_AUDIO_STALL_TIMEOUT_US bound, since that callback is the only thing
that ever signals it.
SDL3 is the odd one: it offers no event, so this polls
SDL_GetAudioStreamAvailable() with the one-millisecond sleep its read
loop already uses, capped at eight laps. A poll is worse than a wait,
but it is bounded and it is still off the frame, which is the point.
Every one returns what it has rather than what was asked for when the
bound expires, and 0 when the device is gone - the contract the header
sets, so a stalled microphone costs a dropped slice rather than a
parked worker.
All six microphone drivers now have the slot filled and all six vtables
line up at fourteen against microphone_driver_t, checked
mechanically, since an entry in the wrong slot is the failure this kind
of change produces.
Compiled where the headers exist - PipeWire, SDL2, WASAPI and SDL3 by
the win32 checker, CoreAudio against the Apple stubs. ALSA's behaviour
is unchanged and still verified end to end with the microphone core:
300 reads, 240000 frames, none starved, threaded and not, ASan and LSan
clean. The other five want a run on their platforms.
One correction found by the compiler rather than by reading: SDL2's
per-microphone state is sdl_microphone_handle_t, not sdl_microphone_t -
that one is the driver context, and mic_context is the handle.
The other half of the pair. The playback driver went when the threaded
pipeline covered what it did; the capture one goes now that threaded
capture does, and for the same reason: a worker inside one driver doing
what the frontend does for every driver that can wait on its device.
microphone_alsa gained wait_readable() in the previous commit, so the
frontend's worker owns the blocking read, the dual-mono up-channel and
the resampler on ALSA - which is strictly more than this driver moved
off the frame, since it only ever owned the read; flush's resampling
still happened in the core's call.
It was also MICROPHONE_DEFAULT_DRIVER for any ALSA build with threads,
so this is not a corner: it is what a Linux user got. That default is
now MICROPHONE_ALSA, which reaches the same worker through the general
path.
"alsathread" in an existing microphone_driver setting is aliased to
"alsa" for a release, as the playback one is - checked, and such a
config now takes threaded capture and delivers every frame.
With the audio half already gone the whole
!__QNX__ && !MIYOO && HAVE_THREADS block was capture-only, so it goes
whole: the worker, the fifo, alsa_thread_info_t and the vtable. 438
lines, and alsa.c is one playback driver and one microphone driver
again.
Checked on the three configurations that matter - a config still naming
alsathread, the default with no key at all, and threaded capture off -
each delivering 300 reads and 240000 frames with none starved, ASan and
LSan clean on all three. win32 syntax and the settings table clean.
The menu label stays in msg_hash, as the playback one did: the packed
translation headers carry it with byte counts and a contiguity check,
and the driver is no longer in the list, so nothing can reach it.
The capture path had both problems the threaded pipeline was built to
remove from playback, and neither had been addressed: the core's
retro_microphone_read() called microphone_driver_flush() in a loop, and
that does a blocking device read, a dual-mono up-channel and a
resample. All three inside a frame.
Now a worker does them and the core's read is a fifo read. Same
setting as the playback pipeline and the same condition - the driver
must be able to wait on its device, or the worker would poll - so
microphone_driver_t gains an optional wait_readable(), the counterpart
of the playback side's wait_writable(), and a driver without it keeps
the frame-synchronous path and says so in the log. Every existing
microphone driver has a short initialiser, so all seven get NULL and
none changes behaviour.
ALSA implements it, mirroring alsa_wait_writable(): snd_pcm_avail() on
a capture stream reports frames ready rather than room, snd_pcm_start()
begins capture where it began playback, and it is bounded the same way
- two periods per wait, eight laps - so a device that stops delivering
returns 0 and the worker loops back to check whether it should still be
running, rather than parking.
Four races, all found by ThreadSanitizer and all mine:
close_mic() ran before the worker was joined, so the worker was reading
a device being closed. The join goes first now, before anything it
touches is torn down.
capture_running was a volatile bool. Volatile orders nothing between
threads; it is an atomic with acquire/release now.
The worker was started in the middle of mic_driver_open_mic_internal(),
while that function was still writing the USE_FLOAT flag, orig_ratio
and both resamplers - which the worker reads immediately. It starts
last now, after every field it touches has been written.
And the worker read microphone->flags on every pass, through the
sample-size macro and the USE_FLOAT tests in flush, while the main
thread writes the ENABLED bit in set_mic_state() and clears PENDING at
the end of open. Those are the same word. The format cannot change
while a microphone is open, so it is snapshotted before the thread
exists and the capture path asks the snapshot; the frame-synchronous
path derives it as before.
Verified with a core written for it, since none of the test cores opens
a microphone: 400 reads, 320000 frames, none starved, identical
threaded and not. ThreadSanitizer clean over four runs after the fixes
and reporting on every run before them; ASan and LSan clean on both
paths; win32 syntax clean; the audio harnesses still pass.
This also puts microphone_alsathread in the position audio_alsathread
was in before it was removed - a worker inside one driver doing what
the frontend now does for every driver. It is left alone for now: it
still works, and unlike the playback case it is not silently disabling
anything.
It did one thing the synchronous driver did not - own the blocking
snd_pcm_writei on a worker thread so the frontend wrote into a fifo -
and the threaded pipeline now does that a layer up, from the audio
thread, and does more of it: the resampler moves off the frame as well
as the write.
Worse, the two could not coexist. alsathread has no wait_writable(), so
selecting it made audio_driver_init_internal() refuse the pipeline and
fall back to the inline path with a log line most people will never
read. It was also AUDIO_DEFAULT_DRIVER for any ALSA build with threads,
so that was not a corner case: it was what a Linux user without Pulse
or PipeWire got, and after the pipeline became the default it was the
configuration that silently did not get it.
The one case that looked like a justification does not survive reading
it. A core that sets an audio callback has the pipeline refused - but
audio_driver_init_internal() installs the audio thread wrapper for that
case too, on the same condition, so the decoupling is there without a
second worker inside the driver.
Measured before removing, four ways, 900 frames of a 60 Hz core: alsa
with the pipeline, alsathread with the pipeline refused, and both with
it off. Same 64 ms buffer and the same pacing in all four, differences
inside the noise of this machine. So this is not a performance change;
it is one mechanism instead of two, and the one that can host the
pipeline.
"alsathread" in an existing config is aliased to "alsa" for a release,
as "coreaudio3" is - checked, and such a config now also gains the
pipeline it was being denied. AUDIO_ALSATHREAD and its default go with
it.
The microphone half of that file stays and is untouched. There is no
threaded pipeline for microphones, snd_pcm_readi blocks the caller, and
that worker is still the only thing between a core's mic read and the
device. The section comment now says which half went and why.
The menu label and its help text stay in msg_hash. Fifteen packed
translation headers carry that entry with byte counts and a contiguity
check; rewriting them to delete a string nothing can reach is a larger
and riskier change than removing the driver was, and the driver is no
longer in the list, so the label is never shown and the help lookup
never matches.
Three console drivers with a real device clock to offer, each in the
shape its hardware gives.
Wii U's AX frame callback fires every 3 ms and takes
AX_AUDIO_SAMPLE_COUNT frames, so it is counted there - but only inside
the branch where the voice is running and written is decremented. A
stopped voice is not taking anything, and a count that ran on
regardless would report the device playing audio it never played; the
estimator discards windows that stall, which is the right reading of a
voice that stopped. It is accumulated under the spinlock the callback
already holds and read under the same one, because a 64-bit read is
not atomic on this CPU and a torn one would look like the device
jumping backwards.
GameCube's DMA callback fires once per CHUNK_FRAMES, so that is the
count. It is a volatile uint32 rather than an atomic for the reason
dma_busy and dma_next beside it already are - single core, no store
buffer to reorder around - and 32-bit so the read cannot tear, which
wraps in about a day at 48 kHz and is handled by the differences the
estimator takes.
The 3DS DSP has no frame count in its callback, only a signal, so this
one unwraps ndspChnGetSamplePos() instead. That has to happen in the
callback: the position wraps every 2048 samples, about 43 ms, and the
estimator reads once every four seconds, which would miss ninety wraps
between looks. Unsigned subtraction and the mask do the unwrap without
a comparison. Checked against a model walking the position round the
ring 58 times - 120000 frames counted, 120000 expected.
Not compiled: there is no devkitPPC or devkitARM here. What could be
checked was checked, because on these platforms a mistake is a crash
rather than a diagnostic: the vtable slot against audio_driver_t for
all three, that each new function is defined before the table that
names it, that no macro is used above its #define - the 3DS callback
had to move below the block that defines the mask - and the wrap
arithmetic against a model.
Deliberately not done: ctr_csnd. It already computes samples_played,
which looks ideal, but from svcGetSystemTick divided by
CTR_CSND_CPU_TICKS_PER_SAMPLE - the CPU's clock, not the audio
hardware's. Measured against the host clock it would return
approximately zero ppm by construction and mean nothing, which is worse
than not measuring: it would look like a working device clock.
The last two desktop drivers with a device clock to offer. Neither has
a callback per period, so both take the shape ALSA and PulseAudio
already use: everything handed to the device, less what has not been
played yet.
OSS asks SNDCTL_DSP_GETODELAY, which reports the queue in bytes and is
the direct equivalent of snd_pcm_delay(); its format is fixed at S16
stereo at init, so a frame is four bytes. tinyalsa has no delay call,
so the queue is the buffer less the room pcm_avail_update() reports,
which is the same quantity by a different route.
Both guard the same three ways, and the guards are the point rather
than decoration: a failed query reports nothing rather than a count
that would read as a stall and drag the measured device rate down; a
queue larger than what was written - which a bogus query can produce -
reports nothing rather than wrapping a uint64 into an enormous count;
and the running total is 64-bit, so a long session does not overflow
into a backwards step that would look like a stopped device.
Those cases are checked directly: nothing written, fully drained, part
queued, entirely queued, a queue larger than the total, a billion
frames in, and monotonicity across two hundred thousand frames with a
steady queue.
Both compile against the real headers, and the vtable slot each entry
landed in was checked against audio_driver_t.
Every desktop and mobile audio driver now reports consumption except
xaudio, which cannot: XAudio2's SamplesPlayed stops while the voice is
starved, so it would read the device slow exactly when the buffer is in
trouble. That one stays as it is, for the reason written in the file.
OpenAL has no callback per period, so this is neither JACK's shape nor
ALSA's: al_unqueue_buffers() already collects the buffers the source
has finished with, and those are frames the device has played. Counting
them where they are collected is the whole change.
A buffer is counted once, when it is first seen as processed, so the
running total is right even though the moment of observation belongs to
the frontend rather than the device: looking late moves when a frame is
counted, not how many there are, and the estimator sums over windows of
seconds.
AL_SAMPLE_OFFSET would add the position inside the current buffer, but
it is per-buffer rather than cumulative and would have to be stitched
to this count to mean anything. It would buy nothing: OPENAL_BUFSIZE is
1024 bytes, which is 256 frames of int16 stereo or 128 of float, and
256 frames is 1333 ppm of a four-second window against the 2500 the
estimator is already specified to tolerate for a 480-frame period. The
count is finer than the thing it feeds.
Checked: compiles against the real OpenAL headers, which are present
here; the frames-per-buffer arithmetic for both formats; and the vtable
slot against audio_driver_t, since an entry in the wrong place is what
broke the Android build the last time this hook was added to a driver.
Three more drivers that had no device count for the sink rate estimate,
so the statistics overlay correctly showed no Sink row and the
correction had nothing to work from. Each gets the shape its API
actually offers rather than one imposed on all three.
JACK and PipeWire are callback-driven, so they are counted the way the
WASAPI pump and the ASIO callback are: the server calls the process
callback once per period and says how long it is, and that is the
device's own clock. No queue to subtract, because the callback is the
device consuming the audio rather than a queue being filled. Silence
during an underrun counts: the period elapsed either way, and elapsed
device time is the quantity. A counter that stalled on underruns would
read the device slow exactly when the buffer is in trouble.
PulseAudio has no such callback, so it takes ALSA's shape instead:
everything handed to the server, less what has not been played yet.
pa_stream_get_latency() reports that remainder - the server's queue
plus the sink's own, which is what should come off - and it is only
meaningful once timing data has arrived, so a stream without it yet
reports nothing rather than a count that would read as a stall. Its
negative flag is handled rather than ignored: on a monitor source or
just after a flush the server can be ahead of the write pointer, and
subtracting a negative latency would inflate the count.
All three compile clean against the real headers, which are present
here, unlike the OpenSL one. The slot each entry landed in was checked
against audio_driver_t as well, since a vtable entry in the wrong place
is what broke the Android build the last time this hook was added.
Wants a run on each to see the Sink row settle on a plausible ppm; a
figure past 500 now refuses itself and says so, so a miscount announces
rather than corrupts.
They are process objects and outlive the thread that made them, and
rtime_sleep_deinit() can only reach the calling thread's own slot, so
the handle of every other thread that ever slept stays open until
process exit. Harmless for a fixed set of workers; a reason not to
sleep from a thread created per event.
A sanitizer switch after make clean kept linking the stale
scan_begin_budget_test.o from the previous instrumentation, because
clean removed the other two test mains but not this one.
task->flags is guarded by property_lock - the worker's handler reads
it through task_get_flags mid-task - and the threaded reset already
takes it for exactly this reason. Cancel was the one site left
setting the flag under running_lock alone, which both races that
read and, being a read-modify-write, can lose a concurrent
task_set_flags update in either direction. The GCD backend shares
this cancel, so it is covered by the same change.
Same gap CoreAudio had: the driver never implemented frames_consumed(),
so the estimator had no device count to compare against what the
frontend offered, and the statistics overlay correctly showed no Sink
row. On Android that is every user of the default audio driver.
The buffer queue callback fires once per block the device has finished
playing, so counting blocks counts device time - the same thing the
WASAPI pump and the ASIO callback count, and with no queue to subtract
as ALSA needs, because a block only comes back once it has been played.
The block size in frames is now kept beside the byte size it was
derived from, and the counter is a retro_atomic add in the callback
against an acquire load in the reader, matching how buffered_blocks is
already handled in this file.
Ordering checked rather than assumed: frames_per_block is assigned at
init before RegisterCallback, so there is no window in which the
callback could fire and add zero.
Not compiled here - no NDK, and the SLES headers are not stubbable
without writing most of OpenSL ES. What could be checked was the slot,
since a vtable entry landing in the wrong place is what broke the
Android build the last time this hook was added to a driver: every
audio driver's initialiser was counted against audio_driver_t, and
opensl's sixteen line up. Wants an Android run to see the Sink row
appear and settle on a plausible ppm.
A title bar drag, a border resize, or an open menu enters a modal loop
inside DefWindowProc that holds the thread until the user lets go, so
no frame runs and no audio is written for the length of the drag.
The loop still dispatches posted messages. For its duration a helper
thread sleeps at the content rate on the high resolution timer and
posts a tick, and each tick runs one iteration. The iteration is told
that an external clock is pacing it, through a new RUNLOOP_PACE_EXTERNAL
source: the frame-limit sleep, frame delay, and the no-window wait then
yield to that clock, so the iteration returns as soon as the frame is
out and the modal loop has the mouse back within a frame's work rather
than a frame's period. Content and audio hold their rate at any refresh
the core reports. Where no thread can be had, WM_TIMER drives the ticks
instead.
The thread is created once and parks between sessions, so a drag does
not leave retro_sleep()'s per-thread timer handle behind. Three things
guard the nested iteration: a flag drops a tick pulled out by the
iteration's own pump, the pump leaves mouse messages in the queue while
a tick runs since the modal loop has the mouse captured, and a quit
that an iteration accepted is not repeated on later ticks. Threaded
video is left alone, as the window then belongs to the video thread
and the run loop is not blocked.
Everything used exists on Windows 95; message and timer constants are
defined locally where an SDK lacks them.
retro_sleep() on desktop Windows sleeps on a high resolution waitable
timer, so it no longer rounds up to the system timer period. Three
comments still justified their surrounding code with that behaviour.
The code stays as it is: the menu path defers to audio backpressure
because two clocks on one loop is double-pacing regardless of timer
resolution, and the pad enumeration waits pump before sleeping because
the pump is what usually ends the wait.
Two things from following up the CoreAudio crackling report.
The buffer model showed the correction was innocent there: a 10.7 ms
buffer with a 490 ppm mismatch first underran at eleven seconds with
the estimator on and off alike, and the earliest a bias can exist is
thirty. It was also useless - it cannot hold a buffer that small - and
nothing said so.
The thirty seconds is a floor rather than a choice. A device that
counts in whole periods is only known to a period, which is 2500 ppm
of noise over four seconds and 330 over thirty, so applying sooner
would apply noise. That makes the cadence fixed and leaves the buffer
as the variable: if half of it empties in less than thirty seconds at
the mismatch just measured, this mechanism cannot help, whatever it
measures. It now says so once, with the figures and the two things
that do fix it - rate control, which acts on the fill every frame
instead of on the clock every thirty seconds, or more latency. The
driver's buffer size is already known and logged at init, so this
costs nothing but the comparison.
No behaviour change beyond the warning. Shortening the cadence for a
small buffer would trade a correction that arrives too late for one
built from noise, and choosing between rate control and latency is the
user's call, not the frontend's.
The second thing came out of asking whether that warning could be
spammed. It cannot - all three sink warnings are one-shot, the flag is
tested first, and the block only runs when a bias is applied. But the
question pointed at the top of the same function, where
audio_driver_sink_update() runs after every write and called
frames_consumed() before the window gate: two hundred and forty calls
at 60 fps to serve one every four seconds, four hundred and eighty at
120. That is an atomic load on WASAPI and CoreAudio but snd_pcm_delay()
on ALSA and an unwrapped play cursor on dsound, made for every frame of
audio and thrown away. The gate moves in front of the call; the
baseline case still asks, since that call establishes the starting
count, so the windows and the counts are identical.
The harness checks both sides of the cadence line, since a warning that
always fires is no more use than one that never does: the reported
10.7 ms buffer at 490 ppm drains in 10.9 s and fires; the same buffer
at 11 ppm lasts 486 s and does not; a 200 ms buffer at 490 ppm lasts
204 s and does not. The four audio harnesses pass under ASan, the
frontend runs clean, win32 syntax is clean.
The CoreAudio report was audio breaking up about a minute in with sink
rate estimation on, and the first bias lands at about a minute, so the
correction was the obvious suspect. The harness measured the estimator
but not what the estimator is for, so it could not answer that.
It models the device buffer now: each second the source delivers at
its rate times the resampler's ratio, the device takes at its own, and
the level moves by the difference. An underrun is a pop. The source
gets a rate of its own too - src_ppm - because a real one need not
produce exactly nominal, and the reported case turns entirely on that:
the device measured +47 ppm, an ordinary crystal, while the source
measured -442.
Run at those figures against the reported 10.7 ms buffer, the answer
is the same with the estimator on and off: the first underrun is at
eleven seconds. A 490 ppm mismatch drains 10.7 ms from half full in
about eleven, and the earliest a bias can exist is thirty. Whatever
breaks that configuration up, the correction is not it - the buffer is
long empty before one is applied. With estimation on the bias does
find the mismatch, +442 ppm, it just arrives far too late to matter.
Both are asserted, so neither can quietly stop being true: the drain
must precede any possible correction, and the correction must still
find the mismatch.
My first pass at this asserted the opposite - that the correction
leaves fewer underruns - and it failed. The assertion was wrong, not
the code: a bias applied thirty seconds after the buffer emptied stops
further drift but cannot refill what has already gone.
A saved view on the Playlists screen carries its .lvw path in the
label slot, so the lane pins the seam that matters: pressing one
must push the deferred Explore list with that path and
MENU_EXPLORE_TAB on the stack, and the plain Explore entry must
push the same list under the canonical label the downstream
view-path check excludes. The entry matcher gains a label mode for
entries shaped this way.
The displaylist-push dispatcher keys on the entry's cbs->enum_idx,
the same key the bind path matches on. Saved Explore views carry
their .lvw path in the label slot, so any label-string lookup sends
them to the archive/file browser instead of the Explore view; the
canonical-label comparison remains only as a fallback for entries
without a live selection buffer.
Found while chasing the CoreAudio crackling, and a defect in its own
right: audio_driver_flush()'s fast path - the one that hands int16
straight to a driver that resamples for itself - never touched
sink_offered and never called audio_driver_sink_update(). On any driver
taking that path the estimator was inert. No Sink row, no measurement,
no bias, and nothing anywhere saying it was doing nothing; the feature
simply was not running and looked like it was.
That is also why CoreAudio showed no Sink row before. It implements
write_raw, so it took this path, and only became visible when the
threaded pipeline turned on by default and moved it to write().
The driver resamples here, so what it will hand the device is the
frames given scaled by the rate ratio and by rate control's
adjustment. sink_offered wants that with the adjustment divided out, as
the other two write sites do, so the bias measures the clocks rather
than rate control's own corrections; sink_offered_raw keeps it, since
that is what was really offered. write_raw returns device frames
accepted, so sink_accepted takes it directly with no byte conversion -
unlike write(), whose return is bytes.
One limit, written at the call site because it is not obvious: on this
path the bias can only reach the driver through the rate_adjust
argument, which compute_rate_adjust() multiplies it into. With rate
control off there is no rate_adjust to carry it, so the rate is
measured and shown but not applied - the same as for a blocking
writer, which the estimator already handles that way.
The arithmetic is checked across matched rates, a 44.1 kHz core on a
48 kHz device and the reverse, with and without a rate control
adjustment: offered comes out as device frames with the adjustment
divided out in every case. The four audio harnesses pass under ASan,
the frontend runs clean, win32 syntax is clean.
The settings table drift, griffin-order unity compile and migrator
self-test no longer gate pushes and pull requests.
tools/settings_check.py and the migrator self-test remain in the tree
for manual runs.
From a CoreAudio log: "the device appears to take 48004.5 Hz against a
nominal 48000 (+94 ppm), which is too far off to be a crystal". Ninety-
four parts per million is a perfectly ordinary crystal, so the line
read as nonsense.
The check is on the ratio between the two counts and the message
printed the device against nominal, which is a different quantity. The
ratio was over the 500 ppm line because the source side was running
about -400; the device was fine.
The warning now prints the ratio it actually judged and both sides of
it, so the next report says which side moved without anyone having to
know how the check works.
Reported on CoreAudio: crackling about a minute after start with sink
rate estimation on. The timing is the diagnosis. The bias is set thirty
seconds after the baseline opens, the clamp is 2000 ppm, and 2000 ppm
empties a 64 ms buffer in thirty-two seconds - so a measurement that
lands on the clamp gives clean audio for a minute and then breaks up,
which is exactly what was seen.
The clamp was the wrong response to a bad measurement. Audio crystals
are specified in tens of parts per million and the worst are around a
hundred; every device measured for this feature came in at eleven. A
ratio of 2000 ppm is not a fast crystal, it is two counts that are not
measuring the same thing - and clamping it still applies 2000 ppm,
which is guaranteed to drain the buffer. The clamp bounded the number
without questioning it.
So a ratio further than 500 ppm from unity - five times the worst
plausible part - is refused. The bias stays where it is, at 1.0 if
nothing has ever been applied, and the rate goes on being measured and
shown in the log and the overlay, which is what a mismeasuring driver
needs in order to be found. One warning names the driver and the
figure. The clamp stays behind it for what gets through.
The trade is deliberate: a device genuinely 600 ppm out is now left
uncorrected and drifts as it did before this feature existed. That is
a worse outcome than correcting it and a much better one than
corrupting a device that was fine, and no such device has been seen.
The sink_rate harness asserted the clamp, so it failed on this change -
correctly. That case now asserts refusal, and two more bracket the line
itself: 400 ppm is still corrected, 900 ppm is not. It also checks the
rate is still reported when the bias is refused, since silently
measuring nothing would hide the driver bug underneath.
This stops the symptom on any driver. It does not explain why CoreAudio
measures what it does - that wants the Sink log line from the affected
machine, which now says so in as many words.
It has been off by default since it landed. The reasons to turn it on
are the ones the sublabel already claims and the harnesses already
check: the same latency as the frame-synchronous path at any Audio
Latency setting - the pipeline moves off the frame, it does not add a
buffer - rate control measured at the device's own pace instead of
once a frame, and the resampler out of the frame budget.
Nothing here forces it on a driver that cannot take it. audio_driver
already requires wait_writable() and no core audio callback, and falls
back to the inline path with a log line naming the driver otherwise,
so this is the default for the drivers that can and a no-op for the
rest. The whole block is inside HAVE_THREADS, so the default follows
it: true with threads, false without, in the shape config.def.h
already uses for DEFAULT_THREADED_DATA_RUNLOOP_ENABLE and
DEFAULT_MENU_SAVESTATE_RESUME.
Only new installs move. An existing retroarch.cfg has the value
written, and configuration.c's migration for the old tri-state leaves
what it finds.
tools/settings_table.reference regenerated, since it records defaults
and the Settings-Checks job compares against it - the flip shows up
there three times, once per emitted table. Verified by running with
the key absent so the default applies, which takes the threaded path
and logs the ring size, and with it explicitly false, which does not.
The four pipeline harnesses pass under ASan and win32 syntax is clean
on the two touched files.
The statistics overlay shows no Sink row on CoreAudio because the
driver never implemented frames_consumed(). The estimator needs the
device's own count of what it has taken to compare against what the
frontend wrote; without it there is nothing to measure and the row is
correctly suppressed. ALSA, WASAPI, ASIO and dsound have had it since
the feature landed - CoreAudio was simply missed, and the
consolidation that merged coreaudio3.m into this file did not add it
either.
The render callback is the device asking for exactly one period, so
counting what it asks for counts device time. That is how the WASAPI
pump and the ASIO callback do it, and unlike ALSA, which has a queue
to subtract from what it wrote. Silence during an underrun counts: the
period elapsed whether or not there was audio to fill it, and elapsed
device time is the quantity. A counter that stalled on underruns would
read the device slow exactly when the buffer is in trouble, which is
the WASAPI shared bug from the original series over again.
The ring is counted in samples and the output bus is fixed at
interleaved stereo, so a frame is two of them. The counter is written
only by the callback and read by the frontend, so it is a retro_atomic
add against an acquire load, like the ring's fill.
Run against the stub device at 48 kHz: 94208 frames over 1.98 s is
47699 Hz, inside a percent of the device rate on a machine with one
core running under ASan; it keeps counting through half a second of
deliberate silence, 23552 frames over 46 callbacks; and it is
monotonic. Syntax clean as gnu89 with mixed declarations an error and
as gnu99, macOS and iOS. The earlier coreaudio harnesses still pass.
Wants a Mac to see the Sink row appear and converge on real hardware.
The app-icon texture cache found in cff40c8 was an instance of a shape,
not a one-off, so here is the sweep for the rest of it.
The shape: every Objective-C file the Apple build compiles is built
twice and differently. The Makefile builds each .m alone, MRC, bar the
four given -fobjc-arc at Makefile:275; Xcode builds griffin_objc.m,
which includes most of them, as one ARC translation unit. A convenience
constructor returns an autoreleased object, ARC retains it into a
strong slot and MRC does not, so the same line storing one in a static
or an ivar is correct under Xcode and a use-after-free under the
Makefile once the pool drains. The build most people use is the one
where it works, which is why these survive.
Thirteen MRC files, thirty-eight long-lived object slots. Every
candidate the scan turns up is a local or a permanent singleton, so
nothing else of that exact kind is left. Two things did come out of it.
ui_companion_cocoatouch_set_app_icon() declared NSString *str with no
initialiser and assigned it only when the icon is not "Default". nil is
the documented way to restore the primary icon and is what the
"Default" case means, but only ARC zero-initialises a strong local; MRC
leaves it indeterminate, so under the Makefile build choosing "Default"
handed whatever was on the stack to -setAlternateIconName:. Written
down as nil now.
And the ownership rules in cocoa_gl_ctx.m's hand-rolled fullscreen path
are right for a reason worth recording rather than changing: it uses
-retain and -release, which do not compile under ARC, and it sits in
the #else of HAVE_COCOA_METAL, which Xcode defines. ARC never sees it.
The scan is kept as tools/apple_mrc_ownership_check.py rather than run
once and thrown away, since the next file added to griffin_objc.m
inherits the same split. It is a lint, not a prover - it reports
permanent singletons too - and it is self-checked: reintroducing the
texture-cache bug makes it name that line.
Not swept, and worth saying: the other half of this split is
uninitialised object locals, which is what the set_app_icon bug
actually was. A heuristic for that missed the very case it was written
for and flagged six that were fine, so it is not included. Clang's
-Wsometimes-uninitialized finds them properly, and that wants a Mac.
Looking at whether this dispatch_once was needed turned up something
worse than the once.
The cache is +dictionaryWithCapacity:, which hands back an autoreleased
object, assigned to a static and used across calls. This file is built
MRC by the Makefile - it is not in the -fobjc-arc list at Makefile:275
- so the pool drains at the end of the run loop pass that created it,
and every later call messages freed memory. Under Xcode, where
griffin_objc.m is compiled ARC, the strong static retains it and the
identical code is correct, which is how this has survived: the build
most people use is the one where it works, and the menu that reaches
it - materialui's app icon list - is not a screen most people open.
-initWithCapacity: is +1 owned and right under both, and the object is
kept for the process lifetime deliberately, as the dock indicator in
dispserv_apple.m is.
The dispatch_once around that create was misleading rather than
wrong. It reads as a thread-safety guarantee the rest of the function
does not make: the next lines mutate the dictionary with no
synchronisation at all, so were this ever reached from two threads the
once would be the one part that was safe. The sole caller is
materialui's icon draw, on the menu's frame, so a plain lazy create
says what is true.
Left alone, the other three in this sweep. WebServer's +sharedInstance
is a public singleton accessor whose contract is that it works from
any thread, and it constructs a real object - that is what
dispatch_once is for, and swapping it for a flag would trade a correct
idiom for a proof obligation the next caller has to know about.
get_app_icons() and get_ios_version() likewise build something once,
from main-thread paths, at no cost worth removing. Unlike the metal
and cocoa_common cases, none of these three is guarding the assignment
of constants into a global.
Not compiled here - no Apple toolchain, UIKit cannot be stubbed.
Linux still builds.
Two sites, neither doing what a once is for.
sendKeyForPress: built an NSDictionary of NSNumber keys to NSArrays of
NSNumbers inside dispatch_once, then hashed a boxed press type against
it on every button press. The mapping is nine compile-time constant
pairs. A switch is the whole of it: no dictionary to build, nothing
boxed per press, no once, no block, and the two-element array indexing
becomes two locals.
The page keys were added to that dictionary under
if (@available(tvOS 14.3, *)), which guarded the wrong thing. The
constants have to exist at compile time either way and did, referenced
unconditionally inside the block, so the SDK floor is unchanged; and a
press type the running system never sends simply never arrives, so a
case for it costs nothing. The runtime check bought a branch and the
appearance of caution.
The web server alert used a once to show itself only once per run.
That is a flag. dispatch_once is a barrier for publishing an
initialisation to other threads, and this is UIKit on the main thread
putting up an alert - it works, but it says something that is not
true, and a once can never be reset while the intent is exactly
something that might want to be. The blocks inside stay: those are
UIAlertAction handlers, which the API requires, not GCD.
What is left of GCD in this file is real and stays. dispatch_after on
the main queue for deferred work, dispatch_async to hop to the main
thread, and a dispatch_semaphore in an actual wait loop. Unlike
coreaudio.c there is no portability argument to make here either: this
file is iOS and tvOS only, where GCD has been present since iOS 4, so
these two changed because they were the wrong tool, not because the
tool was unavailable.
Not compiled here - no Apple toolchain, and UIKit cannot be stubbed.
Linux still builds.
NSStringFromRPixelFormat() filled a file-scope array on first call,
under dispatch_once with a block. There was nothing to initialise.
Every value is a compile-time constant and an NSString literal is a
constant object the compiler places in __DATA - not allocated, not
refcounted, in existence before main() runs - so the once was guarding
the assignment of constants into a mutable global that existed only to
hold them.
It was not about speed either. dispatch_once's fast path is an acquire
load and a compare, which is what a plain flag costs; and the only
caller is -debugDescription, which is a human with a debugger or a log
line, never a frame. Metal is 10.11 and later so the 10.6 floor that
motivated removing GCD elsewhere does not apply here - this is simply
machinery with nothing to do.
A switch over the enum drops all of it: the global, the token, the
block, and the separate bounds check, which the default arm now
covers. It is thread-safe by construction rather than by a barrier,
and because every enumerator is named, a format added later without a
string gets a compiler warning instead of the empty slot the array
would have handed out.
While there: Context's _inflightSemaphore is created with
dispatch_semaphore_create(MAX_INFLIGHT) at init and then never waited
on, never signalled, and never released - the only two mentions in the
file are its declaration and that assignment. Whatever frame-inflight
throttling it was meant for was never wired up, and MAX_INFLIGHT
reaches the layer's maximumDrawableCount on the next line regardless.
Removed. metal.m now references no dispatch API at all.
Not compiled here - no Apple toolchain, and Metal cannot be stubbed.
Linux still builds.
Three things in the macOS block that a 10.4 or 10.5 toolchain could
not build or would refuse at runtime, and one code path each now
rather than two.
The dock progress bar was built inside dispatch_once() with an
Objective-C block, which put it behind RARCH_HAS_CGDISPLAYMODE_API - a
macro about the display-mode API that happens to also mean "10.6 SDK",
so the feature was absent from builds for a reason unrelated to
anything it does.
Neither the once nor the block earned that. dispatch_once's fast path
is an acquire load and a compare against ~0; a plain flag is a load
and a compare. Disassembled both: on x86-64 one is mov/cmp/jne and the
other cmpb/je, and on arm64 the only difference is ldar against ldrb -
the acquire, which is there to publish an initialisation to another
thread, and there is no other thread here. This runs from a task
callback on the main thread, and everything it touches is AppKit,
which must be on the main thread anyway. The block is the worse half:
it is a compiler feature, not a runtime one, so GCC 4.0 - which is
what Xcode 3.1 has - cannot parse it however new the system running
the binary. It is a static bool now.
NSDockTile itself is 10.5, so -dockTile is asked for rather than
assumed and the tile is held as id, which means no SDK has to declare
the class. Its -size returns a struct that performSelector: cannot
carry, so KVC boxes it, with 128x128 as the fallback the dock has
always used. On 10.4 the selector is absent, the function reports
failure and the frontend goes without a dock progress bar - which is
what it did there when this was compiled out, except now that is a
runtime answer instead of a build-time one.
-setStyleMask: is 10.6 and was sent unconditionally: before that a
window's style is fixed at creation, so it is asked for and the
decoration toggle reports failure rather than throwing. The constant
needed no such care - cocoa_defines.h already maps
NSWindowStyleMaskTitled to NSTitledWindowMask, which is 10.0.
And <CoreGraphics/CoreGraphics.h> is a 10.8 umbrella that was included
on new SDKs and ApplicationServices on old ones. ApplicationServices
re-exports the same types and has since 10.0, so it is included
unconditionally and one more SDK test goes.
Left alone: the RARCH_HAS_CGDISPLAYMODE_API blocks around the
resolution list. That gate is about what it says - CGDisplayMode
against the 10.5 CGDisplayAvailableModes dictionaries are genuinely
different APIs with different memory rules - and making it runtime
means writing the second implementation, not deleting a gate.
Not compiled here; there is no Apple toolchain on this machine and
AppKit cannot be stubbed the way the CoreAudio work was. Linux still
builds. This wants a Mac, and ideally a Leopard one.
The last three pacing commits rest on predicates checked with
throwaway models that no longer exist. They are pure functions of
their arguments, they are exactly what a later refactor breaks
quietly, and two of the three failures are silent: a gap limiter that
engages next to another source double-paces and the frontend runs
slow, and one that engages under fast-forward throttles it to 1x -
both read as performance bugs rather than pacing bugs, and neither
trips an assert.
So the three move from runloop.c into runloop.h as inline functions
that runloop.c calls, and samples/runloop/pacing runs those. Not a
copy of the logic: change the header and the test changes with it,
which is the only version of this worth having. They take plain
arguments rather than the runloop state, so the test needs no stubs
and links against nothing.
What it checks. The gap predicate over all 128 combinations of the
five pace bits by nonblocking by fastmotion, asserting that exactly
one of them engages - exhaustive rather than sampled, since the space
is small enough to be. The period over the rates a core can report,
including zero, negative, a thousandth of a hertz and a million, and
for monotonicity, because a clamp written the wrong way round still
passes a bounds check. The sample filter at its boundaries and against
the regression it exists for, by running the same eight-sample average
the runloop keeps: a 0.9 s stall must leave it untouched, a 0.2 s
hitch must still be averaged in.
Confirmed to fail on four plausible regressions, each introduced in
runloop.h and reverted: dropping the fast-forward guard, testing one
pace bit instead of the whole record, inverting the clamp, and
loosening the stall bound back to a second. Each was caught, and the
first line of each failure names the case.
The frontend times the same as before the refactor, 57.8 fps with
both sync options off and 101 with vsync on, and win32 and the
settings table are clean.
Following the measured-rate line to its cause. The runloop sets the
vsync pace bit from the setting, and the previous commit showed the
loop running at 104 fps on a 60 Hz core while claiming vsync was
holding it. This is why: the display reports 0 Hz.
A display with no refresh rate has no vertical blank to present
against - an offscreen or dummy video driver, a remote session, some
virtual machines. What makes it hard to notice is that nothing in the
API says so. SDL grants SDL_RENDERER_PRESENTVSYNC on such a display,
SDL_GetRendererInfo reports it as granted, and the renderer is created
without complaint; the present just returns immediately. So the check
cannot be "did we get the flag we asked for" - we did - and there is
no flag-based fix here, which is worth recording since it is the first
thing anyone will reach for.
The mode is already logged, 0hz and all. Now the conclusion is drawn
next to it, once, when vsync was requested and no display reports a
rate: presentation will not pace the frontend. Silent otherwise.
This changes no behaviour. The pacing gap commit already makes the
loop run at the content rate when nothing claims to pace it, and this
case is one where something does claim to - so a log line, and the
Pacing line's measured rate, is what the user gets. Making the pace
bit itself conditional on a real vblank needs the driver to report
something it currently has no way to know.
The other half of the question the pace record asks, and the loose end
from the previous commit. The bits say which source is holding the
loop. They do not say whether it is: the vsync bit reads the driver's
blocking state, and a driver can be in blocking mode with nothing to
block on - headless SDL2 with no vblank sets it and runs at 104 fps on
a 60 Hz core, and so does an audio driver whose buffer never fills.
"Pacing: VSync" reads as an answer in both cases when it is the
question.
So the loop's own period is measured, smoothed over eight iterations,
and shown next to the claim: "Pacing: VSync (104.2 fps)". A rate near
the content's confirms the named source; a rate well above it says the
claim is empty and the loop is running free, which is the state that
took a field session to notice the first time. One clock read on a
path that already takes several.
Samples over a quarter second are dropped rather than smoothed. They
are not pacing but a stall - a state load, a shader rebuild - and in
testing one 0.9 s sample pulled an eight-sample average from 60 fps to
8 and took several iterations to climb back, so the line would have
lied for a second after every hitch. Nothing that genuinely holds the
loop runs slower than 4 fps.
No behaviour change: the three configurations time the same as before
this commit - 59 fps with both sync options off, and 103 fps with
either switched on, which is precisely the disagreement the line now
makes visible instead of leaving to a stopwatch.
The gap the pace record was built to expose. With vsync off, audio
sync off or not writing, and no scanline lock, the only thing left is
the frame limiter - and it cannot engage, because
frame_limit_minimum_time is zero whenever the fast-forward ratio is
"unlimited", which is the default. The record comes out NONE and the
loop runs as fast as the machine allows. On a 60 Hz core with a light
frame that is 105 fps here, and was 215 fps on the field machine that
turned this up: a burnt CPU core and frames nobody asked for, from a
configuration a user reaches by turning both sync options off.
It is not fast-forward. The user did not ask to run fast; nobody is
saying when the next frame is due. So the timer takes it at the
content's own rate, which is what any of the other sources would have
produced. Only when the record is empty, so it never competes with a
source already doing the job, and never under fast-forward or
FASTMOTION, where running unthrottled is the point.
The period comes from the content rate through a small helper the
no-window path already needed, bounded to 1-100 ms either way: a core
may report a nonsense rate, and neither a spin nor a ten-second stall
is a reasonable reading of one frame.
Measured on a clean build, 900 frames of a 60 Hz core with both sync
options off: 8.56 s before, twice, and 15.12 s after, twice - 105 fps
becoming 59.5. The predicate is checked over all 32 pace combinations
by nonblocking by fastmotion, and the period over seven rates
including zero and a million.
One thing the measurement also showed, which is the environment
rather than the change: with vsync or audio sync switched back on the
runs stay at ~104 fps, because headless SDL2 has no vblank to block on
and ALSA here does not fill. The record is right in both - something
does claim the loop - and this change deliberately does not
second-guess it. Telling "claims to pace" from "actually paced" needs
measurement rather than a flag, and that is a larger change than this
one.
Closing out the retro_sleep() audit. The Wiimote extension-init
sequence sleeps 100 ms between register writes, three times, and the
delay itself is not the problem - the device needs it, and the values
were arrived at against real hardware, clones included.
Who pays for it is. The handshake runs from whichever HID driver
delivered the packet: libusb and wiiusb call it from a per-adapter
poll thread, where a 100 ms wait costs that device's own thread and
nothing else, but iohidmanager schedules its callbacks on
CFRunLoopGetCurrent() at init - the main thread - so on macOS a
Wiimote handshake freezes the frontend for about 300 ms, and btstack's
packet callback is the same shape.
Fixing it means replacing the sleeps with a deadline: record when the
next write may go out, return, and resume from a timer instead of from
the response the sleep currently stands in for. That is a change to a
device protocol sequence, and I have no Wiimote to check it against;
a rewrite that looks right and subtly breaks pairing for clone
controllers is a worse outcome than a 300 ms stall on one platform.
So this commit changes no behaviour at all. It writes the thread
analysis down at the function, so whoever does have the hardware
starts from it rather than working it out again.
The rest of the audit came out clean: the Emscripten audio drivers'
1 ms polls are bounded and say why, the 3DS one likewise, dsound's
mixer thread documents that DirectSound's notifications are not
dependable, and openal prefers a condvar and only polls when there are
no events.
The UWP link failed on an unresolved IsIconic pulled in by
griffin.obj: the presentable() hooks added to the wgl and Windows
Vulkan contexts call it, and UWP has neither the export nor an HWND to
pass it - win32_get_window() returns NULL there, and the stub in
win32_common.c is guarded on _XBOX, which a UWP build is not. The
desktop MSVC projects failed on the same commit for the same reason,
having code-generated the same call.
Both call sites are now behind !__WINRT__. A UWP app's visibility
arrives as CoreWindow events, which these contexts do not see, so
there it reports presentable exactly as it did before the hook existed;
the Vulkan context's swapchain check still applies on both.
win32 syntax clean on both files, and preprocessing each with
__WINRT__ defined leaves no reference to IsIconic at all.
Two defects in the sensor poll thread, both reachable from a core: the
rate is whatever it passes to RETRO_ENVIRONMENT_SET_SENSOR_STATE, and
it arrives here unexamined.
The interval is 1000 / poll_rate in whole milliseconds, so any rate
above 1000 Hz gives zero and the thread reads the sensor as fast as
the CPU allows - and each read is an open/read/close of a sysfs file,
so that is a core of CPU and a hammered filesystem for a device that
cannot answer at that rate anyway. 2000 Hz is not an absurd thing for
a core to ask a sensor API for. The rate is clamped to 1 kHz, at the
two places it enters and again in the loop, so the figure the log
prints is the figure that is used. Zero was already handled, by an
assert in the loop and a default at the entry points; the loop now
defaults rather than asserting, since a release build has no assert
and a division by zero is a worse outcome than 5 Hz.
The loop also tested errno after retro_sleep() without clearing it
first. errno is only meaningful directly after a call that failed, and
the sensor read immediately above goes through open/read/close, each
of which can leave one behind - so an EINTR from something else
entirely would end the sensor thread and the sensor would go quiet for
the rest of the session with a misleading line in the log. Cleared
before the sleep.
Checked against every rate up to 2^20 plus the boundaries and
UINT_MAX: no rate now produces a zero interval, 0 still means 5 Hz,
and anything past 1 kHz becomes 1 ms. Before the patch, 1001 Hz
produced a 0 ms sleep.
dinput_joypad_enum_wait() polls a flag with a 1 ms sleep until an
in-flight pad enumeration finishes, in both the DirectInput and the
XInput hybrid joypad drivers. Two things about it are right and worth
writing down before changing anything, because they look wrong.
It polls rather than waiting on a primitive because the flag is
cleared by the task's own main-thread callback: nothing will clear it
unless this loop keeps calling task_queue_check(). And it is unbounded
on purpose. EnumDevices() walks the HID/PnP tree and a wedged driver
stack can hold it for seconds - the file says so a few lines up, which
is why the enumeration was moved off the startup path in the first
place - and giving up would let the caller free the pad state and the
DirectInput context the enumeration is still walking. A slow exit
traded for a crash is not a trade.
What was actually missing is any sign of it. A user whose Bluetooth
stack is stuck saw the frontend freeze on quit with nothing in the
log. It now warns after a second and every five after that, so a
report of "hangs on exit" arrives with the reason attached.
The loop also pumps before it sleeps, so on the iteration where the
callback runs it leaves without sleeping at all, and it returns
immediately when nothing is in flight. That is not micro-optimisation:
retro_sleep(1) is SleepEx(1) on Windows and rounds up to the timer
period, 15.6 ms by default, so the old shape could add that to every
teardown that had an enumeration finish a moment earlier.
The loop's shape was run against a model with the flag clearing on
various iterations and the sleep costing its real 15.6 ms: it leaves
on the pump that clears the flag, never sleeps when the flag was
already clear, and warns at 1 s then every 5 s. win32 syntax clean on
dinput_joypad.c; xinput_hybrid_joypad.c fails that checker identically
with and without this change, on XInput symbols unrelated to it.
Running the TSan build across more configurations turned up two more
of the same kind as the window title: state shared between the runloop
and the video thread, written under a lock on one side and read
without it on the other.
VIDEO_FLAG_THREAD_WRAPPER_ACTIVE was a bit in video_st->flags, a word
the main thread read-modify-writes for unrelated bits while the video
thread reads this one - from video_driver_thread_wrapper_active(),
reached through video_driver_set_output_size() inside a driver's
frame(). A bit in a shared word cannot be read from another thread
without taking the lock the write side takes, and taking a lock there
would put a mutex on a path called several times a frame. It gets its
own bool instead, written by the main thread only, which is all it
ever needed: it says whether the wrapper is installed, not anything
per-frame. The enumerator stays as _UNUSED so the bit is not handed
out again to something that would inherit the problem.
It was also cleared at the top of video_thread_free(), before the
thread it describes had been joined - a write racing the video
thread's own reads, and a window in which the rest of the frontend was
told the wrapper was gone while its thread was still presenting.
Cleared after the join now.
And at exit, main_exit() cleared DISPGFX_WIDGET_FLAG_PERSISTING while
gfx_widgets_frame() was reading the widget flags on the video thread -
about one run in three. The video thread is brought to a stop first,
with the same video_thread_wait_idle() the driver teardown a few lines
later already uses for the same reason.
Linux-tsan-threaded-video.yml builds with -fsanitize=thread and runs
headless with video_threaded = "true", which nothing else covers: ASan
and TSan cannot share a binary, and the ASan job deliberately runs
unthreaded, so the video thread is never created there and a threading
bug is invisible. It suppresses races inside libSDL2 itself - no
RetroArch frame on either access - and warns if the run produced no
video thread at all, since a green result would then mean nothing.
Soft-fail while the baseline settles, as the ASan job's smoke tier
was.
Five consecutive 400-frame threaded runs clean, unthreaded clean,
win32 syntax clean on the four touched files.
Found by building the frontend with ThreadSanitizer and running it
with threaded video on, which is not something the ASan jobs cover.
video_st->window_title and the VIDEO_FLAG_WINDOW_TITLE_UPDATE bit are
written on the main thread, by the fps block in video_driver_frame(),
and read on the video thread: every driver that sets a window title
does so from inside its frame(), and sdl2_gfx is the one this run
happened to use. Neither side synchronised. The title is assembled in
several statements - a strlcpy, then the separator written a byte at a
time, then a second strlcpy - so a reader could copy it halfway
through and put a torn string in the title bar. Bounded by strlcpy, so
never unsafe, just wrong.
The lock for it was already there and already used by the write side
of the flags word: display_lock, which video_driver_modify_disp_flags()
takes. The read side simply was not covered. Both writers now assemble
the title and set the bit inside it, and the reader tests, copies and
clears inside it, which it has to do together anyway - the bit is a
handoff, and testing it outside the lock lets two readers both see it
set. The bit is cleared inline there rather than through
modify_disp_flags() because slock is not recursive.
Behaviour is unchanged on a build without threads, where the macros
compile to nothing.
TSan is clean on this now, threaded and unthreaded. One race remains
in the run and it is inside libSDL2 itself, with no RetroArch frame on
either access; not ours to fix, but worth knowing it is there when
reading a TSan log of an SDL build.
A correctness hole in the presentable() work. With threaded video the
context belongs to the video thread - it is created, used and torn
down there - and the runloop was calling presentable() on it from the
main thread every iteration. A Vulkan context answers by reading
vk.swapchain, which the video thread writes when it rebuilds the
swapchain; SDL and Cocoa read window state the video thread also
touches. Word-sized reads, so benign in practice, but a data race by
the letter and exactly the kind ThreadSanitizer exists to find.
The wrapper already solved this for alive, focus and has_windowed: the
video thread polls each after every frame and publishes it under
thr->lock for the main thread to read. presentable joins them, with a
getter in the same shape.
The query splits in two so the poll cannot read back its own answer.
video_context_driver_presentable_direct() asks the context and must be
called on the thread that owns it - that is what the wrapper calls, on
the video thread. video_context_driver_presentable() is what everyone
else calls: the published value when the wrapper is active, the direct
question when it is not.
A driver without the hook still reports presentable, and the wrapper's
local starts true, so nothing changes for the twenty context drivers
that do not implement it.
Built and run under ASan with video_threaded both false and true, 300
frames each, clean; win32 syntax clean on the three touched files; the
presentable checker still passes.
The last sleep left in a context driver's swap path. The Vulkan
context slept 10 ms while the compositor reported the surface
suspended, and gfx_ctx_wl_vk_presentable() now reports that same flag,
so the runloop already waits a frame on it - twice-paced, which is the
thing this series set out to remove.
The runloop's wait is the better one on both counts the old sleep was
reaching for. It happens only when nothing else is holding the loop,
so audio still paces a hidden window if it is blocking; and it is
skipped under fast-forward, which is why the sleep was conditional on
swap_interval - a condition that only approximated it, since
swap_interval is not the only way the frontend runs unthrottled.
Everything else in that block stays: skipping the presentation-time
pacing and feedback for a surface that is not being scanned out, and
flushing the display fd so the resume configure is seen.
No context driver sleeps in its swap path now. The one wait left in
the Vulkan common code is the backoff on a second consecutive
VK_ERROR_OUT_OF_DATE_KHR, which is a retry limiter for a different
failure.
The Android build broke on a presentable() that cast data to
android_ctx_data_t, the GL context's type, in the Vulkan context whose
type is android_ctx_data_vk_t. Nothing else could have caught it:
every context driver is compiled by exactly one platform's CI, so a
mistake in one is invisible to the other twenty jobs until that
platform runs.
Two mistakes are easy in this shape and both are silent. Casting the
wrong driver's type is a build break somebody else discovers. Writing
presentable() and never wiring it into the vtable is worse than that -
it compiles everywhere, the hook never runs, and the loop spins in the
state the hook exists to stop, which is precisely the bug nobody
reports.
So, textually and without compiling anything: every presentable() must
cast data to the same type its own swap_buffers() does, and must be
referenced exactly once from a gfx_ctx_driver_t in the same file. A
shared implementation is allowed for - x11_presentable() serves the
three X11 contexts from x11_common.c - by looking for the definition
elsewhere in gfx/ rather than demanding a local one.
The checker has a self-test over three fixtures, and reintroducing the
Android bug makes it fail with the type mismatch named. Fourteen
context drivers pass today.
The Android Vulkan context's presentable() cast to android_ctx_data_t,
which is the GL context's type; this file's is android_ctx_data_vk_t.
A name taken from the wrong driver by the script that added these, and
the only build that compiles the file said so. It also reports
surface_lost now, which is what its own swap_buffers() already checks
and is exactly the state the hook is for: an app in the background has
had its window taken away.
Cocoa gains the hook on both contexts. A miniaturised window has
nothing behind it to present to - -flushBuffer returns at once rather
than blocking to the display's refresh - so with vsync as the only
pacing the loop would spin. AppKit keeps the state, so it is asked of
the window rather than tracked through notifications, and the Vulkan
context keeps its swapchain check behind that for the moment before
the swapchain is rebuilt.
macOS only. On iOS and tvOS the system stops the CADisplayLink when
the app leaves the foreground, so there is no loop to pace, and
UIWindow has no equivalent of -isMiniaturized.
Every presentable() in the tree was then checked against the type its
own swap_buffers() uses and against the struct it dereferences, which
is the check that would have caught the Android one before pushing it.
Fifteen contexts. Linux builds and runs clean under ASan; win32 syntax
clean on the two Windows contexts.
SDL keeps the window state itself, so there is none of ours to keep:
SDL_GetWindowFlags() answers MINIMIZED and HIDDEN directly. A window in
either state has nothing behind it to present to - SDL_GL_SwapWindow()
returns at once rather than blocking to vblank - so with vsync as the
only pacing the loop would spin, which is what the runloop's
presentable() check exists to stop.
Both GL contexts and the SDL3 Vulkan one. The Vulkan context asks the
window first and keeps its swapchain check behind it, for the moment
before the swapchain has been torn down or rebuilt.
SDL 1.2 has no equivalent query - its gfx_ctx_sdl_data holds an
SDL_Surface, not a window - so that path returns true, as it behaved
before.
Thirteen contexts implement the hook now. Linux builds and runs clean
under ASan, and the SDL2 path of sdl_gl_ctx.c compiles clean; the SDL
1.2 path is not compilable here (no SDL 1.2 headers) and fails
identically with and without this change.
The previous commit said X11 could not report presentability because
its map state was folded into g_x11_has_focus, which focus events also
wrote. That was wrong, and reading the file rather than the name shows
it: the flag is written by MapNotify, by UnmapNotify, and once at
input-context creation. Nothing else touches it. There is no
FocusIn/FocusOut handler at all - x11_has_focus() asks the server with
XGetInputFocus() and ANDs the answer with this flag, which is why the
combination worked and why the misnomer survived.
So it is renamed g_x11_mapped, which is what it holds, and the
comment says the two are different things. x11_has_focus_internal()
keeps returning it - the xvideo driver's "is the window usable" test,
which mapped answers correctly.
With that clear, the three X11 contexts - GL, EGL and Vulkan - report
presentable as mapped. An unmapped window is minimised, on another
workspace, or withdrawn; the server discards the drawing and neither
glXSwapBuffers() nor a Vulkan present blocks, so with the display as
the only pacing the loop spins. Unfocused is not unmapped and is
deliberately not tested: a visible window without the keyboard must
keep running at full rate.
Twelve contexts now implement the hook. Linux builds and runs clean
under ASan.
The presentable() hook exists for any context that can tell, not just
Vulkan; a minimised window spins a GL build exactly as it spun a
Vulkan one, and there are more GL builds.
Wayland's GL context already knows: the compositor reports the surface
suspended when it is occluded, minimised or the screen is locked, and
the swap path skips the frame callback and the presentation feedback
in that state for precisely this reason - a suspended surface receives
no frame callbacks, so the throttle that normally paces the loop is
not there. Reporting it lets the runloop pace instead of the swap
running free. Compositors older than xdg_wm_base v6 never send the
state, so this is always true for them, as before.
Windows GL asks IsIconic(). A minimised window has no client area:
SwapBuffers() returns at once instead of blocking to vblank, so with
vsync as the only pacing the loop spins. No state of our own is
needed. The Windows Vulkan context asks the same question in front of
its swapchain check, which covers the moment before the swapchain has
been rebuilt.
Not X11. Its map state is folded into g_x11_has_focus, which
FocusIn/FocusOut also write, so an unfocused but perfectly visible
window would be reported unpresentable and throttled to one frame per
iteration. Reporting presentability from there needs the two to be
separated first, which is a change to x11_common's event handling
rather than to this hook.
Linux builds and runs clean under ASan; win32 syntax clean on both
touched Windows files.
The structural half of the swapchain throttle. Until now a Vulkan
context with no swapchain - a minimised or zero-sized window, a
suspended surface, a create that failed - handled the resulting spin
by sleeping inside swap_buffers() and inside
vulkan_acquire_next_image(). Both sit under the frontend's pacing,
where neither the frame limiter nor the audio block nor the scanline
wait can see them, so the waits stacked with whatever was already
holding the loop; and both were fixed constants unrelated to the
content rate.
gfx_ctx_driver_t gains an optional presentable() at the end of the
struct, so every driver that does not implement it stays NULL without
shifting a vtable slot and keeps the old meaning, always presentable.
The seven Vulkan contexts implement it as "the swapchain exists";
Wayland's also reports false while the compositor says the surface is
suspended, which is the same condition its own documented throttle
tests.
The runloop asks once, where it already builds the pace record, and
waits one frame of content time when nothing else is pacing, the
frontend is not fast-forwarding, and the context says the frame went
nowhere. One wait, at the layer that knows what else is holding the
loop, sized to the core rather than to a constant, and visible: the
statistics overlay's Pacing line reads NoWindow, so a user who wonders
why a hidden window is not spinning a core can see why.
With that in place vulkan_acquire_next_image()'s sleep on a failed
create goes; the runloop covers the same frame. The 10 ms backoff on a
second consecutive VK_ERROR_OUT_OF_DATE_KHR stays - different failure,
and it only fires on a retry.
The predicate is checked exhaustively against all sixteen pace
combinations by two states, and the frame period against six rates
including nonsense ones: it waits only when nothing else paces, and
the wait is always between 1 and 100 ms. Linux builds and runs clean
under ASan, win32 syntax clean on the three touched shared files, the
pacing-adjacent audio harnesses pass.
Not tested against a real minimised window - that wants X11, Wayland
and Win32 in front of a person. The behaviour to look for is a hidden
window costing one core frame per iteration instead of spinning, and
restoring it being noticed within that frame.
An audit of every retro_sleep() in the video, audio, input and task
paths, following the Vulkan swapchain one. Almost all of them are
sound: the runloop's idle sleep has nothing to do, the audio drivers
that sleep-poll say why and prefer an event path where one exists,
dsound's mixer thread documents that DirectSound's notifications are
not dependable, d3d9 backs off three seconds between device-creation
retries at init, and Scanline Sync's sleep-then-spin is deliberate.
One is a defect. network_gfx's init loops on the connect for ever -
socket, connect with a five second timeout, sleep a second, goto - so
a host that is not there hangs the frontend at startup with no way out
but killing it. The video driver never returns and nothing above it
gets a chance to report the failure.
Five attempts and then the driver fails, which the frontend already
knows how to handle. The sleep stays where it is and at a second: the
connect above it already spends up to five seconds per address on a
host that is down, so the wait is really for a host that refuses
instantly - nothing listening on the port - which would otherwise be
hammered.
Also fixes the leak of the network_video_t on that path, which never
mattered before because the path never ended.
Eight Vulkan context drivers slept 10 ms in swap_buffers when the
swapchain was VK_NULL_HANDLE, and vulkan_acquire_next_image(), called
from the very next line of each of them, sleeps again when the create
it then attempts fails. A minimised window therefore cost 30 ms per
iteration from two places that did not know about each other - and
both sit under the frontend's pacing, so whichever of the frame
limiter, the audio block or the scanline wait was active added its own
wait on top of that.
The context-side sleep goes. There is nothing to present without a
swapchain and nothing to wait for: the retry happens in
vulkan_acquire_next_image() and that is where the wait belongs, with
the code that knows whether the create failed.
The remaining one drops from 20 ms to 8. It only has to keep a failing
create from being retried as fast as the CPU allows, and 8 ms is under
one frame at any rate the frontend can be asked to run at, so it never
becomes the thing that paces the loop. It is also under the resolution
of the sleep on Windows, where SleepEx() rounds up to the timer period
- 15.6 ms by default - so a smaller number would not buy anything and
a larger one only delays noticing that the window came back.
Left alone: the 10 ms backoff on the second consecutive
VK_ERROR_OUT_OF_DATE_KHR, which is a retry limiter for a different
failure and only fires on the second attempt; and the Wayland
suspended-surface throttle, which is a deliberate, documented, and
conditional wait for a surface the compositor says is not being
scanned out.
Still a wait in the video driver rather than an event from the window
system - un-minimising is noticed up to 8 ms late - and still below
the layer that ought to be doing the pacing. Both of those want the
context to tell the runloop it has nothing to present, which is a
larger change and belongs with the pacing work.
frontend_darwin_get_os() picked its implementation by deployment
target: NSProcessInfo.operatingSystemVersion when the build targeted
10.13 or later, and otherwise a runtime probe for the same selector
with a Gestalt fallback under it. The property is 10.10, so a binary
built for an older target never used it however new the system it ran
on, and the gate was on MIN_REQUIRED - the deployment target - which
made the choice about who might run it rather than about what is there.
One path now, decided where it runs: the selector if the process has
it, Gestalt if not. Two details make that free of a build-SDK
dependency and free of cost.
operatingSystemVersion returns a struct of three NSIntegers, which
comes back in memory on x86_64 and in registers on arm64 - objc_msgSend
against objc_msgSend_stret - so it goes through NSInvocation, which
gets that right on both without this file choosing. Gestalt is
deprecated since 10.8 and out of the newest SDKs' headers, so it is
resolved with dlsym and its selectors written as the four-character
codes they are, 'sys1' and 'sys2'; a system old enough to need it has
the symbol.
Neither of those is cheap, and neither runs twice: the answer cannot
change while the process runs, so it is cached in statics on the first
call. That matters here - get_os() feeds the menu's system information
list, which is rebuilt every time the list is opened, and the old fast
path was a property read. After the first call this is now two
compares.
Left alone, deliberately: the two AVSpeechSynthesizer gates. Those
guard a class the SDK must declare for the code to compile at all, and
the @available inside each is already the runtime half; making them
dynamic would mean NSClassFromString and message sends for the whole
AVSpeechSynthesizer and AVSpeechUtterance family, which is worse code
for a build configuration nobody uses.
Linux builds. The Apple side is not compiled here.
Three faults in 9a1b0c5's mechanical insertion of the include, all
found by the Apple builds.
The settings_def_*.h fragments are not translation units: msg_hash_us.c
includes them inside a struct body, three times over, to build the
member list, the initialiser and the id table. An #include there is
fatal, and clang said so - extraneous closing brace, redefinition of
uint32_t, and a run of cascading syntax errors. The fragments have no
includes again; the three files that pull them in - configuration.c,
menu_setting.c and msg_hash_us.c - all reach TargetConditionals.h
themselves, the last through msg_hash.h.
In four headers the include landed inside the very #if TARGET_OS_IPHONE
or #if TARGET_OS_TV block it exists to make meaningful, which is
circular: the macro was still undefined where it was tested.
retro_miscellaneous.h, rglgen_private_headers.h, apple_compat.h and
apple_platform.h now include it immediately after their guard.
griffin.c and glsym_es3.h test the macro and had no path to a
definition at all - griffin.c because it is the amalgamation's own root
and includes no header that provides it, glsym_es3.h because it is
included from GL code that does not either.
Checked mechanically rather than by eye, since that is what let the
first two through: no include of TargetConditionals.h sits inside a
conditional that tests TARGET_OS_ anything (0 of 56), the 244 files
reachable from msg_hash_us.h contain no angle include at all, and every
file testing TARGET_OS_IPHONE or TARGET_OS_OSX reaches a definition.
Linux builds and runs clean under ASan.
The companion to the OSX change. The Xcode config passed -DIOS and 126
sites across 40 files tested it.
What -DIOS meant is the important part: OTHER_CFLAGS_IOS_TVOS_SHARE
carries it to iphoneos, iphonesimulator, appletvos and appletvsimulator
alike, so IOS meant "the iPhone family", not iOS as against tvOS.
Apple's name for that is TARGET_OS_IPHONE, which is 1 on iOS, tvOS,
watchOS and their simulators - TARGET_OS_IOS is 0 on tvOS and would
have turned every one of those 126 sites off there. So IOS becomes
TARGET_OS_IPHONE, one for one, and nothing changes which branch it
takes on any target. menu_displaylist.c's "#if !defined(IOS) || !IOS",
which said the same thing twice, becomes one test.
The macro now comes from Apple's header rather than a -D, so
<TargetConditionals.h> is included, under __MACH__ as config.def.h
already did it, by the 42 files that test TARGET_OS_IPHONE or
TARGET_OS_OSX and did not include it. The four settings_def fragments
are not translation units - they are included into configuration.c,
menu_setting.c and msg_hash_us.c, all three of which have it, the last
through msg_hash.h. -DIOS is gone from the Xcode config.
And the simulator, which is the reason to be careful here rather than
after a bug report: iOS SDKs before 9 set TARGET_OS_IPHONE to 0 for a
simulator build and marked it with TARGET_IPHONE_SIMULATOR instead, so
the TARGET_OS_OSX fallback added in the previous commit would have
called an old simulator build a Mac. Both simulator spellings are
checked now, the modern TARGET_OS_SIMULATOR and that one.
The preprocessor matrix from the previous commit grew two rows - an
old iOS-simulator SDK and a modern one - and all thirteen are correct.
Linux builds and runs clean under ASan; the Apple side is not compiled
here.
Both build systems passed -DOSX to say "this is a macOS build" and 95
sites across 24 files tested it. Apple already has a name for that -
TARGET_OS_OSX, from TargetConditionals.h, alongside TARGET_OS_IPHONE
which this tree does use - so the private one goes and Apple's is used
throughout: #ifdef OSX becomes #if TARGET_OS_OSX, #ifndef OSX becomes
#if !TARGET_OS_OSX, defined(OSX) becomes TARGET_OS_OSX. Every TARGET_OS_*
is defined to 0 or 1, so the tests are on the value, never on whether
the macro exists.
TARGET_OS_OSX itself is only in the 10.12 and later SDKs. Older ones get
it from cocoa_defines.h, by the same rule Apple's own header uses: on
Apple, anything that is not the iPhone family is macOS. That needs
nothing but TARGET_OS_IPHONE - which every SDK that has ever had an
iPhone in it defines, and whose absence on a 10.4 or 10.5 SDK that
predates the iPhone means macOS by the same reasoning. tvOS and watchOS
set TARGET_OS_IPHONE and land on the right side. The include of
TargetConditionals.h is under __MACH__, as config.def.h already does it.
The build systems pass -DTARGET_OS_OSX=1 exactly where they passed
-DOSX, for the translation units that include no Apple header at all -
retroarch.c, dir_list.c, menu_displaylist.c and the like. That is the
same reach the old define had, so no file changes which branch it
takes; on a 10.12 or later SDK the header defines the macro identically
and the redefinition is silent.
Checked by preprocessing the header against a stand-in TargetConditionals
for every SDK generation it has to survive: 10.4 with no TARGET_OS_IPHONE
at all, 10.5 with it zero, iPhone SDK 2.0 with it one, 10.6 through 10.11
with no TARGET_OS_OSX, 10.12 and later which define it, a modern iOS SDK,
a modern tvOS SDK, the two oldest and newest of those with the build
system's -D added (the redefinition clean under -Wall -Werror), and a
translation unit with no Apple header at all both with the -D and on a
non-Apple build. Eleven configurations, all correct.
An earlier attempt used a backslash line continuation, which does not
continue across the CRLF endings of that file and silently left the
macro at 0 on macOS; the test matrix is what caught it, and the file
now says so.
Linux builds and runs clean. The Apple side is not compiled here.
The same treatment ui_cocoa.m's activation policy, open panel and
full-screen bit just had, for the rest of the Cocoa code that decided
by the SDK it was compiled against rather than the system it runs on.
Each of these had the failure in both directions: a binary built on an
old SDK lost the capability everywhere, and one built on a new SDK
either crashed or misbehaved on an older system.
The GL and Vulkan contexts had two copies of get_video_size(), one
using -convertRectToBacking: and one not, chosen by
MAC_OS_X_VERSION_10_7 - so a binary built on an old SDK was blurry on
every Retina Mac. One function now, asking the view once and keeping
the answer in a static; the note in the old code about not wanting a
selector lookup per call is honoured, and there was never a reason for
two functions and a third to dispatch between them.
setWantsBestResolutionOpenGLSurface: likewise, asked of the view.
The GL core profile attributes are integers - NSOpenGLPFAOpenGLProfile
is 99, the 3.2 core profile 0x3200, the 4.1 core 0x4100 - so they are
written out and requested unconditionally. A system without the
attribute, or without that profile, fails the pixel format, and the
retry path drops the profile and takes a legacy context; the offline-
renderer retry that was already there now runs after it, since a
caller asking for GL 4 would rather lose offline renderers than the
profile. Before, a 10.7-SDK build simply could not ask for 4.1 and a
10.10-SDK build asked for it on 10.7 and got nil with no fallback.
App Nap's beginActivityWithOptions:reason: and endActivity: are 10.9.
The runtime probe was already there; the SDK gate around it was for
the declarations, so the calls go through objc_msgSend and
performSelector: with NSActivityIdleDisplaySleepDisabled written as
1 << 40, and the gate goes. On a pre-10.9 SDK the screensaver
suppression now works when the binary meets 10.9, instead of being
compiled out.
coretext's kCTFontDefaultOrientation was renamed
kCTFontOrientationDefault in 10.8. Both are zero, so the value goes in
and neither name is spelled.
Not compiled here; wants a Mac build.
The last two build-SDK gates in the file.
The open panel's directory and result moved from paths to URLs in 10.6
- setDirectoryURL: and -URL for -runModalForDirectory:file: and
-filename - and the file picked a generation by whether the build SDK
was 10.6 or later. Now it asks the panel in front of it: setDirectoryURL:
if it answers to it, the path-based pair otherwise. Each generation's
calls go out in a form that compiles whether or not the SDK declares
them - performSelector: where the argument or the result is an object,
objc_msgSend where the result is an integer - and everything else the
panel is told is 10.0 API, so the two branches are now four lines each
rather than two blocks. -filename is deprecated but declared and
working from 10.0 to today.
The full-screen-primary collection behaviour is 10.7, the property
10.5. The constant was already spelled out in the file; the check is
now on NSAppKitVersionNumber rather than the SDK, so a modern binary
run on 10.5 or 10.6 leaves the bit alone instead of setting a
behaviour the system does not implement.
ui_cocoa.m now has no MAC_OS_X_VERSION_10_x conditional at all.
Not compiled here; wants a Mac build.
A bare-binary build starts as a background-only process and never
receives keystrokes until it is promoted to a regular GUI app. The
promotion was chosen at compile time: -setActivationPolicy: when the
build SDK was 10.6 or later, TransformProcessType() otherwise. That
bakes the build machine into the binary - one built on a modern SDK
would send an unrecognised selector on 10.5 - and the fallback was
only ever compiled by the oldest toolchains.
One path now, decided where the binary runs: if NSApp answers to
setActivationPolicy: it is sent, through objc_msgSend with the enum's
value written out so the file needs no SDK declaration of the method;
otherwise the Carbon Process Manager call, declared in every SDK since
10.3, deprecated since 10.9 and still there, with the deprecation
warning quieted at the one call. The ApplicationServices include is
unconditional for the same reason.
Not compiled here; wants a Mac build.
The previous commit marshalled -setTitle: onto the main thread with
dispatch_async under a MAC_OS_X_VERSION_10_6 gate and called directly
below it - correct on 10.6 and later, and the pre-10.6 fallback called
AppKit from the video thread. The gate was there only because GCD was
the tool chosen.
performSelectorOnMainThread:withObject:waitUntilDone: has done this
since 10.0. It signals a run-loop source and the main thread picks the
message up on its next pass, which is what a dispatch to the main queue
does too, and it retains the argument until the selector has run. So:
no gate, one path, the same cost, and the alloc'd title is released
right after enqueueing under MRC - the perform holds its own reference.
The target is the view rather than the window, through a one-line
-setWindowTitle: added to CocoaView for macOS, so that the -window
lookup happens on the main thread beside the set instead of being
evaluated on the video thread to find the receiver.
Not compiled here; wants a Mac build.
An audit of ui_window_cocoa_set_title().
What it had right: data is video_st->display_userdata, which
cocoa_common.m sets to the CocoaView itself, so casting it straight to
the view is correct - even though the six other entry points in the
same table cast data to a ui_window_cocoa_t wrapper and read ->data
through it. Those would misread a view as a struct if called, but only
Win32 calls them, with its own struct; on Cocoa they are dead. Said so
in a comment rather than left as a puzzle. The copy of the title
before dispatching, and the async rather than sync marshalling to the
main thread, were both right and stay.
Two things it had wrong.
The NSString was autoreleased on the calling thread. Under threaded
video that is the video thread, a plain C thread with no autorelease
pool, so under MRC - the Makefile build compiles this file MRC - the
object was leaked on every title change, one per content load. It is
alloc'd now and released inside the block once the title is set, which
the block copy's own retain of the capture balances; under ARC the
release is the no-op macro.
stringWithCString:encoding: returns nil for bytes that are not valid
UTF-8, and -[NSWindow setTitle:] throws on nil. Titles carry content
names, and a filename from a foreign filesystem is whatever bytes it
was written in. Falls back to Latin-1, which takes any byte sequence,
and returns if even that fails.
Not compiled here - no Apple toolchain - so it wants a Mac build.
9fed0dabe1 removed the HAVE_COREAUDIO && HAVE_MICROPHONE block from
griffin_objc.m - the two microphone .m files it included are gone -
but took the block's inner #endif and left its outer one, so every
Xcode target failed at griffin_objc.m:67: #endif without #if. macOS,
iOS and tvOS all build this file and all three fell over on it.
Removed; the file's conditionals are balanced again, checked by
counting them and by running it through the preprocessor.
Two microphone drivers as Objective-C files, 1600 lines between them,
and neither needed to be Objective-C. coreaudio_mic_macos.m had no
Objective-C in it at all - CoreFoundation strings are C. The iOS one
had a single block: the AVAudioSession put into a record category
with the session's rate read back. That is the Cocoa layer's object,
not the driver's, and now lives there as
cocoa_audio_session_begin_record() in ui_cocoatouch.m, declared in a
plain C header; the driver calls it under TARGET_OS_IPHONE and stays C.
The two drivers were the same driver twice: an input-enabled output
unit, HALOutput or RemoteIO, whose input callback pulls each slice
with AudioUnitRender into a fifo the frontend reads, with a lock the
callback only try-locks and a timed condition wait for the reader. The
merged one is that once, ~650 lines at the end of coreaudio.c, opening
its unit through the same runtime-resolved component API as playback.
macOS keeps everything it had: input devices listed with their UIDs
riding in attr.p, a saved setting matched by UID first and name second,
the default input device followed through a HAL property listener with
the reconnect done on the reader's thread, a 256-frame device period,
the client format at the device's own rate. iOS and tvOS keep the
session category with the options and reasons they had. Both gain the
other's fixes: the driver context is its own allocation, latching the
non-blocking state so a mic opened later inherits it, and open() no
longer starts the unit - start() does, as the frontend expects and as
macOS already did; the iOS driver had started at open and again at
start.
Griffin loses the two includes, Makefile.common the object and the
AVFoundation link the macOS microphone never needed. C90: every
declaration at the top of its block, checked as gnu89 with mixed
declarations an error, for macOS and for iOS.
Verified on Linux with the real file linked against stubs of the Apple
calls and a capture thread handing over a sample ramp: the device list
with UIDs; open by UID, by name and by default; 8 KB of blocking reads
with the ramp intact; the default device changing under a running mic
and the reconnect landing on the new one with the mic still alive;
non-blocking reads; stop and close, under ASan and UBSan. The earlier
playback harnesses run unchanged. Wants a Mac and an iOS build; the
Cocoa function is not compiled here.
The driver pulled in <Accelerate/Accelerate.h> - the whole umbrella,
BLAS, LAPACK and the Sparse headers whose Nb collided with the AES
code in griffin and needed an #undef - and linked the framework, for
one call: vDSP_vsmul scaling a few thousand floats a frame by the
volume. A loop does that in microseconds and the compiler vectorises
it. Accelerate exists from 10.3, so this was not a 10.4 break, but it
was a framework, a header umbrella and a workaround for nothing; the
driver now needs AudioToolbox and CoreAudio and no more, and the
HAVE_COREAUDIO link line drops -framework Accelerate. The AVFoundation
camera still links it for its own vDSP use.
TargetConditionals.h defines every TARGET_OS_* macro to 0 or 1, so
#ifdef TARGET_OS_IPHONE is true on a macOS SDK too and takes the iOS
branch there. Two sites tested the definition; the rest of the tree
tests the value.
coreaudio.c: the stereo channel layout for the HAL unit was under
#ifndef, which on a macOS SDK that defines the macro as 0 meant it
was never set. The site is #if !TARGET_OS_IPHONE now, TargetConditionals
is included explicitly with the convention stated once, and the layout
is advisory - the two-channel stream format above it already fixes the
channel count - so a HAL that declines it is logged rather than
failing the open, which it would have done the day it started being
set again.
gl_capabilities.c: GL_CAPS_TEX_STORAGE_EXT returned false under
#ifdef TARGET_OS_IPHONE, which on a macOS build that has the macro
defined disabled EXT_texture_storage on the desktop. It tests the
value now, with TargetConditionals included under __APPLE__.
Checked with the macro defined as 0 and as 1 in both files; the
coreaudio harness runs unchanged.
Two playback drivers for the same hardware: coreaudio.c, C, on the
AudioUnit C API, and coreaudio3.m, Objective-C with ARC and blocks, on
AUAudioUnit and AVAudioFormat, which the Xcode builds defaulted to.
Side by side they differed in what each had missed rather than in
what either did better: 3 had no device list, ignored the device
argument, had no wait_writable() so the threaded pipeline could not
host on it, read its ring a sample at a time with a divide each, and
signalled its dispatch semaphore from every callback with no waiter
count, the pile-up fixed in coreaudio.c three commits ago. AUAudioUnit
is a wrapper over the same I/O unit and DefaultOutput can insert a
sample-rate converter that HALOutput at the device's nominal rate does
not; nothing in 3 was faster or lower in latency, and dispatch made
the file a libdispatch client for a semaphore the kernel already has.
coreaudio.c is the driver. From 3 it takes the three things worth
taking: the converter is rebuilt when the source rate changes or rate
control moves its adjustment by more than half a percent, not only
when the effective rate has drifted that far; the stream format set
is retried once, since RemoteIO has been seen to refuse the first;
and stop()/start() double as the two ends of an audio session
interruption, which is what the Cocoa side already calls at Began and
Ended - start() resets the converter, since one stopped mid-stream
can come back stuck at end-of-stream, the tvOS 13/14 symptom. The
Cocoa handler now reactivates the session before restarting and
honours ShouldResume, without which the units were restarted into a
session the system had deactivated.
The last SDK gate goes with it. Component versus AudioComponent was a
build-time choice on HAS_MACOSX_10_12, baking the build machine into
what the binary could open; both triples are resolved once at runtime
through dlsym, the modern one if the process has it and the Component
Manager's otherwise, with no header from either side needed since the
description structs share a layout and the handles are pointers. The
HAL device query asks AudioObjectHasProperty whether the output scope
answers rather than assuming from the SDK, through one helper both
the device list and the named open now share. Only TARGET_OS_IPHONE
remains, which is an ABI split and not a version.
"coreaudio3" is accepted as an alias of "coreaudio" in the driver
lookup for a release, so a config naming it lands on this driver and
not on whatever is first in the table. HAVE_COREAUDIO3 is gone from
the Makefiles, qb, griffin, the feature list, the menu, the label
tables and the Xcode config - where its OTHER_CFLAGS line had been
overriding the HAVE_COREAUDIO one above it, so Xcode builds had only
ever compiled 3.
Verified on Linux with the real coreaudio.c linked against stubs of
the Apple calls and a render thread at the device rate: the device
list and the named open through both the AudioComponent and the
Component Manager triples, the converter built once across 200 calls
with rate control jittering by a tenth of a percent and once more for
a two percent step, stop and start, and the earlier wait harness -
no stale-signal storm, blocking writes at the device's pace. Syntax
under gnu89 with mixed declarations an error and gnu99, macOS and
iOS. Linux and win32 builds of the touched frontend files clean; the
frontend runs under ASan. Wants a Mac and an iOS build before it is
trusted; the Cocoa change in particular is not compiled here.
coreaudio.c stopped carrying a RARCH_COREAUDIO_LEGACY fallback in
46781ec: every Apple toolchain it builds on gets a real backend from
this header, OSAtomic on the oldest, so it uses the atomics
unconditionally. Pattern 1's example now says so, and says when the
pattern is still the right one - a target that genuinely lands in the
volatile fallback.
An audit of the wait, since consolidation made it the one primitive
the driver has.
The wait itself is not a sleep. coreaudio_wait_ms() blocked on the
Mach semaphore the render callback signals after every pull; the
100 ms was a ceiling for a unit that has stopped rendering and will
never signal, and in play the writer woke when the callback freed
space, after the kernel's wake latency. That is the floor on Darwin -
pthread_cond, dispatch_semaphore and os_unfair_lock's waiters all end
in this same wait - and no primitive with the compatibility this file
keeps wakes sooner.
What the audit found was on the other side. Every callback signalled
whether a writer waited or not, and a bare Mach semaphore counts: with
the writer non-blocking - audio sync off, or the threaded pipeline
between frames - the count climbed by one per callback without bound,
and the next real wait returned at once, and again, until the lap cap
tripped and wait_writable() reported no space coming for a ring that
had plenty. The frontend drops what it cannot deliver. Run on Linux
against a fake render thread at the device rate, 3 s non-blocking and
then 120 blocking waits: 34 of the 120 came back empty. The GCD
version had the same latent defect; dispatch_semaphore counts too.
A waiter count, as dispatch keeps in userspace: the writer raises it,
rechecks the ring - a callback that ran before the raise did not
signal, and the recheck sees the space it freed - waits, and lowers
it; the callback signals only while it is non-zero. At most one stale
count can remain, which the loop around the wait absorbs. Both sides
pair an acq_rel read-modify-write with an acquire load, which no
backend reorders. The same run: 0 of 120 empty, and 120 blocking
frame writes take 2.02 s against 2.00 s of device time. The Mach call
per callback with no writer waiting goes too.
The three sites pass what they wait for: one sample of room for the
two write loops, the capped request for wait_writable().
The file carried two drivers under RARCH_COREAUDIO_LEGACY: the current
one - lock-free ring, AudioConverter resampler, GCD semaphore - and the
pre-rewrite one restored for Xcode 3.1 on Tiger and Leopard, with a
mutex-guarded fifo_buffer_t and scond. Two copies of init, of the
render callback, of write, stop, start, device enumeration; every fix
since has had to land twice or, as with the timed waits and the lap
caps, land in one and be remembered for the other.
What the legacy copy actually needed from the old toolchains was
narrower than a second driver, and on inspection nothing. The
lock-free ring rests on retro_atomic, which picks OSAtomic on those
toolchains - real atomics with barriers, present since 10.4.
AudioConverter and vDSP_vsmul are in AudioToolbox and Accelerate,
which those SDKs ship and which the Makefile links whenever
HAVE_COREAUDIO is set, legacy or not. The one thing the old SDKs lack
is Grand Central Dispatch, and all the driver took from it was a
semaphore with a timed wait for the writer to sleep on between
callbacks. That primitive is in the kernel since 10.0 as a Mach
semaphore - semaphore_signal is lock-free and safe from the real-time
render thread, semaphore_timedwait is the timed sleep - and
dispatch_semaphore is a userspace counter over exactly it, saving a
syscall only when no one waits. At one signal per render callback the
saving is nothing; the gate it cost was a whole second driver. The
writer now sleeps on a Mach semaphore on every toolchain, behind
coreaudio_signal() and coreaudio_wait_ms(), and the file has no SDK
gate left: RARCH_COREAUDIO_LEGACY is gone, dispatch.h and rthreads.h
with it.
On performance. Measured uncontended on x86-64, a mutex-guarded fifo
and the atomic ring cost the same per write-and-read of 512 floats:
50 ns against 65, two lock-prefixed adds against two mutex operations,
a wash. The difference that matters is not throughput but who can
block whom: with a mutex the render callback - a real-time thread with
a deadline - can be made to wait on a normal-priority writer in the
middle of a memcpy, and the legacy callback took that mutex. Now
nothing in the callback can block: the ring is lock-free and the
signal is a Mach call that never waits.
The converter's quality variable moves above the statements so the
function is C90; Xcode 3.1's GCC accepts mixed declarations as an
extension, and there is no reason to lean on it.
Not compiled with an Apple toolchain here. Syntax-checked against a
stub of every CoreAudio, AudioUnit, AudioToolbox, Accelerate and Mach
declaration the file uses, in all four configurations - the Component
and AudioComponent API generations, macOS and iOS - as gnu89 with
mixed declarations an error and as gnu99. Wants a Mac build, and
ideally a Tiger or Leopard one, before it is trusted.
The maintainer of Scanline Sync reports the phase-lock rebuild broken
on his setup and the tree at 302b9546ea working; he has ideas to take
that version forward and none for the new one. Restored at his request.
What returns is the tuner as it stood at 302b9546ea: the beam polled
and a target line nudged by one, two and four with its hold and re-arm
logic, the sleep sized from frame time and core time, the line count
learned from the peak reading. Gone with the phase lock: the vblank
anchor and calibration (c9260c895d), the blanking aim and the
release-to-present measurement (83762306a5, 6abbdb7), the visible line
count from the display mode (c993b7ac34), the VRR disengagement
(985debc1a4), the user offset with its setting, menu row, label string
and table entry (49f07a52ee, 2d6caa73dd, fc23f85979, 6869b9df65,
a9d3b1ca58), and the runloop's audio yielding to the scanline wait
(e88eea9911), which existed for the rebuild's blocking wait and has no
counterpart in this tuner.
What stays is the abstraction from 83762306a5: the tuner reads the beam
through video_display_server_get_scanline() rather than d3dkmt
directly, so video_driver.c carries no HAVE_D3DKMT and a server without
the hook reports -1, which the tuner already treats as unsupported. The
win32 server keeps wait_vblank() beside it for whoever takes the
feature on next; nothing calls it now. The pace record and its overlay
row are runloop work and stay.
The six packed translation headers that had picked up the offset's
label and sublabel from Crowdin are edited to drop the two rows - the
JSON sources are not in the tree - and each still passes its own
contiguity check at compile time. Linux C89 and win32 syntax checks
clean; frontend builds and runs under ASan.
f015d7bc06 hoisted the two implementations' includes into one block
that takes AudioToolbox on iOS and CoreAudio on macOS. The modern path
had included AudioToolbox on both - its resampler is an AudioConverter,
declared there and not in CoreAudio - so the macOS build lost
AudioConverterRef, AudioConverterNew and the rest with it. Included
beside dispatch.h, where the modern path's own includes begin.
The three remaining direct users of C11 atomics under audio/drivers on
Apple, brought onto retro_atomic like coreaudio3 and the rest of the
tree: one header, one backend choice, one set of memory orders.
coreaudio.c: the ring's filled count becomes retro_atomic_size_t; the
acquire loads and the add/sub stay what they were, the add/sub as
acq_rel where they were release, which on both x86-64 and ARM64 is the
same instruction. The comments naming C11 atomics say retro_atomic.
coreaudio_mic_macos.m: the three atomic_bools - is_running,
is_initialized, device_changed - become retro_atomic_int_t holding 0
or 1, with every load an acquire and every store a release. That is
stronger than the relaxed load in the capture callback and the relaxed
clear of device_changed, by one ldar/stlr each, once per callback;
the rest were already seq_cst and are not weakened.
coreaudio_mic_ios.m included the header and used nothing from it.
Verified: the coreaudio.c ring compiles and runs against retro_atomic
under ASan; every macro these files use is defined by every backend in
retro_atomic.h. Not compiled as a whole here - no Apple toolchain - so
it wants a Mac build before pushing, like the coreaudio3 change.
Two CI failures from the last two commits.
ThreadSanitizer: the pump thread read the EXCLUSIVE bit from w->flags,
one byte that start(), stop() and set_nonblock_state() write from other
threads while the pump runs. A read of one bit against a write of
another in the same byte is a race by the letter, and since the pump
now starts in init() it is running when set_nonblock_state() is first
called. The pump reads a bool of its own instead, set at init before
the thread exists and never written after.
C89: the Sink Rate Estimation sublabel was 665 characters against the
509 an ISO C90 compiler is required to accept, and the C89 build has
-Werror=overlength-strings. Cut to 503 saying the same things.
Two things in the ring buffer between the audio thread and the render
callback.
The reader's length load was relaxed. The writer publishes samples
with a release-add on len; a relaxed load on the other side does not
order the buffer reads that follow it after the length on ARM64, so
the callback could observe the new length and still read stale samples
at the wrap. x86 gives the acquire for free, which is why it never
showed there; Apple silicon does not. Through retro_atomic the load is
an acquire - ldar for ldr, once per callback - and the read-modify-
writes compile to the same instruction as before on both: lock add on
x86-64, ldaddal on ARMv8.1. Disassembled with GCC 13 at -O2 for both
to check that this is a change in ordering and not in cost.
And every sample advanced the read pointer with (p + 1) % cap, where
cap is latency * rate / 1000 and no power of two, so each sample in the
render callback paid a 64-bit udiv and msub: 96,000 divides a second
at 48 kHz stereo, in the one thread that must never be late. A
conditional reset wraps it; the write side's advance by n, with n
never above cap, wraps with one conditional subtract.
The ring is verified against a sequence through 3000 random write and
read sizes at a 4801-float capacity, wrapping thousands of times, with
no error under ASan; on ARM64 the interleaved read compiles with no
divide. The Objective-C translation unit itself is not compiled here -
no Apple toolchain - and wants a build on a Mac before it is pushed.
With the threaded pipeline off there was no sound from wasapi at all.
The pump thread - the thing that moves the fifo into the device since
the writer stopped feeding it directly - was created only in start().
The threaded pipeline and the threaded driver call start() from their
wrapper thread and the pump came up. The synchronous path does not:
audio_driver_init() starts the client in init() and leaves start() to
the runloop, which issues it only around pause, menu and thread-wait
transitions. So the client ran, the fifo filled, nothing drained, and
the non-blocking writer dropped everything it was offered. Before the
pump the writer fed the device itself and no start() was needed, which
is why the path had worked.
Bring the pump up in init() beside IAudioClient::Start, as start() does,
and join it on init's error path before the event and client it waits
on are released.
The pacing harness runs an exclusive 8 ms and a shared 64 ms scenario
that never call start(): without the fix 666 of 667 periods go
unanswered and 99.6% of the audio is dropped; with it, none.
Rate control steers the buffer's fill: a fast, proportional term, and
the only one, so a constant difference between the device's crystal
and the host's - tens of parts per million, hundreds on a bad one -
was carried as a constant fill offset, or, with rate control off, as a
drift that ended in an underrun or overrun, a click every so many
minutes that no buffer size removes. GroovyMAME's emusync makes the
case for measuring the sink's rate instead; this is that, as a slow
integral term beside the existing one.
A driver may report frames_consumed(): what the device has taken since
start, on its own clock. Every four seconds the frontend closes a
window and reads two counts over it against the host clock: frames the
device consumed, and frames the frontend offered - each divided by the
resampling ratio in force when it was, rate control's adjustment and
the bias both, so the sum is what the source produces at the nominal
rate. A window in which either side ran more than two percent off the
nominal rate is excluded, not averaged in: a pause, a save state
loading, a menu, a core stalling are all shorter than a stall gate
would catch, and every one of them put the device ahead of the source
and read as a fast clock. Kept windows are summed, and once thirty
seconds of them have accumulated the bias - the factor multiplied into
the resampling ratio - is set to their ratio, and every thirty seconds
after. With rate control off the baseline keeps growing, so a count
that steps by whole periods (480 frames on a shared WASAPI engine) is
known better and better: 330 ppm of noise at thirty seconds, 33 at
five minutes. Clamped to two tenths of a percent, three and a half
cents, which covers any crystal and no ear notices.
Offered, not accepted: a non-blocking writer against a buffer smaller
than its bursts has frames refused, and a ratio built on accepted
frames read the refusal as a fast device and sped the resampler up to
match, which refused more - +1867 ppm against a device measured at
+11. Refused frames are counted apart and warned about once, above a
tenth of a percent, since dropped audio is the thing the user would
want told. Rate control's mean adjustment is not consulted either: with
a non-blocking writer the fill sits pinned full and it says "slow" by
its whole delta forever, the buffer's state and not the clock's.
The bias is applied only when the writer does not block. With audio
sync on the frame limiter stands down and blocking is the pace: the
core runs exactly as fast as the resampler drains into the device, so
the source's rate is the device's divided by the ratio in force and
carries no clock - a bias set from it fed back, the source climbing
+2300 to +3450 ppm with the bias at the clamp and the game a third of
a percent fast. Blocking already locks the core to the device; there
is nothing to correct. Non-blocking - the frame timer or the display
pacing the core - is where the drift lives and where the bias acts.
The rate is measured and shown in either mode.
The log line says both rates - the device's and the source's - so when
the bias disagrees with the device's ppm it says which side moved. In
the field that settled the last question: a bias saturating at +2000
against a device at +11 was the frontend dropping 0.14% of frames under
timer pacing, 800 frames of audio never produced per drop, and the
estimator correctly compensating so the buffer does not drain. The
statistics overlay shows both as Sink and Source rows under Buffer.
Drivers. alsa reports frames written less snd_pcm_delay(). wasapi's
pump counts one device period per event in exclusive mode and one
engine period per event in shared - the event is the clock; released
less padding stopped counting during an underrun, when the engine
plays silence of its own, and read the same pin 400 to 1000 ppm slow
that exclusive read at +11. asio's callback counts its periods. dsound
unwraps the play cursor in its mixer thread. xaudio does not report:
XAudio2 exposes no device clock - SamplesPlayed counts our samples
rendered and pauses whenever the voice runs dry, for a moment between
bursts even with a blocking writer, and read the same pin -1480
climbing to -754 ppm over a growing baseline, a count being diluted. The threaded pipeline's wrapper forwards
the call to the driver it wraps - without that the estimate saw NULL
and never ran on the configuration every reporter uses. Every other
driver leaves it NULL and is as before.
audio_sink_rate_estimation, on by default, turns it off; the row is
under Audio > Synchronization with a sublabel that says what it
measures and why.
Verified on one HDMI pin through ASIO4ALL, WASAPI exclusive at 16 and
8 ms and WASAPI shared: +11 ppm from all of them once the counts were
right. samples/audio/sink_rate drives the estimator on a synthetic
clock: a device counting in 480-frame periods and a writer refusing
half a percent, a 120 ppm device found within 40 ppm after five
minutes with or without refusals, 5000 ppm stopping at the clamp, rate
control pinned at its delta not leaking in, a half-second pause every
ten seconds excluded, a stall and the setting off moving nothing.
wasapi's count advances at the fake device's rate on the pacing
harness's real clock. ASan/UBSan/LSan clean; the frontend clean under
the pipeline; the Windows toolchain clean for both XAudio2 header
paths.
Shared rows at 32 and 24 ms join the run, and every shared row with
the fifo at the setting checks what the driver reports against the
setting, or against the floor - 20 ms of fifo before the engine buffer
the endpoint handed back - where the setting is under it. The fake's
endpoint buffer is in its stats for that. Caught the 66 ms the reporter
saw at every setting.
The floor under the shared fifo was written as two engine buffers. On
the legacy path the engine buffer is itself two periods, and an
endpoint that hands back 22 ms for a 20 ms request made the floor 44
ms: every setting from 16 to 64 became a 44 ms fifo before a 22 ms
engine, 66 ms reported at each, in the reporter's log. The floor is
two periods, and never under 20 ms for the writer's frame-sized burst
with audio sync off; above it the fifo is the setting less the
engine. On that endpoint 64 ms reads as 64 now; 32 and below read as
42, the honest floor there: a frame of fifo before the engine the
endpoint insists on.
The fake's legacy engine buffer rounds up as Windows does, and the
harness runs 32 and 24 ms shared as well; the run showed the 66 before
this and passes after. Clean under ASan/UBSan/LSan and TSan;
syntax-checked on the Windows toolchain.
Shared mode on the fake opens as int16 - the stub settings ask for no
float - and the report divided by the float frame size, halving the
reported buffer it printed. It divides by what use_float() said.
VERIFIED AGAINST THE FAKE DEVICE; not yet run on Windows.
On the legacy shared path - the one an endpoint that offers a single
engine period leaves the driver on, as the reporter's NVIDIA HDMI
output does with default, fundamental, minimum and maximum all 480
frames - the engine buffer was the whole latency setting and the fifo
in front of it the same size again: 128 ms reported against 64, in
the reporter's log. The engine was deep because the writer fed it
only when the fifo was full and had to cover its own frame-sized
cadence; the pump feeds it every period now, so two periods of engine
buffer cover it, and the fifo holds the setting less the engine,
never under two engine buffers so the writer's burst fits. A 64 ms
setting is a 44 ms fifo before a 20 ms engine and reads as 64; under
IAudioClient3 at 3 ms the engine is 9 ms and the fifo 55. The stages
are logged, as the exclusive ones are.
Against the fake, shared at the setting, at 800 frames, under
IAudioClient3 at 3 ms at 64 and 32 ms, and with the engine locked:
nothing dropped, no period unanswered. Clean under ASan/UBSan/LSan and
TSan. Syntax-checked on the Windows toolchain.
The fake's IAudioClient hands out an IAudioClient3 when configured
with a minimum engine period: GetSharedModeEnginePeriod reports it,
InitializeSharedAudioStream opens a three-period engine buffer at the
period asked, and a configured locked period refuses any other with
AUDCLNT_E_ENGINE_PERIODICITY_LOCKED, GetCurrentSharedModeEnginePeriod
reporting it. Three scenarios join the run: shared at 3 ms at 64 and
32 ms settings, and the engine locked at 10 ms; each checks the
engine period the driver ended up at as well as the pacing. The first
two showed the path starving before the pump fed shared mode.
VERIFIED AGAINST THE FAKE DEVICE, NOT REAL HARDWARE; the exclusive
half of this design has since been confirmed clean on a Topping USB
DAC. Shared mode under IAudioClient3 has not been run on Windows.
The IAudioClient3 shared path - the engine at its minimum period, 3
ms on a capable endpoint, in place of the 10 ms default - was in the
tree and never taken on the reporter's machine, and every reason it
might not be was logged at DEBUG or not at all: the block is under
__IAudioClient3_INTERFACE_DEFINED__, and an SDK too old to define it
compiled the path out silently. Every way out is at INFO now, that one
included, with the engine's four period figures when it answers. And
another stream holding the engine at its period, which returned
AUDCLNT_E_ENGINE_PERIODICITY_LOCKED and dropped the driver to the
legacy path, is joined at that period instead.
Then the harness, given an IAudioClient3 engine at 3 ms, showed the
path would have starved once taken: 46% of periods unanswered, 44% of
the audio dropped. The engine buffer there is three periods, and the
writer's frame-sized cadence cannot keep it fed - the same defect the
pump thread fixed for exclusive mode, which under the legacy engine's
setting-sized buffer never showed. The pump feeds shared mode too:
each engine event, what the engine has room for; the writer fills the
fifo only, in both modes, and wait_writable() and write_avail() go
through the same lock. Against the fake: IAudioClient3 at 3 ms, no
period unanswered at 64 ms and eight in a thousand at 32; the locked
engine joined at 10 ms and fed; every earlier row unchanged.
Clean under ASan/UBSan/LSan and TSan. Syntax-checked on the Windows
toolchain.
The driver's text, unmodified but for its three Windows include
lines, built on Linux against a fake IMMDevice/IAudioClient/
IAudioRenderClient with the vtable shapes the driver's macros
dereference. Exclusive and event-driven: the endpoint buffer is one
period; each period the device takes the buffer released for it or
counts the period unanswered, then raises the event. Shared: a
padding that drains a period per period. Events are pthread condvars
with auto-reset. The writer is a frontend as the reporter runs it -
audio sync off, so non-blocking; one frame of core audio, 800 frames,
every 16.68 ms; no rate control - and the harness counts what the
driver did not take and what the device did not get.
make WASAPI_SRC=<file> runs any wasapi.c against the model: the
driver before this series, 3-10% of periods unanswered; the first
smaller-period attempt, 75%. On the audio samples workflow under ASan
and then TSan; PACING_TRACE=1 prints the unanswered count each second.
WRITTEN AND VERIFIED AGAINST A FAKE DEVICE, NOT REAL HARDWARE - see
samples/audio/wasapi_exclusive_pacing. To be run by the maintainer
before it lands.
Two defects, one cause. Whoever called into the driver was the only
thing that fed the device, and only when the fifo was full. In
exclusive mode - event-driven, the endpoint buffer one period - that
was every period while the fifo was one engine buffer, by accident;
in shared mode the engine ran dry while a fifo larger than the
writer's burst filled, and the one fifo size that worked with audio
sync off was a frame exactly, 800 frames, which a reporter found by
hand. With audio sync off the writer never waits, so between its calls
nothing answered at all. The first attempt at a smaller exclusive
period, without changing the feeding, dropped 74% of the audio.
Exclusive mode now has a pump thread that owns the device's event: a
period from the fifo each time, or silence and an underrun counted;
the writer touches only the fifo, under a lock; wait_writable() waits
on the pump. The pump starts before the stream so it is waiting for
the first event, at time-critical priority as the wrapper's audio
thread runs. Shared mode moves what the engine has room for from the
fifo on every write, not only when the fifo is full. And the sizing:
the exclusive period is a quarter of the setting, floored at the
device's minimum and never above the setting, retried at the whole
setting if refused; the fifo holds the setting, two periods at least,
and is what buffer_size() reports.
Against the fake device on a real-time clock, under the reporter's
settings - audio sync off, one frame of core audio per non-blocking
write, no rate control - at 3 ms minimum period and 48 kHz:
before: exclusive 16/32/64 ms 4% dropped and 2-4% of periods
unanswered; shared at the setting 6% and 10%; shared at
800 frames 0.3% and none.
after: exclusive 32/64 ms and shared at any size: nothing dropped,
no period unanswered. Exclusive 16 ms: 4% dropped and a
period in fifty unanswered - a 16 ms buffer against a 16.7
ms frame with no sync, which no driver can hold; the init
log now says to set 20 ms or more.
Clean under ASan/UBSan/LSan and TSan. Syntax-checked on the Windows
toolchain. The event handle no longer leaks when free's wait times
out.
Not run on Windows.
A chained Ogg whose links disagree on rate or channels plays its first
link and stops - one voice is built around what the first link reported,
so the rest cannot come out at the right speed. The length was meant to
follow: bound it to the first link, and what info() states is what the
file plays.
It never did. The bound came from rvorbis_stream_length_in_samples on
the packet-fed handle, which walks Ogg pages in a buffer and has none,
so it answered zero and the bound stayed at the sum over every link - a
length such a file never reaches. A stereo track with a mono one chained
behind it reported 130.44 s and played 74.30.
The page survey already walks every link's final granule to sum them, so
it hands back the first link's on its own and the mismatch path uses
that. The same file now reports 74.30 s, which is what comes out.
Only the mismatch path changes. Files whose links agree, and files with
one link, are bounded by the same sum as before - verified against the
seven tracks in this repository, singly and chained, where the length
reported and the frames emitted agree in every case, and decoding each
track alone is byte-identical to before.
A chained file is several complete logical bitstreams one after
another. The streaming decoder stopped at the first one's end-of-stream
page, so a chained file played its first link and nothing else - and
said nothing about it, because the file had ended as far as the parser
was concerned.
It plays through now. An EOS page ends the link rather than the file;
the parser keeps looking for a page, and a page carrying the
beginning-of-stream flag opens the next link, tearing down the decoder
built from the old link's codebooks and rebuilding it from the new
headers. Only running out of input ends the file.
Granules restart at zero in each link, so positions are carried as the
frames earlier links handed out plus the current granule, and the
end-of-stream granule that trims the final packet's overhang is banked
per link. That accounting happens when the next link opens, not when
one ends: what a link has yet to hand out still belongs to it, and
clearing it early truncates every file by one packet - which is what
the first version of this did, and what decoding an unchained file and
comparing it against the previous decoder caught.
A link that disagrees with the first on channel count ends the stream
instead of changing format underneath a caller that sized its buffers
and picked its mixer from what the first link reported.
Verified by concatenating the seven Ogg tracks in this repository, in
pairs and all seven, and comparing the chain's decode against the
tracks' own decodes: bit-exact, in order, with the total exactly the sum
- 23471717 frames for the seven, which is what audio_transfer's page
survey independently reports. Decoding each track alone is bit-identical
to the previous decoder, byte for byte.
One caveat is in the header comment: nothing in this interface says "no
more input is coming", so a link boundary that lands on the end of the
fed input reads as the end of the file. Callers that hand over a whole
file at once, which is every caller in tree, are unaffected.
The whole-file interface is unchanged: rvorbis_open_memory still decodes
the first link, and its length and seeks answer for that link alone.
This reverts commit 44fcf7523aebd5b6cc4bba21e9be76e4d9eb8fe8.
The window given to ASIOInit was RetroArch's own, in place of the
foreground or desktop window. Under the threaded pipeline ASIOInit
runs on the audio thread while the main thread - which owns that
window and pumps its messages - waits for the audio thread's init to
finish. A driver that sends a message to the window it was given from
inside its init then waits on a thread that is waiting on it. ASIO4ALL
does: on the reporter's machine RetroArch started, ASIO4ALL raised its
control panel, no window was ever drawn, and the process had to be
killed. The same window, destroyed at exit before the parked instance
is released, is the likeliest cause of the exit that no longer
completed. The foreground/desktop handle is owned by a thread that is
never waiting on RetroArch, which is why it worked; it goes back.
The change was made blind, on the plan's claim that some drivers need
the host's window; that claim was not tested against the way this
host initialises, and should not have been landed without being.
audio_driver_flush() consumed reinit_request and ran
CMD_EVENT_AUDIO_REINIT on whatever thread called it. Under the
threaded pipeline that is the audio thread, and the reinit tears the
audio driver down, which stops and joins the audio thread: it joined
itself, and the frontend hung. Reachable by any driver that asks for a
reinit while the pipeline is on - WASAPI on a device change, latent;
ASIO on a buffer size change from its control panel, reported the
first time that path ran on a real device.
The producer, audio_driver_submit(), runs on the core's thread, which
is the one that can stop and join the audio thread: it takes the
request there when the pipeline is on, dropping the batch as the
inline path drops it, and flush() acts on the request only inline,
where it runs on that thread already.
Reproduced: alsa patched, for the run only, to raise the request from
its write path; the frontend before this hung on the first request
and was killed at 20 s, and with it reinitialises - the driver goes
down and comes back up in the log - and the run completes in 4 s.
Clean under ASan/UBSan with the pipeline on and off; the harnesses
that include audio_driver.c pass; C89 checked; the Windows toolchain
clean.
buffer_size() was read only on the rate control path, so with rate
control off the frontend never learned the driver's buffer: the init
log said nothing of it and the statistics overlay showed n/a - as a
report from a machine with rate control off and ASIO up did. The
buffer is what the latency setting became, and it exists whether or
not the controller uses it; it is read whenever a driver is active and
reports one, and logged, the line saying whether rate control holds it
or is off. The rate control branch keeps only its own decision. The
overlay says 'rate control off' in place of a fill it is not holding.
Read live with rate control off and on; the frontend clean under ASan
with the overlay rendering; the four harnesses that include
audio_driver.c pass; C89 checked; the Windows toolchain clean.
The microphone device list was built once, at driver init, only when
the driver had opened, and from the running driver. So a driver picked
in the menu showed the old driver's devices until a restart; a driver
that failed to open - the usual state, a microphone opening only when
a core asks for one - showed nothing to pick instead; and the Device
screen never rebuilt it. The same three defects the audio device list
had, fixed the same way: rebuilt each time the Device screen opens and
after init whether or not it succeeded, from the running driver when
it is the one configured and from the configured driver with no
context otherwise, and released by the driver that built it. Every
microphone driver's enumerator takes a NULL context - PipeWire's
returns no list rather than reading one - and none of their frees read
one.
The device-list harness gains the microphone side, built with
HAVE_MICROPHONE and the alsa and null microphone drivers: with nothing
running, alsa configured builds a list through alsa and null
configured attributes it to no such driver; the free leaves nothing
behind. Clean under ASan/UBSan/LSan and TSan; the frontend clean under
ASan with the change in; C89 checked; the Windows toolchain clean.
The latency setting is a hint; what the driver made of it was in the
log and nowhere the user looks. The statistics overlay's audio block
gains a line under the sample rate: the buffer the driver reported at
init in milliseconds of the output format, the setting it was asked
for - the core's minimum where that is higher - and the half of the
buffer rate control holds the fill near, which is the latency heard
from the driver at steady state. n/a with the setting when the driver
reports no buffer. The block also names the driver the user chose
rather than the wrapper it runs under, as the init log already did.
audio_driver_get_buffer_latency_ms() computes it from the reported
size, the output format and the output rate. Clean under ASan/UBSan
with the overlay rendering; C89 checked; the Windows toolchain clean.
When the ring ran dry mid period the last sample of audio was followed
by a hard zero, and when audio returned after a silent period it
started at full amplitude from silence: a step at each edge, and a
click for each. Both edges fade now, over 32 frames - under a
millisecond at 48 kHz, so the underrun itself is as audible as before,
only its edges are not: audio that runs out ramps to the silence that
follows, and audio that returns after a period that ended in silence
ramps up from it, the callback keeping whether the last period did.
The fades are applied to the driver's scratch in place, which the
converter now takes as such.
The fixture harness checks both ramps - the run-out reaching exactly
zero on its last frame, midway about half, the frame before the ramp
untouched; the return near zero on its first frame and full at the
ramp's end - and that a full period with no fade asked for is
untouched at both ends. On the audio samples workflow.
The converter is unit-tested; the driver is syntax-checked with
tools/mingw_syntax_check.sh and not run - no ASIO here.
The driver's preferred buffer size was always taken. The device stage
behind the ring is about two periods, so a driver preferring 1024
frames - 21 ms at 48 kHz, not unusual on an interface tuned for
recording - put 43 ms in the device alone under a 16 ms setting, and
nothing in RetroArch could lower it.
The period asked for is the driver's preferred size, as before, unless
that is more than a quarter of the latency setting; then it is the
smallest legal size at or above that quarter, legal as the
specification's granularity defines it - a step from the minimum, the
three advertised sizes only, or powers of two - and never above the
preferred size or outside the driver's range. A locked driver gets its
size. At 48 kHz: a driver preferring 1024 opens at 256 for 16 ms, 512
for 32, 1024 from 64 up; one preferring 96 or 128 is unchanged at any
setting, being under the quarter already. The init log says which.
This is not the implementation plan's chooser, which mapped the whole
setting onto the period and would have opened a 96-frame driver at 768
for 16 ms; the period is the device stage, and the ring is what holds
the setting.
Syntax-checked with tools/mingw_syntax_check.sh; the arithmetic
tabulated on the plan's fixtures and both reporters' devices. Not run
- no ASIO here.
Both were unanswered - asioMessage returned 0 for them and did not
list them as supported. A resync request is acknowledged and logged;
the driver has nothing to resync in the host, whose ring fill rate
control steers back. A latencies-changed message sets a flag the
writer acts on, on its own thread, which owns the COM apartment:
ASIOGetLatencies is read again and logged with the new output figure
kept for the next reinit to size the ring from.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here.
An ASIO device presents its outputs as numbered mono channels, and the
driver created its stereo pair on the first two, always. On a device
with more than two outputs the first two are not always the ones the
user is listening to - a digital pair listed before the analog ones is
common - and there was nothing to change it: RetroArch played into the
wrong jacks and the user heard nothing.
audio_asio_output_channel is the first of the two outputs to use,
counted from 0 and stepping by two, shown in the menu as the pair the
device numbers it - 1-2, 3-4 - with the device's own names for the two
beside it when the driver is up to be asked, through two new exports.
A value past the device's outputs falls back to the first pair with a
warning. Changing it reinitialises audio, and the reclaim path rebuilds
the buffers when the pair differs from the one in use, as it does for
the driver's own reset request. Init logs the pair and its names.
One row in the single-source definition file; the label string table
regenerated; the display list shows the row with the ASIO items, under
the configured driver.
Syntax-checked with tools/mingw_syntax_check.sh on every touched
translation unit; settings_check.py table and unity clean. Not run -
no ASIO here.
free() parked the instance on every call, for the next init to reclaim
- right for a core swap or an audio setting change, where the next
init is ASIO's, and where destroying and re-creating the COM object
crashed ASIO4ALL. But a change of audio driver never calls ASIO's init
again, and the parked instance held the device - exclusively, under
ASIO - for the rest of the session while the driver the user chose
tried to open it. The audio driver setting is written before the
reinit that follows a driver change, so free() reads it: still asio,
park; anything else, the full teardown the exit handler already did -
stop, a pause for a callback some drivers still have in flight when
ASIOStop returns, dispose, release, free - now one function both use.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here. The scenario to try: ASIO to WASAPI in the menu, then WASAPI
opens the same device; and ASIO to WASAPI to ASIO, where the second
ASIO init is a fresh CoCreateInstance and the log says so.
kAsioResetRequest and kAsioBufferSizeChange raised the driver's
shutdown flag: the next write failed, the audio thread ended, and audio
was gone for the session - on any change from the driver's control
panel, a sample rate the device followed, or a buffer size the user
picked there. The message callback now does what the WASAPI
device-change path does: it asks the frontend to reinitialise audio,
which the frontend does on its own thread in its own time, and marks
the parked instance so the reclaim path disposes the ASIO buffers and
creates them again at whatever size the driver prefers now.
The block that does that - the buffer-size query, the scratch sized to
it, the channel's sample type, the buffers, the OutputReady probe, the
latencies and the ring sized from them - is one function, run at first
init and on a rebuild, in place of a copy that only first init had.
asio_request_shutdown() has no caller left and goes.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here. The scenario to try is a buffer-size change in the driver's
control panel while content runs: audio should pause and resume at the
new size, and the log should show the reclaim and the rebuild.
The handle passed to ASIOInit was whatever window was in the
foreground at that moment - another application's, as often as not -
or the desktop. Several drivers parent a message-only window to the
handle they are given and never call back without a live one of the
host's; the symptom was a driver that initialised, started, and stayed
silent until the frontend's bounded wait gave up on it. It is given the
video driver's window now, as the DirectInput driver does; the old
lookups remain for a reinit that runs before video is up.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here.
The callback called ASIOOutputReady every period whatever the driver
had said of it, and the driver was asked whether it supported it only
after ASIOStart, too late to shape anything. It is probed once now,
after the buffers are created and before the latencies are read - a
driver that honours it knows from the probe that the host will notify
it, and reports the shorter output latency that follows, which then
sizes the ring - and the callback notifies each half only on a driver
that said yes, silence-filled halves included so the zeros ship.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here.
Two of this week's harness directories were added with their built
executable in them - asio_convert_test and latency_contract_test - and
one older one, gfx_widgets_task_bar_stack_test, had been the same way.
All three leave the index; samples/**/*_test and samples/**/*.o are
ignored so a whole-directory add cannot pick a build up again.
The lines that state a driver's two stages called the first one
measured, meaning the stage rate control measures; read cold it says a
measured latency, and the ring's whole size at that. Each line now
says the ring or fifo is held about half full by rate control and
states the sum that follows - half of it plus the device stage - as
about so many milliseconds from write to the device, which is the
figure the line was being read for. asio, dsound, pipewire, rsound and
the libnx audren thread driver.
pipewire's line read live; asio and dsound syntax-checked on the
Windows toolchain; rsound and libnx are string and arithmetic changes,
balanced.
The check that room comes back while the device plays slept for twice
the setting and read once. A null backend drains in real time but
through its own device buffering - OpenAL Soft's null device holds
some 64 ms - so 64 ms of queued audio took near the whole wait to
clear, and under TSan it missed: one failure in a CI run, openal at
64 ms only. The check polls until room returns, with a deadline of
twenty times the setting; every driver here returns some within 20 ms.
The line printed says how long it took.
samples/audio/asio_convert runs asio_convert.h on every ASIOSampleType
for full scale, zero, half and one 16-bit step in both signs, and
checks each byte against what the specification says the type holds,
written by hand from the type's definition: the Int32LSB16..24 family
as low-aligned sign-extended words, the MSB types big-endian, the
floats as their IEEE patterns including negative zero. Saturation and
the silence fill past the supplied frames are checked as well. On the
audio samples workflow; clean under ASan/UBSan.
The callback's converter wrote Float32LSB, Float64LSB, Int32LSB,
Int24LSB and Int16LSB, and memset the hardware halves to zero for
every other ASIOSampleType, with nothing in the log. The Int32LSB24
family that professional interfaces report was among them: on such a
device the driver initialised, started, and played silence.
The conversion moves to asio_convert.h - pure C, no platform
dependency, so it runs on any host - and covers every type the
specification defines: the Int32 family with 16, 18, 20 and 24
significant bits as low-aligned, sign-extended words, which is what
the alignment in the name means; every MSB type as big-endian stores;
Float32MSB and Float64MSB the same. Every integer conversion is
symmetric full scale, round half away from zero, saturating, in
double - the contract the Int16 and Int32 paths kept, extended to the
rest. A type the specification does not define is still silence, and
init says so once rather than the callback saying nothing.
Syntax-checked with tools/mingw_syntax_check.sh; the conversion itself
is run against fixtures by samples/audio/asio_convert.
retro_spsc rounds its capacity up to a power of two, and the driver
reported that capacity, wrote into all of it, and measured the room
against it. On a Topping native ASIO driver at a 16 ms setting the
request was 648 frames and the ring 1024: 21.3 ms reported against 16,
a third over, and the rate control holding the ring's fill at the
rounded size's half rather than the request's. The ring has a logical
size now - the request - for every purpose: reported, written into,
and measured against; the room is the physical room less the excess
capacity, which goes unused. The physical ring is remade only when
the request no longer fits it or would leave more than half of it
idle.
From the two logs: ASIO4ALL at a 96-frame period with 667 reported,
16 ms setting, a 384-frame ring in a 512 physical, 8.0 ms; the Topping
at 128 and 120, a 648-frame ring in a 1024 physical, 13.5 ms. Heard is
about half the setting plus half the device stage on both, as the
other drivers give.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here.
a4149af took two periods off the setting for the device's double
buffer. A real log showed the driver saying otherwise: ASIO4ALL at a
96-frame period reported an output latency of 667 frames, 13.9 ms -
the WDM-KS pipeline it wraps, three and a half times the double
buffer - so the user heard half the ring plus 13.9 against a 16 ms
setting. The driver knows its stage and says so; the ring is sized
from the setting less the reported output latency, as jack sizes from
its ports' reported latency. That figure exists only once buffers are
created, and the ring must exist before that because some drivers call
back during creation; so the ring is made from two periods first and
resized to the reported figure between ASIOCreateBuffers and
ASIOStart, where no callback runs. One helper does the sizing for the
first init and the reuse path, and the stage log names which figure it
used.
On that machine: 16 ms setting, 96-frame period, 667 reported; ring
512 frames after the four-period floor and the power-of-two rounding,
10.7 ms; heard about 19 ms, from about 25.
Syntax-checked with tools/mingw_syntax_check.sh; the arithmetic is
from the reporter's log. Not run - no ASIO here.
The ring was the latency setting in frames, floored at four device
periods and reported at its power-of-two capacity, with the device's
own double buffer - two periods, about a period and a half of audio
queued at any moment - behind it, acknowledged in a comment and not
accounted for: the user heard half the ring plus that. Two periods
come off the setting before the ring is sized now, so the two stages
add up to about it; the four-period floor stays, being what keeps
main-thread jitter from underrunning the ring. One helper sizes the
ring at first init and on the reuse path, which had the arithmetic
twice, and one logs both stages in the form the other drivers use.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no ASIO
here. Helpers are defined after the struct they take.
samples/audio/latency_contract opens alsa on the null PCM, sdl2 on
SDL's dummy audio and openal on OpenAL Soft's null device, at 64 and
32 ms, and checks each against what audio_driver.h now promises:
buffer_size() is the setting in bytes of the output format, within a
tolerance for the driver's own granularity; write_avail() never
exceeds it; a buffer's worth offered without blocking takes room and
makes none; and where the backend models a fill - the null PCM does
not, it consumes on the spot - the room comes back as the device
plays. Where the buffer starts is left to the driver: sdl prefills and
starts full, the others start empty; both are within the contract.
Against the sdl driver before fdda5f7 it reports 42.7 ms for a 64 ms
setting and 21.3 for 32, and fails. On the audio samples workflow,
which gains libsdl2-dev and libopenal-dev; clean under ASan/UBSan/LSan
and, with OpenAL Soft's own atomics suppressed, TSan.
fifo_new(len) keeps len + 1 slots and holds len bytes; these three
reported the slot count, one byte more than write_avail() can ever
reach, and capped wait_writable() against it. Each reports the
capacity. Found by the latency contract harness, which prints the
reported size against the setting and showed 12289 for 12288.
Under the threaded pipeline the running driver is the wrapper, and the
line that reads the buffer back against the setting named it
audio-thread. It names the wrapped driver.
The node latency asked of the graph - the quantum the process callback
pulls from the ring each cycle - was the whole latency setting, and so
was the ring. Every cycle drained the whole ring, so rate control
sampled a full-scale sawtooth; and the graph runs a quantum or more
ahead, so a whole setting's worth sat in the graph on top of the ring
the user was told about. The quantum asked for is a quarter of the
setting now, floored at 128 frames - a request, which the graph may
hold to its own - and the ring holds the setting, at least two quanta,
and is what buffer_size() reports. The init log states both.
Against a PipeWire daemon at 64 ms, 48 kHz: a 24576-byte ring, 64.0 ms,
in front of a 768-frame quantum, 16 ms; the stream connects and
streams. Clean under ASan/UBSan/LSan with a core producing audio,
pipeline off and on.
The linux-c89 lane runs ./configure without libvulkan-dev, so the Vulkan
driver, the slang shader stack and the Vulkan contexts are dropped before
any of their C files meet -std=c89 -pedantic. Add the package so the lane
covers them, and make the files pass it: non-constant aggregate
initializers become declare-then-assign (the input texture, the scissor
and viewport rectangles, the compute-upload UBO, the translation matrix,
the three uniform_sem_t sites), the two empty-brace initializers become
{ 0 }, the two for-loop declarations use the function-scope index, the
two trailing enumerator commas in vulkan_common.h go, and the vendored
SPIRV-Cross header, whose enumerator lists end in commas, is included
under a -Wpedantic push/pop on GCC and Clang.
slang_pass_build_semantics() also kept its two 64-entry descriptor batch
arrays on the stack, a 5.6 KiB frame against the 4 KiB ceiling. They move
into CommonResources: passes build their semantics one after another on
the thread recording the frame, so one chain-wide pair serves them all.
The driver hands audio to audsrv and has nothing to measure its fill
with; write_avail() returned the whole buffer as free whenever the
driver was running. The rate control read that as an empty device on
every frame and answered with its full upward correction, permanently:
audio a half percent fast, and sharp with it, for as long as rate
control was on. The interface now says to leave write_avail() NULL
rather than report a constant, and this does; the frontend disables
rate control for the driver and logs that it did.
Unverified: no ps2sdk here. One table entry becomes NULL and one
function is removed.
The fifo between rs_write() and librsound's pull callback was a fixed
4 KiB - 21 ms of int16 stereo at 48 kHz, whatever the setting - and
was what buffer_size() reported, so rate control held the same 10 ms
at any setting while the setting itself was asked of librsound for its
own buffer behind the fifo, kept full by the callback: the setting
plus 10 ms heard, 21 ms reported.
The fifo holds the setting, in bytes, and is reported; librsound is
asked for what is left of the setting after half the fifo - half of
it, floored at 16 ms or the setting - so the two add up to it, the
same arithmetic as dsound. The init log states both.
Unverified: librsound is not packaged for this distribution. Existing
calls only; the file balances.
Each counted its ring in frames of int16 stereo - the 3DS DSP driver
advances its position a frame per four bytes written, the Wii U driver
counts written frames, the PSP ring is uint32_t frames - and reported
buffer_size(), write_avail() and wait_writable() in those frames, a
quarter of the bytes the interface asks for. The rate control only
needs the ratio, so it was right; fast-forward's bound on the
resampler's input divides write_avail() by the frame size, so on these
three it discarded four times too much. wait_writable() also compared
the caller's byte count against a frame count. All three report bytes
now, and the wait converts the caller's bytes to frames to compare.
Unverified: no console SDK here. Each change multiplies or divides by
the frame size at a report site; the files balance.
fb58eab had audio_thread_wrapper.c release a device list it cannot
forward with string_list_free(); the harness that links the wrapper
did not link string_list.c, and failed at link. It links string_list.c
and compat_strl.c, which string_list.c needs. Every harness the three
sample workflows build was built and run the way they build it before
this commit; all nine pass.
Both libnx audren drivers sized each wave buffer as the setting's
frame count used as a byte count - a quarter of the setting - and kept
BUFFER_COUNT of them, five, in flight: five quarters queued, of which
buffer_size() reported one, and write_avail() the room in the one
being filled, a sawtooth that says nothing of the four behind it.
Rate control on Switch was steering a 16 ms buffer to 8 while 64 sat
queued.
The direct driver splits the setting across the five buffers, in
bytes of int16 stereo, whole frames each; buffer_size() is all five
and write_avail() the free ones plus the rest of the one being
filled, after an update so a finished buffer counts. The threaded
driver has a fifo in front of the wave buffers the render thread
keeps full, the DirectSound shape: the fifo holds the setting and is
what is reported, the wave buffers together take half the setting -
floored at 16 ms, or the setting when lower - so the two add up to
it. Both log the stages at init.
Unverified: no libnx here. Every call is one the files already made;
the arithmetic is the same as dsound's, tabulated there; both files
balance and every new symbol is defined before its first use.
Two stages sat behind dsound_write(): the staging fifo rate control
measures and holds half full, and the DirectSound ring the notify
thread keeps full. Both were sized to the setting, so the user heard
the ring plus half the fifo - about one and a half times the setting,
and at 64 ms of float, with the fifo rounded up to a power of two by
retro_spsc, 107 ms - while buffer_size() reported the fifo alone.
The fifo holds the setting, rounded up to the power of two the ring
buffer is anyway and reported at that real capacity; the ring takes
what is left of the setting after half the fifo, so the two add up to
it, floored at 16 ms to ride out the scheduler between the thread's
1 ms polls - or at the setting, when that is lower, so no setting
hears more than it did. One helper sizes both, used again on the
float-to-int16 fallback so a changed format resizes both stages, and
the init log states each. At 48 kHz float, heard latency by setting:
8 -> 13, 16 -> 27, 32 -> 37, 64 -> 64, 128 -> 128 ms; before, 13, 27,
53, 107 and 213.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no
DirectSound here. The stage arithmetic was tabulated for every
setting above.
ja_init() creates the condition and its lock, opens the client and
makes the ring before the server has been asked for anything, and on
any failure after that freed only the handle. A server that is not
there - the usual failure - leaked all four; LSan, with jackd stopped.
The error path releases each it finds made.
cde1e88 recorded the driver that built the device list and freed it
through that driver with no context, on the finding that no driver's
device_list_free reads one. The wrapper's does: it needs the context
to find the driver it forwards to, and with none it did nothing. Under
the threaded pipeline the wrapper is the enumerating driver, so every
list it built leaked - LSan, on sdl2 with the pipeline on. The wrapped
driver is recorded as the builder instead, and the wrapper's free
releases a list it cannot forward rather than drop it.
The device buffer was a quarter of the setting rounded up to a power
of two - 16 ms became 21.3 at 48 kHz - the fifo in front of it two
device buffers, and the log claimed four. buffer_size() reported the
fifo: 42.7 ms against a 64 ms setting, the rate control holding it at
21. The fifo now holds the setting, as the other drivers' buffers do,
and never less than two device buffers; the device buffer is the
quarter share rounded down, with a 64-frame floor; the log states the
two stages separately and correctly, the device buffer being SDL's and
unmeasured.
On the dummy backend at 48 kHz the init log reads 64.0, 32.0 and
16.0 ms against settings of 64, 32 and 16, with device buffers of 512,
256 and 128 frames. Clean under ASan/UBSan/LSan with a core producing
audio, pipeline off and on. C89 checked.
The latency setting's range starts at zero, and zero reached the
drivers: xaudio and coreaudio floored it at 8 ms, the libnx drivers
at 5, tinyalsa at its hardware minimum or else 64, pipewire refused
to open, dsound fell back to a fixed 4 KiB fifo, and the rest passed
it through to whatever the device made of it. The frontend applies a
floor of 8 ms once, before any driver is called, and says so in the
log; the init line notes the setting was raised. A driver's own floor
is now for its hardware only.
write_avail() is clamped to buffer_size() at both places it is read.
The rate control's direction term is the sample's distance from half
the buffer, over half the buffer; a driver that counts a stage in one
and not the other - shared-mode WASAPI did, before it reported both -
drove the term past +1 and the adjustment nine times past its band.
The contract forbids it; the clamp is for the driver that has not
caught up with the contract.
samples/audio/rate_control_bounds runs the shipping computation with
a scripted driver: 1.0 at the setpoint, the band's edges when empty
and full, and the upper edge - no more - when write_avail() reports
five times the buffer. Upstream gives 1.045 there. On the audio
samples workflow; clean under ASan/UBSan/LSan and TSan. The floor
verified on alsa: a setting of 0 becomes 8, and the device delivers
8.0 ms.
ja_find_buffersize() computed the frames the latency setting called
for, less JACK's own reported latency, and returned that many
sizeof(float) - one sample a frame, for a ring the writer fills with
interleaved stereo. The ring held half the frames asked for,
buffer_size() reported that half, and the rate control held the fill
at half of it again: a quarter of the setting. The log beside it
divided by one sample a frame too, and printed the frame count the
user asked for over a ring that held half of it.
Against the dummy JACK server at 64 ms, 48 kHz: the init log read a
12288-byte buffer, 32.0 ms, rate control near 16.0 ms; it reads
24576 bytes, 64.0 ms, near 32.0 ms now, the same as alsa and openal
report on the same setting. Clean under ASan/UBSan/LSan with a core
producing audio, pipeline off and on. C89 checked.
One line when rate control is enabled: the driver's buffer_size() in
bytes, what that is in milliseconds of the output format at the rate
the device settled on, the latency setting it was asked for, and the
milliseconds rate control will hold it near - half, being the
setpoint. A driver reporting one stage of several, or in the wrong
unit, shows here as a size that does not match the setting; before,
the mismatch was invisible in any log a user could send.
On this box at 64 ms: alsa, openal and jack report 64.0 ms; pulse the
48.0 ms its server negotiated, which it says so above; sdl2 42.7 ms,
its device count rounded to a power of two and its device-side stage
uncounted - the next things to fix, now that they can be read.
The interface documented write_avail() and buffer_size() with the
word Optional and said nothing of init's latency argument, and the
drivers answered accordingly: bytes in most, frames in four, a
constant in one, zero in two; the whole of what the driver holds in
some, one stage of it in others; the setting honoured, halved,
ignored, or floored at eight different values. The rate control
steers write_avail() to half of buffer_size(), so half of what a
driver reports, in time, is the latency the user gets from it - and
no two drivers meant the same thing by the number.
Stated now, on the three members:
latency milliseconds the driver should hold between write()
returning and the device consuming; the policy
minimum is the frontend's, a hardware minimum the
driver's; a driver that can learn the device's own
latency may subtract it.
write_avail() bytes of the output format the driver takes now
without blocking, over every stage it controls,
never above buffer_size(); NULL when it cannot
measure, rather than a constant.
buffer_size() the most it holds across those stages, in the same
bytes; read once at init, renegotiated through
audio_driver_set_buffer_size(); zero disables rate
control, so a driver with a buffer reports it.
The drivers are brought to this one at a time, each against the init
log that reads the reported buffer back in the setting's own units.
The settings list is built once per session, and the audio builder
added the WASAPI rows only when the running driver was wasapi and the
ASIO row only when it was asio. The audio output page gates each
driver's items on the configured driver, correctly - but it looks
them up in a table that never held the other driver's rows, so
switching driver in the menu could not show them for the rest of the
session, and which set appeared depended on what was running when the
menu came up. The microphone builder added its WASAPI rows on the
configured microphone driver, evaluated once, with the same result
after a switch.
All three sets are built always. The audio output page already gates
by the configured driver; the microphone page had no gate, relying on
the table's filtering, and gains the same one so the rows stay off the
page for every other microphone driver.
Checked with tools/mingw_syntax_check.sh on both files (WASAPI and
ASIO are Windows defines) and tools/settings_check.py against the
table reference; not run - no Windows audio here.
Choosing a driver rebuilds the audio output page without reinitialising
audio, and the WASAPI items appear and disappear with the setting. The
ASIO control panel item did not: it was also gated on a device being
set, which is empty right after switching - so picking ASIO showed
nothing ASIO-specific - or, worse, held over from WASAPI, which the
gate could not tell from an ASIO device. It is gated on the driver
setting alone now; the driver opens its default device when none is
set, so a device was never what the panel needed.
What the panel does need is ASIO running, which it is not until the
next start. audio_asio_open_control_panel() reports whether it opened,
and the menu action puts a message on screen saying to restart when it
did not, in place of a warning that reached only the log.
Syntax-checked with tools/mingw_syntax_check.sh with HAVE_ASIO
defined; the settings checks pass. Not run: no ASIO here.
Picking a driver in the menu writes the setting and rebuilds the page;
it does not reinitialise audio, so the running driver stays what it
was until a restart. The Device list is rebuilt each time its screen
opens, and it was asked of the running driver - so after choosing asio
it still listed wasapi's devices, under asio's name, until the next
start. The Device setting is applied to the configured driver, and its
list now comes from the configured driver: from the running instance
when that is the one configured, from the configured driver with no
context otherwise, which enumeration allows.
A list can now be built by a driver other than the one running when it
is released, so the builder is recorded with the list and its
device_list_free is the one called. Every such free in the tree is a
plain string_list_free today; the record keeps a mismatch from
mattering when one is not.
samples/audio/device_list_driver runs the shipping selection with alsa
running on the null PCM and null configured, and asserts the list is
not the running driver's; the previous selection returns alsa's
devices there. On the audio samples workflow. Clean under
ASan/UBSan/LSan and TSan; C89 checked, and the header's Windows
consumers pass tools/mingw_syntax_check.sh.
d32605d put LIBNX_AUDREN_WAIT_NS and LIBNX_AUDREN_WAIT_LAPS in front of
the write, and the file defines wait_writable() before the write, so
the Switch build failed on the first use. The defines now follow the
includes, ahead of everything in the file.
Checked the same way for every constant the console, web, OpenSL,
ASIO, OSS and CoreAudio changes introduced: this was the only one
defined after a use.
Both the float write and the converting write_raw() already waited
100 ms at a time on the render callback and left when the unit stopped;
the cap covers a unit that reports running but never renders, as
wait_writable() in this driver, and the legacy driver's loops, already
have.
Unverified: no Apple SDK here. Two counters on loops that already
existed.
Already timed at 100 ms per wait and ended by the unit stopping; the
cap covers a unit that reports running but never renders, in line with
the other drivers.
Unverified: no Apple SDK here.
alsa_qsa_write() looped on a variable that does not exist in it. The
file is compiled only through griffin under the QNX guard and by no
workflow, so nothing had tried since. This is the one-token fix that
lets it compile; the driver's behaviour has not been looked at beyond
that, and it should not be assumed to work.
audoutWaitPlayFinish() was given UINT64_MAX - no timeout - and the
loop around it, and the libtransistor loop around svcWaitSynchronization
beside it, ran until a buffer came. A service that has stopped
releasing them - the output device changed, the console on its way to
sleep - held the calling thread there. Each wait is 100 ms now and the
loop is capped; with no buffer in hand after that, the write is
skipped this call and wait_writable() reports no space coming.
Unverified: no libnx here. The timeout argument is the one the call
already took; the cap is a counter on a loop that already existed.
The blocking write waited for room with no end, and on a build without
ASYNCIFY that wait is a spin of resume attempts on the browser's own
thread: an AudioContext the autoplay policy keeps suspended until the
page is clicked, or the browser suspends for a background tab, frees
nothing, and the tab froze. The wait is capped - in sleeps on the
ASYNCIFY builds, in attempts on the spin - after which the audio is
dropped; the external-block helper's async loop is bounded the same
way.
Unverified: no emsdk here. Each bound is a counter on a loop that
already existed, and the constant is defined before its first use.
The notify thread sets the event after every block it moves and clears
thread_alive when the buffer is lost for good, so a period with no
event is a play cursor that has stalled, not a device gone. The
blocking write returned -1 there, which the wrapper reads as the device
gone and ends the audio thread over; it now returns what went, and a
lost buffer still reports through the flag on the next call.
wait_writable() gains a lap cap for a thread that keeps moving blocks
but never frees enough.
Syntax-checked with tools/mingw_syntax_check.sh (win32 gnu99 and the
linux-c89 job's flags); not run - no DirectSound here.
The blocking write retried its wait for room without end, and on the
busy-wait build that wait is a spin with no sleep on the browser's own
thread: an AudioContext the autoplay policy keeps suspended until the
page is clicked, or the browser suspends for a background tab, frees
nothing, and the tab froze. The retry is now capped and the spin has a
ceiling, after which the rest of the write is dropped and said so; the
external-block helper's two async loops are bounded the same way. Init
waits up to thirty seconds for the worklet module - a fetch, so
generous - and treats silence past that as the init failure it is.
Unverified: no emsdk here. Each bound is a counter on a loop that
already existed, and every constant is defined before its first use.
sl_wait_writable() states the full-queue case: with every block on the
device there is no block to fill and the space is zero. The arithmetic
gave zero there only through unsigned wraparound - (buf_count -
buffered - 1) at buf_count wrapped, times buf_size, plus buf_size,
came round to zero because buffer_ptr is always zero in that state,
an invariant the write loop keeps in another function. Correct by
accident is not correct.
sl_write() counts a block as enqueued, and moves the index past it,
only when the device took it; a refused block left the count one high
and the index on the next block.
Unverified: no Android NDK here. The change is confined to one branch
and one reordering.
Each of these drivers waited for its output thread, callback or
hardware position with no bound, and one that stopped - the port
closed, the renderer gone, the system back from sleep with the channel
dropped - held the calling thread there. Each wait is now timed where
the platform offers a timed form of the same primitive, and capped at
a fixed number of them; a blocking write then returns what went and
wait_writable() hands the pass back as no space coming from this call.
psp scond_wait -> scond_wait_timeout, both loops; the write
loop also stops when the thread is no longer running
ps3 sysLwCondWait(cond, 0) - 0 is no timeout - -> a timeout in
microseconds, both loops; the write loop also stops when
the port is not started or the thread is quitting
wiiu the write's OSYieldThread() spin, which a running voice
whose callback had stopped never left, waits on the frame
event with OSWaitEventWithTimeout(); wait_writable()'s
OSWaitEvent() the same
switch compat_condvar_wait_timeout() added to the compat layer
(condvarWaitTimeout on libnx, trn_condvar_wait's own
timeout on libtransistor); switch_thread and the libnx
audren thread driver use it in both loops, the latter
also returning from a write while paused, when its render
thread takes nothing
ctr dsp the write's 100 ms poll of the channel position is capped;
the quit path returned true from a function returning
ssize_t, and returns -1
ctr csnd the write's 1 ms poll of the play position is capped
xenon360 the write's udelay(50) spin on the unplayed count is capped
Not changed, for want of a timed primitive: gx's write, a spin with no
sleep on the DMA state, and its wait_writable() on OSSleepThread();
the 3DS DSP driver's wait_writable() on LightEvent_Wait(); and the
non-threaded libnx audren driver's audrenWaitFrame(). Those need hands
on the platform.
None of this could be compiled here: no console SDK. Each change
reuses a primitive the tree already calls on that platform (rthreads
for psp and the Switch and Wii U timed waits), keeps declarations at
block starts, and every file balances; the platform workflows are the
compile check.
The buffer-switch callback zero-fills and returns before it drains or
signals whenever the stream is paused, so a blocking write or
wait_writable() issued while paused waited on a signal that could not
come, a period at a time, with no end. Both now return - the write
with what went, the wait as no space coming from this call - when the
stream is paused, and both are capped at a fixed number of period-long
waits for a driver that has stopped calling back without shutting
down, which no flag reports.
Syntax-checked with tools/mingw_syntax_check.sh (win32 gnu99); not
run - no ASIO here.
The linux-c89 pass compiles a file whose whole body sits under the
Windows guard as an empty translation unit, which -Werror=pedantic
rejects. asio.c joins wasapi, xaudio and dsound on the list of files
that pass has no business with.
The write and wait_writable() of the AudioUnit driver waited on the
render callback with no timeout except on iOS, which had been given a
300 ms one. A unit that has stopped rendering - device removed, default
output changed while stopped - signals nothing, and on macOS the thread
stayed there. Both waits are the timed one everywhere now, with a lap
cap on wait_writable(); the HAL driver's wait_writable(), already timed
per wait and checking IsRunning, gains the same cap for a unit that
reports running but never renders.
Unverified: no Apple SDK here. The change removes the #if that kept
the untimed arm and reuses the iOS arm as written.
The blocking write returned -1 when no buffer completed within a
period, which the wrapper reads as the device gone; a voice held
through a power event produces exactly that. It now returns what it
staged - the staged bytes stay staged and go out on the next write
that finds a slot - and a failed submit remains what -1 is for.
wait_writable() gains a lap cap for a voice that keeps completing
buffers but never frees enough.
Syntax-checked with tools/mingw_syntax_check.sh; not run - no Windows
audio here.
In exclusive mode the blocking write loop did not react to a wait
timing out at all: nothing was moved, nothing was written, and the
loop came round to wait another period, with no end for an engine that
had stopped signalling. In shared mode it returned -1, which the
wrapper reads as the device gone and ends the audio thread over - for
one period without a signal, which a device held through a power
event routinely produces. Both loops now return what went, and a
device that has in fact gone still reports through the failed padding
query or buffer call that follows, which is what -1 is for.
wait_writable() already returned on the timeout.
Syntax-checked with tools/mingw_syntax_check.sh (win32 gnu99 and the
linux-c89 job's flags); not run - no Windows audio here.
samples/audio/pipeline_stall includes audio/audio_driver.c and runs the
shipping producer and consumer against a scripted driver, reaching the
device through the write_raw() path so no resampler or arena is
needed. The device is made to stop taking audio, and the cases pin the
contract: the consumer takes nothing from the ring it cannot deliver
and reports no progress it did not make; the producer records the
stall after one bounded wait and every frame after that costs it
nothing; and both recover when the device returns, twice over.
Against the consumer before 8f13068 the stall is never recorded and is
cleared with no room made - the defect that paced the frontend at one
frame per failed pass, found until now only by freezing a JACK server
by hand. On the audio samples workflow; clean under ASan/UBSan/LSan
and TSan.
In non-blocking mode a full stream refuses with -1, the normal state
of a stream fed faster than it drains, and the driver returned that -1
- which the wrapper reads as the device gone and ends the audio thread
over. It returns what went instead, as oss does for EAGAIN; a stream
that has failed says so on the next blocking write. In blocking mode a
write that takes nothing and reports no error ends the call rather
than looping on it.
Unverified: libroar is not packaged for this distribution, so the
file could not be compiled here. The change is confined to the return
arms of one loop and uses no libroar symbol the file did not already.
Each of these already waited with a timeout, and looped on it. The
loop ended on shutdown or on space, so a server that stays up but
stops running the process callback - the client deactivated, the
graph frozen, no error raised - kept the write and wait_writable()
cycling through timed waits with no end. Both now give up after a
fixed number of period-long waits, the write returning what went and
wait_writable() handing the pass back as no space coming from this
call, in line with the other drivers.
JACK verified against a server frozen with SIGSTOP mid-run under
ASan/UBSan/LSan and TSan with a core producing audio: the audio
thread cycles its bounded waits and the frontend exits cleanly once
the server resumes. sdl2 clean under both on the dummy backend.
rsound and sdl3 are by review: no roard here, and no SDL3 headers.
C89 checked where the headers allow.
audio_driver_pipeline_consume() sizes the chunk from what the ring
holds, waits for the device to have room for it, and only then reads
it out and tells the producer the ring has drained. A device that
makes no room gets nothing taken on its behalf: the ring stays full,
the producer's one bounded wait finds it so, pipe_stalled sticks, and
frames run at full rate dropping audio until a pass completes. Taking
the chunk first and then finding no room threw it away while reporting
the ring drained and the stall cleared, so the producer waited again
next frame - the frontend paced at one frame per failed pass, around
two a second, for as long as the device was stuck. While paused the
chunk is still taken and dropped, so the ring keeps flowing.
Measured against a JACK server frozen with SIGSTOP mid-run, with a
core producing audio: the stall is reported once, pipe_gen holds, and
the frame rate is unaffected; before, the main thread sat in the
producer's one-second wait every frame. Clean under ASan/UBSan/LSan
and TSan across jack, pulse and sdl2, pipeline off and on. C89
checked.
libpulse has no timed wait, so pulse_wait_ms() makes one from a
one-shot time event on the mainloop that signals the waiter, and every
wait goes through it: 3 s for the context and the stream to become
ready at init, after which the open fails; 1 s for a cork or uncork to
be acknowledged, after which the operation is cancelled - a corked
stream raises no write callbacks, so nothing else would have woken
that wait; and 1 s for the server to make room during a write or
wait_writable(), after which the write returns what went and the wait
hands the pass back. The init waits also re-check the state after each
wake, so a spurious signal cannot pass a stream that is not ready.
Verified with a core producing audio against a real server: a sink
suspended mid-run, then the server killed; the connection failure is
reported and the frontend runs on and exits cleanly. Clean under
ASan/UBSan/LSan and TSan, pipeline off and on. C89 checked.
oss_wait_writable() gives select() a timeout of two fragments' worth of
time, clamped, and caps its loop, so a device that stalls or wakes
without space returns 0 - a pass to skip - rather than holding the
audio thread on a select with no timeout, as alsa_wait_writable() does.
The device's settled rate is kept from init to size the bound.
Clean under ASan/UBSan/LSan and TSan with a core producing audio on
the open-failure path, which is all this container's OSS offers; the
bound itself is by review. C89 checked.
Where the library offers AL_SOFT_events - OpenAL Soft, on Linux,
Android and most Windows installs - the driver arms BUFFER_COMPLETED
and DISCONNECTED and waits on a condition the callback signals, so a
write blocked on a full queue wakes the moment a buffer frees rather
than sleeping a millisecond and asking again. The sleep-poll stays as
the path for a library without the extension, Apple's framework among
them. The callback makes no AL call and the waiter holds the lock
across none: it reads a generation the callback bumps, asks the device
with the lock released, and sleeps only while the generation is
unchanged, so an event landing between the two is caught rather than
missed. Both paths keep the existing bound.
The driver gains wait_writable() on the same wait, so the threaded
pipeline paces on the device and stops falling back to the inline
path on OpenAL.
Over 600 frames with a core producing audio, on OpenAL Soft's null
backend: nanosleep calls fall from 4926 to 989, and the remaining
several hundred are the runloop's own frame pacing, present in both.
Clean under ASan/UBSan/LSan and TSan, menu-only and with a core,
pipeline off and on. TSan reports races inside libopenal.so itself
once the event thread is active - an uninstrumented library's atomics,
with no frame of this driver in any report - so the library is
suppressed for that run and the driver's own code holds clean. C89
checked with and without HAVE_THREADS.
al_unqueue_buffers() starts its count at zero, checks the error state
around the query and the unqueue, and caps the count to the slots
left in res_buf. The count was read from an uninitialised variable
with no check: on a device that has gone, the query fails and leaves
it as it was, and whatever was there became the number of buffers
unqueued into res_buf and added to res_ptr - a write past the array
and an index past it after. A healthy source never reports more than
fit, but no query result is allowed to.
al_get_buffer() sleeps for a free buffer under a 200 ms bound, after
which the write returns what it managed. A device that processes
nothing - disconnected, or its context lost - never frees one, and
the sleep-poll had no end.
Verified under ASan/UBSan/LSan and TSan against OpenAL Soft's null
backend, menu-only and with a core producing audio, pipeline off and
on. The gone-device arms are by review: the null backend cannot be
made to fail on demand. C89 checked.
The non-blocking write loop now stops at a refused or empty write and
returns what it managed. A full device refuses with -1 (EAGAIN on the
fd) - the normal state of a device fed faster than it drains - and the
loop added that -1 to its count, shifted it to walk the source pointer
backwards past the caller's buffer, grew its remaining size and looped,
stopping the device on each pass. The blocking loop waits on the device
for two periods' worth of time and returns short when nothing is ready,
leaving an xrun or a suspend for pcm_writei() to recover; and
tinyalsa_wait_writable() waits under the same bound and a lap cap, so a
device that stalls or wakes without space hands the pass back rather
than holding the audio thread on a poll with no timeout, as
alsa_wait_writable() already does.
pcm_writei() reports the frames its initial write took rather than
zero, which had both callers write the opening chunk twice; and
pcm_wait() retries a poll a signal interrupted instead of reporting it
as a device error (errno is positive, so the check it had could not
match).
samples/audio/tinyalsa_write scripts the kernel at ioctl() and poll()
and runs the shipping loops against a full, unstarted, stalled,
interrupted and gone device; on the previous driver UBSan reports the
negative shift and the first case never returns. On the audio samples
workflow. Clean under ASan/UBSan/LSan and TSan, C89 checked.
Each wait on the loop is a sleep until a callback signals it: the
daemon answering a sync, the stream changing state, the process
callback completing a cycle. A daemon that has gone away or errored,
and a graph that has stopped scheduling a stream that still reads as
streaming, signal nothing, and the thread that was waiting - the
frontend's, for init and enumeration; the audio thread's, for a write
- stayed inside the driver. The waits now run in 100 ms steps under a
bound: 3 s for a sync or a connect, after which the open fails, and
1 s for a write cycle, after which the call reports nothing written,
which the pipeline takes as a pass to skip. The microphone's read and
open follow the same rule.
The core error callback records the error and wakes the waiters
rather than stopping the loop: it runs on the loop's own thread,
which cannot join itself, so the stop it issued left that thread
finished but never joined, and the owner's stop at teardown found
nothing running and skipped the join. The owner is the only one that
stops the loop now.
Verified against a daemon killed mid-run with a core producing audio:
the connection error is reported, the stream drops to unconnected,
the frontend runs on and exits cleanly, and the loop thread is
joined. Clean under ASan/UBSan/LSan and TSan across the PipeWire
matrix, menu-only and with a core, including that scenario - TSan is
what reported the unjoined thread.
audio_thread_wait_writable() passes the wrapped driver's zero through
as what the wait_writable() contract says it is - no space from this
call, skip the pass - instead of ending the thread over it. Only a
failed write ends the thread, and once it has ended, block() and
alive() see alive == false and return rather than waiting for an
acknowledgement that cannot come: a stop() after the thread had left
its loop parked the frontend at shutdown, on every driver whose
wait_writable() returns zero for a stream that is not running.
pipewire_stream_set_active() waits on the stream state with a two-
second bound, and returns at once for a stream already where it was
asked to be, which raised no event to wait for. The pipeline producer
that finds a full ring not draining within its wait marks the consumer
stalled and drops until a pass completes, so a device that never
drains costs one wait rather than one per frame.
Found by a sanitizer sweep of the Linux build with PipeWire and no
session manager: the stream stayed paused, wait_writable() returned
zero, the thread ended, and the exit's stop() waited forever.
samples/audio/thread_handshake gains a device that reports no space
pass after pass and asserts the thread stays, alive() answers and a
stop is acknowledged; the previous wrapper hangs there. Clean under
ASan/UBSan/LSan and TSan, C89 checked.
The MessagePack encoder is shared by the two, and was listed only
under the database, so --disable-libretrodb left bsvmovie.o with
undefined references to rmsgpack_write_* at link. It is now added
when either is built, once. Checked against all four combinations of
the two switches with make -n.
The ratio is the mixer's rate over the source's, and a zero on either
side is not one. The mixer's rate is 0 until audio_mixer_init() runs,
which the frontend reaches only once the audio device has opened;
with the device failing to open and menu sounds enabled, the sounds
loaded anyway, the one-shot output was sized as samples * 0 plus the
pad, and the sinc resampler run on phases / 0.0 wrote frames past it.
A WAV whose header claims no rate gives a ratio of infinity from the
other side, with the same result. Both one-shot resamplers and both
stream-voice paths now fail the load or play instead.
libretro-common/samples/audio/audio_mixer_rate_guard pins the refusal
for an uninitialised mixer and for a zero-rate header at every
quality and both sample formats, and pins that an initialised mixer
still accepts a 336-load matrix of rates, lengths and channel counts.
It overflows on the previous mixer at its first case under ASan; the
target is on the libretro-common samples run list. Clean under
ASan/UBSan/LSan and TSan, C89 checked.
Each harness under those three directories builds and runs on every
push and pull request, under ASan+UBSan and - for the ones with a
second thread - TSan as well, in the per-subtree shape the gfx and
tasks workflows use. Until now none of them ran anywhere but by hand:
mixer_lock, thread_handshake and alsa_wait_writable each fail against
the audio code they replaced, webdav_options against the OPTIONS probe
it replaced, and pipeline_migration against the unmigrated config
read, and that stays enforced only if something runs them.
Every step's run block was executed verbatim from a clean tree before
this commit. The asan-ubsan job's note on which per-sample workflows
it complements lists the three new ones.
The main thread's two waits on the audio thread - for init to return
and for a stop to be acknowledged - now wake every two seconds to
check, and the first time either has run that long it names the
driver and which call it is stuck in. Neither wait is abandoned:
callers past a stop act as though the thread is parked, and free()
joins it either way, so a device that never returns from a call still
holds the frontend where it did - but the log now says what it is
waiting on rather than ending without a word.
samples/audio/thread_handshake gains a stop against a device that
stops returning for longer than the threshold and an init that takes
as long, each asserting the handshake still completes and is reported
exactly once, and asserts prompt handshakes are not reported. Clean
under ASan/UBSan/LSan and TSan, C89 checked with the linux-c89 job's
flags.
The wrapped thread parks in one of two waits, the one before its first
start and the one inside its loop; block() waits for stopped_ack from
whichever the thread is in, and now both give it. A start() clears the
stop flag and signals, and a stop() that sets it again before the
thread has re-checked leaves the thread in the first wait, which
acknowledged nothing - block() then waited forever, with the main
thread parked in a stop that could not complete.
samples/audio/thread_handshake drives the shipping wrapper through a
stop before the first start, two thousand start/stop round-trips, a
stop against a device that has stopped returning, and teardown, on a
watchdog; the previous wrapper hangs in the round-trips on every run.
Clean under ASan/UBSan/LSan and TSan, C89 checked with the linux-c89
job's flags.
The setting briefly held off/automatic/on and was saved as a number.
A bool is written as "true" or "false", so a bare digit on this key
came from that three-way setting, where automatic meant the pipeline
was never asked for - config_get_bool() reads it as an explicit on
instead, starting the threaded pipeline for users who never chose it
and on drivers it was not verified against. Automatic and off now
resolve to off, on to on, and the value is rewritten so the next save
records a bool and the rule stops applying.
samples/settings/pipeline_migration pins each stored value against
both defaults, and pins that "true", "false", absent, empty and
multi-character values are untouched; clean under ASan/UBSan/LSan and
TSan, C89 checked with the linux-c89 job's flags.
The lock is created before the audio thread exists and freed after
audio->free() has joined it, which is what makes the NULL test in
audio_driver_state_lock() safe rather than optional. A caller that
reaches the mixer or the DSP filter from another thread outside an
initialised audio driver gets no lock and no diagnostic, so the
lifetime is recorded at the helper and at both ends that maintain it:
widening who touches that state means widening this with it.
audio_driver_mixer_stop_stream() and audio_driver_mixer_remove_stream()
split into unlocked internals plus locking entry points, and the
callers that already hold the lock - add_stream()'s manual-slot free
path and remove_stream()'s stop arm - call the internals. The lock is
a plain mutex, so reaching an entry point while holding it parks that
thread against itself; loading the first menu sound did exactly that
and left a fully rendered menu with no input.
Each entry point now range-checks before it locks rather than
returning out of the locked region, and the manual slot is checked in
add_stream() where the unlocked internals index the array directly.
remove_stream() tests the stream name pointer before the bytes behind
it: a stream loaded without a basename, which is every menu sound,
carries NULL there.
Fixeslibretro/RetroArch#19485.
samples/audio/mixer_lock drives the real entry points over the
manual-slot reload, the occupied and playing slots, out-of-range
slots and a full two-pass load of every system slot, on a watchdog
that aborts rather than hanging; it deadlocks on the previous code at
the first case. Clean under ASan/UBSan/LSan and TSan, C89 checked
with the linux-c89 job's flags.
All three had NULL in the device_list_new slot, so the menu showed
only "Default" and the device field could only be filled by hand. Each
now lists what its init() actually accepts:
jack physical input ports, from a throwaway client opened with
JackNoStartServer - the "left,right" the stream connects to
oss the /dev/dsp nodes that exist and can be opened for output,
labelled with the card name where SNDCTL_AUDIOINFO is there
tinyalsa /dev/snd/pcmC<card>D<dev>p nodes as "card,dev", labelled
with the card id from /proc/asound/card<N>/id - this file
carries only the PCM half of tinyalsa, so no mixer API
None needs a driver instance. device_list_free() frees the list in
each. All three compile under the linux-c89 job's flags.
Both the legacy and modern branches stubbed device_list_new() to
return NULL. Enumerate through the HAL - kAudioHardwarePropertyDevices
on kAudioObjectSystemObject, the same query coreaudio_choose_output_device()
already makes to match a name - and collect the names instead. Needs
no driver instance. device_list_free() now frees the list. iOS has no
HAL output enumeration and keeps returning NULL there.
No macOS toolchain on this machine; the body's shape is verified
C89-clean against a mock of the four HAL calls it uses.
pwire_device_list_new() returned the list the driver's registry
listener built at init and NULL without an instance. Bring up a
throwaway core with the same listener, wait for the registry sync so
every existing Audio/Sink has been reported, take the list, tear the
core down. Used only when there is no live driver.
pulse_device_list_new() returned the list the driver built at connect
time and NULL without an instance, so a user whose configured sink
failed to open had no list to pick another from. Enumerate through a
short-lived plain mainloop when there is no instance: connect, list
sinks, tear down. The driver's threaded mainloop is not involved.
Also fixes a declaration after statement in pa_sinklist_cb.
The device list was built in drivers_init() only after a successful
init and only through the live context:
if (current_audio && current_audio->device_list_new && context_audio_data)
so a failed init left no list, and the one way out of a bad device
choice - the Device dropdown - showed only "Default". Under the
threaded pipeline the wrapper's forwarder gated on driver_data the
same way. None of this was needed: wasapi, xaudio, dsound and asio
never read the context, and mmdevice_list_new() reaches one Windows
import, CoCreateInstance, on a fresh enumerator.
The contract is now that device_list_new(NULL) is valid. The frontend
builds the list after every init attempt regardless of outcome, the
menu refreshes it when the dropdown opens so hot-plugged devices
appear, and the wrapper forwards whatever context it has. When the
wrapper was selected but never started there is no inner driver to
ask, so enumeration falls back to the configured driver by name with
no context.
Drivers that only return a list cached at init - pulse, pipewire -
still return NULL for a NULL context; context-free enumeration for
those, and the missing render enumerations in coreaudio, alsa, sdl3
and tinyalsa, are separate commits.
With the threaded pipeline on, audio_init_thread() installs the
wrapper as audio_driver_st.current_audio. The wrapper's table had
device_list_new and device_list_free as NULL and its ident is
"audio-thread". Two things broke:
The Audio Device dropdown showed only "Default", because the list
is built from current_audio->device_list_new at driver init and
there was none to call.
Per-driver settings - WASAPI exclusive mode and buffer length,
ASIO's - disappeared, because menu_setting.c keys them on
audio_driver_get_ident() and "audio-thread" matches nothing.
The wrapper holds the real driver in thr->driver. Forward
device_list_new/free to it - enumeration, not streaming, and the
drivers already build the list from the main thread in the
non-threaded case - and have audio_driver_get_ident() report the
wrapped driver's ident when current_audio is the wrapper. The
"Started audio driver" log line still says "audio-thread", which is
true.
338abdf045 logged only the success case. On a system where it fell
back the log showed the legacy path with no hint which of the four
exits was taken. One debug line per exit.
Shared mode passed hnsPeriodicity 0 to IAudioClient::Initialize, which
means the engine's default period - 10 ms on a typical desktop. The
write event fires once per engine period, so a client blocking on it
is released on a 10 ms grid regardless of buffer size. With audio_sync
on that is the grid the whole frame loop is released on, and at 60 fps
it is the largest single source of frame-time jitter under VRR.
Windows 10 1607 added IAudioClient3::InitializeSharedAudioStream,
which lets a shared client run at a smaller engine period, down to the
minimum GetSharedModeEnginePeriod reports - typically 128 frames,
2.67 ms at 48 kHz. Use it when it is there: query the interface, read
the minimum, initialise at it. The event now fires every ~2.7 ms and
the release grid is four times finer.
Backwards compatibility, at each layer:
Build. Everything new sits under __IAudioClient3_INTERFACE_DEFINED__,
the guard the SDK header sets. An SDK without the interface compiles
the IAudioClient path only, byte for byte as before. The IID is
defined locally rather than through the SDK export, which mingw's
libuuid lacks.
Runtime. QueryInterface fails on pre-1607 Windows, or the period
query or the initialise can fail on a driver that does not support
it; in every case the client is released, re-activated fresh, and
handed to the unchanged legacy path. Nothing downstream can tell the
difference except the flag.
Latency budget. The default FIFO sizing takes the engine buffer
size, and under IAudioClient3 the engine sizes that buffer to the
small period - a few ms, no longer reflecting audio_latency. When the
low-latency path engaged, the FIFO is sized from audio_latency
directly, so the user's buffer is what it always was and only the
drain cadence changed. Exclusive mode and the microphone are
untouched; the microphone passes NULL for the new out-parameter.
Both COM macro branches (C lpVtbl and C++) carry the new methods.
The OPTIONS probe reads the DAV header (RFC 4918 10.1) and, failing
that, looks for PROPFIND, MKCOL or PROPPATCH in Allow; either one
means WebDAV. PUT is no longer consulted. Allow describes the target
resource (RFC 9110 5.6.1.2), the target cloud sync probes is always a
collection, and PUT is not defined on a collection (RFC 4918 9.7.1),
so a correct server omits it - Apache mod_dav does - and its absence
says nothing about the endpoint. Only a response carrying an Allow
header with no WebDAV method in it now fails the sync, which is the
plain HTTP server the probe is there to catch; dav_verified, and so
the MKCOL 405 rule that rides on it, follows the same signal.
Fixeslibretro/RetroArch#19486, where Koofr was rejected for a
collection Allow list without PUT while working in other WebDAV
clients.
samples/network/webdav_options drives the probe as shipped over the
mod_dav, Allow-only, DAV-only, HTTP/2-lowercase, plain-HTTP and
plain-HTTP-with-PUT header sets plus the token-matching edges, and
runs clean under ASan/UBSan/LSan and TSan. C89 checked with the
linux-c89 job's flags.
The overlay showed "None" on every frame. It reads runloop_state_t::pace
from inside video_driver_frame(), which runs in the middle of the
iteration - after the reset 3330479c16 put at the top and before the
computation at the end - so it only ever saw the reset value. Before
that commit it saw the previous iteration's computed value, which was
right and one frame stale.
The top-of-iteration reset was the wrong fix for the early-return
paths. Remove it. The reset and the vsync bit live in the compute
block, and the three paths that return before reaching it - the
menu's "rely on vsync throttling" return under vrr_runloop_enable,
POLLED_AND_SLEEP and POLLED_AND_CONTINUE - set the record themselves.
AUDIO_FLAG_WROTE is cleared where it is read, and on those returns.
alsa_wait_writable() now waits at most two periods per snd_pcm_wait()
and a handful of laps per call, returning 0 - skip the pass, retry on
a later wake - when the device delivers no space in that time. A
prepared stream that is short of space is started rather than waited
on: it drains nothing and arms no poll on its own, and an empty
start's immediate underrun recovers to a prepared stream reporting a
full buffer. The wait_writable() contract in audio_driver.h now makes
the bounded wait part of the interface, so a device that stalls behind
a nonconforming plugin (the pipewire ALSA shim among them) costs
dropped audio instead of a parked consumer, wedged wrapper handshakes
and a frozen frontend.
samples/audio/alsa_wait_writable scripts the stalled, dead-prepared,
underrun-storm and progress-free-wake devices through -Wl,--wrap and
runs the conforming path against the null PCM; every scenario is
bounded under ASan/UBSan/LSan and TSan, and the pre-change code fails
the suite through its watchdog.
c8f05b12f2 put the per-iteration pace reset between the local
declarations at the top of runloop_iterate(), ahead of the
HAVE_NETWORKING and HAVE_MENU locals that follow. That is a statement
before declarations, and the linux-c89 job builds with
-Werror=declaration-after-statement:
runloop.c:7942:4: error: ISO C90 forbids mixed declarations and code
Moved below the last declaration.
The tool I check with was running -std=gnu99 only, which accepts this.
It now runs a second pass with the C89_BUILD flags verbatim from the
Makefile - -std=c89 -ansi -pedantic -Werror=pedantic
-Werror=declaration-after-statement - on the native compiler, for
every file that is not Windows-only by path. That pass reports the
error above on c8f05b12f2 and nothing on this commit.
Six of the returns into RUNLOOP_STATE_POLLED_AND_SLEEP are not idle.
Config replace, startup page, list-cache flush, core load (two sites)
and a dialog's pending command each perform a one-shot action and
return so the next iteration starts clean - and then pay the 10 ms
idle sleep that state exists for. A visible stall on every core load
and every dialog confirmation, for nothing.
RUNLOOP_STATE_POLLED_AND_CONTINUE returns to the caller without
sleeping. The six sites use it. The three genuinely idle returns -
unfocused, IDLE in the menu, IDLE with the core running - keep
POLLED_AND_SLEEP unchanged.
APP_CMD_PAUSE/STOP set RUNLOOP_FLAG_IDLE and swap in a stub frame,
with the comment "Avoid draining battery while app is not being
used." The runloop then returns POLLED_AND_SLEEP every iteration and
sleeps 10 ms, and each wake runs android_input_poll(), which drains
the looper with a short timeout, finds nothing, and returns. A
backgrounded RetroArch woke a hundred times a second to discover it
was still backgrounded. Android's doze accounting penalises exactly
that pattern.
The blocking primitive is already in the file: ALooper_pollOnce(-1)
waits until the OS delivers an event, and android_run_events() uses it
for the startup pump. When IDLE is set, android_input_poll() now
blocks on it; the OS's RESUME command is what wakes the loop, and
nothing else needs to. After the first event the remainder of the
burst is drained non-blocking so RESUME and INPUT_CHANGED land in one
call. The runloop's 10 ms sleep is skipped on Android when IDLE, since
the poll already blocked and sleeping again only delays the response.
Unfocused-but-foreground is unchanged: the app is visible and the
short sleep is right there.
No NDK on this machine; runloop.c compiled, the poll body's shape
verified C89-clean in isolation.
RUNLOOP_PACE_AUDIO required runloop_is_libretro_running(), on the
theory that with the core paused nothing writes to the driver. Wrong:
audio_driver_menu_sample() feeds a frame of silence every menu
iteration the core is not running, with the mixer, menu sounds and
thumbnail video audio mixed in, through the same blocking
audio_driver_flush() funnel. Audio holds the menu loop too, and the
overlay said it did not.
Stop predicting and record the fact. AUDIO_FLAG_WROTE is set at the
three driver write sites, cleared at the top of runloop_iterate()
alongside the pace reset, and the bit reads it. Whatever path reached
the write - core, menu silence, mixer - the answer is the same.
Several paths return before the block that computes it - the menu's
"rely on vsync throttling" early return under vrr_runloop_enable, and
POLLED_AND_SLEEP - and the overlay then kept showing the previous
frame's sources on a frame nothing was holding. Reset to NONE first,
and set the vsync bit there too since it is state rather than path and
those early returns do rely on it.
With Scanline Sync and audio_sync both live the loop is held by two
display-rate clocks in series: each frame waits for the audio buffer
to drain, then for the beam to reach the target. Measured at 25 ms per
frame - 40 fps - on a 120 Hz panel with 60 fps content, and the
overlay read "Audio+Scanline". At matched refresh the frame rate is
fine but the two clocks drift against each other with nothing to
arbitrate, the same fight VSync + audio_sync has.
When the scanline wait holds the loop it is the pace. Audio goes
non-blocking and Dynamic Audio Rate Control tracks the display through
the resampler ratio, which is what it exists for. The AUDIO bit is
cleared so the overlay reports what is actually holding the loop.
Restored through the same call every other blocking transition uses
when the scanline wait stops - VRR toggled on, calibration lost,
setting turned off.
Scoped to Scanline Sync only. VSync + audio_sync is the same conflict
with a far larger installed base and is not touched here.
runloop.h consumers checked; the one failure, gfx/drivers/rsx_gfx.c, is
PS3-only and fails identically on HEAD.
Under LATENCY, from runloop_state_t::pace. Names every source that
held the loop on the last frame - VSync, Audio, Scanline, Timer - and
joins them with '+' when more than one was live, since that overlap is
what the bitmask exists to expose. 'None' when nothing held it.
70e5889d9a declared the two entry points as
PFN_vkAcquireFullScreenExclusiveModeEXT and
PFN_vkReleaseFullScreenExclusiveModeEXT inside
#ifdef VK_USE_PLATFORM_WIN32_KHR. That compiled in vulkan_common.c and
broke shader_vulkan.c:
error: unknown type name 'PFN_vkAcquireFullScreenExclusiveModeEXT'
The typedefs live in vulkan_win32.h, which vulkan.h includes only if
VK_USE_PLATFORM_WIN32_KHR is defined the first time vulkan.h is seen.
shader_vulkan.h includes vulkan.h directly, before vksym.h defines the
macro; vulkan.h's include guard is then set, and the later include via
vksym.h - macro now defined - is a no-op. So in that TU the macro is
defined and the typedefs are not. The #ifdef was true and the types
were missing.
Store them as PFN_vkVoidFunction, which vulkan_core.h provides
everywhere, and cast at the two call sites.
The mistake was verifying by compiling the file I edited. A header is
checked by compiling everything that includes it, and the include
order differs per TU, so one passing does not imply the rest.
tools/mingw_syntax_check.sh does that on a mingw cross compiler with
the Win32 feature set:
tools/mingw_syntax_check.sh --consumers-of gfx/common/vulkan_common.h
reports 2 errors on 70e5889d9a and 0 with this fix.
Windowed Fullscreen Mode off was supposed to mean exclusive
fullscreen. On the Vulkan driver it reached the swapchain as
VK_FULL_SCREEN_EXCLUSIVE_ALLOWED_EXT, which is a permission the driver
may decline - and NVIDIA declines it. PresentMon on a 5090 reports
"Hardware Composed: Independent Flip" with the setting in either state:
the swapchain sits on DWM's multi-plane-overlay path regardless, and
the toggle is inert. vkAcquireFullScreenExclusiveModeEXT was resolved
in the vendored header and never called anywhere.
Add a setting beside it. Relaxed is the current behaviour, ALLOWED and
nothing more, and stays the default. Forced sets
VK_FULL_SCREEN_EXCLUSIVE_APPLICATION_CONTROLLED_EXT on the surface and
calls vkAcquireFullScreenExclusiveModeEXT once the swapchain exists.
A failed acquire is a warning and the swapchain carries on where
Relaxed would have left it. VK_ERROR_FULL_SCREEN_EXCLUSIVE_MODE_LOST_EXT
- alt-tab, a mode switch - takes the same rebuild path as
VK_ERROR_OUT_OF_DATE_KHR, and the mode is released before the
swapchain it is bound to is destroyed.
What Forced buys is not asserted here because it has not been
measured: possibly lower present-to-scanout latency than the
compositor path, genuinely immediate presentation that Scanline Sync
could position against, and the application owning the display's HDR
state rather than negotiating it through DWM. This is the switch that
makes those measurable.
The Windows-only pieces are under VK_USE_PLATFORM_WIN32_KHR. The
setting exists on every platform and does nothing where the extension
is absent. tools/settings_table.reference regenerated; the three
Settings-Checks jobs pass locally.
The blanking arithmetic used video_st->height as the display's visible
line count. That field is the output window's height, set by
video_driver_set_output_size(), and it equals the display's visible
lines only in fullscreen at native resolution. Anywhere else the aim
is nonsense: a 1080p window on a 2250-line panel computes 1170 lines
of "blanking" and centres the target inside it. The assumption was
inherited from the previous implementation, but this series built the
aim on it without checking.
Read the mode height from the display server at calibration instead,
falling back to the window height only when the server cannot report a
mode. Stored alongside total and period since all three describe the
display and are re-measured together on reinit.
Reported as a regression: G-Sync on with Scanline Sync also on had
worked before this series and now tears and loses A/V sync.
It worked before because Scanline Sync did nothing - it latched itself
off within two seconds of starting, so leaving it enabled alongside
G-Sync was harmless. c9260c895d made it work, and under VRR a working
scanline sync is wrong by construction. The display holds in blanking
until a frame arrives and only then scans it out. "Wait until the beam
reaches line N" therefore waits for a scan that starts when we present;
the phase lock closes on our own output rate and reports it as the
display period. Every frame is delayed, nothing is synchronised.
There is no software-visible signal that VRR is active. Sync to Exact
Content Framerate is the setting a user enables for VRR, and the
RUNLOOP_STATE_MENU throttle already treats it and Scanline Sync as
alternatives, so it is the proxy: when it is set, Scanline Sync clears
its target and does not engage. That restores the pre-series behaviour
for that configuration exactly.
7da21f0e5f reverted a05198f9dc for its policy. Two bit corrections
were bundled into that commit and went with it; they are right on
their own and are restored here, still with no behaviour change since
the bits are only recorded.
RUNLOOP_PACE_SCANLINE mirrors the gate at the after_frame call site,
which skips the wait under fast-forward. RUNLOOP_PACE_AUDIO also
requires runloop_is_libretro_running(): in the menu with the core
paused the driver stays in blocking mode but nothing writes to it.
This reverts a05198f9dc.
It broke A/V sync on the configuration it was meant to leave alone:
Sync to Exact Content Framerate, G-Sync on, VSync off, audio_sync on,
120 Hz panel with 60 fps content. That is the "VRR, audio blocking"
row of the commit's own case table, the one it claimed changed
nothing observable.
The claim rested on this reasoning: audio backpressure holds the loop
at the core's rate, so the timer's to_sleep is already zero or
negative in steady state and the sleep was not happening. That is
wrong about how audio backpressure works. audio_driver_write() blocks
only when the buffer is full. With a 22 ms buffer and 16.7 ms of audio
per frame, the loop runs free for a frame or two, blocks until the
buffer drains, runs free again. The average rate is right; the
per-frame delivery is bursty. It is a rate limiter, not a frame pacer.
The timer was what turned that into smooth per-frame delivery. Its
phase accumulator held each frame to its own slot, and audio
backpressure only bounded the average underneath. Remove the timer
and frames arrive in clumps - invisible under VSync, where the
display resamples them onto its own grid, and immediately visible
under VRR, where the display shows every frame the moment it arrives.
So RUNLOOP_PACE_AUDIO is not a pace source in the sense the policy
assumed. VSync and Scanline Sync block on every frame; audio blocks on
some. Treating them as interchangeable "hardware clocks" was the
error. The structure from 542c5abdbc stands and the bits are still
honest about what is blocking; what was wrong was the inference that
"blocking" means "pacing every frame".
No replacement policy here. The right one for audio is not "skip the
timer" but something that takes audio as the rate reference and the
timer as the per-frame pace, which is what the code already did and
what this had removed.
First policy on the pace composition from 542c5abdbc. If vsync, audio
backpressure or Scanline Sync is holding the loop this frame, the
timer sleep does not also run. A loop that is already paced by
hardware gains nothing from a second pace off the CPU counter.
Two bits had to be tightened first for this to be safe, both found by
walking the cases:
RUNLOOP_PACE_SCANLINE now mirrors the gate at the after_frame call
site, which skips the wait under fast-forward. Without that the bit
claimed a live wait during fast-forward and this policy would have
dropped the limiter there.
RUNLOOP_PACE_AUDIO now also requires runloop_is_libretro_running().
In the menu with the core paused the driver stays in blocking mode
but nothing calls audio_driver_write(), so nothing is held; the bit
would have suppressed the timer that paces the menu at refresh rate
and let it spin. This is the predicate the RUNLOOP_STATE_MENU
throttle already uses for the same question.
With the bits honest, the cases:
VRR, audio blocking AUDIO timer was redundant -> skipped
VRR, scanline sync SCANLINE timer was redundant -> skipped
VRR, audio non-blocking none timer is the pace -> fires
fast-forward none timer is the limiter -> fires
menu, vsync off, paused none timer paces the menu -> fires
vsync on, focused VSYNC timer never fired -> unchanged
Only the first two change, and in both the phase accumulator already
brought to_sleep to zero or below in steady state, so the sleep was
not happening. The one observable difference is at content start under
VRR with audio_sync: before the audio buffer first fills the timer used
to pace those frames and now audio backpressure takes over a few
frames later. That is how audio_sync already behaves without VRR, where
the timer never ran, so VRR now matches it.
Same defect as 5bd8dc8864, on the other hardware clock.
RUNLOOP_PACE_VSYNC read the video_vsync setting, but what the video
driver is actually doing is decided in driver_set_nonblock_state():
video_nonblock = INP_FLAG_NONBLOCKING;
if (!video_vsync || RUNLOOP_FLAG_FORCE_NONBLOCK)
video_nonblock = true;
Fast-forward sets INP_FLAG_NONBLOCKING and the driver stops blocking on
present while the setting stays true. A policy that treated the bit as
"a hardware clock is live" would have dropped the fast-forward limiter
in that window, exactly as the audio bit would have.
There is no live VIDEO_FLAG_NONBLOCK to mirror, so compute the bit from
the same three inputs the driver call uses.
No behaviour change: still only recorded.
542c5abdbc set RUNLOOP_PACE_AUDIO from the audio_sync setting. The
setting is not the state. Fast-forward switches the driver to
non-blocking for a few frames with audio_sync still true, and during
those frames audio_driver_write() returns immediately and holds
nothing. A policy built on the setting - "a hardware clock is live, so
skip the timer" - would have removed the fast-forward limiter in that
window and let it run unbounded with audio_sync on.
AUDIO_FLAG_NONBLOCK is maintained by audio_driver_set_nonblock_state(),
the same call that flips the driver, so it is the actual blocking
state. Read that.
No behaviour change: the bit is still only recorded, and the timer
gate it sits beside is unaffected.
The loop is held to a rate by four mechanisms, each of which decides
on its own whether to block, and none of which knows about the others:
video_vsync present blocks in the video driver
audio_sync audio_driver_write() blocks on the buffer
video_scanline_sync after_frame waits on the vblank-locked target
frame limit a timer sleep at the end of runloop_iterate()
They are not exclusive and they are not prioritised. Any combination
can be live on the same frame, which is why VSync together with
audio_sync stutters - two hardware clocks holding one loop - and why
the RUNLOOP_STATE_MENU throttle has a comment special-casing audio
backpressure against the timer. Each overlap that has been found so
far was handled where it was found.
This does not change which of them fires. It records the composition
for the current frame in runloop_state_t::pace as a bitmask, computed
once from the same conditions each mechanism already tests, and gates
the timer sleep on its bit. The timer's condition is copied verbatim,
so the gate is boolean-equivalent to the inline test it replaces; the
other three still block inside their own subsystems and are only
recorded here.
A bitmask rather than an enum deliberately. A single value would
assert that the loop chooses one source, and it does not. Making that
choice - hardware clocks over the timer, one hardware clock at a time
- is a behaviour change with a configuration-dependent blast radius,
and it belongs in its own commit argued on its own. This gives it one
place to be made.
The Settings-Checks CI job diffs the emitted configuration tables
against tools/settings_table.reference so that a def row which stops
reaching the tables is caught - nothing else notices. Adding a row
changes the tables, and fc23f85979 did not regenerate the reference,
so the job failed on "Configuration table drift".
python3 tools/settings_check.py table --write tools/settings_table.reference
Verified all three checks in that workflow pass locally: table drift,
griffin-order unity compile for console/desktop/mobile, and the
migration self-test.
Setting the offset to -950 in the menu stored 2050. The row passed
offset_by = 0 with a minimum of -3000, and setting_action_ok_uint opens
a dropdown that maps list index to value through offset_by: index 2050
is -950 counted up from -3000, and with offset_by wrong it was stored
as 2050 literally. crt_switch_center_adjust passes offset_by = -50 for
its -50..50 range; this does the same with the minimum.
Step goes from 1 to 10. A step of 1 made the dropdown 6001 rows and put
the useful value 950 presses away, and the precision this needs is the
half-width of a 90 line blanking interval.
49f07a52ee added the setting - config key, enum, descriptor and strings
all from one settings_def row - and verified those symbols existed, but
never added it to the two display lists in menu_displaylist.c that
decide what actually appears under Settings > Video > Synchronization.
The row was reachable through the config file and by search and
nowhere else.
Add it directly below Scanline Sync in both lists, behind the same
HAVE_D3DKMT guard.
The tear line persists after every CPU-side correction. The aim
arithmetic is right, the release-to-present interval is measured rather
than assumed, the timer sleep was removed and tested and made no
difference, and a two-image swapchain made no difference either. What
remains is a stable residual of roughly 950 scanlines - about 7 ms at
4K60 - between where the CPU believes the flip will land and where it
lands.
That residual is the part the CPU cannot observe. The wait releases,
the core runs, the frame is submitted, and the present call returns -
but the GPU has not finished rendering, and the flip happens when it
does. No timestamp taken on the CPU side sees that moment. Closing the
loop properly needs present-timing feedback from the driver
(VK_KHR_present_wait or VK_GOOGLE_display_timing on Vulkan), which is
the correct long-term fix and is not attempted here.
Expose the offset instead, in scanlines, signed, and dialled by hand
until the tear moves into blanking. This is how RTSS's scanline sync
handles the same limit, and it is honest about what the CPU can and
cannot know. The default is zero. The aim is wrapped to stay inside one
frame since the offset can push it out either side.
Plumbed through settings_def_video_sync.h so the config entry, the
enum, the menu descriptor and the strings come from one row.
The tear was still visible after 83762306a5, moved up near the top of
the screen, and the reported scanline sat around 1474 rather than the
old constant 2025.
The target arithmetic was right - centre of blanking at 2205, minus
core_run_time of 5.26 ms at 730 lines, gives 1475, which is what was
observed. The quantity was wrong. core_run_time is the core's own
duration and nothing else: on a PS2 core at 4K it is 5.26 ms of a
16.21 ms frame, so the remaining 10.95 ms of render, submit and present
- about 1520 lines - was unaccounted for. The flip therefore arrived
1520 lines after the aim point, wrapped past blanking and landed near
the top, which is exactly where it appeared.
Measure the interval instead of naming a parameter and hoping it covers
the work. video_driver_scanline_after_frame() runs immediately after
the present, so the gap between one wait releasing and the next entry
to that function is precisely the release-to-flip time, whatever it
happens to consist of. It needs no knowledge of the video driver, the
shader chain or the swapchain depth, and it tracks changes in any of
them.
core_run_time is no longer used here.
Two things, both surfacing from the same test run.
The tear was visible on screen. The target was a fixed 15/16 of
height - line 2025 at 4K, with the visible area ending at 2160 - so it
sat 135 lines inside the picture. Worse, the wait only releases the
frame loop: the present happens core_run_time later, so the aim has to
subtract the core's own duration for the flip rather than the release
to land in blanking. c9260c895d took core_run_time as a parameter and
then discarded it, which dropped the "corelines" term the previous
implementation had for exactly this. Aim at the middle of the blanking
interval with core_run_time subtracted, which puts the present at line
2205 of 2250 whether the core takes 2 ms or 13 ms.
No smoothing on core_run_time, which is what the measurement said
rather than what was expected. The target moving is the correction and
not error - it moves because the core's duration moves, and that is
what holds the present in place; smoothing would lag a real load change
and put the tear on screen during the transitions this is meant to
absorb. The error is the frame-to-frame delta, since each frame is
aimed with the previous frame's measurement, and that measured 68-142
us on a PS2 core: 9 to 19 lines against a 90 line blanking interval.
Movement is clamped instead, because peaks reach 1268 us in steady
state and 4335 us across a geometry change - 171 and 585 lines - and
without a limit one anomalous frame spoils the next as well.
Second, the two platform primitives move behind
video_display_server_t as get_scanline and wait_vblank. The phase lock
itself is arithmetic and now compiles everywhere with no HAVE_D3DKMT in
video_driver.c; a server that implements neither reports nothing and
the feature disables itself.
Nothing to consolidate elsewhere, which is worth recording. Wayland's
presentation-time protocol reports after the fact rather than blocking;
X11 detects GLX_OML_sync_control but only uses it to pick a swap mode;
the KMS context has a page-flip handler and no drmWaitVBlank. The
equivalents exist in those platform APIs - drmWaitVBlank and
glXWaitForMscOML - but both are vblank waits with no live scanout
position, and Scanline Sync needs the position to calibrate the line
count and to target a line. The interface allows one without the other
so that a vblank-only server can still be wired up for something else.
The tuner this replaces nudged a target line by +-1, +-2 and +-4 based
on where the beam happened to be last frame, with kill/hold/rearm
heuristics to stop it thrashing, and found the beam by polling
D3DKMTGetScanLine. That call measures ~223 us on a WDDM driver, during
which the beam covers ~60 lines at 4K120, so the loop could not resolve
better than 60 lines and spent 5.5 to 9.2 ms per frame doing it -
longer than an 8.31 ms frame period. The heuristics on top produced
three separate latches, all fixed and then reverted in 302b9546ea,
because fixing them only exposed that the loop underneath does not
converge.
Anchor on D3DKMTWaitForVerticalBlankEvent instead and let the display
be the clock. Measured across ten modes on a 4K panel: zero vblank
intervals outside +-20% of the median in 499 samples per mode, p1..p99
spread 21-32 us, period accurate to better than 0.1% at 30, 50, 59, 60,
75, 100, 119 and 120 Hz. Because the period is measured rather than
read from the reported refresh rate, nominal-versus-actual error does
not enter and a variable refresh display is tracked rather than
assumed.
Three parameters, all measured at calibration:
total line count including blanking, from the peak scanline
period median vblank-to-vblank interval
bias residual prediction error at a fixed delay
bias is mode dependent - +27 lines at 4K120, +59 at 4K60 - so all three
are measured together and the calibration is dropped on video reinit,
which is where a mode change lands. It is an offset and not a drift:
swept against a 0-6 ms delay it moves 0.1 lines at 60 Hz and 3.9 lines
at 120 Hz, which is what makes a single calibrated constant valid.
The wait is a timer sleep to the computed moment followed by a short
spin across SCANLINE_MARGIN_US. The sleep is not a timing authority -
the spin decides the moment, so a mistimed sleep costs microseconds of
extra spin rather than putting the beam somewhere it is not.
The anchor is refreshed by reading the beam and back-computing where
the vblank must have been, not by a second blocking wait: that would
block for up to a whole frame period on top of the frame's own wait.
One GetScanLine is ~223 us, 0.17% of a frame at 120 Hz amortised over
16 frames. Calibration is 24 vblank waits, 160 scanline reads and 12
bias samples - 362 ms at 120 Hz, 662 ms at 60 Hz.
Prototype measurements over 3000 frames with a 0.5-5.5 ms synthetic
core load, against the implementation being replaced:
landing error spin per frame
4K120 14.8 lines mean 331 us was +-60 lines, 5500-9200 us
4K60 6.8 lines mean 321 us was +-30 lines, 5500-9200 us
Median landing error is +1 to +2 lines. Frames that cannot make their
target are counted and otherwise ignored; there is no hold state, so a
missed frame does not affect the next one.
Recorded in the header comment because it cost an evening to establish:
none of this can absorb a display refresh that is not the content rate.
Syncing to a scanline means releasing one frame per beam pass, so at
120 Hz with 60 Hz content there are two passes per frame and no single
correct moment to wait for. The wait duration alternates and shows up
as frame time deviation no matter how precisely it is measured -
15% observed, against 0.9% with the feature off. Configure the display
at the content's refresh rate, VSync off, and make sure
video_refresh_rate matches what the panel actually runs at: a nominal
120.000 against a real 120.39 drifts a full frame every ~2.5 seconds.
Reverts cde80df7f3, 3a95d8f1c0 and 0966f2f32d.
The three commits fixed real defects - the disable latching itself on
permanently, a zero frame-time sample reading as deviation, the hold
expiry being consumed by the next hold, and a 1 ms sleep leaving 7 ms
of every frame in a 223 us-per-sample poll loop. All of that was
measured, and none of it is in dispute.
But fixing them only exposed that the control loop underneath does not
converge. With the sync actually running, tune coverage swings between
1 and 121 frames per 120-frame window: kill fires once or twice a
second, holds for 60 frames, releases, and holds again. The result is
visibly jerky output where before the feature was silently inert and
the output was smooth. The poll median also falls to 1.4 samples per
spun frame with the loop frequently exiting on its first sample, so the
tuner is driving scanline_next to targets the beam has already passed -
it runs without synchronising to anything.
A feature that does nothing is better than one that stutters. Reverting
restores the previous behaviour; the reverted commit messages record the
defects and the measurements for whoever takes this on properly. The
work needed is a redesign of the tuner, not a fifth patch to the same
state machine.
0966f2f32d fixed the disable latching itself on permanently. Testing
the result on a PS2 core showed the feature still inert for minutes at
a time, with a different signature:
rearm=40 dec=80 tune=0 | next 0..0 | hold 0..2
scanline_hold cycling 3-2-1-0-3, forty times per 120 frames, and the
recovery never running. Same shape of latch, one level down.
The disable decision ran before the "Allow change" block. A deviation
raised scanline_hold to 3, and the tune test a few lines later saw a
non-zero hold and skipped - so the frame on which the hold expired was
immediately consumed by the next hold, and scanline_next stayed at 0.
after_frame() then takes its "scanline_target <= 0 -> wait = false"
exit, which is why the poll count was zero for window after window.
Move the decision after the tune, and gate it on having tuned this
frame. Tuning now happens on the frame the hold expires, and the new
hold applies from the next one. The "tuned" gate is what stops this
becoming 0966f2f32d again: without it, the decrement at the tail brings
hold to zero and the decision immediately re-arms it, so the countdown
never yields a tuning frame either.
Modelled against the observed deviation rate, tune goes from 18 frames
in 360 to 99.
"Disable if unsupported" (scanline < 0) stays where it was, ahead of
the "Shift overflow" that can itself make scanline negative.
Third latch found in this state machine, second fixed here. The
interlock between scanline_hold, scanline_next and the two decision
sites is fragile enough that a fourth would argue for restructuring it
rather than another patch.
video_driver_scanline_after_frame() sleeps off the bulk of the wait and
polls the remainder, because D3DKMTGetScanLine is the only scanline
access the D3DKMT surface offers and every wait it exposes -
WaitForVerticalBlankEvent, SetSyncRefreshCountWaitTarget - is
frame-granular. A sub-frame wait is therefore a sleep or a spin, and
the spin is expensive: the query measures ~223 us on a WDDM driver, so
the beam covers ~60 lines per sample at 4K120 and whatever is not slept
is paid for at that rate.
The sleep was (frame_time_target - core_run_time) / 1000 - 4
milliseconds. At 120 Hz that is (8306 - core_run) / 1000 - 4, negative
for any core using more than about 4 ms, so it clamped to the 1 ms
floor. Measured on a 4K120 display with a PS2 core: 25 to 41 samples
per frame, 5.5 to 9.2 ms of busy-wait against an 8.31 ms frame period.
The spin was running longer than the frame lasted.
Compute the remaining time from the lines still to travel and the line
period that frame_time_target and the learned scanline_total already
give, and sleep that less a margin. Modelled on the observed case -
beam near vblank, target line 2156 - this is a 6 ms sleep and about 9
samples where it was a 1 ms sleep and 31.
The sleep is an efficiency estimate, not a timing authority. The loop
still exits on a measured scanline, so a wrong sleep costs extra
samples or an overshoot that the existing scanline_total learning
absorbs; it cannot put the beam somewhere it is not. That is the
distinction from predicting beam position off a CPU counter, which
would need the counter and the pixel clock to agree in rate and would
break outright under VRR.
The original sleep is kept for the case where no line count has been
learned yet, or the query failed.
Scanline Sync stopped working ~121 frames into every session and never
resumed. Instrumenting video_driver_scanline_before_frame() showed the
spin in after_frame() running for the first ~31 frames and then never
again - 2400 consecutive frames with zero polls, while the menu still
reported the feature as on.
Two faults, compounding.
The disable tested "scanline_hold &&", so it fired only while a hold
was already in progress, and re-armed scanline_hold to refresh_rate / 2
on every frame of the hold it had just set. The tail decrements by one,
so hold sat permanently one below that value - the diagnostic reported
hold 59 in every window, never anything else. It therefore never
reached zero, the "Allow change" block is gated on !scanline_hold, and
scanline_next stayed at 0. after_frame() computes
scanline_target = 0 from that and takes its "scanline_target <= 0 ->
wait = false" exit, so nothing ever waited again.
The way in was frame_time_deviation, which compares
frame_time_samples[frame_time_count - 1] against
frame_time_target * 0.33f. That ring starts zeroed, and under
video_frame_time_sample_gated a sample failing the clean predicate is
not written and the count is not advanced, so the read can land on a
slot that was never populated. Zero satisfies the comparison, so an
unwritten sample reads as a permanent deviation. The diagnostic
confirmed it: the disable fired on 120 of 120 frames while
core_run_time exceeded its threshold on only 1.
Decide while the sync is running (!scanline_hold) so the countdown
completes uninterrupted, and gate the deviation test on frame_time > 0.
The two severities the old code intended but could not reach are now
distinguished: a core structurally over budget takes the long hold and
surrenders the target, a transient deviation takes the short one and
keeps it.
Verified on a 4K120 display: hold now shows 0..59 rather than pinned at
59, tune fires, scanline_next reaches 2156, and the spin runs again.
Two follow-ups this makes visible, neither addressed here. The short
"rearm" path still pre-empts the recovery on the frame hold expires, so
a trickle of deviations stalls tuning for long stretches. And with the
spin actually running it makes 25-41 D3DKMTGetScanLine calls per frame
at ~223 us each - 5.5 to 9.2 ms against an 8.31 ms frame period.
Two things in the same area.
HAVE_D3DKMT was defined only inside the HAVE_D3D12 block, so a Windows
build configured for Vulkan and OpenGL without D3D12 silently lost
Scanline Sync.
The dependency was never real. D3DKMT is the kernel-mode thunk into
dxgkrnl.sys, not Direct3D - gdi32 exports D3DKMTGetScanLine as a
forwarder to ext-ms-win-dx-d3dkmt-dxcore-l1-1-5.#56, DXCore being the
adapter layer deliberately factored out of DXGI. RetroArch opens the
adapter from a plain GDI HDC:
EnumDisplayDevices() -> CreateDC() -> D3DKMTOpenAdapterFromHdc()
with no D3D device, no DXGI factory and no swapchain anywhere. And
d3dkmt_init() is called from win32_window_init(), which every win32
video driver goes through - w_vk_ctx.c:195 and wgl_ctx.c:678 as much as
the d3d ones, plus gdi_gfx and sdl2_gfx. The feature already worked
under Vulkan; it just could not be compiled unless D3D12 was enabled.
Gate it where the code lives instead: the Win32 block that adds
win32_common.o. The vendored dxsdk include path moves with it, since
win32_common.h includes <d3dkmthk.h> for the two structs. Verified that
the D3DKMT block compiles against those headers with no D3D10/11/12
defines present.
Second, d3dkmt_init() called pD3DKMTOpenAdapterFromHdc without checking
it. pD3DKMTGetScanLine is null-checked at both its use sites; this one
was not, and it is called inside the EnumDisplayDevices() loop, so a
missing export meant a call through NULL on the first display device.
Both entry points are WDDM - gdi32 has no D3DKMT exports at all under
XDDM - and the enclosing "if (!pD3DKMTOpenAdapterFromHdc)" meant a
failed resolve left the pointer NULL and every later d3dkmt_init()
retried the enumeration and crashed again.
Bail out with the scanline state zeroed when either export is missing.
d3dkmt_scanline_get() then returns -1, which
video_driver_scanline_before_frame() already handles as unsupported.
Now reachable on more configurations than before, which is the other
reason to add the check rather than leave it.
The round-robin probe in xinput_joypad_poll() autoconfigured with
vid = 0 and pid = 0, and looked the name up with
xinput_joypad_name(xinput_hotplug_index), which resolves to
g_pads[xinput_hotplug_index].joy_name.
Both are wrong in the same way. g_pads[] is indexed by DirectInput
enumeration order; xinput_hotplug_index is an XInput user index;
g_xinput_pad_indexes[] is the mapping between them. They coincide only
when every enumerated pad is an XInput pad. Add a DirectInput-only
stick alongside an Xbox pad and the probe indexes g_pads[] by the wrong
number and reports another device's name.
Search for the entry that maps to the user being probed, the way
dinput_enum_hybrid_autoconf_flush() already does, and take the name,
vid and pid from it. When no entry maps - the pad appeared after the
last enumeration and DirectInput has not seen it - fall back to the
previous behaviour, since there is genuinely nothing to recover.
Worth being clear about the scope: this path is a fallback. A normal
hotplug goes WM_DEVICECHANGE -> winraw_handle_message() ->
joypad_driver_reinit(), which re-enumerates and autoconfigures through
the flush, already with the real vid/pid. The probe only runs when that
did not happen, and it never produced a generic name - only a
possibly-wrong one, and only with mixed XInput and DirectInput-only
pads.
The two paths still disagree on what "port" means - the flush passes
the enumeration index, the probe passes the XInput user index. Left
alone deliberately: changing port numbering would move existing
autoconfig bindings, which is a much larger question than this fix.
b3bd51a6e2's commit message overstated the impact. It claimed that in
every shipped Windows build the hybrid driver "reads the wrong device
state and overwrites pad 0's rumble parameters" with two or more
controllers. That is not true, and testing with two XInput pads showed
both enumerating correctly on the right ports with the right names.
What the message missed: each file also carried its own static copies
of the accessors - dinput_joypad_destroy, dinput_joypad_button,
dinput_joypad_axis, dinput_joypad_name - and only one joypad driver is
active at a time, since input_joypad_init_driver() selects a single
entry from joypad_drivers[]. So whichever driver was running used its
own view of g_pads[] consistently, and the other never touched the
array. The 432-byte definition meant it was over-allocated relative to
the hybrid's 400-byte view, not overrun.
The change itself stands - two divergent copies of a shared struct
under one guard is a trap, and it had already been sprung once when the
rumble storage fields were added to the definer alone. But it is
cleanup, not a bug fix, and the header comment now says so rather than
leaving the misleading version as the only record.
Connecting a second pad re-announces the first: "Xbox One Controller
configured in port 1" appears again alongside the new pad's
notification.
input_autoconfigure_connect_ex() already suppresses a repeat
notification - it compares the incoming name, vid and pid against
what the port already holds and sets
AUTOCONF_FLAG_SUPPRESS_NOTIFICATIONS when they match and the port is
already autoconfigured. That check never fires here, because
dinput_joypad_destroy() blanks input_device_info[port].name for every
port on the way out, and a device arrival tears the driver down:
WM_DEVICECHANGE -> winraw_handle_message() -> joypad_driver_reinit().
By the time the re-enumeration announces the same pad, the field the
comparison reads is empty.
The clear is not the disconnect mechanism. Both drivers announce
disconnects from poll(), and joypad_driver_reinit() runs poll() one
last time before destroy() specifically to catch them -
input_autoconfigure_disconnect() then clears the name, display name,
config name, driver, phys, vid and pid together. Clearing only the
name at teardown left vid, pid and the autoconfigured flag behind: a
partial disconnect that no other joypad driver in the tree performs,
and the only two that did are these.
Remove it from both. Consequence worth stating: after switching to a
joypad driver that enumerates fewer pads than the previous one, the
higher ports keep the old device name until something overwrites it.
Every other joypad driver already behaves that way.
g_pads[] is defined in dinput_joypad.c and read by
xinput_hybrid_joypad.c, which cross-references DirectInput to recover
the VID/PID that XInput does not expose. Both files carried their own
copy of struct dinput_joypad_data, each wrapped in the same
__DINPUT_JOYPAD_H guard, and the copies had drifted:
dinput_joypad.c 432 bytes
xinput_hybrid_joypad.c 400 bytes
The four missing fields are rumble_axis, rumble_direction,
rumble_envelope and rumble_force, added to the definer when
rumble_props was fixed to stop pointing at a dead stack frame. The
reader never got them.
On the console builds this is harmless: griffin includes
dinput_joypad.c first, its guard fires, and the hybrid's shorter copy
is skipped, so the whole unity build sees one layout. The Windows
build does not go through griffin - Makefile.common compiles
dinput_joypad.o and xinput_hybrid_joypad.o separately whenever
HAVE_DINPUT and HAVE_XINPUT are both set, which is the normal
configuration - and there the two objects disagree on the array
stride by 32 bytes per element.
So in every shipped Windows build, g_pads[i] in the hybrid driver
resolves 32*i bytes below the real element for i >= 1, landing inside
the previous pad's rumble storage. One controller is unaffected. Two
or more and the hybrid driver reads the wrong device state and
overwrites pad 0's rumble parameters while doing it.
Move the struct and the four externs into a real dinput_joypad.h and
delete both in-file copies. Verified with two translation units built
separately against the header: 432/432 and agreeing, where the old
pair reported 432 in the definer and 400 in the reader.
xinput_joypad.c (the non-DirectInput variant) never referenced g_pads,
and input/drivers/dinput.c has no copy of its own, so neither needed
changing.
dinput_poll() cleared di->state with a pointer loop, leaving kb_state
at di->state + 256. The clear on the GetDeviceState() failure path then
reused that same pointer:
for (; kb_state < di->state + 256; kb_state++)
*kb_state = 0;
The condition is false on entry, so the body never ran. The intent -
zero the state when the device cannot be read - was not happening.
It has not caused visible trouble because the first loop already
cleared the buffer before the call. That only holds if a failing
GetDeviceState() leaves the buffer untouched, which DirectInput does
not promise; a partial write before DIERR_INPUTLOST would be read as
live key state and, since dinput_poll() runs once per frame with no
other writer, stay that way until the device is reacquired.
dinput_joypad_poll() in the sibling driver gets this right: it memsets
joy_state on the same failure and re-centres rgdwPOV, with a comment
about D-pad directions latching across focus loss. The keyboard path
now does the same thing.
Both clears become memset, which is what the loop was open-coding, and
kb_state goes away with them.
thread_alive is written by the main thread in dsound_stop() and
dsound_start(), and by the mixer thread itself at dsound_thread():293
when the buffer can no longer be locked. It is read by the mixer
thread's loop condition and by dsound_write(), dsound_write_avail()
and the blocking wait, all on the main thread. Written on both
threads, read on both.
It was a volatile bool, which gives ordering under MSVC /volatile:ms
and none under /volatile:iso - what ARM64 builds get. The data path
beside it already goes through retro_spsc with proper acquire/release,
so this was the one field in the driver not stating its ordering.
xaudio.c made the same change for the same reason and records the
reasoning in a comment; asio.c followed in 49a96fba65.
retro_atomic_int_t with acquire loads and release stores. Unchanged
codegen on x86-64 - the loads and stores are plain movs either way -
and the barriers appear on ARM where the mixer thread needs them.
ds is calloc'd, so it still starts clear.
is_paused, nonblock and use_float are untouched: none of them is read
inside dsound_thread(), so they do not cross.
ra_asio_t::running was assigned in five places and read in none. It
tracked whether the driver was streaming, but g_asio already carries
exactly that: every "running = true" sits immediately after
"g_asio = ad", every "running = false" immediately before or beside
"g_asio = NULL", and the comment above the parked-instance handling in
ra_asio_free() names the real gate outright - "g_asio == NULL gates
it".
Dead state is worse than no state here. It was declared volatile,
which invites the next reader to believe something checks it across
threads, in a driver whose init/park/restart lifecycle is already
subtle enough to need three paragraphs of comment.
No functional change: nothing read it.
asio_cb_buffer_switch() runs on the ASIO driver's realtime thread and
takes its early-out on ad->is_paused and ad->shutdown. Both were plain
scalars - shutdown volatile, is_paused not even that - while every
other cross-thread field in that path goes through retro_spsc_* or
retro_atomic_*.
They are genuinely cross-thread in both directions. is_paused is
written by ra_asio_set_state() and ra_asio_free() on the main thread;
shutdown is written by ra_asio_init(), ra_asio_free() and also by
asio_request_shutdown(), which asio_cb_message() calls from the
driver's own thread on kAsioResetRequest and kAsioBufferSizeChange.
volatile gives ordering under MSVC /volatile:ms and nothing under
/volatile:iso, which is what ARM64 builds get, so make both
retro_atomic_int_t with acquire loads and release stores. On x86-64
this is unchanged codegen - the loads and the store are plain movs
either way - and on ARM the barriers appear where the callback needs
them.
Nothing else in that callback needed touching. It reads the ring
through retro_spsc with the acquire/release pairing already spelled
out, touches the cursors once per callback rather than once per frame,
allocates nothing and takes no lock, and scond_signal() returns after a
single atomic load when there is no waiter.
Noticed while auditing but left alone: ra_asio_t::running is written in
five places and read in none.
main_window.hwnd and video_st->window are the same HWND held twice.
win32_window_create() copies the first into the second at
win32_common.c:1552, and win32_destroy_window() nulls the first.
Nothing nulls the second: the only assignment of zero to it is at
video_driver.c:4537, inside video_driver_init_internal(), which is an
initialisation reset rather than a teardown.
So from win32_destroy_window() until the next video driver init the two
disagree, and video_driver_window_get() returns a destroyed HWND.
Callers pass that straight to Win32: ScreenToClient() and
GetClientRect() in dinput and winraw, and
IDirectInputDevice8_SetCooperativeLevel() in dinput and the joypad
drivers. Those fail rather than crash, but they fail for a reason that
looks like nothing.
Clear it where the window actually dies, so both reach zero together
and the Win32 side owns the invariant at the single point that breaks
it.
USE_CTRULIB_2 now defaults by probing the toolchain's gspgpu.h for
gspPresentBuffer, so plain make builds correctly against both current
libctru and the legacy ctr toolchain with no flag on the make line.
An explicit USE_CTRULIB_2=0/1 still overrides the probe, and a preset
CTRULIB is honored. The salamander makefile gains the same probe, so
the two makefiles agree on any given install.
Legacy libctru remains the out-of-the-box selection, keeping the
ctr-legacy CI lane and existing build setups working unmodified.
Builds on current devkitPro pass USE_CTRULIB_2=1 on the make line,
as the modern CI jobs already do.
The ctr-legacy image carries a pre-2.0 libctru, so the precompile pass
now states the legacy libctru selection explicitly instead of leaning on
Makefile.ctr's default, mirroring the modern job's USE_CTRULIB_2=1. The
retroarch-precompiled artifact then matches the toolchain the static
core jobs compile and link the remaining frontend objects with.
1a2a1fe739 moved winraw_init_mouse_xy_mapping() to winraw_poll(),
because it writes the mouse position and the wndproc no longer owns
that. It reproduced the call site as a per-frame test of
WRAW_INP_FLG_MOUSE_XY_MAPPING_READY, which is not the condition the
wndproc used.
The original ran it only when a MOUSE_MOVE_ABSOLUTE report arrived and
the mapping was not yet built, so for an ordinary relative mouse it
never ran at all. Two unintended consequences followed from the
rewrite:
winraw_init_mouse_xy_mapping() returns early when
video_driver_get_viewport_info() fails, without setting the ready flag,
so while the viewport is unavailable the check re-ran and called it
every frame instead of once.
And when it does succeed it calls winraw_sync_mouse_to_cursor(), which
snaps every mouse's position to the system cursor. That now happened at
startup for relative mice, which it never did before. Harmless in
effect, but not a change that was asked for or written down.
Restore the original trigger: the wndproc raises wr->map_pending when
it sees an absolute report with no mapping, and poll builds it on the
next frame. Same condition as before, one frame later, and nothing runs
for devices that never send absolute reports.
g_mice[i].x/y had two writers on two threads, both doing
read-modify-write. The wndproc accumulated deltas into them and
overwrote them for absolute reports; winraw_poll() overwrote them from
the system cursor and on focus regain. The failure mode is lost
updates, not tearing, so making the fields atomic would have fixed
nothing and only made the code read as though the question were
settled - which is what the comment at the top of the file has been
saying for several commits.
Make the position belong to one thread. winraw_poll() is now the only
writer, and the wndproc neither reads nor writes it. Instead it
publishes what it knows and poll derives the position once per frame
from whichever source applies:
pos_pending sample the system cursor (highest precedence)
abs_pending use the published abs_x/abs_y
dlt_x/dlt_y advance by the frame's delta, clamped to active_rect
All three are already atomic or become so. The wndproc keeps its own
abs_ref_x/abs_ref_y - untouched by any other thread - to turn
successive MOUSE_MOVE_ABSOLUTE reports into deltas without reading the
position it no longer owns.
The client-rect check moves from the wndproc to poll for the same
reason: active_rect is read by the clamp and view_abs_ratio_* is set by
winraw_init_mouse_xy_mapping(), and both now sit on the thread that
owns the position. That also removes rect_check_pending, which existed
only to run the check once per frame from inside the wndproc; running
it directly in poll is the same thing without the flag. The
mapping-not-ready case the wndproc used to handle inline becomes a
check at the top of poll.
Behaviour is unchanged in the ordinary case. Relative motion now
advances by the frame's accumulated delta in one step rather than by
each report in turn, so the active_rect clamp applies once instead of
per report - the endpoint is the same unless a mid-frame position would
have been clipped. Absolute reports and the cursor path land on the
same values they did before, one frame later at worst.
The wndproc's remaining writes are all to atomics, so the only
cross-thread reads left are view_abs_ratio_* and the MOUSE_XY_MAPPING
flag, written by poll on a resolution change and read by the wndproc's
absolute branch. Those are 8-byte aligned doubles, atomic on x86-64;
on 32-bit a torn read would cost one misplaced report from an absolute
pointing device during a resize.
The threading note already said that whoever drains raw input owns it
process-wide, in the context of SDL2 doing that to this driver. It did
not say that the same is true in the other direction, which is the
concrete case someone will hit.
GetRawInputBuffer() drains the queue for the whole thread. A drain in
winraw_callback() therefore takes the joystick and gamepad records that
winraw_joypad registered against its own HWND_MESSAGE window, created
on this same thread by input_driver_init_joypads() right after
winraw_init(). d087a820cd did exactly that and killed pad input;
forwarding the RIM_TYPEHID records back fixed the pads but keyboard and
mouse buttons stayed broken for a reason that was never found, and
7858994d45 reverted it.
Also records the size of the prize, so the next person can decide
before writing the code rather than after: measured at 8.32 reports per
frame with a 1000 Hz mouse at 120 fps, a drain removes roughly seven of
about seventeen syscalls per frame. It cannot touch the other ten -
WM_INPUT is still posted per report and PeekMessage() with PM_REMOVE
has no user-mode fast path, and no registration flag suppresses the
messages while keeping the data.
Comment only, no functional change.
GetRawInputData() has no user-mode path on any Windows version - the
24H2 export is a six-byte jmp to win32u!NtUserGetRawInputData, and XP's
is a four-instruction syscall stub - so the per-message form costs one
kernel transition per report. Measured on a 1000 Hz mouse at 120 fps:
8.32 reports per frame, 999 per second, no coalescing by the input
stack.
GetRawInputBuffer() returns as many records as fit in one call, so a
drain on the first WM_INPUT of a frame collects the backlog and the
later messages find an empty buffer. It also has a user-mode fast path
GetRawInputData() lacks: it reads the raw-input change bit out of
CLIENTINFO.pClientThreadInfo at TEB+0x860 and answers "nothing
buffered" without entering the kernel at all.
That removes roughly seven of the eight per-frame syscalls under
sustained motion. It does not touch the other half of the cost: WM_INPUT
is still posted per report and PeekMessage() with PM_REMOVE has no fast
path, so the pump still makes one transition per message. No
registration flag suppresses the messages while keeping the data.
64-bit only, deliberately. In a 32-bit process under WOW64 the returned
records carry the 64-bit header size, so the block walk goes off by
eight bytes per record from the second one onward and reads garbage.
The correction is possible, but it is delicate code on the
configuration where getting it wrong means silently corrupt mouse
input, and RetroArch ships i686 builds - see Windows-i686-MXE.yml. A
compile-time gate means that code does not exist rather than existing
untested; 32-bit keeps the per-message path unchanged. The comment on
winraw_drain() says not to unify the two without handling it.
The 56-line dwType switch moves verbatim into winraw_handle_record();
read this with --color-moved. Two things changed while extracting it.
The ignored-scancode cases used to "return 0" straight out of the
window procedure, skipping DefWindowProcA(); they now return from the
helper and the message is passed on as MSDN requires for WM_INPUT
cleanup. And the RIM_INPUTSINK test is now a positive check for
RIM_INPUT before either path runs, which is the same condition read
forwards.
NEXTRAWINPUTBLOCK() is spelled out rather than used: mingw-w64's
RAWINPUT_ALIGN() references QWORD, which its headers never define, so
the macro does not compile there and would have broken the MXE builds.
The drain buffer is 16 KiB, allocated once at init and freed on both
teardown paths. A ~200 ms startup hitch at 1000 Hz queues ~200 records
of 40 bytes, so that covers roughly double the worst backlog observed;
the loop runs until the buffer comes back empty regardless, so an
undersized buffer costs an extra call rather than dropped input. malloc
is at least 16-byte aligned, which satisfies the 8-byte alignment the
records need.
rthreads drives the Vita kernel directly and asks nothing of the pthread
port, but the port is still what satisfies libstdc++.a. A C++ core enters
the link under --whole-archive, so its function-local static guards, the
EH allocation pool and std::locale all arrive wanting gthread mutexes,
and FBNeo brings its own on top through r8brain's CSyncObject, which
takes the raw pthread path below C++11. libpthread is a static archive at
the tail of the line, so a build that references none of it links nothing
extra: the flag costs pure-C targets nothing and keeps every C++ core on
the platform buildable.
Write bin2s output to a temporary .s and assemble it as a separate
step, so a bin2s failure aborts the rule instead of being masked by
the exit status of the assembler at the end of the pipeline. The
shader rules also go through CTR_PICASSO and CTR_BIN2S now, picking
up the .exe suffix on MSYS.
Ignore the .shbin and .shbin.s intermediates and drop them on clean,
so a rule that stops early leaves nothing behind that can reach a
commit.
wr->window is created on whichever thread runs winraw_init(), and
WM_INPUT is posted to that thread's queue. Only a pump running on the
same thread will dispatch it, and the pump belongs to whichever thread
owns the main window - the video thread under video_threaded.
On every Windows video driver that supports this input driver the two
match, because the driver is built from the video context driver's own
input_driver callback and so runs wherever the video driver's init()
did. The exception is a video driver that leaves *input NULL: then
video_driver_init_input() picks the configured driver itself, on the
main thread, after the video thread already exists. sdl2 is the only
one that does that today, and sdl2 plus raw is separately broken
because SDL2 drains the raw input stream itself, so the mismatch is
currently unreachable - but it is one video driver away from being
reachable, and the symptom would be keyboard and mouse silently doing
nothing with no indication why.
Compare the two thread IDs at the end of winraw_init() and log an error
naming both. One GetWindowThreadProcessId() call at init.
Skipped when main_window.hwnd is NULL, since that means there is no
win32 main window to compare against and the check cannot conclude
anything.
Run the ctr-shaders recipe as a single shell command that writes
bin2s output to a temporary file and moves it into place only once
picasso and bin2s have both succeeded, so the checked-in
gfx/drivers/ctr_shaders/ctr_sprite_shbin.S survives a partial or
failed regeneration. A trap clears the intermediate .shbin and the
temporary on the way out, leaving no untracked debris in the tree.
Ship the assembled PICA200 shader as
gfx/drivers/ctr_shaders/ctr_sprite_shbin.S and select it whenever
picasso or bin2s are missing from the devkitPro install, so a 3DS
build needs nothing beyond devkitARM itself.
Regenerate the prebuilt shader with 'make -f Makefile.ctr ctr-shaders'
after editing ctr_sprite.vsh or ctr_sprite.gsh.
Several patches in this area were written against an assumed thread
split, and one open question - whether WM_INPUT is dispatched at all
under video_threaded - stayed open for a while because the answer is
not local to this file. Record it here so the next person does not
have to trace it again.
The trace: on every Windows video driver that can use this input
driver, it is built by the video context driver's input_driver
callback, not by the runloop, and under video_threaded the video
driver's init() runs on the video thread via CMD_INIT in
video_thread_wrapper.c. So wr->window, the HWND_MESSAGE window that
RegisterRawInputDevices() targets, is created on whichever thread also
owns the main window and runs the pump. Since
ui_application_win32_process_events() peeks with a NULL hWnd, it
retrieves messages for every window of the calling thread, and
WM_INPUT arrives correctly in both configurations. There was no bug.
The comment also records the fallback path, because it is the case
where that reasoning does not hold. A video driver may leave *input
NULL, and video_driver_init_input() then initialises the configured
driver on the main thread, after the video thread already exists. A
winraw window created there would not share a thread with the pump.
sdl2 is the only Windows driver that does this, and sdl2 plus raw is
already broken and warned about because SDL2 drains the raw input
stream itself, so the case is unreachable in practice - but it is one
video driver away from being reachable.
The split that does exist is between winraw_callback(), on the window's
thread, and winraw_poll(), always on the main thread via
input_driver_poll(). That is why dlt_x/dlt_y, whl_u/whl_d, pos_pending
and rect_check_pending are retro_atomic_int_t.
Also records why x and y are deliberately not atomic. 5e89f9e363 left
them plain and called it "worth a separate look"; the look concluded
that atomics are the wrong tool. They have two writers doing
read-modify-write, so the failure is lost updates, which atomicity does
not address - it would only make the code read as though the question
were settled. Sole ownership of the position by one thread would fix
it, and that is a restructure.
Comment only, no functional change.
file_watch_poll() runs on the frame loop, once per frame while
video_shader_watch_files is on, and its idle path was
if (WaitForSingleObject(d->evt, 0) != WAIT_OBJECT_0)
continue;
per watched directory. A zero-timeout wait reads as a cheap "is it
signalled" test but is not one. WaitForSingleObject has no user-mode
fast path on any Windows version - kernel32 forwards to
KernelBase!WaitForSingleObject, which is
"xor r8d,r8d; jmp WaitForSingleObjectEx", and that translates the std
handle pseudo-values, converts the timeout and calls
ntdll!NtWaitForSingleObject unconditionally. XP and 9x have the same
shape. So the idle cost was one syscall per directory per frame,
whatever the object's state.
HasOverlappedIoCompleted() reads OVERLAPPED::Internal, which the I/O
manager writes when the operation completes, and compiles to a load
and a compare against STATUS_PENDING.
The two are interchangeable here. The event is created manual-reset
and file_watch_arm() calls ResetEvent() explicitly before re-issuing
ReadDirectoryChangesW(), so nothing relied on the wait clearing it.
The overlapped is armed during init before any poll can run, and a
directory whose arm fails is torn down and skipped by the d->dir guard
at the top of the loop.
Not applied to the other zero-timeout wait in the tree,
net_icmp_echo_poll(): its handle is the Event argument to
IcmpSendEcho2(), not an OVERLAPPED, so there is no field to read and
the wait is the correct way to poll it.
winraw_update_mouse_state() opened with an unconditional
GetClientRect((HWND)video_driver_window_get(), &wr->prev_rect);
if (!EqualRect(&wr->active_rect, &wr->prev_rect)) ...
to notice a resolution change. It runs once per raw input report, about
sixteen times per frame with a 1000 Hz mouse, to observe something that
cannot change more than once per frame.
GetClientRect itself is cheap - it resolves the window through the
one-entry CLIENTINFO cache and subtracts, with no kernel transition on
any Windows version - but it gates the part that is not.
winraw_init_mouse_xy_mapping() does a video_driver_get_viewport_info()
and a winraw_sync_mouse_to_cursor(), and the latter is a GetCursorPos()
syscall. During a resolution change the rect differs on consecutive
reports, so that ran up to ten times, and at a high polling rate all
ten landed inside one frame. 963abf2e99 had just removed the same
syscall from the neighbouring path.
Gate the whole check on a flag winraw_poll() raises once per frame and
the first report of that frame consumes, so it happens at most once per
frame and not at all in frames with no mouse activity - which is the
same trigger condition as before, just deduplicated.
One consequence worth stating: rect_delay now counts frames rather than
reports. That is the unit the surrounding comment implies ("Triggering
fewer times seems to fix the issue"), and it removes a settle window
that silently depended on mouse polling rate - roughly five frames at
125 Hz, under one frame at 1000 Hz, now ten frames either way. Longer
than before in both cases, in the direction that comment asks for,
though whether ten is the right number is a separate question.
wr is calloc'd, so the flag starts clear.
Nothing between the stock-shader line and the joypad driver logs
anything, so a crash anywhere in default filter chain construction,
the context input hook, or the joypad init truncates the log at the
same place and the report cannot say which. Issue #19481 is stuck
there.
Log the stages at RARCH_DBG, so they cost a log-level check in
nightlies and cost nothing at the default level. The per-pass line
already existed behind VULKAN_DEBUG, which no shipped build defines;
move it to the same runtime gate as the rest.
dlt_x, dlt_y, whl_u, whl_d and pos_pending are produced by the wndproc
and drained once per frame by winraw_poll(), which under video_threaded
are different threads. They were plain LONG accessed with Interlocked*,
which is a full barrier on x86/x64 but not on ARM, and RetroArch ships
ARM64 Windows builds. They become retro_atomic_int_t with the ordering
stated rather than assumed.
x and y stay plain: they are written and read across the same two
threads but were never accessed atomically, and making them atomic
would change the type of the single-threaded snapshot in
winraw_input_t::mice that shares this struct. Worth a separate look.
Six of the seventeen sites discarded the value InterlockedExchange
returns, so they were paying for a locked read-modify-write to perform
a store. Those become retro_atomic_store_release_int:
old b8 01 00 00 00 mov eax,1
87 41 10 xchg DWORD PTR [rcx+0x10],eax
new c7 41 10 01 00 00 00 mov DWORD PTR [rcx+0x10],1
The hottest of them is the pos_pending store, which runs once per raw
input report. The consumer side keeps a real exchange, since the
test-and-clear and the per-frame drain both need the old value.
The accumulating sites are unchanged in output: GCC already emitted
lock add rather than lock xadd once the fetch_add result went unused.
Verified at -std=c89 and -std=gnu99 that the snapshot assignment,
the int64_t returns in input_state and the init-time memcpy of
winraw_mouse_t all still compile against the atomic member type.
IMM_AddRef/IMM_Release used InterlockedIncrement/InterlockedDecrement
on a plain LONG. The decrement is the one that matters: another
thread's last use of the object has to be ordered before the free()
that follows the count reaching zero. Interlocked* is a full barrier on
x86/x64, so this is correct there, but on ARM it is not, and RetroArch
ships ARM64 Windows builds.
retro_atomic_fetch_sub_int is documented acq_rel and its MSVC backend
emits __dmb around the RMW on ARM, which is exactly the pairing a COM
refcount needs. On x86/x64 it expands to the same Interlocked* call.
The QueryInterface increment discards its result, and
retro_atomic_inc_int returns void, so it now compiles to
f0 83 41 08 01 lock add DWORD PTR [rcx+0x8],0x1
instead of the lock xadd that InterlockedIncrement forced by always
producing the new value. Incidental, but free.
refCount becomes retro_atomic_int_t; mmdevice_common_inline.h is only
reached from C translation units today, so the C++ backend's
std::atomic<int> spelling is not in play, and the struct is never
copied wholesale.
rtime_sleep_state is published by rtime_sleep_init() after it resolves
CreateWaitableTimerExW/SetWaitableTimer, allocates the TLS slot and
stores rtime_sleep_tls. Readers that observe the published state then
dereference those. The store used InterlockedExchange but every read
was a plain load of a volatile LONG:
if (rtime_sleep_state == RTIME_SLEEP_UNKNOWN)
if (rtime_sleep_state != RTIME_SLEEP_HIGHRES ...
On x86/x64 that happens to work - the locked store is a full barrier
and MSVC's default /volatile:ms gives the loads acquire semantics. On
ARM64, where MSVC uses /volatile:iso and Interlocked* is not a full
barrier, neither side is ordered, so a thread can observe
RTIME_SLEEP_HIGHRES while rtime_sleep_tls still reads
TLS_OUT_OF_INDEXES. RetroArch ships ARM64 Windows builds.
Make the pairing explicit: retro_atomic_store_release_int on publish
and teardown, retro_atomic_load_acquire_int on every read, and
retro_atomic_cas_int for the once-only init guard. retro_atomic's MSVC
backend emits __dmb around the ARM RMWs for exactly this reason, and on
x86/x64 it expands to the same Interlocked* calls, so the generated
code there is unchanged.
retro_atomic.h is included unconditionally rather than under
HAVE_THREADS, since the Win32 sleep block is guarded only on
_WIN32 && !_XBOX && !__WINRT__.
No functional change on x86/x64. Verified: same four pre-existing
-Wcast-function-type warnings before and after, at -std=c89 and
-std=gnu99, with and without HAVE_THREADS.
The filtered set is allocated num_parameters entries but filled while
walking reflection.num_float_parameters, one store per live entry. The
std::vector it replaces grew on demand, so the two counts never had to
agree. They agree today only because semantic_float_parameters is
sized off the parameter map, and a pass with no parameters leaves the
array NULL while the walk is still free to store into it.
Bound the walk by both counts and size the allocation off that bound.
slang_pass_init_pipeline_layout() frees bindings and desc_counts on
the set-layout and descriptor-pool paths, but the push-constant
allocation failure and vkCreatePipelineLayout() failure in between
returned straight out, leaking both arrays. Free them there too, and
reindent the push-constant block while it is being touched.
The framebuffer path guarded pass->common while the reflection setup
twenty lines further down dereferenced it unconditionally for the two
semantic maps, so the guard bought nothing and a NULL common would
have faulted there instead. Check it once at the top and drop the
ternary.
gdi32's SwapBuffers does not implement anything. On every Windows
version it locates opengl32!wglSwapBuffers by name and forwards to it,
and it re-resolves it on every call:
Win95 OSR2 4.00.950 GetModuleHandleA("OPENGL32")
GetProcAddress("wglSwapBuffers"), call it
Win98 SE 4.10.1998 byte-identical to Win95
XP SP3 5.1.2600.5512 LoadLibraryA, GetProcAddress, call, FreeLibrary
24H2 10.0.26100 LoadLibraryExA, GetProcAddress, FreeLibrary,
and the resolved pointer is discarded - it is
tested for NULL and the swap dispatches
through a separate runtime-patched slot
So the current desktop path pays, per frame, two loader-lock round
trips and an export-table lookup in a ~750 KB DLL, to reach a function
this process already imports.
opengl32 is already a load-time dependency: Makefile.common links
-lopengl32 for GL builds and this file calls wglCreateContext,
wglShareLists, wglMakeCurrent and wglGetProcAddress unconditionally.
By the time gfx_ctx_wgl_swap_buffers() runs, a context has been created
and made current, so opengl32 is not merely loaded but active. There is
no state where SwapBuffers would succeed and wglSwapBuffers would not.
The branch was not a deliberate choice. mingw-w64's wingdi.h declares
every other wgl entry point used here but omits wglSwapBuffers, so the
desktop path reached for the gdi32 spelling because that was the one
with a prototype. Declare it, as the __WINRT__ block already did, and
the two paths collapse into one.
Untested surface: overlay and capture software hooks the GL swap. Most
GL hooks target opengl32!wglSwapBuffers, which would make this neutral
or better, but anything hooking gdi32!SwapBuffers instead would stop
seeing RetroArch's swaps. That cannot be established from the binaries
and is the thing to look at if an overlay regression is bisected here.
winraw_update_mouse_state() called GetCursorPos() from the wndproc, so
once per raw input report. GetCursorPos has no user-mode path on any
Windows version - it is 'mov edx,1; jmp win32u!NtUserGetCursorPos' on
24H2, an NtUserCallOneParam syscall on XP, and a 32->16 FT_Thunk on 9x -
so that is an unconditional kernel transition per report. A 1000 Hz
mouse at 60 fps makes about sixteen of them per frame where one would
do. ScreenToClient on the following line is free everywhere: it is two
subtractions against the client-side window mapping.
Only the value in place when winraw_poll() takes the frame snapshot is
ever read. mouse->x is write-only between snapshots in this branch -
the MOUSE_MOVE_ABSOLUTE branch and the non-cursor branch are the only
readers and neither runs here - so every query but the last is
overwritten unobserved.
Set a per-mouse pos_pending flag instead and resolve it in
winraw_poll(), at most once per frame however many reports asked and
however many mice are attached. The sample is taken at poll time rather
than at the last report, so it is fresher, not staler. Moving the call
across threads is safe: under video_threaded the wndproc runs on the
video thread while winraw_poll runs on the main thread, and
NtUserGetCursorPos reads global state with no message-queue or TEB
involvement, unlike GetKeyState.
pos_pending is drained with InterlockedExchange, matching how dlt_x and
whl_u already cross the same thread boundary, and calloc zeroes it at
device enumeration.
One case changes. A frame whose reports switch from wanting the cursor
to not wanting it - the menu closing between two reports - now advances
from the previous frame's position rather than from a cursor sample
taken mid-frame. The divergence is bounded by one frame of pointer
motion and only occurs on the transition. Frames that consistently want
the cursor, which is the normal case, are unaffected apart from the
fresher sample.
In winraw_update_mouse_state(), the getcursorpos branch logs and falls
through when GetCursorPos() or ScreenToClient() fails:
if (!GetCursorPos(&crs_pos))
RARCH_DBG(...);
else if (!ScreenToClient(..., &crs_pos))
RARCH_DBG(...);
...
mouse->x = crs_pos.x;
crs_pos is an uninitialised local at that point, so the failure path
stores stack garbage into the mouse position. The logging is debug-only
but the store is not.
Seed crs_pos with the current position before the branch, so a failed
query leaves the mouse where it was.
winraw_update_mouse_state() derived the absolute position from the
frame-accumulated delta:
InterlockedExchangeAdd(&mouse->dlt_x, state->lLastX);
crs_pos.x = mouse->x + mouse->dlt_x;
...
mouse->x = crs_pos.x;
dlt_x is only zeroed by winraw_poll(), once per frame, so every message
after the first in a frame re-adds the deltas of the messages before it.
Position advances by N(N+1)/2 * d over a frame of N reports instead of
N * d:
1 report of +1 per frame -> x advances 1 (correct 1)
2 reports of +1 per frame -> x advances 3 (correct 2)
4 reports of +1 per frame -> x advances 10 (correct 4)
16 reports of +1 per frame -> x advances136 (correct 16)
At one report per frame it is exactly right, which is why it survived
since 044e1ec655. The error scales with reports per frame, so a 125 Hz
mouse at 60 fps overshoots about 1.5x and reads as "too sensitive",
while a 1000 Hz mouse at 60 fps overshoots about 8.5x and pins to the
active_rect clamp.
Use the delta of the message being handled. That is what the lightgun
branch above already does by assigning dlt_x with InterlockedExchange
rather than adding, so both branches now advance identically, and the
result is unchanged for every case that was already correct: lightguns,
and any device seeing one report per frame.
dlt_x itself was always right - winraw_poll drains the true sum - so no
relative consumer changes. Only the absolute readers are affected:
RARCH_DEVICE_MOUSE_SCREEN in input_state and the pointer fallback.
dinput_poll() called GetAsyncKeyState four times to decide whether the
latched shift and alt keys had been released. VK_LSHIFT, VK_RSHIFT,
VK_LMENU and VK_RMENU are all at or above 0x20, which is the cutoff for
user32's async key cache, so each one was an unconditional transition
into win32u!NtUserGetAsyncKeyState on every poll where the matching
flag was set.
di->state already holds the answer. IDirectInputDevice8_GetDeviceState
fills it a few lines earlier in the same block, unconditionally, and it
is a DIK-indexed 256-byte array with 0x80 as the down bit - the form
every other read in this file already uses. The same four DIK codes are
used a little further down to set these flags in the first place.
Four syscalls become four array reads with no additional call.
One behavioural difference is worth stating. The device is acquired
DISCL_NONEXCLUSIVE | DISCL_FOREGROUND, so it is unacquired when the
window leaves the foreground; the reacquire then fails and the block
above zeroes di->state. The alt release therefore fires at focus loss
rather than when alt is physically let go afterwards. The shift pair is
unaffected, since the case it exists for - the missing WM_KEYUP when
both shift keys are held and one is released - happens while focused
and acquired.
8d6bfc4b29 hedged on what GetKeyState returns outside the one binary
that had been read at the time. The encoding has since been checked
against three implementations by disassembling the exports:
user.exe 4.10.2222 Win98 SE, 16-bit mov bx,0xff80
user32.dll 5.1.2600.2180 XP SP2, x86 or edi,0xff80
user32.dll 10.0.26100.7462 24H2, x64 or ax,0xff80
and GetAsyncKeyState in the same three returns 0x8000, bit 7 clear.
So the 0x80 mask the callers used to apply was correct everywhere, it
was simply undocumented and inconsistent between the two calls.
The cost argument was understated for the older targets. On 9x user32
implements neither function: GetKeyState and GetAsyncKeyState are
four-byte stubs that load a thunk ordinal and jump to a shared
dispatcher, which pushes the VK as a 16-bit value and calls into
user.exe through FT_Thunk. Every query was a 32->16 transition. And
there GetKeyboardState is a flat 256-byte copy of the key table in the
layout it is already stored in, so the replacement is also less work
per call and not only fewer calls.
Comment only, no functional change.
thread_prefer_fast_cores was added to settings_def_menu_threaded_data.h
without updating tools/settings_table.reference, so the drift check in
Settings-Checks.yml has been failing on master.
Regenerated with tools/settings_check.py table --write. The only change
is the new row appearing in all three emitted tables, which is what the
def row produces; no other row moved.
keyboard_led() runs on the main thread: led_driver_init from
retroarch.c and led_driver_set_led wired into the core LED interface in
runloop.c. Under video_threaded the main window and its message pump
belong to the video thread, so the main thread's synchronous key table
is never advanced and GetKeyState reports every lock as clear.
That made keyboard_init() record all three locks as off, and every
subsequent set saw status 0 and fired keybd_event to toggle a lock that
was already lit. keyboard_free() then "restored" all of them to off.
Take the lock state from the mask win32_update_keyboard_mods()
publishes from the owning thread, which is correct in both video
threading configurations.
The mask advances when the window procedure sees a key event, so a lock
changed while RetroArch is not focused is still not observed until the
next key reaches the window. That is a smaller window than before and
matches what the rest of the Win32 input path already assumes.
GetKeyboardState reads the calling thread's synchronous key table, and
that table only advances as that thread dispatches keyboard messages.
Which thread owns the main window depends on video_threaded:
video_threaded = false window and pump on the main thread
(runloop.c, guarded on !THREADED)
video_threaded = true window created under CMD_INIT inside
video_thread_loop, and the pump moves with it
to win32_check_window on the video thread
input_driver_poll runs on the main thread either way, so in the second
configuration dinput_poll and winraw_callback were reading a queue that
never receives keyboard input and always got an empty mask. Nothing
calls AttachThreadInput, so the two queues are never bridged.
Split the read from the query. win32_update_keyboard_mods() re-reads the
table and publishes the result; it is called only from the main
window's procedures and from win32_set_video_mode, all of which are on
the owning thread by construction. win32_get_keyboard_mods() now
returns the published value and is safe from anywhere, so the existing
dinput and winraw call sites become correct without changing.
Publishing through retro_atomic_store_release_int keeps the cross-thread
handoff off the sanitizer's list. The mask is seeded at window creation
so the lock states are not reported clear before the first key event.
Only the mask was affected. The synthetic modifier key-up events in
dinput_poll are driven by GetAsyncKeyState, which reads global physical
state and is thread-agnostic.
The default was defined inside the GEKKO/PSP/PS2 branch of the
threaded-tasks block, so every other build had the configuration
row referring to an undeclared name. It now sits after that block,
unconditional and false everywhere: pinning the main and audio
threads to the fast cores trades battery for latency, so it stays
the user's call regardless of platform.
keyboard_led() tested the whole GetKeyState return for truth. A lock key
that is physically held reads as 0xFF80 or 0xFF81, which is truthy even
when the lock itself is off, so the initial state captured at
keyboard_init() could be recorded as lit for a key whose LED was dark -
and then "restored" to on at deinit. The same conflation misfires on
every subsequent set, since state and status are compared as booleans.
Mask to bit 0, which is the documented toggle bit.
Five copies of the same block queried GetKeyState once per modifier and
tested the result with 0x80 (down) and 0x81 (down or toggled).
Neither mask is documented. GetKeyState returns a SHORT in which only
bit 15 is contractual; 0x80 happens to work because user32 builds the
down value as 0xFF80, and on the slow path the encoding comes straight
out of win32k and is not guaranteed at all. GetKeyboardState fills the
documented 256-byte form of the same per-thread table, 0x80 down and
0x01 toggled, which is stable across every Windows and under Wine.
0x81 also conflated the two states: holding CapsLock while the lock was
off reported RETROKMOD_CAPSLOCK. The toggle keys now test 0x01 only.
The single call is cheaper as well. user32's GetKeyState cache covers
virtual-key codes below 0x20 and nothing else, so VK_NUMLOCK, VK_SCROLL,
VK_LWIN and VK_RWIN each took an unconditional kernel transition on
every query - four syscalls per keyboard event, in every copy.
GetKeyboardState is a bare thunk that returns all 256 keys at once.
Consolidated into win32_get_keyboard_mods() in win32_common.c and used
from wnd_proc_common, wnd_proc_common_internal,
wnd_proc_common_dinput_internal, winraw_callback and dinput_poll.
dinput_get_active_keyboard_mods() is removed as a duplicate.
The GetAsyncKeyState calls in dinput_poll are left alone: they read the
global physical state, which GetKeyboardState does not expose.
A "Prefer Performance Cores" toggle, off by default, next to Threaded
Tasks. With it on, the main thread pins itself to the fast cores as
the task queue comes up and the audio thread does the same alongside
its priority raise, through sthread_prefer_fast_cores; on a processor
whose cores are all alike, or a platform without the means, both
calls are no-ops, so the toggle is neutral there. It takes effect on
restart, since neither thread is recreated on the fly. The row lives
in settings_def_menu_threaded_data.h so the enum, strings, descriptor
and configuration passes all derive from the one definition, and the
label string table is regenerated.
sthread_prefer_fast_cores learns three more platforms. On Windows it
reads every logical CPU's EfficiencyClass through
GetSystemCpuSetInformation and hands the top class to
SetThreadSelectedCpuSets - a soft preference the scheduler may still
override under pressure, the right strength for "prefer" - with both
resolved at run time so builds that target older Windows keep
linking and report false there, and the records read by offset since
older SDKs lack the type. On macOS and iOS, where Apple silicon
offers no affinity, quality of service is what steers a thread onto
the performance cores, so it sets QOS_CLASS_USER_INTERACTIVE behind
the same deployment-target gate as the existing QoS override. On
WiiU the three cores are identical except that core 1 carries a
2 MiB L2 against 512 KiB on the others - the reason Cafe OS keeps
the main application thread there - so that is the core it asks for.
Homogeneous parts still report false without touching the thread.
Compiles on the MinGW, devkitPPC and host C89 lanes.
A thread that must not be parked on a slow core - the emulation
loop, the audio feeder - can pin itself to the CPUs whose maximum
clock is the highest in the system. On Linux and Android the set is
read once from cpufreq's cpuinfo_max_freq per CPU, intersected with
the thread's existing affinity, and applied through the raw
sched_setaffinity syscall, so no particular libc vintage is assumed;
on a homogeneous part, where nothing is readable, or where the fast
cores lie outside the allowed set, the thread is left where it is
and the call reports false. Elsewhere it is a no-op that reports
false. The sysfs root is a compile-time constant so the selection
can be exercised against a synthetic topology.
scond on Android goes straight to the futex instead of through
Bionic's pthread_cond_t. Bionic's condvar issues the FUTEX_WAKE
syscall on every signal whether or not a thread is waiting - true of
every Bionic from 4.4 to 14 on every ABI - so a waiter count in
front of the wake makes the common empty signal cost a load and a
compare rather than a kernel round trip. Waits register and sample
the sequence word under the caller's lock, then sleep on
FUTEX_WAIT_BITSET|FUTEX_PRIVATE_FLAG with an absolute
CLOCK_MONOTONIC deadline, so a spurious wake never stretches or
shortens a timed wait and a wall-clock jump never touches it. The
futex constants and the syscall prototype are spelled out in place:
they are kernel ABI, identical from the 2.6-era kernels of Android
2.x on, and the syscall numbers come from the Bionic headers per
ABI. Threads and mutexes stay on Bionic's pthreads, which are
already the same futex primitives underneath.
The scond passes ASan, UBSan, LeakSanitizer and TSan on the host
against real Linux futexes, and the empty signal measures at about
1 ns against roughly 150 ns for the bare wake syscall it replaces.
rc_compat's rc_mutex on Switch is a libnx RMutex - a recursive lock
over the same userspace-fast-path Mutex the rthreads backend uses -
rather than a pthread mutex layered over it through newlib. It sits
next to the GEKKO, 3DS, WiiU and Vita backends and is worth
upstreaming to rcheevos with them; a full Switch build links it in.
Makefile.common no longer forces RC_NO_THREADS onto GEKKO and 3DS:
rc_compat carries native recursive mutexes for both, so threaded
builds through Makefile.wii and the default Makefile.ctr get real
locks around the achievement runtime, as the griffin path already
does. Only a build without HAVE_THREADS compiles them out.
HAVE_THREADS goes on for PS2, riding the native EE kernel backend in
rthreads: the task queue, autosave and the threaded video and audio
plumbing become available. A full ELF builds and links with this
on; runtime behaviour on hardware is the thing to watch when this
lands, and flipping this single line back is the revert if anything
surfaces.
rthreads.c drives the EE kernel directly on PS2. Threads ride
CreateThread on a 32 KiB stack the sthread owns, inheriting the
creator's priority (an explicit rthreads priority maps across
1..127), with the sthread itself as the start argument. The kernel
has no join, and frees only the control block on exit, so the exit
path lifts the thread to priority 0 - the top of the band, which the
kernel never time-slices - before signaling its joiner and calling
ExitDeleteThread: nothing can free the stack out from under those
last instructions. Detached threads park on a reap list swept by
the next create or detach once their DONE bit is set. slock is a
binary semaphore. scond is a FIFO of waiter nodes on the waiters'
stacks parked in SleepThread, using the kernel's per-thread wakeup
count as an auto-reset event so a wake issued before the sleep is
kept; signal wakes one, broadcast the detached list. Timed waits
arm a one-shot timer alarm on the bus clock whose handler marks the
node and wakes the thread from interrupt context; on return the
waiter releases the alarm with interrupts off if it has not fired,
withdraws its node if nobody signaled, and drains the surplus wakeup
with CancelWakeupThread if both the alarm and a signaler did. The
lists are guarded by disabling interrupts, the only way onto them
on a single core with no time-slicing. sthread_cancel joins the
unsupported list.
The pthread and Win32 lanes preprocess to byte-identical code, the
backend passes ASan, UBSan, LeakSanitizer and TSan through a host
shim of the EE kernel API - alarm, wakeup counts and all - and the
worst native frame is 160 bytes against the 2 KiB budget.
HAVE_THREADS went on for WiiU, so rcheevos stopped taking the
RC_NO_THREADS path in Makefile.common and fell through to its pthread
backend. devkitPPC ships <pthread.h> but leaves pthread_mutex_t undefined
without _POSIX_THREADS, so every translation unit that reaches
rc_client_internal.h fails to compile -- cheevos_menu.c is simply the
first one the build gets to.
The fix is not RC_NO_THREADS. That macro compiles rc_mutex_lock and
friends away to nothing, and with threads on the cheevos client is
reached from the task queue and the HTTP callbacks, so it would trade a
compile error for unsynchronised access. WiiU gets a real backend
instead, alongside the Wii, Vita and 3DS ones already there: OSMutex,
which is the recursive Cafe OS primitive and so matches the recursive
pthread mutex the generic path asks for. OSFastMutex, which rthreads uses
for its own slock, is not recursive and would deadlock rcheevos' nested
locking. There is no destroy entry point because the object is
caller-owned storage with no kernel handle behind it.
Verified by preprocessing rc_compat.h under -DWIIU: rc_mutex_t resolves
to OSMutex with no pthread typedef present, against a control on the
unpatched tree where it resolves the other way. rc_compat.c compiled for
that profile has exactly OSInitMutex, OSLockMutex and OSUnlockMutex
undefined and no pthread references.
rthreads.c drives libnx directly on Switch: threads ride threadCreate
with a 128 KiB stack that libnx allocates and maps, inheriting the
creator's priority, with an explicit rthreads priority mapped across
0x1C..0x3F and falling back to inheriting if the kernel refuses it
for this process; slock is a libnx Mutex and scond a libnx CondVar,
both userspace fast paths that arbitrate through the kernel only
under contention, and timed waits hand condvarWaitTimeout nanoseconds
and block once. libnx has no detach - a Thread stays live until
threadClose, and a thread cannot close the stack it runs on - so
detached threads park on a reap list that the next create or detach
sweeps once their handles have signaled, bounding what lingers to the
detached threads finished since the last call. sthread_cancel joins
the unsupported list, and the standing note about thread attributes
on Switch is settled by the backend owning stack size and priority
itself.
The pthread and Win32 lanes preprocess to byte-identical code, the
worst native frame is 80 bytes against the 2 KiB libretro-common
budget, and a full Switch build links into an .nro with no undefined
pthread references - the pthread symbols left in it are newlib's own
definitions, pulled by nothing in rthreads.
griffin.c compiles both filter chains into one C translation unit now that
each is C. In C++ the gl3_shader namespace kept the two apart; in C they
collide on every name they happen to share, and the MSVC and Apple lanes
fail on opaque_vert, opaque_frag, Texture, CommonResources and
common_resources_init/free. The GL chain's copies take a gl3_ prefix:
gl3_opaque_vert, gl3_opaque_frag, gl3_texture_t, gl3_common_resources,
gl3_common_resources_init/free.
Size2D was the other half of the clash -- both files typedef'd it, which
is a C11-only redefinition -- and it never earned being a type. It was two
unsigned fields passed by value and returned by value. Every struct member
becomes a width/height pair, every parameter becomes two unsigneds, and
the two functions that returned one take out-params instead. The struct is
deleted from gfx/video_defines.h; gl3_framebuffer_copy and
gl3_framebuffer_copy_partial in the GL driver take the pair directly.
The gate that should have caught this now exists: a translation unit that
includes slang_process.c, spirv_opengl.c, shader_vulkan.c and shader_gl3.c
exactly as griffin.c does. The previous cohabitation check paired the GL
chain with the two files that could not collide with it and omitted the
one that could, which is why it passed while CI did not.
Verified: the four-file griffin unit compiles with zero errors and zero
warnings, both chains pass the C89 and pointer-conversion gates,
griffin_cpp.cpp and griffin_glslang.cpp still compile as single units, the
Vulkan chain's dump is byte-exact over 1643 shaders, the GL chain renders
543 identical pixel hashes plus 12 unchanged chain-creation failures over
the 555-preset corpus, and both chain-lifecycle smoke suites are clean
under ASan, UBSan and LSan.
HAVE_THREADS goes on for WiiU, riding the native Cafe OS backend in
rthreads: the task queue, autosave and the threaded video and audio
plumbing become available on a console with three cores that the
frontend has so far used one of. A full RPX builds and links with
this on; runtime behaviour on hardware is the thing to watch when
this lands, and flipping this single line back is the revert if
anything surfaces.
rthreads.c drives Cafe OS directly on WiiU. Threads ride
OSCreateThread across all three cores, inheriting the creator's
priority (an explicit rthreads priority maps across the 0..31 band);
the OSThread control block and the 128 KiB stack are one caller-owned
allocation, and the deallocator the scheduler invokes once a thread
is fully dead hands it back, so neither join nor detach can free a
stack that is still live. slock is an OSFastMutex embedded in the
lock. scond is a FIFO of waiter nodes on the waiters' own stacks,
each parked on an auto-reset OSEvent behind a fast-mutex gate:
signal wakes exactly one, broadcast detaches the whole list before
waking it, and a timed wait that loses the race with a signaler
consumes the in-flight wake before its node leaves scope, so
OSWaitEventWithTimeout gives real timeouts with no spurious wakes
and no polling. sthread_cancel joins the unsupported list.
The pthread and Win32 lanes preprocess to byte-identical code, the
backend's lock and condvar paths pass ASan, UBSan, LeakSanitizer and
TSan through a host shim of the Cafe primitives, and the worst native
frame is 88 bytes against the 2 KiB libretro-common budget. The same
source compiles against both the in-tree wiiu/os headers and wut's
coreinit headers, picked by __has_include.
The RAII glslang_include_cache_guard existed for the two filter chains,
and lived in the header rather than either chain's translation unit
because griffin compiles both into one, where two identical definitions
are a redefinition. Both chains are C now and manage the handle
explicitly, so the struct has no consumers.
The C++-only block it sat in stays: the SAL-compat undefs around it are
still needed under CXX_BUILD, where these .c files are compiled as C++
and pull in the STL.
Verified: every drivers_shader object rebuilds clean, both C++ griffin
amalgamations still compile as single units, all three converted .c files
still compile as C++ for the CXX_BUILD lane, and the nine GL chain
lifecycles stay clean under ASan, UBSan and LSan.
Final slice of the arc. gfx/drivers_shader/shader_gl3.cpp becomes
shader_gl3.c: the gl3_shader namespace and all 56 qualifications go away,
C++ headers are dropped, casts and nullptr become C, the header's struct
tags get file-local typedefs, the nested Parameter type moves to file
scope, and the reference parameters become pointers or by-value PODs. The
RAII glslang_include_cache_guard -- the struct slang_process.h keeps under
__cplusplus for this file alone -- becomes an explicit handle freed on
every exit.
Three defects the gates caught, none of them visible on review:
C++ resolved gl3_pass_reflect_parameter by overload. C cannot, so the
texture variant was renamed, and three call sites that had been taking it
silently bound to the semantic-meta function instead. -Wunused-function on
the now-uncalled variant is what exposed that.
The include cache carried a comment claiming its scope close freed it.
Nothing did, and LSan measured 120 bytes leaked per preset. That is the
third variant in this arc of a C++ lifetime whose replacement was
incomplete.
The C89 gate found 14 mixed declarations; the two for-init declarations
needed a separate sweep, since -Wdeclaration-after-statement does not
flag them and only MSVC's C lane would have.
The include moves from griffin_cpp.cpp to griffin.c beside spirv_opengl.c,
which already carries the identical guard, copied verbatim rather than
re-derived. Preprocessing griffin.c resolves it zero times under the ARM64
profile and 873 under the x64 one. Removing it from griffin_cpp.cpp leaves
only optional Qt sources ahead of the vendored SPIRV-Cross block, which is
safe because NOMINMAX is defined at the top of that file rather than
supplied by whichever source happens to come first.
Codegen: the C build is 108 instructions heavier (+1.5%). Pointerizing the
per-frame semantic-texture layer removed the per-frame part of that, and
the demangled per-function diff puts the remainder in create_from_preset,
compile_pass and create_deferred -- the hoisted declarations and the
explicit frees C requires, all of it one-time setup. The per-frame path is
at parity or better: build_offscreen_passes is 17 instructions lighter.
Across the arc the translation unit went from 17407 instructions to 7097.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan with leak
detection on, 543 rendered-output hashes plus 12 chain-creation failures
identical across the 555-preset corpus, both the production build and the
C89 gate at zero, griffin_cpp.cpp compiles as one amalgamated unit, and
slang_process.c, spirv_opengl.c and shader_gl3.c cohabit a single C
translation unit without collisions.
rthreads.c drives the Vita kernel directly: threads ride
sceKernelCreateThread with the creator's priority inherited (an
explicit rthreads priority maps across the 64..191 band), slock is a
SceKernelLwMutex embedded in the lock - userspace fast path, no kernel
object per lock - and scond is a SceKernelLwCond bound lazily to its
lock at the first wait, which the caller's own lock serializes. Timed
waits hand the kernel microseconds and block once; the pthread port
they replace pends in a polling loop, waking every few hundred
microseconds to check a cancellation flag, so every blocked wait on
the platform stops being a periodic wakeup. Detached threads reap
themselves through the same done/detached handshake as the PSP
backend, sthread_cancel joins the unsupported list (its only caller
is Linux-only sensor code), and pthread_init leaves the frontend.
Two other pieces keep the ELF free of the port: rcheevos gets a native
rc_mutex on Vita's recursive kernel mutexes, next to its GEKKO and 3DS
backends and worth upstreaming like them; and data_transfer's
per-thread pool compiles out on the console targets, where __thread
lowers through emutls - a per-access lookup plus a gthread mutex,
costing more than the pool miss it avoids, and on Vita the last thing
asking for pthreads at all. The pthread and Win32 rthreads lanes
preprocess to byte-identical code, worst native frame is 56 bytes
against the 2 KiB libretro-common budget, and a full Vita build links
into a .vpk with no pthread symbols referenced.
Tenth slice, and the file is class-free. Pass becomes struct gl3_pass and
its 24 out-of-line methods and 21 inline ones become gl3_pass_* free
functions. The constructor becomes gl3_pass_new: every default was zero
except final_pass and the frame_direction, total_subframes and
current_subframe trio, so it callocs and sets those four. The in-class
initializers are stripped for the same reason CommonResources' were.
With no class left, the parsed preset moves from new/delete to
calloc/free, which the earlier slices deliberately deferred so that
allocation and release never disagreed mid-arc.
The sanitizer caught the counterpart of the bug it caught last slice: the
destructor released the members and 'delete' released the object, so
gl3_pass_free inherited only half the job and every pass leaked itself
(36 kB across the smoke). Both halves of a C++ destructor have to be
accounted for -- the members it frees, and the deallocation the operator
did behind it.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan with leak
detection on, 543 rendered-output hashes plus 12 chain-creation failures
identical across the 555-preset corpus, and griffin_cpp.cpp still compiles
as one amalgamated unit. Codegen drops 705 instructions (-9.2%).
Only the rename is left.
Ninth slice. CommonResources loses its constructor, destructor and every
in-class initializer, becoming a trivially constructible struct paired
with common_resources_init and common_resources_free. Every default was
zero, so a memset covers what the initializers supplied -- and stripping
the initializers is what makes that memset legal, since an in-class
initializer alone is enough to make the type non-trivial and draw
-Wclass-memaccess.
With no member left that constructs, the chain is calloc'd and freed
rather than new'd and deleted, and gl3_chain_new and gl3_chain_free call
the two functions where the member constructor and destructor used to run.
Codegen is 6 instructions heavier (+0.1%), from the chain's calloc and the
init memset zeroing the same bytes twice. I kept the memset rather than
make common_resources_init depend on its caller having zeroed the storage;
six instructions once per chain creation is not worth that coupling.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan, 543
rendered-output hashes plus 12 chain-creation failures identical across the
555-preset corpus.
Eighth slice. gl3_filter_chain becomes a plain struct and its 34 methods
become gl3_chain_* free functions taking the chain as their first
argument. The constructor and destructor become gl3_chain_new and
gl3_chain_free; the chain is still new'd rather than calloc'd because
CommonResources retains a constructor, and that is the next thing to go.
The create paths drop their unique_ptr wrappers, so every early exit now
releases the chain and, while the chain does not yet own it, the parsed
preset.
The sanitizer caught the one thing review missed: gl3_filter_chain_free
still did 'delete chain', which had been doing the whole teardown through
the destructor that this slice removed, so every chain leaked its passes
and pass_info. It routes through gl3_chain_free now.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan with leak
detection on, 543 rendered-output hashes plus 12 chain-creation failures
identical across the 555-preset corpus, and griffin_cpp.cpp still compiles
as one amalgamated unit. Codegen drops 1090 instructions (-12.4%).
No std:: remains in the file. What is left before the rename is
CommonResources' constructor and destructor, the Pass class, and the C++
syntax the rename itself deals with.
Picks up the Win32 scond reading its waiter flag word through the atomic
API on the backends that type it as a real atomic, rather than punning it
through a volatile LONG.
Seventh slice. The pass list and pass_info become counted arrays, and
deferred_calls goes away entirely -- it was declared and never referenced
anywhere in the file, a leftover of the disposal machinery the Vulkan side
removed in its own first slice.
set_num_passes releases any previous generation before building the new
one, which the vector did implicitly on resize, and the chain destructor
deletes the passes the unique_ptr vector used to own. set_pass_info keeps
growth semantics distinct from the other resizes in this file: it must
preserve entries already written, so it reallocs and zeroes only the new
tail rather than clearing and reallocating.
The two SPIR-V lowering buffers in the ARB_gl_spirv link path become owned
allocations with the early exits routed through one label; spirv_opengl_lower
writes every word it reports, so dropping the vector's zero-fill changes
nothing.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan, 543
rendered-output hashes plus 12 chain-creation failures identical across the
555-preset corpus. Codegen drops 1009 instructions (-10.3%).
What is left is the chain class shell itself and the unique_ptr wrappers in
the create paths.
Sixth slice, and the same port slang_process.cpp already went through.
gl3_cross_compile_program drops spirv_glsl.hpp for <spirv_cross_c.h>: one
spvc_context owns the parsed IR, both compilers and both option sets, and
a single destroy covers every exit including the error paths, so the
try/catch around the whole function is gone with it. The repeated
rename-and-strip-location loops collapse into one helper, the reflected
resource lists come back as plain arrays, and the texture binding fixups
are an owned uint32_t array rather than a vector.
The compiled sources are now borrowed const char* owned by the context
instead of std::string, which removes two copies per program.
This drops the file's dependency on the SPIRV-Cross C++ symbols entirely:
nm reports zero spirv_cross:: and sixteen spvc_ undefined symbols, all
provided by the same spirv_cross_c translation unit slang_process.c
already requires, so no lane gains a provider it did not already link.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan, 543
rendered-output hashes plus 12 chain-creation failures identical across
the 555-preset corpus, and griffin_cpp.cpp still compiles as one
amalgamated unit. Codegen drops 6686 instructions -- 40.6% of the
translation unit, the largest single reduction of either arc.
gx_joypad's reset callback takes the signature of whichever libogc the
build links: (u32, void*) for the system libogc under EXTERNAL_LIBOGC,
(void) for the in-tree fork. libfat's stat fill zeroes the nanosecond
fields on a newlib whose struct stat carries timespecs - st_atime
being a macro there is the tell - and the spare fields on one where it
does not. Current GCC treats both former mismatches as errors, so
these two files decide whether the Wii object set builds at all on a
current toolchain; the remaining distance to a linked .dol is the
prebuilt in-tree libogc archive, whose crt expectations predate
current devkitPPC.
rcheevos' rc_compat carries native mutex backends for exactly these
platforms - a recursive LWP mutex on GEKKO and a RecursiveLock on 3DS -
so griffin lets HAVE_THREADS decide RC_NO_THREADS on its own and
threaded Wii and 3DS builds get real locks around the achievement
runtime instead of compiled-out stubs.
rthreads.c drives libogc, libctru and the PSP kernel directly on these
platforms rather than dressing them up as pthreads: threads, mutexes
and condition variables map onto LWP, LightLock/CondVar and sceKernel
threads with semaphores. sthread_isself answers on GEKKO,
sthread_raise_current_priority moves the caller up within each
scheduler's band, and an explicit rthreads priority maps across the
whole band on all three. GEKKO timed waits hand LWP the timeout in
the form the linked libogc expects - an absolute deadline on its RTC
clock for the in-tree fork under INTERNAL_LIBOGC, a relative timespec
for upstream - so they expire when asked instead of immediately or
never. 3DS thread creation passes its argument through threadCreate
rather than serializing every launch through a static handoff slot,
joins free the libctru thread object, and timed waits ride
CondVar_WaitTimeout's monotonic nanoseconds. PSP drops the pthreads
port from its link: a detached thread reaps itself through a
done/detached atomic handshake so nothing lingers dormant, and the
pthread_init calls in the PSP/Vita frontend are Vita's alone.
The desktop pthread and Win32 lanes preprocess to byte-identical
code. Makefile.ctr defaults USE_CTRULIB_2=1 to match current libctru,
with 0 still selecting the pre-2.0 CondVar backport; the unread
GX_PTHREAD_LEGACY knob is gone from the GEKKO Makefiles; and
stack_budget measures rthreads.c on its portable lane, since -DPSP on
a host compiler cannot see the PSP SDK headers the native backend
needs. Worst native frame is 48 bytes against the 2 KiB
libretro-common budget. Compiles clean on current devkitPPC (system
libogc and the in-tree fork), devkitARM and pspdev; full 3DS and PSP
builds link end to end.
rthreads.c is the only user of the GEKKO and 3DS pthread wrappers, so
they now live inline in its platform include section rather than as
separate headers. psp_pthread.h is referenced by nothing - PSP threading
comes from the toolchain's pthreads - and is removed outright. Comments
across the tree that point at thread stack sizes now name rthreads.c.
Fifth slice. pass_name and Parameter::id become owned C strings, the
parameter vector becomes a counted array, and filtered_parameters becomes
an array of indices into it rather than a vector of copies -- the copies
survived a reallocation of the parameter array and indices do too. The two
SPIR-V module vectors and both uniform buffers become owned allocations
sized on build, and reflect_parameter drops std::string for a fixed-buffer
helper, since the four instance-block prefixes it concatenates are
literals. ~Pass releases all of it.
The buffer allocations can fail where vector::resize would have thrown, so
build() reports that rather than running a pass against a short uniform
buffer.
Verified: nine chain lifecycles clean under ASan, UBSan and LSan --
including the deferred build path the frontend actually uses and the
GL_ARB_gl_spirv binary path, both of which this harness gained while
chasing an unrelated crash -- and 543 rendered-output hashes plus 12
chain-creation failures identical across the 555-preset corpus. Codegen
drops 227 instructions (-1.4%).
The only C++ left in the file is SPIRV-Cross's own API surface.
The statistics overlay formats video_st->current_video->ident and
audio_st->current_audio->ident with no check on either pointer, while the
same file guards the video one at another call site. Whichever of the two
is missing, the fault lands inside video_driver_frame on the first frame
that draws statistics, which is the first frame after startup when the
overlay is enabled.
Both are printed as "n/a" when absent. This is the same class of hazard as
the filter chain guard: a missing driver is a reportable state, not a
segfault.
scond_wait_win32()'s spin peeked at the waiter's flag word through
#define scond_flags_peek(w) (*(volatile LONG*)&(w)->flags)
but flags is a retro_atomic_int_t, and on the C11, C++11 and __atomic
backends that is a real atomic type -- mingw selects C11, so the field is
an atomic_int. Reading one through a volatile LONG lvalue is a type pun:
GCC flags it at -O2,
rthreads.c:668:31: warning: dereferencing type-punned pointer will
break strict-aliasing rules [-Wstrict-aliasing]
three times, once per spin site (lines 952, 969, 984), and it is
undefined besides. The cast is also wrong in kind on those backends,
where the atomic object is not addressable as a plain integer at all.
The cast was not gratuitous, which is why it cannot simply be deleted:
the MSVC, Apple and __sync backends implement the acquire load as a
locked RMW -- InterlockedCompareExchangeAcquire, OSAtomicAdd32Barrier,
__sync_fetch_and_add(p, 0) -- and running one of those per spin iteration
against a line a single waker writes is exactly what the fast path exists
to avoid.
So spell the plain load per backend rather than by casting the field.
The backends that type retro_atomic_int_t as a plain volatile integer
(MSVC, Apple, __sync, volatile) read it directly: same type, no cast, and
the cheap load they needed. The backends where it is a real atomic type
go through retro_atomic_load_acquire_int(), which costs nothing here
because their acquire load is a plain MOV on x86. The SCOND_HAVE_X86
condition is kept in front of the whole thing, so no non-x86 target
silently loses its acquire.
Verified with x86_64-w64-mingw32 at -O2 -Wall -Wextra across
RETRO_ATOMIC_FORCE_{C11,GCC_NEW,MSVC,SYNC,VOLATILE}: the three
strict-aliasing warnings are gone on every one of them, and the C11 and
MSVC expansions were checked to confirm each takes the branch intended
for it. Disassembling scond_wait_win32() before and after shows the same
five locked operations -- no locked op introduced into the spin -- and 204
instructions against 233, the compiler generating better code once the
pun is gone. Also clean at -O2 on native x86_64 and aarch64-linux-gnu,
and under -std=c89 -Werror=declaration-after-statement.
Two pre-existing items on the SYNC and volatile backends are untouched
and unrelated: three -Wdiscarded-qualifiers at lines 825, 967 and 984
from passing the volatile-typed field to non-volatile parameters, and,
on the volatile fallback alone, retro_atomic_ptr_t not being defined at
all. Neither backend is selected on Windows in practice; both only
appear under a forced build.
The frame path picks gl->filter_chain, falls back to filter_chain_default,
and then calls gl3_filter_chain_set_frame_count on the result without
checking it. When a preset fails to build and the default chain is absent
or failed too, both are NULL and the first setter dereferences it, so a
recoverable configuration error becomes a segfault on the first frame --
and on a stripped MinGW build the fault is attributed to whichever
exported symbol precedes the static frame function, which makes it look
like it came from somewhere else entirely.
Report the dead frame and return false instead.
retro_vfs_file_punch_hole_impl() gates its Linux path on
FALLOC_FL_PUNCH_HOLE alone. That flag comes from <linux/falloc.h> and is
defined regardless of feature-test macros, but fallocate() itself is
declared by glibc only under __USE_GNU -- which is to say only when the
*consumer* defines _GNU_SOURCE.
RetroArch defines it, so this never shows up here. A core that does not
gets an implicit declaration instead. libretro-prboom builds with
-D_POSIX_C_SOURCE=199309L -D_DEFAULT_SOURCE and hits exactly that:
vfs_implementation.c:1445:11: warning: implicit declaration of
function 'fallocate'
That is a warning on the compilers in use today, an error on stricter
ones, and on 32-bit it passes the off_t pair wrongly, since an implicit
prototype promotes nothing.
Require the declaration as well as the flag, so a consumer without
_GNU_SOURCE falls through to the existing #else reporting the operation
unsupported -- which is what such a build should have been doing all
along. Builds that do define _GNU_SOURCE are unchanged: preprocessing
vfs_implementation.c with and without it puts the fallocate() call in
exactly the first case and not the second.
This is the same shape as the GetModuleHandle/UNICODE fix a few commits
back: shared code should not depend on which feature-test or platform
macros its consumer happens to set.
Verified with -fsyntax-only under _GNU_SOURCE, under
-D_POSIX_C_SOURCE=199309L -D_DEFAULT_SOURCE, and bare, for both
x86_64 and aarch64-linux-gnu; plus x86_64-w64-mingw32 with -DUNICODE and
with __WINRT__, which this does not touch.
Fourth slice. The three std::vector<Texture> members become counted arrays
behind gl3_texture_array_resize, which allocates fresh zeroed storage --
equivalent here because every resize follows a clear and every element is
written before it is read. The shader_preset unique_ptr becomes an owned
pointer whose setter releases the previous preset; it stays paired with
delete because the create paths still allocate it with new, and those move
to calloc when the chain itself flattens.
CommonResources released all four of these through implicit member
destruction, so its destructor now frees them explicitly. The resize calls
can fail now that they are allocations rather than vector growth, so
init_history, init_feedback and both pass_output sizing points propagate
that instead of continuing with a short array.
Verified: seven lifecycles clean under ASan, UBSan and LSan, and the
rendered-output oracle identical across an enlarged corpus -- 555 presets
now rather than 175, eight per shader family, 543 hashes and 12
chain-creation failures all unchanged. Codegen drops 169 instructions
(-1.0%).
Third slice. The ring has always been sixteen buffers deep, so the
std::vector<GLuint> becomes a fixed GLuint[16] with a count, which removes
a heap allocation for a compile-time constant. The destructor becomes
gl3_ubo_ring_free, called explicitly from ~Pass where the member
destructor used to run, and gl3_ubo_ring_init now frees any previous
generation first rather than stranding sixteen buffer names if a pass is
ever re-initialised -- the vector's resize would have done that silently.
Both counters carry initialisers, since a count standing in for a
container's size is the member most likely to be read before it is set.
Verified: seven lifecycles clean under ASan, UBSan and LSan on a real
core-profile context, 173 pixel hashes identical across the stratified
preset sample, zero build warnings. Codegen drops 96 instructions
(-0.6%).
retro_vfs_file_get_sparse_granularity_impl() reaches kernel32 by name to
resolve GetFinalPathNameByHandleW and friends. Two things are wrong with
how it gets there, and both bite consumers rather than RetroArch itself,
which is why neither has shown up here.
First, the lookup uses the A/W-mapped GetModuleHandle() with a narrow
literal. Whether that compiles depends on whether the *consumer* defines
UNICODE, which is not something a shared tree should be sensitive to.
RetroArch does not define it and so never sees this; a core that does --
libretro/ps2, for one -- fails outright under mingw:
vfs_implementation.c:1282:40: error: passing argument 1 of
'GetModuleHandleW' from incompatible pointer type
Name the A entry point explicitly, as every other module lookup in this
tree already does: rthreads.c:696, features_cpu.c:558 and rtime.c:148/202
are all on GetModuleHandleA. This is the only site that is not, and a
sweep of the whole tree for narrow literals passed to A/W-mapped Win32
entry points turns up no other instance. dylib.c's GetModuleHandle(NULL)
is immune because NULL has no character type.
Second, the block is guarded only by _WIN32 && !_XBOX, so it stays live on
UWP, where GetModuleHandle is in neither form part of the Windows Store
API partition. dylib_proc() states this outright and refuses to call it
there; features_cpu.c, which runs the identical GetModuleHandleA +
GetProcAddress dance against kernel32, spells the guard
_WIN32 && !_XBOX && !__WINRT__ at lines 542, 707 and 839. Use the same
guard here. The POSIX arm below gains !defined(_WIN32) so that a UWP
build falls through to "return 0" rather than into the fstat/st_blksize
path; reporting "unknown" and skipping sparse handling is already the
documented behaviour for any system that cannot resolve these names.
Verified with x86_64-w64-mingw32 -fsyntax-only across the desktop
partition with and without -DUNICODE, desktop with USE_WIN32_THREADS,
__WINRT__ on the desktop partition, and __WINRT__ with
WINAPI_FAMILY_APP; plus native x86_64, aarch64-linux-gnu, and -std=c89
-Werror=declaration-after-statement. No configuration now reports
anything for GetModuleHandle.
Two pre-existing WINAPI_FAMILY_APP gaps are deliberately left alone, as
neither is this defect and neither is reachable from a desktop-partition
build: CreateFileMapping/MapViewOfFile at vfs_implementation.c:1029/1033
want the *FromApp forms under UWP, and rthreads.c has one further
undeclared call in that configuration.
Second slice. Framebuffer becomes gl3_framebuffer: a plain struct built by
gl3_framebuffer_new, which returns NULL instead of a half-constructed
object, with set_size and the shared build path as free functions and the
accessors replaced by member reads.
The ownership that std::unique_ptr had been doing implicitly is now
written out. Pass releases both its framebuffers in its destructor, the
per-pass swap at end_frame is a three-way pointer exchange rather than
std::swap on smart pointers, and the chain gains a destructor at all --
copy_framebuffer and the history entries had no explicit teardown because
the implicit one sufficed. The history ring, previously a
vector<unique_ptr> rotated with move_backward, is a counted pointer array
rotated by index.
Two defects were caught before delivery rather than after. A blanket
member rewrite captured a local variable that deliberately shadowed the
levels member inside the unsupported-format fallback, which would have
fed a possibly-zero level count to glTexStorage2D; the local is restored.
And the replacement for a unique_ptr assignment placed two statements
under a single-statement if, so a final pass would have allocated the
framebuffer it is specifically not supposed to own -- the compiler's
misleading-indentation warning found it.
Verified on a real core-profile context: seven lifecycles clean under
ASan, UBSan and LSan, and 173 pixel hashes identical across the stratified
preset sample. Codegen drops 175 instructions (-1.0%).
First slice of the same conversion the Vulkan filter chain went through.
StaticTexture becomes gl3_static_texture: a plain struct with
gl3_static_texture_init/free, an owned char* id in place of std::string,
and a realloc'd array with an explicit count in place of
std::vector<std::unique_ptr<>>.
The unique_ptr vector used to release every LUT when CommonResources was
destroyed. A plain array does not, so the destructor now frees each LUT and
then the array, and add_static_texture reports allocation failure so the
loader can release the texture it was handing over rather than leaking it.
Verified against the GL filter chain running on a real core-profile context
(llvmpipe): seven chain lifecycles clean under ASan, UBSan and LSan with
leak detection on, and identical rendered output -- 173 pixel hashes over a
stratified sample of the preset corpus, byte for byte. Codegen for the
translation unit drops 268 instructions (-1.5%).
The bound drops from ntdll's 0x90b40 to 64000 TSC cycles, the point
where the measured curve goes flat: at four and eight threads 64k and
592k are within noise of each other, below 32k the spin stops bridging
the interval between dispatches and each one pays a kernel wait
instead, and a pool with two threads per core is steadily better the
shorter the spin, its waiters otherwise taking cycles from the siblings
doing the work. RTHREADS_SCOND_SPIN_CYCLES still overrides it.
The pause loop is the default spin on x86 even where mwaitx or umwait
exist: measured against the native condition variable at 600 dispatches
a frame on a 9950X3D, pause is ahead of mwaitx at every thread count,
because the hardware wait's exit latency is on the order of the whole
interval between dispatches. RTHREADS_SCOND_SPIN=mwaitx or umwait still
selects a hardware wait, on a processor that has it, and the TSC bound
moves to RTHREADS_SCOND_SPIN_CYCLES so a mode and a bound can be set
together.
An scond is one word holding a lock-free list of waiter blocks on the
waiting threads' stacks, each with a flag word of its own. A wait pushes
its block with a compare-and-swap, releases the caller's lock, spins for
a bounded time on its own flag word and only then blocks in the kernel;
a wake takes the block off the list, flips the flag, and issues a kernel
wake only for a waiter that had committed to blocking, so a wake that
lands during the spin costs no syscall. The spin is the hardware wait
where the processor has one - mwaitx on AMD, umwait on Intel, both
parking the core on the flag word's cache line until a TSC deadline so
a sibling hyperthread is left alone - and a pause loop otherwise, with
ntdll's own bound; a single-processor machine does not spin. Signal
wakes the oldest waiter, broadcast takes the whole list in one swap, and
a timed-out waiter unlinks itself under the list lock or, when a waker
has already taken its block, consumes the wake that is on its way.
The kernel wait is resolved once per process from what ntdll exports:
NtWaitForAlertByThreadId with NtAlertThreadByThreadId (Windows 8 and
later), the keyed-event pair (XP and later), and a per-thread auto-reset
event where neither exists (the Xbox targets), so one code path serves
every Windows version with no compile-time floor. RTHREADS_SCOND selects
the sleep primitive by name and RTHREADS_SCOND_SPIN the spin mode or its
TSC bound, for measurement.
retro_atomic_cas_ptr(p, expected, desired) is strong and evaluates to
non-zero on success, on every backend that has the pointer operations:
C11, C++11, GCC __atomic_*, MSVC InterlockedCompareExchangePointer, Apple
OSAtomicCompareAndSwapPtrBarrier and GCC __sync_*. A lock-free list head
is a pointer with flag bits, and a caller that keeps one needs to swap
the whole word.
Picks up the Win32 scond that uses the native condition variable where
kernel32 provides it, so a broadcast to the angrylion workers wakes them
all at once instead of handing a token through them one by one.
android_app_create() waits for the app thread to set 'running' with
neither a timeout nor a test of app_thread_exited - the only one of the
five waits on this condvar without either. 'running' is set in
frontend_unix_init(), a long way into rarch_main(): after
AConfiguration_new(), ALooper_prepare() and ALooper_addFd(), and only
once frontend_driver_init_first() has got that far at all. Any init
failure before that point returns from android_app_entry() with
'running' still clear. The thread then sets app_thread_exited and
broadcasts on its way out, the Java UI thread wakes inside
ANativeActivity_onCreate(), re-tests 'running', finds it still zero and
waits again - and ActivityManager kills the process a few seconds
later. From the outside that is a launch that sometimes comes up and
mostly does not.
Take app_thread_exited as the other way the wait can end, and let
'running' rather than the flag decide the outcome: a thread that
started and then exited quickly can set both before the wait is even
entered, and that is a successful create, not a failed one.
A failed create hands ANativeActivity_onCreate() a NULL instance.
android_app_set_activity_state(), android_app_set_window(),
android_app_set_input() and android_app_write_cmd() already tolerate
that; android_app_free() - the one onDestroy() reaches - dereferenced
it, so guard it too. The three existing failure paths in
android_app_create() free the struct after g_android_early has been
pointed at it, leaving that global dangling for
Java_..._permissionsResolved() to lock a freed mutex through; clear it
there and on the new path.
Compile-verified for aarch64-linux-android31 with NDK r27c clang using
the phoenix-common define set, and for the non-Android unix build.
Reproducing the wait itself needs a device-side init failure, so no
runtime repro accompanies this.
The vendored SPIRV-Cross sources in this amalgamation were protected from
the windows.h min()/max() macros only by accident: shader_vulkan.cpp was
the first RetroArch source in the TU and its first header,
gfx/include/vulkan/vk_sdk_platform.h, defines NOMINMAX. Moving that file to
griffin.c for the C89 conversion removed the definition, so shader_gl3.cpp
now reaches windows.h through retro_miscellaneous.h with the macros live
and spirv_common.hpp fails on std::numeric_limits<int32_t>::min(), which is
not parenthesized the way spirv_cross_containers.hpp is. That is the
C2589/C2059 cascade on the MSVC x64 lanes; the ARM64 lane escaped it only
because it sets neither HAVE_VULKAN nor HAVE_OPENGL_CORE and so had no
RetroArch source ahead of the vendored ones at all.
NOMINMAX is now defined beside WIN32_LEAN_AND_MEAN at the top of the file,
where it does not depend on which sources happen to be enabled, with the
history recorded in a comment. A tripwire before the vendored SPIRV-Cross
includes fails the build with a direct message if the macros are ever
defined at that point again, rather than letting the diagnostics land
inside SPIRV-Cross.
The C89 rename moved the shader_vulkan include from griffin_cpp.cpp to
griffin.c but placed it inside the existing HAVE_SLANG block, dropping the
HAVE_VULKAN half of the guard it had carried. HAVE_SLANG without
HAVE_VULKAN is a real configuration -- the MSVC lanes ship D3D plus slang
and no Vulkan -- so griffin.c compiled shader_vulkan.c into a build where
neither gfx/common/vulkan_common.c nor the Vulkan symbol wrapper is
present, producing 60 unresolved externals at link on ARM64 Release and
every other non-Vulkan MSVC lane. The file compiles fine there, so nothing
short of a link catches it.
The include moves to its own '#if defined(HAVE_VULKAN) && defined(HAVE_SLANG)'
block, matching the guard verbatim from griffin_cpp.cpp, with the reason
recorded in a comment. The now-empty guard left behind in griffin_cpp.cpp
is removed.
Verified by preprocessing griffin.c under the MSVC define set: the include
resolves zero times without HAVE_VULKAN and still resolves with it, against
a negative control on the broken commit that pulls it in either way.
Rename gfx/drivers_shader/shader_vulkan.cpp to shader_vulkan.c and convert
the remaining C++ syntax: 14 autos become indexed loops and typed pointers,
casts become C casts, the extern "C" wrapper goes away, file-local typedefs
cover the header struct tags, and 35 reference parameters become pointers
or by-value PODs. All mixed declarations are hoisted for C89 and the three
for-init declarations are gone, so the file now clears
-Werror=declaration-after-statement clean for the MSVC lanes.
The incompatible-pointer-types gate caught two real conversion bugs before
they could ship: both set_semantic_texture wrappers compared the write_count
POINTER against VULKAN_MAX_DESCRIPTOR_WRITES and forwarded unsigned** into
the flush, and seven build_semantics call sites passed the batch counter by
value. Both would have corrupted descriptor batching at runtime.
Texture, VkViewport and the semantic-texture layer stay pointer-passed on
the per-frame path: by-value conversion initially cost +124 instructions
(+1.6%) in build_semantics/build_commands from struct copies at every call;
with the layer pointerized the final delta versus the C++ object is +4
instructions (0.1%), i.e. codegen parity.
The griffin include moves from griffin_cpp.cpp to griffin.c beside
slang_process.c (single-TU cohabitation of both files compiles clean), and
the Xcode project entry switches to sourcecode.c.c. griffin.c itself does
not compile at origin/master in this environment (pre-existing rc_export.h
include gap and test_input.c collisions), so the cohabitation TU stands in
for that lane.
Verified: 7/7 harness lifecycles under ASan+UBSan+LSan (leaks enabled),
byte-exact parameter/pass dumps across 1643 presets, production build with
zero warnings.
Eighth flattening slice: the last class in the TU. The
vulkan_filter_chain class becomes a plain struct with 55
slang_chain_* functions; the constructor becomes slang_chain_new()
over a calloc'd struct, the destructor slang_chain_free(), with
CommonResources' constructor and destructor likewise becoming
common_resources_init()/free() - the free runs last, preserving the
implicit C++ member-destruction order the flush drain depends on.
The memory_properties reference member becomes a value copy (the
properties are immutable per device, and the copy removes a lifetime
dependency on the caller's create_info). The memory pool's three
member functions become fbpool_* functions and its size cap a plain
define. C can't overload, so the parameterless set_hdr10/set_hdr16
flag setters split from their float-taking HDR-uniform namesakes as
slang_chain_set_emits_hdr10/16, which is what they always meant.
The public C API at the bottom of the file now calls the functions
directly instead of bouncing through methods.
All seven lavapipe smoke lifecycles run ASan+UBSan+LSan clean;
shader oracle byte-exact over 1643 presets; griffin single-TU
compile clean; TU codegen -10.8% instructions.
Seventh flattening slice: after this, no std:: type remains in the
TU. The framebuffer memory pool's block vector becomes a fixed
four-slot array with a count (the pool's own cap), CommonResources'
three per-frame Texture vectors become counted arrays behind one
fresh-zeroing resize helper (every resize site follows a clear, and
every element is rewritten before it is read), the chain's pass_info
vector becomes an overlap-preserving realloc'd array, and the
shader-preset unique_ptr becomes an owned pointer whose setter frees
any previous preset and whose storage moves to calloc/free to match.
The three chain-construction paths drop their unique_ptr wrappers
for explicit deletes on each error exit, and the include-cache RAII
guard in the preset path becomes an explicit handle freed at the end
of its scope and on the shared error label.
The chain smoke test caught the one real mistake in this slice
before it shipped: the pool's new block count was uninitialized
where the vector it replaced had default-constructed empty, and
lifecycle 0 SEGV'd inside lavapipe's vkFreeMemory on the very first
teardown. Zero-initialized, all seven lifecycles run
ASan+UBSan+LSan clean. Shader oracle byte-exact over 1643 presets;
griffin single-TU compile clean; TU codegen -6.3% instructions.
Sixth flattening slice, the largest by surface: the Pass class shell
becomes struct slang_pass with 25 free functions. The constructor's
init list becomes slang_pass_new() over a calloc'd struct (every
default was zero); the destructor becomes slang_pass_free(), and the
trivial inline accessors become direct member reads at their call
sites or tiny INLINE functions where they mutate. The chain's
vector of unique_ptrs to passes becomes a counted owned array, with
set_num_passes freeing the old set and counting successful creations
so a mid-loop failure tears down uniformly, and the chain destructor
picks up the explicit frees the smart pointers used to imply. The
Parameter record moves to file scope as slang_pass_parameter.
All seven lavapipe smoke lifecycles run ASan+UBSan+LSan clean -
LeakSanitizer green over the full create/build/render/teardown set
is the proof that no lifecycle point was lost in the class-to-struct
move. Shader oracle byte-exact over 1643 presets; griffin single-TU
compile clean; TU codegen -14.2% instructions.
Fifth flattening slice: every C++ container and string inside Pass
becomes plain C data, leaving only the class shell for a later slice.
pass_name goes from std::string to an owned C string (get_name still
returns a never-null string); Parameter.id likewise, with the
parameters vector becoming a counted owned array. filtered_parameters
becomes an array of indices into parameters[] rather than copies -
rebuilt by every build(), so it survives parameters growing (and thus
moving) in between, which is the property the by-copy vector was
implicitly providing. The vertex/fragment SPIR-V vectors become
owned word buffers, the push-constant buffer an owned uint32_t array
(zeroed on (re)allocation exactly as vector::resize zeroed it), and
the per-sync-index descriptor set vector a plain array.
init_pipeline_layout's binding/pool-size vectors become
count-then-fill arrays: the bound is the UBO slot plus every
semantic-texture entry, which is preset-dependent and not small, so
no fixed-size shortcut. The destructor picks up explicit frees for
all of it.
All seven lavapipe smoke lifecycles run ASan+UBSan+LSan clean, with
114 gdb-counted parameter registrations across the preset set (both
the preset-override and fresh-from-metadata branches) and the
push-constant upload path driven by the frame-building lifecycle.
Shader oracle byte-exact over 1643 presets; griffin single-TU
compile clean; TU codegen -14.7% instructions - the largest win of
the series, mostly dropped string and vector machinery.
Fourth flattening slice, and the last leaf class. Framebuffer
becomes struct slang_framebuffer: the constructor's
create-then-check-is_valid() protocol becomes slang_framebuffer_new(),
which returns NULL on any failure and frees partial state itself;
init() becomes slang_framebuffer_build() (the (re)build shared by
creation and set_size); the getters become member reads at the call
sites; and the memory_properties reference member becomes a pointer.
Pass holds its framebuffer and feedback framebuffer as plain owned
pointers with an explicit delete helper (the feedback swap is a
three-line pointer swap now), and the chain's history vector becomes
an owned pointer array counted incrementally so a mid-loop creation
failure frees uniformly. Pass and chain destructors gain the
explicit deletes their unique_ptr members used to imply.
All seven lavapipe smoke lifecycles run ASan+UBSan+LSan clean. On a
no-inline build gdb counts 22 slang_framebuffer_new calls and 30
slang_framebuffer_build calls across the full run - the 8 extra
builds are exactly the render lifecycle's 8 set_size resizes,
matching the deferred-disposal count that gate was built to verify.
Shader oracle byte-exact over 1643 presets; griffin single-TU
compile clean; TU codegen -2.9% instructions.
Third flattening slice. StaticTexture - a LUT image with its staging
buffer and semantic id - becomes struct slang_static_texture with
init/free functions, and CommonResources' vector of unique_ptrs to it
becomes a plain owned array. The std::string id becomes a strdup'd
C string (its only readers passed .c_str() into the semantic name
maps), and the never-called set_id() accessor is dropped rather than
ported. add_static_texture keeps its ownership-transfer shape - the
loader fills a local record and hands it over, the chain zeroes the
source - growing the array by one per LUT, which is fine at LUT
counts. slang_static_texture_free is idempotent and safe on a
zeroed record, so the partially-filled array frees uniformly in
~CommonResources on any load failure.
All seven smoke lifecycles (including the two-LUT preset added last
slice, which is what actually exercises this path) run
ASan+UBSan+LSan clean on lavapipe; shader oracle byte-exact over
1643 presets; griffin single-TU compile clean; TU codegen -2.6%
instructions.
A transfer that never reaches a status now reports where it stopped.
net_http keeps the first failing stage ("dns_lookup_failed",
"ssl_connect_failed", ...) and, for the TLS stages, the library's own
error through the new ssl_socket_last_error(), which both backends
implement: the last mbedtls return code from config, setup, hostname
or handshake, and BearSSL's engine error. net_http_failure() hands
them to the task layer, whose error string becomes
"Download failed: ssl_connect_failed (-0x7200)." rather than
"Download failed.", and the webdav failure line appends it whenever
the status is -1.
This is what a log from a console can carry: the transport logger
only exists in DEBUG builds and writes to stderr, so a Vita that could
not complete a handshake had nothing to show but "HTTP -1". The
mbedtls code is printed in the form its error table uses, so -0x7200
reads as MBEDTLS_ERR_SSL_INVALID_RECORD without a debug build
(libretro/RetroArch#19475).
retro_vfs_file_rename_impl() means "replace" - the Win32 branch says
so with MOVEFILE_REPLACE_EXISTING, POSIX says so by definition - and
the write-to-temporary-then-rename pattern that playlists, the config
file and core info rely on is built on it. The Vita branch now calls
sceIoRename() itself and, when an existing destination stands in the
way, moves it aside, moves the new file in, and only then removes the
old one; if the move in fails the old file is put back. Nothing is
ever removed until its replacement is in place.
newlib's rename() reaches the same kernel calls in the other order,
sceIoRemove() of the destination first and the rename after, so a
rename that fails leaves nothing on disk, and a caller that then
discards its temporary has lost the file it was replacing. That is a
playlist emptied after a scan, or a retroarch.cfg gone after a save
(libretro/RetroArch#19480, libretro/RetroArch#19479).
setting_uint_setting() and setting_string_setting() take name and
short_description as given. The callers that build a runtime name -
the per-user _ALT constructors - already pass heap copies and tag the
entry with SD_FREE_FLAG_NAME | SD_FREE_FLAG_SHORT, so the copy the
constructor made on the same flag was a second one that nothing
owned: 448 allocations and about 10 KB per settings-list build, and
a LeakSanitizer report on every run.
The Vita touch panels stand in for L2/R2/L3/R3 in-game only. In the
menu the front panel is the pointer and the rear one is where the
hands rest, so a trigger raised from either is a page scroll nobody
asked for; with the release-wait it was also a button that never
released. query_pad now answers what the other joypad drivers answer -
whether the pad is connected: always for the built-in controls, and
paired-or-not per port on the TV model - instead of whether a button
happens to be held. The poll loop's result variable moves to the top
of its block so the file builds in the C89 lane.
The Online Updater's bundle downloads (assets, overlays, cheats,
autoconfig, core info, databases, shaders, thumbnail packs, content
and system files) now stream to their destination as they arrive,
through the same sink the core updater already uses, instead of
accumulating the whole body - up to 75 MB for assets.zip - in a single
allocation and writing it out at the end. The destination directory is
resolved at push time by a helper shared with the completion callback,
which keeps reading the extraction settings at completion and extracts
beside the archive; a push whose target archive is still being
extracted is refused, and one that cannot open its sink falls back to
the in-memory transfer.
filestream_write_file() and its atomic twin open with
RETRO_VFS_FILE_ACCESS_HINT_SEQUENTIAL_BULK, and the VFS honours that
for writes where VFS_HAVE_DESCRIPTOR_WRITE is defined - on Vita, where
the descriptor path now calls sceIoOpen/Read/Write/Lseek/Close
directly rather than newlib's wrappers, which add a realpath
allocation, a directory stat, a strdup of the path into the fd table
and a mutex round-trip per call, with a FILE, a 64 KiB buffer and an
fstat on top from stdio. A truncating open also skips the seek/tell/
seek size probe on every platform, and the decompress task remembers
the last directory it created so a run of members in one directory
costs one probe. Per extracted member on a Vita SD card that is three
kernel calls where there were nine, most of the removed ones being
directory lookups (libretro/RetroArch#19479).
Second flattening slice. The Buffer class - a host-visible VkBuffer
with backing memory, held through std::unique_ptr by CommonResources
(vbo/ubo), StaticTexture (LUT staging), and the LUT loader - becomes
struct slang_buffer with init/map/unmap/free functions. CommonResources
and StaticTexture now hold it by value: a zeroed struct is the empty
state, slang_buffer_free() is idempotent and safe on any state, and
ownership transfer into StaticTexture zeroes the source struct where
the constructor used to take the unique_ptr by move.
Two lifecycle points that the smart pointer handled implicitly are now
explicit and were verified rather than assumed: the LUT loader's error
path frees the staging buffer it may hold, and ~CommonResources frees
vbo/ubo.
The chain smoke test gained a LUT lifecycle
(crt-hyllian-sinc-composite, two sampler LUTs) since none of the
existing presets exercised the staging path at all: on a no-inline
build, gdb counts four slang_buffer_init calls (vbo, ubo, two LUT
stagings) and seven frees, matching the two post-upload staging
releases, the two now-idempotent StaticTexture destructor frees, the
CommonResources pair, and the ubo reset on (re)build. All seven
lifecycles run ASan+UBSan+LSan clean on lavapipe; shader oracle
byte-exact over 1643 presets; griffin single-TU compile clean; TU
codegen -1.4% instructions.
First slice of flattening the Vulkan filter chain toward C. The
deferred-disposal mechanism was a
std::vector<std::vector<std::function<void()>>> plus a DeferredDisposer
RAII wrapper, all in service of exactly one call site: Framebuffer::
set_size deferring the destruction of a replaced framebuffer's
resources until its sync index drains. The captured state is plain
data - device, pool pointer, and six Vulkan handles - so the
type-erased callables become an array of deferred_fb_dispose records
with one run function, and the per-sync-index queues become a plain
resizable array.
Behaviour notes: the render-pass handle is stored only when the format
changed, folding the lambda's format_changed capture into the record;
and on allocation failure while queueing, the resources are leaked
with an error logged rather than destroyed - an in-flight command
buffer may still reference them, and the std::vector this replaces
terminated the process on the same failure.
Verified on the lavapipe chain smoke test, extended for this change
with a frame-building lifecycle: two frames with different input
sizes through a five-pass preset force intermediate framebuffers
through set_size with live old handles, exercising the queue loaded -
eight disposals pushed, eight executed (gdb-counted on a no-inline
build), all six lifecycles ASan+UBSan+LSan clean. Shader oracle
byte-exact over all 1643 presets; TU codegen -4.3% instructions;
griffin single-TU compile clean.
INP_FLAG_WAIT_INPUT_RELEASE now keeps a per-port set of the RetroPad
buttons that were held on the frame it was armed (by a menu flush, the
bind flow or the dialog Escape path), pruned as they are released. Only
those buttons are withheld; anything pressed after arming is delivered,
and the wait clears once the captured set is empty rather than once
nothing at all is held.
A button that stays down for reasons unrelated to the menu - the Vita
rear-touch L2/R2/L3/R3 mapping with a finger resting on the pad, a
stuck or remapped key - can otherwise keep the wait armed after any
touch or menu action, dropping every digital button while analog-stick
and pointer navigation keep working (libretro/RetroArch#19478).
config_file_new_with_callback tears the conf down with a raw free()
on the read-failure path (ret == 1), but the object from
config_file_new_alloc() carries live internals - the entries hash
map config_file_initialize sets up. Raw free() leaks them: 368
bytes on every failed config read, and RetroArch probes missing
configs constantly (overrides, remaps, per-core and per-game shader
presets), so this accumulates for the lifetime of the process. Use
config_file_free(), as the parse-failure path one branch above
already does.
Found by the new headless-Vulkan chain-lifecycle smoke test: driving
vulkan_filter_chain_create_from_preset with a nonexistent preset
path under LeakSanitizer on lavapipe flagged the allocation, with
the alloc stack pointing into config_file_initialize and a gdb
free-trace showing no config_file_free call anywhere in the
lifecycle. With the fix the full lifecycle set - default chain,
single-pass, feedback, multi-pass, and failed-create - runs
ASan+UBSan+LSan clean.
Two defects in the HAVE_BUILTINSPIRV_CROSS=0 path, both latent since
slang_process moved onto the SPIRV-Cross C API. The configure
fallback accepted a C++-only system installation
(spirv_cross/spirv_cross.hpp + -lspirv-cross-core/-glsl), which
carries no spvc_* symbols and can only fail at link now; drop the
fallback so the spirv-cross-c-shared pkg-config probe is the one way
to satisfy an external build, with the existing notice-and-use-builtin
path handling its absence. And Makefile.common fed
SPIRV_CROSS_CFLAGS to CXXFLAGS alone, but slang_process.c is C - the
header path never reached the C compiler; add it to CFLAGS.
Verified against a real foreign installation
(libspirv-cross-c-shared 0.54.0): slang_process.c compiles against
the system header with the vendored include dir removed, links
against the system library, and the shader oracle over all 1643
libretro/slang-shaders files is byte-identical to the builtin
build's output.
On devkitPPC's newlib, uint32_t is 'unsigned long' while spirv.h
hardcodes SpvId as 'unsigned int': same width, distinct types. The
vendored spirv_cross_c.cpp passes its const SpvId* parameter straight
into Parser(const uint32_t*), an ill-formed conversion that hard-fails
the WiiU lane, and slang_process.c passed uint32_t* word buffers where
the C API declares const SpvId*. Invisible on every toolchain where
the two types are identical.
Cast at both boundaries: a RetroArch-local reinterpret_cast in the
vendored translation unit (commented for the next sync against
upstream SPIRV-Cross, which has the same latent issue), and explicit
casts at the four parse call sites, backed by a compile-time check
that SpvId and uint32_t have identical width - which the SPIR-V
specification guarantees.
Also gate the stage-source assembly helpers under HAVE_GLSLANG: their
only caller lives inside that guard, so slang-without-glslang builds
like WiiU compiled them as dead code and warned under -Wall.
Verified with a WiiU-shaped profile compile (HAVE_SLANG +
HAVE_BUILTINSPIRV_CROSS without HAVE_GLSLANG, -Wall -Werror): clean.
Byte-exact shader oracle over the full corpus, both compile modes
clean, amalgamation symbol closures unchanged.
slang_process.c is C and amalgamated into griffin.c; it includes
<spirv_cross_c.h>, but the SPIRV-Cross directory was only on
LOCAL_CPPFLAGS, which ndk-build applies to C++ compiles alone -
griffin.c failed with 'spirv_cross_c.h' file not found. Move the
directory to LOCAL_C_INCLUDES, which covers both languages, and drop
the now-redundant CPPFLAGS copy so the path has one source of truth.
Every other lane already carries the directory for C: Makefile.common
via its SPIRV_CROSS include block, MSVC and UWP in the vcxproj
include dirs, the Metal project in HEADER_SEARCH_PATHS.
Verified by preprocessing griffin.c with exactly the C include set
this makefile provides: the full include graph resolves.
The C API integration defined SPIRV_CROSS_C_API_HLSL=1
unconditionally in griffin_cpp.cpp, but the amalgamation includes
spirv_hlsl.cpp only under HAVE_HLSL. On every griffin lane without
that flag - Android NDK and the Apple Metal project - the wrapper's
HLSL sections referenced CompilerHLSL methods and its vtable with the
backend never compiled: undefined-symbol link failures. Windows
lanes define HAVE_HLSL, which is why Windows-shaped verification
missed it.
Make the macro mirror the include guard, and document the invariant:
each SPIRV_CROSS_C_API_* macro must be truthy exactly when the
matching backend source is amalgamated (glsl and msl are
unconditional here, so their macros stay 1; the Makefile.common
object list is unconditional for all backends, so its =1 defines are
consistent).
Verified with a symbol-closure check over the amalgamated object
under both feature profiles: compiled without HAVE_HLSL (the
Android/Metal shape) and with it (the Windows shape),
nm reports zero undefined spirv_cross:: symbols in either object -
and the same check on the pre-fix object shows the six undefined
CompilerHLSL symbols, so the closure check now runs on every
drivers_shader delivery as a standing gate.
After the SPIRV-Cross C API port nothing C++ remained but syntax:
reference locals in the offset setters and reflection paths become
pointers, bare enum type names gain the enum keyword, declarations
move ahead of statements for C89 (in the offset setters the pointer
targets are still resolved only after the resize_min call, which may
reallocate the array they point into), and the two bare video_shader
uses gain the struct keyword. The file moves from the griffin_cpp
amalgamation into griffin.c; Makefile object lists are name-based and
need no change, the GL-only Apple project does not build the slang
stack, and the Metal project reaches the file through griffin.c with
SPIRV-Cross already on its header search paths.
The translation unit now compiles as C (and still as C++ for
CXX_BUILD, verified both ways with -Werror=declaration-after-statement
on the C side). Static-name audit over all 752 files amalgamated by
griffin.c found no collisions. Byte-exact against the C++ build over
all 1643 libretro/slang-shaders files; ASan+UBSan+LSan and TSan clean
over the corpus and byte-exact under instrumentation; the
uninitialized-reflection lifecycle repro still faults on the buggy
pattern and passes on the shipped one; the C-struct-member initializer
audit over both chain files is clean. Codegen vs the C++ compile of
the same code: -1.1% instructions (4538 -> 4490), .text 24699 ->
24443 bytes, end-to-end compile+reflect over 400 presets 11.41s vs
11.50s.
The reflection data-model conversion made Pass::reflection a plain C
struct inside a still-C++ class, and neither chain's Pass constructor
initializes it - the previous C++ slang_reflection had a default
constructor that ran implicitly, and removing it silently removed
that guarantee. The first Pass::build() call (and ~Pass() on a pass
torn down before ever building) runs slang_reflection_free() on
indeterminate bytes: free() of wild pointers, which Windows' heap
reports as the c0000374 heap-corruption abort seen from
vulkan_filter_chain_create_default on the very first content load.
The stock chain hits it before any preset is even involved.
Give the member an explicit zero initializer in both the Vulkan and
GLCore chains, which restores the old constructed state as far as the
free/init lifecycle is concerned (slang_reflection_free on the zeroed
struct is a no-op by design). Reproduced deterministically with a
placement-new harness over poisoned memory against the real
slang_reflection_free: the uninitialized pattern faults inside free,
the initialized one is sanitizer-clean.
The shader oracle never constructs Pass objects and no CI lane runs
content, which is how this escaped: chain-object lifecycle now needs
the same explicit-initializer audit as any C-struct member landing in
a C++ class.
Rewrite the SPIRV-Cross usage in slang_process.cpp - cross
compilation, reflection, buffer-range walking and type validation -
from the C++ API onto the vendored spirv_cross_c wrapper. One
spvc_context per slang_process()/slang_reflect_spirv() call owns
every IR, compiler, options object, resource list and compiled
source string, so teardown is a single spvc_context_destroy() on
every path and the try/catch + new/delete + std::string usage is
gone: SPIRV-Cross exceptions are absorbed into error codes inside
the wrapper. The MSL binding remaps keep the same explicit 1:1
mapping the CompilerMSL lambdas set up. Requesting the HLSL
backend without HAVE_HLSL now fails cleanly instead of running the
rest of the function against NULL compilers.
slang_process.h no longer includes spirv_cross.hpp or declares
slang_reflect (now static); shader_vulkan.cpp and shader_gl3.cpp
gain the <string> include they had been getting transitively.
deps/SPIRV-Cross/spirv_cross_c.cpp joins the build with the
GLSL/HLSL/MSL backend sections enabled, in Makefile.common for the
standalone build and beside the other vendored SPIRV-Cross sources
in griffin_cpp.cpp for the griffin lanes.
Verified byte-exact against both the pre-conversion implementation
and the current C++-API implementation over all 1643 .slang files in
libretro/slang-shaders, including full reflection with populated
semantic maps. ASan+UBSan+LSan clean over the corpus and byte-exact
under instrumentation (the vptr checker is disabled for the vendored
spirv_cross_c.cpp only: its opaque handles are 'struct spvc_type_s :
SPIRType' downcasts by upstream design); TSan clean with 4 threads.
griffin_cpp.cpp and griffin_glslang.cpp each compile as one
amalgamated TU, and the glslang bridge closure still reaches none of
the windows.h-bearing headers.
Codegen (x86-64 gcc 13 -O3): the slang_process TU drops 55.3% of
its instructions (10159 -> 4538; .text 53.3KB -> 24.7KB) as the
SPIRV-Cross template instantiations disappear; the program gains the
shared spirv_cross_c wrapper TU (89KB .text) which the planned
d3d/gl3/metal conversions will reuse. End-to-end compile+reflect
over 400 presets measures faster than the original implementation
(11.50s vs 12.10s), parse path unchanged (~3.6s vs baseline ~3.9s).
The glslang bridge is amalgamated with the vendored glslang library
sources in griffin/griffin_glslang.cpp, and <windows.h> breaks that
translation unit from either side. Included before the library (the
data-model conversion made the wrapper pull glslang_util.h ->
video_shader_parse.h -> retro_miscellaneous.h -> windows.h), its
function-like min()/max() macros mangle glslang's
std::numeric_limits uses - the first MSVC breakage. The previous
fix moved the wrapper after the library, which put windows.h after
glslang_tab.cpp's global token enumerators instead, and minwindef.h's
BOOL/INT/UINT/FLOAT typedefs then collide with them - the UWP
C2365 'redefinition; previous definition was enumerator' cascade.
Root cause both times: the bridge has no business dragging platform
headers at all. Its entire RetroArch-facing surface is one enum and
one function. Move those to a new minimal glslang_compile.h
(stdint/stddef/boolean/retro_common_api only), include it from
glslang_util.h so every existing consumer is unchanged, and make the
bridge include only that - restoring the pre-conversion include
hygiene of the deleted glslang.hpp. griffin_glslang.cpp goes back
to its original wrapper-first order, documents the invariant, and
gains a compile-time tripwire on the windows.h/minwindef.h/windef.h
guard macros so any future leak fails the first Windows build with
one diagnostic instead of hundreds of cascade errors.
Verified: the wrapper's include closure reaches zero of
video_shader_parse.h/configuration.h/retro_miscellaneous.h (was
three); griffin_glslang.cpp and griffin_cpp.cpp each compile as one
amalgamated TU; the tripwire fires when a windows.h guard macro is
simulated; full build links; shader oracle output over all 1643
libretro/slang-shaders files is byte-identical.
Since the glslang data-model conversion, the RetroArch wrapper
(gfx/drivers_shader/glslang.cpp) includes glslang_util.h, whose chain
reaches <windows.h> through video_shader_parse.h ->
retro_miscellaneous.h. windows.h defines function-like min()/max()
macros (no NOMINMAX here), and griffin_glslang.cpp included the
wrapper first, so every vendored glslang header after it was parsed
with those macros live: std::numeric_limits<T>::max() and friends
were macro-mangled and all MSVC lanes failed with the C2589/C2059
cascade seen in CI (the earlier syntax damage also left an unbalanced
brace, which is why later errors show glslang nested inside
spvutils).
Move the wrapper to the end of the amalgamation so the vendored
sources are always parsed with pristine macro state, no matter what
the wrapper includes now or in the future. The include set is
otherwise unchanged (same files, same guards, verified as a directive
set diff); only the wrapper's position moves. The prebuilt-glslang
branch is untouched.
The mechanism reproduces on g++ by defining function-like min/max
before including a vendored glslang header (macro 'max' passed 3
arguments); the reordered amalgamation compiles as one TU with the
production flag set.
Replace the C++ slang_reflection (std::vector members, unordered_map
name maps) with a plain C data model in slang_process.h: texture
semantics as counted heap arrays, float parameters as one counted
heap array, and the pass-alias/LUT/parameter name maps as counted
entry arrays with explicit set_unique/free lifecycle. New array
elements and slang_reflection_init() preset every GL location to -1,
matching the previous default-constructed state gl3 depends on.
slang_reflect_spirv() now takes (words, count) pairs per stage.
slang_reflect() stays C++ internally against SPIRV-Cross; both
filter chains and slang_process_reflection are converted, the local
parameter map is freed as soon as reflection returns (it was a
dangling pointer past that point in the old code too), and
slang_process_reflection now hands its uniform/texture arrays to the
caller with a sentinel appended instead of building std::vectors and
malloc-copying them wholesale. The unused slang_texture_semantic_map/
slang_semantic_map pair is gone.
Map lookups carry the entry name length so the find gates its memcmp
behind two byte loads, like the parameter scans.
Verified byte-exact against the pre-conversion implementation over
all 1643 .slang files in libretro/slang-shaders - SPIR-V, meta,
parameters and full reflection, with the harness populating the
parameter semantic map from each shader's own parameters plus
synthetic pass aliases so map hits are exercised for real (73542
dump lines). ASan+UBSan+LSan clean over the corpus, byte-exact
under instrumentation; TSan clean with 4 threads.
Codegen (x86-64 gcc 13 -O3, objdump instruction counts) vs the
previous C++ model: slang_process TU -16.8%, shader_vulkan -5.8%,
shader_gl3 -8.2%; init_alias -243/-244 and Pass::build -188/-178 in
the chains. End-to-end compile+reflect over 400 presets: 13.01s vs
13.48s, peak RSS lower; the parse path stays ~10% ahead of the
original.
The previous commit left slang_cache.cpp written in plain C (fixed
buffers, malloc, C89 declaration placement, guarded extern "C"), so
the rename is mechanical. griffin moves the include from
griffin_cpp.cpp to griffin.c, which is what makes the MSVC lanes
compile it in C mode; Makefile.common's OBJ entry is name-based and
unchanged. Compiles both as C (-Werror=declaration-after-statement)
and as C++ for CXX_BUILD.
Replace the C++ glslang_parameter/glslang_meta/glslang_output structs
(std::string, std::vector) with a plain C data model in slang_process.h:
SPIR-V words as malloc'd arrays with explicit word counts, parameters
as one contiguous malloc'd array (id[64]/desc[64] fixed buffers sized
to the video_shader_parameter fields they merge into), and explicit
glslang_output_init()/glslang_output_free() lifecycle (free
re-initializes, so calling it twice or after a failed compile is safe).
Every consumer - slang_process, slang_cache, the Vulkan and GLCore
filter chains - is converted, with explicit frees on every exit path
replacing the removed RAII. glslang.cpp shrinks to the one
deliberately-C++ TU (the vendored glslang has no C interface), exposed
through a C ABI in glslang_util.h; glslang.hpp is gone.
Verified byte-exact against the previous implementation over all 1643
.slang files in libretro/slang-shaders: SPIR-V words, meta, parameters
and full reflection output are identical. ASan+UBSan+LSan clean over
the corpus; TSan clean with 4 threads compiling concurrently.
Codegen (x86-64 gcc 13 -O3, objdump instruction counts):
slang_process TU -9.2%, shader_vulkan -0.7%; per-function,
create_from_preset -170/-55 and compile_full_pass -128 instructions
in the chains. The parse path measures ~10% faster over the corpus
(4.04-4.24s -> 3.60-4.00s, 30x repetition) with 15x fewer heap
allocations (815662 -> 53062 mallocs) and equal-or-lower peak RSS.
The parameter scans are O(n^2) and Mega Bezel-class presets carry
1000+ parameters with long shared prefixes, so the id lookups gate
their memcmp behind two byte loads - the std::string version got the
same effect from its stored size, and the C model without the gate
measured 22% slower before this was accounted for.
Two deliberate behavior notes: '#pragma name' values now truncate at
63 chars at parse time instead of at the alias[64] copy (downstream-
identical); and spirv_cache_load now bounds on-disk counts (2^24
words per stage, GFX_MAX_PARAMETERS) where the old code resized
vectors straight from untrusted u32s, so a corrupt cache file could
demand a multi-GB allocation and abort via std::bad_alloc. Stage
sources are also now assembled once and reused for both the cache
hash and the compile, where the old path built each stage twice, and
the vertex source is released before the fragment compile so only one
stage source is resident at a time.
Callers probe stored lines with fixed-width prefix compares such as
memcmp(line, "#pragma parameter ", 18). memcmp may read all n bytes
even when an earlier byte differs, so a line shorter than the probe
stored at the very end of the data block was read past the allocation
(heap-buffer-overflow READ under ASan, hit by real presets in
libretro/slang-shaders).
Reserve 32 zeroed bytes of slack beyond buf->len in the capacity
check and clear them after every append, so every prefix probe on the
final line reads defined, in-bounds bytes. Covers the longest probe
literal with margin.
The SET_SYSTEM_AV_INFO no-video-reinit shortcut is taken under CRT
switchres too when the base resolution and field rate hold, since those
are what the switched mode is derived from. A core that re-announces its
av_info only to move the sample rate keeps the mode switchres picked
from the first frame, and only the remaining drivers - audio among
them - are reinitialised to pick up the new rate; a call that moves base
geometry, fps or max geometry still gets the full reinit and the mode
switch that follows from it.
vfs_read_overflow_test drives the two halves of the bound separately: a
wrapping length (mappos=10, len=UINT64_MAX-9) has to come back as a
failed read, since every length able to wrap sits above INT64_MAX and is
refused at entry, and the mapped clamp is then exercised with INT64_MAX,
the largest admitted length, which must return the six bytes remaining
in the mapping.
Restores the ASan discriminator: with the entry guard taken out, the
wrapping read reports six bytes and the test fails.
A Play Console ANR (1.22.2, arm64, Android 15) shows the Java UI
thread parked in android_app_set_window() -> scond_wait() under
surfaceDestroyed() during Activity.performStop, with no app thread
anywhere in the dump - only the idle task-queue worker remains. The
waiter is waiting for an acknowledgement from the only consumer of
the command pipe, and that consumer is gone.
rarch_main() returning from android_app_entry() is how this state
arises: a normal quit never returns (frontend_android_shutdown()
exits the process), but an init failure before the shutdown
machinery runs does, and so does the framework-destroy unwind. After
such a return the activity can live on indefinitely; the next
synchronous lifecycle callback posts TERM_WINDOW into the pipe,
takes a ticket, and waits forever - ActivityManager converts that
into an ANR roughly five seconds later.
The app thread now retires every outstanding ticket and sets
app_thread_exited (under the glue mutex, with a broadcast) as it
leaves android_app_entry(). The synchronous waiters -
android_app_set_window() and android_app_set_input() - treat the
flag as a standing acknowledgement: they stop posting commands
nothing will read and fall through their wait loops, which is safe
because a thread that no longer runs cannot touch the surface or
input queue being handed back. The activity-state notifications
already give up via their timeout; checking the flag there just
saves each of them a three-second stall per callback.
The struct outlives the app thread on every path, so the tail-of-
entry access is sound: android_app_free() joins the thread before
freeing, and deliberately leaks the struct when it orphans a thread
that failed to acknowledge destroy.
Compile-verified for the Android target with NDK r27c clang
(aarch64-linux-android31) and for the non-Android unix build; the
failure needs a device-side init failure plus a lifecycle event to
reproduce, so no runtime repro accompanies this.
The swap-interval block guards video_vsync, video_scanline_sync and
the set_nonblock_state member, but never current_video itself:
if (video_vsync && !video_scanline_sync
&& current_video->set_nonblock_state)
Observed as a Play Console SIGSEGV at menu_driver_toggle+680 on an
Android TV box (1.22.2 / 2026-08-26, armeabi-v7a, BuildId d75406ff)
via retroarch_menu_running <- runloop_check_state; the faulting
instruction is the unguarded set_nonblock_state load at
current_video + 8. The toggle can run while the video driver is torn
down or failed to initialize - the same shutdown-path family as the
existing settings-NULL workaround at the top of this function - and
on TV boxes display loss makes that state reachable in normal use.
A Play Console SIGABRT on Android x86_64 1.22.2 (BuildId 44c13ba3):
bionic FORTIFY aborted fread() with count 18446744071855567113 -
(int)2440982793 sign-extended, i.e. a ~2.3GB value that went negative
through a 32-bit type - reached via sevenzip_parse_file_init ->
filestream_raw_read while scanning an archive from the menu.
Instruction-level verification of the arm64 build of the same commit
shows every length on that path is guarded (the slurp's len <= 0
check and the clamped read-buffer math are all present in the
generated code), and the source compiles identically for x86_64, so
a negative length cannot originate from the 7z path as written. The
crashing stack runs through nested menu action dispatch - exactly
where 1.22.2 performs the stale-menu-handle heap writes since fixed
on master (38234ab96d, 6929c66ec9) - so corrupted stream state or a
stomped spill is the consistent producer, and the root cause is
already fixed.
What remains ours: retro_vfs_file_read_impl() takes uint64_t len and
the stdio path hands it to fread() unclamped, turning any corrupted
or hostile length into a process abort, while the mapped path right
below already clamps its own overflow case. Reject len >
(uint64_t)INT64_MAX at entry so every backend (stdio, cdrom, smb,
mapped) degrades to a failed read - an archive that fails to open -
instead of an abort. Same boundary-hardening class as e8679e552d.
video_driver_free_internal() hands video_st->data to the driver's free()
and leaves current_video pointing at a live vtable, so every path that
reaches the driver through that handle before the next init reads freed
memory. Clear it, and check it at the call sites that only tested the
vtable: the recording viewport query, set_rotation() and
read_frame_raw(). video_driver_get_viewport_info() now reports no
viewport for a torn-down driver, the same answer it already gives for
one that implements no viewport_info at all.
Input drivers are how this gets reached: a pointer or mouse event
arriving while the window is going down runs the coordinate translation
wrappers, which query the viewport, and every GL, Vulkan and D3D
viewport_info dereferences its data argument on entry. On Android the
surface teardown and the queued touch events are serviced from the same
poll loop, which puts the two in the same frame.
The end-of-teardown fps statistics now key off a flag taken before the
free rather than off the handle itself.
Covered by a harness driving the accessor against a driver that
dereferences its handle; the pre-change accessor faults there under ASan
and the current one reports no viewport. C89 and syntax checked.
When onDestroy() times out waiting for the app thread to acknowledge
APP_CMD_DESTROY it returns without joining, leaving that thread to
finish rarch_main() on its own. If the activity is then recreated in
the same process, android_app_create() now joins that thread before
spawning a new one, so two rarch_main() instances never share the
process statics: the second task_queue_deinit() was joining a worker
the first had already joined and freed, which bionic rejects as an
invalid pthread_t and aborts on. The app thread never waits for the
UI thread, so the join always completes.
input_config_get_bind_string and its joykey/joyaxis helpers accumulate
_len from snprintf, strlcpy and fill_pathname_join_delim, all of which
return the untruncated length of what they formatted. Once _len passes
len, every later "len - _len" size argument underflows to a huge
size_t and every "s + _len" write lands past the destination.
The reachable producer is menu search: menu_entries_list_search calls
menu_entry_get without MENU_ENTRY_FLAG_VALUE_ENABLED, and menu_entry_get
still invokes action_get_value (to fill the entry path), passing 0 as
the value buffer length. For a bind entry showing an autoconfigured hat
bind, "Hat #0 " contributes 7 would-be bytes and the direction 2, so
the "(Auto)" suffix snprintf receives (size_t)(0 - 9):
FORTIFY: vsnprintf: size 4294967287 > SSIZE_MAX
#04 input_config_get_bind_string_joykey+664
#09 menu_entries_list_search
observed on the 1.22.2 Play build (armeabi-v7a, Android 12) the moment
a user searched a menu containing an auto-bound entry. bionic aborts on
the underflowed size; libcs without the check format through it, and
the strlcpy segments before it write out of bounds whenever a genuine
overshoot (a long autoconf label or localized key name against a small
destination) rather than a zero length produced the underflow.
Every accumulation point now saturates _len at the last writable byte,
and a zero-length destination returns 0 without touching it, matching
snprintf's own contract. The clamped value is also what the functions
now return, so callers that use the result as an offset no longer
inherit the same underflow.
Verified with the function bodies extracted verbatim into an ASan
canary harness, pre- and post-patch: the exact crash case (len 0, auto
hat bind, suffix) returns 16 before and 0 after; truncating cases that
returned would-be lengths past the buffer now return the written
length; no canary byte is touched in any case.
retro_vfs_file_read_impl handed the caller's full count to a single
fread. bionic's stdio dispatches that through the FILE's int-typed
read callback, so a count above INT_MAX truncates to a negative int
inside libc, __sread sign-extends it back into read(), and FORTIFY
aborts the process:
FORTIFY: read: count 18446744071855567113 > SSIZE_MAX
#05 fread
#06 filestream_raw_read
#07 sevenzip_parse_file_init
18446744071855567113 is (int64_t)(int32_t)2440982793: a 2.4 GB 7z
archive slurped whole by the 7z backend on an x86_64 Android
Chromebook, observed in the wild on the 1.22.2 Play build the moment
the file browser probed the archive for supported cores. Any other
caller reading or writing more than 2 GiB through the stdio-backed
VFS dies the same way on bionic.
The raw-fd paths have the sibling defect without the abort: one Linux
read()/write() syscall moves at most MAX_RW_COUNT (INT_MAX & PAGE_MASK)
and silently returns short, so callers that treat a short count as EOF
or error break on >2GiB transfers.
All four transfer sites (stdio fread/fwrite, raw read()/write()) now
loop in chunks capped at 1 GiB, returning the accumulated count with
unchanged short-read/short-write semantics at EOF and on error.
Verified byte-exact with the cap overridden to 7 bytes: a 1000-byte
pattern written and read back through both the stdio and unbuffered
paths crosses ~143 chunk boundaries per transfer and compares equal,
with a trailing read returning 0 at EOF.
audio_driver_sample_batch_float() takes the state lock, flushes and
releases the lock as one guarded block, so a paused or fast-forward
frame that skips the flush also skips the lock. In
audio_driver_mixer_add_stream() the malloc failure branch now covers
both the unlock and the return, so a successful allocation goes on to
copy the caller's bytes and load the stream instead of returning false
unconditionally. Drop the unused byte counter in the threaded submit
path, and size the firmware key prefix in core_info for any 32-bit
index so snprintf cannot truncate it.
Audit follow-up to 38234ab96d (menu: re-fetch the menu handle after
menu_entry_action()): the same invalidation reaches
runloop_check_state(). The menu handle is cached at function entry,
menu_driver_iterate() runs entry actions that can free and recreate
it, and the render block that follows reads and writes the stale
pointer - menu->state bit updates, driver_ctx->render_messagebox /
render calls, and menu->userdata handed to the old driver. On 64-bit
heaps this lands in freed memory silently on every
menu-reinit-via-action (e.g. changing the menu driver from the menu)
rather than faulting.
The existing re-fetch after display_menu_libretro() (e03f25c569)
covers core_run() invalidating the handle, but sits below this
render block; add the matching re-fetch above it. Both are needed:
iterate and core_run are separate invalidation points with uses
between them.
Found by auditing every menu_st->driver_data cache that survives
across a command_event/action dispatch; the remaining flagged sites
were verified safe (uses precede the dispatch, cross-case switch
noise, or commands that cannot reach RARCH_MENU_CTL_DEINIT).
generic_menu_iterate()'s ITERATE_TYPE_DEFAULT case runs the selected
entry's action handler and then writes
BIT64_SET(menu->state, MENU_STATE_POST_ITERATE) through the menu
handle it was passed. menu_entry_action() can tear down and recreate
that handle - a menu driver change, or any CMD_EVENT_REINIT
dispatched from an action - leaving the parameter and the cached
entries list stale; the tail of the function then reads and writes
menu->state through freed memory.
Observed as a Play Console SIGSEGV at menu_driver_iterate+4088 on a
32-bit Android 11 TV (1.22.2 / 2026-08-26, armeabi-v7a, BuildId
d75406ff); the faulting instructions are the read-modify-write of
the POST_ITERATE bit through the stale pointer. On 64-bit heaps the
same write usually lands in still-mapped freed memory, so this has
been corrupting quietly rather than crashing.
Re-fetch the handle and the entries list after the action returns;
if the menu is gone, there is nothing to post-iterate.
A Play Console SIGSEGV in sl_write_avail on Android 1.22.2
(BuildId 46031fc2, via audio_driver_flush <- audio_driver_sample_batch
<- flycast retro_run) faults on the atomic buffered_blocks load at
context + 0x60 with a near-null context: the DRC helper reached
write_avail() while context_audio_data was NULL.
Two things allow that state to be observed:
- audio_driver_compute_rate_adjust() calls
current_audio->write_avail(context_audio_data) with no context
check; the writability helper earlier in the file already guards
its own write_avail() call against a NULL context
- audio_driver_deinit_internal() frees and nulls the context first
and only touches AUDIO_FLAG_ACTIVE much later (and conditionally),
so an observer that has passed the sample_batch gate can still see
ACTIVE|CONTROL while the context is already gone
Guard the DRC helper (degrade to no rate control for the batch,
matching the half_size<=0 degrade path), and clear
AUDIO_FLAG_ACTIVE|AUDIO_FLAG_CONTROL at the top of deinit before the
context and scratch buffers are released, so gate checks fail before
the state is half-torn.
This narrows the window rather than closing it - the gate check and
the driver calls are still not atomic against a concurrent teardown,
and a full fix is a lifetime lock owned by the threaded pipeline
work. The exact producer of the torn state on the crashing device
could not be pinned from one sample; the guard makes the observed
faulting path impossible regardless of producer.
Two lookups in the xmb_frame icon path index
xmb->horizontal_list userdata with an offset that can point past
the end of the list:
- the "Main Menu" playlists branch uses the entry's entry_idx as
the index while only checking that the horizontal list is
non-empty; if the horizontal list is smaller than the playlist
listing implies (deleted playlist, filtered Explore views, list
rebuilt), the index is stale and out of range
- both name-matching search loops (playlist manager icons and the
sorted-icon-order path) leave offset == horizontal_list.size when
no entry matches, indexing one element past the end
file_list_get_userdata_at_offset() performs no bounds checking, so
either case reads garbage as an xmb_node_t pointer and faults
dereferencing node->icon.
Observed as a Play Console SIGSEGV in xmb_frame via gl2_frame /
video_thread_loop on Android (1.22.2 / 2026-08-26, arm64); the
faulting instruction against BuildId 46031fc2 is the node->icon
load fed by the raw entry_idx lookup.
Clamp both lookups to offset < horizontal_list.size and fall back
to the default texture otherwise.
Track the collections a sync has created and skip the MKCOL for any the
walk has already been through, so the request count follows the number of
distinct directories rather than the number of files. A 143-file sync
over the directory layout in libretro/RetroArch#19475 issues 4 MKCOLs
instead of 357.
The list is per-sync and dropped in webdav_sync_begin, so a directory
removed on the server between syncs is created again. Lookups compare
exactly rather than through string_list_find_elem(), which folds case
where WebDAV does not. Task callbacks are retired on the main thread, so
the list needs no lock.
Covered by a harness driving the walk against a simulated server,
including out-of-order and interleaved completion, MKCOL failure, and 405
from a server that never advertised DAV; clean under ASan+UBSan+LSan and
TSan. C89 checked with gcc -std=c89 -pedantic; cross-compiled for Vita.
file_list_get_userdata_at_offset and file_list_get_actiondata_at_offset
checked the list pointer but not the index, so any caller with a stale
or computed index read item_file fields past the end of the allocation.
Every menu driver resolves nodes through these accessors with indices
derived from a different list than the one being indexed (selection
ordinals, entry_idx values, name-match loop cursors), and each such
site has had to carry its own bounds check; the ones that did not are
the ozone icon lookups fixed in the previous commit, one of which
crashed in the wild. An out-of-range index now resolves to NULL, the
same answer callers already take for a missing node.
The call-site guards stay: they are one compare, they document why the
index can be out of range at each site, and they keep the sites correct
against any other accessor. The header-icon comment that pointed at the
unchecked accessor is reworded to match.
ozone_draw_entries resolves per-entry icons against ozone->horizontal_list,
which is a snapshot built at init or on an explicit refresh, using indices
that come from the entry list being drawn - and the entry list is rebuilt
on every displaylist push. Three lookups did this without an upper bound,
and file_list_get_userdata_at_offset does not bounds-check:
- The depth == 2 Playlists view indexes with the entry's entry_idx, the
ordinal assigned by menu_displaylist_parse_playlists when the view was
pushed. When playlists appear or disappear between the snapshot and the
push (a directory scan finishing, content play creating an auto
playlist, a playlist deleted outside the manager), the tail ordinals
run past the snapshot. The guard only rejected an empty list.
- The same view's ozone_sort_after_truncate_playlist_name name-match loop
and the playlist manager's Ignore-Explore-Views loop both fall through
with offset == horizontal_list.size when nothing matches, reading one
entry past the end of the list.
- The FILE_TYPE_RPL_ENTRY console icon lookup indexes with
categories_selection_ptr - system_tab_end - 1 with no upper bound; the
title lookup for the very same computation already carries one.
The out-of-bounds item_file read itself usually yields garbage rather
than a fault; the garbage userdata is then dereferenced as an
ozone_node_t and crashes there. Observed in the wild as a SIGSEGV in
ozone_draw_entries under video_driver_cached_frame on the 1.22.2 Play
build (armeabi-v7a, Android 12). A missed match or an out-of-range
ordinal now resolves to no node, so the entry keeps its generic icon,
which is also the correct rendering for a playlist the sidebar does not
know about yet.
runloop_check_state() caches menu_st->driver_data in a local at
function entry. When the menu is alive with menu_pause_libretro
disabled, display_menu_libretro() runs a core frame mid-function;
if retro_run triggers a driver reinit (e.g.
RETRO_ENVIRONMENT_SET_SYSTEM_AV_INFO -> CMD_EVENT_REINIT), the menu
handle is freed and recreated, and the subsequent
menu->driver_ctx->set_texture dereference reads freed memory.
Observed as a Play Console SIGSEGV in runloop_check_state on
Android (1.22.2 / 2026-08-26, arm64); faulting instruction confirmed
against BuildId 46031fc2 to be the driver_ctx->set_texture load.
Re-fetch the handle after display_menu_libretro() before touching it.
CTR is the 3DS codename, so ctr_entropy_func reads as 3DS-specific now
that the Vita selects the same callback; platform_entropy_func says what
the hook is for. Give it internal linkage while here, since it is used
only in this translation unit.
Give the Vita a working mbedtls: seed CTR_DRBG from
sceKernelGetRandomNumber via ctr_entropy_func, the way the 3DS already
does, and set MBEDTLS_NO_PLATFORM_ENTROPY so the platform poll stays out
of the build. That macro needs upstream's guards in entropy.c and
entropy_poll.c, which the bundled copy is missing, so restore them.
net_sockets.c gets a self-contained sceNet implementation for the calls
the TLS layer makes -- init, free, send, recv, recv_timeout, blocking
mode and usleep -- using isagain() from net_compat.h, since sceNet
returns its error code rather than setting errno. The listen and connect
half stays out: ssl_socket_init only ever receives a connected socket.
MBEDTLS_TIMING_C is off for the same reason it is off on 3DS.
With HAVE_BUILTINMBEDTLS in Makefile.vita, cloud sync and any other
https consumer can reach servers that do not offer plaintext, which is
by now most of them.
Compile-verified against vitasdk (gcc 15.2.0, arm-vita-eabi): all 61
bundled mbedtls objects plus net_http, net_socket, net_compat and
webdav. Handshakes are untested on hardware.
Cloud sync opens with an OPTIONS request, so inspect that response for
the DAV header (RFC 4918 10.1) and for PUT in Allow (RFC 9110 5.6.1.2)
before doing any work. A server that answers OPTIONS but rejects PUT is
reported at once with a hint that WebDAV endpoints are usually https on a
dedicated host; a missing DAV header alone only warns, since not every
server is strict about advertising it.
MKCOL returning 405 counts as "collection exists" only once the server
has been seen to implement DAV, and an https URL in a build without TLS
now says so. Failure diagnostics are assembled into one buffer and
emitted in a single write, so concurrent transfers no longer splice their
log lines into each other.
Helpers covered by a standalone harness under ASan+UBSan+LSan and TSan,
including the header sets from libretro/RetroArch#19475; C89 checked with
gcc -std=c89 -pedantic.
The location a joypad driver reports at connect is kept alongside the
name, VID and PID in input_device_info_t, so that
input_autoconfigure_reconnect() can offer it to the profile scan and a
profile discriminating on 'input_phys' scores the affinity it would on a
hotplug. Disconnecting a port clears the location, so a port never
inherits the one belonging to whatever occupied it before.
Controller profiles are consumed when a device connects, so the
'Update Controller Profiles' download re-runs autoconfiguration for the
ports that already hold a device and leaves the drivers alone.
input_autoconfigure_reconnect() re-issues a connect from the identity
retained for a port, the same in-place path 'Save Controller Profile'
takes. The asset download keeps its full reinit, which it needs for the
menu textures and fonts it has just replaced.
Ports with no retained device are skipped. The physical location is not
kept per port, so a profile discriminating on 'input_phys' is matched on
name and VID/PID alone here.
With the threaded pipeline a pipeline pass runs on the audio thread,
and it reads two things the main thread changes without any
synchronisation. audio_driver_dsp_filter_free() frees the filter and
nulls audio_st->dsp, from CMD_EVENT_DSP_FILTER_INIT, which does not
reinitialise the driver - so changing or clearing the DSP filter in
the menu while a core runs frees a filter the audio thread may be
inside. The mixer is the same shape: audio_mixer_mix() walks the
global voices from the pass, while add, play, stop, remove and volume
mutate them from the main thread and from the mixer load tasks -
menu sounds, thumbnail video audio and the translation task all do
this while a core is running.
A state lock, held across a pass and by every one of those mutators,
covers both. It is uncontended on the frame-synchronous path, where
both sides are the same thread; on the threaded path a mutator waits
at most one pass, and a pass is a period of audio. It is never held
across a device write: the wait for room happens before it is taken.
Both races are read out of the code, not from a reproducer - a
stress thread that hammered the mutators against a running pass did
not start in the time available, so nothing here is a demonstration
that they fire in practice. What is verified is that the guard does
not deadlock or change behaviour: the threaded matrix, and both
pipelines with a DSP filter loaded, under ThreadSanitizer and
AddressSanitizer.
The three-valued version had an "Automatic" that consulted a list of
audio drivers in audio_driver.c, put there by whoever wrote the list -
me - from what could be measured on one machine. Nothing said what
earned a place on it, users on the drivers left off had no way to know
a list existed, and no other optional driver feature in the frontend
works that way: a driver implements the entry point or it does not.
The fallback for a driver without wait_writable() is the safety that
matters, and it is unchanged. A wait_writable() that turns out to
misbehave is a bug in that driver, reported and fixed like any other.
So: on, off, and a driver that cannot wake on the device keeps the
frame-synchronous path. Default stays off; removing the list is not
the moment to turn it on for everyone.
The enum for it went into msg_hash.h but the string never reached
intl/msg_hash_us.h - I reverted that file while sorting out an
unrelated macro and put back only the macro. msg_hash_to_str() has no
row for an enum it does not know and answers "null", which is what the
dropdown showed between OFF and ON.
menu_list_free() dispatches each driver's list_free hook, and those
reach gfx_animation_kill_by_tag(): directly in xmb_list_clear(), and
again through gfx_thumbnail_reset() when a node owns an icon.
gfx_thumbnail_reset() also unloads the node's texture.
RARCH_MENU_CTL_DEINIT freed the animation and display state ahead of all
that, so those calls read a list that had already gone. Nothing broke
only because RBUF_FREE() leaves the pointer NULL and RBUF_LEN() reads
NULL as zero, and the scan found nothing to kill -- the tween survived
teardown by accident rather than by having been cancelled. The same
ordering ran gfx_display_free() before the texture unloads.
Moving the two calls after the lists rather than moving the lists
earlier keeps menu_cbs_pool_deinit() where it has to be: the driver's
free(), which runs first, releases cbs blocks of its own through
file_list_deinitialize() on its horizontal list.
samples/menu/teardown drives RARCH_MENU_CTL_DEINIT and a reinit after it
under ASan; rgui, xmb, ozone and glui all pass.
menu_list_new() malloc()s each menu_stack and selection_buf entry and
then names the fields it sets. userdata_free was added to file_list_t
without being added here, so every menu list carried whatever malloc()
returned in it, and file_list_free_userdata() called through that on the
first teardown to reach an entry with userdata:
SIGSEGV in file_list_free_userdata libretro-common/lists/file_list.c
#1 menu_driver_list_free menu/menu_driver.c
#2 menu_list_free_list menu/menu_driver.c
#3 menu_list_free menu/menu_driver.c
#4 menu_driver_ctl RARCH_MENU_CTL_DEINIT
xmb and ozone hid it: their list_insert assigns the hook before any
teardown can run. materialui never assigns one, so it went to the
garbage. An arbitrary indirect call on menu teardown, reached by
switching menu driver or by quitting.
The two loops now say userdata_free explicitly rather than being zeroed
wholesale, so the cost stays the same handful of stores on every target
rather than depending on what a given compiler does with a constant-size
memset. Anything added to file_list_t after this has to be added to both
loops; the comment says so.
libretro_free_system_info() free()d library_name, library_version and
valid_extensions, and no path ever puts a heap pointer in any of them.
libretro_get_system_info() copies into the fixed buffers inside
runloop_state and assigns their addresses; the core's own
retro_get_system_info() leaves its static literals, as do the
MSG_UNKNOWN and "v0" fallbacks applied after it. Every one of those is
a free() of something malloc() never returned:
AddressSanitizer: SEGV in __asan::Allocator::Deallocate
#5 libretro_free_system_info retroarch.c:6828
#6 menu_driver_ctl menu/menu_driver.c:7039
RARCH_MENU_CTL_DEINIT runs that block on every teardown where
MENU_ST_FLAG_DATA_OWN is clear, and HAVE_DYNAMIC builds compile out the
frontend_driver_has_fork() guard in front of it. Clearing the struct is
the whole of what the function has to do; both callers already memset()
it immediately afterwards, which is what kept the missing ownership from
being obvious.
Also replaces the memcmp() string comparisons that read a fixed count
regardless of how long their operand is. materialui_list_insert() ran
memcmp(label, "null", 4) against every entry's label, so an empty one --
the shared single-byte string every directory listing labels its rows
with -- was read three bytes past its end:
AddressSanitizer: global-buffer-overflow
#3 materialui_list_insert menu/drivers/materialui.c:12207
menu_driver.c compared entry labels against "resume_content" and
"state_slot" across sizeof() of the literal, reading 15 bytes of
whatever the label was, and did the same for "..."; menu_setting.c did
it for "true" and "false". All become string_is_equal() or
string_starts_with_size(), which stop at the terminator.
Found by a harness that drives RARCH_MENU_CTL_DEINIT under ASan.
The sink was added so a pull-model backend could have a pipeline pass
deliver into its callback's buffer instead of through the driver's
write(). No such backend was written, every one of the eight callers
passes NULL, and the pull path is not something I can measure a case
for: it would remove a thread and a copy per period on the backends
that have their own callback, but the device's own buffering is
unchanged, so it moves no latency. Rather than leave scaffolding for
a design nobody has committed to, take it out; the same few lines can
come back with a caller that needs them.
pipe_underruns and pipe_primed counted passes the consumer had to pad
with silence, in a design where the consumer wrote a fixed period
whether or not the ring could fill it. The consumer has not worked
that way since it started writing what the core produced and waiting
on the device for room; the fields have only been reset ever since,
never read or incremented, and the comment above pipe_pass_int16s
still described the padding. Removed.
The literal conversion left 39 files calling strlcpy_lit() without
including the header that declares it. They built anyway: the files were
already calling strlcpy() without including compat/strl.h either, and on
glibc 2.38 and later that resolves to the C library's own strlcpy(), so
nothing ever forced the include. strlcpy_lit() has no such fallback --
it is an implicit declaration that compiles and then fails to link.
RetroArch itself did not notice, because its translation units pull the
header in through something else. The standalone samples do not, and
samples/tasks/database is where it surfaced:
scan_begin_budget_test.c: undefined reference to `strlcpy_lit'
Adding the include to each of them fixes that and removes the reliance
on the accidental libc fallback for strlcpy() at the same time.
Checked by building every sample rather than the one that failed: 42
under samples/, 38 under libretro-common/samples and libretro-db/samples,
with libxrandr, libdrm and libvulkan headers installed so the X11, KMS
and Vulkan ones actually compile instead of being skipped. All build
except samples/gfx/font_lifecycle, which fails identically on a clean
tree (undefined reference to video_driver_thread_wrapper_active in its
HAVE_THREADS variant) and is unrelated.
The sample binaries were run as their workflows run them, including
"make fixtures" for gfx_thumbnail_preview and a long enough timeout for
the 1000000-iteration SPSC stress in gfx_thumbnail_status_atomic: all
pass. samples/tasks/database builds, passes "test -x database_task" and
exits 1 with its usage line, which is what the smoke step expects.
The setting was a bool that meant "use the threaded pipeline wherever
a driver can", which is right for the drivers whose wait_writable()
has been run against a real device and a guess everywhere else. It is
now three-valued and defaults to Automatic: on for the drivers on the
verified list, off elsewhere, with a log line naming a driver that
could run it and has not been measured. On enables it on every driver
that has a wait_writable() at all, which is how the rest earn their
place on the list; Off is off.
The list starts as pulse, sdl2 and wasapi - measured against a paced
device, inline and threaded, at more than one Audio Latency. ALSA is
deliberately not on it: its wait_writable() has only been exercised
against the null device, which does not pace, so nothing about its
timing has been observed. It belongs there as soon as someone runs it
on hardware, and the same goes for the sixteen drivers whose
wait_writable() has never been compiled.
The key keeps its name, and a stale "true" or "false" in a config
reads as 0, which is Off. That is the wrong sense for "true", but the
setting has never been on by default and the value only survives from
a config written since it was added.
An empty label is not an allocation of its own: file_list.c hands every
one of them the same shared buffer, since a directory listing labels
every entry "" and each used to cost a malloc() for a single NUL. That
made free() on a label unsafe, and four sites outside file_list.c were
still doing it -- xmb, ozone and materialui all relabel the menu stack
top by free()ing the old label and strdup()ing a new one.
Reaching a shared empty label that way is a free() of a static:
AddressSanitizer: attempting free on address which was not malloc()-ed
SUMMARY: AddressSanitizer: bad-free
No push currently gives a stack entry an empty label, so this was not
reachable today; it was one 'info_label = ""' away from being so, with
nothing at the call site to warn about it.
file_list_free_label() releases a label and clears the slot, alongside
the file_list_free_userdata() and file_list_free_actiondata() that
already exist for the same reason. The two sites that free and then
immediately strdup are now file_list_set_label_at_offset(), which did
both correctly all along; the two that free ahead of a switch use the
new call.
Four cases added to test/lists/test_file_list.c: releasing a shared
empty label, releasing and re-releasing an owned one, an out-of-range
index and a NULL list -- the menu computes the index as stack_size - 1,
which is SIZE_MAX on an empty stack -- and relabelling away from the
shared empty string and back. ASan+UBSan+LSan and TSan both 31/31, and
a bare free() of a shared label is a bad-free under ASan.
strlcpy() begins with strlen() of its source. It lives in another
translation unit, so with no LTO that scan happens at runtime even when
the source is a literal whose length the compiler already measured, and
the call itself cannot be folded away either.
strlcpy_lit() passes the length over as sizeof(). When the destination
size is also a constant -- sizeof(buf), the usual case -- nothing is
left but the stores memcpy() compiles to:
strlcpy "core_options" sizeof(buf) 7.01 ns
strlcpy_lit "core_options" sizeof(buf) 0.42 ns
strlcpy 41-char literal sizeof(buf) 7.74 ns
strlcpy_lit 41-char literal sizeof(buf) 0.71 ns
strlcpy "core_options" runtime len 7.11 ns
strlcpy_lit "core_options" runtime len 0.70 ns
The destination bound is kept. When the literal does not fit it defers
to strlcpy(), so the copy truncates as before rather than overrunning;
a form that took only the literal would have turned every one of these
sites into an overflow waiting for a small buffer. The literal is
concatenated with "" so a non-literal source is a compile error rather
than a sizeof() that silently measures a pointer.
818 sites across 104 files were converted by parsing arguments rather
than by pattern-matching text: only a three-argument call whose second
argument is a bare string literal is rewritten, so a macro that happens
to expand to a literal is left alone. Line endings are preserved, so
the CRLF files carry only their own changed lines.
test/compat/test_strl.c compares the two against each other over every
destination size from 0 past the literal's length, for literals that
are empty, single-character, path-shaped, adjacently concatenated and
escape-bearing, checking the return, the terminator and the whole
buffer so an overrun past the terminator is caught. ASan+UBSan+LSan and
TSan both 4/4. The full menu_displaylist_build_list walk over every
displaylist type is byte-identical to before the change
(md5 a2a081bcac09cdc8d72d67efd14ac40e).
Every publish into the threaded pipeline's ring took the lock and
signalled the consumer. For a core that batches per scanline that is
hundreds of lock-and-signal pairs a frame for a consumer that is
already awake, and if the consumer outranks the main thread it
preempts on each one and runs a scanline-sized pass - measured at
59896 passes over 300 frames. The signal now comes once per frame,
from the frame end, and from the two per-frame producers that have no
frame end of their own: menu audio signals itself, and a reversed
rewind frame is published before the frame end that follows it. The
same core now signals 300 times in 300 frames and the consumer makes
one pass per frame, at the same held fill.
audio_thread_priority (default off, Audio > Synchronization, applies
on driver reinit) asks the OS to schedule the audio thread ahead of
ordinary threads, through a new sthread_raise_current_priority(): the
time-critical class on Windows, the audio band on Android, real-time
round-robin at a middling priority where POSIX and the process's
limits allow it. Raised from inside the thread after it exists and
never fatal: a refusal is logged and the default kept. The existing
sthread_create_with_priority() was not usable for this - it sets
scheduling attributes Linux ignores without explicit inheritance, and
a refusal there fails thread creation outright. With the resampler,
DSP and device write on this thread, its scheduling jitter is what
turns into underruns; this is the knob for systems where Audio Latency
is bounded by that rather than by the device.
strlcpy() reports the length of its source, not what it wrote, so an
accumulator built as '_len += strlcpy(s + _len, x, len - _len)' passes
len as soon as one part does not fit. The 'len - _len' handed to the
next call then underflows to a huge size_t, and that call writes at
's + _len' -- already outside the buffer -- with no effective bound.
Four functions did this. fill_pathname_join_special_ext() writes 50
bytes into a 32-byte buffer given a 45-character basename, which is an
ordinary save-state or SRM path built from a long ROM name:
char s[32];
fill_pathname_join_special_ext(s, "/a", "b",
"a_very_long_basename_that_does_not_fit_at_all", ".srm", 32);
fill_pathname_dir() chains two appends with the same flaw.
fill_pathname_join_delim() guards with 'if (len - _len < 2)', which is
the subtraction that wraps, so the guard passes and s[_len++] writes
past the end. fill_pathname_parent_dir() leaves _len past the end after
a truncated copy and hands it to path_parent_dir(), which scans back
from outside the buffer.
path_strlcat() appends and returns the new length, clamped on the way
in so the size argument stays sane and on the way out so the
accumulator stays inside the buffer for whatever the caller does next.
fill_pathname_join() already clamped for this reason and says so in a
comment; these four sat beside it without one.
Checked over 1320 combinations of buffer size against component length,
in exactly-sized heap buffers so a redzone sits immediately after the
last valid byte -- a stack array lets a large overshoot land outside
the instrumented frame and go unreported, which is how three of these
stayed hidden. 336 of the 1320 fault before this change and none after,
and every result is NUL-terminated.
file_list_append() is handed "" constantly -- a directory listing labels
every one of its entries that way, and 99 menu_entries_append() call
sites pass it literally -- and each one cost a malloc() for a lone NUL
byte, which the allocator rounds up to a whole chunk anyway. Every empty
path, label and alt now points at one static buffer, and the release
path skips it.
The entry still holds a valid empty string rather than NULL, so no
reader changes. Storing NULL would have been cheaper still, but it
changes what file_list_get_label_at_offset() returns and would put a
NULL through the strlen() of an entry label in xmb_list_get_entry() and
its ozone counterpart; there are 35 direct reads of these fields to
audit before that is safe, and the failure mode is a crash rather than a
regression.
Nothing writes to an entry's path, label or alt in place -- they are
replaced wholesale by the two setters -- so one buffer is safe to share
across every entry of every list.
Eight cases added to test/lists/test_file_list.c: an empty label reading
back as "" and not NULL, empties sharing one buffer, NULL staying NULL,
the buffer surviving teardown of one list while another still points at
it, surviving clear and pop, both setters crossing between empty and
non-empty in each direction, and file_list_insert() sharing too since it
goes through init_item_file() rather than the append path.
ASan+UBSan+LSan and TSan both 27/27. Dropping the guard from
file_list_strfree() fails them with a free of a non-malloc()ed address.
Swept over the full menu_displaylist_build_list() walk of every
displaylist type via RETROARCH_DISPLAYLIST_DUMP: no reports, leak
profile unchanged at 10402 bytes in 448 allocations, and the displaylist
fingerprint is byte-identical to the pre-series baseline
(md5 a2a081bcac09cdc8d72d67efd14ac40e).
Every row of a list is inserted with the same fullpath -- it is the path
of the menu stack top, one string for the whole list -- and both xmb and
ozone strdup()ed it per row. A playlist paid a malloc() and a full path
copy tens of thousands of times for a string neither driver ever writes
to; node->fullpath is read in exactly one place per driver and never
mutated.
menu_str_ref()/menu_str_unref() hand out a reference-counted shared
copy. The reference is a pointer to the characters rather than to the
header, so node->fullpath stays a plain char* and every read site is
untouched; only the four allocate-and-release sites change. The
consequence is that a fullpath must be released with menu_str_unref()
and never with free().
The one-entry cache compares content rather than the source pointer. The
caller's string lives in a list entry that can be freed and another
allocated at the same address between builds, which a pointer compare
takes for a hit and hands back the previous list's path. A strcmp costs
a fraction of the malloc() and copy it avoids.
Lives in its own translation unit rather than in menu_driver.c so that
it can be tested without linking the frontend.
Measured on a 100k-entry list build, one variant per process: ~116 ->
~89 ns/entry, 11.6 -> 8.9 ms.
test/menu/test_menu_str.c covers sharing across distinct buffers with
equal content, the same address with different content, release in
forward and reverse order, NULL and empty input, repeated flushes, and
alternating strings that defeat the cache. ASan+UBSan+LSan and TSan both
10/10. Re-keying the cache on the source pointer fails two of them,
including the stale-cache case, where the second caller is handed the
first string.
The RPX resolves coreinit symbols through the import table in
wiiu/system/imports.h, and OSInitEvent, OSSignalEvent and OSWaitEvent
were declared in the headers but not imported, so the audio driver's
frame-event wait failed to link. Added next to the condition and
spinlock imports it sits beside.
Also drop the menu_pause_libretro local CMD_EVENT_MENU_PAUSE_LIBRETRO
computed for the microphone alone, on builds without a microphone
driver, which the Wii U compiler flagged as unused.
item_file::userdata was always released with a bare free(). That suits
every file_list_t outside the menu, but the menu drivers' userdata is an
xmb_node_t or an ozone_node_t, each owning further allocations, so the
free() released the node and leaked the strings inside it.
The drivers worked around this by walking a list themselves --
xmb_free_list_nodes() and ozone_free_list_nodes() -- before handing it
to file_list_deinitialize(). That pairing has to be remembered at every
site that can destroy a list, and any path reaching
file_list_free_userdata() without it leaked; file_list_pop() on the menu
stack is one such path.
file_list_t gains a userdata_free hook on the same terms as the existing
actiondata_free: NULL keeps the plain free(), which every non-menu list
gets by being zeroed rather than by saying so. xmb and ozone install
theirs where a node is stored into a list rather than where the list is
built, so a list that holds a node knows how to take one apart however
it is destroyed. The existing free_list_nodes() walks stay -- they also
kill animation tweens tagged by the list -- and are harmless now that
they NULL each slot as they go.
materialui keeps the free() fallback: its node is a plain block, and the
thumbnail reset that has to precede release is driven per-list with a
gfx_thumbnail_cancel_pending_requests() in front of it.
Eight cases added to test/lists/test_file_list.c alongside the
actiondata ones: hook dispatch, the no-hook fallback, NULL userdata and
NULL list, deinitialize and pop, the two hooks not seeing each other's
pointers, and the hook surviving the realloc in file_list_reserve().
ASan+UBSan+LSan and TSan both 19/19. Reverting only the dispatch fails 5
of them.
The player is created with SLAndroidDataFormat_PCM_EX in 32-bit float
representation first, and use_float() reports it, so the frontend
hands float straight to the driver and skips its float-to-int16 pass.
A runtime that refuses that format - an older device, or a
libOpenSLES built without it - fails player creation, and the
existing 16-bit SLDataFormat_PCM player is created instead, so the
driver is exactly what it was on anything that cannot take float.
Block sizes are now computed in frames and the byte size follows the
format, so a block is the same duration either way and the buffer
count the latency setting produces does not change.
The float path is gated on ANDROID and __ANDROID_API__ >= 21 at
compile time on top of the runtime fallback; a build for any other
OpenSL target compiles the file it always did. Compiled here against
the AOSP OpenSL ES headers with the API level at 21, 19 and unset,
under gcc, the C89 lane and clang -Werror, with no new diagnostics;
the headers themselves carry a few pedantic ones. Not run: there is no
Android runtime here. The three includes the file relied on griffin
for are added.
Six drivers labelled their use_float() a stub or a TODO. For five the
answer is the hardware's or the service's: audio(4) encodings are
integer PCM and companded, sceAudio is 16-bit, audout refuses anything
but PCM_INT16, libxenon's sound submit is 16-bit big-endian, and
libroar's stream is opened as 16-bit ROAR_CODEC_PCM_S with no float
codec to open instead. The comments now say that, so nobody spends
time on them again. OpenSL ES is the one that could: API level 21
added float through SLAndroidDataFormat_PCM_EX. Its comment says what
taking that would involve and why it was not done here.
The Audio Interface is fed signed 16-bit big-endian stereo PCM by DMA
and has no format select - AI_CONTROL carries play state, rate,
interrupt bits and the DMA flag only - and libogc's aesnd and asnd
voices are 8- and 16-bit as well. The frontend converts float to int16
before write() for a driver that says false, so returning false is
what a float-capable frontend needs from this hardware. Say so.
The last drivers with a device wake to give, each now exposing it so
the threaded pipeline runs on them instead of falling back.
sdl3: loops the driver's existing token wait on the stream callback.
ctr_dsp: registers an ndsp frame callback that signals a LightEvent,
and waits on it instead of sleeping and polling the sample position;
the wait runs on the audio thread so it does not pump the applet loop
as the write's own wait does. gx: the AI DMA callback signals a
thread queue, the same pattern the Wii video driver uses for the
retrace callback; the write's busy-spin is untouched. wiiu: the AX
frame callback signals an OSEvent every 3 ms. ps3: waits on the
condition the output thread already signals per block, and gains a
real write_avail() and buffer_size(), which it never had, so rate
control can run there in either mode. switch_audio: buffer
acquisition - the wait for a played-out buffer that also takes
ownership of it - is factored out of write() so the wait can acquire
and the following write fills. switch_libnx_audren: waits a renderer
frame at a time for a wave buffer with room.
ctr_csnd stays inline: its play position is estimated from CPU ticks,
there is no hardware position or callback to wake on. So do audioio,
openal, roar, alsa_qsa, audioworklet, rwebaudio, ps2 and xenon360, for
the reasons given before.
None of the seven compile on this host. Every identifier is checked
against its driver and platform headers in the tree, but their first
build is on their platforms' CI.
menu_entries_append() and menu_entries_prepend() took one malloc() per
appended entry, and menu_entries_cbs_free() one free(), on top of the
two strdup()s file_list_append() already does. A cbs is fixed-size, so
both go through a mempool_t instead: allocation becomes a free-list pop
and a rebuild of a list of the same size reaches libc not at all.
The pool grows lazily and is released in RARCH_MENU_CTL_DEINIT, after
menu_list_free() has returned every entry to it.
A pooled block cannot be released with the bare free() that
file_list_free_actiondata() falls back to when a list has no
actiondata_free hook, so the two functions that allocate a cbs now
install the destructor in the same statement. Doing it there rather
than at each list's construction is what makes it impossible to miss:
menu_displaylist_build_list() is handed a stack-local file_list_t by
menu_displaylist_validation_dump(), which is memset() to zero and so
would otherwise reach that free().
Measured on a 100k-entry list build, one variant per process so neither
inherits a heap the other warmed: 755 -> 209 ns/entry cold (75.6 ->
20.9 ms), and ~140 -> ~50 ns/entry in steady state.
Swept with ASan+UBSan+LSan over the full menu_displaylist_build_list
walk of every displaylist type, both passes, via
RETROARCH_DISPLAYLIST_DUMP: no reports, and the leak profile is
unchanged from baseline at 28 leaks / 10402 bytes, all pre-existing.
The displaylist fingerprint is byte-identical to baseline
(md5 a2a081bcac09cdc8d72d67efd14ac40e, 3977 lines).
Serves one block size out of chunks taken from malloc() in bulk, with
released blocks going on an intrusive free list. A caller that builds
and tears down N same-sized objects pays a logarithmic number of calls
into libc instead of N, and a rebuild of the same size touches libc not
at all.
Blocks keep their address while held, so a container that relocates the
pointer -- qsort() in file_list_sort_on_alt(), memmove() in
file_list_insert() -- moves the pointer and never the block.
Covered by test/memory/test_mempool.c: block distinctness under a
whole-block write, alignment for an unaligned block size, free-list
reuse, a rebuild taking no new chunk, deinit with blocks outstanding,
and interleaved churn across chunk boundaries. ASan+UBSan+LSan and TSan
both clean, 10/10.
Opening the menu with the core paused used to stop the audio driver
and closing it started the driver again, unless menu sounds were on,
in which case audio stayed up and the runloop fed the device a frame
of silence per menu frame for the sounds to mix into. That second
model is the only one now. Sound in the menu is ordinary - menu
sounds, the mixer, animated thumbnails with audio - and every one of
those wants a running stream; stopping and starting the device around
the toggle also cost a driver-dependent restart on every menu visit
for nothing, and left the device in a state that is not the one it
plays in.
The menu toggle and CMD_EVENT_MENU_PAUSE_LIBRETRO no longer touch
audio at all; only the microphone is stopped and started. The runloop
feeds silence whenever the core is not running behind the menu,
regardless of the menu-sound setting, so the stream never starves and
rate control keeps its footing across the toggle.
CMD_EVENT_AUDIO_STOP and CMD_EVENT_AUDIO_START lose the gate that
made them consult the menu's pause and sound settings. That gate
existed for the menu toggle, and it also disabled the commands for
callers that mean them: the 3DS sleep and wake hooks stopped stopping
audio whenever pause-on-menu was off.
Closes#12892. The issue asked for the 3DS and Vita to be tested with
the device left running behind the menu; that has not been done here.
The setting's sublabel string is generated from its descriptor row,
but the label-to-sublabel table in menu_cbs_sublabel.c is still kept
by hand and had no entry for it, so the menu showed the setting with
no description. Added.
The audio thread read runloop_state.flags, menu state and a setting
directly, in audio_driver_callback(), the threaded pipeline's consumer
and the accumulator flush a core's audio callback drives. Those words
are written by the main thread without synchronisation, and making
them atomic would touch some three hundred sites for the sake of six
bits. The main thread now publishes those bits - paused, slow motion,
fast-forward, menu alive, menu pauses the core, netplay allows pause -
into an atomic snapshot at every frame end and on every start and
stop, and the audio thread reads only that. A frame of staleness in
"paused" is nothing; a racing read of the runloop's flag word was the
last data race ThreadSanitizer reported on the audio thread, and it
now reports none, on PulseAudio, ALSA and SDL, for the threaded
pipeline and for callback cores alike.
pulse_start() and pulse_stop() never released the operation
pa_stream_cork() returns, one per stop or start; they do now.
ra_asio_wait_writable() slept a millisecond on its no-threads branch,
copied from the write above it. Nothing calls wait_writable() without
threads - the threaded pipeline is its only caller - so that branch
now reports that no wait is possible instead of spinning.
The threaded pipeline setting is no longer advanced, and its text says
what the measurements support: the same latency as the frame-
synchronous path at any Audio Latency, rate control at the device's
own pace, the resampler out of the frame budget, and no effect on
drivers that cannot wake on the device.
Every driver whose write() already sleeps on something the device
signals now exposes that as wait_writable(), so the threaded pipeline
runs on it instead of falling back to the inline path. Each one
mirrors its own write()'s wait exactly.
Compiled here: sdl (the speaker thread's condition), jack (the process
callback's condition), pipewire (the thread loop the process callback
signals), oss (select() on the device, SNDCTL_DSP_GETOSPACE for the
room), tinyalsa (pcm_wait() on the pcm fd). Written against the
drivers' own fields and wake mechanisms, not compiled in this tree, so
their first build is on their platforms' CI: rsound, opensl,
psp_audio, ctr_dsp_thread_audio, asio, dsound, xaudio,
switch_thread_audio and switch_libnx_audren_thread_audio.
SDL also gains a real write_avail() and buffer_size(): it reported 0
and had no size, so rate control could never run on it in either mode.
It now can, which is a change for inline SDL users too.
Left on the inline fallback, on purpose: openal (polls and sleeps),
roar and alsa_qsa (blocking library calls only), audioio (its
write_avail() reports the whole buffer, so there is nothing honest to
wait for), audioworklet and rwebaudio (no threads), sdl3 (push queue),
ctr_csnd, ctr_dsp, gx, ps2, ps3, wiiu and xenon360 (console DMA or
blocking libraries), and the two non-threaded Switch drivers, whose
wait is entangled with taking ownership of the next audout buffer
across two ABIs.
Two wrapper bugs surfaced on SDL. The thread loop only called the
wrapped driver's start() when coming out of a stop, never on the
initial start, so a driver whose init() leaves the device paused
stayed paused - callback cores on SDL have been silent. It now starts
the driver once it is released. And every stop, join and the wrapper's
alive() waited out the consumer's full data timeout because the wake
was a pulse the consumer could miss; audio_driver_pipeline_wake() is
now sticky, and the wrapper raises it before it parks or joins.
is_shutdown is read and written under the wrapper's lock.
On SDL's real-time dummy device the threaded pipeline holds the same
fill as inline at 64 ms and more at 24 ms, at identical wall time.
memmove lands @dir in @s whether or not the two point at the same
buffer, so the copy needs no aliasing test of its own and @s is
always written before the separator test inspects it. Partial
overlap is covered as well, and clamping the copied length to what
fits keeps @s + @_len inside the buffer when @dir does not.
Also fixes a heap overflow when @len is 1 and @dir is not empty.
An ASan sweep over 10 dirs x 8 paths x buffer lengths 1..48, plus
fully aliased and partially overlapping calls, aborts before this
change and is clean after; UBSan, LSan and TSan clean, single- and
multi-threaded. Output strings are identical in every case, and the
return value differs only for @len == 1, where it no longer points
past the end of the buffer.
rstrtod_parse sets the accumulators alongside the sign, the valid
gate and the special tag, so the struct is fully defined on every
path a caller can reach and a compiler that cannot follow the gate
through the flattened parse has nothing left to flag. The cost is
three stores at function entry, outside the digit loop, dead on the
digit path.
200000 randomised inputs give bit-identical doubles, floats and end
offsets against the previous build, and a fixed corpus covering
subnormals, 2.2250738585072011e-308, 9007199254740993, overflow,
inf/nan and partial parses still matches libc strtod exactly.
ASan+UBSan+LSan and TSan clean.
The unused declarations in command_event's CMD_EVENT_CORE_DEINIT
block, in task_overlay_deferred_loading and in
ozone_compute_entries_position are gone, and hw_render_context_type
tests its driver names with string_is_equal so it carries no length
of its own. The 'settings' scope in
menu_displaylist_parse_core_manager_list, the index budget helper in
task_database.c and the cpu_proc_rank helpers in features_cpu.c each
sit under the conditions their only readers do, so a build without
those features declares nothing it cannot reach.
Verified on a Linux target configured with OpenGL, Vulkan and the
D3D backends off, with -Wall -Wextra and no -Wno-unused-variable,
both with HAVE_LIBRETRODB, HAVE_IMAGEVIEWER and HAVE_AUDIOMIXER on
and with all three off.
Each backend already had the device-driven wakeup the threaded
pipeline paces on; this exposes it through wait_writable() so the
setting works on them instead of falling back to the inline path.
ALSA sleeps in snd_pcm_wait(), which wakes at period granularity, and
asks for no more than one period per call, the write's own loop taking
the rest as it frees; -EPIPE/-ESTRPIPE/-EINTR go through
snd_pcm_recover() as the write does. A prepared stream reports its
whole buffer free and returns at once, which is what lets the first
writes reach the start threshold.
WASAPI sleeps on the engine's event until the fifo in front of the
engine has room, and when it has none moves a period into the engine
the same way wasapi_write() does when it finds the fifo full: the whole
fifo in exclusive mode, what the padding allows in shared mode. What it
returns is the free space write_avail() reports and may be less than
asked for when the fifo is only partly free; the write then blocks at
most one period for the remainder, after the fill rate control needs
has already been read.
CoreAudio, both the legacy slock/scond/fifo path and the modern
ring/GCD path, sleeps on what the render callback signals after every
pull until the requested amount fits, capped at half the buffer so the
wait always ends, and returns 0 once the unit has stopped - paused, or
interrupted, which the modern path detects the way coreaudio_write()
does.
ALSA was run on the null device on Linux: correct, though that device
does not pace, so its latency is not measured here. WASAPI and CoreAudio
are written against the drivers' own state and wake mechanisms and are
not compiled in this tree; they need a build and a measurement on their
platforms.
audio_driver_deinit_internal() freed the pipeline's lock and the two
conditions unconditionally; slock_free() and scond_free() do not exist
without HAVE_THREADS, so builds without threads failed to link. Every
other piece of the pipeline was already guarded.
The consumer no longer sleeps a millisecond and retries; there is no
timer anywhere in the pipeline now. It has two waits, each on the
thing that can actually make progress: a condition the producer
signals after every publish, for "the ring has data", and a new
optional driver entry point, wait_writable(len), for "the device has
room for this write". A chunk is written as soon as both hold, sized
to what the core produced and never more than half the device buffer,
so the write inside audio_driver_flush() returns at once and the fill
rate control reads before it is the device's real fill. That is the
inline path's own behaviour, just off the main thread, and it measures
that way: at the same audio_latency the device fill it holds matches
inline within noise at 16 and 24 ms and is 2-4 ms lower at the 64 ms
default, for both core callback styles.
Three shapes did not get there and are worth recording. A blocking
device write as the pacer pins the device full and adds its whole
buffer to the latency. Polling write_avail() for room spins whenever
any room exists and deadlocks against PulseAudio's prebuf. Waking on
the server's request and topping it up is writing to the brim by
another route. What works is writing what the core made, when it made
it, and letting rate control steer the device.
PulseAudio implements wait_writable() by sleeping on its mainloop for
the next write callback until minreq bytes fit; minreq rather than the
full chunk, because before the stream has started playing only the
write's own partial-write loop can fill prebuf. A driver without
wait_writable() cannot host the threaded pipeline and keeps the inline
path, logged once at init, so no backend can be made worse by a path
it never enters. The producer's throttle wait for audio_sync without
vsync is a condition the consumer signals after every pass, checked
against a generation counter under the lock so a pass cannot be
missed; the ring itself stays lock-free.
Also: pipe_threaded was never cleared at deinit; the ring-fill rate
controller from the interim design is gone, rate control measures the
device in both modes; and the wrapper's wait_writable() ends its
thread when the device reports itself gone, as a failed write does.
The main thread sets and clears AUDIO_FLAG_* bits at will - mute,
suspend for run-ahead, non-blocking state, started - while the audio
thread reads the word inside audio_driver_flush(), the threaded
pipeline's consumer and a core's audio callback. Those reads were plain
loads against plain read-modify-writes. Every access now goes through
AUDIO_FLAGS_GET (acquire load), AUDIO_FLAGS_SET and AUDIO_FLAGS_CLEAR
(acq_rel fetch-or / fetch-and) on a retro_atomic_int_t, so a reader
always sees a whole, current word and a writer's earlier stores are
visible after a read that observes its flag. Mechanical: every
`|=`, `&= ~` and `& MASK` on the word is rewritten and no logic
changes. Under ThreadSanitizer the threaded pipeline now reports no
race on any audio driver state.
ffmpeg_free() released the AVFormatContext with a bare av_free() of
the url and the struct, which leaves behind everything the context
owns: the streams avformat_new_stream() created, their codec
parameters, the metadata dictionary and the muxer's private data.
The init-time comment already assumed avformat_free_context() did the
teardown. It does now, and with it the driver's harness run leaves
LeakSanitizer with nothing to report.
encode_audio() allocated an AVFrame for every audio block and then, in
av_frame_get_buffer(), a sample buffer for it that was never used: the
frame's data pointers were immediately repointed at the handle's own
sample buffer by avcodec_fill_audio_frame(), and the whole thing was
freed after the packet loop. The frame now lives on the audio state
with its format and channel layout set once at init, and each block
only sets its sample count and pts and wraps the handle's buffer; the
dry flush sends NULL as before and no longer touches it.
The 8 MiB video and 16 KiB audio "outbuf"s are gone with it. They were
preset on the packet before every encode as if the packet had to own
its output, which the send/receive API stopped needing when this file
moved to it - avcodec_receive_packet() supplies the packet's own
buffer and av_packet_unref() releases it - and nothing ever freed
them, so every recording session leaked the pair. A harness driving
the driver through 120 video frames and 95 audio blocks shows those
8,404,992 bytes gone from the leak report and the decoded audio and
video streams frame-hash identical to before.
Starting an animated thumbnail made four allocations - two job
structs and two frame buffers - and stopping it walked them back
individually. The pair is created and released together and the frames
are the same size, so they now come out of one zeroed block: job A at
its start, job B one cache line in (the worker and main threads pass
the two back and forth, so they must not share a line), then the two
frames on 64-byte boundaries. anim_job addresses the block; anim_job2
and both frame pointers are views, and close pulls both jobs off the
worker queue before the single free. The worker only ever sees the
frame pointers, so its side is untouched. The animation sample test
passes under ASan/UBSan with leak checking and under TSan.
udev_joypad_poll() walked all MAX_USERS slots every frame reading fd
out of each, and a slot is 2.5 KiB, so the fifteen empty slots on a
typical machine cost a cache line apiece before any pad was read. The
open slots are now tracked in one bitmask that udev_pad_set_fd() keeps
in step with fd, and the poll walks only those. Within a slot the
fields the poll and the per-query button and axis reads touch - fd,
the button bitmask, the hats, the axes and the trigger-range flags -
were spread from offset 776 to offset 2300 around the absinfo table,
the keycode maps and the two name strings; they now lead the struct
in its first two cache lines, absinfo follows for the axis events, and
the hotplug-time and sensor fields trail. Behaviour is unchanged.
gl_glsl_set_coords() stages the bound attribute streams into one
contiguous block before handing them to the VBO cache. Draws of up to
four vertices use a stack buffer, but anything larger - every font run
and the menu ribbon's 8064 vertices - malloc()ed the block and freed
it again on each call, so a menu frame paid a heap round trip per line
of text plus a 64 KiB one for the ribbon. The staging now lives on
glsl_shader_data_t, grown to the largest draw seen and released with
the shader; the streams staged, their order and the cache compare that
follows are unchanged.
The slang display path streamed all three attribute arrays through the
scratch VBOs for every menu draw. For the ribbon pipelines that meant
re-uploading the 64x64 grid's 64 KiB of positions every frame although
the array never changes, and, worse, uploading 8064 vertices' worth of
texture coordinates and colours out of the four-vertex default arrays
the display driver substitutes when a stream is absent - a 64 KiB and a
129 KiB read past the end of two static arrays, every frame the menu
was up. The two ribbon shaders declare only the position attribute.
Those draws now bind the position from a static VBO uploaded on first
sight (keyed like the Vulkan driver's, on the source array's address,
length and first and last floats) and leave attributes 1 and 2
disabled; the buffer is released with the scratch VBOs. Every other
pipeline streams exactly as before.
The ribbon pipelines draw the same 64x64 grid every frame: 8064
vertices that the menu driver builds once into the display coord
array and never touches again. gfx_display_vk_draw() nonetheless
interleaved them into the frame's VBO chain on every draw, 258 KiB of
write-combined memory written through a per-vertex loop each frame the
menu is up. The driver now keeps one host-visible vertex buffer for
the ribbon, baked on first sight from the same interleave routine the
chain path uses and rebound thereafter, so a ribbon frame costs one
vkCmdBindVertexBuffers. The buffer is keyed on the source array's
address and length plus its first and last floats, so a menu driver
rebuilding the array at the same address triggers a rebake (after
draining the device, since a submitted frame may still read the old
buffer), and any failure to create it leaves the chain path in place.
The interleave loop itself moves into a shared helper with no change
to what it writes.
With the new audio_threaded_pipeline setting (default off, Audio >
Synchronization, applies on driver reinit) the convert, DSP, resample
and volume pass runs on the audio thread instead of inside the frame.
The main thread publishes raw int16 core audio into a lock-free
retro_spsc ring at frame end - the accumulator flush, the batch
callbacks, rewind reversal and menu audio all go through one
audio_driver_submit() - and audio_driver_pipeline_consume(), called
from the wrapper thread's loop, pulls slices out and runs the existing
audio_driver_flush() on them. Off, every path is byte-for-byte what it
was: submit() just calls flush().
The consumer paces on the device rather than in it. Each pass is sized
to what write_avail() will take, so the write never blocks; a write
that blocked until a slice fit would leave the fill reading full on
the next query and rate control would steer on a pinned number. The
wrapper now forwards write_avail() and buffer_size() to the wrapped
driver so rate control measures the real device from the consumer
side, at a fixed cadence, right before it resamples. The producer only
waits for ring space while the driver is started and its consumer is
alive - never on a wrapper parked by a reinit from inside retro_run(),
and never through alive(), which the wrapper implements by stopping
and restarting the device - with a one-second cap as the only guard
against a device that stops draining.
flush() gained an output sink so the same pass can deliver to the
driver or to a caller's buffer; every present caller passes the
driver. Cores that register their own audio callback keep the inline
pipeline, since they already render on the audio thread and would be
a second producer for the ring. The wrapper thread is now started at
init whenever it is created: nothing restarted it after a mid-session
SET_SYSTEM_AV_INFO, which also stalled audio for callback cores.
The state flags word grows to 16 bits for the three new flags. The
audio thread's reads of that word inside flush() are the same
unsynchronised reads the wrapper has always done for callback cores;
making the word atomic is a separate mechanical change.
xmb_animation_line_ticker() malloc()ed a buffer for the word-wrapped
sublabel and freed it again on every frame the ticker ran, which is
every frame a highlighted entry has a sublabel. The scratch now lives
on xmb_handle_t, grown to the longest sublabel seen and released with
the menu; the wrap, line count and line selection are unchanged.
On a GL without GL_EXT_bgra, gl1_draw_tex() malloc()ed a full
pot-sized RGBA staging buffer for the CPU swizzle and freed it after
the upload, once per core frame and again per menu frame, on exactly
the class of hardware where that path is taken. The staging now lives
on gl1_t, grown on demand and released with the driver. The Vita
build's 3-component vertex expansion in the menu quad and font draws
did the same per draw call through a function-local static that was
never freed; that scratch moves onto gl1_t too, behind one helper
that grows it and fills it, and is released with the driver.
rmp4_open_memory_avail() and rmp4_video_stream_open_avail() grew the
optional need_lo/need_hi stall-range outputs, and the webm core's two
progressive opens were never updated, so any build with HAVE_RWEBM
failed on an argument-count mismatch. The core fills its buffer
sequentially and has no use for the range, so it passes NULL for both,
as the audio transfer does.
Alongside it, two spots that clang rejects under -Werror and that the
griffin check therefore hits: the H.264 chroma DC predictor's closing
`} break;}` on one line reads as a statement hanging off the block
loop (-Wmisleading-indentation) and is split onto its own lines with no
change in meaning, and the HEVC decoder loses rh265_max() and
rh265_clip8(), which had no caller left and warn as unused wherever
RH265_INLINE expands to nothing.
Both decoders copied every slice NAL into a freshly malloc()ed RBSP
buffer (rh265 a second one for the escape positions WPP needs) and
freed it on every exit of the NAL handler, fifteen free() sites in
rh264 and four in rh265. Each rh264_video and rh265_video now owns one
scratch block that grows to the largest NAL seen and is released with
the decoder; the HEVC block holds the escape list first and the RBSP
behind it on a 64-byte boundary. The per-NAL frees are gone, the
parameter-set path in rh264 keeps its own short-lived copy, and the
old rh265 unescape helper is removed as it had no remaining caller.
Decoded output is bit-identical on every stream in the harness.
Every simple glyph walked by the rasterizer took three malloc() calls
for its point flags and coordinates and freed them on the way out, so
an atlas build paid three allocations per glyph on top of the render's
own cell block. The font now records maxp.maxPoints at load, and
rtt_render_glyph() carves flag and coordinate arrays of that size into
the block it already allocates for the coverage cells and row extents,
zeroing only the cells; the walker uses them for any glyph that fits
and falls back to one block of its own for a glyph larger than the
font promised, or when maxp is the CFF short form with no point count.
Atlas output is identical across three fonts at four sizes.
The vibrato allocated a float delay line, an int16 mirror and a Q16
delay LUT per channel, the tremolo a float wavetable and its Q16 mirror
per channel, six and four separate blocks that the free paths walked
back, and neither init checked its allocations. Each filter now
measures its regions once and carves one block, every region starting
on a 64-byte boundary; the vibrato lays both channels' delay lines
first so one memset silences them and the LUTs, which init fills in
full, are written once, and the tremolo block is not zeroed at all
for the same reason. Init now fails cleanly when the block cannot be
had. Output of both the float and the int16 paths is bit-identical.
The int16 sinc driver allocated its coefficient table, its stereo ring
and, on NEON, its Kaiser coefficient scratch separately, where the float
driver has always carved the same pieces out of one 128-byte aligned
buffer. It now does the same: one block owned by buffer_l holding the
rings, then the table, then the scratch, each on a 64-byte boundary,
the rings zeroed as calloc() did before, and one pointer to free. The
alignment is what pays: the table's phase rows are a whole number of
cache lines, so a plain malloc() left every row straddling one line
more than it needed and the inner loop paid for it; aligned, the same
test run takes 0.7% fewer L1 data misses. Output is identical.
alloc_planes() made nine calloc() calls for the three plane sets and
unwound them individually on failure; free walked all nine. The planes
now come out of one zeroed block, each starting on a 64-byte boundary,
with the plane[][] pointers as views into it; the failure path and
rmpeg1_video_free() release one pointer. Decoded output is
bit-identical across I/P/B sequences and a resolution change.
The I, P and B slice readers each began by calloc()ing their
coded-block-flag row, skip and type rows and motion-vector-difference
grids, three more under macroblock pairs, and every early return
freed the set again: five to eight allocations and as many frees per
slice, with 79 free() calls spread over the error paths. The working
frame's map block now carries that scratch, carved by rh264_frame_alloc
in a third mode alongside the plain and reference layouts, and each
reader zeroes exactly the regions it uses at slice start. The readers
only ever see the working frame, so the field view's smaller height is
covered by the frame-height sizing. Decoded output is bit-identical.
The checkpoint encoder allocated its superblock index list up front and
a zero-padded copy of the final partial block lazily, freeing both at
the end. They now come out of one zeroed allocation per call, the
padded block behind the list on a 64-byte boundary; it is only written
when the state size is not a multiple of the block size, and the memset
that clears its tail before reuse is kept behind a flag instead of the
former pointer test.
Each GL driver's overlay_load made four calloc() calls for the texture
names and the vertex, texture and colour coordinate arrays, and its
free_overlay walked the four back. The three coordinate arrays are what
the overlay draw path hands to the vertex setup every frame, so they
now come out of one zeroed block behind the texture names, each region
starting on a 64-byte boundary, with overlay_tex owning it. This also
closes the partial-failure gap where a coordinate calloc could fail
with overlay_tex already set and overlays still zero.
A stream voice took two memalign_alloc() calls in the float path and
two in the s16 path, one for the decode scratch and one for the
resampled buffer, and the partial-failure unwind had to free whichever
of the two had succeeded. Each path now takes one 64-byte aligned
block with the scratch first, a 64-byte pad, then the buffer; the
buffer pointer is a view into it and release frees the block through
the scratch pointer. The pad matters: the scratch is exactly 32 KiB
(float) or 16 KiB (s16), so without it the two would be a multiple of
4 KiB apart and the resampler, which reads the scratch while writing
the buffer, would stall on low-address-bit aliasing.
new_replay() allocated the volume ramp, the channel array, the
background-voice pool and the two IT filter scratch buffers separately
and dispose_replay() freed the five. The mixer walks the channels, the
ghosts and the ramp every tick, so they now come out of one zeroed
block with each region starting on a 64-byte boundary, and the named
pointers are views into it. The pool and the filter scratch used to be
individually optional (a NULL degraded new-note actions to cut and
filtered voices to unfiltered); the replay now either has all of its
buffers or fails to construct. Mixed output is bit-identical for MOD
and IT (filtered) modules.
The streaming encoder allocated its six row buffers and the IDAT chunk
buffer separately and freed all seven on the way out. They are sized
from the same row length and live for one encode, so they now come out
of one block: each row's data starts on a 64-byte boundary with its
filter-tag slot the byte before it, and the chunk buffer follows. The
previous-row buffer is the only one read before it is written and is
zeroed explicitly instead of by calloc(). Encoded output is unchanged.
rh265_alloc_frame() made ten malloc() calls for the per-4x4 maps, the
8x8 QP map and the three per-CTB tables, and rh265_pic_alloc() four
more per DPB slot for the planes and the motion field; the free path
walked them all back. The decoder maps now come out of one block and
each DPB slot out of one block holding its three planes and motion
field, every region starting on a 64-byte boundary, with the named
pointers as views. A slot is allocated whole on first use and its
planes get the same zero and mid-grey fill as before. Decoded output
is bit-identical across 8-bit with B pictures, 10-bit and SAO streams.
rh264_frame_alloc() made ten calloc() calls per frame (three planes and
seven per-block maps), the DPB setup added two more per reference
picture for the motion grids, and rh264_frame_free() walked twelve
pointers back. A frame now takes one zeroed block for its planes and
one for its maps, each region starting on a 64-byte boundary, with the
motion grids of a reference picture carved into the map block behind
the maps; Yb/Ub/Vb and the map pointers are views into those blocks.
The plane block is kept separate from the map block because
rh264_out_push() hands a finished frame picture's planes to an output
slot by swapping the three plane pointers rather than copying; the
swap now moves the plane block along with them, so each frame keeps
owning exactly the block its planes live in. Decoded output is
bit-identical across baseline, main with B pictures, field-coded and
4:2:2 streams.
rvp8_begin() made twelve calloc() calls per frame for the per-MB maps,
the above-row context, the chroma interpolation scratch and the three
reconstruction planes, and three free lists (begin, abort, video_close)
walked them back. They are all sized from the frame geometry and live
and die together, so they now come out of one block laid out by
rvp8_arena_carve(), each region starting on a 64-byte boundary with the
planes last. The block is kept in the decoder state across frames and
only replaced when a frame needs more than it holds, so the persistent
video decoder stops allocating per frame entirely; the maps and context
at the front are zeroed with one memset and the planes are set to 127
as before. rvp8_abort() and the failure path in begin free one pointer.
The persistent decoder's nine reference planes likewise come out of one
block with the same spacing. Since they are now sized once per
geometry, a key frame whose macroblock dimensions differ from the
current reference set retires that set before the copy instead of
copying with a stale plane size.
The reverb made 48 calloc() calls per instance: a float line and an
int64 Q16 mirror for each of eight combs and four allpasses, twice over
for left and right, and reverb_free() walked all 48 back. The echo made
two per channel plus one for the channel table, the equalizer four, and
the FFT it embeds three more. Every one of those lines is walked once
per sample by the process loops, so they were scattered wherever the
allocator put them and any two of them could share a cache line. Each
filter now measures its regions once, takes a single zeroed block, and
carves it with every region starting on a 64-byte boundary; the buffer
pointers in the comb, allpass, echo_channel, eq_data and fft structs
become views into that block and the free paths drop to one call. The
reverb loses its bufcomb/bufallpass mirror arrays, which only existed
so the free could find the lines again. Output of both the float and
the int16 paths is bit-identical to before.
thread_video_t keeps the wrapped driver as const video_driver_t*, and
video_driver_state_t::current_video is video_driver_t*; assigning one
to the other discards the qualifier, which clang under -Werror rejects
and broke the Orbis build. Cast as video_driver.c already does when it
assigns from the video_drivers[] table.
rgui_set_aspect_ratio() allocated the frame buffer, background buffer,
fullscreen thumbnail and both mini thumbnails separately, and three
sites freed the five of them in sequence. They are all sized from the
same menu geometry and always live and die together, so they now come
from one 64-byte aligned block carved with the same spacing the audio
and scaler arenas use; rgui_buffers_free() releases it and resets the
five descriptors. The frame buffer is the menu's texture upload source
every frame, so it now starts on a cache line rather than wherever
calloc() put it. upscale_buf stays separate: it is sized from the
display on demand, not from the menu geometry, and has its own free.
Both buffers already used linearMemAlign() on 3DS and plain malloc()
everywhere else; the softfilter one carried a TODO asking for aligned
output. Every softfilter writes its output with vector stores and the
video driver reads it back for upload, and under threaded video the
main thread copies each core frame into the wrapper's buffer for the
video thread to upload. Starting both on a cache line puts every row
on one for the usual pitches. Same lifetime and sizing as before, now
paired with memalign_free().
video_init_thread() points video_st->current_video at the video_driver_t
embedded in the thread_video_t it allocates. video_thread_free() freed
that struct and driver_uninit() nulled video_st->data but not
current_video, so the second video_driver_free_internal() at exit read
vid->font_backend out of freed memory. Point current_video back at the
wrapped driver's static vtable before freeing, which is the state the
non-threaded path is always in.
scaler_ctx now owns two allocations instead of seven: frame_arena holds
the scaled, input and output frames, filter_arena holds the horizontal
and vertical filter tables with their position arrays, and the typed
pointers in the context are views into them. scaler_ctx_gen_reset()
frees two blocks; allocate_frames() and scaler_gen_filter() each do
one allocation with sizes fully computed before anything is touched,
so a failure leaves nothing half-built. The old filter allocation only
checked the two filter tables and would proceed with a NULL position
array; that path is gone.
Regions start on 64-byte boundaries with a 64-byte pad between them
(SCALER_ARENA_NEXT in scaler_int.h). calloc() only guaranteed 16, so
the SSE2/NEON scaling and conversion passes, which run per frame for
any software-rendered core whose pixel format the video driver cannot
take directly, now work on aligned rows, and a frame and its neighbour
are never a multiple of 4 KiB apart. The arenas come from malloc()
with 63 bytes of slack rather than a platform memalign so the scaler
keeps no dependency beyond the C library.
out_stride is a byte count and output is a uint16_t pointer, so each
row landed two rows down and the second half of every conversion was
written past the end of the destination. Shift by one like every other
16-bit converter in the file.
mic_driver_allocate_frames() made eight allocations for the buffers the
microphone read pipeline flushes through; it now makes two, an int16
block and a float block laid out with the same AUDIO_ARENA_NEXT()
spacing the playback path uses, and the named buffer pointers in
microphone_driver_state_t become views into them. Deinit frees two
pointers instead of eight.
This also closes a gap in the old sequence: it returned false as soon
as one allocation failed but left the earlier ones in place, and its
early-out tested only input_frames, so a later open would report the
buffers as ready with some of them still NULL. The arenas are
allocated together or not at all, and the early-out tests the block
that every buffer depends on.
The s16 path copied its input into input_data_int16 before summing a
synth or running an int16 DSP chain, because the input is normally a
core's const batch buffer. When the input is the single-sample
accumulator it is the driver's own buffer and is discarded as soon as
the flush returns, so the copy was a full frame's memcpy for nothing.
The path now picks a writable view up front: the accumulator itself
when that is the source, otherwise the scratch on first use. Output is
unchanged either way; only where the intermediate lives differs.
audio_driver_init_internal() now makes two allocations instead of
seven: an int16 block holding the output staging buffer, the
single-sample accumulator, the rewind buffer and the s16 scratch, and a
float block holding the resampler input, the synth mix buffer and the
resampler output. The named buffer pointers in audio_driver_state_t
become views into those blocks, so deinit frees two pointers and the
partial-allocation unwind in init collapses to one check. Regions are
laid out by AUDIO_ARENA_NEXT(), which rounds each start up to 64 bytes
for the SIMD paths and inserts one 64-byte pad, so buffers that are
read and written in the same pass (resampler in/out, s16 source and
scratch) are never a multiple of 4 KiB apart and cannot alias on the
low address bits.
The float output buffer and the s16 scratch were previously allocated
only after the driver had started or the int16 resampler was known to
exist; both now have a fixed region from the start and only the
pointer assignment keeps its gate, which is what flush actually tests.
The single-sample core callback now accumulates a whole retro_run()'s
worth of audio in a dedicated buffer (sample_accum) and hands it to
audio_driver_flush() once, from audio_driver_frame_end(), which the
runloop, run-ahead and preemptive-frames paths call right after every
retro_run(). The 512/2048 chunk thresholds no longer decide when audio
leaves the frontend; the 2048-sample bound survives only as an overflow
guard, sized to the batch path's slice so both callback styles reach the
resampler in the same unit. A frame's audio is therefore never held
across a frame boundary, and per-sample cores get exactly one resampler
pass and one driver write per frame, as batch cores already do. A core
that registered an audio callback fills the accumulator on the audio
thread instead, and audio_driver_callback() flushes it there.
audio_driver_frame_end() also arms drc_pending, so the first flush of
each frame recomputes the rate-control factor at a fixed point of the
frame rather than wherever the sample-count gate happened to trip. The
count gate remains for producers that never mark a frame end (menu
audio) and for frames delivering more than one threshold's worth.
Giving the accumulator its own buffer also stops the s16 resampler path
from resampling in place over its own input when fed by the
single-sample callback, and audio_driver_setup_rewind() now asserts the
accumulator is empty at the frame boundary instead of harvesting it.
A server running with pausing disabled answers NETPLAY_CMD_PAUSE from a
client with a NAK and hangs the connection up, and the client sent that
command for any frontend pause at all. Pushing the Quick Menu parks the
runloop for a single frame, which is the path both opening the menu and
opening the chat prompt take, so a client on such a server lost the
connection the instant it opened either.
A client whose server disallows pausing now keeps the announcement to
itself. Its local_paused stays clear with it, so the resume that
netplay_pre_frame would send off the back of the pause is suppressed in
the same stroke, and a server that permits pausing still receives the
command exactly as before.
Fixes#19028
The audio_block_frames row carried no range, a step of zero and no
action handler, so it rendered in the audio menu as a value that left
and right could not move - the setting was reachable only by editing
retroarch.cfg by hand. It now steps in units of 64 up to 1024 with the
range enforced, and carries a sublabel saying what 0 means, since a
block size of zero asking the driver for the device's own value is not
something the bare number conveys.
A playlist entry that names a core no longer installed resolved to that
core's path, failed the validity check and reported a failed load, even
where the playlist itself carried a default core that was present and
able to run the content. Deleting a core therefore left every entry it
had ever been used for unloadable until each was reset by hand.
Such an entry now falls back to the playlist default core and is
rebound to it, the same way an entry that never carried a core is, and
says which core it switched to - which core runs the content decides
the save data and options it sees, so the change is not made silently.
Entries whose core is still installed, and playlists with no default
core set, resolve exactly as before.
These three set no DEFAULT_DIR_CACHE, so cache_directory stayed empty
and archive extraction fell back to writing beside the content. Each
now defaults it to a 'temp' directory alongside the other per-port
defaults, which keeps extracted content out of the user's content
directories.
An archive member was extracted straight into the cache directory, or
beside the archive when none is configured, under its own basename. A
file of that name already sitting there was overwritten by the
extraction and then deleted at teardown along with the other
temporaries, so loading ROMNAME.zip destroyed an unrelated ROMNAME.nes
next to it.
Extraction now goes into a uniquely named directory created for the
load, so the member keeps its own basename - savefile and savestate
paths derive from it, and a uniquified file name would move a user's
saves - while the name it lands on is one nothing else can hold. The
directory is recorded as a temporary after its file, leaving teardown
to empty it before removing it.
Fixes#5954
file_archive_compressed_read() dereferenced the result of
file_archive_get_file_backend() unconditionally, and that lookup yields
NULL for any archive extension whose codec is not compiled in. The
delimiter parser accepts .7z, .zip, .zst and .apk regardless of build
configuration, so a path carrying one of those - a playlist entry, a
command-line argument, an overlay config - could reach the call with no
backend to serve it and crash. Both the backend and the string list the
path splits into are now checked, and either one missing reports a read
failure, which every caller already handles.
archive_zip_test covers it: that harness builds with HAVE_COMPRESSION
alone, so .7z and .zst members there have no backend by construction.
The atomic save swaps the directory entry itself, so writing a config
through a symbolic link replaced the link with a regular file - a setup
like many remap files pointing at one shared layout lost its links on
the first save. config_file_write now resolves the path before building
the temporary, so the temporary lands beside the link's target and the
rename replaces the target while the link stays in place. When
resolution is unavailable or fails - dangling link, over-long path,
platform without realpath - the path is used as given. The resolved
buffer lives on the heap, keeping the frame inside the libretro-common
stack budget.
Fixes#19466
The C90-scoping fix left a stray `}` after the lakka-switch.h include
with no matching `{`, and a missing closing `}` before the
switch_oc/switch_cec/bluetooth_ertm #endif -- breaking any
HAVE_LAKKA_SWITCH=1 build (PROJECT=L4T DEVICE=Switch) with cascading
"invalid storage class" errors. Rebalances both.
frontend_unix_parse_drive_list() appends the shared storage location
twice on Android 11 and later: once as the internal-storage entry, and
again when the StorageVolume loop reaches the primary volume, whose
directory is the same /storage/emulated/0 path. The loop now records
which shared-storage path the earlier block listed - internal storage
when the environment supplied one, the literal fallback otherwise - and
skips a volume whose path matches it, so the root listing names each
location exactly once.
Only that exact plain-path duplicate is dropped: SD cards, USB volumes
and every other entry keep their direct filesystem paths, and the SAF
tree entries are appended separately as before, so no location loses
its fast non-SAF route.
The native derivation that replaced the Java launcher fills an absent
EXTERNAL extra by copying internal_storage_path, so a direct launch sets
the app-external files location to the shared storage root and the file
browser's root listing shows /storage/emulated/0 three times: as
internal storage, as the external application directory, and as the
primary storage volume. The launcher always passed the app-external
files directory here, /storage/emulated/0/Android/data/<package>/files,
so derive that location from Context.getExternalFilesDir(null), which
returns exactly that path and creates it on demand; copying the shared
location remains as a last resort for unmounted storage.
The scoped-storage media-dir branch also runs with the shared location
already supplied, overwriting an SDCARD extra on all-files-denied
devices; it now honors the extras-always-win rule the derivation pass
promises and only fills a still-empty value.
The send side caps the echo payload at 1400 bytes, so a matching
raw-socket reply occupies at most a 60 byte IP header, the 8 byte ICMP
header and the payload; a 1500 byte buffer covers that with room to
spare, matching the Windows tier's reply sizing. Oversized foreign
packets are truncated by recv() and rejected on their leading bytes,
which truncation leaves intact. Brings net_icmp_echo_poll's frame from
65632 to 1600 bytes, inside the 2 KiB libretro-common stack budget.
Pre-Vista SDKs declare the 64-bit Interlocked forms only when
_WIN32_WINNT >= 0x0502, and on 32-bit x86 there is no kernel32 export
behind the names in any case -- the operation is the CMPXCHG8B
intrinsic or nothing. An undeclared use is therefore worse than a
build break: C4013 truncates the 64-bit result to int, and the link
fails on the missing export. On x86 with a pre-Vista SDK and an older
target version, declare _InterlockedCompareExchange64 directly and
build the exchange as a CAS loop over it, which is also how the modern
SDK implements InterlockedExchange64 on that architecture.
VS2003 and the OG Xbox XDK have neither the intrinsic nor an export,
so on those targets the section defines nothing and
RETRO_ATOMIC_HAS_64 stays unset, matching how old 32-bit __sync
targets already degrade.
FILE_NAME_NORMALIZED is a Vista-era constant, the same generation as
GetFinalPathNameByHandleW itself. The function is resolved by name at
run time precisely so that older toolchains can build this file; the
flag has to clear the same bar, so supply the documented value under
the same guarded-define pattern the punch-hole control codes use.
Fixes the MSVC2005 (_WIN32_WINNT=0x0400) lane.
One request in flight per handle: datagram ICMP on Linux and macOS
with a raw-socket second try, IcmpSendEcho2 with an event on Windows,
and a stub tier elsewhere whose open() fails so callers fall into the
unreachable-host path they already have. Wired into no build list --
consumers add the object where they adopt it, so every existing
workflow and platform build is untouched by construction.
Verified: C89 -pedantic on x86-64 and aarch64, the darwin target
against the macOS 15.5 SDK, and a live loopback echo round trip under
ASan and UBSan through the raw-socket path.
Fixes the i686 MXE build, and a Win9x load failure that build was
pointing at.
get_sparse_granularity named four Win32 functions directly:
GetFinalPathNameByHandleW and GetVolumeInformationByHandleW are Vista
and later, GetVolumePathNameW and GetDiskFreeSpaceW are 2000 and later.
Two problems follow from that, and only the first showed up as a
compile error.
mingw does not declare the Vista pair below _WIN32_WINNT 0x0600, which
is what the MXE i686 target builds at, so they came out as implicit
declarations and the build failed.
The worse one is that a static import is resolved by the loader before
main runs. A binary that names GetVolumePathNameW does not start at all
on 95, 98 or ME, because their kernel32 does not export it -- the
feature would not have degraded, the whole frontend would have failed
to launch. HAVE_WIN9X_RUNTIME exists precisely so that does not happen.
All four are resolved with GetProcAddress now, once, into statics. If
any is missing the function returns 0, which callers already treat as
"unknown granularity" and skip sparse handling -- correct on 9x
regardless, since FAT32 has no sparse files. GetVolumeInformationByHandleW
is treated as optional on its own: without it the cluster size is
returned rather than the NTFS compression unit, which is the
conservative answer.
Verified with the compiler the failing job uses, which I did not have
before and have now installed: i686-w64-mingw32-gcc clean at default,
-DHAVE_WIN9X_RUNTIME, -DLEGACY_WIN32, -D_WIN32_WINNT=0x0400,
-D_WIN32_WINNT=0x0501, and all three together; x86_64 mingw and gcc
likewise; libretro-common/Makefile.test now runs the i686 lane instead
of skipping it and passes; parity reports 24 symbols.
And the load-order claim is checked rather than asserted: objdump on a
linked i686 binary shows none of the four names in the import table,
where before this change GetVolumePathNameW and GetDiskFreeSpaceW were
both there. The Linux behaviour is unchanged -- granularity still
matches st_blksize and one aligned granule still frees exactly that.
The function returned the cluster size, which is the wrong answer on
the most common Windows configuration. NTFS deallocates a sparse file
in compression units -- 16 clusters, capped at 64K -- not in clusters,
so on a 4K-cluster volume the real granularity is 65536 and punching a
single 4K cluster frees nothing at all. That is exactly the silent
"file is not sparse and nobody notices" failure the function exists to
prevent.
Windows now also asks GetVolumeInformationByHandleW for the filesystem
name and applies the compression-unit rule only on NTFS; every other
filesystem still deallocates by cluster. POSIX is unchanged.
Found while converting the caller this was written for. PCSX2's ATA
backend does exactly this NTFS mapping itself, in a table keyed on
cluster size, and I had ported only half of what it does with its
native handle -- the cluster size, not the filesystem check. Reading
the whole caller before designing the API would have caught it; I read
the part that matched what I had already decided to build.
Verified: gcc, x86_64-w64-mingw32-gcc, mingw -DLEGACY_WIN32, and a TU
defining INITGUID before windows.h all compile clean; parity reports 24
symbols with both backends agreeing; Makefile.test passes including the
Windows A/W matrix; and on Linux the reported granularity still matches
st_blksize with one aligned granule freeing exactly that many bytes.
The allocation unit of the filesystem holding a file: the smallest span
filestream_punch_hole can actually free. Punching a range shorter than
this, or one not aligned to it, is accepted and frees nothing.
This is the piece that makes punch_hole usable. PCSX2's ATA HDD backend
learns the cluster size by calling GetFinalPathNameByHandle on a HANDLE
it pulls out of its FILE*, and that single call is what keeps the whole
backend on raw stdio: the I/O has VFS equivalents, but the descriptor
does not. The question is about the filesystem rather than the file, so
the backend that owns the handle is the right place to answer it, and
the caller never needs to see a descriptor at all.
Windows resolves the path through junctions with
GetFinalPathNameByHandleW, takes the mount point with
GetVolumePathNameW -- which copes with a volume mounted on a folder
rather than a letter, unlike splitting the string by hand -- and
multiplies out GetDiskFreeSpaceW. POSIX uses fstat's st_blksize.
Anything else returns 0.
0 means unknown, and the header says callers must not read it as "no
alignment needed": a frontend-supplied VFS has no descriptor to ask
about, so it returns 0 too. Assuming 4096 is exactly the bug this
avoids -- 4K is typical for NTFS but 8K, 16K and 64K volumes are
common, and punching 4K units on one of those produces a file that
silently is not sparse.
Verified rather than asserted. On Linux the reported granularity
matches stat's st_blksize (4096 here), and punching exactly one aligned
granule frees exactly 4096 bytes of allocation. Compiled with gcc,
x86_64-w64-mingw32-gcc, mingw with -DLEGACY_WIN32, and inside a
translation unit that defines INITGUID before windows.h, which is the
griffin unity build that the winioctl.h include broke.
The UWP backend gets the symbol too, returning 0: it replaces
vfs_implementation.c rather than extending it, so a missing definition
is a link error. vfs_backend_parity.py reports 24 symbols with both
backends agreeing, and libretro-common/Makefile.test passes including
the Windows A/W matrix.
Reverts the include added in f7b4104 and replaces it with the three
declarations that were actually needed.
winioctl.h defines the storage class GUIDs. griffin.c is a unity build
that compiles this file alongside the DirectX headers, which set
INITGUID, so those DEFINE_GUIDs emit definitions into a translation
unit that already has them -- twenty-odd C2374 "redefinition; multiple
initialization" errors on MSVC x64 Debug. Fixing the missing symbols by
pulling in the whole header traded one build failure for another.
Nothing here needs a GUID. FSCTL_SET_SPARSE and FSCTL_SET_ZERO_DATA are
their documented CTL_CODE expansions, 0x000900c4 and 0x000980c8, and
the zero-data range is a two-LARGE_INTEGER struct. Each is behind an
ifndef, so a translation unit that already saw winioctl.h through some
other path keeps that header's definitions; the struct is spelled
RETRO_FILE_ZERO_DATA_INFORMATION so it cannot collide with the real one
either way.
Checked against the case that broke, not just the plain build. With
x86_64-w64-mingw32-gcc: this file alone; inside a TU that defines
INITGUID before including windows.h, which is the griffin situation;
and inside a TU that has already included winioctl.h, which is the
guard path -- all clean. Plus -DLEGACY_WIN32, plus gcc,
vfs_backend_parity.py at 23 symbols, and libretro-common/Makefile.test
including the Windows A/W matrix.
Three failures from one small commit is a bad record. The first two
came from not running the checks this repository ships; this one came
from fixing a compile error with the first header that declared the
missing names, without asking what else that header brings into a unity
build.
FILE_ZERO_DATA_INFORMATION, FSCTL_SET_SPARSE and FSCTL_SET_ZERO_DATA
are declared in winioctl.h, not windows.h, so the Windows arm of
retro_vfs_file_punch_hole_impl did not compile: eleven errors on MSVC
2022 ARM64 Debug, and the same under MinGW.
This is the second defect from the filestream_punch_hole commit, after
the UWP parity one fixed in f0023de. Both were reachable in seconds
with tools already present -- x86_64-w64-mingw32-gcc, and the
libretro-common Makefile.test that builds three Windows
configurations -- and I compiled only for Linux before sending the
original patch.
Verified: vfs_implementation.c compiles clean under gcc,
x86_64-w64-mingw32-gcc, and mingw with -DLEGACY_WIN32;
vfs_backend_parity.py reports 23 symbols with both backends agreeing;
libretro-common/Makefile.test passes including the Windows A/W matrix.
Fixes the parity failure my previous commit caused. The UWP backend
replaces vfs_implementation.c rather than extending it, so a function
added to the C backend and called from the shared file_stream.c
compiles everywhere and fails to link on UWP alone.
tools/vfs_backend_parity.py exists precisely because
retro_vfs_file_get_mapped_ptr_impl once shipped that way, and it caught
this within a minute of the push. I did not run it, or
libretro-common/Makefile.test, before sending the patch -- I verified
the behaviour of the new call and the build of the two trees I had
locally, and never looked for the repository's own checks. That is the
actual mistake here; the missing stub is just its symptom.
UWP cannot punch holes: FSCTL_SET_ZERO_DATA goes through
DeviceIoControl, which is not in the app container's API surface, and
this backend's handle comes from a StorageFile. So it returns the
documented -1, which callers of filestream_punch_hole already handle by
writing zeroes instead -- the space saving is lost, nothing else.
Checked with the tools the CI uses: vfs_backend_parity.py reports 23
symbols with both backends agreeing, and libretro-common/Makefile.test
passes including the Windows A/W configuration matrix.
Deallocate a range within a file, leaving the length unchanged and
subsequent reads of the range returning zeroes. This is what makes a
sparse disk image sparse: an image that is mostly untouched occupies
only the blocks actually written.
Deliberately NOT an addition to retro_vfs_interface. Extending the
frontend ABI would oblige every frontend to implement an operation most
of their backends cannot do, for the benefit of one emulator's HDD
image. Instead this is a capability of the built-in implementation,
which already carries the descriptor it needs -- int fd, and HANDLE fh
on Windows -- and reports failure when a frontend supplied its own VFS,
because such a handle need not be a file at all. Callers ask, and write
zeroes when the answer is no. The only thing lost is the space saving,
never correctness, and the declaration says so.
fallocate with FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE on Linux;
FSCTL_SET_ZERO_DATA on Windows, preceded by an idempotent
FSCTL_SET_SPARSE because a zero-data range on a file not marked sparse
succeeds while saving nothing. Every other platform returns -1.
Checked on the property that matters rather than the return code -- a
stub returning 0 would look identical. A 16MB solid file, punching 8MB
from the middle: allocated blocks (st_blocks * 512) fall from 16777216
to 8388608, the logical size stays 16777216, and the punched range
reads back as zeroes.
The motivating caller is PCSX2's ATA HDD backend, which today reaches
through FILE* with fileno() and _get_osfhandle() to call fallocate and
DeviceIoControl itself. That is the last thing keeping it on raw stdio
rather than the VFS, and the reason it cannot work on an Android
content:// URI.
Binding a netlink socket is a privileged operation under the SELinux
policy Android applies to applications, so the call is refused with
EACCES and getifaddrs() gives up before it has asked the kernel
anything. The binding is not needed: the kernel assigns the socket a
port ID of its own when the first request goes out, and the replies are
matched against whichever one it picked. Ask for it where it is
permitted and carry on where it is not.
This is the last of the three netlink operations behind Network
Information that an application is not granted.
Reading interface flags is privileged on some platforms. Where the
request is refused every entry comes back reporting no flags at all,
which the IFF_UP test cannot tell apart from a machine whose interfaces
are all down, and Network Information ends up saying there is nothing
there. Count once with the flag test and, only if that finds nothing at
all, count again without it: an address is in this list because it is
configured on an interface, so reporting it is better than reporting an
empty machine.
net_sockets.c is the serious one. On Windows it aliases read(), write()
and close() onto their winsock equivalents and never takes them back, so
in the griffin unity build every later call to those names in the SSL
code, in RetroArch itself and in libsmb2 is quietly rewritten. Undo the
aliases at the end of the file, under the same condition that set them.
x509_crl.c, x509_crt.c and x509_csr.c each give BEFORE_COLON a different
value, 14, 18 and 14. They undefine before defining so it compiles
quietly and each file sees its own value, but the last one still escapes.
Undefine after the info() function that uses them.
aesni.c, padlock.c and timing.c leave asm aliased to __asm for the rest
of the unit. Take it back at the end of each file, but only when the
#ifndef actually fired, since a system header may own the name.
F1, P, S, SHR and the rest are helpers for one function each, but they
were left defined for the remainder of the translation unit. In the
griffin unity build that is a collision with whatever comes next, and
renaming them only moves the problem: an earlier attempt to prefix the
sha512.c set with SHA512_ walked straight into libsmb2's sha384-512.c,
which spells its own helpers the same way.
Scope them instead. Each block now undefines a name before defining it,
so nothing leaking in from an earlier file can clash, and undefines it
again after the last use, so nothing leaks back out. The names go back
to their upstream spelling, which also keeps the round call sites at
their original width.
Drop the #ifndef guards in ripemd160.c along the way. Those were worse
than the redefinition they avoided: had another file leaked an F2, the
guard would have quietly compiled RIPEMD-160 against it.
GET_UINT32_BE and friends were open-coded once per .c file, eleven files
in all, each copy wrapped in an #ifndef guard. In the griffin unity build
that guard is actively harmful: it makes whichever copy is seen first win,
so a file silently compiles against a foreign definition rather than its
own. sha256.c already lost that race - it spells the macros with the
statement-safe do/while form, but sha1.c is included first and its bare
brace form suppressed it.
Move the definitions to mbedtls/int_util.h under MBEDTLS_ prefixed names,
in the do/while form throughout, and have the eleven files include it.
The WANT_IFADDRS branch reaches for IFF_UP without naming a header that
declares it, so the file only translates when something ahead of it in a
unity build happens to have pulled <net/if.h> in already. Include it
alongside <compat/ifaddrs.h> so the file stands on its own.
<net/if.h> may declare a 'struct ifaddr' whose accessor macros are
spelled like the members of 'struct ifaddrs', so move the fixup for that
into <compat/ifaddrs.h> itself, letting either header come first.
Ask git whether the RetroArch source root is itself the repository root
via an empty rev-parse --show-prefix, so the check no longer depends on
make and git agreeing on path syntax. This keeps the git version working
on Windows toolchains where the two disagree, warns when an enclosing
repository is resolved instead, and treats an empty GIT_VERSION in the
environment as unset so an explicit override must carry a real value.
The griffin unity build pulls ripemd160.c and sha512.c into one
translation unit, where both files define F1 and P as file-local round
helpers, so the second one to be included redefines the first and Clang
reports -Wmacro-redefined on every Android ABI.
Prefix the round helpers in each file with RIPEMD160_ and SHA512_ so the
names cannot collide, and drop the #undef and #ifndef guards that were
papering over the clash. The #ifndef guards were the riskier half: they
silently let a leaked definition from an earlier file win instead of
defining the correct one.
Anchor the git version lookup to the RetroArch source root and only
accept the result when the resolved toplevel is that root, so a source
tree without its own .git cannot inherit an enclosing repository's HEAD.
GIT_VERSION may also be supplied from the environment for build systems
that already know the commit being built.
The Apple builds do not use the in-tree libsmb2: retroarch-apple-deps
injects -DHAVE_SMBCLIENT, its own prebuilt headers, and -lsmb2 through
deps.xcconfig, and BaseConfig never adds deps/libsmb2/include, so
smb2/*.h resolves only to the prebuilt copy there. That copy declares
srvsvc_SHARE_ENUM_UNION with an anonymous inner union where the in-tree
fork names it u for C89 -- the single layout difference between the two
header sets -- so the share enumeration in vfs_implementation_smb.c,
written against the fork, failed the Metal builds with no member named
u. The fork header now carries a marker macro next to the struct, and
the one divergent access picks its spelling from it.
Verified by reproducing the exact error with the retroarch-apple-deps
headers under clang targeting arm64-apple-macos11 against a real SDK,
then compiling clean the same way, the full griffin.c with
HAVE_SMBCLIENT included; the builtin path still compiles against the
in-tree headers.
The previous commit made lrc_hash.h's MD5 the one implementation on
every platform, but griffin still carried the matching Apple guard from
the CommonCrypto era around utils/md5.c -- so the Xcode builds called
MD5_Init with the only definition compiled out, and the tvOS and macOS
links failed on the undefined symbol. The guard is gone; libretro-
common's MD5 now compiles everywhere it is called. Makefile.common
already listed the object unconditionally, and this was the only
platform-guarded reference to it in the tree.
Verified by linking the smb2 hmac consumer against utils/md5.c under
-D__APPLE__ with the RFC 1321 vector passing, plus a full Linux build.
The md5.h rewrite in df598e9 dropped the old header's stray
netinet/in.h include, which on Darwin had been the only route by which
time() reached five libsmb2 sources -- their own time.h includes sit
behind HAVE_TIME_H, and no config.h is ever seen by the Xcode build, so
every autoconf guard there is off. Modern Apple clang treats an
implicit function declaration as a hard error where GCC only warns,
which is why exactly the tvOS, macOS, and iOS jobs broke. init.c,
libsmb2.c, ntlmssp.c, pdu.c, and sync.c now include time.h
unconditionally; it is a C89 header and exists on every target.
lrc_hash.h also stops aliasing MD5 to CommonCrypto on Apple. CC_MD5 has
been deprecated since macOS 10.15, nothing had ever actually called the
aliases -- rcheevos hashes through its own md5 -- and the first code to
do so, libsmb2's hmac-md5, immediately painted the Apple logs with
deprecation warnings. The portable implementation in utils/md5.c now
serves every platform; verified against the RFC 1321 vectors.
Reproduced the failure and the fix by compiling the smb2 griffin block
in griffin's include order against a stubbed CommonCrypto with
implicit declarations as errors.
The SMB1/SMB2/SMB3 protocol signatures and the transform header magic
were held in char arrays, so 0xFD/0xFE/0xFF did not fit on targets where
char is signed. They are wire bytes rather than text and become uint8_t;
every consumer reaches them through memcmp or memcpy, so nothing else
changes.
smb2_utf16_to_utf8() writes raw UTF-8 octets through a char cursor,
which put the U+FFFD replacement sequence out of range as well. The
cursor becomes unsigned char while the returned string stays char *.
Conversion output is unchanged: hashing the emitted bytes over every
BMP code unit, supplementary-plane surrogate pairs, lone leading and
trailing surrogates and mixed-width strings gives an identical digest
before and after, clean under ASan and UBSan.
The Winsock writev/readv shims were defined, not declared, in compat.h,
so all 54 translation units that include the header carried an unused
copy. Only socket.c calls them, so the definitions move to compat.c and
the header keeps declarations alongside the __USE_WINSOCK__ prototypes
it already had.
aes_apple.c and krb5-wrapper.c still held C++ comments. Both are
compiled out on the CI hosts, but the preprocessor scans skipped blocks
for character constants, so an apostrophe inside one of those comments
reads as an unterminated literal under C89.
The MSVC-only warning pragma in compat.h gains a _MSC_VER guard; every
other compiler was reporting it as an unknown pragma once per TU.
The C89_BUILD job compiles deps/libsmb2 with -std=c89 -ansi -pedantic
-Werror=pedantic, and the vendored tree relied throughout on C99 and GNU
extensions. Trailing enumerator commas, anonymous unions, zero-size
arrays, non-int bit-fields, // comments, statement-expression
container_of, non-constant aggregate initialisers and mid-block
declarations are all lowered to C89 constructs here, which also clears
the way for the MSVC targets that reject the same extensions.
Single-member anonymous unions are dropped outright, since a union with
one member has that member's layout. Multi-member ones gain the name "u"
and their use sites follow. Zero-size arrays become [1]; all three are
sized through offsetof(), so the allocation arithmetic is unchanged.
Where a _MSC_VER fork already open-coded what a GNU macro did, the fork
collapses into the now-portable macro.
Struct offsets and sizes are unchanged across every touched type, and
the arena allocator carrying the rewritten container_of is clean under
ASan/UBSan/LSan and TSan.
The differential fuzz smoke compiles config_file.c from a source list
written inline in the workflow itself rather than through the sample
makefile, so the makefile sweep never reached it. That inline list now
carries string/rstrtod.c. Grepped every workflow for other inline
compiles of the sources that reference rstrtod; this was the only one.
Reproduced the step exactly: the fuzzer builds under clang with
fuzzer,address,undefined and a 20k-run smoke completes clean.
The previous sweep enumerated makefiles by pattern at the repository
root and under libretro-common/samples, and missed every nested build
tree: libretro-db and its two sample tests (the reported failures), the
top-level samples/ tree for gfx, menu, playlist, and tasks including
the four http stress makefile variants, and the standalone makefiles of
the bundled ffmpeg, imageviewer, and net-retropad cores. This pass
enumerated by find across the whole repository instead, and every
source list linking config_file.c, file_stream.c, or rjson.c now
carries string/rstrtod.c -- 24 more makefiles.
All of them build with the sanitizer arguments the workflows pass, and
the two tests the libretro-db job executes build and run clean. Two
directories fail identically on the pristine upstream tree and are
untouched here: libretro-db/lua's rmsgpack_test wants intfstream
objects it does not link, and samples/gfx/font_lifecycle references
video_driver_thread_wrapper_active, which no longer exists.
The strtod conversion wired Makefile.common, griffin, and the two MSVC
Salamander projects, and missed everything else that compiles
libretro-common sources from its own list: all eight console Salamander
makefiles (ctr, dos, orbis, ps2, psl1ght, psp1, vita, wii) plus the
wiiu one, the libretro-common sample makefiles, and Makefile.test.
Every source list that pulls config_file.c, config_file_userdata.c,
file_stream.c, or rjson.c now carries string/rstrtod.c; lists that do
not need it were left alone, including per-object rules that name those
files as a compile input rather than a link set.
Checked against every entry point in the tree: each console main build
routes through griffin or Makefile.common, the Xcode, Android, and MSVC
projects build griffin, and every workflow under .github/workflows maps
to one of the covered entry points. All 36 sample programs and the
Makefile.test suites build and pass with the exact CI invocation, and
the parser itself runs the differential suite bit-exact under qemu on
big-endian PowerPC and 32-bit ARM, the two architecture families the
console targets add.
The six atof calls join the strtod conversion: both menu dropdown
value paths, the SIXEL_SCALE environment read, the mali fbdev mode
parse, and rjson's number reader. rjson had been sniffing the locale's
decimal separator and rewriting the '.' in the number text so atof
would accept it; rstrtod reads the '.' the JSON grammar requires in any
locale, so that workaround and the reader's decimal_sep state are gone
along with the buffer mutation. The writer keeps its separator fixup,
since that one corrects snprintf output rather than parsing.
sscanf needs no equivalent pass: the first-party sites are all integer
conversions, and filestream_scanf's float path already reads through
rstrtod. Verified with a locale test parsing JSON floats under nl_NL,
per-TU compiles of the conditional sixel and mali drivers with
-Werror=implicit-function-declaration, and a clean full build.
The include landed inside an #ifdef HAVE_MIST block in menu_cbs_ok.c,
inside the HAVE_SLANG/HAVE_SPIRV_CROSS block in video_shader_parse.c,
and inside a generated settings table initializer in menu_setting.c --
so whether the declaration existed depended on the feature set, and in
menu_setting.c the prototypes sat inside an aggregate. All three now
include it at the top with the other libretro-common headers.
Every touched TU recompiles with -Werror=implicit-function-declaration,
and a full build carries zero implicit-declaration warnings.
Every strtod/strtof in first-party code becomes rstrtod/rstrtof: the
signatures match, and the parse no longer depends on the process
locale. Under a comma-decimal locale, strtod reads "59.94" as 59 and
"0.005" as 0 -- demonstrated through config_get_float/get_double
against a real config -- which is the long-standing class of bug where
refresh rates, shader parameters, and overlay coordinates collapse to
integers on locales that RetroArch's C-locale enforcement does not
reach (threads that setlocale themselves, cores that flip it, or
platforms without the enforcement). Touched: config_file and its
userdata list parser, filestream_scanf's float conversions, shader
parameter parsing, overlay coordinates, menu setting input, the
video-processor and mpv bundled cores, and the linux battery reader.
rstrtod.o joins Makefile.common and griffin next to stdstring, and the
Salamander projects that compile config_file.c directly pick it up too.
Deliberately untouched: strtoul/strtol (integers have no locale
problem) and everything under deps/.
rstrtod/rstrtof are locale-independent, correctly rounded float parsers
in one C89 TU: exact power-of-ten fast path, Eisel-Lemire on a 128-bit
product, big-decimal fallback for the ambiguous leftovers. The same
bytes give the same value in every locale on every thread -- strtod
under a comma-decimal locale silently reads "1.5" as 1 -- and they
run 4-5x ahead of glibc strtod. Verified bit-exact against strtod and
strtof over millions of inputs including full subnormal sweeps, clean
under ASan/UBSan, C89 -pedantic, aarch64.
ryaml is a single-TU MSVC-C89 YAML subset parser (block maps and
sequences, flow lists, quoted scalars, anchors rejected as unsupported
rather than mis-parsed), indexed nodes with no per-node allocation.
It reads PCSX2's GameIndex.yaml in LRPS2, where it replaced rapidyaml
at a fraction of the code size.
Neither is wired into the RetroArch build here; the next commit puts
rstrtod behind the float parses.
android_input_poll_event_type_key() indexed the per-port key-state row
with the raw AKeyEvent_getKeyCode() value. Rows hold LAST_KEYCODE bits,
but the platform emits public keycodes above that (AKEYCODE_WAKEUP is
224, AKEYCODE_PROFILE_SWITCH 288) plus unbounded vendor codes, so any
such press wrote outside the row. On ANDROID_KEYBOARD_PORT - the last
row - that lands past the array and corrupts adjacent file-scope state,
including the function pointers the driver keeps there.
Reject bit indices at or above LAST_KEYCODE, matching the bound readers
already apply in android_joypad_button_state(). Nothing in that range is
bindable, so no reachable input is lost.
delete_linebreaks() compacts the certificate body in place but never
terminated the compacted run, and the caller kept passing the span of
the original line-wrapped body to unbase64(). Since unbase64() now
insists on an exact 4-aligned length, every wrapped PEM certificate
was rejected and the BearSSL trust store ended up empty; before that
guard the stale tail bytes decoded as garbage instead. Strip CR as
well as LF, NUL-terminate the compacted run, hand unbase64() its
strlen(), and skip certificates it refuses rather than pushing a
NULL blob into the X.509 decoder.
Also stop scanning when a BEGIN marker has no matching END marker so
a truncated bundle cannot drive strstr() past the buffer.
The two-argument mbedtls_ssl_conf_min_version() and the
MBEDTLS_SSL_MAJOR_VERSION_3/MBEDTLS_SSL_MINOR_VERSION_3 pair are 2.x
spellings that 3.x deprecates and 4.x drops, so a build against a
current system library stops at an undeclared identifier. Only 2.x
needs the override in the first place: its default preset floors the
client at TLS 1.0 whichever protocol versions are compiled in, while
3.0 removed TLS 1.0 and 1.1 outright and floors its own default preset
at 1.2. Guard the call on MBEDTLS_VERSION_MAJOR and pull in the bundled
version.h so that macro is defined on the builtin path too.
Checked with -fsyntax-only against the in-tree 2.6 copy and against
system 2.28.8, 3.6.4 and 4.0.0 headers; the call survives preprocessing
on the first two and disappears on the last two.
The BACK to X compatibility alias sets AKEYCODE_X on both the event's own
port row and the dedicated ANDROID_KEYBOARD_PORT row, but the release path
only clears the former. Pair the clear so the keyboard row cannot latch X
on when the alias fires for a device that occupies a pad slot, which would
otherwise report RETROK_x as permanently held to both RETRO_DEVICE_KEYBOARD
and any joypad bind carrying a keyboard mapping.
Drop the unused device_id local from the same block.
rchd is decode-only, so generating input to test it against needs a
full MAME toolchain and in practice it gets tested against whatever
files someone happened to have. This adds a writer, making the round
trip self-contained: write an image, read it back, compare.
It does no I/O, for the same reason the decoder does none. The reader
is fed bytes the caller fetched (rchd_feed / rchd_read_pending /
rchd_read_step); the writer hands the caller bytes to store, through a
positioned-write callback. Nothing in formats/ should be opening files,
and an earlier draft of this that called filestream_open was wrong.
Positioned rather than sequential because the header and hunk map are
only complete once every hunk has landed -- the map records where each
one went -- so they are emitted last, at offset zero.
rchd_write_prefix_size tells a caller how many bytes to reserve there
if its destination cannot seek.
Uncompressed hunks only: the four compressor slots are written as zero,
which the format defines as an uncompressed file. The codec paths
(LZMA, zlib, FLAC, Huffman) are not exercised by anything this
produces, so it is a fixture generator rather than an archiver, and the
header says so.
The layout follows what rchd_map_v5_raw already reads: an uncompressed
v5 map is a flat array of hunk indices and a hunk's offset is its index
times hunk_bytes, so the file is a sequence of hunk-sized blocks with
the header and map in the first few. Index zero is reserved -- the
reader treats it as a hole -- so an all-zero hunk is stored as a hole,
which costs nothing and exercises the reader's hole handling.
Round-tripped through an in-memory sink: twelve hunks including one
all-zero, correct prefix size, and a well-formed v5 header on the way
out.
The griffin unity build puts this file in one translation unit with
everything included ahead of it, so whatever the C library brought in
earlier owns the uppercase namespace for the rest of the file. B0 is a
baud rate in <asm-generic/termbits-common.h>, which arrives by way of
<sys/ioctl.h>, and a local of that name stops being a declaration.
Lowercase locals stay clear of the macro namespace the C library
reserves. Generated code is unchanged.
The kernel stamps each reply with the port ID it assigned to the socket,
which equals the process ID only when nothing else in the process
claimed that number first. Read the assigned ID back with getsockname()
and match on that, so a netlink socket opened elsewhere in the process
cannot make every reply look like it belongs to someone else and leave
the receive loop waiting for an NLMSG_DONE that has already gone by.
SELinux only grants RTM_GETLINK to system UIDs, so an application asking
the kernel to enumerate links is refused and getifaddrs() has to work
without that half of the picture. Treat the link results as optional:
RTM_GETADDR alone already reports every configured address, an address
that arrives without a label takes its name from its interface index,
and the interface flags come back from a SIOCGIFFLAGS ioctl once the
list is built.
This is what net_ifinfo_new() reads to populate Network Information, and
what netplay LAN discovery and UPnP port mapping consult when picking an
interface to bind to.
Split the two load paths into internal helpers and give the poke entry
points the threaded dispatch every other GPU driver already has, routing
through video_thread_texture_handle() the way vulkan_unload_texture()
does.
An upload allocates, records and frees a command buffer on
vk->staging_pool. VkCommandPool is externally synchronised, and the video
thread allocates from that same pool in vulkan_font_render_msg() for its
glyph atlas uploads; the queue_lock around the submit covers neither the
allocation nor the free. Running the upload on the caller's thread
therefore left two threads free to enter vkAllocateCommandBuffers on one
pool.
The compressed path loses its (void)threaded now that the flag reaches a
real branch.
Three values reach an array subscript here without being checked. The
port indexes analog_state[DEFAULT_MAX_PADS] and is in range only
because android_input_get_id_port maps a touchpad source to port 0 or
to a live pad slot - an invariant that lives in another function and
would be lost the moment that mapping changes. The pointer index that
rides in the action word is not bounded by anything on this side of the
framework, and AMotionEvent_getPointerId indexes the event's pointer
array with it and returns whatever is there, which then subscripts
touchstate. The resulting id is subscripted with no range check of its
own.
Check the port the way the sibling handlers do, drop an event that
names a pointer it does not carry, and skip an id outside touchstate.
Decoding the action once outside the loop drops three
AMotionEvent_getAction calls per pointer and retires the unused
raw_action copy along with them.
AInputQueue_preDispatchEvent hands the event to the IME and returns
non-zero to say so; the caller has to drop it there, and the framework
puts it back in the queue if the IME leaves it unconsumed. The default
poll path reads source, type and keycode off it afterwards and still
routes key events for non-keyboard devices, so an event owned by
another thread is read and acted on, and a soft keyboard press reaches
the core as a button at the same time.
Abandon the event where the gingerbread path already does, which also
lets the branch structure and finishEvent stop tracking the flag.
onInputQueueDestroyed hands a NULL down through APP_CMD_INPUT_CHANGED,
and a LOOPER_ID_INPUT raised before that command was serviced is still
delivered in the same pollOnce batch - detachLooper does not retract an
identifier the looper has already handed out. Both poll paths take that
pointer straight into AInputQueue_hasEvents/getEvent, while
android_input_discard_events guards it for exactly this reason.
Guard the queue, and the app pointer it hangs off, in both paths.
Key the display_lock acquisitions and the scale readback on
VIDEO_FLAG_THREAD_WRAPPER_ACTIVE. What makes video_st->width/height and
the scale pair shared state is a second thread driving the video driver,
so the wrapper's presence is the predicate; the session's threading mode
reads false from SET_HW_RENDER onwards while that thread is still
running, leaving the reads and writes unsynchronised.
video_driver_build_info() keeps caching the answer in a local so its
closing unlock still pairs with the lock it took, and the scale readback
now reuses that same local rather than recomputing a different
predicate mid-function.
display_lock is allocated unconditionally in video_driver_lock_new(), so
the existing null checks stay as they are.
Convert video_driver_build_info()'s userdata, both texture entry points
and video_driver_get_ident() to VIDEO_FLAG_THREAD_WRAPPER_ACTIVE alone.
Each of these asks who owns the render context, which the wrapper's
presence answers; the session's threading mode does not.
The two disagree between a core calling SET_HW_RENDER and the video
driver reinit that follows. video_info->userdata matters most: ozone_frame
and gfx_widgets_frame read it and pass it to dispctx->draw(), which casts
it to the concrete driver type, so a thread_video_t* returned there is
read at the wrong offsets - on Vulkan that reaches the vendor driver as a
garbage device or command buffer.
The texture paths forward the same answer to the drivers, where GL, GL3
and Metal use it to decide whether to marshal the upload, and Vulkan uses
it to keep a texture destroy from racing command buffer recording.
Add video_driver_thread_wrapper_active() and route the three font
dispatch decisions through it: both video_thread_font_init() call sites
and the renderer release. Font atlases are GPU resources owned by
whichever thread drives the driver, so the question that decides where
the work runs is whether the wrapper is installed.
The previous tests read is_threaded together with
!video_driver_is_hw_context(). Both go false the moment a core sets
SET_HW_RENDER, which is before the video driver is reinitialised, so a
widget layout landing in that window built and freed font textures on
the main thread while the wrapper's thread was still rendering. The
recorded font->is_threaded goes stale the same way by release time.
video_thread_font_init() already refuses when the wrapper is absent, and
the callers treat that as an ordinary init failure.
Gate video_driver_get_ptr()'s unwrap on VIDEO_FLAG_THREAD_WRAPPER_ACTIVE
alone. Whether the session should be running threaded video is a
different question from whether the wrapper is installed right now: a
core setting SET_HW_RENDER answers the first one 'no' while the previous
session's wrapper is still up, and video_st->data stays a
thread_video_t* until the video driver is reinitialised.
Every caller casts the result to a concrete driver type or hands it to
font init, so a wrapper handle returned raw is read at the wrong offsets
- on Vulkan that reaches the vendor driver as a garbage VkDevice.
video_thread_get_ptr() already falls back to video_st->data when the
wrapper is absent, so no other state changes.
Take the threaded-video idle barrier before RARCH_MENU_CTL_DEINIT runs
the outgoing driver's context_destroy(). That handler frees the textures
and fonts an in-flight frame is still drawing with, and the menu renders
on the video thread under threaded video, so the two must not overlap.
driver_uninit() and the widget toggle-off path already take this barrier
for the same reason; this is the third context_destroy() call site and
the only one reaching it without one. It is a no-op unless threaded
video is active.
Everything past the rectangle the core last wrote is already blank, so
the only pixels a shrink can strand are the ones between the old
rectangle and the new one: a strip down the right of the new width and
a strip along the bottom of the new height. Uploading those two leaves
the texture in the same state as blanking all of it, and a 384x288
core dropping to 320x200 moves 182 KiB instead of a megabyte.
Clamp both rectangles to tex_w and tex_h first. A core may hand over a
frame larger than the geometry it declared - nothing above the texture
bounds was ever written, and such a width must not come back as the
width of a read out of empty_buf. Rows are packed tight and each
alignment describes its own strip, so both uploads read exactly the
span they cover.
Every texture in the ring comes up blank from gl2_init_textures and
stays that way outside the rectangle the core last wrote, so a frame
that grows needs nothing cleared: gl2_copy_frame covers the difference
on the same call, and what lies beyond is still the blank the previous
shrink left. Only a shrink can strand pixels of the old frame where
edge filtering and clamping still reach them.
Track that and skip the upload otherwise, and start the ring at zero
so the opening frames grow into empty textures rather than each paying
for a full-texture clear. That upload is tex_w * tex_h at base_size -
a megabyte for a core with 384x288 geometry, and it lands on the crop
and region toggles that change geometry mid-run.
gl2_update_input_size uploads a full tex_w * tex_h rectangle out of
empty_buf whenever a core reports a new frame size. At 32bpp that read
covers the allocation exactly to its final byte, while the unpack state
describes the frame instead of the rectangle: the alignment comes from
the frame width at a hardcoded four bytes per pixel, and the row length
is whatever the last upload left set. Either can widen the rows the
driver walks, and a driver that copies the source in blocks has nothing
to spare past the last row - a fault inside glTexSubImage2D once the
allocator guards large mappings, which the Android one does.
Derive the alignment from tex_w * base_size, pin the row length to the
upload width where the extension is available, and keep a spare row in
empty_buf so the tail of the copy stays inside the mapping.
menu_driver_displaylist_push() reads an entry back out of the menu
stack once menu_driver_displaylist_push_internal() has returned, to
find the deferred-push callback. Since that call is free to reach
RARCH_MENU_CTL_DEINIT - a driver switch, a language change, a video
reinit - and menu_list_free() every menu_stack[i] before menu_list_new()
allocates replacements, the stack is re-derived from menu_st at that
point rather than reused from entry. generic_menu_entry_action() takes
the same precaution one frame up.
The menu_stack lookup on entry tolerates a NULL menu list, matching the
lookups the callers perform.
The staging buffer is sized once at thread_init from the maximum
geometry the core declared, and a core that later hands over a larger
frame than that was copied straight past the end of it.
video_thread_frame() now derives how many rows the allocation can hold
from the stride it is about to write and publishes at most that many.
The clamp is applied before the copy rather than inside it, because the
worker renders thr->frame.height out of the same buffer whether or not a
duped frame brought pixels with it.
EGL bindings are per-thread, and under threaded video the context is made
current on the video worker. Both surface entry points now dispatch there
through video_thread_texture_handle(), which calls straight through when
the wrapper is inactive or when the caller already is the video thread.
Issued from the command handler, the eglMakeCurrent() inside
egl_destroy_surface() released the calling thread's binding rather than
the worker's, leaving the destroyed surface current on the worker, and
the create_surface() that followed a resume bound the same context on a
second thread. Drivers are entitled to fault on the next draw.
A duped frame arrives as data == NULL, and the pointer from the previous
frame is kept so the menu and the paused view still have pixels to
re-present. The dimensions published alongside it are the new frame's,
so a core that changes geometry and then dupes leaves the cache
describing more pixels than the retained buffer holds - and
video_driver_cached_frame() then hands that tuple to the video driver,
which reads height * pitch bytes off the end of it.
The pointer is now dropped when a duped frame arrives with dimensions
that do not match it. The dimensions still publish unconditionally, for
the viewport and aspect-ratio readers that need the live geometry, and a
replay with no data re-presents the frame the driver already holds.
file_list_pop() returns alt, userdata and actiondata alongside path and
label, matching file_list_deinitialize_internal(). The two helpers clear
the slot as they go, so a caller that has already released userdata or
actiondata itself passes through them - menu_list_pop_stack() does,
via the driver's list_free hook, which is what left alt as the one
field with no owner: it is set by file_list_set_alt_at_offset() and
nothing released it, so every menu stack pop of an entry carrying an
alt leaked the string.
The directory_ptr read at the end is gated on the backing array, which
a list that has never held an entry does not have.
android_app_set_window() and android_app_set_input() take a ticket from a
command sequence counter for every command they post, and block until the
app thread has advanced the completion counter to it. The framework reuses
ANativeWindow and AInputQueue addresses, so the handle a callback asked for
is not a sound completion signal: it can already match while the command
sits in the pipe, which lets the callback return - and the surface go live
- with a teardown still queued behind it. Replacing a live window posts two
commands, and the ticket covers both.
android_app_write_cmd() reports a failed write so that no ticket is taken
for a command that was never queued, and both ends of the pipe retry when a
signal interrupts them.
ozone_change_tab and both xmb_list_cache relabel sites require an entry
on the menu stack to write to, so each checks the stack depth before
indexing. stack_size - 1 on an empty stack indexes SIZE_MAX, and the
free() that follows releases whatever that read returns. MaterialUI's
equivalent in materialui_switch_tabs already gates on the same
condition.
The menu_stack lookups in the same expressions are NULL-tolerant now
too, matching the selection_buf lookup beside them.
The buffer gl_glsl_set_vbo() keeps is a copy of the last upload, held so
that an identical set of coords can skip glBufferData(). A failed grow
therefore costs a redundant upload and nothing else: the data goes to the
driver straight from the caller, and the cache is marked empty so the
length comparison in gl_glsl_set_attribs() short-circuits before reading
it and the next call retries the grow.
realloc() leaves the existing allocation intact when it fails, so the
buffer is kept rather than dropped.
sthread_setname applies a short label to the calling thread, so the
frame loop, video, audio and task worker threads identify themselves in
debuggers, tombstones and ANR reports. Threads that go unlabelled
inherit the name of whichever thread created them, which on Android
leaves an ANR report holding several identically-named entries and a
tombstone naming no thread at all.
Linux and Android go through prctl, which takes the name as a buffer
and so truncates at the kernel's 16-byte limit rather than rejecting an
over-long label; Apple and the BSDs use their respective
pthread_setname_np spellings. Platforms with no naming interface get a
no-op.
android_run_events handles every ident ALooper reports, dismissing
queued input rather than discarding the wakeup. The queue's descriptor
stays readable until each event is finished, so an ident that is not
LOOPER_ID_MAIN would otherwise spin the window and permission waits in
platform_unix.c and leave the window unresponsive to the input
dispatcher for as long as the wait lasts.
Events are finished unhandled because the input driver is not up yet;
the framework's fallback handling is what should see them.
APP_CMD_PAUSE arrives on the command pipe, which is drained by
android_input_poll(). That is the input driver's poll callback, so a core
using late polling reaches it from inside retro_run() - and the save it
triggers reads core memory through core_get_memory() and reparses and
rewrites the whole config, on the core's own stack mid-frame.
The command handler now records that a save is wanted and
runloop_iterate() performs it, which is the nearest point that is
unambiguously outside the core. The cost is up to one frame of latency,
well inside the window Android allows after onPause() returns.
The 'switch' and 'reset' developer hooks now read from the app's private
data directory rather than /sdcard, and are built only when
HAVE_ANDROID_LIFECYCLE_HOOKS is defined, which no shipped configuration
defines. Shared storage is writable by anything holding all-files access,
so a hook there lets another app choose what runs in this process, with
this process's permissions; the private directory is 0700, and an
ownership check on shared storage cannot substitute for that because the
FUSE layer synthesises the uid it reports.
Both hooks run on the app thread - 'switch' from the APP_CMD_START
handler after the acknowledgement, 'reset' from android_app_destroy() -
so a script that takes its time delays the frame loop rather than a
lifecycle callback the framework is timing.
onStart/onResume/onPause/onStop hand the command to the app thread and
wait for it to take it. The app thread reads the pipe from the looper
between frames, so it answers when it is next idle, and a wait with no
upper bound is a UI thread pinned for as long as whatever the app thread
is doing takes.
Three seconds is the ceiling, well inside the five ActivityManager allows
before it reports an ANR. activityState is a notification the app thread
writes and nothing else reads, so a timed-out caller returns having lost
nothing. The window and input-queue handshakes stay unbounded: those
publish object lifetimes the app thread still holds.
setSustainedPerformanceMode() is reachable from the native thread -
config_load_file() applies it on every config and override load - as well
as from onResume(). Window attributes still have to be set on the UI
thread, so the work is posted there, but the caller no longer waits for
the post to run.
Waiting was a lock cycle: the UI thread sits in onPause() waiting for the
native thread to acknowledge APP_CMD_PAUSE, so a native thread parked on
the UI thread's looper is a deadlock that ActivityManager reports as an
ANR.
The header carried a 16-bit leading-zero count and a 32-bit trailing
one, which is not the set a caller needs, so five files each wrote
their own and none of them included this one. They disagreed on the
part that is easy to get wrong: what a zero argument answers, which
the compiler builtins leave undefined and _BitScanReverse reports
through its return value rather than its output.
compat_clz_u32(), compat_clz_u64(), compat_ctz_u64() and
compat_highbit_u32() join the two already here, all four answering the
width for zero, and rh264, rh265, rzstd, deflate and glslang_util take
them. _BitScanReverse64 and _BitScanForward64 are 64-bit only, so the
32-bit Windows build splits the word rather than losing the path.
Checked against an independent bit-at-a-time reference over every
32-bit and every 16-bit input, and over every single-bit and adjacent
pair value at 64 bits, three times: through the compiler builtins,
through the software fallback with every builtin forced off, and
through the _BitScan arithmetic. The replaced forms answer identically,
so mip level counts, zstd table construction and the deflate match
extension do not move.
The first four digest bytes are copied into a uint32_t, which keeps the
lookup free of the alignment and aliasing constraints that a cast on the
char array carries. Declarations move to the top of the block so the
function builds as C89.
eglGetProcAddress_functionNamePatch() resolves Piglet exports through
the SHA1() entry point, so patches.c needs sha1.h, which maps that
entry point onto libretro-common's SHA1Digest().
GetLogicalProcessorInformation gives both the core count and which
processors share a core, so Windows answers
cpu_features_get_core_amount_physical() from the RelationProcessorCore
entries and ranks an SMT sibling behind the processor it shares a core
with rather than falling back to ascending order.
The export arrives in XP SP3 and Server 2003, so it is resolved through
GetProcAddress rather than linked: on Win9x, NT 4 and 2000 the lookup
comes back empty, as it does on WinRT and Xbox where the call is not
reachable at all, and each caller keeps the answer it had before. The
Ex form carrying EfficiencyClass is Windows 7 and later and would rank
an efficiency core below a performance one, which this does not
attempt.
Compiled for x86 and x86-64 at _WIN32_WINNT 0x0400 and 0x0501, at the
toolchain default, and under LEGACY_WIN32 and __WINRT__. The mask walk
and the ranking were exercised against synthetic topologies covering
SMT, no SMT, cache and NUMA entries interleaved with core entries, and
a set carrying no core entry at all.
A caller pinning a thread wants the strongest processor still free,
which needs the operating system identifiers ranked rather than
counted. cpu_features_get_processor_order() ranks by descending core
performance and places an SMT sibling behind the processor it shares a
core with, so taking the first few entries takes the best few.
Linux ranks on the maximum frequency the kernel publishes per
processor and groups siblings by the sibling list, the pair that
distinguishes a prime core from a little one. Elsewhere the order is
ascending, which still names each processor once.
Every processor is ranked before any is returned: ranking only as many
as the caller asked for would sort a set chosen by identifier and hand
back the little cluster on the layouts that number it first.
An empty SMB Share Name makes smb://<server> a browsable location: opendir
enumerates the disk shares the server exports over srvsvc and returns them as
directory entries, so a share is chosen while browsing rather than typed in
beforehand. Hidden, administrative, printer, device and IPC entries are left
out of the listing.
The share a path resolves against is now derived per path, taken from the
configured share when there is one and from the leading URL component
otherwise, and the pool binds to it through the existing connection key.
Enumeration runs on a context of its own because srvsvc needs a tree connect
to IPC$, and drives smb2_share_enum_async through socket_poll since libsmb2
offers no synchronous entry point.
cpu_features_get_core_amount() answers with logical processors on
every backend it has -- dwNumberOfProcessors, _SC_NPROCESSORS_ONLN,
HW_AVAILCPU, the cpu/present mask -- so its header now says so, and
work that wants a core to itself has
cpu_features_get_core_amount_physical() to size by instead.
Darwin reads hw.physicalcpu. Linux counts distinct SMT sibling groups
under /sys/devices/system/cpu, each of which is one core, which also
answers for ARM where /proc/cpuinfo carries no core count. Anywhere
the distinction cannot be drawn the thread count stands, being never
larger.
A second entry point rather than a changed one: the existing name is
load-bearing for callers sizing thread pools, and redefining it would
move them silently.
The sublabel states that a changed server, share, credential or auth mode is
picked up the next time a share is browsed, and that content already running
from a share holds its connection until it is closed.
Only the English source row changes; the other intl/msg_hash_*.h copies keep
their current text until Crowdin catches up.
Both accumulate in YMM, so each answers only where the operating
system preserves that state, which is the condition RETRO_SIMD_AVX
already establishes and which the two now share. FMA3 comes from
CPUID.(EAX=01H):ECX[12] and hw.optional.fma under Darwin; FMA4 is a
separate AMD encoding at CPUID.(EAX=80000001H):ECX[16] rather than a
superset, so it takes its own bit.
frontend_driver_parse_drive_list() enumerates local volumes only, so the root
of every file and directory browser gains an entry for the configured share.
That reaches the scan and manual-scan directory pickers, which is what
building a playlist from a share needs.
menu_displaylist_build_smb_root() becomes the single place the
smb://<server>[/<share>][/<subdir>] URL is assembled, and rejects a truncated
result rather than returning a partial path.
Loop counters are declared at block scope, the auth-mode locals move to the
top of smb_init(), and C++ style comments become block comments so the file
builds under -std=c89 -pedantic.
The pool teardown helper becomes static smb_reset(), keeping a symbol named
reset out of libretro-common's exported namespace.
A context stays bound to one share for the lifetime of its tree connect, so
the server, share, credentials, workgroup, auth mode, timeout and connection
count the pool was built from are captured in a snapshot. smb_init() compares
the current settings against that snapshot and brings the pool down and back
up when they differ, making a share change visible on the next browse.
Recycling is held off while any stream or directory handle is outstanding,
since those carry raw context pointers. Healing a dead context reads the same
snapshot so a replacement joins the share the rest of the pool is on.
Twelve writes in runtime_log_get_last_played_str() size themselves as
len - _len, and _len is whatever strlcpy() returned for the label at
the top, which is the length of its source rather than the length it
wrote. A translation longer than the buffer therefore turns every one
of those into a very large size_t with the destination already past
the end, so the label offset is clamped once where it is taken.
The AGO branch holds two octets back and so needs room for them and
for the separator before it writes either, and the helper it calls
takes its own offset from snprintf(), which reports the same way.
strlcpy() and snprintf() both report the length they were asked for
rather than the length they wrote, so a running offset built from
their return values passes the buffer size as soon as one append
truncates, and the next size argument, computed as len - _len, wraps
to a very large size_t with the destination already past the end.
Each of these appends is now made only once it is known to fit:
- the menu title join, which eight search terms of
MENU_SEARCH_FILTER_MAX_LENGTH overrun by four times over, and which
is reached by typing into the search box;
- the core licence sublabel, which takes the bounded join in
string_list rather than its own copy of one, and which wrote its
": " separator at an offset it had not checked;
- the subsystem sublabel, over content names;
- the shader preset texture list, over names a preset supplies;
- the disc drive list, whose offset comes from a translated format
string.
strlcpy() reports the length it was given rather than the length it
wrote, so an element that does not fit carries the running offset past
the buffer size, and the next iteration computes len - _len as a very
large size_t with a destination pointer already past the end. Checking
once before the loop does not hold across it, and the _special variant
was not checking at all. Each element and each delimiter is now
appended only once it is known to fit, and the one that does not fills
what is left before the join stops.
The joins reachable this way carry names from outside the frontend:
subsystem content paths off the command line, and the category, author,
licence, permission and firmware lists a core info file supplies.
sha1_calculate() and intfstream_crc_step() both walk a whole file
handed to them by a scan, so they ask for it in the same 256 KB
blocks rather than each picking a size.
strlcpy() reports the length it was given rather than the length it
wrote, so adding its return after it truncates carries the running
offset past the buffer size and every later size argument, computed as
len - _len, wraps to a very large size_t while the destination pointer
is already past the end. The names now come from a table and each one
is appended only once it is known to fit.
An empty mask writes a terminator rather than nothing, since the
caller in retroarch_main_init() prints the buffer it passed without
initialising it, and that buffer is wide enough for every name at
once.
MBEDTLS_SHA1_ALT and MBEDTLS_SHA256_ALT are the hook mbedtls provides
for an integrator supplying its own digests, so sha1.c and sha256.c
keep only their one-shot wrappers and everything above them, the md
layer and X.509 and the TLS transcript included, hashes through
lrc_hash and its dispatch onto SHA-NI or the ARMv8 SHA instructions.
sha_alt.c carries the bodies. The context is the libretro-common state
in each case, so cloning a stream is an assignment, and the raw block
entry point leaves the message length alone for a caller driving its
own padding. SHA-224 is a starting state on the SHA-256 path.
Checked against mbedtls' own implementation over 840 runs covering
lengths across the block and padding boundaries, input split at 1, 7,
63, 64, 65 and 1000 octets a call, and a clone taken at five offsets
including mid-block: byte-identical for the one-shots, the streaming
path, the cloned stream, the md layer for all three digests, and
HMAC-SHA256. Identical again on aarch64, and matching an independent
reference throughout. Clean under ASan, UBSan and LSan.
The one-shot entry points stay the short way to a digest; a caller that
feeds one in pieces, copies a stream mid-digest, or drives the padding
itself now has init/update/final, a struct it can assign to clone, and
a raw block entry point. Both states live in the header so a caller can
hold one by value, laid out largest member first.
SHA-224 comes with them: it is SHA-256 under a different starting state
publishing seven words of the eight, so it costs an initial vector and
a count rather than an implementation, and the hardware block functions
serve it unchanged.
Verified against reference digests over 140 runs spanning the block and
padding boundaries with the input split at 1, 7, 63, 64 and 1000 octets
a call, plus the raw block entry point against a full-block update and
an assigned state against the stream it was copied from. Clean under
ASan, UBSan, LSan and TSan.
sha1_calculate() prefers a mapping and copies only where the platform
gives it none, and on those platforms a read is a round trip rather
than a memcpy out of the page cache. A 64 KiB block asks for the same
bytes in a sixteenth of the calls.
Measured on the copying branch over a 64 MB file: the read loop alone
goes from 9123 MB/s to 14855 MB/s and from 15626 calls to 978, and the
whole function from 991 MB/s to 1079 MB/s, the digest now being the
limit rather than the reads. The gain is larger wherever a read costs
more than it does against a warm page cache.
A kernel queue overflow (IN_Q_OVERFLOW) means events were dropped, so
something may have changed without a record of it; the mask test
silently discarded it. Report a change instead, matching the
notification-buffer-overflow handling in the Windows branch, so a
poll can be relied on for conservative cache invalidation: a consumer
may see a spurious change after an event storm, but never misses one.
MD5 exists in this build for the TLS 1.0 and 1.1 PRF, so turning those
two protocol versions off takes mbedtls_md5 out with them and leaves
one MD5 compiled in the tree. Certificate verification is unaffected:
mbedtls_x509_crt_profile_default accepts SHA-2 and, where enabled,
SHA-1, and has never accepted MD5 for a signature. CBC record
splitting goes too, being a TLS 1.0 mitigation.
The default preset floors the client at TLS 1.0 whichever versions are
compiled in, so net_socket_ssl_mbed.c now names 1.2 as the floor and
the handshake refuses older versions rather than reaching a PRF that
is not there.
buildbot.libretro.com, lobby.libretro.com and retroachievements.org
answer a TLS 1.0 ClientHello with a protocol_version alert and all
serve TLS 1.2.
SHA1Digest() takes the same path as sha1_calculate(), so Apple reaches
the SHA-1 in this file and the ARMv8 SHA-1 instructions behind it
rather than a second implementation in CommonCrypto. lrc_hash.h keeps
the CommonCrypto include for MD5.
Editing the originally loaded preset, or any intermediate in a
#reference chain, now live-reloads the shader; previously only the
root preset and the pass sources were watched, so live-editing a
simple preset did nothing until the root was touched.
Watch construction moves out of the root-config parser into a helper
called at both load sites, fed from the reference path list the
loader already gathers for override application; the parser loses its
watch parameter and the parse loop is no longer duplicated per watch
mode.
sha1.h now presents SHA1() over SHA1Digest(), so the piglet export
lookup in patches.c hashes through libretro-common and picks up the
hardware path on a CPU that has one. The trailing NUL the caller's
21-byte buffer expects is still written.
The two implementations produce the same digest for every length from
0 to 300 octets, so the export hashes the lookup depends on do not
move.
md5.h now presents MD5Init/MD5Update/MD5Final over MD5_CTX, the way
rcheevos already reaches the same implementation, so hmac-md5.c and
ntlmssp.c hash through libretro-common and the tree carries one fewer
copy of the algorithm. The header keeps the includes its callers reach
it through.
mbedtls and bearssl keep theirs: both publish MD5 through their own
abstraction, mbedtls_md5_info feeding md_wrap.c and br_md5_vtable
feeding the TLS 1.0/1.1 PRF in md5sha1.c, including chaining-state
export, so redirecting them would mean carrying a fork of each.
Clang pulls the per-feature sub-headers of immintrin.h in only when the
baseline already has the feature on targets defining __SCE__, and
shaintrin.h refuses to be included on its own, so a Sony toolchain
aimed at a CPU without the instructions leaves the intrinsics
undeclared however the function carrying them is attributed. The
capability test now asks for that, and such a target takes the scalar
path along with everything else lacking SHA-NI.
Verified against clang with __SCE__ defined, with and without the
feature macros, and on ordinary clang and gcc x86-64 where the path
stays compiled in. encoding_crc32.c includes wmmintrin.h directly,
which carries no such guard, and is unaffected.
File watching is filesystem functionality, not frontend-environment
functionality: it needs nothing from the frontend context, yet lived
as two frontend_ctx_driver_t slots that every platform file had to
carry and that buried three implementations inside the platform
monoliths. It is now a libretro-common module, retro_dirent-style:
one header, one TU with inotify, GCD dispatch-source and
ReadDirectoryChangesW branches plus a stub, behind an opaque handle -
file_watch_new / file_watch_poll / file_watch_free / _supported -
which also retires the teardown-through-NULL-list calling convention
and the path_change_data_t wrapper.
The three implementations move essentially verbatim with their
lifecycle guarantees intact; the Darwin branch converts its block
handlers to the _f function-pointer variants with the watch entry as
dispatch context, so the TU stays plain C, and the inotify branch's
watch-descriptor list becomes a plain array sized to the path count.
The two driver slots, their wrappers and the capability query leave
the frontend layer, and the shader file watcher talks to the module
directly.
lrc_hash.h is where MD5_CTX and the three entry points come from, and
utils/md5.c implements them against it, deferring to CommonCrypto on
Apple. The unit test takes the same header as every other caller.
The two declarations had drifted apart: one switched to CommonCrypto on
Apple and the other to OpenSSL under HAVE_OPENSSL, so a translation
unit that took both would not compile, and the test built against a
struct layout the implementation did not share.
lrc_hash.c carries the SHA-1 the tree actually uses, reached through
SHA1Digest() and sha1_calculate(), and it is the copy that gets the
hardware block functions and the batched input path. utils/sha1.c is a
separate implementation of the same algorithm that no makefile, no
griffin unit and no header names, so nothing can reach it without
including the .c directly.
Callers wanting SHA-1 take lrc_hash.h.
Fills the two frontend slots that have been NULL since the watch API
was introduced, so video_shader_watch_files works on Windows instead
of silently doing nothing. Windows watches directories rather than
files, so the watched paths are grouped by parent directory and each
directory gets an overlapped ReadDirectoryChangesW; the per-frame poll
is a zero-timeout wait on the completion events, with completed
notifications filtered against the watched basenames.
A rename onto a watched name counts as a modification, because
editors that save atomically surface the save as a rename rather than
a write; a notification-buffer overflow reports a change rather than
miss one; and every completion re-arms the watch, since an unre-armed
directory silently stops reporting. Idle cost is one event check per
watched directory per frame.
The bit count advances once per call and the octets move by memcpy into
whatever space the current block has left, which is the arrangement
sha256_chunk() already uses. Compression is where the work belongs, and
on a CPU with the SHA-1 instructions it is cheap enough that a per-octet
copy loop would set the throughput instead.
Measured on x86-64 with SHA-NI over a 16 MiB buffer: 307 MB/s to 1189
MB/s with the instructions, 116 MB/s to 167 MB/s without them. Verified
byte-exact on both dispatch arms across the block and padding
boundaries, and against an independent SHA-1 over a 500 KB file through
sha1_calculate's chunked path. Clean under ASan, UBSan, LSan and TSan.
Both digests compress a 64-byte block through a fixed round schedule,
which is exactly what the ARMv8 SHA1/SHA2 instructions and x86 SHA-NI
implement, so each block function gains a hardware form and the
dispatcher asks cpu_features_get() once per block. The scalar round
loop stays as the answer for a CPU whose probe comes back empty, and
for every target without either instruction set.
Nothing in tree passes +crypto on aarch64, so the ARM path compiles
under a function-level target attribute and is chosen at runtime, as
in encoding_crc32.c. SHA-NI enumerates both digests through one CPUID
bit. MSVC takes the scalar path either way, since neither the ARM64
nor the x86 intrinsics have been exercised there.
Verified byte-exact against SHA-1 and SHA-256 reference digests over
23 lengths spanning the block and padding boundaries, on x86-64 with
SHA-NI and on aarch64, with each dispatch arm exercised separately.
Clean under ASan, UBSan, LSan and TSan; no frame over the
libretro-common stack ceiling.
A message whose final block holds 55 octets has room for the 0x80 and
the eight length octets at 56..63, so it completes in the block it
started in. The capacity test runs after the index has advanced past
the 0x80, so 56 is the first value that overflows.
Digests for any input of length 55 mod 64 change, and the boundary is
now covered at 55, 56 and 119 for both digests.
RETRO_SIMD_SHA1 and RETRO_SIMD_SHA256 cover FEAT_SHA1 and FEAT_SHA256 on
AArch64, read as 'sha1' and 'sha2' from the /proc/cpuinfo Features line
and as hw.optional.arm.FEAT_SHA1 / FEAT_SHA256 under Darwin. x86
enumerates both digests through the single SHA-NI bit in
CPUID.(EAX=07H,ECX=0):EBX[29], and Darwin lists it as SHA in
machdep.cpu.leaf7_features, so both bits report together there.
Two bits rather than one because AArch64 may implement either digest
without the other, and a caller selecting a code path needs the answer
for the digest it is about to compute.
RETRO_SIMD_SHA512 covers FEAT_SHA512 on AArch64, read as 'sha512' from
the /proc/cpuinfo Features line and as hw.optional.arm.FEAT_SHA512 under
Darwin, and the x86 SHA512 instruction group enumerated by
CPUID.(EAX=07H,ECX=1):EAX[0]. Leaf 7 EAX gives the highest sub-leaf the
part implements, so it gates the sub-leaf 1 query.
The two are separate instruction groups reached through one bit because
what a caller wants to know is whether SHA-512 has hardware behind it.
AArch64 exposes the AES instructions as 'aes' in the /proc/cpuinfo
Features line and as hw.optional.arm.FEAT_AES under Darwin, so
RETRO_SIMD_AES now answers for ARM as well as x86. They belong to the
optional Cryptographic Extension, which an implementation may omit or
hold in reset, so the bit stays a runtime question on 64-bit ARM.
The Features line is a space separated list in which several names are
prefixes of others: 'aes' sits inside 'sveaes', 'sha3' inside 'svesha3',
'asimd' inside 'asimddp'. Bounding the match on both sides makes a query
answer for the feature asked about rather than for whatever longer name
happens to share its spelling.
The System Information CPU Features line enumerates the mask that
cpu_features_get() returns, so it names PCLMUL and LZCNT on x86 and
CRC32 on aarch64. PCLMUL and CRC32 are the two bits that select the
carry-less multiply and ARMv8 implementations in encoding_crc32.c, so
the line reports which of the three CRC paths a device takes.
Drives the task path of gfx_widgets.c -- alternative_look, title
churn through msg_new, and deinit with live tasks -- under ASan,
UBSan, LSan and TSan.
The shader file-watch poll runs once per frame while watching is
enabled, and paid for it with a 32 KiB stack buffer zeroed on every
call plus an fsync of the changed file - through a throwaway
open/close - on every match. The buffer is now 4 KiB, owned by the
watch state and allocated once, never cleared (the read length bounds
what is parsed), and the fsync is gone: the page cache already
guarantees a re-read sees the new contents, and flushing to disk is
the writer's durability concern, not the watcher's. The wd-to-path
lookup existed only to feed that fsync and goes with it.
Measured on a live inotify watch: the idle poll - the per-frame case -
drops from 1193 ns to 175 ns, and the match-path poll from 594 us to
56 us on fast storage; on device flash the removed fsync is the
difference between a frame-time spike on every watched-file save and
none. The poll's frame drops from 32880 bytes to 32.
The derivation pass received the intent object where every helper
expects the activity, so a launch without extras - the launcher alias,
any direct start - looked Context methods up on android.content.Intent
and derived nothing; the version-code helper additionally issued its
second method lookup with the first lookup's NoSuchMethodError still
pending, which aborts the runtime. The helpers now receive the
activity instance, and every throwing JNI call in the version helper
is followed by its own exception check before the next one, matching
the rest of the derivation block.
NativeActivity resolves android.app.lib_name and android.app.func_name
from the launched component's own ActivityInfo, and an activity-alias
does not inherit meta-data from its target - so a launch through the
alias fell back to loading libmain.so and threw before onCreate
completed. The two entries are repeated on the alias in every
variant, matching the target activity byte for byte.
With the launcher alias starting the native activity directly and
every launch parameter derived natively, the Java launcher code has no
remaining callers: the three MainMenuActivity variants, the unused
RetroActivityIntent, and the config-path, intent-extra and audio
helper methods in UserPreferences are removed, leaving getPreferences
as its only member. CoreSideloadActivity launches the native activity
itself with just the ROM and LIBRETRO extras; everything else is
derived, and the activity resolves the storage permission on its own
like any direct launch.
The launcher entry is now an activity-alias carrying the old Java
launcher's component name and intent filters, targeting the native
activity, so a cold launch skips the Java trampoline entirely - one
full Activity lifecycle and a startActivity round trip less on every
start - while pinned shortcuts, leanback rows and anything else
resolving the old component name keep working unchanged. Launcher
starts carry no extras, which is the direct-launch path: parameters
are derived from the application context and the storage permission is
resolved by the activity itself, gated ahead of any filesystem work.
The hardware volume keys bind to the game audio stream from
RetroActivityCommon now that the Java launcher no longer runs.
Explicit intents that target the trampoline by class name resolve
through the alias; the Java launcher code itself is no longer
reachable from the manifest and can be removed once this has soaked.
The native thread now waits, right after the window wait in
frontend_unix_init, until the activity signals the startup storage
permission as resolved through a permissionsResolved JNI upcall, so
config probing, storage tests and asset extraction never run against
an unsettled permission state. The wait pumps lifecycle commands the
same way the window wait does, so a pause or rotation while a
permission dialog is up cannot block the UI thread's synchronized
window teardown; the looper handoff is published under the app mutex
and ALooper_wake is sticky, so a signal cannot be lost, and an
activity torn down mid-flow releases the gate through the normal
destroy path.
RetroActivityCommon resolves the state after super.onCreate: a launch
through the Java launcher (recognized by its CONFIGFILE extra), a Play
Store build, or an already-granted permission signals immediately and
never waits. Only a direct launch missing the permission asks - the
all-files-access dialog on Android 11+, runtime storage permissions on
6 through 10 - and every outcome including denial and cancel releases
the gate, with denied storage falling back to the app-private paths
the storage probe already selects.
Every device- and installation-derived value the Java launcher passes
as an intent extra can now also be read natively: DATADIR and APK from
ApplicationInfo, SDCARD/EXTERNAL through the same scoped-storage probe
the launcher uses, IME from Settings.Secure, AUDIO_RATE/AUDIO_FRAMES
from AudioManager, VERSIONCODE from PackageManager, and CONFIGFILE by
the launcher's global-config probe order with creation deferred to the
first save. Extras always win; each derivation runs only for a value
the intent left unset, so every existing launch path is unchanged.
A launch of the native activity without the full extra set - a
third-party frontend firing a bare intent, or a future direct launch -
now comes up with the same paths, bundle extraction and audio defaults
as one through the Java launcher. The per-core config selection tied
to the legacy Java preference is intentionally not derived; the extras
path preserves it.
Rumble amplitude changes arrive per frame during effect ramps, and the
native side calls doVibrateUSB for every controller before falling
through to the VibratorManager path. Each Java call resolved the
device from scratch: an InputDevice binder lookup plus a full
UsbManager.getDeviceList() marshal on the USB probe - including for
controllers that will never match - and another device lookup plus
getVibratorManager() on the API 31 path, with per-change logging and a
fresh AudioAttributes on top.
The USB probe now caches its resolution per input device id including
negative results, the API 31 path caches the VibratorManager and
vibrator ids per device, and both are invalidated from the
InputDeviceListener callbacks, with a stale-manager exception dropping
the entry and taking the single-vibrator fallback. AudioAttributes is
built once and shared, and the remaining rumble log fires once per
device at cache fill rather than per amplitude step. Steady-state
cost of a rumble change is the vibrate() or HID transfer itself.
A forced CMD_EVENT_CORE_INFO_INIT (info-file download, core
install/delete/restore) goes through CMD_EVENT_CORE_INFO_DEINIT,
which frees the current-core entry alongside the list it borrows
from, and nothing re-established the entry for a core that is still
running: presence integrations and the AI service lost the core's
identity for the rest of the session, and the savestate support
gate fell back to the permissive no-core default, un-gating
features the core's info file blocks.
After the list is rebuilt the entry is now resolved again through
core_info_load() when a core path is set, restoring the info-file
answer (or the reset-and-stamped fallback when the fresh list has
no entry). core_info_deinit_list() documents the borrow invariant
its callers must uphold.
video_notch_write_over_enable and input_auto_mouse_grab are pushed to
the activity through a setWindowSettings JNI call on config load and
whenever either setting changes, following the sustained-performance
pattern. The Java callbacks read the pushed fields, so they always
see the live values - a mid-session toggle now takes effect
immediately, where reading the config file could only ever see the
last saved state - and the notch mode is re-applied on the spot
through an overridable hook instead of waiting for the next resume.
With no readers left, the Java-side ConfigFile parser and the cached
accessor in UserPreferences are removed; config parsing now happens
in exactly one place, config_file.c.
The emitted mobile-lane table no longer carries the bundle asset keys
that are supplied from the launch intent, so the reference is
regenerated to match.
When the loaded core has no info-list entry, core_info_load() left
the current-core entry either as the zeroed shell from
core_info_init_current_core() or holding a previously loaded core's
data, since core_info_list_get_info() does not touch the output on a
miss. Consumers of core_info_get_current_core() then read a NULL
path or another core's identity.
On a miss the entry is now reset and stamped with the requested core
path, held in state-owned storage so nothing new needs freeing:
every pointer member of the entry is borrowed from the info list,
and the savestate support level keeps the init-time default.
materialui_render_menu_entry_savestate_list passed its entry matrix
to materialui_draw_icon without ever running the rotate that every
sibling render function performs; the display drivers consume
draw->matrix_data verbatim as the MVP, so savestate-list icons were
drawn through an uninitialised matrix on every driver that does not
handle its own transform. The function now rotates the matrix like
its siblings, and the declaration is zero-initialised so the
transform-handling and no-context paths hand deterministic data.
RetroActivityFuture reads native-owned settings from the config file
in callbacks that fire often - every window focus change and every
display cutout update - and each read was a full re-read and re-parse
of the file on the UI thread. UserPreferences.getConfigFile() now
serves those reads from a cached parse that is refreshed only when
the file's mtime or size changes, so the steady-state cost is a
stat(). The atomic native save replaces the file by rename, which
updates the mtime the check keys on.
The display context is NULL around a video driver reinit while
frames can still be submitted; the handles_transform probes in
gfx_widgets_draw_icon, ozone_frame and six materialui render
functions dereferenced p_disp->dispctx directly, the one unguarded
dereference in functions whose draws all check dispctx. Same class
and same shape as the xmb fix in d6ee7ca: gfx_widgets_draw_icon
bails without a context, the menu drivers skip the rotate and
zero-initialise the matrix the skipped rotate would have filled.
gfx_thumbnail_draw already rejects a NULL context at entry; its
probe now reads the checked local so the invariant is lexical.
config_file_write dumps to a temporary beside the target and renames
it over the target only once the write is complete and error-checked,
via filestream_rename for POSIX overwrite semantics on every platform.
An interrupted save - process kill, power loss, full disk - leaves the
previous file intact instead of a truncated one, and a save that hits
a write error now reports failure instead of silently replacing a good
config with a partial one.
The launch intent carries a VERSIONCODE extra with the app version
code; the frontend combines it with the existing APK and DATADIR
extras to set bundle_assets_src/dst/dst_subdir and
bundle_assets_extract_version_current directly, the same way the
Apple frontend supplies them. These four values are no longer bound
to the config file on Android: stale keys in existing configs are
ignored, and only bundle_assets_extract_last_version remains
persistent state.
The Java side stops parsing and rewriting retroarch.cfg at startup:
UserPreferences.updateConfigFile and the unused readback helpers are
replaced by putDeviceIntentExtras, which attaches VERSIONCODE and the
device-optimal AUDIO_RATE/AUDIO_FRAMES values to the launch intent
from every launcher path. ConfigFile itself stays, as
RetroActivityFuture still reads the config at runtime.
The launch intent carries AUDIO_RATE and AUDIO_FRAMES extras with the
AudioManager-reported optimal sample rate and buffer size. The
frontend stores them in g_defaults and config_set_defaults applies
them as the audio_out_rate and audio_block_frames defaults, so a value
saved in the config file still overrides them on load.
The icon-thumbnail path indexed the selection buffer with a
selection that can be stale against a list just rebuilt smaller, and
took the entry's node without a NULL check. The lookup is now
bounds-checked and a missing node leaves the thumbnail at the margin
position.
The display context is NULL around a video driver reinit while
cached frames can still be submitted; every draw in xmb_frame() and
xmb_draw_icon() already checks dispctx before use except the
handles_transform probe, which dereferenced p_disp->dispctx
directly. xmb_draw_icon() now bails without a context, xmb_frame()
skips the rotate, and mymat is zero-initialised so a skipped rotate
cannot copy indeterminate values into per-icon matrices.
ozone_set_header indexes the horizontal list twice with
categories_selection_ptr - system_tab_end - 1: once for the title of
a sidebar playlist tab and once for its header icon. Both are gated
on the list being non-empty but not on the index being inside it, and
the accessor does not bounds-check, so a selection pointer that is
stale relative to a refreshed horizontal list - a playlist removed or
hidden while a tab past it was selected - reads past the end and
dereferences the garbage. Same crash signature as the missed-match
read fixed in the previous commit.
Make the bound the branch condition; an in-range index implies a
non-empty list, so the separate size check folds into it. The
selection pointer is only known to exceed system_tab_end inside these
branches, so the icon branch keeps that comparison ahead of the
subtraction.
Play Console reports a SIGSEGV in ozone_set_header, reached from
ozone_populate_entries when a displaylist is pushed for an entry
selected under Main Menu -> Playlists. With the header icon in its
default Dynamic mode, the function scans the horizontal list for an
entry whose path matches the pushed playlist. Explore views are not
in that list (it is built from .lpl files only) and neither is a
playlist hidden from the sidebar, so the scan can legitimately find
no match and leave offset == size. The ternary that follows only
means to skip an empty list - its cast binds to the size operand, so
the condition is merely size != 0 - and on a missed match it indexes
one entry past the end through an accessor that does not
bounds-check. The garbage userdata pointer is then dereferenced for
its icon.
Resolve a missed match to no node by conditioning the lookup on
offset < size. The per-iteration pathname fill inside the scan wrote
to a buffer nothing reads; it goes with it.
Unloading a texture destroyed it immediately behind vkQueueWaitIdle.
That drain only covers submitted work: command buffers still being
recorded, and menu state on another thread that observes the texture
handle in the short window before the caller zeroes it, can still
reference the texture afterwards. Writing such a texture's view into
a descriptor set then faults inside the ICD (observed on Adreno as a
crash in vkUpdateDescriptorSets under vulkan_draw_triangles during
menu rendering).
Park unloaded textures on a per-driver deferred list instead and
destroy them on the frame-recording thread once a full swapchain
cycle of submissions has elapsed, so every prior reference has been
fenced out. The list is drained eagerly at existing queue-idle
points (swapchain recreation, driver teardown) and mutations are
serialised with queue_lock so reinit-window fallbacks that unload
from the main thread stay safe. This also removes a full GPU drain
per texture unload, which previously stalled thumbnail-heavy menu
scrolling.
The current-core info entry is a zeroed shell until core_info_load()
matches the loaded core against the info list; with no info file for
that core (or no list), its path stays NULL. With a core loaded
previously and core_suggest_always off, menu_content_find_first_core()
forces the current-core route, took the shell as a valid single match
and passed its NULL path to strlcpy: a SIGSEGV in strlen.
A current-core entry without a path now falls back to the scanned
core list, and the function only reports success when it actually
holds a core path - otherwise callers received success with an empty
core path and pushed a load-content task that cannot start.
ConfigFile.parseLine took substring(1, lastIndexOf('"')), so a value
consisting of a lone opening quote - a truncated retroarch.cfg - threw
StringIndexOutOfBoundsException through MainMenuActivity.onCreate on
every launch. An unterminated literal now takes everything after the
opening quote as the value and a lone quote yields an empty value that
is dropped, matching config_file.c semantics.
Two guards keep any remaining malformed input recoverable: append
drops a line whose parse throws instead of aborting the whole load,
and updateConfigFile treats a non-numeric
bundle_assets_extract_last_version as no recorded version, so assets
are re-extracted rather than crashing on NumberFormatException.
task_check_decompress() answers 'is this archive currently being read
by an extraction task': its callers use it to refuse duplicate
extractions and to avoid overwriting an archive a task is still
reading. Subdir and target-file extractions read their source
archive just the same, and every handler variant keeps a
decompress_state_t with source_file in task->state, so the finder
matches any of the three handlers rather than only the plain one.
Trim the comment on the SSL teardown in net_http_conn_pool_free() to
what CODING-GUIDELINES asks of comments: state the invariant going
forward -- ssl_socket_close() closes the descriptor in entry->fd, so
the fd is marked consumed to keep a second close from racing
descriptor reuse -- without narrating the prior behavior. No code
change.
The APP_CMD_PAUSE persistent-state flush issued CMD_EVENT_SAVE_FILES
unconditionally. PAUSE can be delivered while frontend_unix_init is
still pumping events waiting for the native window (activity created,
then immediately backgrounded or screen locked), long before any core
exists - and, in the overlapping-activity case, while a previous
rarch_main instance in the same process is still tearing its core
down, leaving runloop_state.current_core in a transient state.
CMD_EVENT_SAVE_FILES exists to persist SRAM and game-specific cheats,
both of which require loaded content, so gate the flush on
RETRO_CORE_FLAG_GAME_LOADED. Together with the preceding runloop
ordering fix this closes the crash observed on Android 15
(Lenovo Legion Tab, 1.22.2) where the pause flush during the startup
wait loop called through a stale retro_get_system_info pointer into
an unmapped core library.
The config_save_on_exit path is left unconditional: it self-guards on
an empty RARCH_PATH_CONFIG and is meaningful in menu-only sessions.
uninit_libretro_symbols closed the core library first and zeroed
current_core second, leaving a window in which current_core held
function pointers into an already-unmapped .so. On Android, where
overlapping activity lifecycles can run a second rarch_main instance
against the same global runloop_state while the first instance is
mid-teardown, anything performing a NULL check on those pointers
(core_get_system_info being the observed case) would pass the check
and then jump into unmapped memory.
Observed in production as a SIGSEGV with an unresolvable frame #00
(pc in unmapped memory) returning into
cheat_manager_get_game_specific_filename, via the APP_CMD_PAUSE
save-file flush firing during frontend_unix_init's wait-for-window
loop:
#00 Unknown
#01 cheat_manager_get_game_specific_filename
#02 event_save_files+52
#03 android_input_poll_main_cmd
#05 frontend_unix_init
#06 rarch_main+708
Zero the struct before dropping the library reference so the race
degrades to a benign NULL-pointer check miss instead of a wild jump.
No behavioral change for the single-threaded caller.
Google Play Console crash: SIGSEGV in __strcmp_aarch64 via
task_decompress_finder <- retro_task_threaded_find <-
task_push_decompress <- cb_generic_download <-
retro_task_threaded_gather (Android 16, 1.22.2).
task_decompress_handler_finished() ran on the worker thread, inside
the task handler, and freed the entire decompress_state_t (including
source_file) at FINISHED time. The task itself, however, stays
reachable long after that: it sits in the running queue while the
handler returns, is then moved to the finished queue, and remains on
the finished/retiring lists - findable by design - until the main
thread fully retires it, typically many frames later.
task_decompress_finder() dereferences task->state->source_file on
every candidate whose handler matches, so any
task_push_decompress() issued during that window - exactly what
cb_generic_download does when it pushes the next archive from within
a retiring download's completion callback - called strcmp() on freed
memory. The same lockless window also exists while the handler's
final tick is still executing, since the worker invokes handlers
with no queue lock held.
This is the same bug (and the same fix) as task_http.c's finder:
teardown of task->state moves out of the handler into a
task->cleanup callback, which the queue invokes during retirement,
after the task has been pruned from every list. The state - and
thus source_file - now stays live for as long as the task is
findable, which also makes task_check_decompress() truthful through
the retiring window instead of allowing a duplicate extraction of
an archive whose completion callback has not yet run.
Details:
- The completion callback owns decompress_task_data_t and frees
data->source_file (see cb_decompress_task_core_updater_download,
bundle_decompressed), so it now receives a strdup'd copy rather
than taking over the state's buffer; on allocation failure a task
error is set and no data is attached, matching the existing OOM
handling (all consumers NULL-check task_data).
- dec->callback_error is NULLed after task_set_error() hands the
buffer to task->error (freed by the queue at retirement), so the
new cleanup does not free it a second time.
- The finder NULL-checks task->state and source_file as
belt-and-braces, mirroring task_http_finder.
- task_decompress_cleanup() also frees dec->target_file, which the
old FINISHED-time teardown never released (a leak on every
decompress-to-target-file task).
Verified with a harness linking the real task_decompress.c and
threaded task_queue.c, extracting real zip archives while a second
thread hammers task_check_decompress() and completion callbacks
push follow-up extractions from within gather: ASan flags
heap-use-after-free in task_decompress_finder before this change
(from both the gather path and the hammer thread) and is clean
after it, with LeakSanitizer clean across 5 runs and ThreadSanitizer
clean across repeated and concurrent-instance runs. The touched TU
also compiles cleanly under the C89 lane flags.
Play Console reports a SIGSEGV inside the Adreno driver's
vkCreateBuffer (Galaxy S25 Ultra, Android 16), reached from
vulkan_font_init -> vulkan_create_texture via materialui_layout.
materialui_layout only re-runs mid-session on a DPI or dimension
change - sleep/wake, rotation, split screen, a resolution switch -
which is exactly the window where the Android surface is being torn
down or brought back up, so the font initialiser can walk into GPU
object creation while the device state is not trustworthy.
vulkan_font_init currently has no validation at all on that path:
* font->vk, font->vk->context and the device are dereferenced
unconditionally, even though vulkan_font_free guards on exactly
this triple and the HDR-atlas block half-acknowledges the state
with its own NULL checks three lines earlier.
* The staging texture's creation result is never checked, although
vulkan_create_texture fails soft and returns a zeroed texture on
creation or allocation failure - a realistic outcome for the
first allocations against a device recovering from a wake. The
code then calls vkMapMemory on VK_NULL_HANDLE memory, and the
glyph upload path later memcpys through the NULL mapped pointer.
* The optimal texture's creation result is not checked either.
Refuse the init when the device is absent, and turn any texture
creation or map failure into a failed font init: destroy the partial
state (nothing has been submitted to a queue on behalf of the font
at that point, so immediate destruction is safe and
vulkan_destroy_texture unmaps before freeing) and return NULL.
A NULL font is already well-handled upstream - materialui_init_font
keeps the previous font when gfx_display_font_file returns NULL and
font_driver_render_msg tolerates a NULL font - so the failure mode
becomes "keep the old font / draw no text this frame, rebuilt on the
next layout pass" rather than a native crash.
The NULL checks inside the VULKAN_HDR_SWAPCHAIN block are dropped as
dead code now that the top-of-function guard subsumes them.
ssl_socket_close() closes the underlying descriptor itself: both
backends keep the fd handed to ssl_socket_init() -- net_ctx.fd in
net_socket_ssl_mbed.c, state->fd in net_socket_ssl_bear.c -- and
socket_close() it on the way out. net_http_conn_pool_free() then
closed entry->fd, the same descriptor, a second time, on every SSL
pool-entry teardown path (net_http_conn_pool_remove,
net_http_conn_pool_remove_expired, net_http_deinit).
Between the two closes the kernel is free to hand the same fd number
to whoever asks next, so the second close hits a descriptor this code
does not own. On Android that aborts as soon as Binder wins the
race -- fdsan trips with "attempted to close file descriptor N,
expected to be unowned, actually owned by Parcel" with
net_http_conn_pool_free on the stack, seen in Play Console vitals on
1.22.2 (Android 16 / SDK 36) 14 hours after release. On platforms
without fdsan the same race silently closes an unrelated file or
socket under another thread.
Mark entry->fd consumed after the ssl_socket_close()/ssl_socket_free()
pair so exactly one close ever happens; the non-SSL path and the
ssl && !ssl_ctx path (fd never handed to an SSL context) keep closing
via socket_close() as before.
Verified with an LD_PRELOAD close() interposer around an HTTPS
transfer followed by net_http_deinit(): pre-patch reports one
close() -> EBADF from the pool teardown, post-patch zero. ASan/UBSan/
LSan clean; TSan with two concurrent HTTPS transfers reports no races
in net_http.c (remaining reports are pre-existing lazy-init races in
vendored mbedtls). C89 lane compile of the TU passes with and
without HAVE_SSL.
Treat non-positive left/copy/right widths in rvp9_build_mc_border as
empty runs, so the memset/memcpy bounds are provably non-negative for
the compiler and for hostile clamp inputs alike.
Compile a small TU with the UUT's flags that exports
offsetof(settings_t, bools.menu_thumbnail_preview_audio) and
sizeof(settings_t), and poke the stub config blob through those
exports so the poke tracks whatever layout configuration.h produces.
The texture stub reads width/height through the real struct
texture_image, and the stream-state stub returns the declared enum
type.
driver_uninit() only released the menu driver's GPU-held resources
(fonts, textures) under DRIVER_MENU_MASK, and drivers_init() only
rebuilt them when both DRIVER_VIDEO_MASK and DRIVER_MENU_MASK were
set. A video-only reinit - the VIDEO_REINIT network/stdin command
(command_video_reinit) or the CRT switch path
(video_driver_reinit(DRIVER_VIDEO_MASK)) - therefore freed the video
driver instance that every menu font's cached driver pointer refers
to while leaving the fonts themselves alive and never rebuilding
them. The next menu frame then flushed a font block against the
freed instance: a guaranteed use-after-free matching the Android
crash reports faulting in gl2_raster_font_setup_viewport via
gl2_raster_font_flush_block under materialui with threaded video.
Pair the masks up: on a video-only teardown, call the menu driver's
context_destroy so its GPU resources are released with the instance
that owns them (menu state itself persists exactly as it does across
a full reinit, via MENU_ST_FLAG_DATA_OWN), and on the way back up
call context_reset against the new instance. The destroy runs after
the existing video_thread_wait_idle() barrier, so no in-flight
threaded frame still references the resources being freed. All four
menu drivers NULL-guard both hooks, so a never-initialised menu
degrades to a no-op.
Exercised headless under ASan/UBSan/LSan with the menu alive (rgui,
null video, threaded and unthreaded): VIDEO_REINIT round-trips
through the new destroy/reset pair with no sanitizer findings and a
leak summary byte-identical to a run with no reinit at all.
gl2_frame() initialised 'chain', 'width' and 'height' from 'gl' at
declaration, before the 'if (!gl) return false;' guard ever ran, so a
frame call issued with no driver instance faulted at function entry
instead of returning false. Google Play crash reports show exactly
this signature on Android: a fault at the top of gl2_frame reached
from video_driver_frame via command_event_reinit's cached-frame
replay (first-run asset extraction firing CMD_EVENT_REINIT), where
video_driver_frame passes video_st->data to vid->frame() unchecked.
Declare the locals NULL/0 and resolve them immediately after the
guard. gl3_frame already orders its declarations this way; gl2 was
the outlier.
The stream callback sets a data_moved token under the driver lock and
waiters consume it, so a signal raised between the queue-level test and
the condition wait leaves the token set instead of going unheard. The
removal watch is registered before the stream is primed, flag re-arm
belongs to the reopen path, and comments capture the SDL contract the
reopen relies on: logical devices opened as the system default never
receive REMOVED - SDL migrates them itself - so only explicitly
selected devices reach the reopen path.
Syntax-checked against SDL 3.2.20 headers with MSVC C89 declaration
rules enforced; TSan and ASan/UBSan/LSan sweeps with unplug cycles
still required on an SDL3-capable host before push.
android_input_state dereferences the driver data unconditionally on
the MOUSE/LIGHTGUN/POINTER paths. When called with NULL data (the
current_data teardown/reinit window on surface loss, e.g. sleep/wake
or app-switcher resume) this crashes with a segfault in
android_input_state under input_driver_poll, matching field crash
reports from Android touch users where the overlay pointer poll runs
every frame. Fail closed instead.
Also bounds-check the core/frontend-controlled idx against MAX_TOUCH
before any pointer[] access in the POINTER and LIGHTGUN paths.
Previously only POINTER_PRESSED range-checked (via pointer_count);
the X/Y/IS_OFFSCREEN paths would read past pointer[MAX_TOUCH] for
out-of-contract idx values. Those reads stayed inside the struct, so
this is contract hardening rather than a crash fix.
input_poll_overlay dereferences input_st->current_data through the
input driver's POINTER paths every frame while an overlay is alive.
current_data can be NULL in the window around driver teardown/reinit
(drivers_deinit NULLs it before drivers_init repopulates it, and on
Android a surface-loss reinit can be triggered from inside the input
poll itself), which faults inside the driver's input_state.
Skip driver input for the frame when current_data is NULL.
Also evaluate the touch loop bound before issuing the
RETRO_DEVICE_ID_POINTER_PRESSED query: the previous operand order
issued one query with idx == OVERLAY_MAX_TOUCH per poll once all
slots were occupied, relying on every driver to range-check idx
internally, which is not part of the driver contract.
Return false when the core's function-pointer table is not yet
populated. The Android pause-time state flush can run before any core
is loaded - while the frontend is still waiting for its first window -
and its cheat-path caller needs a clean failure here so both the
game-specific cheat load and save reduce to no-ops.
Kill a stalled apt-get update at 90 seconds and pass Acquire
retry/timeout options on the refresh itself, matching the install
lines, so the three-attempt loop reaches its later attempts inside
the step's five-minute cap.
The threaded lane's pump delivers samples of the progress value as it
stands at each check, so seeing a value strictly between 0 and 100
requires the worker to still be mid-stream on at least one check.
Give that sighting a long runway - a fixture spanning many pump
intervals, written by a chunked fixture writer - and bound it with
re-runs when the sighting alone is missed. Every unconditional
invariant stays asserted on every run, and a delivery-path regression
misses on every attempt, so the re-runs cannot mask one.
Extends the pattern from Linux-libretro-common-samples.yml to all
remaining workflows: skip apt when the packages are already present on
the runner image, retry the index refresh, pass Acquire retry/timeout
options on install, and cap each dependency step at five minutes
(eight for the wine/mingw set) so a stalled mirror fails fast with a
step-level error instead of a whole-job cancellation.
Skip apt entirely when the requested packages are already present on
the runner image, retry the index refresh, pass Acquire retry/timeout
options, and cap every dependency step at five minutes so a stalled
mirror fails fast with a clear step-level error.
task_cloudsync.c holds an in-flight count that only the driver's
completion handler releases, so a true return from read/update/delete
is a promise that the handler runs exactly once; any path that skips
it stalls the sync task forever at 100% (#19435). gdrive_do_patch()
had two such paths - a zero-byte local file and a failed malloc - and
all fourteen task_push_http_*() call sites had another, since those
helpers return NULL without ever calling back. Every failure exit now
reports through the handler, via gdrive_cb_state_finish() /
gdrive_walk_fail(), and a zero-byte file is uploaded normally so a
non-empty server copy gets truncated. The begin path and the entry
points' calloc()s instead return false, which task_cloudsync already
fails cleanly.
Also: drop the manual Content-Length on PATCH uploads
(net_http_send_request() emits it for every P* method, so it went out
twice), and clear poll_pending when the OAuth poll request cannot be
dispatched so device authorization retries instead of stalling.
Sanitizer sweep: ASan+UBSan+LSan clean, TSan clean over 31 runs, on a
15-scenario harness driving the driver against a scripted HTTP stub
(success, empty file, create-then-upload, folder walk, 401->refresh->
retry, and every dispatch-failure path); exactly one completion per
operation, zero leaks. gcc -std=gnu89 -Wall -Wdeclaration-after-statement
clean.
Broke the compile matrix: its mem_stats platform arms section syntax
checks every arm, giving each one a stub directory holding the handful of
SDK symbols that arm names, and PS2 was the single arm needing no
directory at all - stdlib.h and nothing else. Including <kernel.h> for
one prototype took that away, and the check that exists to catch exactly
this drift failed with the header not found.
kernel.h is a declaration wall for the whole EE kernel API. What is
wanted out of it is one prototype, and the function behind that prototype
is four instructions emitted by the SYSCALL_SPECIAL macro in ps2sdk's
ee/kernel/src/kernel.S:
GetMemorySize: li , 0x7f ; __NR_GetMemorySize
syscall
jr 1
nop
It lives in libkernel, which every EE build already links for crt0, and
s32 is int on this target. So declare the symbol here and drop the
include: the same call, the same library, no header, and the PS2 arm goes
back to being checkable on a runner with no SDK on it.
Emitting that stub in this file instead was the other option and is the
worse one - it would duplicate a symbol the build already links, through
a hand-written asm path proven on nothing, to avoid a dependency the link
has anyway.
No behavioural change: 32MB and 128MB boards report as before.
compile-matrix.sh green, all 15 arms and all 17 codec combinations.
Sanitizer sweep clean: ASan + UBSan + LSan no findings, TSan no findings.
C89 verified with gcc -std=c89 -pedantic on the PS2 arm with no stub
directory on the include path.
The previous commit bounded the free probe by a hardcoded 32MB, which is
what retail hardware has and what mem_stats_total() had always claimed.
That is right for the machine nearly everyone runs, and wrong for every
other PS2 that exists: a TOOL devkit (DTL-T10000) has 128MB, and a retail
unit switched into 64MiB mode via SetMemoryMode has 64MB. On a devkit the
overlay would have read 0.00/32.00MB with roughly 100MB actually free -
consistent, no longer wrapping, and understating the machine fourfold.
Capping a wrong total is not the same as knowing the total.
The kernel has answered this since the first BIOS: GetMemorySize(),
syscall 0x7f, declared in ps2sdk's kernel.h and used by ps2sdk itself to
decide the TLB layout. Ask it once, cache it, and drive both the total
and the probe's budget from that. The 32MB literal survives only as the
fallback for a kernel that answers with nothing, where understating is
the safe direction for a caller sizing an allocation.
Nothing else moves: the probe still cannot exceed the total, because the
budget it draws down is now the real total rather than an assumed one.
Verified against a stub GetMemorySize reporting 32MB, 64MB and 128MB. On
each, free tracked the board size with the heap unconstrained, and the
pair stayed consistent with the heap capped below it - a 128MB board with
40MB of heap left reports 91.55/128.00MB, with 100MB left reports
31.00/128.00MB. No case produced free > total.
Sanitizer sweep clean: ASan + UBSan + LSan no findings, TSan no findings.
C89 verified with gcc -std=c89 -pedantic on both the PS2 and host arms.
The memory overlay on PS2 read
MEM: 17592186044416.00/32.00MB
which is 2^44, the float rendering of an unsigned subtraction that had
wrapped.
mem_stats_total() on PS2 returns a fixed 32MB, the retail machine's RAM.
mem_stats_free() has no accounting to consult, so it finds out by trying:
three mallocs, each halving from a starting size until one succeeds, the
sizes added up. Every one of the three restarted its search at the full
32MB, so the probe's ceiling was 96MB - three times the total it is
paired with. On a machine that can satisfy all three at the top size the
probe returns exactly 96MB, and gfx/video_driver.c's
last_used_memory = last_total_memory - mem_stats_free();
underflows uint64_t to 18446744073642442752, which the %.2f conversion
rounds to 2^64 and divides down to the figure above.
Give the probe a budget instead. It starts at the same constant
mem_stats_total() reports, and each grab searches down from what is left
of it rather than from the top, so the sum can never exceed the total and
the later grabs converge in fewer steps. Both figures now come from one
MEM_STATS_PS2_RAM_BYTES define rather than from two copies of the same
literal. A probed flag replaces the truthiness test on the cached value,
so a machine that legitimately has nothing free stops re-probing on every
call.
Two callers pair the figures and are fixed to survive them disagreeing,
which the header already warns is possible - they are snapshots of a
moving machine, and a probing platform can only approximate:
- the memory overlay clamps the subtraction to 0
- RETRO_ENVIRONMENT_GET_MEMORY_STATUS clamps free to total, so a core
sizing a pool off the pair cannot inherit the same wrap
Verified with a harness driving the old and new probes side by side:
unconstrained, before free=100663296 against total=33554432 giving the
17592186044416.00 line, after free=33554432 giving 0.00/32.00MB. With
the heap capped to 24MB and to 12MB the two probes agree to within one
halving step (11.00 vs 11.25MB used, 22.75 vs 22.89MB used), so the
bound costs no accuracy where the old code was not already wrong.
Sanitizer sweep clean: ASan + UBSan + LSan no findings, TSan no findings.
C89 verified with gcc -std=c89 -pedantic on both the PS2 and host arms.
Same family as the Vulkan (892e42ab) and d3d12 fixes, latent rather
than reported: with a core supplying RETRO_PIXEL_FORMAT_HDR10_2101010
and the menu open, the composite's D3D11_ST_FLAG_SOURCE_HDR10 branch
passes the back buffer through unencoded. But the back buffer was
cleared to transparent black before the UI drew into it, so the
composite source holds only the SDR UI regardless of what the core
supplies -- the game reached the swapchain through the back-buffer
pass above, and the composite alpha-blends the UI over it. The
passthrough therefore lands the UI's SDR code values raw in the PQ
swapchain, where code 1.0 decodes to 10000 nits, and menu_nits goes
unread.
The reason this stayed latent while the identical d3d12 defect was
reported is the flag: an HDR-emitting shader preset does not set
SOURCE_HDR10, so the reporter's repro fell through to the else-branch
encode and worked. An HDR core would not have.
Encode unconditionally in the non-scRGB branch, at menu_nits, exactly
as the else-branch already did. The scRGB branch is untouched.
Needs on-hardware confirmation (Windows-only TU, cross-compiled here):
HDR10 swapchain, an HDR10-supplying core, menu up -- UI brightness
should track (HDR) Menu Brightness, game underneath unaffected.
Under an HDR10 swapchain with the menu, an overlay, the OSD or widgets
up, the UI renders at the panel's PQ ceiling -- 10000 nits -- and the
(HDR) Menu Brightness setting does nothing. This is the d3d12 half of
the report fixed for Vulkan in 892e42ab, and it fails the same way for
the same reason.
The menu composite's source is the back buffer, and by the time the
composite runs, that holds only the SDR UI: the UI block cleared it to
transparent black before the menu texture, sprites and overlay drew
into it. The game reached the swapchain earlier, through the
back-buffer copy pass (or directly when no back buffer is in use), and
the composite alpha-blends the UI over it -- the STOCK_HDR pipe is the
blend-enabled variant.
4f103505 switched the composite's HDR10 branch to passthrough on the
premise that the back buffer held the PQ-encoded game and re-encoding
it drove the menu background to black. Whatever held at that commit,
it does not describe the current frame: the clear guarantees the
composite source is SDR, so the passthrough hands the UI's SDR code
values raw to the PQ swapchain, where code 1.0 decodes to 10000 nits.
menu_nits is assigned to paper_white_nits just above, but the
passthrough branch never reads it -- hence the setting's inertness.
Encode the UI instead: inverse_tonemap and hdr10 on, which routes
PSMain through HDR10(HDR(To2020(...))) at menu_nits with alpha
preserved for the blend. This is byte-for-byte the configuration of
the d3d11 driver's working else-branch, which is why the reporter saw
the bug survive on d3d12 only. The scRGB branch already encodes at
menu_nits and is untouched.
Needs on-hardware confirmation (Windows-only TU, cross-compiled here):
HDR10 swapchain, menu up -- with and without an HDR-emitting preset --
UI brightness should track (HDR) Menu Brightness, game underneath
unaffected.
retro_vfs_file_rename_impl passed old_path as both arguments of
uwp_move_path, and DeleteFileFromAppW's nonzero-on-success return was
checked inverted, so any rename whose source existed deleted it and
returned failure, while a missing source fell through to success.
Playlist writes on Xbox hit this through edb8fac's write-to-temporary
and reported success with nothing on disk.
Pass new_path through, fix both delete checks, fail on a missing
source, and treat a self-rename as a no-op success. Needs a UWP build
to verify; this TU is not compilable outside MSVC/WinRT.
Loading an HDR shader preset pins the UI at the panel's PQ ceiling --
10000 nits -- and the (HDR) Menu Brightness setting stops doing
anything. The same holds for a core supplying
RETRO_PIXEL_FORMAT_HDR10_2101010. Without an HDR source the setting
works, which is why this read as a recent settings regression; it is
not, and the setting itself is fine.
The menu/overlay composite reads the offscreen buffer. By the time it
runs, that buffer holds only the SDR UI: the UI pass begins
sdr_render_pass on it, whose loadOp is CLEAR, so anything the filter
chain wrote there earlier is gone -- and the buffer is B8G8R8A8_UNORM,
which cannot carry PQ10 or FP16 content in the first place. The game
frame reaches the swapchain separately, through the game pass when
use_offscreen_buffer is set or directly from the filter chain when its
last pass emits HDR natively, and the composite alpha-blends the UI
over it.
fdadb4f5 changed the composite to derive its mode from the game source
via vulkan_hdr_source_mode, on the premise that the pass composites the
game frame too. It does not, and did not in that commit's own tree.
With a PQ-emitting preset or core the helper answers passthrough, so
the menu-nits PQ encode never runs and the UI's SDR code values land
raw in the PQ swapchain, where code 1.0 decodes to exactly 10000 nits
-- the reported symptom, including the setting's inertness, since
menu_nits is consumed only by the encode that gets skipped. On an
scRGB swapchain the helper answers mode 3 instead and PQ-decodes the
UI, corrupting it a different way.
Derive the composite mode from the swapchain alone: SDR -> PQ under
HDR10, SDR -> scRGB under scRGB. The source is SDR by construction,
so nothing else is correct. fdadb4f5's genuine fix -- the game pass
accounting for VK_FLAG_SOURCE_HDR10 -- is untouched, and the readback
pass already derives its mode this way. The helper's docblock now
says it answers for the game pass only, so the composite does not
drift back onto it.
Needs on-hardware confirmation (no Vulkan device here): HDR10
swapchain, an HDR10-emitting preset, menu up -- UI brightness should
track Settings > User Interface > Appearance > (HDR) Menu Brightness
again, with the game underneath unaffected.
The pump finishes a deferred read, installs the cache, and rebuilds
the displaylist; the rebuild asks for the same playlist again. Both
bisect points are one defect through two drivers: the reuse check
judged the playlist just installed unusable, so the rebuild freed it
and began the read again - under 5e4ded7 a list that never filled,
under 63f184f one restart per completion, a placeholder flash each.
Small playlists hide the cycle by finishing inside one slice; the
attached 3315-entry playlist yields on parse cost alone.
Two ways playlist_cached_is_reusable() wrongly said no:
- The recorded base_content_directory was compared against the
config's unconditionally. The attached playlist records one;
opened with autofix inactive, the parse leaves the record alone,
so the cache is exactly what a re-read would produce yet never
matched the config's empty base. The comparison now applies only
when config->autofix_paths is set, mirroring the parse's own gate.
- The autofix pass saves the file it fixed, leaving the size stamp
taken at open stale - the format-conversion hazard on the read
path's other write, unreachable by playlist_cached_after_write()
because the playlist is not installed yet. Now refreshed after
the save.
While a read is pending the placeholder now says Loading (the
deferred dirwalk's entry) instead of asserting the playlist is
empty; the pump replaces it on completion.
Tests: the parity oracle gains a rebuild-reuse lane (pump frame
shape, re-request must be served from cache, sentinel intact) and an
after-autofix-save cache-reuse case; both fail at the parent, four
failures. The menu harness gains a based-playlist-loads-once lane
driving the shipping menu with an OK press and frames alone; against
the parent's playlist.c it fails as reported - still pending after
600 frames, a re-pend on all 30 stability frames, list stuck at the
placeholder.
Verified: the reporter's attached playlist restarts its read on the
rebuild before this change and is served from the cache after it,
file byte-identical throughout. playlist.c and menu_displaylist.c
pass the strict C89 gate. Playlist oracle: plain, twice
ASan+UBSan+LSan (CI env verbatim), twice TSan. Playlist manager
oracle: plain, ASan+UBSan+LSan, twice TSan. Scan oracle:
ASan+UBSan+LSan, twice TSan; its big-DAT pacing lane exceeds budget
on this single-core container identically at the unmodified parent
(environmental), functional lanes pass. Dirwalk oracle: plain.
Full non-Qt Linux build links; the menu harness passes all four
lanes against the shipping objects.
The deferral gate mirrors the load's BLCK_NEED_FULLPATH decision
and declines content the load will not read into memory: a
need_fullpath core (ppsspp + chd, the reported repro) is handed the
path, so a prefetch fills an allocation nothing consumes. The
decision reads the live system info and override list - LOAD_CORE
has run by the gate; the caller's content_ctx is built before it,
without sys info. The progress callback is delivered on the
thread that pumps the queue under either queue mode - a
task->progress_cb trampoline dedups per change on the pumping
thread, the unthreaded handler delivers directly, and
task_queue_push_progress() fires the callback for any non-mute
task while messages keep the title gate. The parked continuation
is stamped with the content and core paths it belongs to and
stands down on a mismatch, since in-flight tasks survive a
competing load's queue reinit.
The oracle gains a threaded progress-callback lane pinning
delivery to the pumping thread; the pre-fix code fails it. Sweep:
ASan+UBSan+LSan pass (9 lanes), TSan 35/35 clean, oracle also run
on aarch64/bionic (NDK r29, static, qemu-user), NDK arm64-v8a
griffin.o compiles clean, desktop griffin error set unchanged.
End-to-end differential (headless build, real ppsspp core, 400 MB
chd via the from-menu entry point): peak RSS 415 MB before, 25 MB
after.
Fixes https://github.com/libretro/RetroArch/issues/19426
0e45cec3 and b9faccab added font_driver_free_deferred() and
font_driver_matches() calls to gfx_widgets_font_init() without adding
them to the gfx_widgets_sanitize stubs. That sample compiles
gfx/gfx_widgets.c for real against a stub layer, so it stopped
linking:
gfx_widgets.c:956: undefined reference to `font_driver_matches'
gfx_widgets.c:981: undefined reference to `font_driver_free_deferred'
My oversight: I checked the tree built and the font_lifecycle harness
passed, and did not check the other samples that compile the files I
had touched.
font_driver_free_deferred() is the same no-op as the font_driver_free()
stub beside it, there being no frame clock or GPU atlas here.
font_driver_matches() answers false, which is the honest answer given
gfx_display_font_file() in this stub layer never produces a font, and
keeps gfx_widgets_font_init() on the build path these tests exercise
rather than short-circuiting it.
Checked the rest rather than just the one that broke. Of the sample
harnesses, only gfx_widgets_sanitize and gfx_widgets_msg_queue_race
mention gfx_widgets.c and only the former compiles it - the latter
says in a comment that it deliberately does not, which is what the
grep matched. gfx_animation_sanitize stubs font_driver symbols but
compiles only gfx_animation.c. No sample under menu/, tasks/,
runloop/, playlist/ or formats/ compiles any file in this series.
Also ran, all clean: every buildable sample under samples/gfx, the
three menu-harness samples, the settings table and griffin-order unity
checks, and the overlap copy check. display_servers,
display_servers_kms and vulkan_validation fail here for want of
Xrandr, libdrm and Vulkan headers, and compile nothing from this
series.
The concatenation risk of a unity build is real for these files -
griffin includes video_driver.c, font_driver.c and gfx_widgets.c, and
the griffin-order check in Settings-Checks only covers the settings
TUs - so those three were compiled as one translation unit in griffin
order to confirm the new statics and macros collide with nothing.
font_driver_reinit_osd() has carried its own copy of the question
font_driver_matches() answers: is the font already the one being asked
for. Same exact size compare, same empty-path-is-no-path rule, same
reasoning in its comment about why the float compare is exact. b9faccab
added the shared version for the menu and widget layout paths without
noticing this one predated it.
Two copies of a comparison this fiddly is what drifts. Use the shared
one.
Behaviour is unchanged. Checked exhaustively rather than by eye: over
every combination of stored and requested path drawn from {NULL, "",
two distinct paths} and size drawn from four values, 256 in total, the
two agree everywhere. The one input where they diverge is an
un-normalised empty request against a pathless font, which
reinit_osd() cannot produce because it normalises "" to NULL first -
and there the shared version gives the right answer, which is why the
layout callers can hand it "" directly.
The substitution also gains the liveness walk, which declines to read
through a handle that is no longer live rather than answering from
freed memory.
Existing coverage in samples/gfx/font_lifecycle already pins this down
- unchanged settings rebuilding nothing and not bumping the
generation, size alone, path alone, a cleared path, an unreadable
font, and no registered OSD font at all. Green under ASan+UBSan+LSan
and TSan.
The deferred free changed when a superseded font handle is released,
not how often one is created. Every layout pass still tears down and
rasterises its fonts unconditionally, and most layout passes are not
asking for a different font at all:
ozone_set_layout runs on thumbnail scale and padding changes, and
rebuilds all six fonts; each font also has its own
scale factor, so moving one moves all six
materialui_layout runs on a nav bar toggle and a landscape
optimisation change, rebuilding three
gfx_widgets_layout runs on any video dimension change, but the size
it asks for derives from last_scale_factor alone,
so a resize at unchanged scale rebuilds six
xmb_set_layout the scale setting steps in hundredths while
font_size is an integer, so most steps of the
slider move nothing
None of that is cached today. font_driver.c's font_file_refs caches
the file bytes per path, which saves the read, but a rebuild still
rasterises the atlas and re-uploads the texture, and that is the
expensive part.
font_data_t already remembers both the size it was built at and the
path it was built from, so the question is answerable without new
state. Add font_driver_matches() and have the four layout paths return
early when it says yes.
It is conservative in one direction only. It answers no for a NULL
handle, for any size or path difference, and for a handle that is no
longer live — the list walk means a caller that freed a font and left
the pointer behind gets a no rather than a use-after-free read. So a
context reset, which frees and clears its fonts, always rebuilds. A
wrong no costs only the work that would have been done anyway; a wrong
yes would keep a font that should have been replaced, which is why the
test covers the no cases far more heavily than the yes case.
Two ordering details this depends on. The size compared is the one
that would actually be built, so the callers' own clamps (9 for ozone
and widgets) run first. And the early return precedes the approximate
glyph metrics those functions write before building, which are
overwritten by measured values afterwards — returning after them would
leave the estimate in place of a measurement.
The wide-glyph sample follows the menu language and is not refreshed
by font_driver_sync_impl(), so a change there must not be skipped past
even when the face is unchanged. XMB recomputes the derived widths
against the loaded font, which is all that is needed. Ozone and
MaterialUI compute those widths inside the build, so they take the
conservative route and fall through to a rebuild; a language change is
rare enough that the simpler code is worth more than the saving.
XMB's hand-rolled guard in xmb_set_layout() collapses into this.
Moving it into xmb_rebuild_fonts() puts it next to the path resolution
it needs, and last_font_path is now recorded before the test rather
than after: a menu font setting can change to a path that resolves to
the file already loaded, and leaving it stale would have xmb_render()
detect a change every frame and call straight back in.
Covered in samples/gfx/font_lifecycle, under ASan+UBSan+LSan and TSan:
match and every flavour of mismatch, a face re-resolved in place by a
language switch no longer matching the path it was built from, and a
freed handle answering no both after a direct free and after the
deferred queue drains.
XMB was not the only caller freeing a font from a path that runs before
the video driver's frame function. The same free-then-create sits in
four more places, all reached per frame:
ozone_init_font ozone_render -> ozone_set_layout, six fonts
materialui_init_font materialui_render -> materialui_layout, three
gfx_widgets_font_init gfx_widgets_iterate -> gfx_widgets_layout, six
menu_screensaver menu_screensaver_iterate, on a dimension change
All four fire on a menu scale factor change, which is the operation that
crashed D3D12 in XMB for years before 45f8bf4a. Whether they have been
crashing quietly or merely could is not something the code can answer;
the structure is the same either way.
Build the replacement before releasing the old handle and retire the old
one through font_driver_free_deferred(). The screensaver has no
replacement to build at that point — it clears the font and lets the
effect init rebuild it below — so it only needs the retire.
Building first has a second benefit at three of the four sites: a font
that fails to load now leaves the previous one in place rather than
leaving the caller with none. Ozone's init in particular returns false
on failure and its caller clears HAS_ALL_ASSETS, but it was returning
with font_data->font already NULL.
The teardown frees in these files are left alone: they run when the
context is going away, where there is no later frame to wait for and
the queue is drained wholesale.
Drops xmb_font_retire(), xmb_font_retire_tick(), the font_retire[] array
and XMB_FONT_RETIRE_MAX in favour of the font_driver.c equivalents. No
behavioural change: same two-frame interval, same build-before-retire
ordering in xmb_rebuild_fonts().
The two flush calls go with them. xmb_context_destroy() and xmb_free()
no longer need to drain anything, because the queue is drained in
video_driver_free_internal() before the driver that owns the atlases is
released, which is the point that actually matters — and a menu driver
reaching into a queue shared with the widget and screensaver fonts to
flush it would be overreach.
45f8bf4a found that releasing a font from XMB's render path could crash
D3D12, because xmb_render() runs before the video driver's frame
function in the same runloop iteration, so the atlas could still be
referenced by a command list that had not been submitted. Its commit
message noted the exposure is structural and applies to any driver with
deferred submission. 007c8b8a narrowed XMB's fix to what is actually
unsafe — building the replacement first and releasing the superseded
handle two frames later — but did it with a queue private to xmb.c.
Nothing in that queue is XMB-specific, and XMB is not the only caller
with the problem, so lift it into font_driver.c:
font_driver_free_deferred(font) retire a superseded handle
font_driver_free_pending(flush) age the queue, or drain it
font_driver_free() already marshals renderer->free to the video thread
when threaded video is active, which covers a main-thread free racing
the video thread. It cannot cover the other case: non-threaded video on
a driver that records commands and submits later, where the race is
against the GPU rather than another CPU thread. That is the gap this
fills.
The queue is aged from video_driver_frame(), immediately after the
frame has been handed to the driver, so the count measures frames
actually submitted rather than iterations of the runloop. It is drained
in video_driver_free_internal() before the driver owning the atlases
goes away, next to the existing OSD font teardown. Global rather than
per-caller because only one menu driver is live at a time and the
widget and screensaver fonts share the same frame clock; what needs
sizing is handles in flight, not callers.
Two frames is the interval XMB's countdown used and what the D3D12
report was tested against. Overflow warns and frees rather than leaking:
it needs more than eight rebuilds between two frames, but a silent leak
would be worse than a loud fallback.
Covered in samples/gfx/font_lifecycle: the handle survives its window
and is released after, an empty queue is a no-op, a retire-per-frame
drag keeps in-flight handles bounded and leaks nothing over 200 frames,
flush releases what is still queued, and retiring NULL is allowed.
Passes under ASan/UBSan in both the plain and HAVE_THREADS builds.
Two horizontal offsets are cached rather than recomputed per frame, and
both are derived from values xmb_layout() changes:
xmb->x = icon_size * 1.1f * -(depth * 2 - 2)
xmb->categories_x_pos = icon_spacing_horizontal * -categories_selection_ptr
Neither is written by xmb_layout(). They are only assigned on a depth or
category change, from xmb_list_open() and
xmb_context_reset_horizontal_list(). So applying a scale change in place
rescaled everything those two feed into while leaving the offsets
themselves at the old scale, sliding the category icon and the entry
column sideways. Navigating out to root and back rewrote them, which is
why re-entering the same menu at the same value looked correct.
The error is exactly the difference between the two scales. Going 1.00x
to 0.72x at 1920 wide, icon_size drops 128 to 92.16 and
icon_spacing_horizontal 192 to 138.24, so xmb->x is stale by 78.85 and
categories_x_pos by 53.76 with the Settings tab selected. The entry
column, which reads only xmb->x, lands 79 px off; the category icon,
which reads both, lands 133 px off. That matches what the setting looks
like on screen in both directions, and the entry icons and labels — which
read margins_screen_left and margins_setting_left, both recomputed — do
not move at all.
Resettle both at the end of xmb_layout(), alongside the node->y pass that
already resettles the vertical spacing. Tweens in flight on either are
killed first, since their targets were computed at the old scale and
would otherwise land on it; xmb_context_reset_horizontal_list() already
does the same before assigning these directly. The depth term is clamped
to 2 to match the settled value produced by the other two writers — the
opening animation is only pushed for depth <= 2, so deeper stacks keep
the depth-2 offset.
xmb->x and categories_x_pos are the only cached geometry with this
problem. node->x is a list-transition offset that animates to zero and
self-corrects, and the remaining animation subjects are alphas.
a61a2423 stopped the flash by holding the new scale back until the
deferred reset fired, so that the geometry and the values the draw path
reads could never disagree. That fixed the flash but made the setting
feel dead: the detection compares against the committed last_* values,
so a scale factor still being dragged re-arms the counter every frame
and nothing on screen moves until the user lets go. Trading a flash for
no feedback at all is not an improvement.
The premise both attempts shared — that a scale change requires the
deferred context reset — is wrong. Almost none of what
xmb_context_reset_internal() does is scale-dependent: the thumbnail,
wallpaper, horizontal-list and screensaver work exists for an actual
graphics context reset. What a scale change needs is the geometry
recomputed and the two fonts rebuilt at the new sizes, which is exactly
what ozone_set_layout() does for Ozone, synchronously, without a flash.
So do that. xmb_set_layout() applies the change in place from
xmb_render(), and the next frame drawn is the first one at the new
scale.
That leaves the hazard 45f8bf4a was chasing. It is narrower than the
rebuild: the problem is never computing new state, only releasing GPU
resources that a command list already submitted may still reference.
Dropping the thumbnail and screensaver work removes the texture frees
from this path entirely, and the font handles are handled by building
the replacements first and retiring the superseded ones through a small
queue that frees them two frames later. Same two-frame guarantee as
before, but nothing on screen has to wait for it, and there is no longer
a window in which XMB has no font at all.
Font path resolution moves into xmb_rebuild_fonts() so that the context
reset and the in-place rebuild cannot drift apart, and the rebuild is
skipped when neither font size nor the font path actually changed — the
setting steps in hundredths, so most steps do not move an integer font
size, and a title-margin change never does.
Retire queue verified against a model of the render loop: 93k rebuilds
including sustained every-frame bursts, peak of four handles in flight
against a queue of eight, no free inside the two-frame window, no
overflow, no leak.
xmb_context_destroy() already clears pending_dynamic_icons_repopulate so
that xmb_render() cannot fire deferred work in the window between destroy
and the matching reset. pending_context_reset has the same hazard and was
not covered: if its counter reached zero in that window, xmb_render() would
call xmb_context_reset_internal() with reinit_textures false, rebuilding
the fonts against a context that had just been torn down while leaving the
textures unloaded.
Clear it in the same place. Dropping the latched scale costs nothing, since
the matching xmb_context_reset() performs a full reset and xmb_render()'s
detection re-arms on the next frame if the scale factor still differs.
Pre-existing since 45f8bf4a; split out from the scale latch fix because it
is a distinct failure mode.
45f8bf4a moved xmb_context_reset_internal() behind a two frame countdown,
because xmb_render() runs before the video driver's frame function in the
same runloop_iterate() and freeing font and texture handles there raced
the in-flight command list. That deferral is correct and is untouched here.
What it also did was keep committing the new scale immediately:
xmb->last_scale_factor = scale_factor;
xmb->pending_context_reset = 2;
xmb_layout() derives icon_size, font_size, the margins, icon_spacing and
scale_mod[] from that same value, and it does not run until the counter
reaches zero. So for two frames the draw path mixed new-scale reads of
last_scale_factor with old-scale cached geometry. 2ef44bda papered over
the result with a goto past everything but the background gradient, which
is the flash visible when changing XMB Scale Factor and which no other
menu driver has.
Latch the new scale and layout into pending_* fields and commit them in
the countdown block, immediately before the reset that consumes them. No
frame is ever internally inconsistent, so the frame skip has nothing left
to hide and is removed. The detection still compares against the committed
last_* values, which preserves the existing debounce: a scale factor still
being dragged keeps re-arming the counter, and the reset fires once on the
value the user settled on, with the menu rendering consistently at the old
scale until then.
xmb->use_ps3_layout was published a frame early for the same reason and is
now a local in xmb_render(), latched and committed with the scale.
Four draw sites recomputed the clamp from settings->floats.menu_scale_factor
directly rather than going through last_scale_factor: the ticker limit in
xmb_draw_item(), the title icon size, and the battery and clock x offsets.
Those track the live setting instantly and would still have skewed by a few
pixels for the two frames, so they now read a scale_cap snapshot taken by
xmb_layout() alongside scale_mod. That also de-duplicates the identical
clamp expression previously computed in both xmb_layout_ps3() and
xmb_layout_psp().
No functional change once the reset has settled; the only difference is
which of the two scales the intervening frames are drawn at.
Both drivers already rebuild their fonts in place when the scale
factor moves, so they watch their font path the same way. Ozone
re-runs ozone_set_layout(); XMB schedules the deferred
xmb_context_reset_internal() its scale changes use, which keeps the
two-frame wait before releasing fonts the GPU may still be reading.
Each records the path it built from where the fonts are created, so
context reset and the in-place rebuild agree on what is loaded.
The rows drop CMD_EVENT_REINIT, and the scan actions behind Delete
drop the reinit they called directly - the path alone is now the
signal.
Resetting the notification font with Delete goes through the scan
action rather than the write handler, so it takes the same route:
font_driver_reinit_osd(), reinitialising only where a driver keeps its
own font.
The three OSD colour settings get a dropdown. Their list comes from
the generic ST_FLOAT builder and is labelled by their representation,
0-255 for a 0..1 fraction, so generic_action_ok_dropdown_setting()
derives the value from the selected index stepped off min instead of
parsing it back out of the label. Settings without a representation
label their own value and keep parsing it.
The notification font and size settings rebuild the OSD font in place
instead of reinitialising every driver. font_driver_reinit_osd() takes
a path and a size and returns false only where there is no shared OSD
font to rebuild - sdl, sdl2, xvideo, vg, omap, oga and exynos keep
their own - in which case general_write_handler() falls back to
CMD_EVENT_REINIT. Widget fonts are built from the same path at their
own sizes, so gfx_widgets_iterate() watches it alongside the scale
factor and re-runs its layout when it moves.
The swap is factored into font_driver_rebuild(), shared with
font_driver_reload_fonts(). It reaches the backend through
font_init_first() so a rebuild is marshalled to the video thread when
its creation was, which font_data_t now records. The lifecycle sample
covers that with a second HAVE_THREADS binary, and links again with a
RARCH_WARN stub.
The flag arrived with about forty lines of block comment restating its
own commit message, including the cost-asymmetry table, plus twelve
more in fill() and six in seek(). Cut to what a reader at the
declaration needs: what the flag means, why the allocation arm is not
latched, why the clear is on the spanless arm. The argument for the
clear stays in the commit that made it, which is where it belongs and
where it is already written out in full.
The clear also carried "if (stream && plain != VFS_ERROR_RETURN_VALUE)"
one line after filestream_raw_seek() dereferenced stream->hfile
unconditionally, so the guard could never fire. filestream_seek()
already has that inconsistency at the top of the function; a second one
does not improve it.
No behaviour change: all nine cases still pass under
-fsanitize=address,undefined and file_stream.c still compiles clean
under -std=c89 -ansi -pedantic -Werror=declaration-after-statement.
Two gaps, one of them a claim the code does not actually make.
The ladder is justified by allocation scarcity being a moment rather
than a property, but a handle granted 1 KiB during that moment keeps
1 KiB for life: rbuf != NULL short-circuits the block and there is no
path back up. case_transient_alloc_failure_recovers only covers the
case where the whole ladder was refused and rbuf stayed NULL, so
nothing observed the downgrade being permanent. The behaviour is fine
- 1 KiB is still a thousandth of the byte path's VFS traffic - but the
comment implied a recovery the implementation does not have. Comment
corrected to state the limitation, and case_smaller_buffer_is_kept
asserts it: take 4 KiB under pressure, lift the pressure, and the
handle stays on the 4 KiB read count with nothing reallocated.
alloc_calls > 1 was the one loose assertion in the file where every
other case pins a number, and > 1 passes for an implementation that
retries once and gives up. Now pinned exactly, derived from the model
rather than from the observed constant: three sizes per fill, one fill
per gets() plus one per byte the fallback consumes.
Verified: all nine cases pass under -fsanitize=address,undefined.
Falsified: latch the allocation arm as well and both new assertions
fail alongside the transient case, three in total.
The seam as written put a non-static mutable function pointer into a
core libretro-common translation unit and gated it on a bespoke
FILESTREAM_RBUF_TEST_HOOKS. It compiled out, but it was a test-only
symbol living in file_stream.c, and the sample Makefile defined the
gate in the shared CFLAGS - so all six binaries in that directory were
linking a file_stream.o that was not the one the tree ships. A suite
that builds its subject differently from production is worth slightly
less than it looks.
Drop the pointer. FILESTREAM_RBUF_MALLOC is now a plain #ifndef
override, and filestream_rbuf_fault_test builds its own
file_stream_rbuf_fault.o with -DFILESTREAM_RBUF_MALLOC pointed straight
at its allocator, force-including a two-line header to declare it. No
new symbol, no runtime branch, no gate macro, and the other five
targets go back to the unmodified object.
#undef after the only use: file_stream.c is part of the griffin unity
build and there is no reason for the macro to outlive the function.
Verified: all six sample binaries build and pass under
-fsanitize=address,undefined in the MMAP=1 lane, and
vfs_seek_contract_test / vfs_mapped_ptr_test / filestream_rbuf_fault_test
pass under MMAP=0. file_stream.c compiles clean under
-std=c89 -ansi -pedantic -Werror=declaration-after-statement.
filestream_rbuf_fill() allocates before it calls tell(), so a handle
that can never report a position - pipe, FIFO, socket, a frontend VFS
supplying read but not tell - takes 16 KiB, latches on the very next
line, and holds that allocation until filestream_close() with not one
byte ever passing through it. Before the latch the buffer at least
stayed live for a handle whose tell() might start working; now it
provably will not until something seeks.
Free it at the latch. rbuf_len is necessarily 0 there - a span only
exists after a fill that succeeded, which means tell() succeeded - so
there is nothing to discard and no position to restore. If a later
successful seek clears the latch, the next fill allocates again, which
is one malloc per clear and is the same bound the clear already had.
This is 16 KiB per degraded handle on exactly the memory-constrained
targets the rest of this series argues about.
case_latch_rearms_once_after_seek covers it, and covers the thing the
suite was missing besides: case_tell_latch_clears_on_seek lets tell()
start working before the seek, so it only shows the clear happening,
never what a wrong clear costs. The new case leaves tell() failing
across the seek - the backend-implements-seek-but-not-tell case, where
clearing buys nothing - scans the whole file, and pins the cost at two
tell() calls for the run rather than one per byte. Two allocations for
the same run pins the free.
Verified: all nine cases pass under -fsanitize=address,undefined.
Falsified: drop the free and the allocation assertion fails alone; drop
the clear in filestream_seek() and the two tell-count assertions fail.
* fix(filestream): latch a failed tell() in the gets() lookahead
filestream_rbuf_fill() gives up in two places and returns -1 from both,
and filestream_gets()/filestream_getc() fall back to the byte path when
it does. Neither -1 is self-clearing, and getc() calls fill() again for
every byte, so the fallback re-derives the same failure per byte:
- tell() failed. filestream_raw_tell() calls the same callback on the
same handle with nothing changed in between, so the second attempt
is bit-for-bit the first. The degraded path therefore costs one
failed tell() plus one 1-byte read per byte, where the code before
the lookahead cost just the 1-byte read. The fallback is slower
than the thing it replaced rather than equal to it.
- the allocation failed. rbuf stays NULL, so the next fill() reissues
the same 16 KiB request, once per byte, under whatever condition
refused it the first time.
Latch the first, not the second. A handle that cannot report a position
is a handle whose backend does not implement the operation - a pipe, a
FIFO, a socket, a frontend VFS with read but no tell - and that is a
property, not a moment, so one bool records it and fill() returns early
ever after. It is never cleared: seek and tell are separate VFS
callbacks and a backend may implement either without the other, so a
successful seek is not evidence the latch was wrong, and clearing there
would re-arm one failed tell() per byte after every seek.
An allocation failure is a moment, and it is likeliest on exactly the
memory-constrained targets the lookahead exists for. Latching it would
pin such a handle to per-byte reads for good. Retrying it unchanged is
already what the code does and does not fix the cadence, so fill()
steps the request down instead - 16 KiB, then 4, then 1 - and keeps
whichever size the allocator grants. rbuf_cap was already the only
place the size is consulted after allocation, so nothing downstream
changes. A handle that gets 1 KiB is a buffered handle, which is
strictly better than bounding the cost of a degraded one; the nominal
size is unchanged and the ladder runs only after a refusal.
The buffer stays at 16 KiB and there is no new size knob.
Measured on the fixture the new sample builds - 40,000 bytes of
generated ASCII, read line by line through a VFS shim that counts
callbacks (Apple M1, macOS 26.5, clang, -fsanitize=address,undefined):
tell() always fails 41,097 tell calls -> 1
16 KiB refused, less OK 40,002 one-byte reads -> 12 buffered reads
every size refused 41,097 allocation attempts -> 123,291
The last row is a cost, not a win: with nothing allocatable the ladder
tries three sizes per fill instead of one. That state is a process that
cannot obtain 1 KiB, and the row above it is what the ladder is for.
sizeof(RFILE) is 64 bytes before and after on LP64 - the bool lands in
err_flag's existing padding - measured on this host, not computed. No
32-bit toolchain here, so ILP32 is not measured.
Not addressed here, and not caused by this change: filestream_raw_tell()
sets err_flag on failure, so filestream_error() reports true after an
ordinary successful gets() on a non-seekable handle. The latch reduces
that from once per byte to once per handle; it does not remove it.
samples/file/vfs/filestream_rbuf_fault_test drives both arms, because
neither is reachable otherwise and a failure path no test can enter is
not verifiable. The tell arm needs nothing from the implementation -
filestream_vfs_init() already takes a caller-supplied VFS - but a
refused allocation has no seam, so that one allocation goes through
FILESTREAM_RBUF_MALLOC(). Undefined, which is every build including
RetroArch's, the macro is malloc() and the indirection is not compiled.
Built against unmodified master with only that seam grafted on, the
test reports 5 failures; against this change, 0. All fixtures are
generated; nothing here reads or requires any real data.
* fix(filestream): clear the tell() latch on a successful seek
The latch as first written was never cleared, and the comment justifying
that gave two reasons. One is true and is a cost bound rather than an
objection; the other does not hold.
True: seek and tell are separate VFS callbacks and a backend may implement
either without the other, so a successful seek is not proof that tell() will
now work.
Not true: "clearing there would re-arm one failed tell() per byte after every
seek". The latch re-arms on the FIRST failure after being cleared, so one
fill() asks tell() and every later fill() short-circuits without calling
anything. The cost of clearing wrongly is one failed tell() per successful
seek, not one per byte. That is now asserted rather than argued -- the new
case reports 2 tell() calls across two scans, and reports 1 when the clear is
removed.
What the never-cleared version got wrong is that tell() fails for two
unrelated reasons and only one of them is a property of the handle. A backend
with no tell callback -- pipe, FIFO, socket -- fails every time, and cannot be
seeked either, so its latch is never cleared and it behaves exactly as before.
But retro_vfs_fp_tell64() falls through to ftell() where none of
RETRO_VFS_HAVE_FSEEKI64 / RETRO_VFS_HAVE_FPOS64 / HAVE_64BIT_OFFSETS is set,
and ftell() returns -1 past LONG_MAX. There the failure is a property of WHERE
the handle is, and seeking back under 2 GiB makes tell() work again. A
permanent latch would pin an ordinary seekable file to per-byte reads for the
rest of its life because it once crossed 2 GiB -- the same permanence this
change refuses for the allocation case, reached by a different route.
The clear is in both arms of filestream_seek(), and the spanless arm is the one
that matters: a latched handle has no span, so it takes the bottom path and not
the branch guarded by rbuf_len != 0. Clearing only in that branch would have
been dead code for exactly the handles the latch is set on.
The struct comment and the existing test case both enumerated the no-callback
backends as though they were the whole population of tell() failures. Both are
corrected; conflating the two is what makes a permanent latch look safe.
Verified: all cases in samples/file/vfs/filestream_rbuf_fault_test pass, and
the new case fails on its load-bearing assertion when the clear is removed.
Co-Authored-By: Claude Opus 5 <[email protected]>
* fix(filestream): drop the unreachable clear, and test that a failed seek holds the latch
Two defects found by asking whether the previous commit was actually correct
rather than whether its tests passed.
The clear was written into both arms of filestream_seek(). The arm guarded by
rbuf_len != 0 can never run with the latch set: rbuf_len is assigned only by a
fill that succeeded, a successful fill means tell() succeeded, and both gets()
and getc() zero rbuf_pos and rbuf_len when the span drains. A latched handle
therefore always has rbuf_len == 0 and always takes the other path. Removed,
and the invariant is stated where the remaining clear lives so the next reader
does not re-add it.
Nothing asserted that a FAILED seek leaves the latch alone. Only a successful
seek is evidence the position may have moved; a failed one leaves the handle
where it was, and on a pipe -- which can neither tell nor seek -- clearing on a
failed seek would re-arm a tell() the code already knows will fail. An
implementation that cleared unconditionally passed the entire suite before this
case existed. It now fails on exactly one assertion, verified by removing the
success guard and running it.
The new case makes the seek fail by seeking to a negative offset rather than
through a fault hook: it fails through the real path with no injection, which
is one less thing the test has to be trusted about.
Verified: all seven cases pass; all six VFS sample binaries pass, including
vfs_seek_contract_test, which exercises filestream_seek directly and covers
files past 4 GiB; libretro-common's test_hash, which links file_stream.c, runs
clean under libcheck. Against unmodified master with only the allocation seam
grafted on, the suite now reports 8 failures.
Not verified, and it is the limitation that matters: this is a 64-bit build on
one platform. The ftell()/LONG_MAX case the seek clear defends against can only
occur where long is 32 bits, which is precisely the configuration never
compiled here. The clear does not rest on that case alone -- see the struct
comment for the asymmetry that justifies it independently -- but the case
itself is reasoned, not reproduced.
Co-Authored-By: Claude Opus 5 <[email protected]>
* ci: register filestream_rbuf_fault_test in RUN_TARGETS
The sample was added to the Makefile but not to the run manifest, so CI
built it every run and never executed it. The job classifies that as an
error rather than a skip, which is why it surfaced immediately -- and the
comment above NORUN_TARGETS says why the check exists: a silent
not-in-allowlist skip is indistinguishable from an omission, and seven
regression tests were once compiled every run and never run.
It belongs in RUN_TARGETS, not NORUN_TARGETS: it is self-contained, takes
no arguments, generates its own fixture and exits non-zero on failure.
---------
Co-authored-by: Claude Opus 5 <[email protected]>
The section named 128Kb as the low end, which is off by more than an
order of magnitude: GEKKO workers get 8 KiB (STACKSIZE in
rthreads/gx_pthread.h) and 3DS 32 KiB. Replace the figure with the
stacks the tree actually creates, and state the budget that
tools/stack_budget.py already enforces in CI for libretro-common,
along with the reasons for it - small console stacks, cumulative
depth along a call chain, and the __chkstk/guard-page behaviour that
makes one page the natural ceiling on Windows.
Also documents how to measure a frame, why -DPSP is part of that, why
absolute sizes are not comparable across target triples, and what to
do with a frame that is too large.
With the scanner reading spans and never writing, the include cache
no longer has any reason to duplicate what it holds.
A hit previously malloc'd a buffer the size of the whole file and
memcpy'd into it, purely because the caller used to write into what it
was given. It now hands back a pointer to the bytes it already holds.
A miss previously read the file and then allocated and filled a second
copy to retain; it now retains the buffer it just read.
Ownership is explicit: slang_include_cache_read() reports whether the
caller owns the bytes, which happens only when the entry could not be
recorded - no cache, an empty file, or an allocation failure - so a
cache that cannot grow still degrades to the old behaviour rather than
failing the load.
On a preset whose passes share a 26 KB header, expanding six passes
four times over: 959,600 bytes allocated before, 329,583 after, with
allocation count down from 162 to 114. Output is byte-identical to
upstream master across the corpus, and 60 allocation-failure injection
points run clean under ASan/UBSan/LeakSanitizer.
The line scanner walked the file buffer destructively: it planted a
NUL at buf_len, compacted \r out of each line in place, shifted the
line delimiter down to the compacted end and then wrote a temporary
NUL over it to terminate the line. The buffer therefore had to be
private to the caller, which is exactly why the include cache hands
out a fresh malloc'd copy of the whole file on every hit.
Nothing downstream actually required any of that. shader_line_buf
_append() is length-based - it memcpy()s the span and appends its own
terminator - and every keyword test is a memcmp() already guarded by
an explicit length check. Only two consumers wanted a C string: the
include filename (copied out in the preceding commit) and the invalid
include diagnostic, which now uses a bounded "%.*s".
Scan spans instead. memchr() finds the line end; \r costs nothing in
the two cases real files produce - none at all, or one trailing from
CRLF, where the span is simply one byte shorter. A line with an
interior \r is compacted into scratch sized to that single line, so
the previous behaviour of stripping every \r wherever it sits is
preserved without a full-file copy.
The "#version " test becomes a length-guarded memcmp() rather than a
strncmp() that relied on the planted terminator to stop; a line too
short to hold the prefix is rejected exactly as before, when the
terminator made the comparison fail.
Dropping the write at buf_len also removes this code's dependence on
filestream_read_file() having allocated the extra byte.
Output is byte-identical: verified line for line over a corpus of LF,
CRLF, interior-\r and lone-\r sources, nested and repeated includes,
present and missing #pragma include_optional, empty roots and empty
includes, a missing #version, a malformed include, and a file with no
trailing newline.
slang_get_include_file() terminated the quoted include name by writing
a NUL over its closing quote and returning a pointer into the file
buffer. That is one of the reasons the buffer has to be writable, and
the include cache pays for it: every cache hit hands out a fresh
malloc'd copy of the whole file precisely because the scanner writes
into what it is given.
Copy the name out instead. The helper takes a const span and a
bounded destination; an over-long name is truncated to fit, which is
what fill_pathname_resolve_relative() already did with it.
The destination is the existing tmp scratch rather than a new local:
tmp's only use is the #line directive built after the recursive call,
so it is free at this point, and this function recurses once per level
of include nesting - a third PATH_MAX_LENGTH array per frame is not
free on platforms with small stacks.
No output change; this only removes one writer of the file buffer.
update_cand() derives a track's span from last_index - cand_index on
signed int64_t values, assuming the sheet lists indices in ascending
order. A malformed or truncated sheet can violate that: most simply,
a sheet missing its final newline makes the tokenizer discard the
trailing INDEX token, so the last track's index is never recorded and
last_index stays behind cand_index. The signed subtraction then goes
negative and, stored into the uint64_t largest and handed on as a
size_t size, becomes a near-SIZE_MAX value.
Downstream that value reaches the serial-probe staging malloc in
intfstream_file_get_serial_fd(). The span is clamped to
SERIAL_PROBE_SPAN only inside the bounded branch (offset != 0 ||
size < file_size); a whole-file candidate that skips the clamp
requests the full underflowed size. malloc() returns NULL and the
read is checked, so there is no overflow, but a crafted cue in a
scanned directory can still provoke a multi-exabyte allocation
request - needless work driven entirely by a garbage length.
Guard the subtraction: a candidate whose last_index is below its
cand_index has no valid span and is skipped, exactly as a zero-length
track would be. Well-formed sheets are unaffected.
INTFSTREAM_BUFFERED's line and character APIs looped
intfstream_read(intf, &c, 1). Since the BUFFERED backend fills a
resident window, those one-byte reads were already window hits rather
than VFS calls - the underlying file was read one fill per window
either way - but every byte still paid a full call into
intfstream_read() plus the type switch and bounds checks, and
getc/gets consumers issue enough calls for that dispatch to be
measurable (the window's own read path keeps its hit test inline for
exactly this reason).
Serve both directly from the window instead:
- gets copies line spans: one hit test, one memchr for the newline
and one memcpy per window visit, refilling only at window edges.
- getc performs the inline hit test and returns the byte from the
window, refilling on miss.
Semantics are unchanged: gets stores at most len - 1 characters plus
the terminating NUL, includes the newline when it fits, returns NULL
at EOF with nothing stored, and returns the partial line when a fill
fails mid-line; getc returns EOF past the end or on a failed fill.
Underlying reads remain one filestream_read per window fill; what
this removes is the per-byte dispatch on top of it.
Resolving a cue or GDI sheet's data track used to open the track
binary once per question asked of it: the parse opened it to learn
its size and closed it (intfstream_get_file_size), the CD-i
AUDIO-track check opened it again for twelve magic bytes, and the
serial or CRC probe then opened it a third time. On the library scan
path that is two to three opens per referenced binary per sheet, and
the serial-to-CRC fallback repeats the whole sequence.
Keep the handle instead of re-opening the file:
- cue_track_stream_open() replaces the open-for-size-and-close
helper: one open reports the size and stays open.
- cue_find_track_internal() / gdi_find_track_internal() carry the
current file's stream through the parse and, whenever the candidate
data track changes, move that stream into the candidate slot
(cue_cand_adopt_stream). On success the winning track's open
stream and its size are handed to the caller; non-winning streams
are closed as they are displaced. The existing public
cue_find_track()/gdi_find_track() signatures remain as wrappers
that decline the stream.
- The CD-i magic probe reads through the stream the FILE line just
opened instead of opening the track separately. Later consumers
seek before reading, so the position disturbance is harmless.
- intfstream_file_get_serial() / intfstream_file_get_crc_and_size()
are split into _fd cores that run on a caller-owned stream and
accept a pre-measured file size (skipping the seek-END round trip
the parse already paid for), with the by-path functions kept as
thin open-delegate-close wrappers for their existing callers in
task_database.c.
- task_database_{cue,gdi}_get_serial / _get_crc_and_size consume the
retained stream; if the parse could not open the winning track (so
no stream was retained) they fall back to the by-path probe, which
fails the same way it always has.
A cue with one data track now costs exactly one open of the binary
for the size + serial (or size + CRC) sequence, and the ~600 KiB
serial staging window plus the 256 KiB CRC fold both run over the
handle the parse already held. Serial and CRC results are unchanged;
only how the bytes are reached changed.
Every token in a cue or GDI sheet was assembled one character at a
time with intfstream_read(fd, c, 1): a full VFS round trip per byte
of every sheet touched by a library scan. The sheets are KB-scale
text, so the scan-side cost of parsing them was almost entirely
per-call overhead, multiplied across a disc-heavy collection.
Feed the tokenizer from a small refill window instead. The window
is 1 KiB on the caller's stack: a typical single-bin cue is consumed
in one bulk read and a large multi-track sheet in a handful, while
the parse itself allocates nothing and holds no more than the window
no matter how large the sheet is - deliberately not a whole-file
slurp. The token state machine is carried over unchanged, including
its historical quirks (leading whitespace skipped even inside quotes;
a final token with no trailing newline is discarded at EOF), so parse
results are byte-identical.
cue_next_file() and gdi_next_file() operate on a caller-owned stream,
are re-entered with the same handle, and gdi_next_file() keys its
header skip off intfstream_tell(). For those, cue_tok_detach()
rewinds whatever the window read ahead before returning, so the
caller-visible stream position is exactly what byte-at-a-time
tokenizing produced. cue_find_track() and gdi_find_track() own their
stream for the whole parse and need no repositioning.
The tokenizer's stored-length cap carries over the previous commit's
bound: at most len - 1 characters plus the terminating NUL.
task_database_cue_get_token() stores each character at *c and only
then checks the length cap:
_len++;
c++;
if (_len == (int64_t)len)
{
*c = '\0';
return _len;
}
When a token reaches len characters, c has advanced to s + len and
the terminating NUL is written one byte past the caller's buffer.
Every caller passes a 255-byte stack array (tmp_token[MAX_TOKEN_LEN]),
so any cue/gdi sheet containing a token of 255 or more characters -
a long quoted FILE path is enough - writes a NUL over whatever the
compiler placed after tmp_token. Cue sheets are downloaded alongside
romsets, so this is attacker-reachable input on the library scan path.
Stop storing at len - 1 characters so the NUL lands on the last byte
of the buffer. The longest representable token shrinks from 255 to
254 characters; delimiter-terminated tokens are unaffected (their NUL
overwrites the delimiter slot, which was always in bounds). Also
reject len < 2 outright rather than looping on a buffer that cannot
hold a character and a terminator.
Found while preparing the buffered-tokenizer conversion of this
parser.
Serial/system detectors resolve within the first 600000 bytes of a
data track (DISC_DATA_SIZE_PS2 is the largest probe, 0x9320 the
deepest magic-number offset), so intfstream_file_get_serial() stages
at most SERIAL_PROBE_SPAN of a bounded span rather than the whole
track. On a 650 MB single-bin rip this takes the probe from 4523 ms
and a 622 MiB peak allocation (cold cache) to 1.5 ms and 2.9 MiB per
scanned disc; detection is byte-identical, verified on Saturn and
Sega CD fixtures covering spans longer than the window, tracks
shorter than it, and data tracks at non-zero offsets.
Follow-up to the formats I/O decoupling: of the three reworked
modules, rbmp was the only one whose remaining intfstream use was
convenience rather than necessity - BMP has no compression stage, the
primary rbmp_save_image_string() already fills its exact-size buffer
directly, and the only stream user in the TU was
rbmp_save_image_stream(), whose callers (the deprecated path adapter
and the screenshot task's allocation-failure fallback) are edge code
that is allowed to speak streams.
Publish the row layer the encoders were already built on:
- rbmp_row_size(): padded on-disk size of one row.
- rbmp_encode_row(): produce one padded output row into caller
scratch, or hand back the source row unchanged when its bytes can
be emitted as-is.
and move rbmp_save_image_stream() - signature, symbol and behaviour
unchanged, including the whole-block single-write fast path - into
file/rbmp_file.c, reimplemented over those primitives next to the
path adapter. Callers keep linking exactly as before.
formats/bmp/rbmp_encode.c is now genuinely dependency-free: its only
includes are stdlib.h, string.h and its own header - no stream, VFS
or libretro-common linkage of any kind - so the encoder can be
vendored as a single file. cdfs keeps intfstream because random
access over large, possibly CHD/RZIP-backed images is the module's
purpose; rpng keeps it because the deflate encoder needs an
incremental sink and replacing a just-published public stream API for
a measured <0.1% gain is churn without payoff.
rbmp_encode_test gains direct coverage of the primitives (sizing,
pass-through vs scratch conversion, padding determinism, argument
validation) alongside the existing string/stream/path byte-identity
and decoder round-trip checks. Swept clean under ASan/UBSan/LeakSan
(strict) and TSan; both touched TUs pass the C89 lane; full frontend
rebuild links with all four public symbols present.
Follow-up to 253c16e ("formats: decouple file I/O from
rpng/rbmp/cdfs, bulk-write screenshots"):
Relocate the deprecated path-convenience adapter TUs from
formats/png/rpng_file.c and formats/bmp/rbmp_file.c to
libretro-common/file/, where the rest of the path-layer code lives.
With this move the formats tree contains no path I/O at all - every
TU under formats/ speaks only buffers and streams - and the
grep-level invariant is trivial to audit. Filenames, symbols and
signatures are unchanged; only the location moves, with
Makefile.common, griffin and the sample Makefiles updated to match.
Also register the three sample binaries introduced by 253c16e in the
Linux-libretro-common-samples workflow's target lists, fixing the
currently red job: cdfs_pure_api_test and rbmp_encode_test as
pass/fail runs, and screenshot_io_bench as an argless run - the
benchmark exits 0 and, under the job's ASan/UBSan build, doubles as
end-to-end sanitizer smoke coverage of both the old and new
screenshot encode/write paths at its default 4K size (its timing
output is informational only).
Make rpng_encode, rbmp_encode and cdfs pure format modules - bytes or
streams in, bytes or streams out - with path open/close reduced to
thin adapters, following the pattern already applied to rchd, rm3u,
logiqx_dat and rxml. The file I/O that moves to the call sites is
redesigned rather than relocated: the screenshot task now encodes to
memory and hits the filesystem with a single bulk write.
rpng: rpng_encode.c already had pure stream/string entry points; the
four path wrappers (rpng_save_image_argb / _bgr24 / _rgba /
_rgb48_hdr) move to a new adapter TU, formats/png/rpng_file.c, with
unchanged signatures and symbols. rpng_save_image_stream_fmt() and
enum rpng_pixfmt gain declarations in rpng.h so adapters and callers
can drive the encoder without duplicating per-format plumbing. The
pure TU no longer includes file_stream.h.
rbmp: rbmp_encode.c is reworked into a pure encoder. New primary API
rbmp_save_image_string() encodes the whole image into a single
exactly-sized allocation (a BMP's size is fully determined by its
dimensions); rbmp_save_image_stream() encodes onto a caller-provided
intfstream_t with one padded row of scratch and byte-identical
output. The old rbmp_save_image(path) survives unchanged as a
deprecated adapter in formats/bmp/rbmp_file.c. Two latent hazards in
the old writer are fixed: the BGR24 path copied stride-many bytes per
row (negative strides went through filestream_write as negative
lengths; over-wide strides corrupted the fixed row layout) and is now
bounded by the row payload with padding explicitly zeroed, so
tightly-allocated viewport readback buffers with unaligned widths are
never read past their final row; unknown source types now emit a
valid zero-filled image instead of a truncated header-only file.
cdfs: cdfs.c is restructured into a documented pure layer and a path
adapter layer (the physical TU split is deliberately not taken -
every core vendoring cdfs.c would need build churn for no behavioural
gain). cdfs_track_from_stream() publishes the previously private
stream wrap with documented ownership and a checked allocation.
cdfs_parse_cue() extracts cue interpretation as a pure, bounded,
non-mutating parser, removing several out-of-bounds hazards of the
old in-place parser (reliance on unchecked NUL termination,
atoi(track + 6) past short final lines, mis-tokenised tab-separated
cues). The cdfs_open_*_track family remains as adapters with
unchanged signatures; the cue adapter now does one bulk read of the
cue and releases that handle before opening the bin; CHD stays
confined to the adapter layer.
cdfs positioning fixes: the byte-granular seek layer had three
compounding bugs that made cdfs_seek / cdfs_tell unusable on files
opened by name (first_sector != 0): cdfs_seek compared a
file-relative sector against the disc-absolute current_sector, so
every seek rewound reads to offset 0; the post-seek cache refill
ignored current_sector_offset; and the cache-hit read fast path never
advanced file->pos, so cdfs_tell / SEEK_CUR drifted. All three are
fixed and locked in by the new sample test.
task_screenshot: all PNG paths and the RBMP fallback encode into an
in-memory buffer via the *_string APIs and write with a single
filestream_write_file() - one open, one sequential write, one close -
instead of interleaving small VFS writes through the whole
compression run while holding the handle. The BGR24 no-resize fast
path feeds the viewport readback straight to the encoder with a
negative stride, deleting a full-frame flip-and-copy (~48 MiB of
allocation plus copy per 4K screenshot). If the whole-image
allocation fails, the task falls back to streaming the encode with a
few rows of scratch, preserving the old memory profile as a degraded
mode instead of failing the screenshot. Encoding already runs on the
task worker thread, so the UI-facing cost is one bulk write.
Measured at 3840x2160 (local fs, avg of 3-5 runs): PNG 182.0 ->
168.6 ms/frame with peak RSS 49.6 -> 25.8 MiB on compressible
frames (the string encoder's worst-case bound is virtual; only
compressed pages are touched) and 689.6 -> 672.0 ms at equal RSS on
incompressible noise; BMP 28.4 -> 25.8 ms with write syscalls per
frame dropping 4322 -> 3. The BMP buffered transient equals the
file size, which on the !HAVE_RPNG platforms that actually reach the
BMP path (low-resolution builds) is a few hundred KB, with the
stream fallback as the safety net. On slow or contended media (SD,
network mounts, Android SAF) the single-write pattern is the point;
the in-tree benchmark below reproduces the comparison on target
hardware.
Tests and tooling: cdfs_pure_api_test drives the pure cdfs layer
entirely in memory (cue parsing incl. quoted names, CRLF, tabs,
pregap sector arithmetic, audio-track skipping, truncation fuzzing;
raw MODE1/2352 ISO-9660 walk, multi-sector reads, cross-sector seeks,
tell accounting). rbmp_encode_test covers all four source types,
positive and negative pitch, unaligned-width padding, header fields,
string/stream/path-adapter byte identity, and pixel-exact decoding
through the in-tree rbmp decoder. rpng_roundtrip_test gains
XRGB8888/RGB565 coverage of rpng_save_image_stream_fmt, the exact
call shape the screenshot fast paths use. samples/formats/bench/
screenshot_io_bench replicates the old and new task I/O patterns for
reproducible measurement. All suites run clean under ASan, UBSan,
LeakSan (strict) and TSan; touched libretro-common TUs compile clean
under the C89 lane, including -DHAVE_CHD.
External ABI: rbmp_save_image, the rpng_save_image_* path wrappers
and the cdfs_open_*_track family keep their signatures and symbol
names; out-of-tree users keep linking so long as they pull in the
adapter TUs, which are wired into Makefile.common, griffin and the
sample Makefiles here.
Cores issue RETRO_ENVIRONMENT_GET_SCREEN_10BPC_CAPABLE from
retro_load_game while choosing their pixel format, i.e. during
CMD_EVENT_CORE_INIT. On a cold start into content that runs before any
video driver exists for the session being started, and on an in-process
core switch the outgoing instance has been torn down, which clears
ctx->get_flags and video_st->poke. The handler answered purely from
live instance flags via video_driver_test_all_flags(), which cannot
distinguish "driver says no" from "no driver exists to ask": with no
instance it reported "no" on every machine, purely because of when the
core happened to ask.
A core that trusts the answer then commits to an 8-bit format before
anything better is known, and the refusal is final -- SET_PIXEL_FORMAT
follows immediately. The visible result with an HDR10-capable core on a
Vulkan HDR10 swapchain:
[Environ] GET_VARIABLE: <core>-color_format = "30bits (HDR)"
Color Format: HDR10 requested, but the frontend does not present a
10-bit source natively -- using 24-bit truecolor instead.
[Environ] SET_PIXEL_FORMAT: XRGB8888.
[Vulkan] Source is SDR (32-bit), swapchain HDR10.
i.e. the frontend inverse-tonemaps an 8-bit SDR frame onto the HDR10
swapchain instead of taking the core's native PQ frames -- while the
same content negotiated HDR10 fine when loaded from a running menu,
because the menu session's live instance answered the query. That
asymmetry made the failure look driver- or core-specific when it is
neither.
This is the same load-time window already fixed on the SET side:
SET_PIXEL_FORMAT's HDR10_2101010 gate consults the configured driver
ident through video_driver_supports_10bit_source() precisely because no
live-instance state is trustworthy at that point (see the rationale
above that helper). The GET handler was missed in that fix.
Answer from live flags when an instance exists -- gl2/gl3/gl1 gate the
flag on actual context capability, so a live answer is strictly better
-- and fall back to the configured driver ident when none does,
matching the SET-side gate. The two errors remain asymmetric in the
fallback: accepting and discovering otherwise is recoverable (the
driver tonemaps PQ for its actual output, and SET still refuses when
HDR output is off), refusing is final.
Verified on Linux with prboom (color_format "30bits (HDR)",
video_hdr_mode 1), cold start into content: HEAD reproduces the
XRGB8888 fallback verbatim; with this change the core negotiates
HDR10_2101010 ("Color Format: HDR10, paper white 200 nits"). HDR-off
and non-capable-driver paths still degrade to XRGB8888 through the
existing refusals. ASan/UBSan/LeakSan/TSan clean (leak reports
byte-identical to HEAD control run; TSan run includes 4 core render
threads).
* netplay: bound the wait for tunnel information
Stage 1 of making netplay host init non-blocking: the hang becomes an
error, with no new states and no restructuring.
Hosting through a relay waits for the lobby server to answer with a
tunnel address. The wait was already narrowed to that one query, but
it was still unbounded - and it is a round trip to a machine we do
not control. A lobby server that is down, blackholed or merely slow
left the frontend hung with no way out but killing it. It now gives
up after ten seconds and reports failure like any other query, so the
caller falls back to direct mode with the warning it already has.
Generous on purpose: timing out a query that would have succeeded
costs the user their session, while waiting a few seconds longer
costs only patience.
task_queue_wait_timeout() is the bound. It wraps the caller's
condition in one that also goes false at a deadline and hands that to
the existing wait, so every queue implementation - regular, threaded
and GCD - keeps its own waiting behaviour and no platform-specific
code is touched. Adding a variant to the implementation vtable would
have meant editing the GCD path blind, which is the shape of a recent
break and not worth repeating for this.
A timeout alone would have been a bug, not a fix. The query is still
in flight when we give up, and its callback writes
host_room->mitm_address and mitm_port unconditionally - so it would
land after the fallback and publish a tunnel address for a session
running direct. Each query now carries a generation token and the
callback ignores results that are no longer current; abandoning a
query bumps the generation. That also closes the pre-existing case
of two queries overlapping.
The oracle covers the primitive rather than netplay's HTTP, because
that is where the risk is: the netplay side is a few lines routing
into an else branch that already existed. Five lanes over both queue
implementations assert that a satisfiable wait returns as soon as it
is satisfied - not after sitting out the timeout, which would pass a
naive check - and that an unsatisfiable one gives up near the
deadline rather than early or never. Removing the deadline hangs the
lane until CI kills it, which is the behaviour being removed.
Verified: builds with networking ENABLED and DISABLED, the latter
being the configuration class that broke recently; task_queue.c links
for ARM32 without NEON; all nine task oracles still pass, since that
file is shared; the new lanes pass plain, under ASan+UBSan+LSan and
twice under TSan.
Not verified: behaviour against a real unresponsive lobby server.
* netplay: split the tunnel query into begin / await / result
Stage 2 of making netplay host init non-blocking: a refactor that
changes no behaviour, so it can be reviewed on its own before
anything moves.
netplay_mitm_query() welded "ask the lobby server" to "wait for the
answer". Those are now netplay_mitm_query_begin(),
netplay_mitm_query_ready(), netplay_mitm_query_await() and
netplay_mitm_query_result(), with netplay_mitm_query() calling them
back to back at the same call site. Nothing is deferred and nothing
moves; the point is that the next step - issuing the query when the
user opens the host UI, so the answer is already there when host
setup runs - becomes a change of WHERE begin() is called rather than
a restructuring of host init.
Every path was walked against the original:
handle NULL/empty false begin() false
custom, decode fails false begin() false
custom, decode ok address set, result begin(), await(), result()
http push fails pending cleared,false begin() false
http ok, reply in time wait, then result await() waits, result()
http ok, timeout warn, bump gen, false await() identical
One deliberate difference, because the split exposed an edge the old
shape hid. The custom-relay path answers synchronously and the old
code returned without ever consulting the pending flag - so an
earlier query left outstanding could not make it wait. Routed
through await() it would have, so the custom path now supersedes any
query in flight: it bumps the generation and clears the flag. That
also stops a late callback from overwriting the address it just set,
which the old code would have allowed.
Verified: builds with networking enabled AND disabled; the
wait_timeout CI step replayed verbatim; the netplay NAT, task
ordering and watchdog oracles still pass.
Not verified: behaviour against a real lobby server.
* netplay: start the tunnel query when the user commits to hosting
Stage 3, and the one that removes the stall the user actually feels.
Host setup needs the relay address, and the query for it was issued
at the moment of need - so the wait was the full round trip, every
time. It is now issued at CMD_EVENT_NETPLAY_ENABLE_HOST, where the
user commits to hosting, which is well before the address is wanted:
on the content-reload path, before content has even started loading.
The round trip overlaps work that was going to happen anyway, so by
the time host setup asks, the answer is usually already there and the
wait costs nothing.
Nothing is deferred and host init is not restructured. The wait
stays exactly where it was, as the backstop for the case where the
answer has not arrived yet - which is what makes this cheap to
reason about and cheap to revert.
Prefetching only matters if asking twice is avoided, so the query
records the handle it was issued for, and host setup reuses an
answer that is already in hand or on its way. It deliberately does
NOT reuse a prefetch that came back empty: a transient lobby failure
at prefetch time would otherwise doom a session that asking again
would have got, and one extra request is the better trade. Every
state was walked:
no prefetch query issued as before
prefetch in flight, same reused, one query, overlapped
prefetch answered reused, no wait at all
prefetch came back empty re-issued
relay changed since re-issued, picks the new server
custom relay answered synchronously, reused
The prefetch is best effort in the strictest sense: skipped when the
relay is off or unconfigured, and if it fails for any reason host
setup asks exactly as it did before. It is not speculative - the
user has already chosen to host, so no request is made for anyone who
merely opens a menu.
Verified: the call site sits inside #ifdef HAVE_NETWORKING, computed
from the preprocessor nesting rather than eyeballed, and both builds
were made - networking enabled AND disabled, the configuration class
that broke recently. The wait_timeout CI step was replayed verbatim
from the workflow, and the netplay NAT and task ordering oracles
pass.
Not verified: behaviour against a real lobby server, including
whether the overlap is long enough in practice to hide the round
trip on the content-reload path.
* content close: pin the save wait, and say what it is waiting for
Stages 0 and 1 of unblocking content close. Nothing about the wait
changes; this establishes what it guarantees and stops it looking
like a hang.
Stage 0, the oracle lane. Closing content waits for in-flight save
and load tasks because their handlers call into core function
pointers - retro_serialize via content_get_serialized_data,
retro_unserialize via core_unserialize - while
runloop_event_deinit_core() unloads the dylib underneath them. A
worker mid-call when the library goes away dispatches into freed
code. That is why this wait, unlike the others removed in this
series, cannot simply be deleted.
The lane pins the property the unload depends on: the file is
checked incomplete before content_wait_for_save_state_task() and
complete after, so a wait that returned early - on a budget
boundary, say - fails here rather than becoming a crash on a device.
Neutering the wait fails it, as it should.
Stage 1, legibility. The wait cannot be shortened without deferring
the teardown itself, which is later work, but an unexplained frozen
frame and a frame that says "saving state" are very different
experiences - and on slow storage this is seconds, not milliseconds.
The message is pushed and one frame forced out before blocking,
since nothing draws once the wait starts.
It fires only when there is something to wait for, using the same
condition the wait itself uses, so an ordinary close with no save in
flight is untouched and shows nothing. That predicate had to become
public to be asked from here; its load counterpart already was.
Verified: builds with networking enabled and disabled; the
save_state oracle passes, including the new lane, and fails when the
wait is neutered.
Not verified on a device: whether the message reads well in the case
that matters, which is the slow-storage close it exists for.
* runloop: introduce the content-closing state, observed by nothing
Stage 2 of unblocking content close. A no-op by construction: the
state is set and cleared around the teardown that already exists, so
while it is true the main thread is inside that teardown and no
frame runs to ask about it.
It is introduced on its own precisely because the work that makes it
observable - returning to the frame loop instead of blocking on the
save wait - is a lifecycle change, and this is the piece the rest
will key off. Reviewing the two separately means a problem with the
lifecycle change is one revert, not an unpicking.
A plain bool in runloop_state_t rather than a RUNLOOP_FLAG bit.
Bits 0-30 of that word are taken and bit 31 was deliberately vacated
to avoid a cross-thread race with the AppIntents entity query;
reusing it would undo that reasoning for no gain, and this is
main-thread only.
The clear is placed at the block's single exit, which was checked
rather than assumed: there is no early return or goto between the
set and the break, so one clear covers every path. A leaked true
would be harmless today and would wedge the frame loop at stage 3,
which is exactly the kind of thing worth getting right while it
still cannot bite.
Verified: nothing reads the state except its own accessor, which
nothing calls - grepped rather than asserted, since "no-op" is the
entire claim here; builds with networking enabled and disabled; the
save_state oracle passes and the wait_timeout CI step was replayed
verbatim from the workflow.
* task_queue: fix two data races TSan found in CI
Two separate races, one mine and one older.
Mine, in the oracle: work_pending was cleared by the task HANDLER,
which runs on the worker thread, and read by the waiting main
thread. A plain bool across two threads - a race on inspection,
with no tooling needed to see it. It was also a misrepresentation
of the thing under test: the code being modelled,
netplay_mitm_query_cb(), clears its flag in the CALLBACK, which runs
on the thread pumping the queue. Clearing it there fixes the race
and makes the test match what it claims to model.
The older one is in the queue itself. retro_task_threaded_reset()
set RETRO_TASK_FLG_CANCELLED under running_lock, while the worker
reads task->flags under property_lock - two locks for one field.
It needed a reset to land while a worker was mid-task, which is why
it sat undisturbed until a lane that cancels an unfinished task
turned up. The flags are now set under property_lock as well.
Nesting is safe: property_lock is never held while any other lock is
taken anywhere in this file, checked by walking every lock/unlock
pair, so it can only be an inner lock and no cycle exists.
Both are load-bearing, measured rather than assumed: 40 TSan runs
report 0 races with both fixes, and reverting the queue fix alone
brings it back at 6/40.
On the reporting of this: the commit that introduced the oracle said
it passed twice under TSan. That was true - I ran it, three times
including the CI replay, and it passed each time. The race
reproduces about once in twelve runs here, so three clean runs were
about a 78% chance of missing it. The claim was accurate and the
verification was not adequate, which is the more useful way to say
it. Repeat counts for anything threaded are now sized against how
rare the failure can be, not against how many runs feel like
enough - and a shared mutable flag written from a handler should
have been caught by reading it, which is where this should have
stopped.
Verified: 40 TSan runs clean on the changed lane; 8 each on the
other threaded oracles, since task_queue.c is shared by all of them;
every task oracle passes on a plain build; desktop build clean; the
CI step replayed verbatim from the workflow.
* runloop: don't run the core while content is closing
Stage 3 of unblocking content close. Still inert, and deliberately
so: it establishes what the frame loop must do while the closing
state is set, before anything can set it with frames running.
retro_run() must not be entered once closing has begun, because the
teardown unloads the library that function lives in. The guard
therefore has to exist before the wait moves out of the close path,
not with it - otherwise the change that lets frames run would be
introducing both the opportunity and the protection at once.
That it cannot fire today was checked rather than assumed: both
core_run() call sites are inside runloop_iterate(), and the content
close block in CMD_EVENT_CORE_DEINIT reaches neither - it calls no
iterate, no core_run and no run_ahead, so the main thread cannot be
in the frame loop while the flag is set.
It polls input and presents a cached frame rather than returning
bare, matching the netplay-paused case immediately below, and for
the same reason: a frame that simply stops being produced reads as a
hang, which is the whole complaint being worked on here.
Verified: builds with networking enabled and disabled; the
save_state oracle passes, including the lane that pins the close
wait; the wait_timeout CI step replayed verbatim from the workflow.
Worth stating before stage 4: at that point this guard stops being
inert, and video_driver_cached_frame() will be running while a core
is being torn down. That is the first thing to look at if the frame
loop misbehaves during a close, and it is why the harness lane for
content close belongs before that change rather than after it.
* runloop: fix C89 violation in core_run
The content-closing guard was placed before netplay_preframe's
declaration, so that declaration followed a statement:
runloop.c:8808: error: ISO C90 forbids mixed declarations and code
netplay_preframe is now declared with the other locals and assigned
where it was, so the guard still returns before netplay's pre-frame
call and the declarations stay at the top of the block.
My own standing rule is that every C file gets the strict C89 gate
before commit, and I did not run it on this one - the ordinary
desktop build does not use those flags, so it compiled clean and I
took that as sufficient. It is not, and the gate exists precisely
because it is not.
Verified with the flags the C89 build actually uses, on runloop.c
with networking both enabled and disabled, and with a full
C89_BUILD=1 compile of the tree; reinstating the original ordering
reproduces the reported error, so the check is real rather than
vacuous.
* samples: drive the content-closing guard against the real frame loop
Stage 4 will stop blocking on the save wait and let frames run while
a close finishes. The guard that makes that safe - core_run()
refusing to enter retro_run() while content_closing is set - is
already in place, and is at present completely unexercised: closing
is synchronous, so no frame runs during it, and the guard would go
live in the same change that first creates the situation it protects
against. If it were wrong, the symptom would be a call into an
unloaded dylib.
So it is driven here first, directly: the real core_run(), in a real
frontend brought up headless, with the flag set. Links the shipping
objects with only main() replaced; nothing is stubbed.
Three assertions, and the first exists to stop the others passing
vacuously: with the flag CLEAR the core must actually be entered, or
this harness would be proving nothing about a core_run() that never
reaches retro_run() in this environment anyway. Then with the flag
set it must not be entered, and once cleared it must resume - a
sticky guard would leave the frame loop dead after a close. A
second lane runs 240 frames with the flag set, the shape of a close
on slow storage, checking core_run() returns cleanly every time
rather than faulting or wedging.
Removing the guard fails both lanes with exactly the message they
exist to produce.
It joins the existing menu-harness workflow, which already makes the
non-Qt build these harnesses need - with Qt enabled ui_qt.o defines
main and the link collides - so it adds about two seconds rather
than another full build. The step was replayed verbatim from the
workflow before commit, and the menu harness alongside it still
passes.
Verified: strict C89 gate on runloop.c with networking on and off,
which is the check I skipped last time and should not have.
* retroarch: extract the content-close teardown into its own function
Preparation for removing the close freeze, and nothing else: the
sixty lines that run after the save/load wait move into
command_event_finish_content_deinit(), still called from exactly
where they used to run.
They have to become resumable. The wait above them is the freeze
this work is about, and removing it means this half runs later -
from the frame loop, once the tasks have finished - so it needs to
be callable from somewhere other than the middle of a switch case.
Doing the extraction on its own makes the next change a question of
WHEN it is called rather than what it does, which is the difference
between a revert and an unpicking if that goes wrong.
Every local is re-derived from its accessor rather than passed in,
so the function carries no dependency on the caller's frame - which
is what lets it be called from the frame loop later.
The extracted body was compared statement by statement against the
original block: 53 statements, zero differences. That check was
worth writing, because the first attempt at this anchored on a
comment that appears twice in the file and silently swallowed an
unrelated case - it failed to compile, but a subtler overshoot would
not have.
Verified: strict C89 gate on retroarch.c with networking on and off;
builds with Qt and headless; both full-program harnesses pass, and
the save_state oracle with them.
---------
Co-authored-by: LibretroAdmin <[email protected]>
GameCube has no salamander and frontend_gx_exec() is a no-op on
HW_DOL, so in-menu core switching is impossible: each core dol is
launched directly from the loader (Swiss etc.). On such a
platform the strict core-info gate in
menu_displaylist_parse_supported_cores() can only ever produce a
dead end: frontend_driver_get_core_extension() returns "dol"
(non-empty) on GEKKO, so when core info matching fails - missing
or renamed info files, stale core_info.cache, misconfigured
directories - content selection ends at 'No Cores Available'
with no recourse, even though the only core that could ever run
the content is already statically linked and running.
Wire the existing LOAD_WITHOUT_CORE_INFO knob (already present
in Makefile.wii, Makefile.wiiu, Makefile.ctr and Makefile.psp1)
into Makefile.ngc, defaulting to 1 since those other platforms
can switch cores at runtime and NGC cannot. With the define set,
the fallback offers the currently running core whenever core
info matching finds nothing; when info files are set up
correctly, matching succeeds and the fallback never fires.
Also move the 'exts' declaration inside the
!defined(LOAD_WITHOUT_CORE_INFO) block in menu_displaylist.c:
it is unused when the define is active, which previously
produced -Wunused-variable on every platform enabling the knob.
Follow-up to 38414d1 ('(GX/GameCube) Support SD2SP2 / Serial
Port 2 SD adapters'), from the same Discord report: the 'no
cores found' half of the issue.
Official GameCube builds link against external devkitPro
libogc/libfat (EXTERNAL_LIBOGC=1), where the GameCube disc
interface table is:
{ "sd", __io_gcsd2 }, /* Serial Port 2 (SD2SP2 etc.) */
{ "carda", __io_gcsda },
{ "cardb", __io_gcsdb },
so an SD2SP2 card mounts as "sd:" and, being first in the table,
becomes the default device. Two pieces of HW_DOL code predate
this and only knew about SD Gecko slots A/B:
1) frontend_gx_get_env() unconditionally ran
chdir("carda:/retroarch"), clobbering the launch directory
that fatInitDefault() had already chdir()'d to from argv[0]
(as provided by Swiss and other modern loaders) whenever a
slot A card with a retroarch folder was present. Only fall
back to a fixed location when the loader supplied no usable
argv[0], and prefer "sd:/retroarch" over "carda:/retroarch"
in that fallback, mirroring libfat's own device priority.
The chdir chain is harmless under the internal libogc, where
no "sd" device exists and the first chdir simply fails.
2) frontend_gx_parse_drive_list() offered only carda:/ and
cardb:/ on HW_DOL, so SD2SP2 users could not browse to their
card's root from Start Directory / Load Content and had to
hand-edit retroarch.cfg. Append sd:/ as well, guarded by
EXTERNAL_LIBOGC since the vendored libogc has no SP2 driver.
Reported on Discord: RetroArch launched from Swiss on an SD2SP2
card resolved sd:/ paths fine once typed into the config, but
the directory browser only ever offered carda:/ and cardb:/.
playlist_get_cached() is NULL while a deferred cached init is part
way through its parse, and every reader handles that except
xmb_set_title()'s icon fallback, which dereferenced the NULL
playlist_get_config() result and crashed the frontend when
switching playlists under XMB (#19418).
Keep the default texture for the frame and arm
current_menu_icon_retry, like the async sidebar/db-node icon loads
in the same switch arm: xmb_render() calls back in and the icon
resolves once the parse lands and the cache holds the new playlist.
Falling through to the history-suffix checks instead would be
guessing at a property of a playlist that does not exist yet.
The snapshot mirrors DEFAULT_MENU_TIMEDATE_STYLE from config.def.h
across all three define configurations, so it now carries
MENU_TIMEDATE_STYLE_YMD_HM. Regenerated with
tools/settings_check.py table --write tools/settings_table.reference.
Both record initialisers created their output directory up front:
disk_index_file_init() stat'ed (and if needed mkdir'ed) the
savefile directory on every content load, and runtime_log_init()
did the same for the runtime log directory on every call. The
runtime log case is the expensive one - the playlist sublabel path
initialises a log per playlist entry purely to *read* recorded
runtime, and each of those inits paid a stat for a directory that
a read never needs: if the directory does not exist, neither does
any log inside it, and the read correctly comes back empty either
way. The disk index case is one stat per content load for a
record most content (anything single-disc) never writes.
Create the directory in the save functions instead, derived from
the record path, immediately before the write that actually needs
it. Read paths pay nothing; write paths pay the same stat they
always did, at the moment it is meaningful.
One behavioural edge moves with it: an un-creatable directory no
longer fails init (runtime_log_init() returned NULL, disabling
runtime tracking for the session; disk_index_file_init() disabled
the record). Reads now proceed - correctly finding nothing - and
the failure is reported by the save that actually hits it.
The two small-record rjson writers - the disk index record and the
runtime log - opened their output file first and then streamed the
JSON into it through the buffered writer adapter. For documents
of a few hundred bytes that shape has no streaming benefit and one
real hazard: the open truncates the previous record before a
single byte of the replacement exists, so every failure past that
point - writer allocation, a write error, a crash or power loss
mid-save - destroys the old record and leaves a zero-length file
where it stood. Those empty files are exactly what the read side
was just taught to tolerate; this removes the machinery that
manufactures them.
Serialise into rjsonwriter_open_memory() and commit the finished
document with a single filestream_write_file() call: one open, one
write, one close, and the previous record is only replaced once a
complete replacement exists. The failure window collapses from
the whole serialisation to the single write call.
This also makes the disk index writer's failure handling honest:
it logged rjsonwriter_free() reporting a write error, then cleared
the 'modified' flag and returned success anyway, so a failed write
was never retried and the caller never knew. Failures now leave
the record modified - a later save retries - and return false.
The writer-allocation failure path also no longer costs the old
record its contents.
Both translation units lose their last use of the
formats/rjson_stream.h adapter; the include goes with it.
Two shapes of broken record survived indefinitely, reporting the
same error pair on every content load without ever being repaired:
A zero-length file - the residue of a save interrupted between
truncation and flush (the writer opens the output before any JSON
exists) - reached the parser and was reported as a JSON format
error at line 1, column 1, on every launch. Detect it before the
parser and treat it like an absent file: one warning, and the
record is regenerated instead of re-reported forever.
A record that fails to parse was worse than noisy: the jsonsax
reader returned failure on a parse error, which made
disk_index_file_init() mark the record modified so the next save
replaced the broken file. The rjson migration (ba1ed2da4b)
dropped that - the error was logged, then the code fell through,
copied whatever values the parser had extracted before failing,
and returned success. The record stayed unmodified, the save-side
'nothing changed' short-circuit kept the corrupt file in place,
and the same parse error greeted every subsequent load. Discard
partial values and return failure, restoring the self-healing
path.
In both cases the repair is only committed when a save is
permitted (multi-disc content); single-disc content never writes
a record, where the empty-file case now costs one warning line
instead of a WARN/ERROR pair plus parser diagnostics.
1dfed08090 converted five small-file rjson readers from the
stat + open + chunked-callback pattern to filestream_read_file()
+ rjson_open_buffer(), on the premise that "filestream_read_file's
failure covers missing files". It does, but not the way the
conversion tested for it: the function returns 1 on success and 0
on failure - never a negative value - and all five call sites
checked '< 0', a condition no return value satisfies.
The failure branch was therefore dead. For any content whose
record file does not exist - which is the common case the commit
message itself called out - the reader sailed past the check with
buf == NULL and len == 0, handed that to the parser, and logged a
JSON format error for a file that was never there:
[WARN] [Disk index file] Error parsing disk index file: "...".
[ERROR] [Disk index file] Error: Invalid JSON at line 1,
column 1 - reached end without any data.
Reported against the disk index record (single-disc content never
writes one, so every load logged the pair), but all five readers
were affected the same way: the runtime log spams one per playlist
entry without recorded runtime, and the test input drivers and
Remote RetroPad lose their quiet no-file path. Worse than the
noise: on the exists-but-unreadable branch filestream_read_file
documents len as unspecified (in practice -1), and the missed
check would have sent (size_t)-1 as a buffer length into the
parser alongside a NULL buffer.
Test the return as the boolean it documents and every other caller
in the tree already treats it as. The failure branches behave as
the conversion intended: silent return for a missing file,
path_is_valid classifying the failure worth logging otherwise.
Verified: all five translation units rebuilt clean; standalone
harness over file_stream + rjson confirms a missing file now takes
the quiet branch, a present record parses to RJSON_DONE, and that
the old '< 0' check misses both failure shapes.
Complements the write-path guards: the joypad driver's entry points
are called with RetroArch user ports up to MAX_USERS (16), but
analog_state[], hat_state[] and android_key_state[] only carry
DEFAULT_MAX_PADS (8) pad rows.
android_joypad_axis() performed no bounds check at all, reading
analog_state[] out of range for ports 8-15. android_joypad_state()
checked joy_idx but dereferenced the android_key_state[] row derived
from the unchecked port argument, and android_joypad_button()
computed that row pointer before its bounds check. Reject
out-of-range ports before any array access.
Android hands out fresh input device ids after every suspend/resume
cycle. android_input_get_id_port() matches by id only, so a device
waking up from sleep was treated as a brand new controller: it got
autoconfigured onto the next free port on every wake-up until the
pad slots ran out.
The existing name-based recovery in android_input_recover_port() was
gated behind the opt-in android_input_disconnect_workaround setting
because blindly rebinding by name would steal the port of a second
identical controller. Make the safe subset of that recovery
unconditional: rebind by name only when the previously mapped device
id verifiably no longer exists (InputDevice.getDevice() returns NULL
for it). A live old id means a genuine duplicate controller, which
still falls through to hotplug and gets its own port, preserving the
old default behaviour for multiple identical pads.
Also refresh g_android->id[] on rebind - previously only
pad_states[].id was updated, so rumble kept targeting the stale
device id after a reconnect even with the workaround enabled.
The read side (android_joypad.c) rejects ports at or beyond
DEFAULT_MAX_PADS, but the write side never did: after a failed
hotplug the poll loops dispatched events with port == -1, and an
over-allocated port index wrote straight through analog_state[],
hat_state[] and android_key_state[] into neighbouring memory.
Guard engine_handle_dpad_default(), engine_handle_dpad_getaxisvalue()
and android_input_poll_event_type_key() against out-of-range ports.
The key handler accepts up to ANDROID_KEYBOARD_PORT inclusive, since
android_key_state[] carries one extra row for the keyboard.
handle_hotplug() assigned ports up to MAX_USERS (16) and then kept
writing pad_states[pads_connected] unbounded, but every consumer of
the port index on the event path - analog_state[], hat_state[] in
struct android_app and the static android_key_state[] - is sized
DEFAULT_MAX_PADS (8) on Android. Registering a 9th device therefore
aliased the keyboard row of android_key_state[], and further devices
trampled memory adjacent to those arrays, including the jmethodID
members that directly follow hat_state[] in struct android_app,
crashing on the next JNI call through them.
Refuse to register a new device once all DEFAULT_MAX_PADS slots are
occupied and leave the port at -1 so event handlers skip it.
This condition is easy to hit in practice: Android re-enumerates
input devices with fresh device ids on every suspend/resume cycle,
so each wake-up used to burn one pad slot until the limit was
exceeded.
The RustyNES core builds cleanly for all Apple targets on the buildbot
(ios-arm64, tvos-arm64, osx/arm64) but is absent from appstore_cores, so
it is not bundled into the App Store builds and is unavailable to users
on iOS, iPadOS and tvOS.
Inserted in alphabetical order between reminiscence and sameboy.
I broke the Miyoo armv5te build and the arm-linux-gnueabihf crc32
sample:
features_cpu.c:907: warning: implicit declaration of
'arm_enable_runfast_mode'
ld: undefined reference to `arm_enable_runfast_mode'
arm_enable_runfast_mode() was DEFINED under __ARM_NEON__ && __arm__
but, after the aarch64 NEON work, CALLED under __arm__ alone. On an
ARM32 target with no NEON the call compiled and the definition did
not, which is a link error - and armv5te arm926ej-s is exactly that
target.
I had actually spotted this hazard while sweeping the NEON spellings,
checked that the NDK's armv7 compiler defines both macros, and
concluded nothing could break. That check only covered NEON-enabled
ARM32; the case that breaks is ARM32 with NEON absent, which I never
tested. Wrong conclusion from a real observation.
The definition and all three call sites now share one condition,
CPU_ARM_RUNFAST, so they cannot drift apart again. It stays
NEON-gated on purpose: the function writes FPSCR, so it needs an FPU
worth configuring, and NEON implies VFP - a VFP-less core would fault
on the write.
Verified by linking, not compiling, across the three ARM shapes with
the NDK's cross compilers: armv7 without NEON (-mfpu=vfpv3-d16, which
reproduces the failing configuration - it defines neither NEON macro),
armv7 with NEON, and aarch64. All three link; reverting the fix makes
the first fail with the same undefined reference. The crc32 sample
that CI reported now passes, aarch64 still reports NEON, and the
desktop build is unaffected.
The comment read as though Android, Linux ARM, iOS and tvOS were on
the table path. That is the situation this branch exists to fix, not
the situation with it, and a reader cannot tell which is meant.
Reworded to say that without the branch those targets would take the
table path, that with it they take the hardware path, and that which
one a build gets depends on the compiler rather than the target -
the last part being what the comment never said at all. NDK r27c
(clang 18) compiles the branch in for aarch64-linux-android; an older
clang fails the predicate and falls back, so CRC32 speed can change
on a toolchain bump alone. Checked with that compiler, not assumed.
Comment only, three lines longer. Verified: compiles clean with the
NDK aarch64 compiler, desktop build unaffected.
A sweep of every __ARM_NEON__ use in the tree, after the same
spelling turned out to be why System Information reported ASIMD
without NEON.
__ARM_NEON__ is the 32-bit spelling. Checked against the NDK's own
compilers rather than from memory:
armv7a : __ARM_NEON 1 __ARM_NEON__ 1
aarch64 : __ARM_NEON 1
So __ARM_NEON is the superset - every ARM target defining the legacy
name also defines it - and testing the legacy name alone silently
drops aarch64. Seven gates around NEON C intrinsics did exactly
that, and the code behind them was dead on every 64-bit ARM build
unless HAVE_NEON happened to be set:
audio/audio_driver.c (3)
libretro-common/audio/conversion/s16_to_float.c
libretro-common/audio/conversion/float_to_s16.c
libretro-common/audio/resampler/drivers/sinc_resampler.c (2)
Also the non-Linux fallback in the feature detector, which would
have missed NEON on aarch64 targets that do not read /proc/cpuinfo,
and the resampler quality default in config.def.h, which reached
aarch64 only via the explicit ANDROID and IOS cases and so missed
64-bit ARM Linux boards.
Four legacy-only gates are deliberately left alone. s16_to_float_neon.c
and float_to_s16_neon.c are 32-bit inline assembly carrying .arm and
.thumb directives, which must not be built for aarch64; the two in
features_cpu.c guard arm_enable_runfast_mode(), a 32-bit VFP control
with no aarch64 equivalent, and armv7 defines both spellings so
nothing there can break.
Verified with the real NDK (r27c, clang 18), not a stub: every
touched translation unit compiles for aarch64 and armv7a, and the
gates were probed directly - on aarch64 the conversion path reports
DEAD before the change and ACTIVE after, while armv7a reports ACTIVE
both ways, so 32-bit behaviour is unchanged. Desktop build
unaffected.
Not measured on hardware: this makes vectorised paths reachable that
were being compiled out, so the effect should be a speedup, but no
benchmark was run here.
System Information listed ASIMD alone on an S22 Ultra, with no NEON
and no VFP.
ASIMD *is* NEON. Advanced SIMD is what aarch64 kernels call it in
the Features: line, where 32-bit ARM says "neon", and it is
architecturally mandatory in ARMv8-A - a CPU reporting asimd has NEON
by definition. The detection set RETRO_SIMD_NEON only inside
#ifdef __ARM_NEON__
which is the legacy 32-bit spelling. aarch64 toolchains define
__ARM_NEON, without the trailing underscores. Checked against the
NDK's own compiler rather than from memory:
$ aarch64-linux-android21-clang -dM -E -x c /dev/null | grep __ARM_NEON
#define __ARM_NEON 1
#define __ARM_NEON_FP 0xE
No __ARM_NEON__. So the guard was false on every 64-bit Android
build and NEON was never reported, on hardware that has had it since
it was designed. That is a reporting bug in the menu and, more
importantly, a capability bug for anything branching on
RETRO_SIMD_NEON at runtime.
asimd now implies NEON directly. The runfast-mode call stays gated
on __arm__, since it is a 32-bit VFP control with no aarch64
equivalent.
VFPv3 and VFPv4 are absent for a different reason and are left
alone: those are 32-bit VFP feature bits that aarch64 kernels do not
put in the Features: line at all, because AArch64 mandates floating
point and has no separate VFP level to advertise. Running the real
parser over an aarch64 Features line shows neon, vfpv3 and vfpv4 all
absent while asimd and crc32 are found - so inferring NEON from asimd
is the only way any of it gets reported.
Verified: compiled with the actual NDK (r27c) for aarch64 at API 21
and 30 and for armv7a, not a stub of it; the parser was driven
directly over a real aarch64 Features line; desktop build unaffected.
Not verified on hardware - the menu should now list both ASIMD and
NEON.
samples/android/compile_check shipped a hand-written imitation of
jni.h and the android/ headers so the Android translation units could
be syntax-checked without an NDK. That does not belong in the tree.
They are a fake of a real SDK: the signatures are approximated rather
than correct, so they can disagree with the NDK while still
compiling, and anything checked against them carries a confidence it
has not earned. Worse, they are the kind of file that gets copied,
included by something else, or quietly relied on later. A real NDK
is a download away and is the only thing worth checking against.
The declaration-order fix that came with them is unaffected and
stays: android_last_mode_id and android_last_mode_hz remain declared
above their first use, which is what the Android build actually
needed.
Verified: only the stub directory is removed, the fix is intact, and
the desktop build is unaffected.
android_last_mode_id and android_last_mode_hz were declared next to
the function that re-applies them, which sits below the function that
records them. The Android build failed on both:
dispserv_android.c:260: error: use of undeclared identifier
'android_last_mode_id'
They move above their first use, which is the resolution list.
The more useful half of this commit is the check. The
ANDROID-gated translation units are not built by the desktop build,
so nothing here compiled them and a missing declaration had no way to
surface before a device build. Four Android patches went out that
way; this one broke.
samples/android/compile_check/ stubs enough of the NDK surface -
jni.h and the android/ headers platform_unix.h reaches for - to
syntax-check those sources with an ordinary compiler. It is
deliberately modest about what it proves: declarations and syntax,
with approximated signatures, nothing linked and nothing run. That
is exactly the class of mistake that should not reach a device build,
and it is now one command:
samples/android/compile_check/check.sh
Verified by reproducing the failure rather than by inspection: moving
the declarations back below their use makes the check report the same
five undeclared-identifier errors the NDK reported, and the fix
clears them. The desktop build is unaffected.
Still not verified on a device - the stubs are not the NDK, and this
proves nothing about behaviour.
Both halves of a mode change are window state, and neither survives
the app going to the background:
- the frame rate belongs to the ANativeWindow, and a resume brings a
brand new one. APP_CMD_INIT_WINDOW is exactly that moment.
- preferredDisplayModeId lives in the window's LayoutParams, which
the framework re-evaluates for the new window.
So a mode the user chose is simply gone after a trip to the Android
UI, and the display falls back to its default. Both are now
re-asserted where the new window appears: the native side from the
INIT_WINDOW handler, which re-requests the mode and re-sets the
frame rate, and the Java side from onResume(), which restores
preferredDisplayModeId and keeps the rate preference cleared.
Both no-op when no mode was ever chosen, so a default install is
untouched, and the Java side also no-ops when the attributes already
say what they should, so a resume that changed nothing does not churn
the window. Both source trees get the onResume hook; the state and
the method live in the common activity they share.
Scope, stated honestly: this is not what was making 120 Hz fail on
the reporting device. That turned out to be Android refusing modes
in a different resolution group - dumpsys says
allowGroupSwitching=false, and the log reads "preferredDisplayModeId
set to 8 ... display now reports mode 3", 8 being 1080x2316@120 and 3
being 1440x3088@60. The request was refused immediately, not lost on
resume. This commit fixes a real gap that would bite once a mode
does apply; it does not by itself make one apply.
Verified: every layer lines up - the choice is recorded only on an
accepted request, both the native and Java paths are reached, and
both no-op paths hold; the desktop build is unaffected.
NOT verified on a device: there is no NDK or JDK here, so neither
the Java nor the Android C was compiled. A logcat now prints a
re-apply line on each resume, so whether the mode is being restored
is directly visible.
Selecting a 120 Hz mode still left the panel at 60. The logcat says
why, and it is a second mechanism we were not using at all:
LayerHistory: ...RetroActivityFuture#7352 Max (can't resolve refresh rate)
SurfaceFlinger: Touch Control - choose 60.00 Hz
SurfaceFlinger: DFS = 60
Asking the framework for a display mode and telling SurfaceFlinger
what the window wants are separate things. The mode request moves
the display; the frame rate on the window is the vote our LAYER
casts. With no frame rate set the layer cannot resolve one, so the
compositor picks - and on this hardware, through Samsung's touch
control, it picks 60 and holds the panel there whatever mode was
requested.
ANativeWindow_setFrameRate() is the call for that and was never made
anywhere in the tree, though the native window has been sitting in
g_android->window all along. It now accompanies an accepted mode
request, with FIXED_SOURCE compatibility: the content really is
produced at a fixed rate, which is what an emulated system is, and
that value permits a mode switch rather than asking the compositor
to fit the content to whatever it is already running.
API 30, guarded - older devices do not have the call, and there the
mode request is all there is.
The result is logged next to the mode request, so the two can be
told apart rather than guessed at.
One thing this cannot beat: the same logcat shows Samsung's Game
Optimizing Service pinning applications it classifies as games -
SGM:SemGameManager: isForegroundGame(), ret=true
GOS:FpsController: requestFpsFixedValue(60, GOS)
- to 60 fps, and DisplayModeDirector reporting
mAlwaysRespectAppRequest: false. If that is what is holding the
rate, no app-side call overrides it; the device's Game Booster
settings decide. Worth ruling out by excluding RetroArch from Game
Booster before reading any further failure as ours.
Verified: the desktop build is unaffected.
NOT verified on a device - no NDK here.
video_display_server_set_refresh_rate() - the frontend's path for
changing refresh rate, as distinct from picking an entry out of the
resolution list - calls set_resolution() with width and height both
zero and only the rate filled in:
current_display_server->set_resolution(data, 0, 0, (int)hz, hz, ...)
The Android implementation matched candidate modes on width and
height, so a request for 0x0 matched nothing and every rate-only
switch failed silently. Zero now means "whatever size is current",
which is what the caller intends, and the search proceeds normally
from there.
Also logs the rate the display reports after an accepted request.
Acceptance and effect are different things: a logcat from this device
shows Samsung's Game Optimizing Service pinning applications it
classifies as games to 60 fps -
SGM:SemGameManager: isForegroundGame(), ret=true
GOS:FpsController: requestFpsFixedValue(60, GOS)
SurfaceFlinger: DFS = 60
- which holds the panel at 60 whatever the app asks for. Logging
only the acceptance made that indistinguishable from our own request
failing, which is exactly the confusion this cost.
Verified: the desktop build is unaffected; the size match is
unchanged for real sizes, and the list is still released on every
path.
NOT verified on a device - no NDK here.
Selecting a 120 Hz mode changed nothing. A logcat says why, and it
is not the system declining the request - it is RetroArch asking for
the opposite at the same time:
PRIORITY_APP_REQUEST_REFRESH_RATE_RANGE ->
Vote{minRefreshRate=60.0, maxRefreshRate=60.0,
disableRefreshRateSwitching=true}
PRIORITY_APP_REQUEST_BASE_MODE_REFRESH_RATE ->
Vote{baseModeRefreshRate=60.0}
-> appRequestRefreshRateRange=[60 60]
-> commit modeset : 1440x3088@60phs
Those are app requests. RetroActivityFuture.onResume() sets
WindowManager.LayoutParams.preferredRefreshRate from the REFRESH
intent extra - the configured video_refresh_rate, 60 - and the
framework turns that into a range vote pinning the display at 60
with switching disabled. preferredDisplayModeId is a separate
request, so asking for a 120 Hz mode while still asking not to leave
60 gets exactly what it asks for: 60.
Two changes, because the pin is applied in two places:
- Choosing a mode now clears preferredRefreshRate. The mode id
names a rate already, so a rate preference alongside it is
redundant as well as contradictory.
- onResume() no longer re-applies the rate preference once a mode
has been chosen explicitly. Without this the pin came back every
time the app moved to the foreground and quietly undid the
selection.
Both source trees carry that onResume block, so both are fixed;
the state and the accessor live in the common activity they share.
This also explains the other half of the report - 60 Hz shown on a
120 Hz panel. That reading was correct: the display really was at
60, because the app had asked for it.
Verified: the two trees stay consistent, and the desktop build is
unaffected.
NOT verified on a device - still no NDK or JDK here. The logcat
after this change should show the app requesting the chosen mode
without the [60 60] range vote alongside it; if a range vote is
still there, something else is setting preferredRefreshRate.
The last two NULL slots in dispserv_android. With them unset,
video_driver_get_next_video_out() fell through to the video driver's
poke interface, which does not provide them either, so stepping
through outputs did nothing on Android.
"Adjacent" means the next entry with a DIFFERENT resolution, which
is how the win32 server reads it and the only reading that behaves
sensibly here: a phone lists the same size at several refresh rates,
so stepping one entry at a time would give several presses that
change nothing visible. Rates are chosen from the resolution list;
this control moves between sizes.
Both directions share one stepper. It walks the whole list once
from the current position, so the search terminates whether or not a
different size exists, and wraps at both ends so holding a direction
cycles rather than sticking. With only one distinct size it reports
failure, which is what lets the caller fall through rather than
appearing to succeed while doing nothing.
The index arithmetic was checked exhaustively rather than by
inspection - for every list size, starting position and direction it
visits each other entry exactly once, takes the true neighbour
first, and never leaves the array. A wrap that skips or repeats an
entry is exactly the kind of thing that looks right and is not.
Also verified: all sixteen vtable slots line up with the header
field order, which a misplaced entry would corrupt silently rather
than fail to compile; the desktop build is unaffected.
Selection still goes through preferredDisplayModeId, so the same
caveat as the resolution list applies - the system may decline a
mode whose resolution differs from the current one, and the logging
added alongside get_video_output_size is what will say whether it
did.
NOT verified on a device: no NDK here, so the Android translation
unit was not compiled.
"Screen Resolution" showed N/A and never changed after selecting an
entry, even with the mode list populated. The label does not read
get_resolution_list at all: it calls
video_driver_get_video_output_size(), which reaches a THIRD display
server callback - get_video_output_size - and that one was still
NULL. A complete list could not have fixed it, and selecting a mode
could not either; the label had nothing to read on any path.
It now reports the mode marked current by the enumeration, with the
refresh rate in the description, because a phone panel offers the
same resolution at several rates and the resolution alone would not
say which is running.
That leaves the other two symptoms - 60 Hz reported on a 120 Hz
panel, and a selection that appears to do nothing - which are one
question, not two: the rate reported IS the mode currently in
effect, so if it says 60 the display is in a 60 Hz mode and the
request to leave it did not take. Android treats a mode change as a
request it may decline silently, and declining is common for a mode
whose resolution differs from the current one - many devices list
such modes and will not switch an ordinary app into them, while
switching refresh rate at the same resolution is generally honoured.
Rather than guess again at which of those it is, both sides now say
what happened: the C side logs the mode id requested, the size and
rate it was chosen for, and whether the call was accepted; the Java
logs the id it set and the id the display reports immediately
afterwards. A logcat from the failing selection distinguishes "the
request never arrived", "the system declined it", and "it was
accepted but the panel stayed put" - which need different fixes, and
which no amount of reading the code here can separate.
Verified: the desktop build is unaffected; the callback is wired and
the three symptoms traced to their respective call paths.
NOT verified on a device - still no NDK or JDK here. The N/A fix is
structural and should simply work; the mode-switch behaviour is what
the new logging is for.
dispserv_android left both get_resolution_list and set_resolution
NULL, so the resolution menu had nothing to show and nothing to do.
How far back this reaches, since it decides the shape: mode
enumeration is API 23. Display.getSupportedModes(), Display.Mode
and WindowManager.LayoutParams.preferredDisplayModeId all arrived
together in Marshmallow, and nothing before it exposes more than the
size currently in effect - nor any way to ask for a different one.
So:
API 23+ every mode the display declares, and selection by id.
earlier a single mode describing the current state, and no
selection - there is nothing to select into.
That is a real limit of the platform rather than a shortcut: KitKat,
Lollipop and the API 16 floor the jelly-bean tree still builds
against genuinely cannot answer. Reporting one current mode there
keeps the caller's shape identical everywhere - always at least one
entry, always exactly one marked current - so the menu has a
selection to draw on every device.
Details worth knowing:
- The rate crosses JNI as millihertz. The config carries an integer
rate beside the float one, and 59.94 must not arrive as 59.
- video_display_config_t::idx carries the Android mode id, not the
array position, because set_resolution has to hand that exact
value back to the platform.
- Selection matches width and height, then picks the closest refresh
rate among modes of that size - which is what keeps a 60Hz and a
120Hz entry of the same resolution distinguishable.
- Setting the mode is a request. The system may stay where it is;
getCurrentDisplayModeId() is what says whether it took.
- Window attributes are written on the UI thread, since that is
whose they are.
- Every JNI local ref and array is released on all paths, including
the allocation-failure ones.
The Display lookup getRefreshRate() introduced is now shared: one
getActiveDisplay() preferring the activity's own display over the
deprecated getDefaultDisplay(), used by all three queries.
Verified: the desktop build is unaffected; the set_resolution
signature matches the interface exactly, which a vtable slot would
otherwise corrupt silently; every layer checked to line up - struct
fields, bindings, JNI signatures, vtable slots, and the API gates on
each Java tier.
NOT verified: anything on a device. There is no NDK or JDK here, so
neither the Java nor the Android C was compiled. Worth checking on
hardware that the list matches what the display really offers, and
that selecting an entry moves the reported refresh rate.
"Set display reported refresh rate" showed 0 Hz. Not a deprecated
call that stopped working: dispserv_android never implemented
get_refresh_rate at all - the slot was NULL.
video_driver_get_refresh_rate() asks the display server first and
falls through to the video driver's poke interface, which does not
provide one here either, so the value was always 0, and the setting's
action returns early on exactly that. Nothing had ever been wired
up.
The Activity is the only thing that knows the answer. Android
switches display mode underneath the app - refresh rate switching,
battery saver, or the app's own preferredRefreshRate, which
RetroActivityFuture already sets - so the rate has to come from the
platform and has to be read live rather than cached, or it goes stale
the moment the system switches.
RetroActivityCommon gains getRefreshRate(), in three tiers, because
the way to reach the Display changed twice and this ships back to
API 16:
API 30+ Activity.getDisplay(). getDefaultDisplay() is deprecated
from here, and on a non-visual context it can hand back
something that reports nothing useful - a plausible way to
get a zero even where a query exists.
API 23+ the display's current Display.Mode, which is the exact
rate of the mode in effect rather than a rounded nominal
figure.
earlier Display.getRefreshRate(), present since API 1, which
covers KitKat and the API 16 floor the jelly-bean tree
still builds against.
Each tier falls through to the next, the whole thing is wrapped
against exceptions, and every failure path returns 0 - which is what
the caller already treats as "unknown", so behaviour is unchanged
wherever the query genuinely cannot answer.
The C side adds the jmethodID, a CALL_FLOAT_METHOD to go with the
int and void ones already there, and the display server callback.
Verified: the desktop build is unaffected; every layer was checked to
line up - struct field, binding, "()F" signature, vtable slot, and
the three Java tiers.
NOT verified: anything on a device. There is no NDK or JDK in this
environment, so neither the Java nor the Android C was compiled, let
alone run. The value should now appear in the setting; if it reads 0
on a specific device the tier that device lands in is where to look.
The ST_INT and ST_UINT handlers in both DISPLAYLIST_DROPDOWN_LIST and
DISPLAYLIST_DROPDOWN_LIST_SPECIAL walked their ranges with a float
accumulator and cast each value back with (int)i.
These are correct today: integer settings use a step of 1 over maxima
well below 2^24, so float addition is exact and every entry lands on the
value it should. But the same shape is what made the preceding commit's
bug possible - a float accumulator silently drops the final entry once
the range and step stop being exactly representable, and a fractional
step would make (int)i emit duplicate entries instead. Nothing enforces
that an integer setting cannot be given such a step.
Derive int32_t bounds from the setting once and iterate with an integer
counter, so the arithmetic is exact by construction rather than by
coincidence. A step below 1 clamps to 1, since a sub-unit step over an
integer range can only produce duplicates.
No behavioural change for any setting currently in the tree. The clang
analyzer's security.FloatLoopCounter count over griffin.c drops from 12
to 4; the remaining 4 are the two ST_FLOAT handlers, where a float
counter is the point.
Built with NDK r27c on arm64-v8a and armeabi-v7a; -Wall warning set over
griffin.c unchanged from master.
DISPLAYLIST_DROPDOWN_LIST_SPECIAL's ST_FLOAT handler computes
float half_step = step * 0.5f;
and uses it for the "is this the current value" comparison, but not for
the loop bound, which is a bare `i <= max`. Accumulating a float step
overshoots max whenever the step is not exactly representable, so the
final entry is never appended and the setting's maximum cannot be
selected from the dropdown.
DISPLAYLIST_DROPDOWN_LIST's ST_FLOAT handler already iterates
`i <= max + half_step` for exactly this reason. The guard was applied to
one of the two and not the other.
Measured against real ranges in the tree:
0.2 .. 5.0 step 0.01 480 entries, should be 481
-1.0 .. 1.0 step 0.05 40 entries, should be 41
0.0 .. 1.0 step 0.10 10 entries, should be 11
Ranges whose step happens to be exact (0.0 .. 0.5 step 0.01, and
anything with step 1.0) are unaffected, which is why this went
unnoticed.
Use the same bound as the non-special path. Found by clang's static
analyzer flagging the float loop counters; the other ten warnings it
reported are the ST_INT/ST_UINT loops, which are correct today because
their steps and ranges are exactly representable, and are left alone
here so this commit stays a pure bug fix.
Scrolling a thumbnail playlist by touch while on-demand downloads are
enabled snaps the view back to the top as soon as a thumbnail lands.
When a download completes, cb_task_pl_entry_thumbnail_refresh_menu()
calls the driver's refresh_thumbnail_image. MaterialUI's version resets
the on-screen thumbnail textures, but also calls
materialui_update_list_view(), which ends by raising
MUI_FLAG_NEED_COMPUTE. The next frame acts on that flag by killing any
scroll animation and assigning
mui->scroll_y = materialui_get_scroll(mui, p_disp);
materialui_get_scroll() never reads the current scroll position. It sums
entry heights up to menu_st->selection_ptr and returns the offset that
centres the *selection*, with an early `return 0.0f` when the selection
falls in the first half of the list.
Under touch, scroll position and selection are decoupled: flicking
through a playlist moves scroll_y and leaves selection_ptr alone. So a
user who scrolled without tapping still has selection_ptr == 0, and the
recompute takes that early return - straight back to the top. Having
tapped an entry first merely changes the destination from the top to
that entry; it is the same defect.
That recompute exists for populate_entries(), where the list really did
change and re-centring on the selection is correct. A thumbnail arriving
is not that case: nothing was added, removed or reordered.
Split the two concerns. preserve_scroll_on_compute marks a pending
compute as geometry-only, and the handler then restores the view instead
of re-deriving it. It cannot simply retain scroll_y, because entry
heights can change across the recompute - materialui_compute_entries_box()
reads MUI_FLAG_SECONDARY_THUMBNAIL_ENABLED, which a newly available
thumbnail can flip - so the pixel offset would drift. Instead it anchors
on first_onscreen_entry plus that entry's clipped offset, and
reconstructs scroll_y from the new heights, keeping the same entry in the
same place. No clamping is needed: scroll_y is already clamped every
frame once dragging and overscroll are inactive.
The flag is cleared in materialui_layout(), materialui_switch_list_view()
and materialui_list_insert(), so a genuine list or layout change
occurring in the same frame still re-derives from the selection.
A bool member rather than a MUI_FLAG_ bit: bits 0-30 of the flag word are
taken and (1 << 31) is not representable as int, so it would not be C89
clean. The handle already carries transition_alpha_lock in the same way.
XMB, Ozone and RGUI are unaffected - their refresh_thumbnail_image only
reloads textures and never touches scroll.
Built with NDK r27c on arm64-v8a and armeabi-v7a; -Wall warning set over
griffin.c unchanged from master. Not verified on a device: reproducing it
needs touch scrolling a playlist with downloads pending.
retro_vfs_readdir_saf() crossed into Java once per directory entry, and
each crossing was a sequence: PushLocalFrame, readdir, getDirentIsDirectory,
getDirentName, NewGlobalRef, GetStringUTFChars, then ReleaseStringUTFChars
plus DeleteGlobalRef on the following call, PopLocalFrame - with an
ExceptionOccurred after nearly every one. Seventeen JNI transitions per
entry to retrieve a name and a boolean.
Add SafDirectory.readdirBatch(), which drains the cursor into a String[]
plus a parallel boolean[], and pull a batch of 256 across in two calls.
The names are copied into native memory immediately and the Java
references dropped, so stepping through a batch costs nothing at all -
the amortised cost is 2/256 crossings per entry rather than 17.
MEASURED. There is no Android device or emulator here (no KVM), so this
was measured on a host JVM with a harness replicating the exact JNI call
sequence of both shapes against an equivalent Java object:
entries per-entry shape batched speedup
1000 584.8 ns/e 107.7 ns/e 5.4x
10000 580.4 ns/e 110.4 ns/e 5.3x
50000 642.7 ns/e 119.2 ns/e 5.4x
Two caveats on that number. It is HotSpot, not ART. And it isolates the
JNI transition component: the harness does not model ContentResolver,
Binder or Cursor work, which batching does not remove, so the end-to-end
improvement on a real SAF tree will be smaller than 5.3x. The measurement
bounds the part this change actually addresses.
The same harness settles a question left open earlier: ExceptionOccurred
costs 13.5 ns and ExceptionCheck 11.3 ns. At eight per entry that is
~18 ns of 585 ns, about 3%. Swapping them is an idiom improvement, not a
performance one, and is deliberately not bundled here - only the new code
uses ExceptionCheck.
A short batch marks the stream exhausted so the cursor is not queried
again after it runs dry. dirstream is now calloc'd rather than malloc'd
so a failure before the first fill cannot free garbage pointers, and
dirent_name_object is gone since names are no longer Java-owned.
Built with NDK r27c on arm64-v8a, armeabi-v7a and x86_64. The batch
method's boundary behaviour (full batch, short final batch, then empty)
was verified by compiling and running it standalone.
5adbff1d accidentally added android_vk_ctx.plist and platform_unix.plist,
1052 lines of clang static analyzer report. `clang --analyze` writes a
.plist per translation unit into the working directory unless told
otherwise, and they were swept up by a `git add -A`.
Delete both and add an ignore rule. The pattern is anchored to the
repository root - a bare `*.plist` would be a trap for anyone adding a
new Info.plist or ExportOptions.plist under pkg/apple, where 26 tracked
plists live. No legitimate plist sits at the root.
Every macro in the JNI helper block expanded to two statements with no
do/while(0) wrapper, so the trailing JNI_EXCEPTION escaped any unbraced
guard at the call site:
if (env != NULL)
CALL_BOOLEAN_METHOD(env, jbool, ..., isScreenReaderEnabled);
expands to a guarded call followed by an unguarded
if ((*env)->ExceptionOccurred(env)) ...
which dereferences env on exactly the path the guard existed to protect.
jni_thread_getenv() returns NULL when AttachCurrentThread fails, so
is_screen_reader_enabled() and accessibility_speak_android() would take
a null pointer dereference there rather than the intended no-op.
android_input_keyboard_hide() has the same shape.
Six further sites guard on a cached jmethodID instead of env
(getPowerstate, getBatteryLevel, setSustainedPerformanceMode,
setScreenOrientation, inputGrabMouse, and the KeyCharacterMap lookup
added in 25b9de8d). Those do not crash, but they run an exception check
for a call that never happened, which can report and swallow a stale
exception belonging to something else.
Wrap all seventeen macros. This fixes every call site at once and makes
the pattern impossible to reintroduce, rather than adding braces at the
nine sites that happen to be wrong today.
One call site relied on the old expansion and omitted its semicolon;
CALL_BOOLEAN_METHOD for isPlayStoreBuild now terminates properly.
Found with clang's static analyzer, which reported three
core.NullDereference paths in platform_unix.c; the count is zero after
this change. Built with NDK r27c on arm64-v8a, armeabi-v7a, x86 and
x86_64, -Wall warning set over griffin.c unchanged from master.
Nothing in the tree ever wrote APP_CMD_DESTROY. The enumerator exists,
android_input_poll_main_cmd() handles it by setting destroyRequested,
and both pollers turn that into RARCH_CTL_SET_SHUTDOWN - but no producer
was left, so the whole path was dead and the app thread was never told
to stop.
Meanwhile android_app_free() took android_app->mutex and then joined the
app thread while holding it. android_app_destroy(), which is what the
thread runs on the way out, takes that same mutex. So if onDestroy()
ever arrived with the thread still live, the Java UI thread blocked
until ActivityManager gave up on the activity.
Write the command, then wait on the condvar rather than joining under
the lock, so the thread can take the mutex it needs to finish.
WHEN THIS ACTUALLY FIRES. Not on 'Quit RetroArch': there the app thread
has already exited by the time finish() reaches onDestroy(), so the join
returned immediately and the bug was invisible. RetroActivityFuture
declares configChanges for orientation, screenSize, screenLayout,
uiMode, locale, fontScale and more, so those do not recreate the
activity either. What is left is density and layoutDirection - changing
display size in system settings, or switching to an RTL locale - plus
any other finish() with the thread still running. Rare, but currently a
guaranteed UI-thread hang.
WHAT THIS DOES NOT DO. It is not a graceful teardown. main_exit() ends
in frontend_driver_shutdown(), which on Android is exit(0), and that
runs a few calls after android_app_destroy() sets destroyed. So the UI
thread's join is racing process exit and will usually lose. The
behaviour this buys is a prompt exit(0) out of onDestroy() instead of a
hang followed by an ANR kill - which is the better of the two, and the
config and SRAM flush added on APP_CMD_PAUSE has already run by then, so
nothing is lost either way.
Two guards, since this makes a path execute that effectively never has:
- destroy_from_framework suppresses the onRetroArchExit() callback in
android_app_destroy(). That callback calls finish() on the activity,
which is pointless when the destroy came from the framework and we are
already inside onDestroy().
- The wait is bounded at five seconds, comfortably inside the window
ActivityManager allows a destroying activity. On timeout the thread
may still be running and still referencing this struct, so skip both
the join and the free and return: leaking the allocation is the right
trade against freeing memory a live thread is using, and the process
is going away regardless. An unbounded wait would only reproduce the
hang this commit exists to remove.
Built with NDK r27c on arm64-v8a, armeabi-v7a, x86 and x86_64; -Wall
warning set over griffin.c unchanged from master. Not exercised at
runtime: reaching it needs a density or RTL-locale change on a device.
android_keycode_to_unicode() resolved everything from scratch on every
key event: PushLocalFrame, FindClass("android/view/KeyCharacterMap"),
GetStaticMethodID for load, CallStaticObjectMethod to build the map,
GetMethodID for get, CallIntMethod, PopLocalFrame. Seven JNI transitions
per keystroke, two of them string-keyed symbol lookups and one a
class-by-name lookup, to read a single character.
Resolve the class and both method IDs once and hold the class as a
global reference, since the method IDs are only valid while the class is
reachable and a local reference would not survive the call. Cache the
per-device map objects in a small round-robin table; devices are few and
long-lived, so four entries cover the realistic case. The steady-state
path is now a single CallIntMethod, and it creates no local references,
so the local frame is gone as well.
Invalidation: the table is dropped on APP_CMD_INPUT_CHANGED, which is
the signal that the attached device set changed and that an id may now
refer to different hardware, and again when the input driver is freed
(which also clears the resolve-failed latch so a re-init retries).
The tradeoff is that a hardware keyboard layout changed in system
settings mid-session will keep resolving through the previously loaded
map until one of those two points is reached. Reloading per keystroke to
cover that is not a trade worth making; there is no callback that would
let us invalidate precisely.
All state is touched only from the input thread - the lookup runs from
android_input_poll_event_type_keyboard(), the two teardown points from
the same thread - so no locking is required.
If NewGlobalRef fails the slot is left empty and the next event simply
re-resolves, rather than caching a null and returning stale results.
Built clean with NDK r27c on arm64-v8a, armeabi-v7a, x86 and x86_64;
-Wall warning set over griffin.c is unchanged from master.
Two fields of struct android_app are shared across threads with no
synchronisation on at least one side.
content_rect is written by the Java UI thread in onContentRectChanged()
with no lock at all, and read by the video thread in
android_gfx_ctx_vk_check_window() and android_gfx_ctx_vk_set_resize().
Beyond the missing barriers the store order was backwards: @changed was
published before @width and @height, so a reader could observe the flag
alongside the previous frame's dimensions and rebuild the swapchain at
the wrong resolution. Store the dimensions first, publish @changed with
a release store, and acquire it on the reader.
unfocused is written on the app thread under android_app->mutex and read
on the video thread in dispserv_android.c without it. Nothing waits on
that condvar for a focus change - onWindowFocusChanged() posts the
command and returns - so the lock and broadcast were guarding a single
scalar for no reader's benefit. Make the field atomic and drop them.
This is a correctness fix; the removed lock/broadcast pair is incidental.
The remaining mutex and condvar uses are rendezvous - block until the
peer acknowledges - which is what a condvar is for and which atomics
cannot express without spinning, so they are left as they are.
android_kbd_lock likewise stays: it guards a char[] staging buffer that
cannot be swapped atomically, and it is per-keystroke, not per-frame.
retro_atomic_int_t is atomic_int in C and std::atomic<int> in C++, so
embedding it in this struct requires that the header stay unreachable
from C++ translation units. It is: griffin.c pulls both platform_unix.c
and android_input.c into a C TU, and no .cpp includes platform_unix.h.
Noted in the header so it is not broken later. Verified sizeof equals
sizeof(int) on both the C11 and the volatile fallback backend, so the
struct layout and the existing calloc zero-initialization are unchanged.
onSaveInstanceState() wrote APP_CMD_SAVE_STATE, woke the app thread, and
blocked the Java UI thread on the condvar until the handler acknowledged
by setting stateSaved. The handler did nothing else.
The blob being synchronised has no producer: android_app->savedState is
only ever populated from the state handed to ANativeActivity_onCreate,
and nothing in the tree writes a new one. So the first call returned the
already-restored create-time blob, re-saving state that is never
consulted, and every subsequent call returned NULL after a full
rendezvous.
Android invokes this on every backgrounding and on every configuration
change, which means every screen rotation paid a pipe write, an app
thread wakeup and a condvar handshake to hand back nothing.
Return no state directly. This removes the last waiter on stateSaved, so
drop the field and the dead command handler; APP_CMD_SAVE_STATE itself
stays in the enum so the following enumerators keep their values.
The remaining handshakes are left alone. android_app_set_window() and
android_app_set_input() must keep blocking - ANativeWindow and
AInputQueue are invalid once the callback returns, so proceeding without
the acknowledgement is a use-after-free on the video thread.
jni_thread_getenv() called AttachCurrentThread on every invocation, then
stored the resulting JNIEnv in thread-local storage - which nothing ever
read back. The key is created with a destructor and written at 437 and
453, but sthread_tls_get was never called, so the cache was inert.
A JNIEnv is valid for the lifetime of the thread it was issued to, and
this runs on only two threads, so a TLS read answers virtually every
call. AttachCurrentThread on an already-attached thread still enters the
VM to resolve the thread, and there are ~25 call sites, the hottest
being the per-keystroke KeyCharacterMap lookup in
android_keycode_to_unicode() which calls it once per key event.
Read the slot first and only attach on a miss. Guard both the read and
the write on whether sthread_tls_create_with_dtor() actually succeeded:
the return value was previously only logged, and pthread_getspecific on
an uninitialized key is undefined. On that failure path the env is still
returned, since it is usable even though it can no longer be cached or
auto-detached.
No functional change to attach/detach semantics: threads that miss are
attached exactly as before, and jni_thread_destruct still clears the
slot on detach.
-applicationDidEnterBackground: flushes SRAM but never writes the config.
The only Apple path that reaches retroarch_main_quit(), and therefore the
only one that saves settings, is -applicationWillTerminate:, which iOS
does not guarantee: a suspended app reclaimed by the system, or one the
user swipes out of the app switcher, is killed without it. Settings
changed during a session are silently discarded in both cases.
There is no user-facing workaround on these targets either, because the
'Quit RetroArch' menu entry is compiled out by the #if !defined(IOS)
guard in menu_displaylist.c, which covers tvOS as well since -DIOS is
applied to appletvos*/appletvsimulator* via OTHER_CFLAGS_IOS_TVOS_SHARE.
Save the config alongside the existing SRAM flush, gated on
config_save_on_exit, mirroring what is now done for APP_CMD_PAUSE on
Android. -applicationDidEnterBackground: is the last callback reliably
delivered before suspension, so it is the correct hook.
A double write when -applicationWillTerminate: does fire is harmless and
already the existing behaviour for CMD_EVENT_SAVE_FILES.
No ARC/MRC exposure: the hunk introduces no Objective-C object
expressions, and the file contains no manual retain/release/autorelease
to begin with.
506f5dad30 defaults DEFAULT_MENU_SHOW_QUIT to false on Android, which
moves one row in the emitted configuration table:
-bool menu_show_quit_retroarch 1 1
+bool menu_show_quit_retroarch 1 0
The reference the drift check compares against was not regenerated
with it, so Settings-Checks has been failing on every push since.
Regenerated with the command the check itself prints.
One line changes, and it is the row that moved - nothing else in the
table drifted, which is what makes the check worth having: an
intentional default change shows up as exactly the row it touched.
Verified: all three steps of Settings-Checks replayed - the table
drift check now reports a match, the Griffin-order unity compile is
clean for console, desktop and mobile, and the migration tool
self-test passes.
The harness workflow failed at link with
/usr/bin/ld: cannot find -lX11-xcb
The runner carries x11 and xcb but not libX11-xcb, and configure -
finding the first two - enabled an X11 build that then could not
link. Chasing that with more -dev packages is the wrong direction:
the harness drives the menu with the null drivers, so everything that
draws is dead weight, and building it is the entire cost of this
workflow.
Configure now disables Qt (which owns main(), and the harness
replaces main()), X11, all three GL drivers, Vulkan, cheevos,
networking and ffmpeg. What is left is the menu, the playlist reader
and the task system - which is all the harness exercises.
Measured on a clean tree, not estimated: 200s to build, 2s to build
and run the harness, 202s total, against roughly ten minutes before.
The timeout comes down from 40 to 25 minutes, which is still ample
headroom for a slower runner while no longer being an open-ended
budget for a job that should be quick. The extra -dev packages come
out of the install step too, since nothing needs them now.
All three lanes pass against the reduced build.
The 'Quit RetroArch' and 'Restart RetroArch' main menu entries are
already compiled out on Apple mobile targets by the #if !defined(IOS)
guard in the MENU_SETTINGS_LIST case, and -DIOS is applied to
appletvos*/appletvsimulator* as well as iphoneos*/iphonesimulator* via
OTHER_CFLAGS_IOS_TVOS_SHARE, so tvOS is covered by the same guard.
The corresponding menu_show_quit_retroarch and
menu_show_restart_retroarch toggles are nonetheless still listed under
Settings -> User Interface -> Menu Item Visibility, where they have no
observable effect. Gate them with the same condition.
Apple's position here is unchanged: QA1561 states there is no API for
gracefully terminating an iOS application and that calling exit skips
-applicationWillTerminate:, so data may not be saved. Apple DTS has
reconfirmed in recent developer forum threads that the document is still
valid and that a UI control for quitting is an App Review risk, and the
same reasoning is applied to tvOS. The hard compile-time guard is
therefore kept for Apple targets rather than being relaxed into the
default-off toggle used for Android.
Android's navigation model expects the user to leave via Home or the task
switcher rather than an in-app control, and Google's Android TV
guidelines state outright that an exit item should not be shown in the
menu. There is no equivalent Play Store policy line for phones and
tablets, but the guidance points the same way and iOS is already
excluded from this entry in menu_displaylist.c.
Default menu_show_quit_retroarch to false on Android rather than
compiling the entry out, so the toggle remains available under
Settings -> User Interface -> Menu Item Visibility and existing configs
that already write the key are unaffected.
This depends on the preceding commit: until config is flushed on
APP_CMD_PAUSE, 'Quit RetroArch' is the only code path on Android that
saves the config at all, and hiding it by default would otherwise make
settings unsaveable.
Android only guarantees onPause(). onDestroy() is not guaranteed to run
at all, and swiping the task away from Recents never delivers it, so the
process can be reclaimed with nothing written back.
Config saving currently only happens in retroarch_main_quit(), which is
reachable on Android exclusively via the 'Quit RetroArch' menu entry.
Every other way of leaving the app therefore discards all settings
changes made during the session.
Treat APP_CMD_PAUSE as the flush point instead: write SRAM and, when
config_save_on_exit is set, the current config. The flush is guarded by
a latch that is cleared on RESUME/START so a duplicate PAUSE does not
rewrite the config twice, while a genuine pause -> resume -> pause cycle
still flushes each time.
The flush is deliberately performed after activityState has been set and
the condvar broadcast, so the Java UI thread waiting in
android_app_set_activity_state() is released before the file writes
begin and onPause() latency is unaffected.
Auto-savestate is intentionally not included here: it is unbounded in
size and would put a multi-megabyte write on the pause path.
The harness built its playlist displaylists directly, which covered
the two faults that live in the list itself but not the first one
reported - a playlist showing the previous playlist's entries - so
that one stayed asserted at the reader level, below the menu.
It now navigates: build the Playlists screen, find the entry, and
press OK on it through menu_entry_action(), which dispatches whatever
action_ok the menu bound to that entry. That is what a tap does.
Two things were needed to make that work, both mine to get wrong:
- MENU_ENTRY_INITIALIZE() zeroes entry.flags, and menu_entry_get()
fills only what the flags enable. Without them the action is
handed an empty path and the rebuild that follows dereferences it -
a crash in path_basename_nocompression() reached through
generic_menu_entry_action(). The harness now enables the same set
the menu enables before it dispatches.
- The switch assertion has to be made WHILE the read is still
pending. Checked after waiting for the load, it passes even with
the fix reverted, because by then the right playlist has arrived.
The bug was never about the end state: it is what the user looks at
for the whole of the load. Checked in that window it reproduces
exactly - "the list showed /games/n64/game00000.bin" while the NES
playlist was loading.
All three fixes now falsify against the real menu: reverting the
stale-cache drop shows the previous playlist's entries mid-load,
reverting the placeholder leaves zero entries, and reverting the pump
leaves the read unfinished after 100000 frames.
Verified: three lanes pass, repeated; the full build links; the
harness build and run take about three seconds on top of it, which is
what its own workflow pays.
The menu navigation harness links the shipping RetroArch objects, so
it needs a full build of the program before it can be built at all.
I put that build inside Linux-samples-tasks, whose job budget is ten
minutes and which already runs a dozen other sample steps. It blew
the budget and the job was cancelled - taking every other sample with
it, twice, at ten minutes a go. It also reconfigured the tree with
--disable-qt part way through a shared job, which is not something a
step should do to the steps around it.
The step is removed from that job and the harness gets its own
workflow, where a long build can only affect itself. Three things
keep the cost honest:
- it is path filtered, so it runs when menu/, playlist.c/.h,
task_content* or the harness itself change, not on every push;
- the timeout is 40 minutes, sized for a cold full build rather than
hoping one fits;
- the harness build and run take about three seconds once RetroArch
is built, so the whole cost is the build, and it is paid only by
changes that could actually break what the harness covers.
workflow_dispatch is on so it can be run by hand without touching a
watched path.
No change to the harness or to what it asserts.
Three field reports came out of the seam between the playlist reader
and the menu, one after another: a stale playlist's entries listed
under a new heading; then, once the cache was dropped promptly, a
blank list the back button could not leave; then, once a placeholder
kept it non-empty, a list that filled only after the user pressed
something. Every oracle in the tree stopped at the playlist reader,
so none of them could see any of it - each fix was reasoned from
menu_driver.c rather than exercised against it, and each left the
next fault in place.
This links the SHIPPING objects with only main() replaced. Nothing
is stubbed: menu_displaylist_ctl(), menu_driver_iterate(), the entry
list, the refresh gate and the action dispatch are the real ones.
retroarch.c is recompiled with main renamed so the harness can supply
its own without excluding the object that holds most of the frontend,
and both the compile and the link reuse the project's own command
lines taken from make -n, so the harness cannot drift from how the
program is really built. It needs a non-Qt build, because with Qt on
main() lives in ui_qt.o and drags the whole Qt UI with it.
Reproducing Android is the part that matters. The harness installs a
VFS through filestream_vfs_init - the seam the frontend uses for SAF
- returning short reads WITH a per-read delay. Short reads alone
reproduce SAF's syscall count but not its latency: on local storage
1500 entries still finish inside one budget slice, the read never
yields, and the whole class of fault is unreachable. That is
precisely why none of the three reports reproduced on a desktop.
With the delay the read yields, and the lanes can see what a phone
sees.
Two lanes, both falsified against the real menu:
- opening a playlist must never leave the list empty. Reverting the
placeholder gives "opening a playlist left 0 entries" - an empty
list is unrecoverable, because the rebuild is gated on
selection_buf_size and the back action needs a selected entry.
- a read that yields must finish on frames alone. Reverting the pump
in menu_driver_iterate() leaves it "still pending after 100000
frames"; with it, twelve frames and no input at all.
One deliberate limitation: which playlist's entries are on screen is
asserted in samples/playlist, not here. This harness pushes the
displaylist directly rather than going through the action_ok path a
user takes, so the label mapping is not the one real navigation
produces and asserting on it would be checking the harness. Driving
real navigation is the next step for it.
Third report from this path: opening a playlist showed "No Playlist
Entries Available", and backing out and opening the same playlist
again showed the entries. Sometimes back did not work on the empty
screen either.
The missing piece was a pump. ENTRIES_NEED_REFRESH is consumed in
exactly one place - generic_menu_entry_action() - which runs when the
user presses something. There is no per-frame equivalent, so a
displaylist that yielded mid-build stayed unfinished until the next
keypress: the placeholder was drawn, nothing advanced the read, and
the list only filled when an action happened to drive the rebuild.
Re-entering the playlist worked because that WAS the input.
menu_driver_iterate() now advances a pending read, once per frame,
under the same shared I/O window the directory walks and the scanner
use, and rebuilds the list itself the moment the read completes -
without waiting for input. With nothing outstanding it costs one
branch, so the ordinary case is unchanged.
This is only reachable at Android/SAF speed. With local stdio a
playlist is read inside the first slice and the yield path is never
taken, which is why none of these three reports reproduced on a
desktop.
The oracle gains a pump lane that runs over the short-read VFS
standing in for SAF: it asserts the first slice really does yield -
so the lane cannot stop covering its own case - and then advances
only by repeated frames, no requests and no input, until the read
completes and the requested playlist is installed with its own 600
entries. Removing the advance leaves it pending forever, which is
the reported symptom.
Verified: strict C89 gates on playlist.c and menu_driver.c; full
Linux build links with HAVE_QT=1; the playlist oracle passes plain,
twice under ASan+UBSan+LSan and twice under TSan; the playlist
manager oracle still passes.
Still not exercised here: the menu-side rebuild itself. The lane
drives the continue/finish pair the pump calls, not
menu_driver_iterate(), because nothing links the real menu yet. A
harness that links the shipping objects with only main replaced is
the missing piece for that, and this path has now earned it.
During fast-forward the audio driver is non-blocking and, unless the
"Fast-Forward Audio Speedup" pitch tracking is enabled, the resampler
still converts every frame the core delivers at the unscaled ratio.
The device buffer drains at real-time rate while flushes arrive at
fast-forward rate, so the buffer is pinned near full and the write
stage drops almost everything: at 40-50x on a lightweight core, over
90% of the convolution output - and of the conversion, clamp and
write-copy stages after it - is spent on frames that are never heard.
This is why speedup-off measures notably slower than speedup-on at
high fast-forward multipliers: the speedup path multiplies the ratio
by wall-delta/expected-delta (clamped at AUDIO_MIN_RATIO = 1/16), so
it emits ~16x fewer output frames, while the speedup-off path emits
them all and discards them at the fifo. Reported as a several-hundred
fps loss on cores fast-forwarding in the thousands of fps, scaling
with resampler quality (HIGHEST's 256 taps against NORMAL's 16).
Bound the resampler's input instead: when fast-forwarding without
speedup, query write_avail() first and cap the input frame count to
what the device can accept at the current ratio, plus a slack margin.
Frames the write would have dropped from the tail of the chunk are now
never resampled. The discard moves from after the resampler to before
it; the same head-of-chunk fragments reach the device either way (seam
samples within the filter width differ, since the ring no longer sees
the dropped tail - both are equally discontinuity artifacts inherent
to dropped-frame fast-forward audio).
Deliberately an input cap and not a ratio bound: shrinking the ratio
would time-compress the chunk into the writable space, which is
exactly the pitch-raising behaviour the user declined by leaving the
speedup option off.
Applied to both the float path and the deterministic s16 path (the
same waste recurs under the integer hint; local fast-forward has no
determinism consumer, and the s16 stream stays bit-exact whenever the
bound passes through). The write_raw fast path is untouched - the
frontend does no resampling there. When the buffer is not saturated
(heavy cores, modest multipliers, or 1x) avail exceeds the chunk and
the bound passes through unchanged, so non-fast-forward behaviour and
latency are unaffected.
The slack (AUDIO_FF_DISCARD_SLACK_FRAMES, 64 frames) covers device
drain between the avail sample and the write - a fast-forward flush
interval is a few hundred microseconds, i.e. tens of frames at typical
output rates - plus driver-side avail rounding. Overfilling by the
margin reproduces today's drop-at-fifo behaviour exactly; underfilling
would starve the device and open audible gaps, so the margin errs
toward a little discarded work (~1 us per flush at 16 taps). It also
keeps the bound from ever returning zero, so the resampler ring stays
warm and no bypass/reinit interactions are introduced.
write_avail() returns bytes (wasapi: fifo + engine free bytes; alsa:
FRAMES_TO_BYTES(snd_pcm_avail); dsound: ring bytes); output frame size
follows AUDIO_FLAG_USE_FLOAT, matching what the write stage below
actually submits on each branch. Under threaded audio this is the
same locked accessor the rate-control path already calls once per DRC
interval; the helper itself is stateless and introduces no new shared
state. Drivers without write_avail() keep the old behaviour.
Measured on the SSE sinc kernel, 735-frame chunks at 44.1k->48k, at
the saturated operating point (device accepting ~16+64 frames/flush):
NORMAL 6.96 -> 1.16 us per flush, HIGHEST 75.1 -> 11.6 us, with the
conversion/clamp/write stages shrinking by the same output factor. A
pipeline model using the real resampler, swept over ratios {1.088,
0.919, 1.0, 2.0, 0.0625} x buffer sizes {1536, 4096, 16384} x drain
rates {0, 1, 16, 100, 900} frames/flush, shows device intake counts
bit-identical with and without the bound across the entire sweep, with
resampled work at 8-10% of baseline when saturated and exactly 100%
(pass-through) when the buffer never fills. The model is
kill-verified: removing the slack makes the intake-identity check fail
(one-frame starvation), and the sweep runs clean under
ASan/UBSan/LeakSanitizer. Touched TU compiles with
-Werror=declaration-after-statement and links into the full binary.
Second field report from the playlist navigation path: after the
previous fix, opening a playlist showed a BLANK list, and the back
button did nothing - the only way off the screen was the tab bar.
An empty entry list is not merely bare, it is unrecoverable. Two
things in menu_driver.c require an entry to exist:
cbs = selection_buf && selection_buf->size ? ... : NULL
so with no entries MENU_ACTION_CANCEL is never dispatched, and
if (MENU_ENTRIES_NEEDS_REFRESH(menu_st) && selection_buf_size)
so nothing ever rebuilds the list either - including the rebuild
that would resume a read still in progress. The list cannot fill
and the user cannot leave.
menu_displaylist_parse_playlist_generic() only set
MD_FLAG_NEED_PUSH_NO_PLAYLIST_ENTRIES from INSIDE
if (playlist_get_cached()), so when the cache is deliberately empty
- which is what the previous fix made it while a different playlist
loads - no placeholder was requested and DISPLAYLIST_PLAYLIST, which
has no count == 0 fallback of its own, left nothing at all. The
flag is now set on that branch too. One placeholder entry keeps
both mechanisms alive: there is something for back to act on, and
the refresh keeps returning, which is what resumes the read.
Why neither report reproduced here is the important part. With
local stdio a playlist is read inside the first budgeted slice, so
the yield path is never taken and everything in it is invisible.
Android reaches content outside the app sandbox through SAF, which
RetroArch drives through a VFS implementation, and it is slow enough
that the parse really does stop part way - so what is an unreachable
branch on a desktop is the ordinary path there.
The oracle now installs its own VFS through filestream_vfs_init(),
the same seam the frontend uses, returning short reads the way a
slow remote provider does. Over it the same navigation takes 180
reads and yields four times, where local files yield none, and the
switch invariants are asserted under that timing. The lane checks
that its VFS was actually used, so it cannot quietly degrade into
another local-stdio test: pointing it back at the ordinary read
implementation fails it with "the slow VFS was not used".
Verified: strict C89 gate on menu_displaylist.c; full Linux build
links with HAVE_QT=1; the playlist oracle passes plain, twice under
ASan+UBSan+LSan and twice under TSan.
Not verified here: the menu-side behaviour itself. These lanes pin
the playlist reader under Android-like timing, and the menu fix is
reasoned from the two gates quoted above rather than exercised - a
harness that links the real menu is the missing piece, and is worth
building before anything else lands on this path.
Reported from the field, MaterialUI on Android: open "A", go back, open "B", and the A titles are listed under the B heading. Not a cosmetic mismatch - selecting one of those rows launches content from the wrong system.
The cause is a window I introduced with the deferred cached init.
The blocking playlist_init_cached() replaces the cache inside a
single call, so no caller can observe an in-between state. The
deferred variant yields mid-parse, and it left the OLD playlist
installed until the new one was ready - while the menu reads
playlist_get_cached() to build its entry list. For every frame the
parse had not finished, the menu drew the playlist the user had just
navigated away from, under the new one's heading.
The cache is now dropped the moment a different playlist is
requested, not when its replacement is ready. That costs nothing
which was not already being spent: this path is only reached because
the cache could not be reused, so it was going to be replaced
regardless. Callers see NULL until the parse completes, which they
already handle - menu_displaylist_parse_playlist_generic returns
early on exactly that, giving an empty list for the frame or two the
parse takes, and the refresh re-issues it.
Two regression lanes, both encoding the reported steps rather than
the mechanism:
- switching from one playlist to another asserts, on EVERY yield,
that the cache is either empty or already the requested playlist,
and that the finished result carries the requested playlist's own
entries. It also asserts the parse actually yielded, so the lane
cannot quietly stop covering the window it exists for.
- the impatient case - open one playlist, back out and open another
before the first finished, and again - asserts the same invariant
as each request supersedes the last.
Both fail with the fix reverted, naming the wrong playlist in the
message. They run in the existing samples CI step, whose comment
now records what they are for; that step has no TSan flavor, because
nothing on this path is threaded.
The gap that let this ship is worth naming: the cache lane already
there covered REUSING a playlist and never covered SWITCHING away
from one, which is the far more common action. Any deferred
replacement of a globally cached object needs a lane asserting the
old object is unobservable while the new one loads, not only that
the new one eventually arrives.
Verified: strict C89 gate on playlist.c; full Linux build links with
HAVE_QT=1; the CI step replayed literally; the oracle also passes
three TSan runs locally, and the playlist manager oracle, which
drives the same reader, still passes.
task_push_overlay_load_default() builds the image list with
string_list_new(), which allocates the list and a 32-entry element
array, but each early return between there and the task push
released only the list with free(). Every failed push leaked the
element array - 768 bytes on LP64.
The paths are reachable in ordinary use: an overlay config that
cannot be read out of its archive, one that is not valid config,
and one with no "overlays" key all land here, and a user cycling
through overlay packs hits them repeatedly.
Use string_list_free(). The list is empty at every one of these
returns, and the loader is released directly rather than through
task_overlay_free(), so there is no second owner.
Checked with LSan over samples/tasks/overlay driving a config with
no "overlays" key: 768 bytes leaked in 1 allocation before, clean
after. ASan+UBSan+LSan and TSan clean over the suite.
A successful overlay load hands the overlays, the overlay path and
the image list to the consumer through overlay_task_data_t, and the
task queue retires a task by running its callback and then its
cleanup. So input_overlay_loaded() releases the image list first,
and task_overlay_free() released it again straight after:
Abort message: 'Scudo ERROR: invalid chunk state when
deallocating address 0x200006d36fbb9c0'
#05 string_list_free+84
#06 task_overlay_free
#07 retro_task_internal_retire_cleanup
#08 rarch_main
Every successful overlay load aborts, which on Android means every
startup that brings up the touch overlay, and thereafter every
overlay switch and every content load.
The release was made unconditional to plug what looked like a leak
of the list container on the success path. It is not one - the
consumer frees it, on both of its paths (input/input_driver.c) -
so the conditional it replaced was right about the list. What the
old condition did get wrong is that RETRO_TASK_FLG_CANCELLED is not
what decides whether the loader still owns anything: a load that
finishes but cannot allocate its overlay_task_data_t returns
without handing off, and the cleanup then walked away from the
overlays, the path, the list and every texture in it.
Drop the loader's references as the hand-off takes them, and key
the cleanup on the pointers instead of on the flag. A pointer
still set is one nobody took, whichever way the load ended, so both
the second release and the missed one go away.
The oracle could not see this: its stub consumer took the overlays
and the path but left the image list alone, so the loader's release
was the only one in the run. Make it take the list too, the way
the real consumer does.
ASan+UBSan+LSan and TSan clean over samples/tasks/overlay; with the
oracle corrected and the fix reverted, ASan reports the
use-after-free above at task_overlay.c:1299.
TSan failed the content prefetch oracle in CI. The race is real and
is in the test, not in the task: the progress sampler read
task->progress directly while the worker wrote it through
task_set_progress(), which holds property_lock. There is a locked
accessor for exactly this, task_get_progress(), and the sampler now
uses it.
The reason it only appeared in CI is a second bug. The threading
mode is a global that task_queue_check() reconciles against on every
call, so the threaded lane - which sets it - silently converted every
lane that ran after it into a threaded run, including the two
callback lanes that believed they were unthreaded and were sampling
progress from the main thread while a worker wrote it. Each lane now
states the mode it wants rather than inheriting whatever ran before,
the threaded lane restores the default when it finishes, and it runs
last so a future mistake in restoring cannot quietly convert the
lanes above it into something other than what they claim to test.
Sweeping the other oracles for the same shape found it once more, in
the scanner's threaded lane: it read both found_task->title and
->progress raw, from the main thread, while its worker was live.
That one has simply been getting lucky - it is the same race, and it
now goes through task_get_title() and task_get_progress() too.
Verified by reproducing the failure rather than by inspection: the
merged prefetch oracle, built with TSan here, races on two runs in
three - which is why it passed the pre-merge sweeps and failed in
CI, and a reminder that a single clean threading run proves much
less than a repeated one. The fixed file is clean over eight
consecutive TSan runs, and over three runs each plain and under
ASan+UBSan+LSan; the scanner oracle is clean over five TSan runs.
No product change: task_set_progress() and task_set_title() were
already locking correctly, and what the tasks report is unchanged.
The progress field was added between system_name and icon_directory,
but the widget's static initializer is positional and was not
updated. Every initializer after that point therefore landed one
slot early: the brace meant for icon_directory initialized progress -
which is what the "braces around scalar initializer" warning was
reporting - and the two remaining braces initialized icon_directory
and icon_file, leaving icon_path to implicit zeroing.
The arrays all ended up empty either way, so nothing visibly broke,
but progress defaulted to 0 rather than -1. Nothing showed a
spurious "0%" only because gfx_widget_load_content_animation_reset()
assigns -1 and runs at the start of every animation - the default was
wrong and happened never to be read. That is luck, not design.
The field moves to sit with the hot per-frame fields, next to
has_icon, where it belongs: it is read every frame while a load
streams, not at layout time like the cold buffers it had been placed
among. Its initializer moves with it.
Verified: the translation unit compiles with -Wall -Wextra
-Wmissing-field-initializers with no initializer diagnostics at all,
and passes the strict C89 gate; the resulting slot mapping was
checked directly - progress reads -1 and every path buffer is empty;
a forced rebuild of the file is warning-free; the gfx_widgets
sanitize and msg-queue-race samples and the content prefetch oracle
all still pass.
Second half of the load progress work. The prefetch task reports a
read percentage; this puts it on screen, after the content name, in
the launch notification that already exists.
The notification used to fire AFTER retroarch_main_init() returned
true - a card celebrating a load that had already happened, which
cannot show progress for it. On the deferred path, where the
content streams in ahead of the load while the frontend keeps
running, it now starts when the read is pushed instead, carries the
percentage as the bytes arrive, and is left alone by the post-init
site rather than being restarted there. The percentage is cleared
once the read is over, so the card finishes as the plain launch
notification it has always been. Display widgets persist across
driver reinits by default, so the card started before the load
survives into the loaded core.
Gating is unchanged and is the setting's existing meaning: with
"Load Content" startup notification off, nothing is shown and no
progress is even reported - the prefetch task takes a NULL progress
callback and never calls it. Loads that do not defer - a bare
archive, a content:// URI, anything with argv - are untouched and
still get the post-init card with no percentage.
The percentage is the READ only. Core init, SRAM and shader setup
follow it, so it reaches 100 before the load is finished rather than
tracking the whole operation. That is honest for what it measures
and matches where the time actually goes on a large ROM, but it does
mean the last stretch sits at 100 - phase weighting, if wanted, is a
follow-up rather than something to fake here.
The oracle gains two lanes for the callback path, which is separate
from the task's progress field: a caller that never sees a callback
shows nothing however correct the field is. One asserts several
updates arrive, in order, with an intermediate value, ending at 100;
the other that a NULL callback is accepted and never called, which
is what every existing caller passes.
Verified: strict C89 gates on all three touched translation units;
full Linux build links with HAVE_QT=1; all eight oracle lanes pass
plain, under ASan+UBSan+LSan and under TSan, repeated in each
flavor; the gfx_widgets sanitize and msg-queue-race samples still
pass.
Not verified here, and worth a look on real hardware: how the card
reads across the driver reinit in the middle of a load, and whether
the percentage wants phase weighting. Neither can be judged from a
build.
Groundwork for showing load progress in the "Load Content" startup
notification. The prefetch task - the path a menu content load
already defers to, streaming the bytes a budgeted slice per tick
while the frontend keeps running - knew how many bytes it was moving
but never said so. It now reports a whole-number percentage through
the task's own progress field, which is what a UI can read without
any new plumbing.
Sizes are learned per item as each is opened rather than up front,
because an archive entry's uncompressed size is only known once its
header is read, so the denominator grows as a multi-file run
proceeds. For the single-file case the frontend actually defers, it
is exact from the first tick. The value is pushed only when the
integer percentage changes: a 4 MiB tick against a large ROM would
otherwise report the same number many times, and the queue wakes a
progress callback on every update.
The task had no oracle, which for a path that hands buffers to the
loader by ownership transfer is the wrong place to have a gap. The
new one (samples/tasks/content_prefetch) compiles the shipping
translation unit against the real queue and the real data_transfer
spine and pins: the deposited bytes are identical to the file; a
file several times the per-tick budget takes several ticks, so the
budget is really applied; progress reports an intermediate value and
ends at 100 without going backwards; several files all arrive;
an unreadable path is skipped and reported not-all-ok rather than
failing the run; and abandoning a transfer mid-stream deposits
nothing and leaks nothing.
One lane drives the THREADED queue on purpose. The others run the
handler on the calling thread, where a TSan run proves nothing about
the worker-to-callback hand-off - which is the only place in this
design a race could actually live.
Two things worth recording from writing it. The progress lane
initially asserted only that the first value was below 100, which is
trivially true of the initial zero; it now requires a value strictly
between 0 and 100, without which a "percentage" is just a two-state
flag. And the threaded lane first polled in a busy spin, which
starves the worker on a single-core machine - CI runners and this
container both - and failed under ASan while passing plain and under
TSan. That looked like a product bug and was the test hogging the
CPU; it yields between polls now.
Verified: strict C89 gate on the task, in both the default
configuration and with DEBUG defined; full Linux build links with
HAVE_QT=1; all six lanes pass plain, under ASan+UBSan+LSan and under
TSan, repeated runs in each flavor to shake out threading flake;
removing the progress call fails the progress lanes.
Closing content waited for in-flight save state tasks twice. The
first wait sat inside the savestate_auto_save branch, immediately
after the auto-state save it kicked off; the second runs
unconditionally a few lines below, against the same condition, with
nothing between them that can push a task. The first could
therefore never be the one that did any waiting.
Only the conditional wait is removed. The unconditional pair stays
exactly as it is: those waits protect a real invariant - both
task_save_handler and task_load_handler call into core function
pointers, and runloop_event_deinit_core unloads the dylib
underneath them - and eliminating that block is a separate question
about deferring content deinit, not a cleanup. The comment
explaining them absorbs the auto-save case rather than referring to
a wait that no longer exists above it.
With the wait gone the branch is a single statement, so its braces
go too, per the brace rule in CODING-GUIDELINES.
No behavioural change: the same condition is still waited on, at the
same point, before anything else happens.
Verified: strict C89 gate on retroarch.c, in both the default
configuration and with DEBUG defined - the conditional-code
configuration a previous change here slipped through; full Linux
build links with HAVE_QT=1; the save_state oracle, which covers the
tasks being waited on, still passes, as does the task ordering
oracle.
Two of the interactive task_queue_wait sites, both around the movie
tasks, and both resolved by the same underlying discovery.
runloop.c waited for the ENTIRE task queue to drain before starting
a movie recording, under a FIXME that admitted it did not know why
the guard inside task_moviectl_record_handler was insufficient and
guessed at either enqueue timing or a wrong finder call. The guess
about the finder was right, and the mechanism is worth stating
plainly: retro_task_regular_gather() lifts every running task off
tasks_running into a local list, then puts each one BACK as it
processes it. Whether content_load_state_in_progress() - a
task_queue_find() - sees a pending load therefore depends on where
that load sits in the pass relative to the caller. It is not
reliably blind, which is worse than if it were: it answers correctly
most of the time and wrongly some of the time, which is exactly the
shape of an intermittent ordering bug.
content_load_state_in_progress() now reads a main-thread flag, set
when a load task is pushed and cleared when its callback has applied
the state, including on the push-refused path where no callback will
run. That is the same correction already made in
task_database_info.c for the same class of window. With the guard
working, the whole-queue wait in runloop.c is deleted rather than
kept - the record task defers itself correctly now. The finder it
replaced is removed.
command.c's PLAY_REPLAY_SLOT handler waited on the whole queue so
that the movie task's callback would have installed the replay
handle before the reply was formatted. The handler it waits for is
trivial and finishes on its first invocation, so the wait was
already only about letting one tick happen - but a NULL condition
means "until the queue is empty", so a network command blocked on
every unrelated scan or download in flight. It now waits on a
predicate for that task alone. Producing the reply from the movie
callback instead would remove the wait entirely, but that needs
deferred-reply support in the command layer and is left for that
work.
A new oracle (samples/tasks/ordering) drives the shipping queue and
pins the contract this rests on: that a finder called from inside a
handler disagrees with the truth on at least one invocation, that a
main-thread flag does not, and that a task guarded by the flag runs
after the task it waits on. Reinstating the finder-based guard
makes the ordering lane fail with the dependent task running first -
the bug the FIXME described, reproduced.
Writing it corrected my own diagnosis: I expected the finder to be
uniformly blind for the whole pass, and the first version of the
lane asserted that and failed. Position dependence is the accurate
property and is what the lane now pins, so a future change to the
gather's put-back behaviour trips it deliberately.
Verified: strict C89 gates on task_save.c, task_movie.c, command.c
and runloop.c; full Linux build links with HAVE_QT=1; three lanes
pass plain, under ASan+UBSan+LSan and under TSan; the save_state
oracle - which covers the translation unit whose predicate changed -
still passes, as do the dirwalk, playlist, playlist_manager,
overlay, scan, NAT and watchdog oracles.
The slow-handler reporter held the last handler in a void* and cast
a function pointer into it. ISO C allows neither the conversion nor
the comparison, so any DEBUG build with the strict gate failed:
error: ISO C forbids comparison of 'void *' with function pointer
error: ISO C forbids conversion of function pointer to object
pointer type
There is already a typedef for exactly this - retro_task_handler_t -
so the fix removes both the cast and the void*, holding the handler
in its own type. No behaviour change.
The oracle carried the same construct; it is aligned, which also
keeps the assertion type-correct rather than comparing through a
cast.
My verification for the original commit claimed retroarch.c was
syntax-checked with DEBUG defined. It was - under gnu99, not the
strict C89 gate applied to every other file touched. The DEBUG path
is the only code here behind a conditional, so it was also the only
code the normal build never compiled.
Verified: the strict C89 gate now passes on retroarch.c with DEBUG
defined, and reproduces both errors exactly when the void* form is
put back; the same gate is clean on the other DEBUG block added by
this work in menu_displaylist.c; full Linux build links; the
watchdog oracle passes.
The guidelines covered C89 declarations, VLAs and brace style, but
said nothing about the rule that the last several rounds of task
work have been enforcing case by case: the UI and the main thread
must never block. That is the constraint a contributor is most
likely to violate without realising it, because when Threaded Tasks
is disabled - which it is on several platforms, and which users turn
off deliberately - task handlers run on the very thread that drives
the frame loop.
Four rules, each of them one that was actually broken in the tree
and had to be fixed:
- Do not call task_queue_wait() from anything reachable while the
menu is visible or a core is running; cancel and guard against a
stale completion, or continue from the task's callback.
- Never pass a NULL condition to task_queue_wait(). It does not
mean "wait for my task" - both queue implementations read it as
"until the queue is empty", so the caller waits out every
unrelated task in flight.
- A handler processing a collection consults the shared per-frame
I/O window between items and always completes at least one item,
and must not chunk by a count derived from the workload: "half the
entries" is not pacing, because a bigger job then just produces
bigger chunks over the same number of invocations.
- Putting work on a background thread is not a substitute for pacing
it, since the threaded queue is optional.
It closes by pointing at task_queue_set_slow_handler_cb(), which is
how a contributor can find out in a debug build whether the handler
they are writing respects any of this, rather than discovering it
from a user report.
Every statement was checked against the tree rather than written
from memory: no NULL-condition waits remain under network/ or
tasks/, none of any kind under menu/, and both APIs referenced exist
under the names given.
Also drops nbio_handle_t::pos_increment, found by sweeping for more
of the workload-derived chunking the overlay loader had. Nothing
else of that shape turned up; this one was assigned once and never
read, a leftover of the design the sweep was looking for.
Verified: strict C89 gate on the touched translation unit; full
Linux build links with HAVE_QT=1.
Both playlist manager handlers - clean playlist and reset cores -
were already per-entry state machines, doing one playlist entry per
handler invocation, with a single exception: PL_MANAGER_BEGIN read
the entire playlist in one blocking playlist_init(). For a large
collection that is tens of milliseconds of JSON in a single call,
and with Threaded Tasks off that call runs on the thread driving
the frame loop - while the progress bar the user is watching, for
the operation they just started from the menu, cannot repaint.
The resumable parse added for the menu's playlist first-touch is
exactly what this needs, so the load becomes two phases: BEGIN opens
the parse, and a new parse phase steps it under the shared per-frame
I/O window until the playlist is ready. Everything after it is
unchanged. Cancelling the task while the playlist is still being
read now releases the in-flight parse along with the partially built
playlist.
A new oracle (samples/tasks/playlist_manager) compiles the shipping
translation unit against the real task queue, the real playlist
reader and the real slice module, with a virtual clock so the
assertions do not depend on how fast CI happens to be. Its pacing
lane isolates the phase under test: the per-entry iteration costs
the same number of invocations either way - it was already one entry
per invocation - so the difference between a paced and an unpaced
run over the same playlist is the load, and nothing else.
Both changes were falsified against it. Stepping the parse with a
NULL budget - the old blocking behaviour - makes the paced and
unpaced runs identical, 3132 invocations each. Removing the cancel
cleanup leaks 19 KB across 225 allocations under LSan.
The cancel lane needed a correction worth recording: task_queue_reset
only FLAGS a task cancelled, and the unthreaded queue's deinit does
not drain, so a test that stops pumping there leaves the task and
everything it owns reachable from the queue's own list - where
LeakSanitizer sees no leak whatever the code does. The lane pumps
until the queue is empty and the cleanup has actually run; before
that correction it passed with and against the fix, which is to say
it proved nothing.
Verified: strict C89 gate on the task; full Linux build links with
HAVE_QT=1; all four lanes pass plain, under ASan+UBSan+LSan and
under TSan; the CI step was replayed literally; the playlist,
overlay, watchdog, dirwalk and scan oracles all still pass.
The overlay loader looked chunked and was not. Its work per handler
invocation came from pos_increment, computed as half of an overlay's
descriptor count and a quarter of the overlay count - a chunk
derived from the workload itself, which means a bigger overlay just
produces bigger chunks. The job always landed in the same two to
four invocations however long each one took, so the pacing did
nothing that pacing is for.
That matters because each descriptor item is an image load off
disk, and with Threaded Tasks off these invocations run on the
thread driving the frame loop. Overlays reload whenever the user
switches one or loads content, so this is a stall on an ordinary
interactive path, scaling with the size of the overlay - exactly the
shape the new task handler watchdog exists to report.
Both loading loops now consult the shared per-frame I/O window
between items, the way the budgeted directory walks, playlist parse
and scanner do: small overlays still finish in one invocation, large
ones spread over as many as they need, and neither case is decided
by the size of the job. Each loop always completes one item before
consulting, so a window that is already exhausted still makes
progress rather than stalling. pos_increment becomes dead and is
dropped from both structures.
Building the oracle turned up a second, older bug. loader->conf
and the overlays are released on every path, but loader->image_list
- the dedup index whose elements point at texture_images the
consumer takes ownership of - was freed only when the load was
CANCELLED. Every successful overlay load leaked the list container,
once per overlay switch and once per content load. It is now freed
on both paths; the textures it indexes are still released only on
the cancelled path, because on success they belong to the consumer.
The new oracle (samples/tasks/overlay) compiles the shipping
translation unit against the real task queue and the real slice
module, with image loading stubbed and a virtual clock so the
assertions are exact rather than dependent on CI machine speed. Its
lanes pin completeness (the same overlays and descriptors as an
unpaced run), pacing (the load spreads over more invocations),
progress under an exhausted window, and the property that actually
distinguishes this fix: work per invocation must not grow with the
size of the overlay.
Both fixes were falsified against it. Restoring the size/2 chunking
fails the scaling lane - 1.23 descriptors per invocation at 32
descs against 0.57 at 8, the linear growth being the defect - and
reverting the image_list free reproduces the leak under LSan.
Verified: strict C89 gate on task_overlay.c; full Linux build links
with HAVE_QT=1; all four lanes pass plain, under ASan+UBSan+LSan and
under TSan; the CI step was replayed literally; the dirwalk,
playlist, scan, NAT and watchdog oracles all still pass.
Phase C guardrail for the non-blocking work. With Threaded Tasks
off, task handlers run on the thread that also drives the frame
loop, so a handler that does not return promptly is a visible stall
- and until now nothing detected one. A regression in a budgeted
handler (the directory walks, the playlist parse, the scanner) would
show up as "the menu feels sticky", found by a user rather than by
the build, and attributing it back to a specific task meant
profiling.
The queue is the only place that can make that attribution, so the
measurement lives there: task_queue_set_slow_handler_cb() registers
a callback and a budget, and the untheaded gather reports any
handler invocation that exceeds it. Two properties keep the cost
honest. With no callback registered - the default, and what
release builds ship - the queue reads no clock at all. And only the
untheaded queue is measured: on the threaded queue handlers run on a
worker, where taking a long time is the entire point.
RetroArch registers it in debug builds only, with a one-frame budget
at 60Hz, and only when Threaded Tasks is off. The report names the
task and is rate-limited per handler, because a task that stalls
once usually stalls on every invocation and a warning per frame
would bury the first and most useful one.
The budget is deliberately loose relative to what the sliced
handlers aim for: they consult the shared per-frame I/O window and
finish far inside it, so anything crossing a whole frame in a single
call is either unsliced work or a slice that has stopped honouring
its budget - which is exactly the regression worth a warning.
A new oracle (samples/tasks/watchdog) compiles the shipping queue
and pins all four properties: a handler inside its budget is silent
however often it runs; one that overruns is reported once,
attributed to that task, with a duration above the budget; the
threaded queue produces no reports; and an unregistered watchdog
never fires, before or after being registered and cleared.
Verified: strict C89 gate on task_queue.c; retroarch.c
syntax-checked with DEBUG defined so the registration path is
compiled, not just skipped; full Linux build links with HAVE_QT=1;
all four lanes pass plain, under ASan+UBSan+LSan and under TSan;
the lanes were falsified by disabling the comparison, which they
catch; the dirwalk, playlist, scan and NAT oracles still pass.
task_queue_wait(NULL, ...) does not mean "wait for my task". A NULL
condition makes both queue implementations loop until the queue is
EMPTY, so a caller blocking on a single HTTP round trip also waits
out every unrelated task in flight. Starting a hosted session
through a relay server therefore blocked the UI not for the tunnel
query but for whatever else happened to be running - a content scan,
a core download, a thumbnail pack - which is minutes, not
milliseconds, and is a far worse freeze than the one the site looks
like it has.
The MITM query now waits on a condition that describes its own work:
a flag set before the transfer is pushed and cleared by its
callback, on every path that callback can take (the queue runs the
callback of every task it retires, so no completion leaves it set).
The same change closes four latent copies of the pattern in
natt.c's blocking variants, which now wait on the device's own busy
flag - every one of those callbacks clears it. Those four are a
trap being closed rather than a live freeze: no in-tree caller
passes block = true today, because the NAT task drives them
non-blocking and steps its own state machine.
What this does NOT do is make hosting non-blocking. Host setup
reads the tunnel address decided by the query - sockets,
announcement and lobby registration all depend on it - so a wait
remains, now bounded by that one query. Removing it means
deferring the rest of netplay host initialisation into the query's
callback, which is a restructuring of netplay init rather than a
change to this wait, and is not attempted here.
A CI guard keeps the unbounded form from returning to network/ or
tasks/, where every remaining wait now names a condition. It does
not sanction waiting: the menu paths are held to the stricter
no-wait guard alongside it.
Verified: strict C89 gate on natt.c, netplay_frontend syntax-checked
under its feature set; full Linux build links with HAVE_QT=1; the
NAT, playlist and dirwalk oracles still pass; the guard was
falsified by restoring an unbounded wait, which it catches.
Reviewed rather than exercised: neither site is reachable from a
test harness here - the MITM path needs a lobby server and lives in
a translation unit too large to link standalone, and the natt.c
blocking variants have no in-tree caller. The reasoning above is
the evidence, and the queue semantics it rests on are the ones the
NAT oracle already pins.
Both NAT entry points began by blocking the calling thread until any
in-flight NAT task finished. The exclusion is real - the two
requests mutate the single shared nat_traversal_request object, so a
second concurrent task would race the first over the same struct -
but the enforcement was a full stop of whichever thread asked, and
the thread that asks is the one driving the menu. The work being
waited on is UPnP discovery: SSDP with timeouts, seconds of frozen
UI for anyone who starts hosting and stops again before discovery
completes.
Requests are queued instead and started from the task callbacks,
which run on the main thread after the task has been popped off the
queue, so the resumed push finds the queue free. The close task
gained a callback for this - it had none, so nothing would have
resumed a request queued behind it.
Order is preserved rather than coalesced: a close followed by an
open has to run as both, in that order, or the port mapping the
close was meant to release stays open. Requests beyond the queue
depth are refused, which is the same answer callers already handle
from any other push failure.
A new oracle (samples/tasks/netplay_nat) compiles the shipping
translation unit against the real task queue with the natt_* device
layer faked by a deterministic stand-in for the network round
trips, and pins: a push issued while a task is in flight returns
without having run that task to completion; a close then an open
runs as "close, open" and not one merged operation; queued requests
actually start from both callbacks; overflow is refused rather than
dropped silently.
Writing it caught a defect in the first version of this change. A
re-pushed request can still be legitimately refused - a close whose
validation fails because the open ahead of it found no device - and
a refusal starts no task, so no further callback arrives to start
anything else. Stopping at the first refusal stranded every
request behind it forever. The flush now drains until something
actually starts.
The no-wait CI guard widens to cover this translation unit, which
is what should happen as each remaining wait site is converted; the
core deinit save/load and netplay MITM sites are still outstanding
and still outside it.
Verified: strict C89 gate on the task; full Linux build links with
HAVE_QT=1; all four oracle lanes pass plain and under
ASan+UBSan+LSan with zero reports; the lanes were falsified by
restoring the blocking wait, which they catch and name.
Menu deinit blocked the main thread on two task_queue_wait() calls
before freeing the explore state and the database info cache. Both
are reachable while the user is looking at the menu, and the work
being waited on is unbounded: a database info scan is a full
libretro-db query, seconds of frozen UI on slow storage, and on
Android an ANR.
Neither wait was protecting a worker mid-write into the state being
freed - the reason such waits usually exist. Both tasks build their
result into their own handle and install it only from their
main-thread callback: menu_explore_build_list touches neither the
global explore state nor the icons (those load in the callback, on
the main thread), and the database scan writes only its own
db_list. What the waits actually prevented was a LATE callback -
one arriving after teardown - installing into state that had just
been freed, which for explore means loading icons through a
torn-down video driver.
That is a stale-completion problem, and the tree already has the
answer for it: the generation token menu_dirwalk uses. Teardown now
calls menu_explore_cancel_init_task() and menu_dbinfo_cancel_task(),
which bump a generation and flag the in-flight task cancelled; a
completion carrying an older generation installs nothing and lets
the task's own cleanup free what it built. The main thread returns
immediately.
Two details worth naming, both found while checking rather than
assumed:
- The queue runs a task's callback whether or not it was cancelled,
so a stale completion does reach the callback. It must not clear
the slot or the pending flag belonging to a NEWER task pushed
after the cancel - for dbinfo that would drop the single-scan
invariant and let two scans fight over the one cache slot. Both
callbacks now clear only when the retiring task is still the one
owning the slot.
- While a cancelled explore task drains, the find-based
in-progress check still reports true, so a fresh initialisation
pushed in that window is refused. This self-heals: the push sits
in the explore displaylist build, which re-runs every rebuild
while showing the "initialising" placeholder, and succeeds once
the old task drains.
A CI guard keeps the property: task_queue_wait() must not appear
under menu/ or in the two task translation units the menu drives.
Its scope is deliberately narrow - the remaining wait sites (core
deinit save/load, NAT traversal, netplay MITM) are separate work,
and widening the guard is what should happen as each is converted.
Verified: strict C89 gates on both task TUs; full Linux build links
with HAVE_QT=1; the dirwalk, playlist and scan oracles all still
pass, the scan one under TSan as well; the guard was falsified by
reintroducing a wait on the deinit path, which it catches.
The cached playlist is reused only while the size recorded when it
was read still matches the file on disk - the VFS exposes no
portable modification time, so that stamp is the backstop against
outside edits. Two things were wrong with it.
First, a real bug, and the reason this started: when the cached
init finds the on-disk format or compression disagreeing with the
settings, it converts the file immediately - and a conversion
changes its size substantially. The stamp was left describing the
file that existed before the rewrite, so every later reuse check
compared it against the new size, never matched, and the playlist
was re-read and re-parsed on every single menu navigation, for as
long as the file disagreed with the settings. Any user whose
playlists predate a format or compression setting change has been
paying a full parse per navigation. playlist_write_file() cannot
fix this itself: it defers to playlist_cached_after_write(), which
only knows how to refresh the playlist already installed as the
cache, and this one is installed immediately after. The install
path now refreshes the stamp itself. This is long-standing
behaviour, not something the resumable parse introduced - it
reproduces on the code before that series.
Second, the micro-optimisation this commit set out to make: the
read path stat'd the playlist for its size and then opened it, and
now that the parse owns the stream from the start, the open handle
can answer instead. Only for uncompressed playlists, though - and
this is the trap. A compressed stream reports the UNCOMPRESSED
size from its rzip header, a completely different number from the
on-disk size the reuse check compares against, so taking it from
the stream unconditionally would have defeated the cache for every
compressed playlist exactly as the bug above does. Compressed
therefore still pays the stat; uncompressed - the default
configuration - saves one filesystem round trip per playlist read,
which is a real one on the network VFS backends, on the path the
menu takes for every playlist it has not cached.
The oracle gains a cache-reuse lane covering JSON, an old-format
file converted on load, and a compressed file, asserting reuse by
marking the cached playlist in memory: pointer identity is not a
sound observable here, because a re-parse frees the old playlist
and the allocator hands the same address straight back, which is
how the first version of this lane passed against the bug. Each
fix was falsified against its own lane: reverting the stamp
refresh fails the converted-old-format case, and capturing the
compressed size from the stream fails the compressed case.
Verified: playlist.c passes the strict C89 gate; all eleven oracle
lanes pass plain and under ASan+UBSan+LSan with zero reports; the
scan oracle (which links playlist.c) passes plain, ASan+UBSan+LSan
and TSan; the CI step was replayed literally; full Linux build
links with HAVE_QT=1.
The rm3u decoupling added libretro-common/samples/formats/m3u, and
the samples workflow treats any built target that appears in
neither RUN_TARGETS nor NORUN_TARGETS as an error - deliberately,
so that a new self-testing sample cannot be silently compiled and
never executed, which is how seven regression tests once came to
be dead weight. That guard did its job here: the job has been
failing since the rm3u commit merged.
rm3u_test belongs in RUN_TARGETS: no arguments, self-contained,
exits non-zero on failure. It builds and passes under the
workflow's default SANITIZER=address,undefined.
An audit of every sample Makefile against both lists finds no
other unclassified target.
Verified: the workflow's exact build and run commands
(make clean all SANITIZER=address,undefined, then the binary under
timeout) replayed locally - builds warning-free, all four lanes
pass, exit 0; YAML parses.
Reading a playlist is tens of milliseconds of JSON parsing and six
allocations per entry, and until now all of it happened in one
blocking call on whichever thread asked - including the menu
thread, on the first touch of a large playlist. The dirwalk work
capped the directory-enumeration half of that stall; this is the
parsing half.
playlist_read_file becomes a resumable state machine behind
playlist_parse_begin / _step / _end / _abort. begin() opens the
file, sniffs the format and (for JSON) opens the parser; step()
advances the parse, consulting an optional budget callback between
batches of work and returning 0 when it runs out; end() yields the
finished playlist. Crucially playlist_init() is now exactly
begin + one unbudgeted step + end, so the blocking and budgeted
paths run the same code and cannot drift apart.
The JSON phase replaces the rjson_parse() callback driver with a
caller-side rjson_next() loop. This is a transcription, not a
rewrite: the driver's only private state was in_object and the
member-vs-value discrimination, and both are recoverable from the
public context queries (a string at object level with an odd
context count is a member name), so the loop is stateless between
events and can stop anywhere. The old six-line format budgets at
line-group granularity, and the autofix path-rewriting pass -
previously an unbounded loop in playlist_init - becomes a third
budgeted phase over the same handle.
Wiring: menu_displaylist_set_new_playlist calls the deferred
cached init under the shared per-frame I/O window, exactly as the
dirwalk sites do. Small and medium playlists complete inside that
one pass and behave as before; a large one yields, the entries are
marked for refresh, and the re-issued request resumes the same
parse rather than restarting it. RARCH_MENU_CTL_DEINIT abandons a
pending parse before the cache it would install goes away. The Qt
grid view keeps its blocking cold path - it must return a
populated model to its callers, so budgeting it needs Qt-side
continuation machinery rather than a drop-in swap; the comment
there now says so.
Aborting mid-document exposed the same staged-entry hazard the
oracle found in the finish path: an entry reserved one past the
committed length with members already strdup'd into it. Both
paths now clear it.
The oracle gains four lanes: budgeted JSON (5000 entries, 23
yields, output identical to the blocking parse entry by entry),
budgeted old format, budgeted autofix, and an abort lane that
abandons a parse mid-document with entries staged.
Verified: playlist.c passes the strict C89 gate; all ten oracle
lanes pass plain and under ASan+UBSan+LSan with zero reports; the
scan oracle (which also links playlist.c) passes plain, ASan+UBSan
+LSan and TSan; the dirwalk oracle still passes; the CI step was
replayed literally; full Linux build links with HAVE_QT=1.
First commit of the resumable playlist parse series. Before the
reader is restructured (callback driver to pull loop, then a
budgeted parse for interactive first-touch loads), this pins its
observable behaviour in a new oracle
(samples/playlist/playlist_parity_test) compiled from the shipping
translation unit, with hand-derived expectations per fixture:
- the full JSON schema: every header meta field, the scan record,
entries carrying every member the handler accepts including
subsystem rom arrays and runtime / last-played values, and
unknown members (flat and nested) skipped for forward
compatibility
- the surprising malformed-JSON contract, pinned deliberately:
a parse error WARNS and keeps everything parsed up to the
error, and playlist_init still succeeds - only allocation
failure fails the read
- empty JSON string values are treated as absent (no allocation,
member stays NULL)
- the old six-line format with mixed CRLF/LF endings and the
trailing metadata block (default core, label display mode,
thumbnail modes as "right|left", sort mode)
- the same JSON document behind the RZIP interface
- a missing file reads as an empty playlist, successfully
- entries beyond config.capacity are dropped, entries within it
survive
Writing the truncated lane flushed out a real leak:
JSONStartObjectHandler reserves the next entry slot with
RBUF_TRYFIT but the commit (RBUF_RESIZE) lives in
JSONEndObjectHandler, so a parse that stops inside an items
object - malformed input or OOM - leaves an entry staged one past
the committed length with members already strdup'd into it.
playlist_free walks the committed length only; the staged
members leaked. playlist_read_file now frees the staged entry on
that path, identified precisely as current_entry sitting at the
one-past-length slot (between complete entries it legitimately
points at the last committed one). LSan gates the fix through
the new CI step permanently.
Verified: playlist.c passes the strict C89 gate; all six lanes
pass plain and under ASan+UBSan+LSan with zero reports (no
threads on this path, so no TSan flavor); the CI step was
replayed literally; full Linux build links with HAVE_QT=1.
The rename swept every *.c / *.h, Makefile.common, griffin and the
Apple project files, but the residual grep never matched
extensionless sample Makefiles - and the scanner oracle's Makefile
still compiled formats/m3u/m3u_file.c, a path that no longer
exists. The oracle runs in Linux-samples-tasks CI, so that step
has been failing since the rename merged.
One line: m3u_file.c becomes rm3u.c. A tree-wide grep across
Makefile* finds no other stale reference to any of the renamed or
removed format entry points.
Verified: scan_begin_budget_test builds and passes at this tip.
Audit of every rjson / rxml call site's surrounding file I/O,
following the codecs' decoupling. The rxml side needs nothing:
its production consumers parse buffers that arrive without the
adapter (logiqx through the scanner's budgeted incremental read,
natt from network payloads), and the filestream adapter is used
only by the sample harness and the treehash tool. On the rjson
side, the large streaming consumers are already shaped right -
playlist, core info cache and the explore state read through
buffered (and RZIP-aware) streams with no redundant pre-open
checks, cloud sync opens tolerantly with no extra stats, and the
writer flushes in buffered chunks - but the small-file readers all
shared one wasteful pattern: stat the path (path_is_valid), open
it, let the stream adapter fstat it again for io block sizing,
then read a few hundred bytes through the chunked callback
machinery.
Five readers convert to filestream_read_file() +
rjson_open_buffer(): the runtime log (the hot one - playlist
sublabels showing per-core runtime build one of these per entry),
the disk index record, the two test input drivers, and the Remote
RetroPad core (the latter three being the reference code this
pattern kept getting copied from). One open/size/read/close per
file, parse from memory, no callback crossings - and the
up-front stat is gone: filestream_read_file's failure covers
missing files, and path_is_valid now runs only inside the failure
branch to decide between the old quiet no-file path and the old
error message, so every log line and return value is preserved
while the common case pays one filesystem operation less. These
files are a few hundred bytes to a few KB and always parsed in
full; holding them in memory for the parse is what the parser's
own input buffer was already doing, minus the chunking.
Verified: strict declaration-placement gates clean on the C89
TUs; the net-retropad translation unit syntax-checked standalone;
full Linux build links with HAVE_QT=1; no rjson_open_filestream
readers remain outside the write-side handles the buffered writer
still streams through.
Same contract as rjson and rxml before it: a format in
libretro-common/formats imposes no file I/O through its interface.
rm3u was the most entangled of the three - rm3u_init read the file
from inside the constructor, rm3u_save opened and wrote from inside
the codec, and rm3u_is_m3u answered "is this an m3u" with two
filesystem stats. The codec now performs no I/O at all:
rm3u_init creates an empty handle, rm3u_parse consumes a
caller-supplied buffer, rm3u_dump renders the file contents as a
single caller-owned string, and rm3u_is_m3u is a pure extension
check. formats/rm3u_stream.h is the opt-in filestream wiring,
reproducing the old load / save / is-m3u behaviour for the two
task callers; cheevos' include turned out to reference nothing and
is dropped.
Decoupling exposed where the I/O was wasteful, so the adapters do
less of it - which matters most through network VFS backends,
where every operation is a round trip:
- rm3u_is_m3u_filestream answers extension + exists + non-empty
with ONE stat where the old check used two: a missing file has
no size, so path_get_size() alone settles both questions. The
scan and playlist-cleanup tasks call this per candidate path.
- Saving now renders everything and writes ONCE where the old
writer issued one or two filestream_printf calls - each an
unbuffered write - per entry.
- The parse itself became a single bounded cursor pass: the old
loader split the file into a string_list (one allocation and
copy per line) before copying each line again into stack
buffers. The rewrite walks the buffer in place with
length-bounded copies, drops the codec's string_list dependency,
and preserves the old truncation and whitespace-trim semantics
exactly.
A new oracle (libretro-common/samples/formats/m3u) pins the
rewrite: a parse lane covering every construct the old loader
understood (#LABEL:, #EXTINF:, path|label, plain paths, comments,
CRLF endings, surrounding whitespace, empty lines, dangling
labels) with relative-path resolution asserted; a dump lane
asserting the rendered bytes equal the old per-line writer's
output literally, for all four label types; round-trips; and an
adapter lane against real files (wrong extension rejects, missing
file loads empty, empty file is not an m3u, save/load is
lossless, bytes on disk equal rm3u_dump). The codec and adapters
have no threads, so the sweep is ASan+UBSan+LSan; a TSan run
would exercise nothing.
Verified: rm3u.c and the adapter header pass the strict C89 gate
standalone; strict gates clean on both migrated task TUs; full
Linux build links with HAVE_QT=1; oracle passes plain and under
ASan+UBSan+LSan with zero reports.
Aligns the M3U codec with the naming convention every other format
in libretro-common/formats follows (rjson, rxml, rbmp, rpng, ...):
the translation unit becomes formats/m3u/rm3u.c, the header
formats/rm3u.h, and the full API surface renames with it -
rm3u_t / rm3u_entry_t, enum rm3u_label_type with RM3U_LABEL_*,
RM3U_EXT, and rm3u_init / _free / _get_path / _get_size /
_get_entry / _add_entry / _clear / _save / _qsort / _is_m3u. The
internal struct tag becomes rm3u, which also retires the comment
apologising for the old workaround tag.
Purely mechanical: no behavioural change anywhere. The three
in-tree consumers (task_database, task_playlist_manager, and
cheevos' include - which uses nothing from the header) migrate,
along with Makefile.common, griffin, and the two Apple project
files that list the header. Local variables in the task callers
tidy from m3u_file to m3u so the rename reads complete rather than
half-done.
Verified: rm3u.c passes the strict C89 gate standalone; strict
gates clean on both migrated task TUs; full Linux build links with
HAVE_QT=1 after invalidating the four stale dependency files that
still named the old header; zero references to any old identifier
remain in the tree.
Same contract as the rjson commit before this: a format in
libretro-common/formats imposes no file I/O through its interface.
rxml's one violation was rxml_load_document(path), which opened,
sized and read the file through filestream inside the codec
translation unit - its only reason to include the stream layer.
The loader moves to an opt-in, header-only adapter
(formats/rxml_stream.h) as rxml_load_document_filestream(),
byte-for-byte including the negative-size hardening (a -1 from
filestream_get_size otherwise flows into malloc(0) plus a far
out-of-bounds NUL write), now expressed through the public
rxml_load_document_owned() with the identical ownership contract.
rxml.c and rxml.h reference no I/O at all; the buffer entry points
and the incremental rxml_parse_* API were already pure, so a caller
with its own I/O reads the bytes however it likes and hands them to
rxml_load_document_owned. logiqx_dat, the other formats consumer
of rxml, was already buffer-only and needs nothing.
The two in-tree callers of the path loader - the rxml sample
harness and the rxml_treehash tool - migrate to the adapter.
natt.c includes rxml.h for the buffer API only and is untouched.
Verified: rxml.c passes the strict C89 gate standalone; the
adapter header compiles standalone under the same gate; full Linux
build links with HAVE_QT=1; the sample harness parses a fixture
through the adapter printing the full tree; the treehash tool
builds and hashes; no references to the removed loader remain
anywhere in the tree.
The contract for libretro-common/formats is that a codec imposes no
file I/O through its interface: parsing and writing run over
caller-supplied buffers or caller-supplied callbacks, and how bytes
reach the parser - filestream, intfstream, or anything else - is
the caller's decision, not the format's. rjson violated it four
ways: rjson_open_stream / rjson_open_rfile and their writer
equivalents lived inside rjson.c, coupling the codec translation
unit to both stream layers for the sake of four thin shims.
The shims move to a new opt-in, header-only adapter
(formats/rjson_stream.h) as static INLINE functions with names that
say what they couple to: rjson_open_intfstream /
rjson_open_filestream and the rjsonwriter equivalents, preserving
the size-scaled io block heuristic. Including the adapter couples
the including translation unit - and only it - to the stream
layers; rjson.c and rjson.h now reference no I/O at all, and a user
with their own I/O wires rjson_open_user / rjsonwriter_open_user
directly, exactly as the in-tree callers now demonstrate.
All sixteen call sites across the nine in-tree callers migrate
mechanically (core_info, net-retropad, disk_index_file, the two
test input drivers, menu_explore, playlist, runtime_file,
cloudsync). One latent bug surfaced: rjson.c used INLINE while
reaching retro_inline.h only transitively through the stream
headers it no longer includes; it now includes it explicitly.
Verified: rjson.c passes the strict C89 gate standalone (it could
not before - the missing include was masked); full Linux build
links with HAVE_QT=1; strict gates clean on the migrated C89
callers; net-retropad translation unit syntax-checked; no
remaining references to the removed constructors anywhere in the
tree.
* dir_list: expose the sort comparators
dir_list_sort() and dir_list_sort_ignore_ext() each pick one of four
static qsort comparators. dir_list_sort_cmp() now hands the chosen
comparator to callers that need the identical ordering through a
different sorting strategy - concretely, a menu-side incremental
sort that must produce the same listing order as today's blocking
qsort while doing bounded work per invocation. The two sort entry
points route through the selector, so there is exactly one source
of the ordering and no behaviour change.
Two listings sorted with the same comparator agree everywhere
except among entries that compare equal (case-insensitive twins in
one directory), whose relative order qsort never specified in the
first place; the header says so.
Verified: strict C89 syntax pass; full Linux build links; the
dirwalk oracle added alongside the first consumer asserts listing
parity against dir_list_sort() across every flag combination the
menu uses.
* menu: add menu_dirwalk, a budgeted cancellable directory walk
Every interactive displaylist that lists a directory - the file
browser's three dir_list_initialize variants plus its sort, and the
four playlist-directory walks - performs the enumeration and the
sort synchronously on the thread that drives the UI, a stall
proportional to directory size. menu_dirwalk is the mechanism the
follow-up commits route those sites through; this commit adds the
module, its build wiring, and its oracle, with no callers yet.
A request first runs the walk synchronously under the shared
per-frame I/O window (tasks/task_nbio_slice.c): dir_list_iter
consults the budget between entries with a one-entry floor, so small
and medium directories - the overwhelming case - complete on the
spot with the identical single enumeration and single sort as today,
merely capped. A directory too large for the window continues on a
silent retro_task and, when Threaded Tasks is on, leaves the main
thread entirely; completion fires a registered refresh callback (the
task_menu_explore hand-off pattern), and the repopulation that
callback triggers re-issues the identical request and consumes the
finished listing. Nothing is enumerated or sorted twice. One walk
is pending at a time, identity is the full parameter set plus a
consumer tag, a superseding request or menu_dirwalk_cancel() ends
the previous walk, and a generation counter makes late completions
of cancelled walks no-ops.
Sorting turned out to be the larger half of the stall: qsort of a
40k-entry listing costs ~15ms of main-thread time, measured as an
18.5ms worst gather when the sort ran whole in the handler. Above
a synchronous threshold the sort is therefore resumable too - fixed
runs qsorted one or more per window, then bottom-up merge passes
with the budget consulted every batch of copied elements, ordered
by the exact comparator dir_list_sort_cmp() exposes. A walk that
fits the window but produces an oversized listing defers just the
sort. Scratch allocation failure falls back to the one-shot qsort:
a stall is a lesser evil than a lost listing.
The oracle (samples/menu/dirwalk, wired into Linux-samples-tasks)
compiles the shipping TU with shrunken sort constants and provides
the clock itself: real time for the pacing and bench lanes, a fixed
virtual step for deterministic window exhaustion in the deferral,
cancel and threaded lanes. Parity asserts a consumed listing equals
dir_list_new() plus the same sort across the seven flag/sort
combinations the displaylist callers use, on a flat 40k-file fixture
(the walk is non-recursive; flat is what a large ROM directory looks
like). Pacing: worst main-thread stall 4.17-4.25ms across three
runs against a 16ms gate, versus 18.5ms before the resumable sort.
Bench: best-of-three wall clock to consumed listing 34.2-35.6ms
versus 28.6-29.0ms blocking (ratio 1.18-1.24, gate 1.35) - the
honest price of per-entry budget checks and the merge's copy
traffic, paid only on the deferred path; a regression to a second
enumeration or a quadratic measures 2x and up. The cancel lane
supersedes and cancels mid-walk and drains stale completions with
LeakSanitizer judging teardown; the threaded lane is the TSan
target for the worker-handler to main-callback hand-off.
Verified: strict C89 syntax pass on the new TU; full Linux build
links; oracle three consecutive plain passes plus
ASan+UBSan+LSan and TSan runs with zero reports, CI step literal-
replayed locally.
* menu_displaylist: drive the file browser through menu_dirwalk
filebrowser_parse walked and sorted its directory synchronously on
the thread that drives the UI - three dir_list_initialize variants
(subsystem extensions, MANUAL_SCAN_DAT/SIDELOAD without compressed
files, and the default) plus a dir_list_sort - so opening a large
content directory froze the menu for as long as the walk took. The
three variants now route through menu_dirwalk_request with the
dirs-first sort, and the outcomes map exactly onto the existing
control flow:
DONE moves the completed heap listing into the local string_list
the append loop reads (the loop itself is untouched), with the
module's sort making the local dir_list_sort call redundant for
that path; the archive branch keeps its own listing and its own
sort, unchanged - an in-memory enumeration after the archive read
is a different cost class. PENDING appends a single inert
"Loading" placeholder and jumps to the parent-directory prepend, so
backing out stays possible while the walk continues in the
background - off the main thread entirely when Threaded Tasks is
on. FAILED falls into the same directory-not-found entry a failed
dir_list_initialize produced before.
The hand-off back is the explore-task pattern: the registered
refresh callback marks ENTRIES_NEED_REFRESH | PREVENT_POPULATE on
the main thread, the repopulation re-issues the identical request,
and menu_dirwalk hands over the finished sorted listing - one
enumeration, one sort, ever. The list's contents change exactly
once, at that refresh, so cursor and scroll state are never
invalidated mid-fill. Navigating to a different directory
supersedes the pending walk through request identity;
RARCH_MENU_CTL_DEINIT cancels outright, and the generation guard
makes any late completion a no-op.
Small and medium directories complete inside the synchronous
budgeted pass and never see any of this: same single walk, same
single sort, same single tick, no placeholder, no refresh.
Verified: full Linux build links; strict declaration-placement gate
clean on both touched TUs; dirwalk oracle re-swept after the tag
enum addition - plain, ASan+UBSan+LSan and TSan all pass with zero
reports. Manual smoke on Ozone/XMB/RGUI (large directory open,
placeholder, refresh-to-full-list, navigate-away mid-walk) still
owed on real hardware before the series goes upstream.
* menu_displaylist: budget the playlist-directory walks
The four remaining directory walks in interactive displaylists all
enumerate the playlist directory synchronously: the Playlists tab
(menu_displaylist_parse_playlists, dir_list_initialize + a
sort-ignore-ext), and the three DIR_LIST_COLLECTIONS listings
behind add-to-playlist, the playlist manager and the thumbnail
downloader (dir_list_new_special + a dirs-first sort each). A
playlist directory is rarely huge, but every one of these runs on
the thread that drives the UI on every visit, and after the file
browser's conversion they were the last unbounded dir_list_new*
calls on any interactive menu path except the core list, which is
converted separately.
Each site now issues the equivalent menu_dirwalk request - the tab
keeps its NULL-extension, directories-included walk with the
ignore-extension sort; the three COLLECTIONS sites request exactly
what DIR_LIST_COLLECTIONS resolved to, "lpl" with no directories
and no compressed files, dirs-first sort - under its own consumer
tag. DONE hands the module-sorted listing to the untouched
consumption loop (the in-loop sort calls are gone with the walks
that needed them); PENDING appends the same inert "Loading"
placeholder the file browser shows and returns, with the refresh
callback rebuilding the list on completion; a failed walk leaves
the same early-out or NULL-list path each site already had. All
five converted consumers share the single pending slot by design -
the menu shows one list at a time, and a request from a different
list supersedes whatever was in flight.
Verified: full Linux build links; strict declaration-placement
gate clean on the TU; anchored replacements verified unique before
application. Manual smoke of the four lists rides with the
file-browser smoke already owed on real hardware.
* menu_displaylist: budget the core-list directory walk
menu_displaylist_parse_cores slipped through the first inventory
because it builds its listing with dir_list_append rather than
dir_list_new - same unbounded synchronous walk of the cores
directory plus a dirs-first sort on the thread that drives the UI,
behind "Load Core". A cores directory is usually a few hundred
entries, so this is the fast path almost always; the conversion
buys the bound, not a visible change.
The common path now issues the equivalent menu_dirwalk request
(extension filter from the displaylist info, directories included,
no compressed files, dirs-first sort) under its own tag, with the
usual outcome mapping: DONE hands the module-sorted listing to the
untouched loop and skips the local sort; PENDING shows the inert
placeholder after the parent-directory entry so backing out works;
a failed walk lands in the same directory-not-found entry a failed
append produced. UWP keeps its one-shot construction and local
sort unchanged - it folds the optional core packages into the same
listing, a platform special case of a handful of directories that
menu_dirwalk's single-directory contract does not cover.
Verified: full Linux build links (object and binary mtimes checked
against the source); strict declaration-placement gate clean on the
TU; anchored replacements verified unique before application.
* materialui: document why the icon-list walk stays synchronous
The Phase 2 plan flagged materialui's playlist-directory
enumeration for conversion pending investigation of how hot the
path is. Investigated: it runs at driver init and on
RESET_HORIZONTAL_LIST only - not per navigation - over the playlist
directory, and both call sites expect the icon list populated on
return (RESET_HORIZONTAL_LIST destroys, refreshes and re-uploads
textures in one synchronous sequence). Deferring it would also
put an init-time walk into menu_dirwalk's single pending slot,
where the first displaylist build would supersede it and the icon
list would silently never fill, since the driver has no refresh
channel to re-enter through.
A cold path over a small directory, with a correctness hazard and
a driver rework as the price of conversion: it stays blocking, and
the comment at the site says so for the next audit that greps
dir_list_new_special.
* menu/qt: playlist first-touch through the cache
Audit of every interactive playlist_init call in the menu and the
Qt UI, per the Phase 2 plan. One menu site and four Qt sites
parsed a playlist from disk synchronously on the thread that
drives the UI; three convert to cached-first, one converts with
mutation through the cached instance, and one stays with its
reason recorded at the site.
The menu site (menu_displaylist_parse_database_entry) reads the
database playlist for CRC matching and is reached from that very
playlist, so the cached instance is the common case: reuse it when
playlist_get_conf_path matches, parse otherwise, free only what
was parsed.
Qt's PlaylistModel::getPlaylistItems is the first-touch hitch the
plan targets - the grid view parsed the whole file on the UI
thread at every open. It now borrows the cached playlist when the
paths match; every field the loop reads deep-copies into QStrings,
so nothing outlives the borrow. The single-entry update and
delete-index handlers adopt the cached-mutate template the file
already uses for core association (mutate the object the menu
reads, write to disk, free only if loaded), replacing a full parse
per one-entry edit. The bulk add-files dialog stays on a private
parse: its cancel path discards the half-modified playlist by
freeing it unwritten, which a borrowed cached instance cannot
offer - the comment at the site says so.
The remaining first-ever-open of a huge playlist with a cold cache
still parses whole; a resumable rjson playlist parse is Phase 2.5
scope, per the plan, and the cached path covers the in-session
requirement. The nine menu_displaylist_build_list calls under
ui_qt_widgets.cpp were re-checked during this audit: all settings
enumerations, no directory or playlist I/O behind any of them, so
no Qt displaylist conversion is needed - the CoA's Qt exposure was
the playlist parses handled here.
Verified: full Linux build links with HAVE_QT=1 (ui_qt_widgets.cpp
compiled clean) and with the menu TU under the strict declaration-
placement gate; anchored replacements verified unique before
application.
* menu_displaylist: flag over-budget builds in debug builds
The watchdog the Phase 2 plan closes with: with Threaded Tasks
off, every displaylist build runs on the thread that drives the
UI, so any build past frame scale is a user-visible hitch of
exactly the kind the preceding commits converted away. Debug
builds now time menu_displaylist_ctl end to end - the driver
list_push runs inside a declaration initializer, so the existing
entry point became a thin wrapper around the renamed internal -
and log a warning with the displaylist type and the measured time
when a build on a non-threaded queue exceeds 16 ms. A newly
added blocking walk, a cold-cache playlist parse, or a large
archive listing identifies itself in the log instead of surviving
as an anonymous stutter.
Threaded queues are exempt: their heavy lifting is off-thread by
construction, and the main-thread callback cost is already bounded
by the same window the tasks step under. Release builds compile
to the plain internal call.
Verified: full Linux build links (object mtime checked against the
source); strict declaration-placement gate clean both with and
without DEBUG defined.
---------
Co-authored-by: LibretroAdmin <[email protected]>
rvp9_decode_frame rejected any frame with seg_enabled set, returning
-4 before the compressed header was touched. That made a large class
of ordinary VP9 unplayable: libvpx turns segmentation on for every
adaptive-quantisation mode, so a clip encoded with aq-mode 1, 2 or 3 -
or by anything that enables cyclic refresh, which realtime and CBR
encodes do by default - failed outright. In RetroArch the failure was
silent: rwebm_video_decode_packet flattens the decoder's return to -1
and nothing is logged, so an affected .webm simply never produced a
thumbnail while other .webm files in the same directory did.
The uncompressed header already parsed the whole segmentation element;
nothing below it existed. This adds the rest:
- Feature table persistence. rvp9_parse_uncompressed wipes hd every
frame, but a frame with seg_update_data clear inherits the
previous frame's features, which is the common case: AQ signals
the table once per key frame and every inter frame between
inherits it. The surviving copy lives in rvp9_dec beside the loop
filter deltas, is replaced wholesale when the flag is set (libvpx
clears every feature before reading, so an absent feature is
absent rather than inherited), and is cleared by
rvp9_past_independence - which runs ahead of the frame's own
segmentation header, matching libvpx's ordering of
setup_past_independence before setup_segmentation.
- The segmentation map, two buffers swapped after each decoded frame
while segmentation is enabled. The intra and inter read paths
differ: an intra frame has no previous map to predict from, so an
absent seg_update_map means segment 0, while an inter frame
inherits the co-located id - the MINIMUM over the block's mi
footprint, not the top-left cell - and may code a bit saying "same
as last frame" whose context is how many of the above and left
neighbours took that shortcut. Either way the footprint is
written, because the next frame reads this one as its predictor.
- Per-segment dequantisation. The segment's qindex, absolute or as
a delta on base_qindex, replaces base_qindex as the origin the
y_dc/uv_dc/uv_ac deltas apply to; token decode indexes the pair by
the block's segment.
- Per-segment loop filter levels. lvl gains a segment dimension and
SEG_LVL_ALT_LF moves the segment's base level before the ref and
mode deltas apply on top.
- SEG_LVL_SKIP forces skip and ZEROMV and codes neither bit, and is
rejected on sub-8x8 blocks where the per-sub-block modes would
have nowhere to come from. SEG_LVL_REF_FRAME names the reference
outright, which also implies the intra/inter bit.
The segment id is read ahead of skip, so the block header bit order
follows libvpx. The helpers that touch no pixels live in rvp9.c
rather than rvp9_recon.inc, so the high-bit-depth instantiation shares
them instead of emitting a second copy.
Verified byte-exact against libvpx (raw I420 md5, ffmpeg -c:v
libvpx-vp9) on 83 conformance vectors and a 40-clip local corpus.
Feature coverage across that set: ALT_Q, SKIP (vp90-2-19-skip,
-skip-01, -skip-02, all three of which previously failed with -4),
temporally predicted maps, key-frame-only maps, inherited maps, tiles,
lossless, and non-8-aligned dimensions. vp90-2-09-aq2 and
vp90-2-15-segkey/-segkey_adpq pass. ALT_LF, REF_FRAME and absolute
(non-delta) feature data are implemented to spec but no vector or
libvpx encoder setting was found that emits them, so they remain
unexercised.
Unrelated failures that remain in the vector set, both predating this
change: the scaled-reference vectors still return -6 as documented,
and vp90-2-10/-20-big_superframe differ only in how many times a frame
is emitted for show_existing_frame - every decoded frame is
bit-identical and in order, and the repeat is the caller's output
policy rather than reconstruction.
Sanitizer sweep: ASan+UBSan+LSan clean and TSan clean over 126 streams
covering the full lifecycle - repeated open/decode/close, a single
decoder instance driven across streams of differing geometry to
exercise the map reallocation, an idempotent double rvp9_free, and a
zero-initialised decoder freed without decoding.
rvp9_past_independence only set frame_ctxs_init when it reset all four
frame contexts - key frame, error-resilient, or reset_frame_context 3.
reset_frame_context 2 resets just the context the frame header names,
and left the flag clear, so rvp9_decode_frame_impl rejected the frame
with -10 before decoding a single block.
That context is precisely the one the frame is about to load into
d->fc, so it is valid and the frame is decodable. libvpx draws no
distinction here: cm->frame_contexts is calloc'd, the reset writes the
named entry, and decoding proceeds. The other three entries stay
zeroed in both decoders, so a stream that referenced one without
resetting it would be equally malformed either way.
A stream may legitimately begin without a key frame - three invisible
intra-only frames that later show_existing_frame displays is a normal
alt-ref construction. The vp90-2-16-intra-only conformance vector
opens exactly that way and failed at its first packet.
Verified byte-exact against libvpx on vp90-2-16-intra-only.webm (was
-10, now 1064448 bytes matching) with no change to any other stream in
the corpus or vector set.
Sanitizer sweep: ASan+UBSan+LSan clean and TSan clean, including the
lifecycle case where the decoder is freed after a stream that never
carried a key frame.
rvp9_build_intra clamped its left-column and above-row extension to
d->yw/d->yh, the frame's crop dimensions. libvpx's
dec_build_intra_predictors clamps to xd->cur_buf->y_width and
y_height, which vp9_alloc_frame_buffer sets to the crop size rounded
UP to a multiple of 8.
The rows and columns between the crop size and that alignment are
coded and reconstructed like any other part of the frame. Clamping to
the crop size made an intra block in the last mi row (or column) of a
frame with a non-multiple-of-8 dimension replicate the last visible
line instead of reading the reconstructed samples above it, so the
block came out wrong and the error then spread through the loop filter
and into every frame that referenced it.
Only frames whose width or height is not a multiple of 8 are affected,
and only their final mi row/column, so the damage is a thin band at
the edge that the loop filter smears a few pixels inward - subtle
enough that it never showed up in the existing corpus, whose clips are
all 8-aligned in at least the dimension that mattered.
Found on a 400x710 VP9 clip: mi_rows is 89, so the last superblock row
holds a single mi row, and 116 of 720 frames differed from libvpx by
1-4 in the bottom two mi rows of the luma plane. A content-matched
re-encode at every width/height residue mod 8 (401x711, 402x713,
407x719, 398x706, 400x705, 393x710, 640x361, 641x360, 63x65, 33x17)
reproduces it on 6 of 20 clips before this change and on none after.
Verified byte-exact against libvpx (vpx via ffmpeg -c:v libvpx-vp9,
raw I420 md5) on the 64 vp90-2-02-size-WWxHH conformance vectors and
the vp90-2-03-size-* set, which sweep the alignment residues
systematically, plus the local corpus.
Sanitizer sweep: ASan+UBSan+LSan clean and TSan clean over 126 streams
including full open/decode/rewind/close lifecycles and geometry
changes on a shared decoder instance.
* playlist: extract playlist_path_ids_match from the entry matcher
playlist_path_matches_entry() interleaves two concerns: lazily
building the entry's cached path ID, and comparing two path IDs
under the playlist's matching rules (exact real-path equality with
the case-insensitive OS variant, then the fuzzy bare-archive vs
inside-archive equivalence on the parent archive path). The
comparison now stands alone as playlist_path_ids_match(), symmetric
in its arguments and operating on const IDs; the entry matcher
keeps the lazy cache initialisation and delegates.
No behaviour change; this is preparation for a dedup index that
must verify hash-bucket candidates with exactly these rules against
path IDs it owns itself rather than against live entries.
Verified: strict C89 syntax pass; full Linux build links; the
scanner oracle's plain run passes (its playlist parity lane goes
through this matcher for every fixture entry).
* playlist: split playlist_push; add playlist_push_unchecked
playlist_push() bundles three separable stages: the validation
preamble (core path check, path-ID construction, real core path
resolution, core name derivation), the linear scan for an existing
matching entry (update-in-place / bump-to-top semantics), and the
append tail (capacity policy, front insertion, field duplication).
The preamble and the append tail now live in playlist_push_prepare()
and playlist_push_new_entry(); playlist_push() keeps the scan and
delegates around it, with identical error messages, path-ID
ownership transfer and modification flagging.
The new public playlist_push_unchecked() composes the two helpers
without the scan, for callers that have already proven the content
path absent. The manual scan's END flush probes every result with
playlist_entry_exists() before pushing, then pays playlist_push()'s
own linear scan a second time for the same answer; with a dedup
index answering the first probe, the second scan is pure O(n) waste
per added entry, and this entry point removes it.
One latent hazard fixed in passing: playlist_push() read
entry->core_name in an initialiser before its NULL check of @entry;
the shared preamble guards the dereference.
Verified: strict C89 syntax pass; full Linux build links; scanner
oracle plain run passes (parity lane exercises playlist_push for
all 5000 fixture entries, m3u overwrite path included).
* playlist: add playlist_dedup, an O(1) content path existence index
The manual content scan's END flush answers "is this path already
in the playlist?" once per scan result via playlist_entry_exists(),
whose linear matcher scan makes the flush quadratic in playlist
size - 48% of total scan wall time under gprof for a 5000-entry
scan, and growing with the square of collection size.
playlist_dedup_t is an open-addressing hash index over content path
IDs. Correctness rests on a superset property of the matching
rules: any two IDs accepted by playlist_path_ids_match() share at
least one hash value between their {real, archive} pairs, because
an exact match implies equal real-path hashes and the fuzzy
bare-archive vs inside-archive match implies equal parent-archive
hashes (a bare archive's archive hash equals its real hash).
Storing every ID under both of its hashes and probing under both of
the query's hashes therefore yields a candidate superset, and every
candidate is verified with playlist_path_ids_match() itself - the
index can change only the cost of a query, never its answer.
The index owns all stored IDs, re-deriving them from entry path
strings during seeding instead of borrowing the entries' lazy
caches, so entry deletion (the scanner's m3u overwrite path calls
playlist_delete_by_path() mid-flush) cannot dangle it, and teardown
order relative to the playlist is free. Seeding is resumable with
a caller-supplied budget callback and a one-entry-per-call progress
floor, matching the budgeted iteration idiom of the scan's
directory walk. Any allocation failure permanently degrades the
index: queries fall back to the linear playlist_entry_exists() scan
and remain correct.
No callers yet; the scanner flush is wired in the next commit.
Verified: strict C89 syntax pass; full Linux build links; the
scanner oracle's differential lane (added two commits ahead)
asserts index answers equal linear-scan answers over a synthetic
corpus of plain, bare-archive, inside-archive and case-variant
paths with fuzzy matching both enabled and disabled.
* task_database: dedup the END flush with playlist_dedup
The flush paid two linear matcher scans per added result: the
playlist_entry_exists() probe, then playlist_push()'s own duplicate
scan reaching the same conclusion. Together they were 48% of scan
wall time under gprof and quadratic in collection size.
Each playlist the flush opens (or borrows, in single-playlist mode)
now gets a playlist_dedup_t. Pre-existing entries are indexed by
budgeted seeding on the shared per-frame window, resuming across
gathers exactly like the directory walk - flush_pos does not
advance while seeding, so a resumed gather re-enters the same
group. Existence is then answered by playlist_dedup_check_add()
with will_add, which records the path as present precisely when the
result is pushed: an absent path is always pushed, and the m3u
overwrite path (delete + re-push of the same path) keeps it
present. The push becomes playlist_push_unchecked(), absence
having been proven by the check. Index allocation failure at any
point falls back to the original linear probe and checked push.
The index is freed at group close, at flush completion, and in the
mid-flight teardown a cancelled task takes; it owns all of its
state, so ordering against the playlist frees is immaterial.
Scanner oracle, plain build: total 73-84ms for the 5000-file
fixture against ~230ms before this commit and a ~266ms blocking
reference - the scan is now ~3x faster than before the index and
~0.3x of blocking, with max gather CPU 7.6-11.7ms inside the 16ms
budget, first feedback 0.02ms, and the parity, threaded and
14-point cancellation lanes passing.
Sanitizer sweeps (ASan+UBSan+LSan, and TSan separately, both via
"./scan_begin_budget_test sanitize"): PASS with zero reports across
all lanes, covering group switches, cancellation mid-flush and full
teardown. Without the sanitize argument the wall-clock pacing
assertion trips under instrumentation on this tree and on pristine
master alike - the still-single-gather DAT parse inflates ~3.5x
under ASan past the 16ms budget - which is the documented rxml
residual addressed later in this series, at which point even
uninstrumented-style pacing should hold under sanitizers.
* samples/tasks/database: dedup lanes; tighten the throughput factor
Two lanes pin the playlist dedup index:
The differential lane builds a playlist of plain, bare-archive,
inside-archive and case-variant paths (path IDs are syntactic, so
none needs to exist) and asserts playlist_dedup_check_add() answers
exactly like the linear playlist_entry_exists() scan over a probe
corpus covering exact hits, misses, both fuzzy archive
equivalences, a case variant whose answer is OS-dependent - exactly
why the lane is differential - and the empty-path guard, with fuzzy
matching both off and on. An absolute anchor keeps the
differential from rotting into vacuity by asserting the fuzzy
equivalences themselves; it is build-aware, since the sample
compiles without RARCH_INTERNAL, where the matcher's config gate is
compiled out and fuzzy matching is unconditionally on. The lane
also pins the will_add contract: a recorded path is immediately
visible to the index, and playlist_push_unchecked() then brings the
playlist into agreement.
The no-overwrite rescan lane runs the full scan a second time into
the playlist the first pass wrote: all 5000 results must hit the
index against the pre-existing entries - driving the budgeted
seeding path at scale - and the playlist must still hold exactly
5000 entries afterwards.
THROUGHPUT_FACTOR drops from 1.60 to 0.80: with the index the
measured ratio against the blocking reference is ~0.35-0.41, and
0.80 keeps double headroom for runner noise while rejecting the
pre-index quadratic flush, which lands at ~1.1.
Verified with the CI step's literal command sequence: plain build
PASS (total 77.2ms, max gather CPU 8.1ms, all lanes), ASan+UBSan
+LSan sanitize-mode PASS with zero reports, TSan sanitize-mode PASS
with zero reports.
* rxml: incremental owned-buffer parsing
The manual content scanner reads its DAT file in budgeted chunks
but then parses the completed buffer with one
rxml_load_document_owned() call: a multi-hundred-MB MAME list still
parses inside a single task gather on a non-threaded queue,
freezing the frame for the duration. The scanner needs to buy the
parse a slice at a time.
The parser was already shaped for it: rxml_parse_document() is
fully iterative, with the cursor, accumulator, frame stack and open-
name budget all in struct rxml_parser, and exactly one program
point where no function-local state is live - the between-
constructs point at the head of the content loop, reached each time
an element opens or a child closes. That point now honours a stop
cursor: past it the parser records the suspension and returns a new
code, and the next invocation jumps straight back in. The one-shot
path sets the stop past the end of input, where the check can never
fire (the cursor never moves beyond the terminating NUL), so its
code path is one always-false comparison per construct.
rxml_parse_begin_owned() / rxml_parse_step() / rxml_parse_end() /
rxml_parse_abort() expose this: begin takes the buffer under the
same ownership contract as rxml_load_document_owned(), each step
consumes roughly max_bytes of input (bounded by max_bytes plus one
construct), and the terminal verdict applies the same
RXML_OPT_STRICT_EOF and RXML_OPT_LINES rules as the one-shot
wrapper, including the tolerated-EOF partial tree.
Verified: strict C89 syntax pass (including the resume jump under
-pedantic); differential harness compares stepped trees against
one-shot trees over documents covering nested elements, attributes,
entity and character references, comments, PIs, CDATA, deep
nesting, truncation at two points, an invalid close and a trivial
document, at step sizes 1, 2, 3, 7, 64, 4096 and run-to-verdict -
byte-identical dumps and verdict parity throughout - plus a 64-
point mid-parse abort sweep and an end-before-verdict discard, all
under ASan+UBSan+LSan with zero reports.
* logiqx_dat: incremental initialisation; hash-based name index
Two single-gather spikes remained in the scanner's DAT handling:
the one-shot rxml parse of the completed buffer, and the search
index built inside the first logiqx_dat_search() - two full child
walks plus a qsort, tens of milliseconds for MAME-sized lists,
whichever gather happens to issue the first search.
The index becomes an open-addressing hash keyed on the game name,
power-of-two capacity at a load factor of at most 1/2, with first
occurrence winning on insert so duplicate names keep the leftmost-
match answer of the linear scan. Exact lookup is all
logiqx_dat_search() ever asked of the sorted array, and unlike the
sorted array the build is a chunkable O(n) fill with no qsort
spike. The count, allocation and fill stages are extracted as
cursor-advancing chunk helpers; the lazy build on first search
composes them unbudgeted, and the allocation-failure fallback to
the authoritative linear walk is preserved.
logiqx_dat_parse_begin_owned() / _step() / _end() / _abort() layer
incremental initialisation over rxml's new incremental parser: the
XML phase forwards the byte budget to rxml_parse_step(), the count
and fill phases translate it into node visits (with a progress
floor), and the verdict applies exactly the validation
logiqx_dat_init_owned() performs, under the same buffer ownership
contract. A DAT built incrementally marks the index attempt
complete, so a later search neither rebuilds nor bypasses it.
Verified: strict C89 syntax pass; differential harness compares a
2000-game DAT (with every tenth name duplicated, an anonymous game
node and a non-game node) built incrementally at step budgets 1,
300, 4096, 65536 and unbudgeted against logiqx_dat_init_owned() -
2100 search queries answer identically at every budget, duplicates
resolve to first occurrence, the iteration API sees all 2001 game
nodes, and four invalid/truncated documents reach the same verdict
- plus a 40-point abort sweep and an end-before-done discard, all
under ASan+UBSan+LSan with zero reports. Scanner oracle plain run
passes unchanged through the reworked lazy index.
* task_database: budget the DAT parse and index build across gathers
MANUAL_SCAN_BEGIN_DAT_LOAD read the DAT in budgeted chunks and then
handed the completed buffer to logiqx_dat_init_owned() in one call:
the whole rxml parse in a single gather, with the search index
following in whichever later gather issued the first search. A
multi-hundred-MB MAME list froze the frame for the duration of each
on a non-threaded queue - the last single-gather work proportional
to input size anywhere in the scan.
The completed buffer now goes to logiqx_dat_parse_begin_owned(),
and the tick steps the parse and index build 256 KiB at a time
under the shared window - one step per gather as the floor, more
while the window lasts - resuming across gathers until the verdict.
The invalid-document error path and user-facing message are
unchanged, and a cancelled task aborts the in-flight parse in the
teardown alongside the stream and buffer it already released.
Scanner oracle, plain build (three runs): total 79.1-82.4ms, max
gather CPU 7.7-8.4ms (the DAT parse gather no longer contributes a
spike), all lanes passing. ASan+UBSan+LSan and TSan sweeps
(sanitize mode): PASS, zero reports, the 14-point cancellation
sweep covering the new mid-parse abort path.
For the record, instrumented builds still exceed the 16ms pacing
budget without the sanitize-mode skip, at ~29ms max gather: a
temporary probe attributes 16.8ms CPU of that gather to the final
playlist sort + write alone under ASan (~5ms plain), which is the
deliberately retained one-blocking-write-per-group parity noted in
the original split. The budget is a production-build contract; the
sanitize-mode treatment of it is unchanged.
* samples/tasks/database: big-DAT pacing lane
The 5000-game fixture parses in a couple of budgeted steps, which
cannot distinguish an incremental DAT parse from a single-gather
one. The new lane scans 50 files against a generated 300000-game
DAT (~40MB of XML, one streamed write to build) and asserts the
same 16ms pacing budget over that scan: before the incremental
parse this lane's whole document parse - and the search-index build
after it - each sat inside one gather, and any return to that fails
the assertion by an order of magnitude.
Correctness in every mode: exactly 50 playlist entries, with the
content files sampling every 6000th game so the labels span the
whole list - the final sample's label can only come from an index
that covered all 300k entries, so a build that silently stopped at
a prefix mislabels the tail and fails. Three additional cancel
points (32, 128, 512 gathers) land inside the budgeted read and
parse of the large document, phases the small fixture's sweep
passes through in single digits; the mid-parse abort path they
exercise is the one the task teardown gained with the incremental
parse. Under sanitizers the lane runs for memory coverage with the
wall-clock assertion skipped, like every other pacing check.
Lane cost: ~2s plain, ~3s ASan, ~7s TSan on a development machine,
well inside the CI step's timeouts.
Verified with the CI step's literal command sequence: plain PASS
(big-DAT pacing held), ASan+UBSan+LSan sanitize-mode PASS with zero
reports, TSan sanitize-mode PASS with zero reports.
---------
Co-authored-by: LibretroAdmin <[email protected]>
* rxml: add rxml_load_document_owned
A public owned-buffer entry point next to the copying string one:
the caller hands over a NUL-terminated heap buffer and the document
keeps it for its lifetime (the tree points into it), freeing it in
rxml_free_document - or immediately on a failed parse. This is
exactly the contract rxml_load_document's tail already had with its
own file read; exposing it lets a caller that read the document
bytes itself (e.g. a task handler filling a buffer under a time
budget across gathers) parse without doubling peak memory the way
rxml_load_document_string's mandatory copy does. DAT files reach
hundreds of megabytes, so the copy is not a rounding error there.
First consumer is logiqx_dat, which is losing its file I/O in a
following commit.
Verified: strict C89 syntax pass (-std=c89 -ansi -pedantic
-Werror=declaration-after-statement); full Linux build links; the
rxml sample parses; series tip swept under ASan+UBSan+LSan and TSan
separately via samples/tasks/database/scan_begin_budget_test
(threaded lane and a 14-point cancellation sweep included), clean.
* lists/dir_list: add a resumable directory iterator
dir_list_new() is a blocking recursive walk: whoever calls it pays
for the whole tree in that call, which is why the manual content
scanner freezes the frame that starts a scan of a large library
when Threaded Tasks is off. dir_list_iter_new()/_step()/_free()
is the same walk with the recursion made explicit - one frame per
open directory on a growable stack - so it can stop between any
two entries and resume on the next gather under a caller-supplied
budget callback, with a floor of one entry per step so a spent
budget still makes progress. Entries land in the caller's list as
they are read: it is the only walk, nothing is enumerated twice.
The enumeration is a transliteration of dir_list_read_ctx(), and
two behaviours of the recursion that read like accidents are
contract and preserved deliberately: the recursive call's return
value is ignored, so an unreadable subdirectory is skipped silently
while its owed post-order append still happens; and a directory
itself is appended (under include_dirs) after its contents, because
the append sits after the recursive call. The platform special
cases (System Volume Information, the IOS/OSX .framework-as-file
rule) are carried over verbatim. Equivalence with dir_list_new()
on the same tree - same entries, same order, pre-sort - is pinned
by samples/tasks/database/scan_begin_budget_test, whose playlist
parity lane fails on any drift.
Verified: strict C89 syntax pass; full Linux build links; series
tip swept under ASan+UBSan+LSan and TSan separately (iterator
lifecycle covered mid-walk by the harness's cancellation sweep,
which frees iterators at every phase), clean.
* tasks: extract the shared I/O window into task_nbio_slice.c
The per-frame window that bounds file-transfer work per gather -
at most 4 ms of budgeted work per 16.6 ms of wall time, shared by
every participating task rather than handed to each one, with a
floor of one work item per open so a spent window still makes
progress - moves from file-local statics in task_file_transfer.c
into a translation unit of its own, and its type plus the three
entry points (task_nbio_slice_open/close/within_budget) are
exported through tasks_internal.h.
Two reasons. The manual content scanner's BEGIN and END states
are about to become budgeted, and per the window's own design
rationale they must share this window, not invent a parallel
per-task budget that multiplies with the task count. And a TU
whose only dependencies are the clock and the task queue lets test
harnesses link the real window instead of a reimplementation -
samples/tasks/file_transfer already compiles the shipping transfer
TU and now links this one with it; the scanner oracle does the
same.
Behaviour is unchanged: same constants, same threaded-queue
opt-out (each task gets a whole slice on the worker thread, where
there is no frame to protect and a shared static would be a race),
same floor-then-deadline accounting. task_file_transfer_test
passes unmodified against the extracted TU, including its
budget-exhaustion and concurrency lanes.
Verified: strict C89 syntax pass on both TUs; full Linux build
links (Makefile.common + griffin wiring included); series tip
swept under ASan+UBSan+LSan and TSan separately, clean.
* task_database: fix empty-path strdup leak on scan handle teardown
content_database_path and playlist_directory are strdup'd
unconditionally when a manual content scan task is pushed, but the
teardown freed each only when it was non-empty - and strdup("") is
a real allocation. Any scan pushed while the content database
path setting is empty (every scan in a bare configuration, and
every scan in the new oracle's fixture) leaked both, 16 bytes per
push under glibc.
Found by LeakSanitizer under the cancellation sweep of
samples/tasks/database/scan_begin_budget_test, which pushes 16
scans per run: 16 identical 16-byte leaks, all from
task_push_manual_content_scan. The leak predates the scanner
rework this series performs; it lands first so the sweeps behind
the rest of the series run clean on their own merits.
Verified: strict C89 syntax pass; full Linux build links; series
tip swept under ASan+UBSan+LSan (leak detection on) and TSan
separately, clean - the sweep that found this reports zero leaks
with it applied.
* logiqx_dat: make the parser I/O-free
logiqx_dat.c performed two kinds of file I/O: logiqx_dat_path_is_valid()
stat'd the path for existence and size, and logiqx_dat_init(path)
read the whole document off disk (blocking, via rxml_load_document)
before parsing it. The second is why the manual content scanner's
BEGIN state could stall a frame for the duration of a DAT read -
DAT files reach hundreds of megabytes - with no way for the task
layer to budget it: the read was buried inside the parser.
The module now performs no I/O of any kind. Its API is:
logiqx_dat_extension_is_valid() - the extension whitelist (.dat /
.xml, case-insensitive), a pure string check. The whitelist is
logiqx domain knowledge, so it stays here; existence and size
checks belong to whoever owns the path.
logiqx_dat_init_owned(buf, len) - parses a document the caller
read, taking ownership of the buffer (heap, len + 1 bytes,
NUL-terminated) via rxml_load_document_owned: kept for the
document's lifetime on success, freed here on failure, and no
copy either way.
Call sites move accordingly. manual_content_scan.c gets a static
content_scan_dat_path_is_valid() with exactly the old validation
semantics (extension, path_is_valid, path_get_size > 0, optional
size out-param for the free-memory check). task_database.c
validates then reads the file itself and hands the buffer over -
still one blocking read in this commit, kept deliberately so this
change is purely a custody move; a following commit spreads that
read across gathers under the shared I/O window, which is the point
of the custody move.
Verified: strict C89 syntax pass on all four touched TUs; full
Linux build links (no other callers of the removed entry points
exist, griffin included); series tip swept under ASan+UBSan+LSan
and TSan separately - the DAT ownership transfer, including the
parse-failure path that frees the handed-over buffer, is exercised
by the scanner oracle's cancellation sweep - clean.
* logiqx_dat: index game names for logiqx_dat_search
logiqx_dat_search() walked every child of the root per call. The
manual content scanner calls it once per content file to resolve
the playlist label, so a DAT-backed scan was O(files x games):
gprof put logiqx_dat_is_game_node at 29% of a 5000-file scan's
wall time, and the product is quadratic into MAME-sized lists.
The search now builds, lazily on first use, an index of every game
node keyed by its 'name' attribute - pointers into the document's
own storage, so the index owns no strings - sorted by (name,
document position) and answered by a leftmost-equal binary search.
Sorting by position second means duplicate names resolve to the
first occurrence, which is the answer the linear scan gave; nodes
without a non-empty name are exactly the nodes the linear matcher
could never match, so omitting them changes no answer either. On
any allocation failure the index is marked failed and the linear
walk remains authoritative, and logiqx_dat_get_next()/reset()
iteration is untouched.
Cost: 24 bytes per game (~120 KB for the 5000-game fixture, a few
MB for a full MAME list), and the build-plus-qsort runs once inside
whichever gather issues the first search - milliseconds at fixture
scale; on a very large DAT under a non-threaded queue it is a
single one-off hitch far below the per-file scan it replaces, noted
in the scanner oracle alongside the parse-tick residual.
Measured on the oracle's fixture (5000 files, 5000-game DAT,
regular queue, release build): total scan 364.9 ms -> 183.6 ms,
which is 0.87x the blocking dir_list_new walk + whole-file DAT
parse reference - the incremental scanner is now faster end to end
than the code it replaced, not merely smoother.
Verified: strict C89 syntax pass; full Linux build links; oracle
passes all lanes including playlist parity; series tip swept under
ASan+UBSan+LSan and TSan separately, clean.
* database_info: add database_info_dir_init_from_list
database_info_dir_init() owns three things: the blocking recursive
dir_list_new() walk, the cue/gdi-prioritising sort, and the handle
construction. The scanner's BEGIN split needs the walk performed
incrementally by the task under a time budget, but the sort and the
handle belong with the database layer - the comparator
(dir_entry_compare) is static here for a reason.
database_info_dir_init_from_list() is the second and third pieces
over a content list the caller already built: same sort, same
handle fields, borrows the list without taking ownership, NULL only
on allocation failure. The directory variant remains for callers
that want the one-call blocking form.
Verified: strict C89 syntax pass; full Linux build links; series
tip swept under ASan+UBSan+LSan and TSan separately, clean.
* task_database: split MANUAL_SCAN_BEGIN into budgeted sub-states
MANUAL_SCAN_BEGIN performed the entire recursive directory walk
(dir_list_new via database_info_dir_init or the blocking content
list getter), the whole DAT file load and the playlist setup in a
single handler invocation. Under the regular task queue that
invocation runs on the frame thread, so starting a scan of a large
library froze the frame for the duration - hundreds of
milliseconds at 5000 files, unbounded above - and the freeze is
exactly what Threaded Tasks off is supposed to avoid paying.
BEGIN is now a sequence: BEGIN (setup and branch selection) ->
BEGIN_DIR_LIST (the walk, via the resumable dir_list iterator) ->
BEGIN_DAT_LOAD (a chunked 256 KiB-per-read fill of a single
buffer, then one owned-buffer parse) -> BEGIN_PLAYLIST (playlist
open, parameter recording, and the final status selection, moved
verbatim). Every sub-state runs under the shared per-frame I/O
window (task_nbio_slice_*, the same window the file-transfer spine
uses - deliberately shared rather than a parallel per-task budget,
which would multiply with the task count), each with a floor of
one work item so a spent window still progresses. Completed
sub-states collapse into one invocation while the window lasts, so
a small library finishes the whole BEGIN sequence in a single tick
- the latency it had before the split. The task's public identity
(handler, state) is unchanged, so duplicate-scan suppression via
task_queue_find() is unaffected.
The same total work is performed: the walk is incremental rather
than done twice, the DAT is read into one buffer whose ownership
passes to logiqx_dat_init_owned, and the DB branch's rdb-list
setup moves verbatim into the setup sub-state. The non-DB
branch's content list creation moves to
manual_content_scan_content_list_iter_new(), which shares its
extension/compressed-inclusion policy with the blocking getter
through one helper so the two cannot drift; the caller owes the
completed list the same empty-reject and dir_list_sort the getter
applied. Error semantics match the old code path for path: same
messages, same terminal exits, including dir walk allocation
failure mirroring dir_list_new() returning NULL.
Cancellation mid-walk or mid-read is now possible, so the handle
teardown releases the in-flight resources: the directory iterator
(which closes any open directory handles), the DAT stream, and the
read buffer when its ownership has not yet transferred.
Residual, documented in the oracle: the rxml parse of a completed
DAT buffer and the (following commit's) search-index build remain
one operation inside a single gather - budgeting the read bounds
the dominant I/O latency, but a pathological multi-hundred-MB DAT
still parses in one tick on a non-threaded queue. rxml is not
resumable today; the sub-state structure leaves the seam for it.
Measured on the scanner oracle's fixture (5000 files across 50
dirs, 5000-game DAT, regular queue, release build): worst gather
attributable to the scanner's begin path drops from the full
walk+load+setup cost to inside the 4 ms window plus slack; first
task feedback 0.01 ms after push; playlist parity exact.
Verified: strict C89 syntax pass (HAVE_LIBRETRODB both ways); full
Linux build links; database_scan_test links and passes against
this commit standalone; series tip swept under ASan+UBSan+LSan and
TSan separately, including a 14-point cancellation sweep landing
in setup, mid-walk, mid-DAT-read, iteration and flush, clean.
* task_database: make the MANUAL_SCAN_END flush resumable
With BEGIN split, the largest single freeze left in a scan was the
gather that reached MANUAL_SCAN_END: it added every accumulated
scan result to the playlist - each add a linear
playlist_entry_exists() probe over the growing entry array, so the
batch is quadratic - then sorted and wrote the file, all in one
handler invocation. Measured at 166 ms for the 5000-file fixture
on a release build; gprof attributes 48% of the whole scan's wall
time to playlist_path_matches_entry alone.
scan_results_batch_update_playlists() becomes
manual_scan_end_flush_tick(): the same loop with its locals -
position, the open playlist for the group being written, the group
key, the added counter, the path scratch buffer - promoted to the
scan handle, yielding on the shared per-frame window between any
two results with a floor of one result per gather. The
MANUAL_SCAN_END case drives it until completion before running its
terminal message block. Group-close writes for multi-playlist
scans and the final sort+write remain single blocking operations:
a playlist file write is atomic by design, and a partial write
would corrupt the playlist on a cancelled scan.
One behavioural fix rides along, forced by the loop conversion:
the old for loop's continue on a playlist that failed to open
advanced the index implicitly; the while loop must advance the
position itself, or the same unopenable playlist would be retried
every gather forever.
Cancellation mid-flush is now possible, so the handle teardown
closes the flush's open playlist - before manual_scan->playlist is
released, because in single-playlist mode flush_playlist borrows
that handle and the identity comparison preventing a double free
needs the pointer still live to say so - and frees the scratch.
Measured on the oracle's fixture (regular queue, release build):
worst gather over the whole scan 166 ms -> 5.6 ms, inside the 4 ms
window plus bounded slack; total scan time unchanged by this
commit (the quadratic probe cost still runs, spread out - its
removal is the playlist-side dedup index left as follow-up, with
the gprof profile attached to the oracle).
Verified: strict C89 syntax pass; full Linux build links; series
tip swept under ASan+UBSan+LSan and TSan separately, with
mid-flush cancellation covered by the sweep's tail points, clean.
* samples/tasks/database: add scan_begin_budget_test
Pacing + throughput oracle for the manual content scanner's
BEGIN/END split, compiled from the tree so it exercises the
shipping translation units. It builds its own fixture - 5000
content files across 50 subdirectories plus a Logiqx DAT naming
each - so it needs nothing but a writable directory (honours
TMPDIR).
Three wall-clock assertions guard the plain build, because any one
alone can be satisfied by a scanner that is worse: pacing (no
task_queue_check() attributable to the scanner over the shared
4 ms window times a 4x slack multiplier - the pre-split tree fails
this at >160 ms), first feedback within two frames of the push,
and total scan time within 1.60x of the blocking walk + DAT parse
+ 40 us/entry floor it replaced, so a scanner that no longer
freezes but takes twice as long fails. Playlist parity - every
fixture file present exactly once - is asserted in every mode, and
is also the equivalence pin for the dir_list iterator against
dir_list_new(). Measured at the series tip (release, regular
queue): total 183.6 ms = 0.87x reference, max gather 5.6 ms, first
feedback 0.01 ms.
A 'sanitize' mode runs every lane but skips the wall-clock
assertions a sanitized build cannot honestly meet; the sanitizers
are the assertion there. Its lanes are where the split's
in-flight resources are made to die at every phase: a threaded
lane runs the same scan under the threaded queue (the TSan target
- the shared window deliberately opts threaded queues out of its
static state, and a mistake in that opt-out surfaces here), and a
14-point cancellation sweep lands a cancel in setup, mid-walk,
mid-DAT-read, content iteration and mid-flush, with LeakSanitizer
deciding whether teardown released everything. The sweep already
paid for itself: it found the pre-existing empty-path strdup leak
fixed earlier in this series.
A 'bench' mode prints the metrics and skips assertions, for
regression-guard sampling.
The CI workflow gains the corresponding step (plain assertions,
then ASan+UBSan and TSan in sanitize mode) and its C89
syntax-check list gains the translation units this series touched.
Known residuals, deliberate and documented in the source: the rxml
parse of a completed DAT buffer and the logiqx index build each
run inside a single gather; playlist-side entry dedup remains
quadratic (48% of scan wall time per gprof) with the flush now
spreading rather than removing it - the fix is a path-hash dedup
index in playlist.c verifying candidates with the existing fuzzy
matcher, left as follow-up.
* samples/tasks/image_decode_slice: link task_nbio_slice
The window extraction moved task_nbio_slice_open/close/
within_budget out of task_file_transfer.c into their own
translation unit, and updated the two sample Makefiles built
during that work - file_transfer and database. This oracle also
compiles the transfer spine (for the fill path its decode window
budgets), so it now fails to link on the three slice symbols,
which is exactly how CI reported it.
Add the slice TU to its source list. No flag or include changes
are needed: the slice TU's includes are a subset of the transfer
TU's, and its external calls resolve against what the sample
already links - task_queue_is_threaded from task_queue.o, and
cpu_features_get_time_usec from the sample's own fake clock. The
latter is not an accident to paper over but the point: the
oracle's budgets are deterministic because the clock is the test's
instrument, and the extracted window now runs against that same
instrument, as the in-tree window did before the extraction.
A repo-wide sweep confirms these were the only three sample
Makefiles compiling the transfer TU, so no further consumer is
left to fail one CI job at a time.
Verified: image_decode_slice_test builds and passes in both its CI
lanes against this Makefile - plain and ASan+UBSan (leak detection
and stack-use-after-return on).
* logiqx_dat: make the parser fully path-agnostic
logiqx_dat_extension_is_valid() performed no I/O - it was a pure
string check - but it was still the module reasoning about file
paths, and it forced logiqx_dat.c to keep including file/file_path.h
for path_get_extension(). With the file system checks already moved
to the callers, the .dat/.xml whitelist was the last piece of path
knowledge left in a parser whose contract is bytes in, structs out.
The whitelist moves into manual_content_scan.c, folded into the
validator introduced when the parser lost its I/O, which is now
exported as manual_content_scan_dat_path_is_valid(): extension,
existence, size > 0, optional size out-param. Both menu-side call
sites use it, and task_database.c's budgeted DAT loader validates
through it instead of composing the three checks itself - so the
task can no longer drift from the acceptance rules the menu applied
when the path was chosen, which is a small correctness win on top
of the layering one. logiqx_dat.c drops the function and the
file_path.h include; its remaining includes serve the parse alone.
Verified: strict C89 syntax pass (-std=c89 -ansi -pedantic
-Werror=declaration-after-statement) on all three touched TUs; full
Linux build links; scan_begin_budget_test passes all lanes
(playlist parity, threaded, 14-point cancellation sweep) and was
swept under ASan+UBSan+LSan and TSan separately, clean; repo-wide
grep confirms no caller of the removed entry point remains.
* samples/tasks/database: build the budget oracle from all; pace on CPU time
The new CI step follows the workflow's house pattern - `make clean
all`, then probe for the binary, then run it - but `all` in this
Makefile only built database_task, so the step's silent
`test -x scan_begin_budget_test` failed under set -eu right after
a successful database_task link, which is exactly the shape of the
CI log: link echo, no error text, exit 1. It passed locally only
because a stale binary from an earlier explicit
`make scan_begin_budget_test` was still sitting in the directory.
The oracle joins the all target, so every build invocation in the
step - plain and both sanitizer configs - produces it, and plain
`make` in this directory builds everything runnable.
While correcting the step, the pacing lane switches its per-gather
measurement from wall time to thread CPU time
(CLOCK_THREAD_CPUTIME_ID, falling back to the wall clock where
unavailable). The property under test is how much work the
scanner chooses to do in one gather; on a shared CI runner a cheap
gather can be preempted mid-flight and charged tens of wall-clock
milliseconds it never spent, failing the lane spuriously, while
the regression the lane exists for - the pre-split tree doing the
whole walk, DAT load and flush in one invocation - burns its
>160 ms as CPU and is caught by either clock. Wall time remains
the clock for the total/throughput and first-feedback lanes, where
elapsed time is the property.
Verified with the step's literal command sequence in all three
build configurations: plain (`make clean all`, test -x, full
assertion run - PASS, max gather 8.2 ms CPU), ASan+UBSan+LSan
sanitize mode, and TSan sanitize mode, all clean.
* comments: describe the code as it is, not as it was
Review feedback on the scanner series: comments narrating how the
code previously behaved add nothing to reading it now - that story
lives in the commit messages. Rewrite the affected comments in
present tense (the BEGIN-family and flush headers, the enum and
handle-field notes, the logiqx index rationale, the DAT-path
validator, the shared-window deadline note), keeping only what
states a live contract: the parity targets against the still-
present blocking entry points, the atomic-playlist-write floor,
the teardown ordering constraint, and the advance-before-continue
invariant in the flush loop. The leak-fix guards lose their
comments entirely - a NULL-checked free needs no explanation - and
the database sample Makefile's comments go with them.
The regression oracle and the CI workflow keep their historical
descriptions: there, the previous behaviour is the documented
subject.
Comment-only change apart from the Makefile whitespace; verified
no object-code effect is possible, strict C89 pass on all touched
TUs, and the oracle's full plain run passes.
---------
Co-authored-by: LibretroAdmin <[email protected]>
sonninnos reports thumbnails coming out blurrier with the upscaler on
than off, on Amiga save state screenshots and on downloaded boxart.
That is exactly what it does, and the reasoning in the comment that
introduced it was wrong.
The argument was that duplication does not improve the picture - at 4x
it leaves 75% of pixels identical to their left neighbour and steps
edges 4 levels at once, where interpolation steps 1. True of the
texture in isolation, and irrelevant, because nobody sees the texture.
The menu hands it to the GPU, which samples it bilinearly into a
thumbnail box that is a different size again. Softening it here means
softening it twice, and interpolating twice is worse than letting the
GPU interpolate once - so the upscaler was not merely failing to
sharpen, it was making thumbnails worse than leaving it off.
Measured through the real scaler, then through a 2x2 bilinear sample
into the thumbnail box, mean gradient magnitude against the upscaler
off:
360x256 Amiga state, 480px box bilinear -0.5% point x4 +21.6%
360x256 Amiga state, 960px box bilinear -3.9% point x4 +11.9%
360x256 Amiga state, 1600px box bilinear -5.2% point x4 +8.8%
720x256 Amiga state, 960px box bilinear -2.2% point x2 +3.3%
350x240 PS1 shot, 960px box bilinear -0.9% point x4 +2.5%
Every case negative, the wide one and the photographic one included.
Duplication survives the GPU's pass instead of compounding with it:
the hard steps land on texel boundaries the bilinear sample softens by
one output pixel, which is the crisp look the setting exists for. It
is also the only faithful option for 10-bit, where point sampling
copies the packed words untouched instead of filtering them.
This restores the two settings to being meaningfully different: off is
smooth, on is crisp. Before, on was a slightly worse version of off.
Point sampling needs whole-number runs, so the caller now takes the
largest whole factor that stays under the downscale cap rather than
aiming exactly at the cap. A 720x256 screenshot at a 1080p cap goes
to 1440x512 rather than 1920x682 - fewer pixels, but even runs, and it
still never trips the sinc downscale that the cap fix removed. Cost
is unchanged, ~1.7 ms either way.
ASan+UBSan+LSan clean. C89 clean.
SIGSEGV inside raac_decode_frame, previewing a trailing-moov MP4 from
the file browser (mingw-w64, non-threaded audio: the whole stack on
the main thread through audio_driver_menu_sample). Reproduced on
Linux/ASan frame for frame: the mixer's AAC decode read access units
off pages the preview window never committed. Five defects, one
family - the resident bound that windowed audio decode rides on:
- audio_transfer_set_avail compiled to a no-op without HAVE_RWEBM: the
whole body sat behind a HAVE_RWEBM gate, so on an RWEBM-less build
every bound an M4A voice was given - including the one that keeps
rmp4's open off unpopulated pages - was accepted by the caller's
guard (audio_mixer_play_stream admits RAAC+RMP4 alone, and its
comment says why) and silently dropped by the callee. The gate is
now the union of the arms that honour a bound, the demuxer pokes
inside carry their own guards, and the FLAC case - absent from the
switch entirely, so the mixer's FLAC lane fell through to a break -
exists. The Vorbis and Opus contexts kept avail (and now the wall
flag) inside their HAVE_RWEBM block, which would have made the
unconditional stores a compile error on RWEBM-less builds; those
fields are unconditional now.
- The trailing-moov path had no wall at all. The open needs
avail == the file length so the box walk can reach the committed
moov island, and it was left there: over the packets, safety rested
on the feeder always being ahead, which a driver prefill or a fat
video-interleave stride outruns - and the budgeted feeder made that
practical. gfx_thumbnail now clamps the bound down to the resident
head the moment add_stream returns, and the per-tick feeder follows
the feed frontier in both directions, because after a loop's rewind
the frontier drops back to the head and a bound left at its old
height admits reads of the pages the rewind decommitted.
- rmp4_set_avail and rwebm_set_avail refused to lower ("bytes never
un-arrive"), as did the rmp4_video/rwebm_video mirrors above them.
That is a growing-buffer model; a windowed caller's bytes do
un-arrive, and the refusal silently discarded both the post-open
clamp and every post-lap drop. All four are exact stores now.
- An empty read at the wall returned AUDIO_PROCESS_END, which a
looping mixer voice answers with a mid-file rewind and a second
stall with releasing the voice - so fixing the wall alone would
have traded the crash for the preview audio dying a second in.
The windowed arms (AAC over rmp4, ADTS, and Matroska; Vorbis, Opus
and FLAC over Matroska, where the AAC and FLAC pulls conflated
RWEBM_READ_AGAIN with end of stream) now note the wall when they
stop at it, audio_transfer_read_* reports the resulting empty read
as AUDIO_PROCESS_NEXT - starved, retry - and both mix_stream
variants contribute silence for that call instead of rewinding.
The read contract in audio.h says so.
- rmp4_consumed reported its monotonic high-water across rmp4_rewind,
so after the first true lap the feeder's byte tell froze at the
file end: the window stayed parked on the tail while the rewound
decoder walked the head into pages the window had decommitted
behind itself, and playback degenerated into a stutter-loop of
whatever stayed resident. Audio and video both - rmp4_consumed is
the byte tell under each. rmp4_rewind now resets the frontier to
the merged next sample offset, the same merge rmp4_read_packet
does.
The new preview_audio_decode_test drives the REAL chain - audio_mixer
-> audio_transfer -> rmp4 -> raac - over a budget-fed window, at
budgets small enough that the decoder reaches the frontier, with the
feeder modelling the fixed gfx discipline. The sibling harness in
the same directory stubs the audio driver edge, so the mixer's decode
path never ran under it: for the third time in this sample's history,
the defect lived inside the layer a harness had modelled. The
_norwebm build of the same test is the RWEBM-less lane that catches
the set_avail gate. Both builds crash-kill on the unfixed tree
(SEGV in raac_getbits), and the lap assertions - the voice is never
FINISHED, rewinds only at true laps, the byte tell returns to the
head after each - kill the stall-conflation and frozen-tell defects
separately. Wired into the existing preview CI step, ASan and TSan.
Both window feeders in gfx_thumbnail.c run on the main thread, and
data_transfer_window_extend performs its read synchronously, so the
first animate() tick after a windowed open - and the first tick after
every lap of a looping video - read the whole lookahead in a single
filestream_read. Measured at the video feeder's constants (4 MiB
head, 8 MiB lookahead): 4.26 MiB in one call on the first tick, up to
the full 8 MiB depending on where the anchor lands, repeated at every
loop of every looping preview, forever. On fast storage that is
single-digit milliseconds and invisible; at 20 MB/s it is a ~210 ms
frame stall per burst, which is the hitch this bounds.
data_transfer grows data_transfer_window_feed_budget(): the same
policy as window_feed - same rewind detection, same advance, same
tell + lookahead target - but while the frontier covers the consumer,
one call commits and reads at most 'budget' new bytes and later calls
carry the frontier the rest of the way. window_feed becomes the
budget == 0 case. Two decisions in the design are the point:
- The budget is NOT applied while the frontier is behind the
consumer (a lap landing past the head, a rebased open). There the
bytes at tell itself are what is missing, and pacing THAT read
over ticks does not shrink a burst, it leaves the consumer's very
next read on unresident pages: that catch-up stays one unbudgeted
extend, exactly what window_feed has always done. Nothing about
which bytes become resident changes for any layout - leading moov,
trailing moov, media past the head - only how many ticks the
read-ahead portion is spread across.
- The call reports the frontier it left resident through an
out-parameter, and the feeders raise their consumers' avail to
THAT, never to the lookahead target: under the budget the target
can lag, and avail past the frontier is a read of reserved pages.
The single-owner contract's refusal to expose the frontier as an
accessor stands - this value is feed-owner state, handed to the
one owner inside the one call the contract already gives it.
gfx_thumbnail.c feeds with 512 KiB per tick for video and 256 KiB for
audio. Those are rates, not tastes: at 60 ticks/s they sustain
30 MB/s and 15 MB/s of catch-up, an order of magnitude above anything
a thumbnail previews (a 100 Mbps master needs 12.5 MB/s), so the
feeder gains on the decoder every tick it is behind and the steady
state - where each tick reads only what the last tick consumed - is
untouched. Measured: the worst single-call read drops from 4.26 MiB
to exactly the 512 KiB budget over the same simulated playback.
Separately, the same file's other main-thread read: deciding whether
a PNG animates. gfx_thumbnail_anim_open answered it by opening the
file and reading a 4 KiB prefix - one open+read+close on the main
thread for every PNG the menu highlighted, moments after the still
task had read the entire file into memory on the worker. The task
now answers from the bytes it holds: task_image_png_probe() runs
rpng_is_apng_ex over the completed transfer's buffer (a chunk walk,
microseconds, and over the WHOLE file, so it is at least as
conclusive as the prefix probe it replaces) and hands the verdict to
gfx_thumbnail_anim_open_probed(). Only a completed read is a
verdict - a partial buffer can only say "not yet", and "not yet"
mistaken for "still" would silence an APNG - so anything short,
detached, or not a PNG image task reports unknown, and unknown falls
back to the file probe exactly as before. The historical 2-argument
gfx_thumbnail_anim_open remains (the anim oracle globalizes and
drives that symbol directly), as does the probe inside it, for every
caller with no task verdict to offer.
What this deliberately does not touch: the open-time head read, the
trailing-moov prime, and every admission cap are exactly as they
were. The multi-gigabyte preview paths - leading and trailing moov
both - are load-bearing and verified below; bounding them at open is
a separate question with history behind it.
Verification, all on this tree:
- samples/formats/data_transfer: four new checks in the window test
pin the budget contract - per-call read bound, the reported bound
never behind the consumer with byte-content verification against
the file, convergence on the unbudgeted target (and budget == 0
identical to window_feed), and the catch-up exemption closing a
behind-the-consumer frontier in one call. Kill-verified against
two mutants: applying the budget during catch-up fails the fourth
check (bound at 131072 with the consumer at 6291456), and a 64x
over-read fails the first (196608 read against a 32768 budget).
The first kill attempt ran a stale binary from a failed relink and
"passed"; caught, rebuilt clean, re-run.
- Full sanitizer sweep, every suite touching a modified TU:
ASan+UBSan with LeakSanitizer strict (detect_leaks=1), a
standalone -fsanitize=leak lane, TSan, and plain builds:
data_transfer window/prefix/source/pool-scrub tests;
audio_transfer_window_test; gfx_thumbnail_anim_test (3 distinct
frames in 8 paced vsyncs, ASan and TSan); the full
gfx_thumbnail_preview_test fixture matrix including both 7.28 GB
layouts - no feeder stalls, the mixer receives the whole
container, RSS stays inside the window budget on every layout,
under ASan and TSan; task_file_transfer_test including the
32-thread concurrent lane (ASan and TSan); and
image_decode_slice_test, which links the modified task_image.c,
plain and ASan. The one red lane is pre-existing: the window
test's /proc-based residency bound trips under TSan's shadow
memory on the unmodified tree identically, and CI runs that suite
under ASan+UBSan only.
- Full RetroArch build links clean; gfx_thumbnail.c and
task_image.c also compile with the menu and widget defines
stripped, matching the headless lane.
Measured on one core: the ASan pass ran 128 s and the TSan pass 210 s,
~5.6 min for this step alone, against a job whose timeout-minutes is
10 and which already carries a dozen other steps. It would have gone
red on time rather than on a defect - and I only found that by running
the step end to end instead of its pieces, which is what I should have
done before landing it.
120 ASan frames and 60 TSan frames bring it to ~1.5 min with every
assertion intact: the feeder advances at AUDIO_BYTES_PER_TICK rather
than per frame, so the window boundaries the audio checks rest on are
still crossed, and TSan's value is instrumented access pairs rather
than iteration count. Job timeout raised to 20 min for headroom on
slower runners.
Verified from a clean checkout of master by running the step verbatim,
both halves: PASS with 0 failures on all four layouts.
The CI step ran the sanitizer passes as
make clean all SANITIZER=address,undefined; make fixtures; run
make clean all SANITIZER=thread; run
and clean did `rm -rf fixtures`, so the second run globbed nothing,
passed the literal "fixtures/*.mp4" through as a path, and reported
status=3 alpha=0.000 tex=0 uploads=0
R1 FAIL R2 FAIL R3 FAIL R4 ok R5 ok
A6 FAIL A7 ok A8 ok
Status 3 is MISSING - the thumbnail was never requested, because the
path does not exist. Worse than the false red is that A7 and A8 went
GREEN on it: file_len() of a missing file is 0, so "the mixer got the
whole container" compared 0 against 0 and passed, and a feeder that
never ran recorded no stalls. A test that half-passes on a file it
could not open is the same defect class it exists to catch.
Two fixes. clean keeps fixtures/ - the passes rebuild with different
SANITIZER= values and each needs them - with distclean to drop them,
and the CI step regenerates before the TSan run anyway so it cannot
depend on that. And the test now checks the fixture is readable
before anything else, failing with a single unambiguous F0 rather than
a spread of results that invites reading the tea leaves.
Verified: the exact broken invocation now yields "F0 fixture readable
FAIL" and nothing else; fixtures survive a clean+rebuild under a
different sanitizer; ASan+UBSan+LSan and TSan both green on all four
layouts.
The windowed video open primes the window at the media floor and
raises the demuxer's avail to match, but only when the open had to
jump for a trailing moov. A LEADING moov small enough to sit inside
the 4 MiB resident head opens with no jump at all:
stream OPEN after 0 jump(s)
MEDIA FLOOR = 7816986202
so avail stayed at the head while the first sample sat 7.8 GB away.
The stream installed, the first decode found nothing, the rewind retry
found nothing, and animate closed the animation - texture never
uploaded, status stuck at PENDING, no preview. Same standstill as the
leading-moov audio bug, reached from the other side.
What decides whether the prime is needed is where the media starts,
not how the open got there. Run it unconditionally: rebase declines
when the frontier already covers the floor, so a file whose media
follows its moov immediately is unaffected, and set_avail is monotonic.
This is the red case the gfx_thumbnail_preview sample landed with. It
now passes on all four layouts - leading moov with media past 4 GiB,
trailing moov at 7.28 GB, trailing moov at 12 MB and at 148 KB - under
ASan+UBSan+LSan and under TSan.
Every fault in the file-browser video preview shipped because the
tests around it measured the wrong layer. A decode-level harness
proved rmp4 returned frames while the thumbnail drew at zero opacity;
a thumbnail-level harness stubbed the mixer and so modelled the two
functions three of the mixer defects lived in. Each fix was verified
against the layer below the one that was broken.
This links the REAL gfx/gfx_thumbnail.c and the REAL
libretro-common/audio/audio_mixer.c, and asserts what the menu and the
audio thread actually need:
R1 a texture is uploaded
R2 status reaches AVAILABLE
R3 alpha is non-zero, or a fade was pushed to raise it
R4 no whole-file still load is queued for a video
R5 RSS stays bounded, independent of file length
A6 preview audio starts
A7 the mixer is handed the whole container length
A8 the feeder keeps the decoder fed, no stalls
R3 is the alpha bug; A6/A7 are the trailing-moov silence and the
int32_t truncation; A8 is the dropped set_avail raises. R1/R2/R4/R5
all passed while R3 failed, which is precisely why the invisible-
preview bug survived a green sweep.
The stubbed audio thread consumes at a settable rate and touches every
byte it is told is resident, so an over-claimed avail faults under
ASan instead of passing quietly. Stalls are counted only below the
file end, where hitting the wall means a starved feeder rather than
the loop point.
The fixtures are container LAYOUTS rather than clips, because layout
is what every one of these faults turned on: leading moov, trailing
moov, moov larger than the resident head, media starting past 4 GiB,
co64 chunk offsets. mkfixtures.py builds them from a three-second
seed by box surgery, sparse, so two 7.28 GB files occupy ~400 KB.
Known red on first landing: leading_huge fails R1/R2/R3. Preview
audio is correct on all four layouts, but the VIDEO path does not
render on a leading-moov file whose media begins past 4 GiB. That is
a real defect this fixture set found immediately and the ad-hoc one it
replaces did not - left visible rather than papered over, so the fix
lands against a test that already proves it.
Two failures, one root cause: the head was being asked to do a job a
head cannot do.
A LEADING moov was covered by sizing the head to the moov's end - but
the first sample begins just past that, sixteen bytes past it on a
real file, so the bound landed below the media floor. The decoder
could not reach packet one, and the feeder anchors at tell==0, so
nothing ever raised the bound: silence from a head that looked
generous. The head must clear the floor as well as the moov.
A TRAILING moov cannot be reached by a head at all without holding the
file, which is why it was previously refused past a ceiling. Commit
the moov's island instead and open against the full length; from there
the feeder keeps the committed band straddling the decoder, which is
the contract the video path has always run under for this layout.
Residency is the head plus the island plus the slide - a few MiB
whatever the file weighs - so the ceiling is gone with it.
The head is still capped. A file whose media starts far past its moov
would otherwise drag the permanently resident region out to the first
sample; there the head stays on the metadata and the island bound
carries the decoder out to the media.
Every layout in the fixture set now plays, with no size limit
anywhere: leading moov at 7.28 GB, trailing moov at 10 MB, 41 MB,
170 MB and 7.28 GB, co64 with offsets past 4 GiB, a moov larger than
the resident head, and media starting 7.65 GB into the file. The two
that previously declined - 41 MB and 170 MB trailing - now stream.
Preview audio was silent on every trailing-moov MP4. The head was
sized from image_transfer_anim_stream_media_floor, but that is the
first sample's offset: it sits just past a LEADING moov and is 44 on a
file whose moov is at the far end. The decoder's open takes a PREFIX
bound, so it can only reach the moov if the head reaches it too, and a
2 MiB head never does. Probed on the reported file:
@0 ftyp 28
@36 mdat 29423357
@29423393 moov 112244 <- trailing
MEDIA FLOOR = 44 -> head = 2 MiB
avail=2097152 start=FAILED
avail=29535637 start=OK ch=2 rate=48000
Locate the moov instead, by walking top-level box headers through
window_peek - a handful of 16-byte reads whatever the file's size -
and grow the head to cover it. A leading moov usually needs LESS than
the floor-derived figure, so those files get a smaller head too.
Bounded by GFX_THUMB_AUDIO_HEAD_MAX. A trailing moov past that would
mean holding most of the file resident, which is the cost the window
exists to avoid; such a file now declines audio explicitly rather than
handing the mixer a stream whose decoder cannot open. That is the
same outcome it had before this commit, reached deliberately.
Verified across the fixture set: leading moov at 7.28 GB and trailing
moov at 10 MB both play; trailing moov at 41 MB and 170 MB decline.
The reported file is a 29.5 MB trailing moov and falls inside the
ceiling.
b0099a4 moved the audio_mixer_sound_set_avail call in add_stream out
from under #ifdef HAVE_ROPUS - correctly, since the resident bound is
not Opus-specific - but left the DEFINITION behind that same guard.
Every target built without Opus then compiled cleanly and failed to
link:
audio_driver.c:(.text.audio_driver_mixer_add_stream+0x3ec):
undefined reference to `audio_mixer_sound_set_avail'
make: *** [retroarch_3ds.elf] Error 1
The sound's 'avail' field is unconditional, so the body needs no
guard either. Only audio_mixer_sound_set_end_granule stays under
HAVE_ROPUS, which matches its call site - that one is still guarded.
Checked by linking a translation unit that calls every symbol
add_stream calls unconditionally, with no codec defines at all: it
reproduces the 3DS error exactly on the parent commit and links clean
here. audio_driver.c and audio_mixer.c both compile at C89 across
the empty config, RWAV, RAAC, ROPUS, RWEBM+RVORBIS, RAAC+RMP4 and the
full set.
ef19c2f handed the mixer a windowed container. Four things in the
mixer could not take one, and between them they produced a preview
that looped the same second forever and a SIGSEGV on WebM.
1. audio_mixer_voice_set_avail had arms for OGG and OPUS only. A
windowed M4A voice therefore accepted every feeder raise and
dropped it: the decoder kept the bound it was given at add_stream,
reached it a second or so in, reported end of stream, and a looping
voice restarted. That is the repeat. M4A and FLAC arms added and
the guard widened - it required HAVE_RWEBM with Opus or Vorbis, so
the whole function compiled to nothing in an AAC-only build.
2. audio_mixer_load_weba sniffed the codec with the file's whole
length. It runs before the sound exists, so sound_set_avail cannot
have been applied, and on a windowed buffer the segment walk went
straight off the committed head:
#0 rwebm_open_memory_avail
#1 audio_transfer_webm_audio_type
#2 audio_mixer_load_weba
#3 audio_driver_mixer_add_stream
Added audio_mixer_load_weba_avail, which bounds the sniff; the
plain form is now a wrapper passing 0, so no other caller changes.
3. The load_* API took int32_t size and the sound stored it as
unsigned. A 7,817,178,301-byte container arrived as -772,756,291
and was stored as 3,522,211,005 - the demuxer then treated every
sample past 3.5 GB as out of range, which truncates the stream
regardless of what the feeder does. Both widened to size_t.
4. sound_set_avail was applied under #ifdef HAVE_ROPUS, so a build
without Opus dropped the resident bound entirely and opened the
decoder against the whole file with only the head committed. The
bound is not Opus-specific; it now applies unconditionally, with
only the end-granule injection left under HAVE_ROPUS.
On the testing, which is the part that failed here: the previous
harness stubbed audio_driver_mixer_add_stream and
audio_driver_mixer_stream_set_avail, so it modelled the two functions
that contained three of these four defects and could not have caught
any of them. samples/audio/mixer_real links the real audio_mixer.c
against a genuine reserved mapping - everything past the resident
bound is PROT_NONE, so a read past it faults instead of passing - and
checks the >4 GiB length survives the load and that the bounded sniff
stays inside the head.
AAC PCM equivalence re-checked and still byte-exact over 144000
frames with the window starved to force backpressure. audio_mixer.c
builds at C89 with RAAC+RMP4, with RWEBM+RVORBIS, with RAAC alone and
with none.
The read-back this replaces held the whole container in memory, which
is why it had to be bounded by gfx_thumb_anim_mem_ok and why a
multi-gigabyte recording still had no audio: it was the one file the
whole windowing effort was for.
Now that the AAC arm honours set_avail (ba45b55), the mixer can borrow
a window instead. A second, independent window over the same file -
data_transfer refuses to let two consumers share committed ranges, and
these two decoders run on different threads - handed over as
params.buf with params.avail saying how much is resident, then
followed forward from audio_driver_mixer_stream_byte_tell.
The bound is what makes this safe rather than merely well-timed. The
decoder refuses to read past avail, so avail is raised only to what
the feed has already committed, never to where the decoder is headed:
a feeder that falls behind stalls the decode instead of touching
reserved pages. Feed first, then raise - the other order advertises
pages that are not there. That is also why the feeder can live in
gfx_thumbnail_animate rather than needing a task of its own; the
failure mode of a menu that stops rendering is a silent preview, not
a fault.
The permanently resident head must cover the container metadata for
the stream's life: the AAC arm borrows the AudioSpecificConfig out of
the moov, and a loop rewinds the demuxer back through it. The video
open already found where the media starts, so its floor is reused
rather than discovered twice - but clamped, because that floor is the
first sample's offset, not the end of the metadata. They coincide in
an ordinary file and do not in one whose media starts far in, where an
unclamped head would have committed gigabytes; past the clamp the
decoder's bounded open declines and the file simply gets no audio.
Cost, measured against the same fixtures with audio disabled: +32.0
MiB on a 7.28 GB leading-moov file (the clamped head), +2.1 MiB on a
40 MB one, +2.2 MiB at 1080p. Independent of file length, which is
the property the read-back could not have.
Feeder margin, measured rather than assumed: the modelled decoder
consumes at a settable rate while the feeder ticks once per frame. At
255 kbps - 532 bytes per tick at 60 fps - the 2 MiB lookahead is ~66
seconds of audio and the bound never moves. Driven to 64 KiB and then
1 MiB per tick (~500 Mbps, some 2000x a real stream) the feeder still
records zero stalls with residency flat, so crackle would need the
menu to stop rendering entirely rather than merely run slow.
The regression harness models the audio thread rather than stubbing
it: it touches every byte it is told is resident, so an over-claimed
avail faults under ASan instead of passing quietly, and counts stalls
- excluding the file end, where hitting the wall is the loop point and
not a starved feeder. ASan+UBSan+LSan, standalone LSan, strict UBSan
and TSan all clean across the fixture set. C89 clean with and without
HAVE_AUDIOMIXER.
The AAC arm could not be told that its buffer was partial.
audio_transfer_set_avail had arms for Vorbis and Opus only, so an
M4A stream handed a window read wherever its sample tables pointed -
into reserved, uncommitted pages. That is what forced the thumbnail
preview to read whole containers into memory, and what capped it at
GFX_THUMB_ANIM_ABS_MAX_FILE: a multi-gigabyte recording could not
have audio at all.
Everything needed already existed one layer down and was simply never
wired up: rmp4_open_memory_avail bounds the box walk, rmp4_set_avail
raises the bound, rmp4_read_packet already withholds a sample whose
extent is past it with RMP4_READ_AGAIN, and rmp4_consumed reports the
compressed frontier. This connects them:
- struct audio_transfer_aac gains 'avail', taken exactly as the
Vorbis and Opus arms take theirs.
- set_avail gets an AAC arm, forwarding to rmp4_set_avail and, for
Matroska-carried AAC, rwebm_set_avail.
- the MP4 open becomes bounded. Unbounded, it walks the box tree to
the end of the file - for a leading-moov movie that is a read past
the resident head before a single packet is pulled.
- the pull reports 2 for RMP4_READ_AGAIN, and the ADTS walk reports
it rather than stepping past the prefix.
- buffer_tell reports rmp4_consumed / rwebm_tell. Without it a
windowed M4A could be given a bound that no feeder could ever
move, since audio_driver_mixer_stream_byte_tell already routes
M4A here and the arm returned 0.
Every consumer of the new code had to be taught it, and two of them
would otherwise have hung rather than misbehaved: read_s16 and
read_f32 test 'r == 0' to stop, so a 2 fell through with pend_frames
at 0, took 0 frames, and spun forever on a window the feeder had not
caught up with. The seek walk had the same shape plus an
uninitialised read, 2 being > 0 there: it used pdata/plen the pull
never set. A stall now refuses the seek, which is what the windowing
contract in this file already specifies - the caller retries once the
bound has moved.
audio_mixer's set_avail call was compiled out unless WebM with Vorbis
or Opus was built, so an avail passed for an M4A stream was silently
dropped; the guard now admits RAAC+RMP4 as well.
Verified for PCM equivalence, not just for playing: the same file
decoded whole-file and through a window raised in steps produces
byte-identical s16 over 144000 frames, with the window deliberately
starved (1 KiB steps) to force the backpressure path - 4 stalls, still
byte-exact. A windowed decode that dropped or repeated one access
unit per boundary would pass a does-it-play test and fail this one.
ASan+UBSan+LSan and strict UBSan clean. Builds at C89 with RAAC
alone, with RAAC+RMP4, with the full set, and with none.
9650f04 stopped the SIGSEGV in the mixer hand-off by skipping it for
windowed handles. That was the right emergency stop and the wrong
resting place: every video now takes the windowed route, so preview
audio was effectively off for all of them.
The mixer needs the whole container - it demuxes the audio track and
seeks its own index - and a window cannot supply that, since only the
resident head plus the sliding window is committed. So read the file
a second time, independently of the window, and hand that over.
The read is opened in SOURCE mode rather than prefix mode, which is
load-bearing rather than incidental: a prefix transfer's buffer is a
memreserve() mapping wherever the platform can reserve address space,
and the mixer releases what it owns with free(). A source transfer is
one exact malloc the consumer may adopt, so the completed buffer is
donated straight across - no second copy, and no multi-hundred-MB
memcpy on the menu thread.
It is pumped by gfx_thumbnail_animate on the same per-frame budget the
adopted read uses, so the menu never blocks on it, and nothing starts
until the read completes - a partial container would have the mixer
demux a truncated index. Admission is gfx_thumb_anim_mem_ok on the
file length, so this restores exactly the pre-window footprint: files
that had audio before the windowed path existed keep it, and only
those past the cap - which previously had no preview at all - stay
silent.
Also frees params.basename on the success path. audio_driver.c:2589
strdups it into the stream slot, so ours was ours to release, and it
was only being freed when add_stream failed - leaking the name on
every preview that actually started.
Regression test gains two assertions, because the earlier ones passed
throughout this bug: preview audio starts, and bufsize equals the file
length exactly. Files past the cap assert the opposite, so the policy
is pinned rather than merely observed. The mixer stub uses the real
audio_mixer_stream_params_t - reconstructing the field order silently
misreads the donated pointer.
Sweeps on the fixture set (leading-moov 7.28 GB, trailing-moov
7.28 GB, co64 past 4 GiB, 160k-sample tables, 1080p and 4K):
ASan+UBSan+LSan clean over 300 frames each, standalone LSan clean,
standalone UBSan with pointer-overflow and signed-integer-overflow
clean, TSan clean both on the TU and on the worker/feeder pair.
Builds clean at C89 with and without HAVE_AUDIOMIXER.
A windowed open ends at whatever avail the last island jump asked for.
With a TRAILING moov that jump reaches the end of the file, so the
stream is created already seeing every byte and never blocks. With a
LEADING moov it is the end of the moov island - which sits immediately
below the media floor, so the first sample of the first frame is
outside the demuxer's valid range from the moment the stream is
installed.
Nothing raised it afterwards. The feeder in gfx_thumbnail_animate
committed pages but never called set_avail, and it was gated on
'tell > 0' - a condition the decoder could only satisfy by decoding a
frame it was unable to reach. No decode, so no tell; no tell, so no
feed; no feed, so no avail. stream_next returned NULL on the first
call, the rewind retry returned NULL too, and animate closed the
animation: install succeeded, every probe reported healthy, and not
one frame ever appeared.
Observed on a 7.28 GB 3840x2160 file whose moov leads at offset 32 and
runs 5.79 MB, past the 4 MiB resident head, so the open needed the
island jump that exposes this:
jump 1: [24,5789233)
media_floor=5789249 prime_hi=18372161
stream OPEN
install: frames=8192 3840x2160 charge=20971520 mem_ok=1
Prime avail to the primed range once the media floor is resident, and
have the feeder carry it forward, anchoring at the floor until the
decoder has consumed something. set_avail is monotonic in both
demuxers, so a stream that already sees the whole file is unaffected
and the trailing-moov path is byte-identical.
Fixtures added to the preview regression test, all built by splicing a
real encode so the tables are genuine: leading moov past the resident
head at 7.28 GB, trailing moov at 7.28 GB, co64 with sample offsets
past 4 GiB, 160k-sample 44-minute tables, and moov larger than KEEP.
The leading-moov case fails on the parent commit and passes here.
ASan+UBSan+LSan clean over 120 frames on each; TSan clean on the
worker/feeder pair.
A video opened over a window installs its stream and reports PENDING,
with no still decode behind it: the first frame uploaded by
gfx_thumbnail_anim_upload is what makes the thumbnail drawable, and it
flips PENDING to AVAILABLE there. Nothing on that route ever raises
alpha.
gfx_thumbnail_reset zeroes alpha, and both request paths only reach
gfx_thumbnail_init_fade through their end: label, which is guarded on
the status NOT being PENDING - exactly the status the windowed open
leaves behind. The still route has no such gap: handle_upload calls
init_fade unconditionally once it has a texture.
So every windowed video decoded correctly, uploaded a texture, reached
AVAILABLE, and drew at zero opacity on every frame. The file browser
showed nothing at all for any video - not only large ones, since after
the reservation-aware gate every video takes the windowed route.
Start the fade where the status flips, which is the only point on this
route that knows the thumbnail just became drawable.
Regression test (samples/gfx/gfx_thumbnail_preview) links the real TU
against frontend stubs and asserts what the menu driver needs in order
to draw: texture uploaded, status AVAILABLE, alpha non-zero or a fade
pushed, no whole-file still load queued for a video, and RSS bounded.
The alpha assertion fails on the parent commit and passes here; the
other four pass on both, which is why the fault was invisible to a
decode-level test.
gfx_thumbnail_preview_audio_start copies container_size bytes out of
anim_buf to give the mixer bytes that outlive the thumbnail. On a
windowed handle anim_buf is a reservation and anim_buf_len is the
file's full length, so that copy reads the whole file out of a mapping
where only the resident head plus the sliding window is committed: it
walks off the head into unmapped pages and faults.
Thread 1 received signal SIGSEGV
#0 msvcrt!memmove
#1 gfx_thumbnail_anim_audio_begin
#2 gfx_thumbnail_anim_install
#3 gfx_thumbnail_anim_open
#4 gfx_thumbnail_request
rbx/r12 = 0x1d1f0acbd (7817178301) - the file length, verbatim
The malloc succeeds first, which is what makes this a fault rather
than a graceful bail: a multi-gigabyte commit charge is satisfiable
from the pagefile, so the allocator hands back a destination and the
read side is what dies. Reproduced identically under ASan on Linux
with overcommit enabled - SEGV in memcpy at the first uncommitted page
past the head.
anim_install calls audio_begin whenever anim_read_pending is clear,
and a window never sets it (a window has no completion to wait for),
so every windowed video reached this. It was unreachable before the
previous commits only because gfx_thumbnail_request_file - then the
sole windowed entry point - is the savestate path and never sees
anything but .png, whose type the audio gate rejects.
Skip the hand-off for windowed handles. Streaming the mixer from a
window needs the feeder the video decoder already has, driven from the
audio thread, which is the cross-thread change anim_install documents
as out of scope; until then a windowed preview is silent rather than
fatal. The fully-resident path is untouched and keeps its audio.
Sweeps, on a 7.28 GB 3840x2160 H.264 + AAC file with a trailing moov
past the 4 GiB boundary: ASan+UBSan+LSan and standalone LSan and UBSan
(with pointer-overflow and signed-integer-overflow) clean over open,
30 frames of decode, and close; TSan clean over a worker/feeder pair
modelling the decode-vs-commit race.
The imageviewer special case in gfx_thumbnail_update_path() - the
branch that lets a browsed WebM or MP4 serve as its own thumbnail -
gated on gfx_thumb_anim_mem_ok(path_get_size(...), 0). That test
short-circuits on file_len <= GFX_THUMB_ANIM_ABS_MAX_FILE, so any
video past 1 GiB failed it, thumbnail_path was left empty, the status
fell through to MISSING, and the file browser showed no preview at
all: no log line, no decode attempted, nothing to distinguish it from
an unsupported format. A 7.28 GB 4K H.264 recording is refused this
way even though rmp4 parses it perfectly well - 64-bit box headers and
co64 chunk offsets are both handled - and even though it is well
inside the sample-table caps.
The comment above the gate explained itself with "unlike an image the
entire file must be read to decode it", which stopped being true when
the sliding-window open landed. A reserved window commits its head
plus a fixed KEEP+AHEAD+BACK budget and nothing else, and jumps
straight to a trailing moov by committing the island the demuxer names
rather than paging the mdat in between. Length is simply not a cost
on that path.
Charge the window where the window is what will be used, and keep the
length test only where data_transfer_open_window degrades to a
whole-file read, matching the admission split gfx_thumbnail_anim_open
already performs internally. With the previous commit putting the
windowed open on gfx_thumbnail_request(), a multi-gigabyte video now
previews from ~20 MiB of committed pages.
The heap-scaled arm of gfx_thumb_anim_mem_ok is unchanged and still
governs the no-reservation fallback, so no console target starts
slurping a file it previously refused.
rgui is unaffected: it drives task_push_image_load directly with its
own callback and has no animated-thumbnail path to reach.
A WebM or MP4 browsed in the file browser is its own thumbnail: it
resolves through the imageviewer special case in
gfx_thumbnail_update_path() and is then loaded by
gfx_thumbnail_request(). That function had no video step at all - it
handed the path to task_push_image_load unconditionally, i.e. the
whole-file still decode, with gfx_thumbnail_anim_open() only reached
afterwards from gfx_thumbnail_handle_upload() to adopt whatever stream
the still happened to leave behind.
So the file-browser preview paid the file's length in resident bytes
where gfx_thumbnail_request_file() - the savestate entry point, and
the only caller with the windowed open - paid a fixed window. A long
recording therefore either spiked the allocation or spent a minute of
disk reads producing one frame, which is what the size gate in
update_path exists to stop it doing.
Try the window first, exactly as request_file does, and fall through
to the still task when it declines. The animation status is PENDING
either way, so the end: fade logic needs no change.
Files under the update_path size gate are the only ones that reach
here today; they now preview from the window instead of a full read.
Lifting that gate is a separate change and depends on this one.
gfx_thumbnail_request_file() opens WEBM/MP4 over a sliding window
before queueing the still task. gfx_thumbnail_request() - the path
every file-browser and playlist entry takes - has no such step, so the
same file goes straight to task_push_image_load and its whole-file
read. Sharing the block is a prerequisite for giving it one.
Pure refactor: the extracted helper performs the same type gate, the
same gfx_thumbnail_anim_open(), and the same list_id/PENDING store as
the inline block it replaces, and reports whether it took so the
caller can fall through unchanged when it did not. No behavioural
change on any path.
Mbed TLS 4.0 split the crypto code out into TF-PSA-Crypto and removed the
legacy low-level interfaces from the public API. mbedtls/ctr_drbg.h and
mbedtls/entropy.h now live under mbedtls/private/, and every function
that used to take an RNG callback pair had those parameters dropped -
all randomness comes from the PSA subsystem, which needs a single
psa_crypto_init() instead of an entropy context plus a seeded DRBG.
net_socket_ssl_mbed.c included both removed headers unconditionally on
the system-mbedtls path, so distributions that have moved their default
mbedtls package to 4.x (Arch went to 4.1.0 on 2026-08-13) fail the build
outright:
libretro-common/net/net_socket_ssl_mbed.c:54:10: fatal error:
mbedtls/ctr_drbg.h: No such file or directory
Gate the legacy RNG plumbing behind SSL_MBED_LEGACY_RNG, derived from
MBEDTLS_VERSION_MAJOR, and use psa_crypto_init() on 4.x. The builtin
mbedtls copy (2.6.0) and the _3DS entropy callback keep the legacy path
unconditionally. psa_crypto_init() is idempotent, so calling it per
socket is safe, and it is deliberately not paired with a teardown since
the PSA subsystem is global and shared with any other live connection.
No other translation unit in the tree consumes system mbedtls headers.
Verified by compiling the translation unit against real installs of
Mbed TLS 4.1.0 and 3.6.5 (-std=gnu99 -Wall -Wextra, zero warnings
attributable to the file in either case); pre-patch the same TU
reproduces the reported error against 4.1.0 and builds against 3.6.5.
Every mbedtls_/psa_ symbol the object references was confirmed to
resolve against libmbedtls/libmbedx509/libtfpsacrypto from 4.1.0.
Link flags need no change: mbedcrypto.pc in 4.x resolves to
-ltfpsacrypto, and the existing pkg-config query picks that up.
Fixes#19394
task_push_to_history_list() takes the history entry label from
runloop_st->name.label, which is global state that persists until the
next content load overwrites it.
Every load path that reaches the history list sets or clears that field:
task_push_load_content_from_playlist_from_menu() copies the playlist
entry's label, task_push_load_content_with_new_core_from_menu() clears
it, and the companion UI path does one or the other. Only
task_push_load_subsystem_with_core() touched neither, so a subsystem
launch pushed a history entry carrying the label of whatever content
happened to be loaded before it.
Play a game from a playlist, then start subsystem content, and the new
history entry is written with the previous game's name - the paths,
subsystem ident and subsystem ROM list are all correct, only the label
is wrong. The same stale value is also what "Add to Favorites" picks up
for the running content.
Give the function the same label parameter as its siblings and set or
clear name.label accordingly. The playlist-relaunch path passes the
entry's cached label so an existing subsystem entry keeps its name; the
file browser and netplay lobby paths pass NULL, and the menu then falls
back to the base name of the first ROM, as it does for any other content
loaded without a label.
Note that playlist_push() only fills in blank fields on an already
matching entry, so history entries written with a wrong label before
this fix keep it and have to be renamed or removed by hand.
Same leak as the gl2 driver, reachable through a different path.
gl3_frame() has a "reset rendering state which libretro core could
easily have overridden" block, but it lives inside the chain.active
branch, which the slang filter chain path never enters, and it does not
cover scissor either way. gl3_filter_chain_build_offscreen_passes()
only touches GL_SCISSOR_TEST under GL3_ROLLING_SCANLINE_SIMULATION, so
on a default build nothing between the core's return and the menu draw
clears it.
Reset it at the hardware-render texture setup instead: that runs on
every frame a hardware-rendering core is loaded, independent of which
shader path is active.
No user report against glcore yet - this is the same class of bug found
in gl2, closed pre-emptively. gl1 needs no equivalent change: the GL
hardware render context type resolves to gl2 or gl3 in
video_driver_find_driver(), so gl1 never hosts a core that can dirty
the context.
gl2_frame() resets the rendering state a libretro core "could easily
have overridden" after a hardware-rendered frame, but GL_SCISSOR_TEST
was missing from that list. Scissor is global context state, so any
hardware-rendering core that returns with the test still enabled leaves
every subsequent frontend draw clipped to the core's last scissor
rectangle. Nothing else in the gl2 frame path disables it: the only
other glDisable(GL_SCISSOR_TEST) calls in the driver are
gfx_display_gl2_scissor_end() and the scRGB UI-layer FBO bind.
The visible symptom is a menu clipped to a rectangle anchored at the
bottom-left of the window (GL's origin) whose size tracks the core's
internal draw area rather than the window. Which menu drivers are
affected depends purely on whether they happen to call
gfx_display_scissor_begin()/scissor_end(): Ozone does, so its first
clipped draw resets the box to the full viewport and disables the test,
masking the leak. XMB and RGUI never call it, so they stay clipped for
as long as the core is loaded.
Reported against Beetle PSX HW, where the clip rectangle matched the
PS1 GPU drawing area (640x480 on the BIOS boot screen, 384x240 in
Castlevania: SotN's file select) scaled by the internal upscale factor.
vulkan_readback() may clamp the copy extent below the staging texture's
own dimensions (negative viewport origin via translate_x/y, or a clamp
against the swapchain size). With the row pitch now stated explicitly
the surviving rows land where the host expects them, but the columns and
rows outside the copied extent are still untouched staging memory, and
vulkan_read_viewport() walked the full vk->vp rectangle regardless.
Record the extent the copy actually used and clamp the host walk to it,
blanking the destination first when the two differ, so a clamped readback
yields deterministic black borders instead of whatever the staging
allocation happened to contain.
No functional change in the common case where the extent is not clamped:
copied_width/height then equal vk->vp.width/height and neither the memset
nor the clamp is taken.
vulkan_readback() left region.bufferRowLength/bufferImageHeight at 0,
which per spec means the copy is tightly packed at imageExtent.width.
The staging buffer, however, is created at vk->vp.width x vk->vp.height,
and every host-side consumer (vulkan_read_viewport,
vulkan_read_viewport_hdr, and the recording scalers via
ctx->in_stride = staging->stride) walks it at staging->stride, i.e.
vk->vp.width texels per row.
Up to 1.15 the two agreed by construction:
region.imageExtent.width = vp.width;
region.imageExtent.height = vp.height;
c847b02206 ("Integer overscale GPU screenshot crash fixes") changed that
to vp.width + vk->translate_x, and 680214bca3 ("Vulkan validation layer
fixes") added a further clamp against the swapchain dimensions. Either
of those can make imageExtent.width smaller than vk->vp.width while the
staging stride stays put, so the GPU packs rows narrower than the host
reads them. Each output row then starts (vp.width - imageExtent.width)
pixels late, producing a constant shear across the image and an
unwritten (black) tail once the copied bytes run out.
Reported against Beetle PSX HW / Vulkan on a Radeon Vega 11: a 1199x900
screenshot came back with a true row pitch of 957 px (242 px of shear
per row), and a 1050x720 one with a pitch of 660 px (390 px per row).
Both are exactly vp.width minus a clamped extent. The software Beetle
PSX core was unaffected only because its viewport never tripped the
clamp.
Set the destination pitch explicitly instead. bufferRowLength is
required to be 0 or >= imageExtent.width and bufferImageHeight 0 or
>= imageExtent.height, both of which hold since the extent is only ever
clamped downwards from the viewport, and the addressing formula stays
inside the allocation because it is sized at exactly
vp.width * vp.height * bpp.
allocate_frames() multiplied an int stride by an int row count to get
each buffer's byte size. Past 2 GB that product does not merely
overflow, it wraps to something small and plausible, and calloc()
happily returns it - so scaler_ctx_gen_filter() reports success on a
context whose scaled frame is orders of magnitude too small, and the
first pass writes through it.
A 262144-wide output over an 8192-row input needs 16 GB and got one
byte:
gen_filter: SUCCEEDED
scaled.stride=2097152 scaled.height=8192 -> needs 16.00 GB
ERROR: AddressSanitizer: heap-buffer-overflow
WRITE of size 8
#0 scaler_argb8888_horiz scaler_int.c:408
#1 scaler_ctx_scale scaler.c:458
0x502000000011 is located 0 bytes after 1-byte region
Products that wrap to a large value instead just fail the allocation,
which was always the harmless half of this.
Compute the sizes in size_t, and reject anything whose byte size will
not fit in an int. That bound is not arbitrary: the frame descriptors
carry int strides and the passes walk rows with int arithmetic -
scaler_argb8888_vert() forms its row base as filter_pos[h] *
(scaled.stride >> 3) - so a frame larger than INT_MAX would index
wrongly even where the allocation succeeded. A degenerate rectangle
is rejected up front for the same reason: the size arithmetic assumes
positive extents.
The ceiling is 2 GB per frame, against 42 MB for a 4K screenshot
reduced to 1440p, so nothing real comes near it. Verified: 8K,
11000x11000 and 15360x8640 sources still scale unchanged; the
bilinear size matrix, the sinc path over nine real 4K screenshots and
the 128-tap fallback are all bit-exact against upstream in x86-64
SSE2, x86-32 SSE2 and SCALER_NO_SIMD builds. ASan+UBSan+LSan clean.
C89 clean under -std=c89 -pedantic -Wall -Wextra.
The 2-tap specialisations left the generic path - every sinc scale -
still paying for two things per output pixel that neither the tap
count nor the pixel position justifies.
Coefficient construction. Both passes widened a tap pair with two
_mm_set1_epi16 and a join, a movd and two shuffles each. The taps are
adjacent int16_t, so one 32-bit load carries both and two unpacks
spread them: three instructions against roughly seven, for the
identical register. Factored out as scaler_coeff_pair().
Repetition. The vertical taps depend only on the output row, yet were
rebuilt for every pixel in it; they are now built once per row into a
small stack array (32 pairs, covering sinc_size for downscale ratios
up to 8:1 - beyond that the per-pixel path still applies, just with
the cheaper construction). The horizontal pass also gets the adjacent
source pair in one movq instead of two 32-bit loads, a shift and an
or, as the 2-tap version already did.
Bit-exact by construction: same coefficient values, same accumulation
order, same taps. Nothing is reassociated - processing two pixels per
register would use the lanes better but reorders the saturating adds,
and unlike the 2-tap case that is not free to assume.
This is the path a 4K screenshot takes when the thumbnail is capped to
a smaller display, which is where it shows. Real save state
screenshots through the sinc stage, gcc -O2 -msse2, mean of 5 warm
runs:
3840x2160 -> 2560x1440 8 taps 70.11 ms -> 45.77 ms 1.53x
3840x2160 -> 1920x1080 16 taps 88.41 ms -> 55.17 ms 1.60x
2880x2160 -> 1920x1440 8 taps 52.50 ms -> 33.41 ms 1.57x
2880x2160 -> 2560x1920 8 taps 78.67 ms -> 51.30 ms 1.53x
2160x3840 -> 1440x2560 8 taps 67.98 ms -> 44.84 ms 1.52x
Verified bit-exact against upstream on nine real 4K screenshots at two
caps (8- and 16-tap kernels), on the 11-case synthetic size matrix,
and on 128-tap kernels that take the fallback - in x86-64 SSE2, x86-32
SSE2 and SCALER_NO_SIMD builds. ASan+UBSan+LSan clean. C89 clean
under -std=c89 -pedantic -Wall -Wextra. scaler_argb8888_vert's frame
grows to 624 bytes, inside the 2048 budget, and the array is inside
the SSE2 path so the small-stack targets never see it.
scaler_ctx_gen_filter() bails from a dozen places, several of them
after allocating buffers or binding a scaling function, and left all
of it behind on the way out. Two consequences, both live.
The buffers leaked. Only rgui cleaned up after a failure, with a
defensive scaler_ctx_gen_reset() and a comment guessing there might be
leftovers; both call sites in tasks/task_image.c, the gl2/gl3/vulkan
readback paths and the camera drivers simply returned.
Worse, a context could come back false with scaler_horiz and
scaler_vert bound but their filters freed or never validated, and
scaler_ctx_scale() cannot tell that apart from a working context.
Callers that ignore the return value - tasks/task_translation.c does,
twice, and scales immediately afterwards - then run through it.
Scaling a 1x1 source, where validate_filter() rejects every generated
position, reads past the end of the source row:
ERROR: AddressSanitizer: stack-buffer-overflow
READ of size 4 at 0x7f16c0400124
#0 scaler_argb8888_horiz scaler_int.c:181
#1 scaler_ctx_scale scaler.c:403
#2 main crash.c:28
Both predate the 2-tap work: the same repro faults identically at
eea592a, one 4-byte read rather than one 8-byte read.
Move the body to a static and have the wrapper release the
allocations and unbind every function pointer when it returns false,
so a failed context is inert rather than half-built. scaler_ctx_scale()
gains a matching guard: with nothing bound there is nothing to scale,
and it must return before the pixel converters, which are called
unconditionally on their formats and would otherwise run against the
buffers the failure just freed.
An ignored failure is now a no-op instead of a wild read. Callers that
do check are unaffected, and rgui's defensive reset stays correct - it
is simply redundant now.
Verified with a harness that deliberately leaks - it never cleans up
after a failed gen_filter - across 1x1, 1xN, Nx1, and working sizes:
ASan+LSan report a heap-buffer-overflow and leaked filters before,
nothing after. Output bit-exact across the 11-case size matrix in
x86-64 SSE2, x86-32 SSE2 and SCALER_NO_SIMD builds. C89 clean under
-std=c89 -pedantic -Wall -Wextra.
The same generality tax the vertical pass was just relieved of, in
three places. The coefficient pair is rebuilt with two
_mm_set1_epi16 and a join for every output pixel of every row, though
it depends only on the column. The two source pixels are assembled
with a pair of 32-bit loads, a shift and an or, though they are
adjacent in memory and can be loaded together. And the product is
accumulated into a zeroed register, though with a single tap pair
there is nothing to accumulate against.
Build the coefficients from one 32-bit load of the tap pair widened by
two unpacks, load the source pair with one movq, and use the product
directly. The store becomes _mm_storel_epi64, which unlike
_mm_cvtsi128_si64 needs no 64-bit GPR, so the 32-bit fallback that
split the result into two dwords through a union goes away with it.
Bit-exact: same taps, same mulhi, same lane fold, minus an add of
zero. Verified by hashing an 11-case size matrix against upstream in
x86-64 SSE2, x86-32 SSE2 and SCALER_NO_SIMD builds - identical in all
three.
Precomputing the coefficients for the whole image rather than per row
was also measured, and is not worth it: 2.56x against this version's
2.49x on 360x256 -> 1440x1024, in exchange for an allocation and a
failure path in a hot function.
Full bilinear scale, gcc -O2 -msse2, minimum of 5 interleaved runs of
20 iterations, against the two preceding commits:
before +vert +horiz
360x256 -> 1440x1024 6.26 2.07 1.51 4.14x
720x256 -> 1920x682 5.89 2.17 1.43 4.13x
320x240 -> 1280x960 5.11 1.51 1.07 4.78x
720x256 -> 2880x1024 12.50 4.28 3.01 4.15x
256x256 -> 1024x1024 4.36 1.29 0.93 4.71x
641x481 -> 321x241 0.87 0.66 0.45 1.94x
1920x1080 -> 640x480 4.37 3.48 2.52 1.73x
ASan+UBSan+LSan clean. C89 clean under -std=c89 -pedantic -Wall
-Wextra with and without SSE2.
scaler_argb8888_vert() is written for an arbitrary tap count and pays
for that generality once per output pixel. It rebuilds both
coefficient vectors from filter_vert[] inside the innermost loop even
though they depend only on the output row, and it spends the whole
128-bit register on one pixel's two source rows, so it retires one
32-bit pixel per pass.
With the tap count known neither is necessary: the coefficient
broadcasts hoist out to the row, and both lanes can carry a pixel
instead, halving the iteration count. Dispatch to a 2-tap variant
when vert.filter_len is 2, which is every bilinear scale.
This is where the menu thumbnail upscaler spends its time - 3.9 ms of
4.8 ms filtering a 360x256 save state screenshot to 1440x1024 - so it
is also the remaining cost of browsing a playlist with
menu_thumbnail_upscale_threshold set. The texture cannot simply be
made smaller instead: the sidebar and fullscreen views share one
texture and the fullscreen toggle only raises a flag, so it has to
stay display-sized.
SSE2 only, deliberately. The same specialisation written in C
measured ~20% *slower* than the generic loop (7.4 ms -> 9.2 ms on
256x256 -> 1024x1024, consistent across runs): a two-iteration loop is
something the compiler already unrolls, and hand-unrolling it only
adds truncations. Scalar builds keep the generic path untouched.
Output is bit-exact - the arithmetic is the same saturating add of the
same two mulhi terms, reassociated across lanes rather than across the
accumulator. Verified by hashing the result of an 11-case size matrix
(upscale, downscale, odd dimensions, single-pixel, unscaled) against
upstream, in both SSE2 and SCALER_NO_SIMD builds: identical in every
case.
gcc -O2 -msse2, full bilinear scale, 20 iterations:
360x256 -> 1440x1024 5.97 ms -> 1.91 ms
720x256 -> 1920x682 4.83 ms -> 2.06 ms
320x240 -> 1280x960 3.94 ms -> 1.29 ms
720x256 -> 2880x1024 9.96 ms -> 3.57 ms
256x256 -> 1024x1024 3.36 ms -> 1.07 ms
641x481 -> 321x241 0.78 ms -> 0.60 ms
1920x1080 -> 640x480 3.58 ms -> 2.88 ms
ASan+UBSan+LSan clean in both configurations. C89 clean under
-std=c89 -pedantic -Wall with and without SSE2.
clamp_8bit() returns uint8_t, which promotes to int before the shift,
so the alpha channel of the scalar vertical store overflows int for
any value above 127:
scaler_int.c:225:32: runtime error: left shift of 138 by 24 places
cannot be represented in type 'int'
Shift as uint32_t instead. Every target we build for produces the
same store from either form, so this is UB on paper only - but it is
UB that fires on ordinary opaque pixels, and so buries the sanitizer
log of anything else in this file under a finding per output pixel.
Found by an ASan+UBSan sweep of the scaler; predates the current
work. Output verified unchanged across an 11-case size matrix
(upscale, downscale, odd dimensions, unscaled).
The thumbnail upscale factor is derived from the shorter side, so on
anything other than a square image the longer side lands at the
threshold times the aspect ratio. On a wide image that overshoots the
downscale cap, and the two stages then fight each other: the bilinear
upscale invents pixels the sinc downscale immediately filters back
out, and the sinc pass costs more than the upscale that made it
necessary.
The width alone decides whether this happens, which is why it looks
like an aspect ratio bug. A 720x256 Amiga save state screenshot at
menu_thumbnail_upscale_threshold = 960 scales x4 to 2880x1024, is cut
straight back to 1920x682 on a 1080p display, and takes 77 ms. The
same shot at 360x256 scales to 1440x1024, stays under the cap, skips
the downscale entirely, and takes 10 ms. Raising the cap to 3840
removes the difference (21 ms vs 11 ms, purely the pixel count), which
confirms the cap crossing rather than the ratio as the cause.
Aim the upscale at the capped size directly instead of overshooting
and correcting. upscale_image() now takes output dimensions rather
than an integer factor, and the caller clamps that target to
downscale_cap before calling it. Where the clamp leaves the target at
or below the source, the upscale is skipped and downscale_image()
handles the image alone.
Output geometry is unchanged in every case; the image simply gets
there in one resampling step rather than two. 720x256 at a 1080p cap
goes from 77 ms to 9 ms, and the result is marginally sharper (47.7 dB
PSNR against the old output, the difference being the softening the
discarded intermediate used to contribute). Unaffected sizes are
unchanged.
Measured with a harness around the real scaler, 15 iterations,
gcc -O2 -msse2, ARGB8888:
720x256, cap 1920: 77.39 ms -> 9.02 ms (1920x682, same)
720x256, cap 2560: 105.81 ms -> 15.36 ms (2560x910, same)
720x256, cap 3840: 20.99 ms -> 20.19 ms (2880x1024, no cap crossing)
360x256, cap 1920: 9.96 ms -> 9.50 ms (1440x1024, unaffected)
libchdr's Zstandard codec was reachable only through the reference
library. Everything gating it - the wrapper's include, the eight sites
in libchdr_chd.c that declare the codec data, register the two codecs
and select them for a hunk, the object in Makefile.common and the
griffin include - tested HAVE_ZSTD alone. A build configured with
HAVE_RZSTD instead therefore compiled no zstd codec into libchdr at
all, and chd_open() rejected any image using CHD_CODEC_ZSTD or
CHD_CODEC_CD_ZSTD as an unsupported format. That is not a decode that
comes out wrong somewhere deep in a hunk; the image does not open, so
it neither runs nor survives a playlist scan.
That combination is not unusual, it is the default. qb/config.params.sh
ships HAVE_CHD=yes, HAVE_RCHD=no, HAVE_ZSTD=no, HAVE_RZSTD=yes, so a
plain ./configure selects libchdr as the reader and rzstd as the only
Zstandard decoder present. pkg/android's Android.mk defines HAVE_CHD
and HAVE_RZSTD without HAVE_RCHD and lands in the same place. Both have
been unable to open a Zstandard image. Makefile.webos sets HAVE_RCHD=1
and was unaffected, as is anything else on the built-in reader.
rchd has had this since it landed, through rzstd. libchdr is what a
build gets without HAVE_RCHD, and both readers are configurable, so the
codec has to be reachable from either one.
Widen every gate to test both, matching the shape already used for
FLAC, and give the wrapper an rzstd path. rzstd wins when both are
configured: it is where the tree is going, and it keeps <zstd.h> - six
long long declarations under -std=c89 -pedantic - out of the
translation unit. No configuration in the tree currently sets both, so
that precedence changes nothing today.
The rzstd path decodes in one call rather than streaming. Both callers
hand over one whole frame and already know how large it expands to:
zstd_codec_decompress() gets a hunk, and cdzs_codec_decompress() splits
its blob on the stored base length and passes each part exactly. That
matters because rzstd_decode() treats bytes left over after a frame as
an error instead of ignoring them, so the lengths have to be exact, and
they are. Nothing is carried between hunks, so init and free have
nothing to do and the codec struct holds a placeholder member, an empty
struct not being C89.
With <zstd.h> gone from an HAVE_RZSTD build, one mixed declaration in
cdzs_codec_decompress() was all that stood between this file and the
standard the tree targets, so complen_base moves to the top of its
block. The assignment stays where it was, after the bounds check that
has to run before the value is read. That arm now builds clean under
-std=c89 -pedantic.
The comment in rzstd.h claiming no caller can be served without a
streaming interface described this file and was wrong about it: a hunk
is one whole frame however it is fetched. Corrected rather than left to
justify an absence it no longer explains.
Verified by round-tripping reference-compressed hunks - 8 CD frames,
mixed raw, RLE and match-heavy content - through both builds of the
wrapper, plain and cdzs. Output is byte-identical: plain digest
8576c74f8291f69d, cdzs digest 9ad1f68b3999d76c on both. Reuse of a
codec across hunks, truncated frames, oversized stored base lengths and
short blobs all behave the same. ASan, UBSan and LSan clean; TSan
clean; no leaks. In the default configuration libchdr_zstd.o resolves
rzstd_decode against encoding_rzstd.o and retains no ZSTD_ symbol, and
libchdr_chd.o still references all six codec entry points, so the table
registration is live rather than compiled out.
One deliberate difference in strictness. Asked to decode a frame into a
destlen smaller than its true content, the reference path stops when
the output buffer fills, sees output.pos == output.size and reports
success, silently truncating; rzstd rejects it. Valid images always
pass the true hunk size, so this cannot fire on one, but on a corrupt
image the rzstd build fails the hunk where the reference build returns
a partial one as good.
cdzl_codec_decompress(), cdlz_codec_decompress() and
cdzs_codec_decompress() discarded the return value of both the base and
the subcode decode. A hunk that failed to decompress therefore fell
straight through to the reassembly loop, which copied whatever
cdXX->buffer happened to hold into dest, and the function returned
CHDERR_NONE. The caller has no way to tell that apart from a good hunk.
The buffer is malloc'd in the codec's init, so on the first hunk that is
uninitialised heap. After that it is the previous hunk's contents.
Decoding a good hunk of 0x41 and then a hunk of 0x42 with one byte
flipped inside the compressed frame returns success with all 2352 bytes
of frame 0 still reading 0x41 - the earlier hunk handed back as valid
data. With WANT_RAW_DATA_SECTOR the loop then runs ecc_generate() over
those bytes and produces a sector that looks structurally sound.
This is not reachable from a well-formed image, where every hunk
decodes; it needs a corrupt or truncated one, which is exactly when
silently substituting unrelated bytes is worst. It is also
backend-independent: the zstd codec behaves identically whether built
against the reference library or rzstd.
cdfl_codec_decompress() has always checked its subcode decode. Bring
the other three into line rather than leave the inconsistency looking
deliberate.
Good hunks are unaffected - round-trip digests over the zstd codec are
unchanged at 8576c74f8291f69d plain and 9ad1f68b3999d76c cdzs, both
backends. The corrupt-hunk case now returns
CHDERR_DECOMPRESSION_ERROR with no stale bytes in the output. All three
files build with WANT_SUBCODE both on and off, with no unused-variable
warning in the latter. ASan, UBSan and LSan clean.
net_http_new() allocates response.data up front as the receive buffer,
64KiB of plain malloc(). On a transport failure -- DNS, connect(), or
a socket error before the status line -- nothing is written into it
and response.len stays 0, but the pointer is non-NULL, and
net_http_data()'s accept_err skipped the status check and returned it.
So every failed download published uninitialised, unterminated heap as
http_transfer_data_t::data. On a body torn mid-transfer it was worse:
response.len is the Content-Length remainder there, not the bytes that
landed, so a consumer copying data->len read heap that never arrived
off the socket.
Every give-up path already resets response.status to -1, and
net_http_new() initialises it so; both accessors now key off that and
return NULL, leaving ownership with the handle so net_http_delete()
reclaims the buffer.
task_http.c gated task_set_error() on RETRO_TASK_FLG_MUTE, which only
ever suppressed the on-screen notification in
task_queue_push_progress(); task->error is a separate channel handed
to the callback unconditionally. Muted is the common case for internal
traffic, and task_core_updater.c's list callback computes
`data && (!err || !*err)`, so an unreachable host looked like a
successfully downloaded empty core list.
Adds samples/tasks/http/http_transport_failure_test.c and wires it
into the samples-tasks-http CI job: four failure lanes entering
net_http_update() at four different points, two contrast lanes (a 200
and a 404 must still deliver their bodies) and two task lanes, muted
and unmuted. getaddrinfo is wrapped at link time so the DNS lane
cannot quietly start testing a successful lookup. Twelve checks fail
on the pre-fix tree; both contrast lanes pass there.
Verified: new test 47/47 under ASan+UBSan with leaks on, and TSan;
existing http samples unchanged; C89 gate clean.
Two more places supplied stdio a buffer of their own, for the same
reason retro_vfs_file_open_impl() did - a comment in each observing
that C89 has no way to request a buffer of a given size without
providing one. Both now pass NULL and let the library allocate, own
and release it with the stream.
config_file_io.c wrote its config through a 16 KiB buffer it malloc'd
and freed around the fclose, having first moved it off the stack
because the autoconfigure path reaches this from a task thread with a
16 KB frame. With the library owning it there is nothing to keep
alive across the writes and nothing to free, so the allocation, the
failure branch and the stack note all go away. The stream is
write-only, so unlike the VFS change this is not about Apple's read
fast path; it is one allocation per config write and a smaller
function.
vfs_implementation_uwp.cpp is the same code as the portable VFS with
the same buffer, and is changed for the same reason plus a duller one:
two implementations of one interface that differ in how they buffer is
a trap for whoever reads one and edits the other.
verbosity.c was checked and left alone - it deliberately installs no
buffer at all, because every log line is flushed as it is written, and
the comment there already explains it.
Verified: full build clean; config_file_test passes under
ASan+UBSan+LSan; C89 diagnostics unchanged in config_file_io.c. The
UWP translation unit needs WinRT headers and does not compile on this
host, so that half rests on the edit being the same one-line
substitution made twice elsewhere, and on the MSVC job.
Every buffered open handed stdio a 64 KiB buffer of our own:
stream->buf = malloc(bufsize);
setvbuf(stream->fp, stream->buf, _IOFBF, bufsize);
Who owns that buffer is not a detail on every C library. Apple's
fread() has a fast path that reads a large request straight into the
caller's buffer instead of copying it through the stream, and gates it
on __SMBF - "this buffer is mine". setvbuf() sets __SMBF only on the
branch where it allocates the buffer itself, so passing one in cleared
the flag and disqualified the fast path (Libc stdio/FreeBSD/setvbuf.c
line 125 sets it; fread.c tests it, citing radars 5980080 and
6180417).
The buffer was added for write throughput - 8 MiB in 1 KiB pieces went
from 14.02 ms to 3.05 ms - and it has been costing Apple targets
roughly half their read speed ever since, which is how a whole-file
read of a 2.4 MB font measured 0.3097 ms buffered against 0.1636
unbuffered on a MacBook Pro.
Passing NULL keeps the size and moves the allocation into the library,
so writes keep the buffer they were given and reads on Apple stop
paying for it. It also removes a malloc and a free from every open.
A library that will not allocate for a NULL buffer leaves the stream
on its own default: slower for small writes, correct, and the same
outcome a failed malloc used to produce.
The whole-file read path already avoids this by not using stdio at
all. What this reaches is everything else - intfstream, the archive
walkers, the image decoders - which is most of the reading RetroArch
does and none of what the earlier change covered.
Measured reading a 2.4 MB file repeatedly from an open handle, so the
open cost is out of it and the only variable is who allocated the
buffer:
Apple 0.1702 ms -> 0.0641 ms -62.3%
glibc -8.3%
musl -0.5%
Bionic +0.7%
Neutral on every library that has no ownership gate, and 2.65x on the
one that does. Larger than the -47% the same buffer cost a whole-file
read, because that figure included the open and this does not: what is
left here is the copy itself.
The 3DS and Wii U branches keep their own buffers: Wii U aligns to
0x40 for its DMA, which a library allocation would not promise.
Verified: full build clean; the vfs contract tests at 113 / 110 / 112
checks across the three path configurations under ASan+UBSan+LSan,
with the seek, mapped-pointer, overflow and large-file samples passing
alongside; C89 diagnostics unchanged; arm-vita-eabi, mingw-w64 and the
Android NDK all compile it.
oga_gfx.c had not compiled for some time. It reached for
&stb_font_renderer and called that backend's init() by hand:
vid->font = vid->font_driver->init("",
settings->floats.video_font_size, FONT_ATLAS_FORMAT_A8);
three arguments against a five-argument prototype. Calling a backend
directly means tracking its signature, and this driver stopped doing
so - the argument list has been wrong across more than one change to
that interface, and nothing noticed because HAVE_ODROIDGO2 is off by
default and no CI job builds it.
It goes through font_renderer_create_default() now, like every other
video driver: that resolves the path, reads the file, chooses a
backend and hands back a driver to free it with. A NULL path still
ends at stb's built-in glyphs when no font is configured or found,
which is the behaviour the old call wanted, and a configured font is
now honoured, which it was not. Teardown guards the driver pointer as
well as the handle and clears both.
Three other things kept it from compiling once the font call was
fixed, all rot rather than anything recent: errno.h, unistd.h and
sys/mman.h were used and not included - errno, close(), mmap() and the
PROT_/MAP_ constants all came from somewhere else once and no longer
do. Two variables computed from the game geometry at the top of
oga_gfx_init() have no remaining reader and are dropped.
Checked with a stand-in for <rga/RgaApi.h>, since librga is not
something a build host has: the file now compiles clean under -Wall
-Wextra with only the diagnostics that stub's own gaps produce, and
those go away when its rga_info_t is filled in to match. It is a
Linux-only driver and, like its neighbours, does not build under C89 -
it uses designated initialisers, and the DRM headers are not C89
either. Untested on hardware; nobody here has an ODROID-GO Advance.
The font file cache guarded its list with retro_atomic_cas_int(), which
is not defined by every backend in retro_atomic.h. The fallback -
selected when __GCC_ATOMIC_INT_LOCK_FREE says the compiler may emit a
call rather than a lock-free instruction - has no compare-and-swap at
all, and the PS2 toolchain lands there, so gfx/font_driver.c stopped
compiling for it.
Gated on the primitive rather than on a platform list, so a target that
gains lock-free atomics gains the locking with them and none has to be
named here. Where it is absent there is nothing to guard: that backend
is chosen exactly on the single-core targets, and PS2 in particular
builds with HAVE_THREADS=0, so the gate is already closed for it on
the threading condition alone.
Compiles clean with every backend forced in turn - volatile, C11,
gcc-new, sync - and with threads on and off, plus arm-vita-eabi. The
lock also has no wrapper functions any more; a one-line static that
exists to hide a single call, and that has to vanish entirely in the
unlocked build, is better written as the macro it wanted to be.
Behaviour is untouched: four threads building and destroying the
nine-font set concurrently stay clean under TSan and under
ASan+UBSan+LSan, now linked against the real rthreads rather than a
stub, and a context reset still reads 4.93 MiB where the fonts are
released before the new ones are built and nothing at all where they
overlap.
Ozone builds six fonts and gfx_widgets three, and between them they
name two files. Every font instance read its whole face from storage:
nine reads of two faces per context reset, and a context reset happens
on content load and again on close. On a Vita that is 21.91 MiB off
the memory card each time, of which 17 MiB is the same bytes over
again, and the same volume held in resident buffers.
The bytes of a path are now read once and shared by every font built
from it, and released when the last of them is gone. Measured on the
real nine-instance workload through font_renderer_create_default():
before 22971643 bytes (21.91 MiB)
after 5173059 bytes ( 4.93 MiB)
The rendered result is untouched: the atlas of the first font hashes
identically before and after, which is the point of sharing the bytes
rather than the parsed face.
A renderer says whether it can borrow. stb and freetype only read
what they are handed, so they take the shared buffer and no longer
free it - stb already had font_data_owned for the Wii U shared font,
and freetype now leaves file_data alone, its face being torn down
first since the face holds a pointer into those bytes. coretext hands
the buffer to CGDataProviderCreateWithData and CoreGraphics releases
it on a schedule of its own, so it keeps ownership and gets a private
copy; it needs no change at all, taking the new field's default.
The reference cannot be dropped by whoever destroys the font. Some
thirty video drivers hold the driver pointer and the handle and call
'font_driver->free(font_data)' themselves, and most of those files
cannot be built outside their own platform, so thirty edits would be
thirty chances to break a driver nobody here can compile. Instead the
driver handed back by font_renderer_create_default() is a copy of the
backend's with its free replaced, and the release rides on the free:
the reference is dropped wherever the font is destroyed, by callers
that know nothing about any of it. The backend's own teardown runs
first, and outside the lock.
Each wrapper slot has its own trampoline so that it knows which
backend it stands for without consulting the bookkeeping record. That
matters on one path: if the record's allocation failed at creation
there is no reference to drop, but there is still a font holding an
atlas and whatever else the backend hung off it, and it has to be torn
down anyway. Returning early there would have turned a failed calloc
of a few bytes into a leaked font.
Locking is a statically initialised word rather than an slock_t,
because an slock_t has to be created and there is nowhere race-free to
create it: rthreads has no one-time-init primitive, and the first font
is not reliably built on a known thread - with threaded video, init is
dispatched to the video thread. Spinning is only tolerable because
the file read happens outside the lock and every critical section is a
walk of a list holding one entry per distinct font file. Two threads
wanting the same uncached path both read it and one loses the insert,
which costs a redundant read in a case this codebase does not produce.
The placement of the release is the whole correctness question, and
the first version got it wrong: the hook sat in
font_driver_release_renderer_state(), which frees the raster font
rather than the glyph renderer and is nothing like the only caller of
the latter. A harness replaying three context resets caught it -
4.93 MiB, then nothing, then nothing, the references never returned.
LeakSanitizer does not catch that shape, the entries being reachable
from a static list, so the cycle count is the assertion that matters.
Verified: full build clean; three context resets reading 4.93 MiB
each; four threads building and destroying the nine-font set
concurrently, clean under ASan+UBSan+LSan and under TSan; C89 pedantic
diagnostics unchanged; arm-vita-eabi cross-compiles.
Noticed while surveying the callers and left alone: gfx/drivers/
oga_gfx.c calls font_driver->init() with three arguments against a
five-argument signature and cannot have compiled for some time. That
predates this and wants its own fix.
The macOS leg of the cross-libc job failed test_directory_path in the
buffered configuration, and the test was right.
A directory opens and seeks like a file on every platform checked,
and the platforms then disagree about what it looks like. Linux
reports INT64_MAX from lseek, which the size ceiling turns away, so
this never showed there. Darwin's buffered path reports a small
plausible length and then reads zero bytes - and zero bytes after a
non-zero size was being taken for a file that shrank, which is a
real case the loop has to tolerate. So filestream_read_file()
returned success with an empty buffer: a directory made
indistinguishable from an empty file, differently per platform.
Two guards, because a directory can arrive in two shapes.
Where the size is non-zero and nothing at all is read, that is not a
file that shrank - a shrunk file still hands over the bytes it has,
and the loop already accepts a short count - it is a stream that
cannot be read. That fails without needing to ask what the path is.
Where the size is zero the bytes say nothing, since an empty file
looks the same, so this one costs a stat. It is paid only on the
read-to-EOF branch, which a file with a known size never reaches, so
no read on the paths that matter for throughput touches it.
Empty files, single-byte files and procfs entries all still read
successfully; that is the regression this could have caused and it
is checked directly.
test_directory_path now forces the three lengths a platform can
report for a directory (0, 128, 4096) through the wrapper VFS, each
with the read answering an error and again with it answering
end-of-stream, so every machine exercises every other machine's
behaviour rather than only its own. The Darwin shape reproduces on
Linux with those forced: six failures before this change, none
after.
glibc 113 / 110 / 112 checks across the three path
configurations, clean under ASan+UBSan+LSan and TSan
Bionic 113 / 110 / 112
mingw-w64 113 under Wine
C89 pedantic diagnostic count unchanged
Both non-Linux legs of the new cross-libc job failed on their first
run, each for a reason that was written into the job and never
executed against the platform it described.
Windows died at the first compile. The leg ran a plain `make`, and
the vfs sample Makefile defaults MMAP to 1, which puts -DHAVE_MMAP on
the command line and makes vfs_implementation.c include
<sys/mman.h>. That header does not exist under MSYS2, so both the
UCRT64 and MINGW64 legs stopped on the shared sources.
The flag was never wanted there. VFS_HAVE_FILE_MAPPING comes from
_WIN32 inside vfs.h, so the Win32 CreateFileMapping backend - and
with it descriptor I/O and the mapped-pointer path - is on with or
without it; HAVE_MMAP only selects the POSIX half of that pair. The
job's own comment said exactly this and then failed to pass the flag.
Fixed in two places. The job passes MMAP=0 explicitly, and the
Makefile now defaults it from uname, so a plain `make` in an MSYS2
shell does not fail on a missing POSIX header either - the sample
directory should build for someone who just walked into it. All
five vfs samples build and pass under mingw-w64 that way, so
vfs_mapped_ptr_test is added to the leg while it is open: it covers
the Win32 mapping backend, which nothing else in the tree exercises.
macOS built everything and then aborted on the run line. The job
asked for ASAN_OPTIONS=detect_leaks=1, and LeakSanitizer is not
implemented on Darwin - the runtime does not ignore the request, it
prints "detect_leaks is not supported on this platform" and calls
abort(), so the leg exits 134 with every test unrun. The option now
comes from the matrix: full strictness on Linux, off on macOS, whose
leg is there for Apple's libc behaviour rather than for leak
coverage. Leak coverage comes from the Linux leg on identical
sources.
Linux is untouched. The three path configurations still report
101 / 98 / 100 checks with no failures, under both detect_leaks
settings, with vfs_seek_contract_test and vfs_mapped_ptr_test
passing alongside.
filestream_read_file() opened with RETRO_VFS_FILE_ACCESS_HINT_NONE,
which pins the stream to buffered stdio. For a function whose whole
job is "open it, take all of it, close it", the buffer is dead weight:
the bytes are going straight into storage the caller keeps, and no
part of the stream is ever read twice.
What that costs depends on the C library, and they disagree:
glibc fread() reads into the caller's buffer once the request
reaches the block size, whoever owns the buffer
(libio/fileops.c, _IO_file_xsgetn).
musl drains the buffer, then reads the remainder direct.
Bionic breaks out of the copy loop and reads direct as soon as
the remainder exceeds _bf._size (libc/stdio/stdio.cpp).
newlib refills through the stream buffer and memcpy()s every
refill into the caller's buffer.
Apple has the bypass, but skips it when setvbuf() supplied the
buffer - __SMBF is then clear (Libc stdio/FreeBSD/fread.c,
radar 5980080 / 6180417) - which is exactly what
retro_vfs_file_open_impl() does.
So the two families that pay for it are the newlib consoles and Apple,
and on Apple we cause it ourselves by handing stdio our own buffer.
Measured on VitaSDK's own libc.a (linked against a Linux syscall shim
and run under qemu-arm), one 2.4 MB read of assets/ozone/regular.ttf:
before 39 read() calls, 2525124 bytes of libc-internal memcpy
after 1 read() call, no internal copy
Over one RetroArch context reset, where ozone and gfx_widgets build
nine font instances between them, that is 355 read() calls and 22 MB
of redundant copying down to 9 calls and none.
The descriptor path this selects already existed - it is what a file
mapping falls back to - but RFILE_HINT_UNBUFFERED was only ever set
inside '#ifdef VFS_HAVE_FILE_MAPPING', so every target without mmap or
Win32 file mappings could not reach it no matter what the caller
asked for. The gate was over-broad: only the stream->mapped block
needs a mapping, read(2) needs nothing. VFS_HAVE_DESCRIPTOR_IO names
the targets where the path has been exercised, and is wrapped in
#ifndef so a port can opt in with a -D rather than a header edit.
A separate hint rather than widening FREQUENT_ACCESS, because the two
want opposite things: FREQUENT_ACCESS means "I will seek around in
this, map it if you can", and mapping every playlist and config read
would be a regression on desktop. The new hint is added after the
existing block and never touches the mapping hint, so a stream asking
for both still maps and behaves as before. It is also restricted to
VFS_SCHEME_NONE: the descriptor branch knows about plain files and
(on Android) SAF, and has no case for cdrom:// or smb://.
Two correctness fixes have to land with it.
First, the read is now looped. filestream_read_file() issued one
filestream_read() for the whole file and treated the count it got back
as the answer. fread() loops internally, so on the buffered path the
count was always complete and the missing loop never showed; read(2)
has no such obligation, and neither does a frontend-supplied VFS.
Routing this function onto the descriptor path without the loop would
turn a latent bug into silently truncated fonts, shaders and ROMs.
Requests are capped at FILESTREAM_BULK_READ_MAX so no single read
exceeds what a platform's count argument can express - Windows _read()
takes an unsigned int and returns int, Bionic caps its own direct
reads at INT_MAX, and Vita's _read_r hands len straight to sceIoRead.
A file that shrank between the size query and the read still succeeds
with the byte count that was there, as it always has.
Second, an implausible size is refused rather than handed to malloc().
A directory opens like a file on Linux through both fopen() and
open(), and seeking to its end reports INT64_MAX; the old
'malloc((size_t)(content_buf_size + 1))' did that addition in int64_t
first, which is signed overflow - undefined behaviour, and UBSan says
so. Ordinarily the allocation merely fails, but AddressSanitizer
treats a request past its ceiling as fatal, so every sanitizer build
inherited an abort from a path that only needed to return an error.
The arithmetic now happens in the unsigned domain after the bound
holds, and FILESTREAM_READ_FILE_MAX rejects the size without a
syscall. This was reachable before this series and is fixed here
because this is the function it lives in.
Tested with a new contract test, vfs_bulk_read_test, which covers path
selection, round trips across fourteen sizes straddling both buffer
sizes, short reads and read errors injected through a wrapper VFS, the
file-shrank case, binary transparency (CRLF, LFCR, 0x1A and embedded
NULs - the descriptor path opens with open()/_wopen() rather than
fopen("rb"), so it depends on O_BINARY), a directory as the path, and
eight threads reading concurrently. It discriminates: against the
un-looped version it reports "chunk 7: got 7 of 1048576 bytes".
glibc x86-64 101 checks, clean under ASan+UBSan+LSan and TSan
musl x86-64 clean
Bionic (NDK r27c) 101 / 98 / 100 across the three path configs
mingw-w64 msvcrt 101 under Wine
mingw-w64 ucrt 101 under Wine
arm-vita-eabi compiles; VFS_HAVE_DESCRIPTOR_IO on, mapping off
Interleaved A/B timings, 15 trials of 100 reps, median:
2.4 MB 4 KB
glibc -2.7% -21.5%
musl -1.9% -89.9%
Bionic -1.4% -60.7%
Apple -47.2% -12.2%
msvcrt -1.9% -1.8%
UCRT -1.8% -1.4%
Two different mechanisms show up in that table. On the Linux
libraries the win is in the small files, because what is being
skipped is the 64 KiB setvbuf allocation per open - which glibc
serves from a free list but musl and Bionic satisfy with mmap - and
that matters to the thousands of small whole-file reads a scan does
rather than to the handful of large ones. On Apple it is the
reverse, and it is the copy: an unbuffered stream there measured
0.1657 ms against 0.1636 for the descriptor path and 0.3097 for the
buffered one, so setting __SNBF is enough to recover the whole
difference. That is the __SMBF gate above, confirmed on hardware -
the 64 KiB setvbuf, added for write throughput, has been costing
Apple targets roughly half their whole-file read speed by
disqualifying fread's bypass.
Both Windows runtimes are neutral, and measure the same doing it -
0.1457 ms against 0.1456 on the large file. ucrtbase is what an
MSVC build links, reached through the api-ms-win-crt-* sets, and it
reads a large request straight into the caller's buffer: on an open
handle with the open cost excluded, its four arms (caller-owned
buffer, libc-owned buffer, unbuffered, descriptor) sit within 2.1%
of each other at about 20 GB/s. Its
absolute numbers also say why nothing subtler shows on Windows: a
4 KiB open-read-close costs about 0.021 ms there against roughly
0.0015 ms on Linux, so a userspace allocation or an extra copy is
rounding error beside the kernel's open path.
One more data point arrived by accident and is worth keeping. A
binary built by the MSYS2 Cygwin compiler rather than the UCRT64
toolchain - the two are distinguishable only from the PE import
table - gave 13.4 GB/s buffered against 19.8 GB/s unbuffered on the
same machine where ucrtbase gave 20 GB/s throughout. Cygwin's stdio
is newlib-derived, so that is the same refill-and-copy behaviour as
VitaSDK's newlib above, on entirely different hardware, and it
settles the ownership question for that family: newlib copies either
way, where Apple copies only when the buffer is not its own.
The read-to-EOF path for size-0 files, the one shape where the stream
buffer could have won, measured faster too (glibc -5% to -8%).
Cross-libc CI is added for the libraries that cannot be reached from a
Linux runner: macos-latest for Apple libc, MSYS2 UCRT64 and MINGW64
for the two Windows runtimes. Apple, msvcrt and ucrtbase have all
been measured on real hardware; nothing above is inferred, and the
jobs are there to keep it that way.
ozone_draw_messagebox() called gfx_display_draw_texture_slice()
without first establishing a blend state. Unlike every other
gfx_display draw helper, the slice helper issues its nine
dispctx->draw() calls raw and never brackets them with
blend_begin()/blend_end() of its own, so it inherits whatever
blend state the previous draw happened to leave behind.
The immediately preceding call is ozone_draw_backdrop() ->
gfx_display_draw_quad(), which terminates with blend_end().
With content running the video driver's frame path leaves a
different state than in menu-only mode, so the inherited state
differs and the dialog's alpha channel is composited wrongly.
dialog-slice.png is pure white in RGB across the whole image;
only the alpha channel carries the rounded rectangle shape.
Any source factor other than SRC_ALPHA therefore paints solid
white into the four alpha-0 corner regions instead of leaving
them transparent.
Bracket the slice draw in blend_begin()/blend_end(), matching
ozone_draw_cursor_slice() and xmb_render_messagebox_internal(),
which already do this correctly.
Also initialise draw.scale_factor and draw.rotation in
gfx_display_draw_texture_slice(). They were left uninitialised
on the stack while d3d9cg, d3d9hlsl, d3d10, d3d11 and d3d12 all
read both fields, applying garbage scaling and rotation to the
nine sub-quads.
Signed-off-by: LibretroAdmin <[email protected]>
win32_display_server_get_metrics() acquires a screen DC via GetDC(NULL)
before the metric switch, but the default branch returns without
releasing it. Every query for a metric this display server does not
implement therefore leaks a DC.
The two early returns above the switch are unaffected -- both happen
before the DC is acquired.
* autodetect: enhance Bliss-Box pad type detection and updated pad types.
logging: fixed an issue with null values creating spurious file system garbage.
* autodetect: harden the Bliss-Box hidraw probe and unify pad type lookup
Follow-up to the hidraw probe path, addressing correctness issues found
by static analysis and a sanitizer sweep.
Fixes:
- Double close of the hidraw descriptor. The success path closed fd and
broke out of the scan without clearing it, so the post-loop
"if (fd >= 0) close(fd)" closed the same descriptor a second time.
Because autoconfig can run while other threads are opening files, the
second close could reap an unrelated descriptor rather than simply
failing with EBADF. Reported by -Wanalyzer-fd-double-close.
- Reported device name was overwritten with "Bliss-Box 4-Play" when no
pad was found. Bliss-Box devices are excluded from the VID+PID match
in input_autoconfigure_tuple_affinity(), so the name is the only
signal a profile can match on, and the shipped fallback profile keys
on the name the adapter itself reports. Overwriting it left the pad
with no autoconfig at all, and the sentinel cached in blissbox_pads[]
prevented recovery on a later probe. The reported name is now left
untouched.
- struct hidraw_devinfo was read on the path where HIDIOCGRAWINFO had
failed, so an uninitialised stack buffer was formatted into the log.
It is now zeroed, and the failure and id-mismatch cases are separate
so the struct is only read when the ioctl succeeded. Its fields are
__s16, so they are also cast through uint16_t to stop ids at or above
0x8000 printing sign extended.
- The libusb error paths skipped libusb_release_interface() and
libusb_attach_kernel_driver(), leaving the adapter detached from the
kernel driver until it was physically replugged. Cleanup is now
unconditional through a single exit label.
- The libusb control transfer omitted the report type from wValue. The
HID specification requires (type << 8) | report_id, so the request
asked for input report 17 rather than feature report 17.
- The hidraw and libusb backends disagreed on where the pad type lives
in the feature report: one read answer[1], the other answer[0]. Both
now share blissbox_feature_report_index() and blissbox_pad_from_index(),
which also replaces three copies of the lookup loop. A numbered HID
feature report carries its report id in byte 0 and the payload in byte
1; transports that strip the id put the payload in byte 0. Both shapes
are accepted.
- Pad type 55 was named "Unassigned", which is a non-empty name and so
passed the sentinel check, producing the device name "Bliss-Box 4-Play
Unassigned". It is now a NULL entry.
- Pad type 227 (0xe3) is the PlayStation protocol id for the JogCon and
was the only value mapped previously. It is kept alongside the newer
127 so both firmware generations resolve.
Other changes:
- The sysfs uevent read goes through the VFS (filestream_*) instead of
stdio, and path construction uses strlcpy instead of snprintf, which
also clears two -Wformat-truncation warnings.
- Per-device skip messages are demoted to RARCH_DBG; the scan ran once
per port and logged every unrelated HID device on the system at
ordinary log level. A failed open() now warns and mentions that a udev
rule may be required, since that is the actionable case.
- qb: HAVE_BLISSBOX no longer requires libusb on Linux. The hidraw path
is the preferred backend there and libusb is only a fallback, so a
--disable-libusb build previously lost Bliss-Box support entirely.
- Dropped the log_directory initialiser in verbosity.c; master already
zero-terminates that buffer a few lines below.
Note: blissbox_pads[] is read and written without synchronisation. TSan
reports a race when the handler is entered concurrently for the same
port. Every current caller reaches it from a joypad driver polling on
the main loop, so this is latent rather than live, and it predates this
series; left alone here.
Sanitizer sweep: ASan+UBSan+LSan clean, valgrind memcheck clean (0
errors, no leaked blocks or descriptors) across 15 cases covering both
report shapes, match-on-later-node, malformed and oversized sysfs uevent
data, missing sysfs entries, ioctl failures at each stage, denied
opens, hidraw-lookalike /dev names, short and zero-length buffers, all
256 pad indices cross-checked between backends, and 1000 repeated
probes. Negative controls confirm the harness detects each fixed defect.
Builds clean at -Wall -Wextra with and without HAVE_LIBUSB, C89-pedantic
clean, and -fanalyzer clean.
* autodetect: fix Win32 build, restore loop counter in Bliss-Box HID scan
The Win32 pad type lookup loop was replaced by the shared
blissbox_pad_from_index() helper, and the 'unsigned i' declaration went
with it. That counter is also used by the SetupDiEnumDeviceInterfaces()
enumeration loop earlier in the same function, so the Win32 build broke
with 'i' undeclared.
Restores the declaration. No functional change.
Verified with the real MXE CI invocation
(./configure --host=x86_64-w64-mingw32 && make), plus MinGW x86_64 and
i686 at _WIN32_WINNT 0x0500 and 0x0601, native gcc/clang/g++, C89
pedantic, and -O0/-O2/-O3.
---------
Co-authored-by: Rendman <[email protected]>
rjpeg rejected any frame that was not 1 or 3 components, so Adobe CMYK
and YCCK streams failed at the SOF gate. Everything downstream of that
gate was already written against img_n rather than a hardcoded 3 --
img_comp[4], order[4], coutput[4], scan_n <= 4, the SOF length check,
the img_x * img_n * img_y overflow guard, both arena cuts,
rjpeg_plane_ring_undo and the progressive finish walker all scale
already -- so the change is the gate, the Adobe APP14 parse, and a
colour branch.
Two clamps in the output pipeline hardcoded 3 (decode_n in
rjpeg_resample_setup and rjpeg_output_rows). Both were identities for
the 1 and 3 component cases they were reached with, so dropping them to
img_n is a no-op for existing streams and gives the K plane its own
resample state, including its own sampling factors.
APP14 is parsed in place in the marker skip arm, which previously
blind-skipped every APP segment. The transform byte selects the colour
path; the cursor is untouched and the existing clamped skip still
consumes the segment, so a truncated or forged APP14 cannot move the
parse. The peek requires its 12 bytes to be both declared and resident,
which also keeps it correct under prefix feeding, where img_buffer_end
is the resident frontier rather than the end of the file.
The colour handling is the part worth stating precisely, because both
layouts are stored inverted and only one of them wants the complement
of the YCbCr result:
CMYK (transform 0, or no APP14) -- the byte on disk is 255-C, and
libjpeg hands those raw values to the caller rather than undoing it,
so consumers have always reconstructed R = v_c * v_k / 255
themselves. rjpeg's output contract is BGRA/RGBA, so it does that
here.
YCCK (transform 2, or anything unrecognised, matching libjpeg's
warn-then-assume) -- the same inverted CMYK with the forward YCbCr
transform on the first three planes. Running the ordinary
YCbCr->RGB kernel over them recovers the complement of the stored
bytes, so this path inverts before the same multiply. This is why
libjpeg's ycck_cmyk_convert emits MAXJSAMPLE minus the YCbCr result
while its CMYK path is a null convert.
Getting that backwards yields a photographic negative rather than
anything that looks like a decode failure, so it is asserted on
directly (see below).
A four-component frame with no APP14 at all is read as CMYK and still
treated as Adobe-inverted. Decoders disagree here -- libjpeg leaves the
inversion to the caller, so ffmpeg reads such planes literally while
Pillow always inverts -- and four-component JPEGs in the wild are
overwhelmingly of Adobe origin even when a tool has since stripped the
segment, so inverting is the better bet. Called out in the code and
pinned by a test so a future revisit fails loudly rather than silently.
RJPEG_MUL255 is an exact rounded x*y/255 with no divide.
The 3-component fused hv_2 kernel is gated on img_n == 3, so
four-component frames fall out to the separate resample and convert
path with no change to the YCbCr hot path.
Also bound the DHT symbol count, found by the fuzz sweep written for
the above. The 16 count bytes of a DHT segment are summed into n
without any check, so a malformed segment can declare up to 16*255 =
4080 symbols where the format allows 256. Both consumers are sized for
256: rjpeg_build_huffman fills h->size[], a uint8_t[257] and the last
member of rjpeg_huffman, and the value read memcpy()s n bytes into
values[256]. A crafted DHT therefore overruns size[] by 3823 bytes and
values[] by 3824, corrupting the neighbouring Huffman tables and the
fast_ac acceleration tables. The write stays inside the rjpeg_jpeg
allocation, which is why ASan does not see it -- the huffman tables are
interior members and the slack behind them happens to absorb the
overrun. That is a property of the current field order, not a bound,
and it does not survive a reordering or a change to FAST_BITS. The
upstream stb_image sources this decoder derives from carry the same
check; the fork lost it. Reachable from any JPEG the frontend decodes:
thumbnails, menu wallpapers and overlay images all reach
rjpeg_process_marker before anything validates the stream. Verified
with a crafted DHT declaring all sixteen counts as 255: before,
-fsanitize=bounds-strict reports "index 4080 out of bounds for type
'uint8_t [257]'"; after, the segment is rejected.
test_rjpeg gains five 24x17 streams -- YCCK 4:4:4, YCCK with the K
plane subsampled under Y, progressive YCCK, CMYK transform 0, and a
four-component stream with no APP14 -- which extends the existing path
equivalence and abandoned teardown sweeps over them, plus two tests for
the colour stage itself. Those assert neutral-patch bounds rather than
golden checksums, so they hold in either byte order and under any IDCT
or colour kernel the host dispatches, and they assert that the
transform byte reaches the colour stage at all. Path equivalence cannot
see an inversion error, because both drivers share the colour stage.
Verification: decoded against libjpeg (via Pillow) over a 193x131
corpus chosen not to be MCU-aligned -- baseline RGB, grayscale,
progressive, YCCK 4:4:4, YCCK subsampled K, progressive YCCK, CMYK
transform 0, and no-APP14 -- max channel delta 3 across all eight in
both byte orders, the same order as the pre-existing YCbCr paths.
Output is byte-identical to the previous build for every 1 and 3
component stream in both byte orders. The three tests were mutation
checked: dropping the YCCK complement, treating every four-component
frame as YCCK, and ignoring the transform byte each fail the intended
assertion.
Sanitizers: the suite passes under ASan, UBSan and LeakSan with
CK_FORK=no. A truncation and bit-flip sweep -- every truncation point
of every stream across both drivers and both byte orders, plus 20000
single-byte corruptions per stream weighted toward the header -- is
clean under ASan, UBSan, LeakSan and -fsanitize=bounds-strict with
halt_on_error. Prefix feeding at every resident step size from 1 to 64
bytes matches the whole-buffer decode for all streams. TSan over 8
concurrent instances x 8 files x 40 iterations x 4 modes reports no
races and no checksum mismatches against the single-threaded reference.
Size: +554 bytes of .text at -Os, unchanged with SIMD disabled, so the
path is unconditional rather than behind a define.
A survey of all 36 audio_driver_t vtables found the drivers disagreeing
on what these three entry points mean, in ways that matter because the
frontend acts on the answers.
Two asymmetries make the disagreement consequential. audio_driver_stop()
refuses to call the driver at all when alive() reports false, so a driver
whose alive() lies about being stopped gets its stop() skipped. And
audio_driver_start() clears AUDIO_FLAG_ACTIVE when start() returns false,
so a driver that reports failure for a recoverable condition silences
audio for the rest of the session rather than for one call.
The contract the frontend actually relies on, now written at the vtable:
stop() and start() report the resulting state rather than the work done,
so calling either on a driver already in that state succeeds; alive()
agrees with the last successful stop() or start(); and start() returns
false only when the stream genuinely cannot be resumed, because that
answer is not a soft failure.
Two shapes of violation turned up. The first is a paused flag updated
only inside the branch that talks to the hardware, so alive() reports
the opposite of the truth on a device where that branch is skipped -
usually one without a hardware pause primitive, where stop() correctly
does nothing but then also fails to record that the driver is stopped.
The second is start() returning false when a pause capability the device
advertised turns out not to work, trading a missing pause for silence.
tinyalsa had both, and is fixed here: the flag is maintained on every
path, a refused pause or unpause warns and revokes the capability rather
than failing, and both entry points return early when already in the
requested state. With no hardware pause the driver simply stops feeding
the device, which is what the capability was worth in the first place.
Three drivers violate the contract and are deliberately not touched
here. alsa has both faults; the fix is entangled with the unbounded
snd_pcm_wait() it shares a cause with, and is held until that can be
tested on the hardware that reproduces the freeze. alsa_qsa (QNX) has
both faults in exactly tinyalsa's shape, and
switch_libnx_audren_audio.c has an alive() hardcoded to true so it never
reports the driver as stopped; neither compiles on a Linux host, and an
unbuildable behavioural edit is worse than a documented one.
psp_audio.c returns false unconditionally from stop() under ORBIS. That
is contract-violating but inert - nothing acts on a false from
audio_driver_stop() - and the reason for the guard is not recoverable
from the code, so it is left alone rather than changed blind.
The rest conform, including several that a mechanical scan flags as
suspicious but which are correct on inspection: coreaudio, ps2, ps3 and
psp assign their flag inside a null check or an already-in-state check
rather than inside a capability branch, which is the difference that
matters. wasapi.c is worth reading as the reference: on a failed
transition it returns whether the device is already in the requested
state, which is exactly what the contract asks for.
Verified by compiling every driver that builds on this host - alsa,
tinyalsa, oss, rsound, sdl, openal, jack, pulse, pipewire - plus
audio_driver.c and audio_thread_wrapper.c against the changed header.
jack and pipewire carry pre-existing warnings; the counts are unchanged
from master.
rpng_save_image_argb() takes a host-order uint32_t of 0xAARRGGBB. Every
caller holding a GL_RGBA / VK_FORMAT_R8G8B8A8 readback or texture-cache
surface holds the opposite word, 0xAABBGGRR, so today it has to allocate
a frame-sized scratch buffer and swizzle red and blue across it purely so
that copy_argb_line() can swizzle them back one line later.
RGBA is the on-wire channel order for PNG colour type 6, so that path
needs no per-pixel work at all - only the move into the filter scratch
line, which cannot be elided because the winning filter's tag byte is
written at chosen_filtered[-1] and would otherwise land one byte inside
the caller's surface.
bpp cannot select this: ARGB32 and RGBA32 are both four bytes per pixel
and differ only in channel order. The encoder worker therefore takes an
explicit enum rpng_pixfmt and derives bpp from it. rpng_save_image_stream()
stays as a bpp-taking wrapper - it is not declared in any header, but it
is not static either, so out-of-tree callers keep compiling and keep their
exact behaviour.
Measured, 1920x1080, mean of 5:
swizzle scratch + rpng_save_image_argb 764.5 ms
rpng_save_image_rgba (direct) 745.5 ms
19 ms, 2.5%. The encode is deflate-bound, so the time saved is small; the
8100 KB of scratch that no longer has to be allocated per image is the
part that matters for a texture cache dumping many images in sequence.
Verified byte-exact. rpng_save_image_rgba() and swizzle +
rpng_save_image_argb() produce identical files at 1x1, 2x3, 17x5, 64x64,
320x240 and 255x129, with positive and negative pitch, clean under ASan
and UBSan. An independent decoder confirms valid chunk CRCs, IHDR depth
8 / colour type 6 / non-interlaced, and pixels decoding back byte-identical
to the source surface. argb, bgr24, rgb48 (with and without HDR
metadata) and both _string entry points are byte-identical to pre-patch
output, so the dispatch change is behaviour-neutral.
retro_vfs_file_rename_impl used _wrename()/rename(), which cannot
replace an existing destination on Windows (no POSIX overwrite
semantics). That silently breaks every atomic save pattern of the
form write tmp -> rename over original, in the frontend and in every
core routing file I/O through the VFS: the rename fails, the new data
is stranded in the temporary, and the stale original is what loads
next session. This is the frontend-side half of the SwanStation
stale-save bug (saves left as .tmp on Windows).
The wide branch now uses MoveFileExW with MOVEFILE_REPLACE_EXISTING -
the Win32 spelling of the POSIX overwrite guarantee, and in the app
API partition so WinRT builds take it too. The ANSI branch cannot use
it (the flag is documented unsupported on 9x, and the Xbox XDK has no
MoveFileEx at all), so replace there is delete-then-rename: plain
rename first, which is the only path taken when the destination does
not exist, removing the destination only after it fails.
While here, finish the LEGACY_WIN32 migration for the last two
stragglers still keying the ANSI/wide split off
_WIN32_WINNT < 0x0500: retro_vfs_file_rename_impl and
retro_vfs_file_remove_impl. As the migration note in encodings/utf.h
lays out, that test was wrong in both directions - it forced ANSI on
Unicode-native NT4, and a real 9x build (which has no reason to set an
NT macro low; mingw-w64 defaults _WIN32_WINNT to 0x0A00) took the wide
branch on the exact platform the test existed to protect - where
MoveFileExW is a stub failing with ERROR_CALL_NOT_IMPLEMENTED. Both
functions now select like the rest of this file: LEGACY_WIN32_RUNTIME
probes per call, LEGACY_WIN32 is compile-time ANSI (Xbox/9x), default
is wide.
Verified with mingw-w64 -fsyntax-only across the full matrix (default
wide, _WIN32_WINDOWS=0x0410, LEGACY_WIN32, HAVE_WIN9X,
HAVE_WIN9X_RUNTIME - the last compiling both arms plus the probe) and
gcc on Linux; only comment text still mentions the old guard.
video_init_thread() sets *out_driver and *out_data before calling
video_thread_init(), and leaves both published when it fails. The
caller passes &video_st->current_video and &video_st->data, and
video_thread is a member of thread_video_t rather than a static vtable,
so current_video points into the allocation. Nothing clears it when
that allocation is freed: video_driver_free_internal() and
driver_uninit() clear data only, and current_video is not NULLed until
retroarch_deinit_drivers(). Probe, forcing a video init failure with
video_threaded = true:
[PROBE] thr=0x51d000001e80 vtable=&thr->video_thread=0x51d000002040
Threaded video stats: Frames pushed: 0 <- thr freed
[PROBE] after driver_uninit: current_video=0x51d000002040 data=(nil)
Restore drv (static, never freed) and NULL the data, matching the
non-threaded failure path; the probe then reports the static vtable.
Free with video_thread_free() rather than free(): init can fail at the
CMD_INIT round-trip with the worker thread running and frame.buffer
allocated at tens of MB. It handles every partial-init state and
clears VIDEO_FLAG_THREAD_WRAPPER_ACTIVE itself.
Found triaging libretro/libretro-prboom#208 (core-side cap fixed
separately), but this does not claim to fix that segfault: the dangling
pointer is confirmed, a dereference of it is not. An ASan build of the
repro reports no use-after-free, because nothing in that reduced path
calls through current_video in the window.
Verified: forced-failure repro before and after under ASan
(detect_leaks=1), no findings either way; success path unaffected; unix
build clean.
fill_pathname_join_special() was byte-identical to
fill_pathname_join() except for omitting the @s != @dir alias guard,
with a one-line header note ("@dir and @s CANNOT be identical")
carrying the whole contract. That precondition is the worst kind:
its violation is silent on the platforms developers build on (the
portable strlcpy_retro__ fallback tolerates exact aliasing) and
fatal only in the field on macOS, where strlcpy resolves to the
fortified __strlcpy_chk and aborts via __chk_fail_overlap - the
defect class of bd79aee/e942ee2, and e942ee2 fixed exactly one such
violation of this contract (the HAVE_LAKKA url_path join).
The guard it omitted is one pointer compare ahead of two strlcpy
calls and a strlen walk through fill_pathname_slash(): never a
measurable saving. Meanwhile the symbol is libretro-common API with
313 call sites in RetroArch alone and is vendored into downstream
cores that never run the overlap_copy_check CI gate, so caller-side
enforcement cannot protect the ecosystem - the safe semantics have
to live in libretro-common itself.
Removing the name outright was rejected (it would break every
downstream consumer at their next libretro-common sync). Instead
the prototype becomes a function-like macro in file_path.h
expanding to fill_pathname_join(), and the function body is removed
from file_path.c - the compat/strl.h pattern. Each macro argument
expands exactly once, so arguments with side effects are safe.
Preconditions checked: no code in the tree takes the function's
address or redeclares it, and fill_pathname_join_special_ext() is a
distinct preprocessor token the macro cannot touch (it already
routes through fill_pathname_join() and inherits alias-safety).
Version skew between file_path.h and file_path.c in a partial
vendor sync produces a clear link error rather than silent
misbehaviour; whole-directory syncs are unaffected.
The aliased HAVE_LAKKA call site keeps fill_pathname_join(); its
contract-explaining comment from e942ee2 is reduced to a brief
overlapping-buffers note now that the distinction is gone.
Partially overlapping (unequal) buffers remain undefined - the
equality guard matches the only aliasing shape meaningful for a
path join - and the docs say so explicitly.
Verification:
- equivalence harness (ASan+UBSan+LSan, system strlcpy via
-DHAVE_STRL): the macro and fill_pathname_join() produce
byte-identical strings and return values to the historical
distinct-buffer join_special body across 28 dir/path combinations
(trailing slash, leading slash, empty components, bare root), for
both distinct and aliased @s/@dir - 4 variants per combination
- existing fill_pathname regression sample passes under sanitizers
- file_path.c compiles under gcc -std=c89 -pedantic; nm confirms
the fill_pathname_join_special symbol is gone while
fill_pathname_join and fill_pathname_join_special_ext remain
- all 56 in-tree TUs calling the macro compile: 48 build in the CI
configure, 3 networking TUs (task_pl_thumbnail_download,
task_core_updater, webdav) additionally syntax-checked with
-DHAVE_NETWORKING; the remainder are platform-SDK-gated
(QNX/CTR/SDL3/connman) with plain call syntax
- overlap_copy_check: clean (274 helpers tracked, 37 allowlisted)
TSan: not applicable (no concurrency change).
* sdl3: Add OpenGL and Vulkan contexts
* sdl3: Fix comment, style and build gate
* sdl3: Flush unconsumed SDL events
* sdl3: Vsync reprobe and monitor fallback
* sdl3: Clean up code comments
Co-authored-by: Rob Loach <[email protected]>
* sdl3: Simplify the GL context driver
* sdl3: Fall back when cached ctx fails
* sdl3: Small cleanups
* sdl3: Clean up comments
* sdl3: Fix context list order, and gl1 shader flag
* sdl3: Add sdl3_ctx_enabled() to allow detection if SDL3 ctx should be used
* sdl3: Fix extension truncation by moving to 16 available slots
* sdl3: Add input driver passthrough
* font: Fix stale OSD font crash
The Windows build emits 35 -Wsign-compare warnings from the bundled
libsmb2. Most are cosmetic, but one is not, so they are worth clearing
rather than muting.
smb2-cmd-query-directory.c, both loops in
smb2_encode_query_directory_reply(), is an actual bug:
in_remain = fslen;
do {
...
in_remain -= PAD_TO_64BIT(sizeof(struct smb2_fileidbothdirectoryinformation));
} while (in_remain >= sizeof(struct smb2_fileidbothdirectoryinformation));
in_remain is int and sizeof is unsigned long long, so the comparison
converts in_remain to unsigned. The moment it goes negative - which it
does whenever fslen is not an exact multiple of the padded entry size -
it reads as a huge positive value, the condition stays true and the loop
keeps walking past the end of the buffer. Comparing against
(int)sizeof(...) makes it signed and the loop terminates.
The rest are type mismatches with no behaviour change:
- libsmb2.c, sync.c: NTSTATUS values are 32-bit unsigned and the
SMB2_STATUS_* constants above 0x7fffffff have unsigned int type, but
smb2_command_cb carries status in an int. The comparisons were
already being done in unsigned arithmetic by the usual conversions;
the casts just say so. Changing the callback signature would be an
API break across the whole library, so the casts stay at the
comparison sites.
- dcerpc.c, dcerpc-lsa.c: loop counters declared int while walking
uint64_t/uint32_t counts that had already been used to size the
matching allocation. Widened to match the count. For counts above
INT_MAX the old code was signed overflow, so this is also a small
correctness win.
- ntlmssp.c: alloc_len held in an int and compared against uint32_t
offsets. Made uint32_t, which is what the two assignments and both
malloc() calls already treat it as.
- smb2-data-file-info.c: name_len in
smb2_encode_file_normalized_name_info() made uint32_t to match
fs->file_name_length; the two comparisons in
smb2_decode_file_stream_info() get explicit (size_t) casts spelling
out the conversion that was already happening; the tv_usec sentinel
check is cast to unsigned long, which preserves today's behaviour on
both 32-bit and 64-bit long.
- socket.c: smb2_accept_connection_async() takes an int fd but
SMB2_INVALID_SOCKET is SOCKET-typed on Windows. Cast at the
comparison. Worth noting separately that taking a Windows SOCKET
through an int truncates on Win64 - that is a public prototype in
libsmb2.h and is left alone here, and RetroArch does not use the
accept path.
Deliberately not touched: the decode-side arithmetic in
smb2_decode_file_stream_info() still computes offsets in int from
network-supplied values. The casts silence the diagnostic and preserve
current behaviour, they do not harden the parser.
Verified with x86_64-w64-mingw32-gcc 13: all 55 sources in
Makefile.common's builtin libsmb2 object list go from 35 -Wsign-compare
warnings to 0, with 0 failures. A full Linux ./configure --enable-libsmb
rebuild links clean and produces an identically sized binary. Note the
53 -Wunknown-pragmas warnings my harness also showed are not present in
real builds - Makefile.common already passes -Wno-unknown-pragmas.
Worth sending upstream to libsmb2, the query-directory loop especially.
Building the bundled libsmb2 on MSYS2/MinGW64 fails:
./deps/libsmb2/include/asprintf.h:32:19: error: static declaration of
'vasprintf' follows non-static declaration
C:/msys64/mingw64/include/stdio.h:298:5: note: previous declaration of
'vasprintf' with type 'int(char **, const char *, char *)'
./deps/libsmb2/include/asprintf.h:36:13: error: implicit declaration of
function '_vscprintf_so'
./deps/libsmb2/include/asprintf.h:55:19: error: static declaration of
'asprintf' follows non-static declaration
Two problems, one cause.
MinGW-w64 declares asprintf() and vasprintf() in <stdio.h>, gated on
_GNU_SOURCE at all four of its declaration sites. lib/libsmb2.c defines
_GNU_SOURCE itself before it includes anything, so on that toolchain the
declarations are always in scope by the time asprintf.h is reached, and
the static inline definitions in the header collide with them. The
existing "#ifndef vasprintf" / "#ifndef asprintf" guards cannot catch
this, because those names are functions and not macros.
The header did already carry a !defined(__MINGW32__) exclusion, but only
on the _vscprintf_so() helper, not on the two fallbacks that call it. So
on MinGW the helper was suppressed while its callers were still emitted,
which is where the implicit declaration comes from - the build was broken
in both directions at once.
Gate the helper and its users on a single condition so the two halves
cannot drift apart again: when MinGW's libc supplies asprintf() and
vasprintf(), use those; otherwise define our own exactly as before.
Scope is narrow. asprintf.h is only included by lib/libsmb2.c, and only
under _WIN32 / _XBOX / __AROS__, which is why Linux builds never hit
this. MSVC is unaffected - it does not define __MINGW32__ and still gets
the fallbacks, including the _vscprintf_so() rename that exists for its
sake. _XBOX and __AROS__ paths are unchanged.
Verified with x86_64-w64-mingw32-gcc 13 (mingw-w64 11): before the change
lib/libsmb2.c fails as above, after it compiles clean, and all 55 sources
in Makefile.common's builtin libsmb2 object list compile with 0 failures.
Checked both with and without -D_GNU_SOURCE on the command line, so the
fallback path is exercised too. On Linux the header is not reachable and
obj-unix/release/deps/libsmb2/lib/libsmb2.o still rebuilds clean.
Note this is a local change to a vendored dependency and is worth sending
upstream to libsmb2 as well.
SMB support was already implemented, but reaching it from the qb
configure system required knowing which of the two providers to ask
for: --enable-smbclient hard-requires a system libsmb2 via pkg-config
and errors out when one is not installed, while --enable-builtinsmbclient
selects the copy bundled in deps/libsmb2. There was no single switch
that just says "build SMB in".
Add HAVE_LIBSMB / --enable-libsmb as an umbrella switch that resolves
to a concrete provider:
- if pkg-config found a system libsmb2, that is used;
- otherwise the bundled deps/libsmb2 is built in.
The existing --enable-smbclient / --enable-builtinsmbclient switches are
untouched and still pin a specific provider for packagers who need one.
--enable-libsmb layers on top of them, so the combinations behave
sensibly:
--enable-libsmb --disable-smbclient
skips the system library and builds the bundled one
--enable-libsmb --disable-builtinsmbclient
errors out when no system libsmb2 is present, instead of
silently producing a build without SMB
--enable-libsmb --disable-networking
errors out, matching the other networking-dependent features
The switch defaults to no, so the default configure output is unchanged
apart from an inert "HAVE_LIBSMB = 0" line in config.mk. Makefile.common
needs no changes: it already keys the -DHAVE_SMBCLIENT define, the
network/cloud_sync/smb.o and vfs_implementation_smb.o objects and the
deps/libsmb2 object list off HAVE_SMBCLIENT / HAVE_BUILTINSMBCLIENT.
Verified on Linux/gcc: ./configure --enable-libsmb followed by a full
build links retroarch with the bundled libsmb2 statically built in, and
all four switch combinations above resolve as described. Both modified
scripts pass a POSIX shell syntax check under dash.
retro_vfs_dirent_get_name_impl() decoded each entry's name and wrote the
UTF-8 back over the find-data's own cFileName, on the strength of a
comment claiming that buffer "is always at least as large as the name it
held". That holds only for ASCII, and three bugs followed.
cFileName is 520 bytes on the W path and 260 on the A path, but UTF-8
needs up to 3 bytes per BMP unit, so a 174-character CJK or Thai name
overflowed the W buffer and strlcpy() silently cut it (CP1251 went at
~130 on the A path). NTFS allows 255, so files really could appear
under the wrong name. Overwriting cFileName also destroyed the source
encoding, making a second call on the same entry return garbage -
nothing calls it twice today, but nothing said so either. And both
paths allocated, memset and freed for every entry in the listing.
One scratch buffer on the handle fixes all three. MAX_PATH * 3 + 1
cannot truncate - a surrogate pair costs 2 units for 4 bytes, so 3 bytes
per unit bounds every case - and the source stays read-only.
utf16_conv_utf8() writes into a caller buffer, so the W path now
allocates nothing; utf16_to_utf8_string_alloc() and utf16_to_char_string()
both malloc internally. But utf16_conv_utf8() and WideCharToMultiByte()
disagree on malformed input: the first stops at an unpaired surrogate,
the second substitutes U+FFFD and carries on. NTFS permits unpaired
surrogates, so taking the first unconditionally would truncate such a
name, in the worst case to nothing. Its return value therefore gates a
fallback to the allocating form, which costs an allocation only for
names that are already broken.
For the A path, add local_to_utf8_string() beside the _alloc form in
encodings/. It carries the same guard, so where that guard resolves to
a plain copy - Xbox, and any UNICODE build - there is nothing to convert
and now nothing to allocate either; the old code ran strdup, strlcpy and
free per entry to achieve nothing on those targets. Where conversion is
real it delegates unchanged. Keeping the guard in encodings/ rather
than restating it in the VFS layer is the point: the VFS layer should
not have an opinion about which codepages need converting.
One behaviour change falls out: the A path returned NULL for an empty
name, and now returns "". dir_list_read_ctx() dereferences this without
a NULL check, so "" is the safer of the two, and it matches what the W
path does.
Verified on mingw-w64 x86_64 and i686 across default/UNICODE,
LEGACY_WIN32, LEGACY_WIN32_RUNTIME, VFS_FRONTEND and LEGACY_WIN32 with
UNICODE, both translation units, warning-clean at -Wall -Wextra; 20
configurations. The converters, extracted verbatim into a harness, pass
ASan+UBSan+LSan and TSan against max-length ASCII/Cyrillic/CJK/Thai,
surrogate pairs, lone surrogates and 200k randomised names; the pre-fix
logic and a version without the surrogate fallback both fail it.
local_to_utf8_string() is covered separately for NULL and zero-length
destinations, NULL and empty sources, and strlcpy-shaped truncation.
The real file is clean over a 4002-entry tree, single-threaded and 6-way
concurrent, under all four sanitizers. Built for Win32 and run under
wine, the W path matches WideCharToMultiByte byte for byte on all eleven
direct cases - 259 CJK at 777 bytes, 259 Thai, 129 surrogate pairs, and
unpaired surrogates at every position including one at offset 250 of a
long name - and both the W and the forced-LEGACY_WIN32 A path list a
real directory correctly. MAX_PATH * 3 + 1 was measured against
sizeof(cFileName) rather than assumed: 781 on both forms.
Not covered: MSVC, and sanitizers do not run on Win32, so the Windows
path has cross-compilation, wine and the harness but no sanitized
Windows run. Xbox and 9x are reasoned about, not executed.
GCC warns that 'current_inputs' may be used uninitialized at the
BIT256_GET_PTR(p_new_state, j) expansion in the mapper switch of
input_driver_poll():
./libretro-common/include/retro_miscellaneous.h:362:40: warning:
'current_inputs' may be used uninitialized [-Wmaybe-uninitialized]
input/input_driver.c:7550:25: note: in expansion of macro
'BIT256_GET_PTR'
input/input_driver.c:7361:23: note: 'current_inputs' declared here
The read is in fact safe: the mapper switch only touches p_new_state
for RETRO_DEVICE_KEYBOARD/JOYPAD/ANALOG, which are exactly the cases
in which the preceding device switch runs BIT256_CLEAR_ALL_PTR() on
it; every other device falls through both default labels. GCC cannot
correlate two separate switches on the same value, and the state is
written and read through the p_new_state alias rather than directly,
so it conservatively assumes the clear may be skipped.
Hoist the clear out of the device switch case so 'current_inputs' is
zeroed unconditionally right after it is declared. The dominating
store removes the warning, and the object is now defined on every
path through the remap block rather than only on the paths that
happen to consume it. For KEYBOARD/JOYPAD/ANALOG the work is
identical - the same memset, just moved a few lines up; other device
types (mouse, lightgun, pointer, none) now pay one 80-byte clear per
user per poll when remapping is enabled, which does not show up
against the input_state_wrap() calls in the same loop.
No functional change.
MSYS2 updated mingw-w64-ffmpeg to 9.0, which removes the deprecated
AVCodec.sample_fmts / AVCodec.supported_samplerates arrays and breaks
the MSYS2 CI build in record_ffmpeg.c.
record_ffmpeg: add accessors that use avcodec_get_supported_config()
where available (lavc >= 61.13.100, FFmpeg 7.1+) and fall back to the
legacy AVCodec struct members on older versions. NULL now means
unrestricted instead of being dereferenced, hardening both paths.
camera/ffmpeg: version-gate const-ness of the AVInputFormat/AVCodec
pointers; lavf 59 (FFmpeg 5.0) made that API const-correct, lavf 58
(FFmpeg 4.x, the driver's minimum) warns on const pointers.
Verified: compile matrix with production flags against FFmpeg 3.4.13,
4.4.5, 5.1.6, 6.1.2, 7.1.1, 8.1.2 and 9.0 all clean; full configure +
make against FFmpeg 9.0 builds and links; resolver logic runtime-tested
under ASan/UBSan against FFmpeg 9.0 (new path) and 6.1.2 (legacy path)
with identical results and no sanitizer findings.
dir_list_read_ctx() calls retro_dirent_is_dir() once per entry, and on an
smb:// listing that landed in retro_vfs_dirent_is_dir_smb(), which rebuilt
the full path and called retro_vfs_stat_smb() -> smb2_stat(). smb2_stat()
is libsmb2's blocking sync wrapper (smb2_stat_async() + wait_for_reply()),
so every entry cost a serialized CREATE/QUERY_INFO/CLOSE round trip to the
server. A 4000-file share at ~6.5ms RTT took ~26s to list.
The information was already on hand. smb2_opendir() performs the whole
QUERY_DIRECTORY enumeration up front and libsmb2 attaches a populated
smb2_stat_64 to each entry, so retro_vfs_readdir_smb() can and does fill in
smbc_dirent::type for free. retro_vfs_readdir_impl() then discarded it,
copying only de->name into rdir->smb_path, and a few lines later we went
back to the server to ask a question we had just been handed the answer to.
Cache the type on the RDIR handle at readdir time and answer
retro_vfs_dirent_is_dir_impl() from it. That drops the entire per-entry
round trip; browsing cost collapses to the single smb2_opendir()
enumeration we were already paying for.
retro_vfs_dirent_is_dir_smb() has no callers left and is removed.
retro_vfs_stat_smb() is untouched and still serves retro_vfs_stat_impl(),
which has no dirent to read from.
smbc_dirent::type was an undocumented 0/1 with a single producer. Now that
a second site consumes it, give it RETRO_SMB_DIRENT_FILE/_DIR names: if the
field is ever extended toward libsmbclient's SMBC_* numbering, where 1 is
SMBC_WORKGROUP rather than a directory, a bare truthiness test in the new
consumer would silently misclassify every entry.
Not addressed here: get_smb_context() round-robins over the context pool,
so the four connections provide no parallelism for any remaining
synchronous path. That is moot for directory listing once the per-entry
stat is gone, and fixing it properly means async fan-out.
Reverts, newest first:
- 05db1e076 "smb2: rebuild the pool when settings change, move cloud
sync onto it, fix signing negotiation"
- cea330ebe "smb2: allocate nothing at all when no SMB server is
configured"
- 3c7447d9d "smb2: serialize context use, replace cached context
pointers with generation handles"
- 36766173e "smb2: publish loaded settings to the SMB backend, harden
the pool paths"
05db1e076 was not part of the original series but depends on it
(smb_cfg publication, smb_shutdown(), generation handles), so it has
to go with the other three; a three-commit revert does not apply
cleanly and would not compile.
All touched files are restored byte-exact to their state at 558b24a68,
the parent of the series. Unrelated later work in configuration.c
(282ca43c0, gl_direct_spirv setting rows) and vfs_hybrid.c (c46b14a79)
is untouched. No in-tree references to the removed pool API
(smb_pool_acquire/release/heal_if_dead) remain; retroarch_smb_init and
struct smb_settings predate the series (e9f9dd443) and stay.
Note this reintroduces the issues the series addressed, including the
signing-mode negotiation fix and the late publication of the numeric
SMB settings fields.
strdup is POSIX, not C89: under a strict-C89 toolchain string.h hides
the prototype, the call becomes an implicit declaration returning int,
and on LP64 the pointer is truncated. It compiled quietly so far only
because every current consumer builds with gnu language modes. Caught
by the gcc -std=c89 -pedantic gate while wiring vfs_hybrid.c into the
frontend build.
Copy manually with malloc+memcpy from the headers already included.
NULL-on-OOM semantics for f->path are unchanged from strdup.
Mirrors the identical fix submitted to upstream libretro-common.
Verification: gcc -std=c89 -pedantic -Wall -Wextra compile, zero
warnings. ASan+UBSan+LSan and TSan harness (hybrid install via fake
frontend VFS, 1000x open/read/close cycles through the front door
exercising the path copy alloc/free, plus the no-frontend init path):
clean. TSan run is single-threaded; no concurrency is touched by this
diff.
Cores are adopting vfs_hybrid_init() in place of wholesale
filestream_vfs_init()/dirent_vfs_init() adoption. On STATIC_LINKING
platforms cores exclude their bundled libretro-common and resolve
against the frontend's copy - which provided every symbol the old
adoption pattern needed, but not vfs_hybrid_init, so statically linked
cores calling it fail at final link (seen on ecwolf across all ten
static targets: psp, ps2, vita, ctr, ngc, wii, wiiu, libnx, djgpp,
emscripten).
Add vfs_hybrid.o to Makefile.common and the matching include to
griffin.c, placed directly after vfs_implementation to keep griffin
order aligned with the object list. Compiled unconditionally: the
hybrid only references the public retro_vfs_*_impl API, and all 16
impl symbols it uses are also provided by vfs_implementation_uwp.cpp,
so no __WINRT__ guard is needed. The frontend itself never calls
vfs_hybrid_init; the object exists so statically linked cores resolve
it, same as other core-support facilities. Cores that ship their own
vfs_hybrid.o in the archive coexist fine: the frontend object wins
resolution and the archive member is never extracted, so there is no
duplicate-symbol hazard in either direction.
Verification: gcc -std=c89 -pedantic -Wall -Wextra compile of
vfs_hybrid.c, zero warnings; ASan+UBSan+LSan and TSan harness per the
preceding commit, clean. Build-system wiring itself alters no code
paths in the frontend.
C89 7.9.5.3: on an update stream, output followed by input - or input
followed by output short of EOF - requires an intervening file
positioning or flush call. filestream_read() directly after
filestream_write() (and the reverse) never inserted one, a latent
defect predating the lookahead: whether it bit depended on the libc
and the stdio buffer configuration behind the VFS. The r+
interleaved-write shadow harness caught it immediately when pointed
at an older vendored VFS tree (FBNeo's), where a gets after a write
returned stale pre-write bytes.
Track the last data direction in RFILE and interpose a zero-byte
SEEK_CUR on transitions, at this layer so every backend is covered -
local stdio, fd, mapped, and frontend-supplied VFS callbacks alike.
A successful seek or flush from any path clears the state, so the
lookahead discard's rewind doubles as the read-to-write transition
seek and the common sequential paths pay nothing. Cost is one no-op
seek per direction change, only on update streams that actually
alternate.
Verification: conformance harness re-run on this tree - 50,160,118
checks with HAVE_MMAP, 50,160,380 without, 0 failures - and against
FBNeo's older vendored VFS combination where the failure reproduced
before this fix (now 50,160,118 clean). ASan+UBSan+LSan clean in
both mapping configs; TSan clean. C89 pedantic adds no warnings
over baseline. Benchmark medians unchanged (n=15 alternated,
260.9ms upstream gets loop vs 12.3ms buffered): the read-only
sequential path never transitions.
bd79aee and e942ee2 fixed three instances of the same defect class:
copy calls whose source and destination overlap. Nothing in the
existing CI can catch this class - the portable strlcpy_retro__
fallback tolerates exact aliasing on Linux/Windows, so the fortified
__strlcpy_chk abort only ever fires on a user's Mac, and overlapping
memcpy usually "works" until a memcpy direction change or sanitizer
run says otherwise. This adds a static gate so the class cannot
regress.
tools/overlap_copy_check.py runs two passes over the tree (~19s,
1739 files):
1. Direct: strlcpy/strlcat/strcpy/strncpy/strcat/strncat/memcpy
calls whose dst and src normalize to the same base buffer
expression (exact alias or same base at different offsets), and
sprintf/snprintf calls whose destination reappears as a format
argument. memmove is the sanctioned fix and is never flagged.
2. Helper-mediated: function definitions whose body copies from one
pointer parameter to another via those primitives WITHOUT an
alias guard (no dst==src / dst!=src comparison between the two
parameters in the body), cross-referenced against every call site
passing the same base expression for both - including
function-pointer call spellings such as (ozone->word_wrap)(...),
which is the exact form of the OSK bug. Guarded helpers such as
fill_pathname_join()/_join_delim()/_parent_dir() are recognized
as safe and their in-place callers are not flagged.
Audited-intentional same-buffer copies (overlap-guarded LZ match
runs, disjoint element/row/member copies, single-element shift
loops, one-stride row duplication) are recorded in
tools/overlap_copy_allow.list as file|func|dstbase|srcbase keys -
line-number-free so the list survives churn, with stale entries
reported so it cannot rot. Exact-alias findings can never be
allowlisted. The current list is exactly the 37-key residue of the
e942ee2 hand audit; the tree checks clean against it.
Regression demonstration - the checker flags all three fixed bugs
when run over the pre-fix trees:
ozone.c:7383: word_wrap(cursor_message, cursor_message)
[helper-aliased] (bd79aee)
gfx_widget_leaderboard_display.c:475/576:
memcpy(&info[i], &info[i + 1]) [same-base] (e942ee2)
menu_cbs_ok.c:5443: fill_pathname_join_special(url_path,
url_path) [helper-aliased] (e942ee2)
.github/workflows/Overlap-Copy-Check.yml wires it up in the
Settings-Checks style: fixture self-test first (exact-alias flagged,
overlapping shift flagged, memmove ignored, unguarded helper flagged
while guarded helper is not, allowlist honored but never for exact
aliases, stale entries reported), then the tree check.
Verification: self-test passes; tree check clean and byte-identical
across repeated runs (deterministic ordering); workflow YAML parses;
checker output verified against the by-hand audit results from
e942ee2 (same finding set). Python tool - sanitizer sweep not
applicable; no C sources touched.
Follow-up to bd79aee (in-place word_wrap OSK crash): audited every
strlcpy/strlcat/strcpy/strncpy/memcpy/sprintf call in the tree for
overlapping source/destination, both directly and through helper
functions that copy from one parameter to another. Two defects found:
1) gfx_widget_leaderboard_display.c: removing a tracker/challenge
entry shifted the remainder of the array down with
memcpy(&info[i], &info[i + 1], (count - i) * sizeof(info[i]));
which copies between overlapping regions whenever more than one
element moves (dst == src - stride). This is undefined behaviour
and is reported by ASan as memcpy-param-overlap; converted both
sites to memmove(). All other element-shift loops in the tree
(netplay chat history, cheat list insert/delete) move a single
element per iteration in the correct direction and are fine.
2) menu_cbs_ok.c (HAVE_LAKKA): MENU_ENUM_LABEL_CB_LAKKA_LIST called
fill_pathname_join_special(url_path, url_path, ...);
violating the documented contract ("@dir and @s CANNOT be
identical" - the helper strlcpy()s @dir into @s unguarded), the
same fortified-strlcpy abort class as the OSK crash. Switched to
fill_pathname_join(), which explicitly supports @s == @dir and is
otherwise identical; verified byte-identical output and return
values for the aliased call across representative dir/path
combinations.
Audit notes for everything else flagged and cleared:
- fill_pathname_join()/_join_delim()/_parent_dir() carry alias guards;
their in-place callers (runloop save dirs, cheevos badge path,
platform_gx, task_database, parent-dir navigation) are safe.
- LZ-style self-copies in r7z_lzma, encoding_vcdiff, encoding_rlz4 and
tasks/task_patch (BPS TARGET_COPY) are deliberate overlapping runs
with explicit disjoint/periodic-chunk guards.
- Decoder-internal same-buffer copies (ropus SILK resampler and LPC
rewind, rmp3 L12 scf spread, raac SBR persistence, rvp8 reference
rotation, rhmap slot compaction, ntsc_crt/sdl_rs90 row duplication,
ctr_svchax cache flush) are disjoint by construction or mirror
upstream reference implementations.
- No direct strlcpy/strlcat/sprintf calls with aliased arguments
remain anywhere in the tree.
Sanitizer sweep (ASan+UBSan+LSan):
- old leaderboard shift pattern reproduces
"AddressSanitizer: memcpy-param-overlap"; memmove version verified
element-exact against expected post-removal contents
- fill_pathname_join(buf, buf, path) output verified byte-identical
(string and return value) to fill_pathname_join_special() with
distinct buffers over dir/path edge cases including trailing slash
and empty components
- gfx_widget_leaderboard_display.c and menu_cbs_ok.c compile clean;
HAVE_LAKKA branch additionally syntax-checked with gcc
TSan: not applicable (no concurrency change).
ozone_draw_osk and the OSK caret rendering in xmb/materialui all
called word_wrap() with the same buffer as source and destination:
(ozone->word_wrap)(cursor_message, sizeof(cursor_message),
cursor_message, strlen(cursor_message), ...);
word_wrap()/word_wrap_wideglyph() copy via strlcpy(), whose buffers
must not overlap. On macOS, compat/strl.h force-defines HAVE_STRL, so
the system strlcpy is used; with _FORTIFY_SOURCE this resolves to
__strlcpy_chk, which aborts on any overlap (including exact aliasing)
via __chk_fail_overlap. As a result, typing a single character into
any OSK field (e.g. the IP Address setting under SMB Network
Settings) hits word_wrap's short-string early return
`strlcpy(s, src, len)` with s == src and raises EXC_BREAKPOINT.
Crash frame: len=2048, src_len=1, line_width=84, s == src == "1";
video_thread_loop -> ozone_draw_osk -> word_wrap -> __strlcpy_chk ->
__chk_fail_overlap. Linux/Windows never crashed only because the
strlcpy_retro__ fallback tolerates exact aliasing by accident.
In addition, in-place use of word_wrap_wideglyph() silently corrupts
CJK OSK text on all platforms: the lastwideglyph wrap branch
overwrites a source byte with '\n' before rewinding src onto it.
In-place wrapping of the kana string
E3 81 82 .. E3 81 93 (10 glyphs, line_width 4, wideglyph_width 200)
produces glyph 1 + 24 bare newlines + glyph 10 instead of one glyph
per line.
Fix: stage the OSK text in a separate cursor_src buffer at all three
call sites and wrap into the destination, matching the tmpstr pattern
already used for ozone's sidebar metadata. Document the
non-overlapping-buffers requirement for word_wrap()/
word_wrap_wideglyph() in stdstring.{c,h}. All other in-tree callers
(rgui, gfx_widgets, message boxes, sublabels) already use distinct
buffers.
Sanitizer sweep (ASan+UBSan+LSan, gcc, system strlcpy via
-DHAVE_STRL on glibc 2.39):
- fixed caller pattern yields "1" for the crashing input, clean
- separate-buffer output verified byte-exact against the previous
in-place output for ASCII inputs across line widths 5/10/30/84
(no rendering change for existing users)
- libretro-common word_wrap_wideglyph overflow regression tests pass
- ozone.c, xmb.c, materialui.c, stdstring.c compile clean
TSan: not applicable (single-threaded string path, no concurrency
change).
filestream_gets() fetched one byte per filestream_getc() call, and
each of those was a full crossing of the VFS interface - for a core
running under a frontend-supplied VFS, an indirect call across the
libretro ABI per byte of text. Line-scanning large files this way is
what makes FBNeo's 13MB cheat.dat load take seconds on Android
(#19372).
Give RFILE a lazily allocated lookahead buffer (FILESTREAM_RBUF_LEN,
16KB, overridable) that only filestream_gets()/filestream_getc() ever
fill. gets() finds the line bound with memchr() over the buffered
span and copies it out in one go; getc() serves single bytes from the
same buffer. Every other operation keeps the logical stream position
exact:
- read() drains unconsumed lookahead first, then reads through. If
the follow-on raw read fails after bytes were already delivered,
the short count is returned with err_flag set, as a partially
satisfied stdio read would.
- tell() answers from the buffer bookkeeping without a VFS call.
- seek() translates SEEK_CUR by the unconsumed span and DISCARDS the
lookahead on success rather than serving targets inside it. This
deliberately matches stdio, where fseek() drops the read buffer:
rewind-and-reread of synthetic files (procfs/sysfs pollers such as
the linux illuminance sensor) must return fresh bytes, not cached
ones. A failed seek leaves the buffer, and therefore the observed
position, untouched.
- write()/truncate()/flush() rewind the underlying handle over the
unconsumed span before proceeding, so r+ interleaving stays
byte-exact.
Handles that never touch gets/getc never allocate the buffer, and the
whole change is internal to RFILE - no API or ABI change, so every
core picks it up through its vendored libretro-common at resync with
no call-site changes. Code holding the raw handle from
filestream_get_vfs_handle() (cdrom, disc tasks) bypasses the layer,
and never mixes it with gets/getc on the same RFILE in-tree.
When the handle carries a file mapping (FREQUENT_ACCESS hint or a
frontend mapped-ptr callback), gets() instead memchr()s the mapping
directly and copies exactly the line - zero intermediate copy - as
proposed by asieveking in #19372. rfopen() keeps
RETRO_VFS_FILE_ACCESS_HINT_NONE: mapping everything a core fopens is
not free on 32-bit targets (the reporter measured 1.85GB peak VA on
Android with wholesale mapping), and the lookahead already removes
the per-byte cost for unmapped handles, so the mapping stays an
opt-in fast path rather than a default.
Also fixed: filestream_gets(stream, s, 0) decremented the size_t
length past zero and wrote unbounded output; it now returns NULL like
fgets() with n <= 0.
Two behaviors documented while testing, both pre-existing and
unchanged: a FREQUENT_ACCESS mapping permits seeking past EOF but its
read then returns -1 where the fd/stdio paths return 0; and glibc's
fgets() and fortified __fgets_chk() disagree with each other at
n == 1 (mid-file s vs NULL), so the n == 1 contract is asserted
natively (returns s, stores the NUL, consumes nothing - identical
before and after this change).
Verification: conformance harness mirroring stdio ground truth over
random op interleavings (gets/getc/read/seek/tell), sequential
whole-file gets sweeps at lens {2,3,4,17,80,65536}, and an r+
interleaved-write shadow test with final byte-exact file compare,
across 6 corpora (text, binary with NULs, no trailing newline, empty,
200KB single line crossing fills, newlines-only) x hints
{NONE, FREQUENT_ACCESS}. Harness first validated against the
pristine implementation: 50,160,106 checks, 0 failures. Patched:
50,160,118 checks, 0 failures with HAVE_MMAP, 50,160,380 without.
ASan+UBSan+LSan clean in both mapping configs; TSan clean. C89:
gcc -std=c89 -pedantic -Wno-long-long adds no warnings over the
pre-existing baseline.
Benchmark (25MB, 259,381 lines, filestream_gets(512) to EOF, n=50
alternated runs, medians): 260.6ms before, 11.9ms after (21.8x);
with the FREQUENT_ACCESS mapping 11.8ms. Buffer-size sweep 4K/16K/
64K all land within 1.7ms of each other on a local filesystem - the
size only matters where the per-call cost is high (frontend VFS
across the ABI, Android SAF), which is what 16KB is sized for.
Adds video_gl_direct_spirv, a boolean gating the GL_ARB_gl_spirv path in
gl3_spirv_binary_supported(). The row only appears when glcore is the
running video driver, since the extension is an OpenGL (Core) feature and
the entry would be inert anywhere else.
Worded as a hint on purpose. The extension may be absent, and a shader
whose SPIR-V cannot be lowered to the OpenGL environment falls back to
cross compilation on its own, so switching this on is a request rather
than a guarantee. The sublabel names the extension and says as much.
The setting is read on every call rather than latched alongside the
capability probe, so toggling it takes effect on the next preset load
without a driver reinit.
Defaulting to off, on the strength of the numbers rather than caution:
measured on llvmpipe over 146 presets, the SPIR-V path builds programs
about 4x faster against a cold driver shader cache, but Mesa's on-disk
cache serves the cross compiled path and not the SPIR-V one, so once the
cache is warm the GLSL path drops to ~11ms per program while SPIR-V stays
at ~120ms. That is a ~4x median regression on every load after the first,
which is the common case for a user reloading presets. Whether it holds
for other drivers is unknown, so this stays opt-in until there is data to
key a per-driver default off. RETROARCH_GL3_DISABLE_SPIRV still forces
the old path regardless of the setting.
Wired through the single-source descriptor mechanism:
settings/settings_def_video_gl_direct_spirv.h feeds configuration.c,
msg_hash.h, intl/msg_hash_us.{c,h} and menu/menu_setting.c.
tools/settings_table.reference regenerated.
Verified: settings_check.py table and unity both clean; the three enums,
both strings and the config key all appear after preprocessing;
menu_setting.c compiles both with and without HAVE_OPENGL_CORE/HAVE_SLANG
with no unused-const-variable warning when the guard is off; gl3.c error
counts identical to HEAD for desktop and GLES3.
Three related changes to how SMB connections are owned:
1. Settings changes now take effect without a restart. The pool
snapshots the parameters it was built with (server, share,
credentials, workgroup, timeout, context count, auth mode - subdir
excluded, it only affects path building and reads the live
configuration). A publication whose parameters differ tears the
pool down; the next smb:// access rebuilds it lazily with the new
values, and an identical republication leaves a live pool
untouched. Publications run on the main thread and smb_shutdown()
takes every lock, so in-flight operations finish first and their
generation handles fail closed - the rebuild rides entirely on the
machinery the pool-locking commit introduced. Combined with
publish-on-save, the full menu flow now works end to end: change a
setting, save, and the next SMB access uses it.
2. The cloud sync SMB driver used to connect a private context per
sync session and hold it in unlocked driver state. It now draws
from the shared pool: smb_pool_acquire()/smb_pool_release() hand
out a connected context under its slot lock per operation, and
smb_pool_heal_if_dead() gives cloud sync the same
heal-once-and-retry policy the VFS operations use at each
operation's first server contact - previously a transport drop
simply failed the whole sync cycle. Per-session connection setup,
the driver's duplicate parameter plumbing, and its unsynchronized
context field are all gone; connection ownership has exactly one
home. Acquisition also prefers idle slots now (slock_try_lock
across the pool, blocking on the round-robin slot only when every
slot is busy), so a long cloud sync transfer holding one slot does
not stall unrelated VFS operations that round-robin onto it.
3. smb2_set_security_mode() takes the SMB2_NEGOTIATE_SIGNING_* flags,
but the VFS backend passed the SMB2_SEC_* authentication constants
into it, so selecting Kerberos (2) silently demanded
signing-required (2) and the auth probe's first attempt did the
same. Signing negotiation and authentication are independent:
signing is now always ENABLED (the cloud sync driver already did
this correctly) and the authentication call carries the auth mode.
Verified offline before commit:
- Sanitizer harness extended with: identical republication leaving a
live pool untouched (zero reconnects), a changed parameter tearing
it down and the next access rebuilding with the new values, clearing
the server shutting SMB down entirely, signing always negotiated
ENABLED with authentication carrying the configured constant
(including KRB5, where the old coupling requested REQUIRED), and the
public acquire/heal/release API single-threaded and inside the
8-thread hammer with heal chaos. ASan+UBSan+LSan clean and TSan
clean across all of it; the negative test (locks compiled out) still
fails immediately under TSan.
- Full Makefile build with --enable-builtinsmbclient, threads,
networking and cloud sync compiled in; clean link, no new warnings
outside vendored deps. Settings table drift check, griffin-order
unity compile, and the migration tool self-test all clean.
The stale-handle behavior on rebuild is the generation design working
as intended: streams opened before a settings change fail closed on
their next operation and callers reopen against the new pool, which is
the correct semantic for "the server I was talking to changed".
The migrator produced the July settings regressions through three
emission defects, each of which restored-by-hand rows and the CI drift
check now guard against but the tool itself would have reproduced on
the next migration batch:
1. default_enable was never parsed. The configuration row regex
consumed SETTING_*() rows without inspecting the third argument, so
a row deliberately registered without a default (state_slot,
replay_slot, netplay_start_as_spectator, ...) was deleted and
re-emitted through the generated grammar, which always applies one
- resetting the state and replay slots to 0 on every config load.
The tool now splits the consumed row's arguments; default_enable ==
false keeps the hand-written configuration row and wraps the
generated row in #ifndef SETTINGS_DEF_CONFIG_PASS, the
accessibility_enable idiom.
2. The def row's default came from the descriptor row alone; the
configuration row's default expression was discarded. A descriptor
that transcribed DEFAULT_VIDEO_SMOOTH as a literal 'true' therefore
erased the macro's platform conditional (menu_linear_filter), and a
genuine value divergence silently took the descriptor's side
(cloud_sync_sync_saves/configs came back false). The tool now
reconciles the two spellings: textually equal passes; a macro
spelling versus a literal that expands identically against
config.def.h under the base define set keeps the macro; anything
else aborts with both values for the operator to reconcile in-tree
first - divergence is a source defect the migration must not
launder.
3. The CONFIG_PASS guard exemption existed only in the strings-free
emission branch, and only for one mismatch direction. Rows whose
base-language strings are themselves guarded took the raw
descriptor guard stack, narrowing their configuration registration
to the descriptor's platform (the audio_block_frames /
keyboard_gamepad_mapping_type class - twelve rows in July). Guard
reconciliation is now uniform and bidirectional: the config-pass
condition always derives from the configuration row's own guards
(an inner '#if !defined(SETTINGS_DEF_CONFIG_PASS) || (cfg)' term
whenever they differ from the descriptor's), and the outer
descriptor guard gains '|| defined(SETTINGS_DEF_CONFIG_PASS)' in
both string topologies whenever the configuration row was
registered more broadly.
tools/settings_migrate_selftest.py builds a miniature real-shaped tree
(descriptor table, string/label/config rows, surgery anchors) and runs
the migrator over it with a new --stop-after-emit flag that stops
after extraction, reconciliation and surgeries - the gate battery and
enum stage need the full tree and session harness and stay exercised
on real migrations. Fixture rows cover: a healthy row, the no-default
class, the macro-spelling class, descriptor-guard narrowing in both
string topologies, a config-only guard, and a genuine divergence that
must abort. Wired into the Settings-Checks workflow as a third step;
pure text fixtures plus one gcc -E probe, seconds per run.
Detection power verified against the pre-fix tool (with only the
neutral --stop-after-emit graft so its emission logic runs): the
self-test fails on its output at the first defect-class assertion, and
a side-by-side audit shows the old strings-guarded branch narrowing
the config pass to '#ifdef HAVE_ODROIDGO2' where the fixed tool emits
'#if defined(HAVE_ODROIDGO2) || defined(SETTINGS_DEF_CONFIG_PASS)'.
Also verified: settings table drift check against the committed
reference, griffin-order unity compile for all three profiles, and a
full Makefile build, all clean - the tool change emits nothing into
the tree by itself.
No sanitizer sweep is applicable: build tooling only; no executable
code paths in RetroArch are modified.
The pool-locking commit created the backend's bootstrap mutex whenever
the configuration was published, whether or not SMB would ever be
used: one allocation at startup on every HAVE_SMBCLIENT build. Remove
even that.
Intent is precisely expressible: SMB connections use only the
configured server address - never the host embedded in an smb:// URL -
so a publication carrying no server address means SMB cannot function
this session and needs nothing. smb_init_cfg() now creates the
bootstrap lock only when the published configuration carries a server
address. A session with no SMB server configured performs a pointer
store and a handful of scalar copies per publication and allocates
exactly nothing: no mutex, no slot locks, no contexts, no sockets.
Every publication site runs on the main thread (startup, configuration
loads, and now saves), so gated lock creation stays race-free, and the
happens-before argument for the slot locks is unchanged: operations
reach a slot lock only through a handle, handles only exist after
initialization, and initialization runs under the pool lock.
The one flow that needs care is configuring a server through the menu
mid-session. The menu writes land in settings->arrays, which the
published struct aliases, so the backend can see a server address that
no publication has carried yet - and without a publication there is no
lock to serialize on. Operations refuse in that state with a message
saying a configuration save activates SMB, and config_save_file() now
publishes on success exactly like config_load_file() does: the menu
flow reaches a save (manual or the default save-on-exit), which both
refreshes the numeric snapshot and creates the bootstrap lock. Before
the pool-locking work this same flow only half-worked anyway - the
frozen numeric snapshot meant a mid-session server ran with fallback
values until a republication - so the save requirement aligns
activation with the point where the settings become correct at all.
Verified offline before commit: the sanitizer harness gains scenarios
asserting the zero-footprint contract behaviourally - a publication
with an empty server sets up nothing and every operation refuses with
zero connection attempts; a server visible only through the aliased
strings still refuses until a publication carries it, then works after
one does. Full battery clean: ASan+UBSan+LSan and TSan, single- and
multi-threaded with heal chaos, zero contexts and handles alive after
shutdown. Full Makefile build with --enable-builtinsmbclient and
threads, settings table drift check and griffin-order unity compile
all clean.
libsmb2's synchronous API is not thread-safe per context: each
smb2_context owns a socket and a PDU queue, and two overlapping
synchronous calls on one context corrupt both. The VFS SMB backend
handed contexts out round-robin with an unsynchronized cursor and let
every open file and directory cache its context pointer for its whole
lifetime, so concurrent VFS access from task threads interleaved
operations on shared contexts, two threads could race the lazy
smb_init() into double initialization, and smb_shutdown() could tear
the pool down under in-flight operations.
Worse, the pointer caching was unsafe even single-threaded: healing a
dead connection destroys the old context, so every other stream still
caching that pointer dangled, and the smb2_context_active() liveness
probe those streams ran was itself a read of freed memory.
The pool is now an array of slots, each holding a context and a
generation counter, with a lock per slot and a pool lock:
- Every libsmb2 call happens under the owning slot's lock, which
serializes use of the non-thread-safe context.
- Streams and directory handles store a packed (slot, generation)
handle instead of a context pointer. Destroying a slot's context -
heal or shutdown - bumps the generation, so every outstanding handle
on it fails closed under the slot lock without touching freed
memory. This fixes the stale-pointer use-after-free in non-threaded
builds too, where the locks compile to no-ops but the generation
logic remains.
- Initialization is serialized by the pool lock, so a lazy first use
from two threads initializes exactly once; the round-robin cursor
and shutdown are under the same lock. Healing happens in place under
the slot lock via the same create-and-connect helper initialization
uses, removing the init/heal configuration drift fixed piecemeal
before (the heal path now shares the timeout fallback by
construction).
- The pool is a fixed-capacity static array (64 slots; the menu caps
the setting at 20), which removes the pool allocation failure paths.
Lock objects are created when the configuration is first published -
on the main thread during startup, before any thread can reach an
smb:// path, since operations refuse to run unconfigured - and are
deliberately never destroyed, so no thread can block on a freed
lock.
- retro_vfs_readdir_smb() returned a pointer to a function-local
static, which two threads reading different directories raced on
regardless of pool locking; the entry now lives in the directory
handle.
The lock order is pool before slot where both are needed; operations
on an open handle take only the slot lock, so reads and writes on
different contexts do not contend.
Resource cost when SMB is never used: publication of the configuration
(at startup and after each config load) creates exactly one bootstrap
mutex and performs a handful of scalar stores - no contexts, no
sockets, no network initialization. The per-slot locks are created
under the pool lock on first actual smb:// access, sized to the
configured pool, so a session that never touches SMB never allocates
them; the heavy machinery remains fully lazy.
Verified offline before commit:
- Sanitizer harness linking the real translation unit and the real
rthreads slock against scripted libsmb2 stubs (real vendored headers
supply the prototypes; stub handle bookkeeping mirrors libsmb2
freeing outstanding handles with their context). Single-threaded
scenarios cover the lifecycle: unconfigured refusal, explicit values
respected, zero-value fallbacks, capacity clamping, the KRB5 probe
and explicit-KRB5 failure, and file/directory operations across a
forced heal, including stale-generation reads, closes after the
context was destroyed, and directory reads after shutdown. The
multi-threaded scenario runs 8 threads x 400 iterations of
stat/open/read/seek/tell/close and opendir/readdir/closedir against
a chaos thread toggling transport death every 2 ms to force
continuous heals, starting from unserialized lazy initialization.
ASan+UBSan+LSan clean and TSan clean on all of it, with zero
contexts and zero handles alive after shutdown.
- Negative test: the same multi-threaded harness against the
translation unit compiled without HAVE_THREADS (locks compiled out)
fails immediately under TSan with a data race in smb_slot_acquire,
confirming the harness detects the defect class this commit removes.
- Full Makefile build with --enable-builtinsmbclient and threads,
clean link, no new warnings outside vendored deps. Settings table
drift check and griffin-order unity compile clean.
Known limitation, unchanged: settings changes only take effect for
contexts created after publication; the pool is not rebuilt while
initialized.
Commit e9f9dd443b moved retroarch_smb_init() to the top of rarch_main()
so the SMB backend has a registered configuration before anything can
touch an smb:// path. That intent is kept, but the move exposed an
asymmetry in the published struct: the string fields alias
settings->arrays and therefore follow every configuration load on their
own, while timeout, num_contexts and auth_mode are copied by value.
Since configuration loading happens later, inside retroarch_main_init(),
the numeric fields froze at the calloc-zero settings struct: the user's
configured values were never applied, the zero-fallbacks in smb_init()
always fired, and auth_mode always resolved to SEC_UNDEFINED, forcing
the KRB5/NTLMSSP probe on every first connection regardless of the
configured mode.
Reading the settings from inside the backend is not an option -
vfs_implementation_smb.c lives in libretro-common and must not depend
on RetroArch's configuration - so the fix is on the publisher side:
config_load_file() now re-publishes at its single success return, which
covers the main configuration, the salamander pass, appends and
per-core overrides alike, and keeps working if startup ordering changes
again. retroarch_smb_init() gains a declaration in retroarch.h (it had
none; configuration.c needs one) and comments documenting the
alias-vs-snapshot contract at both the definition and the declaration.
The early registration call stays. The SMB backend initializes lazily
on first smb:// access, which in every startup path happens after the
configuration has been loaded and re-published, so the correct values
are in place by the time the context pool is built.
Two hardenings in the backend paths this touches:
- reset() was a non-static function in a file that griffin.c includes
into the unity translation unit, i.e. a global symbol named 'reset'
waiting to collide at link time. Now static as smb_reset_pool(), with
the loop lowered to C89.
- smb_create_context(), used when healing a dead connection, applied
smb_cfg->timeout raw, without the zero fallback smb_init() applies,
so a healed context could be configured with timeout 0 where the
original pool got the default. Both paths now agree.
Verified offline before commit:
- Full Makefile build with --enable-builtinsmbclient (compiles the
backend, the vendored libsmb2 and the new hook), clean link, no new
warnings outside vendored deps; 'reset' no longer an exported symbol.
- Sanitizer sweep of the backend lifecycle: the translation unit linked
against scripted libsmb2 stubs (real vendored headers supply
prototypes and struct layouts, so signature drift is a compile
error), driven through nine scenarios via its public entry points:
unregistered config, empty server, normal init with explicit values
respected and round-robin use, zero num_contexts/timeout fallbacks,
the auth-probe path, context-creation failure mid-loop, connect
failure mid-loop, retry after failure, and re-publish with changed
values between init/shutdown cycles (the config_load_file refresh
this commit adds). ASan+UBSan+LSan clean, TSan clean, no leaks on
any failure path.
- Settings table drift check and griffin-order unity compile both
clean; the emitted configuration tables are unchanged.
Known limitations, unchanged by this commit: values only take effect
for contexts created after publication - the pool is not rebuilt when
settings change at runtime - and get_smb_context()'s round-robin index
is not synchronized, so concurrent VFS access from multiple threads can
hand the same non-thread-safe smb2_context to two callers. Both are
follow-up material.
The single-source migration changed the emitted configuration tables in
four distinct ways, none of which produce any diagnostic. This restores
the pre-migration behaviour and adds the checks that would have caught
each one, verified offline before commit: full Makefile build, a
griffin-order unity compile of the settings TUs at -Werror for three
profiles, and an entry-by-entry diff of the emitted tables against the
pre-migration commit (31f3a60641) for desktop and Android define sets.
1) Transcribed default values. cloud_sync_sync_saves and
cloud_sync_sync_configs were registered true and came back false;
menu_linear_filter was registered DEFAULT_VIDEO_SMOOTH and came back a
literal true, discarding the platform conditional. Restored.
2) Guard breadth. Twelve rows had their configuration registration
narrowed to the descriptor's platform guard, so on other platforms
they get no default and are neither saved nor loaded:
audio_block_frames, microphone_block_frames,
keyboard_gamepad_mapping_type (worst: guard moved from ANDROID to
TARGET_OS_IPHONE, so its Android default of 1 was never applied),
menu_show_reboot, menu_show_shutdown, sustained_performance_mode,
ui_menubar_enable, video_ctx_scaling, suspend_screensaver_enable,
video_disable_composition, accessibility_narrator_engine, and
network_remote_enable (registered under HAVE_NETWORKGAMEPAD before,
HAVE_NETWORKING && HAVE_NETWORK_CMD after). The guards gain
SETTINGS_DEF_CONFIG_PASS terms so the configuration row emits where
the hand-written table emitted it; descriptor and menu rows keep
their current platform restriction.
3) default_enable. The generated grammar always applies a default, but
six rows were deliberately registered without one:
rgui_show_start_screen, netplay_start_as_spectator,
network_remote_enable, ui_companion_toggle, state_slot, replay_slot.
Applying a default resets the state and replay slots to 0 on every
config load and forces the others back to compiled-in values. These
rows are excluded from the configuration pass in their def files and
registered by hand in configuration.c, the same idiom
accessibility_enable already uses. Deliberately, no new emitter
spelling is introduced anywhere: a previous attempt added _ND row
names via self-aliasing '#ifndef' blocks, which leak into any
translation unit that includes a def file before configuration.c
defines the name for real - in the griffin unity build that is
msg_hash.h, and MSVC flagged the redefinition (C4005). With no new
names there is nothing to collide and nothing to undefine.
4) Duplicate registration. video_fullscreen_x/y were registered twice in
the uint table, by the surviving hand-written rows and the generated
ones. The hand-written pair is removed and its __WINRT__ exclusion
moved onto the generated rows. This requires the config passes to
define SETTINGS_DEF_CONFIG_PASS before the video_fullscreen include
rather than after it; that ordering applied to all five passes.
tools/settings_check.py carries both regression checks:
table - preprocesses configuration.c for desktop, android and console
profiles and diffs every emitted (kind, ident, default_enable,
default) against tools/settings_table.reference. Intentional
settings changes regenerate the reference in the same commit,
putting the table delta in the diff for review.
unity - compiles configuration.c, msg_hash.c, intl/msg_hash_us.c and
menu/menu_setting.c concatenated in griffin/griffin.c order at
-Werror for three profiles. This is the configuration in which
the C4005 collision manifested; per-file builds cannot see it.
Negative-tested by planting a leaked guarded alias: the check
fails with the redefinition error, and passes after removal.
Both run in the new Settings-Checks workflow on every push and pull
request: preprocessor and -fsyntax-only only, no configure step, no
dependencies beyond the runner's gcc and python3, well under a minute.
The remaining table deltas against 31f3a60641 are intentional and
verified benign: settings added since July, the separate config.def.h
change to audio_fastpath_s16, and ORIENTATION_NORMAL spelled as its
value 0.
No sanitizer sweep is applicable: preprocessor-level changes and
build-time tooling only; no executable code paths are modified.
tools/settings_table_check.py landed without anything invoking it, so a
row that stops reaching the configuration table still produces no signal
on a pull request. Wire it up.
The job needs no configure step, no build and no dependencies past the
gcc and python3 already present on the runner: it only preprocesses
configuration.c for the three platform profiles and diffs the emitted
(ident, default_enable, default) triples against
tools/settings_table.reference. It finishes in seconds.
A settings change that intentionally alters the tables fails the job
until tools/settings_table.reference is regenerated in the same commit,
which is the point - it puts the table delta in the diff where a reviewer
can see it.
Nothing today notices when a single-source row stops reaching the
configuration table. The row still compiles, the menu entry still works,
the string tables are still complete, and the only symptom is that the
setting silently reads back as zero and stops being written to
retroarch.cfg. Every regression fixed in the preceding commits in this
series had that shape.
Add tools/settings_table_check.py, which preprocesses configuration.c for
three platform profiles, extracts every emitted (ident, default_enable,
default) triple from the five populate_settings_* tables in order, and
diffs the result against tools/settings_table.reference. Only the
preprocessor runs, so it needs no configured build tree and finishes in a
couple of seconds per profile.
Adding or intentionally changing a setting is a diff too; the reference is
meant to be regenerated in the same commit, which makes the table delta
visible in review rather than invisible.
tools/settings_table_check.py --check tools/settings_table.reference
tools/settings_table_check.py --write tools/settings_table.reference
No sanitizer sweep is applicable: the script is a build-time tool and no
shipped code is touched.
The hand-written table registered network_remote_enable under
HAVE_NETWORKGAMEPAD. The migrated row inherited the descriptor guard of
the group it landed in, HAVE_NETWORKING && HAVE_NETWORK_CMD, so a build
with the network gamepad but without the command interface no longer
registers the key at all and stops persisting it.
Restore the original condition for the configuration pass only; the
descriptor and menu rows keep the guard they have now.
No sanitizer sweep is applicable: preprocessor-only change with no
executable code touched.
The single-source grammar has no way to express default_enable == false:
every generated row emits SETTING_x(n, ptr, true, d, false). Six rows
that the hand-written table registered with default_enable == false now
have a default applied on every config_set_defaults(), which resets
state_slot and replay_slot to 0, and forces netplay_start_as_spectator,
network_remote_enable, ui_companion_toggle and rgui_show_start_screen
back to their compiled-in value instead of leaving whatever the earlier
initialization or the loaded config left there.
Add _ND (no default) spellings for the five row forms those six rows
use. The mechanism follows the existing _H idiom exactly: each def file
self-aliases the _ND name to its base form under #ifndef, so the string,
enum and menu passes see no change at all, and only the five
configuration passes in configuration.c define the _ND names for real,
emitting default_enable == false. settings_def_rows_end.h undefines them
alongside the rest.
Verified by preprocessing configuration.c against 31f3a60641: the six
rows are back to 'if (0)' in the emitted table, and no other entry moved.
menu_setting.c, intl/msg_hash_us.c and menu/cbs/menu_cbs_sublabel.c
preprocess with no unexpanded _ND token, confirming the aliases resolve
in every consumer.
No sanitizer sweep is applicable: preprocessor-only change with no
executable code touched.
video_fullscreen_x and video_fullscreen_y were registered twice in the
uint table: once by the hand-written rows that predate the migration and
once by the generated rows in settings_def_video_fullscreen.h. The
duplicate is harmless at runtime because both entries carry the same
pointer and default, but it burns two of the 320 uint slots and makes
the save-path index pairing between uint_settings and uint_defaults
harder to reason about.
Remove the hand-written pair and carry their __WINRT__ exclusion over to
the generated rows, which had lost it.
Doing so needs SETTINGS_DEF_CONFIG_PASS to be visible inside
settings_def_video_fullscreen.h, and it was not: all five config passes
defined the marker immediately after including that header rather than
before it, so it was the one generated table that could not test which
pass it was in. Move the define above the include.
Verified by preprocessing configuration.c with and without __WINRT__:
the uint table has no duplicate idents on either, and the fullscreen
size rows are absent on WinRT exactly as before.
No sanitizer sweep is applicable: preprocessor-only change with no
executable code touched.
The single-source migration attached each row's descriptor guard to the
configuration row as well, so a row that the hand-written table
registered unconditionally is now registered only on the platforms that
show it in the menu. On every other platform the setting gets no default
applied and is neither saved to nor loaded from retroarch.cfg.
Affected on a desktop build: audio_block_frames, microphone_block_frames,
keyboard_gamepad_mapping_type, menu_show_reboot, menu_show_shutdown,
sustained_performance_mode, ui_menubar_enable, video_ctx_scaling,
suspend_screensaver_enable, video_disable_composition and
accessibility_narrator_engine. keyboard_gamepad_mapping_type is the
worst of these: its guard moved from ANDROID to TARGET_OS_IPHONE, so on
Android its default of 1 is never applied and it reads back as 0.
Widen the guards with SETTINGS_DEF_CONFIG_PASS so the configuration row
is emitted wherever the hand-written table emitted it, while leaving the
descriptor and menu rows restricted exactly as they are now. The two
menu power-view rows are additionally qualified with HAVE_MENU, which is
what gated them before the migration.
Verified by preprocessing configuration.c on 31f3a60641 and on this
commit for both a desktop and an ANDROID/RARCH_MOBILE define set and
diffing the emitted populate_settings_* tables; no row present before
the migration is missing afterwards. Table occupancy stays well inside
the fixed bounds (bool 392/512, uint 177/320).
No sanitizer sweep is applicable: preprocessor-only change with no
executable code touched.
The single-source migration transcribed three defaults incorrectly.
cloud_sync_sync_saves and cloud_sync_sync_configs were registered with
a true default in the hand-written config table and came back as a
literal false; menu_linear_filter was registered with DEFAULT_VIDEO_SMOOTH
and came back as a literal true, which discards the platform conditional
in config.def.h.
Verified by preprocessing configuration.c on both the pre-migration
commit (31f3a60641) and master and diffing the emitted
populate_settings_* tables entry by entry; these three rows no longer
differ.
No sanitizer sweep is applicable: preprocessor-only change with no
executable code touched.
Three regressions from ec6dc34 and e0e13f5, all mine.
1. rglgen_headers.h defined GL_NUM_EXTENSIONS. That token is not just a
token: gl_capabilities.c and gl2.c use #ifdef GL_NUM_EXTENSIONS as a
feature test to decide whether to call glGetStringi(). Defining it in a
header every GL platform includes switched that path on for GLES class
targets, where glsym carries no glGetStringi, breaking webOS, Vita,
Android and Emscripten. All four tokens are reverted out of the shared
header and declared locally in the desktop-only paths that need them.
2. spirv_opengl.c was added to Makefile.common but not to griffin.c, so
every griffin build (MSVC, Android, Apple) linked shader_gl3.cpp
against a missing spirv_opengl_lower(). It is now included under
exactly the guard griffin_cpp.cpp uses for shader_gl3.cpp, which is
its only consumer.
3. shader_gl3.cpp is also built for GLES3, where glsym has neither
glShaderBinary nor glSpecializeShader. The SPIR-V path now compiles
out behind GL3_HAVE_SPIRV_BINARY, with gl3_spirv_link_program()
reduced to a stub that returns 0. gl3_spirv_binary_supported() already
returned false there, so behaviour is unchanged.
Verified:
- gl_capabilities.c: the implicit glGetStringi declaration reproduces at
HEAD under -DHAVE_OPENGLES2 and is gone after this change.
- shader_gl3.cpp compiles clean for both desktop and GLES3.
- gl3.c reports identical diagnostics to HEAD in both configurations.
- spirv_opengl.c compiles clean under -std=c89 -pedantic -Wall -Wextra
-Wshadow, and as C++, and has no GL dependency at all.
- A C++ caller links against the C object, which is the exact failure
mode of the MSVC and macOS link errors.
- Lowering behaviour unchanged: the reference shaders still lower, still
pass spirv-val for both opengl4.5 and vulkan1.0, still link on
llvmpipe with blocks at bindings 0 and 1, and the array, gl_VertexIndex
and separate-sampler guards still reject.
Pass::init_pipeline() now tries handing both SPIR-V modules straight to
the driver before falling back to the existing cross compiled path. That
skips SPIRV-Cross's GLSL emission and, more significantly, the driver's
entire GLSL front end, which is most of the cost of bringing up a
multi-pass preset.
gl3_spirv_binary_supported() gates this. It requires desktop GL (the
extension is explicitly not written for OpenGL ES), GL 4.6 or the
extension string, a resolvable glSpecializeShader, and
SHADER_BINARY_FORMAT_SPIR_V_ARB actually appearing in
SHADER_BINARY_FORMATS: advertising the extension is not on its own enough
to guarantee ShaderBinary will take the format. The result is latched.
RETROARCH_GL3_DISABLE_SPIRV in the environment forces the old path, for
bisecting driver specific breakage without a rebuild.
The fallback is the real safety net here. Any failure to lower, specialize
or link returns 0 and the pass is rebuilt through
gl3_cross_compile_program(), so a driver that dislikes a module costs a
slower compile rather than a black screen.
Consequences of SPIR-V shaders that the surrounding code has to respect:
- There is no name reflection, so glGetUniformLocation() cannot find
individual block members. Pass::reflect_parameter() returns early and
every location stays -1, which is exactly the behaviour that routes
semantics into the uniform buffers instead. This is deterministic
rather than lucky: the RARCH_* names those lookups use do not exist
in the SPIR-V under any circumstances.
- Block bindings are immutable and come from reflection rather than
glGetUniformBlockIndex(), per issue 24 of the extension.
- BindAttribLocation has no effect; the Location decorations are
authoritative. slang already pins Position to 0 and TexCoord to 1,
which is what the vertex arrays are set up for, so this matches what
the cross compiled path was arranging by hand.
- Sampler bindings likewise come from the decoration, which is what
set_semantic_texture() was already using for glActiveTexture().
The lowered push constant block needs a uniform buffer of its own, so
Pass gains a second UBORing and gl3_buffer_locations two more indices.
It is allocated rounded up to 16 bytes because a std430 block's declared
size need not be a multiple of 16 the way a std140 block's is, and the
buffer must not come up short of the block size the driver derives from
the SPIR-V offsets. The offsets themselves are untouched by the lowering,
so build_semantics() keeps writing at the offsets reflection reported.
Only the filter chain passes take this path. The menu pipelines and the
quad program pass flatten=true and depend on the UBO being flattened into
vec4 uniforms, which SPIR-V cannot express; they are compiled once at
startup and are not worth the complication.
Also skips a redundant glBindBufferBase() when the vertex and fragment
bindings coincide, which they now routinely do.
Not yet benchmarked on hardware. Preset cold compile time across a 10 to
20 pass preset is the number that decides whether this is worth keeping.
glslang emits SPIR-V for .slang shaders under Vulkan rules, which the
OpenGL SPIR-V execution environment of GL_ARB_gl_spirv will not accept.
This adds a small, dependency free C89 pass that rewrites such a module
into one OpenGL can consume, or rejects it so the caller can fall back to
cross compiling to GLSL. Nothing uses it yet.
Only two things actually differ for the modules the slang filter chain
produces:
- The PushConstant Storage Class does not exist in OpenGL SPIR-V;
Appendix A.spv.4 limits Storage Class to UniformConstant/Input/
Uniform/Output/Workgroup/Private/Function/AtomicCounter/Image. The
push constant block is therefore turned into an ordinary Uniform
Block and given the Binding decoration ARB_gl_spirv requires on
uniform block variables. Skipping shaders with a push block was not
an option: that is where every #pragma parameter lives.
- DescriptorSet must be 0 if present. slang mandates set #0 already,
so this is validated rather than rewritten.
Two traps worth recording, since neither is caught by spirv-val:
Layout. A push constant block packs std430, a uniform block std140.
Section 7.6.2.spv only requires the explicit Offset/ArrayStride/
MatrixStride decorations to obey the alignment rules, not the packing
rules, and std430 and std140 alignment are identical for scalars,
vectors and matrices. They diverge only for arrays and nested structs,
whose base alignment std140 rounds up to 16, so a push block containing
either is rejected rather than relaid out.
Pointer identity. Rewriting the Storage Class of an OpTypePointer often
produces a declaration identical to one the module already has for the
UBO, and SPIR-V section 2.8 requires non-aggregate types to be unique
(only SPV_KHR_variable_pointers relaxes this for pointers, and that is
not in play). Redundant declarations are dropped and their uses
redirected. This is sound because the Logical addressing model, which is
validated here and which ARB_gl_spirv mandates, confines pointer values
to OpVariable and the access chain instructions; spirv_opengl_ptr_uses_ok()
rejects the module if anything else names one.
Everything that cannot be proven safe returns 0. A slower compile costs
nothing; a wrongly rewritten module is a broken shader.
Testing, against all 1599 shaders in libretro/slang-shaders that glslang
accepts:
- 1599 lowered, 0 rejected.
- All lowered modules pass spirv-val --target-env opengl4.5.
- All lowered modules also pass spirv-val --target-env vulkan1.0,
which is the meaningful layout result: the opengl4.5 target does not
check block layout at all, while vulkan1.0 enforces the std140
uniform buffer rules that section 7.6.2.spv restates.
- Negative tests confirm the guards fire: an array in a push block, a
gl_VertexIndex reference and a separate texture/sampler pair are each
rejected. With the array guard disabled the lowered module does fail
vulkan1.0 layout validation, so that guard is load bearing rather
than merely conservative.
- A module with no push constants is passed through byte identical.
Builds clean under gcc -std=c89 -pedantic -Wall -Wextra -Wshadow.
glsym's generated tables stop at GL 4.5, so glSpecializeShader (core in
4.6, and the entry point GL_ARB_gl_spirv is built around) was missing.
glShaderBinary was already present.
Both the core and the ARB suffixed name are resolved. Drivers exposing
the extension without advertising 4.6 may only export the suffixed one,
and rglgen_resolve_symbols() simply leaves whichever is unavailable NULL,
so callers can pick at runtime.
Also adds the ShaderBinary related enums to rglgen_headers.h, following
the existing pattern there for tokens that older platform GL headers may
predate.
These are hand written rather than regenerated from gl.xml, to keep the
diff to the four symbols actually needed.
Fixes#18818: the desktop menu's grid view is aligned correctly on
launch, but once content has been started the spacing goes wrong and
stays wrong until the user adjusts something.
GridView caches the item rectangles. calculateRectsIfNecessary()
derives every rectangle from viewport()->width(), stores them in
m_rectForRow and clears m_hashIsDirty; subsequent calls return
immediately while the hash is clean. Only setModel(), dataChanged(),
rowsInserted() and refresh() mark it dirty again, and resizeEvent() is
what normally calls refresh().
A hidden widget receives no resize events. The desktop menu is hidden
while content runs - ui_companion_qt_toggle() shows the window again
afterwards - so anything that changes the viewport width in between
leaves the cached rectangles describing a width the viewport no longer
has. They are reused on the next paint, and nothing marks the hash
dirty, which is why the misalignment persists rather than correcting
itself on the next frame.
GridView overrides resizeEvent() but not showEvent(). Adding it, so
the layout is recomputed against the viewport the view is actually
about to be drawn into.
Instrumented to confirm the mechanism rather than infer it: a build
with a probe in calculateRectsIfNecessary() logs the width each layout
pass is computed from, and it does change between passes
(viewportW=98 then 84 in a headless run), while
setDefaultCustomProperties() runs only once at startup and is not
re-entered on content launch - so the reset of the default spacing is
not what is happening here.
Not confirmed end to end: reproducing the visible misalignment needs
the icon view active in a real window manager, and the headless
harness gives the Qt window no meaningful geometry. The caching
behaviour and the missing showEvent are from the code and the probe,
the connection to the reported symptom is not proven.
Fixes the crash in #17819:
EXC_BAD_ACCESS (SIGSEGV)
KERN_INVALID_ADDRESS at 0x0000000000000030
0 RetroArch input_driver_collect_system_input + 636
1 RetroArch runloop_iterate + 1052
2 RetroArch -[RetroArch_OSX rarch_main] + 60
A byte read at offset 0x30 from NULL, with the crash registers pointing
at input_config_binds and input_autoconf_binds - the two arrays that
loop walks.
Both are [MAX_USERS], and both indices come from the config file
without being checked against it:
- input_max_users bounds the loop over input_config_binds[port].
- input_joypad_index[port] indexes input_autoconf_binds directly.
Neither is validated on load, so a config with a larger value walks off
the end. The same bound is already applied elsewhere for exactly this
reason - input_driver_set_rumble(), input_config_get_bind_auto() and
the autoconfig scan all test joy_idx >= MAX_USERS first - so this is
the missing case rather than a new rule.
Applied at the three places that index with a config-supplied joy_idx
(collect_system_input, state_wrap, and the turbo/remap init), and the
loop bound clamped in the three functions that read input_max_units.
input_keys_pressed() takes its joy_idx from the joypad_info the caller
filled in, so it is covered by the clamp at the call site.
A bad index falls back to slot 0 rather than skipping the port: the
wrong binds are recoverable, no input at all is not.
Also null-checks input_st->libretro_input_binds[port] before
dereferencing it in the menu analog path. That pointer is set when a
core declares its controller layout, and the reporter's crash happens
while loading content - the window where it can still be unset.
Not reproduced here: no macOS, and the App Store build's symbols are
needed to map +636 to a line. The bounds are wrong regardless of which
of them faulted.
cond_cmd multiplexes two unrelated predicates - the command reply
(pkt->type == reply_cmd, video_thread_wait_reply) and frame consumption
(!frame.updated, video_thread_frame and video_thread_wait_frame) - and
both signallers use scond_signal(), which wakes exactly one waiter.
There is no scond_broadcast() anywhere in the wrapper.
That is correct today only because at most one thread ever waits on
cond_cmd: all four wait sites run on the user thread, and
video_thread_wait_frame() bars the worker with an explicit thread id
check. The requirement is nowhere stated, and nothing catches a
violation.
Add a second waiter and a wake-one signal can wake the waiter whose
predicate did not change. It re-tests, sleeps again, and the signal is
gone - the intended waiter is never woken. Only the frame pacing wait in
video_thread_frame() is timed and would recover; the other three are
unbounded scond_wait() and would hang. That failure would present as an
intermittent hang on frame submission, which is an unpleasant thing to
bisect, so state the requirement where someone about to add a waiter
will see it and assert on it in debug builds.
The check deliberately keys on thread identity rather than nesting
depth. On Apple, video_thread_pump_wait() drains the cocoa main thread
trampoline while waiting, which can re-enter the wrapper on the waiting
thread; a plain depth counter would abort on that. Same-thread nesting
cannot lose a wakeup to a different waiter, so it is counted and allowed.
A second distinct thread is the case that breaks the invariant and the
case that fires.
thr->lock is held across every wait, so the counter needs no atomics.
Release builds compile both macros to nothing; the two fields exist
unconditionally only so the struct layout does not vary with the build
type.
No behavioural change.
Verified: compiles clean with and without DEBUG under gnu89, and under
-std=c89 -pedantic (the pre-existing implicit roundf warning at the
frame pacing code is unrelated and unchanged). All four wait sites
confirmed bracketed with no return or goto between enter and leave, so
the counter cannot leak. The macro logic was exercised standalone across
four cases: sequential waits on one thread, same-thread nesting,
a second thread entering while the first waits (fires), and a second
thread entering after the first released (does not fire).
The fast forward / VRR / menu limiter divided its remaining budget down
to whole milliseconds before sleeping:
to_sleep_ms = (frame_limit_last_time + frame_limit_min - now) / 1000;
frame_limit_min is a retro_time_t in microseconds and is rarely a whole
number of them - a 60 Hz cap is 16666 us and a 144 Hz cap is 6944 - so
the divide discarded up to 999 us of every wait. Now that retro_sleep_us()
exists (4f3513e), sleep the value that was already computed.
What this does and does not fix, since the two are easy to conflate: the
average rate was already correct. The "combat jitter" line advances
frame_limit_last_time by exactly frame_limit_min rather than to now, so a
short sleep leaves a correspondingly larger budget next iteration and the
schedule self-corrects. Simulating 4000 frames of the pacing arithmetic
puts the achieved mean within 0.00% of the cap both before and after.
What the truncation cost was per-frame jitter. With a 1 ms timer period
the old code alternated between N and N+1 ms sleeps, giving a mean
absolute interval error of 445 us at a 60 Hz cap, 381 us at 144 Hz and
465 us at 120 Hz, worst case 744 us. In microseconds all three go to
zero: every frame interval lands exactly on the cap.
The second effect is subtler. When the remaining budget fell between 1
and 999 us the truncated value was zero, the > 0 gate failed, and control
fell through to the else branch, which resets the pacing anchor with
frame_limit_last_time = end_frame_time instead of advancing it. At a
144 Hz cap that happened on half of all frames. Acting on the
microsecond value keeps the anchor on the intended schedule.
None of this helps when the sleep primitive itself is coarse - with a
15.6 ms period both old and new are dominated by Sleep() granularity, as
expected. It is the high resolution waitable timer from 5c30a403 that
makes the microsecond value worth passing on, which is why this comes
after it rather than before.
Emscripten is left on the old truncation: it paces through a deferred
main loop timeout expressed in whole milliseconds
(platform_emscripten_deferred_sleep takes an int ms), so it has nothing
to do with a sub-millisecond remainder. The gate is split accordingly so
that platform's behaviour is unchanged.
Also drops the inner `if (sleep_ms > 0)`, which was unreachable dead code
under the outer `to_sleep_ms > 0` gate.
Verified: pacing arithmetic replayed over 4000 frames for {60, 144, 120}
Hz caps against {2.0, 1.2, 3.7} ms of core work, at 1 us, 1 ms and
15.6 ms sleep granularity, checking achieved mean rate, mean absolute and
worst case per-frame interval error, and anchor reset counts. C89 clean.
Cocoa Touch foreground guard and the return 1 contract are unchanged;
runloop_iterate() returning 1 only causes the caller in retroarch.c to
iterate again, and since core_run() precedes this block every iteration
still produces exactly one frame.
Not verified: no Windows or Apple host here, so this is arithmetic
replay rather than measurement, and no sanitizer sweep was run.
5c30a403 removed the accidental dependency on the WinMM MIDI stream by
sleeping on a high resolution waitable timer, but that timer needs
Windows 10 1803. On anything older - Windows 7, 8, early Windows 10, and
the XP era builds still shipped - retro_sleep() falls back to Sleep(),
whose granularity is the global timer period, so those systems are still
riding on the WinMM MIDI stream holding that period at 1 ms. Making the
MIDI driver lazy or defaulting it off would fix the cold start stall for
everyone and silently break Frame Delay and Scanline Sync for everyone
below 1803. Close that gap before touching the MIDI default.
Lowering the period is what winmm's timeBeginPeriod() does, but pulling
in winmm is the dependency this series exists to remove, so call what it
calls. winmm!timeBeginPeriod is an export forwarder to winmmbase, which
range checks the period, inserts it into a per-process list under a
critical section, and when the list minimum drops calls
ntdll!NtSetTimerResolution with the period in 100 ns units;
timeEndPeriod removes the entry and only issues
NtSetTimerResolution(0, FALSE) once the list drains. (Call chain from
public reverse engineering of winmmbase, not from a dump taken here.)
The tradeoff is explicit and worth recording, because it is a real
regression against timeBeginPeriod: that user-mode list and its critical
section ARE the reference counting. The kernel holds exactly one
resolution request per process and SetResolution=FALSE clears it
regardless of who asked. Bypassing winmmbase therefore means an
unrelated timeBeginPeriod/timeEndPeriod pair anywhere in the process - a
driver DLL, dsound, a core - drains winmmbase's list to empty, clears
the process request, and takes ours with it. That is precisely the
silent degradation this series is fixing, so instead of setting once and
trusting it, the request is reasserted on every fallback sleep.
NtSetTimerResolution replaces rather than accumulates, so reasserting is
idempotent, and one cheap syscall in front of a sleep of at least a
millisecond does not register.
Gating, on the same terms as 5c30a403:
- NtSetTimerResolution and NtQueryTimerResolution are resolved through
GetProcAddress on an already loaded ntdll, so no import is added and
nothing needs to be present in the SDK. Object still carries no import
newer than Windows 95 and no winmm.
- Only reached when the high resolution waitable timer is unavailable,
including the case where the timer probe succeeds but TlsAlloc fails.
- Win9x has no ntdll and its default period was 1 ms anyway; it lands on
plain Sleep(), as does any system where either export or either call
fails.
- The request is released in rtime_deinit() with SetResolution=FALSE, so
repeated core load/unload does not leave it pinned.
Note that this only covers retro_sleep(). WaitForSingleObject timeouts
round to the same period, so scond_wait_timeout() users - the Vulkan
mailbox and the threaded video wrapper on Windows - are still period
dependent and, on 1803+, no longer get a 1 ms period for free once MIDI
stops opening a stream. Their timeouts are coarse enough that this
should not matter, but it wants checking on hardware alongside the MIDI
default change rather than being assumed.
Verified: C89 clean (-std=c89 -pedantic -Wall -Wextra) under
i686-w64-mingw32 and x86_64-w64-mingw32 with _WIN32_WINNT unset and at
0x0400, 0x0501, 0x0600 and 0x0A00, with and without HAVE_THREADS; POSIX
path of rtime.c unchanged; import table confirmed to add nothing. The
availability state machine was replayed against a stubbed Win32 surface
covering all seven branches (timer available, no CreateWaitableTimerExW,
TlsAlloc failure, no ntdll, missing exports, query failure, set failure)
asserting the resulting state, whether a kernel request is held, that
reassertion happens on every fallback sleep, and that deinit releases.
Not verified: no Windows host available here, so no sweep was run and
the actual timings are untested. This needs a run on a pre-1803 machine
with the MIDI driver set to null before the MIDI default is changed.
The Frame Delay target is derived from the frame time and is therefore
rarely a whole number of milliseconds: 3/4 of a 60 Hz frame is 12.5 ms,
and 100% of a 144 Hz frame is 6.944 ms. It was being truncated into a
uint8_t millisecond value, so up to 999 us was discarded on every single
frame - 6% of a frame at 60 Hz, and 14% at 144 Hz. For a feature whose
entire purpose is shaving milliseconds off input latency, that is a
large constant giveaway.
Add retro_sleep_us() to libretro-common and use it here.
libretro-common/include/retro_timers.h:
retro_sleep_us() mirrors the retro_sleep() platform chain condition for
condition, in the same order, so it resolves on exactly the same set of
platforms. Most targets already have a microsecond- or nanosecond-native
primitive and honour the request as given (Vita, 3DS, Xenon, PS3, Wii,
PSL1GHT, QNX, and the nanosleep default). Targets that can only express
whole milliseconds - WinRT, Xbox, Wii U, Emscripten - round to nearest,
so a sub-500 us request degenerates to a yield rather than being rounded
up into an unwanted full millisecond of added latency.
libretro-common/time/rtime.c:
On desktop Windows retro_sleep_us() becomes the primitive and
retro_sleep() is expressed in terms of it. The waitable timer added in
5c30a403 takes 100 ns units natively, so this needs no new mechanism -
only the multiplier changes, from usec*10 rather than msec*10000. The
Sleep() fallback rounds to nearest, which for a whole-millisecond
request is exact, so the fallback path is unchanged in behaviour. The
millisecond wrapper guards the *1000 conversion rather than wrapping
around, so a bogus value cannot turn into a short sleep.
gfx/video_driver.c:
The auto tuner works in whole millisecond steps (delay_decrease is 1, 2
or 3) and frame_delay_effective is displayed in the menu as an integer,
so both are deliberately left alone. Only the truncated fraction is
recovered: it is derived from the same float expression that produces
the integer target, which is left character for character unchanged, so
the target itself is bit for bit identical.
Computing the target directly in microseconds instead was tried first
and rejected - reordering the float operations changes the rounding, and
90% at 60 Hz came out as 14 ms rather than 15 ms, silently shifting the
value fed to the tuner and shown in the menu.
The remainder is only added while the delay is still at the target. If
the auto tuner has reduced the delay it did so because frames were being
missed, and time must not be handed back.
Verified: integer target identical to the previous code across
{59.94, 60, 50, 75, 144, 30} Hz x {20, 25, 50, 66, 75, 90, 100}% and the
auto 3/4 rule, remainder always within [0, 999]. C89 clean
(-std=c89 -pedantic) for rtime.c under i686-w64-mingw32 and
x86_64-w64-mingw32 with _WIN32_WINNT unset and at 0x0400, 0x0501, 0x0600
and 0x0A00, with and without HAVE_THREADS; POSIX path unchanged; import
table still carries nothing newer than Windows 95. The retro_sleep and
retro_sleep_us preprocessor chains were diffed branch by branch to
confirm they resolve identically.
Not verified: no Windows host available here, so the actual sleep
timings are untested and no sanitizer sweep was run on the Windows-only
code. Both need doing before merge.
Sleep() rounds up to the global timer period, which is ~15.6 ms by
default - longer than one frame at any common refresh rate. Frame
Delay (video_frame_delay(), gfx/video_driver.c) and Scanline Sync
(video_driver_scanline_after_frame(), same file) both call
retro_sleep() with values as low as 1 ms and are therefore entirely
dependent on that period being lowered.
Nothing in the tree lowers it. The only timeBeginPeriod(1) call is in
rthreads.c, guarded on _WIN32_WINNT < 0x0500 and so compiled out of
every modern build. What has actually been lowering it is the WinMM
MIDI driver: MIDI_DEFAULT_DRIVER is MIDI_WINMM on Windows and
DEFAULT_MIDI_OUTPUT is "Microsoft GS Wavetable Synth", so
winmm_midi_init() opens and starts a MIDI stream on every startup
whether or not a core ever emits a MIDI byte, and a running WinMM
stream holds the process at a 1 ms period.
That is an accident, and it means anything that stops the stream from
being opened - a null MIDI driver, lazy MIDI init - silently disables
both features. It also explains the pre-existing FIXME in
midi_driver_set_all_sounds_off(): DirectSound and XAudio2 sit on
stacks that lower the period as a side effect, WASAPI does not, so on
WASAPI the MIDI stream was the only thing holding it down.
Break the dependency without reintroducing winmm. Rather than request
a global period via timeBeginPeriod(), which is only refcounted in
winmmbase user-mode over a single per-process kernel slot and can
therefore be cleared out from under us by an unrelated timeEndPeriod()
anywhere in the process, use a high resolution waitable timer. It
affects only our own waits and cannot be cancelled by third-party code.
Gating:
- CreateWaitableTimerExW() and SetWaitableTimer() are both resolved
through GetProcAddress(), so the object carries no import newer than
Windows 95 and builds against SDKs predating either.
- CREATE_WAITABLE_TIMER_HIGH_RESOLUTION, TIMER_ALL_ACCESS and
TLS_OUT_OF_INDEXES are defined locally when absent.
- Availability is probed at runtime by attempting the create, not by
version check: pre-1803 kernels reject the flag with
ERROR_INVALID_PARAMETER rather than ignoring it. Any failure at any
stage falls back to plain Sleep(), which is exactly the current
behaviour.
- A plain waitable timer is deliberately not used as an intermediate
fallback; it is no more accurate than Sleep().
- _XBOX and __WINRT__ keep their existing paths untouched.
The timer object is per thread via TlsAlloc(), because
SetWaitableTimer() on a handle another thread is already waiting on
reschedules that thread's wait. The TLS index and the calling thread's
handle are released in rtime_deinit(), so repeated core load/unload
cannot exhaust TLS indices.
retro_sleep(0) keeps Sleep(0) semantics (yield the remainder of the
time slice) rather than being turned into a zero-length timer wait.
Implemented in rtime.c rather than inline in retro_timers.h because of
the per-thread state; rtime.c is already in the unconditional object
list and in griffin.c, so no build system changes are needed.
Verified: C89 clean (-std=c89 -pedantic) under i686-w64-mingw32 and
x86_64-w64-mingw32 with _WIN32_WINNT unset and at 0x0400, 0x0501,
0x0600 and 0x0A00, with and without HAVE_THREADS, and the POSIX path
of rtime.c unchanged. Import table checked to confirm no static
CreateWaitableTimerEx/SetWaitableTimer/winmm dependency. Sanitizers
were not run: the added code is Windows-only and unreachable on the
Linux build - it needs an ASan/TSan sweep on a Windows host before
merge.
apply_patch_content() returned true unconditionally. The failure branch
only logged, then fell through to the same "return true" the success
path used, so try_ips_patch() answered "handled" for any .ips that
merely existed - applied or not.
patch_content() picks a format with
if ( try_ips_patch(allow_ips, name_ips, buf, size)
|| try_bps_patch(allow_bps, name_bps, buf, size)
|| try_ups_patch(allow_ups, name_ups, buf, size)
|| try_xdelta_patch(allow_xdelta, name_xdelta, buf, size))
so a rejected .ips short-circuits the chain and the .bps or .ups sitting
beside it is never tried. This is visible in libretro/RetroArch#18774,
where the reporter had to delete the .ips before the .bps of the same
hack would load. The indexed "*.ips1" loop below reads the same return
value and had the same problem in reverse: it kept iterating past a
patch that had not been applied.
Return whether the patch actually applied.
Also push the failure to the OSD. Until now a patch that was found and
rejected produced one RARCH_ERR line and nothing else: the content boots
unpatched and looks like the patch was ignored, which is the confusing
part of that report. The applied-patch notification stays behind its
setting - a failure is an error, not a notification the user opted into.
Both IPS appliers bounded a copy record's payload with
if (off > patch_len - len)
len is a 16-bit field read from the patch and patch_len is the size of
the patch file, so a patch smaller than the length its own record
declares wraps the subtraction to a value near SIZE_MAX. The test then
passes for every off and the record is accepted with a payload that
extends past the buffer.
In tasks/patch_stream.c the consequence is direct: the record is entered
into recs[] and off += len walks past patch_len, so the scan stops on
the next iteration having already committed it. patch_stream_ips_finish
then overlays it and memcpy() reads off the end of the patch allocation
into the target. ASan, on a mutated but header-valid .ips of 3.5 kB:
ERROR: AddressSanitizer: heap-buffer-overflow
READ of size 35156
#2 patch_stream_put tasks/patch_stream.c:135
#3 patch_stream_ips_overlay tasks/patch_stream.c:352
#4 patch_stream_ips_finish tasks/patch_stream.c:405
Those bytes land in the buffer handed to the core, so this is heap
contents disclosed into the running content, not only a fault. Since
soft patching applies whatever .ips sits beside the content, the patch
file is untrusted input.
tasks/task_patch.c carries the same wrapped comparison in both of its
passes. There it is currently masked - ips_alloc_targetdata() runs the
same walk first and returns PATCH_PATCH_INVALID before ips_apply_patch()
gets to write anything - but the guard is wrong in the same way and is
one restructuring away from mattering, so correct it in place.
Use the remaining-bytes form instead. off is at most patch_len at that
point in each of the three loops, so patch_len - off cannot wrap, and
comparing the declared length against it is exact.
ips_alloc_targetdata() took the size named by the IPS truncation
extension and made it both the size of the allocation and the reported
length of the result. ips_apply_patch() then copied the whole source
into that buffer:
memcpy(*targetdata, sourcedata, (size_t)sourcelength);
For any patch that shrinks the content - EOF followed by a three-byte
size smaller than the source - that is a heap write of sourcelength
bytes into an allocation of the smaller truncated size. Records sitting
past the truncation point, which the format permits, overran it as well.
A twelve-byte .ips is enough:
50 41 54 43 48 45 4f 46 00 00 10 "PATCH" "EOF" truncate-to-16
Soft patching applies whatever .ips sits next to the content, so the
patch file is the input here, not something the user has vetted.
Split the two lengths. targetlength stays what the content becomes;
alloclength is what the applier may touch, and covers the source and
every record end so that both the wholesale copy and the writes past a
truncation point stay inside the buffer. The copy is clamped to the
allocation rather than trusting sourcelength on its own.
Allocate with calloc(). A patch that grows the content need not write
every byte it adds, and the gap between the end of the source and the
first record beyond it was previously uninitialised heap that went
straight into the buffer handed to the core. tasks/patch_stream.c
zero-fills the same gap, so this also stops the two appliers disagreeing
on the output for the same patch.
Reference behaviour matches Floating IPS, which allocates
max(outlen, outlen_min_mem) and zero-fills beyond the source.
Building with a GLES2-only configuration, e.g.
./configure --enable-opengles --disable-opengl_core \
--disable-vulkan --disable-opengl1 --disable-qt
failed to compile cores/libretro-ffmpeg/ffmpeg_core.c, with a long list
of undeclared functions and enums out of the FFT visualizer code
(glGenVertexArrays, glTexStorage2D, glMapBufferRange, glBlitFramebuffer,
GL_RGBA8, GL_RG16I, GL_PIXEL_UNPACK_BUFFER, ...).
The visualizer was guarded by
#if defined(HAVE_OPENGL) || defined(HAVE_OPENGLES3)
which reads as "desktop GL or GLES3", but HAVE_OPENGL does not mean
"desktop GL" here. Makefile.common defines -DHAVE_OPENGL for any modern
GL context by way of HAVE_GL_MODERN, and HAVE_GL_MODERN is also set when
HAVE_OPENGLES is 1. In a GLES2-only build HAVE_OPENGL and HAVE_OPENGLES2
are therefore both defined, glsym.h pulls in <GLES2/gl2.h>, and the
visualizer - which needs VAOs, sized/integer internal formats, buffer
mapping and framebuffer blits, and whose shaders are #version 300 es -
got compiled against the GLES2 headers.
Introduce a single HAVE_FFMPEG_FFT feature macro that is only enabled
for GLES3 or for an actual desktop GL build, and use it for all fourteen
guard sites. GLES2 builds now simply drop the visualizer; audio-only
playback falls back to the existing no-video path, and the FFT core
options are hidden.
Also move the CG/GLSL shader-string macros out of the visualizer block:
the plain video path uses GLSL() as well, and it previously only
compiled because the block was unconditionally active.
Compile-tested in four configurations - GLES2, GLES3, desktop GL, and no
GL at all - all clean under -Wall -Wextra
-Werror=implicit-function-declaration.
Fixes: https://github.com/libretro/RetroArch/issues/19364
font_driver_reload_fonts() rebuilt every live font on every language
change, whether or not the file behind it differed. Five languages
have a face of their own - Arabic and Persian, Chinese, Korean, Thai -
and the other twenty-nine share the menu font, so English to French,
or either to Japanese, resolved to the file already loaded and
rebuilt it anyway.
That is not free. Each rebuild frees the glyph atlas and the GPU
texture behind a font, reads the same TTF back - 2.4 MB for ozone's
regular face - and builds a new atlas, six times over for ozone alone,
plus the widget and xmb fonts. It then bumps the generation, so every
derived metric in every menu driver is recomputed on the next frame
too. All to arrive exactly where it started.
Compare the resolved path against the one the font is already built
from, and skip when they match. Fonts that do not follow the language
at all - the widgets' regular and bold faces - resolve to their own
path and so are skipped as well, which they should always have been.
This narrows what the function does: it rebuilds where the face
changes rather than unconditionally. That is what its only caller
wants, and the tests are updated to match - the ones that were
checking the swap, the release of the old renderer state and the
failure path now force a real face change so they still exercise a
rebuild, rather than relying on the old rebuild-everything behaviour.
New cases cover the point directly: English to French and French to
Japanese rebuild nothing, leave the renderer state untouched and do
not bump the generation, while a switch to Thai still rebuilds.
hunkcount is logicalbytes / hunkbytes, both header fields, so a two
kilobyte image can claim a terabyte and send both map-decoding loops
around 268 million times. Measured at 64 seconds on a 2 KB file, after a
3.2 GB allocation, before anything is decoded and before any codec runs.
The bound used here is not an arbitrary cap on image size, which is what
made this awkward to fix earlier: guessing a limit on how large a
legitimate CHD may be is exactly the kind of judgement that turns into a
compatibility bug later. bitstream_overflow already reports when the map
bitstream has been consumed past its end, and past that point every read
returns zero, so the remaining iterations decode nothing whatever the
header claims. A well-formed map never overflows - a map that does cannot
describe the hunks it says it has.
64 seconds becomes 0.016. All nine codecs still decode their hunks
afterwards, verified with a probe at the decompress call so the check is
known not to be cutting the loops short on valid input.
What is deliberately left alone: a header can still ask for a large but
correctly computed rawmap allocation, which then fails and is handled -
every such input exits cleanly through CHDERR_OUT_OF_MEMORY in a build
without AddressSanitizer's allocator cap. Bounding that would need a
limit on legitimate image sizes, and self-referencing hunks make hunkcount
a poor proxy for file size.
huffman_import_tree_huffman builds a local decoder for the small tree and
frees it on the way out, but the early return when
huffman_assign_canonical_codes rejects the lengths skips that free. That
is the path a malformed huffman-compressed hunk takes, so the leak is per
hunk read rather than per image: opening a six-hunk image leaked six
decoders, 4464 bytes in 18 allocations.
This showed up on a well-formed image rather than a fuzzed one - the
first sanitised run of a valid huffman-coded CHD reported it - which is
worth noting, because it means it had simply never been exercised.
huff_codec_init returned CHDERR_NONE whether or not create_huffman_decoder
succeeded, leaving a NULL decoder that huff_codec_decompress passes
straight into huffman_import_tree_huffman, which dereferences it without
checking. Reports the allocation failure instead.
create_huffman_decoder's result is also checked at the top of
huffman_import_tree_huffman, where the very next statement wrote through
it.
bitstream_peek refills its accumulator while bits <= 24, so after a peek
there are at least 25 bits available. bitstream_remove will take up to 31
- huffman_decode_one removes lookup & 0x1f - so a code longer than the
data remaining drives bits negative. The next refill then computes
24 - bits, which for bits at or below -8 is a shift of 32 or more on a
32-bit type: undefined.
The deficit itself has to stay. bitstream_overflow is
(doffset - bits / 8) > dlength
so a negative bits is precisely how over-consumption is reported;
clamping it at zero would suppress that and turn a detectable error into
silent garbage. Only the OR is skipped, and doffset and bits advance
exactly as before, so overflow reporting is unchanged. The accumulator's
contents are already meaningless once the stream has been over-consumed,
and a well-formed stream never reaches the refill with bits below zero.
Found fuzzing a CHD corpus that reaches the huffman codec. Distinct from
the promotion fix on the same line: that one was the byte being promoted
to int before the shift, this one is the shift count itself.
cdzl, cdlz and cdzs each read a base-block length out of the hunk and
then split the hunk on it without checking it:
complen_base = (src[ecc_bytes + 0] << 8) | src[ecc_bytes + 1];
...
zlib_codec_decompress(..., &src[header_bytes + complen_base],
complen - complen_base - header_bytes, ...);
complen_base comes from the compressed data, so a value larger than the
hunk makes the pointer land past the end of src and the length underflow
to near 2^32. The subcode decompressor is then handed an out-of-bounds
pointer and an enormous length. AddressSanitizer catches it as a
heap-buffer-overflow read in rinf_need, four frames down from
cdlz_codec_decompress.
The length bytes themselves are read at src[ecc_bytes ...] before
anything has established that the hunk is even header_bytes long, so
that needs checking first and is checked first here.
cdfl has the same shape by a different route: the split point is the
offset flac_decoder_finish reports, which a malformed stream can drive
past complen, giving src + offset out of bounds and complen - offset
underflowed.
Reachable by opening a downloaded image. All four are fixed the same way,
rejecting the hunk rather than reading past it.
Verified that this rejects nothing legitimate, which needed a real CD
hunk rather than an approximation: a cdzl image was built with the actual
frame layout the codec expects - ecc bytes, then the base length, then
zlib of the sector data, then zlib of the subcode - and decodes to a
byte-identical hash before and after the change, all four hunks read.
An earlier synthetic corpus that carried plain zlib in the hunks is now
correctly refused instead of overread, which is the intended behaviour
and not a regression.
Found by fuzzing with a corpus that reaches these codecs for the first
time. Reaching them at all required writing a valid CHD v5 compressed map
- the earlier corpus stopped at the huffman tree import in
decompress_v5_map and never entered a codec.
Two faults, together meaning rinflate_set_stop_at_block has never done
anything since it was written.
The status was thrown away. Both boundary reports did
status = RDEFLATE_PROCESS_BLOCK;
goto suspend;
and the suspend label assigns RDEFLATE_PROCESS_NEXT before falling into
done, so what the caller received was an ordinary "give me more input".
An index builder driving this - the reason the entry point exists - got
no access points out of any stream, whatever its size. Both now jump
to done, which is where the block_ready path meant to go.
The first boundary was missing. block_ready is set when a block ends,
so the position in front of the first block was never offered. For a
stream that fits in one block that is the only boundary there is; for
every other one it is the natural first access point. zlib reports it
under Z_BLOCK and deflate.h says this status matches Z_BLOCK, so a
stream now reports there too, once, before the first block header is
read.
Found while making gzipped PS2 disc images load: with the gzip
container added but this unfixed, the index came back empty and the
image stayed unloadable. With it, the index builds, the extractor
seeks to its points, and the images load.
Streams that do not ask for boundaries are untouched - stop_at_block
is off by default, and raw, zlib and gzip all still inflate
byte-identical to what gzip(1) and python's zlib produce.
deflate.h said window_bits followed zlib's convention and then offered
two of its four cases: negative for raw, non-negative for zlib. zlib's
other two - 16..31 for gzip, 32..47 for detect-it-yourself - fell into
"non-negative", so a caller passing 47 got a decoder looking for a
2-byte zlib header, met 1f 8b, and failed.
That is not hypothetical. The PS2 core opens gzipped disc images with
z = rinflate_new(47); /* automatic zlib or gzip decoding */
and every .gz image failed to load, its index builder reporting a data
error for a file gzip(1) is perfectly happy with. The comment
described zlib's behaviour; the decoder never had it.
Decoding gzip: the 10-byte fixed header, the optional FEXTRA, FNAME,
FCOMMENT and FHCRC fields, then the deflate data, then a trailer whose
crc32 and length are both checked. Every field is resumable in the
same style as the rest of the decoder - a header arriving split across
two calls suspends and continues. Auto-detect looks at one byte: gzip
starts 1f, and a zlib header cannot, since its low nibble has to be 8.
Encoding gzip: the minimal header zlib writes when given no gz_header -
no mtime, OS 255 - and the crc32/length trailer, little endian, unlike
the big-endian adler32 the zlib wrapper uses. Compressing with 32..47
writes gzip, that being what a zlib caller passing 47 would have got.
The checksum is encoding_crc32(), already here.
No existing caller changes meaning: everything in the tree passes -15
or 15, and 16..47 had no defined behaviour to change.
Verified against the reference tools rather than against itself: a 6 MB
file compressed by gzip(1) inflates byte-identical at both 31 and 47;
a file this compresses at 31 passes gzip -t and gunzip(1) returns the
original byte for byte; our decoder reads back what our encoder wrote.
zlib and raw are unchanged - a zlib stream still inflates at 15 and at
47, a raw one at -15, and what we write at 15 is still accepted by
python's zlib.
92509d3 regressed the menu font on Windows: ozone asks for
assets/ozone/regular.ttf and got C:\Windows\Fonts\consola.ttf, the
first entry on stb's candidate list.
Before that commit the renderer was only asked where to look when no
path had been given:
if (!path)
path = font_backends[i]->get_default_font();
Giving the resolver the requested path meant calling it every time,
and the answer was then used in place of the request. stb and coretext
ignore the argument - they have no reason to resolve against it - so
they answered with their defaults and those defaults won.
A resolver returns NULL now to mean "the request stands", and all
three do that when a path was asked for. freetype is the exception
that the argument exists for: it hands the request to fontconfig when
the caller asked for a fallback font, which is the case where
overriding is the point.
font_renderer_create_default() no longer treats the request as a
fallback for the resolver's answer; the request is the answer unless a
resolver replaces it.
The create test grows the case, and the stub records which path was
actually read so the check is on the file opened rather than on
whether init succeeded - the old behaviour still produced a working
font, just the wrong one, which is why nothing caught this. Verified
by reinstating the regression: the test fails with it and passes
without.
76f8821 moved the reads out and 266285f dropped the headers, but the
renderers still dealt in paths: freetype kept streams/file_stream.h for
the fontconfig branch, coretext still opened its font itself through a
CFURL, and every init() took a font_path it mostly no longer used.
They take bytes now, and nothing else.
void *(*init)(uint8_t *font_data, size_t font_data_len,
unsigned face_index, float font_size,
enum font_atlas_format fmt);
const char * const *(*get_default_fonts)(const char *requested,
unsigned *face_index);
The two things that kept fontconfig in init() are what the new
arguments are for. It resolves against the request and the user's
locale, so the resolver is given the requested path; and it answers
with a face index as well as a path, which matters for a TrueType
collection - a CJK system font selects the wrong face at index 0 - so
the resolver hands one back. What is left in init() is
FT_New_Memory_Face on bytes somebody else read.
coretext went from CFURLCreateWithFileSystemPath and
CGDataProviderCreateWithURL to CGDataProviderCreateWithData over the
buffer it was given, with a release callback so CoreGraphics discharges
the ownership when it is finished. It was the last renderer opening a
file, and the only one that still needed the path.
stb needed nothing but the signature: it had stopped reading the path
already.
What is left in the directory that matches a grep for "path" is the
static candidate list freetype returns to
font_renderer_create_default(), and the fontconfig result string being
copied out of the pattern. Neither opens anything.
The fontconfig branch compiles here for the first time -
libfontconfig1-dev, -DHAVE_FONTCONFIG_SUPPORT - rather than being
carried untested. stb, freetype with and without fontconfig, and
font_driver are all clean; both sample suites pass under ASan, UBSan,
LeakSanitizer and TSan.
coretext is Apple-only and still unbuildable here, and it is the
biggest change of the three: CI is the check on it.
41bcd2a broke the message widgets: "Press again to quit" and the rest
render with no background and their text pushed off the bottom of the
screen.
That commit moved the derived-metric computation out of
gfx_widgets_font_init() and into gfx_widgets_font_sync(), which is
guarded:
if (font_data->metrics_generation == font_driver_get_generation())
return;
At init the two are equal. A fresh dispgfx_widget_t has
metrics_generation zero, and font_driver_generation is also zero until
something frees a font - which, on the first layout, nothing has. So
the guard decided nothing had changed and returned, leaving
line_height, line_ascender, line_descender and glyph_width at zero.
Everything laid out from them collapsed.
The guard and the computation are now separate. Callers that know the
font is new - init - compute unconditionally; the per-frame path keeps
the check. Both still run the same code, so they cannot drift.
xmb had the same trap from ea1f6b5, one commit later and by the same
hand: xmb_context_reset() set wideglyph_generation to zero and then
called the guarded sync, which compared zero against zero and returned
without ever working the width out. Split the same way.
ozone and materialui were never exposed: they compute inline and stamp
the generation afterwards, rather than calling the guarded path.
The lifecycle test gains a case for the trap itself - that a fresh
impl at generation zero does not end up with a zero line height - so
the next person to fold these together has something to fail against.
Not confirmed on screen: the harness could not get the quit prompt to
appear. The mechanism is exact and the fix follows from it, but the
screenshot that started this is still the test that matters.
retroarch-assets 6a1bbde added pkg/thai-fallback-font.ttf, so Thai has
a face at last. One case in font_driver_language_font_file() picks it
up, alongside the Arabic/Persian, Chinese and Korean ones.
The label drops "(Restart required)" with it. That note was kept back
in a147b6d specifically because a restart would not have helped Thai -
there was no font to restart into. Now there is, and 42a711f already
made a language change rebuild the menu fonts in place, so selecting
Thai takes effect immediately like the other four.
Confirmed on screen rather than inferred, which is worth stating given
how much of this subsystem is hard to see: RetroArch on lavapipe at
3840x2160 with user_language=36 renders the ozone sidebar, entry labels
and sublabels in Thai, with Latin words inside the same strings -
"Core", "Netplay" - still rendering, which was the thing that ruled out
Noto Sans Thai as the asset.
The lifecycle test grows a Thai case next to the Korean one: what it
checks is that the path re-resolves to thai-fallback-font.ttf on a
language change rather than the font merely being re-read.
Requires the assets update. Without it Thai falls back to a path that
does not exist, font_renderer_create_default() finds nothing to read,
and the built-in glyphs stand in - the same as any missing asset, no
worse than the tofu it replaces.
2026-08-08 22:26:06 +02:00
1790 changed files with 310283 additions and 133684 deletions
# traversal, netplay MITM) are tracked separately and are
# NOT covered here - widening this guard is what should
# happen as each of those is converted.
hits=$(grep -rn 'task_queue_wait' menu/ \
tasks/task_menu_explore.c \
tasks/task_database_info.c \
tasks/task_netplay_nat_traversal.c || true)
if [[ -n "$hits" ]]; then
printf '%s\n' "$hits"
printf '::error title=Blocking wait on a menu path::task_queue_wait() must not be reachable from the menu. Use cancellation with a stale-completion guard, or a deferred continuation.\n'
exit 1
fi
printf 'No task_queue_wait on the guarded menu paths.\n'
- name:Build and run playlist_parity_test (plain, ASan + UBSan)
shell:bash
working-directory:samples/playlist
run:|
set -eu
# Parity oracle for playlist.c's read path (playlist_init /
# playlist_read_file), compiled from the shipping
# translation unit. Pins the reader's observable behaviour
# ahead of its restructuring into a budgeted resumable
# parse: the full JSON schema, the
# malformed-JSON-keeps-partial-data contract, the old
# six-line format with its trailing metadata block, RZIP
# compressed input, missing-file-reads-empty, and capacity
# clamping. The budgeted lanes then assert the resumable
# API (playlist_parse_begin/step/end/abort) produces
# playlists identical to the blocking path while yielding
# repeatedly, for JSON, the old format and the autofix
# pass, and that abandoning a parse mid-document leaks
# nothing. The cache-reuse lane guards the size stamp the
# cache keys on: it must be the on-disk size for every
# format and compression combination - including after the
# autofix pass saves the file it fixed - or the cached
# playlist is never reused and the menu re-parses on every
# navigation.
#
# The switch and supersede lanes cover a field report -
# MaterialUI on Android showing one playlist's titles under
# another's heading, where selecting a row launches content
# from the wrong system. The deferred cached init yields
# mid-parse, and the menu reads playlist_get_cached() while
# building its list, so leaving the previous playlist
# installed until the new one is ready is user-visible and
# wrong. The lanes assert on EVERY yield that the cache is
# either empty or already the requested playlist, including
# when a request is superseded before it finishes, and that
# the parse really yielded - a lane that stops exercising
# its own window is worse than no lane.
#
# The pump and rebuild-reuse lanes cover issue #19427
# (large playlists flashing the empty-list placeholder):
# a yielded read must complete on frames alone, and the
# rebuild that follows completion must be served from the
# cache just installed - a restart there loops, one flash
# per completion. Pinned: the recorded
# base_content_directory under an autofix-off config, and
# the size stamp left stale by the autofix pass's save.
#
# The truncated-JSON lane also guards the staged-entry
# cleanup in playlist_read_file: an entry interrupted
# mid-object is reserved but never committed, and its
# already-parsed members must be freed on the error path -
# the leak the lane originally flushed out. LSan gates it
# here permanently. No threads on this path, so no TSan
Write-Host "::notice::Runner predates 10 1803; the placeholder job tests the legacy path too"
}
# vcvarsall through vswhere, which ships with every Visual Studio and
# needs no third-party action: this repo uses only
# microsoft/setup-msbuild, and one job is not a reason to add
# another dependency to the supply chain.
- name:Build the area sample
shell:cmd
run:|
for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do set VSPATH=%%i
if not defined VSPATH exit /b 1
call "%VSPATH%\VC\Auxiliary\Build\vcvars64.bat"
cd libretro-common\samples\memmap\memshm_area_test
RetroArch is primarily written in C89. Preserve the existing language and style of the file being edited.
- New C code must be C89-compatible unless the surrounding code explicitly requires otherwise.
- Do not introduce C99/C11 features into C89 code.
- Existing files written in C++, Objective-C, or another language should remain in that language.
- Follow the conventions of surrounding code rather than introducing a new style.
- Prefer the smallest change that completely solves the task.
- Preserve existing architecture
RetroArch contains deliberately complex, cross-file systems. Do not simplify, consolidate, refactor, or redesign an existing subsystem merely because its structure appears unnecessarily complicated.
Complexity may exist for compatibility, platform, driver, or architectural reasons.
When implementing a feature, find an existing feature with similar behavior and follow its implementation pattern. The current source tree takes precedence over historical examples.
Do not make unrelated cleanup or refactoring changes.
## Comments
Keep comments as short as possible.
- Prefer no comment when the code is self-explanatory.
- When a comment is necessary, prefer a few words or a few lines at most.
- Explain why something is non-obvious, rather than describing what obvious code does.
- Do not add comments merely because code was changed.
- Do not describe the previous implementation or behavior unless that historical context is necessary to understand the current code.
- Do not use comments as miniature documentation or change logs.
- Prefer clear names and straightforward code over explanatory comments.
Comments should describe information that will remain useful after the current change is forgotten.
## Commit messages
Keep commit messages concise and focused.
- Explain what changed and why.
- Prefer a short subject line and a brief body when additional context is useful.
- Do not enumerate every changed file or implementation detail.
- Do not describe intermediate attempts, discarded approaches, or the development process.
- Do not explain the previous implementation unless it is necessary to understand the change.
- Keep the complete commit message below approximately 600 words.
- A commit message is not a changelog or development diary.
## Menu changes
Adding or modifying a menu option commonly requires changes in several parts of the codebase.
Do not assume that a menu option can be implemented by modifying only the file where it is displayed.
### Before adding a menu option:
- Find an existing menu option with similar behavior.
- Trace its complete implementation through the menu, settings, configuration, and message systems.
- Identify every part of that pattern that applies to the new option.
- Implement the new option consistently across all required layers.
- Do not modify unrelated menu code.
#### Useful reference commits:
- 6e9aa66aea2b2b3d0687585e41bac658ac9ae252 — SMB timeout/context settings exposed as menu options.
- 434f41b935232325ed13024786653f9dfef49643 — CRT menu additions.
- 23170b82ec58c2b283a7132b85fa73e18050a5ae — MIDI device dropdown menu items.
05faba73e3fed69a5e118df2cc28d31309089243 — menu implementation example.
These are reference implementations, not fixed lists of files to modify. The required files depend on the feature.
## Shader and video backend changes
Shader functionality frequently crosses shared shader code and multiple video backends.
### When adding or modifying shader functionality:
- Find an existing shader feature with similar behavior.
- Trace the feature through the shared shader infrastructure.
- Identify every applicable shader implementation and video backend.
- Keep backend implementations consistent where the feature is supported.
- Do not update only the backend currently being tested if the feature is expected to work elsewhere.
- Check platform/backend-specific differences rather than blindly duplicating code.
#### Useful reference commits:
- da5ecaa45a862e51783eca1280f46fbfb7984ede — OriginalAspect and OriginalAspectRot uniforms.
- 37cfa4857d064464754e800bad89ce69e81230fe — sensor shader uniforms across multiple shader and video/input backends.
These commits demonstrate the breadth of changes that may be required. They are examples of implementation patterns, not instructions to reproduce their exact file lists.
## Historical commits
Historical commits may be used to understand established implementation patterns.
- Treat examples as references, not specifications.
- Prefer the current source tree when it differs from an historical commit.
- Do not copy historical changes mechanically.
- Do not reproduce obsolete workarounds merely because they appear in an old commit.
- Inspect surrounding current code before applying a historical pattern.
## Scope of changes
Keep changes focused.
### Do not:
- reformat unrelated code;
- rename unrelated symbols;
- modernize C89 code;
- replace established patterns with preferred patterns;
- refactor neighboring code without a direct need;
- add documentation or comments solely to explain the current change;
- make speculative improvements.
If a task requires a broad change, make the breadth necessary for the feature rather than artificially minimizing the number of files changed.
A change touching many files is not inherently a problem in RetroArch. Missing a required layer of an established multi-file system is a problem.
Loaded 100 of 1790 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
LibretroAdmin