Android tells the activity when an input device goes away - a pad
switched off, out of range or unplugged - but nothing passed that on
to the input driver, and no input event marks a removal. A pad
therefore stayed connected for as long as the app ran:
- its port kept its device name, so an overlay hidden while a
controller is connected never came back;
- whatever it was holding stayed held, since no release arrives from a
device that is gone;
- no disconnect notification was shown, and pausing on disconnect
could not happen.
RetroActivityCommon.onInputDeviceRemoved now hands the device id to
the driver. It is left in a small lock-free mailbox and taken out at
the next input poll, which clears the pad's key, axis and hat state
and reports the disconnect through input_autoconfigure_disconnect, as
the other joypad drivers do.
The pad's slot keeps its name and is marked removed instead of being
emptied, so the device gets its old port back when it returns, under
whatever id Android gives it; its return is reported as a connect. A
removed slot is matched before a live one of the same name, so of two
identical controllers the one that went away is not taken for the one
still connected.
A removal is read at the next poll, which can be long after it
happened if the app was in the background. If the id resolves again by
then, the device has come back under it and nothing is reported.
This is the device-removal part of PR #19077, reworked. That version
emptied the slot's name, which the driver uses to find a returning
device's port: every reconnect would have moved the pad to the next
free port.
Compiled with the NDK through the Android unity build. Not run on a
device.
Co-authored-by: f4mrfaux <[email protected]>
* android: Add Samsung S-Pen stylus input support
Stylus input for the Android driver, squashed from PR #19076 and
rebased onto current master (three conflicts, all additive: the
hover-guard block in android_input.c, an include in runloop.c and the
stylus rows in tools/settings_table.reference).
- toolType-based stylus/finger classification with an input-source
fallback
- hover, contact and click handled separately; hover can move the
pointer without pressing
- phantom-touch suppression for Samsung pens (hover guard, proximity
window)
- both barrel buttons read
- four Android-only settings: input_stylus_enable,
input_stylus_require_contact_for_click,
input_stylus_hover_moves_pointer, input_stylus_pressure_sensitivity
- android_mouse_calculate_deltas computed its fallback delta from the
zeroed delta instead of the event position; fixed
Closes#15490Closes#18194
* overlay: hide for a stylus without unloading the pack
The S-Pen support unloaded the on-screen overlay from the run loop
whenever a stylus became active and loaded it again two seconds after
the last pen event. A pen that leaves and returns does that over and
over, and each round is the pack's file and texture work, triggered by
input while a core runs.
It was a full unload because a soft-hidden overlay still took over
pointer input: with input_overlay_pointer_enable the overlay poll
tracks every touch as its own pointing device and sets
INP_FLAG_BLOCK_POINTER_INPUT, so input_state never reaches the driver
and the pen's native state (hover included) is lost.
Add INPUT_OVERLAY_STYLUS_HIDDEN and make it complete. While it is set
the overlay:
- is not drawn (alpha 0, as the gamepad soft-hide does);
- reads no touches, so the poll releases whatever a finger was holding
the way lifting it would, and no hitbox or pointing device sees the
pen;
- does not answer mouse, lightgun or pointer queries and does not
block pointer input.
The run loop sets and clears the flag each frame on the current
overlay, the way the gamepad soft-hide is driven. Nothing is unloaded
or reloaded.
* android: report a pen as one pointer, its barrel button as the right mouse button
The S-Pen support gave pointer indices a meaning of its own: index 0
was the cursor, index 1 the tip pressing and index 2 the barrel
button, with the pointer count raised to 2 or 3 to match. Only a core
written for that contract can read it. Everywhere else an index is a
separate touch, so a pressing pen looked like two fingers and a press
with the barrel button like three - to cores that count pointers, to
the on-screen overlay, and to this driver's own lightgun mapping,
where two pointers are turbo and three are reload. With the barrel
button held but the tip not pressing, index 1 was stale and still
inside the count.
Map the pen onto the devices the driver already reports instead:
- position is pointer 0, and moves without a press while hovering
(input_stylus_hover_moves_pointer);
- a press is pointer 0 pressed with a count of 1: the tip pressing
past the pressure threshold, or, with
input_stylus_require_contact_for_click off, the barrel button,
touching or hovering;
- the barrel button is the right mouse button.
This also puts two settings to work. The contact-for-click option and
the pressure threshold only fed a latch nothing read; they now decide
the press, as their menu text says.
The pen no longer switches the driver into mouse mode
(mouse_activated). That flag is never cleared, and once set the
lightgun position is read from the mouse coordinates, which a pen does
not update, so lightgun aim went stale after the first pen contact.
The mouse_x_viewport/mouse_y_viewport values it wrote alongside are
not read anywhere.
A cancelled pen gesture now releases the press like a lift does;
before it fell through to the touchscreen path and left the pen's
contact latched.
Compiled with the NDK through the Android unity build. Not run on a
device: it needs a pass on S-Pen hardware before merging.
* android: let a mouse's hover events reach the mouse path again
The S-Pen support drops every hover event before the source is looked
at, so hover cannot be taken for a touch. A mouse moved with no button
held reports its motion as hover events too, and they were dropped
with the rest: the pointer only moved while a button was down.
Drop hover events after the mouse branch instead. Mouse motion is
handled as before the stylus work, and hover from any other source is
still kept out of the touchscreen path.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The Apple display server answers get_window_refresh_rate from the screen
the RetroArch window is on: on macOS the current mode of that display,
or, on a built-in panel whose mode carries no rate, the screen's own
maximumFramesPerSecond; on iOS and tvOS that screen's
maximumFramesPerSecond, which follows an iPad onto an external display
and a TV through a mode change. With no window yet the frontend falls
back to get_refresh_rate as before.
The reading is taken again whenever the window changes screen or a
screen changes mode: macOS posts NSWindowDidChangeScreenNotification
and NSApplicationDidChangeScreenParametersNotification, iOS and tvOS
the UIScreen mode, connect and disconnect notifications, and an
observer the display server installs at init passes each on as
video_driver_window_output_changed().
With "Use System Keyboard" on, the text being typed lives in a
Java-side EditText and the menu line is a copy of it: every change to
the field sends the whole text across. A soft keyboard commits most
text straight into the field, but sends some keys as key events
instead - Backspace when nothing is being composed (so always in
something like an IP address), digits on the AOSP keyboard.
RetroActivity is a NativeActivity, so those key events come out of the
native input queue first, and the views only see one that native code
finishes unhandled. android_input_poll_input_default finished every
key event handled, so a Backspace never reached the field. Its text
kept the characters, and the next character typed sent them back:
"192.168.17888", Backspace twice, "." gave "192.168.17888.".
While a system keyboard session is open, a key event the soft keyboard
sent - AKEY_EVENT_FLAG_SOFT_KEYBOARD or the virtual keyboard's device
id - is now finished unhandled and kept out of the native key state
and line editor, so the framework forwards it to the field. BACK stays
native: the field takes it ahead of the IME to cancel, and one left
unhandled would fall through to the activity's own back handling.
Hardware keyboards and gamepads are untouched.
The decision sits in input/drivers/android_kbd_route.h, free of NDK
types, so samples/input/android_kbd_keys can build it on a host. That
harness runs the reported sequence through the queue-then-views
pipeline and asserts the old behaviour is caught; it is wired into
the samples/input workflow.
Menu Frame Rate sets the rate the menu runs at while content is loaded,
in every sync mode: 'Display Rate', the default, runs it at the refresh
rate of the monitor the window is on, and 'Content Rate' at the
content's frame rate. At 'Display Rate' the silence the menu feeds is
written without blocking and measured out by the clock, so audio no
longer holds the menu to the content's rate; a core left running behind
the menu keeps its own clock, running when its next frame is due while
the menu re-presents its last frame in between; and behind the threaded
video wrapper without the display pacing hold, the timer holds the menu
to the display's period. A configuration with Sync to Exact Content
Framerate and Throttle Menu Framerate on keeps the content rate it had,
and the old key is dropped on save.
Display servers gain get_window_refresh_rate, the refresh rate of the
output the window is on, for Win32, X11 and SDL, and Wayland reports it
from surface enter and leave; the frontend keeps the reading until the
window moves, its output changes mode or the menu opens. The pace
decision, the menu pacing table, the threaded video harness and the X11
live sample cover both rates.
'Show Help' (menu_show_help) gated a main menu Help entry that no
longer exists; nothing read the setting, and the previous commits
took it out of the Views list and removed the Help list wiring.
Drop the def row, its default, the settings_t field and the label
string, and regenerate tools/settings_table.reference (the key is
the only change). An old retroarch.cfg line for it is ignored.
The generated intl tables still name the toggle's VALUE and SUBLABEL
ids, so those two stay in msg_hash.h as retired rows until the next
Crowdin fetch regenerates the tables without them; a follow-up
deletes the two rows then. help_list stays: its "Help" string is the
title of the per-setting help popup.
With the Help list wiring gone, the only remaining use of the Help
icon in XMB, Ozone and MaterialUI was loading it: every menu init
still read and uploaded menu_help.png / help.png for an entry that
is never drawn. Drop the texture slots and their file names.
The main menu stopped adding the Help entry long ago, and nothing
builds items for DISPLAYLIST_HELP_SCREEN_LIST, yet the plumbing for
it stayed: the display list type, its deferred push, the ok/title/
sublabel table rows, and the XMB/Ozone/MaterialUI icon cases. The
"Show Help" toggle under Menu > Views was also still listed even
though nothing reads menu_show_help, so it did nothing.
Remove the wiring and take the toggle out of the Views list. The
help_list and menu_show_help setting rows and their strings stay for
now: the generated intl tables reference their message ids, so
dropping them has to follow a Crowdin sync. DISPLAYLIST_HELP (the
per-setting help popup) is unrelated and untouched.
In Information > Database Entry, Release Date Month showed the raw
number ("1"). Show the month name ("January") instead, through twelve
new translatable MENU_ENUM_LABEL_VALUE_RDB_ENTRY_RELEASE_MONTH_*
strings; languages without a translation yet fall back to English as
usual.
A month outside 1-12 in the database keeps the old behaviour and
shows the number, rather than an unresolved label.
Reworked from #19114: squashed, trailing-whitespace churn dropped so
it applies to current master, invalid-month fallback added, helper
returns enum msg_hash_enums from a static const table.
With no start directory set, Load Content on webOS opens on nothing
useful. Default the menu content directory to /media/internal, the
TV's shared storage, when it exists. Only a fresh config picks this
up; an existing retroarch.cfg keeps its saved rgui_browser_directory,
and nothing else about the config location changes.
The previous commit stopped offering "/" in the webOS drive list, but
Parent Directory could still walk up into it. Rather than special-case
webOS in the file browser, let the frontend say whether its drive list
offers the root: a new optional root_in_drive_list hook on
frontend_ctx_driver_t (NULL means it does), wired for the unix
frontend to frontend_unix_root_in_drive_list().
filebrowser_parse() then hides Parent Directory when going up would
land on the root and the root is not offered. Back returns to the
drive list as before. Rooted users can still open "/" deliberately
when the platform offers it; on webOS and Play Store Android, where
it is not offered, the browser no longer wanders into it.
Every other frontend leaves the hook NULL and is unchanged.
Under jailer the root is not user storage, and opening it from Load
Content has been reported to freeze the file browser (PR #19203).
The webOS drive list already lists the real storage locations
(/media/internal, /tmp/usb and, in developer mode,
/media/developer/temp); stop appending "/" after them.
The root decision moves into frontend_unix_root_in_drive_list(),
which also carries the existing Play Store Android rule, so the next
commit can expose it to the file browser.
Playlist core names are copied from .lpl entries without a length
limit. strlcpy returns the source length on truncation, so using it
as the next offset can place the runtime text write beyond the
sublabel buffer. A crafted playlist with runtime data can corrupt
memory when its entry is displayed.
Stop appending when the prefix or core name exhausts the buffer.
The suite drives the whole frontend - runloop, task queue, autodetect - and fails on any sanitizer report, so it gains a sanitized lane for each, and a ThreadSanitizer warning now fails it as an error does.
The preference is stored with release ordering on the main thread and loaded with acquire ordering by the worker as it starts, so a worker started while the setting is applied reads it without a data race.
A core that reports no frame rate, such as the dummy core, adds nothing to the core runtime counter, and neither does one whose rate is so small that a frame would outlast a day; the frame time is no longer derived from a division by zero and a conversion of infinity. Every other rate is counted exactly as before.
IT's Pxy shared the XM handler: right by x and left by y per tick, in
the engine's 0-255 units, and no fine slides. IT slides the other way
- Px0 left, P0y right - in its 0-64 units (four engine steps each),
and PxF / PFy are single fine steps on the row. A song panning with
Pxy had its channels drift the wrong way at a quarter of the speed:
its left/right balance was a median 2.8 dB off OpenMPT, now 0.1 dB.
Three ways IT notes landed in the wrong place in the stereo field,
compared with OpenMPT and Impulse Tracker:
- Pan envelope nodes are signed (-32 left .. +32 right, 0 centre), but
the loader read them unbiased like volume nodes, so 0 meant hard
left and negative nodes clipped to it. Songs with pan envelopes
played lopsided and 2-2.7 dB loud.
- An instrument's default pan was ignored, and a sample's default pan
overwrote the channel pan for good. IT gives each new note its own
pan - instrument default if set, else the channel's, the sample's
over either, then pitch-pan separation - and leaves the channel pan
alone until a pan effect sets both.
- Pitch-pan separation was not applied.
On the 40-file corpus the worst stereo case, where instruments carry
default pans, goes from a median 4.5 dB left/right balance difference
to under 1 dB, and the three pan-envelope songs from +2.0/+1.6/+1.1 dB
to within 0.2 dB.
Three note-level divergences from OpenMPT, found rendering 40 IT files
from modarchive against libopenmpt:
- A tone portamento on a channel with nothing sounding plays as an
ordinary note in IT; rmodtracker kept it a slide with no source, so
the note never sounded. An Impulse Tracker 2.15 song that opens every
part this way was silent for its first 36 s.
- Exx, Fxx and Gxx share one memory in IT (Gxx its own with the
compatible-Gxx header flag); rmodtracker kept three, XM-style. The
volume column's Gx now uses IT's speed table and that memory, and its
Ex/Fx pitch slides, which the loader dropped, are played.
- The instrument keyboard's note byte was ignored, so instruments that
map keys to other notes (drum kits, transposed instruments) sounded
at the key pressed - several semitones off in one song. Notes now
also reach IT's C-0, eleven semitones below the engine's lowest key.
On the 40 files the share of 50 ms windows more than 6 dB off goes
from 67.6% to 2.5% for the portamento song (spectral match 0.66 ->
0.93), and the two keyboard-mapping songs' spectral match from 0.47
to 0.77 and 0.59 to 0.91.
Files ModPlug Tracker wrote (or OpenMPT in its ModPlug mode) are played
by OpenMPT at ModPlug's mix level, which pre-attenuates by a table of
the channel count; rmodtracker played them at Impulse Tracker's level,
3.4 dB (4-5 channels) to 9.5 dB (26 channels) louder, so they clipped.
Detect them with OpenMPT's header rules (cwt/cmwt pairs such as
0x217/0x200 and 0x888, or an OpenMPT version with "OMPT" in the
reserved field), read the mix-level mode ("PMM.") and sample pre-amp
("SPA.") from OpenMPT's song extensions when present, and scale the
gain as OpenMPT does: x64 / table[channels / 2] for ModPlug's levels,
x2 for 1.17RC3, the 1.17RC1/RC2 and FT2 variants likewise. The gain is
capped at 255 to keep the volume products within 32 bits.
Also accept patterns up to 1024 rows: Impulse Tracker stops at 200 but
OpenMPT writes more, and a longer pattern was replaced by 64 empty
rows (one OpenMPT song played 3 s of 98).
Across 40 IT files from modarchive rendered against libopenmpt 0.7.3,
the eight ModPlug/OpenMPT-levelled ones go from +3.4..+9.5 dB (one
OpenMPT 1.17RC3 file -6 dB) to within 1 dB.
Every connect and disconnect queued for a port takes the next number in
that port's sequence, and only the operation holding the newest number
writes the port's state when it completes, so a multi-pass connect scan
can no longer overwrite a disconnect or a different device queued after
it. A connect is skipped only when the same device's connect is already
the newest thing pending on the port, and a disconnect only when a
disconnect is; a device that is swapped or replugged while an earlier
scan is still running is configured rather than dropped.
The autoconf suite gains a phase that runs the connect/disconnect cycle
in several instances at once, so every step is queued before the first
scan as it is on a busy CI runner, and checks that the device connected
last is the one configured.
The mailbox added for #19665 spelled its slots as volatile LONG with
InterlockedCompareExchange/InterlockedExchange. Use retro_atomic_int_t
with retro_atomic_cas_int/retro_atomic_exchange_int instead, like the
rest of the tree: same lock-free behaviour on Windows, and the slots
are real atomics to the compiler and to TSan rather than volatile.
With threaded video the RetroArch window is created on the video
thread, so the menu bar's WM_COMMAND is handled there. Window Scale
calls CMD_EVENT_REINIT, whose driver teardown sends CMD_FREE to the
video thread and joins it - from the video thread itself. It waits
for itself forever, the window stops pumping, and the process has to
be killed from Task Manager.
win32_menu_loop() now parks the command when it is not on the main
thread, in a small lock-free mailbox, and the main thread's pump
(ui_application_win32_process_events, which the run loop calls every
iteration) runs it. That is the same thread the in-menu Windowed
Scale setting already runs on. The video thread also pumps through
that function, so draining is gated on task_is_on_main_thread().
Non-threaded video is unchanged: the window is on the main thread and
commands run immediately as before.
This covers every menu-bar entry, not just Window Scale: the others
(fullscreen toggle, save/load state, disc control, core/content load)
were also running run-loop work on the video thread.
The IT loader never set linear_periods, so every IT file slid with
Amiga periods, where a slide's pitch change depends on the note. With
header flag 3 set (the default in Impulse Tracker, and set in all four
test files) Exx, Fxx and Gxx move xx * 4 sixty-fourths of a semitone a
tick on any note, which is exactly the engine's XM linear-period
arithmetic. Select it from the flag.
With the two pitch fixes before it, the pitch offset to it2play goes
from about 3 cents to 0, and over the three patches the share of 50 ms
windows more than 6 dB off it2play goes 0.3% -> 0.0% for Deus Ex - The
Illuminati, 1.0% -> 0.1% for Landing (mostly the pitch envelope fix)
and 0.8% -> 0.9% for through_the_infinite (OpenMPT: 0.5% -> 0.4%).
The linear-period frequency multiplied by c2_rate >> 4, dropping the
rate's low four bits (8363 Hz became 8352), and the IT loader
truncated each sample's C-5 speed to 1/128 of a semitone; together
every linear-slide note played about 3 cents flat of Impulse Tracker,
OpenMPT and FastTracker 2's 8363 * 2^((4608 - period) / 768).
Add the dropped low bits back as a second 32-bit product, and round
the IT tuning to the nearest 1/64 semitone, the step the period
arithmetic keeps. Across a spread of C-5 speeds and octaves the error
goes from -2.3..-4.0 cents to -1.3..+0.7. This also corrects XM, whose
linear periods use the same formula.
Impulse Tracker resets each voice's frequency every tick and applies
the pitch envelope on top: an envelope value v puts the note v/2
semitones away from its pitch (v * 32 / 768 octaves). rmodtracker
instead multiplied the period by it every tick, so any non-zero value
kept sliding the note further away for as long as it held - a flat
+4 envelope swept the note up by two semitones a tick rather than
holding it two semitones up.
Keep the offset per channel and apply it in the frequency calculation,
for linear and Amiga periods. Landing.it, whose instruments 6 and 17
carry pitch envelopes, goes from 1.0% to 0.1% of 50 ms windows more
than 6 dB off it2play.
The bench gate timed the blocking reference once, before the task
queue existed, and the deferred walk much later; drift between those
two moments on a shared CI runner failed the gate on unchanged code
(24.0ms vs 17.1ms, ratio 1.40 against a 1.35 limit).
Time a blocking walk immediately before each frozen deferred round
and gate on the best per-pair ratio over five pairs: a real
regression is slow in every pair, noise only has to miss one. Raise
the factor to 1.5, which still rejects what the gate exists to catch
(a second enumeration or a quadratic, 2x and up). The paced rounds
stay at three on the real clock.
S7x was ignored for IT. Apply the note-related half as Impulse Tracker
does:
- S70, S71 and S72 cut, release or fade the notes this channel has
moved to the background (fade approximated as a release, as for the
new-note action);
- S73-S76 override the playing note's new-note action (cut, continue,
off, fade) until the next note, which a portamento does not count
as.
S77-S7C (switching the envelopes per note) are still ignored. The
three test files without S7x render byte-identical.
Impulse Tracker's four volume-column volume slides (fine up, fine
down, up, down) share one memory per channel, and a slide of 0 recalls
it. rmodtracker treated 0 literally, so a run of B0 after B1 - a fade
written as one fine step down and then repeats - did nothing.
through_the_infinite.it fades six channels that way from 147 s to 158 s
and played them up to 14 dB loud there.
Keep the memory for IT; XM's volume column has none and is unchanged.
Against it2play the share of 50 ms windows more than 6 dB off drops
from 2.4% to 0.8% (OpenMPT: 1.8% to 0.5%); the other three test files
render byte-identical.
Files whose compatible-with version is below 2.00 store instruments in
the IT 1.x layout, but the loader always read the 2.x one: NNA, DCT,
DCA and fadeout came from the wrong bytes, the global volume was read
from where 1.x keeps its fadeout, and the volume envelope from where
1.x keeps an unused 200-byte table, so the envelope was garbage or
silent. Such songs played up to 27 dB quiet and largely wrong.
Read the 1.x layout when the header's cmwt is below 0x200, as Impulse
Tracker and it2play do: NNA and DCT at 0x1A/0x1B, fadeout at 0x18 in
half the 2.x units, the volume envelope as up to 25 tick/value byte
pairs at 0x1F8, and no DCA, instrument global volume, filter, pan or
pitch envelope. 2.x files are unaffected.
Tested on landing.it and through_the_infinite.it rewritten into the
1.x instrument layout, against it2play rendering the same files: the
share of 50 ms windows more than 6 dB off drops from 41.8% to 0.1% and
from 28.5% to 2.6% (the latter's original renders at 2.4%).
The server answered NETPLAY_CMD_NETPACKET from a client that is not
playing with a NAK, which closes the connection. A client reaches that
state without misbehaving: it calls the core's netpacket start() at the
end of the handshake while still a spectator and only then asks to
play, so an older client whose core sends from start() or in the frames
before its promotion lands, or a client set to start as a spectator,
was disconnected.
Read the packet off the stream as before, then drop it without passing
it to the core or relaying it, so spectators still cannot reach core
packet parsers but stay connected. Logged at debug level, since a
spectator's core may send every frame.
The server accepts NETPLAY_CMD_NETPACKET from connected spectators,
including clients authenticated with a spectator-only password. It
forwards their payloads to the core or other players, exposing core
packet parsers to unauthorized input.
Reject packets from non-playing clients before dispatch and prevent
spectator clients from sending core packets. Leave client reception
unchanged because cores can send during player admission before the
mode change reaches the client.
libsmb2 reworked libsmb2-share-enum.h in place upstream (48e2461,
2026-10-01): the srvsvc_* structures, SHARE_INFO_1 and
SRVSVC_SHARE_TYPE_* became a flat struct smb2_share_enum_reply,
SMB2_SHARE_INFO_1 and SMB2_SHARE_TYPE_*. The Linux libsmb2 CI lane
builds libsmb2 from upstream master, so smb_enum_shares stopped
compiling there.
Map the names once, keyed on SMB2_SHARE_TYPE_DISKTREE, which only the
new header defines; the async call and the level-1 netname/type
fields are the same in both. Builds against the headers before and
after that commit, and with HAVE_RETROSMB, which does not use this
path.
The task queue refuses a blocking task while another runs, and four
pushers ignored the refusal. content_load_state() had already marked the
load pending, so a load pushed while a save state, a screenshot or a
Wi-Fi or Bluetooth task ran left the flag set for the rest of the
session: closing content waited on it forever, and a movie recording
never started. It now clears the flag, frees the task and returns false,
which every caller already handles. The undo save state, the Wi-Fi and
the Bluetooth pushers leaked the refused task and reported success; they
free it and return false.
samples/tasks/save_state checks that a load pushed while a save runs is
refused and not left pending - both fail without this change - with
LeakSan covering what the refusal frees.
The IT loader scaled the mix-volume byte by 4/3 to sit on libxmp's IT
levels, which run about 2.5 dB over Impulse Tracker itself. Against
it2play and OpenMPT every IT file came out 2.2-3.8 dB hot, and loud
songs clipped: Deus Ex - The Illuminati hard-clipped 7065 samples
(OpenMPT: 430), Jazz Jackrabbit 2's Streetwize 1941 (44).
Map the mix volume straight onto the gain, honouring its full 0-128
range (128 was masked to 0 and played at the default). Four test
files now sit within 0.7 dB of it2play and OpenMPT, and clip about as
often as OpenMPT does (398 and 37 samples).
IT2 runs its resonant filter on the raw sample, scaled to 15 bits, and
clamps each output to 16 bits before the voice volume applies, so a
resonant peak clips at twice the sample's full scale. rmodtracker
filters after the volume, with headroom that never clips, so high-Q
sweeps through a loud sample came out several dB over IT2 and OpenMPT:
through_the_infinite.it's sweeps peaked about 4 dB hot.
Clip both filter paths at twice the side's gain, which is where a
full-scale sample lands in the buffer. Against an it2play render the
share of 50 ms windows more than 6 dB off drops from 3.7% to 2.5%
(OpenMPT: 4.8% to 1.9%); files without filters are unchanged.
The float path also kept its memory in the integer fields scaled by
32768, which overflows once a clipped value passes 65535 buffer
units; give it float fields of its own.
The Win32 companion went through the ANSI entry points with the core's
UTF-8 strings, so on NT every name outside ASCII was read in the local
code page: CJK, Korean or Thai file names in the browser and playlists,
and translated labels in menus and titles, came out as mojibake, and a
path the code page cannot hold could not be opened, picked or dropped.
Its windows, controls, menus, list rows and columns, tabs, status text,
file and folder pickers, dropped files, icons and message boxes now go
through the W entry points, converting at the edge - the virtual list
filling the control's own buffer row by row - and Windows does the font
fallback. A Win9x build (LEGACY_WIN32) keeps the ANSI entry points and
converts to and from the local code page instead of passing UTF-8
through.
The Win32 companion test reads rows as UTF-16 and browses a directory
holding a file named in CJK, Hangul and Thai, which now shows as itself;
the test script runs Wine in a UTF-8 locale so the file can be made.
Where there is mmap(), a font file is mapped rather than read whole: a
face's pages are read in as its glyphs are drawn and stay the file's,
so the 4.4 MB CJK fallback face costs what is drawn from it - 2 MB with
900 distinct characters on screen, less in a menu that shows a few.
Win32, the consoles and builds without mmap() read the file as before.
font_driver.c includes config.h first, as vfs/vfs.h - which
streams/file_stream.h pulls in - decides from HAVE_MMAP whether the VFS
maps at all.
A mapping does not survive its file being truncated, so archive
extraction now writes each member beside its file and renames it into
place, falling back to writing in place where a rename cannot be done.
An Update Assets run while fonts are in use leaves them reading the
file they mapped, and an extraction stopped part way leaves the old
file whole rather than cut short.
The integer resonant filter took its error term as acc - ( y << 14 )
and stored its clamped output back as y << 2, both undefined for a
negative y (UBSan flags them on any filtered IT). Multiply instead;
the result is the same on every target.
New-note actions move continuing notes to a shared pool of background
voices, stealing the quietest when it is full. Songs with continue or
fade actions on many channels exhaust 32 voices: through_the_infinite.it
keeps around 14 voices per channel ringing, and with 32 the stolen
voices left parts of each phrase several dB short of Impulse Tracker.
At 64 its output stops changing (128 and 256 give identical renders),
for 32 more struct channel copies per replay.
IT subtracts the instrument fadeout from 1024 each tick (it2play and
OpenMPT both load the IT 2.x value as is). The engine fades from
32768, so the step is fadeout * 32, not * 64; instruments faded out
twice as fast as they should.
The IT loader halved the global volume into the S3M/XM 0-64 range and
shared their effect handlers, so Vxx clamped at 64 (V40, half volume
in IT, played at full) and Wxy slid by XM rules: WF1, a fine slide
down by 1, became a slide up of 14 per tick. through_the_infinite.it
ends with WF1 / W00 to fade out and instead stayed at full volume.
Keep IT's global volume in 0-128 and scale it accordingly, let Vxx
set up to 128, and slide as IT does: W0y and Wx0 on the non-row ticks,
WFy and WxF once on the row.
IT starts the fadeout when a volume envelope reaches its last node,
but a sustain loop keeps it from getting there while the key is held,
even when the sustain point is the last node. rmodtracker only
excluded normal loops, so notes on such instruments faded out while
still held; channels 9-11 of through_the_infinite.it were silent for
most of the song. Exclude sustained envelopes too.
Impulse Tracker starts a new voice for every note, whether or not the
row has an instrument number: the envelopes restart, the key goes down
again, fadeout resets and the instrument's initial filter applies.
rmodtracker did this only when an instrument was given, as XM does,
so on a run of bare notes the volume envelope ran out once and the
channel faded to silence for good. Channel 2 of through_the_infinite.it
was silent from 16 s to 56 s this way.
Do the reset for IT on any non-porta note without an instrument; the
volume still carries over, as in IT.
The IT loader ignored Zxx, so songs that sweep the resonant filter
with it played unfiltered: through_the_infinite.it uses Zxx 1861
times and came out 12-17 dB too bright between 3 and 8 kHz.
Reduce the MIDI macros to the filter controls the engine has. Without
an embedded configuration Impulse Tracker's defaults apply: SF0
makes Zxx 00-7F set the cutoff, and Z80-Z8F set the resonance in
steps of 8. An embedded configuration (special bit 3, after the
optional edit history) is parsed for the same forms: "F0F000" for
the cutoff and "F0F001" for the resonance, followed by z or a fixed
hex value; other macros are ignored. SFx picks the macro per channel.
The winraw_abs_pos_test step installed gcc-mingw-w64-x86-64 and
wine64 with a bare apt-get install: no timeout, no download-first
retry, and -qq hiding all progress. A stalled mirror left the step
silent until the job's 20-minute timeout cancelled the run.
Add both packages to the "Install dependencies" list so they go
through the same bounded, retried download as the rest, and drop the
in-step install. Also kill wineserver on exit so a lingering server
cannot keep the step's output open.
The 3DS swizzled the atlas into its font texture once, at init, and
never again, so every glyph rasterized afterwards - all fallback-font
text, and whatever took a cell another glyph had held - drew from cells
the texture never received. Each message now swizzles the atlas's dirty
rectangle into the texture first. The 3DS, Wii U and PS3 drivers then
let the atlas grow - to 1024, 1024 and 2048 px - making texture memory
of its new size, and for the Wii U a texture-size block too, and keeping
what it replaces until the font is freed: a frame in flight may still
draw from it, and freeing it there means waiting for the GPU, which
none of them does for this. The atlas growing at most twice keeps that
to a third more memory. The 3DS's never-enabled VRAM variant of the
texture goes.
After the INT_MAX check, an XM sample header could still claim close
to 2 GB of data. A small file could make the loader allocate ~4 GB of
silence, and once that succeeded, offset + count in data_sam_s8 (and
offset + count * 2 in data_sam_s16le) overflowed, so their end-of-data
clamp failed and they read past the file buffer; the running offset
sum after each sample could overflow as well.
Clamp the sample size to the bytes remaining after the sample data
offset, counting the ADPCM table and two samples per byte for ModPlug
ADPCM. Samples that fit in the file are unchanged; a truncated file
now gets a sample ending at the end of the file rather than one padded
with silence.
The IT loader bounds the frame count, but reads the loop start into a
signed int and only checks that the loop end lies after it and within
the sample. A loop start of 0x80000000 or above becomes negative and
passes, leaving the loop sentinel copy and the mixer reading before
the sample buffer. Drop the loop when its start is negative.
The S3M loader read the 32-bit sample length, loop start and loop end
into signed ints. Large values turned negative and passed the loop
checks: a length of 0xFFFFFFFF with a looped sample left loop_start at
-1, so after calloc(0) the loop sentinel was written before the sample
buffer, and a negative loop start with a small loop end left the loop
indexing before the buffer at load and during playback.
Keep the fields unsigned until they are bounded, limit the length to
0x1000000 frames as the IT loader does, clamp the loop end to the
length and drop the loop when it is empty. Well-formed files load the
same loop as before.
GDI already made its atlas DIB again when the atlas's size changed, and
the Switch blits glyphs from the atlas in memory; both offer it room to
grow and ask for it before drawing. The PS2 makes its EE copy of the
atlas again at the new size and has the GS sent it again, as it does
for the menu texture, with the atlas kept to 512 px for the GS's 4 MB.
The Vita makes a texture at the new size and keeps the one it replaced
until the font is freed - a frame in flight may still draw from it, and
freeing a texture there means waiting for the GPU - which the atlas
growing at most twice keeps to a third more texture memory.
XM loop fields are read into signed ints. An oversized loop length can
become negative and pass the end-only check, causing a heap write
before the sample buffer when the loop sentinel is copied.
Keep the fields unsigned until they are checked against the sample
length. Reject sample sizes that do not fit the decoder's signed
counts, and guard ping-pong loop doubling against integer overflow.
connmanctl_tether_toggle pasted the AP name and password from the
user's tether config unquoted into a popen() command, so a space,
quote or ';' in either one split the arguments or ran as shell syntax.
Pass each as a single-quoted word, with embedded quotes escaped, and
refuse to run a command that would not fit rather than truncate it.
Turning tethering off no longer appends the two empty arguments the
callers pass; the command was already "connmanctl tether wifi off"
once the shell dropped them. Also stop on a failed popen().
connmanctl_get_connected_servicename used its malloc'd line buffer
without checking for NULL, and only freed it when an online service
was found; when the loop ran out of services the buffer leaked. Bail
out on a failed allocation and free the buffer on the not-found path.
connmanctl_valid_wifi_service_id returned false from inside its loop
on a disallowed character and again after it when no terminator was
found. Break out of the loop instead so both cases use the single
return false below it. No change in behaviour.
D3D12 makes its font texture again when the atlas grows and retires the
old one behind the queue fence - the value the frame being recorded
signals at its end - releasing it and its descriptor once the fence has
passed, so nothing waits on the GPU and no command list in flight loses
what it names. D3D8 and both D3D9 drivers already remade their texture
when the atlas's size changed; they offer the device's largest texture
and ask for the atlas before drawing. Metal makes its managed buffer and
the texture on it again, a command buffer in flight keeping the old pair
alive. The SDL1, Xv, Exynos, OMAP and OGA drivers blit glyphs straight
from the atlas in memory and only offer it room to grow.
The GL1, SDL2, SDL3, D3D10 and D3D11 drivers offer their fonts' glyph
atlas room to grow, up to the largest texture each can make - from
GL_MAX_TEXTURE_SIZE, the renderer's reported limit, D3D10's, and
D3D11's for its feature level - and ask for the atlas before laying a
message out, remaking their texture when it has grown: GL1 respecifies
its texture, the SDL drivers already remade theirs at the atlas's size,
and D3D10 and D3D11 release and make it again, the device keeping the
old one alive for draws already issued. On GL1 and SDL2, thirty titles
of thirty CJK characters each in XMB's History now draw every glyph
where SDL2 drew the stand-in past the 256th.
ConnMan prints discovered SSIDs directly in its service list. An SSID
with embedded newlines can forge a row whose final token is treated as
a service ID. Connect or disconnect then interpolates that token into
a popen() command, allowing a nearby AP to execute shell commands.
Accept only bounded Wi-Fi service IDs made of ASCII letters, digits,
and underscores. Check IDs again at command sites, including names
found in the settings directory.
webdav_create_digest_auth set cnonce to the constant "1a2b3c4f" for
every login. The cnonce is the client's own input to the digest
response; with it fixed, a hostile server (or anyone in the middle of
plain HTTP) chooses every other input and can test password guesses
against precomputed tables.
webdav_create_cnonce fills a 33-byte buffer in the state with 16 bytes
from crypto_random_bytes as hex, or, without HAVE_CRYPTO or when no
entropy source answers, an MD5 of the microsecond clock and a counter.
The buffer replaces the const char pointer, so nothing is allocated.
Tested against a local server that checks the digest response: before,
every login sent cnonce=1a2b3c4f; after, each run sends a new 32-digit
cnonce, and the server accepts the response both times and the sync
completes. Builds clean with make; webdav.c compiles as gnu89 with and
without HAVE_CRYPTO.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The comment added with the GL_STREAM_DRAW change quoted a single
radeonsi measurement. Those figures will go stale and are already
recorded in that commit's message; keep only the reason for the hint.
gl_glsl_set_vbo re-specifies the GLSL backend's vertex buffer with
glBufferData on every draw whose coords changed - each font flush and
menu quad, many times a frame - but declared it GL_STATIC_DRAW, the hint
for data written once and drawn many times. GL_STREAM_DRAW is the hint
for this pattern; the glcore driver's scratch VBOs already use it.
Measured: Ozone's idle menu on the gl driver spends 2.7% of its time in
this upload (perf, radeonsi). A windowless EGL bench on the same GPU
(RX 6600, radeonsi) of the font path's pattern, twelve ~6 KB uploads
and draws a frame: STATIC 278 us/frame wall, 99 us CPU; STREAM 228-236
us wall, 84-87 us CPU, in two runs. GL_DYNAMIC_DRAW measured like
STATIC. The hint changes no results, only placement.
Checked: the Ozone menu runs on the gl driver (Xvfb, llvmpipe) with no
GL errors.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
screenshot_dump only writes new_screenshot_dir when screenshot_dir is
set, then tests !*new_screenshot_dir to fall back to the content
directory. With the screenshot directory left at its default (empty),
that test reads uninitialised stack, so the screenshot can go to a
directory named from leftover bytes, which path_mkdir then creates.
Repro: null drivers, screenshot_directory = "", content loaded
(fceumm, an NES zip), --max-frames=30 --max-frames-ss. Under valgrind,
master reports 55 errors in 35 contexts, all in screenshot_dump
(conditional jumps, mkdir/openat on uninitialised bytes), and the PNG
was written to a new directory named with garbage bytes. With this
change: 0 errors, and the PNG lands beside the content.
Co-authored-by: Claude Opus 5.5 <[email protected]>
A font's glyph atlas grows when a frame needs more glyphs than it has
cells for: at the next frame, before anything is drawn from it, its
columns and rows double - to 1024 cells, then 4096 - with every cell
staying where it was, so no glyph's offsets change. A video driver
offers it by setting the atlas's max_width and max_height to the largest
texture it can make and asking for the atlas before laying a message
out; it then remakes its texture at the new size. The GL, GL core and
Vulkan drivers do - GL respecifying its texture, GL core making a new
immutable one, Vulkan making a new staging and sampled pair and parking
the old one for the frames still drawing from it. Thirty titles of
thirty CJK characters each, in XMB's History, draw every glyph on all
three; a driver that has not opted in keeps the atlas at its first size
and draws the stand-in past it, as before.
The font_lifecycle frame test covers the growth: deferred to the next
frame, every earlier glyph in place, then the whole frame drawn, and a
second growth past 1024 cells.
The stub's real-file branch declares its length before the read log
is taken under its lock, so the block opens with its declarations as
the coding guidelines ask.
A glyph cache hands out its least recently used cell for a new glyph,
and once a frame drew more distinct glyphs than the cache holds - 256 a
font - it handed out cells that glyphs earlier in the same frame were
already queued to draw from, so those drew as other characters, and
every glyph past the 256th was rasterized again and the atlas uploaded
again every frame. A cell looked up in the current frame is now left
alone until the next one begins: past what the cache holds, the rest of
a frame's glyphs go undrawn - their stand-in is drawn - and nothing is
rasterized again until the text changes. 400 distinct CJK glyphs a frame
cost 27 us instead of 1.4 ms, and nothing is drawn from a cell that has
changed under it.
font_driver_frame_begin() marks a frame: the video thread marks it
before the driver's frame when video is threaded, video_driver_frame()
otherwise. Until the first mark a cache is plain LRU, as it was.
samples/gfx/font_lifecycle gains a test of it.
The fallback tests read fonts on threads of their own, and the stub's
record of which files were read was written by two threads at once in
the fallback test, which TSan reports. It is kept under a mutex, and the
chain test asks the stub whether a file was read instead of walking the
record itself.
Ethiopic, its supplement and its extensions are drawn from
ethiopic-fallback-font.ttf in the assets' pkg/ when the font has no
glyph for them; the general fallback has none. The syllabary needs no
shaping, so the face renders it correctly a glyph at a time. Assets
without the file leave Ethiopic as undrawn as it was. The chain test
checks Ethiopic is looked for in its own face.
A codepoint the font has no glyph for is drawn from the fallback face
its block picks, and where that face has none either, from a second one
known to carry the range with the fallback fonts the assets ship:
Hangul jamo, circled numbers and vertical and small form variants from
the CJK face, circled Hangul from the Korean one, Yijing hexagrams from
the general one. That is close to 400 more glyphs drawn - the circled
numbers in Japanese titles among them - and a codepoint no face is
known to carry still has no other file read for it.
samples/gfx/font_lifecycle gains a chain test, DejaVu Sans Mono standing
in for the general face and DejaVu Sans for the CJK one, run under ASan
and TSan in Linux-samples-gfx.yml.
font_driver_resolve_params() fills in what a video driver's font draws
a message with: the caller's font_params, or for the on-screen message
the position and colour from the settings, left aligned, full screen,
opaque, under the usual 2 px shadow at 0.3 of the colour - with the
colour both as floats and as bytes, and the caller's float colour passed
through. Every video driver that drew from font_params resolves them
there instead of copying them field by field and reading the settings
itself; what is each driver's own stays with it - its packed colour
layout, the "alpha 0 means opaque" fix-up where it had one, and the 3DS
and PS2 drivers' own shadow for the message - and every driver ends up
with the same values as before. SDL2's and the Switch's on-screen
message take its position, and SDL2 its colour and shadow, from the
settings as the others do, where both drew it at the screen's corner.
The compile matrix's clang font layout lanes reject a declaration after
a statement, apart from the PS2 and Wii U drivers' C99 toolchains.
core_option_manager_new allocated opt->cats with calloc(_len, ...),
_len being the number of options, then filled it by category count.
A core declaring more categories than options writes past the end of
the heap block.
Repro: a minimal test core passing RETRO_ENVIRONMENT_SET_CORE_OPTIONS_V2
with four categories and one option, null drivers, under valgrind:
master reports 35 errors in 13 contexts, all invalid writes from
core_option_manager_new; with this change, 0. Every read of opt->cats
is bounded by cats_size. fceumm, which uses categories, loads with no
valgrind errors.
Co-authored-by: Claude Opus 5.5 <[email protected]>
font_renderer_get_message_width() in font_driver.c measures text through
a font's glyph cache - the UTF-8 walk, the lookup with its '?' stand-in
and the summed advance - and every video driver's get_message_width
returns it, where each walked the text itself. Its loop is
gfx/font_measure.h, a statement block in the manner of
gfx/font_layout.h; Vulkan and Metal, which upload an atlas cell as each
glyph is looked up, include it with that upload as their hook, so no
loop pays a call per glyph. The widths and uploads are the same as
before; the 3DS driver no longer dereferences its font before checking
it.
The GDI driver's font lays text out through gfx/font_layout.h, which
leaves no video driver with a text layout of its own. The layout block
splits the message into lines and hands each one over, through
FONT_LAYOUT_SKIP, to be drawn as its shadow run and then its text run,
so the text still lands on the shadow; a run walks its glyphs through
the block too. Untinted text is AlphaBlended a glyph at a time as
before, and tinted text baked into the scratch DIB and blitted per line;
a tinted line that is also right or centred is measured once rather than
twice. The Windows 95 path, which hands the line to TextOut, is as it
was, and the calls into GDI and the scratch contents are the same as
before.
The layout block documents fl_m, the current line's first byte, as
readable from its hooks, which is how a driver drawing a line in passes
of its own gets the line. The compile matrix's clang font layout lanes
cover GDI.
The software drivers that blit their own on-screen messages walked them
a byte at a time, so a multi-byte character came out as one Latin-1
glyph per byte - nothing a fallback font could help with - and a line
break was looked up as a glyph. They lay messages out through
gfx/font_layout.h instead: UTF-8, a codepoint the font cannot draw shown
as '?', and each line one line height below the last. Each blit is
unchanged; a glyph left out at the screen's edge now advances the pen,
where the next one used to be drawn in its place.
The OGA driver hung on a codepoint its font had no glyph for, never
stepping past it, and wrote past its message surface once the text
filled it. It wraps a glyph before it would cross the right edge, draws
nothing once a row would cross the bottom, and reports no more height
than the surface has.
The Metal driver's font lays text out through gfx/font_layout.h, one
method where it had a per-line method, a wrapper splitting the lines and
a width method called per line for right and centred text. It keeps its
pixel-snapped line origin, its six-vertex quads in the vertex range and
its atlas-cell update after every glyph lookup, the measuring pass's and
the '?' stand-in's included. The message length is passed down rather
than rescanned, and the vertices and cell updates are the same as before.
tools/platform_stubs/metal stands in for the parts of the macOS SDK the
driver uses - Foundation, AppKit, Metal, MetalKit, QuartzCore, simd - in
the shapes the SDK gives them, and the compile matrix compiles
gfx/drivers/metal.m as Objective-C against them with clang. A selector
the stand-ins do not declare is an error there, so a new Metal call
needs its declaration added with it.
The SDL3 driver's font lays text out through gfx/font_layout.h, as SDL2's
does: one function where it had a per-line function and a wrapper
splitting the lines, keeping its top-left pixel origin, floating-point
pen, atlas upload at the start of each line and SDL_RenderGeometry
batches of up to 64 glyphs on the stack. A right or centred line is
measured in the layout's own pass rather than by its width function,
with the '?' stand-in known before it, and the vertices and draws are
the same as before.
The compile matrix's clang font layout lanes cover it where the SDL3
headers are installed.
Both D3D9 drivers' fonts lay text out through gfx/font_layout.h. Each
looks a glyph up once for a line's shadow run and its foreground run,
drawn in that order as before, where it used to measure and walk the
line again for each run, and the draw both runs repeated is a function
of its own. Empty lines are passed over before any lookup, as they were.
The vertices and draws are the same as before.
The compile matrix's clang font layout lanes cover both.
D3D10, D3D11, D3D12, the Wii U and the Switch leave out a line too long
for their buffer, or past their limit, and before drawing through the
shared font layout they never looked its glyphs up. Through the layout
they did, measuring and walking a line they then skipped - rasterizing
glyphs nobody would see and evicting ones that were in use. Each now
names the line in FONT_LAYOUT_SKIP, so it is passed over before any
lookup, as it was.
The fallback fonts load on a worker that needs threads and a lock-free
compare-and-swap. Where the atomics backend has none (the PS2 one), the
loader was compiled out but its file table and reference slots were not,
and -Werror stopped the atomic-backend lane on them. FONT_FALLBACK_LOADS
is now decided before both and guards them with the loader.
The lane compiled gfx/drivers/vulkan.c against the system's
<vulkan/vulkan.h>, which the runner does not have, and failed there. It
now takes the Khronos headers the tree carries in gfx/include, as the
real builds do, and runs on any runner.
The 3DS, PS3 and D3D8 drivers' fonts lay text out through
gfx/font_layout.h. The 3DS keeps its vertex cache, its capacity check
ahead of each line and its per-line draw, now a function of its own; the
PS3 keeps its integer line origin, six-vertex quads and 64-glyph chunked
draws. D3D8 looks each glyph up once for a line's shadow run and its
foreground run, drawn in that order as before, where it used to measure
and walk the line again for each; its scissor cull now runs before the
line is measured, so a line out of view is not looked up at all. All
three produce the same vertices and draws as before.
The layout block takes an optional FONT_LAYOUT_SKIP, for a driver that
can tell a line will not be drawn before looking any of it up. The
compile matrix's clang font layout lanes cover D3D8.
A right or centred line is measured before its first glyph is drawn, so
the glyph that stands in for a codepoint the font cannot draw has to be
known by then. It was looked up as each line started, after the
measuring pass: the first such line was measured without it, and the
text drawn off by the stand-in's advance. It is looked up once, ahead of
the layout.
The PS2, Switch, Vita and Wii U drivers' fonts lay text out through
gfx/font_layout.h, one function where each had a per-line function and a
wrapper splitting the lines. Each keeps its own line origin and the way
it draws a glyph: PS2 its per-line alpha state and gsKit sprites, the
Vita its atlas copy and tinted draws, the Wii U its vertex cache, its
truncating pen and its per-line draw and capacity check, the Switch its
software blit and its line-length limit. The text length is passed down
rather than rescanned.
The Switch blit counted rows with the line's own y origin, so every glyph
after the first on a line was drawn at the height the previous one ended
on; it has counters of its own. The compile matrix's clang font layout
lanes cover the four, and the Vita driver no longer leaves its shader
patcher's result set but unread.
The GL1 and SDL2 drivers' fonts lay text out through gfx/font_layout.h,
one function where each had a per-line function and a wrapper splitting
the lines: GL1 keeps its integer line origin, six-vertex quads and
64-glyph chunked draws; SDL2 keeps its top-left pixel origin, its
floating-point pen, its atlas upload at the start of each line and its
SDL_RenderGeometry batches. The text length is passed down rather than
rescanned, SDL2 measures a right or centred line in the layout's own pass
instead of calling its width function per line, and both draw the same
pixels as before.
The compile matrix's clang font layout lanes cover both.
D3D10 measures a line with a shadow or an alignment first, then writes
each glyph once as its shadow and its foreground sprite, straight to
their slots - the line's shadows ahead of its glyphs, as before - where
it used to measure and walk the line again for each of the two. D3D12
keeps its single interleaved pass and its retroactive shift of right and
centred lines; the line splitting, glyph lookup and '?' stand-in are the
shared block's. Both keep their own truncating pens and their rule for
lines too long for the buffer, and write the same sprites as before.
The compile matrix's clang font layout lanes cover both.
A driver that draws through gfx/font_layout.h but has no use for one of
the values it reports - a line's glyph count for Vulkan and the GL
drivers, the width and the shared pen for D3D11, which keeps its own -
reads it as void in its macro, so GCC 15 and clang no longer warn that
the layout's counter is set but never used.
The compile matrix builds each driver drawing through the layout with
clang and the warning made an error.
The layout block also reports each line's glyph count and length. D3D11
measures a line first when it has a shadow, so every sprite is written
straight to its slot - the line's shadows ahead of its glyphs, as before
- instead of emitting the glyphs, cloning them and swapping the two
runs; right and centred lines are still shifted once their advance is
known, with their shadows taken from the shifted glyphs. The sprites are
the same as before, byte for byte, and lines too long for the buffer
are still left out.
The GL core driver's scRGB/HDR10 state lives outside the slang guard,
and what needs the slang pipelines is inside one: the context's HDR
framebuffer is taken up only with slang, the HDR10-to-SDR downconvert
and the end-of-frame scRGB encode are compiled only with it, and the
ribbon VBO helper only where its caller is. A build without slang - the
C89 lane is one - compiles again, and draws SDR as it always has.
The compile matrix checks gl3.c both with and without slang.
The GL core driver's font lays text out through gfx/font_layout.h, as the
Vulkan and GL drivers do: the line splitting, glyph lookup, alignment
width and pen advance are the shared block's, and the driver keeps its
own line origin, its six-vertex glyph quads and its 64-glyph chunked
draws. The text length is passed down rather than rescanned, and the
output is the same pixels as before.
After the first narration, SIGCHLD was set to SIG_IGN for the whole
process, so every later wait on a child - nmcli's connect and
disconnect among them - failed with ECHILD and read as failure. The
narrator stays a direct child and is reaped with waitpid(WNOHANG):
is_narrator_running_unix() reaps it once it has exited, so a finished
narrator never reads as running and its pid is not kept past its exit,
and one sent SIGTERM that has not exited yet is kept to reap on a later
call. Nothing waits, and SIGCHLD keeps its default action. As found in
#19647, which reparented the narrator instead; a finished narrator then
read as running until whatever adopted it reaped it.
gfx/font_layout.h is the text layout every video driver's font repeats:
splitting lines, walking the UTF-8, looking each codepoint up with '?'
standing in for one the font cannot draw, measuring a line for right and
centred alignment, and advancing the pen. It is a statement block that a
driver includes in its own render function, with macros for what is the
driver's own - where a line starts, how a glyph's quad is written in its
vertex format, and when its atlas is uploaded - so the loop compiles to
what the driver wrote out by hand, with no call or buffer per glyph.
The Vulkan and GL drivers draw through it, and produce the same pixels as
before. GL measuring a line for alignment now counts '?' for a codepoint
the font cannot draw, as drawing it does.
The compile matrix checks gfx/drivers_font_renderer/coretext.c and the
font driver's Apple branch, macOS and iOS, against stand-ins for the
CoreFoundation, CoreGraphics and CoreText declarations they use, so a
change to the rasterizer interface is caught before the Apple jobs.
samples/gfx/font_lifecycle gains a fallback test: two fonts with no
glyph for a set of codepoints, on two threads, ask for the same fallback
face at once, with real stb and real rthreads behind font_driver.c. The
file has to be read once and every glyph has to arrive. It runs in the
ASan/UBSan check and again under TSan in Linux-samples-gfx.yml.
The glyph cache - the 16 x 16 cell atlas, the codepoint hash, least
recently used eviction and the dirty rectangle - lives in font_driver.c,
and stb, freetype and CoreText are rasterizers behind it: they open a
face on bytes font_driver.c read for them, say whether it has a glyph
for a codepoint, and draw that glyph into the cell they are handed. The
video drivers keep calling get_glyph and get_atlas as before; a hit
costs the same hash walk it did in each renderer's own copy.
A codepoint the font has no glyph for is drawn from a fallback face
picked by its Unicode block - the CJK, Korean, Thai and general fallback
fonts in the assets' pkg/ directory - so a Chinese, Korean or Thai file
name shows in an English menu instead of boxes. Each fallback file is
read the first time a codepoint needs it, on a thread of its own, and
shared by every font afterwards; until it arrives that codepoint draws
blank and is not cached. A font that never meets such a codepoint reads
nothing and keeps nothing more.
* wifi: pass discovered SSIDs to nmcli without a shell
The nmcli scan path interpolated each nearby SSID into a system()
command. A nearby access point could execute commands as the
RetroArch user when the Wi-Fi network list was opened.
Query saved profiles and connect or disconnect using argument-based
nmcli calls, so SSIDs and passphrases remain literal arguments.
* wifi/nmcli: fill the nmcli argument vectors by assignment
C89 takes only constant expressions in an aggregate initializer, and the
argument vectors held the SSID and passphrase: the C89 lane rejected all
five. The vectors are now declared and filled in by assignment.
---------
Co-authored-by: libretroadmin <[email protected]>
9b5f6ed768 ("net/tls: certificates for IP addresses; no server name
for them") made net_http_connect() copy the TLS host name with
strlcpy(). compat/strl.h maps strlcpy to strlcpy_retro__, which lives
in compat_strl.c, but Makefile.dns_lifetime never linked that file.
The non-SSL build drops the call, so only the HAVE_SSL build failed
to link:
undefined reference to `strlcpy_retro__'
Add compat_strl.c to the link line. The test passes again with
SSL=1, SSL=0 and SANITIZER=address,undefined.
A nearby Bluetooth name can contain a newline that forges a line in
bluetoothctl devices output. The next status check or a connect action
used the forged line's second token in a popen() command, allowing
shell commands to run as the RetroArch user.
Accept only lines with a valid Bluetooth address, use that address
for status, connect, and remove commands, and cap the list to the
fixed status arrays.
A menu driver's context_reset loads every texture into its slot without
looking at what the slot held, so calling it on a live menu leaks the
whole icon set already loaded. menu_driver_context_rebuild() runs
context_destroy first and then context_reset against the running video
driver - the pair the menu goes through when it survives a video driver
swap - and the in-place resets use it: the Dynamic Wallpaper, Show
Sublabels and XMB Entry Icons settings, Start on the wallpaper entry,
and Update Assets.
Update Assets also reloads the notification widgets with
gfx_widgets_reload_assets(): the message queue and volume icons are
reloaded under the widget state lock, asynchronously so nothing waits
on the video thread while the lock is held, and the next layout pass
rebuilds every widget font on the thread that drives the widgets,
retiring the old ones, so the fonts from the new assets replace the
built-in one the widgets fell back on.
The camera poll in runloop_iterate() tested RUNLOOP_WORK_CAMERA and
then re-tested five more things every frame: settings->bools.
camera_allow, cb.caps, driver, driver->poll, data. The bit is meant to
mean "poll this every frame", so those conjuncts belong at the edge
that raises it, not on the frame path.
driver_camera_start() now raises the bit only when the start
succeeded, the driver can be polled, and the core registered caps
(data and driver are preconditions of reaching start at all). Stop
and teardown already drop it. The one condition that could change
underneath - the user flipping camera_allow off mid-session - gets
its own edge: the setting row's command trigger fires
CMD_EVENT_CAMERA_ALLOW_APPLY, which stops a running camera (and so
drops the bit). Allowing again does nothing by itself, as before: the
core has to call start() for the camera to come up.
A trigger rather than a change handler because the generated
settings glue (msg_hash, configuration.c, the string passes) only
defines the base S_BOOL variant for this row; other rows already pass
a CMD_EVENT_* through the same consumers, which discard it.
Two visible changes: a disallowed-while-running camera is now
actually stopped instead of merely no longer polled while still
capturing, and the frame path is one bit test. Frontend-overhead
harness passes.
The password challenge salt came from an LCG seeded with time(NULL):
anyone who saw one salt could predict every later one. It now comes from
crypto_random_bytes(), the platform entropy source on Windows, UWP,
POSIX and the 3DS, Vita and Switch; builds without the crypto library,
or a platform without a source, keep the LCG seeded from the
microsecond clock and the connection's address. The client's password
hash is compared with crypto_memeq_ct() rather than memcmp(), whose
early exit told how much of it matched. The salt travels in the same
field, so older clients are unaffected. As found in #19649.
The four-block NEON ChaCha20 ran on AArch64 only, though the Vita and
32-bit Android have NEON as well and are where ChaCha20-Poly1305 is now
negotiated first. It now runs wherever NEON is: ChaCha20-Poly1305 on
ARMv7 takes 14.7 instructions an octet, down from 32.
No job ran the crypto suite on 32-bit ARM, the consoles' architecture.
crypto-arm-check runs it under qemu-arm in their two shapes, ARMv7-A
with NEON as on the Vita and ARMv6K with VFP only as on the 3DS, where
every portable path is taken, and the libretro-common job runs it.
A TLS connection to an IP address - https://[::1]:8443/ or
https://192.0.2.1/ - was treated like one to a host name, which is
wrong in three places:
- net_http handed the TLS layer the host as it appears in the URL, so
an IPv6 literal kept its brackets ("[::1]"), which nothing can match.
The brackets belong to the URL and the Host header only; the TLS
layer now gets the bare address.
- The built-in client sent the address as server_name. RFC 6066
allows host names only there; for an IP address the extension is now
left out.
- x509_match_hostname() compared an address against dNSName entries
and the commonName as text, so a name certificate whose CN or a DNS
entry happened to read "10.0.0.1" vouched for that address, while a
certificate carrying the address as a proper iPAddress entry never
matched. An IP address now matches only iPAddress entries, byte for
byte (RFC 9525 6.3); a host name matches as before.
x509_parse_ip() is the new, exact parser behind that: dotted IPv4
without leading zeros, IPv6 with "::" compression and a trailing dotted
IPv4, brackets and a zone id allowed.
On the way: the built-in client kept the caller's pointer for the host
name, and net_http passes one that dies with the transfer while the
connection lives on in the pool - where a TLS 1.3 session ticket
arriving later is filed under that name. It keeps its own copy now.
BearSSL and mbedTLS get the bare address too; their own IP handling is
theirs (BearSSL's minimal verifier has none, so such a connection fails
verification, as it did).
Tests: the crypto suite parses valid and malformed literals and matches
a SAN with DNS, IPv4, IPv6 and a DNS-spelled address, plus an IP-reading
commonName. The local openssl matrix connects to 127.0.0.1 against a
leaf with IP:127.0.0.1 (refused before this change) and one with
DNS:127.0.0.1 (now refused).
Where AES runs in software - the 3DS, the Vita, older phones - AES-GCM
costs about four times what ChaCha20-Poly1305 does (116 against 32
instructions an octet on 32-bit ARM), yet the client offered it first,
and servers that follow the client's order chose it. The client now
offers ChaCha20-Poly1305 first when crypto_aes_hw() reports no AES
instructions; the TLS matrix checks a build without them negotiates it
on 1.2 and 1.3, through ssl_socket_retro_suite().
X25519 on machines without a 128-bit integer ran on the generic
Montgomery arithmetic, 6.1 million instructions an operation on 32-bit
ARM. It now has arithmetic of its own mod 2^255 - 19 in ten 32-bit
limbs of 26 and 25 bits, every product one 32x32->64 multiply: 1.9
million on the Vita's ARMv7, 2.1 million on the 3DS's ARMv6.
A download to a file - task_push_http_download_file(), which the core
updater and playlist thumbnails use - that lost its connection part way
through failed outright and deleted what it had. On a flaky link a
large core or asset bundle could fail at 90% every time.
It now picks up where it stopped, inside the same task:
- On the first response, a 200 that advertises "Accept-Ranges: bytes"
and a strong ETag (or else Last-Modified) makes the download
resumable. A weak ETag cannot validate a byte range.
- When the transfer then fails in transport mid-body - not an HTTP
error, not a full disk, not a cancel - the task opens a new request
for the same URL with "Range: bytes=N-" and that validator in
If-Range, N being what is already on disk, and keeps writing to the
same file.
- A 206 must continue exactly at N (Content-Range is checked); a 200
means the server sent the whole resource again - it ignored the range,
or If-Range found it changed - and the file starts over; anything
else fails the download and removes the file, as before.
- At most three resumes, each only after progress.
A completed resumed download reports status 200 to the callback, since
the whole file was delivered (the thumbnail callback checks for exactly
200), and progress covers the whole file rather than the last slice.
net_http gains net_http_header(), a lookup of a response header while
the transfer is still running, which is what the sink needs to decide.
Two task_http fixes on the way: a failed download to a file set its
error string twice and leaked the first, and a task cancelled before
its connection became a transfer leaked the connection.
New samples/tasks/http/http_resume_test drives the real task_http.c,
net_http.c and task queue against a scripted server: resumed once,
resumed twice, resource changed (the new body alone, not spliced),
wrong Content-Range, no validator (no resume tried), and a connection
that drops every time (fails after three resumes). Six of its checks
fail on the previous code, where every one of these downloads failed.
The Emscripten backend gets response headers from the browser as one
text blob and turns it into net_http's header block itself. Its test
could not build task_http_emscripten.c (emcc only), so it carried a
copy of the parser - and the copy had already drifted: it still split
into a string_list, while the backend had moved to the block.
The conversion is now net_http_headers_compact() in net_http.c: header
text in, the block net_http_headers_take() returns out, in place, with
CR and trailing spaces trimmed, blank lines dropped and the final line
kept without its newline. The Emscripten backend calls it on the buffer
the browser filled; http_emscripten_headers_test links net_http.c and
calls it too, so the response side has no copy left to drift. The
request side stays a copy, being static to the emcc-only file, as the
test's header says.
http_emscripten_headers_test: the same response cases (splitting,
blank and final lines, lower-cased names, NULL and empty input) against
the real function, under ASan and UBSan.
A response that ran past its Content-Length failed the transfer:
status -1, no body, although the whole body had arrived and was sitting
in the buffer. It can happen whenever the excess lands in the same read
as the body - with the header block, or with a sink, whose receive
window is not trimmed to what is left of the body.
The body is now complete at Content-Length. What follows it is garbage
or a response nobody asked for, so it is dropped together with the
connection - the same treatment as bytes after a response with no body.
net_http_semantics: "Content-Length: 2" followed by "okEXTRA-BYTES"
delivers "ok", with and without a sink, and the next request goes out
on a new connection. Both delivery checks fail on the previous code.
A 3xx whose Location is going to be followed already kept its body
out of the caller's sink, but without a sink it was still read in
whole: the receive buffer grew to the advertised Content-Length - up to
the 256 MiB ceiling, on the server's say-so - only for net_http_redirect()
to throw it away.
Such a body is now handled the way a sink body is - drained after every
read, never sized to Content-Length - except that it goes nowhere. One
predicate, net_http_body_streams(), replaces the sink tests on the
body path. The redirect body is still read through, so the connection
stays usable for the redirected request.
net_http_semantics: a 302 with an 8 MiB body is followed with no
allocation reaching 1 MiB (it reached 8 MiB before), on the same
socket.
net_http_send_request() wrote the request with blocking sends - a
dozen for the head, then the whole body - before returning. net_http's
sockets are non-blocking, so once the body outgrew the socket buffer,
socket_send_all_blocking() spun on EAGAIN until the server had taken
all but the last buffer's worth. Measured against a server reading at
2 MB/s, a 16 MiB PUT spent 6057 ms in one net_http_update() call, at
100% CPU. In an unthreaded build that call is on the video thread; in a
threaded one it holds up every task queued behind it. Cloud sync
uploads of saves and states are exactly this request.
The request now goes out like the response comes in:
- The head is built once into one buffer, byte for byte what the old
send sequence wrote: one send where there were a dozen.
- Each net_http_update() sends as much as the socket takes, at most
NET_HTTP_SEND_BUDGET (256 KiB, the drain budget's twin) per call,
then returns. Postdata goes straight from the caller's buffer; a
streamed body is pulled a 64 KiB run at a time into one buffer kept
for the transfer.
- A full socket marks the transfer blocked, so net_http_wait() waits
for it to become writable - no spinning, no sleeping; stopping on the
budget does not. Over TLS the last bytes are flushed out of the TLS
layer before the request counts as sent.
- The stale-pool replay and a redirect start the send over, after the
source rewind they already did.
Same 16 MiB upload: longest call 6057 ms -> under 1 ms, total time
unchanged (8.07 s), and CPU for the whole transfer 6.0 s -> 0.01 s with
a caller that waits between updates. Requests that fit in the socket
buffer go out in the first call as before.
cheevos_login_test: a 12 MiB PUT through net_http to a server that
stops reading for a second, plain and over TLS, must never spend
250 ms in one update; the longest is 4-8 ms. Plain and BearSSL spent
1020 and 1033 ms on the previous code. The request bytes the login
checks validate one for one are unchanged.
ssl_socket_send_all_nonblocking() had no callers, and none of the
three implementations was usable:
- mbedTLS took MBEDTLS_ERR_SSL_WANT_WRITE - an ordinary full socket -
for an error, and on success reported the whole length as written
whatever mbedtls_ssl_write() had actually taken.
- The built-in client sent the whole buffer blocking.
- BearSSL was close, but copied from a NULL buffer when the engine was
full and could not say whether bytes it had taken were still waiting.
The contract is now written down in net_socket_ssl.h: take as much as
fits without blocking and return that (0 when the socket is full, -1
on error); after a 0, call again with the same buffer, at least as
long; once the last byte is taken, call the new
ssl_socket_flush_nonblocking() until it reports 1.
- mbedTLS remembers the length of a write that returned WANT_WRITE and
repeats it, as mbedtls requires; a positive return already means the
records are on the wire, so its flush has nothing to do.
- The built-in client seals one record from the bytes it takes and
keeps whatever the socket did not accept in a per-connection buffer,
sending it before sealing the next. A blocking send drains it first
so records leave in sequence; close_notify is skipped behind a
half-sent record rather than wait on a peer that stopped reading.
- BearSSL pushes its pending records before taking more, refuses a
closed engine, and flushes in a loop while the socket keeps taking.
samples/network/cheevos_login gains a built-in TLS lane (SSL_BACKEND=
retro, trusting the test CA through ssl_socket_retro_set_trust_pem) and
a check that drives each backend's non-blocking send directly: a
12 MiB PUT to a mock /upload that stops reading for a second before
taking the body. Every call must return at once and the server must
receive every byte. mbedTLS fails it on the previous code.
The AArch64 AES-GCM path still did as x86 did before it changed: CTR
four blocks at a time, round keys reloaded and byte-swapped every round,
each counter block rebuilt through a scalar swap, and GHASH reduced
after every block. CTR now keeps eight blocks in flight with the keys
loaded once, the counter stepped by one add in a lane-reversed vector,
and GHASH folds eight blocks per reduction with Karatsuba multiplies:
AES-128-GCM takes 40% fewer instructions, 3.4 per byte down from 5.7.
ChaCha20 ran scalar on AArch64, where it is what TLS chooses on cores
without the AES instructions. It now runs four blocks at once on NEON,
which every AArch64 core has: ChaCha20-Poly1305 takes 60% fewer
instructions, 9.6 per byte down from 24. Counts are executed AArch64
instructions under QEMU, start-up subtracted.
net_http now waits on an eventcount, so tls_http, which links net_http
over the built-in TLS client, links rthreads/retro_eventcount.c as the
HTTP task samples do; without it the tool failed to link and the TLS
matrix stopped there.
An IPv6 address in a URL is bracketed because it contains colons
(RFC 3986 3.2.2): http://[::1]:8080/x. Both places that split host
from port took the first colon:
- net_http_connection_iterate()/_done() stopped the host at "[" and
refused the rest as a bad port, so such a URL could not be requested
at all;
- net_http_redirect() split the joined URL the same way, so a redirect
to a literal failed (or, with digits in the right place, went to a
junk host and port).
Both now treat a leading "[" as a literal running to its "]", with the
port colon only after it; an unterminated, empty or trailing-garbage
literal is refused. The domain keeps its brackets, which the Host
header needs ("Host: [::1]:8080"), and the resolver strips them before
getaddrinfo(), which wants the bare address. url_join needed nothing:
it never looks inside the authority.
Not covered: TLS to an IPv6 literal still passes the bracketed form
as the server name, as it passes any other host.
http_limits_test (net_http.c included whole, resolver stubbed): URL
split with and without a port and with a zone id, the three malformed
forms, the resolver's argument, and a redirect to http://[::1]:8443/v6.
All fail on the previous code.
An interim response (100, 102, 103) is followed by the real status line
on the same connection, and its headers were cleared before reading it
- but not what they had set. bodytype, len and content_len survived:
- "Transfer-Encoding: chunked" on a 100 stayed T_CHUNK, and since
Transfer-Encoding wins over Content-Length, the final response's
"Content-Length: 2" was ignored and its body parsed as chunks.
- "Content-Length: 5" on a 103 stayed T_LEN 5, so a close-delimited
final body was cut after five bytes and the connection pooled with
the rest unread.
Interim blocks now reset the framing along with the headers. Costs
three stores per 1xx block; nothing on the body path.
Two new net_http_semantics checks, both failing before.
A received record was decrypted in place, copied to a plaintext buffer,
and copied again into the caller's. The plaintext now stays where it was
decrypted, and when the caller's buffer holds the whole record - every
net_http read, which asks for 32 KiB or more - it is decrypted straight
into it: a download takes 31% fewer instructions. The AES-GCM key
schedule is built once per key rather than once per record.
The end of the stream came back as 0 bytes, which net_http takes for
'nothing yet': a body that runs until the server closes the connection
never finished, every byte received and the transfer polling forever.
It is now -1 with the error flag, as plain sockets and the library
backends report it. tls_http drives net_http over the built-in client in
the TLS matrix, four version and cipher pairs, checking the body by hash
and failing a transfer that never completes.
The WWW-Authenticate challenge is whatever the server sends with a
401, and the parser trusted its shape:
- realm, nonce and opaque took strchr(ptr, '"') + 1 - ptr as a length.
With no closing quote strchr() returns NULL, the length is a huge
value, and malloc() of it fails or the strlcpy() that follows writes
through NULL.
- An unknown unquoted parameter was skipped with
while (*ptr != ',' && *ptr != ','), which never checks for the end
of the string, so one at the end of the line ("stale=FALSE") read on
past it until it happened to meet a comma.
- qop and unknown quoted values had the same missing-quote hole.
Every value is now found with a bounded scan, and a challenge that
cannot be parsed is refused - which the caller already handles as
"cannot re-authenticate". Quoted values go through one helper that
copies them out and replaces any earlier copy, so a repeated parameter
no longer leaks the first; the HA1 hash is recomputed rather than
leaked when realm repeats. qop is read as a list whose members may be
spaced ("auth-int, auth" was refused), and a quoted algorithm is
accepted.
webdav_reauth_test parses well-formed and malformed challenges from
exactly-sized buffers under ASan. The previous parser aborts on the
first malformed one.
webdav_needs_reauth() now walks the response's header block with
net_http_header_next(), which hands out const char *, and passed each
line to webdav_create_digest_auth(char *) - a -Wdiscarded-qualifiers
warning. The parser only ever reads the challenge (every field it
keeps is copied out with strlcpy/strdup), so its parameter and cursors
are const now.
Update Assets rebuilds the menu's own context once the archive is
extracted - context_destroy then context_reset against the running
video driver, the same pair the menu goes through when it survives a
video driver swap - instead of CMD_EVENT_REINIT. The menu reads its
icons and fonts from the assets directory when that context is built,
so the new assets show at once without tearing down and recreating
the video driver, its context and its swapchain.
Texture and font releases go to the video thread under the threaded
wrapper, as on the driver-swap path. The notification widgets keep the
assets they loaded until the video driver is next initialised.
getaddrinfo() with AF_UNSPEC lists IPv6 addresses next to IPv4 ones,
often first. On a system without IPv6 - a kernel or console network
stack built without it - socket() refuses AF_INET6, and that ended the
attempt in two places:
- net_http_new_socket() created the socket for the first address only,
so an IPv6 address listed first failed the transfer outright with
"socket_create_failed", IPv4 addresses untried;
- the connect loop moved on with socket_next(), which returns -1 when
socket() fails for the next address, and the loop read that as the
end of the list, so an IPv6 address mid-list hid every address after
it.
Both now go through net_http_socket_for(), which creates the socket for
the first address at or after the current one that the system accepts,
and the connection remembers that address's family so connect starts
there. An extra socket() call happens only on a refused family; the
common case is unchanged. Full Happy Eyeballs (racing connects) is not
attempted.
http_limits_test, with socket() stubbed to refuse AF_INET6: an IPv6
address first connects over the IPv4 one after it, and with the first
IPv4 address refusing the connection and an IPv6 one next, the IPv4
address after that is still tried. Both fail on the previous code.
Every keep-alive connection a finished transfer handed back stayed in
the pool until the server closed it and select() noticed. Nothing
bounded how many - each one a descriptor here and a slot on the
server - and nothing aged them. A connection whose NAT mapping or
server keep-alive timer expired silently is never reported readable,
so it was reused: the request went into a dead socket and the transfer
waited out its timeout.
A finished transfer now hands its connection back through
net_http_conn_pool_release(), which stamps it idle and, past 8 idle
connections, closes the one idle longest. The pool sweep that already
runs when a transfer looks for a connection closes any idle for more
than 30 s before it even asks select(). Reconnecting costs a handshake;
a dead socket cost the whole timeout.
Neither touches the receive path: the release runs once per finished
transfer and the reap once per pool lookup. With the pool owned by the
driving thread, neither takes a lock.
http_limits_test: 20 finished connections leave 8 idle (all 20 stayed
before), the longest-idle go first, and a connection is reused at 15 s
idle and closed at 30 s.
Every distinct host:port added a cache entry, and a successful one had
its timestamp bumped on every hit, so it lived for as long as it kept
being used. A session that touches many hosts - thumbnail mirrors,
cloud sync, netplay lobbies, core and asset downloads - grew a list
that is walked on every connect, with no bound.
Inserting a new host:port now evicts the least recently used entry
once there are more than 32. A lookup still in flight is never the
victim: its resolver owns the entry until it publishes. The cap costs
one walk on insert only; a cache hit and the receive path are
untouched. The expiry sweep also reads the clock once per walk rather
than twice per entry.
New samples/tasks/http/http_limits_test (net_http.c included whole,
clock and resolver stubbed), threaded and unthreaded, under ASan+UBSan:
100 distinct hosts leave 101 entries before this change and 32 after,
the newest kept and their addresses freed as they go.
net_http took a mutex around every DNS cache and connection pool
access and parked DNS waits on a condition variable, on the premise
that any thread might drive a transfer. None does: in RetroArch every
net_http_new/update/wait/delete comes from task_http on the task
queue's worker, or on the main thread with threaded tasks off, and
switching between the two stops the queue first, so the handover is
ordered by the join. The only real concurrency is the DNS resolver
threads, each of which fills in one cache entry.
So the cache and pool now belong to the driving thread, and nothing
takes a lock:
- A resolver reads its entry's domain and port in place (they are set
before the thread starts and never change), writes addr, publishes
valid with a release store and touches the entry no more. The owner
reads valid with an acquire load, and reads addr, frees the entry or
joins its thread only after that. The strdup the resolver took under
the lock, a copy of something nobody could change, is gone.
- The pool loses its lock outright; only the driving thread uses it.
- connect no longer pins the cache entry with a user count: the pin
covered another thread's expiry sweep running mid-connect, which
cannot happen.
- net_http_wait() parks on a retro_eventcount the resolver notifies
after publishing, re-checking the lookup after registering so a
publish in between is never missed, and bounded by the caller's
timeout as before. net_http_init() creates it; there are no locks
left to create.
- net_http_deinit() can no longer deadlock: resolvers wait on nothing,
so joining one only lasts as long as its getaddrinfo().
The sweep still never joins a resolver that has not published - that
used to deadlock on the lock and would now stall the driving thread for
the whole lookup - and find() no longer reads addr before valid is
published, which TSan would report as a race now that no lock orders
the two.
The rule is documented next to net_http_init() in net_http.h. The
stress test's concurrency case drove six transfers from six threads,
which RetroArch never does; it now interleaves them on one thread as
the task queue does, with the resolvers running alongside under TSan.
The DNS lifetime test loses its nested-connect case (the cross-thread
sweep it modelled) and the init-failure test its resolver strdup case
(the strdup is gone). The tests that build net_http with HAVE_THREADS
link retro_eventcount.c, as Makefile.common and griffin already do
under HAVE_THREADS. The samples network job now fails if net_http.c
calls an slock_ or scond_ function.
Stress, transport-failure, task-lifetime, DNS-evict and the semantics
harness pass under TSan and ASan+UBSan.
Every transfer allocated a string_list for its response headers: the
struct, a 32-slot array, and a strdup per header line, then freed all
of it when the task finished - on the task thread, or on the video
thread in an unthreaded build. A typical response has 10 to 20 lines;
a 64-line one cost 67 allocations before a byte of body arrived.
The headers now go into one block as they are parsed: each line
NUL-terminated, an empty line at the end, grown by doubling from 1 KiB
and bounded by the existing 64 KiB header cap. It is allocated with
the first header line (a failed connect allocates nothing) and handed
over whole. In the harness, 64 header lines cost one allocation more
than one line does. The Location offset replaces the pointer into the
list, since the block may move as it grows.
API:
net_http_headers_take() the block, ownership to the caller (free());
same NULL rules as net_http_headers_ex()
net_http_header_next() walk the lines
net_http_header_value() first field by name, case-insensitive
net_http_body_is_framed() takes the block
net_http_headers() / net_http_headers_ex() remain for code outside
RetroArch, building a string_list from the block on demand; nothing
in-tree calls them.
http_transfer_data_t.headers is the block. task_http hands it over
and frees it; task_core_updater frees it; the emscripten backend
compacts the browser's header text into the same layout in the buffer
it already had, so it also drops to one allocation. WebDAV, S3 and
Google Drive walk it with net_http_header_next(), and S3's ETag lookup
uses net_http_header_value(). The WebDAV, S3, transport-failure and
cheevos harnesses build or take blocks; the S3 harness links net_http.c
for the helpers.
The menu acts on input collected before the frame polls, so the state
menu_event() reads is one poll old. The runloop stamps each menu poll
with the frame's time and hands menu_event() the stamp of the poll whose
state it is reading (input_time_us); auto-repeat measures the hold on
that clock. Time spent after a poll - Ozone and XMB build a whole
playlist synchronously when the selection lands on it - is not counted
as time the button was held, so one press moves the selection once,
while a button held across the stall repeats as soon as a later poll
sees it down. Steady-state repeat timing is unchanged.
samples/menu/nav_repeat_stall drives the real runloop with a scripted
input driver whose poll stalls, and runs in the Linux menu harness
workflow.
net_http_redirect() had three ways to read a Location. An absolute URL
allocated a whole connection object just to parse it; an absolute path
replaced the path; anything else went through
fill_pathname_resolve_relative, a filesystem helper that knows nothing
of URLs. "../up" was not resolved, "?q=1" was treated as a file name
and replaced the last path segment, a "//host/path" reference stayed
on the old host, and a fragment went out in the request line.
It now rebuilds the current URL, joins the Location onto it and parses
the result: one path for every form, and the place to add IPv6
literals or userinfo if a caller ever needs them. The fragment is
dropped, the port is validated, and the Location is not decoded -
"%20" must stay encoded in the request line. One scratch allocation
replaces the connection object and its copies on absolute redirects;
file_path.h is no longer needed.
Runs on 3xx only. 5 new checks, 4 failing before (the %20 one guards
against decoding).
Master could encode a URL but not decode one, and had no way to
resolve a reference against a base URL: redirects did it by hand in
three branches, one of them borrowing fill_pathname_resolve_relative,
a filesystem helper. The ideas come from the 2018 index_parse branch;
the code does not.
net_http_url_join() resolves per RFC 3986 section 5 into caller
storage with no allocation: empty reference, absolute http(s) URL,
"//" network path, "/" absolute path, "?" and "#" references, and
relative paths under the base directory, followed by "." / ".."
removal in place. index_parse strdup'd the base to punch NULs into
it; this walks pointers. A reference with any other scheme, or a
buffer too small, returns -1 with an empty result.
net_http_urldecode() and net_http_urldecode_inplace() decode %XX,
reading both digits before writing so the in-place form is safe
(the 2018 version rewrote the source nibbles). A bad escape is copied
as is, and '+' is not a space outside form bodies. Documented as not
for URLs about to be requested: decoding %20 there would put a space
in the request line.
Neither runs on the body path. The unit matrix is in the
net_http_semantics harness.
net_http_new_socket() still created the DNS cache and pool locks on
first use, the unsynchronised initialisation net_http_init() exists to
replace: two threads arriving together could each create one and then
lock different objects. RetroArch calls net_http_init() before the
task queue starts, as do the threaded test harnesses, so the lazy path
only ever served programs that skipped it - and served them racily.
It is gone; such a program now runs unlocked (slock_lock(NULL) is a
no-op), which is documented in net_http.h and next to the sample,
which now calls net_http_init()/net_http_deinit(). No lock is added:
this removes three creations from the connect path.
net_http_connection_new() compared the scheme with memcmp, reading 7
or 8 bytes of a URL that may be shorter. strncmp stops at the NUL.
http_transfer_stress_test's silent-peer case called net_http_deinit()
to drop pooled connections and carried on relying on the lazy
creation; it now calls net_http_init() after it, as its re-init case
already did.
One new check (an exactly-sized "ht" URL), which ASan reports as a
heap-buffer-overflow on the previous code.
Content-Length was already capped at 256 MiB, but nothing else was.
A header block with no end, a single header line with no newline, or
an endless run of 1xx blocks doubled the buffer for as long as the
peer kept sending; so did a close-delimited or chunked body. Headers
now fail past 64 KiB in total, interim blocks included, and the
receive buffer stops at the Content-Length ceiling plus one window
for every framing.
A chunk-size line with no hex digits parsed as 0, the last chunk: the
transfer "succeeded" with what had been decoded so far and the
connection went back to the pool with the rest unread. It is an error
now, and the connection is dropped.
The stale-pool replay only looked at the receive position, which is
back at 0 once a complete status line has been consumed. A pooled
connection that died right after it was replayed, appending a second
header set to the first - against the function's own promise that
nothing is replayed once response bytes have been seen.
A buffered Content-Length body no longer takes the drain loop's
window-floor growth. Its buffer is exactly the body, so near the end
the loop doubled it - 16 MiB to 32 MiB for the final 32 KiB - and
P_DONE shrank it back: two reallocs, and two whole-body copies on
allocators without mremap, per download. This removes work from the
drain loop; nothing is added to it.
6 new checks, all failing before; the buffer-size check reads the
largest realloc through --wrap=realloc.
request.path is kept without its leading slash, since the request
line writes " /" before it, but an absolute-path redirect stored
"Location: /foo" whole and went out as "GET //foo". The slash is
dropped now.
Redirects were followed with no limit, so two URLs pointing at each
other kept a transfer going forever. After 8 the transfer fails
("redirect_limit").
A streamed request body was not rewound before the redirected send.
The source was already drained, so the resend on the pooled socket
went out as headers with a short body, and only the stale-pool replay
happened to rescue it on a second connection. It is now rewound first,
the same contract as that replay, and fails cleanly when the source
cannot rewind. The 3xx's own body is no longer handed to the caller's
sink - a download to a file used to start with the redirect page -
and the sink bookkeeping starts over for the new URL. A failed
net_http_connection_new() on an absolute Location is checked.
POST is still not rewritten to GET on 301/302/303: WebDAV PUT and
MKCOL would change meaning silently.
Costs one counter and a flag on 3xx only. 8 new checks, 5 of them
failing before (the others guard behaviour that already held).
Both headers are comma-separated lists, and both were matched as one
exact line. "Connection: Keep-Alive, Close" or "Connection:close"
did not close, so the socket went back to the pool; "Transfer-Encoding:
gzip, chunked" was not chunked, so the body was read to a close that a
keep-alive server never sends. Both are now token scans on those two
lines only.
HTTP/1.0 closes unless the server says keep-alive; the version is
taken from the status line, and such a connection no longer goes to
the pool to cost a failed send and a replay on the next request. A
close-delimited body ended the connection too, and it is no longer
pooled either. A Content-Length arriving after "chunked" reframed
the body; Transfer-Encoding now wins whichever comes first.
Location is captured during the same walk, as a pointer into the
string_list copy of the line (not the receive buffer, which is
memmove'd afterwards), and it now accepts "Location:" without the
space. P_DONE used to walk the whole header list twice, once for
Connection and once for Location; it reads flags now.
net_http_body_is_framed() takes the same list test.
GET 200 pays one extra prefix compare, on the Connection and
Transfer-Encoding lines only. 7 new checks, all failing before.
A response with no body was read as if it had one. A HEAD answered
with Content-Length, or a 304 carrying the cached copy's length,
waited for that many bytes; a 204 without a length was taken as
close-delimited and waited for the close. On a keep-alive server
none of that ever arrives, so every one of them sat until the caller
timed out - and had it arrived, it would have been the next
response on that socket.
Such a response now ends at the blank line: HEAD, 204 and 304 by
definition, 101 too, with the connection marked unusable. A 1xx
other than 101 is an interim block and the real status line follows
on the same connection; only 100 was handled that way, so 102 and
103 were reported as the final status with the rest of the stream as
their body. Interim headers are dropped rather than reported as the
final response's, and redirects reuse that clear instead of freeing
and reallocating the header list. Bytes left over after a bodyless
response keep the socket out of the pool, since the next request
would read them first.
GET 200 does not take this path: one status test at the blank line.
New samples/network/net_http_semantics harness: the real net_http.c
against a scripted in-process server that logs which connection each
request arrived on, run under ASan+UBSan and TSan in the samples
network job. 11 of its 18 checks fail on the previous code.
Signed SMB 3 traffic, every read reply included, carried an AES-CMAC,
which chains each block on the one before and so runs a block at a
time: a 256 MiB read from Samba with signing mandatory came at 380 MB/s.
NEGOTIATE on 3.1.1 now offers SIGNING_CAPABILITIES, GMAC before CMAC,
and where the server takes GMAC the signature is a GCM tag over the
message with the MessageId, sender and CANCEL bits as nonce: the same
read comes at 1.3 GB/s, and a server's own signing is as much cheaper.
Servers without GMAC keep CMAC.
The Samba matrix checks which algorithm was chosen, GMAC by Samba's
default and CMAC from a server that allows no other, through
rsmb_get_sign_alg().
The handshake's public-key work trailed: X25519 took 0.18 ms where
BearSSL takes 0.13, P-256 key generation 0.43 where both take 0.31, and
an ECDSA check 0.90 where BearSSL takes 0.83. X25519 ran on the generic
Montgomery arithmetic, with an inversion that multiplied for every set
bit of p - 2; on 64-bit machines it now has arithmetic of its own mod
2^255 - 19 in five 51-bit limbs and the inversion by the usual addition
chain. The curve multiply went a bit at a time, an addition per bit; it
now goes four bits at a time from a table on the heap, chosen by
scanning every entry, a zero digit adding anyway and dropping the sum,
and a signature check does both of its products in one pass. X25519
0.045 ms (BearSSL 0.13, mbedTLS 0.67); P-256 key generation 0.26 (0.32,
0.33), ECDH 0.27 (0.53, 0.84), ECDSA verify 0.35 (0.94, 1.43). RSA-2048
verification ties mbedTLS with a fresh key, as a handshake has one.
On x86 with AES-NI, AES-GCM ran at 1.3 GB/s against BearSSL's 3.1, and
ChaCha20-Poly1305 at 0.39 against 0.46. CTR now keeps eight blocks in
flight, loads its round keys once per call, and steps its counter with
one vector add on the byte-reversed block, where the scalar rebuild of
every counter had been the cost; GHASH folds eight blocks per reduction
with Karatsuba multiplies. ChaCha20 runs four blocks at once on SSSE3,
chosen at run time, and Poly1305 uses 44-bit limbs where the compiler
has a 128-bit integer. Per 16 KiB record: AES-128-GCM 4.3 GB/s (BearSSL
3.0, mbedTLS 0.23), AES-256-GCM 3.3 (2.8, 0.22), ChaCha20-Poly1305 0.68
(0.47, 0.29), SHA-256 1.2 (0.27, 0.28). Other machines keep their paths.
test_bulk_paths checks the bulk paths against the block functions over
lengths around each group size and counters across the 32-bit wrap.
The compile line searches -I., libretro-common/include, deps and the
other in-tree dirs first, then pkg-config's paths, then the absolute
dirs configure found on the system (/usr/local/include on the BSDs),
so a same-named header installed on the system cannot stand in for one
of ours. tools/include_order_check.sh plants colliding headers in both
kinds of system path and compiles retroarch.c against them.
The Visual Studio 2005 to 2017 projects build the network stack as the
2019 and 2022 ones do: the TLS client, the keychain, and the SMB and NFS
clients. What kept them out was the CA bundle, one 255 KB string, where
MSVC before 2019 stops at 65535 bytes (fatal error C1091). net/cacert.h
now holds it in five arrays, each under 60000 bytes and ending where a
certificate does, and x509_trust_load_pem_parts() loads them all into
one trust store with no copy. tools/cacert_split.py writes the header,
from a new cacert.pem or from itself, byte for byte the same bundle.
The rest already kept to what those compilers take: C89, the LL/ULL
suffixes code they build uses, and Windows entry points resolved at run
time. The compile matrix builds the stack for 32-bit Windows in C89 at
the 0x0410 and 0x0501 floors, and checks no part of the bundle reaches
the limit.
Both context drivers share one resize step: the compositor's
configures count from the first resize whatever the scaling, and the
surface takes the buffer scale where it can. With fractional scaling
or an older compositor the GL window follows the compositor's size
once the splash is gone, and on whole scales it is redrawn at the
scale it is on.
A task swiped away from Recents is killed as soon as it drops to the
background, which is right after onPause() returns, and a stopped
process can be reclaimed once onStop() has returned; neither case
delivers onDestroy(). APP_CMD_PAUSE and APP_CMD_STOP are therefore
acknowledged only once the one-time background flush of SRAM, core
options and retroarch.cfg has run at the top of the next runloop
iteration, so the lifecycle callback waiting for the acknowledgement
cannot return with the settings unsaved. STOP performs the flush
itself when a timed-out PAUSE left it owed, a start or resume
supersedes a held acknowledgement, and the startup pump acknowledges
at once since nothing is loaded yet.
samples/input/android_lifecycle_flush reproduces the handshake with
a real UI thread and app thread, with sabotage lanes for
acknowledging before the save and for holding the acknowledgement in
the pump, and runs under ASan/UBSan and TSan in the samples/input CI
workflow.
DEFAULT_VIDEO_THREADED is now true for ANDROID as well as the Switch,
so a fresh Android install runs the core on one thread and presents on
another without the user having to turn it on.
Only the default changes. A config that already has video_threaded
saved keeps its value, and the option is still there under
Settings -> Video -> Output to turn it off.
tools/settings_table.reference regenerated; only the android profile's
video_threaded row changes.
DEFAULT_INPUT_ANDROID_SYSTEM_KEYBOARD goes from false to true, so menu
text entry on Android opens the system (IME) keyboard out of the box
instead of the built-in on-screen keyboard. That gives clipboard paste
and password managers without the user having to find the option first.
Only the default changes. A config that already has
input_android_system_keyboard saved keeps its value, and devices whose
input method cannot be driven from a gamepad can still turn the option
off under Settings -> Input to get the navigable on-screen keyboard
back.
The key is registered on every platform but only read under ANDROID, so
the reference row changes in all three profiles. The SDL3 option of the
same name (input_sdl3_system_keyboard) is left as it was.
tools/settings_table.reference regenerated.
The smb_client_readahead setting went in without the reference being
regenerated, so the "Settings table drift and griffin-order unity
compile" job fails on master:
+uint smb_client_readahead 1 0
in all three profiles. Regenerated with
python3 tools/settings_check.py table --write tools/settings_table.reference
No code change.
The 3DS, Vita and Switch build smb://: the SMB and Kerberos clients
reach the network only through libretro-common's socket helpers, and
the crypto they rest on draws on each console's hardware random number
generator. The Wii U, whose crypto has only a clock to stir, is left
out: SMB derives its signing and sealing keys from that randomness. The
3DS keeps its SMB transfers at 64 KiB, as it does NFS's.
The compile matrix builds the SMB client, the Kerberos client and the
SMB VFS backend for each of the three under their Makefiles' defines.
The hold picks the vblank nearest the frame's due time on a grid from
the display, and aims at the due time itself when the grid is only
the video thread's clock (vsync off, no present timestamps). A 120 fps
core on a 120 Hz display holds 120.
The threaded-video harness gains a pacing-rate lane: every shipped
pairing of content and display rate, with and without timestamps,
with a core that takes 1.3 ms and a render that varies, must hold
within 2% of its rate.
Over 4.2 the client reads with READ_PLUS, whose reply sends the holes
of a sparse file as their extent rather than as zeros: a 32 MiB image
holding 1 MiB of data costs 1.1 MB on the network instead of 33.6 MB.
The runs are written straight into the reader's buffer, data copied
and holes zeroed. A reply no READ could have given - no progress short
of the end, an end before the size the file was opened with, runs out
of order - is a server's broken READ_PLUS, not a failed read: the
connection goes back to READ and the range is asked again as that.
nfs-ganesha 4.3 answers every range with one empty data run.
local_read_plus_test.sh reads a sparse file from fake_nfs42.py, a
minimal NFSv4.2 server that reports the holes the filesystem really
holds, and has Wireshark's own dissector decode the replies to check
their data and holes add up to the file. The Ganesha matrix checks the
fall back to READ; the network workflow installs tshark and runs both.
The 16x16 PNG and ICO are hand-pixeled from the 96x96 logo at native
resolution, with 1px outlines and hard edges so the invader stays crisp
at icon size.
With no cursor-shape-v1 the context draws the cursor from a theme of
its own. It now loads the theme and size the toolkits use, at the next
whole scale up from the surface's (fractional scales included), with
the cursor surface's buffer scale set to match, and loads it again
only when that scale changes.
A server that restarts grants a client it knew its opens back during
its grace period and refuses new ones until grace ends - 90 seconds on
Linux. A file open for writing went through an ordinary open by path
after the restart, which a slow machine could not wait out, so its
next write failed. The client keeps its files that hold open state,
and whenever it identifies itself again it claims each back with
CLAIM_PREVIOUS - for 4.1 before RECLAIM_COMPLETE - where a server that
did not restart refuses harmlessly and the state it kept goes on. A
WRITE built before the call that re-dialled carries the old state, so
one whose file's state changed during it is sent again with the new.
The Ganesha restart rounds bring the server back with a 60 second
grace period, longer than any retry waits, so they test the claim.
A library that opens but is not a core (no retro_init, say) fails the
core stage after the old session is gone. The stage now runs again for
the dummy core: the drivers come back, the menu runs, and the failure
is said. On webOS a surface kept across the reinit that no init claims
is released.
Supersedes #19503.
The 3DS, Vita, Switch and Wii U build nfs:// - the NFS client needs none
of the crypto the RAM-starved consoles leave out, and it reaches the
network only through libretro-common's socket helpers, which these
consoles already build. Where a console's socket layer may not sleep
in a select() with no descriptors, the grace-period wait takes the
platform's own sleep (RARCH_CONSOLE); the 3DS keeps its transfers at
64 KiB, where four connections at the 1 MiB other platforms grow to
would hold 8 MiB of buffers for Wi-Fi that gains nothing from them.
The compile matrix builds the client, its VFS backend and the
prefetcher for each of the four under the defines their Makefiles
set: Wii U against its own wiiu/include, Vita against new stubs of
vitasdk's socket headers.
An smb:// file open while its server restarted - a NAS rebooting under
a paused game - stayed dead afterwards: only a new open, stat or
listing mended a connection, and a stream held on to the one it had.
Each stream now keeps how to open its file again (share, path, and
its flags less create and truncate), and a read, write or seek that
finds its connection or its handle gone mends the connection, opens
the file again and goes on from its own record of the position, once.
A dropped context frees its files with it, in the built-in client as
libsmb2 documents for its own: struct smb2fh is the client's file, with
no wrapper of its own, and rsmb_free() frees the files still open. For
libsmb2, whose context does not, the handle's close is queued before
the dead context is let go, which answers it and frees it.
local_samba_test.sh restarts smbd under smb_idle_test, the read and
then the write first, over the built-in client and libsmb2.
Each download attempt gets four minutes; a stalled or failed one moves
the sources off the Azure mirror to archive.ubuntu.com before the
next. The install then runs once from the downloaded packages.
A game paused, or between disc reads, while its NAS rebooted found its
nfs:// files dead afterwards: the client never dialled again, a 4.1
session lost while idle was not set up again on the read path, and a
4.0 client whose lease ran out could not open its files again. A call
that fails now drops the connection and the next one dials afresh -
same identity, so a server that kept this client's state gives it back
- and reads, lookups and writes, which change nothing when repeated at
their offset, go again on the new connection; other calls fail once
rather than risk running twice. A lost 4.1 session is set up again, an
expired 4.0 client identifies itself again, and a file open for
writing whose state the server lost is opened again by its path.
Arguments built in the shared buffers are set aside before a dial,
which would otherwise send what the dial left there.
nfs_idle_test holds a file open for reading and one for writing across
a pause; the Ganesha matrix runs it past a short lease and across a
server restart for 4.0 and 4.1, and the unfs3 matrix across a restart
with the read and then the write as the first call after it.
The v4 client spoke 4.0 alone, which servers that have dropped it -
Windows Server among them - refuse. With version 4 chosen it now asks
for 4.2, then 4.1, and settles on 4.0 only when the server answers
neither. From 4.1 the client identifies itself with EXCHANGE_ID, opens
a session with CREATE_SESSION and says RECLAIM_COMPLETE, and every
request after that leads with a SEQUENCE on a session slot, which also
keeps the lease alive. Pipelined READs each take a slot of their own,
as many as the server grants. A session lost while idle is set up
again once; the server lets a session lapse with its lease when the
connection closes.
The Ganesha matrix runs every case against servers offering 4.0, 4.1,
4.2 and all three, each round on a fresh export, and checks the
version the client settled on.
A client could tell what was running but not find anything to run:
LOAD_CONTENT takes a core path and a content path and nothing reported
either. LIST_CORES gives the installed cores and their paths,
LIST_PLAYLISTS the playlists, and GET_PLAYLIST a playlist's entries -
index, label, content path, core path - 200 at a time with MORE <next>
to page on. All three are read-only commands on every interface and
tools for MCP clients.
The playlist commands read files, so they answer from a task: the text
is made on the task thread and sent from its callback through the
asking interface's reply_to, which holds an MCP request open until
then, and is dropped if the interfaces were re-created meanwhile.
apt-get install is retried up to three times with a fresh index, and
the install steps get 15 minutes (20 for the network clients' larger
set) instead of 5 or 8, which a slow mirror ran past mid-download.
The threaded gather a wait runs from inside a main-thread handler
leaves the main-thread tasks to the pass further up the stack. They
ran twice and were freed twice: a load that failed with Threaded Tasks
on crashed falling back to the dummy core.
Without threads audio_driver.h declared slock_t and scond_t itself,
which rthreads.h declares with or without threads, so a file including
both - task_save.c - repeated the typedefs and make C89_BUILD=1
--disable-threads stopped there. The header includes rthreads.h in
every build; it holds declarations only. The compile matrix builds the
tasks and the audio driver that way, C89 and without threads.
The lane hooks the driver's frame() with a 30 ms present and checks
that every frame the video thread draws behind it is newer than the
one before. It also asked for at least ten such draws out of the 60
frames the core pushes - a race between this thread and the slow
present on whatever GPU is there: 32 draws on an M1, 10 on the CI
runner's Apple Paravirtual device in one run and 8 in the next, which
failed the macOS metal-harness job's XRGB8888 pass. The count says
nothing about the wrapper; the staleness of the draws does.
The core now keeps pushing after the 60 until the driver has drawn
ten, up to 600 more frames; a driver that never gets there is still a
failure. The grant and zero-copy counts stay measured over the first
60. Linux on Vulkan: 29 drawn, all lanes pass.
Whether a read-ahead window helps could not be seen from outside. Each
SMB and NFS stream with a prefetcher counts the octets it served from
its windows and those it missed and read directly, and adds them to the
backend's totals when it closes; once the core has closed its files
the frontend logs them - [SMB] Read-ahead: 812 of 900 KiB read came
from its window (90%) - and takes them back to zero. Nothing is counted
or logged while read-ahead is off.
The SMB and NFS thread tests read the test file through and check the
counts add up to it with read-ahead on and stay empty with it off.
On macOS every lane that counts frames the core ran failed - the
zero-copy lanes saw 0 of 60 asks granted, the lent-window lane found
no grant and skipped, and "Paused" sat in the corner of the window
throughout. The harness window is never the active one: it is run
from a terminal, and under Cocoa the application is never made key.
pause_nonactive defaults on, so the frontend paused the core while
the window was inactive, and the core ran nothing; the lanes that
count what the driver presents (dupes count there) passed and hid
it. The harness config now turns pause_nonactive off. The same on
Linux, where the Xvfb window happened to report focus.
NFSv3 resolved a path with a LOOKUP per component and then a GETATTR,
so each open three directories down cost five round trips - paid on
every file by content that opens many, as ScummVM does. Each
connection keeps the handles of the directories it walked through
recently, and a walk starts from the deepest one on its path: an open
in a known directory is a single LOOKUP, whose own attributes stand in
for the GETATTR. A handle the server calls stale empties the cache and
the walk starts again from the root; a rename, an unlink or a new
connection empty it too. NFSv4 already walks in one COMPOUND.
The client test counts the round trips of a repeated open three
directories deep, and opens through a renamed directory's old and new
names.
The Metal driver loads default.metallib from the executable's own
directory - a bare binary has no .app bundle for newDefaultLibrary to
look in - and the tree's make builds it next to retroarch. The
harness binary lives in samples/gfx/threaded_video, so the driver
looked there and found nothing:
[Metal] Could not load ".../threaded_video/default.metallib": library not found.
[Metal] Failed to load main pipeline shader functions.
[Video] Cannot open video driver. Exiting...
On the Cocoa build path build.sh copies the tree's default.metallib
next to the harness binary, and says so if the tree has none. clean
removes the copy.
The harness's one argument is its cycle count. AppKit hands every
command-line argument that is not a flag to the delegate as a file
to open (application:openFiles:) while the application finishes
launching, before rarch_main() is reached; with no core up yet the
delegate stores it as the content path, and retroarch_main_init()
then tried to load a file named "1":
[Content] Loading content file: "1".
[Content] Could not read content file: "1".
FAIL: retroarch_main_init failed
The Cocoa rarch_main() clears the content path before the lanes run.
Servers may root the v4 namespace below their export paths - Linux's
fsid=0, a Ganesha Pseudo of / - so /media exported as the root is "/"
to v4 and /media/user is "/user"; the export field, given the
directory's path on the server as for v3, found nothing. When the
export is not there and its first component is not in the namespace
either, that component is taken to lie above the v4 root and dropped,
down to the root itself. A first component the namespace does have
means the path is wrong, and it is refused as before.
The Ganesha matrix mounts the export by its server path under v4 and
checks a wrong path under an existing export is still refused.
On macOS the harness build "succeeded" and left no threaded_video_test:
the link line's -o was rewritten with \?, a GNU sed extension that
BSD sed treats as a literal, so the pattern never matched, the link
produced retroarch in the tree root, and the script said "built".
check-driver then ran ./threaded_video_test and got "No such file or
directory".
The rewrite is two plain patterns now (-o retroarch and
-o retroarch_debug), and build.sh checks every object and binary it
was supposed to produce, so a substitution a sed does not take fails
the build where it happened instead of at the run. The script was
run under BusyBox's POSIX sed, grep, awk and sh as well as GNU's on
the Linux tree; both build.
SMB and NFS read one request per read again unless a read-ahead is
set: both default to 0, and 0 now means off. It had meant 1 MiB - the
backends substituted their window for it and still started a
prefetcher on every read-only open, each holding a second connection
and a thread, which content that opens many files at once paid for on
every open. With read-ahead off no prefetcher starts; with it on, both
behave as before. The ranges start at 0, NFS's in SMB's 64 KiB steps.
The SMB and NFS thread tests run with read-ahead off and on, checking
a read-only open starts a prefetcher only when it is on, and the menu
harness checks both settings can be set to 0 and default to it.
The harness did not build on macOS. There the frontend's main() is
Cocoa's (HAVE_MAIN, ui_cocoa.m): it stands up NSApplication, the
delegate and the window, and calls rarch_main() from
applicationDidFinishLaunching:, and the Metal and Cocoa GL contexts
need all of it. build.sh replaced retroarch.c's main() with the
harness's own, which on macOS is a second main() against ui_cocoa.o
and an application that was never started.
On a tree whose compile line carries HAVE_MAIN the harness now keeps
Cocoa's main and takes rarch_main() over instead (harness_cocoa.m,
compiled with the tree's own Objective-C line; retroarch.c gets its
rarch_main renamed, the harness's main becomes harness_main): the
lanes run in rarch_main's place, on the main thread with the
application and window up. Each frame turns the main run loop once,
so the window's layer tree is committed and the drawables the driver
presents come back round. And the process leaves with the harness's
status: the Cocoa companion's deinit, reached from main_exit(),
terminates the application, which is exit(0), so an atexit handler
installed once the status is known ends the process with it. Nothing
changes off macOS - the hooks are under HAVE_COCOA and the other
build path is as it was. check-driver no longer wants $DISPLAY on
Darwin.
MacOS.yml gets a metal-harness job: a Makefile build without Qt on
the runner, then the harness on DRIVER=metal with the core pushing
RGB565 and again XRGB8888 - every lane the Vulkan and D3D legs run,
including the lent-window lane over the Metal lend of 771a19c8a1,
on the runner's GPU. It is the first end-to-end run of the Metal
driver anywhere in CI.
build.sh also says why it could not find the compile line when it
cannot: the usual reason is a config.mk that fell behind configure
after a pull ("config.mk is outdated or non-existing"), which make
reported and the script threw away - that was the failure reported
against it today.
The Linux path was rebuilt and rerun on Vulkan under ASan with this:
all lanes pass. The macOS path is compiled and run only by the new
CI job; no Apple toolchain here.
A setting's list starts at its minimum, but a pick was turned back
into a value from 0 (plus offset_by, which few settings set to their
minimum), so every range above 0 was a step short: NFS Version 4 set
1, an SMB timeout of 20 set 19, a read-ahead of 64 set 0, Minimum
Audio Latency and the rest likewise. The pick is taken back from the
minimum in whole steps, exactly as the list is built. The network
settings harness picks through the real dropdown to hold it there.
The SMB timeout goes to 60 seconds as NFS's does, and NFS Network
Settings gets the icon SMB Network Settings has in Material UI.
tools/console_objects_check.sh, run by the compile matrix, reads what
each console Makefile builds and fails on the MCP server anywhere or
the crypto library on GameCube, Wii and PS2.
The Metal driver was the one desktop driver with no
get_current_software_framebuffer: a core's ask was declined, it
rendered into its own buffer, and every frame was copied into the
texture by the host (replaceRegion:). Vulkan and D3D12 lend; the
threaded wrapper lends; a core written for the loan (beetle-psx's
software renderer since 3f5be07) got nothing from it on macOS with
threaded video off.
FrameView lendFramebuffer: hands the core the contents of a
shared-storage MTLBuffer sized to its ask, in the frame's pixel
format - BGRA8 for XRGB8888, and R16Uint for RGB565, which is what
_src is, so the existing compute conversion applies unchanged - with
a linear texture over the buffer at the tight row pitch. On Apple
silicon that is the memory the GPU samples. updateFrame:pitch: then
tests the pushed frame against the loan the way vulkan_frame_window
and d3d12_sw_fb_window do (7510039d5d): the whole loan, or a window
into it at the loan's pitch and whole, is blitted out of the loan
texture with a source origin into the frame texture (or into _src
for RGB565), on the blit command buffer the context commits ahead of
the frame's own - one GPU copy, no host copy, and the cached frame's
pointer into the loan stays valid. The lend waits for the command
buffer of the last such blit before handing the buffer back to the
core, since the core is about to write over what the GPU read; a
resize, and the view going away, retire the cached frame first.
Declined, as the Vulkan and D3D12 lends decline, when the row pitch
the device needs for a linear texture (its minimum linear texture
alignment, at least 16) is wider than width * bpp - a core that
ignores fb.pitch would shear - and for a 10-bit source, which has no
such texture. At an alignment of 16 the widths cores use (320, 640,
beetle-psx's 700) all lend at the tight pitch in both formats.
samples/gfx/metal_texture_update gets a third lane, run on the macOS
CI leg's Metal device: for those widths in both formats, the shared
buffer and its linear texture at the tight pitch, a CPU-written
pattern, a window blitted out with an origin and read back at its
four corners, and a second write after the first blit's command
buffer completed that the second blit must show - the ordering the
lend relies on. The three counters of 7510039d5d cover this path too,
and the threaded-video harness's lent-window lane runs on it with
DRIVER=metal on a Mac.
Compiled and run only by CI here (no Apple toolchain on this box).
The MCP server was listed with the networking objects, so every
platform Makefile with networking compiled it, the consoles included,
and the 3DS - networking without the command interface it is built on
- failed to. It is HAVE_MCP now: a ./configure option, on by default,
dropped without networking, and built only with the command interface;
no console Makefile sets it. The server, its settings and its menu
entries all follow it.
A core that renders into the buffer GET_CURRENT_SOFTWARE_FRAMEBUFFER
lends it may push back a pointer partway into that buffer, at the
buffer's pitch, with the size of the window it wants shown: that is
how an overscan crop is done by offset, and beetle-psx does exactly
that since 3f5be07. The threaded wrapper has accepted such a push as
a lend since 9ed0db58c3, the vulkan and d3d12 drivers since 7510039.
This driver lends frame_surface->map and knew a lent frame only by
that exact pointer, so a window into it took the copy path: memcpy
from the surface onto itself, one row pitch apart, shredding every
row the copy overtook, and then blitting from (0,0) so the crop was
ignored as well. Most visible on a paused frame or a screenshot,
where the frontend replays the cached window pointer.
Recognise the window the same way the other two drivers do: inside
the mapping, on the mapping's pitch, starting on a pixel. Then skip
the copy and blit from the window's origin, clamping the size to
what the surface holds from there. A pointer inside the mapping that
is not addressable as a window (another pitch, or mid-pixel) is not
copied either -- that would be the same self-overlapping memcpy --
the loan is shown from its origin instead.
RetroArch serves the Model Context Protocol over Streamable HTTP as one
more command interface beside UDP and stdin, polled with them and
waiting on nothing. Its tools are the commands in command.h, listed
from the rows HELP prints with their read-only and destructive hints,
and a tool call runs the command's own handler; a command that answers
later holds its connection until it does. Both protocol eras are
answered: 2026-07-28 statelessly, with server/discover and the header
checks, and earlier revisions through initialize.
It is off by default and listens on 127.0.0.1. Every request carries a
bearer token, made on first start and kept in the keychain, and a
browser Origin other than this machine is refused. MCP Server, its port
and its token are under Settings > Network; docs/mcp-server.md covers
connecting a client.
task_push_keychain_passphrase() was declared inside the networking-only
part of tasks_internal.h, so a build with the keychain and without
networking called it from the menu undeclared. It is declared under
the keychain's own gate. The compile matrix builds the menu and the
task in that shape with implicit declarations as errors.
The hold measured from the vblank the video thread expected at its
last present. After a stall that vblank had passed, and every frame
was released at once until the presenter caught up: the core ran at
the display's rate. It is carried forward on the display's grid.
Both command tables carry a one-line description and flags saying
whether a command only reports or may lose data or load code, so the
tables in command.h are the one description of what RetroArch can be
told to do. HELP answers with every command - name, argument, what it
does, [read-only] or [destructive] - or with one when named; interfaces
that hand the commands to a client can be generated from the same rows.
tools/command_table_check.py, run by CI, fails on a command added
without a description.
A command sent without an argument no longer has its argument pointed
one byte past the end of the line, and SHOW_MSG is documented as
taking the message it shows.
Every NFS setting, SMB read-ahead, the SMB Kerberos realm and KDC, and
TLS certificate verification had help text but nothing that bound it
to the entry, so they showed without a sublabel. They are bound in the
sublabel map, each under the gate its setting is built under.
A texture released through vulkan_unload_texture is parked for the
frame thread to destroy once a swapchain's worth of frames has gone by,
because a frame already handed over, or menu code still looking at the
handle, may yet use it. Parking allocated a list node per texture, and
when that allocation failed the texture was destroyed on the spot,
after waiting for the frames already submitted - which covers neither
of those two.
The link and the countdown are in struct vk_texture now. Parking is the
compare-and-swap push it already was, on the texture itself: no
allocation, so no failure, so no path that destroys early or waits.
vulkan_texture_defer() loses the same fallback, and every unload loses
a malloc and a free.
What is left is vulkan_texture_defer_copy(), which parks a texture
embedded in another object (the font's two) by moving it to a heap
copy; if that copy cannot be allocated it still waits on the driver's
own fences and destroys. That runs on the frame thread.
Two comments that still described a vkQueueWaitIdle under queue_lock in
the unload path are brought up to date; there has been none for a
while.
There is no test that fails before this and passes after: the path it
removes needed an allocation to fail. The threaded-video harness lanes
that load and unload textures pass on lavapipe under the validation
layer (software core, hardware core, hardware core with its own
device), and a running RetroArch with the ozone menu toggled eight
times and quit, threaded video on and off, reports nothing - a texture
destroyed early or never would show there.
docs/network-stack.md describes what users meet of the built-in TLS,
SMB and NFS clients and the keychain: certificate verification, the
SMB and NFS settings and content addresses, read-ahead, where saved
passwords live, and how to move a keychain to another machine with a
passphrase; with the configure options that shape the stack. CHANGES
lists the stack under NETWORK.
The keychain's key comes from the key file and this machine's identity,
so a configuration carried to another machine lost every saved
password. Settings > User > Keychain Passphrase wraps that same key
under a passphrase in the key file, next to a wrap for this machine;
nothing is sealed again, and a key file without one is unchanged. On
another machine the keychain comes up locked, saves write nothing in
the clear, and entering the passphrase once unlocks it, wraps it for
the new machine and puts the credentials back into the running
settings. The key derivation runs as a task a slice at a time, so the
main thread never waits on it, with or without threads.
A sealed value that cannot be opened is now taken out of what the
loader sees even when there is no key at all, instead of becoming the
setting's value; and only the values that failed to open are carried
through a save, so clearing a password reaches the file and a normal
save no longer reports credentials from another machine.
The AES round tables, the inverse S-box and the InvMixColumns table
were built on first use behind a plain flag. Two threads setting up
keys at once both wrote them while each read the flag - a data race
and, on a weakly ordered CPU, a thread that sees the flag before the
tables and encrypts with half of them. They are now constants generated
from the S-box, so there is nothing to set up; the AES-NI check caches
its answer atomically.
crypto-tsan-check first uses every primitive with a table or a CPU
dispatch from eight threads released together, under TSan, and checks
they all compute the same bytes. It runs in the libretro-common tests
workflow and reports this race on any runner, where the SMB test saw
it only when two connections started together on two cores.
Emulated mailbox - FIFO kept while fast-forwarding with vsync on, with
a second thread doing the acquiring so the presenting thread never
blocks on the display - passed a request and an answer between its two
threads under a mutex, with a condition variable that carried the
request, the answer and the teardown all at once.
Each of the three is a word of its own now, written by one side and
taken by the other, and each direction has an eventcount:
request the presenting thread wants an image; the acquire thread
takes it and acquires
acquired the acquire thread has an answer. result and index are
written before it is raised, and not again until the
presenting thread has lowered it and asked once more
dead teardown
The acquire thread sleeps on `work` and looks at request and dead again
between announcing the wait and committing to it; the blocking acquire
on the presenting side sleeps on `answered` the same way, against the
one deadline it already had. Whether a request is outstanding was only
ever the presenting thread's business and is a plain bool of its. The
finite waits that let the acquire thread notice the teardown are as
they were.
Nothing the driver does is different: same requests, same answers, same
bounds. This was the last lock in the Vulkan code that was not
queue_lock.
samples/gfx/vulkan_validation had no case for the mailbox at all. It
has one now: emulation switched on in place, forty frames drawn through
it under the validation layer, switched off with a request possibly in
flight - which must end with an image in hand and the acquire thread
joined - three times over, the last torn down with emulation on. It
passes on the mutex version and on this one, and on this one under
ThreadSanitizer.
samples/gfx/vulkan_mailbox_init_leak keeps verbatim copies of the
mailbox's init and deinit, with the primitives mocked, to hold the
partial-init leak fix in place; the copies and the mocks follow: the
failure stages are the first eventcount, the second, and the thread.
tools/lock_acquire_budget.sh measures the acquire thread (none), the
present (queue_lock, once) and the acquire (queue_lock, only in the
path that empties a full stale-acquire list). Against the code before
this change the first reads four and the last two.
With vsync on, fast-forward toggled six times in a running RetroArch
with Beetle PSX HW, threaded video on and off: emulation entered and
left three times each, no validation error.
The command returns at once; the reply follows once the movie task has
installed the replay, to the sender that asked. command.c joins the
no-wait guard.
Three idle waits were still on the idle-wait allowlist, one list
borrowed queue_lock because it was the lock at hand, and the present
held it across its own failure handling. queue_lock is the
one lock this driver cannot lose - the libretro interface hands it to
hardware cores, and a VkQueue shared by two threads needs it - so what
it guards should be the queue call and nothing else.
vulkan_submit_deferred_cmd_fenced
With no node or fence to defer an upload with, it submitted and
then called vkQueueWaitIdle, under queue_lock. That drains a
hardware core's work too and keeps the core out of lock_queue for
as long as the GPU takes. The upload is waited for on a fence now
- the caller's, else one the driver keeps for the purpose - with
the lock dropped after the submit.
vulkan_acquire_next_image
With the stale acquire list full it called vkDeviceWaitIdle and
destroyed a semaphore. That is a use of every queue made without
queue_lock on the presenting thread, while a core may be
submitting; it waited on the core's work; and an acquire is not
queue work, so it never waited for the signal it destroyed the
semaphore under. One batch that waits on the whole list, with a
fence, consumes the signals; the lock is held for the submit call.
vulkan_present
Kept queue_lock from the present call to the end of the function,
across the failure handling. A failed present tears the swapchain
down, the teardown waits for the present with an empty fenced
submission behind it, and that submission takes queue_lock: where
the mutex is not recursive - everywhere but Windows - the thread
waited on itself, and an out-of-date present never returned. The
lock is dropped when vkQueuePresentKHR returns.
slang_chain_flush
The vkDeviceWaitIdle fallback for a chain created without
wait_submissions. Every chain in the tree is created with it. The
callback is required now and the fallback, with the chain's copy
of the lock callbacks that only it used, is gone.
the parked-texture list
vulkan_unload_texture parks textures for the frame thread to
retire, and a reinit fallback can park one from the main thread,
so the list was serialised - with queue_lock, which made parking
a texture wait behind a present or a core's submit. Parking is a
compare-and-swap push onto one atomic pointer now, and the list
the frames count down is private to the thread that records
them, which takes what was pushed with an exchange each tick.
tools/vulkan_idle_wait_allow.list is down to its one legitimate entry,
the device teardown; the existing idle-wait check fails on the three
removed sites if any comes back.
samples/gfx/vulkan_validation gains two cases. A present that fails:
the driver has to come back from it with the swapchain torn down. The
code before this change never returns from it on Linux (the test gives
it ten seconds). And the acquire case: frames presented
for more than two lists' worth without anything waiting on their
acquires. The driver has to get through it without one queue or device
drain, and without the validation layer objecting to the batch that
consumes the list or the semaphores destroyed behind it (the layer's
MissingAcquireWait on each such present is the scenario itself, and is
set aside by name). The code before this change drains 19 times there;
this one, none.
The threaded-video harness passes on lavapipe under the validation
layer with a software core, a hardware core, and a hardware core that
creates its own device.
The slang filter chain makes one sampler per filter, mip filter and
address mode. The address modes had a fifth entry, mirror-clamp-to-edge,
that no wrap mode maps to - a preset can name border, edge, repeat and
mirrored repeat, and rarch_wrap_to_address() returns one of those four -
so its four samplers were created on every chain and never bound.
Creating them is only valid with VK_KHR_sampler_mirror_clamp_to_edge
enabled. The frontend adds that, as an optional extension, to a device
it creates itself. A core that creates the device through the
negotiation interface's create_device is handed only the required list,
so on every such core (Beetle PSX HW, for one) each chain raised
VUID-VkSamplerCreateInfo-addressModeU-01079
VUID-VkSamplerCreateInfo-addressModeU/V/W-parameter
four times over. Drivers tolerate it; it is invalid all the same.
The entry and its case are gone. The optional extension stays in the
device list: nothing in the tree uses it now, but a core drawing on the
frontend's device may.
The threaded-video harness core can now be that kind of core:
HARNESS_CORE_HW_VULKAN=2 registers a version 1 negotiation interface
and creates the device with exactly what the frontend passes. The
Vulkan validation workflow runs the harness that way as well. Before
this change that run reports 48 validation errors and fails; with it,
none.
libsmb2 keeps every context on one process-wide list that
smb2_init_context() pushes to, smb2_destroy_context() unlinks from and
smb2_context_active() walks, with no lock of its own. Connections made
and dropped on several threads at once can lose an entry, after which a
live connection reads as dead and its seeks and reads are refused, or
break the list outright. Every one of those three calls in the backend
now goes through a wrapper holding a lock for the list operation alone;
none of them does network I/O. The cleanroom client has no such list,
and single-threaded builds none to guard: both call straight through.
The libsmb2 lane runs the SMB tests once more under TSan against a
TSan-built libsmb2, which reports this on any runner rather than only
when two cores happen to collide.
Debug builds report slow handlers with Threaded Tasks on as well. The
load stages that run the core's own load and the driver rebuild are
titled and exempt: they are one-off by design. The content_load
harness gains the acceptance run - menus, playlists, overlays, the
Explore index and a load, with no handler over a frame.
The slow-handler watchdog now also measures RETRO_TASK_FLG_MAIN_THREAD
handlers when Threaded Tasks is on: they run on the frame thread there
too, so an overrun is the same visible stall.
retro_vfs_readdir_smb handed every caller the same function-static
entry, so listings on different threads overwrote one another's name
and type while the caller copied them out. The entry now lives in the
directory handle, for the cleanroom client and libsmb2 alike; the
libsmb2 threads test is clean under TSan with it.
vfs_threads_test names each failed operation with its thread, round
and what came back, so a failure on a runner says where it happened
rather than only how many.
7a8b2ab made the Vulkan ring fence re-signalable by giving each one a
mutex and an "armed" flag. That was a lock added to a path that is
meant to lose its locks. It is gone, along with everything else that
commit added to vulkan.c: the driver's ring fence is a bare VkFence
again.
What the lock was for is done in the wrapper instead, once, for every
driver, without one (gfx/video_thread_hw_fence.h). A slot has three
driver fences and each is only ever touched by the thread that owns it
at that moment; ownership moves by an atomic exchange:
- the video thread, after a frame drawn from the slot, takes an idle
fence, signals it and exchanges it into `armed`. If an earlier
arming comes back out - a dupe: nobody waited it - the video thread
owns that one, waits it out and returns it to idle;
- the core's thread exchanges `armed` out, waits what it got, returns
it to idle, and looks again.
At most one fence is armed and at most one is with the waiter, so one
of three is always idle. A driver's signal is only ever called on an
idle fence and its wait on a signalled one.
That also closes the same hole in the GL drivers, which needed no
change: gl2 and gl3 kept one sync object per slot behind a bare
pointer that the video thread deleted and replaced while the core's
thread could be waiting on it and then deleting whatever the pointer
held. And it retires the slot's in_flight flag, a plain bool written
by one thread and read by the other.
samples/gfx/hw_ring_fence runs the real header with two threads at
one slot and fake fences that count a signal on a fence not idle, a
wait on one never signalled, and two calls inside one fence at once;
under ASan+UBSan and under TSan. The single re-signalled fence it
replaces is built too and must fail. The Vulkan hardware-core lane of
the threaded-video harness (dupes in game and under the menu) passes
on lavapipe with the validation layer, as it did with the lock.
The check scans for readers with deps/ and the other vendored trees
pruned, but built its set of removed members from the whole diff, so
deleting a bundled library counted every member that library declared
- tv_sec, ss_family, session and the rest - as removed RetroArch API,
and the removal of deps/mbedtls, deps/bearssl-0.6 and deps/libsmb2
failed on 430 readers of the platform's own structs. The pruned trees
are excluded from the diff as they are from the scan: a vendored
library's members are its own API, read through its own headers. A
member removed anywhere else is still caught, readers and all.
The regression lane for 53ca1ac (a hardware core's image is let go of
before the core destroys it), and the second bug the lane ran into.
The bug. Under the threaded wrapper the video thread signals a ring
slot's fence after every frame it draws from the slot, and a dupe
draws from the slot last presented again: one frame in two for a
30 fps game, every frame while the menu is up over a hardware core.
The fence was a bare VkFence, signalled by an empty vkQueueSubmit, and
nothing reset it between a frame and its dupe, so every dupe submitted
a fence that was already signalled or still pending. That is invalid
usage (VUID-vkQueueSubmit-fence-00063/-00064); lavapipe aborts on it.
The ring's fence now records whether it is armed. A signal that finds
it armed waits the earlier arming out and resets it first - a wait on
work submitted a frame ago - and a lock orders that against the main
thread's wait-and-reset, so a reset cannot land on a fence the other
thread has just submitted. Direct3D 12's ring fence counts and never
had this.
The lane. The threaded-video harness core gains a Vulkan hardware
mode (HARNESS_CORE_HW_VULKAN=1): one image of its own, set_image and a
hardware frame per retro_run with every third a dupe, destroyed in
context_destroy without a word to the frontend, and an export that
withdraws it with set_image(NULL) and destroys it in mid-session. It
calls Vulkan through the frontend's interface and links nothing.
With that set the harness runs one lane, hw image lifetime:
- unthreaded: frames in game, then a staged close from the menu;
- the core again, under the wrapper: hardware frames reach the
driver; the core withdraws and destroys its image, and no install
names an image until it presents again; then the staged close, and
no install names an image after the core's context_destroy.
It also requires that the close presented at least one frame between
the core's teardown and the new drivers, so it cannot pass by no
longer testing the window.
Run in the Vulkan validation job on lavapipe:
with both fixes pass, no validation errors
53ca1ac taken out VUID-vkDestroyImageView-imageView-01026
(view in use), then a segfault
this fence change taken out VUID-vkQueueSubmit-fence-00063,
then lavapipe aborts
The software-core lanes are unchanged and pass on the same build.
Every build uses the built-in TLS, SMB and NFS clients, so the
bundled deps/mbedtls, deps/bearssl-0.6 and deps/libsmb2 are removed.
A system library can still stand in for a built-in client, off by
default: --enable-mbedtls and --enable-bearssl replace the TLS client
with a system mbedTLS or BearSSL, and --enable-libsmb the SMB client
with a system libsmb2; a library asked for and not found is a
configure error, and two TLS libraries at once are refused. The
older builtin/system switches go, as do the platform files' bundled
selections, and the libsmb2 backend follows the library's current
share-enumeration API.
Linux-library-net-backends.yml replaces the legacy lane with one lane
per library: each configures with its option, builds RetroArch to
the link, checks the library's backend is linked and the built-in
client is not, and operates it - the cheevos login transport over a
local HTTPS server for mbedTLS and BearSSL, the SMB VFS tests against
a local smbd for libsmb2 (built from upstream; Ubuntu has no package).
The cheevos_login sample links the system BearSSL, the smb_backup
sample carries stand-in libsmb2 headers for its fake share, and the
samples workflow installs libbearssl-dev.
Closing content with a Vulkan hardware core under threaded video could
freeze or crash the frontend inside the GPU driver.
A staged content close keeps the video driver up after the core is
gone and presents dupes of the last frame until the next session's
drivers are built. For a hardware core the last frame is the core's
own image, which the core destroys in context_destroy. Nothing told
the driver: video_driver_free_hw_context() called context_destroy and
left the driver holding the image, and every dupe after it sampled a
destroyed VkImageView. Seen on NVIDIA as a fault inside the driver,
called from the filter chain's final pass on the video thread, with
the main thread waiting on that frame in the ring drain.
Under the wrapper two more things made it certain rather than likely:
- the wrapper had no invalidate_hw_render_cache, so the runloop's
invalidate (core_reset) stopped at the wrapper and never reached
the driver;
- a core's set_image(NULL) - how a core withdraws its image before
destroying it, and what LRPS2 does in context_destroy - only wrote
the ring slot the core fills next. A dupe re-reads the slot last
presented, so the withdrawal never took effect.
Now:
- video_driver_free_hw_context() invalidates the driver's hardware
render cache on both sides of context_destroy: before, so nothing
is drawing from the image while the core tears it down, and after,
because a core may present once more on its way out;
- the wrapper implements invalidate_hw_render_cache: drain the frame
ring, have the driver wait out its submissions and drop the image
on the video thread, then clear the ring's slots and the slot a
dupe would re-read;
- set_image(NULL) under the ring does the same.
A dupe after that draws the driver's blank texture, as it already does
without the wrapper after a reset.
The task carrying the deferral's deadline had no cleanup, so its
state leaked once setup ran; LeakSanitizer on the content_load
harness's host-setup lane reports it.
Hosting through a lobby relay needs the relay's address, and host
setup waited for the lobby's answer inside CMD_EVENT_NETPLAY_INIT - up
to ten seconds on a slow lobby. When the query is still out, the
command now puts host setup off to a main-thread task that checks each
frame and reissues CMD_EVENT_NETPLAY_INIT once the answer is in, or
goes direct once the bound has passed; the frame loop keeps running.
netplay_mitm_query no longer waits at all.
The room reset at the start of host setup also cleared an answer
already in hand, so the prefetched query was asked - and waited for -
again every time; the answer is now carried across the reset.
content_load harness lane: with a relay configured and the query out,
the command returns in microseconds with host setup queued, frames
keep presenting, and setup runs once the query settles.
The guard now greps tasks/task_content.c for task_queue_wait, forbids
the close's blocking helpers (content_wait_for_*_state_task, which
wait inside task_save.c where the first grep cannot see) anywhere in
menu/ and tasks/, and checks the load still closes through
retroarch_main_deinit_pending. The menu-harness job builds and runs
samples/tasks/content_load and samples/menu/explore_build on its
non-Qt menu build.
The lent-window lane merged in 7510039d5d failed where a driver lends
nothing: d3d11 has no framebuffer to lend, and d3d12 lends only for a
row pitch on a 256-byte boundary, which the harness core's RGB565 at
320 wide (640 bytes) is not - so on both Windows legs the core was
granted nothing and the lane failed on the grant count. A driver that
lends nothing has no loan to push a window into: the lane now reports
a skip there instead.
That also left the D3D12 half of 7510039d5d - the window box copied
out of the loan's footprint - with nothing running it. The harness
core can push XRGB8888 (HARNESS_CORE_XRGB8888 set at start; the pixel
values are the same in the low bits either way, so every lane reads
as before): at 4 bytes a 320-pixel row is 1280 bytes, on the boundary,
and d3d12 lends. The d3d12 matrix entry runs with it. The Vulkan legs
run the harness under both formats, since the two take the driver's
two upload paths - RGB565 through the compute conversion of a staging
buffer, XRGB8888 through a linear image sampled directly and, for a
window, vkCmdCopyImage out of it; the second is the one beetle-psx
takes. Both pass on lavapipe under ASan with the validation layer
silent: 59 grants, 39 windows, 1 host copy (the core's own oversize
frame), 3 frame textures over 60 frames, and the paused redraw runs
through.
A core that renders into the buffer GET_CURRENT_SOFTWARE_FRAMEBUFFER
lends it may push back a pointer partway into that buffer, at the
buffer's pitch, with the size of the window it wants shown: that is
how an overscan crop is done by offset, and beetle-psx does exactly
that since 3f5be07 - the whole 700x480 scanout surface is rendered
into the loan and the visible part pushed. The threaded wrapper has
accepted such a push as a lend since 9ed0db58c3. The drivers' own
loans, the ones handed out with threaded video off, did not.
The Vulkan driver knew a lent frame only by the exact pointer of its
swapchain-index texture's mapping, so a window into it was a frame
of another size than the texture: the texture was recreated at the
window's size every frame (and again at the loan's size at the next
lend), the loan went to the deferred list, and the window was copied
out of the parked mapping by the host. That worked while the core
kept pushing. The cached frame kept naming the parked mapping,
though, and when the core stopped - the menu opened over a paused
core, or the window lost focus with pause-on-focus-loss on - every
iteration redrew the cached frame; after the deferral window the
mapping was unmapped and freed, and the next redraw read it. That is
libretro/RetroArch#19652 (Beetle PSX on an RTX 4060: memcpy fault
from vulkan_frame's row copy on entering core options or on losing
focus).
vulkan_frame() now tests a pushed frame against the texture's mapping
(vulkan_frame_window): inside it, on its pitch, whole, word-aligned.
Such a window leaves the texture alone - it keeps the loan's size for
the next lend and its mapping stays mapped, which is what the cached
frame needs - and is cut out on the GPU into texture_optimal, which
is what gets sampled: vkCmdCopyImage with a source offset for a
streamed linear image (the XRGB8888 case), vkCmdCopyBufferToImage
with a buffer offset and row length for a staging buffer, and a word
offset in the RGB565 compute upload's UBO (rgb565_to_rgba8888.comp,
.inc regenerated). No host copy, no texture per frame. texture_optimal
is kept at the frame's size whenever it exists, since it is sampled
whenever it exists; a window-sized one left over from a window is
recreated for a staging texture and retired for a streamed one. A
frame inside the mapping that is not a window (another pitch) is
still copied out, and the cached frame is then retired, so a paused
redraw can never follow a parked mapping again.
The D3D12 driver's loan is a persistent buffer of its own, so it was
memory-safe, but it decided by a flag set when the loan was asked
for, not by what was pushed: a window pushed into it was copied as
the whole footprint into a window-sized texture, and a core that
asked and then pushed its own buffer, or a dupe, was shown the loan.
It now makes the same pointer test (d3d12_sw_fb_window) and copies
the window's box out of the loan's footprint; the flag is gone.
Three GFX_INSTRUMENT counters cover the path: frame textures
created, frames copied by the host, frames read out of a loan.
The threaded-video harness gets a lane for it: threaded video off on
the real driver, the harness core's cropped-window lend for 60
frames, then the menu opened over the paused core for longer than
any deferral window. On the old driver on lavapipe under ASan the
lane counts 59 grants and 0 frames read out of the loan, then the
paused redraw aborts with the use-after-free; on the new one 39 of
40 pushes are read out of the loan (the 41st is the core's own
oversize frame, copied as it should be), no host copy, 3 frame
textures over 60 frames, and the paused redraw runs through with
the validation layer silent. It runs on the release, ASan and TSan
legs of Linux-vulkan-validation.yml as they are.
menu_explore_build_list was one handler call: every playlist parsed,
every RDB walked, every category sorted - on the frame thread when
Threaded Tasks is off. The build is now begin/step/end/abort over
three phases (playlists through the resumable playlist parser, RDB
cursor items, one sort per step), consulting a budget between work
items; build_list is begin + one unbudgeted step + end, so the two
cannot drift. The explore init task steps it under the shared
per-frame I/O window and aborts cleanly on cancel.
Harness samples/menu/explore_build writes a collection of playlists
and databases, checks the index built one item per step hashes the
same as the one built in one go, and that the real task returns
unfinished across checks and installs the same index.
Each stage of the staged load is a task of the frontend's queue with
RETRO_TASK_FLG_MAIN_THREAD: its handler runs one stage per check and
returns unfinished while the stage waits, its callback pushes the next
stage, and the last one runs the after-work. The hand-stepped
task_content_load_check and its hook in runloop_iterate go; the
prefetch continuation starts the load from its own callback.
The close, now a handler, asks whether a save state task is inside
the core from inside the gather, where the unthreaded runner has
lifted every task off the queue and a finder sees none: it judged
nothing pending and unloaded the core under a save still writing.
content_save_state_in_progress is now a flag that transitions on the
main thread - set at the push, cleared in the callback - as the
load-state one already is. The harness lane that closes content
over a save in flight is what found it.
Makefile.retrofw asked for HAVE_RETROSSL with no HAVE_NETWORKING:
Makefile.common set HAVE_SSL and the define anyway, retroarch.c called
ssl_socket_set_verify_mode() under HAVE_SSL, and the object holding
it is only built with networking, so the RetroFW link failed. A
makefile without networking now gets no TLS backend at all, and the
call site is gated on networking as its definition is.
tools/platform_makefile_check.sh takes every platform makefile built
on Makefile.common - RetroFW, the OpenDingux family, webOS - without
its toolchain: the graph is generated with stand-ins, the DEFINES it
hands the compiler are read back and checked for consistency (TLS,
SMB and NFS need networking, the clients and the keychain need the
crypto, HAVE_SSL needs an SSL object in the graph), and the frontend
units are compiled by the host compiler under exactly those defines
with implicit declarations as errors. On the old Makefile.retrofw it
fails on the gate; on all eight it passes now. The compile matrix
runs it.
A save or load state task still inside the core holds the close:
the task calls the core's serialize, and the close unloads the
library it lives in. CMD_EVENT_CORE_DEINIT and RARCH_CTL_MAIN_DEINIT
split into a begin half (options, unpause, RAM state, the auto-save),
a pending predicate and a finish half; the commands still wait
between the halves, and the staged load returns to the frame loop
between them, asking each frame, so the frames keep presenting
"Saving state" and core_run's content_closing guard keeps retro_run
out. The harness core carries a 64 MiB state so a save spans
several checks; the lane closes content over one and counts the
frames presented while the close waits.
The last builds carrying another TLS library: the 3DS and Vita
makefiles compiled the bundled mbedtls, the Switch linked devkitPro's,
and the OpenDingux family had the bundled mbedtls (dingux) or a
HAVE_OPENSSL that linked a library no backend used (rg350, retrofw).
All six now select HAVE_CRYPTO, the keychain and HAVE_RETROSSL, as the
desktop, mobile and Windows builds do: the TLS 1.2/1.3 client with
certificate verification and the encrypted keychain, from sources the
tree already has, with nothing to link. The crypto's random source
already had a path for each of them (PS_GenerateRandomBytes,
sceKernelGetRandomNumber, randomGet). The 3DS makefile's note that TLS
could not go through griffin no longer holds, the unity lane proving
it, and the mbedtls include path goes with it.
The compile matrix compiles the crypto and keychain in the 3DS and
Vita shapes and the whole TLS client in the Switch shape, with the SDK
stand-ins the random sources need added to the stubs, and the whole
set with a MIPS cross compiler for the OpenDingux family; the matrix
job installs that compiler. The Vita, 3DS, Switch and RetroFW
workflows build the real things.
aes_ccm_core checked that the message length fits the L octets the
nonce leaves with len >> (8 * L), and for the 11-octet nonce SMB 3.0.2
uses L is 4: on a 32-bit size_t that is a shift by the word width,
undefined, and on MIPS it comes back unshifted, so every sealed
SMB 3.0.2 message was refused there. The same holds for 32-bit x86
and, by the letter of the language, every 32-bit target. The check is
done as 64 bits.
The crypto suite now also runs under qemu-mipsel: a 32-bit,
strict-alignment target, the one such a Linux host can run, standing
in for the 32-bit ARM and x86 builds. It fails on the old line and
passes on the new; the libretro-common tests job installs the MIPS
cross compiler for it.
A task pushed with the flag has its handler run by task_queue_check()
- the main thread - under every runner: the threaded worker and the
GCD dispatch pass such tasks by, and the gather runs the due ones once
each within the handler budget, settling them as the worker settles
its own. Work that must stay on the main thread (calls into the
core, the drivers, the menu) gets a task's lifetime: pushed, found,
cancelled, retired through callback and cleanup.
Sample task_queue_main_thread_test covers both runners: where the
handler runs, one call per check while unfinished, a future 'when',
task_queue_wait(), and plain tasks interleaved with main-thread ones.
tools/platform_stubs/gcd carries a dispatch.h stand-in so the HAVE_GCD
arm syntax-checks with clang -fblocks off Apple.
Reads each distinct ./configure flag set out of .github/workflows,
configures it in a worktree of HEAD and builds: --objects compiles the
frontend units and reports a symbol they call that the default build
defines but this set compiled out (a function inside #ifdef HAVE_MENU
called from runloop.c, the --disable-menu lanes' failure), the
default mode links the whole program. Sets this host cannot
configure are skipped, cross-host sets listed.
The queue was torn down and rebuilt at every load, which joined the
worker thread from a menu action. retroarch_init_task_queue now
splits: startup builds the queue, and a load onto a running session
calls retroarch_task_queue_configure, which applies Threaded Tasks
(the queue swaps at the next check, between frames), the fast-cores
preference, the slow-handler probe and the budgets. Every task in
flight, and the worker, survive the load; the load can become a task
of that queue.
Harness lane: under the threaded queue, a thread-local mark set by a
task before a reload is still on the worker after it, and a task in
flight across the reload retires afterwards.
BS0 and BS1 are baud-rate constants in the termios headers, which the
Android unity build has in scope by the time lrc_hash.c is reached,
so clang reported the SHA-512 sigma macros as redefinitions. They are
SHA512_BSIG0/BSIG1/SSIG0/SSIG1 now, the names RFC 6234 gives them;
the hash and crypto suites pass unchanged.
The Android and Apple builds failed as soon as they carried the SMB
client: their griffin unit also holds rvorbis.c and iec61937.c, whose
static get32 and put16 collide with net_smb2.c's. The helpers are
now smb_get16/32/64 and smb_put16/32/64, crypto.c's aes_ccm_crypt
becomes aes_ccm_core so it cannot meet libsmb2's either, and the
NFS backend's window size is cast to the uint32_t the client takes,
which Xcode warned about.
The host griffin lanes never saw the clash because none of them
carried a codec beside the network stack. The compile matrix gains a
full-unity lane, griffin with every codec, image reader and subsystem
a host can hold beside the whole network stack, which is the shape
the Android, Apple, webOS and MSVC targets ship; on the old file it
fails on exactly these two names.
The MinGW builds have carried the stack since configure took it up,
but the MSVC 2019 and 2022 projects, which the x64 and ARM64 jobs
build, had networking with no TLS or SMB client at all, and the UWP
project had libsmb2 and no TLS. All three are griffin builds, so the
change is their preprocessor definitions: HAVE_CRYPTO, the keychain,
HAVE_RETROSSL, HAVE_RETROSMB and HAVE_RETRONFS on every configuration
line that has HAVE_NETWORKING, giving those builds the TLS 1.2/1.3
client with verification, the encrypted keychain, the SMB2/3 client
with signing, sealing, Kerberos and the read-ahead window, and
nfs://. UWP loses the libsmb2 configuration defines and include
directories with it and links bcrypt for the random source its
WinRT path draws on.
The stack is C89 and MSVC-clean by design; every translation unit
compiles at the _WIN32_WINNT=0x0400 floor with MinGW, and the Wine
lanes run the Windows binaries. The projects themselves are the
MSVC and Windows ARM64 workflows' to build.
The Apple targets still had griffin pull in the bundled mbedtls for
TLS, and the webOS build both mbedtls and libsmb2. BaseConfig.xcconfig
and Makefile.webos now select HAVE_CRYPTO, the keychain, HAVE_RETROSSL
and HAVE_RETROSMB as the desktop and Android builds do, and add the
NFSv3/v4 client, so macOS, iOS, tvOS and webOS get the same TLS
1.2/1.3 client with verification, the encrypted keychain, the SMB2/3
client with signing, sealing, Kerberos and the read-ahead window, and
nfs://; neither bundled library is compiled in any more. The Apple
MBEDTLS_BUILTIN_FLAG stays a variable a deps.xcconfig may still set:
with HAVE_RETROSSL defined griffin takes the built-in client either
way. The keychain's Darwin path resolves gethostuuid at run time
already, so the old SDK floors hold.
The nine objects of the stack compile through Makefile.webos with its
cortex-a9 softfp ARCHFLAGS on an ARM cross compiler with neither
warning nor error; the Xcode build is the Apple workflows' to run.
The APK still had griffin pull in the bundled mbedtls for TLS and
libsmb2 for SMB. Android.mk now sets HAVE_CRYPTO, HAVE_KEYCHAIN,
HAVE_RETROSSL and HAVE_RETROSMB as the desktop builds do, so the APK
gets the same TLS 1.2/1.3 client with certificate verification, the
encrypted keychain, and the SMB2/3 client with signing, sealing,
Kerberos and the read-ahead window, and neither bundled library is
compiled in any more; the libsmb2 configuration defines and include
paths go with it. The mbedtls path stays selectable by turning
HAVE_RETROSSL off.
The compile matrix gains a lane per translation unit of the stack
compiled by clang for aarch64 with the cross headers standing in for
the NDK's: the same compiler family and target the APK is built
with, which is where the AES/PMULL path takes its +crypto target
attribute and which the gcc lanes never exercised. The matrix job
installs clang and the aarch64 cross toolchain for it.
A hardware core makes its SET_HW_RENDER request during the core
stage, while the previous session's drivers are still up. Freeing
those drivers ran video_driver_free_hw_context on that request:
the new core's context_destroy before any context_reset, and the
request zeroed, so drivers_init built no context and the core ran
with none. The driver stage now takes the request out before the
teardown and puts it back for drivers_init, the way a reinit
carries a running core's; the teardown moves into
retroarch_main_init_drivers with it. Context flags set in that
window (glcore's core-context flag) defer to the next context
instead of landing on the one being freed.
The harness core makes such a request at load; a lane checks it
survives the rebuild with nothing destroyed, and that flags set
mid-load defer even on a context that takes them.
HAVE_RETRONFS in the Android makefile: the built-in NFSv3/v4 client
and its backend, which griffin already carries under HAVE_NFSCLIENT,
need nothing but sockets, so the APK gets nfs:// beside the smb:// it
had. The compile matrix compiles the client, the backend and the
prefetcher through the Android platform stubs.
The read-ahead window still stalls the reading thread once per
window, for a round trip and a window's worth of transfer, which on a
slow link is a visible pause every megabyte. vfs_prefetch.c is a
prefetcher a stream opened read-only reads through: a thread of its
own fetches aligned windows ahead of the reader on a connection of
its own (a second pool slot, or a private one), so a run of
sequential reads is served from memory and the reader never waits
for the network. The reader's side takes no lock: slots move
EMPTY -> FILLING -> READY under the thread and READY -> EMPTY under
the reader alone, so every write the thread makes to a slot is
ordered after the reader's CAS and every read the reader makes after
the thread's release; a seek out of the fetched range retargets the
thread by generation, a miss retargets it when it has fallen behind
rather than let it fetch what the reader pulls itself, and the thread
parks on a retro_eventcount when it has nothing to do. Windows are
the client's read-ahead size, two of them. On a miss the stream reads
the network as before, so without HAVE_THREADS, where the prefetcher
is stubs, nothing changes.
The nfs:// close now sends the NFSv4 CLOSE on a connection, which the
connectionless close skipped. The one object is built whenever a
network backend is, in griffin and the sample Makefiles too. A stress
test of random reads and seeks with every octet checked runs clean
under TSan and the address sanitizers, and the three server matrices
pass under TSan with the prefetcher active in the VFS lanes.
Links the shipping objects and drives the menu's load entry point on
a reload of the harness core, hooking the driver's frame() to count
what is presented. Lanes: a frame drawn with the old core down and
another on the old drivers once the new core is up (a synchronous
load presents nothing between and fails), a second request refused
while one is in flight, a reinit deferred to the load's own driver
stage, the dummy-core fallback, and Close Content's reload landing
after the dummy session rather than in the middle of it.
Run by hand against a non-Qt build:
./configure --disable-qt && make
make -C samples/tasks/content_load check
The menu load ran the whole close-core/init-core/init-drivers cycle
inside one runloop_iterate, so the window went dark - or dropped to
the compositor on webOS - for the length of the read. content_load
is now a job of three stages the frame loop steps through one per
frame: the old core closes with the drivers kept up, the new core
comes up behind them, then one driver reinit follows. Between stages
the loop presents the cached frame under the menu, so the window
keeps drawing.
retroarch_main_init splits into a core phase and a driver phase for
this: the drivers are found in the driver phase, after the previous
session's are freed, so both sessions never hold a driver at once.
Core deinit keeps the drivers while switching, a reinit the core asks
for mid-load defers to the driver stage, and the pending menu actions
(config replace, close, core reload) wait for the frame after.
Startup and the CLI still load in one go, before any frame runs.
The threaded-video harness already counts heap calls over the frame
path, but its core pushes no audio, so the sample-batch callback -
where the fold and remap staging used to grow on the callback - was
never under that gate. Nothing in the tree measures what the frame
loop itself costs, either; the frame-path audit had to build that by
hand to learn the answer (about 5 us per iterate with an empty core,
on the null drivers).
samples/runloop/frontend_overhead links the shipping release objects
with main() replaced, the way the threaded-video harness does, boots
on the null drivers, and loads a core whose retro_run() does the
least a real one does: one 320x240 RGB565 frame, one 735-frame stereo
batch, one input poll and read. Two lanes:
- heap: zero general-purpose heap calls over 300 frames of iterate
+ video frame + audio batch, counted by glibc interposition. A
single malloc per frame in the core fails it with the count
(checked: 300 over 300).
- cost: microseconds per frame under fast-forward, engaged the way
the hotkey does it so the pacer sees real facts. Printed so a
trend is visible in the log; tripwire at 1000 us, loose on
purpose, to catch a blocking call or a sleep landing on the frame
path rather than to tune. 3-5 us here.
Linux-frontend-overhead.yml runs it on a default non-Qt release build
- no sanitizer, since one would replace malloc and be counted.
The script runs under set -e, and the oracle was called as a bare
statement: when Samba's own client failed on the runner, the script
ended on that line, exit 1 and not a word, instead of skipping with
the report. The oracle is now called in an if, and the report is
printed by a helper that turns errexit off, so a log that does not
exist yet cannot cut it short either. Both paths were forced locally
and print what they should.
Three follow-ups on the product mask.
Cheevos was the product left on a settings bool: both rcheevos_test()
sites in the iterate read settings->bools.cheevos_enable and called
into cheevos.c every frame, whether or not anything was attached.
Put it on RUNLOOP_WORK_CHEEVOS with an honest lifetime: rcheevos_load()
raises it once it has passed the enable / core-support / account gates
and committed to loading content, rcheevos_unload() drops it. Between
those the tick runs exactly as before (the bg-thread mailbox, then
rc_client_do_frame or rc_client_idle). One visible change: flipping
cheevos_enable off mid-content no longer silently stops the tick,
which previously left a game attached but untested until the next
load; the attach/detach now follows content, as rcheevos already
assumes.
presence_update(PRESENCE_GAME) ran every frame while the bit was on.
"In a game" is a state, not a per-frame event, and its sinks are
Discord (whose pump is already 10 Hz) and Steam. Add
presence_poll(presence, now_us), a 100 ms throttle on the frame's
existing clock, and call that from the iterate. No edge is lost:
each sink compares against its own last status, so a change between
polls is written at the next one.
Document the threading contract on runloop_frame_work_set(): frame_work
is a plain word the iterate reads without an acquire, so every setter
must be on the main thread. All current callers are (init/deinit,
commands, setting handlers); a worker must post a command instead.
A game reading a disc image in small pieces paid one network round
trip per read, and over a real link that is the stutter reported
against the SMB backend. Both clients now keep a read-ahead window
per open file: a read smaller than the window is served from it, and
a miss refills the window from the current offset with pipelined
READs, up to eight in flight (SMB within the credits the server has
granted, NFS by RPC xid, matched as they come back in any order), so
a run of small sequential reads costs one round trip per window and
the fill runs near line rate. A short or empty reply ends the
contiguous run there whatever later chunks answered; reads at least
a window long go straight to the caller's buffer; a write, truncate
or seek outside the window invalidates it; the window is allocated
on the first small read and freed with the file. The named pipe the
share enumeration talks to keeps single-shot reads, since a pipe
holds extra reads pending. On the loopback, 2 KiB reads of a 256 MiB
file went from 37 to 347 MiB/s; with latency the gap is the ratio of
window to read size.
The window defaults to 1 MiB and is a setting per client -
smb_client_readahead and nfs_readahead, KiB, 64 to 16384, under the
SMB and NFS client settings - since it is that much memory per open
file. The nfs_test and smb_test samples read their file through a
4 KiB window in small pieces, across its edges, after seeks in and
out of it and at the end, so every server in the matrices exercises
it.
net_smb2.c calls into net_krb5.c since the Kerberos session setup,
and the SMB fuzzer's link line did not carry it. The four fuzz
binaries also depended only on net_fuzz.c, so a change in any client
they link left a stale binary in place and a local run passed on it;
they now depend on everything they link.
The push no longer grows the region on the callback, so the lane
checks that it never allocates and that a state without staging drops
the audio while the int16 same-layout bypass still records.
Makefile.wiiu compiles against wiiu/include, not wut, so the event
and time headers live under wiiu/os and the tick conversion is spelled
OSMicroseconds there. tools/wiiu_syntax_check.sh compiles the rthreads
units with that build's include directories and defines.
The runner's domain controller provisioned but refused the ticket at
SESSION_SETUP, which passes here; the difference is in the runner's
environment, not visible from one status line. Before the built-in
client is tried, Samba's own smbclient now logs in to the DC with a
ticket from kinit: when it cannot, the DC is not usable there and
the lane skips, printing smbclient's and the DC's log tails; when it
can and the built-in client cannot, that is a real failure, and the
DC's logs are printed with it. The daemon runs with the krb5.conf
its provisioning wrote, and the workflow installs smbclient.
The 3DS arbiter is the futex shape with a less-than comparison: it
waits while the word is strictly below the value given. The epoch only
climbs, so waiting while it is below key + 1 is waiting while it still
reads key, and a notify signals every waiter on the word. INT_MAX has
no successor to compare against; a wait registered there is answered
as a spurious wake, once per wrap of the counter. The unbounded wait
is a long bound re-armed while the epoch still reads key.
The test harness gains a CTR=1 build of the backend on the host, the
two arbiter wrappers stood in for over futex, and a lane at the epoch's
wrap: a wait registered at INT_MAX comes back, and one registered just
past the wrap is still woken.
The pooled backend with an OSEvent in auto-reset mode in place of the
semaphore: a signal that arrives before the wait sets the event and the
wait then returns at once, which is the counted property the protocol
needs, since it signals at most once per park. The bound goes through
OSWaitEventWithTimeout in timer ticks.
The domain controller's provisioning failed on the runner with
vfs_init_custom failed for acl_xattr: the acl_xattr VFS module its
file server needs is in samba-vfs-modules, which samba only
recommends and --no-install-recommends leaves out. Reproduced by
removing the package. The login wait in local_ad_dc_test.sh also
retries a refused session setup for a while, since the KDC and file
server come up in their own time after the user exists.
Since the horizontal (driver-owned) list enumerates synchronously,
walk_status is only assigned on the interactive-playlists branch.
Initialise it to MENU_DIRWALK_FAILED so the variable has a defined
value on every path, matching the file browser site.
The Vita's pooled semaphore backend, on the PS3's lv2 semaphore: both
SDKs' spellings, as rthreads.c has them for its mutex and condition
variable, and an lv2 wait's microseconds with zero standing for no
timeout. Any failure of the wait is answered as a timeout, so the
block comes off the list before its frame goes.
smb_client_realm and smb_client_kdc join the SMB Client settings
(descriptor rows next to the authentication mode, configuration rows
kept literal as the cloud sync strings are). They reach the backend
through smb_settings and the connection key, and the compat shim's
Kerberos mode is now real: smb2_set_kerberos() hands the realm and
KDC to the client, and a KRB5 request needs a realm and succeeds
only when the session came from a ticket, so a pool trying Kerberos
first records the truth and its NTLMSSP attempt still runs, while an
NTLMSSP request never touches the KDC. Share enumeration takes the
same path. A libsmb2 build reads its realm from krb5.conf as before,
so the setter is HAVE_RETROSMB only.
The fold staging and the recorder remap staging are carved out of the
arenas at init now, at fixed sizes, and the batch callback no longer
grows them. The harness stands the state up by hand, as init does it,
and gave the state neither, so every fold answered as having no room
and the lane counted 165 failures. Both stand-ups carve the two as
init does, and the two checks that watched them grow now watch them
stay.
The NFS Network Settings entry opened an empty "null" browser: none
of the NFS client labels had a row in msg_hash_lbl.h, so the label
its deferred list is pushed with resolved to "null" and the push
found no list. Add the rows and their identifier literals, as the
SMB ones have.
Both entries leave the Quick Menu: they are global settings that
already sit under Settings > Network, and a per-game menu is the
wrong place for a client configuration.
samples/menu/network_settings links the shipping objects with only
main() replaced, resolves the SMB and NFS labels, opens Settings >
Network and both screens through the real dispatch (this reproduced
the empty screen before the fix), and checks the Quick Menu no longer
carries the entries. The menu-harness workflow runs it and keeps
networking enabled in its configure so the screens are compiled in.
Provisioning the domain controller sets POSIX ACLs on SYSVOL and
failed in setsysvolacl on the runner, where the temp directory does
not take them. The script now probes for ACL support with setfacl
and provisions in the first place that has it - the temp directory,
the repository's own filesystem, then /var/tmp - and skips with a
clear message when none does. A provisioning failure prints the
log's tail so the reason is visible. The workflow installs acl for
the probe.
The semaphore backend kept a semaphore per thread in a thread key,
which the console ports' threads do not have. The Vita takes the same
backend with the semaphores drawn from a pool instead: a lock-free
stack of kernel semaphores, one taken for the length of a park and put
back on return, which the park protocol allows because a park never
returns with a signal still counted on it. A pool that has run out
answers a park as a spurious wake after a yield. The waiter list's
lock bit is now spun for a bounded count before yielding, so a holder
that is not running gets back on when the threads outnumber the cores.
The test harness gains a SEMPOOL=1 build of the pooled variant over
POSIX semaphores, and a crowd lane: more threads than the pool has
semaphores, all parking with a bound against a moving notifier, then a
concurrent probe on the semaphores handed back and forth, which must
expire clean on any semaphore backend.
rsmb_set_kerberos(realm, kdc, port) makes the session setup obtain a
TGT and a cifs/<server> ticket through net_krb5 and send the RFC 4121
context token in a NegTokenInit with the Kerberos mechanism; the
server's NegTokenResp carries its AP-REP, which is verified before the
first sixteen octets of the context key become the SMB session key
(MS-SMB2 3.2.5.3.1) that the per-dialect signing and sealing keys are
derived from, now in one rsmb_derive_session_keys. An unreachable KDC
or a refused ticket falls back to NTLMSSP as before, and when that
fails too the error names both. The KDC defaults to the server, which
is right for a domain controller; service principals are named by
host, so a server given by address goes the NTLMSSP way.
rsmb_used_kerberos() reports which one the session came from.
smb_test takes the realm, KDC and port from SMB_KRB_REALM, SMB_KRB_KDC
and SMB_KRB_PORT, with SMB_KRB_REQUIRE failing a run that did not get
in by ticket. local_ad_dc_test.sh provisions a Samba Active Directory
domain controller on the loopback with NTLM disabled and mandatory
signing, creates a user, and runs the login, a wrong password, an
unknown user, and NTLMSSP alone (refused, so the login was the
ticket); the standalone matrix gains a lane where Kerberos is
configured but no KDC answers, so the fallback is exercised. The
network-clients workflow runs the DC matrix natively, under the
sanitizers and with the Windows tools under Wine, installing the AD DC
packages that samba only recommends.
With savestate_automatic_interval set, content_save_state_automatic()
called time(NULL) every frame to see whether the interval had passed.
The iterate already has the frame's cpu_features_get_time_usec()
reading in hand; pass that in and keep the last-save stamp in the
same units, so the check is a subtract and compare with no clock call
of its own.
audio_driver_record_push() grew record_remap with realloc() on the
core's sample-batch callback the first time a recording needed a
layout or format remap - the same grow-on-the-callback the fold
buffer had before it was moved into the float arena. Recording is
exactly when a hitch on the audio path is visible in the capture.
Both halves are bounded: the slice is at most AUDIO_CHUNK_SIZE_
NONBLOCKING >> 1 frames, and neither a core's layout nor a recorder's
can declare more than AUDIO_RECORD_REMAP_CHANNELS channels. Carve two
halves of that size out of arena_int16 in audio_driver_init_internal()
next to the rewind region, drop the separate free, and make the size
test in the push a bound check that skips the batch rather than a
grow.
The refill moves the frontend ring into the loop once per notification,
and the ring was one fragment: supply exactly what a fragment plays, so
a pass that came late left a deficit nothing repaid and every fourth
pass was short for good. The ring is two fragments now, so a pass can
move the fragment and the deficit. The margin a resync leaves is
capped at half the loop rather than a quarter, which cut it under the
FIFO on a loop of two small fragments. The loop's sizing is in one
function, wdmks_rt_geometry, so the harness runs the driver's numbers.
samples/audio/wdmks_loop gains the lane that runs them: for a latency
setting, the loop and ring the driver would make, the position moving
half a loop per notification, the refill once per notification, the
consumer feeding chunks of half the buffer it is told, one pass late by
a fragment and a half. The loop must never run dry on time, cost at
most one short pull for the late pass, and heal. It fails on each
geometry that shipped and crackled: a loop four times the setting, a
ring of one fragment, a margin under the FIFO.
video_driver_convert_xrgb2101010() is the fallback for a driver that
cannot present a 10-bit source surface, and it ran on the core thread
right before present with two costs it did not need. It grew its
scratch with realloc() the first time a frame arrived - a heap call on
the frame path - and its inner loop extracted each channel to its own
10-bit value before shifting it back into place, a form the compiler
did not vectorise.
Size the scratch in video_driver_init_internal() from the core's
declared max geometry, so the grow in the convert is only reached by
a core that delivers more than it declared. Rewrite the narrow as one
shift and one mask per channel directly into the output position;
output is bit-identical (checked against the old loop over a full
frame of random pixels) and at -O3 the compiler vectorises it. On a
2.1 GHz Xeon at 1920x1080: 736 us -> 631 us, the same as hand-written
SSE2 (604 us) - the loop is memory-bound past this point. For
comparison the 0RGB1555 scaler path, which the audit also flagged, is
7 us at 320x240 and 31 us at 640x480 on the same box, which is why it
is left alone.
With fps_show on, video_driver_frame() ran snprintf("%6.2f") on the
core thread every frame to print a value that only changes once per
fps_update_interval; with memory_show on it did the same for two
figures that only move once per memory_update_interval. That is
floating-point formatting on the frame path, right before present,
for a string that is identical to the previous frame's almost every
time.
Keep each piece as a static string: the FPS text is reformatted when
last_fps actually changes (a float compare per frame), the memory
text on the interval tick that already refreshes the figures. The
per-frame work is a strlcpy of a short cached string into
status_text. The frame counter is left as it was, since it changes
every frame when shown, and the time text was already cached.
The probe that slept 40 ms at init to measure how the position moves
is gone: a driver does not learn about a device by blocking on a
timer, and the refill did not need to pace on one either. The loop is
two fragments, each the latency setting's worth: one is held ahead of
the hardware, the driver's two notifications a loop land once a
fragment as the hardware leaves it, and the refill wakes on that and
fills what it left. The hardware's clock paces; the timer is a
watchdog. The position can be stale by no more than a fragment by
construction, and a pass late by up to a fragment is still inside the
lap, which is what the loop the size of the setting could not give.
The post-core_run product block called into four translation units
every frame to find out nothing needed doing: cheat_manager_apply_
retro_cheats() loads a cold global to see there is no cheat list,
presence_update() calls discord_update() and steam_update_presence()
which each compare a status word and return, the camera poll reads
settings_t for camera_allow before the caps check, and
bsv_movie_next_frame() pulled two settings through config_get_ptr()
before checking whether a movie handle exists at all.
Give each an up/down owner and a bit on runloop_state_t::frame_work,
raised and dropped by the product itself through
runloop_frame_work_set():
RUNLOOP_WORK_CHEATS - cheat_manager_new/realloc raise it when a
list is allocated, cheat_manager_free drops it
RUNLOOP_WORK_PRESENCE - raised with Discord init and with Steam
(mist) init; dropped with whichever of the
two is the last compiled-in sink
RUNLOOP_WORK_CAMERA - driver_camera_start raises it when the core's
start succeeds, driver_camera_stop and the
driver teardown drop it. The remaining
camera_allow/caps/driver checks stay behind
the bit; a camera the core registered but
never started is no longer polled.
BSV needs no bit: the movie handles live on input_st, which the frame
already has in cache, so the iterate tests the pointers directly and
bsv_movie_next_frame() checks its handle before touching settings.
A pin with no position register answers PlayOffset from its driver,
and a driver that reports from its fragment interrupts moves it a
fragment at a time - on a loop of two fragments, half the loop. Audio
held ahead of a position stale by that much is audio the hardware may
already have passed: it plays the previous lap's bytes there, and the
count, which trusts the position, says nothing. The step is measured
once the pin runs, 40 ms at 200 us, and what is held ahead is floored
at the margin and two steps before the frontend is told what that is.
Where the position moves finely nothing changes.
Even throttled, the iterate reached the Discord pump by calling into
discord.c and loading a cold global to learn that Discord was not in
use. A product that is compiled in but idle should cost the frame one
bit test on a word the loop already has in cache, not a cross-TU call.
Add runloop_state_t::frame_work, an unsigned in the hot front of the
struct next to pace, with one bit per optional product that hooks the
iterate. Discord is the first: retroarch_main_init() raises
RUNLOOP_WORK_DISCORD when CMD_EVENT_DISCORD_INIT succeeds (which it
does not when discord_enable is off) and retroarch_main_quit() drops
it with inited. The iterate tests the bit and only then calls
discord_poll(). The other per-frame products (presence, cheats, BSV,
camera) can move behind the same word one at a time.
The notification event fires twice a loop, and the refill waits on it
where the pin gives one. With the loop decoupled from the latency
setting - four times it - a notification comes every two settings'
worth of audio, and a refill that waits for it has run the held audio
out before it wakes: constant crackle on any pin with an event. The
event is not taken when the loop is larger than what is held; the
timer path, whose cadence is sized from the held amount, paces instead.
samples/audio/wdmks_loop checks the choice.
runloop_iterate() called Discord_RunCallbacks() and
Discord_UpdateConnection() at the top of every visible frame whenever
Discord was inited. UpdateConnection does a non-blocking read on the
RPC pipe (and a clock read plus reconnect backoff check while it is
not open) - a syscall on the emulation thread, every frame, for
callbacks that arrive a handful of times per session.
Add discord_poll(now_us), which runs both at most every 100 ms using
the frame's existing timestamp, and call that from the iterate
instead. Presence changes are queued by discord_update() and flushed
by the next pump, so nothing is lost; join/spectate callbacks arrive
within 100 ms, which is far inside what Discord itself tolerates. The
shutdown path in retroarch.c still calls Discord_UpdateConnection()
directly so the final ClearPresence goes out unthrottled.
The WaveRT loop was the size of the latency setting, so a 2 ms setting
made a 2 ms loop. Two things went wrong with that. A pass of the
refill thread 2.5 ms late was a lap the position could not be followed
across, so the whole loop was cleared and the cursor resynced, an
audible discontinuity for half a millisecond of missed audio. And the
margin a resync leaves for the codec's FIFO was cut to a quarter of the
loop - 24 frames against a FIFO of 46 - so every resync, the start
included, wrote its first frames into bytes the hardware had already
fetched.
The loop is now four times the setting and 8 ms at the least, and the
setting is what is held ahead of the hardware beyond that margin: the
frontend's chunking, the ring it writes and the latency it is told are
all the setting's worth, a late pass has laps to spare, and the margin
is the FIFO's.
samples/audio/wdmks_loop gains the lane: a short setting on a large
loop fills the margin and the setting's worth and no more, and a pass
three settings late is an overrun on the held audio, not a lost lap.
Runahead is N + 1 core steps per visible frame plus a serialize and a
deserialize, and the serialize cost is the core's state size, not
something the frontend controls. For a 1-4 MB state that is nothing;
for a core whose state is tens of MB, or one that runs near the frame
budget on its own, N frames of runahead no longer fit and the result is
dropped frames and stretched audio - worse latency than not running
ahead at all. There was no gate: the requested count ran every frame
regardless of what it cost.
Measure the cost of one core step every frame (the whole of
runahead_run() divided by the steps it made, so the save and load are
folded in), IIR-average it (1/8), and clamp the frame count to what fits
in 70% of the core's frame period. The count climbs back only once the
request fits within 60%, so a core sitting on the boundary does not flip
between N and N - 1 every frame. Samples above 250 ms (a JIT warm-up, a
page-in) are dropped rather than dragging the average. Both instance
paths cost N + 1 steps, so one bound covers both.
A clamp is logged and, unless run_ahead_hide_warnings is set, shown once
on the OSD when it changes; recovery is logged only. The measurement and
the clamped count live in runloop_state_t next to the other runahead
state and reset with it in runahead_clear_variables().
audio_driver_multi_fold_room() grew multi_fold with realloc() whenever a
batch slice was larger than any seen before. That is a heap operation
on the core's sample-batch callback - the one place in the audio path
that must not allocate.
Every multichannel entry (the discrete and folded paths of both the
int16 and float batch callbacks, and the rewind path) already slices its
batch to at most AUDIO_CHUNK_SIZE_NONBLOCKING >> 1 frames before
folding, so the buffer has a fixed upper bound of 1024 frames. Carve it
out of arena_float in audio_driver_init_internal() alongside the other
scratch regions, and turn the room check into a pure bound test. The
region is freed with the arena, so the separate free() goes too.
Time-critical is 15 in the normal class, and a display driver's worker
threads sit above it, which on a machine driving a 4K HDR swapchain is
where a 2.5 ms hole in a thread following a 2 ms loop comes from. The
refill thread now takes the Pro Audio scheduling class under the same
setting the WASAPI pump takes it, falling back to time-critical where
the class is off, absent or refused; the teardown line names which.
With Audio Sync and rate control off the pipe's fill is regulated by
nothing: it is whatever priming put there, and priming filled it to one
publish - which is a frame just after the core hands one over and
nothing just before the next, so a frame late by any amount reached the
device as silence. The target on that path is two publishes now, a
frame's worth held at the worst phase, which is what a late frame costs.
Two things then kept the margin from lasting. The trim after an
underrun cut the pipe back to the target without the device's share
priming had added on top, so the first pass after every prime threw
that share away as late audio and the fill settled a device buffer
short; the same figure now serves both. And a stall that emptied the
pipe left it empty - a core paced to the display never runs ahead to
refill it - so silence found with the pipe short primes it again, the
wait the first pass made, while the device is silent anyway.
samples/audio/pipeline_clocked gains the lane: a 60 ms stall the core
does not catch up from, then three seconds of frames up to 12 ms late,
which must not reach the device. It retries a run the host held the
process through, which its own wake lateness shows.
The driver's count of silence says the device went unfed; it does not
say by whom. The threaded pipeline's consumer now keeps the other half
of the account - the passes after priming that found the pipe short and
had to wait for the core, the longest such wait, and the least the pipe
held at the start of a pass - and says so once beside the driver's line,
so a session with silence in it names the side that was late.
playlist_push_runtime() shifted every entry up and then filled the new
top one with strdup() results it never checked, so a copy that failed
partway left the list one entry longer with no path or no core on top,
and the next write put that on disk. The strings are made first now,
and a push that cannot make them leaves the list as it was.
samples/playlist gains a lane that fails the copies at every point.
The refill thread follows the hardware's position round the mapped
loop, and a pass that lands a loop late is a lap it cannot follow, which
the driver counts as a period of silence and resyncs across. At a loop
of a few milliseconds that is one ordinary scheduling delay next to a
busy core, and the thread ran at normal priority. It now raises itself
to time-critical where audio_thread_priority asks, as the WASAPI pump
does under the same setting, and a pin with no notification event is
sampled every quarter of the loop rather than every half, so a sample
late by up to three quarters is still inside the lap.
The thread keeps its own account of how it woke - the worst gap between
two passes and how many were a loop or more - and says so once at
teardown, beside which scheduling it ran under.
samples/audio/wdmks_loop gains a lane that runs the thread on the fake
register under both settings.
The consumer could read the count, find it drained, then see the done
flag a producer set after adding one more item, and leave that item
behind: a fail one run in thousands, on any backend.
Whether a bounded wait ever expires inside a burst depends on the
machine: a loaded runner under a sanitizer wakes every wait before its
bound, and the win32 event tier rounds the bound past the notifier's
period. The held-notifier probes show the bound is honoured; the lane
now fails only when those all end early and no burst expired either.
svcWaitForAddress/svcSignalToAddress are the futex shape, so the Switch
build takes the lockless path on firmware 4.0.0 and newer and keeps the
condition variable only on older firmware, decided once per object at
init.
The test harness gains a SWITCH=1 build of the arbiter protocol on the
host, with the two system calls and the firmware query stood in over
futex; check runs it as current firmware and as pre-4.0.0.
The PS4 toolchain defines __FreeBSD__, but its libc carries neither the
POSIX clock the semaphore backend's bounded wait needs nor a semaphore
that backend can rely on, so it is kept off that backend the way
rthreads.c and retro_procbarrier.c keep it off the FreeBSD paths.
The Windows backend's waiter list - stack blocks on the object, a spin
bit in the head pointer - gains a second sleep: a counting semaphore
of the waiting thread's own, made on its first park and kept in a
thread key for its life. A signal that lands before the wait is
counted rather than lost, the property the Windows ALERT tier has, so
no rendezvous is needed; a park is one wait and a wake one signal per
parked waiter, and nothing on either side takes a mutex. A bounded
wait that expires while a waker has taken its block consumes the
signal in flight, so the count never carries into the thread's next
park. The list is shared with Windows; only the sleep differs.
Darwin spells the semaphore in Mach - semaphore_create, semaphore_wait,
semaphore_timedwait, semaphore_signal - every call of which is in Mac
OS X 10.4's headers, so one binary runs from there up with no runtime
resolution; the BSDs spell it in POSIX. RETRO_EVENTCOUNT_FORCE_SEM
builds the POSIX spelling on any host, and the eventcount test's check
runs it beside the futex and scond backends. A new lane paces a
notifier to the consumer's bound so expiry and wake coincide, and
probes for a carried-over count after each burst with the notifier
held; on the semaphore backend the probe must expire.
The bench and test binaries that had been committed are untracked and
ignored.
The wrapper's lock existed only to give its condition variable
something to sleep under. Both threads park on one retro_eventcount:
the audio thread in its stopped loop and its idle wait, the main thread
for the stop acknowledgement and for init() to return. Every change to
the words they test - alive, stopped, stopped_ack - is a release store
followed by a notify, and the re-check between prepare and commit is
what makes a request that lands in the window visible.
The stop acknowledgement answers the flag it read: block() clears the
ack before it raises the flag, so a thread that finds the ack clear
inside its wait window knows a start and a second stop landed since it
last answered, and answers again before parking. How init() ended, and
who owns the state when the frontend gives up waiting, is one word
moved by compare-and-swap from either side, so exactly one of them
frees it.
audio_thread_loop's ceiling in tools/lock_acquire_budget.ref is 0.
The frame ring's bookkeeping - the slot the video thread claims next,
how many filled slots wait, whether one is being rendered - is one
atomic word, frame.state, moved by compare-and-swap on both sides: the
push takes back the newest unclaimed frame with a CAS, the video thread
claims and completes with a CAS, so neither can win the same slot. The
ring's waiters park on a per-ring eventcount and the video thread on a
work eventcount, notified by whoever gives it something to do; the
menu-texture drain and idle wait slice their parks on the Apple main
thread so marshalled jobs still run. Slot contents stay ordered by the
publish and the claim.
The presenter's inputs to display pacing - period, next vblank, render
time - travel in the stats seqlock, so the hold reads them as one
snapshot; drain requests, repeat requests, the content period and the
main thread's core time and pacing flag are atomic words; the wrapped
driver's poke table is installed on the video thread through a
blocking command, the thread that reads it every pass. Released
textures ride an mpsc_stack to the next push. thr->lock covers the
command channel and the upload lists only, and video_thread_frame
takes no lock at all: its ceiling in tools/lock_acquire_budget.ref is
0, and video_thread_loop is measured there too.
The threaded_video harness reads the video thread's own fields and
swaps its drivers in through blocking commands on that thread, and
takes the repeat and swap counts in one read.
The griffin unit includes net_krb5.c next to x509.c and net_smb2.c,
which have static der_peek and der_len of their own with other
signatures, so the build failed on the conflicting types. Every DER
helper, type and tag macro in net_krb5.c now carries a krb5_ / KRB5_
prefix; nothing else changes.
The tamper case flipped the low bit of the third-last base64
character, and one time in 64 that character was 'A', whose flip
'@' is outside the alphabet; unbase64 does not reject it and reads
it as 'A' again, so the sealed value decoded unchanged, opened, and
the assertion failed at random. Swap the character for a different
alphabet character instead, so the decoded octets always differ and
the AEAD tag always refuses them.
Full-scale float to 32-bit integer with a valid-bits mask for
24-in-32, saturating with NaN to 0, in the shape of
convert_float_to_s16: an SSE2 arm, a NEON arm enabled from
cpu_features at init, and a scalar tail. The wdmks driver's write path
converts through it for the wide integer pins.
libretro-common/samples/audio/float_to_s32 checks every arm against a
scalar reference on the edges and at every tail length; it runs
natively for SSE2 and cross-built under qemu-aarch64 for NEON, and is
registered in the samples workflow.
A pin whose ranges name only 32-bit PCM - a USB Audio Class 2 DAC as
usbaudio2.sys reports it - had nothing proposed to it and the driver
gave up with no format tried. Both wide integer shapes are proposed
now: the render side converts the frontend's float frames into them
on the write path, the capture side converts them back into float, and
either is the same size as float so nothing about a write or a read
changes. The order is float, 32, 24 in 32, 16. Float is also offered
once more as a last resort to a pin with a 32-bit-wide range and no
float subtype, since such pins have taken it when asked.
samples/audio/wdmks_loop gains a lane for the pin shape and both
conversions.
On the enctype layer, the protocol: a DER writer that closes
constructed values by moving their content up under the header (the
lengths stay minimal, as the KDCs require), a matching reader, and
the three exchanges. krb5_get_tgt sends an AS-REQ without
preauthentication and, when the KDC answers KDC_ERR_PREAUTH_REQUIRED
with ETYPE-INFO2, retries with PA-ENC-TIMESTAMP under the key from
the salt and enctype it named; the KDC's own time in the KRB-ERROR
corrects our clock for everything after. krb5_get_service_ticket
runs the TGS exchange with an AP-REQ of the TGT in PA-TGS-REQ (the
req-body checksum with usage 6, the authenticator with 7, the reply
under the TGT session key with 8). krb5_gss_init_token wraps an
AP-REQ for the service in the RFC 4121 initial context token with the
0x8003 checksum, mutual authentication and a fresh subkey, which is
the mechToken a SPNEGO NegTokenInit carries; krb5_gss_accept_token
takes the AP-REP and settles the context key. All over TCP with the
four-octet length, one connection per exchange, timeouts throughout;
no snprintf, frames under 1.4 KiB.
samples/network/krb5_retro/krb5_test drives the client against a KDC
and, given the service key, proves the ticket in the token is real by
decrypting it with that key (usage 2) and finding the client name
inside. local_kdc_test.sh stands up a MIT krb5kdc on a fixed port with
a realm, a user that requires preauthentication, one that does not,
and a cifs/ service whose key it reads back from its keytab, then
runs both paths, a wrong password against each, and an unknown
service. The network-clients workflow installs krb5-kdc and runs the
matrix natively, under ASan/UBSan/LSan and with the Windows build
under Wine. The crypto and keychain suites now link the socket layer
alongside net_krb5.c.
The capture side of kernel streaming, as microphone_wdmks: the same
filters enumerated under the capture alias, a pin audio flows out of
opened for reading, mono first and stereo folded down where the pin
captures nothing narrower. WaveCyclic pins are read with
IOCTL_KS_READ_STREAM packets; WaveRT pins through the mapped loop, read
from behind the hardware's position, with lost laps and overruns moving
the read cursor up to the hardware. Every wait is bounded, and the
packet reclaim on close is the render side's.
The WaveRT buffer, register and notification-event requests and the
device scan and pin open take the pin handle and direction, so both
sides share them. samples/audio/wdmks_loop gains lanes for the fold and
the capture loop model.
A pin's data ranges are kept apart rather than folded into one box, so a
candidate format is proposed only when a single range holds its rate,
width, channel count and sample type together: a pin offering 48 kHz
stereo and 192 kHz 8-channel no longer has 192 kHz stereo asked of it.
Past twelve ranges the rest widen the last kept range of their type.
Pin records move to the heap with the device that owns them.
The WaveRT position register is read at the width the pin reports. A
64-bit register is a byte count, and its low word alone cannot place
the hardware once the count passes 2^32 on a loop that is not a power
of two; a 32-bit build reads the high word twice so it is never torn.
samples/audio/wdmks_loop gains lanes for both.
The whole-block ciphertext-stealing swap copied one block of the
output buffer over another. The two blocks are adjacent and do not
overlap, but the overlap-copy check keys on any copy within the same
buffer, so take both blocks into temporaries first. The RFC 3962 CTS
vectors pass as before.
Every gfx_widgets_msg_queue_push() caller is main-thread code -
runloop_msg_queue_push() defers off-main callers before it gets there
and the task queue's progress push runs from its main-thread gather -
and the consumer is the thread that owns the widgets, so the pending
ring has one producer and one consumer. It is now the retro_spsc
shape: the producer owns msg_queue_tail and the consumer
msg_queue_head, free-running counts indexed by mask, each published
with a release store and read across with an acquire load.
msg_queue_lock is gone.
gfx_widgets_msg_queue_race_test builds the production push and pop
out of gfx_widgets.c and drives them from one producer thread and one
consumer thread, checking order, loss, full-ring refusal and the
consumer handover at deinit, under ThreadSanitizer in CI. The
snapshot-ownership test models a full ring through the cursors.
tools/libretro-common-link-check.sh also scans tests-other/.
The first layer of Kerberos for the built-in SMB client: the AES
enctypes of RFC 3962 (aes128/aes256-cts-hmac-sha1-96), which every
Active Directory domain and MIT/Heimdal realm offers. net_krb5.c
carries n-fold, DK, CBC with ciphertext stealing, string-to-key,
encryption with the confounder and truncated HMAC, checksums and the
PRF, on the cleanroom AES and SHA-1 alone; it is built with
HAVE_RETROSMB and the protocol (AS/TGS/AP exchanges, GSS tokens,
SPNEGO in SESSION_SETUP) goes on top of it next.
CTS needs the inverse cipher, which no mode so far did, so crypto.c
gains aes_decrypt_block on all three paths: the portable one runs the
standard inverse order on the encryption schedule with InvMixColumns
on the state, AES-NI applies aesimc to the round keys per block and
ARMv8 vaesimc likewise, so struct aes_ctx and every existing caller
stay as they are and no decryption schedule is cached. kdf.c gains
HMAC-SHA1 and PBKDF2-HMAC-SHA1 for the enctypes' checksum and
string-to-key, in the shape of the SHA-256 ones.
The crypto suite pins all of it: FIPS 197 C.1-C.3 for the inverse
cipher plus round trips through this build's own path, RFC 2202 and
RFC 6070 for the SHA-1 pair, all seven RFC 3961 A.1 n-fold vectors,
all six RFC 3962 B CTS vectors (encrypt, and decrypt in place), all
RFC 3962 B string-to-key vectors, and encrypt/decrypt round trips
with tampering, wrong usage and short input refused. The suite runs
natively, under qemu-aarch64 on both -march variants and as the
Windows binary under Wine.
rzip_stream.c parks its chunk-writer pool on retro_eventcount, so the
content_prefetch sample and the companion test scripts (core, cocoa,
cocoa_macos, win32), which compile it with HAVE_THREADS, link
rthreads/retro_eventcount.c alongside rthreads.c.
tools/libretro-common-link-check.sh builds every sample Makefile in
the tree that names a given libretro-common unit and lists the
tools/*.sh harnesses that do, so a unit gaining a dependency on
another libretro-common TU is caught at link time across all of
them before delivery.
Each pool slot is owned by one worker and touched by exactly two
threads, so the slot status is an acquire/release atomic word that
orders the slot's inputs and outputs, and parking goes through
retro_eventcount: one per slot for its worker, notified only by the
writer, and one for the writer, notified by whichever worker finishes.
A dispatch or a completion wakes exactly the thread waiting for it.
Output stays byte-identical to the serial writer.
rzip_parallel_test builds rzip_stream.c with HAVE_THREADS, drives the
pool at every width (the serial fallback, two, eight, and a count past
RZIP_MAX_THREADS) with the core count wrapped at link time, and
compares each result to the serial file byte for byte, both codecs,
including a stream written in pieces across pool wake-ups and two
hundred short pool lifetimes. It runs in the samples workflow under
ASan/UBSan and under ThreadSanitizer. The other sample Makefiles that
build rzip_stream.c with HAVE_THREADS link retro_eventcount.c.
rzstd_encode()'s predefined-table pointer array is a table of address
constants, so it is initialised once as const and concurrent encoders
only ever read it. Clears the ThreadSanitizer report on parallel
Zstandard writes.
The other local_*_test.sh scripts cd to their own directory before
building; this one did not, so invoked from the repository root (as
the workflow does) its bare make hit RetroArch's top-level Makefile
and stopped on the missing config.mk.
nfs_version had a config key but no menu entry, so choosing NFSv4
meant editing retroarch.cfg. Add it to the NFS Client group next to
the ports as a 3..4 range with a sublabel explaining what each
version needs from the server.
Add NFSv4.0 (RFC 7530) to the built-in NFS client, selected per
connection with rnfs_set_version(c, 4): one COMPOUND per operation
over a single TCP connection to port 2049, no portmapper or MOUNT
service, the export given as the server's pseudo-filesystem path.
SETCLIENTID/CONFIRM establishes the client, OPEN/OPEN_CONFIRM with a
per-connection open-owner and advancing seqid creates files, READ and
WRITE use the anonymous stateid, SETATTR truncates, CREATE/REMOVE/
RENAME (SAVEFH: source dir saved, target dir current) and READDIR
with fattr4 cover the rest. NFS4ERR_GRACE and DELAY are retried on a
fixed bound so a restarting server never blocks a caller forever;
the wait is a bare select so no timer header is pulled in. Large
COMPOUND buffers are heap-allocated so every frame stays under 2 KiB
including where the helpers inline into rnfs_connect. The client
verifier and open-owner are derived without shared state, and the
VFS backend's last-error string now goes through a ring of slots
claimed by an atomic counter, so the pooled threads test is clean
under TSan for both versions. Nothing here depends on the crypto
code, so the file still links without HAVE_CRYPTO.
The nfs:// backend gets nfs_settings.version and RetroArch a
nfs_version config key (default 3, unchanged behaviour). The NFS
samples take the version as a fifth argument, local_ganesha_test.sh
runs the same four lanes as the unfs3 matrix against nfs-ganesha,
and the network-clients workflow runs it natively, under ASan/UBSan/
LSan and as the Windows tools under Wine.
der_read_time multiplied the day count by 86400 in long, which is 32
bits on Windows on both architectures, so any notBefore/notAfter past
2038-01-19 wrapped negative and the certificate read as expired. The
leaf-policy test vectors run to 2046 and tripped this in the Wine lane
while the Linux run, with a 64-bit long, passed. Do the arithmetic in
int64_t, and where time_t itself is 32 bits saturate rather than wrap
so a far-future notAfter stays in the future.
The job installed wine64 alone, which on the runner brings the 64-bit
loader but not the wine command the Windows lanes run the tools
through; the unit-test job had always installed wine as well. With
the launcher missing the Wine step's own check skipped cleanly, but
the matrices then ran their MinGW builds through nothing and failed
with command not found. The job installs wine with wine64 and
asserts the launcher is present before the lanes run.
The certificate policy the verifier gained from the audit - unknown
critical extensions refused, an extendedKeyUsage without serverAuth
refused, a keyUsage that cannot sign refused - was covered by the
server matrix only, which needs openssl at hand. The crypto suite now
carries an independent P-256 CA and six leaves under it, valid for
twenty years: serverAuth with digitalSignature and an unknown
non-critical extension pass; an unknown critical extension, a
clientAuth-only EKU, a codeSigning-only EKU and a keyUsage without
digitalSignature are refused. Runs natively, on both limb widths,
under qemu-aarch64 and as the Windows build under Wine.
Both backends had one lock around every operation, held across the
network I/O, so a read on one connection blocked every other thread's
access to that backend and a stalled server held up all of them. The
pools are now lock-free. Each slot is a connection and a busy flag; a
caller takes a slot by swapping the flag from 0 to 1, uses the
connection alone, and swaps it back - no waiting anywhere. When every
slot is busy the caller connects privately for its own use and drops
that connection afterwards: slower, but never blocked behind another
thread. The pool itself is published with a CAS by whichever caller
builds it first, and a pool whose slots are still held is left alone
by a settings change (callers connect privately until it is idle).
The two protocols place their handles differently: an SMB file or
directory handle belongs to its session, so an SMB stream holds its
slot from open to close and a listing from opendir to closedir, while
stat and share enumeration borrow one per call; an NFS file handle is
valid on any connection to the export, so an NFS stream holds nothing
and every read borrows a slot. Slots connect on first use, and the
SMB auth-mode probe settles per pool from the first connection.
Both backends are clean under ThreadSanitizer with four threads on a
two-slot pool, which exercises the private-connection path; the SMB
matrix gains that test beside the NFS one. Both pass native, under
ASan and as the Windows builds; without threads the atomics compile
to plain operations and the objects carry no thread references.
The libretro-common test job has a fifteen-minute budget, and the
network lanes that accumulated in it - three server matrices, the
Windows builds under Wine, the sanitizer rerun, the fuzz targets and
building unfs3 from source - now take it past that, so the runner
cancels whichever step is running when the clock runs out; the last
run died in the Cocoa companion harness, which has nothing wrong with
it. The six network steps move to Linux-network-clients.yml with a
budget of their own and only the packages they need; the unit-test
job is back to what it was before the network stack arrived.
Trust store and curves: the X.509 trust store is process-wide and
the first handshakes on the updater, achievements and cloud-sync
threads raced to build it - free, reallocate and publish it
field by field. One lock now covers building, freeing and every
chain walk over it, and a load of the source already loaded is the
no-op every later handshake takes. The same test found a second
first-use race in pk.c: the curve constants were computed on first
use under a "every thread writes the same values" argument, which
lets a thread publish the ready flag while another is still writing
a constant a third one reads. The computation runs under a lock,
with an acquire load of the flag on the fast path. Six threads making
their first connections at once are clean under ThreadSanitizer
(seven reports before), and the case is in the matrix.
Certificate policy: an extension marked critical that the verifier
does not understand now makes the certificate unusable, as RFC 5280
requires; extendedKeyUsage is parsed and a leaf whose EKU lacks
serverAuth (client-auth or code-signing only) is refused, as is a
leaf whose keyUsage cannot sign the handshake or that is itself a CA
in a chain. Six matrix cases cover the refusals and the two shapes
that must still pass. The 155-certificate bundle still loads.
Staging: a core without VFS support had its remote content copied to
<cache>/<basename>, which overwrote any file of that name already in
the cache and deleted it on unload. The copy now goes into a private
directory made for the connection, under a part name until complete,
then renamed; file and directory are registered for removal only
once they exist, the directory after its file. Verified with the
real binary: an unrelated <cache>/vfs.bin survives the run.
Also from the audit: the smb:// / nfs:// scheme check read six
octets of any path (string_starts_with now); the built-in TLS
backend's verification mode is an atomic that each connection copies
at creation, like the mbedtls backend already did; and net_http
refuses a source callback claiming more bytes than the buffer it was
handed.
libFuzzer harnesses under test/fuzz for the four parsers that read
bytes off the network: the X.509 parser (the input as a DER
certificate and as a PEM bundle), and the TLS, SMB and NFS clients'
receive paths - a listener in the harness accepts every connection
the client makes and answers with the fuzzer's input, so the real
socket and parsing code runs against arbitrary bytes. Seed corpora
were recorded off openssl s_server (1.2 and 1.3, RSA and ECDSA), Samba
and unfs3 through a small relay, which is what gets the fuzzer past
the first framing checks. Two minutes per target under ASan and UBSan
found nothing; the libretro-common CI job runs thirty seconds per
target and fails on any artifact. The SMB client gains rsmb_set_port()
for the harness, which is also what a server on a non-standard port
needs.
The third TLS 1.3 suite runs its whole key schedule over SHA-384:
48-octet secrets, HKDF and HMAC over that hash, the SHA-384 transcript
the client already keeps for 1.2, a 32-octet AES key. The 1.3 code
now takes the hash length from the suite everywhere it had assumed
SHA-256 - the secrets, Derive-Secret, the Finished MACs, the
CertificateVerify content, the ticket PSK, the binder (over the hash
of the suite the ticket was issued under) and KeyUpdate - and kdf.c
gains HKDF over SHA-384, checked against an independent computation
in the crypto suite. Servers restricted to this suite, which
compliance profiles such as CNSA call for, now connect and resume.
Two matrix cases: the suite on an RSA certificate, and on ECDSA with
three rounds resuming.
The 1.3 KeyUpdate handling had no test: nothing in the matrix ever
rotated keys mid-connection. tls_keyupdate drives openssl s_server's
console - lines to the client, 'K' for a KeyUpdate with update
requested, 'k' for one without - and exchanges a line with the server
across each: the client must keep reading under the server's new keys
and, on the requested one, rotate its own so the server can read the
reply, which the server's output shows. Both directions pass, native,
under the sanitizers and as the Windows build under Wine.
The bench and this tool no longer connect the socket before handing it
to the client, which connects it itself: a second connect passes on
Linux but Winsock refuses it, which the Wine run caught.
X25519 is the group most servers prefer and some allow alone; the
client offered P-256 and P-384 only, so a 1.3 server wanting X25519
had to retry, which the client refuses, and a 1.2 server restricted to
it could not connect at all. pk.c gains RFC 7748 X25519 - the
Montgomery ladder over 2^255 - 19 on the field arithmetic the curves
already use, scalar clamping, the all-zero result refused - checked
against the RFC's vectors in the crypto suite on both limb widths.
The ClientHello offers x25519 ahead of the NIST groups and carries an
X25519 share beside the P-256 one, so no 1.3 server needs a retry;
the 1.2 ServerKeyExchange accepts the group too. Four matrix cases
pin each group alone on each version.
With 1.3 negotiated everywhere, every connection had gone back to a
full handshake. The client now keeps the resumption secret from each
1.3 handshake, turns the server's NewSessionTicket into a PSK
(Expand-Label over the ticket nonce, the ticket as identity, its
age obfuscated as the server asked) and offers it in the next
ClientHello with psk_dhe_ke and a binder computed over the truncated
hello. When the server selects it the early secret comes from the
PSK and the server's flight is EncryptedExtensions and Finished
alone; when it does not, the handshake runs in full as before. The
cache entry for a host is either a 1.2 session or a 1.3 ticket. Real
hosts resume on the second connection; the matrix gains three cases:
three rounds on AES-GCM, two on ChaCha20, and a server issuing no
tickets, where no round resumes.
Measuring the resumed handshakes with a request and reply attached
showed the 1.2 abbreviated case at 44 ms: the first request went out
while the client's Finished was still unacknowledged and Nagle held
it for the server's delayed ACK. TLS sockets now set TCP_NODELAY
where the platform has it - everything RetroArch does over TLS is
request-then-reply, so there is nothing for Nagle to coalesce -
which puts a resumed 1.2 connection with its first exchange at
0.7 ms and a resumed 1.3 one at 1.6 ms on loopback.
Wine refuses to create a prefix inside a directory it does not own,
and $RUNNER_TEMP belongs to the runner user, so the SMB matrix under
sudo could not start its Windows tools. The root run now uses a
prefix under /tmp with root's own HOME, initialised with wineboot
before the first tool runs.
The ClientHello now offers TLS 1.3 ahead of 1.2 - supported_versions,
a P-256 key_share, the two SHA-256 suites (AES-128-GCM and
ChaCha20-Poly1305) and the rsa_pss_rsae signature schemes 1.3
requires - and the ServerHello decides which handshake runs. For 1.3
that is the RFC 8446 1-RTT flow: the HKDF key schedule from the
shared secret and the running transcript, handshake traffic keys for
EncryptedExtensions, the 1.3-shape Certificate (verified through the
same chain code), CertificateVerify over the transcript hash (ECDSA
P-256 / P-384, or RSA-PSS with MGF1 and a hash-length salt, which
pk.c gains and the crypto suite checks against OpenSSL signatures in
both hash sizes plus four refusals), the Finished MACs both ways, then
the application keys. Records take the 1.3 shape: outer type
application_data, the real type inside, iv XOR sequence as the
nonce, the header as AAD; the compatibility ChangeCipherSpec is sent
and tolerated. After the handshake, tickets on the application stream
are dropped (no PSK resumption yet) and a KeyUpdate rotates the
server keys and is answered when asked. A HelloRetryRequest is
refused since every 1.3 server takes P-256; a server offering only a
suite we do not carry gets no downgrade, just no connection.
Offering the PSS schemes lets a 1.2 server sign its
ServerKeyExchange with them too, so the 1.2 verifier takes them as
well. The 1.2 matrix is unchanged and green; nine 1.3 cases join it
(RSA and ECDSA certificates, both suites, a P-384 signature, an
intermediate chain, untrusted CA and wrong host refused, the
unsupported-suite case), all passing native, under the sanitizers and
as the Windows build. Real hosts negotiate 1.3 with it; tls_fetch
now prints the version.
The runner's samba package comes up as a service on 445, so the
matrix's own smbd could not bind and every case reached the
distribution server instead: it maps an unknown user to guest (the
unsigned-session failure seen earlier) and has no [share], hence
tree connect refused with STATUS_BAD_NETWORK_NAME. The script now
checks the port before the first case, stops the service and any
other smbd holding it, and fails plainly if it cannot be freed.
Reproduced with a second smbd on 445 and confirmed the guard takes
it over.
tls_bench host port ca get path fetches a file from an openssl
s_server -WWW and reports the receive rate through the client's
record path. Against a local server: AES-128-GCM 560 MB/s and
AES-256-GCM 552 MB/s, the same server limit openssl s_client reaches
(565 MB/s); ChaCha20-Poly1305 238 MB/s, the software cipher, still
well above any home link. No client change came out of the
measurement.
Both clients moved data 64 KiB per request, so a large read was a
round trip per 64 KiB: fine on loopback, the bottleneck on a real
network. The SMB client now does multi-credit I/O on 2.1 and later -
CreditCharge of one credit per 64 KiB, the message id advanced by the
charge, a running credit balance from each reply's grant, requests
clipped to what the server has granted - up to 1 MiB, the negotiated
MaxRead / MaxWrite, or 64 KiB on 2.0.2, which has no multi-credit;
the connection's buffers grow to the negotiated size once the
NEGOTIATE reply has been read. The NFS client asks FSINFO for the
server's rtmax / wtmax after the mount and sizes its transfers and
buffers to that, up to 1 MiB.
On loopback, where round trips are nearly free, SMB reads go from 256
to 343 MB/s and writes from 248 to 320 MB/s against Samba; NFS
against unfs3 from 1.5 to 2.1 GB/s reads and 181 to 414 MB/s writes.
Against a server across a LAN the gain is the sixteen-fold drop in
round trips per megabyte. Both matrices pass native, under ASan
(which caught the NEGOTIATE parser reading its reply after the
buffer had been grown, now done afterwards) and as Windows builds.
A guest or anonymous/null session is returned unsigned even when the
server requires signing, so a server that maps failed or empty
credentials to guest answers the final SESSION_SETUP with SUCCESS and
no signature. The client read that as a bad signature and refused the
connection - intermittent in the field, since whether a login lands on
a real or a guest session depends on the server's mapping. The session
setup now reads SessionFlags before checking the response: on
SMB2_SESSION_FLAG_IS_GUEST or IS_NULL it turns signing off for the
session and skips the check, as MS-SMB2 3.2.5.3.1 requires. The samba
matrix gains a guest-mapping server (map to guest, mandatory signing)
that reproduced the failure exactly before the fix.
The SMB matrix runs under sudo for port 445, and Wine refuses to use a
prefix owned by another user, so the Windows build of the SMB tools
failed before connecting. That run now gets a root-owned prefix, and
its clean runs under sudo too so the root-built tools can be removed.
The shim's fixture registration is a macro, which stops the unused
function warning in suites that register none.
Every request cost 44 ms on loopback. The four-octet NetBIOS length
prefix went out as its own write and the frame as a second one, and
the frame sat in Nagle's algorithm behind the unacknowledged prefix
until the server's delayed ACK - the same stall the TLS handshake
had, here on every SMB request: each directory listing, stat and read
the menu browser makes. The transmit buffer now keeps four spare
octets ahead of the frame and prefix plus frame leave together;
0.02 ms per request on the same loopback, and the samba matrix that
used to run for minutes takes fifteen seconds.
The SMB and NFS test tools print their per-request latency so a stall
of this shape shows in the CI log. NFS was already a single write.
A full handshake took 47 ms against a local server and a resumed one
0.1 ms, which no amount of public-key work explains. The client wrote
ClientKeyExchange, ChangeCipherSpec and Finished as three small
records, and the second of them sat in Nagle's algorithm waiting for
the server's delayed ACK of the first: a 40 ms stall on every full
handshake, on every platform, to every server. The flight is now
corked into one buffer and written once, and the resumed flight's
CCS and Finished the same way; the socket keeps its normal options.
Full handshakes drop to 2.9 ms on the same loopback, RSA and ECDSA
suites alike.
samples/network/tls_retro/tls_bench measures full and resumed
handshake time against a server; it is what found this.
The MinGW builds of the crypto and keychain suites and of the three
network test tools were only ever linked; now they run. Makefile.test
gains win32-wine-check, which builds the crypto and keychain suites
with the libcheck shim as Windows binaries and runs them under Wine -
RtlGenRandom, the MachineGuid machine identity, the Win32 cpu-feature
dispatch onto AES-NI - and the three server matrices take RUNNER=wine
EXE=.exe so the MinGW tools drive the same local openssl, Samba and
unfs3 over Winsock, the four-thread NFS test on the Win32 thread
backend included. All of it passes; the libretro-common CI job runs
it with wine64 installed.
Two things the run needed: the shim now runs checked fixtures around
each test, as libcheck does, since the keychain suite sets its key
file up in one (nothing ran it, so the suite saw an empty path); and
that suite takes its temp dir from TMP / TEMP before falling back to
/tmp, which a Windows build cannot write.
The VFS is entered from several threads at once - content loading,
cloud sync, thumbnail and playlist scans - and the smb:// and nfs://
backends hand those threads contexts from a shared pool whose
connection is not safe to use from two of them, with the pool's own
bookkeeping racing on top. ThreadSanitizer reports 72 data races from
four threads on a two-slot pool, and the reads come back corrupted.
Each backend now takes one lock around every entry point (HAVE_THREADS
only; the lock is published once with a CAS so two first callers
cannot each install their own): calls to one server serialise, which
is no loss on a network-bound path. The TLS client's session cache,
the one thing its connections share, is guarded the same way, and the
ClientHello copies the cached id and ticket out under the lock rather
than reading the slot while it is written.
The unfs3 matrix gains a four-thread test through the nfs:// VFS
backend, clean under ThreadSanitizer with the locks and failing without
them. Without threads every lock compiles to nothing; the three units
build with the tree's --disable-threads defines and carry no rthreads
symbol.
Every case so far had the leaf signed by the trust root directly; real
servers send leaf plus one or two intermediates and the client trusts
only the root. Four cases cover that: an RSA intermediate, a P-384
intermediate under the RSA root, the intermediate itself as the trust
anchor, and a server that omits the intermediate, which must be
refused. The intermediate is handed to s_server with -cert_chain, as
its -cert takes only the first certificate of a file.
The hex loop used snprintf, which the strict C89 lane rejects since
-ansi does not declare it; a nibble table writes the same 64 lowercase
digits and the terminator. Verified against the FIPS 180-4 "abc"
vector; the one caller (cloud sync's S3 signing) already sizes for it.
The NFS menu declarations - the MENU_LABEL rows, the deferred-list
enum, the display-list and action enums, the settings-list enum and
the deferred-push / ok-list cases - had been added inside the
neighbouring HAVE_SMBCLIENT blocks, and the netplay action def file
that carries the NFS settings entry was only consumed under
HAVE_SMBCLIENT. Every shape with NFS and without SMB failed to compile,
which is what the compile matrix's NFS-with-menu gate and the griffin
HAVE_RETRONFS shape build. Each declaration now sits in its own
HAVE_NFSCLIENT block and the def-file regions take either client.
The sanitizer lane builds unfs3 from source with the sanitizer CFLAGS
still exported, and a server built that way does not come up, so the
NFS run could not mount. The script now builds it with the flags
unset, and the test tool frees its context on a failed check so a
real failure is not buried under a leak report.
The legacy-backend job rejected x509_verify_chain in the binary; it is
part of HAVE_CRYPTO, which the keychain needs whichever TLS backend is
selected. The check now asserts on client symbols only.
The TLS, SMB and NFS server matrices were run with the clients built
under ASan, UBSan and LeakSanitizer. UBSan caught one thing: the
SHA-384 PRF path hands the HMAC an empty second seed as a NULL pointer
with length zero, and the SHA-256 and SHA-512 stream updates passed
that straight to memcpy, which is undefined even for zero bytes. Both
updates now return at once on an empty length. Everything else -
14 TLS cases, 8 SMB dialect and sealing cases with the VFS on top,
the NFS run with the VFS on top - came through clean, no leaks.
The libretro-common CI job now runs the three matrices a second time
under the sanitizers, so any memory error a real server can drive is
caught there rather than on a user's machine.
The built-in clients are now the defaults, so nothing in CI builds the
mbedtls backend or the bundled libsmb2 any more - the configuration a
packager pinning a system library, or a platform that keeps one of
them, actually uses. This job configures with every built-in client
off (system mbedtls, bundled libsmb2, no NFS), builds to the link, and
checks the binary carries the legacy backends and none of the built-in
client symbols. Locally, full builds with --disable-networking and
with --disable-retrossl both link, and the flag-sensitive units
(configuration, retroarch, task_content, the VFS, the menu units,
cloud sync) compile under --disable-crypto, --disable-networking and
the clients disabled one by one.
Every reconnect to the buildbot or the achievements server was a full
handshake: certificate chain, two signature verifications and an ECDHE
exchange. The client now keeps a small per-process cache (eight
hosts, six hours, never on disk) of the master secret, suite, server
session id and RFC 5077 ticket from each completed handshake. A
ClientHello offers the cached id and the ticket in the session_ticket
extension (empty when there is none, so the server knows tickets are
taken); when the ServerHello echoes the id, the abbreviated flow runs
- key expansion from the cached secret, the server's ticket, CCS and
Finished, then ours - and skips everything the cache already vouched
for. A NewSessionTicket after a full handshake is stored, so servers
that only do tickets resume as well. ssl_socket_retro_was_resumed()
reports which happened, for tests.
The fetch tool takes a round count and requires every round after the
first to resume; the server matrix runs three rounds with tickets,
three with -no_ticket (session ids), and two on the ECDSA + ChaCha20
suite. Against a real host three rounds went full, resumed, resumed.
A core without VFS support hands the content path to its own fopen(),
where an smb:// or nfs:// URL means nothing. task_content.c now copies
such content into the cache directory first and passes the copy on,
registering it as a temporary so it goes when the content is unloaded
- the same shape the UWP fallback already takes, and the staging rule
the plan carries over from PR #19269. A core that reads through the
VFS still streams the original, which is what keeps large disc images
usable; compressed content was extracted through the VFS earlier and
is local by then. Without a cache directory the load fails with a
message saying to set one. Verified end to end: a need_fullpath core
without VFS support loads nfs:// content through the real binary and
opens the staged copy.
Two things that run found. Both network clients took their settings
in rarch_main() before the configuration file was read: the string
fields point at the live buffers so those followed, but the numeric
ones (timeouts, pool sizes, the NFS ports) were copied as zero.
retroarch_smb_init() and retroarch_nfs_init() run again right after
config_load(). And with no export configured, the nfs:// backend took
only the first path component as the export; it now mounts prefixes
shortest first until the server accepts one and remembers the split
per connection, so nfs://server/srv/nfs/roms/file works for an export
of /srv/nfs/roms.
Settings > Network gains an NFS Network Settings group (server,
export, sub directory, timeout, maximum connections, NFS and MOUNT
ports) and Load Content gains Browse NFS Export, which opens the
configured server as a content list the way Browse SMB Share does.
The group entry is an S_ACTION row in the netplay action definitions
so its label string is generated; the settings rows themselves are
descriptor holdouts in menu_setting.c (free-input strings, plain uint
ranges), and the list, ok, deferred-push, sublabel and title handlers
mirror the SMB ones under HAVE_NFSCLIENT. Every deliverable of the
network stack plan now has its surface in the tree. The compile
matrix gains the NFS + menu shape over the touched units.
nfs_server, nfs_export and nfs_subdir, plus nfs_timeout,
nfs_num_contexts and the nfs_port / nfs_mount_port pair for servers
without a portmapper, under HAVE_NFSCLIENT in the settings struct,
config.def.h and configuration.c, handed to the VFS backend from
retroarch.c beside the SMB settings at startup and released at exit.
An nfs://server/path URL then resolves against the configured export;
nfs://server/export/path keeps working with nothing configured. The
keys round-trip through retroarch.cfg. Menu rows for them follow; the
compile matrix gains configuration.c with NFS and no menu.
vfs/vfs_implementation_nfs.c implements the nfs:// backend contract
carried over from PR #19269 (nfs_init_cfg, retro_vfs_*_nfs, the
nfs_dir_handle and nfs_dirent shapes, probe and last-error) on
net_nfs3.c, with a small pool of connections keyed by export so a
stream keeps the one it opened on. A URL carries everything a mount
needs - nfs://server/export/path - so it works with no settings at
all; the settings, when present, supply a default export and subdir
(nfs://server/path is then relative to them), the timeout, the pool
size and fixed NFS / MOUNT ports for servers without a portmapper.
vfs_implementation.c dispatches the scheme everywhere it dispatches
smb://: open / read / write / seek / tell / flush / close, stat,
opendir / readdir / dirent name, type and stat, closedir, and the
copy path's native-or-not decision.
HAVE_RETRONFS turns it on (configure: yes whenever networking is on;
the console Makefiles do not set it) and defines HAVE_NFSCLIENT for
the surface, replacing the vendored-libnfs approach of #19269. The
unfs3 matrix now also drives the real VFS - filestream and
retro_dirent over nfs:// URLs - and the compile matrix gains the
griffin HAVE_RETRONFS shape. A full RetroArch build links with it on.
Menu and configuration rows for the settings come next.
An NFSv3 client (RFC 1813) over ONC RPC on TCP with AUTH_UNIX, one
translation unit with a small synchronous API (rnfs_*): the portmapper
for the MOUNT and NFS ports unless given, MOUNT v3 for the export's
root handle, then paths resolved component by component with LOOKUP
and GETATTR, READ and WRITE (FILE_SYNC) chunked to 64 KiB, CREATE,
SETATTR for truncation, REMOVE, MKDIR, RENAME, and directory listing
with READDIRPLUS, falling back to READDIR plus a LOOKUP per entry on a
server that answers NOTSUPP (unfs3, some embedded ones). The XDR is
built and unpacked by hand. The socket is non-blocking and every send
and receive is bounded by the context timeout; reply records may
arrive fragmented. This is the transport for the vfs_implementation_nfs
contract carried over from PR #19269; hooking it in comes next.
samples/network/nfs_retro holds a test tool and local_unfs3_test.sh,
which builds unfs3 from its repository when no unfsd is installed and
runs the tool against it on fixed ports: mount, a 200 KB write and
read-back, seeks, stat, mkdir, rename, truncate, listing, unlink, and
a refused export. The libretro-common CI job runs it.
rsmb_enum_shares() runs NetrShareEnum level 1 through the srvsvc named
pipe on an IPC$ connection: a DCERPC bind (srvsvc 3.0 over NDR 2.0)
and one request, both built and unpacked by hand since this is the one
RPC the frontend makes. The VFS backend's share listing for smb://server
is back on the built-in client with the same filtering as before
(disk shares only, hidden, temporary and $-suffixed ones left out), so
the share browser works again.
The connection socket is now non-blocking and every send and receive
goes through the timeout variants bounded by the context timeout: a
request to a server that stops answering fails after that long instead
of holding the caller forever, which the frame loop cannot afford.
The samba matrix enumerates shares at the API level (the share in use
listed as a disk share, IPC$ typed as IPC) and through the VFS with a
hidden share in the config, and its cleanup now also stops the
samba-dcerpcd that smbd spawns per config - a stale one answers the
srvsvc pipe for nobody. The compile matrix gains configuration.c with
SMB and no menu, and with neither.
settings_def_settings_show_smb.h reads DEFAULT_SETTINGS_SHOW_SMB_CLIENT,
which config.def.h defines inside HAVE_MENU, but configuration.c was
including the row under HAVE_SMBCLIENT alone. That was latent while
SMB support needed a libsmb2 the menu-less jobs did not have; with the
built-in client on by default it broke every --disable-menu build.
The five includes are now under HAVE_MENU as well, like the other
settings-visibility rows.
The threaded video's display pacing judges a frame queued behind
another - and drains a period for it - when its push-to-present
latency reaches a period and a half of the display. On a display faster
than the content that period is the wrong one: a 60 fps frame on a
120 Hz panel is due every other vblank, and a render of a few
milliseconds plus the wait to its vblank reached 12.5 ms on its own, so
every longer frame drained a queue that was never there and the core
lost a content period each time. The judgement is now against the
content's period, which the hold already computes, or the display's
where that is longer. On a display at the content's rate nothing
changes.
samples/gfx/threaded_video gains the fast-display lane: a presenter on
a 120 Hz grid with the core at 60 fps and the render alternating 2 and
7 ms must show no drains and keep the core at 60, where the display's
period drained 29 of 120 frames and ran it at 48.
HAVE_RETROSMB puts net_smb2.c behind the VFS SMB backend and the SMB
cloud-sync driver in place of libsmb2. configure turns it on whenever
HAVE_CRYPTO is on and no libsmb2 was asked for, so main builds stop
carrying the bundled library; --enable-smbclient / --enable-libsmb
still select a libsmb2. Both consumers keep their libsmb2-shaped source
through net/net_smb2_compat.h, which presents the calls they make -
context and credential setup, connect, open / read / write / lseek /
ftruncate / close, stat / unlink / mkdir / rename, opendir / readdir /
closedir, echo, the I/O size limits, error text - over the built-in
client with the same return conventions (lseek's -EINVAL, libsmb2's
type and stat structs). net_smb2.c gains unlink, mkdir, rename (SET_INFO
FileRenameInformation) and ftruncate (FileEndOfFileInformation) for
them. A Kerberos request fails at connect so the backend's NTLMSSP
fallback runs; share enumeration (SRVSVC over IPC$) is not carried yet,
so browsing starts at a named share.
The samba matrix now also drives the real vfs_implementation_smb.c
(stat, open, seek, read, tell, opendir / readdir) over each dialect,
and the compile matrix gains the griffin HAVE_RETROSMB shape. A full
RetroArch build links with the client selected and no libsmb2 symbol.
Both scripts run under set -e and end with a trap that kills the
server they started; when it has already gone, kill (or pkill)
returns non-zero, the trap aborts on it and the script exits 1 after
every case has passed - which is how the libretro-common CI job failed
on a green TLS matrix. The kill in each trap is now allowed to fail.
An SMB2/3 client on the cleanroom crypto, one translation unit with a
small synchronous API (rsmb_*): connect to \\server\share over TCP
445, open / read / write / seek / close, stat, opendir / readdir /
closedir, echo. NEGOTIATE offers 2.0.2 through 3.1.1 with the 3.1.1
preauth-integrity (SHA-512 chain) and encryption-capability contexts,
GCM preferred over CCM; SESSION_SETUP carries NTLMv2 inside SPNEGO
(NEGOTIATE, CHALLENGE, AUTHENTICATE with the target info echoed and a
MIC over all three; no NTLMSSP integrity flags, so SPNEGO needs no
mechListMIC leg and no RC4); keys come from the session key through
the SP 800-108 KDF on 3.x with the preauth hash as context on 3.1.1.
Every request after setup is signed - HMAC-SHA256 on 2.x, AES-CMAC on
3.x - and replies are verified; when the server or the share requires
it every message is sealed in a TRANSFORM_HEADER with AES-CCM (3.0,
3.0.2) or AES-GCM (3.1.1). Reads and writes are chunked to the
negotiated limits (64 KiB cap); STATUS_PENDING interim replies are
skipped. Two 70 KiB buffers per context on the heap; the largest
frame is under a kilobyte.
samples/network/smb_retro holds a test tool and local_samba_test.sh,
which runs smbd on the loopback through a matrix: every dialect with
signing mandatory, CCM and GCM sealing required, and a refused logon,
each writing, reading back, seeking, stat-ing and listing a 200 KB
file. The libretro-common CI job installs samba and runs it. Hooking
the client up behind vfs_implementation_smb.h in place of libsmb2 is
the next step.
The pieces the cleanroom SMB2/3 client needs that the stack did not
have: AES-CCM (RFC 3610, any nonce 7-13 and tag 4-16, the 11/16 shape
SMB 3.0 and 3.0.2 seal with) in crypto.c beside GCM and CMAC; and in
kdf.c the SP 800-108 counter-mode KDF over HMAC-SHA256 that SMB 3.x
derives its signing, sealing and application keys with, MD4 and
HMAC-MD5 for NTLMv2, and ntlm_ntowf_v2() computing the NTLMv2 key from
password, user and domain with the UTF-16LE and upper-casing rules of
MS-NLMP. MD4 and HMAC-MD5 exist for that one protocol and say so.
Every one is checked against published vectors in the suite: RFC 3610
packet vector 1 plus the SMB3 CCM shape, RFC 1320 A.5, RFC 2202, a
KBKDF vector cross-checked against a reference implementation, and the
NTOWFv2 vector from MS-NLMP 4.2.4 including the case rules. kdf.c now
uses utils/md5.c, so the test, bench and sample link lists carry it.
The fetch tool's Makefile now adds -lws2_32 -liphlpapi when built with
MinGW (MSYS2 or a cross gcc) and compiles compat/fopen_utf8.c, which
vfs_implementation.c calls on Windows; without them the link failed on
every net_socket.c and net_compat.c symbol. The libretro-common CI job
cross-links the tool with x86_64-w64-mingw32-gcc after running the
local server matrix, so the Windows link line stays checked on Linux.
The built-in client now also offers ECDHE-ECDSA and ECDHE-RSA with
AES-256-GCM, whose PRF and transcript run on SHA-384: kdf.c gains
HMAC-SHA384 (RFC 4231 vector in the suite), tls_prf() and the Finished
computation take the hash from the negotiated suite, and both digests
run over the transcript until the ServerHello picks one. That is the
full set of AEAD suites a TLS 1.2 server can require, so a server
configured for AES-256 only no longer turns us away.
The libretro-common CI job runs samples/network/tls_retro/
local_server_test.sh - the openssl s_server matrix over every suite,
both curves, RSA / P-256 / P-384 certificates and the verify-mode
refusals - and the compile matrix gains the griffin HAVE_RETROSSL
shape. A full RetroArch build links with the client selected and no
mbedtls or BearSSL symbol in the binary; the matrix also passes with
the fetch tool built under ASan and UBSan.
A TLS 1.2 client behind the existing ssl_socket_* API, on the
cleanroom crypto: ECDHE over P-256 or P-384, RSA or ECDSA server
authentication through crypto/x509 against net/cacert.h, AES-128-GCM
or ChaCha20-Poly1305 records, SHA-256 transcript and PRF, SNI,
signature_algorithms and renegotiation_info sent; no resumption,
renegotiation or client certificates. The handshake runs inside
ssl_socket_connect() as the other backends' do; application reads
hand back one decrypted record at a time, and the verify-mode and
logging hooks behave as with mbedtls (required fails closed, optional
logs and continues, disabled logs once). Handshake messages accumulate
in one 64 KiB buffer that is not compacted until the handshake ends,
since the parsed Certificate points into it. x509.c exposes its ECDSA
signature decoder for the ServerKeyExchange.
HAVE_RETROSSL selects it: configure prefers it whenever HAVE_CRYPTO is
on, so main builds stop needing mbedtls or BearSSL, both of which stay
selectable under their own flags. samples/network/tls_retro carries a
fetch tool and local_server_test.sh, which runs an openssl s_server
matrix: every suite, both curves, RSA / P-256 / P-384 certificates
under a CA made on the spot, and the refusals that must happen (wrong
hostname, untrusted CA) with verify required, optional and disabled.
The tool also completes real HTTPS round trips against public hosts.
pk.c's elliptic-curve code now runs over a curve descriptor - prime,
order, b, generator, word count, and the Montgomery constants derived
from them on first use - with P-256 and P-384 as the two instances;
keygen, ECDH and ECDSA verification are exposed for both. P-256 stays
on the TLS fast path, so its word count is spelled as a constant on
that branch and the compiler keeps its unrolled Montgomery clones:
the bench numbers do not move. x509.c accepts P-384 subject keys and
ECDSA signatures over them, so every one of the 155 certificates in
net/cacert.h now loads as a trust anchor; the 15 P-384 roots (DigiCert
G3 and G5, GlobalSign E46, Amazon Root CA 3 and 4, Entrust EC1 and the
others) were being skipped before. Verified against reference P-384
ECDH and ECDSA vectors on both limb widths, the qemu-aarch64 lane, and
the shipped bundle.
crypto/x509.c gives the cleanroom stack the certificate layer its TLS
client needs: a DER reader (definite lengths only, non-minimal
encodings refused), x509_parse() for the fields a client acts on
(issuer, subject and its commonName, validity, SubjectPublicKeyInfo
for RSA and P-256, basicConstraints, keyUsage, subjectAltName,
signature algorithm), x509_verify_signature() over RSA PKCS#1 v1.5 and
ECDSA with SHA-256/384/512, hostname matching on dNSName entries with
a single leftmost-label wildcard (commonName only when there is no
subjectAltName), a trust store loaded from the PEM bundle in
net/cacert.h, and x509_verify_chain(): from the leaf, each step is a
certificate from the server's chain or a trust anchor whose subject
names the current issuer and whose key verifies the signature, with
validity, cA, keyCertSign and pathLenConstraint checked on the way, in
whatever order the server sent the chain. Names compare as DER octet
strings. Everything points into the caller's DER; the parsed chain is
heap-allocated so a frame never carries it.
140 of the 155 bundle certificates load as anchors; the rest carry
P-384 keys, which pk.c does not have yet, and are skipped rather than
refused so the store still loads. test_crypto.c gains a suite built
from a generated three-level chain (RSA root, P-256 intermediate with
pathLen 0, RSA leaf with wildcard SAN): signatures under each
algorithm, hostname rules, expiry, a bad signature, a rogue CA under
the pathLen-0 intermediate, an out-of-order chain with the root
included, and loading the shipped bundle. Built under HAVE_CRYPTO
beside the other crypto units.
crypto.c gains the ARM counterpart of its x86 AES-NI + PCLMULQDQ path:
the block cipher, CTR with four blocks in flight and GHASH on the
ARMv8 crypto extension, gated the way lrc_hash.c gates its SHA path -
used unconditionally when the baseline carries the extension, else
compiled in under target("+crypto") on toolchains whose arm_neon.h
declares the intrinsics regardless of the baseline (Clang 16+, GCC 9+)
and chosen at run time on RETRO_SIMD_AES. The GHASH multiply is the
byte-reversed-operand algorithm the x86 path uses, with the lane
shifts spelled as vext. Apple Silicon, Android, Switch and Linux ARM
builds get hardware AES-GCM; everything else keeps the round-table
cipher and 4-bit GHASH tables.
Nothing on an x86 runner executes this, so Makefile.test gains
crypto-aarch64-check: with an aarch64 cross gcc and qemu-user on PATH
the crypto suite runs under emulation in both the dispatched and the
baseline-crypto shape, built against test/check_shim.h since libcheck
is not installed for the target; without the tools it says so and
passes. The libretro-common CI job installs both and runs it.
The PowerPC job builds against an SDK whose unistd.h does not declare
gethostuuid(), so the Apple machine-identity branch failed to compile
under -Werror=implicit-function-declaration. The function is looked up
with dlsym() instead, with the prototype declared locally: no header
has to know it and a host without the symbol (a 10.4 libSystem) simply
takes the key-file-only key, which is what every non-Apple platform
without a machine identity does already.
Every sensitive value written to retroarch-keychain.cfg is now sealed:
"$kc1$" followed by base64(nonce || ciphertext || tag), ChaCha20-Poly1305
under a master key with the setting's own name as associated data, so a
blob cannot be moved from one key to another. The master key is
HKDF-SHA256(salt = retroarch-keychain.key, a 32-octet random file
created owner-only beside retroarch.cfg, ikm = the machine identity:
MachineGuid on Windows, /etc/machine-id on Linux, gethostuuid() on
macOS 10.5+, nothing elsewhere, where the key file alone is the key).
A keychain file or a copied config directory on its own no longer
carries usable secrets. Plaintext values from before this change load
as they always did and are sealed on the next save; a keychain file
whose key file is missing or belongs to another install keeps its
blobs untouched through load and save, with a warning, rather than
having them overwritten by empty values. Values are sealed and opened
through heap buffers so nothing path-sized lands on the stack. The
layer is HAVE_KEYCHAIN (default on) and needs HAVE_CRYPTO: a build
without the crypto library, such as the small consoles, keeps the file
in the clear as before. No user-facing setting, there is no trade-off
to expose.
libretro-common/file/keychain.c holds the format and key derivation
and is exercised by test/file/test_keychain.c (round trip, per-setting
binding, tamper, plaintext pass-through, key file persistence, loss and
corruption) from Makefile.test; configuration.c opens the entries after
merging the keychain file and seals them in config_save_credentials.
Adds libretro-common/crypto in three translation units, built under
HAVE_CRYPTO (qb default on; the GameCube, Wii, PS2 and other console
Makefiles do not set it, so nothing lands in their RAM). crypto.c:
AES-128/192/256 (forward cipher on round tables, CTR, GCM with 4-bit
GHASH tables, CMAC), ChaCha20, Poly1305, an AEAD front (ChaCha20-
Poly1305, AES-256-GCM), constant-time compare and clear. kdf.c:
HMAC-SHA256 over the streaming SHA-256 in lrc_hash, HKDF, PBKDF2, a
ChaCha20 fast-key-erasure DRBG and crypto_random_bytes() over the
platform entropy source (RtlGenRandom / CryptGenRandom resolved at run
time on Win32 with no header or link dependency, BCrypt on WinRT,
getrandom then /dev/urandom on POSIX, the kernel RNG on 3DS, Vita and
Switch, a clock-mixed fallback elsewhere). pk.c: fixed-width Montgomery
bignum with 64-bit limbs where the compiler has 128-bit products and
32-bit limbs otherwise, RSA PKCS#1 v1.5 verification, and P-256 ECDH,
key generation and ECDSA verification. SHA-384/512 join SHA-256 in
lrc_hash. Pure C89 with no third-party code; on x86 with GCC or Clang an
AES-NI + PCLMULQDQ path is compiled in under the target attribute and
chosen at run time, the way lrc_hash already does for SHA-NI, so HMAC,
HKDF, PBKDF2 and the DRBG inherit that SHA-NI path as well.
This is the base the keychain, TLS, and SMB signing and encryption
build on. test/crypto/test_crypto.c checks each primitive against
published vectors (FIPS 197, RFC 4493, the GCM spec cases including 8-
and 60-octet IVs, RFC 8439, RFC 4231, RFC 5869, PBKDF2, FIPS 180-4) and
the public-key side against reference-generated ones, on both the
vector and scalar AES paths and both limb widths, from Makefile.test in
the libretro-common CI job. samples/crypto/bench compares throughput
with the mbedtls the Linux build links: on an AES-NI host AES-256-GCM
runs 5.8x, ChaCha20-Poly1305 1.2x, HMAC 2.4x, P-256 ECDH 2.5x, ECDSA
verify 1.8x, and RSA verify 1.0-1.1x per fresh key. Every frame stays
under the 2 KiB libretro-common budget; RSA keeps its operands on the
heap. The compile matrix gains two griffin lanes, crypto on and off,
since griffin is where a file-local name can collide with another
unit's and only the MSVC job builds it.
Reject cache entries whose hostname allocation fails, check DNS entry and resolver-thread creation, and roll back failed startup without leaving a pending cache entry. Complete resolver hostname-copy failures as failed lookups. Close newly created sockets when connection-pool allocation fails.
Add deterministic allocation and thread-start failure tests, cleanup checks and recovery coverage for threaded and non-threaded builds, and register both variants in CI.
Connection setup uses addrinfo after releasing the DNS cache lock. An expiration sweep can reclaim it while a slow connection attempt still uses it. Count active connect users under the cache lock, skip their entries during eviction, and release the reference on every exit path without holding the lock across socket or TLS operations.
Add deterministic eviction tests for plain and TLS success/failure, early exits and overlapping users, and register both build variants in task CI.
Copying frontend_userdata leaves the widget keyed to the finished download until extraction publishes progress. A delayed start can let the widget expire without unlinking the extraction task.
Transfer the link under the widget state lock after extraction creation succeeds, reusing the existing owner-change logic to clear terminal state and cancel expiration. Cover pending and displayed widgets, delayed progress, and teardown before the new task runs.
The regular runner executes on the pumping thread but skipped running-task progress pushes as though a separate threaded gather would deliver them. Publish updates for unfinished tasks after their handler, while leaving final delivery to retirement. This restores notifications and progress callbacks when threaded tasks are disabled.
Test intermediate updates and exactly one final update; clarify prefetch's existing callback deduplication.
Rapid title updates queued multiple completion callbacks. After the first transferred msg_new into msg, a later callback could free the displayed text with no replacement. Give title transitions their own animation tag, cancel before replacing them, and cancel that tag at widget teardown.
Exercise queued callbacks and unrelated movement animations in the ownership test, and run that test in graphics CI.
A muted task or failed final title allocation can suppress its last progress push, leaving a widget linked to a freed task. Send an empty completion message when frontend data is attached. The runloop updates widget lifetime state without forwarding empty text to notifications or speech.
Cover muted and missing-title completions, callbacks outside the property lock, and real task retirement with an attached widget.
Classify task_queue_snapshot_test as runnable so the sample workflow executes it instead of rejecting the new target. Link the production task queue in both widget harnesses to resolve task_get_progress_snapshot, and initialize its lifecycle around the tests.
Compare snapshots with the displayed title when no transition is pending. Transferring the pending string on animation completion clears msg_new, so comparing only that pointer restarts identical titles on later progress pushes. Cover repeated snapshots after completion in both widget styles.
A cancelled or failed task may finish with progress between 1 and 99. Clearing its finished flag while drawing makes widget cleanup treat its retained task pointer as live after the task has retired. Keep the completion state authoritative regardless of the displayed progress.
Add a production-renderer regression covering 38 completion, cancellation, error and live-task cases across both widget styles. Include the compatibility string header required by the shared test stubs on Windows.
On macOS, -[MTLDevice newDefaultLibrary] only resolves default.metallib
from inside a .app bundle's resources. For a bare executable (CLI builds,
non-.app installs, a dev build run in place) it returns nil, leaving the
Metal driver with no shader library -- every -newFunctionWithName: then
returns nil.
That nil function is passed straight to
-newComputePipelineStateWithFunction:error: when building the pixel-format
conversion filters, which asserts ("computeFunction must not be nil") and
aborts the process. Depending on timing it can instead surface as a
half-initialized driver or a plain "Cannot open video driver".
Two fixes:
- When newDefaultLibrary returns nil, fall back to loading
default.metallib explicitly via -newLibraryWithURL:error: from the
executable's own directory (where `make install` places it). The normal
.app bundle path is unaffected.
- Guard -newFunctionWithName: in +[Filter newFilterWithFunctionName:...]:
a nil function now returns gracefully with an error log instead of
tripping the NSAssert inside newComputePipelineStateWithFunction.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Reuse snapshot strings for widget creation and title updates, and move the pending string when its animation completes. Check the one remaining creation duplicate before publishing the task association. This removes redundant allocations and avoids NULL dereferences on allocation failure.
Exercise the production message path with allocation-failure injection, ownership accounting, immediate and animated updates, terminal flags, error text and full-queue rollback.
Copy title, error, flags and progress together under the task property lock. Widgets consume the owned copy after that lock is released, preserving the existing task association and widget-state locking. Keep terminal flags available on allocation failure so retiring tasks do not leave live widget references.
Add regression coverage for concurrent string replacement, retained snapshots, allocation failures and an unthreaded runner.
Replace concurrent live-SRAM reads and the uninitialized frame sequence with worker-requested snapshots. Preserve nested netplay scopes, service paused iterations, and keep disk I/O outside the publication lock. Retain failed writes for retry instead of advancing the saved-image baseline.
Add regression coverage for snapshot ownership during slow I/O, nested scopes, paused checks, unchanged data, write failures, compressed routing and shutdown.
The default GameCube/Wii build links the prebuilt in-tree libogc, whose
SYS_Init installs its heap, lock, exit and time handlers into newlib's
__syscalls dispatch table -- a table devkitPPC r38 and later no longer
provide, so the EXTERNAL_LIBOGC=0 path has not linked on any current
toolchain (CI never sees it because it builds EXTERNAL_LIBOGC=1).
Defining bare storage for the symbol would link but misbehave:
libsysbase's default sbrk would place the heap by linker symbols
instead of libogc's MEM1 arena, colliding with SYS_GetArena1Lo
allocations, and malloc would run unlocked under lwp threads.
wii/libogc_syscalls_compat.c therefore defines the table with the slot
layout the prebuilt SYS_Init actually stores to (offsets 0..32,
verified against the archive's relocations) and provides strong
definitions for the per-symbol weak __syscall_* hooks that modern
libsysbase dispatches through -- sbrk, the four lock operations, the
malloc locks, exit and gettod all route to the same libogc handlers
the old toolchain reached, with the old pre-SYS_Init NULL-slot
behaviour preserved. The file is compiled only under
EXTERNAL_LIBOGC=0, external-libogc builds are untouched, and the
default make -f Makefile.ngc links again on the CI toolchain
(devkitPPC r38, GCC 10.2.0).
24 MB of MEM1 makes resident code weight count, and the dol carries all
of it, so features that cannot do their job on this hardware are
compiled out of the GameCube build: libretrodb (defaulted on by
Makefile.common; database scanning is impractical here and the explore
menu goes with it), the webp/dds thumbnail decoders, the mod/s3m/xm
tracker, 7z (LZMA dictionaries alone outsize the console's free RAM, so
7z archives could never extract anyway) and runahead (the second
savestate never fits beside a running core).
Measured with the CI toolchain (devkitPPC r38, GCC 10.2.0) in the CI
configuration, the frontend's static image shrinks from 4.43 MB to
3.95 MB -- about 490 KB of MEM1 returned to content on every core --
and the link map confirms no libretrodb, explore, webp, dds, mod
tracker, 7z or runahead code remains in the image.
A host can send an out-of-range client number in SYNC and later grant
play mode, causing the client to write past fixed per-client arrays.
Reject invalid assigned numbers during SYNC and require MODE/YOU
updates to name the assigned client.
Fixes#19075.
Every selection-pointer change in a playlist bumps the thumbnail
generation and cancels all in-flight icon loads. An entry whose load
had already been dispatched on the previous step is left in the
PENDING state with no task behind it: gfx_thumbnail_handle_upload
refuses to touch a thumbnail through a stale tag, so nothing ever
clears it.
The icon dispatcher in xmb_render then skips that entry on every
subsequent pass because its status is not UNKNOWN, and the recovery
in gfx_thumbnail_request_stream (gfx_thumbnail_reset_if_orphaned) is
never reached. Holding Down for a few seconds therefore leaves a
random subset of the visible entries permanently blank until the
list is repopulated.
Call gfx_thumbnail_reset_if_orphaned before the status early-out so
orphaned entries drop back to UNKNOWN and are re-requested.
webdav.c uploads through task_push_webdav_put_stream, so the three
harnesses that link it need that stub in place of the plain PUT one.
The re-authentication stub pulls the body from the source in send-buffer
sized pieces, so its retry check still sees the bytes that would go on
the wire.
A dupe (video_refresh(NULL)) from a hardware core reached the worker
with no HW slot, so the driver drew whatever it had retained from the
last real frame - the core's texture or ring copy - without the slot
being installed or its fence re-signalled afterwards. The core's wait
on that slot at its next publish covered only the first draw, not the
dupe's repeat of it.
The ring now remembers the slot the worker last claimed a real frame
from. A dupe installs and fences that slot around the driver's NULL
frame exactly as the first draw did, so the slot's fence covers every
read of it. The push side waits, bounded, before handing the core a
slot that a still-queued dupe is about to re-read, since that dupe has
not re-signalled the fence yet.
Frontend-internal; the core-facing interfaces and their versions are
unchanged.
When both ring slots are claimed, the incoming push replaces the newest
frame the worker has not taken yet. A dupe (video_refresh(NULL))
carries no image, so letting it replace a real frame threw away the
only new image the core produced in that period - a 30 fps game sends
real, dupe, real, dupe - and the driver showed the frame before it for
two periods. Without the wrapper a NULL push can never lose a frame.
When the incoming push is a dupe and the slot it would take holds a
real frame, drop the dupe instead: count the miss, queue nothing, and
leave the real frame pending.
A core that has nothing new sends video_refresh(NULL). The driver then
draws the core's texture from the previous frame again through
hw_direct.current, but frame_took_texture is only set inside the
handoff block that a NULL frame skips, so no hw_v2.done signal was
queued behind the dupe's command list.
wait_sync_index promises the core that the frontend is finished with
every use of the texture. With one sync index and one texture in the
non-threaded path, the core's next wait saw the previous value already
complete and started writing the texture while the dupe's draw could
still be reading it on the GPU.
Set frame_took_texture whenever the frame draws the core's texture
directly, so the done fence covers repeats as well. No interface
change: the header already promises this.
Distribution packages install the libretro data sets under
<prefix>/share/libretro/<name>: FreeBSD's games/retroarch-assets and
games/libretro-core-info put assets and info in
/usr/local/share/libretro/, and Debian uses /usr/share/libretro/ for the
same. The FreeBSD port has to sed those paths into the installed
retroarch.cfg because a stock RetroArch never looks there; anyone
building from source on such a box starts with an empty ~/.config
tree and a menu without assets or core info even though both are
installed.
The Unix defaults already probe /usr/{local/,}share/{games/,}retroarch
for assets, so extend that to the share/libretro layout: when no
LIBRETRO_* environment override is set and <prefix>/share/libretro/
{assets,info,shaders,autoconfig} exists, it becomes the default for
assets_directory, libretro_info_path, video_shader_dir and
joypad_autoconfig_dir. Only read-only data sets are affected; cores,
saves and everything RetroArch writes keep the per-user default, so
the online updater still works. Existing configs are untouched (the
defaults only apply to keys the config does not carry).
samples/frontend/packaged_dirs/run.sh runs the built retroarch
headless with a throwaway HOME and config_save_on_exit and checks the
written config: per-user paths when /usr/local/share/libretro is
absent, the packaged paths when it is present (that lane needs root to
create the directory and skips otherwise). It fails on the previous
platform_unix.c and passes with this one. The FreeBSD CI job runs it
after the build.
udev_input.c had a half-written kqueue alternative to epoll behind a
"#if 1 / TODO" that hard-selected epoll everywhere, which is why the
FreeBSD games/retroarch port has to link the udev input driver against
libepoll-shim (UDEV_LIB_DEPENDS/UDEV_CFLAGS/UDEV_LDFLAGS in the port).
Complete it: on FreeBSD, OpenBSD, NetBSD, DragonFly and macOS the evdev
fds are registered with EV_SET(EVFILT_READ, EV_ADD, udata = device)
(the placeholder referenced an undefined LISTENSOCKET), the poll uses a
zero timespec like the epoll_wait(..., 0) it mirrors (the placeholder
referenced an undefined `timeout`), and the event loop tests
filter == EVFILT_READ && !(flags & EV_ERROR) instead of the epoll-only
`events & EPOLLIN`. kqueue is level-triggered by default so the
drain-until-EAGAIN read loop is unchanged, and closing a device fd
removes it from the kqueue just as it does from the epoll set.
NetBSD's kevent.udata is intptr_t rather than void *, hence the
UDEV_KQ_UDATA cast.
Linux still uses epoll; that object is unchanged. The kqueue variant
was compiled with libkqueue's <sys/event.h> to catch type errors.
CI: the FreeBSD job now fails if the linked retroarch binary still
imports any epoll_* symbol.
A stock FreeBSD install could compile RetroArch only after
`pkg install evdev-proto v4l_compat`, the Linux header shims:
input/common/wayland_common.h:23: fatal error: 'linux/input.h' file not found
camera/drivers/video4linux2.c:35: fatal error: 'linux/videodev2.h' file not found
cores/libretro-video-processor/video_processor_v4l2.c:49: fatal error: 'linux/videodev2.h' file not found
Neither package should be needed:
- FreeBSD ships the evdev button/key codes in base as <dev/evdev/input.h>,
and udev_input.c already includes that header on __FreeBSD__. Do the
same in input/common/wayland_common.h and input/drivers_joypad/udev_joypad.c,
the two remaining unconditional <linux/input.h> users on this platform.
Every BTN_/KEY_/EV_/ABS_/FF_/EVIOC symbol they use exists in FreeBSD
14.3's dev/evdev/ headers.
- configure turned V4L2 on because libv4l2.pc was installed (libv4l is a
common FFmpeg dependency), but the sources include the kernel header,
which on FreeBSD only v4l_compat provides. After the libv4l2 check,
verify <linux/videodev2.h> actually compiles and disable V4L2 when it
does not, like the EGL check already does.
CI: the FreeBSD job no longer installs evdev-proto or v4l_compat, enables
Wayland (previously disabled for exactly this reason), and asserts that
configure auto-disables V4L2 with libv4l but no headers. v4l_compat is
then installed and only the two V4L2 objects are recompiled so that
code path stays covered without a second full build.
Reported-by: gouchi
With the threaded presenter holding each push to the display's vblank,
the frame no longer acquires the next drawable right after presenting;
the next frame acquires its own when it first draws. Held from the
present, it sat idle for a period with one drawable on screen and one
queued, so the video thread blocked until the next vblank on every
frame - read by the pacer as a frame queued behind another, draining
every few presents and stalling the core. Without the pacer the
acquire-after-present is unchanged.
video_frame_info_t::threaded_display_pacing is set only while the
wrapper runs, so a driver can rely on it meaning the push is being
timed; its one existing reader is the wrapper itself. threaded_video
gains a lane checking the flag against the setting and the wrapper.
get_last_present_time now answers from each drawable's presented
handler (macOS 10.15.4, iOS/tvOS 10.3, probed at runtime; the selectors
are declared locally so any SDK builds it). presentedTime is on the
host clock and is bridged to cpu_features_get_time_usec() with one
paired read in the handler, since the frontend's CLOCK_MONOTONIC counts
sleep and the host clock does not.
The threaded presenter then lays its vblank grid on the display's own
timestamps, so display pacing holds the push against real vblanks with
vsync on instead of an estimate that drifts off phase.
A core asking for GET_CURRENT_SOFTWARE_FRAMEBUFFER while one frame is
queued and the other is being rendered - the steady state under a
driver whose present blocks until vblank, as Metal's drawable acquire
and a FIFO MoltenVK swapchain do - is now lent a third buffer kept
outside the ring. The push that returns it swaps it by pointer into
whichever slot it picks, and the buffer it displaces becomes the next
spare, so the frame is still published without a copy. The spare is
allocated on the first ask that needs it; ring-slot loans are
unchanged.
threaded_video gains a lane with a driver whose present blocks past a
display period: nearly every ask must be granted, published zero-copy,
and each drawn frame must be newer than the one before.
A driver that lost its GPU device set VIDEO_FLAG_GPU_DEVICE_LOST and
the runloop rebuilt the video driver at once, every time. That is the
right answer to a device lost once - a TDR, a driver update - which
comes back on the first rebuild. A device that is lost again straight
after being rebuilt is a failing driver or GPU, and rebuilding on it
every frame was a busy loop of full driver reinitialisations, each of
which failed the same way.
The first loss is still rebuilt immediately. A loss following it is
retried with a doubling delay, from one second up to sixteen, and
after six in a row the rebuild is abandoned with an on-screen message
until a minute has passed. A loss a minute or more clear of the
previous one is a fresh incident and starts the count again.
Falling back to a different video driver is left as a policy choice:
this only bounds the retry loop.
A WebDAV upload read the complete file into a buffer, and
net_http_connection_set_content() then copied that buffer again, so
two copies of the file sat in memory for the whole transfer. Saves are
small, but a save state is not, and several uploads run at once.
net_http gains a request-body source: net_http_connection_set_content_source()
takes a Content-Length, a read callback and an optional rewind
callback. net_http_send_request() pulls the body from the source in
64 KiB runs and sends each as it arrives, so one run is all that is
ever held; a source that delivers fewer bytes than the advertised
length fails the request as a transport error would, since the header
has already promised them. When a request has to be replayed on a
fresh connection the source is rewound first; a source with no rewind,
or one whose rewind fails, is not replayed.
task_push_webdav_put_stream() wraps that for the task queue, and the
cloud-sync WebDAV driver uploads straight from the open RFILE through
filestream_read, rewinding with filestream_seek. The existing seek to
the start before the upload is kept, as the Digest-challenge retry
still reaches this path with the file at its end.
Checked with a 3 MB body PUT to a local server: the received body is
byte-identical (SHA-256) and the request completes with 201.
getifaddrs and freeifaddrs are declared with C linkage under C++, so a
C++ caller resolves them against compat_ifaddrs.c rather than looking
for mangled names that nothing defines.
On Windows, memzero_pages() decommits the range and commits it again to
get fresh zero pages. If the decommit succeeded and the recommit then
failed, control fell through to the generic memset over a range that
was no longer backed, which faults.
memzero_pages() now returns bool. On Windows, a failed recommit after
a successful decommit returns false and leaves the range alone: it is
decommitted, and the caller must treat the allocation as lost. Every
other path succeeds or falls back to memset and returns true.
There are no callers of this API yet; this settles its contract before
one arrives.
ssl_authmode is written from the settings thread and read on whichever
thread opens a TLS connection. It was a `volatile unsigned`, which
promises nothing about visibility between threads and is a data race
under the C memory model, whatever the hardware happens to do with an
aligned word.
It is now a retro_atomic_int_t, stored with release and loaded with
acquire. retro_atomic.h supplies a C89 fallback on the console
toolchains without <stdatomic.h>, which is why volatile was chosen
originally, so nothing is lost there. Compiles under -std=c89
-pedantic.
The per-frame YUV key buffer is grown with realloc when a frame is
larger than the buffer was sized for. If that realloc fails, the key
pass was skipped but the workers still ran, indexing the keys buffer by
the new frame's width and height - a read past the end of a buffer
sized for the smaller frame.
When the buffer cannot be grown, the workers are now given a plain 2x
pixel-doubling callback for that frame instead of the edge detector,
so the output is filled sensibly and nothing is read out of bounds.
The next frame tries to grow the buffer again.
Checked with the realloc wrapped to fail: every output pixel is the
doubled source under ASan, and the ordinary path runs once the growth
succeeds.
A core that took a framebuffer loan may push a pointer anywhere inside
the lent slot (cropping overscan by offset) or somewhere else entirely
(it rendered into its own buffer and the loan lapses). The check was
`f >= b && f < b + buffer_size` on the raw pointers, and ordering two
pointers into different objects with the relational operators is
undefined in C. It happens to work on every compiler in use, but the
ownership test should not rest on that.
Both addresses are converted to uintptr_t and the frame is accepted
when it is at or past the slot start and its offset is inside the
slot. The row clamp for the zero-copy case is reworked to compute the
rows that fit as (buffer_size - offset) / stride rather than forming
height * stride, which can wrap.
The streamed recording ring records a copy of each frame's back buffer
into one of three readback buffers, inside that frame's command list,
and signals the queue fence behind the list. d3d12_record_free() then
released all three buffers immediately - when recording stops, when the
swapchain changes size or format, and at teardown. D3D12 requires a
resource to stay alive until every command list naming it has
executed, so the ordinary stop and resize paths could free a buffer a
still-queued copy was about to write into.
d3d12_record_free() now moves the ring onto a small retired table,
tagged with the highest fence a capture into it was signalled with,
and d3d12_record_reclaim(), run once per frame, releases a generation
once GetCompletedValue() has passed its tag. A ring nothing was ever
captured into is released outright. Should the table fill under a
storm of resizes, the oldest generation is waited for with the same
bounded wait the queue drain uses. Final teardown, which already
drains the queue, releases the ring and every retired generation
through d3d12_record_teardown().
Nothing waits on the GPU in the stop or resize paths, so the stall
the ring was introduced to remove does not come back.
Two walks can read the same slot as free at the same time; if both then
wrote their fields and one lost the compare-and-swap, the winner's slot
would carry the loser's count and target, its acknowledgement would go
to the wrong walk, and the winner would wait on it for good. The
compare-and-swap now comes first and the fields are written only into a
slot the walk owns. Nothing reads them before the signal that is sent
after them, and a signal from an earlier holder cannot match the new
generation.
DJGPP ships a sys/mman.h with the protection bits and mprotect and no
mmap, so HAVE_MMAN is defined on DOS while MAP_ANONYMOUS is not.
memjit_alloc and memjit_free now take the same gate as every other mmap
user in this file -- HAVE_MMAN, not Emscripten, and MAP_ANONYMOUS
defined -- and on DOS, the consoles and Emscripten memjit_alloc is a
plain NULL, as its documentation says.
A thread can block the signal between the walk's look at its mask and
the signal's delivery, and then hold it queued for as long as it likes;
the walk now closes such a slot at its recheck instead of waiting. For
that to be safe the slot must outlive the signal, which may arrive
whenever the thread unblocks: slots move from the walker's stack to a
static pool of 128, each claimed under a generation, and the queued
signal carries index and generation rather than an address. One
compare-and-swap in the handler both checks that the slot is still the
signal's and closes it, so a delivery that comes after the slot was
closed, released or claimed again is ignored, and the pointer to the
walker's count is only ever followed while the walker still waits on it.
retro_procbarrier_test gains a check with a busy thread flipping its
mask between all-blocked and open every few microseconds while barriers
run; thousands of its signals are delivered after their slot was closed,
and none may be waited on or land anywhere.
A tid is reused as soon as its thread is gone, so a walker rechecking a
target it is still owed cannot take the tid's existence as the thread
still being there: with threads starting and exiting around it, a later
thread under the same number would keep the walk waiting on a signal
the kernel discarded. The walk now records each target's start time
from the stat line it already reads to test for runnable, and at the
recheck closes a slot whose thread is gone or whose start time no longer
matches.
The recheck sleep starts at 250 us and doubles to 10 ms, so a walk that
caught a thread in its last microseconds is over in a fraction of a
millisecond, while one owed by a thread that is merely slow to be
scheduled costs a syscall every 10 ms.
memjit_alloc gives a mapping the caller both writes code into and
executes: PAGE_EXECUTE_READWRITE on Windows, an executable anonymous
mapping elsewhere, and on macOS on Apple Silicon the MAP_JIT kind that
memjit_write_begin/end already serve, under the same TARGET_OS_OSX
test so iOS and tvOS keep the plain mapping. It returns NULL where no
such mapping exists, and memjit_free releases it.
memzero_pages zeroes a page-aligned range as cheaply as the system
guarantees: MADV_DONTNEED where that zero-fills (Linux, Android),
decommit and recommit on Windows, fresh pages mapped over the range
where the advice would only be advice (Darwin, the BSDs), and memset
where nothing else exists. memdecommit's documentation now says the
same about what a non-strict range reads back as.
Android's seccomp filter answers a syscall outside its allowlist with
SIGSYS, which ends the process, and membarrier is on that allowlist
from Android 10 (API 29) on: on Android 8.0 through 9 the probe's query
alone would kill the process at its first asymmetric eventcount. The
API level is read from ro.build.version.sdk and, below 29 or unreadable,
the probe is not made, so those systems resolve to the signal tier,
whose rt_tgsigqueueinfo, futex and tgkill are allowed there.
A thread with the barrier's signal blocked would never acknowledge it,
and a GL or Vulkan driver blocks every signal in each worker it creates,
so a walk that picked one waited on it for good. The walk now reads
SigBlk from /proc/self/task/<tid>/status, in small pieces so the frame
stays within budget, and passes such a thread by. That is sound under
the contract the header now states: a thread that notifies an
asymmetric eventcount must not block the signal, and every thread
started through rthreads inherits an unblocked mask.
retro_procbarrier_test gains a check that runs barriers past a busy
thread with every signal blocked. Under ThreadSanitizer walks run one
at a time, spinning for the turn rather than blocking: the tool keeps
one deferred signal per number per thread and drops a second, and runs
a deferred handler only once the thread is back in instrumented code.
With that, the signal tier completes under the tool.
A thread the Linux signal tier picks can exit before its handler runs,
and a signal queued to an exiting thread is discarded, so nothing would
ever answer for it. Each target now gets a slot of its own on the
walker's stack, carried by the queued signal: the handler closes it, or
the walker does when tgkill(pid, tid, 0) reports ESRCH, and a CAS on the
slot makes sure only one of them counts it. The walker still sleeps on
the futex and is woken by every acknowledgement; the wait times out
after 10 ms only so that a target that died is noticed. Targets are
walked in batches of 32 slots to keep the frame bounded.
retro_procbarrier_test gains a check that runs barriers while two
threads keep starting and joining short-lived threads, which stalled
the signal tier every time; the concurrency check only hit it by chance
on SMP runners. Under ThreadSanitizer the slot handoff is annotated
with __tsan_release/__tsan_acquire, since the syscall that orders it is
invisible to the tool.
oss lists its nodes with the card's name after the path - "/dev/dsp0
(HDA Intel)" - so the user can tell them apart. The menu stores the
entry as shown, and it comes back to init as the device string; init
opened that string as a path and failed. A device picked from the
list could not be used on OSS4 and the BSDs, where SNDCTL_AUDIOINFO
is there to name the card.
audio_device_label_path() takes the path off the front of an entry -
everything before the first " (" - and leaves a bare path alone; oss
and audioio open what it gives.
audioio listed nothing, the one driver left whose kernel offers a
choice of nodes without a list to choose from. It now walks
/dev/audio and /dev/audio0..15 as oss walks /dev/dsp*, with the name
AUDIO_GETDEV gives after the path. That needs no driver instance,
which is what the menu has when init failed and the user needs to
pick another device.
samples/audio/device_label_path compiles both drivers whole with
open() redirected to a recorder and checks each reaches it with the
path of an entry, a bare path as it is, and the default with none;
and pins the helper on its own. It fails on the previous oss and
audioio at "init opens the path of a list entry" and passes now.
Any number of threads can be inside retro_procbarrier() at once, and no
caller waits for another. On Darwin the walk takes each running thread's
state with thread_get_state, which holds the thread, waits in thread_stop
until it is off its CPU and releases it before the call returns, so two
callers stopping each other can never be left parked on each other's
hold; KERN_ABORTED is retried. The Linux signal tier queues a real-time
signal per thread with rt_tgsigqueueinfo carrying the walk's own
acknowledgement counter, which the walker sleeps on with a futex, and
init probes that path on the calling thread and refuses non-real-time
signal numbers. The page flip claims one of four locked pages with a CAS
and flips a page of its own when all are busy.
retro_procbarrier_test gains a concurrency check: four threads issuing
barriers against each other under a watchdog, run for every tier by the
existing CI lanes. The header documents the reentrancy guarantee and
the Linux tier's real-time signal requirement. Pre-10.11 Darwin SDKs
spell MAP_ANONYMOUS as MAP_ANON, which the page flip now accepts.
Every GameCube crash during content load is reported as "black screen,
then the console reboots" (RetroArch #19637 and friends) and nobody
can supply the register dump / stack trace we ask for. Reason: the
video driver is freed and re-initialised around each content load,
gx_free() called VIDEO_SetBlack(true), and libogc's default exception
handler only swaps VI to its own console framebuffer; it never clears
the black flag. So a fault in retro_load_game or driver re-init prints
its registers and stack trace to a display nobody can see, then the
__exception_setreload(8) countdown resets the console.
gx_free() now clears XFB 0 to black, points VI at it and leaves the
display enabled instead of blanking it. Looks identical to the user;
lets the exception screen show through.
For that to be safe VI must keep a valid framebuffer after the driver
instance is gone, so the two XFBs move from per-instance allocations
to process-lifetime statics (g_framebuf), allocated once in
setup_video_mode and never freed. They were already fixed at the
largest mode's size, so nothing changes per mode; it also removes a
1.4 MB memalign/free pair from every content load on a 24 MB box.
If gx_free() runs before any XFB exists (early init failure) it
blanks VI as before.
video_hdr_send_luminance is read solely by the Wayland context driver,
yet the HDR settings list showed it on every platform once HDR was on.
Gate the entry on HAVE_WAYLAND so Windows/macOS/X11-only builds don't
expose a toggle that does nothing there.
The PS4 toolchain defines __FreeBSD__ but Sony's libc has no
pthread_np.h, so the include added for FreeBSD's pthread_set_name_np()
broke the Orbis build. Exclude __ORBIS__/ORBIS, as retro_procbarrier.c
already does for the same reason.
retro_procbarrier.c gated <unistd.h>/<sys/mman.h> on PB_FREEBSD_MEMBARRIER,
but that macro is only defined further down the file (after the
FreeBSD version check), and the x86 page-flip tier uses mmap()/sysconf()
on plain PB_FREEBSD regardless. So on FreeBSD the headers were never
included and the file failed with undeclared PROT_*/MAP_*/_SC_* symbols.
Gate the POSIX headers on PB_FREEBSD like the code that needs them.
rthreads.c calls pthread_set_name_np() on FreeBSD/OpenBSD/DragonFly,
which those systems declare in <pthread_np.h>, not <pthread.h>; include
it so the call is no longer an implicit declaration.
Found by the CI FreeBSD job.
audio_driver_start() and audio_driver_stop() log the driver through
audio_driver_get_ident(), which resolves through the threaded
pipeline's wrapper, so the lines name the device driver in use
("alsa", "wasapi", ...) rather than "audio-thread". The buffer
report in audio_driver_init_internal() uses the same accessor in
place of its own copy of that lookup.
The three HDR settings added to settings_def_video_hdr.h ship a
sublabel string, but menu_cbs_sublabel.c never mapped their labels to
it, so the menu showed nothing under them. Add the missing rows to the
sublabel table next to the other HDR entries.
Reporter (SD2SP2 + Swiss): "lighter" cores like Nestopia or SNES9x crash
on the first content load but work after the console restarts; PrBoom
and ECWolf go to a black screen and the GameCube reboots.
The attached log is not a crash. It is a clean retroarch_main_quit():
two "Saving salamander config" lines, "Saved config", "Unloading core",
performance counters. RetroArch chose to exit.
Root cause, in three parts:
1) retroarch_main_init() reads retroarch-salamander.cfg and sets
RARCH_PATH_CORE from its libretro_path *before*
frontend_gx_process_args() runs, and process_args only fills in
argv[0] when RARCH_PATH_CORE is still empty. The salamander config
names whichever core DOL ran last, so launching a different core
DOL from Swiss leaves RARCH_PATH_CORE pointing at the wrong core.
2) On static builds every content load asks RARCH_CTL_IS_CORE_LOADED
whether the chosen core matches the loaded one. With the stale
path it does not, so the load goes down the "fork a new instance"
path (command_event_cmd_exec -> frontend_driver_set_fork).
3) GameCube has no fork/exec support (set_fork is NULL, exec is
HW_RVL-only). command_event_cmd_exec ignored the false return and
still reported success, so the caller set RARCH_CTL_SET_SHUTDOWN
and the frontend exited; with nothing to return to, libogc resets
the console. That quit rewrote the salamander config with the
correct core, which is why the *second* boot of the same DOL
loads content in-process and works.
Fixes:
* platform_gx.c: on HW_DOL always take the core path from argv[0].
The core linked into the DOL the loader started is the only core
this process can run, so argv[0] is authoritative over the
salamander config. Wii keeps the fill-if-empty behaviour.
* task_content.c: command_event_cmd_exec() now returns false when
the frontend cannot fork, logging which core was requested and
pushing MSG_FAILED_TO_LOAD_CONTENT; the two menu callers that
unconditionally issued CMD_EVENT_QUIT afterwards return to the
menu instead of shutting down. A core mismatch on a no-fork
platform is now an on-screen error, not a reboot.
* task_content.c: "Always reload core on run content" needs fork
support; on frontends without it the running core is all there
is, so load in-process when it matches instead of exiting.
Host syntax-checked task_content.c with HAVE_DYNAMIC/HAVE_DYLIB
undefined and platform_gx.c with -DHW_DOL against the vendored
libogc headers. No devkitPPC on this box.
FreeBSD's PipeWire 1.6.9 headers declare pipewire_init(int *, char ***),
so the camera driver's static pipewire_init() fails to compile there:
"static declaration of 'pipewire_init' follows non-static declaration".
This is what the games/retroarch port's patch-camera_drivers_pipewire.c
worked around.
Rename the driver's five static entry points to pwire_camera_* so they
cannot collide with anything libpipewire exports, matching the pwire_
prefix the audio driver already uses. No behaviour change.
GitHub has no FreeBSD runners, so run the build inside a FreeBSD 14.3 VM
via vmactions/freebsd-vm with the games/retroarch port's default feature
set minus Qt and Wayland. The job first checks that --disable-pipewire
leaves HAVE_PIPEWIRE_STABLE off with libpipewire installed, then builds
with udev (libudev-devd), X11, SDL2, FFmpeg, PipeWire, ALSA, PulseAudio
and OSS forced on so a missing dependency fails loudly.
HAVE_PIPEWIRE_STABLE only qualifies HAVE_PIPEWIRE (it gates the camera
driver), but it was probed independently, so --disable-pipewire on a box
with libpipewire installed still ended up with HAVE_PIPEWIRE_STABLE=1 in
config.mk and config.h. Tie it to HAVE_PIPEWIRE: when PipeWire is off,
record PIPEWIRE_STABLE as off without probing.
While here, make griffin's include of the camera driver require
HAVE_PIPEWIRE_STABLE like Makefile.common and camera_driver.c already do,
so a griffin build against pre-1.0 PipeWire does not try to compile it.
Reported via the FreeBSD ports patch-qb_config.libs.sh ("Fix build if
multimedia/pipewire installed").
linux_common.o is only compiled on Linux, but udev_input.c called its
illuminance-sensor helpers unconditionally, so --enable-udev on FreeBSD
(libudev-devd) failed to link. The sensor reads sysfs IIO nodes, which
only exist on Linux, so keep it Linux-only: the udev driver on FreeBSD
now reports the illuminance sensor as unavailable and everything else
works as before.
This is the last of the source patches the FreeBSD games/retroarch port
still had to carry; the ALSA errno.h and FFmpeg 7.1+ ones were already
fixed in master, and the camera pipewire_init clash went away when the
audio driver was renamed pwire_init.
image_blit_bands() ran a decoder's frame conversion as row bands on a
thread pool and malloc'd a group - the bands and their counters - for
every blit, freed by whichever holder let go last. That is one heap
round trip per decoded video frame, on the frame's path (the audit's
"multi-band image conversion allocates a coordination group per
blit").
Groups now come from a static pool of eight, taken by CAS and given
back when the last holder lets go, so a runner that starts late is as
safe as it was with the heap; the heap is the fallback when every
slot is taken. And the caller, once its bands are done, runs the
runners it posted that are still sitting in the pool's queue
(tpool_help) rather than leaving them: each finds no band left and
lets go. Before, a caller faster than the pool left thousands of no-op
runners queued behind it, each holding a group - which is also why the
pool alone was not enough.
image_blit_bands_test (new, wired into Linux-libretro-common-samples)
wraps malloc through the linker and checks: every row covered once for
1080 and 719 rows at align 2, 500 blits with zero mallocs, four
callers blitting at once, and the no-pool path. The old code mallocs
once per blit and fails the count lane.
06aed75 made mutter_displayconfig.c call video_mode_list_finish but
only added gfx/common/video_mode_select.o under HAVE_WAYLAND, so every
Linux build without Wayland (the plain Linux job and linux-c89) failed
at link time with an undefined reference from
mutter_displayconfig_get_resolution_list.
video_mode_select.c is plain C with no platform dependency, so it goes
in the base object list next to video_driver.o, and the same include
moves next to video_driver.c in griffin.
The D3D recorders read the frame back synchronously, every frame:
re-render it (video_driver_cached_frame), create a fresh staging
texture (D3D11) or readback buffer (D3D12), copy the back buffer into
it, then Map with a blocking read - D3D12 drained the whole queue
first. The frame loop stalled for the GPU on each recorded frame,
which is the audit's "generic recording fallback can synchronously
serialize GPU readback". GL and Vulkan already stream theirs.
Both drivers now do what gl2's PBO ring does:
- While recording, each frame's render queues a copy of the back
buffer into the next slot of a 3-deep ring, right after the render
and before Present. D3D11 uses staging textures and CopyResource;
D3D12 records the copy into the frame's own command list and
signals a fence value behind it.
- read_viewport() takes the slot copied three frames ago: D3D11 maps
with D3D11_MAP_FLAG_DO_NOT_WAIT and declines the frame on
DXGI_ERROR_WAS_STILL_DRAWING; D3D12 checks the slot's fence against
the queue and declines until it has passed. The recorder already
treats a declined frame as "try again next frame". No re-render, no
drain, no per-frame allocation, no wait.
- The ring is (re)created when the swapchain changes size or format,
and freed when recording stops or the driver goes down.
HDR swapchains are decoded on the CPU from the ring copy, as the GPU
tonemap pass renders from the live back buffer and waits on it;
screenshots keep the GPU path.
The recorded frame is three frames behind, as with gl2's ring.
Compile-checked under MinGW; needs a real D3D11/D3D12 recording run.
Mutter, KWin and wlroots each carried their own copy of the same two
decisions - the listed mode a requested size and rate mean, and the
order a resolution list is shown in. gfx/common/video_mode_select.c
holds them once and all three call it, reading their own mode
structures through a callback rather than copying them.
The rules are the ones Mutter already applied and the two Wayland
backends lacked: a rate further off than half a hertz is not the mode
the caller asked for unless it carries that whole-hertz label, a
progressive mode comes before an interlaced one, and the mode in use
wins a tie. samples/gfx/video_mode_select checks each of them.
The two unthreaded parity lanes relied on the unthreaded gather
reversing the running list every pass, which e8c5c81 removed: the
list keeps its order now, the HTTP task always ticks ahead of the
updater and is still allocated when the updater sees it finished.
Built against the old handler under ASan those lanes pass 9/9, so
they guard nothing.
The window that remains is the threaded one: the queue frees the
HTTP task on the main thread right after its callback, and with a
single worker the free can only land between the HTTP task's last
tick and the updater's next one. The test hits it deterministically
instead of by timing. task_set_flags() is wrapped at link time; when
the HTTP task marks itself finished, on the worker, the wrapper
pushes the updater's 'when' two seconds out. The worker only looks
at the front of the running list, so it sleeps; the main thread
checks until nothing but the updater is findable, i.e. the queue has
freed the HTTP task; then the updater ticks holding a freed pointer.
'when' is written and read on the worker thread, so there is no
race, and TSan agrees.
Against the old handler ASan reports heap-use-after-free in
task_get_flags from task_core_updater_get_list_handler on every run;
against the fixed handler the lane passes plain, ASan+UBSan and
TSan. One plain unthreaded lane stays so the direct-read path runs
under the sanitizers too.
core_updater_sub_task_running() reads the sub-task through
task_queue_find() so the threaded queue cannot free it underneath
the worker. On the unthreaded queue that read is a no-op: the
gather detaches the running list before it ticks anything, and
retro_task_regular_find() only walks tasks_running, so from inside
a handler no sub-task is ever findable and the download and
decompress progress never reached the updater's progress bar.
The unthreaded queue has not needed find() since e8c5c81: it runs
handlers in list order (a sub-task pushed from the updater's tick
always sits ahead of it) and only retires finished tasks after the
pass. So when the updater reads a sub-task it has already ticked,
alive, at worst finished but not yet freed; by the next pass its
callback has set the complete flag and the updater stops looking.
Read flags and progress directly there.
The list and download handlers polled their sub-tasks with
task_get_flags()/task_get_progress() and only then checked the
*_complete flag the sub-task's callback sets. The queue frees a task
right after that callback, so a tick that runs after the free still
dereferences the stale pointer. This happens in CORE_UPDATER_LIST_WAIT,
CORE_UPDATER_DOWNLOAD_WAIT_TRANSFER and, on decompress_task,
CORE_UPDATER_DOWNLOAD_WAIT_DECOMPRESS. In the unthreaded queue each
gather reverses the running list, so the updater runs ahead of its
sub-task every other pass. In the threaded queue the window is
narrower but still open.
Moving the complete check first is not enough on its own: the worker
can read 0, the gather can then free the task, and the read that
follows still hits freed memory. The flags and progress are now read
through task_queue_find(), which scans under the queue locks, and a
task only leaves the retiring list under those locks before it is
freed. The complete flag is still checked first, so a reused address
costs one wrong progress value at most. The *_task_finished bools
had no other use and are gone.
get_list_refresh_flags_test gains two unthreaded lanes. They hold the
loopback response until a gather pass of each parity, so one of them
always frees the HTTP task ahead of an updater tick. The ASan build
reports that read as heap-use-after-free on the old handler.
Fixes#19633, reported by milnet01.
The GPU Index entry lists the devices the running driver found rather
than a fixed sixteen: an index past them names no device, and choosing
one leaves the frontend rendering on a GPU it cannot display from, with
nothing on screen to say why.
Where that has already happened - a GPU that answers that it can
present, on a compositor that then cannot import its buffers - Vulkan
makes swapchain after swapchain and throws each away. It now says so
once, naming the GPU the index picked and where to set it back, instead
of looping in silence.
The zero-copy lend only counted when the core pushed exactly the
pointer it was given. A core that renders its whole surface into the
loan and then crops by pointer offset - beetle-psx trims overscan that
way, with its own stride - pushed a pointer past the slot's start,
the loan lapsed, and every frame was copied as if it had never asked.
A pushed frame that lies anywhere inside the lent slot is now
published from there, with the pitch the core pushed; the slot
records the offset and the worker presents from it. Rows past the
slot are clamped as before, now counting from the offset. A push
outside the slot still lapses the loan.
threaded_video_test: the harness core gains a mode that renders the
loan and pushes a cropped window of it (8 pixels in, 4 rows down, the
loan's pitch); the zero-copy lane runs in both modes. The cropped
lane publishes 0 of 59 grants zero-copy on the old wrapper and 39,
the same as the plain lane, on this one.
A GPU index is a position in the list its driver enumerates, which a
driver update, a BIOS change or another card reorders; the saved index
then names a different GPU, and the frontend renders on a device that
may not be able to reach the display at all. Every index - Vulkan,
OpenGL, Direct3D 10, 11 and 12, and Metal - now remembers the device it
was chosen as: where that device has moved in the list its new position
is used, where it is gone the driver's first device is used with a
warning, and index 0 means the driver's own choice as it always did.
The name is kept up to date with what was resolved, and an index set by
hand takes the device at that position and remembers it.
task_queue_init() set task_threaded_enable only when asked for a
threaded queue and never cleared it, so init(false) after a threaded
session left the flag on and the next task_queue_check(), seeing the
flag and an unthreaded queue disagree, re-initialised the threaded
one behind the caller's back. RetroArch never hit it because its
settings path clears the flag through task_queue_unset_threaded()
first; any other user of the queue did.
The flag now follows the argument both ways. task_queue_budget_test
drops its workaround and checks that init(false) after a threaded
session stays unthreaded across a check; that lane fails on the old
queue, and the two unthreaded lanes after it fail with it.
Two audit P1s, both about task_queue_check() taking as much of the
frame as it likes on the thread that drives the frame loop:
- Retirement had no bound. Finished tasks retire (progress push,
callback, cleanup, free) on the checking thread, and a burst of
completions - a thumbnail scan, a bulk download - retired in one
check however long the callbacks took.
- On the unthreaded queue every running task's handler ran once per
check, so the frame cost grew with the running list.
task_queue_set_budget(retire_usec, retire_max, handler_usec) bounds
one check. Retirement stops once retire_max tasks or retire_usec have
gone by, after at least one; what is left stays on the retiring list,
findable, for the next check (the threaded early-out is told there is
work pending). Handlers stop once handler_usec has gone by, after at
least one due handler; the tasks not run go back to the front of the
running list, the ones that ran behind them, so the list rotates
instead of starving its tail. The unthreaded gather now keeps FIFO
order rather than reversing the list each pass, which the rotation
needs. Zero means unbounded and is the default, so nothing changes
for a caller that does not opt in.
RetroArch sets 2 ms or 32 callbacks for retirement and 4 ms of
handler time per check, leaving most of a 16.7 ms frame to the core.
task_queue_wait() loops on the check, so a blocking wait is slowed,
never changed.
task_queue_budget_test covers the count and time retire budgets and
the unbounded case on the threaded queue, and the handler budget,
rotation fairness and the unbounded case on the unthreaded one. Four
of its seven lanes fail against the old queue. Wired into
Linux-libretro-common-samples.
The clock/capacity table was unsigned long[256] (2 KiB on 64-bit)
beside a 512-byte path, a 2704-byte frame that the stack-budget
check refuses for the 8 KiB thread stacks on PSP and GX.
32-bit values are enough (kHz clocks stay below 10^7, capacities at
1024) and the path buffer is 256 bytes; the frame is now about 1.3
KiB. No behaviour change; fast_cores_test still passes.
The two CCDs of an X3D part are one performance class and within a
few percent of one clock, so cpu_features_get_processor_order() put
the frequency CCD first and a core pinning its emulation thread to
"the strongest processor" landed on the die without the V-cache.
Within a class the ranking now prefers the larger last-level cache,
then the higher clock. On Linux that is cache/index3/size per
processor (96M behind the V-cache CCD, 32M behind the other); on
Windows the level-3 RelationCache record whose mask covers the
processor, from the same GetLogicalProcessorInformation call the
core list already comes from. Parts with one L3 size are unaffected.
fast_cores_test: the X3D fixture with and without the cache figures,
and confined to the frequency CCD.
cpu_features_get_processor_order() is what a core uses to pin its
threads to the strongest processors (lrps2 pins EE, VU and GS to the
first three). It ranked by raw cpuinfo_max_freq on Linux and not at
all on Windows, and it ignored the process affinity mask. rthreads'
fast-core classification had just been fixed to use the kernel's own
P/E topology; this brings the ranking onto the same source.
- The per-processor class read moves into cpu_class.h, a private
header included by both features_cpu.c and rthreads.c (neither may
link the other: rthreads.c is built alone by the sample harnesses,
features_cpu.c by the cores). Linux: cpu_core/cpu_atom, then
cpu_capacity within 70%, then max clock within 85%. Windows 10
1607+: CPU Sets EfficiencyClass per processor.
- The ranking sorts by class first, then clock, then SMT sibling,
then id. An E-core can no longer outrank a P-core on clock, and on
Windows a hybrid part is now ranked by class instead of by
enumeration order.
- Processors outside the calling thread's affinity mask are left out
of the order: a pin from a parent process, a container or the user
is a boundary, not a target.
- The Linux gather ranks every processor the kernel publishes rather
than the first N by id, where N was the thread count.
fast_cores_test now also drives the ranking through the fixture tree:
Intel hybrid (P, P siblings, then E; confined to E), ARM three-tier,
x86 favoured cores (lead the order, no class of their own; confined
to four), and an X3D part (clock decides within one class).
RARCH_SOFTFILTER_THREADS_AUTO gave a filter one worker per logical
CPU. That is wrong twice over: the pool competes with the emulation,
video, audio and task threads for the cores those need, and the light
filters get slower as workers are added (EPX at 256x224 measured ~26%
slower on eight workers than on one; LQ2x and 2xSaI lose their
four-worker gain at eight). A filter sits on the frame's critical
path - every worker must finish before the frame presents - so a few
unhindered workers beat many contended ones.
Automatic now resolves as:
- physical cores the process may use (sthread_get_core_topology, or
cpu_features_get_core_amount_physical where there is no topology),
- minus the reserved frame-critical threads: the emulation thread,
plus the video, audio and task threads where each runs on its own
thread (video_driver_init_filter tells the filter the count),
- minus one of headroom for the GPU driver and the OS once above two,
- capped at RARCH_SOFTFILTER_AUTO_MAX (8),
- then capped by the plugin's workload: light filters (epx, lq2x,
crop_borders, darken) run alone, medium ones (2xsai, super2xsai,
supereagle, phosphor2x) stop at four, heavy ones (ntsc, ntsc_crt,
2xbr, blargg_ntsc_snes, and any plugin not in the table) take the
budget.
On a 4-core with everything threaded that is one worker; on a
16-core X3D with everything threaded, 8 for the heavy filters, 4 for
the medium, 1 for the light. An explicit Video Filter Threads value
is passed through untouched, as before.
The arithmetic is rarch_softfilter_auto_budget(); filter_lifecycle
checks it across eleven machine/reserve/plugin combinations and that
the live answer stays within bounds.
"Prefer Performance Cores" (thread_prefer_fast_cores) pinned only the
main and audio threads. The two other frame-critical threads never
asked: the threaded video worker (video_thread_wrapper.c) and the task
queue worker (task_queue.c), which runs the completion handlers the
frame loop retires.
Both now take the same sthread_prefer_fast_cores() placement when the
setting is on:
- video_thread_set_prefer_fast_cores() is set from video_driver.c
before video_init_thread() and applied at the top of
video_thread_loop().
- task_queue_set_prefer_fast_cores() is set in retroarch.c before
task_queue_init() and applied at the top of threaded_worker(). It
is a setter rather than a task_queue_init() argument so the samples
and stubs that call task_queue_init() are untouched. No effect on
GCD, which has no worker thread.
The setting's sublabel now names all four threads.
Counts the physical cores the calling thread may run on, split into
fast and slow by the same classification sthread_prefer_fast_cores()
pins by. SMT siblings count once. On a homogeneous part every core is
fast and slow is 0.
This is what a worker pool needs to size itself: how many cores are
left once the frame-critical threads have theirs, and on a mixed part
whether growing further means spreading onto the slow cluster.
cpu_features_get_core_amount() gives the thread count and knows
nothing about classes.
Sources:
- Linux/Android: the process affinity mask, thread_siblings_list for
the core grouping, and the class from the topology read (cpu_core/
cpu_atom, cpu_capacity, or clocks within the band).
- Windows: CPU Sets, one core per (Group, CoreIndex), fast when its
EfficiencyClass is the highest present. Before 1607,
GetLogicalProcessorInformation core count, all fast.
- Apple: hw.perflevel0/1.physicalcpu on Apple silicon; hw.physicalcpu
otherwise, all fast.
- Elsewhere: returns false and the caller falls back to the thread
count.
The Linux class read is split out of the classifier so both share it.
fast_cores_test gains topology cases: Intel hybrid with SMT, the same
confined to either cluster, ARM three-tier, an X3D 16c/32t part and
an affinity of six threads on it, and an empty sysfs.
sthread_prefer_fast_cores() on Linux took as "fast" exactly the CPUs
whose cpuinfo_max_freq equalled the highest in the system. That is
only right when the fast and slow clusters differ by a lot, as on
classic ARM big.LITTLE. On x86 it goes wrong:
- Favoured cores (Intel Turbo Boost Max 3.0, AMD CPPC preferred core)
report a few percent more than their siblings, so the fast set
shrank to those two cores and the main and audio threads were both
pinned onto them.
- On an X3D part the V-cache CCD boosts lower than the other, so the
emulation thread was pinned to the frequency CCD - the opposite of
what a game wants.
The classifier now takes its answer from, in order:
1. the kernel's own hybrid classification (/sys/devices/cpu_core and
cpu_atom on Intel hybrid parts);
2. cpu_capacity on ARM, keeping every core within 70% of the best so
a three-tier part (prime + big + little) keeps its big cores;
3. cpuinfo_max_freq, within 85% of the highest, so favoured cores and
X3D CCDs are one class while a real slow cluster (20-40% lower)
still splits off.
The classification is split from the affinity read so it can be
driven from a fixture sysfs tree. fast_cores_test covers the three
sources, the affinity-already-inside cases and the two x86 regressions;
against the old logic it fails the favoured-core, X3D and three-tier
ARM cases. Wired into Linux-libretro-common-samples.
* Add SDL3 device sensor support
* Gate sensors to port 0, zero on failure
* sdl3: Sensor comments
* sdl3: simplify sensor input
* sdl3: Fix comments
* sdl3: Fix comment
* SDL3: Log sensors on joypad connect
* SDL3: Restore sensor state on reconnect
* SDL3: Track sensor state in pad struct
* sdl3: Update comment
* input: route sensor disable requests to both the joypad and input drivers
input_driver_set_sensor stops at the first driver that reports the
action as handled. That is right for an enable, where the sensor ends
up held by exactly one driver, but wrong for a disable: joypad drivers
are documented to report the disable of a sensor they never had as a
success, so when the host sensor is actually held open by the input
driver (e.g. the SDL3 input driver's accelerometer/gyro fallback for a
pad with no sensors), the disable never reaches it. The sensor stays
open, keeps being polled, and keeps feeding the core after it asked
for it to stop.
Send disable actions to both drivers. Disabling a sensor a driver does
not hold is already defined as a no-op success, so this is safe for
every existing driver pair.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The "CPU Cores" row printed cpu_features_get_core_amount(), which is
the logical processor count, so a 16-core/32-thread 9950X3D read as
"CPU Cores: 32".
It now prints the physical count from
cpu_features_get_core_amount_physical(), and adds the thread count
when SMT makes it larger: "CPU Cores: 16 (32 threads)". Parts without
SMT keep the plain "CPU Cores: 4" form.
* Add pen support to SDL3 input driver
* sdl3: Report pen as pointer input only
* Skip pen density lookup out of range
* Cache SDL_Window in sdl3_input_t
* sdl3: Update pen comments
* sdl3: infer pen proximity from events
* sdl3: Unneeded comment
* sdl3: map pen barrel buttons to mouse
uwp_main.cpp defines several win32_common.h functions under extern
"C" and does not include the header, so when a prototype changes no
compiler compares the two: the C callers are updated and pass, the
C++ body keeps its old parameters and links, and the mismatch is only
found at run time. That is how 0b2613d crashed Xbox on boot, and how
win32_set_video_mode was left taking four parameters against a
three-parameter prototype.
tools/extern_c_prototype_check.py reads the tree as text: every
function defined inside an extern "C" region of a .cpp or .mm file
is looked up in the .h files, and its parameter list is compared with
each prototype's, type by type with the names stripped. A definition
with no prototype anywhere is not judged. --selftest runs it over
three fixtures: a match, the 0b2613d parameter-count drift, and a
parameter-type drift. The new workflow runs both the self-test and the
tree check on every push and pull request.
Same drift as win32_check_window: the header and every caller
(uwp_egl_ctx, d3d11, d3d12) moved to (data, dims, fullscreen) when
the size became one word, and the UWP definition in uwp_main.cpp,
which never sees the header, kept (data, width, height, fullscreen).
Under the C calling convention it took the packed word as width, the
fullscreen flag as height, and read fullscreen from a register no
caller set - so the launch view size and windowing mode UWP asks for
were nonsense. Found by tools/extern_c_prototype_check.py.
The body unpacks the word with VIDEO_SCALE_W/H; nothing else changes.
dc44cc7b0a changed win32_check_window to take one unsigned* for the
size, and every caller passes one. uwp_main.cpp carries the UWP
definition of that function, and it does not include win32_common.h,
so the compiler never saw the prototype change: it kept taking width
and height as two pointers. The callers pass four arguments, the
body reads a fifth that was never passed and writes through it, and
on Xbox it writes both axes on every call - RetroArch crashes at the
first alive() check on boot. Bisected by Nejsplitter to 0b2613d, the
first build after that change to get through the MSVC lane.
Both paths now pack width and height with VIDEO_SCALE_PACK and hand
the word back, matching the prototype and win32_common.c.
CPU video filters always ran on one worker per detected core
(RARCH_SOFTFILTER_THREADS_AUTO). That suits the heavy filters (NTSC,
NTSC CRT, 2xBR) but over-parallelises the light ones: at 256x224 EPX
is ~26% slower on eight workers than on one, and LQ2x/2xSaI lose
their four-worker gain at eight.
video_filter_threads (Settings > Video > Output, under Video Filter;
advanced) picks the count: 0 is Automatic, the previous behaviour and
the default; 1-16 forces that many workers. Changing it while a filter
is running rebuilds the filter on the new count at once; with no filter
loaded the value applies at the next load.
video_driver_init_filter() now frees a live filter before building the
new one. Re-init while content was running (the new setting, and also
a video filter path change) previously overwrote state_filter and
state_buffer, leaking the old filter, its worker pool and its output
buffer.
tools/settings_table.reference is regenerated; it had drifted on
master (HDR, custom viewport and window size rows) independently of
this change.
cloud_sync_http_body_is_framed lived in network/cloud_sync_driver.c,
which the webdav_options, webdav_reauth and gdrive_backup samples do
not link, so all three failed with an undefined reference once
webdav.c and google_drive.c started calling it (#19631).
The check restates the two tests net_http itself uses to pick the body
framing, so it belongs beside them: it is now net_http_body_is_framed
in libretro-common/net/net_http.c, which every one of those samples
already links. cloud_sync_driver.c/h lose the copy and its includes.
webdav_reauth's stale-nonce test answered the retried GET with a 200
that carried no headers, which the read callback now correctly refuses
as unframed; the test's response gets a Content-Length like a real one.
All three samples build and pass under ASan/UBSan again.
The header declares ssize_t-returning functions but only pulled in
stdlib.h, boolean.h and retro_common_api.h, so it compiled only when
the includer had already brought net_compat.h in. network/tls_log.c
includes it first, and the 3DS and Vita builds fail there with
"unknown type name ssize_t". Include net_compat.h, as net_socket.h
does for the same declarations.
Under the default Required policy a device with an unset or badly
wrong clock rejects every server's certificate as expired or not yet
valid. The mbedTLS verify text alone reads like a server-side problem,
so log an explicit hint that the device's date and time are the likely
cause, and name the setting that restores connectivity meanwhile.
The mbedTLS backend set MBEDTLS_SSL_VERIFY_OPTIONAL on every connection
and ignored the verification result. A handshake therefore
succeeded against any certificate, including self-signed and expired
ones, so anyone able to intercept traffic could impersonate the
servers used by the online updater, RetroAchievements and cloud sync.
Add a tls_verify_mode setting (Required, Optional, Disabled) with
Required as the default:
- Required: the handshake fails on a bad certificate, the connection is
refused and the reason is logged.
- Optional: the previous behaviour, now with a logged warning.
- Disabled: no verification, with a warning on every connection.
The mode is applied at startup and whenever the setting changes, and
takes effect on the next connection. Logging goes through two hooks
that the backend defines as weak no-ops, so libretro-common still
builds on its own; network/tls_log.c routes them into the RetroArch
log. Griffin builds skip the weak definitions, since tls_log.c lands in
the same translation unit. BearSSL already verifies and fails closed,
so it only gains a no-op setter.
Tested on Linux with the bundled mbedTLS: full build, and the touched
files also compile with C89_BUILD=1. A small program linked against
the built objects connected successfully to buildbot.libretro.com,
thumbnails.libretro.com, retroachievements.org and the Google APIs
hosts in Required mode, and refused self-signed.badssl.com and
expired.badssl.com with the reason logged. Optional and Disabled
connected to the self-signed host and logged their warnings.
Co-authored-by: Claude Opus 5.5 <[email protected]>
LOAD_SAVESTATE carries a peer-controlled output size. The client used it
to grow and zero every rollback buffer before checking decompression,
letting a remote host request excessive memory with a small packet.
Bound output size to the fixed per-session zbuffer limit before any
rollback buffer is resized.
This reverts commit e4ed924601.
The rationale was stale: credentials are no longer written to
retroarch.cfg. config_save_file strips every sensitive key into
retroarch-keychain.cfg, and config_save_credentials already calls
path_set_private() on that file after writing it. The chmod on the
main config protected nothing.
The main config stores plaintext credentials (WebDAV, Google Drive and
streaming tokens, netplay passwords). With the usual umask a freshly
written file is world-readable. When config_save_file creates the file
it now chmods it to 0600 on POSIX platforms.
A config that already exists is left with the mode it has: users share
configs across accounts and read them from scripts, so changing the
mode of an existing file on every save would break those setups.
rcheevos_filter_url_param removes a parameter by copying the tail of
the URL over it with strcpy. Source and destination are overlapping
ranges of the same buffer, which strcpy does not define; memmove does.
rcheevos_patch_address returns a pointer with no length, so
READ_CORE_RAM served whatever byte count the request asked for and
WRITE_CORE_RAM wrote every byte in the payload, both running past the
end of the core's memory region. READ_CORE_RAM also computed its reply
buffer as `40 + nbytes * 3` in unsigned int: a request for
~1431655766 bytes wrapped that to a few bytes and the reply loop then
wrote 4 GiB through it.
Add rcheevos_patch_address_avail, a thin wrapper over
rc_libretro_memory_find_avail that also reports how many bytes remain
in the region. READ_CORE_RAM clamps nbytes to that, WRITE_CORE_RAM
stops there and logs that the rest of the payload was dropped. Both
READ_CORE_RAM and READ_CORE_MEMORY additionally refuse requests above
16 KiB: the reply is 3 characters per byte in one UDP datagram, so
larger requests could never be delivered anyway, and the cap keeps the
size arithmetic well clear of the wrap. READ_CORE_MEMORY already
clamped to the descriptor length.
Also size the " -1" reply against the allocation instead of
sizeof(reply), which was the size of the pointer.
The network command interface has always bound to every interface, so
with network_cmd_enable on, any host that can reach the port can send
LOAD_CORE, WRITE_CORE_RAM or WRITE_CORE_MEMORY. Many setups rely on
exactly that (phone remotes, cabinet front-ends, home automation), so
the default cannot change without breaking them.
Add network_cmd_bind_address. Empty, the default, keeps the historical
all-interfaces bind; "127.0.0.1" restricts the interface to the local
machine. command_network_new takes the address and passes it to
socket_init, which only sets AI_PASSIVE for a NULL server, so an
explicit address binds to that interface alone. The startup log now
names what the socket is bound to.
Three places trust data that comes from outside RetroArch:
- replay_set_serialized_data reads the embedded replay length from the
save state. A negative value, or one shorter than the replay header,
was passed on to the seek and write that restore the replay. Refuse
such states.
- natt_parse_desc_node walks the IGD description XML a router serves.
On a node without children it recursed with a NULL child and then
read child->next, so a leaf node or an empty root crashed it. It
also recursed only into childless nodes, so a <service> nested in
<root>/<device>/<serviceList>, the normal UPnP shape, was never
found. It now always recurses into children and checks the
serviceType and controlURL text before using it.
- task_cloud_sync_fetch_server_file took the local path from a
server-supplied manifest key as strchr(key, '/') + 1. A key without
'/' read from address 1, and a key containing ".." or an absolute
path was joined onto the sync directory, so a hostile server could
write outside it. Such keys are now refused and counted as a
failure.
The description walk moves to network/natt_desc.c and the key check
to tasks/task_cloudsync_path.c, so the new sample tests
(samples/tasks/natt_desc, and cloudsync_path_safety_test beside the
existing cloudsync test) compile the shipped code directly. Both are
added to Linux-samples-tasks.yml.
Tested on Linux: full build; touched files also compile with
C89_BUILD=1 without warnings. The workflow's commands for both new
steps pass, and the existing cloudsync test still passes under
SANITIZER=address. With the old parser or the old key handling
substituted, each test segfaults.
Co-authored-by: Claude Opus 5.5 <[email protected]>
net_http fails a transfer that ends before its Content-Length or its
final chunk. A response with neither is delimited by the connection
closing, and net_http documents that it cannot tell a complete body
from one cut off by a dropped connection. The WebDAV and Google Drive
read callbacks accepted such a body as a successful download and wrote
it over the local save. A connection that drops mid-download could
therefore replace a save with a truncated copy, which the next sync
then uploads.
The read callbacks now treat a 2xx response without Content-Length or
"Transfer-Encoding: chunked" as a failed download and leave the local
file alone. The check is shared as cloud_sync_http_body_is_framed and
matches the headers net_http itself uses to pick the body framing.
Also NULL-check allocations that were used unchecked: the WebDAV
digest-auth chain (HA1, HA2, response, header), webdav_delete's
callback state, and the Google Drive context in
gdrive_begin_with_token. Each now fails the operation instead of
writing through NULL.
Tested on Linux: full build; touched files also compile with
C89_BUILD=1 without warnings. A small program linked against the
built cloud_sync_driver.o returned true for Content-Length and for
chunked (any case), false for no framing, "gzip, chunked" and a NULL
list. Not tested against a live WebDAV or Google Drive server.
Co-authored-by: Claude Opus 5.5 <[email protected]>
wl_data_device_handle_drop set offer_data->dropped before checking
whether offer_data was NULL, so the check never protected anything.
Move the assignment after the check.
uint32s_index_insert_exact declared a shadowing 'uint32_t idx = 0' in
the RHMAP_HAS branch, so every object whose hash collided with an
existing bucket was recorded in that bucket as index 0 instead of its
own index. Lookups through the bucket then compared against the
all-zeros pattern rather than the new object, and pop/remove could
never find the entry to take it back out. Use the caller's idx.
Small, independent fixes for crashes and leaks on reachable paths:
- wayland: gfx_ctx_wl_get_video_size_common dereferenced a NULL output
when no window output was set and all_outputs was empty. The DnD
drop handler leaked the received buffer when fmemopen failed, and
always leaked the line getline allocated.
- drm: drm_plane_setup logged a NULL drmModeGetPlaneResources result
and then dereferenced it. init_drm read `connector` uninitialised
when the card reported no connectors.
- uint32s_index (replay statestream): uint32s_bucket_free left the
freed pointer and length in the bucket. RHMAP_CLEAR keeps value
slots, so a later RHMAP_PTR on the same slot could see them and free
the vector again. uint32s_index_pop now also refuses an empty index
and checks RHMAP_HAS before RHMAP_PTR, which would otherwise create
a bucket for a missing hash.
- ozone: ozone_context_reset_horizontal_list allocated a node for an
entry without one but never stored it in the list, so it leaked.
- discord: discord_json_next_strdup leaked the previous string when a
key repeated.
- core_backup: core_backup_add_entry leaked an empty filename.
- audioworklet, caca, sixel, vga: the init functions leaked their
struct when the second allocation or the font renderer failed.
Tested on Linux: full build. The touched Linux files show no new
warnings; the C89 build of the Wayland and DRM files fails on this
host in system headers and existing lines, with no diagnostic on a
changed line. drm_gfx.c was compiled although HAVE_PLAIN_DRM is off.
samples/tasks/core_backup passes. The audioworklet, caca, sixel and
vga drivers are not buildable here and were not compiled.
Co-authored-by: Claude Opus 5.5 <[email protected]>
Screen resolution on PS3 lived in the ps3 context driver: a list of
resolution ids it probed for at init (leaked on every reinit - the
malloc'd list was never freed), stepped through with left/right on
the menu entry and applied with OK by reinitialising the driver. On
PSL1GHT that did nothing at all: the RSX driver configures the video
output itself, from the mode the system menu has it in, and never
looked at the chosen id.
dispserv_ps3 is now the display server for __PS3__. It lists the
modes the connected display takes (480, 576, 960x1080, 720, the
1080-line scalers, 1080), marks the one in use, and switches between
them. The menu entry now opens the same resolution dropdown the
desktop platforms use, and Start resets to the system menu's mode as
it does elsewhere. A choice is stored in current_resolution_id - the
video output resolution id, as before, so existing configs keep
their modes - and the video driver is reinitialised to configure the
output in it; picking the mode already in use reinitialises nothing.
A refresh-rate-only set_resolution (the autoswitch) is refused.
ps3_display_server_resolution() is what configures the output: the
chosen mode when the display takes it, otherwise the system menu's.
The RSX driver's rsx_init_screen now uses it in place of the system
mode, which makes the setting work on PSL1GHT; the PSGL context uses
it for its device size in place of the unchecked stored id, so a
choice the current display cannot show (a config moved to another
TV) falls back instead of failing.
The mode table and those rules are in dispserv_ps3_modes.c, which
takes the display's answers through a callback and so has no SDK
dependency. The context driver loses its list and its output
size/prev/next callbacks, and global_t loses the list, count, check,
idx and initial fields nothing reads any more.
Test: samples/gfx/dispserv_ps3 links the real server and mode rules
against stubbed PSL1GHT video output queries (declared in the tree's
PS3 stub <sysutil/video.h>, now carrying PSL1GHT's layouts) for
displays taking different sets of modes with the system menu in each:
the list is exactly what the display takes with the mode in use
marked; each entry, through the label the menu prints and the parse
the menu does, selects its mode, stores its id, reinitialises once
(never for the mode in use) and reports its rate and size; the mode
the driver configures follows the choice and falls back to the system
mode for one the display lacks or an unknown id; refresh-only and
unavailable modes are refused without touching anything. Wired into
Linux-samples-gfx (ASan + UBSan); compile-matrix gains a PSL1GHT
syntax lane for the server.
Screen resolution on GameCube/Wii was a private mechanism of the gx
video driver: a fixed table in gx_gfx.c, stepped through with
left/right on the menu entry and applied with OK, with no idea of
what each mode would actually run at. Issue #19380 is a user on a
240p CRT working out modes and aspect by trial and error.
dispserv_gx is now the display server for GEKKO. It lists the modes
the console's TV standard can show, each with the refresh rate and
scan type it really runs at (640x240 at 59.826 Hz, 640x480 at
59.94 Hz interlaced, 50.0801/50 Hz on PAL, progressive on a
component cable with 480p on), and switches between them. PAL's own
preferred mode (640x574) is in no fixed entry and is listed as the
default. The menu entry now opens the same resolution dropdown the
desktop platforms use. A choice is stored in current_resolution_id
- the table keeps its order, so existing configs keep their modes -
and handed to the driver, which still programs the VI since it owns
the framebuffers and the menu surface a switch has to redo. A
refresh-rate-only set_resolution (the autoswitch) is refused: the VI
cannot change rate without changing mode.
The mode table and the rules for what a mode becomes (double strike
at or below half the standard's lines, interlaced or progressive
above, clamping, field rate) move to dispserv_gx_modes.c, which has
no libogc dependency. gx_set_video_mode and the server's list both
use it, and the TV standard detection is shared, so the list cannot
promise a mode the driver programs differently.
Fixes a bug in the resolution dropdown on every platform along the
way: action_cb_push_dropdown_item_resolution parsed the label with
strtoul() inside VIDEO_SCALE_PUT_W/H, whose VIDEO_SCALE_PACK
evaluates its arguments twice. The height's second strtoul() started
where the first had left 'end', returned 0, and every mode picked
from the list went out as Wx0. Each axis is now parsed into a local
first.
Test: samples/gfx/dispserv_gx links the real server and mode rules
against stubbed libogc queries and runs NTSC, NTSC 480p, PAL 50,
EuRGB60 and MPAL: every listed mode fits the standard, exactly one
is current, and each entry - through the label the menu prints and
the parse the menu does - selects that mode, stores its id and
reports its listed rate; refresh-only and unknown modes are refused
without touching anything; config ids keep their meaning. It failed
on every selection against the old dropdown parse. Wired into
Linux-samples-gfx (ASan + UBSan) with a grep guard against the
double-evaluation shape; compile-matrix gains Wii and GameCube
syntax lanes for the server.
The core updater list's cancel handler flushed the menu stack back to
'Online Updater'. When the list was opened from 'Load Core' there is no
'Online Updater' entry on the stack, so the flush popped all the way to
the Main Menu.
The list only ever occupies one stack level, so use the default pop
instead - it already handles removing core updater search terms and
returns to whichever menu the list was opened from.
Fixes#19625
midi_driver_open() called midi_driver_free() when the device failed to
open, which also dropped the device lists and cleared
rarch_midi_drv_inited. After that the MIDI menu lists came back empty
and the next GET_MIDI_INTERFACE could not retry until a driver reinit.
Split the device teardown out into midi_driver_close() and use that on
the failure path: the driver stays selected, the enumerated lists stay
valid for the menu, and the next request from the core retries the
open. midi_driver_free() still does the full teardown for driver_uninit.
Also make midi_driver_set_input()/set_output() a quiet no-op while no
device is open: with lazy open the menu can change the device before
any core has asked for MIDI, and the new value is picked up when the
device is opened, so this is not the error the DEBUG log claimed.
drivers_init enumerated and opened the MIDI devices on every start,
whether or not a core would ever use them. With the default WinMM
driver and "Microsoft GS Wavetable Synth" output that means a MIDI
stream for every Windows user, and on Windows 11 the first WinMM MIDI
call after boot starts the Windows MIDI Service and blocks until it is
up. That is the slow first launch in #19247.
Keep driver selection in drivers_init, and enumerate and open the
devices when a core calls GET_MIDI_INTERFACE. The request survives
driver reinits (SET_SYSTEM_AV_INFO, audio latency changes) and is
dropped when the core is unloaded. The menu device lists are filled
on demand without opening a device.
Frame Delay no longer relies on the MIDI stream holding the timer
period down since 5c30a403be, and the WASAPI workaround for it was
removed in 8d4bccac55.
Fixes#19247
Vita nightlies since the audio rework crackle in PCSX, UAE4ARM, blueMSX,
VICE, Fuse and more (#19624). The worker released a window to the writer
as soon as the output call that handed it over returned, on the reading
that the call returns once the device has taken the window. It does not.
sceAudioOutOutput / sceAudioSRCOutputBlocking wait for the *previous*
window to finish, queue this one and return with it still in flight: the
SDKs document passing NULL as the way to wait for that last window, and
the PSP driver, traced on hardware for PPSSPP (docs/sceAudio.md), arms a
descriptor pointing at the caller's memory, waits for one descriptor to
retire and returns - nothing is copied, the DMA reads the window out of
the ring over the next period. A write landing in it in that period is
heard.
The old driver had the same early release, one call earlier still, but
an 8192-frame ring that rate control kept about 85 ms from the edge, so
the writer never reached the window in flight. The reworked ring is the
latency setting - five periods at 64 ms - and the threaded pipeline
keeps it full, so the writer sits on the freshly released period every
period and lands in audio being played.
The worker now keeps its own cursor for the next window to hand over, and
publishes read_pos one window behind it: after each call returns, the
window that call handed over is the device's and everything before it is
the writer's. The window in flight is therefore held inside the ring, so
the ring is sized to the setting directly with no device period beside
it, buffer_size() is the ring, and the floor goes to six periods - five
of room, which still holds a 30 Hz video frame of audio beside the
window in flight. Same fix on all three drivers; they share the code and
the semantics.
samples/audio/psp_ring's mock device now reads a window across the period
after the call that queued it, the way the hardware does, rather than
inside the call. On the previous drivers every lane fails with a break in
the middle of a window - frame 384 of 512, the writer's overwrite - and
the old unified psp_audio.c fails the same way under a flat-out writer.
The harness also gains the check target the unclaimed-samples job looks
for, since without one it was never being run.
Follow-up to 368b842. Two more ways the replay state machine could
get stuck, both found by the test added here, plus the test.
1. A recording halted before its first frame produced a file with a
header and a checkpoint and nothing after it. The player rejected
it: bsv_movie_reset_playback() reads the first frame's events at
init, hit EOF, raised BSV_FLAG_MOVIE_END and failed - the same path
the truncated-file deadlock came from, now landing as "failed to
load movie file" for a file the recorder itself wrote. Treat it as
what it is, a replay of zero frames: leave the checkpoint pending
at init and let the first frame restore it and hit EOF, which ends
playback through the normal end-of-replay path.
2. movie_stop_record() only looked at the promoted handle. The record
task installs the handle as next_handle and the run loop promotes
it on the next frame, so halting from the menu before a frame ran
(core paused) found no handle, returned false and left the
RECORDING flag set and the pending handle allocated: the session
was stuck "recording" and every later Record/Play Replay refused
until restart. Finish a pending recording like any other; it is a
recording of zero frames.
samples/tasks/bsv_replay_init: links the shipping tasks/task_movie.c
and input/bsv/bsvmovie.c against a stub core through the real task
queue, intfstream and VFS, in the iOS configuration (HAVE_RZSTD on,
HAVE_STATESTREAM off) the issue was filed against, and mirrors the
run loop's two-line replay protocol. Lanes: record/play round trip
(raw and zstd, core restored to the checkpoint), truncated first
checkpoint (start fails, user told, no MOVIE_END left, nothing pauses,
Close Content clears), zero-frame replay (plays and ends on frame one),
stray MOVIE_END (movie_stop clears it). Against b40db22 the truncated
lane reproduces the report exactly - paused after every frame, carried
across Close Content - and the empty lane fails on both bugs above.
Clean under ASan+UBSan with leak detection. Wired into
Linux-samples-tasks.yml next to bsv_replay_bounds_test.
Not changed: retro_vfs_file_truncate_impl calls ftruncate() with no
fflush() first. The recorder truncates every frame, so this was a
suspect for the reporter's short file, but the preceding
fseek(0, SEEK_CUR) flushes on glibc and BSD libc alike and the round
trip lane cannot make it lose a byte. The origin of the short file on
iOS is still unconfirmed; the new OSD message and the [Replay] log
lines will name the read that came up short.
Signed-off-by: LibretroAdmin <[email protected]>
bsv_movie_init_playback() reads the first checkpoint and the first
frame's events before the handle is installed. Every short-read branch
in bsv_movie_load_checkpoint() and bsv_movie_read_next_events() (the
latter called with end_movie=true from bsv_movie_reset_playback()) sets
BSV_FLAG_MOVIE_END on the global input state. When init then fails, the
handle is freed and never enqueued, so BSV_FLAG_MOVIE_PLAYBACK is never
set alongside MOVIE_END.
The run loop checks MOVIE_END after every frame and issues
movie_stop() + CMD_EVENT_PAUSE. movie_stop() only clears MOVIE_END via
movie_stop_playback()/movie_stop_record(), which it never reaches
without the PLAYBACK/RECORDING flag, so the flag stays set forever:
every pause toggle runs exactly one frame and re-pauses, input looks
dead, and since Close Content goes through the same movie_stop() the
flag carries into the next core and content. Only a process restart
clears it. A truncated replay is one trigger; a replay halted before
any frame was recorded (EOF at the first frame) is another.
Fix:
- bsv_movie_init_internal(): clear MOVIE_END on the error path. A
handle that failed to initialise leaves no flag behind.
- movie_stop(): also clear MOVIE_END when no movie is active, so a
stray flag never survives Close Content.
- bsv_movie_start_playback(): push the "failed to load movie file"
OSD message on failure. It runs from the task callback after
CMD_EVENT_PLAY_REPLAY already returned success, so the command's own
failure message never fired for an unreadable file.
Signed-off-by: LibretroAdmin <[email protected]>
configure generates the Wayland protocol sources, so a tree configured
before a protocol was added has no way to make the new object and stops
on a missing file. config.mk depends on the generator script too, so
make asks for the reconfigure that produces it, as it already does for
configure and qb.
Send Frame Luminance, off by default, describes the GL drivers' HDR
frame to a Wayland compositor with its own luminances instead of as
Windows-scRGB: the same extended-linear sRGB frame at BT.709
primaries, with the reference white at the Peak Brightness setting's
paper white and the peak at what the display or that setting gives, so
the compositor maps it from what the frame holds rather than from an
assumption about scRGB content. The parametric description is used
only where the compositor advertises the parametric and set_luminances
features with the transfer function and primaries it needs; anywhere
else, and with the setting off, the surface carries the scRGB
description exactly as before.
samples/gfx/wayland_color_management gains the parametric path against
its compositor, which rejects a creator made without the feature, a
property set twice and a description created with one unset, and the
fallback to scRGB where the features are missing.
The Metal driver sets its layer's EDR metadata, as the D3D and Vulkan
drivers set theirs: the peak is the display's where Use Display Peak
supplies one, else the user's Peak Brightness, with the optical output
scale the layer's own colour space calls for - 10,000 nits for PQ, 80
for extended-linear scRGB. A display that reads it maps the picture to
what the frame actually contains rather than to its own guess; one that
ignores it behaves as before, as does an OS below CAEDRMetadata's, and
the metadata is dropped again when HDR goes off.
struct netplay keeps its fourteen boolean states - server or client,
CRC checking, pausing, replay and catch-up, the forced rewind, reset
and savestate send, and the host settings - as bits of one uint16_t,
enum netplay_flags, rather than as separate bools. Conditions test the
bit; the one value sent to peers, whether pausing is allowed, is
converted to 1 or 0 before it goes on the wire.
display_servers_kms expects the connector's modes from
kms_display_server_modeline_enum(), which lists them so the engine can
score one when generation is off (#19619), rather than the empty list
of before: the mode's whole DRM timing, tagged DRM/KMS, and marked the
desktop mode where it is the live one, with an empty list and no
failure while there is no connector. Each failure exit frees the
display server it took, so a failing check reports itself rather than
the leak behind it.
The bundled wlr-output-management protocol leaves out the enum
attributes that name another interface's enumeration
(wl_output.transform, zwlr_output_head_v1.adaptive_sync_state).
wayland-scanner resolves such a reference only against the protocol it
is given, so scanners before 1.15 stop with "could not find
enumeration", which failed every build generating the bundled
protocols. The attributes annotate what an argument means and change no
generated C; the arguments keep the types the protocol gives them.
Four GPU index descriptor files carry the row of another driver than
the one their name gives: settings_def_gpu_index_d3d11.h holds
Direct3D 10's, _d3d12.h Direct3D 11's, _gl.h Direct3D 12's and
_vulkan.h Metal's. They are renamed for what they hold, and the four
sources that include them follow.
video_frame_menu_settings_t keeps its 29 boolean settings as bits of
one uint32_t flags word, enum video_frame_menu_flags, rather than as
separate bools. video_driver_build_info sets the word from the same
settings as before, and the menu drivers read the bits through
'? true : false', so a flag stays true when bool is a narrow type.
filter_plugins_test --bench, or make -C samples/gfx/filter_plugins
bench, times each video filter over 256x224 frames at one worker and
at as many workers as there are CPUs online (--threads N or THREADS=N
to choose), and prints the median per-frame time of each and the
speedup. The workers are kept alive across frames and woken once a
frame, as RetroArch's dispatcher does, so the times include what the
wake and join cost. It is not part of make check.
A mode switch through Mutter returns once it is asked of Mutter's
worker, which applies it later, so the Wayland display server no
longer follows it with a roundtrip on its connection. The roundtrip
waited on the compositor for the new mode's wl_output events, which
cannot come before the worker has applied the switch; they arrive
through the display server's pump when they come, as they already did.
The Wayland display server lists and switches modes on sway, Hyprland,
river and the other compositors that offer wlroots' output management,
as it does on GNOME and KDE. gfx/common/wayland_wlr_output.c speaks
zwlr_output_manager_v1 on the display server's connection: the heads
and their modes as the compositor describes them, and a switch as a
configuration carrying the last done serial and every head exactly
once, as the protocol requires - the chosen head with its new mode,
the other enabled heads as they are, the disabled ones disabled. The
compositor's succeeded, failed or cancelled answer is logged when it
comes, so nothing waits.
Mutter and KWin keep their place, and compositors with none of the
three still have no resolution list. The protocol is bundled from
wlr-protocols in deps/wayland-protocols/wlr. samples/gfx/wayland_wlr_output
checks the backend against an in-process compositor that rejects a
configuration leaving a head out or naming one twice, at manager
versions 4 and 2; the samples linking dispserv_wl.c link it too.
vga_set_texture_frame downscales ARGB8888 menu frames into the BGR332
menu buffer the way it does RGBA4444 ones, taking each channel's top
bits as the core frame path does, rather than leaving the buffer
holding whatever it held before.
The Wayland display server lists and switches modes on KWin as it does
on GNOME, so Screen Resolution and the refresh rate autoswitch work
under KDE Plasma. gfx/common/wayland_kwin_output.c speaks KWin's own
output protocols on the display server's connection: the outputs and
their modes from kde_output_device_v2, published through its registry
from version 21 and as globals before it, and a switch through a
kde_output_management_v2 configuration, whose applied or failed answer
is logged when KWin gives it, so nothing waits. The modes are those of
the output the window is on, matched by connector name.
Mutter keeps its place on GNOME, and every other compositor has no
resolution list, as before. The two protocols are bundled from
plasma-wayland-protocols (MIT-CMU) in deps/wayland-protocols/kde, so
nothing of KDE's is needed to build. samples/gfx/wayland_kwin_output
checks the backend against an in-process compositor speaking both
forms; the samples linking dispserv_wl.c link it too.
Demodulation runs in two passes. crt_demodulate_begin adds the noise,
finds vertical sync and, in line order, the horizontal sync and colour
carrier that carry from one line to the next, recording each active
line's decoding parameters. crt_demodulate_lines then equalises and
resamples a range of lines from those parameters alone, with its own
copies of the equalisers and its own line scratch in place of the
function-static one.
The filter's packets pass converts the input, modulates it and runs
the first pass once a frame; each worker decodes a band of lines and
copies the band of output rows those lines own. The output is the
single-worker output frame for frame, and about three quarters of a
frame's work is spread across the workers.
The colour burst phase advances one step a row, so each worker's band
of rows starts at the frame's phase plus its first row, where a single
pass would be on that row. The per-frame toggle of the phase, used when
fields are not merged, happens once a frame before the workers run
instead of once per worker, which also leaves the workers nothing
shared to write. The filter runs on as many workers as RetroArch
creates it with, giving the single-worker output frame for frame.
The horizontal stretch writes the pixel past the last source pixel as
black instead of blending it from whatever the buffer held, so the
partial clear of the output that it relied on - its length taken in
pixels as bytes, covering only part of each band - is gone. The blue
phosphor pass stops at the end of the line rather than writing one
pixel past it. The image is unchanged.
The filter runs on as many workers as RetroArch creates it with,
giving the single-worker output.
Each pixel's upper and lower neighbours are the rows above and below it
in the frame, repeating the edge row only at the frame's own top and
bottom. The last worker's band had used its own row as the one below
for every row, which with one worker is the whole frame, so the filter
compared neighbours only across and above. It now runs on as many
workers as RetroArch creates it with, giving the single-worker output.
The XRGB8888 averages clear the lowest bit of every byte before
halving, so an odd sum in one channel no longer shifts into the top of
the channel below it: averaging green 1 with green 0 had given blue 128.
kms_display_server_modeline_enum() returned 0, so the engine started
on KMS with no modes at all. With modeline_generation off there is
no generated candidate either, so mode selection had nothing to score
and failed with "Could not find a video mode that meets your specs".
Fill the list from g_drm_connector->modes, each with its whole DRM
timing, tagged MODELINE_TIMING_DRMKMS, with the entry equal to
g_drm_mode marked as the desktop mode - the same shape the XRandR
server already provides. Setting one goes through the existing path:
modeline_set() copies the timing into crt_switch_st and drm_ctx
rebuilds the drmModeModeInfo from it, so an existing mode round-trips
exactly.
g_drm_connector is NULL between a context teardown and the reinit;
the list is empty then rather than an error, so a generated mode
still goes through.
Not a regression from the in-tree engine: the previous KMS path
used switchres's "dummy" display, which enumerated nothing either.
With generation on the existing modes now compete in scoring the
way they already do under X11.
These filters split the frame into row bands across as many workers as
RetroArch creates them with, one per core, rather than one; each band
reads its neighbouring rows from the frame, so the output is the same
as with a single worker. A single-core machine still gets one worker.
The filter plugin harness runs every video filter that takes more than
one worker with 2, 3 and 4 workers at once at an odd frame size and
checks the output byte for byte against a single worker's, and the
Linux-filter-plugins workflow runs it under ThreadSanitizer as well.
The core/directory/game .switchres.ini files are loaded through
crt_adjust_ini(), then switch_res_crt() writes the H-Size, H-Shift
and V-Shift sliders onto the generator right before modeline_get(),
so an h_size/h_shift/v_shift the files set never reached the mode:
with the sliders at their defaults the override became 1.0/0/0.
The sliders are now the base the files refine. crt_adjust_ini()
seeds the generator from the sliders before loading the files,
records which of the three the files changed and their values, and
switch_res_crt() writes the recorded value for those until that
slider is moved, after which the slider wins for that one value.
The write still happens on every switch because the generator
clamps the geometry in place per mode.
Two things this uncovered:
- current_content_name was never filled, so the content-change
check fired on every mode change and re-parsed switchres.ini and
all the override files each time. It is now filled from the
content basename the way crt_adjust_ini() records it.
- On the first switch the slider values carried in from the config
counted as a move; crt_adjust_ini() now takes the slider state as
seen, so a config value does not clobber a file's override on the
first mode.
samples/gfx/crt_switch_modeline gains a case with a game override
file: its h_shift/v_shift reach the generator with the sliders at
default, survive the next mode change, are parsed once per content
rather than per switch, a moved slider takes only its own value
back, and other content without a file returns to the sliders.
Once a frame, before the workers run, the filter fills a buffer with
one YUV key per source pixel, RGBtoYUV of its RGB565 value. The edge
decisions load the keys of the 5x5 neighbours with the same clamped
indices as the pixels and compare them by neighbour name, so each
comparison is a subtraction instead of two conversions and table
lookups, and the outer ring of the neighbourhood, used only in
comparisons, is no longer loaded as pixels. The output is unchanged;
at 256x224 the XRGB8888 path takes about 60% of its previous time.
The XRGB8888 path makes its edge decisions at RGB565 precision through
the same YUV table as the RGB565 path, two lookups per comparison in
place of a double-precision YUV transform, and blends at full 8-bit
precision as before. At 256x224 it runs in about the RGB565 path's
time, around a fifth of what the floating-point comparisons took.
df8 and eq8 are macros local to the file rather than exported symbols.
The video filter Makefile defaults to a release build, as the DSP
filter Makefile does, so the plugins are built with -O2 unless
build=debug is given.
The video filter and audio DSP filter Makefiles name libm after the
objects, so --as-needed keeps it and the plugins that use math
functions record it; they loaded only into a host that had libm
already.
samples/gfx/filter_plugins loads every plugin with RTLD_NOW into a
host that does not link libm and runs a frame or a run of audio blocks
through it at its default settings. The Linux-filter-plugins workflow
runs it as built and again under ASan, UBSan and LeakSanitizer.
Each pixel reads the two rows above and below it in the frame, and at
the frame's edges the rows and columns repeat the edge rather than
reading outside it. With the filter on its single worker the row step
had been zero for every row, so the neighbours above and below were
the pixel's own row, none of 2xBR's edge rules could fire and the
output was plain pixel doubling, identical to normal2x. It is now
2xBR's.
The CRT output buffer is cleared when it is allocated and whenever it
is resized. crt_demodulate does not write every pixel of it, and what
it leaves is copied to the screen, so without the clear those pixels
showed whatever the heap held and changed from run to run.
The noise generator wraps in unsigned arithmetic, and the negative
filter coefficients and luma samples are scaled by multiplication
rather than left-shifted; the output is unchanged.
The NTSC filters widen RGB565 channels to 8 bits by replicating each
channel's top bits into the new low bits, as the rest of RetroArch
does, so white enters the filter at full scale and greys stay
neutral, rather than as 248/252/248 with green a step ahead of red
and blue.
The scale loops find each source row pitch bytes into the frame
rather than width pixels, so a core frame whose rows are padded is
read at its own stride instead of coming out skewed. Tightly packed
frames, and the menu frames these drivers pack themselves, read
exactly as before.
vga_gfx.c is built against stub dpmi.h and pc.h carrying the DPMI
register block, the real-mode interrupt call, dosmemput and the port
I/O it uses, in the shapes DJGPP gives them, with declarations after
statements rejected. Outside the DJGPP job nothing compiled it.
conv_argb8888_rgba4444, conv_argb8888_0rgb1555 and conv_rgba4444_rgb565
have SSE2 paths, and conv_argb8888_rgba4444 a NEON one. The 32-bit
sources narrow with _mm_packs_epi32, the 4444 results sign-extended
first so values past 0x7fff are not saturated. conv_bgr24_argb8888
reads four pixels as three 32-bit words on little-endian hosts that
have no NEON path. conv_bgr24_rgb565 keeps its byte loop, which the
word loads did not beat.
The compile matrix builds gl1.c's Vita branches against a vitaGL stub
over the host GL headers, rejecting declarations after statements the
way the PSP lane does; nothing else compiles them outside the Vita
toolchain.
screenshot_dump() without a thread writes the file and then releases
its state and, once written, the caller's buffer - the same ownership
the task path has, where a failed write leaves the buffer the caller's
to free. It also releases the state when the screenshot directory
cannot be created. --max-frames-ss and save-state thumbnails both take
the synchronous path, and each call left the state and a full captured
frame behind.
The ASan/UBSan workflow takes a screenshot on exit under
LeakSanitizer, with only the system libraries suppressed, and fails on
any leak left over.
gl1_draw_tex updates its texture in place with glTexSubImage2D, reading
the frame at its own pitch through GL_UNPACK_ROW_LENGTH, and respecifies
the storage, cleared, only when the power-of-two size or internal
format changes. XRGB8888 frames go up straight from the core's buffer,
RGB565 frames as GL_UNSIGNED_SHORT_5_6_5 on GL 1.2 and newer, and
RGUI's RGBA4444 framebuffer as it is; the no-GL_EXT_bgra swizzle writes
just the frame, tightly packed. GL 1.1 without packed pixels, the HDR10
tonemap and the Vita build, which has no GL_UNPACK_ROW_LENGTH, keep
their staging buffers.
samples/gfx/gl1_upload runs the driver under Xvfb with a test core and
checks the screen pixel for pixel down every upload path, threaded
video and frame size changes included, in the Linux-gl1-upload
workflow.
conv_rgb565_0rgb1555, conv_0rgb1555_rgb565, conv_rgba4444_rgb565,
conv_bgr24_argb8888, conv_bgr24_rgb565, conv_argb8888_0rgb1555,
conv_argb8888_bgr24, conv_abgr8888_bgr24 and conv_yuyv_argb8888 have
NEON paths, so every converter the scaler dispatches to is vectorised
on ARM. YUYV keeps the scalar path's 16-bit fixed point, with
vqmovun_s16 doing the clamp. The SSE2 path of conv_rgb565_0rgb1555
shifts red and green down one bit, matching its MMX and scalar paths.
The pixconv test covers every converter, including the 24-bit and
YUYV sources, so each SIMD path is checked against a reference on
x86 and, in the cross-arch lane, on aarch64, armv7 and PowerPC.
vulkan_common.c asks the video driver for the peak its HDR metadata
carries; the sample's stub answers with the driver's own value, as the
frontend does with no display peak in use.
Use Display Peak, off by default, makes the peak luminance the display
reports stand in for Peak Brightness wherever a peak is used: what
cores are told through GET_HDR_MAX_NITS, and the HDR metadata the
drivers send - DXGI's for D3D11 and D3D12, Vulkan's vkSetHdrMetadataEXT
(which reaches the compositor under Wayland), and the KMS blob. Off, or
with no reported peak, every value stays as it was: the setting for
cores, each driver's fixed 1000 nits for its metadata.
Windows reports the panel's peak through DXGI_OUTPUT_DESC1, read where
the HDR check already reads it and passed on only when it changes, and
Android through HdrCapabilities (API 24), queried once by its display
server; Wayland and KMS already did. Apple's EDR APIs give a headroom
over the current SDR white, never an absolute peak, so macOS and iOS
report none.
The Wayland and KMS contexts learn the display's peak luminance where
it is on offer - the compositor's description of the output
(wp_color_management_output_v1, its target luminance) and the sink's
EDID - log it, and hand it to the video driver, whose Peak Brightness
entry shows it beside the setting, as "1000 (display: 650)". The
setting's own value is unchanged and nothing applies the display's:
peak luminance is something users calibrate around.
The compositor's description is read without waiting, from events the
context already dispatches: its target luminance, not the transfer
function's range - 10,000 nits for PQ - which says nothing about the
panel. samples/gfx/wayland_color_management covers the query against
its compositor, with and without a target luminance.
conv_0rgb1555_argb8888, conv_rgba4444_argb8888, conv_0rgb1555_bgr24
and conv_rgb565_bgr24 convert eight pixels per step on ARM NEON. Each
channel is shifted to the top of its lane with its own top bits
inserted beneath, so narrowing gives the bit-replicated 8-bit value;
the BGR24 variants store through vst3_u8. Output matches the scalar
path bit for bit.
The cross-arch samples lane runs the pixconv test under qemu on
aarch64, armv7 with and without NEON, and big-endian PowerPC, and the
scalar test build leaves out NEON as well as SSE2 and MMX.
The KMS context drives an HDR10 sink when HDR is on: where the sink's
EDID declares SMPTE ST.2084 and the connector has HDR_OUTPUT_METADATA
and a Colorspace with BT2020_RGB, it sets static metadata built from the
user's peak (capped at the sink's), Rec.2020 and 10 bits, then scans out
XRGB2101010 from a 10-bit EGL config. The properties are set before the
surface exists; if the connector refuses them, or there is no 10-bit
surface or config, what changed is put back and the output stays SDR,
so a PQ picture never reaches a sink not told to expect it. The
connector is restored on the way out. GFX_CTX_FLAGS_HDR10_FRAMEBUFFER
tells the video driver its framebuffer is Rec.2020 PQ.
gl and glcore render it through their scRGB offscreen and encode pass,
finished in PQ: the encode shaders gain an HDR10 output switch - a
uniform in gl2's, a UBO field in glcore's, off everywhere else - that
rotates the composited scRGB to Rec.2020 and PQ-encodes it at absolute
luminance, a PQ source passing through as it came. The native HDR
screenshot read-back, which reads an FP16 backbuffer, stands aside.
gfx/common/drm_hdr.c holds the EDID parse and metadata build as pure
functions, which samples/gfx/drm_hdr checks, including the blob layout
against the kernel's own struct.
The X11 EGL context chooses its GPU as the Wayland one does: it lists
EGL's devices, makes its display on the chosen one through
EGL_EXT_explicit_device, falls back to the default display where the
device cannot drive it, and publishes the list to the menu. The KMS
context's display follows the card it opens, through GBM, so there the
chosen device's DRM card (EGL_EXT_device_drm, now kept per device by
egl_common) is tried before the others, which follow in their usual
order when it has no usable output.
The GL drivers' GPU Index entry appears only while the context has
published such a list; GLX, and any context without device enumeration,
has nothing to choose from and shows none.
The GL drivers' scRGB frame carries alpha - their frame target clears to
transparent black and the encode pass keeps its alpha - and on Wayland
an RGBA16F surface's alpha is blended by the compositor, the opaque
region being only a hint, so the letterbox around a core's frame showed
whatever lay behind the window. The context now creates that surface
with EGL_EXT_present_opaque, which egl_common applies to the next window
surface when the context asks, and scRGB is offered only where the
display has it.
compile-matrix.sh builds configuration.c with EGL on and Vulkan off, and
with Vulkan on and EGL off. Each video API owns a GPU index setting and
its default under its own gate, and a build has any mix of APIs; a
default defined under another API's gate builds only where both are on,
as the GL GPU index default did before it moved under HAVE_EGL, which
broke every EGL build without Vulkan, the PS Vita's among them.
DEFAULT_GL_GPU_INDEX is defined wherever the gl_gpu_index setting is,
under HAVE_EGL, rather than inside the Vulkan block, so EGL builds
without Vulkan build again.
The GPU list the Wayland context publishes for the GL drivers has a slot
of its own, for OpenGL and OpenGL ES alike, so the GPU Index entry names
each device and steps through them as it does for Vulkan and Direct3D.
A device's label leaves out the vendor where its renderer string
already begins with it, as NVIDIA's does.
pixconv.h provides single-pixel expanders from RGB565, 0RGB1555,
RGBA4444 and ARGB4444 to 32-bit that move every channel into its byte
lane and widen them together with one shift and mask (a multiply by
0x11 for 4-bit channels). The pixconv scalar paths, dxgi_copy, the PNG
and BMP encoders and the HUB75 driver use them. conv_argb8888_rgba4444
takes the high nibble of each channel, dxgi_copy reads and writes
B5G5R5A1 alpha at bit 15 and widens it to full scale, and the MMX
paths load and store through memcpy since rows need not be 8-byte
aligned.
libretro-common/samples/gfx/pixconv runs every 16-bit input through
each converter with SSE2, with MMX only and with no SIMD, and is part
of the libretro-common samples workflow.
The GPU Index entry is a row per video driver, bound to that driver's
own index and added only while it runs; gl, glcore and gl1 get theirs,
bound to gl_gpu_index, so the GL GPU list the Wayland context publishes
reaches the menu. The entry's label reads the bound index again, as it
does for every other driver.
The GL drivers on Wayland take a GPU index as Vulkan does. The context
lists EGL's hardware devices through EGL_EXT_device_enumeration - under
glvnd every vendor's, so an NVIDIA GPU beside a Mesa one - labels them
by vendor, renderer and DRM node, and makes its display on the chosen
one through EGL_EXT_explicit_device. Index 0 leaves the choice to the
EGL implementation, as before; a device that cannot drive the display
gives way to the default with a warning. The list reaches the GPU Index
entry like Vulkan's, whose label now shows the running API's own index.
egl_common gains the device list and a device hint for the next display
it makes, usable by any EGL context.
The GL drivers get HDR on Wayland the way they have it on Windows: an
FP16 framebuffer composited as scRGB, which gl, glcore and gl1 already
render to when the context reports GFX_CTX_FLAGS_SCRGB_FRAMEBUFFER. No
EGL driver tags a Wayland surface's colour space for GL, so the context
does it through wp_color_manager_v1: where the compositor offers the
windows_scrgb feature with a perceptual intent, EGL has an RGBA16F
config (EGL_EXT_pixel_format_float) and HDR is on, the context takes
that config, marks the surface opaque and tags it Windows-scRGB. The
tag is set from the image description's ready event, as the protocol
requires, during the context's own dispatch, so nothing waits for it.
The HDR settings are offered where the compositor and EGL could do it;
GL HDR is scRGB only, so HDR10 maps to it. The Vulkan context binds the
manager but never tags its surface, which the Vulkan driver's WSI owns.
gfx/common/wayland_color.{c,h} holds the protocol client, built on the
colour-management protocol now bundled in deps/wayland-protocols.
samples/gfx/wayland_color_management runs it against an in-process
compositor that records every fatal rule a client could break -
Windows-scRGB without the feature, a description set before it is
ready, an intent not offered, a second colour-management object on a
surface - and CI runs it under ASan and UBSan.
The Wayland display server's DRM lease report is only a log line, so
it is started once per instance through wl_display_server_report_lease()
and only where the log is on: video_display_server_init() starts it when
it makes or keeps dispserv_wl with verbosity enabled. The early display
server init runs before the command line turns the log on, and its
report used to race that, appearing on some starts and not others;
now the report comes with the video driver's context, after
"Found display server", on every start, and without the log no report
connection is made at all.
The display server set up early from WAYLAND_DISPLAY, before the video
driver has a window, is kept when the driver's context declares Wayland
too, instead of being destroyed and made again. dispserv_wl keeps a
connection of its own whatever window the driver makes, so the early
one serves the session as it is, and startup makes one connection and
reports the DRM lease once.
Since Linux 6.12 a time-shared thread may ask for its own EEVDF slice
through sched_setattr(), with no capability or rlimit. A new backend,
frontend/thread_elevation/eevdf.c, asks for the kernel's shortest, 0.1
ms, so the thread gets an earlier deadline and the CPU sooner when it
wakes, with its share of the CPU unchanged. It leaves a real-time,
batch or idle thread alone, and counts only when the slice reads back as
asked, which kernels before 6.12 do not do. Under two busy threads on
one CPU, an audio-shaped thread's p99 wake-up lateness drops from about
3 ms to under 0.1 ms, and a burst longer than the slice completes no
later than before.
The chain gains additive backends: self-acting, tried first, and never
ending the chain, since a scheduler hint complements a raised priority
rather than standing in for it; a continuation never reaches one.
thread_elevation_raise_current() reports one that applied, and the
audio thread logs it. samples/frontend/thread_elevation covers additive
ordering and continuation, and runs the real backend on the kernel it
finds: granted with the slice read back where it is supported, and a
real-time thread left as it is.
The Wayland contexts declare RARCH_DISPLAY_WAYLAND once their display
is up, as the X11 ones do, so a native Wayland session gets dispserv_wl
- the output's refresh rate, size, metrics and EDID, and on GNOME
Mutter's resolution list and refresh-rate switching - rather than the
null display server. dispserv_wl keeps a connection of its own, used
only on the thread that calls it, and makes no roundtrip: init asks for
the registry and returns, and every call first takes whatever the
compositor has sent without waiting for more, so the output is bound
and described over the first few calls. The DRM lease report, which
needs answers, runs on a detached thread with a connection of its own.
compile-matrix.sh builds the D-Bus, Mutter and RealtimeKit units and
the elevation chain as a target with no POSIX headers, against
tools/platform_stubs/no_posix, where any of them reaching <unistd.h>,
<poll.h>, <fcntl.h>, <dlfcn.h> or <sys/*.h> outside its platform gate
fails. samples/gfx/wayland_drm_lease builds with threads, as RetroArch
does, and waits for the report; samples/gfx/mutter_displayconfig checks
that the output is described without a roundtrip.
Where retro_atomic_int_t is volatile (the MSVC backend), the address
of a waiter's flags or of the audio reinit request travels through
volatile-qualified pointers: the runtime-resolved wait-on-address
entry points and the MONITORX/UMONITOR helpers take a volatile
address, and wasapi hands the reinit request to the watcher thread
through an explicit void pointer, which mmdevice_thread casts back.
The autodetect index parser keeps the strchr() result in a const
pointer, so the qualifier-preserving strchr of C23 glibc has nothing
to discard, and the X11 modeline path drops a crtc counter it never
reads.
Griffin mirrors Makefile.common, which adds dbus_runtime, dbus_common,
mutter_displayconfig and rtkit only for Linux and BSD targets, so
griffin.c includes them under the same platform set instead of under
HAVE_THREADS alone. Windows, consoles and other griffin targets no
longer see these units at all.
mutter_displayconfig.c keeps <unistd.h> inside RARCH_HAVE_MUTTER_DC,
next to the pipe/poll code that needs it, so the file stays an empty
translation unit on any platform where the Mutter backend is off.
Mutter's org.gnome.Mutter.DisplayConfig is used by every threaded Linux
and BSD build, through libdbus loaded at run time, and nothing that
calls into it waits on the bus. One worker for the process connects,
subscribes to MonitorsChanged and publishes Mutter's state, reading it
again after every change; it waits in poll() on the bus socket and a
wake pipe. The resolution list and availability are answered from the
published state under a lock the worker takes only to swap it, so the
menu, the refresh-rate check and the runloop's autoswitch no longer
make a D-Bus round trip. A switch is checked against that state at once
- a size or rate the head does not list is refused without asking
Mutter - and applied by the worker against the state Mutter reports
then, with the serial, layout and carried-over settings as before.
Until the worker has answered, or where Mutter is not there, callers
keep their own behaviour.
dispserv_wl and dispserv_x11 ask for Mutter at each call rather than
once at init. The runtime libdbus table gains the container, append,
match and dispatch entry points this needs, checked against libdbus's
headers by samples/frontend/dbus_runtime. samples/gfx/mutter_displayconfig
builds on the loader and waits for the worker where it used to get a
synchronous answer, and its mock announces MonitorsChanged as Mutter
does.
Screensaver inhibit over org.freedesktop.ScreenSaver works in every
threaded Linux and BSD build, on libdbus loaded at run time. One worker
for the process loads the library, opens and closes the session-bus
connection and makes the Inhibit and UnInhibit calls; the main thread
only publishes the state it wants under a lock the worker never holds
across a call, and never waits for an answer. A generation that moves
on every open keeps a slow answer from being reported for a newer
request, and losing the session bus ends the connection rather than
RetroArch.
On X11, XScreenSaver is asked alongside D-Bus on the main thread, where
its display lives. xdg-screensaver, whose suspend lasts as long as the
window, starts only once D-Bus has failed: at once when that is already
known, otherwise from x11_check_window() when the worker's answer
arrives. Wayland compositors without idle-inhibit use the same worker.
RealtimeKit is a backend of every threaded Linux build: libdbus is
loaded when first needed through gfx/common/dbus_runtime, which mirrors
libdbus's public, ABI-stable structs and constants under names of its
own, so no D-Bus headers or link dependency are needed and a system
without the library runs as before. The rtkit helper loads it on its
own thread, so neither the audio thread nor the main thread waits on
the dynamic loader or the bus, and whichever thread loads first
publishes the table without anyone waiting. HAVE_DBUS no longer gates
the backend.
The helper now tells apart what can happen: libdbus not available, no
system bus, rtkit not running, rtkit refusing with its D-Bus error
name, and each grant. samples/frontend/dbus_runtime checks the mirror's
sizes, alignments, field offsets and constants against libdbus's own
headers, and the no-pkg-config CI job runs it.
Without pkg-config, configure finds the packages it used to need
pkg-config for through check_nopkg(): the include directories are
looked for under the include and library directories, where D-Bus keeps
its arch header, and a probe program must build and link before the
package counts, with the CFLAGS and LIBS pkg-config would have given.
Qt is described by its own qmake -query, headers, libraries and mkspec
alike, and wayland-scanner answers for its version, with the protocol
generator using deps/wayland-protocols. With pkg-config present none of
this runs and configure is unchanged.
A package asked for by name stops configure when nothing can find it:
check_pkgconf() dies on a forced package without pkg-config unless a
library check or a check_nopkg() probe follows, and a forced Wayland
stops configure when its libraries or scanner are missing rather than
being disabled with a notice. The no-pkg-config CI job installs these
packages, configures with --enable-dbus and checks each is found and
built, then checks that a forced rsound fails and a forced pulse, which
has a library behind it, still configures.
The custom viewport origin and size, the window size pairs and the
desktop menu window geometry each keep both axes in one settings_t
word, and each menu row owns one half of it. The generic uint/int
display strings, the custom viewport width/height display strings,
the dropdown builders and commits, and the Qt spin box, combo box and
radio button widgets all dereferenced value.target directly, so these
rows showed and edited the whole word: a 960x720 viewport displayed
its width as 62915280, stepped in units of 65536, clamped to 9999 in
Qt, and a Qt edit or dropdown choice overwrote the partner axis.
setting_uint_get/set and setting_int_get/set now live in
setting_list.h so the menu and the desktop UIs share them, and all of
those paths go through them. samples/settings/packed_rows drives the
rows through their display strings, a right press and the dropdown,
and runs in the menu harness workflow.
thread_elevation_raise_current() tries each backend in
frontend/thread_elevation.c's list until one grants, and returns at
once. A self-acting backend changes the calling thread and answers
there; a brokered one acts on a thread id from any thread and may
answer PENDING, finishing on a thread of its own and carrying on with
thread_elevation_continue() if it is refused. Every self-acting backend
precedes every brokered one, so a continuation only ever meets brokered
backends. The list is rthreads' kernel request, then RealtimeKit on
D-Bus builds; the audio thread asks once and names none of them.
RealtimeKit moves out of gfx/common/dbus_common.c into
frontend/thread_elevation/rtkit.c as a brokered backend, unchanged in
what it asks for and logs. samples/frontend/thread_elevation drives the
chain runner with a scripted list: order, the thread id and resume
point a brokered backend is given, and a pending request refused on its
own thread continuing past a self-acting backend placed after it.
Both device clock fits, from the audio timestamp and from the position,
start again from the writer's next sample whenever the device stops and
restarts: alsa_start(), alsa_stop() and every playback xrun recovery
raise an atomic flag that alsa_clock_sample() consumes. Through a pause
or an underrun the position stands still while time runs on, so a fit
spanning one reads the device as slow; the last published estimate
stands until the new window publishes its own.
alsa_clock_test covers a 2 s pause and a 300 ms underrun mid-window.
When sthread_raise_current_priority() is refused on a Linux build with
D-Bus, the audio thread hands its tid to dbus_rtkit_raise_current_thread(),
which runs the whole RealtimeKit exchange on a detached thread of its own
and returns at once; nothing waits on the bus and the outcome is only
logged. The helper asks for real-time round-robin at rtkit's
MaxRealtimePriority, capped at the middle of the band, after bringing
RLIMIT_RTTIME within rtkit's RTTimeUSecMax as rtkit requires, and falls
back to a nice value no lower than -11 or rtkit's MinNiceLevel. One
request is in flight at a time.
audio_thread_wrapper.c includes config.h so that HAVE_DBUS reaches it.
On KDE 4 and later and on GNOME 3, xdg-screensaver suspend hands the
inhibit to a detached Perl helper that needs Net::DBus and
X11::Protocol. The probe reads the desktop from the environment the way
xdg-screensaver does and, on those desktops, checks that both modules
load; without them screensaver suspension is reported as disabled
instead of launching a helper that fails on stderr while the command
itself exits 0.
sthread_raise_current_priority() asks for SCHED_RR at the middle of the
band first. When that is refused on Linux it retries at RLIMIT_RTPRIO
if the limit grants a lower real-time priority, and failing that lowers
the calling thread's nice value to -11, or as far as RLIMIT_NICE lets
it. A process that holds CAP_SYS_NICE, or whose limits allow the full
request, gets exactly what it got before.
audio_thread_loop() checks its argument before touching it.
- Change screen switch combo to LShift+Tab
- Default pointer mode to pointer
- Restart UDP socket on IP option change
- Call GET_SENSOR_INTERFACE only on page switch and not on every frame in sensor page
- Hide pointer and sensor debug under define instead of comment
- Cleanups and nits
struct record_params carries out_dims and fb_dims as VIDEO_SCALE_PACK
words; ffmpeg scales and evens the output size in locals and stores
the result as a word, and sizes its buffers with VIDEO_SCALE_AREA.
The viewport parameter snapshot keeps the custom viewport's origin and
size as the VIDEO_POS_PACK and VIDEO_SCALE_PACK words the settings
store, and vulkan's font pass sets its viewport from the video size
word it already holds.
The scRGB offscreen in gl1, gl and glcore, glcore's streamed textures
and gl3_filter_chain_texture keep their sizes as VIDEO_SCALE_PACK
words, so frame, history, feedback and framebuffer sizes pass through
the filter chain as the words they are stored in. The offscreen target
helpers, gl3_update_cpu_texture(), gl3_framebuffer_set_size(),
gl3_pass_get_output_size() and the size semantic builders take words.
Every driver with a pump or a callback counts the periods it filled
with silence for want of audio and hands the number to the statistics
overlay; wdmks handed nothing, so it was the one driver where the
screen could not say whether it was keeping up.
The loop knows exactly when it runs out: the hardware running past the
write cursor, or laps going by unobserved, which is where the write
cursor is resynced. That is counted - once per gap in the audio. A
start is not a gap, and neither is every further sample of a pause
finding the loop empty again: the count moves only when audio had been
placed since the last resync, so a pause of any length is one
underrun, the one where the audio stopped.
The packet path still reports nothing: a packet the device found
nothing behind is the port driver's to fill, and it does not say so.
The note on the position register that ended with "a thread of this
driver's own - which is not here yet" is brought up to date: the
refill thread has been there since it landed.
samples/audio/wdmks_loop asserts the count through the contract: nil
after a start, one when the audio runs out, still one as the pause
goes on, two after laps are lost with audio queued, and unchanged by
a restart.
SpriteVertex carries its colour as UShort4Normalized, packed by
rgba16_pack(): a vertex is 24 bytes against 32, and the menu bake,
the font quads and the overlay store one word per vertex. The vertex
functions still read a float4, so the shaders and the prebuilt
metallib are unchanged.
The overlay's colour stream is UNORM16, bound as normalised
GL_UNSIGNED_SHORT: eight bytes a vertex against sixteen for four
floats, and set_alpha writes the one alpha channel it changes.
struct vk_vertex carries its colour as R16G16B16A16_UNORM, packed by
rgba16_pack() in gfx/common/rgba16_pack.h, which replaces rgba8_pack.h.
Sixteen bits keep the high-precision menu text colour and every float
fade finer than any swapchain the menus draw to, so a fade on a 10-bit
or scRGB target is not held to 256 steps; a vertex is 24 bytes against
32 for four floats.
samples/gfx/rgba16_pack holds the SSE2, NEON and scalar paths to the
same reference, including a lane that reads every 10-bit level back at
10 bits, which an 8-bit vertex colour fails.
After the hardware runs past the write cursor - and at every start -
the write cursor is put a margin ahead of the register, so the bytes
are in place before the DMA fetches them. That margin was a fixed two
milliseconds, a guess, and on the loop it is also the floor the
reported latency can reach: a queue cannot be driven below it because
every resync puts it back.
The pin says how far past the register the DMA has already fetched:
the FIFO in its hardware latency report, which the driver was already
reading for the statistics. The margin is now that FIFO plus half a
millisecond for the write itself, and two milliseconds only where the
pin reports no FIFO. On a codec reporting a 1 ms FIFO that takes the
floor from 2.7 ms to about 1.5 ms; on a pin whose FIFO is longer than
two milliseconds the old margin was a write into bytes already gone,
a click on every resume, and the margin is now sized to it.
The latency report is taken before the pin is started, because the
first sample after a start resyncs by the margin.
samples/audio/wdmks_loop runs the contract with no FIFO report, a
1 ms one and a 10 ms one, and pins the margin's arithmetic for each.
video_driver_frame packs the core's width and height once and hands
that word to the driver, the threaded wrapper passes its slot's word
through, and the drivers that compared or stored the size packed
compare and store it directly. sdl2's texture record and gl2's pass
sizing take the word too.
Pausing a core on the wdmks driver, or anything else that stops the
frontend writing - the menu with the core silent behind it, a stall -
played the last few milliseconds of audio round and round at full
level for as long as it lasted.
A WaveRT pin plays a mapped loop; unlike a packet pin it has no notion
of running out. The frontend writes a fade to silence on a pause and
then nothing, on the understanding that a device with nothing to play
plays nothing. The loop kept what it held and the hardware kept
reading it, so the tail of the fade and whatever came before it in
the loop repeated at the loop's own rate until writes resumed.
Now every byte the cursor passes is cleared as soon as it has been
played, so from then until it is written again it holds silence, and
whatever the hardware reads past the end of the audio is silence.
Two more things fell out of the same bookkeeping:
The two positions alone could not say whether a loop with the cursors
together was full or empty. The model took it for full, so at start
the first audio was only ever placed behind the hardware as it moved
off, one loop late; and after the hardware overran the write cursor
the next write went in behind it too, to be played a loop late for
good. Room is now the loop less a count of what has been placed and
not played, and when that count shows the hardware past the write
cursor - or the cursor could not be watched round - the write cursor
is moved a small margin ahead of it.
start() takes the pin through ACQUIRE, which puts the hardware back at
the top of the loop; the model now starts over from the first sample
after it rather than believing a write cursor left for the old one.
samples/audio/wdmks_loop compiles the driver whole and drives the loop
by hand on both the caller-driven path and the refill thread's pump,
under wine; it fails on the previous driver at "the audio ran out: the
whole loop is silence, not its tail again", among others, and passes
now.
ledN_map reaches the image at the slot it names whatever the desc
there does when pressed, as it did before the _led naming: an LED
overlay may put its lights on keys or buttons, and a config
calibrated on such a pack keeps working. A pack that names its LED
images with _led is still not mapped, and a hidden image still stays
hidden while its desc is pressed.
overlay_visibility_bounds_test pins every mapped slot, control or
"nul", as hidden while its LED is off; overlay_led_hidden_test now
expects the mapped "up" square hidden, pressed or not.
gfx_thumbnail_get_draw_dimensions() unpacks the thumbnail's size once,
folds the core-aspect correction into one ratio that is 1 when it does
not apply, and writes the two results once at the end rather than
reading them back through the pointers between steps. Bit-identical
results. vulkan_init_readback() reads the viewport size once.
With Integer Scale Axis set to Y + X (or any of the modes that touch X),
the horizontal multiplier was applied to the aspect-corrected width
rather than the frame's own width. A 256x224 Snes9x frame with its core
aspect of 64:49 is 293 wide after correction, so at 1080p the scaler
picked 293 x 4 = 1172 wide over 224 x 4 = 896 high: a 4.58x horizontal
scale, pixel columns alternating 4 and 5 wide, and the fraction the
statistics overlay shows as Scale X/Y. It now picks 1280x896, 5x/4x.
The "reset width to exact width" step used to read the frame cache
fields directly. When those became a local snapshot (69efa8a894) the
reset started reading content_width/content_height, which by that point
hold the aspect-corrected width and the rotated height. The frame's
size is now kept in its own pair and the reset reads that; rotated
content takes the frame height as its width, as it did before.
Driver-agnostic: the scaler is in video_driver.c, so d3d11, Vulkan and
the rest all showed it.
Regression test: samples/gfx/integer_scale_axis runs the scaler out of
video_driver.c (extracted by awk, as asio_clock does) over a sweep of
viewports, frame sizes, axis and scaling modes, rotated and hi-res
content, and checks each drawn size is a whole multiple of the frame
size; the first check is the reported 1080p 256x224 case. On the old
code it fails 386 of 778 checks, reporting 1172x896 for the report
case; on the new code all 783 pass. Wired into CI Linux samples/gfx
under ASan + UBSan.
Reported-by: sonninnos
Functions that unpacked the same dims parameter many times read its
halves into two locals at the top instead. Only where the object code
came out the same size or smaller.
vulkan_create_swapchain() and vulkan_surface_create() take the wanted
size as one VIDEO_SCALE_PACK word, and so does KHR_display's mode
search, from vulkan_display_surface_info through the chosen mode. The
context drivers pass the words they hold rather than their halves.
The step started the build with no display and required exit status 1.
On the runners it fails at once with nothing in the run log, on every
driver and sanitizer, and the same run passes on a runner-shaped build
elsewhere, so it tests the runner rather than the code. It goes; the
glcore teardown guard it was written for stays in gl3.c.
A context driver's set_resize takes the output size as one
VIDEO_SCALE_PACK word, which every caller already holds; the D3D8 and
D3D9 drivers' own set_resize helpers take it the same way. The X11
contexts log the size as unsigned.
With XInput 2, the driver's own connection takes the master pointers'
motion, button and crossing events on the window, and the poll reads
each pointer's position, buttons and whether it is inside from them,
so it sends no request. A pointer outside the window reads nothing
held, a drag out of it included. That connection holds the window's
button events, so it latches the wheel and buttons 8 and 9. A grabbed
pointer is warped back on the same connection, and a position older
than the warp is ignored. Each pointer is queried once at init; a
server that refuses the selection, or no XInput 2, leaves the query.
samples/input/x11_key_poll becomes x11_input_poll and drives the
pointer too: buttons, the wheel, a drag, a new driver over a held
button, and grabbed motion across a warp held behind a server grab.
x_input_poll() reads the keyboard from key events on a connection of
the driver's own, drained at the poll, so it sends no request and
never waits behind what the video thread presents through the shared
connection; keys still read as of the poll. KeymapNotify after each
FocusIn replaces the key vector and a focus change clears it, except
under another client's grab, which keeps the window's focus. Detectable
auto-repeat keeps a held key from ever arriving as a release. Without
that connection the poll queries the server as before.
samples/input/x11_key_poll drives the driver through XTEST on Xvfb:
no requests over 200 polls, a key held before init, auto-repeat, focus
changes, a keyboard grab and the fallback.
vulkan_filter_chain_texture keeps its size as one VIDEO_SCALE_PACK
word, so the driver's texture, framebuffer and history sizes pass
through as words. slang_pass_get_output_size(),
slang_framebuffer_set_size() and the texture-size semantic builders
take the words they were unpacked from.
The Mutter path in dispserv_x11 (17c7c6a) recognised XWayland only by
RandR output names starting with XWAYLAND. That holds for older
XWayland and for compositors that do not name their outputs, but under
Mutter XWayland takes each connector's name - DP-1, HDMI-1, Meta-0 -
like any X server. On a GNOME desktop the check never matched, the
XRandR path ran, and the Screen Resolution list stayed XWayland's:
every mode at the desktop's current rate, no 60 Hz on a 240 Hz panel.
XWayland 23.1 and later advertise the XWAYLAND extension, which is
what xrandr tests for its "running against an Xwayland server"
warning. That is now checked first, the name prefix kept as the
fallback for older XWayland. A real X server has neither, so it still
never talks to D-Bus.
Regression test: a new mutter_displayconfig case runs dispserv_x11 on
the XWayland of a real headless Mutter (one 2560x1440@60 virtual
monitor, output named Meta-0) and requires Mutter's list - one mode at
exactly 60.000 Hz - rather than XWayland's CVT list. Against the
previous dispserv_x11 it fails with XWayland's 33 entries; mutter joins
the step's packages.
struct vk_texture keeps its size as one VIDEO_SCALE_PACK word. The
readback staging check compares it with the viewport's word directly;
elsewhere the halves are read where Vulkan wants them apart.
samples/input/winraw_abs_pos is cross-built with mingw-w64 and run under
Wine, and no workflow named it, so the job for unclaimed samples picked
it up with only build-essential installed and failed on a missing
compiler. The input job now runs it, installing mingw-w64 and Wine when
the runner lacks them, which also takes it off the unclaimed list.
The unclaimed job installs the same two the first time a leftover
sample's Makefile uses them, so the next Windows sample without a home
runs there instead of failing for want of a compiler.
Left to probe with no display, the context the GL drivers find depends
on the runner rather than on the code: a build with SDL2 brings up its
offscreen EGL backend on llvmpipe, and a VM with a DRM device can bring
up KMS. The step then runs a live driver instead of the refusal it is
there to test, and its exit status follows the image.
The step sets video_context_driver to null, so both drivers are handed
the null context on every host, and requires the log to name it before
checking the exit status. With the teardown guard in gl3.c reverted the
glcore run still faults with 139.
switch_video_t keeps the size o_size shows the frame at as one
VIDEO_SCALE_PACK word. Its width/height, and the context's
width/height, native_window and resize, were written and never read;
they go, and the context crops from the size word it already has.
check_window's locals are declared C89-first.
The Vulkan context keeps its swapchain size as one VIDEO_SCALE_PACK
word, and khr_display its own size likewise; the retained backbuffer
and the window size compare and copy the word rather than repacking
its halves.
On GNOME's Wayland session RetroArch's X11 driver runs on XWayland.
XWayland lists every mode at the desktop's current rate - a 240 Hz
panel shows no 60 Hz mode - and a "switch" only scales the fullscreen
window; the real modes belong to the compositor. The Wayland driver
had no resolution list at all. Under Mutter no X11 or Wayland protocol
reaches the real modes; its org.gnome.Mutter.DisplayConfig D-Bus
interface (what GNOME Settings uses) does.
gfx/common/mutter_displayconfig.c (HAVE_DBUS only) reads
GetCurrentState and applies ApplyMonitorsConfig:
- The list is the target head's real modes, one entry per size and
rate (a fixed and a variable rate twin are merged), sorted, the
scanned-out mode current.
- A switch picks the listed mode of that size nearest the rate, the
same rule as the XRandR path; a zero size keeps the current one (the
refresh rate autoswitch). It is applied as a temporary configuration:
not saved to monitors.xml and no "keep these settings?" prompt.
Every other monitor keeps its mode, position, scale and transform;
every monitor keeps its colour mode (HDR), RGB range and
underscanning; a mirror switches all its monitors or none; heads to
the right of or below the switched one move with its new size; the
layout mode goes along only where Mutter allows choosing it.
- The target head is the connector (wl_output v4 name), else the head
under the RetroArch window (XWayland root coordinates are Mutter's
logical layout), else the monitor index, else the primary.
- Each call opens and closes its own private session bus connection,
bounded by a 2 s timeout. With no bus address and no
$XDG_RUNTIME_DIR/bus it does nothing, so no bus is ever autolaunched.
Backwards compatibility - every existing path behaves as before:
- Builds without HAVE_DBUS compile exactly the previous code.
- dispserv_x11 only asks Mutter when the output is XWayland's
(XWAYLAND<n>); a real X server never talks to D-Bus. On XWayland
without Mutter (KDE, sway, ...) XWayland's own list and switch are
used as before. Only when Mutter answers is its answer final.
- dispserv_wl gets set_resolution, get_resolution_list and get_flags
in HAVE_DBUS builds. A new display server flag,
DISPSERV_CTX_NO_RESOLUTION_LIST, lets it report "nothing to list
right now": set when Mutter was not on the bus at init, and
video_display_server_has_resolution_list() then answers false, so
the Screen Resolution entry and the refresh rate autoswitch stay off
on non-GNOME Wayland exactly as before. No existing server sets the
flag, and its bit is not DISPSERV_CTX_MODELINE's.
Checked against a real headless Mutter 46: GetCurrentState parses and
ApplyMonitorsConfig is accepted.
Regression test samples/gfx/mutter_displayconfig (ASan + UBSan, new
Linux-samples-gfx step): mock_mutter.py carries Mutter's signatures and
its ApplyMonitorsConfig checks (serial, connectors, modes, scales,
every lit monitor configured) on two monitors - a 240/120/60 Hz HDR
panel and an underscanned one beside it. Cases: no session bus and a
bus without Mutter (unavailable, nothing autolaunched); the list, a
60 Hz switch with every other setting carried over, rate only, whole
hertz, a smaller mode with the neighbour moved, another head by
connector, unlisted size and rate refused without an apply; dispserv_x11
on a real Xorg dummy server with Mutter on the bus (XRandR list, Mutter
never asked); dispserv_x11 on Xwayland under a headless Weston with
Mutter (Mutter's list and switch) and without (XWayland's list as
before); dispserv_wl with Mutter (list, switch, not flagged) and
without (flagged, no list, as before).
dispserv_x11 left set_resolution and get_resolution_list NULL, so
video_display_server_has_resolution_list() was false on X11: the
Settings > Video > Output > Screen Resolution entry was hidden, and
the automatic refresh rate switch and its PAL threshold never ran.
Both are implemented on XRandR 1.2, gated on HAVE_XRANDR like the
rest of the RandR code:
- get_resolution_list lists the modes of one head - the one under
the RetroArch window, or under the Xinerama screen
video_monitor_index names, else the primary output, else the first
lit one (the rule modeline_open already uses). The field rate is
computed from the timing with interlace and doublescan accounted
for, the same size and rate listed under two names is merged into
one entry, exactly the scanned-out mode is marked current, and the
list is sorted by size then rate.
- set_resolution switches that head to the listed mode of the asked
size nearest the asked rate (within half a hertz, or matching the
whole-hertz menu label), progressive over interlaced. A zero size
keeps the current one, which is how the refresh rate autoswitch
asks. The framebuffer grows before the crtc takes a larger mode
and shrinks to the lit area after, so each step is a configuration
the server accepts; other heads are left where they are, the DPI is
kept, and the root window geometry is read for the current size
rather than Xlib's cached DisplayWidth(). X errors are trapped, a
failed switch puts the framebuffer size back, and the kept refresh
rate is invalidated. An unlisted size is refused and changes
nothing. No libm: the driver still links without it.
Regression tests:
- samples/gfx/display_servers_x11_live (Xorg dummy server, ASan +
UBSan, already run by Linux-samples-gfx): the list must describe
the lit head with one current entry, in order, without
duplicates; switches to a smaller mode, a larger mode, a rate only
(size 0), a whole-hertz rate, and back must land on the wire with
the framebuffer following; an unlisted size must fail and change
nothing. Fails on the old tree (callbacks NULL).
- samples/gfx/display_servers (stubbed RandR): every failing query
and a disconnected head give an empty list and a refused switch,
with no crtc configured, no leak and no bad free.
The orbis image is Alpine 3.15, whose musl cannot load the runner's
Node 24, so checkout and upload-artifact run on the host and the
dependency install, build and toolchain version steps run inside the
image through docker run.
opendingux, mali and videocore keep their size as one VIDEO_SCALE_PACK
word and hand it back as it is. vivante, vita and orbis stored a size
nothing read; those fields go, with vivante's unused default size and
EGL format and mali's unused set_video_mode locals.
Without pkg-config, check_val falls back to a plain link test and takes
the include flags from the directories it is given. check_lib now accepts
a list of include subdirectories, each of which has to exist for the
library to be enabled, and PipeWire passes both pipewire-0.3 and spa-0.2.
A system that has the library but not the headers ends up with PipeWire
disabled rather than enabled with no include path, and PIPEWIRE_STABLE
takes its 1.0.0 floor from PW_CHECK_VERSION in the headers.
A CI lane configures with PKG_CONF_PATH=none and builds to completion,
checking that PipeWire is enabled with both include directories.
The main thread publishes the resume mark as one word, a sequence and
an armed bit, and is its only writer; the consumer records the
sequence it acted on instead of clearing the flag, and takes the
position between two reads of the mark that agree. A pause and resume
landing while the consumer reads can no longer have their mark
cleared unseen, so the resumed audio keeps its ramp.
samples/audio/pipeline_fade_mark runs the shipping publish, pending
and clear functions with a pause and resume delivered in every gap of
the consumer's read.
d3d11 records the back buffer size its copy was made at in one word,
as the d3d12 driver does. Both font renderers hold the cell every
glyph is rendered into the same way, so the atlas layout, the copy
clamp and the blank-glyph clear all read their axis from one place.
onContentRectChanged() publishes the size as one VIDEO_SCALE_PACK
word, so the Vulkan context never pairs one change's width with
another's height, and check_window() takes the changed flag with an
exchange, so a change raised while it reads is not cleared unseen.
The context keeps its own size as one word too.
samples/gfx/android_content_rect runs both shipping functions on the
host, taken out of their files at build time, with the UI thread's
change delivered in every gap between the reader's operations.
The report handler publishes an absolute report's x and y as one
VIDEO_POS_PACK word, so winraw_poll() on the main thread reads a
position a report gave rather than one report's x with the next
report's y.
samples/input/winraw_abs_pos runs the real driver under Wine and
delivers a second report between poll's loads.
With no display only the null GL context is found, and the glcore
driver refuses it. Its teardown now touches the vertex array object
only when one was made, so the refusal reaches the frontend as a
failed video init and an exit with an error, as the gl driver's does,
instead of a call through a GL entry point the null context never
resolved.
The llvmpipe harness job starts the build it tests with no display,
directly and threaded, on both GL drivers, and requires exit status 1.
Explore already reads "tags" from database entries, but c_converter had
no mapping for it, so a tags field in a DAT never reached the .rdb.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
Each readback slot records the extent its copy was taken at in one
word, and the SDL2 font holds its atlas texture's size the same way,
so the glyph scale and the upload stride read their axis from one
place.
The two padded structs in vulkan's uniform block stay as they are:
the shaders read that layout.
The deepest shortfall and the ring's fill in that same callback share
one atomic word, in frames, 16 bits each: one store records both and
the teardown log reads them together.
coreaudio_write runs underruns of different depths and checks the
word keeps the deepest with its own fill.
With the mouse grabbed, the X input driver takes the window's centre
from the size the event pump records and sends the warp back to it
without waiting: the next poll's pointer query follows the warp on
the connection, so it reads the warped position. A poll costs the
keymap query, the pointer query and the warp. The recorded size is
one word, written by the pump and read by the poll on the runloop
thread; the server is asked for the size only for a window the pump
has not seen.
The threaded video harness moves the pointer from a connection of its
own between polls, directly and under the wrapper, and requires the
exact motion from each poll and no more than those three requests.
alive, focus, presentable and has_windowed are bits of one atomic
word the video thread publishes with a single store after each frame.
The window answers lane drives each answer on its own through a
swapped driver and reads it back on the main thread, so an answer
landing in another's bit, or one that does not move, fails.
switch_video_t holds the frame size the scaler was last built for,
the window size the hardware scaler asks the compositor for, and the
menu texture's arriving and on-screen sizes one word each, so the
rebuild test is a single comparison and the two nwindowSetDimensions
calls read their pair from one place.
Five declarations in this file move to the top of their block, two of
them out of a for statement, so it builds as C89.
The driver keeps the size the core renders at in src_dims, and
sunxi_setup_scale takes it as that word. Four menu fields go with the
change: menu_rotation, menu_width, menu_height and menu_pitch were
declared and never touched, and menu_active is the only one the
driver reads.
The two __disp_ rect types above stay as they are: they are the
kernel's disp ioctl layout, not ours.
A dumb buffer's pixel size sits beside the stride the scanout walks,
and the driver holds the surface's internal size, the mode the
display is running and the size the core last handed over the same
way. drm_surface_setup takes the size as the word its menu caller
already had.
struct drm_rect went with them: nothing has ever used it.
The copy of the last presented back buffer records the size it was
made at, and the software framebuffer the size it was laid out for,
one word each, so each recreate test is a single comparison against
the size in hand.
The OutputSize pair in the uniform block stays two floats: the shaders
read it as a float2, so its layout is not ours to change.
gfx/common/wayland_common.c is what both Wayland contexts call into
and was the one Wayland unit no lane compiled. It gets three, for the
plain build, the libdecor build and the backport build, each carrying
the declaration-after-statement gate the PSP lane already uses.
The libdecor lane defines HAVE_DYLIB alongside HAVE_LIBDECOR_H: the
function pointers that half calls through are declared under both, so
a lane naming only the first compiles a file with no libdecor calls
left in it.
struct vk_vertex carries its colour as R8G8B8A8_UNORM, packed by
rgba8_pack() in gfx/common/rgba8_pack.h: a vertex is 20 bytes instead
of 32, and the menu's per-vertex bake packs with one SSE2 or NEON
sequence rather than four float stores. The font and quad paths pack
their colour once per string or quad.
samples/gfx/rgba8_pack holds the vector path and the scalar one to
the same reference: every 8-bit level, both sides of every rounding
boundary, saturation past either end, infinities and NaN.
The workflow runs it after the menu-mouse test, installs libxtst6 for
its XTEST key presses, and also triggers on led/drivers/led_overlay.c.
The overlay_visibility_bounds_test step in the tasks samples workflow
describes what the test now covers: input_overlay_alpha.c's LED hiding
and its per-poll alpha pass.
The X Vulkan context opens a second connection to the server for the
swapchain and hands it to the XCB surface; the window stays the
frontend's. A software WSI writes each frame into its connection as
image data, and on the frontend's connection every request the
frontend makes - the input driver's keymap and pointer queries, the
event pump - waited behind that frame on whichever thread presented.
The connection closes after the swapchain and surface; if it cannot
be opened the frontend's is used. vulkan_surface_create() takes the
XCB connection itself.
The threaded video harness counts the requests on the frontend's
connection over 60 presented frames with the input poll left out and
requires none. The compile matrix builds the context with XCB, as
the real build does.
The reader thread asks for the rate on g_x11_dpy, the connection the
requests are counted on. After the switch its ask can still be reading
the server when the count starts, or can withdraw a rate it kept across
the pump's invalidations, and either lands requests inside the count.
The count now starts only once an ask that began after the pump has
finished, so every earlier ask is done and the kept rate is final.
The X display server reads the refresh rate from the server once per
display change and answers from memory in between, so a core polling
the throttle state every frame, or the menu showing the rate, sends
nothing to the server. On g_x11_dpy it selects RandR screen, crtc and
output changes on the root window; the event pump drops the kept rate
on each of them, and the display server drops it after its own mode
sets. The rate is kept only once the pump drains that connection, and
a reader that raced a change withdraws what it kept. A NULL output
from XRRGetOutputInfo is skipped, and opening the display no longer
writes the display server's flags, which two threads ask at once.
samples/gfx/x11_refresh_rate runs the real pump and display server on
an Xorg dummy server against mode switches from another client, with
a second thread asking throughout, under ASan+UBSan and TSan.
A driver's device-clock estimate and whether there is one share a
single int, AUDIO_CLOCK_PPM_NONE until the first estimate: one store
publishes it, one load reads it, and a reset landing inside a read can
no longer pair an old flag with the reset's value (ASIO, JACK, Pulse,
CoreAudio, WDM-KS, 3DS DSP).
asio_clock runs the driver's own publish, reset and reader, taken from
asio.c at build time, with a reset between the reader's loads;
live_drivers checks Pulse and ALSA report no clock before any audio.
gfx_ctx_wayland_data_t holds the surface's size, the buffer behind it
and the size to restore when the window floats again one word each,
and the latched xdg_toplevel configuration holds the size the
compositor asked for the same way. Both configure paths scale one
axis into each half of the buffer word in a single pack.
Four declarations in this file move to the top of their block so it
builds as C89 under the libdecor path as well as without it.
output_info_t carries the mode the output is running in dims, which
is the word both readers of it were building anyway. The physical
size the geometry handler also stored goes with them: nothing read
it, and the display server keeps its own copy for the DPI it reports.
The browser's canvas size is stored as one packed word, so a read on
the main thread can no longer pair a width from one resize with a
height from another; platform_emscripten_get_canvas_dims returns it
and platform_emscripten_set_canvas_size takes it. The EGL and WebGL
contexts keep their framebuffer size the same way. The JavaScript
side keeps its two numbers.
A content rate slightly past what the display presents at the chosen
interval - a 60.0988 Hz core at swap interval 2 on a 120 Hz panel, or
60 Hz content on a 59.94 Hz VRR ceiling - is paced by vsync with audio
skewed as it is without VRR, rather than forcing the loop nonblocking.
Only a content rate beyond audio_max_timing_skew keeps its own rate
with vsync off.
The decision is runloop_sync_plan_for() in runloop.h, pinned by a
table and an invariant sweep in samples/runloop/pacing. Fixes#19600.
With display sync off a repeated present is not held for another
display frame, only presented again unpaced. set_nonblock_state now
takes the interval to 1 in nonblocking mode, as Vulkan's does.
x11_check_window() and x11_has_focus() run on the frame path and must
not wait on the server. Keyboard focus is tracked from FocusIn/FocusOut
on the window (grabs ignored), so the GLX, EGL and Vulkan X contexts
select FocusChangeMask. The window size is recorded when the window
maps, since without a window manager no ConfigureNotify follows; it is
cleared with the window, so a new window never reports the old size.
The main thread no longer shares a round trip with a software Vulkan
WSI presenting a whole frame through the same connection.
samples/gfx/x11_frame_poll runs the poll on Xvfb and requires it to
send no requests, with size and focus still following the window.
The display mode's pixel size sits beside the stride it scans out
with, and the driver holds the frame the core hands over, the size it
presents at, the menu frame and an overlay image's size the same way.
An overlay's tex_w and tex_h stay floats: they are a fraction of the
image, not the image.
tools/__pycache__ is written by the Python checkers whenever they run,
so a copy tracked in the tree shows up as a modified file after any
run of them, and the one that was there was built by a different
interpreter version than the tree's lanes use. .gitignore keeps it
out.
The members a change removes, minus the ones something still declares,
against every reader in every language the tree is written in. A
reader the Linux compilers never see - an Objective-C driver, anything
behind a platform #ifdef - is named by file and line here rather than
by whichever platform job happens to build it.
The checker carries fixtures of its own: a member removed with its
readers, one left behind, one that merely moved, one whose name
another struct still declares, a rename, a name appearing only in a
comment or a log string, a declarator list, and a name too common to
reason about.
ui_cocoa reads and writes settings->uints.window_position_pos, so a
window placed on a display left of or above the main one keeps the
negative origin AppKit gives it.
runloop_msg_queue_off_main() is built only with HAVE_THREADS, like its
callers. WASAPI's error-string helper is built only with
HAVE_MICROPHONE, the one path that calls it, and the driver casts
GetProcAddress straight to its function-pointer types, so it builds in
the Windows C89_BUILD lane. The ffmpeg core's slot helpers for the GL
path are built only with OpenGL, and its channel-layout declaration
sits at the top of its block. The IIR filter's RIAA CD case marks its
fall into the high shelf, and the ffmpeg_live sample's GL frame
callback has the libretro signature, which GCC 14 and later require.
The compile matrix installs mingw-w64, which turns its Direct3D lanes
on, and compiles WASAPI with the C89 flags both with and without the
microphone, treating unused functions as errors. That lane compiles
for real rather than with -fsyntax-only, which never reports unused
static functions.
ozone_draw_icon and xmb_draw_icon already take video_dims and read the
screen height from it; the width and height they were also handed
are gone. xmb_draw_text, xmb_draw_dark_layer and the MaterialUI
scrollbar helpers take the packed size their callers hold instead of
its two halves, and xmb_frame reads video_info->dims where it rebuilt
it from its own halves. The locals that only fed those arguments go
with them.
The module's window state keeps its origin and its frame size one
word each, and a pos_set flag says whether an origin has arrived yet,
so the case where Windows chooses the position is a flag rather than
a sentinel coordinate sitting in the same field as a real one.
Carrying the origin between the setting and the window is now a
single assignment either way.
window_position_pos holds where the window sits in VIDEO_POS_PACK's
layout, and its two config keys are signed rows over the halves of
it, so a display left of or above the primary one saves the negative
coordinate it actually has and reads it back the same way. The keys
keep their names and a non-negative position carries over from an
existing config unchanged.
Each pass holds the frame size it last built vertices for in
last_dims, and the ring of previous-frame sizes is one array of those
words rather than a pair of parallel arrays, so carrying a size
between the ring and pass zero is a single assignment.
LinkInfo carries the size of the texture a pass renders into as
tex_dims, and set_pass_size in both d3d9 chains takes that word in
place of two axes, so the test for whether a pass has to be
respecified is a single comparison against the size the caller
computed.
camera_driver_t init takes the requested frame size as one word, which
the frontend packs from the setting or the core's request. The FFmpeg
driver keeps its requested and target sizes that way, V4L2 the size
its device negotiated, and the PipeWire format builder takes the
size packed. The FFmpeg live test checks that frames arrive at the
size asked for.
companion_rect_t is an origin packed with VIDEO_POS_PACK and a size
packed with VIDEO_SCALE_PACK, and companion_dock_state_t holds a dock
row's position and size the same way; the rows read and write as
before. companion_place_window takes the needed and minimum sizes
packed. The dock layout works in signed locals and packs each
rectangle once, clamping the content width and a pane's height to
zero before they are stored. The Win32, Cocoa and Qt companions fill
and read the packed rectangles.
The soft scissor keeps its rect as an origin word and a size word, so
the rect the menu hands scissor_begin is stored as the word it
arrives in. The render chain holds the frame size it last built
vertices for and the size of the texture behind them the same way,
and the font's atlas mirror does too, which makes the respecify test
a single comparison against the size the atlas reports.
overlay_t carries the size of the texture behind it in tex_dims, so
the menu overlay's size test in every d3d driver is a comparison
between the word the frame arrives as and the word the texture was
made from, rather than a pair of axis comparisons either side of it.
The d3d9 HLSL and Cg font renderers start color_dark at zero, as d3d8
does. It is only read by the drop-shadow pass, under the same has_drop
test that sets it, but once the quad emitter is inlined GCC no longer
ties the read to that test and reports -Wmaybe-uninitialized at -Os.
ct_anim_first is built only where the decode threads that ask it
are. The modeline goldens and the EDID parse test drop the engine's
log lines through a do-nothing function rather than a macro that
swallowed its arguments, so the buffers the engine formats them into
are used, and the golden tool needs no variadic macros.
set_alpha, tex_geom and vertex_geom compare the index with the page's
count and return with no page loaded; the "&overlay[image] is NULL"
tests they had can never be true.
tools/overlay_setter_bounds.py now reads every driver with an overlay
interface, lets a stub setter with an empty body pass, and fails a
driver in gfx/drivers that defines the setters without being in its
table.
overlay_t keeps the quad last written to the image's vertex buffer
(vert_sent). The draw builds the quad as before and locks and fills
the buffer only when it differs, or when the buffer is new; a failed
lock leaves it to be written next frame, and a NULL lock is no longer
written through. D3D9 Cg draws from user memory and is unchanged.
tools/overlay_setter_bounds.py fails a D3D8 or D3D9 HLSL overlay draw
that locks its buffer before comparing the quad with vert_sent.
video_modeline_t lists its mode under one packed size beside the
refresh label, the EDID standard timings hold theirs the same way,
and modeline_get and modeline_set_user_mode take the requested size
packed. The timing fields stay per axis. The SDL display servers pass
the packed size through to their apply, and the Windows, X11, KMS and
VideoCore servers fill and read it where they list and set modes.
The goldens are unchanged.
vulkan_render_overlay() takes one uniform range for the MVP and one
vertex range holding the page's whole vertex array, binds the vertex
buffer once, and draws each image's quad at its offset (firstVertex
4 * i). Each image keeps its own descriptor set for its texture, still
written in batches of 16.
GFX_INSTRUMENT counts the pages drawn and the buffer ranges they take
(overlay_draw, overlay_draw_alloc); the threaded_video overlay page
lane holds that to two ranges per drawn page on drivers that report
it, which the Vulkan validation leg runs.
parseSwizzleSelector() bounds its checking loop by the decoded length
as well as by the selector's size. The selector never holds more than
that many entries, and the decoded length is visibly at most
MaxSwizzleSelectors, so the compiler can see every index stays inside
the selector and field-set arrays.
TokenizableIncludeFile's notifyActivated() and notifyDeleted() are
noinline under GCC and Clang. They are the only overrides of those
tInput hooks, so GCC speculatively inlines them into every pushInput()
and popInput() call and then bounds-checks the guarded body against
the smaller input objects those calls push. They run once per include.
The CRT consumer keeps the requested and last-applied core sizes as
one word each and passes sizes packed from crt_switch_res_core down
to the aspect switch. Two framebuffer fields nothing read are gone,
as are the aspect switch's two unused mode-size parameters. The
harness checks the size published for each switch.
downscale_box takes the source size packed and hands the result back
the same way; a source too large to pack goes straight to the sinc
stage. task_image_process no longer returns the decoded size, which
it already stores in the image and no caller read.
input_overlay_set_scale_factor and input_overlay_auto_rotate_ take the
packed output size their callers already hold, and
task_overlay_load_desc takes the base image's size packed once by the
loader.
The GDI driver's remaining size pairs join the ones already packed on
gdi_t: a loaded texture carries its pixel size in gdi_texture_t::dims,
and the font renderer's atlas mirror and tint scratch DIB each keep
their capacity in one word. Both DIBs are created from the same word
their extents are read back out of, so the surface and the size
recorded for it cannot drift apart, and the pixel-count multiplies go
through VIDEO_SCALE_AREA so a buffer size widens to size_t before the
bytes-per-pixel factor.
Either helper refuses a size past what a word holds rather than
recording a capacity smaller than the surface it describes.
The binary dump of a double copies its bits into an unsigned 64-bit
word with memcpy rather than reading the double through a long long
pointer, which broke strict aliasing and tripped -Wstrict-aliasing at
-Os. The word is unsigned so shifting its top bit out is defined, and
the loop only shifts it, so each printed digit is the matching bit of
the value.
companion_thumbs keys its cache, requests, animations, deliveries and
scaler on one VIDEO_SCALE_PACK size, and refuses a size with an empty
axis. The Win32, Cocoa and Qt companions carry it the same way through
their own pane and pixmap helpers. A decoded image that does not pack is
not scaled rather than scaled from a clamped size.
A page's sprites live in a CPU copy (overlays.shadow): set_alpha,
vertex_geom and tex_geom write their sprite there and mark the page
dirty, and the overlay draw uploads the whole page in one map when it
is - a WRITE_DISCARD on D3D10/11, so the frame before keeps the buffer
it drew from. A page load resets the copy and leaves the upload to the
first draw. The setters still refuse an index off the page.
tools/overlay_setter_bounds.py fails a D3D10/11/12 setter, or the
helper it goes through, that maps the sprite buffer.
rarch_softfilter_new, rarch_softfilter_get_output_size and
rarch_softfilter_process take the size packed, the wrapper keeps its
maximum as one word, and video_thread_filter hands the size back the
same way. The plugin interface in softfilter.h keeps its two axes.
ozone_alloc_node() zeroes the node's packed attr word in one store
instead of going through the height, sublabel and wrap setters, each of
which reads the word first. Bits outside those three fields are now
defined too, so the bitwise node copy in ozone_snapshot_visible() never
carries indeterminate data.
companion_core_browse_name() bounds the volume label with a precision
in the drive-name format, so the letter suffix always fits in the
64-byte buffer. Real volume labels are far shorter than that bound.
video_viewport_get_scaled_aspect2 and video_viewport_get_scaled_integer
take the packed size; the frontend hands them vp->full_dims as it is,
and gx passes its own full_dims.
The desktop companion's saved window geometry and the camera's
requested capture size were the two width/height pairs left in
settings_t. Six unsigned fields, three words.
The companion's geometry is a rectangle four UI drivers agree on --
Qt, win32, Cocoa and the two companion test harnesses -- written as
four axes at a time and read back as four. It is an origin and a size
now. Its origin packs as a signed word even though nothing negative
reaches it: both writers clamp at zero and every row is bound to
32767, so VIDEO_POS_PACK's range covers it with the sign intact
rather than laundered.
The four config keys are unchanged, which matters more here than
usual: tools/companion_qt_persist_test.sh reads
desktop_menu_window_width and _height straight out of the retroarch.cfg
RetroArch writes on quit, and companion_win32_test / companion_cocoa_test
assert on the saved values. The four rows move from S_UINT_EX, which
binds by field name, to S_UINT_AT_EX with a half tag, so they are
literal in configuration.c now; S_UINT_AT_EX is added to the one label
pass in msg_hash_us.h that did not define it.
The camera pair needed none of that. No config key and no menu row
binds either axis -- it is read once, in retroarch.c, to ask the camera
driver for a size.
The video thread keeps the float bits it last handed the driver for
each overlay image (alpha_applied, its own) and an apply passes on
only those that differ, so one press on a page is one driver set. A
page load marks them all unknown and the next apply sets every image;
without the array every image is set at every apply.
The overlay page lane checks one change is one set, a repeated value
is none, and a new page gets all of its images.
input_overlay_alpha_pass() works out every image's alpha for the poll
- opacity, a pressed desc's highlight, 0 for an LED image that is off -
into scratch and hands the driver only the images whose alpha differs
from the last one it was given. A page load, through load() or
load_textures(), forgets what was sent, so the next pass sets them
all. The per-image sent and scratch arrays are one block sized for the
pack's largest page; without it every image is set every pass.
The LED hiding moves with it into input/input_overlay_alpha.c, and the
immediate hide of an LED going out goes through the same record, so a
light that goes out and back on between two polls is shown again.
overlay_visibility_bounds_test pins no-change, press/release, forget,
opacity change, that LED case and the no-block fallback.
The statistics overlay's text and the threaded wrapper's copy of it are
both VIDEO_STAT_TEXT_SIZE (1664), room for every section at once. Each
line goes through video_driver_stat_appendf(), which keeps the text
terminated within the buffer and takes nothing once it is full.
The threaded video harness's stat text bounds lane appends past the
buffer and checks the length, the terminator and the byte after it.
The window's own size and the ceiling its auto-resize honours were
four unsigned fields in settings_t. Both pairs are one word now, using
the half-aware binding the custom viewport got: the config keeps
video_windowed_position_width/_height and video_window_auto_width_max/
_height_max as four keys and the menu keeps four rows, each carrying
which half it is.
The four menu rows move from S_UINT, which takes a field name, to
S_UINT_AT_EX, which takes an offset -- that is where the half tag
rides. Two of the blocks that include this header define only the
macros their rows used, so S_UINT_AT_EX is added to the enum pass in
msg_hash.h and the label pass in msg_hash_us.h. The auto-resize pair
was config-bound through the S_UINT pass, which binds by field name;
with the field gone those two rows are literal in configuration.c like
the size pair beside them.
win32_common is the reason this pair was worth packing. It copied all
four axes out of the window, then subtracted the border from the width
and the border, title bar and menu bar from the height -- four writes
to a stored pair, one axis at a time, with the struct holding a
half-adjusted size in between. The chrome comes off two locals now and
the pair is stored once.
Not touched: window_position_x/y. They launder a negative coordinate
through an unsigned field -- SDL and win32 both hand over a signed
position and cast it in, and sdl3_common casts it back out -- so
packing them is a question about what the config file should hold for
a window on a monitor left of the primary, not a mechanical change.
UIntRadioButtons walks a setting's range with unsigned counters
instead of a float, which stops advancing past 2^24. The bounds go
through qt_setting_bound(), negative bounds clamp to zero, a step
below one is one, and the walk stops before it would pass the maximum
or wrap. At most QT_RADIO_BUTTONS_MAX buttons are made, so a row with
no enforced maximum lays out a bounded list instead of hanging.
video_viewport_settings_t held x, y, width and height as four members
while video_viewport_t next to it held the same rectangle as two. It
was the one pair in the tree the campaign had been leaving alone, with
a comment on the struct saying why: the menu and the config bind a
setting by address, and a packed half has no address to bind.
So the binding learns about halves. A config row carries CFG_HALF_HI
or CFG_HALF_LO and its ptr addresses the whole word; every read and
write of a uint or int row goes through cfg_uint_get/set and the
signed pair rather than dereferencing ptr. A menu row carries the same
in SD_FREE_FLAG_PACKED_HI / _LO, seeded from two spare bits in the
descriptor's value_offset -- settings_t is nowhere near 2^30 bytes --
and setting_uint_get/set and the signed pair stand in front of
value.target for the generic paths: the typed-value setter, the
default left and right handlers, and setting_reset_setting. Rows
without a tag read and write exactly as before.
The config file is unchanged: custom_viewport_x, _y, _width and
_height are still four keys, so a config written before this still
loads.
x and y went through the uint rows before, with a cast, because a
whole-word store round-trips a negative fine. A half store does not --
the unsigned accessor clamps, so -100 would have come back 0 -- so
they move to the int rows, where the half accessors sign-extend
through VIDEO_POS_X/Y.
Most of the consumers get shorter: video_driver.c and rgui.c were
packing these four into a video_viewport_t's two words and unpacking
them back, and the publish of the seqlock'd parameters was doing the
same. Those are word copies now.
Checked by exhausting both accessor pairs over the ranges the setting
rows allow -- 50015 size cases and 53865 origin cases, including
negative origins -- confirming each axis round-trips and that writing
one never disturbs its partner.
A setting's range is float, and a spin box with no enforced maximum
falls back to INT_MAX; that ternary is a float, and 2147483648.0f
converted back to int is undefined - at run time on x86 it is
INT_MIN, which collapses the box to a single value of -2147483648.
The resync in paintEvent then never matches the stored value and
re-sets it on every paint, repainting without end (Auto-Shader Delay,
SaveRAM Autosave Interval).
UIntSpinBox, IntSpinBox and SizeSpinBox now take their bounds through
qt_setting_bound(), which clamps in double and gives an unenforced or
NaN bound the widget's default. Their paint-time resync compares
against the stored value clamped to the box's range, so a value the
box cannot hold is shown at the nearest bound once.
Off mobile the two portrait-bias slots were a copy of the two
landscape ones: published every time the parameters changed, loaded by
every read, and never looked at. Both sites that reach for the portrait
pair pick it on the window being taller than it is wide, and both sit
inside #if defined(RARCH_MOBILE).
They are not published off mobile now, and the pair sits last so every
other slot keeps its index on both platforms. Eight slots on desktop,
ten on mobile.
The two fields of the snapshot struct are guarded the same way. Off
mobile nothing assigns them, so a reader added outside the #if has to
fail the build rather than pick up whatever the stack held -- which is
what it would have done had the fields stayed.
On packing the bias pairs into one slot each, which is the other way to
spend these four slots: measured rather than assumed, and it does not
work. The consumers are (int)(padding * bias) and a roundf of the same
shape, so the only question is whether a sixteen-bit bias ever moves
the pixel they land on. Over the two-decimal values the menu and its
typed entry can produce, against paddings from 1 to 8192, it moves
15382 of 827392 -- including at bias 0.53 -- and 1.6% of six-decimal
values, the precision a config file keeps. A pixel of viewport offset
is not worth two slots.
hub75_t held the panel chain's own size, the core frame's, the menu
frame's, and where the last frame landed on the panel as an origin and
a size; hub75_overlay_t held an overlay image's size. All five pairs
are one word each now.
The panel width is the row stride, so it is read three times per pixel
in hub75_refresh_once and once per pixel in the overlay blend. Both
take it into a local once rather than through the macro each time.
Left as they are: the overlay's tex_w/tex_h and vert_w/vert_h, which
are normalised 0..1 coordinates rather than sizes, and three blocks of
function locals that never leave the frame they are computed in.
A pack names the image that shows an LED with overlayN_descM_led = K
(1-based, as ledN_map counts): under led_driver = "overlay" that image
shows while the core has LED K lit, and a pack that names any LED image
is not mapped at all. For a pack that names none, ledN_map still picks
slots of the page, but only the image of a "nul" button answers to it:
a desc that does something when pressed is never an LED, so a config
made for an LED overlay leaves a gamepad pack's controls where they are.
The LED driver reports the core's LEDs (input_overlay_set_led) and
input/input_overlay_leds.c decides what they hide. The per-slot
visibility array, its enum and the parameter threaded through the
alpha and poll paths are gone.
overlay_visibility_bounds_test now builds input_overlay_leds.c with
the LED driver and pins the map/named/off cases and the page bound;
overlay_led_hidden_test runs a mapped pack and a named pack under Xvfb
with Up held.
set_alpha, tex_geom and vertex_geom compare the index with the page's
count and return with no page loaded. load() refuses a failed calloc
and frees a page whose texture could not be made, so no size is left
without an array behind it; enable() sets its flag once rather than
inside a loop that never runs for an empty page, and full_screen()
takes a NULL page. tools/overlay_setter_bounds.py now reads d3d8.c.
runloop_check_state() read video_driver_get_output_dims() separately
in the overlay block and the ASPECT_RATIO_FULL block. Read it once
into the existing output_dims snapshot (now declared unconditionally)
and use it in both.
Resolve the TODO/FIXME above the FULL aspect block by documenting why
the rest of it stays as is: each block keeps its own last_dims since
they are gated independently, and the check is polled from the run
loop because video_driver_set_output_dims() runs on the video thread
under threaded video, where the blocking driver pokes done by
video_driver_set_aspect_ratio() cannot be issued.
No functional change.
The seqlock'd viewport parameters were fifteen atomic ints, walked in
full by the publish (a compare against each, then a store to each) and
by every read. The video thread reads them once per frame through
video_driver_update_viewport, and every shader bind reads them again
through video_driver_get_rotation_snapshot.
Ten now. The custom viewport was four slots holding one rectangle:
its own setting rows bound the origin to -9999..9999 per axis and the
size to 1..9999, so VIDEO_POS_PACK and VIDEO_SCALE_PACK cannot reach a
clamp, and it is two. Four bounded selectors -- scale_integer,
aspect_ratio_idx, video_scale_integer_scaling and
video_scale_integer_axis -- shared thirteen bits of one slot, with a
negative-array-size check that fails the build if any of those enums
grows past the field it goes into.
rotation and core_req_rotation stay a slot each and are deliberately
not narrowed. Both trace back to a core's RETRO_ENVIRONMENT_SET_ROTATION,
which a core may set to any unsigned, and shader_glsl and shader_gl_cg
pass the value to a shader uniform without reducing it -- so a field
too small, or a modulo four on the way in, would change what a shader
sees for a core that sets something unexpected.
The four bias floats keep a slot each. Two of them are a mirror of the
other two off mobile, which is real waste, but folding a pair into one
slot means sixteen bits an axis, and that is a quantisation rather than
a repacking; left alone here.
Checked by exhausting the flag word over all 1456 legal combinations of
the four selectors and the custom viewport over its range, both
round-tripping exactly, and by confirming the sizing check fails the
build when a field is made too narrow.
osk_textbox_x, _y, _w and _h are one rectangle held as four ints. The
whole of it is written in one place, when the text box is drawn, and
read in one place, by the pointer hit test that decides whether a tap
landed inside it. Two fields now: an origin in VIDEO_POS_PACK's layout,
because the box's x goes negative when a wide message pushes it off the
left edge, and a size in VIDEO_SCALE_PACK's.
The pressed-desc highlight in input_overlay_post_poll() asks the page's
visibility first: an image the LED driver has hidden (led_driver =
"overlay", ledN_map) stays hidden while its button is held, instead of
showing for exactly as long as it is pressed.
An overlay alpha is opacity times a desc's alpha_mod, which packs set
above 1 for pressed buttons. Drivers that pack it into a byte (d3d8,
d3d9 cg/hlsl, d3d10/11/12, gdi, gx, gx2, gxm, sdl2) now convert through
VIDEO_ALPHA_BYTE(), which saturates, so a pressed button no longer
wraps to a dimmer alpha than a released one. tools/overlay_setter_bounds.py
fails any driver that packs one without it; overlay_led_hidden_test holds
Up through XTEST under Xvfb and checks the hidden image stays off screen.
The statistics text is built in video_driver_frame_statistics(), a
VIDEO_NOINLINE helper called only while the overlay is shown, so
video_driver_frame carries none of its formatting.
The Scanline Sync helpers are built on every platform. When the sync is on, video_driver_init_internal() and a CRT mode switch read the output lines directly, so the first frame after startup or a mode change no longer spends time on the lookup. Only switching the sync on at runtime leaves the read to its first frame, and nothing is read while it is off.
video_driver_scanline_before_frame() reads the display's active and total lines itself, on its first frame and again after a video driver reinit or a CRT mode switch clears them, so nothing is fetched while Scanline Sync is off and each change is read once. An output size that cannot be read is retried about once a second.
The lookup is file-local to video_driver.c and built only with HAVE_D3DKMT, the one configuration with a beam position, and the display-change handler and d3dkmt_init() no longer touch it.
The menu turned its mouse off whenever any overlay was alive, so that
a press on the overlay is not also a click in the menu. An LED overlay
(led_driver = "overlay") is a page of "nul" buttons whose images are
lit and unlit - nothing on it responds to a press - yet it took the
menu's cursor away all the same: fullscreen with such an overlay shown
in the menu, no cursor was drawn and the mouse did nothing.
The desc loader now marks a page OVERLAY_TAKES_INPUT when any of its
descs acts on a press: every type but a button, and a button that
binds something other than "nul". The menu gives its pointer up only
to an alive overlay whose active page carries that flag, or whose own
pointer mode (input_overlay_pointer_enable) is on. struct overlay's
flags widen to 16 bits for the ninth flag.
overlay_menu_mouse_test runs the real binary under Xvfb in the overlay
on-screen workflow: with a "nul" pack the RGUI cursor must follow the
X pointer, with the same pack bound to "a" it must not, each with
threaded video off and on. It fails on the old code (no cursor with
the "nul" pack) and passes on the new.
configuration.c expands the single-source setting rows from
settings/settings_def_menu_filebrowser.h in every build, so the
default of menu_file_browser_extension_display names
MENU_FILE_BROWSER_EXTENSION_DISPLAY_ALWAYS even when the menu is not
compiled in. Include menu/menu_defines.h ahead of the HAVE_MENU gate;
it only carries enums and retroarch_types.h already pulls it into
menu-less builds, so builds with the menu see no change.
* Config: Convert Ozone 'Color Theme' setting to array
* Ozone: keep the color theme list in one place
The array conversion left the list of themes spelled out three times
- the identifier-to-theme table in ozone_set_color_theme(), the
identifier-to-label table in menu_setting.c, and the hand-written
'|'-separated values string in settings_build_menu() - all of which
had to be kept in step by hand for every theme added.
Move the list into menu/drivers/ozone_color_themes.h as an X-macro
and expand it in both places; the values string is now built from
the same list. The frozen legacy numeric table in configuration.c
stays separate on purpose.
While here:
- Left/right on a value that is not in the list (an out-of-range
number from an old config, a hand edit) did nothing, leaving only
the dropdown to get out. Step from the default theme instead,
which is what such a value renders as; the label shown for it is
now derived from DEFAULT_OZONE_COLOR_THEME rather than hard-coded.
- Find the color theme row to turn into ST_STRING_OPTIONS by its
enum rather than by assuming it is the first row of
settings_def_ozone_sidebar.h.
* Ozone: drop stray struct member, guard a NULL color theme
- Remove the unused 'unsigned draw_entry_delay' that the array
conversion's conflict resolution left in struct ozone_handle.
- ozone_set_color_theme() now strlcpy()s the identifier into
last_color_theme, so a NULL would crash. The frame path only runs
while the menu is alive, when video_info carries the pointer, but
fall back to the default identifier rather than depend on that.
---------
Co-authored-by: LibretroAdmin <[email protected]>
filebrowser_parse() backs every file picker, not just Load Content,
so Duplicates Only / Never also stripped extensions in the shader,
overlay, config, font and playlist pickers. There the extension is
often the only thing telling files apart (crt.slangp vs crt.glslp),
so apply the setting only to content lists (DISPLAYLIST_DEFAULT and
DISPLAYLIST_CORES_DETECTED, minus the overlay, image, video font and
playlist-collection pickers that also come through there). Say so
in the sublabel.
Also move the Always / Duplicates Only / Never strings in
msg_hash_us.h from after the multimedia group to just ahead of it,
beside the file browser group, and regenerate
tools/settings_table.reference for the new setting.
Follow-ups to the File Extension Display setting (#19611):
- Move the helpers out of menu_cbs_get_value.c, where they were not
value callbacks, and make them static next to their only callers:
the prepare step in menu_displaylist.c and the label formatter in
menu_driver.c. Only menu_file_browser_stem_length() stays shared,
declared in menu_entries.h; menu_cbs.h goes back to how it was.
- Replace the bare 0/1/2 entry states with
enum menu_file_browser_extension_state.
- Duplicate detection worked out every name's length-without-
extension again inside the qsort comparator, each time going
through path_basename() and its archive-delimiter scan. Work it
out once per entry and sort an array of {cbs, path, length}
instead, which matters in large ROM folders on slow hardware.
- Compare names with TOLOWER instead of strncasecmp, which nothing
else in the frontend relies on.
- Build the " (ext)" hint with offset-tracked strlcpy calls instead
of chained strlcat, which rescanned the buffer on every append.
- Put the rich-label condition in menu_entry_get() back to plain
cbs->action_label; every entry gets a label callback, so the extra
file_extension_state test never changed anything.
No change in what gets drawn.
Add a File Browser setting with Always, Duplicates Only, and Never modes. Duplicate detection uses the filtered view and preserves list order. Format only the drawn label so stored paths and menu entry paths remain unchanged.
gdi_t carried eight width/height pairs and struct gdi_overlay a ninth:
the core frame's size, the screen's, the window surface's, the DDB's,
the menu frame's, the menu compositing surface's, the capacities of the
two cached scratch DIBs, and an overlay's texture size.
Two of them are there specifically to be compared against a size and
acted on. bmp_dims is the clearest, and its own comment says why it
exists: it tracks the DDB's real size because comparing against the
core's frame size instead triggers a destructive DeleteObject +
CreateCompatibleBitmap every frame, racing WM_PAINT and producing
visible flicker. That comparison is now one word.
The scratch DIB slots said they kept width and height apart rather
than a pixel count "because BITMAPINFOHEADER cares about both". It
still does; both axes are still there, in one field.
Two "use this size, or fall back to that one" reads in the paint path
picked each axis independently. Both fields in each pair are only ever
written as a whole, so the fallback is one ternary over two words.
The overlay's zero-size guard is left per-axis: it exists to reject a
malformed overlay, and a word test would accept one whose width alone
came back zero.
Also corrects three comments still naming vp.full_width/full_height
and bmp_width/bmp_height, which went to single words earlier.
gl2_t held the output size, the streaming texture's size, the viewport
the last frame went out at, and the size of the retained backbuffer
copy as eight fields. All four are one word each now, which finishes
the GL family -- gl1, gl3 and the FBO chain have already been through.
tex_w and tex_h were the pair with reach: thirty-five sites, most of
them sizing a buffer from the product of the two. Those go through
VIDEO_SCALE_AREA, so the byte counts widen before the bytes-per-pixel
multiply rather than after.
The retained backbuffer's pair was compared a half at a time to decide
whether the texture could be reused, then stored in two statements;
it is one word either way. Five sites clamped the readback rectangle
to the output size with (a > b) ? b : a written out per axis; they use
MIN.
VIDEO_SCALE_W(d) * VIDEO_SCALE_H(d) had grown to forty sites across
twenty-three files, nearly all of them sizing an allocation. Both axes
clamp at 65535, so the product alone still fits in a 32-bit unsigned --
0xfffe0001 -- but the thing being computed is almost never the pixel
count on its own. It is the pixel count times the bytes per pixel, and
that multiply wraps. A wrapped allocation size does not fail; it
succeeds at the wrong size and the write past it comes later.
VIDEO_SCALE_AREA(d) widens to size_t first, so the bytes-per-pixel
multiply that follows is done at pointer width on any 64-bit target.
Three call sites had already been written with a (size_t) cast in front
of the first half for exactly this reason; the rest had not.
gl3_t kept the output size, the viewport the last frame went out at,
the hardware render target's maximum size and the menu texture's size
as eight fields. All four are now one word each.
The menu texture's pair was the one that could drift: its guard
compared each half against the incoming size to decide whether to
reallocate, then stored the two halves separately. It is one word
either way now.
Four sites clamped the viewport to the output size with
(a > b) ? b : a written out per axis, twice over; they use MIN.
video_driver_convert_xrgb2101010, the threaded wrapper's frame
conversion and video_thread_record_take take the frame size as one
packed word. The recording readback keeps its encoder size and its
last source size as words, so a viewport change is a word compare and
the recorder's gpu_dims reaches it as it is stored.
last_width[GL_CORE_NUM_TEXTURES], last_height[GL_CORE_NUM_TEXTURES],
hw_render_last_width and hw_render_last_height are declared in the
chain struct and never written or read, here or anywhere else in the
tree.
gfx_surface_new and gfx_surface_new_static take the surface's size as
one packed word, stored as given, and the thumbnail's surface helper
passes it through. VIDEO_SCALE_FITS says whether a size packs
unclamped; the thumbnail and overlay loaders check it before sizing a
surface for a decoded image, since a clamped axis would allocate less
than the decoder writes.
gl1_t kept the last frame's size, the screen's, the menu frame's and
the viewport the last frame went out at as eight fields. All four pairs
are now one word each.
Three of the four existed to be compared against an incoming size and
then written back, a half at a time, to decide whether to reallocate:
the frame path in gl1_gfx_frame, the one in the render-target setup,
and the menu path guarding MENU_SIZE_CHANGED. Each is now one word
compared and one word stored.
The font viewport's fallback reads better for it as well. It wanted the
screen size, or the frame size when the context driver has not reported
one yet, which took a VIDEO_SCALE_PACK over two ternaries picking each
axis on its own; it is one ternary over two words.
Left alone: inv_win_width/inv_win_height, which are reciprocals for the
vertex math rather than a size.
vk_t held six width/height pairs: the output size, the streaming
texture's size, the viewport the last frame went out at, the size of
the retained backbuffer copy, the extent the last readback actually
copied, and the size the hardware-render interface last saw.
Three of them were being compared a half at a time to decide whether to
rebuild something. The retained backbuffer is the clearest: two
separate comparisons against the swapchain's width and height decided
whether the copy could be reused, and the pair was written back in two
statements after the reallocation. That is now one word compared and
one word stored, so a reallocation cannot leave the struct claiming a
size the image does not have.
Two viewport clamps read (a > b) ? b : a spelled out per axis over
three lines each; they use MIN, which is what they were.
No video driver fills read_frame_raw - GL2's implementation only built
under a test define that nothing sets - so the slot, its frontend
wrapper, GL2's implementation and the screenshot path that consumed it
are gone. The screenshot task always takes the raw frame from the
frame cache, and prefers a viewport read on any hardware-context core.
The translation driver's translate hook and its HTTP and Apple
implementations take the frame size as one packed word, which
run_translation_service and the response handler keep as they get it
from the frame cache. The software-convert context's size fields,
which the callback no longer reads, are gone.
The GL3 half of the same change: gl3_framebuffer's size and the size a
gl3_pass last rendered at are one word each, and gl3_pass_get_output_size
returns the size rather than writing it through two out-pointers.
Its float-to-pixel conversion goes through VIDEO_PX for the same reason
as the Vulkan one -- a plain cast of roundf() is undefined outside int
range.
Two blocks here declared a pair of locals only to copy the input
texture's size into them and hand them straight to
gl3_framebuffer_set_size, with the size comparison above spelled out a
half at a time. Both are now one packed comparison and a direct call.
Six declarations across the file were indented a level too deep, which
is how those blocks looked from a distance; they are gone with the
locals and fields they belonged to.
slang_framebuffer, slang_pass and vulkan_filter_chain between them held
four width/height pairs as eight fields -- a target's size, the size a
pass last rendered at, the chain's maximum input size, and the source
size accumulated across a per-frame build. All four are now single
words.
Two of them were being passed through out-pointers, which is what made
the packing worth doing rather than merely tidy. slang_pass_set_pass_info
wrote its result into &source_w and &source_h for the caller to feed
into the next pass, and slang_pass_get_output_size did the same one
level down; at the deferred call site those out-pointers were aimed
straight at the chain's own deferred_source fields. Both now return the
size as a word, so the chain walk carries one value per pass instead of
a pair the callee reaches back through two pointers to set.
slang_pass_get_output_size also converted its two floats with a plain
cast of roundf(), which is undefined for anything outside int range.
It goes through VIDEO_PX, which rounds and bounds.
Four declarations in those structs were indented one level too deep;
they are gone with the fields they belonged to.
The screenshot task keeps the frame's size and the size it writes out
as two packed words; screenshot_dump, the rotate step and the PNG/BMP
wrappers take them as words and unpack only where libretro-common's
encoders and scaler want width and height. The output buffer is
allocated straight into the task state, and the scaler pointer sits in
the PNG branch that uses it.
video_driver_cached_frame_info, the cached_frame_read callback and
video_driver_cached_frame_publish pass the frame's size as the packed
word the cache already stores. The reinit snapshot keeps it as one
word from read to republish, and the screenshot copy carries it the
same way.
gfx_display_gl2_discard_draw_rectangle took width and height apart and
kept the last pair it saw in two statics, comparing them a half at a
time to decide whether the scissor rectangle needed rebuilding. It now
takes the size word its only caller already holds and keeps one static,
so the test is a word comparison. That also retires the two locals
gfx_display_gl2_draw was unpacking solely to pass them back in.
The same block had a declaration after a statement, which C89 does not
allow. It has been there a long time: the whole workaround sits behind
#ifdef MALI_BUG, which is defined only on __arm__ and __aarch64__, so
no lane in the compile matrix -- all of which run on x86-64 -- has ever
preprocessed it. Compiling gl2.c with __arm__ forced is what found it.
struct video_fbo_rect held six unsigned fields -- img_width/height,
max_img_width/height and width/height -- three pairs describing one
pass of the shader chain: what it renders this frame, the largest it
renders at any input size, and the texture that holds it. They are now
img_dims, max_img_dims and dims.
The struct goes from 24 bytes to 12, and each driver keeps
GFX_MAX_SHADERS of them, so gl2 and gl3 each drop 768 bytes of state.
gl2_renderchain_recompute_pass_sizes and its gl3 twin are where this
matters. A shader pass scales its two axes independently -- absolute on
one, relative to the input on the other -- so the old code wrote each
axis of each pair in a switch of its own, clamped the four halves to
GL_MAX_TEXTURE_SIZE one at a time, then carried four loop variables to
the next pass. Six of those eight writes could land without their
partner. The axes are still found separately, because they genuinely
are separate, but into plain locals; the pair is joined only where it
is stored, and the loop now carries two words instead of four.
Verified over the 156 compile-matrix lanes, which include six for these
two drivers (desktop, GLES2, GLES3 and the C89 and no-threads gates).
video_output_info_t carries a connected output's size as one packed
word, which the display servers fill in one store and the menu and CRT
switch lists read their labels from.
The fast-path and sort-only deferral lanes pin the mechanism - a walk
that fits the window completes inline with no task and no refresh, and
a completed walk too large to sort inline hands only the sort to the
task. They now run with the instrument frozen, so the outcome no longer
depends on whether the small tree walks inside 4ms of wall-clock, which
a TSan build on a CI runner does not. The pacing lane keeps the real
clock and owns the wall-clock claim.
video_display_config_t carries a mode's size as one packed word. The
display servers fill and match their resolution lists on it and keep
the current mode's size as a word; the SDL servers sort the list on the
word, which orders by width and then height as before.
set_resolution on the display server, its frontend wrapper and every
display server's implementation take the requested size as one packed
word; a zero axis still means the current one, filled in on the word.
The win32 server keeps the mode it restores as one word, and the menu's
resolution picker parses into one before writing the fullscreen
settings.
materialui_render_menu_entry_savestate_list is a copy of
materialui_render_menu_entry_default with the smooth ticker switched
off -- if (0 && mui->flags & MUI_FLAG_USE_SMOOTH_TICKER) -- so only
the plain ticker has ever run there. The dead branch was also the only
writer of value_x_offset, which was initialised to zero and added to
the value's x position on every frame.
The default renderer is untouched: its ticker branch is live and stays
exactly as it is.
The shadow config sat behind if (0 && shadows_enable) with an else that
set GFX_THUMBNAIL_SHADOW_NONE, so the branch has only ever taken the
else. The comment above it said why -- the thumbnails carry their own
background border -- and that reason is now the whole of it.
rgui_render_mini_thumbnail carried a drop shadow behind if (0 &&
shadow_enable) and, in the savestate placeholder branch, an "N/A"
label behind an unconditional return. Neither has drawn a pixel.
The label's TODO said what it was waiting on -- a caption belongs on
a missing state file, not on a missing image file -- so that survives
as a comment on the background it would have sat on. With the label
gone the text position, the message lookup and the function's fb_width
unpack have no remaining reader.
gfx_widgets_init, the widgets' context_reset and the volume and
load-content implementations of it take the output size as one packed
word, which becomes last_video_dims as given. The screenshot widget
keeps its box and thumbnail sizes as words, and the thumbnail scale
factor is computed from the destination and image words.
thumbnail_t held four unsigned fields: max_width, max_height, width and
height. Both are pairs that have to agree -- rgui_process_thumbnail
sets width and height together from the decoded image and the copy loop
below walks exactly that rectangle, so a field updated without its
partner reads off the end of the buffer. They are now one word each,
max_dims and dims, which makes that impossible to write.
One of the two was already packed at its source: mini_thumbnail_max_dims
is a packed word that was being taken apart into max_width and
max_height for two thumbnails and read back as halves. Those four
assignments are now two word copies.
rgui_downscale_thumbnail's max_width/max_height parameter follows, since
its only caller now holds the word.
The halves are still what most of the arithmetic wants, so the two
functions that read them repeatedly unpack once into locals rather than
reaching through the macro at each use.
This costs about 270 bytes of text on x86-64 -O3, from the clamp
VIDEO_SCALE_PACK applies at the two sites that build a word from
unrelated values. That clamp is the point: it is what stops one axis
writing over the other.
Eleven functions in rgui.c took the framebuffer size as a separate
fb_width and fb_height, from the two primitives that write pixels
(rgui_fill_rect, rgui_color_rect) up through the render phases that
call them. Every one of those calls started from p_disp->framebuf_dims,
which is already a packed word, so the pair was unpacked at the top of
rgui_render and carried down seventy call sites only to be compared
and multiplied as two halves again.
The parameter is now a single fb_dims. Functions that need the halves
unpack them locally, which costs a shift and a mask once per call
rather than a second argument at every level, and the call sites pass
p_disp->framebuf_dims straight through.
Two consequences worth naming. rgui_render no longer unpacks fb_height
at all, and the framebuffer-resize test it feeds -- last_dims against
a freshly packed pair -- becomes a comparison of the two words it was
always really comparing.
gfx_display_reset_textures_list and its buffer variant report the
loaded image's size as one packed word, which the AI service overlay
and the screenshot widget keep as they get it. The icon loader's size
outputs, which no caller reads, are gone, and draw_cursor places the
cursor from the size word it is already given.
The poke interface's set_texture_frame and every driver's
implementation of it take the menu texture's size as one packed word;
drivers that clamp it to their texture do so on the word's axes, and
Metal's menu view takes it as updateDims:. The threaded wrapper stores
the word it is handed, and rgui passes its framebuffer's word.
The twenty-two draw helpers in materialui.c each took the output size
as a separate width and height, and each of them handed that pair
straight back to gfx_display, which packs it again. Both halves of the
round trip are gone: the helpers take a single video_dims word and pass
it through untouched.
Two of the call sites are function-pointer variables rather than
definitions -- materialui_render_menu_entry and
materialui_render_selected_entry_aux carry the callback shape -- so
their declared types move with the functions they point at.
The frame-level locals that fed these calls were only ever unpacked
copies of video_info->dims, so they go too, and the seventeen sites
that used them read the packed field directly.
The Wayland protocol headers are generated into gfx/common/wayland by
configure, so the Wayland lanes probe for them as well as for the
system's wayland-client.h, and the Vulkan one for vulkan.h;
platform_video takes a list of probes and skips on the first missing.
b1e88d6f62 changed the set_viewport slot and every driver's own setter
to take the output size as one packed word, but the two TARGET_OS_IPHONE
calls in gl2_frame and gl3_frame still passed width and height as two
arguments. Those lines only build on iOS and tvOS, so no Linux, Windows
or macOS lane saw them, and the tvOS build stopped compiling.
Pack the two axes with VIDEO_SCALE_PACK, as the other call sites do.
font_renderer_t's flush and the GL, Vulkan, D3D11/12 and RSX
implementations of it take the output size as one packed word, as do
the GL and RSX font viewport setters behind them. render_msg builds
that word from the driver's own size once, and font_flush, the widgets,
xmb and the screensaver pass the size word they already hold.
Server and local manifest entries under a directory whose sync toggle
is off are carried forward rather than synced, and the diff steps over
them in one go. Progress now leaves them out of both the done and the
total counts, and is taken after each diff step rather than before it,
so the step that ends the diff also lands the bar on 100%. Previously
a disabled directory sorting last (thumbnails after saves and states)
was skipped in the final step and the bar stayed at the in-scope share,
e.g. 5% (#19606).
The directory-scope test moves into task_cloud_sync_key_in_scope(),
shared by the skip and by the count taken when the diff starts, so the
two cannot disagree.
The writer marks the stamp odd with a store-release and then writes the
four fields relaxed. A store-release orders the accesses before it, not
the ones after, so nothing stops those field stores from being hoisted
above the odd stamp. On a machine that reorders store with store - ARM,
PPC, so Android, iOS, the Apple Silicon builds, Wii, Wii U, PS3 - a
reader can then load an even stamp either side of a tuple that is half
written, which is the one thing the stamp exists to prevent.
The other two seqlocks in this file already carry the fence, and the
frame cache's says why in a comment. This one was the odd one out.
The harness reproduced the same shape, so it could not have caught
this; it carries the fence now. x86 does not reorder store with store,
so no lane there can make the hole visible - which is the point worth
recording next to it.
The video_driver_t set_viewport slot and each driver's own viewport
setter take the size as one packed word; the GL, Vulkan, GDI, RSX, GXM
and Metal setters store it as the viewport's full size directly, and
d3d8's rect helper returns the fitted viewport's word. Metal's
setViewportDims: replaces the width/height selector.
The menu drivers, widgets and screensaver hand set_viewport the frame
info's size word as they receive it.
Sixteen of the driver's draw functions took video_width and
video_height as a pair. They take the word.
A hundred and fifty-eight repacks went with them - each one a
VIDEO_SCALE_PACK of two locals that came from the same word a few
lines above, which is the residue the layers below this one left as
they were packed. Ten locals had no reader afterwards and went too.
259 lines out, 205 in.
Matrix clean, the file native, under mingw and under C89, all at zero
warnings.
render, osk_ptr_at_pos and osk_pointer_over_textbox receive the output
size as one packed word, and the menu drivers pass it on whole: to the
screensaver, to materialui's entries box and to xmb's layout choice.
The menu drivers' init reads the output word without unpacking it, and
the pointer DPI cache keys on the same word.
The ozone_color_theme_defer sample hands the frame info's size word to
render.
Builds without HAVE_DYLIB bind pQueryDisplayConfig to the SDK function
directly; it takes the file's own DISPLAYCONFIG structs, so it is cast
the way the GetProcAddress path already casts it.
Six of the driver's draw functions took video_width and video_height as
a pair, and the draw context it fills for each item carried the same
pair again. Both take the word.
Thirty-six repacks went with them: each was VIDEO_SCALE_PACK of two
locals unpacked from the same word a few lines above, which is what
every call site turned into as the layers below this one were packed.
Three locals had no reader left afterwards.
xmb_draw_item reads its width and height off the context rather than
its own parameters, so it keeps a pair of locals - they are the halves
of the context's word now rather than two fields.
Matrix clean, the file native, under mingw and under C89, all at zero
warnings.
get_video_output_size - on the display server, the poke interface and
the context driver alike - fills one packed size word, as do the
frontend wrappers over it and the Cocoa and PS3 helpers behind their
implementations. Callers that set the video mode from it pass the word
straight through; the auto window cap keeps its limit packed.
The thumbnail sample stubs include video_defines.h for the size macros
they use.
Four helpers in gfx_display.c and gfx_widgets.c still took the output
size as a pair: the quad batcher, the indicator draw and the two
message draws. The batcher took a second pair for the quad's own size,
and its one caller was handing it VIDEO_SCALE_W(draw.dims) and
VIDEO_SCALE_H(draw.dims) - the halves of a word it already held.
With those four taking words, sixty-two repacks in the two files had
nothing left to do. Each was VIDEO_SCALE_PACK(video_width,
video_height) where the two locals came from VIDEO_SCALE_W and
VIDEO_SCALE_H of the same word a few lines above, so the word goes
through instead. Ten of those locals had no other reader and went with
them.
Matrix clean, both files under mingw, C89 pass, call_arity_check at its
baseline.
The Wayland GL and Vulkan contexts, the GL context as webOS builds it,
and the webOS shim each get a lane. The shim's protocol headers are
wayland-scanner output of LG's webos-wayland-extensions (Apache-2.0),
kept under stubs/webos since the SDK only generates them at build time.
Every lane rejects incompatible pointer types and int-to-pointer
conversions, as gcc 14 does, so a callback whose signature no longer
matches its slot fails here and not only on newer toolchains.
fpga_init takes the input driver pointer the video_driver_t slot hands
it, and the opendingux and vivante fbdev contexts cast their native
window to the pointer egl_create_surface takes, as vita and wayland do:
Khronos' eglplatform.h makes EGLNativeWindowType an integer on unix
builds without X11, which gcc 14 refuses to pass as a pointer.
font_flush took the output size as a pair at all twenty-three of its
call sites, and gfx_widgets_draw_icon took two pairs - the output size
and the icon's own - at eighteen.
The icon draw gains the most: it was unpacking the icon pair only to
pack it straight back into the descriptor's dims, and packing the
output pair again for the draw call under it. Both go through whole.
The packs that appear at call sites here are the menu drivers' own
video_width/video_height parameters, which are the next pair along and
are left for their own change.
Matrix clean, C89 pass over every file touched, call_arity_check at its
baseline.
The webOS get_video_size hands back a single packed size word like the
common Wayland one, and wayland_ctx.c's webOS prototypes for it and for
check_window match the context driver's get_video_size slot.
cocoa_vk's live-size query passes the word through, the WinRT window
size no longer shadows its out parameter, and the emscripten EGL init
takes its axes from the word it fills.
sdl3_vk and switch fill a local size word and take their axes from it;
the UWP context's local no longer shadows its out parameter, and the
Cocoa GL live-size query passes the word through.
The unclaimed-samples job runs make clean before make check in every
sample directory and never builds the frontend. The gamemode sample
now has a clean target (run.sh keeps its stub in a temporary
directory it removes itself), and run.sh skips when there is no built
./retroarch to run.
gfx_display_draw_quad carried three width/height pairs in one
signature and two of them were output sizes with nothing in the names
to tell them apart: video_width/video_height went to the driver, and
width/height was the space the quad's coordinates are measured
against, which the body divides x and w by and flips y against. They
are video_dims and ref_dims now, and the quad's own w/h is dims.
At 133 of the 141 call sites the two are the same value passed twice.
The eight where they are not are the ones the names were hiding: the
menu screensaver measures against its own last size, and four draws in
xmb measure against the view rather than the window.
draw_cursor and draw_keyboard take the output size as a word with it,
which is the shape draw_texture_slice and the scissor calls already
had.
The line count is the argument lists: a pair that took two lines takes
one, so eighteen files lose 837 lines and gain 558.
Full matrix clean, the menu drivers and widgets under mingw, a C89 pass
over every file touched, and call_arity_check at its baseline.
gfx_thumbnail_draw takes the output size and the rectangle the
thumbnail is fitted into each in one word, and
gfx_thumbnail_get_draw_dimensions takes its rectangle the same way.
draw and draw_pipeline were the two members of gfx_display_ctx_driver_t
still taking the output size as a pair, beside the scissor calls that
already take it as a word. Both take it as one now, and so does
gfx_display_draw, which is the shim every caller reaches them through
and which passes the word straight down rather than unpacking it on the
way.
Twenty-two backends implement draw and six implement draw_pipeline.
Most read both halves and get them back from the word at the top of the
body; the few that read neither take the word and ignore it, as they
ignored the pair.
The call sites that still hold a pair of their own pack it at the call.
Those pairs are the next layer down - the gfx_display_ * helpers and
the menu drivers' own parameters - and are left for their own change
rather than dragged into this one.
gfx_display_draw_texture_slice takes the output size, the texture's
size, the size to draw it at and the area it is placed in each in one
word; the body reads the axes it needs once at the top.
video_info_t's width and height became dims; metal.m's init was still
packing the two fields that had gone away. iOS and tvOS both stopped
there, six errors from the one line. applyVideoMode already takes the
word, and the other two callers already pass it.
Under the threaded wrapper x11_alive() pumps X events on the video
thread while the input driver polls on the runloop thread, and both
touch g_x11_entered and the wheel/extra-button latch. Both are now
retro_atomic words; a wheel bit is read and cleared in one fetch_and,
so a notch latched between the two is kept.
The threaded-video harness gains an X11 event-pump lane: it sends
crossing and button events from its own connection while a frame is
on the worker and polls against it, so TSan sees the unordered
accesses on every run rather than when the server's pointer happens
to sit over the window.
Nothing here compiled sdl2_gfx.c or sdl3_gfx.c. That is how both went
on reading video_info_t's width and height after the pair had been
packed into dims: every job was green and the build broke.
Each lane skips where its headers are absent, as the dingux and exynos
lanes do, so the SDL2 one runs wherever libsdl2-dev is installed and
the SDL3 one wherever libsdl3-dev is. Putting the old field back fails
the SDL2 lane with the error the build gave.
sdl3_poke_set_video_mode gets the leftover from the interface change
with it: it took the size word apart into two locals and packed them
back together twice, once for the call it forwards to and once for the
field it stores, and nothing else read them. The word goes through.
gfx_display_scissor_begin, the display context's scissor_begin and
scissor_end and all seventeen driver implementations take the output
size and the rect's size each in one word; a driver that needs the axes
takes them once at the top.
Configure builds can now leave out GameMode support with
--disable-gamemode; it stays enabled by default. Builds that do not
use config.h (the handheld Makefiles, griffin) keep GameMode as
before.
frontend_unix_deinit() leaves GameMode unconditionally, so shutdown
loads libgamemode.so.0 and queries gamemoded even with gamemode_enable
off. Track whether this process entered GameMode and make the leave
request a no-op when it did not.
samples/frontend/gamemode runs the built binary against a stub
libgamemode.so.0 and checks that nothing calls it with the setting
off, and that GameMode is entered and left with it on.
get_video_size was the last of the size interfaces still split: every
context driver wrote two unsigneds through two pointers, and every
caller supplied a pair of locals to catch them. Thirty-one
implementations, forty call sites, and the same shape at each one.
It writes the word now. The window checks are where that reads best.
Each of them used to hold a pair, fill it, pack it to compare against
the word the caller passed, and pack it again to store:
unsigned new_width, new_height;
get_video_size(x, &new_width, &new_height);
if (VIDEO_SCALE_PACK(new_width, new_height) != *dims)
{
*dims = VIDEO_SCALE_PACK(new_width, new_height);
*resize = true;
}
which is now one local, one comparison and one assignment. Eleven
drivers carried that.
Two of them were taking apart something already packed. The Cocoa
contexts publish their backing size to a thread-safe word as
(width << 16) | height, which is VIDEO_SCALE_PACK's layout exactly, and
their _ts readers existed to split it into the pair this interface
wanted; they hand the word over whole now. The Android Vulkan context
built its pair from two atomic loads and compared the halves
separately.
The driver locals that fed it go the same way: mode_width/mode_height
and temp_width/temp_height in gl1, gl2, gl3, vulkan, vg and gdi, which
is where the line count comes from. One test kept its two halves on
purpose - the fallback that asks video_driver for the output size when
the context reports nothing runs when either axis is zero, and the
packed word being non-zero does not mean both are, so it reads
VIDEO_SCALE_W() != 0 && VIDEO_SCALE_H() != 0 rather than dims != 0.
Full matrix clean, the Win32 drivers under mingw, the X11, EGL,
Wayland, DRM and OSMesa translation units the matrix does not reach
compiled here directly, and a C89 pass over every plain-C file the
change touches. The Cocoa contexts have no lane; their hunks were
reviewed by hand.
Start on the Resolution entry sets the video mode full screen only when
the frontend already is - from the setting or the forced-fullscreen
flag - so a windowed desktop session stays windowed; GameCube/Wii and
PS2 stay full screen as before.
samples/menu/resolution_reset links the shipping objects, swaps the
null driver's poke table for one that records the mode, and presses
Start in a windowed, a fullscreen-setting and a forced-fullscreen
session; it runs in the menu harness workflow.
video_info_t::dims carries both axes, so the size the frontend resolves
travels to every driver's init as one word and the drivers read the axes
they need with VIDEO_SCALE_W and VIDEO_SCALE_H. The Direct3D drivers'
stored copy compares and updates as one word.
The poke sizes the window or takes it full screen through the same
-applyVideoMode: that driver init uses, on the main thread via
cocoa_main_thread_sync so threaded video can reach it, and shows the
cursor only when windowed - matching the Cocoa GL and Vulkan contexts.
A zero axis keeps the view's current size.
gfx_ctx_mode_t holds VIDEO_SCALE_PACK(width, height), and the Cocoa
GL/Vulkan contexts hand it through their main-thread argument blocks
the same way. The size copies those contexts kept but never read are
gone, as are the mode locals in gl1, vulkan and sixel that only
repeated the frame size or were never used.
gfx_display_draw_text, its high-precision sibling and the internal they
share take the area the text is placed in as one word; the menu drivers
and the widget layer hand over the word they already hold.
Six video drivers open with the same twenty lines: take the requested
size out of the frame descriptor, fall back to the detected screen size
when both axes are zero, fall back again to the configured fullscreen
size when fullscreen had to be forced, hand the result to the context
driver. With set_video_mode taking the size as one word, the two locals
that carried it between those steps have nothing to be two of.
The zero test is the part worth reading twice:
if (video->fullscreen && (win_width == 0) && (win_height == 0))
becomes a test of the word, because neither axis set is the whole word
clear. The OpenVG driver gains the most - its fallback already asked
the video driver for a packed output size and took it apart to fill the
pair, and now assigns it.
gl1 and gdi keep their mode_width/mode_height, which are read
separately further down; they take the halves back out of the word.
sdl3_vk_ctx keeps its pair: it passes both to
sdl3_window_get_video_size by address, and a packed half has no
address, which is the same reason the settings viewport is not packed.
Full matrix, a native build and a C89 build from a clean tree, all at
zero warnings, plus the GDI driver under mingw.
Three interfaces in one call chain each carried the same loose pair.
video_driver_set_video_mode takes it from the frontend, hands it to
video_poke_interface's set_video_mode, and the video driver hands it on
to gfx_ctx_driver_t's - 59 declarations and 35 call sites, all of them
(unsigned width, unsigned height, bool fullscreen). The thread wrapper
already stored the pair packed in its mode packet and unpacked it for
no reason other than to make that call, which is what the chain looked
like from the inside.
All three take the size as one word now, and so do the two helpers
under them, win32_set_video_mode and sdl3_window_set_video_mode.
Most of the payoff is at the call sites. The thread wrapper passes
pkt.data.new_mode.dims straight through. d3d10, d3d11 and d3d12 were
unpacking their own viewport to pass the halves and now pass
->vp.full_dims. Twelve bodies that did nothing but forward the pair
pass the word instead of taking it apart and putting it back together.
sdl3_window_set_video_mode is why the rewriting worked from the
declarations rather than the calls: it takes five arguments and its
pair is not the one before the last, so anything that guessed the
position from the shape of the call would have packed its height
together with its fullscreen flag.
Full matrix, a native build and a C89 build from a clean tree, all at
zero warnings, plus the Win32 drivers under mingw. The Cocoa contexts
and the Metal driver have no lane here; their hunks are the parameter
rewrite and were reviewed by hand.
gfx_widgets_iterate, iterate_layout, the per-widget iterate callback,
gfx_widgets_draw_text and gfx_widgets_flush_text all carry the output
size as one word. The two scale caches compare one word, and the layout
check compares against last_video_dims directly.
Every menu metric is a constant times a DPI scale that is almost never
whole, and the plain conversion to the int it is stored in truncates.
That loses up to a pixel off each one, always downward, and it stacks:
at the 1.3333 scale Ozone's 50px row came out 66 rather than 67, which
is thirteen pixels over a twenty-row list. Ozone's three spacers
already rounded by hand - someone knew - while the other twenty-two
metrics beside them did not.
All of them go through VIDEO_PX now, in Ozone, MaterialUI and the two
integer metrics XMB keeps, so there is one way to turn a scaled length
into a pixel and it is the same one the draw origin uses. The 1px
spacer keeps its floor of one: a hairline that rounds to nothing is not
a hairline.
MaterialUI's node->entry_height is left alone deliberately. It is a
float that accumulates several terms before anything reads it, so
rounding each term into it would be rounding the same value twice.
Pixels move, which is the point: a metric that was a pixel short is a
pixel taller. ENTRY_HEIGHT gains one at the 1.3333 and 1.7778 scales,
ENTRY_ICON_SIZE at 1.7778 and 2.6667, HEADER_HEIGHT at 1.7778. At the
whole scales - 1.0, 1.5, 2.0 - nothing moves at all.
Full matrix, native build and C89 build from a clean tree, all three at
zero warnings.
The descriptor carried its size as a packed integer pair and its origin
as two floats - the same quad's geometry in two number systems - and
every driver had to bridge that itself. None of them rounded. gl1, gl2
and gl3 hand the float to glViewport, which takes a GLint; gdi and d3d8
cast with (int). So a position that came off a scale factor and landed
at 99.99998 drew at 99, while its width was exact. gx2, d3d10, d3d11,
d3d12 and the two SDL drivers did not truncate at all - they divide by
the viewport to get normalised coordinates - so the same descriptor
drew at a different sub-pixel position depending on which driver was
up.
The origin is one signed pair now, in VIDEO_POS_PACK's layout, like a
viewport's. Whole pixels, decided once where the layout is known
instead of differently by each driver on its way out.
VIDEO_PX is what decides. It rounds rather than truncates, which is
worth a pixel on every value that came off a DPI scale, and it bounds
the conversion, which the plain cast does not: converting a float past
INT_MAX is undefined. That was not theoretical. gfx_widgets_draw_icon's
coordinate math underflows in the first frames after an icon loads, and
the float the descriptor carried took that value as far as
SDL_RenderGeometry, where it became NaN vertex positions and an SDL
error that poisoned the renderer state for every draw after it - the
widget simply vanished. sdl2_gfx and sdl3_gfx each carried a
hand-written range test against it. Both are gone: a signed 16-bit
origin and an unsigned 16-bit size cannot hold a value they would
reject, and a NaN reaching VIDEO_PX lands at the far edge, off-screen
and over the following frame.
The twelve positions that genuinely animate keep their float - the
tween is a pure function of the clock and wants sub-pixel motion - and
round on the way into the descriptor, which is where a sub-pixel scroll
position has to become a pixel regardless.
Checked with the full matrix, a native build and a C89 build from a
clean tree, both at zero warnings, plus the Win32 drivers under mingw
and sdl2_gfx against the host SDL2. metal.m and the 3DS and RSX
drivers have no lane here; their hunks are the mechanical read rewrite
and were reviewed by hand.
Thread CPU time on Windows advances in whole 15.625 ms scheduler
ticks, so one submit that straddles a tick reads as a full tick and
fails a per-sample percentile. The lane now budgets the mean over all
submits, which tracks the time actually spent on any clock and still
fails when every submit carries a 4K copy. The p50/p95/p99 figures
stay in the baseline line.
RGUI blits into an integer framebuffer, and these were the only two
floats in the file. They existed to halve a glyph: icon_padding is
font_width_stride / 2, and every offset built from it was truncated
back to int one to five lines later.
Most of those halves cancelled - the padding appears in pairs that add
back up to a whole glyph - so the icon origins were always integers
taking a detour through float. The two that did not cancel are the hit
box's top edge and its height, and there the truncation threw the half
away. The integer forms keep exactly what the truncation produced:
(size + 1) / 2 for the edge, (size * 5) / 2 for the height, checked
against the float arithmetic across every font stride from 1 to 39 and
every framebuffer geometry RGUI runs at.
No pixel moves. The point is that a software blitter has no half
pixels to place anything on, so nothing here should be asking for one.
The cap is a third of the screen width, written as a multiply by
0.3333333f. That literal is one ulp below the nearest float to 1/3, so
the cap lands just under the third it is meant to be - 639.99994 rather
than 640 at 1920 wide - and the scale that comes out of the division is
off by the same amount. Dividing says what it means and is exact to the
float.
The name went with it. new_width is not a width of anything; it is the
largest the sidebar is allowed to be, which is what the code below it
compares against.
video_frame_info_t::dims carries both axes beside the scale_dims and
custom_vp_dims words already there, and the two writers hand over the
packed word they already held. The widget tick takes it straight
through; the netplay widgets read their axes once into locals.
gfx_animation_update, gfx_animation_update_widgets, the update_time_cb
the menu drivers set, and gfx_display_get_dpi_scale all take both axes
in one word. The scale caches compare one word where they compared two,
and the widget path hands its packed size straight through.
ps2_gfx.c is the other console video driver nothing here compiles, for
the same reason as the Vita one: it reaches a compiler only inside the
PS2 toolchain. The viewport work touched it in two places, and neither
has been through a compiler.
This adds the gsKit and PS2SDK stubs it needs - gsKit.h with dmaKit.h
under it, kernel.h for the semaphores its vsync handler takes, and
tamtypes.h for the scalar names the rest lean on - and a lane that
compiles it against them. gsToolkit.h and gsInline.h are the two other
headers the driver includes; gsKit ships both behind its master header,
so here they are one line each.
Same rule as the Vita stubs: the declarations the driver names and only
those, each in the shape the real header gives it, so the lane rejects
what the SDK would reject. Dropping an argument from
gsKit_prim_sprite_texture fails it.
The lane takes the window offset, which Makefile.ps2 turns on and the
driver reads; without it the settings struct has no such member and the
lane would fail on the tree rather than on a change to it.
It is also the only lane that compiles retro_atomic.h's EE
interrupt-mask backend. No other platform selects it, so DIntr/EIntr
and every operation built on them were unreachable from any job here.
ExitHandler() is the one declaration whose body cannot run on the host:
it is the R5900 sequence the SDK writes, and the lane gets away with it
because it parses rather than assembles. A lane that compiled to an
object would need a stand-in; the note in kernel.h says so.
gxm_gfx.c reaches a compiler only inside the Vita toolchain. Nothing
here builds it, so a change to a struct or a signature it touches is
green in every job until that console job runs - which is how four
lanes broke in a row on the viewport work, one round trip at a time.
The driver is one of the files that work rewrote most: its viewport
origin, its two size pairs and the output size all travel as packed
words now, and none of that has been through a compiler.
This adds the psp2 stubs the driver needs - gxm.h, display.h,
kernel/sysmem.h, common_dialog.h, and the scalar types the rest lean
on - and a lane that compiles it against them.
The stubs carry the declarations gxm_gfx.c names and only those, each
in the shape the real header gives it, so the lane rejects an argument
list the SDK would reject rather than only a malformed one. Dropping
an argument from sceGxmSetRegionClip fails the lane; so does dropping
one from the driver's own gxm_set_viewport. The enums keep the SDK's
values and lose the members nothing here names, which is why a format
constant may be missing: take it from the real header rather than
inventing a value.
Same narrowing suppression as the psp1 lane above it - the driver
folds pool addresses into pointers that a 64-bit host widens, and the
arithmetic is 32-bit on the target.
Both compile here already - gx2_gfx.c against the wiiu headers the tree
vendors, hub75_gfx.c against plain POSIX - so the only reason neither
was checked before delivery is that neither had a lane. They are the
drivers this campaign kept breaking unseen.
HOSTOFF moves up to the flags every lane shares. It was defined two
hundred lines below the first platform_video lanes, so a console lane
written above it expanded to nothing and compiled with the host
identity left on, taking the Linux branch of every #if ladder and
proving nothing.
vg_init's fullscreen branch takes the window size from the output word
and writes it straight to win_width/win_height; its two declarations of
out_dims become one. The driver has had neither since the output size
started travelling as one word: nothing here compiled it, so the
statements left in the middle of an if condition went unseen.
tools/platform_stubs/openvg carries the OpenVG surface the driver names
- generated from what it uses, not from the specification - and the
compile matrix gains a lane for it, so the next change to this driver
is compiled rather than read.
A signature change is caught by the compiler wherever it compiles. The
call sites it does not compile are found by whichever CI lane builds
that platform, one round trip each: changing check_window's parameters
broke the PS3 driver, two Win32 contexts and the webOS window check
that way, one after another. tools/call_arity_check.py reads the tree
as text instead, so a branch behind GEKKO, WEBOS, VITA or _XBOX is
examined like any other, and reports a call whose argument count does
not match the declaration.
It declines what text cannot settle: varargs, calls through a pointer,
function-like macros, an argument list carrying a preprocessor
conditional, and any name a harness stub redeclares with a shape of its
own.
Its first finding goes with it. gx_efb_screenshot() called
intfstream_open() with intfstream_open_file()'s three arguments, so the
GameCube and Wii builds would not have compiled it - nothing did: it
sat behind a TAKE_EFB_SCREENSHOT_ON_EXIT that an "#if 0" defines, and
nothing called it. It goes, with the stream include it was the only
user of.
gfx_ctx_wl_check_window_webos is the webOS build's stand-in for the
shared handler, so it takes the same one word and compares it whole.
Its declaration in wayland_ctx.c follows.
The frontend stops presenting on a queue of its own, and the negotiation
interface goes back to version 2. This removes 5d950a7288 (the second
queue), dadbd009d9 (v3 in libretro_vulkan.h), c265d78ee9 (the Android
carve-out) and 9f686efdfa (the comment on when it is used), and adapts
4823918abe, which stays.
What it bought was narrow: on one vendor, in exclusive fullscreen,
vkQueuePresentKHR blocks inside the call waiting on the display, and
with one queue under queue_lock a core's submissions sat behind that.
What it cost is on record. Its first act was to break every frontend
older than four days against cores that took the queue (libretro/ps2
#167), it needed a per-platform exception within five days because a
window system could not present on the second queue, and it is the only
change in the path where the same issue now reports random aborts at
start and heap corruption at exit - on the v3 path every time. None of
that is proof, and none of it is a measurement in its favour either.
Three places had to agree about it, and the feature was on for some
systems and silently off for others. That is not worth the stall it
removed.
4823918abe - a swapchain rebuild waits out the presents it retires - is
kept, because the bug it fixes is not the second queue's. A present's
semaphore wait is a queue operation behind the last frame's fence, not
under it, on any queue; Mali's Android WSI (#19601) fails every later
present when the semaphore goes out from under one. The empty
submission that fences the presents now goes to the graphics queue,
under queue_lock like any use of it.
A core that asks for version 3 gets 2 and the graphics queue as its
presentation queue, which is what every build before the 17th did.
The vulkan_validation sample builds and passes, the teardown-behind-a-
present case included, on llvmpipe here.
wgl_ctx and uwp_egl_ctx ask win32_check_window for the size and hand
their own two out-parameters back from its halves. Both also write
through those parameters rather than over them, so the Xbox display
resolution reaches the caller.
tools/mingw_syntax_check.sh reports these files at all now. A header
this box lacks made the compile stop at the include, and the
missing-header filter then reported the file clean: HAVE_D3DKMT pulls
in d3dkmthk.h, so every consumer of win32_common.h went unchecked that
way. The define is dropped when its header is absent, and a parse that
stops early is named and counted rather than folded into the ok line.
'Auto' derives the swap interval as the whole multiple the display rate
is of the content rate, up to MAXIMUM_SWAP_INTERVAL, which the manual
setting's range reads too so the two cannot drift apart. A 480 Hz panel
running 60 fps content holds each frame for eight display frames;
driver_adjust_system_rates() then divides the target by that interval
and the audio rate follows the display.
The multiple has to be one the driver can present: the D3D APIs carry
the interval as a present parameter that stops at four, so those drivers
report that bound through the new get_swap_interval_cap poke and the
ceiling comes down to it. A multiple past the cap falls back to 1 rather
than clamping short, since a short multiple paces the loop fast.
The decision itself moves to runloop_video_swap_interval_for() in
runloop.h, next to the other pure pacing decisions, and
samples/runloop/pacing covers the multiples, the driver cap, the ratios
that are not whole multiples, and the degenerate rates.
Six drivers read the output size in the else branch of their init:
that branch takes the three statements it needs as a block. The Win32
Vulkan context, the SDL3 Vulkan context and the SDL3 shared handler
pass their own size word on rather than the two names it replaced.
xdg-toplevel-icon-v1 lets the client destroy the icon object right
after set_icon without the toplevel losing its icon, and the buffer
only has to outlive the icon object; wl_buffer.release is never sent
for it. So wl_create_toplevel_icon() now destroys both straight after
set_icon, and nothing about the icon has to live with the window.
That drops the libdecor_icon, xdg_toplevel_icon and icon_buffer fields,
the shm_buffer back-pointer and the release handler special case. The
failure path also destroys the icon object instead of leaking it.
gfx_ctx_wl_destroy_resources_common() released every other Wayland
object but left the libdecor frame, context, toplevel icon and dlopen
handle behind, so every video driver reinit dropped a set and built
new ones. Release them before the surface is destroyed and the display
is disconnected: the frame owns xdg objects built on that surface, and
the context still talks to the display.
The icon's shm buffer leaked the same way. It is freed on
wl_buffer.release, which a compositor that holds the icon need never
send, so the window owns it now; an early release clears the
reference.
video_driver_get_output_dims() hands back the word the state already
holds and video_driver_set_output_dims() takes one, so a reader gets a
coherent pair from a single atomic load rather than two halves split
out of it. VIDEO_DRIVER_OUTPUT_WIDTH/HEIGHT were aliases for
VIDEO_SCALE_W/H and go with them.
The window check hands its size back the same way: check_window on the
context-driver vtable, win32_check_window and x11_check_window each
take one unsigned* where they took two, and a driver passes the field
itself - d3d10, d3d11 and d3d12 give &d3dN->vp.full_dims straight to
win32_check_window. The drivers' own size locals travel packed
alongside, and the runloop's two aspect checks compare one word
against the size they last laid out for.
The preview's sound had been held to itself - decoded, fed, not
stalled - and never to a reference, while its first seconds are
reported as distorted in the field. preview_audio_oracle_test decodes
a track through the chain a preview uses, audio_mixer to
audio_transfer to the demuxer to the AAC decoder, from the first
sample against ffmpeg's decode of the same track: at the track's rate
the two agree to the rounding of a float decoder, and through the
mixer's resampler, the filter's delay aligned for, to a third of a
percent RMS in every half second from the first. So the chain from
the file to the mixer's output is not where the first seconds go
wrong; what remains is downstream of the mixer, or the feed. In the
check target and CI.
gl3_viewport is this backend's own, so its origin and its drawn area
pack like the frontend's and glViewport, glScissor and the pass sizing
take the halves back out. gl3_set_viewport hands the frontend viewport
over as two word copies.
rsx_viewport_t is this driver's own, so its origin and its drawn area
pack the same way the frontend's viewport does and rsxSetViewport takes
the halves back out. A viewport handed straight over is a word copy -
rsx_set_viewport and rsx_frame take vp.dims from rsx->vp.dims as it
stands. The depth range, scale and offset stay floats: they are not
pixel counts.
tools/viewport_shadow_check.py reports the remaining names that mean a
video_viewport_t in one place and another type elsewhere, skipping the
ones already packed, so what is left is the types whose layout is not
ours to change - VkViewport, MTLViewport, wp_viewport and the GL
handles.
rsx_viewport_t is the RSX's own float x/y/w/h that rsxSetViewport and
rsxSetScissor take; only rsx->vp and gcm->vp are the frontend's packed
viewport, so the local takes its two axes out of those and passes them
on as floats.
tools/viewport_shadow_check.py reports the names this confusion needs:
one that means a video_viewport_t somewhere in the tree and some other
type in the file at hand. It names rsx_viewport_t, VkViewport,
MTLViewport, gl3_viewport, wp_viewport and the GL handles, by
declaration rather than by compiling, so the drivers no lane here
builds are covered too.
An atomic beside anim_sess says whether the animation thread has one
published, so a poll with nothing playing takes no lock at all. The
audio window is still started and fed under the lock, where the
session stays valid.
Two atomic counters track the items in the queue and the items queued
or in progress, written under work_mutex beside the queue itself and
read without it. tpool_help returns on an empty queue and tpool_wait
returns on an idle pool without touching the lock, so a caller looping
in either while it waits on work in other hands leaves the queue's
lock to the threads that need it.
Every menu driver keeps the video size it last laid out for and
compares the incoming one against it; that pair is one word in
VIDEO_SCALE_PACK's layout, so the check is a single compare against
last_dims rather than one per axis. RGUI's glyph cell, its window size
and its mini-thumbnail cap, and MaterialUI's nav-bar layout go the same
way.
The pairs whose axes are derived apart from each other keep two
members: the widgets' bg_width/bg_height come from text metrics and
line height separately, and MaterialUI's thumbnail caps are clamped one
axis at a time, so packing either would buy one store and cost eleven
read-modify-writes.
video_viewport_t holds x and y in VIDEO_POS_PACK's layout, x in the high
half and y in the low, each a signed 16-bit offset that sign-extends
back out through VIDEO_POS_X/Y - an origin goes negative wherever
integer scaling overscans the window. VIDEO_POS_PUT_X/Y set one axis
where a caller has only that one, and an offset past a half saturates,
so neither axis can put the image on the far side of the display.
The whole viewport is now three words. The threaded wrapper's seqlock
publishes the origin as one slot, so VIDEO_THREAD_VP_SLOTS drops to
three and a reader takes three relaxed loads between the sequence
checks. VkViewport keeps its own float x and y, as does the overlay's.
samples/gfx/viewport_pack gains an origin lane: the round trip over
negative offsets, a sign bit that stays in its own half, one axis put
without moving the other, and the saturation. Dropping the sign
extension from either half fails it six times.
d3d9_make_d3dpp takes the back buffer's size out of d3d->vp.full_dims,
and the GDI window's paint handler takes its viewport rect out of
gdi->vp.dims. Both files reach a viewport through a member their own
header declares, so neither names video_viewport_t.
video_viewport_t carries the drawn area and the window that holds it in
VIDEO_SCALE_PACK's layout, one word each, so a viewport reaches a driver
in two stores and every producer holding both axes writes them together.
VIDEO_SCALE_PUT_W/H set a single axis where a caller has only that one,
and a viewport passed straight through is a word move per pair - the GL
backends hand gl->vp.dims to their shader parameters as it stands.
The custom viewport the settings own keeps its axes as two members,
under video_viewport_settings_t: configuration.c's SETTING_UINT rows and
the menu's offsetof rows bind an unsigned by address, and half a word
has no address to give them. video_driver_init_internal() seeds it
through a viewport of the packed type and takes the two axes back out.
VkViewport and MTLViewport keep their own width and height.
samples/gfx/viewport_pack holds the layout: the round trip on both
words, one axis put without disturbing the other, and the clamp that
keeps an oversized axis out of its neighbour's half. Transposing the two
halves fails it 29 times.
Row publication notifies a retro_eventcount: a release store for the
row count and a notify that touches no lock unless a thread is parked.
The reference-row wait, the DPB output wait and the context join
register, re-check their own predicate and only then sleep. The
lossless sample links retro_eventcount.c for its threaded build and
keeps it out of the no-threads one.
Row publication notifies a retro_eventcount: a release store for the
row count and a notify that touches no lock unless a thread is parked.
The three waits on it - a reference picture's rows, the output pop and
the context join - register, re-check their own count and only then
sleep. The lossless sample links retro_eventcount.c for its threaded
build and keeps it out of the no-threads one.
The frontend may present on queue even when a core hands back another
queue of the graphics family: where the window system is known not to
tolerate a present on another queue, it does, still serialised with the
core's submissions through lock_queue/unlock_queue. Say so, so a core
does not depend on the separate queue being used. Comment only.
Since 5d950a7288 the frontend presents on the graphics family's second
queue wherever the family has one. Behind Android's WSI the loader
turns a present's wait semaphore into a sync fd through the driver's
QueueSignalReleaseImageANDROID, and Mali (G615 MC6, r44p1, which
reports two queues in family 0) fails that call when the semaphore was
signalled on another queue: every present fails, the failed present
destroys the swapchain, the next acquire rebuilds it, and the menu
freezes in that loop (#19601). A CI build from before 3df4bd91b7 - the
swapchain-teardown hazard fixed in 4823918abe - already froze, so that
fix alone does not cover it.
The choice is made in one place now, vulkan_select_present_queue_index():
queue 1 where the family has a second queue, queue 0 behind Android's
WSI. Android's WSI is VULKAN_WSI_ANDROID, and any context driver in an
ANDROID build, the SDL3 one included. A core that provides its own
presentation queue in the graphics family gets the graphics queue
there as well; one in another family is still refused as before. The
desktop drivers keep the second queue and its lock-free presents.
The vulkan_validation sample checks the policy ahead of its display
check, so it runs on every lane: second queue elsewhere, the graphics
queue behind Android's WSI at one, two and four queues. The old
selection behind the same helper fails it on the Android cases.
A present is a queue operation that vkQueuePresentKHR returns ahead
of: the presentation engine still has to wait on the frame's swapchain
semaphore, and no frame fence covers that wait - the fences are
signalled by the render submission, which the present comes after.
Since 5d950a7288 the present goes to a queue of its own where the
family has one, with nothing on it that a frame fence orders, and
since 3df4bd91b7 a swapchain rebuild or teardown waits on the frame
fences instead of draining the device. Together, a resize or an
out-of-date acquire destroyed the swapchain semaphore, and the
swapchain, under a present still waiting on them.
Mali's Android WSI (r44p1, MT6897) does not survive that: every
present after it fails inside QueueSignalReleaseImageANDROID, the
failed present destroys the swapchain, the next acquire rebuilds it,
and the menu freezes in that loop (#19601: the 09-16 nightly clean,
09-18 failing every frame).
vulkan_context_wait_frames() now also waits out the presents when one
has been queued since it last ran: an empty submission on the present
queue, behind them, signals a fence the rebuild waits on with the
frame fences. One submission per rebuild, nothing per frame, and only
the present queue - a hardware core's work is still not drained. The
shared-queue case takes queue_lock for the submission as any use of
that queue must.
The vulkan_validation sample gets the case: acquire, a fenced
submission signalling the swapchain semaphore, the present, and a
teardown behind it. The validation layer retires a present's waits
when vkQueuePresentKHR returns, so it passed the old code; the sample
now tracks presents on the symbol wrapper's function pointers and
fails when a semaphore or swapchain a present is queued against is
destroyed before a fence behind that present has been waited on. Fails
on the old code (both objects), passes on this.
They go through platform_video with the header each one needs as its
probe, so a runner without the Vulkan or X11 development headers says
so and the matrix passes, the way every other lane whose headers may be
absent already does.
A Linux desktop configuration builds no Vulkan, no Cg and none of the
framebuffer or X11 back ends, so a change to the frame descriptor or the
shader parameter block those read is green here until a job that selects
them runs. Five lanes compile them, and the UI companion's own tests
join its core.
The setting's help described the threads as converting frames to
pixels, which was all they did when it was written; they carry the
decode itself now, and the preview keeps to one thread while a core
is running. The help says so, in terms of what turning it up buys
and what it costs, and names no format.
With pictures decoding concurrently the decoder declined catch-up
drops, on the argument that a non-reference picture runs beside the
chain of references and dropping it buys no time. That holds when the
threads outnumber the pictures in flight and nowhere else: on a
machine whose pool is saturated by a 4K stream, the non-reference
pictures are half the work, and a preview that fell behind stayed
behind. The drop now happens after the slice header is parsed and
before any of the picture's state is taken on - the first slice of a
nal_ref_idc 0 picture and every slice after it - which leaves the
decoder exactly as it was, one picture at a time or four in flight,
as the HEVC decoder already did.
The catch-up oracle gains a build with four pictures in flight on a
pool, and CI runs both - it had built the oracle and never run it.
Every picture shown while catching up is the full decode's, at the
same count threaded as not, under ASan and TSan.
video_frame_info_t carries the content scale and the custom viewport as
packed words. video_thread_get_scale() hands back the word the video
thread published rather than splitting it for a caller that stores it
whole, so the threaded path carries the scale across without touching
either half.
The custom viewport's full size went unread by anything, and goes.
video_shader_ctx_params_t carries the viewport, input, texture and
output sizes as four packed words, so the GL drivers fill four members
per pass and the GLSL backend unpacks them once into the locals its
uniforms are built from.
The last of the three fan-outs that waited for the whole pool to go
idle after handing it a frame's work: the blit's bands and the HEVC
wavefront went before, and the VP9 tile columns go now. The frame's
jobs share a count; the last to finish wakes the caller, who takes
queued work from the pool meanwhile rather than sit idle, and waits
for the frame's own jobs and nothing else. Harmless while VP9 was
alone on the pool, a stall per frame the moment anything shares it.
The tiled fixture hashes the same at every thread count, under ASan
and TSan.
Two built harness binaries had been committed - the ignore pattern
covered <dir>_test and not the _test_threaded and _test_norwebm
names beside it - so every make clean there showed as a deletion;
they are untracked, and the patterns cover names built on _test_ too,
sources excepted.
companion_core, companion_thumbs and companion_dock reach a compiler
only in a Qt or Cocoa build, so a desktop job configured --disable-qt
compiles none of them and a change to a struct they read stays green
until one of those runs. Two lanes compile all three, with and without
threads.
The last patch found the HEVC decoder applying SAO to nothing in a
build without threads, the flag that asks for it having been set
inside a HAVE_THREADS block; the oracle could not have caught it,
being built with threads only. Each oracle's Makefile now builds the
same test a second time without HAVE_THREADS - no pool, no rthreads,
the concurrent legs skipped, every one-thread and rotated leg held
to ffmpeg as before - and the CI sample runner runs both. A decoder
path right only with threads, or wrong only without them, shows in
this lane rather than on a DOS or console build.
Since pictures decode concurrently, a pool given to the decoder is
the pictures', and the wavefront fan-out - a picture's CTB rows dealt
to the pool, with shadow decode states, row progress counters and
the group on an eventcount - could no longer be reached: the two
threadings shared the pool and the one for pictures won, being ×N on
any content rather than on wavefront-coded content alone. What could
not be reached goes, 350 lines and the eventcount dependency with
it; wavefront-coded pictures decode their substreams in order as
they always did on one thread, and the fixture's hash is the same.
With it a bug that came in with the row hook: the flag that says a
picture asked for SAO was set inside the fan-out's HAVE_THREADS block,
so a build without threads applied SAO to nothing. It is set where
the slice data starts, in every build. The header describes the
decoder as it is - Main and Main 10, P and B, WPP, bypass, constrained
intra, pictures concurrent - and what it still refuses.
gfx_anim_preview_t carries the animation's size as one word, packed
from the two the stream fills, and gfx_animation_ctx_line_ticker_smooth_t
its field size. gfx_anim_preview.h takes video_defines.h along, so a
consumer of the header has the accessors.
The plain smooth ticker keeps its field_width: it has no height beside
it, so there is no pair there to fold.
The oracle had no interlaced case: pair scanning, field motion
compensation, the per-pair deblocking and, with pictures in flight,
the rows of a MBAFF picture published only at its completion, were
held to nothing. Three x264 MBAFF cases join it - CAVLC, CABAC with a
B-pyramid, weighted prediction - decoded in all four modes against
ffmpeg, and an MBAFF fixture joins the preview oracles, which run it
plain and concurrently under both sanitizers. All byte-exact as the
decoder stands; a comment claiming B field pictures refused, stale
since the field machinery landed, goes.
Two Main-profile features the decoder refused, so that a stream using
either got no picture at all. Transquant bypass: the CU's flag is
parsed, its residual is its coded levels with no scaling and no
transform, its transform_skip_flag is not read and its signs are not
hidden, and the loop filters leave its samples as they are - the
deblocking filter runs the edge and puts the bypass side back, SAO
passes the samples over. Constrained intra prediction: a reference
sample from a CU that is not intra-coded is unavailable to intra
prediction, the motion field's pred saying which. Byte-exact against
ffmpeg in the four decode modes: lossless 8-bit, 10-bit, with the
wavefront and with B-frames, bypass CUs mixed with coded ones under
deblocking and SAO, and constrained intra in P and B pictures; the
streams that never used either decode as before, to the hash.
HEVC decoding is specified to the sample, so for any conforming
stream the decoder's output is ffmpeg's to the byte, lossless or not;
the decoder had only ever been held to its own output across modes.
rh265_lossless_test builds its cases with x265 - deblocking and SAO,
B-frames and a pyramid, the wavefront and none, 10-bit, a tall
picture with a partial last CTB row, 16-sample CTBs - and decodes
each four ways, one thread, four contexts in rotation, four pictures
concurrently, and concurrently with every row's publication held
back, every way against ffmpeg's frames. Transquant bypass the
decoder refuses, out of its profile; the oracle holds it to
refusing whole rather than decoding wrong.
It found the row hook filtering a CTB row the moment it was
reconstructed, when the next row's intra prediction reads this row's
last line as reconstructed and unfiltered. The hook filters the row
above now, once nothing will predict from it, and SAO runs two rows
behind; a row is final below what SAO has been over. Both preview
fixtures were wavefront-coded, where the hook never runs, which is
how the hashes stayed equal: a non-wavefront fixture joins them, and
the preview oracles run it plain and with rows held back. The CI
sample runner takes the new oracle.
The viewport and video-mode commands, the menu texture and the frame
slot each carry their size as one packed word, so a frame's size
reaches the video thread in one store and is read back in one load.
The packet union's unused 'output' member goes with them.
The field file that found the multi-slice fault kept a residue of
1,166,172 differing samples at one thread, all in the last picture.
It was the reference: ffmpeg duplicates pictures to hold a file's
declared rate against its timestamps, so a file of 127 packets gave
129 frames, and the decoder's last picture - correct to the byte -
was compared against a copy of the one before it. Every reference
the oracle makes now takes -fps_mode passthrough, and the file mode
makes its own reference that way from the file alone, RH264_REF
naming one made already; its exit status is the comparison. The
field file decodes byte-exact at one thread, four contexts and four
threads.
The fourth step, and the arc's point: with a pool and more than one
thread, HEVC pictures decode concurrently. A picture's slices are
queued on its context as they arrive - each job takes the unescape
buffer it was parsed from, the decoder draws another from a pool, and
the slice header and lists go with it - and go to the pool when the
next picture opens, at which moment the picture's place in the
sequence is settled, out of the decoding slot and into the output
order, with its rows still counting up. A reference read sleeps for
the rows it reaches and no more; a picture holds the slots it reads
so the DPB hands none of them to a new picture; the output holds a
picture until it is complete and waits on its last row only when the
queue is full; the drain and the close land everything first. The
wavefront stays off while pictures are on the pool - the pool is the
pictures'. The delay knob holds every row's publication back at
random, so that the readers wait rather than nearly always find their
rows, and the output must not change under it.
Three things the sanitizers found on the way, all outside the
scheduler: the pool entry and the close changed state under pictures
in flight, and join first now; the catch-up flag crossed from the
caller's thread to the decode's as a plain int in both decoders, and
is atomic in both; and the preview's resident-bytes diagnostic read
the window's bookkeeping while the feeder moved it, and reads a value
the feeder publishes instead.
Byte-exact against one thread on both HEVC fixtures at two and four
threads, plain and with rows held back, under ASan and TSan; the
threaded preview harness at four threads, both codecs, under both.
The check target and CI run the concurrent and the held-back lanes.
struct gfx_display carries its framebuffer size, the gathered batch's
first rectangle and the batch's video size as three packed words;
dispgfx_widget_t its last video size and the AI service overlay's; and
gfx_thumbnail_t its own. The batch compare that decides whether a quad
can join asks one word per pair, and a thumbnail takes its size from
the surface as a single copy.
The AI overlay's loader keeps two locals for the decoder's out-params
and packs the answer, since the decoder fills them separately.
The decode state - the parameter sets and slice header the picture
decodes with, its reference lists, its maps of intra modes, coded
depths, edge strengths, QPs and SAO parameters, its scratch, its
SAO bands - was one struct in the decoder; it is one per context now,
eight at most, and a new picture takes the next context round, which
readies its own maps for the decoder's geometry while the DPB is
remade only when the geometry changes. With one context that is the
decoder as it was. The pictures still decode one after the other, so
rotating changes which memory a picture uses and nothing else: the
WPP fixture and the B-frame fixture hash the same at four contexts as
at one, and the WPP oracle now decodes at four - a difference would
be a piece of picture state still in the decoder rather than the
context, shared between pictures the moment they decode concurrently.
This is the third of H.264's four steps; the scheduler is what
remains.
VIDEO_SCALE_PACK converts an axis before it compares it against the
limit, so a caller holding its sizes in int or float packs without a
sign-compare warning and without a cast of its own. The clamp is
unchanged: at the limit, past it, negative and fractional inputs all
land where they did.
Each picture in the DPB counts the CTB rows of it that are final -
reconstructed, deblocked, SAO'd, written by nothing again - counted
up by the row hook as the filters pass and set to the whole picture
at the finish. What is final is what SAO has been over, since the
row above's SAO reads this row's first line and the edge at the top
of the row below still touches this row's last; without SAO, what is
deblocked below the row still being filtered against. Every read of
a reference asks for the row it reaches: the motion compensation for
the block's last line displaced by its vector and the eight-tap
filter's four lines past it, temporal prediction for the co-located
row. On one thread a reference is complete before it is read and the
ask is answered at once; a picture on another thread will wait
there, and nowhere else. The WPP oracle asserts that no read on one
thread or on the wavefront finds its rows short, the bench reports
the count beside the hash, and both fixtures hash as before.
The deblocking filter ran over the whole picture at the finish, and
SAO after it. Both go a CTB row at a time now: when a row is
reconstructed its edges are filtered - vertical, then horizontal from
its top edge down, which reads only lines the band above has already
filtered vertically and reaches no line the band below has yet to -
and SAO runs over the row above, whose last line reads this row's
first as deblocked. Done band by band from the top that is the
picture's filter to the sample; the WPP fixture and the B-frame
fixture hash the same as before at every thread count. The finish
does what the hook left: the last row's SAO, or the whole picture
when the hook could not run - the wavefront, or more than one slice.
This is the first of the four steps H.264 took towards pictures
decoding concurrently: rows that become final one at a time, for a
later picture to wait on.
The headless OpenGL context includes the headers that declare
gfx_ctx_driver_t, gfx_ctx_proc_t and input_driver_t, so a build that
configure enables from its pkg-config probe compiles. bind() takes its
address as a struct sockaddr, and the unused second sockaddr goes.
compile-matrix gains eleven context-driver lanes, osmesa among them:
these answer the frontend's window and size questions and no other job
here compiles them, so each lane names the headers it needs and says so
where the runner has none. vivante_fbdev stays out, since only that
vendor's EGL headers declare the functions it calls.
A file whose every picture is coded as eight slices rendered at one
thread and not at two or more. rh264_video_decode fed a sample's NALs
until a picture was shown and stopped - one coded picture per call,
which sequentially is exact: the picture shown is this sample's,
complete at its last slice. With pictures in flight the picture shown
at a slice's open is an earlier one, and stopping there left the
sample's remaining slices unfed: rows of the first slice decoded and
the rest never, the picture never complete, every picture predicting
from it waiting for rows that could not come. The call now ends with
the sample. Two more that the same file found: a continuation slice
was refused because its start was checked against a picture end the
job had not yet reached - the slices must arrive in raster order at
submit, and the job checks each against the last one's end when it
runs them; and a picture that refused published no rows on the
inline path, which was the hang. Every picture is published whole
once its run is over, however it ended.
The oracle gains an eight-slice case with B-frames, decoded in all
four modes, and a mode that decodes one file against its ffmpeg
reference at one thread and at N - the file from the field, and the
next one. The preview sample's threaded target was listed in all:
before its variable was set, so make read an empty prerequisite and
CI found no binary; the definitions come first now.
The lane for the two preceding fixes. It asks each filter which
parameters it reads, then re-inits it once per parameter per poison
value - zero, negative, huge, infinity, NaN, and the int edges - across
five sample rates including zero, and runs audio through both the float
and the int16 path. Discovery is by callback, so a parameter a filter
gains later is covered without touching the harness.
Each filter is swept in its own process, so one that dies still leaves
the other fourteen reported, and an alarm turns a loop the preset sizes
into a named failure rather than a hang. Built with
-fno-sanitize-recover, a sanitizer report is a trap, which the parent
reports the same way.
Run it as: make SANITIZER=address,undefined,float-cast-overflow check
Not wired into .github/workflows/Linux-samples-audio.yml, which names
its directories one at a time.
The same class as the vibrato and tremolo fix, across the remaining
filters: a preset value reaches a Q16/Q24/Q30 conversion, a divisor, an
allocation size or a loop bound without one.
Clamp each parameter at init to the range its buffers and fixed-point
format hold, with the NaN-safe ordering. The bounds that are not simply
the control's own range: chorus takes the modulated delay against the
ring, since the int16 LUT carries it as Q16 samples; chorus lfo_freq
sets the LUT length and the loop that fills it; eq block_size_log2 is a
shift count, an FFT order, an arena size and a block divisor at once;
iir quality divides the biquad alpha, as wahwah resonance does.
Refuse a zero input rate, which is what a filter is handed when the
audio device never opened, and floor the reverb delay lines at one
frame - at a low enough rate every comb length rounds to zero and
comb_process reads past a one-byte arena.
Saturate the Q16 state in echo and reverb. Both let a preset set a loop
gain of exactly 1, and echo's amp sits inside the feedback path, so the
state grows until the int64 product overflows. The float paths go loud
instead; the ceiling is 2^20 times full scale, so a setting that decays
never reaches it and its output is unchanged.
A .dsp preset is a text file the user picks, and rstrtof parses "inf"
and "nan", so every parameter reaching these filters can be zero,
negative, enormous or non-finite.
Vibrato divides by an LFO period it re-derives per sample from freq
without the clamp vibratocore_carve applies, so a freq above the sample
rate truncates it to zero and the modulo raises SIGFPE; take the clamped
core->maxphase instead, which also drops a division from the per-sample
path. Tremolo takes its wavetable index modulo a length derived the same
way, with the same result on both the float and the int16 path.
Overdrive, early reflection reverb and WSOLA clamp their parameters, but
the clamp returns NaN unchanged because NaN compares false against both
bounds. In overdrive that reaches the Q20 bias, and the tanh LUT is then
indexed out of its 4097 entries - a segfault. Order the comparison so a
NaN lands on the low bound.
Clamp vibrato and tremolo's freq and depth to the ranges their buffers
are sized for, floor the vibrato delay line at the four samples the
Hermite tap reads, and refuse a zero input rate, which is what the
filter is handed when the audio device never opened.
The VITA, Wii U and PS3 centring path offsets y by the icon's height,
so a non-square icon drawn with a scale factor below 1 sits centred on
both axes.
The VITA, Wii U and PS3 path takes its scale in the icon size, so the
size is packed already scaled rather than multiplied afterwards.
compile-matrix gains five lanes that compile the shared menu and video
files under the console platform macros, against the stubs already in
the tree. The blocks those macros select are not compiled by any
desktop job, so a change to a struct they touch stays green until a
console job runs; these fail by name instead.
A streaming voice sized its resampler output buffer as
AUDIO_MIXER_TEMP_BUFFER * ratio plus a fixed 16 samples, then stored
whatever process() reported into it. The pad does not scale, but the
overshoot does: a call ends on a phase it did not start on, so roughly
one input frame's worth of output lands on top of the estimate, and
that is ratio samples, not 16.
Measured with the mixer's own constants and the sinc driver: at ratio 6
(8 kHz source into a 48 kHz mixer) the buffer holds 49168 samples and
process() reports 49166 - two to spare. At ratio 12 it holds 98320 and
process() reports 98330, and at ratio 24 it is short by 34. Both are
ordinary sources; the buffer is on the heap, so the excess lands in
whatever follows it.
buf_samples was written at both allocation sites and never read - the
bound was recorded and not enforced.
Scale the pad with the ratio, as one_shot_resample() already does for
the non-streaming path, and clamp the reported count to buf_samples at
both fill sites, so a backend that reports more than the estimate
truncates rather than overruns.
The audio_mixer_rate_guard harness gains a streaming lane driving
64 mix calls per voice at 44.1k, 8k, 4k, 3k and 2k sources into 48 kHz
and 96 kHz mixers, both sample formats. Reverting this change makes it
report a heap-buffer-overflow in resampler_sinc_process_sse_kaiser
under ASan.
exynos_data carries the framebuffer size as dims, exynos_video the
core's current mode, and the image defaults table its own sizes, all in
VIDEO_SCALE_PACK's layout. The frame path compares one word to decide
whether the core changed resolution, and the scaling maths unpacks once
into locals.
compile-matrix gains an exynos lane against libdrm's exynos module and
its G2D headers, which nothing else in the tree compiles; it skips
where those headers are absent. platform_video's failure output now
goes through show_out, so a lane that fails leads with its error rather
than whatever warning came first.
The sample adaptive offset reads the deblocked samples around each one
and writes in place, and it did that from a copy of the whole picture
- three planes malloc'd, twelve megabytes at 4K, copied and freed for
every picture with SAO on. A band of one CTB row and a line above and
below, kept with the decoder, serves the same reads: the line above
comes from the previous band, where it is still as deblocked, the rest
from the picture ahead of the pass. The output is the same to the
hash on the WPP fixture, at every thread count. The preview bench
takes BENCH_HASH, a hash over every frame's pixels, which is how two
builds of a decoder are held to the same output on a file.
The WPP rows went to the pool and the caller waited for the pool to
fall idle - a wait for everything on it, which is fine while a pool
carries one picture's rows and nothing else, and a stall on every
picture once pictures decode concurrently on the same pool, as the
H.264 decoder's do and this one's are to. The rows belong to a group
on the heap now: the runners claim them, the caller too, and the
caller waits on a process-wide eventcount for the rows in other hands
and then for the runners to let go - a runner still queued behind
other work finds its rows gone and leaves at once, but until it has,
the shadows and counters it can reach are not the next picture's to
reset; that was the race the first shape of this had. Byte-exact on
the WPP oracle across the thread counts, under ASan and TSan.
The lane's non-blocking check timed six posts against a fixed 5 ms
budget, which on a two-core WARP runner under gdb measures how often
the poster is descheduled rather than whether the post waits on the
video thread: it reports ~19 us here and failed CI at 5649 us.
The first load now parks the worker inside the upload, where it runs
with no lock held and with nothing on the main thread waiting on it.
The remaining five are posted into that, and the lane asserts they
came back while the worker was still in there - the worker's own
state answers, so the check holds however loaded the machine is. A
post made to wait on the video thread fails it with the hold's own
bound.
gfx_display_ctx_draw holds dims in VIDEO_SCALE_PACK's layout, so the
size reaches every display backend in one member. The producers pack
where they already had both axes in hand, and each backend unpacks
where it builds its viewport or its quad.
samples/gfx/gfx_widgets_sanitize gains an icon-draw lane: it draws at
64x16 and checks both halves of the word the widget layer hands the
driver, so a transposition on the way in fails there. The Direct3D
lanes in the compile matrix cover the six backends no other lane
compiles.
The plan for threaded previews had the worker count defer to a
running core, and the setting was taken as it stood: a 4K preview at
four threads under a menu open over running content took the cores
the game was on. The count is the setting unless content is loaded
and not paused, in which case one - the preview decodes as it did
before the setting, and the pool is remade as the count changes,
which it already was.
Both drivers validate the nominal ratio when the instance is made and
then take a different number on every call: audio_driver_effective_ratio()
multiplies by slow motion and fast-forward, and audio_driver_bound_ratio()
returns the ratio untouched wherever the caller reports no capacity of
its own. Nothing checked that one.
sinc derives its phase step as phases / data->ratio and nearest as
1.0 / data->ratio, so a ratio of zero, a negative one, a NaN or one
larger than a pair of sample rates can name leaves a step that never
advances the loop, and the inner loop emits past data_out until it
leaves the buffer. The integer drivers reach an int64 cast on the same
value, where the conversion is undefined.
Each dispatcher now holds the range its own arithmetic needs before it
reaches any of that. sinc uses sinc_resampler_ratio_valid(), the
predicate its init already refuses on - it classifies by bit pattern,
so it survives the float driver's fast-math mode - and nearest bounds
the step the same way the CC driver does.
resampler_reset drives all three backends through the ratios a clamp
would have caught, instead of CC alone with a note that the other two
were theirs to answer for. Reverting either fix turns that lane into an
ASan heap-buffer-overflow, in resampler_sinc_process_c_kaiser and in
resampler_nearest_process.
Each slice's job was made with calloc and freed after - two kilobytes,
a B context inside, zeroed and returned per slice on the submitting
thread. A finished job goes to a pool on the decoder beside the
unescape buffers, and the next slice takes it from there, cleared;
its buffer is settled - pooled or freed - before the job is offered,
since a pooled job is another thread's to take. With this the per-
picture path on the submitting thread makes no allocation in steady
state: blocks, buffers and jobs all cycle.
The preview harness built the thumbnail without HAVE_THREADS: the
decode ran inline on the poll, no worker, no pool, and the paced
stall check could say nothing about the app, where the picture
decodes on the worker and, above one thread, on the pool. The same
test now builds a second way, gfx_thumbnail_preview_test_threaded,
with rthreads, the pool and the eventcount real and the stubs' thread
shims compiled out; THREADS sets the count as the menu setting would.
The check target and the CI job run it at four threads, paced, with
a bound on the longest stall, under ASan and under TSan - the first
time the worker, the pool and the poll have been under a sanitizer
together. Clean.
Three sums of file-derived sizes were unchecked: twice what the decoder
took since the last feed, the decoder's position plus the lookahead,
and the same for the audio window. On a normal file none can wrap; on
one that can, the first would budget a decoder that took the most next
to nothing and the others would clamp a wrapped bound against the
length and repair nothing. Each saturates.
Seven lanes compile d3d8, both d3d9 backends, d3d10, d3d11, d3d12 and
gdi against the DirectX headers the tree vendors in gfx/include/dxsdk,
through a mingw-w64 cross compiler. These drivers read the same frame
and draw descriptors every other video driver does, and a descriptor
field they are not carried along with fails all seven by name.
The cross compiler is the only thing needed beyond the tree; where
there is none the lanes say so and the matrix passes as before. A
runner installing gcc-mingw-w64-x86-64 turns them on.
The queued job copied the unescaped slice out of the context's scratch
into a buffer made for it - a megabyte moved and an allocation freed,
per slice, on the submitting thread, for bytes that already sat in
memory nobody else needed. The job takes the buffer the slice was
unescaped into - whichever context's was current then, the one before
the rotation when the slice opened a picture - and that context draws
its next from a pool of buffers on the decoder; a finished job returns
its buffer there. Workers return and the submitter draws, under the
blocks lock when there is a pool. Byte-exact in every mode, clean
under the sanitizers.
gfx_surface_t holds dims in VIDEO_SCALE_PACK's layout. The thumbnail
animation path compares one word to decide whether a surface still fits
the frames it is being handed, and the overlay and swizzle paths unpack
where they compute a frame length.
rarch_softfilter_get_output_size() and get_max_output_size() hand the
size back as one word in VIDEO_SCALE_PACK's layout. The plugin ABI is
untouched: a filter still answers through two pointers, seeded from the
word so one with no size query of its own leaves it as the caller had
it.
video_driver_init_filter() carries the packed size from the geometry
through to the buffer allocation, and the frame path, the threaded
wrapper and the recording setup each hold one local for the pair.
samples/gfx/filter_lifecycle checks the output dimensions it gets back,
so a halved unpack fails there.
Every output slot was allocated a sample block of its own - twelve
megabytes at 4K, zeroed - to have a frame picture's block shared over
it on the spot, and once a picture had been shown the slot kept the
block until it came round again, which with eighteen slots is a dozen
pictures later. So the free list stayed empty and every picture paid
a fresh block: a mapping from the kernel and its zeroing on the
submitting thread, per picture, and the memory of a dozen pictures
pinned in slots nobody read. A slot for a frame picture is now made
as a shell, bookkeeping only, and a slot that has been shown and is
not on show lets its block go the moment the next picture is pushed.
Field pairs, which copy, keep a block. The free list holds sixteen.
CC_RESAMPLER_PRECISION above 4 selects a trig kernel only the reference
implements. Two things followed from that and neither was intended.
It did not compile: cc_int() uses M_PI and sinf() at that setting while
the file included neither <math.h> nor retro_math.h, so the mode the
header calls full precision has been unbuildable under -std=c89. Both
includes are there now, and 0, 1 and 5 compile clean on x86 and armhf.
And since the arms became a runtime choice, building it would have left
one binary resampling differently depending on which arm the mask
picked - the reference on its trig kernel, the vector arms on the
polynomial they are the only implementation of. Above 4 the vector arms
are no longer built, so the setting means the same thing whatever the
mask says.
resampler_simd reads the same condition, so at that setting it reports
that there is no vector arm rather than comparing the reference against
itself.
What the setting buys, measured at 32 to 48 kHz: about 21 dB of THD+N
at 100 Hz, 8 dB at 500 Hz, 3 dB at 1 kHz, and nothing at all above
2 kHz, where the width of the kernel is the limit rather than the way
it is evaluated.
The feed runs once per frame, on the decoding thread, and read half a
megabyte a tick. A decoder with pictures in flight takes its samples
in bursts of three and four a frame, more than that refills, so the
lookahead ran down each few seconds - and then the feed's answer was
to extend unpaced: two seconds of 4K read from disk in one go, on the
thread the frame was waiting on. A pause every few seconds, in a
preview that decodes faster than it plays.
The budget is now at least twice what the decoder took since the
last feed, so a decoder that takes four samples a frame is kept ahead
of without the lookahead ever running down; and a lookahead that has
run down is made up a quarter of the shortfall a tick rather than at
once. Only a decoder already past the frontier gets the unpaced
extend, since its next read would fault.
4d5f87f added audio_resampler.c to this sample's link, and that unit's
resampler_config table points at config_userdata_get_float() and the
rest of the config_userdata family. Without a sanitizer --gc-sections
drops the table and nothing notices; under
SANITIZER=address,undefined it survives and the link fails with
undefined references, which is how the Linux audio samples job broke.
Stub them the way resampler_options_test does, aborting since nothing
here passes backend config userdata, and mark them externally_visible
so -fwhole-program on extra_capacity_test.o does not internalise
them.
audio_driver_flush() points the resampler at the core's own buffer when
nothing needs to be produced into input_data, and fell back to the copy
unless that pointer was 16-byte aligned. The requirement was the CC
resampler's ARM assembly, which loaded its input with an explicit
alignment hint; that file is gone and nothing has taken its place.
Every consumer of src_data.data_in reads it without one: sinc pushes it
into its own ring a sample at a time, nearest copies frame structs, CC
takes it with _mm_loadl_pi and vld1_f32, and the bypass clamp uses
_mm_loadu_ps and vld1q_f32. So the check only decided whether a batch
was copied, and a core-owned buffer under an allocator that aligns to 8
paid for it every flush.
Worth about 1% of the resample where it fires - 512 stereo float frames
at 48/32, best of 9, 0.0862 s against 0.0871 s through CC and 0.1266 s
against 0.1283 s through sinc. Small here, and this is the host with the
fastest memcpy and the batch already in L1.
The harness builds its test object with -fwhole-program and overrides
retro_resampler_realloc_hq(), so it has always supplied the resampler
layer rather than linking it. When retro_resampler_int16_new() arrived
the layer was added to its sources instead, which links
audio_resampler.c's driver table and with it the config_userdata_*
callbacks the harness does not define. The plain build survived because
-fwhole-program lets the linker drop the table; the sanitizer build did
not, so the suite could not be sanitized at all.
It supplies the factory too now, against the sinc int16 driver its lanes
already drive, and links neither resampler-layer unit. Both build modes
pass, ASan and UBSan among them.
sinc_resampler_int16.h promises NULL when the nominal ratio cannot
safely advance the phase clock, and both sinc drivers enforce it before
they configure anything. CC took any ratio and only refused per call,
through the output bound, so a core reporting no rate gave a resampler
that created cleanly and then produced nothing, where sinc gives the
frontend a failed init and a log line.
Both CC drivers now hold the same range at init that process() holds:
positive, and no larger than a pair of sample rates can name. The
predicate is the one the output bound already used, named once.
resampler_options pins it for the float driver and stretch for the
integer one; without the check each reports three failures.
Introduce a new "release month" explore category. Adds EXPLORE_BY_RELEASEMONTH and its explore_by_info entry in menu/menu_explore.c, and the corresponding MENU_ENUM_LABEL_VALUE_EXPLORE_BY_RELEASE_MONTH in msg_hash.h to enable browsing by release month.
omap_video_t carries the current mode as dims and omapfb_data_t the
native screen size as nat_dims, in VIDEO_SCALE_PACK's layout. The
frame path compares one word to decide whether the core changed
resolution, and the font, viewport and menu-texture paths unpack once
into locals.
The overlay's six references to the cached frame's dimensions unpack
from the word frame_cache_peek() hands back, so the block carries one
local for the pair.
Two threads gave a picture in flight and nothing else: the pool had
one worker and the submitter sat idle at the join. The pool gains
tpool_help(), which runs the job at the head of the queue on the
calling thread, and the join takes queued pictures that way before it
sleeps - the one it waits for, or an earlier one the workers have not
reached; the head is taken as a worker takes it, so a picture never
waits on one still queued. The contexts number the threads plus one,
up to eight, so the pictures in flight are the threads that can
decode them and one being built.
Catching up dropped the pictures nothing references. Decoding
concurrently those cost nothing - they run beside the chain of
references, the critical path either way - so dropping them bought no
time and emptied the pool: a preview that fell behind stayed behind,
on the pictures it had left, in fits. Threaded, the drops are
declined; on one thread they are what they were - except that a
dropped picture now says so, which lets the container pass its slot
on the timeline: the flag was never set here, so every H.264 preview
that caught up read its frame durations one slot behind.
The mixer's repeat rewinds an AAC stream through audio_transfer_seek's
own frame-zero path, not through the seek-to routine, and only the
latter withdrew the window bound on a rewind. So a looping preview
still read the head at once - the faulting instruction is the first
byte of the head packet, inside its declared size, on a page the
window gave up a lap ago. Both rewinds withdraw the bound now, through
one helper, and the reads stand at the wall until the feeder brings
the head back and publishes a bound for the new lap.
The bound the feeder publishes is what lets the decoder stand at the
wall on a loop instead of reading the head out of pages the window
gave up - and it was published at open only for the island layout.
For every other file the decoder trusted the demuxer's offsets
outright the whole lap, and at the lap's end looped and read the
head before the feeder's next tick brought it back: the access
violation in the AAC decoder, in the faulting instruction's own
terms, a byte inside the packet's declared size on a page that is
not there. The bound is published for every windowed stream from
its first read.
frame_cache_snapshot(), peek() and store() hand the pair across as the
one word it is stored as, so the seqlock body is a straight assignment
and the four consumers that need separate axes unpack once, at the
point of use. The two that only forward the pair and the one that reads
only the pointer keep it packed.
video_driver_cached_frame_publish() compares one word where it compared
two, and the lock-based branch holds one field where it held two.
samples/gfx/threaded_video's size-pair round-trip lane covers this
path: a halved unpack anywhere along it fails the lane by name.
The preview's audio decodes on the audio thread out of a sliding
window the menu thread feeds, and the feeder lets the head go as the
lap goes on. When the audio reaches the end it loops, on its own
thread, and reads the head at once - pages the feeder gave back long
ago, and can only bring back on its next tick. That read is the
access violation in the AAC decoder when a preview reaches its end.
On a loop the decoder now stands at the wall: the bound the feeder
published for the old lap is withdrawn, so every read waits, and the
feeder, seeing the position fall back, forgets the bound it published
too, brings the head back and publishes one for the new lap. The
decoder resumes on it. A whole-file buffer has no window and no bound,
and loops as it did.
dc98c1a moved image_blit_bands onto retro_eventcount, but
tools/companion_thumbs_test.sh links image_blit_bands.c without
retro_eventcount.c, so the link fails with undefined references to
retro_eventcount_init / notify / prepare_wait / commit_wait /
cancel_wait. Add it to the script's source list.
CC picked its kernel with a preprocessor chain: SSE under __SSE__, the
external NEON assembly under HAVE_ARM_NEON_ASM_OPTIMIZATIONS, the C
reference otherwise. That macro is defined nowhere, so every ARM build
took the scalar path while sinc - which keeps a NEON intrinsics kernel
beside its assembly one and dispatches on resampler_simd_mask_t - was
vectorised on the same hardware.
Every arm is now built and init() picks the pair the mask names, as
sinc does. The kernels are NEON intrinsics rather than assembly:
vzipq_f32 gives the lane spreads _mm_shuffle_ps built and vcombine the
half splices, so the two arms are the same arithmetic in the same
order. Multiply and add are kept separate rather than vmlaq_f32, which
would round once where the other arms round twice.
cc_resampler_neon.S goes with it. It addressed struct offsets that no
longer held, read its ratio with vldr s4, [sp] under a hard-float
platform, and had no output bound; it is in the history if the
alignment hints are ever wanted back.
samples/audio/resampler_simd drives every arm a build has over the same
stream - upsampling, downsampling, unity, rate control's excursions -
against the scalar reference, and skips an arm this build lacks rather
than comparing the reference against itself. The arms agree to one
ulp, not to the bit: cc_int()'s polynomial has double literals, so the
reference rounds through double where the vector arms stay in float.
Swapping the two NEON lane spreads moves the worst difference from
1.19e-07 to 1.05.
Built armhf with -mfpu=neon and run under qemu-arm: same frame counts
as the reference, same one-ulp agreement. Measured on x86, SSE is 4.84x
the scalar arm upsampling and 2.98x downsampling; the qemu figures say
nothing about speed and are not offered as such.
No AVX arm. One was written and measured: correct to the same ulp, and
6% slower than SSE at 48/32. The kernel is four taps wide, so a 256-bit
multiply saves one instruction and costs three - an insert to spread the
weights, an extract and an add to fold the halves. sinc gains from AVX
because it streams 128 taps and more.
The width/height pairs that travel as a single word all use
VIDEO_SCALE_PACK and its accessors now: the threaded wrapper's
viewport slots, the cached frame's dimensions, the output size, and
record_video_data, which carries both axes as one member. The macros
move to video_defines.h so the record drivers reach them. Clamping
applies everywhere the macro does, so an axis past 65535 saturates
rather than wrapping into a small number.
samples/gfx/threaded_video gains a size-pair round-trip lane: it
publishes at dimensions no square frame can hide, reads both axes back
through the public accessors, and checks the clamp at the top of the
range. A store and a load that disagree on which half holds which axis
fails it by name.
5f19526 added samples/rthreads/tpool_parallel_test, but the samples
workflow fails any built target that is in neither RUN_TARGETS nor
NORUN_TARGETS, so the job has failed since. The sample is a
self-contained pass/fail test that runs in about 1.2 s and passes on a
one-core machine, so it goes in RUN_TARGETS beside tpool_wait_test.
The row bands of a blit went to the pool and the caller waited for
the pool to fall idle - which, once the pool also carries pictures
decoding for the frames to come, is a wait for every one of those on
every frame. A sixteen-core run of the concurrent H.264 decoder had
three pictures in flight and one core's worth of decoding, and this
was why.
No band belongs to a runner now. Whoever runs - the caller included -
claims the next band from an atomic index and does it until none are
left; a runner the pool starts late, behind a picture, finds them all
claimed and leaves. The caller never sleeps while there is a band it
could do itself, and waits only for the last few in other hands, on a
process-wide eventcount: no lock, no allocation but the group's, and
a syscall only when there is nothing left to do. The group is on the
heap and the last holder frees it, so a late runner has something to
find. Byte-exact on the blit-bands oracle under ASan and TSan.
The integer counterpart of a float backend was selected by an
if-chain on short_ident, written out three times in audio_driver.c -
the front pair, the extra-channel pairs and the microphone - and a
fourth site did not select at all: audio_mixer.c called
sinc_resampler_int16_init() outright, for both the one-shot WAV
conversion and the streaming s16 voice. So with CC or nearest chosen,
the game stream went through the selected backend while menu sounds
and BGM went through sinc, and those voices honoured Resampler Quality
while the stream did not.
retro_resampler_int16_new() is that selection, once, in
libretro-common. It resolves the name through the same lookup
retro_resampler_realloc() uses, so NULL or an unknown name lands on the
fallback, and it hands back the instance with its process, reset and
free - zeroed when the backend has no integer implementation, which is
the caller's signal to keep the float path. The quality mapping onto
sinc's own enum, which audio_driver.c and audio_mixer.c each carried a
copy of, lives there too.
It goes in its own unit rather than audio_resampler.c: a target that
never asks for an integer backend links none of the three int16
drivers, which putting it beside the driver table would have forced on
every consumer of audio_resampler.o.
audio_mixer_play_s16() takes the resampler_ident its float twin has
always taken, and the s16 voice carries the whole handle rather than a
bare pointer with sinc's functions assumed around it.
The HQ policy now reads RESAMPLER_CAP_HQ_OVERSAMPLE rather than
comparing the backend against &sinc_resampler, so extra_capacity's
stand-in for "another backend" clears that bit instead of relying on
its address.
Resampler Quality and HQ Oversampling were shown whenever the resampler
was not "null" (menu_displaylist.c:9149), while only sinc reads either.
With CC or nearest selected both rows were live controls that changed
nothing, and Quality cost a CMD_EVENT_AUDIO_REINIT to do it. On PSP,
where CC is the default driver, config.def.h even defaults Quality to
RESAMPLER_QUALITY_LOWEST.
retro_resampler_t gains caps, a RESAMPLER_CAP_* mask, last so a backend
that leaves it zero offers no control. sinc sets both, nearest and CC
zero. audio_resampler_driver_caps() answers for a name through the same
lookup realloc uses, so an unknown name reports the fallback's. The two
audio-page rows and the microphone page's quality row are shown against
it, and the HQ policy in audio_driver_resampler_realloc() asks the mask
rather than comparing the backend against &sinc_resampler.
resampler_options pins the mask per backend and through the name lookup,
including the fallback.
The ident-lookup suite built only sinc and nearest, so nothing checked
that "cc" and "CC" reach CC_resampler through find_resampler_driver(),
or that the HQ request and the quality tier - neither of which CC reads
- still give one stream. It links CC now and covers both.
The suite freed its handle at the end of the nearest block and left the
pointer, which the next realloc would have freed again; the new block
clears it first.
Dead with it: CC_RESAMPLER_IDENT, defined three times and referenced
nowhere; HAVE_CC_RESAMPLER in Makefile.griffin, a make variable the
file never reads, since each platform passes -DHAVE_CC_RESAMPLER in
PLATCFLAGS; and cc_resampler_neon.o on Vita, which is built and linked
there although Vita defines neither HAVE_CC_RESAMPLER nor takes CC in
griffin.c, under PLATCFLAGS carrying -mfloat-abi=hard - the ABI those
kernels' vldr s4, [sp] is wrong for.
_ftime64_s fills a __timeb64 - seconds and milliseconds - and the
timed wait handed it the timespec it was about to give
pthread_cond_timedwait, which did not compile under a recent MinGW
and would have read garbage under an older one. The clock goes into
its own struct and is converted.
A sixteen-core run had the pool's CPU time equal to the wall time:
however the pictures were posted, they decoded one at a time. The
decoder counts the most pictures its pool had decoding at one moment,
and the bench prints it beside the row waits. And rthreads gets a
sample that settles the pool by itself: four jobs each burning a spell
of CPU, posted to a pool of four, must run together and finish in
about one spell; jobs that never overlap fail it, and a machine that
gives the process one core is told apart from a pool that does not
wake its workers.
Four variables become two, and each site that used both now reads one.
The content scale the viewport maths produces was two fields that the
threaded wrapper then packed by hand into scale_packed; it is one field
in that same layout, so the wrapper publishes it with a copy and only
one place knows the packing.
The recording state's two pairs go the same way. A resize check that
compared width and compared height is one comparison against a freshly
packed viewport; the GPU size drops four size_t-to-unsigned casts with
the field; and --size's override is one test for "was it given" rather
than two. That override is now clamped at 65535 an axis by the pack,
which is past any size a recorder here opens.
video_frame_info_t keeps its own scale pair: it feeds the float
arithmetic the shader chain wants, where a pack would only be unpacked
again.
Verified: make check on plain, address+undefined and thread builds;
compile-matrix 112/0; C89_BUILD from a clean tree; hot-function-budget;
the tree builds release and C89 clean.
Width and height go in one slot and the full size in another, width in
the high half, the layout scale_packed in the same struct already uses.
Four slots instead of six: two stores a publish, two loads a read, and
each pair is whole before the sequence is even checked.
Verified: make check on plain, address+undefined and thread builds; the
forced-atomic-backend loop over C11/GCC_NEW/SYNC/PS2/VOLATILE;
compile-matrix 112/0; C89_BUILD from a clean tree; hot-function-budget.
The S3 cloud-sync backend only griffin includes, and nothing defines
HAVE_S3 for it at all; the Lakka wifi driver needs HAVE_LAKKA. Neither
is in the C89 build's object list, so neither was checked, and both had
a mixed declaration - s3_parse_url declaring a size_t after a log call,
connmanctl_tether_start_stop an int after a popen.
A failing lane also used to print the first twelve lines of the
compiler's output, which for a file with a page of warnings ahead of
its one error showed nothing to act on. The reporting moves into one
helper the four call sites share, and it prints the errors when there
are any.
Verified: 112 lanes, 0 failures, from a clean tree; C89_BUILD clean;
putting either declaration back fails its own lane.
The pipeline counters showed a sixteen-core run with one picture in
flight and nothing blocking the submitter, which leaves the pictures
serialising on each other. Two more numbers say whether that is so:
the reference reads that had to wait for rows, and how many rows short
they stood when they did, process-wide and reset on reading. And the
bench splits the CPU time of a run between the calling thread and the
pool (CPU_SPLIT=1), so a submitter busy the whole time is told apart
from one that only waits.
The two newest binaries were left behind by clean, the .exe forms
along with them, so a rebuild after a pull could report the bench up
to date against sources it had never seen.
A four-core run of the concurrent decoder showed no gain, and nothing
in the tree could say why. The decoder counts what its pipeline does:
pictures posted to the pool, the pictures in flight at each post,
posts that found the most possible in flight, joins that had to wait
for a context, output pops that held a due picture for being
incomplete, and pops that waited for one with the queue full.
rh264_video_stats() reads them, the container and the image layer hand
the decoder out for a bench to ask, and the preview bench prints the
line under each thread count. Those six numbers tell a submitter
blocking on a context from pictures serialising on their reference
waits, which is the question a flat bench leaves open.
cc_resampler_neon.S compiled on __ARM_NEON__ alone, while the C file
selects the kernels on __ARM_NEON__ && HAVE_ARM_NEON_ASM_OPTIMIZATIONS
and every sibling .S - sinc_resampler_neon.S, float_to_s16_neon.S -
guards on both. Nothing in the tree defines that macro, so every ARM
build took the C reference and carried two exported functions nobody
called. It also left the reverse open: the macro defined, the file
assembled without the flags the kernels need.
Checked with clang -target armv7-linux-gnueabi -mfpu=neon
-mfloat-abi=softfp: no symbols without the macro, the two kernels with
it, and the file assembles cleanly either way.
Reviving the path needs more than this. The kernels take their ratio
with vldr s4, [sp], which is the soft-float convention and wrong under
-mfloat-abi=hard, and the upsampler has no output bound, so they should
not be switched on without a device to test them on.
resampler_CC_init() allocates with memalign_alloc(32, ...), but the
function pointer sat first, so buffer landed at offset 8 on a 64-bit
host and 4 on a 32-bit one - never 16-byte aligned, and the SSE arms
had to take it with _mm_loadu_ps/_mm_storeu_ps for a state the
allocation had gone out of its way to align.
It is also the layout the NEON kernels address: cc_resampler_neon.S
documents "[r2+0] --> buffer, [r2+32] --> distance" and acts on it at
:84, :103, :187 and :188. With the pointer leading, the 32-byte store
at :187 writes buffer[0].l over re->process and distance is read and
written at buffer[3].r. Nothing compiles that path today, so this has
been latent rather than broken, but the offsets are now the ones it
was written against: buffer at 0, distance at 32, on both widths.
The SSE accesses become _mm_load_ps/_mm_store_ps. init() is the only
place the struct is allocated, so the alignment holds. No measured
speed change on x86 - best-of-7 over 20000 calls sits in the same
0.174-0.177 s band either way, which is what movups on aligned data
costs on anything recent; the older in-order targets this driver
exists for are where it is not free.
cc_resampler_int16.c mirrors the float driver, including both of the
things just fixed in it.
The upsample loop had no output bound, and its phase step is quantized:
ratio = (1/data->ratio) * 2^24 is zero once data->ratio passes about
3.4e7, so re->distance never advances and the loop writes past data_out
forever. A negative ratio walks the other way, and a ratio that is not a
number reaches the int64 casts, where the conversion is undefined. Both
arms now refuse a ratio outside (0, 65536] before those casts, and the
upsampler stops at what the ratio asks for.
The direction is also latched at init from the nominal ratio while the
runtime one is multiplied by slow motion, and cc_i16_downsample() emits
at most one frame per input frame. Same move as the float driver: above
a ratio of one the stream goes to the upsampler and stays.
samples/audio/stretch gains both lanes for CC, float and int16 - eight
calls at 2.0 on an instance initialised at 0.5, and the five ratios no
pair of sample rates can name, into an exact heap buffer. Without this
commit the int16 half segfaults. sinc and nearest do run past data_out
on those ratios; the suite says so where it drives only CC.
resampler_CC_init() picks the direction once, from the nominal ratio
the frontend passes as bandwidth_mod. The ratio the backend is then
handed is a different number: audio_driver_effective_ratio() multiplies
it by the slowmotion setting, 3.0 by default.
resampler_CC_downsample() emits at most one frame per input frame - the
store sits inside the once-per-input body - so above a ratio of one it
cannot produce what it is asked for. Measured over 2,048,000 input
frames, an instance initialised at 0.5 and run at 2.0 returned 2,047,999
frames where 4,096,000 were due: half the audio, gone, for any core and
device whose nominal ratio is under 0.75 the moment slow motion starts.
So the stream moves to the upsampler, which serves either direction, and
stays there. The two carry distance differently, so the move restarts
the stream; it happens once. Measured at 48000 to 32000, the two arms
agree to two decimals on gain and THD+N at every frequency from 100 Hz
to 14 kHz, so nothing is given up by staying.
resampler_reset gains the lane: eight calls at 2.0 on an instance
initialised at 0.5. Without this commit it reports 7999 frames of 16000.
The function reports the length it wanted, so >= len is how a caller
detects a join that did not fit; callers sizing a path buffer from a
bound held elsewhere depend on that and on nothing outside the buffer
being written. Neither was covered.
Checked against an injected defect that clamps the return to len - 1:
the case fails there and passes once reverted. ASan clean.
The inline feature block predates the shared header by one commit and
does the same job, leaving two idioms in the tree for it. The object is
byte-identical under the default profile.
The PSP path uses SceCtrlData, the PSP_CTRL_* bits and three sceCtrl
calls and includes none of them. It compiles because griffin includes
psp_input.c, which does include <pspctrl.h>, ahead of it - the VITA
path in this same file includes <psp2/ctrl.h> for itself, so the
omission is one-sided.
A compile-only pspctrl.h joins the seven psp stubs already there, and
two lanes follow: psp_input and psp_joypad, neither compiled anywhere
else. SceCtrlData carries what pspsdk's does in the same order and no
more - the PSP has one analog stick, and a stub offering Rx/Ry would
let a lane pass what the real SDK rejects. Taking the include back out
fails the new lane.
The 3ds audio drivers stay out: they want fifteen more symbols whose
libctru signatures cannot be checked from here, including a Result
returning entry point, and a guessed one is a lane that passes code
the SDK refuses.
Verified: 110 lanes, 0 failures, from a clean tree; the tree builds;
the stub header compiles standalone under -std=c89 -pedantic.
griffin includes each of these behind its platform's #ifdef and the
desktop makefiles never build them, so a C89 slip or a missing
declaration in one waits for the console job. The stubs to reach them
are already in the tree; nothing joined the two.
Ten lanes, on the files the existing stubs reach today, with the same
warnings the driver lanes above them use - C89 declarations included,
because that is the rule these files are furthest from anyone
checking. GEKKO takes the vendored libogc headers ahead of its stub,
which carries only what libogc does not. dispserv_android already had a
lane and gains the declaration check rather than a second one;
psp1_gfx is covered by platform_video.
gx_joypad.c needed a fix to pass: WPAD_StickX declared its double
mid-function, where WPAD_StickY immediately below it declares the same
variable at the top. Reintroducing it fails the new lane, as do
injected implicit declarations in ctr_input.c and ps4_audio.c.
Files the stubs cannot reach are left out rather than papered over -
gx_audio, gx_gfx, ctr_gfx, wiiu_audio, gx2_gfx and the rest still want
headers no stub here supplies, and two of them collide with the host's
own nanosleep.
Verified: 108 lanes, 0 failures, from a clean tree; the tree builds;
hot-function-budget unchanged.
<stdio.h> carries ssize_t only for a profile asking for the BSD or
POSIX.1-2008 set, so a consumer building under a stricter one gets
"unknown type name 'ssize_t'" out of any public header here that uses
it, chd_stream.h among them. Include the header POSIX puts it in.
Scoped to __unix__ and Haiku alongside the existing Apple arm; console
toolchains define none of those and keep the path they have.
strdup(), strcasecmp(), fstatat(), O_CLOEXEC, M_PI, sigaltstack() and
syscall() all sit behind __USE_XOPEN2K8 or __USE_MISC, so a consumer
compiling the library under a stricter feature profile than the default
sees none of them. The Debian/Launchpad core packaging passes
-D_XOPEN_SOURCE=600, where ten of these translation units reach an
implicitly declared function; at -D_POSIX_C_SOURCE=200112L strdup joins
them, and an implicit strdup returns int, so the pointer is truncated on
LP64 rather than merely warned about.
Add retro_posix_source.h, which asks for that profile, and include it
first in the files that need it. The guards only raise the profile and
ask for what a build passing no feature macros already gets, so no path
the library selects changes: every touched object is byte-identical
under the default profile.
nbio_linux.c defined _GNU_SOURCE after its first include, where it had
no effect; retro_posix_source.h replaces it.
Verified: all 221 standalone-compilable libretro-common units clean
under no macros, _XOPEN_SOURCE 600 and 700, _POSIX_C_SOURCE 200112L and
200809L, and _GNU_SOURCE, with implicit declarations and int-conversion
as errors; the fifteen samples suites that link these units build and
pass; compile-matrix.sh gives results identical to before the change.
Building with -DRETRO_POSIX_SOURCE_CHECK_ORDER turns a late include into
an error, and was itself checked both ways.
Given a thread pool, the decoder keeps up to four pictures in flight.
A picture's slices are queued as they arrive, on its own context, and
go to the pool when the next picture opens; the sequence's side - an
IDR emptying the DPB, reference marking, the DPB and the output queue
- is settled then, on the submitting thread, with the picture's rows
still counting up. Every place a picture reads a reference - the
inter prediction, the B predictions, and temporal direct's co-located
cells - works out the row the read reaches and waits on the block's
counter for it, and nowhere else. A picture due for output leaves
only once complete; the reorder depth is the pool's head start. Field
pictures and intra pictures stay on the submitting thread: a pair
fills one block a field at a time, and an intra picture's grid is
marked at completion. The waits sleep on a condition; nothing spins.
Without a pool, or with one thread, this is the decoder as it was.
The lossless oracle decodes every case four ways and holds each
byte-exact against ffmpeg: on one thread, with four contexts rotating,
with four pictures decoding concurrently on a pool, and concurrently
with every row's publication held back by a random spell so that the
readers wait for their rows rather than nearly always finding them.
That last run, and the sanitizers, found what they were for: a hook
on the submitter's picture rather than its own, a completion asking
the sequence's flag, reference counts that were not atomic, a field
view reaching half the rows it read, and the B predictions reading
their references past the wait altogether.
The MP4 container hands its pool to the H.264 decoder as it does to
the HEVC and VP9 ones, so the thumbnail's thread setting reaches all
three; a bench on one core is what it was, and the number that matters
comes from a machine with cores to give.
The VFPU register file is per-thread context on PSP, and the two ends
of this backend are on different threads: audio_driver_init_internal()
creates the resampler on the main thread, and the threaded audio
pipeline runs process() on the audio worker. The constants pi and 1/pi
that init() issued, and the accumulator and position in c720/c730,
never reach the thread that resamples - it starts on whatever its own
context holds. One register file also cannot hold several instances,
so the frontend's resampler, the extra-channel ones and every mixer
voice were the same stream.
So the handle carries them: init() allocates it, process() issues the
constants per call and loads c720/c730 from it and stores them back
around the loop, and reset() and free() become the plain C the other
arms already have. Two vcst.s and a pair of quad accesses per call,
against a loop that runs per frame.
This is the crash reported on PSP: a stale position kept the loop
emitting after the input was spent, and the scalar float stores walked
out of the output buffer and off the end of user RAM. The bound in the
previous commit stops the walk; this stops the stale position.
The asm is compiled here only through gcc -fsyntax-only against the
PSP stub headers, which does not assemble the VFPU mnemonics. It wants
a devkitPSP build before it goes near master.
struct resampler_data hands a backend an output pointer and no
capacity, so the frontend's own clamp is all that keeps a ratio inside
data_out - and audio_driver_bound_ratio() returns the ratio untouched
wherever the caller reports no capacity. The arms that can emit more
than one frame per input frame now stop at what the ratio asks for
plus the frame a carried phase adds, and a ratio no pair of sample
rates can name resamples nothing rather than guessing at one.
This is what keeps a bad phase inside the buffer instead of walking
memory: the upsample loops exit only on their own accumulator, so a
ratio or a position that never brings it back under one emits forever.
resampler_reset gains the contract: every backend's reported frames
against what the ratio asks for, across several calls so a carried
phase is in play, and CC against the ratios a clamp would have caught,
into an exact heap buffer so a sanitizer build sees a write past it.
Without this commit that lane is an ASan SEGV in the SSE upsample arm.
sinc and nearest run past data_out on the same ratios and are left for
their own change; the suite says so where it skips them.
The bound is free: best-of-7 over 20000 calls at 48/32 on the SSE arm
lands in the same 0.174-0.177 s band with and without it.
Three things in one function. memalign is called with no declaration -
<stdlib.h> does not carry it - and this driver is its own translation
unit in the emscripten build, not part of griffin's, so nothing hides
it: emscripten's clang makes an implicit declaration an error by
default. audioworklet.c, the sibling driver in the same directory,
already includes <malloc.h> for the same call.
The two mixing buffers were used unchecked. rwebaudio_write()
interleaves into them on every call with no test of its own, so a
refused allocation handed back a context that faults on first use;
init now fails and tears down instead.
And the driver context leaked when RWebAudioInit() failed, between the
calloc and the error return.
The allocator itself stays memalign: the buffers cross into the JS
side, and libretro-common's memalign_alloc keeps a header word below
the pointer it returns, which is not a swap to make on a platform this
cannot build.
Verified: the file compiles standalone under gnu99 and c89 with
-Werror=implicit-function-declaration, where it failed before, and the
tree builds. No samples/audio suite references it, so there is nothing
to run.
librsound.c calls both and includes no stdio.h. It reaches a compiler
only through griffin's unity build, where stdio arrives 900 lines
earlier from compat_snprintf.c, so the two implicit declarations have
never fired - they wait for the include order to change or for the file
to be built on its own. Compiled standalone it is two
-Werror=implicit-function-declaration errors, which is a hard error on
devkitPPC rather than a warning.
The rsound driver's own comment also still pointed at librsound's
callback lock as the thing its ring avoids touching. That lock is gone;
the ring is what the handoff runs on.
Verified: librsound.c standalone clean under gnu99 and c89 with
-Werror=implicit-function-declaration, where it failed before; the tree
builds; and the new include is provably a no-op in the unity build,
which already has stdio.h by then.
O_CLOEXEC, fstatat() and dirfd() are POSIX.1-2008 and syscall() is a
glibc/musl extension, so a consumer compiling this file under a
stricter feature profile than the default sees none of them; the
Debian/Launchpad core packaging passes -D_XOPEN_SOURCE=600 and stops
on undeclared O_CLOEXEC. Request the profile the file needs before its
first system header instead, raising it only when what the consumer
asked for is lower.
What is requested matches a build passing no feature macros at all, so
every path this file selects stays as it is: the object code is
byte-identical under the default profile, and a -D_XOPEN_SOURCE=600
build now produces that same object.
The snapshot is published once per frame as the catch-all for settings
toggles, and the video thread reads it every frame. What it carries
changes on a settings toggle, a rotation or an aspect change - never
during play - so the unconditional store spent a cache line transfer a
frame to say nothing, and took the line away from the reader to do it.
Compare first: fifteen relaxed loads, which leave the line where it is,
and the sequence bump and the stores only on a difference.
The slot count stops being a literal in two places, with headroom left
in the array so a parameter can be added without moving anything after
it.
Nothing notices if the compare goes - the values are still right and
every lane passes - so the harness watches the sequence counter, which
is the one thing that stands still exactly when the publish did
nothing: ten quiet frames must not move it, a changed aspect index
must, and the new value must reach the slot. Publishing
unconditionally fails the first; a slot that stops following its
setting fails the others.
Verified: compile-matrix 98/98, C89_BUILD from a clean tree, the
RARCH_MOBILE branch of the publisher, hot-function-budget,
stack-frame-sweep, lock_acquire_budget, thread_read_audit,
video_thread_state_check, and make check on plain, address+undefined
and thread builds plus 15 cycles.
Padding epoch and waiters apart is the obvious-looking fix and it is
the wrong one, so record the measurement rather than leave the next
reader to rediscover it. Both sides read both cursors on every
operation, so separated, each pays two lines where one served.
Measured on 32 cores, shared against padded, medians over nine rounds:
where the operations punctuate real work, which is every caller in the
tree, no difference at all - 1.00 with the bracket inside a percent,
with one waiter or four. In a tight loop the direction is not even
consistent: one notifier and one waiter lose a third of the notify rate
separated, four waiters gain a seventh. The video ring is the former
shape, so the padding would cost it something for nothing.
retro_spsc pads its cursors because each of its sides reads only its
own; the note points at that difference.
SYSCLOCK_ARM11 falls back to the SoC clock's multiple where libctru does
not define it, and the frame wait polls LightEvent_TryWait in short
slices where LightEvent_WaitTimeout is absent; the ctr-legacy image has
neither.
Three places in thread_video_t had fields written by different threads
on one 64-byte line, so every write to either pulled the line across:
the snapshots the video thread publishes every frame sat with the head
of cmd_data, which a sender rewrites for every synchronous command; the
per-frame published flags sat with nonblock and the deferred ring's
head; and the ring's producer and consumer indices were four bytes
apart. A line of padding each way, so the separation holds wherever the
fields land rather than depending on an offset, at 192 bytes on a 22 KiB
struct.
Measured on a 6-core Windows workstation with a bench that models these
exact offsets, so its "shared" variant is the layout as it shipped.
Publishing during a burst of synchronous commands goes 2.13x
[1.93 .. 2.19 over 9 rounds] with no cost to the sender, 0.99
[0.96 .. 1.00]; the ring's throughput goes 1.20x [1.10 .. 1.33] on both
ends. A quiet frame - one command between two publishes - came out as
noise on both machines, so that part is not claimed.
The padding is the whole of the mechanism and nothing else notices when
a field added above one of the pads moves the field below it back onto
the shared line, so the harness asserts each separation by offset
against the struct the build actually has. Removing pad_published,
pad_flags or pad_ring each fails the lane by name.
Verified: compile-matrix 98/98, C89_BUILD from a clean tree, the
forced-atomic-backend loop over C11/GCC_NEW/SYNC/PS2/VOLATILE,
hot-function-budget, stack-frame-sweep, lock_acquire_budget,
thread_read_audit, video_thread_state_check, and make check on plain,
address+undefined and thread builds plus 15 cycles.
The finish comes apart into the picture's own completion - the slice
map for the filter, the rows the row hook did not filter, the intra
marks in the grid, the block's row count set to the whole picture,
the references let go - and the sequence's side: an IDR emptying the
DPB, reference marking, the picture entering the DPB and the output
queue. The first takes only the context and is what a pool thread will
run; the second runs in decode order on the thread that submits.
Allocation comes apart the same way: the DPB and the output queue are
remade only when the geometry changes, and a context is readied for
that geometry when its picture opens, not before its header is read.
With that a picture can take any context: the decoder holds four, a
new picture takes the next one round, and a field pair stays on the
context its first field took. rh264_video_set_contexts() sets how many
are in rotation; one is the decoder as it was. The pictures still
decode one after the other, so rotating changes which memory a picture
uses and nothing else - and that is the proof this sequence of
changes was for. The lossless oracle decodes every case at one context
and at four and holds both byte-exact against ffmpeg: a difference
would be a piece of picture state still in the decoder rather than the
context, shared between pictures the moment they decode concurrently.
There is none, and the check stays.
A CI run failed with threaded video off: rows 377-418 of the bottom
right square were not magenta and its centre read (22, 22, 22). That
is 0x161616, BG_COLOR_DEFAULT, the background of a widget notification
box, which at 640x480 lands over that square. The runner reinitialised
a driver during startup, and the notification it raised was still up
when the screen settled and was grabbed.
The test only looks at the overlay, so set menu_enable_widgets to
false in its config.
The statistics the overlay reads, the viewport an input driver asks for
on every poll, the refresh rate the runloop asks for every iteration
and the audio thread's two loop conditions all carry their values in
atomics so the read costs no acquisition. Nothing stops the next change
from putting an slock_lock back, and a reinstated one breaks no test:
the suite passes and it shows up as a slower frame on someone's
machine.
So count them. The count is read off the relocations inside each
function's own block in the release objects, which catches a lock taken
through a helper the compiler inlined and misses one taken through a
call it kept - a floor, like tools/video_thread_state_check.py, and
worth having because every acquisition it reports is a real one. Three
of the counts are deliberately not zero: viewport_info writes read_vp
when the viewport has changed, the audio loop takes the lock on the
pass that parks or leaves, and the gather takes it once it has
something to retire.
Run by hand after a plain make. It reports which architecture it can
count on rather than passing silently where it cannot.
Negative-tested: an acquisition put back into presenter_stats, into
viewport_info's poll, into the audio loop's every pass, and into
video_driver_frame each fails its own line.
A slice decoder read its reference lists through pointers into the
DPB slots. On one thread that is sound; with pictures decoding
concurrently a slot can be taken by a later picture while an earlier
one still decodes from it, and the reference would be pulled out from
under the read. Each picture's context now snapshots the frames its
lists name - a copy of the frame holding its own reference on the
sample and motion blocks - and the lists point at the snapshots.
Entries naming the same picture share one, and the finish releases
them. The counted blocks are what make that a pointer swap and a
count, not a copy.
With this a picture's decode depends on nothing in the decoder a
later picture can change: the slice decoders take their frame, their
parameter sets and their lists as arguments, the lists are owned, and
the row counter is on the block. What remains for pictures to decode
concurrently is the sequencing. Byte-exact on every case of the
lossless oracle, with no reference read short of its rows.
The loop took thr->lock every pass to read two booleans that are false
while audio is playing. They become atomics: the loop tests them
without the lock and takes it only on the pass that has something to
coordinate - leaving, or parking on a stop. A writer still holds the
lock while it changes one and signals, which is what keeps the change
and the wakeup atomic against a waiter, and the stop path re-tests
under it so a start that lands in between is not stopped and started
again for nothing. alive() and apply_control() read them as atomics
too, where they read a plain bool before.
That plain read was a race, and samples/audio/thread_handshake now
reaches it: the scripted device refuses a write, which is the one
place the audio thread marks the device gone, and the frontend then
asks whether it is alive. The lane also feeds the callback through the
wrapper's own write() rather than the inner driver's, which is the
path the frontend uses. On the old code TSan reports the race between
audio_thread_alive() and audio_thread_write(); with the atomics it is
clean, and the lane holds the device-gone contract - reported, stop
returns without waiting for an acknowledgement, free() joins.
Verified: thread_handshake and pipeline_wakeups, the two suites that
compile the wrapper, on plain, address+undefined+leak and thread
builds; compile-matrix 98/98; C89_BUILD from a clean tree; the
forced-atomic-backend loop over C11/GCC_NEW/SYNC/PS2/VOLATILE;
hot-function-budget; stack-frame-sweep; thread_read_audit; and
samples/gfx/threaded_video make check.
The Linux samples job runs retro_procbarrier_test once, so it only
ever exercises membarrier. The page-flip and signal fallbacks, the
Windows FlushProcessWriteBuffers path and the Darwin paths never ran
in CI at all; the Darwin code had never been compiled.
Add procbarrier-tiers.yml, which builds retro_procbarrier_test and
retro_asym_eventcount_test on Linux x86_64/arm64, Windows x64/arm64
and macOS arm64/x86_64, and runs both once per tier the platform has.
run_tiers.sh checks the "tier:" line of each run, not just the exit
code: a forced tier the platform lacks falls back to the natural one
without an error, which would otherwise pass unnoticed. With
REQUIRE_SMP=1 it also fails on a one-CPU runner, where the fence
checks cannot catch a missing barrier.
Everything that belongs to the picture being decoded - the working
frame, its motion grids, the slice bookkeeping, the slice scratch, the
NAL unescape buffer - moves into rh264_pic_ctx, and the decoder holds
one of them for now. What stays in rh264_video is the sequence:
parameter sets, the DPB, picture order, reference marking, the output
queue. The slice decoders already took their frame, parameter sets and
reference lists as arguments and touched the decoder only for the
sequence, so one of these per picture in flight is what lets pictures
decode on different threads, and a scheduler is all that is missing.
The block behind a picture's planes counts the macroblock rows that
are final. The row hook counts them up as each row's filtering
completes - after row r is filtered everything above it is final,
while r's own last lines are still touched through the edge it shares
with r+1 - and the finish counts the whole picture. The one place a
reference is read works out which row the read reaches, the block's
last line displaced by its vector and the six-tap filter's three past
it, and asks the block for that row. On one thread a reference is
complete before it is read and the ask is answered at once; a picture
decoding on another thread will wait there, and nowhere else. The
lossless oracle asserts that on one thread no read ever finds its
rows short, across every case: a short read would have a threaded
decoder waiting on a row it had already been handed.
Both come from the wrapped driver on the video thread, which is their
only writer, and both are read from elsewhere every frame - an input
driver asks for the viewport on each poll, the runloop asks for the
rate on each iteration. Publish them instead of locking for them: the
viewport as a seqlock over six slots, the rate as one word of float
bits. read_vp still has to follow what was reported, because
CMD_READ_VIEWPORT compares the driver's own viewport against it, so
viewport_info writes it when it changes - a resize - and the steady
state takes no lock at all. Every reporter writes the same published
value, so a compare that races another's write converges on it.
thr->vp goes with them; the snapshot is what callers read now.
The harness lane asserts the reported values arrive, that read_vp
follows a resize, and that the read touches no lock: a helper thread
holds thr->lock for 200 ms and the viewport, the rate and the four
statistics readers are timed inside that window, so a reader that
still takes the lock is delayed rather than deadlocking. Putting
either read back under the lock fails it at 199 ms; swapping the
width and height slots, dropping the per-frame publish, never
publishing the rate, and stopping read_vp from following each fail
their own check.
Verified: compile-matrix 98/98, C89_BUILD from a clean tree, the
forced-atomic-backend loop over C11/GCC_NEW/SYNC/PS2/VOLATILE,
hot-function-budget, stack-frame-sweep, thread_read_audit,
video_thread_state_check, and make check on plain, address+undefined
and thread builds, plus 15 cycles with no lane skipped.
winsock2.h declares struct pollfd when _WIN32_WINNT >= 0x0600. Gate
the libsmb2 compat.h declaration on that, so Vista+ targets take the
SDK's and the Xbox XTL, legacy winsock.h and pre-Vista SDKs take the
compat one. Fixes C2011 on the UWP build.
net_compat.h supplies the same declaration for pre-Vista Win32
targets, where socket_poll() needs it and no builtin SMB client is
compiled: msvc2003 builds at 0x0400, msvc2005 at 0x0410.
tools/win32-pollfd-check.sh cross-compiles both headers in both
include orders from 0x0400 to 0x0A00, plus net_socket.c on its own.
Run it after touching either header.
Every DPB slot carved two full motion-vector grids into its meta block
when the decoder was set up - thirty-two megabytes a slot at 4K - and
since a frame picture shares the working grid into its slot, nothing
read them: they were allocated and zeroed for the field pairs alone,
which copy into grids of their own. The slot makes its two grids the
first time a field is copied into it and keeps them with the frame;
a stream of frame pictures never allocates them.
The preview bench takes BENCH_MAX, to profile a stream's start on its
own: what it showed on a 4K recording was that a second picture taking
most of a second was a 1.2 MB P-frame's entropy decode, not the
decoder's setting up.
The statistics overlay reads four groups of wrapper numbers once a
frame, and each read took thr->lock - a write-share on the busiest
lock in the wrapper, from the runloop thread, for data it only reads.
Publish them instead: the video thread stamps the snapshot inside the
region it already holds that lock for, so publishers stay serialised
by it, and a reader only loads. 64-bit counters travel as two
int-wide halves, low first. core_time and display_pacing come from
the main thread, so they ride a frame behind; everything else is this
frame's.
samples/gfx/threaded_video gets a lane that brackets each published
field against the field it came from, and asserts the swap count
across a carry out of the low half. The flags word is only meaningful
where a bit is set, so the repeat, display-phase and display-pacing
lanes call the same check with their own bools true. Negative-tested
per bit and per slot: each dropped half, swapped slot pair and
dropped flag bit fails the lane.
Verified: compile-matrix 98/98, C89_BUILD from a clean tree, the
forced-atomic-backend loop over C11/GCC_NEW/SYNC/PS2/VOLATILE,
hot-function-budget, stack-frame-sweep, thread_read_audit,
video_thread_state_check, and make check on plain, address+undefined
and thread builds.
The windowed feed reads half a megabyte a tick, sized for a decoder
that takes one sample a tick. A decoder catching up takes three or
four - it passes over the pictures nothing references - and a fast
one takes them at whatever rate it decodes, so either eats through the
two seconds of lookahead faster than the feed refills it and then
stands at the wall for the forty ticks a refill takes: on a 4K
recording, a stall near a second at every group of pictures. The
pacing holds only while the feed is comfortably ahead now; once the
resident lookahead is below half its target the tick extends unpaced,
the burst a lap or an open already pays, and the decoder never sees
the wall. On the recording that showed it, the stalls at each group
went from eight hundred milliseconds to two hundred, which is one
picture's decode.
The preview sample's paced check bounds the stall now, on the
B-frame fixtures, and its audio checks apply only when audio was asked
for. The bench names any frame that took over a third of a second.
WSAPoll is Vista+ only. 9x, XP, and Xbox XTL do not export it.
Keep socket_poll() on every platform. On Win32 and Xbox implement it
with select() and Microsoft POLL* bits so socket_wait() and SMB keep
working. POSIX/Vita/Gekko/PSL1GHT backends are unchanged.
libsmb2: stop mapping poll to WSAPoll. Use the existing select()-based
poll() in compat.c on all Windows targets, including desktop Winsock2.
The CloudKit driver:
- reported success for a read whose local file could not be written
(with no file, which cloud sync takes as "not on the server" and
drops from the server manifest) and for a record with no readable
data (with an empty file, which cloud sync takes as the save);
- saved a new record when the lookup before an upload failed, leaving
a duplicate of a record that may well exist;
- reported a delete as done before the deletes were sent, and as a
success whatever became of them;
- ignored non-destructive sync: deletes removed records for good and
uploads replaced their data.
Reads now fail unless the data was loaded and written in full. Uploads
fail on a failed lookup. Deletes report once every record has been
dealt with, with the result. With destructive sync off, a delete
renames the record's path to <path>-<yymmdd-hhmmss> and an upload that
replaces a record first saves one under that name holding the old
data, not replacing anything if that fails, as the other drivers keep
their copies. The server manifest is not kept. A duplicate-record log
line also passed an NSString to %s.
samples/network/icloud_cloudkit includes icloud.m and replaces
CloudKit with an in-memory database by overriding +[CKContainer
defaultContainer] in a category, using real CKRecord, CKAsset and
CKQuery objects. It checks each case above and that every request
reports once. Against the old driver 8 checks fail. Runs in CI macOS
under ASan+UBSan.
A presentation_queue that is another queue of queue_family_index is only
valid for a frontend that takes one, so it gets a version of its own.
GET_HW_RENDER_CONTEXT_NEGOTIATION_INTERFACE_SUPPORT reports 3, and a core
may hand back that queue only when it sees 3 or higher. Frontends below
that fail context creation on any presentation_queue that is not the
graphics queue.
Acceptance stays version-agnostic: a core that provides the queue without
asking is still honoured when the family matches.
With destructive sync off, a Google Drive delete keeps the server's
copy, renamed in place to <name>-<yymmdd-hhmmss>, but an upload
replaced the file's content outright, so the setting protected against
deletes and not against overwrites.
An upload to an existing file now asks Drive to copy it (files.copy,
server-side) to <name>-<yymmdd-hhmmss> in the same folder, the name a
delete uses, before the PATCH. A failed copy stops the upload rather
than overwrite without a backup. A file that does not exist yet has
nothing to keep, and the server manifest, rewritten every sync, is not
backed up.
A 401 on the PATCH now retries only the PATCH. It used to go back
through the whole update, search included, which with the backup would
take it a second time; the file ID is known by then.
samples/network/gdrive_backup includes google_drive.c and plays the
Drive API request by request: copy before PATCH, from the file being
replaced, named and placed as a delete names it; a failed copy stopping
the upload with one failure report; a 401 on the copy and on the PATCH
each refreshing the token once; destructive sync, a new file and the
manifest not copied. Against the old driver 5 checks fail. Runs in CI
Linux samples/network under ASan+UBSan and TSan, as does the SMB
harness.
smb_update() opened the server's file, truncated it and wrote. A write
that failed part way - a dropped connection, a full share - left the
server holding a truncated save, in either sync mode. And with
destructive sync off, a delete kept the old file as
<path>-<yymmdd-hhmmss> while an upload overwrote it in place, so the
setting protected against deletes and not against overwrites.
Uploads now write <path>.rauploading and rename it into place once it
is complete. Before that rename, the file being replaced is renamed to
the backup name a delete uses (non-destructive) or removed
(destructive); if the final rename fails, it is put back. A failed
write, a backup that cannot be made or a failed final rename all leave
the old file exactly as it was and no temporary file behind, reported
as one failure. The server manifest is never backed up.
Also: the local file is read from its start rather than wherever the
stream was left, a short read fails the upload, and a zero-byte write
from the server no longer loops forever.
samples/network/smb_backup runs smb.c against an in-memory libsmb2
share and checks what the share holds after each case: new content in
place and the old kept beside it; no backup for destructive sync, a new
file or the manifest; the old file intact after a failed write, backup
or final rename; an empty upload; the configured subdirectory. Against
the old driver 13 checks fail.
The S3 driver never read cloud_sync_destructive: a delete removed the
object for good and an upload overwrote it, so turning destructive
sync off did nothing for S3 users. WebDAV, Google Drive and SMB keep
the server's copy in deleted/ on delete, and WebDAV now on overwrite.
With the setting off, a delete or upload first copies the object to
deleted/<path>-<yymmdd-hhmmss>, the name WebDAV uses, with CopyObject
(a PUT on the new key naming the source in x-amz-copy-source), which
is server-side. The source is /<bucket>/<key> for both addressing
styles: a path-style endpoint's canonical URI already is that, a
virtual-hosted one gets the bucket in front. A 404 means there is
nothing to keep; any other failure, including a 200 carrying an
<Error> body, which CopyObject can answer, stops the delete or upload
rather than lose the copy. The server manifest is not backed up.
S3 requires every x-amz-* header to be signed, so the SigV4 code takes
an optional copy source and signs it in its sorted place. The
signature for a CopyObject request and for a plain GET was checked
against botocore's SigV4 for the same inputs and matches.
samples/network/s3_backup includes s3.c and plays the server request
by request: copy first, into deleted/, with the right source for
virtual-hosted and path-style endpoints and x-amz-copy-source signed;
404 not an error; a failed or <Error> copy stopping the operation with
one failure report; destructive sync and the manifest not copied.
Against the old driver 6 checks fail. Runs in CI Linux samples/network
under ASan+UBSan and TSan.
Three things the 4K preview was doing wrong on Windows, none of them
visible to a decode oracle.
The shared plane and motion blocks are recycled rather than freed.
The working frame takes a fresh block for nearly every picture - the
one it holds is a reference's or a queued picture's by then - and a
4K block is twelve megabytes: on the C runtime MinGW builds against
that is a page-zeroed mapping from the kernel and back, twice per
picture, where the swap it replaced allocated nothing. A released
block goes on a short free list by length and the next of that length
comes from there, so the steady state allocates nothing again.
A picture passed over to catch up left the presentation index where
it was: the decoder returns 0 for it as it does for a picture held
back for reordering, and the container took it for one. From then on
every shown frame read its duration from the slot before its own. The
decoders say when a call passed a picture over and the container lets
its slot go by, as it does for a refused one.
The preview sample's oracle animated flat out, which decodes six
hundred 4K pictures twice and shows nothing about playback. PACE_HZ
polls it at a display's cadence, which is how the thumbnail is driven,
and reports the picture changes and the longest run of polls in which
the picture did not change - the stall a viewer sees.
With destructive sync off, a WebDAV delete moves the server's file
into deleted/, but an upload overwrote it in place: the setting
protected against the server's copy being removed and not against it
being replaced, which is the more likely way to lose a save (an older
or broken save from another device overwriting a good one). The code
carried a TODO/FIXME for exactly this. The local side already keeps
cloud_backups/ before a fetch replaces a local file.
An upload now copies the server's file to deleted/<path>-<yymmdd-
hhmmss>, the name a delete uses, before the PUT. COPY rather than
MOVE, so the server keeps its file until the PUT replaces it, and an
upload that fails after the backup leaves the server as it was. A 404
means there is nothing to back up yet; any other failure stops the
upload rather than overwrite without a backup. The COPY and the PUT
each get their own Digest retry. The server manifest is rewritten
every sync and is not backed up; its name moves to
cloud_sync_driver.h as CLOUD_SYNC_SERVER_MANIFEST so the driver can
recognise it.
Adds task_push_webdav_copy() beside task_push_webdav_move() in both
HTTP backends, sharing the Destination-header code.
samples/network/webdav_backup includes webdav.c and plays the server
request by request: backup before PUT, into deleted/ named as a delete
names it, 404 not an error, a failed backup stopping the upload with
one failure report, destructive sync and the manifest not backed up,
and a Digest retry each for COPY and PUT. Against the old driver 5
checks fail. Runs in CI Linux samples/network under ASan+UBSan and
TSan; the other WebDAV harnesses gain the new stub.
Both overlay passes skipped before they reached a single assertion:
the null driver has no overlay_interface, so the lane got its
textures and then stopped at "no load_textures". Everything the lane
describes - the page of borrowed textures, the switch that uploads
nothing, the disable-before-unload order - was never run in CI, and
neither was any of the wrapper code behind it.
So give the driver an overlay back end. It goes on video_null itself
and BEFORE the mode switch, because the wrapper decides at init
whether it has an overlay to offer from the driver it is about to
wrap: with one there, the threaded pass runs the real path, main
thread to command ring to the video thread's captured table, and
what the driver is told can be compared with what the frontend
asked for.
That comparison is what the pass now makes. Two of the checks are
against wrapper code nothing else in the tree touches, and both were
negative-tested:
- an alpha set cannot be lost. The set is fire-and-forget: the
value goes into an atomic and raises a flag the video thread
clears with an acquire exchange BEFORE reading the values, so a
set that lands mid-apply is applied whole next pass. The lane
fires one from inside the apply - the driver's set_alpha runs on
the video thread, after the clear and after image 0's value was
read - and asserts it still arrives. Clearing the flag after the
read instead loses it: "an alpha set from inside the apply was
lost: the driver last saw 0.500, not 0.375".
- the pack's textures are retired, not released. Unloading under
the wrapper must NOT hand the driver a release, because the
in-flight frame may still name the texture; it goes on the retire
list and the video thread frees it once it has drawn the frame
that carries it. Releasing inline instead fires "3 of the pack's
textures were released on the main thread instead of being
retired", and the named handles are then each asserted released
once the frames have run.
The third check covers the page switch itself: the handles that
arrive are the frontend's own, in order, and the uploading load()
never runs. Forcing the fallback fires six lines, from "load_textures
refused page one" through "page two arrived as (0, 0), not (5002,
5003)".
With GFX_INSTRUMENT=1 the overlay baseline reports real numbers for
the first time - 6 loads, 6 unloads for 4 pages over 2 passes - where
it used to print zeroes, which also means the pre-existing bound on
uploads per page switch is finally being exercised.
Verified: make check clean on plain, address+undefined, thread and
GFX_INSTRUMENT=1 builds, and over 25 cycles; the harness TU clean
under -Wall -Wextra -Werror on the tree's own compile line; all three
negative tests fail as quoted.
The harness now has no skipped lane left on the null driver.
Cloud sync counts its outstanding requests and waits for each one to
report, so a driver must call the completion handler exactly once per
request. The iCloud Drive driver broke that both ways:
- a read whose local destination could not be created never called
it, so the sync waited forever;
- a read whose cloud file existed but could not be loaded, and an
upload whose local file could not be read, reported failure and then
carried on and reported again, decrementing the outstanding count a
second time.
It also treated an unavailable iCloud container as "file not
present": a read reported success with no file, which cloud sync now
takes as the server not having the file and drops from the server
manifest, and a delete reported success. Nothing was checked, so all
three now report failure. A short write of the downloaded data is a
failure too.
samples/network/icloud_drive includes icloud_drive.m and redirects
the container to a temporary directory by overriding
URLForUbiquityContainerIdentifier: in a category, so the driver runs
unchanged; a nil answer stands for iCloud being unavailable. It checks
that every request reports once with the right result, for the paths
above and the ordinary read, missing file, upload and delete. Against
the old driver 8 checks fail. Runs in CI macOS under ASan+UBSan.
Every WebDAV request callback retried when the server answered 401
with a Digest challenge, and nothing limited how often. That is right
for a stale nonce, but with a wrong password the server answers every
retry with a new challenge, so the first request of a sync - the
OPTIONS probe - looped forever and the sync never reported anything.
Allow one retry per request (webdav_retry_auth()); a second challenge
now fails the request with an error naming the credentials. Each
MKCOL in a directory walk counts as its own request.
A retried PUT also sent garbage. webdav_do_update() reads the upload
into memory, leaving the file at its end, and the retry read it again
from there: it got nothing and sent a buffer of the right length full
of uninitialised memory, replacing the save on the server whenever a
nonce expired mid-upload. Rewind before reading, and fail the upload
on a short read.
samples/network/webdav_reauth includes webdav.c and plays the server
for OPTIONS, GET, PUT, DELETE, MOVE and MKCOL: a server that keeps
rejecting the credentials must get exactly one retry and one reported
failure, a single stale-nonce challenge must still be retried and
succeed, and every PUT must carry the file's bytes. Against the old
code every request type loops and every retried PUT sends the wrong
bytes. Runs in CI Linux samples/network under ASan+UBSan and TSan.
A finished frame picture was copied plane by plane into the DPB when
it was a reference and swapped into an output slot when it was to be
shown, and its motion-vector grid was copied into the slot twice, once
per parity. At 4K that is twelve megabytes of samples and two grids
per reference picture, and five per cent of the decode was memcpy.
The plane block and the working grid now carry a reference count in a
header ahead of the data. A frame picture's block is shared into the
DPB slot and the output slot rather than copied, its grid is shared
into the slot for both parities, and the working frame takes fresh
blocks on its way into the next picture only while the ones it holds
are still someone else's. Field pairs, which fill a block a field at
a time, keep the copy, into grids and samples the frame owns - a
field can never land in a block a queued picture still reads. The
lossless oracle holds every case byte-exact against ffmpeg, plain and
under the sanitizers, with every block released by its last holder.
Beyond the memcpy this is the shape frame threading needs: a picture
that lives in one place for its whole life is one a row counter can
sit on, and one a picture decoding on another thread can wait for a
row at a time. Under the copy model a reference did not exist until
the copy at the finish, and there was nothing to wait on.
The surface lane stopped at "driver made no texture" on the null
driver, which has no load_texture at all, and returned before the
three things it exists to check: that a streaming surface loads ONE
texture and keeps it across every frame, that each queued submit is
released exactly once, and that freeing a surface with a frame still
in flight releases its texture rather than leaking it. Everything
after the early return - the 30-submit loop, the in-place handle
check, the whole direct half, both instrument baselines - was
unreachable in CI.
So lend the driver a back end. Handles carry a tag and are tracked
one by one, so the checks name the surface's own texture and are not
disturbed by what else the frontend uploads through the same poke
(the menu's framebuffer comes through it). It goes on per video mode,
because each switch reinits the driver and rebuilds the poke from the
real one: threaded it sits on the wrapped driver, so the worker's
uploads run through it; direct, on the driver's own poke.
Two checks the back end makes possible are new, and are the ones that
have teeth:
- the surface's texture is named and asserted released after a free
with a frame in flight. ASan cannot see this leak: a texture
handle is a driver-side resource, not an allocation.
- the load count is bounded at one for the whole run, in both
halves.
Both were negative-tested. Dropping the unload from the dying branch
of gfx_surface_done() fires the first ("1 textures still loaded after
the threaded half (1 loads, 0 unloads)"); clearing can_update on
every completion fires the second ("31 texture loads for one
streaming surface") along with the pre-existing in-place check, which
had never once run.
With GFX_INSTRUMENT=1 the lane now reports a real budget for this
driver instead of nothing:
surface, threaded: 1 loads, 31 updates, 32 posts (0 allocated), 0 copies
surface, direct: 1 loads, 10 updates, 1 unloads, 0 posts, 0 copies
Verified: make check clean on plain, address+undefined, thread and
GFX_INSTRUMENT=1 builds; the harness TU clean under -Wall -Wextra
-Werror on the tree's own compile line; both negative tests fail as
quoted.
The two overlay lanes still skip: they need an overlay interface with
load_textures, which the null driver also lacks. That is a separate
piece.
webdav_read_cb(), webdav_update_cb(), webdav_delete_cb() and
webdav_backup_cb() dereferenced webdav_cb_st->path to log a failure,
and read and delete again on the reauth path, before the NULL check
that came further down. Check once at the top and return, as
webdav_stat_cb() already does, and drop the checks that can no longer
fail.
A file listed in the server manifest but missing on the server (such
as one uploaded under a name cut short at '#' before the WebDAV path
encoding fix) came back from the driver as success with no file.
task_cloud_sync_fetch_cb() logged that as "Failed to write file from
server", kept the entry in the server manifest and failed the sync,
so the same fetch failed on every sync after.
That result was ambiguous in the drivers too: WebDAV and Google Drive
also returned success with no file when the download could not be
written locally, and when the file on the server was empty. Both now
hand back an open RFILE for any found file, empty included, and
report a local open or short write as a failure. Success with no file
now means only "not on the server", which is what SMB, S3 and
iCloud Drive already meant by it.
task_cloud_sync_fetch_cb() then drops such an entry from the server
manifest and marks it for upload, leaving the local file and local
manifest alone: if this device has the file, the next sync sees it
as new and uploads it, which repairs the server.
Since an empty server manifest now arrives as an empty RFILE rather
than none, treat an empty one as no manifest, as an empty body was
before; rjson rejects empty input.
The local manifest was rewritten in place, so a crash, a full disk or
a power cut part way through left it empty or truncated. The last fix
makes the next sync discard such a file, but that costs the sync its
record of the previous state, and every differing file then goes
through conflict resolution.
Write manifest.local.tmp and rename it over manifest.local only once
it is complete. filestream_rename() replaces an existing destination
on every platform, including Windows. If the write or the rename
fails, the temporary file is removed and the old manifest is left in
place, which is the safe outcome task_cloud_sync_commit_local_manifest()
already describes: the next sync re-examines the same files.
libretro VFS has no fsync, so on filesystems that do not order a
rename after the data it replaces, a power cut can still lose the
new manifest; the old one, or the discard path, covers that.
task_http passes the failure reason (stage plus code, such as
"ssl_connect_failed (-0x7780)") to every transfer callback as err,
but every S3 callback discarded it with (void)err, so a transport
failure was logged only as "HTTP -1". Pass err through to
s3_log_http_failure() and s3_multipart_fail() and print it when there
is no status, the way webdav_log_http_failure() already does; the
"Did not get HTTP data" warnings include it too.
net_http_failure() gave a code only for the TLS stages, so a plain
connection refused, timed-out connect or failed send reached the log
as "socket_connect_failed" with nothing to say why.
Record errno (WSAGetLastError on Windows) for socket_create_failed,
socket_connect_failed and socket_send_failed, right after the failing
call and only for the first failing stage, as the TLS code already is.
task_http prints positive codes as "(111)", so the log line becomes
"Download failed: socket_connect_failed (111)."
socket_connect_with_timeout() now leaves the reason in errno too: a
non-blocking connect reports a refused or unreachable peer through
SO_ERROR, and a timeout through nothing at all, so both are copied
over (SO_ERROR value, or ETIMEDOUT/WSAETIMEDOUT). This is limited to
platforms that keep socket errors in errno or WSAGetLastError; PS3,
Vita, Wii U, GameCube/Wii and 3DS keep theirs elsewhere and still
report 0 (no code).
Receive failures are left without a code: a peer closing the
connection sets no errno, so any value there would be stale.
The decoder reconstructed a picture whole and then filtered it whole,
one sweep over every macroblock at the end. Each slice decoder now
calls a hook as it completes a macroblock row, and the row above it is
filtered then: intra prediction reads unfiltered neighbours, and by
the time row r is complete it has read row r-1's last line, so the
filter can have it. Macroblock order is what it was, so the picture
is the whole-picture pass's, macroblock for macroblock - the lossless
oracle holds every case byte-exact against ffmpeg - and the row is
filtered while it is still in cache. Pair-scanned and field pictures
keep the whole pass, as does whatever a second slice leaves unfiltered.
What the boundary strengths read matters here: pic_mvg is the
picture's motion-vector grid, folded from the working grid once each
slice is done, and mid-slice it still holds the last picture's. The
hook reads the working grid; the finish reads pic_mvg, which by then
is the same.
A row that is filtered is final, and rows become final one at a time
from the top: that is what a picture decoding on another thread will
wait on, and the reason for this step.
samples/gfx/threaded_video gains a menu-texture lane. The harness driver
had no set_texture_frame at all, so nothing in it exercised the handoff
or the three texture lanes that skip; this one supplies the receiving
end and a slow frame() to make the timing visible, and asserts what
actually holds: with the menu texture enabled, video_driver_frame()
returns with the ring drained, and the driver is handed one push's
pixels rather than a mixture. Removing that drain fails the lane on 12
of 12 menu frames.
That drain is also the correction to what the previous commit claimed.
It said a main-thread set_texture_frame() had been parked on a frame and
its swap on every RGUI frame. It had not: the drain leaves the worker
idle before the next iteration's push, so the lock was never contended
on that path. Narrowing it still stands - set_texture_enable() and
apply_state_changes() arrive on a user action with no push behind them
and were waiting out a render - but the comments now say what is true
instead, and name the drain the lane depends on.
The lane checks the tear case too, cheaply, though that half has no
demonstrated teeth: with the drain removed and the handoff unlocked, the
window between the push's memcpy and the driver's read stayed too narrow
to reproduce here.
* cloudsync: reject invalid manifests
Check the JSON parser result before sorting a manifest. Discard partial manifests and stop the sync with an error instead of dereferencing an incomplete entry.
* webdav: handle missing files and encode sync paths
Treat HTTP 404 as a missing file without writing the server's error page to disk. Encode sync paths separately from the WebDAV base URL so filenames containing '#' are sent as filenames, not fragments.
* webdav: keep '/' as separator in deleted/ backup path
fill_pathname_join_special(dest, "deleted", path) appends
PATH_DEFAULT_SLASH when the directory has no slash of its own. On
Windows that is a backslash, which net_http_urlencode then sends as
%5C, so every backup MOVE targeted "deleted%5C<path>" instead of the
deleted/ collection. Pass "deleted/" so the existing '/' is kept.
* cloudsync: recover from an invalid local manifest
manifest.local is rewritten in place, so a crash, full disk or power
loss during the write can leave it empty or truncated. rjson rejects
empty input, so with the new manifest validation every later sync
stopped with "Invalid manifest" and the only way out was switching
sync drivers, which is what deletes the file.
Log a warning, delete the bad local manifest and continue as a first
sync. With no local manifest, files that differ between local and
server go through conflict resolution instead of being overwritten.
An invalid server manifest is still fatal.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The video thread takes frame.lock for the thread_update_driver_state()
that applies the menu texture and apply_state_changes, and releases it
before the render. A driver takes the texture's pixels inside its own
set_texture_frame(), uploading or copying them there rather than keeping
the pointer, so the staging buffer is free as soon as the update returns
and the render needs nothing the lock guards.
That is what a main-thread set_texture_frame() now waits for: the
handoff, not a frame and its swap behind it. RGUI pushes a texture every
frame the menu is up, so with the software menu open the main thread was
parked on a full render and vsync each frame - hardest on the handhelds
where RGUI is the default. The repeat path drops the lock too: it
presents a retained frame and touches no menu texture.
samples/gfx/threaded_video passes under ASan+UBSan and under TSan, but
it does not cover this handoff: its null driver has no set_texture_frame,
so the video thread clears the update flag without ever reading the
buffer, and removing the lock outright still passes. Giving the harness
driver texture entry points would cover it, and would also un-skip the
three texture lanes that skip today.
The sample stopped at its first file on Windows: setenv and unsetenv
are POSIX, and the C runtime MinGW builds against has _putenv, where
"NAME=" removes the variable. One helper takes both. Every binary in
the sample also took -DHAVE_MMAP, which to the VFS and memmap means
POSIX mmap and sys/mman.h; without it they take the Win32 mapping
calls, the way the tree's own Windows build does, so the define is
set off Windows only. The windowed-open oracle reads /proc and asks
madvise for huge pages, and is built and run on Linux alone.
With that the decoders, the bench, and the catch-up and blit-band
oracles cross-build as PE, which is where the thread-scaling numbers
have to come from: the hosts that produced the first table had one
core.
Whether a 4K preview is fast enough has been answered by watching it.
preview_decode_bench decodes each file given through the stream the
thumbnail uses, at one, two and four threads, and prints frames per
second, the megapixels per second that implies, and the speed-up over
one thread; with --catchup it runs with the stream told it is behind,
so the gain from dropping droppable pictures sits beside the gain from
threads. `make bench` runs it over the fixtures. It is not a test - it
exits 0 unless a decode fails - and it is never built by check, since
numbers from a sanitized build mean nothing.
On one core, single-threaded: VP9 at 300 Mpix/s, H.264 at 190, HEVC
at 85, against the 250 a 4K stream at 30 fps needs. The thread columns
say nothing on a one-core host and everything on a real one: HEVC's
wavefront rows and VP9's tile columns should scale, and H.264 has no
intra-frame parallelism to scale with.
A preview whose decode is slower than its frame rate showed every
frame late - a 4K H.264 stream played in slow motion - because the
scheduler noticed it had fallen behind and did nothing about it. It
now tells the stream to catch up, and clears that the moment a frame
lands on time. Catching up means passing over the pictures nothing
predicts from: H.264's non-reference pictures (nal_ref_idc 0), and
HEVC's sub-layer non-reference pictures in the highest sub-layer,
which needs sps_max_sub_layers_minus1 kept from the SPS the parser
read and discarded. Each is consumed without being decoded and the
decode call returns 0, the contract the container already keeps for
a picture held back for reordering, so its presentation slot passes
and the clock catches up. What is still shown decodes exactly as it
would have: a B-frame-heavy 4K stream does about a third of the work
while behind, and no setting, since this only ever engages once late.
WebM, APNG and WebP reference every frame and take no action.
samples/gfx/gfx_thumbnail_preview gains two fixtures with three
non-reference B-frames between references and no pyramid, and an
oracle that decodes each twice: every frame the catch-up run shows
must be a frame of the full run, byte for byte and in order, or a
dropped picture was referenced after all, and it must show fewer.
H.264 shows 18 of 60 and HEVC 17 of 60 - the I- and P-frames exactly.
The oracle that already lived there animated a fixed twelve frames
and asserted a texture had arrived; with two more fixtures ahead of
hevc_wpp.mp4 in the glob, their jobs were still draining when it
opened and under a sanitizer its first picture missed the window, on
this tree and on master alike. The twelve frames stay as the
measurement; the arrival is now waited for.
The message is built from the task's title, progress and flags under the
lock that guards them, and the push that carries it to the frontend runs
with that lock released. msg_push() inserts into the widget queue and, on
a build with accessibility enabled, reaches a speak backend that forks;
a worker's task_set_progress() has no business waiting behind that.
The push therefore carries the formatted string and nothing the lock
guards, which makes the title and mute tests in
runloop_task_msg_queue_push() the caller's to make: re-reading the title
there would be reading a pointer a worker is free to free, so they go.
The widgets' one remaining read, ALTERNATIVE_LOOK, is set while the task
is built and never changed after, and says so where it is read.
progress_cb() stays under the lock - it reads the task's progress and
finished flag, and pokes the display server rather than going through the
message path.
samples/tasks/progress_push holds the contract. msg_push() there calls
task_get_progress(), which takes that same non-recursive lock, so a push
made under it deadlocks and a watchdog names the phase; the worker churns
the title free-then-set throughout, so a push reading task->title is a
race TSan reports against task_free_title(). Both halves were checked by
injecting each defect. Clean under ASan, ASan+UBSan and TSan; the thread
read audit stays at 0 findings.
filter_lifecycle checks a processed frame's pixels, which reads like
coverage for the fan-out/join and is not: deleting the join outright
leaves it passing, because its slices finish while the verification loop
is still reading them. It also asks for four worker threads and gets
one - Darken and ntsc are the only builtin filters whose
query_num_threads returns the count it was given, every other one pins
filt->threads to 1 - so nothing in samples/gfx has been running the
pool at all.
This lane includes video_filter.c the way samples/audio/mixer_lock
includes audio_driver.c, creates a real pool over Darken, and swaps
filt->impl for slices that sleep on a skewed schedule. On return from
rarch_softfilter_process() it asserts the contract itself: every slice
ran in this generation, outstanding is zero, and no worker still holds a
packet. A watchdog turns a stranded join into a named failure naming the
phase rather than a job that hangs to its limit.
It names four injected defects: the join deleted, the count armed after
the fan-out, a worker that decrements without notifying, and a worker
that tests its flag outside the wait window - that last one a race,
caught on 5 of 5 runs by the no-sleep loop. Clean under ASan, ASan with
UBSan, and TSan. Run by hand; no workflow lists it.
Each worker parks on an eventcount and takes its packet from a release
store, so the fan-out is a store and a notify per worker with no lock
either side. The join waits on one shared eventcount against an atomic
count armed before any worker is woken, and only the worker that takes
that count to zero notifies, so a frame costs one wakeup rather than one
per thread. Workers test their go flag inside the wait window, which is
what lets a fan-out publishing concurrently release them.
Counted with strace over 1000 frames of the same shape: futex calls drop
from 8919-10574 to 5113-5507 at two threads, 11928-12503 to 7596-7863 at
three, and 12569-14759 to 9460-9856 at four. Wall time is not reported:
on a two-core box the run-to-run spread is wider than the effect.
samples/gfx/filter_lifecycle links retro_eventcount.c, which
video_filter.c now needs.
dispmanx, sunxi and the videocore context carry their pageflip state as
an atomic and wait on retro_eventcount. Each waiter opens its wait
window before the work that a vsync callback answers - the swap, or the
flip test - so a callback landing from there on either shows up in the
re-test or makes the commit return at once, and the frame is never held
for a period it already had. The callbacks themselves now take nothing:
dispmanx's runs inside a VideoCore display callback and sunxi's inside a
thread that should be back in its ioctl.
sunxi's nextPage handoff rides the release store that clears
pageflip_pending, so the page update_main() blits into is published
rather than ordered by coincidence. The videocore context also clears
vsync_callback_set when it stops the callbacks, so raising the swapchain
depth and lowering it again re-arms them instead of leaving a waiter
with nothing to wake it.
A voice publishes its type with a release store when it is claimed and
clears it the same way once released, so the mix loops pass over an idle
voice without taking its lock and a fully idle mixer takes none at all.
Both loops read type again under the lock, where an idle voice lands on
its NONE case, and audio_mixer_play()'s slot search keeps its locked
re-check.
audio_mixer_has_float_voices() and audio_mixer_has_s16_voices() read the
same field, which makes their lockless scans well defined; play_s16()
sets is_s16 ahead of the store that publishes the voice, so those two
see the format flag of any voice they see claimed.
android_display_has_focus() returns one acquire load of an atomic the
focus commands publish with a release store, which is all the ordering
there is to carry; the input poll reaches it every frame. A dispmanx
page carries its used flag as an atomic, claimed by the frame thread and
released in the vsync callback - the form the free-page scan already
reads it in - so a page needs no mutex of its own.
task_queue keeps one lock over the running queue: running_lock guards
every walk and every splice of it, task_queue_remove() included, and
task_queue_push()'s blocking-task scan takes it like the push_running()
that follows. librsound calls the client's audio callback directly;
rsd_callback_lock() and rsd_callback_unlock() go with the lock they
were the only users of.
The sample was running in the leftover job, which installs only
build-essential, so wayland-scanner and the client/server libraries
were missing and the run failed at protocol generation. It belongs in
samples/gfx alongside the other display-server lanes; naming it there
also takes it out of the leftover job, which skips whatever a workflow
claims.
No job built the tree with GFX_INSTRUMENT, so the harness lanes that
print what a run cost and hold the budgets - one texture per streaming
surface, no allocation per post, no copy, a 4K submit that stays a
descriptor hand-off - compiled those checks out on every push. The
numbers existed only on whoever ran them by hand. The plain Vulkan
validation leg builds instrumented now and passes the flag to the
harness; it is the one leg for it, since the timings mean nothing
under a sanitizer and the counters change no behaviour.
The surface plan holds that nothing lands on architectural
cleanliness alone, and no gate in it had ever been run. The harness
gains a lane that streams a 3840x2160 surface through the wrapper for
sixty frames and takes the numbers: the time a submit costs the main
thread and the frames it takes for the slot to come back, as
p50/p95/p99, with the budgets asserted - one texture, no allocation
per post, no copy, and a submit that stays a descriptor hand-off. It
is thread CPU time, not wall clock: on a one-core host the video
thread's 33 MB upload preempts the main thread inside the submit, and
the clock on the wall charged that to the hand-off.
The counters were missing half the work. A load or update posted
through the asynchronous list never passed through the synchronous
entry point where the counter sat, so every upload the surfaces make
- the point of them - went uncounted, and the baselines printed so
far undercounted by that much. They are counted on the thread that
runs them now. With the true figures the surface lane's own
assertion, two loads for two surfaces, turned out to have passed only
on the undercount: the mode switch between its halves reinits video
and loads the menu's textures through the same counters. Each half is
measured on its own now, with the reinit outside both.
Without the row in intl/msg_hash_lbl.h the label resolves to "null",
so the deferred push matches nothing and stays on its default, which
is a file browser: OK on CRT Super Resolution opened Select File
instead of the list.
What it does is hold the display's own line count and move only the
rate, which fits any fixed-pixel display - an OLED as much as an LCD.
The label named a panel technology instead, so an OLED owner would
read past the one mode that serves them. It now says what it does,
and matches the wording already in the sublabel.
The ini keeps monitor lcd: that is switchres's name for the preset
and it is in people's configuration files.
The log said the display server cannot apply a modeline, and
separately which connectors are up for lease, and left the reader to
join them. It now names the setting that takes the offer, and only
where there is one to take.
The display server reported leasable connectors with its own copy of
the listeners, and the lease module then grew the same ones to take a
lease. Reporting moves to the module, on the connection the display
server already has, so the protocol is spoken once - a sequencing
mistake has one place to be made rather than two.
samples/gfx/wayland_drm_lease covers both sides already and is
unchanged, which is what says the reporting still behaves.
A revoked lease reaches the DRM context as a page flip that will not
queue, and looked like any other flip failure. The question is asked
there, where it already went wrong, rather than on every frame: the
lease reads its connection non-blocking - dispatch what is queued,
then read the socket only when it has something - and the context
says so once.
samples/gfx/wayland_drm_lease revokes a live lease from the
compositor's own event loop, since its resources belong to that
thread, and waits on the client side with the same non-blocking call
a caller would use.
A mingw cross build puts them in a binary the audit can walk - WASAPI,
WDM-KS, DirectSound, XAudio, ASIO, raw input, the DirectInput
enumeration job, the modeline resync and the D3D11 and D3D12 frame
contexts, none of which a Linux build contains. It audits 62 entries
with no findings, and the note now carries the two commands.
Between that and a wide Linux build's 76, the entries neither can reach
come down from 52 to 27: Apple, Android, emscripten, the consoles, Steam
and a few drivers whose library is not packaged here. Those still want
their own toolchain.
A Wayland client cannot put a modeline on the wire, and a card opened
inside a session cannot modeset because the compositor holds master.
A DRM lease settles both: the compositor gives up a connector, its
CRTC and a plane, and hands back a descriptor that is master for
exactly those, on which an atomic modeset carries a full
drmModeModeInfo.
With that descriptor the existing path serves it unchanged. The DRM
context scans out on it - resources on a lease are only the leased
objects, so the head is already chosen - and the display server is
the KMS one, which applies the engine's modelines to that connector
the way it does on a bare KMS session. A leased connector is one the
compositor has given up, so the desktop keeps the heads it did not
lease: this is a second output, not a takeover.
Reached by setting the video context driver to the DRM one inside a
Wayland session; without a lease that context cannot come up there at
all, so nothing changes for anyone else. The monitor index counts
through the offered connectors as it does elsewhere.
samples/gfx/wayland_drm_lease grants leases now as well as
advertising them, so the request, the descriptor, the index and the
refusal are all exercised against libwayland rather than reasoned
about.
Revocation is not handled: a compositor may take the lease back, and
noticing its finished event needs the connection's descriptor in the
poll the DRM context runs for flip events. Until then a revoked lease
shows up as DRM calls failing.
Left and right were the only way through NATIVE, DYNAMIC, 1920, 2560
and 3840, so the setting takes OK to a list like the rest of them.
The values are widths, not an index - 0 and 1 mean native and best
fit - which the generic uint dropdown cannot express, since it walks
min to max by step and writes the index back. So the list is its own
DISPLAYLIST_DROPDOWN_LIST_CRT_SUPER_RESOLUTION beside the resolution
one, each row carrying its value, with the entry index only ordering
them. The stored value is unchanged, and left and right still work.
No ST_UI_TYPE: the Qt companion's uint combobox populates from min to
max, which are zero on this row, so the hint would give the desktop
menu an empty box.
"For CRT displays only" is not true of the engine: the EDID and LCD
modes take their limits from whatever the display reports, and a
panel gets the content's refresh rate the same way a tube does. The
sublabel now says what the setting generates and which modes are for
which display.
English source only; the locales follow through Crowdin.
Two properties with a history, one harness, both against the shipping
translation unit rather than a model of it.
ps3_audio_wait_block() has to hold cond_lock across sysLwCondWait():
that call wants the lwcond's own mutex and fails with EPERM without it,
which is what turned the write's bounded waits into microseconds of
spinning that returned zero and dropped the audio on a full ring. The
device mock refuses on the same precondition and counts it, so the check
is a run of blocking writes longer than the ring: they must all land,
with no refusal counted. Removing the lock again fails at round 16.
quit_thread is written by the thread tearing the driver down and read by
the output thread's loop every pass, with no lock either side. Against
the volatile bool it used to be, ThreadSanitizer reports the read at
ps3_audio.c:89 against the write at :302; it is clean as an atomic.
tools/audio-suite-check.sh finds this for ps3_audio.c on its own, so the
driver's pre-commit check covers it without a workflow change.
"INI" and "EDID" said nothing about what they do, and the one thing
the engine can do for a panel - hold its resolution and move only the
rate - had no entry at all. The dropdown now reads Custom
(switchres.ini), Match Display (EDID) and Match Refresh Only (LCD);
the strings are literals, so no locale is disturbed.
The new mode selects the lcd preset, whose refresh band is seeded
from the display's EDID once the server is bound - its own default is
the desktop rate either side by one, which switches nothing - and the
preset is filled again from that band, since setting the monitor
filled it from the old one. A display that states no band keeps the
desktop rate and says so.
The value is appended, so a saved crt_switch_resolution keeps its
meaning.
samples/gfx/crt_switch_modeline covers it: 70.086 Hz content on a
panel reporting 50-75 Hz lands at [email protected], the panel's own
line count at the content's rate.
The flag became an atomic with nothing to reproduce the race, and no
existing suite could: mixer_lock stubs the mixer, and the two mixer
suites under libretro-common are the wrong layer for a tasks file.
audio_mixer destroys a voice from the mixer thread as well as the
caller's, so the borrow's release callback sets dead on one thread while
the feeder task's handler reads it on another, with no lock either side.
This drives that pair directly - including the task's translation unit to
reach them, as samples/audio/psp_ring does for the console drivers, since
both are static. The handler and the release are the shipping ones.
The stubs alongside keep a pass cheap: with no transfer window the
handler bails on its first tick, and the classic loader it falls back to
gets no task, so a round costs the read of dead and little else. The
codec defines matter - without one of them the whole windowed path is
compiled out, and an earlier measurement of this seam was taken against
a build that did not contain it.
Against the volatile bool the field used to be, ThreadSanitizer reports
the read at task_audio_mixer.c:1265 against the write at 1225. Clean as
an atomic, and clean under ASan with leak detection and UBSan.
The engine is display-agnostic and the EDID preset takes its limits
from whatever the display reports, so a panel that says it syncs to
75 Hz is handed the content's rate rather than the desktop's. The
case builds a 50-75 Hz block with the tree's own generator, so the
block the parser reads is the one the writer emits, and asks for
70.086 Hz content.
Nothing in the menu says this works on a display that is not a CRT,
which is exactly why it wants a lane: with the display reporting no
EDID the preset falls back to generic_15 and the applied mode is not
at the content's rate.
Content loading tears the display server down and builds it again
with the engine alive across it. The rebind does not re-enumerate and
an already flushed mode is not flushed again, so the new instance is
asked to switch to a mode it was never handed.
Every backend is fine with that for its own reason - X11 finds the
RRMode still in the X server, KMS reads the timing out of the struct,
the Win32 backends registered it with the driver - and all of them
rely on the mode carrying its whole timing at set(). That is what the
new case pins, together with the switch going through, so a backend
that grew a dependency on its own add() shows up here rather than as
a black screen after a core loads.
pipe_consumer_gone became an atomic without anything to reproduce the
race it was: the suite's ThreadSanitizer lanes passed the same either
way, because none of them has the consumer leave its loop while a
producer is still waiting on it.
CONSUMER_EXIT drives exactly that. The producer fills until it has to
wait, the device then refuses every write, and audio_thread_write()
clears alive under thr->lock and the wrapper leaves its loop - reading
alive under the same lock, but publishing pipe_consumer_gone under none,
to a producer reading it under none either. Nothing joins between the
two, because the producer is parked rather than tearing the driver down.
Against the volatile bool the field used to be, ThreadSanitizer reports
the pair: the read in the wait at audio_driver.c:5596 on the submitting
thread against the write in audio_driver_pipeline_consumer_exit() on the
wrapper thread. It is clean as an atomic.
The lane also asserts the producer comes back at all, since a consumer
that has gone is one it must stop waiting for.
A Wayland client cannot put a modeline on the wire: every modeline op
in dispserv_wl.c is NULL, and the protocol that changes a mode carries
width, height and a refresh rate, nothing that can express a porch.
A DRM lease hands a connector over whole, which is the one route that
can, so the display server now says at init whether the compositor
offers one and which.
Discovery only. The lease is not taken and no scanout changes; what
the log answers is whether a given compositor could ever serve a CRT,
which today nothing tells a user.
The non-master fd the compositor sends for device picking is closed
as it arrives, and the device is released once the connectors are
read.
drm-lease-v1.xml is bundled, and generate_wayland_protos.sh falls
back to the bundled copy per protocol, so a wayland-protocols older
than the one a protocol first shipped in no longer fails the build.
samples/gfx/wayland_drm_lease runs the real display server against a
compositor built on libwayland-server: the mistakes this code can
make are protocol errors, which kill the connection at startup and
are invisible to a compile. Run by hand:
make -C samples/gfx/wayland_drm_lease check
Under HDR the last slang pass is force-allocated as the composite's
input, so it renders with the unrotated mvp_last_pass like any other
pass that owns an RT. Pass the content rotation to the composite for
that source as well, so TATE content is rotated by the composite's UV
transform whether or not a shader chain is active.
crt_engine_init() runs on every geometry change, so the lines that
describe the context rather than the switch repeat for as long as a
core keeps changing mode. The context identity and the engine-alive
line now belong to the engine's start.
Vulkan direct-to-display says what it cannot do at bind, beside the
no-modeline-path line, which leaves the per-switch branch as the
apply alone.
samples/gfx/crt_switch_modeline covers both contexts over three
geometry changes.
A server whose modeline ops are all NULL - Wayland - now says so
once, when the engine binds it, naming itself, and the per-switch
apply is skipped rather than reported as a failure. The mode list is
still generated, so the aspect and the timing the consumer publishes
are unchanged.
A server that does have ops and whose set() fails still reports it.
samples/gfx/crt_switch_modeline links the consumer and the engine
and stubs the display server, which is the only way to stand up a
server with no modeline ops on demand. It also pins the connector
the EDID install hint names. Run by hand:
make -C samples/gfx/crt_switch_modeline check
The firmware-EDID hint names the connector the monitor index lands
on, resolved the way the Monitor Index setting itself resolves a
head; "auto" is named only where there is a single output, and the
list is read up to eight heads deep.
A second line gives the debugfs edid_override route, which puts the
block in place without a reboot. Not every driver exposes
trigger_hotplug, so the line says so.
retro_timers.h includes <pspthreadman.h> for PSP, whose only use in this
header is sceKernelDelayThread, and then named PSP in neither sleep
ladder - so both calls fell through to the generic nanosleep body and the
include was dead. Frame Delay and the runloop's pace to content frame
time are both sub-frame sleeps that go through retro_sleep_us, and on PSP
they were getting libc's resolution rather than the microsecond delay the
kernel offers.
PSP joins VITA, which takes microseconds through the same call with the
same signature. Preprocessing -DPSP now yields sceKernelDelayThread where
it yielded the nanosleep body; GEKKO, WIIU, _3DS, VITA and the generic
branch are unchanged.
The macro also drops a libc symbol where there was one, which the
statically linked platforms care about more than most: nothing in the PSP
salamander object set includes this header, and the one salamander TU
that does is the Wii's, whose branch does not move.
opensl.c asks audio_layout_channels() which layout the frontend wants,
and the suite builds the driver itself, so its link needs the unit that
answers - the lane fails at link without it.
wait_readable() returns a count, not a yes or no: it is what the device
has now, and that is regularly less than the slice asked for. Every
driver returns short when its bounded wait expires with something in
hand, and alsa returns a period even where the slice spans several -
a small device cannot hold a whole slice at all. read() blocks until it
has everything it was asked for, so the worker must ask only for what
wait_readable() just reported. Otherwise it sits in a read that only
more samples can end, and teardown - which joins this thread
unconditionally - hangs with it, taking mic close, driver switch and
shutdown down too.
The lane goes in samples/audio/mic_handshake, which already drives the
shipping worker against a clocked device. Its read returns what it has,
so it could never show this; the case adds one that blocks as the
header specifies, lets the device go quiet under it, and tears down
with a watchdog so a stuck join fails in five seconds instead of
hanging. The measurement runs keep the non-blocking read and their
numbers.
The nonblocking halves of alsa, sdl2, sdl3 and coreaudio's mic reads
went with the hook that selected them, so every read takes the blocking
path the header describes.
That empties the alsa and sdl2 driver contexts: what init() and free()
manage is ALSA's global state and SDL's audio subsystem, and the
frontend only needs a handle to give back, so both hand back the
driver's own vtable rather than an allocation of nothing. CoreAudio's
keeps the microphone it tracks.
The init gate's comment says rate control needs write_avail and
buffer_size, but it only tested buffer_size. A driver with a size and
no fill took AUDIO_FLAG_CONTROL, and that flag is the only guard on
audio_driver_compute_rate_adjust()'s write_avail() call, so the first
batch past the DRC threshold called through a null pointer. audioio
has exactly that shape on the platforms whose audio_prinfo carries no
seek.
The regression lane goes in samples/audio/pipeline_format, which
already drives audio_driver_init_internal() against a patched
audio_null: rate control must stay off without write_avail and come on
with it. Without the fix it segfaults. That case also needs the
wide-then-float one to release the driver it leaves initialised, which
it now does.
Nothing in the frontend has ever called it: audio_driver_set_nonblock_state()
reaches the playback vtable only, and there is no setting, flag or menu
entry behind a non-blocking microphone. The reads the header describes
as the default - blocking until the frontend has what it asked for - are
the only ones a driver is asked for, and the threaded capture worker
wants exactly those, with wait_readable() bounding the wait instead.
Six drivers and the null driver lose the slot, and the three harnesses
that drove it by hand lose the call. The nonblock each driver latches
stays where another path still writes it, so wasapi keeps the state it
takes from audio_sync at init and pipewire keeps what the playback
setter leaves on the shared core.
roar and xenon360 both answered write_avail with a literal 0. A driver
that cannot measure its fill should leave the slot NULL rather than
report a constant: NULL disables rate control, while a constant feeds
it a fill that never changes and it corrects against that forever.
switch audout returned the whole size of the buffer it was filling
instead of the room left in it, so a nearly full buffer read as an
empty device. wait_writable in the same file already had the right
expression; write_avail now uses it, and reports nothing when no
buffer is in hand, since acquiring one may block.
write_avail() returned buffer_size() - the whole buffer, every call,
whatever was queued in it. Rate control reads that as a device that
never drains and answers with its full upward correction for as long
as it is on, which is the half percent of pitch the PS2 driver carries
a note about having had.
audio(4)'s play.seek is the count of bytes pending, so the room is the
buffer less that. It comes from AUDIO_GETBUFINFO, which exists to be
the cheap read of exactly these fields.
Both are NetBSD's: the Solaris audio_prinfo has buffer_size and
samples and no seek, so there the hook is not offered at all. The
interface asks for nothing rather than a constant, and the frontend
turns rate control off and says so, which is the truth.
buffer_size() returned false from a size_t on a failed ioctl.
audio/drivers/audioio.c compiles on the systems whose toolchain ships
<sys/audioio.h> - NetBSD and Solaris - so nothing here has held it to
a syntax gate. A compile-only stand-in, shaped after NetBSD's:
gcc -fsyntax-only -std=c89 -pedantic -Wall audio/drivers/audioio.c \
-isystem tools/platform_stubs/bsd -Ilibretro-common/include -I. \
-DRARCH_INTERNAL
AUDIO_GETBUFINFO is NetBSD's alone, which is what the driver keys its
play.seek branch off; BSDSTUB_NO_GETBUFINFO compiles the other side of
that gate on a host.
sdl3_counters drives the two callbacks directly and then runs the
driver against SDL's dummy device, asserting what the frontend reads:
a covered callback is not a shortfall, microphone puts never reach the
count, feeding leaves it alone, starving moves it, and a reopen keeps
what it has. The shape sdl1_counters uses for the same contract.
Hand-run like the other two sdl3 suites - tools/audio-suite-check.sh
audio/drivers/sdl3_audio.c builds and runs all three, plain and under
ASan+UBSan and TSan.
The output stream gets its own get callback for the count, and the
shared wake goes back to waking. A put callback carries the audio the
microphone just delivered in additional_amount, so counting in the
function both streams call has a working microphone reporting an
underrun per device iteration on its own context.
The count also starts at init rather than at the stream, which is how
audio_driver.h specifies it and how the frontend reads it: the pipeline
compares underruns() against what it last saw and treats any change as
fresh silence, so a count reset by a device reopen costs it a pass and
the audio it was holding.
The process callback fills a quantum with silence in two places: when
the ring comes up short, which it already computes zero_frames for,
and when it cannot take the lock at all - and that one printed a
warning from the render callback and returned. Both are one atomic add
now, and underruns() hands the count to the frontend, which shows it
and says it once at teardown.
The short-ring case counts only while the driver is running: one that
is not running is not being starved.
memalign() reached this file through <memory.h>, which emscripten's
own webaudio.h pulls in; <malloc.h> is included here now, so the
driver stands on its own.
audio/drivers/audioworklet.c compiles only under emscripten, whose
headers refuse to be included from its source tree - they want a
generated sysroot - so nothing here has held that driver to a syntax
gate. Stand-ins for <emscripten/webaudio.h>, <emscripten/wasm_worker.h>
and <emscripten/atomic.h>, and the EM_ASM_* macros beside the
emscripten.h stub that was already here:
gcc -fsyntax-only -std=gnu99 -Wall audio/drivers/audioworklet.c \
-isystem tools/platform_stubs/emscripten \
-Ilibretro-common/include -I. -D__EMSCRIPTEN__ -DRARCH_INTERNAL
The webaudio stub includes emscripten.h as emscripten's own does, so
the driver reaches the EM_ASM_* macros the way it does in a real
build. Struct shapes follow emscripten's, because the driver
initialises the options positionally and indexes the sample frames.
check_pcm_status() runs from the write path and said SND_PCM_STATUS_
UNDERRUN to the log every time the device ran dry - a line during
play, on the path whose margin it is reporting on. It is one atomic
add now, and underruns() hands the count to the frontend, which shows
it on the statistics overlay and says it once at teardown.
The file also reached memset, calloc and the errno values through
QNX's <sys/asoundlib.h>, which pulls them in; they are included here
now, so the driver stands on its own includes.
audio/drivers/alsa_qsa.c compiles nowhere but QNX, whose headers ship
with its toolchain, so nothing in this tree has ever held it to a
syntax gate. A compile-only stand-in for <sys/asoundlib.h> is enough
to, as the android and ps3 stubs do for theirs:
gcc -fsyntax-only -std=c89 -pedantic -Wall audio/drivers/alsa_qsa.c \
-isystem tools/platform_stubs/qnx -Ilibretro-common/include -I. \
-D__QNX__ -DRARCH_INTERNAL
Shapes and spellings follow QSA rather than ALSA; the values are
arbitrary where the driver only assigns or compares them.
sdl3_mic_wait drives the put callback by hand against a real unbound SDL
stream and asserts what the capture worker depends on: a silent device
answers within one device period, a period satisfies a slice-sized
request, a put ends a parked wait, and an unplug ends one with nothing.
Hand-run like sdl3_raw beside it - tools/audio-suite-check.sh
audio/drivers/sdl3_audio.c builds and runs both, plain and under
ASan+UBSan and TSan.
sdl3_microphone_wait_readable() takes its per-wait bound from
sdl3_microphone_wait_ms() - two device periods, clamped to 20-200 ms,
the shape alsa_microphone_wait_ms() uses - and caps what it waits for at
one period. The capture worker re-reads its exit flag only between these
waits, so that bound is also how long closing a microphone holds the
main thread in sthread_join(); and a target above what buffer_size lets
the stream hold can never be reached, the put callback dropping the
backlog past it. The read loop's wait takes the same bound, and
period_frames carries what SDL reports at open for all three streams.
The player was created with two channels whatever the setting said.
The PCM_EX format the float path already uses carries a channel mask
as well, and the frontend's mask is that mask - both are the
WAVEFORMATEXTENSIBLE bits - so it goes across as it is and the frames
arrive in its ascending-bit order either way.
Tried once, ahead of the stereo attempts, and only where PCM_EX is:
API level 21, the gate the float path is already behind. A player that
refuses the wider format falls through to those attempts untouched,
which is also what an older API level does, and layout() then reports
stereo.
The renderer was built for two channels whatever the setting said.
audrvDeviceSinkAdd() takes up to six, so 5.1 is the widest this
renderer has, and it now opens that when the layout asks for it. The
mix buffer count follows the channel count, which is why the config
moves into the driver: audrenInitialize() and audrvCreate() have to be
handed the same one. Anything that is not two or six - a four channel
mask - opens as stereo, and layout() reports what was granted.
use_float() says why it is false while it is here. audrvVoiceInit()
takes PcmFormat_Int16 and PcmFormat_Adpcm and rejects the rest,
PcmFormat_Float among them, so this is libnx's limit and not a
setting this driver declined to read.
The port was opened with two channels whatever the setting said.
libaudio takes six and eight as well, so it now opens what the layout
asks for and falls back to two when the port refuses it or the count
is not one libaudio has - a four channel mask is not. layout() reports
what was granted, so the frontend upmixes to that and no further.
The block the output thread hands to audioAddData() moves to the heap
on the way. That thread is created with 4 KiB of stack, which one
stereo block already half fills; six channels would have run straight
past the end of it.
Both clock fits read the platform's tick register directly -
gettime() on the one, svcGetSystemTick() on the other.
cpu_features_get_perf_counter() is those two calls, on those two
platforms, and is where a counter read belongs.
The scale stays per-platform: features_cpu.h offers the counter and
no frequency beside it.
Both know the device's position - the DMA callback counts chunks, the
DSP callback reads the channel's sample cursor - and neither took a
timestamp beside it. With one, the slope of position against time is
what the hardware is really running at, against the rate the driver
announced.
The AI's is not a crystal difference but a divisor: it divides its
source clock by a figure that does not land on the rate it is asked
for, and the gap has never been reported anywhere. The DSP's is a
straight comparison of two domains, the channel's cursor against
svcGetSystemTick().
gx fits on the frontend's thread from a pair the callback publishes
under a sequence counter: that callback is a DMA interrupt, and
floating point does not belong in one. The pair has to come from
there even so - sampling the position from the frontend is a chunk
out, which over any usable window is more than the figure being
measured. ctr_dsp fits in its callback, which is an ordinary thread,
and publishes the result as one int.
ctr_dsp takes svcGetSystemTick() rather than
cpu_features_get_time_usec(), which is osGetTime() on this platform
and resolves in milliseconds - a hundred parts per million of
quantisation on a figure measured in single parts.
Measurements, both. Nothing acts on either.
The driver handed audio to audsrv and asked it nothing back, so it had
no write_avail() and rate control stayed off for the session. audsrv
answers all three questions: audsrv_available() is the room in its
ring, audsrv_queued() is what it still holds, and the ring's size is
what it reports free before anything is queued - so buffer_size()
becomes the real ring rather than a constant, write_avail() a
measurement, and frames_consumed() what was handed over less what is
waiting.
wait_writable() polls that room with a bounded sleep and a lap cap
rather than calling audsrv_wait_audio(), which blocks with no bound of
its own and would park the audio thread on a server that stopped
draining.
nonblock is still not implemented; the frontend's own policy now has
the room figure to work from.
The latency-update callback already holds the server's timing info and
read only sink_usec from it. read_index is what the sink has played,
in bytes, and timestamp is when that was true - the pairing of
position and time device_clock_ppm asks for, arriving on its own
without a call to fetch it.
Fitted over the window rather than between two points, and restarted
when read_index steps back on a rewind or a flush. A measurement:
nothing acts on it.
The rest of the drivers that detect a starved device and do nothing
with it. tinyalsa recovers from EPIPE; rsound's callback returns a
short count librsound zero-fills; SDL3 is handed the shortfall as the
get-callback's additional_amount and ignores both its integer
arguments; OpenSL's buffer-queue callback decrements to zero and drops
the result; the PS3 output thread memsets a block it had nothing for;
the GX DMA callback retires each chunk by erasing it, so a chunk the
writer has not reached goes out as the silence left behind; the AX
frame callback already stops the voice when the buffer is short.
Each adds one to a counter where it happens and reports it through
underruns(). GX and AX use the plain volatile counters beside them
rather than atomics, as those files do for their frame counts.
rsound, ps3, gx and opensl stopped short of the slot, so the members
before it are named NULL rather than left to position.
init reported 32730 Hz to the frontend and set the channel to
32728 Hz. The channel interpolates with NDSP_INTERP_NONE, so the
frontend resampled to a rate the DSP was not playing at and the ~60
ppm between them had nothing to correct it. Both now come from one
constant.
Also drops the second `if (!ctr)`: ctr is checked, with ndspExit(),
where it is allocated, and the callback and the event have already
been given it by the time the second check is reached.
rs_init() calls RARCH_LOG without the header that declares it, so a
build with HAVE_RSOUND=1 stopped on an implicit declaration under
C89. It reached the file transitively on no include path this tree
uses.
The device callbacks, with JACK, PipeWire and PulseAudio built, take a
wide Linux build from 59 audited entries to 76, all clean. The note's job
is to stop a narrow build's zero being read as a clean tree, so the
figure in it should be the one a wide build actually reaches.
The grab waits for two polls to agree on a screen something has been
drawn to, up to a deadline, instead of trusting a fixed wait. A
frontend that is still starting has drawn nothing, or has drawn into a
window short of its final size, and either is a picture of the startup
rather than of the overlay.
A wrong picture settles at once and fails as it always did: a driver
squeezing the quads is caught on the first pair of polls, and an
overlay that never arrives fails on the menu screen beneath it. A run
that stays in motion to the deadline says so and is checked anyway. On
a healthy machine both lanes now finish sooner than the old wait alone
took.
Each of these drivers already knows when the device went without
audio and did nothing with it. ALSA recovers from -EPIPE and drops the
fact; PulseAudio registers an underflow callback whose body is #if 0'd
around a log line; PipeWire fills the quantum with silence when the
ring is empty; DirectSound's mixer thread has a branch commented
"nothing in FIFO (underrun)"; OpenAL finds the source STOPPED and
restarts it. All five now add one to a counter where it happens and
report it through underruns().
The frontend shows the count on the statistics overlay and says it
once at teardown, which is where a line about audio belongs - pulse's
disabled callback was disabled because it logged from the mainloop's
thread while audio played.
Only the event each backend really reports: -EPIPE and not the -EINTR
and -ESTRPIPE beside it, a PipeWire quantum with nothing at all rather
than a short one, and AL_STOPPED rather than any state that is not
AL_PLAYING, so the first write of a session does not count itself.
The gate walked from the threads this tree creates and from the audio
pipeline's callback. A driver's device callback belongs to the backend
- SDL's device thread, libpulse's and PipeWire's loop, JACK's process
thread, the ASIO driver thread, CoreAudio's listeners, the DSP and AX
frame interrupts - so no create call here names it and the binary walk
has no edge to reach it. Those callbacks are worker threads by every
other measure, and nothing was auditing them.
They are declared, for the same reason the driver frame-context
functions are. A driver's init, open and free stay off the list: they
run on the main thread, where a settings read is legal and every one
of them would be a false finding.
The device callbacks reached for audio_state_get_ptr() to raise
reinit_request: the ASIO driver thread on a reset or rate change,
JACK's server threads on a rate or period change, the HAL's listener
when the default output moves. The singleton is static and the field
is an atomic, so nothing was unsafe, but a worker reaching into
frontend state is the shape the thread-read audit exists to reject,
and wasapi already avoids it - it captures the same pointer on the
main thread at sthread_create and hands it to the worker.
Each driver now takes the pointer at init and its callbacks store
through it.
Both waited without a timeout. audrenWaitFrame() is eventWait() with
UINT64_MAX, and LightEvent_Wait() has no bound at all, so a renderer
or a DSP that stops signalling - stopped, reset, or the channel
dropped coming back from sleep - parks the caller for good. On
wait_writable that is the threaded pipeline's audio thread; audren's
blocking write could hold the core's thread the same way.
Both now take the event with a timeout and a lap cap, as every other
driver's wait does, and report a short count when it expires. The
timeout is never reached while the device runs, so it only decides how
long a stopped one takes to be noticed.
Casting alcGetProcAddress()'s object pointer straight to a function
pointer is not legal C, and `make C89_BUILD=1` with HAVE_AL=1 stopped
on it. al_init_events() a hundred lines above already resolves its two
entry points through a union; the device clock now does the same.
Both drivers park a poll thread on a flag the teardown sets from another
thread, with no lock either side, as volatile bools that ordered nothing
between the two. The flags are initialised before their thread is
created, and on the Wii USB side before the first goto error, since free
stores through it.
wiiusb_hid's device_detected goes the other way. It is raised by
wiiusb_hid_change_cb, which libogc runs in interrupt context, so an
atomic is not what it wants - the writer is not a thread, and this file's
other interrupt-shared state is volatile for that reason. It was a plain
bool, which left the poll thread free to hold it in a register across the
loop; volatile is the smallest thing that fixes that.
Both files build against the headers already in the tree - libogc under
wii/, wut under wiiu/ - on a 32-bit PowerPC compiler and on all five
retro_atomic backends, unmodified first and then with the change.
samples/input/android_haptic_defer reproduces the defer protocol and
pins it: the callback never enters Java, repeats inside one iteration
coalesce, a press is not carried across a pause, stop, window teardown,
focus loss or destroy, and the delivery guards catch a press raised
after the command in the same poll batch. Two sabotage modes put the
pre-defer dispatch and the missing clears back and are asserted to be
caught. Run by hand, like samples/input/android_kbd_mailbox.
doHapticFeedback() is called from the native app thread, and
View.performHapticFeedback() is not documented as thread safe. Drop the
per-keypress log line with it: it runs on an interactive path.
The flag is input-driver state, set and read only on the app thread.
Hold it as a file static beside android_state_flush_pending rather than
in struct android_app, which is shared with the Java thread under the
app mutex. Clearing it before the delivery guards also stops a request
raised before g_android exists from sticking.
runloop_iterate() already carries the Android work that must happen
outside the core, next to the APP_CMD_PAUSE save that defers for the
same reason. Dispatch the keypress haptic there instead of from
retroarch.c's outer loop, which is compiled out under HAVE_MAIN.
Record pending keypress feedback in Android app state and dispatch it after runloop_iterate() returns. Coalesce requests per iteration and clear pending feedback on pause, stop, focus loss, window removal, and destruction.
Guard delivery by activity, window, focus, and JNI method availability.
A failing lane prints the rectangles on the screen grab, the
rectangles the config asks for, and the lines of the log that say
which context, driver and resolution drew them. A geometry failure is
then read off the CI output rather than reproduced: two bands of two
spans are the two squares, and their rows and columns say what moved
them. A passing lane prints the line it always did.
Set by the thread tearing the driver down and read by the output
thread's loop on every pass, as a volatile bool that ordered nothing
between them. The same flag is already an atomic in the PSP, Vita and
PS4 drivers; this one was left behind.
tools/platform_stubs/ps3 grows enough of PSL1GHT's audio, thread,
event-queue, mutex and cond headers to compile the driver on a host, so
the change is checked rather than read: the driver builds unmodified
against them first, then on all five retro_atomic backends. The headers
ps3_defines.h includes unconditionally are there as empty placeholders,
marked as such.
The volatile fallback backend declares retro_atomic_cas_int_fb as
returning bool, the header's only use of the type, and nothing here
provided it. A translation unit that includes retro_atomic.h before
anything else fails to compile on that backend - two lines are enough to
show it - which is every platform without real atomics.
The forced-backend loop in the gfx samples workflow did not catch it
because all four files it compiles reach bool through another header
first. boolean.h is what the sibling headers use, is header-only so the
salamander object sets are unaffected, and is a no-op under C++, where
this header is included inside extern "C".
The vtable scan, the shared-declaration scan and the driver count run
over a copy with comments and literal contents blanked, so a slot
spelled `NULL, /* presentable */` - the tree's way of naming every
unimplemented slot - is a NULL slot and nothing more. Blanking keeps
every offset, so the brace walks still line up with the text.
Three selftest fixtures hold it: a NULL slot named in a comment, a
comment beside a wired hook naming a different one, and the hook's
name inside the driver ident string. Each of the three reports the
checker's defined-nowhere error without this change.
Both are written by one thread and read by another, and volatile orders
nothing between them, so neither reader had a guarantee of seeing the
write at all. This is the same reason capture_running in
microphone_driver.h is an atomic, where ThreadSanitizer reported the
race on the field itself.
pipe_consumer_gone is set by the wrapper thread as it leaves its loop
and read by the producer deciding whether to keep waiting; it takes a
release store and an acquire load.
audio_mixer_wfeed's dead is set by the sound's release, which
audio_mixer runs from the mixer thread as well as the caller's, and read
by the feeder task's handler on the task thread.
Checked on all five retro_atomic backends, C11 through the volatile
fallback, and on a clean C89 build.
Each unit targets a single SDL major version and exports its own vtable:
gfx_ctx_sdl1_gl and gfx_ctx_sdl2_gl. Makefile.common selects the object
inside the existing HAVE_GL_CONTEXT guard, griffin includes it in the
matching arm of its SDL chain, and gfx_ctx_gl_drivers[] lists whichever
one the build defines - SDL2 first, as the merged file's own #ifdef chain
resolved it. Both keep the "gl_sdl" ident, so video_context_driver settings
carry over untouched.
sdl1_gl_ctx.c holds the SDL 1.2 surface path alone: no GL context or shared
context members, no webOS hints, and no settings lookups, since the monitor
index and windowed-fullscreen flag only ever fed SDL_CreateWindow. Its
presentable slot is NULL, which gfx_ctx_driver_t documents as "always
presentable"; bind_hw_render stays an empty function rather than NULL
because gl3.c calls it unguarded whenever a core requests hardware
rendering.
Each unit targets a single SDL major version and exports its own vtable:
sdl1_joypad (ident "sdl") and sdl2_joypad (ident "sdl2"). Makefile.common
selects the object alongside the matching input driver on HAVE_SDL2,
griffin includes it in the same arm, and joypad_drivers[] lists whichever
one the build defines.
sdl1_joypad.c keeps only the plain joystick API: no GameController, haptic
or sensor members, so the pad accessors collapse into their call sites and
set_rumble, set_sensor_state and get_sensor_input are NULL entries rather
than stubs that always return false - both are null-checked by their
callers. The unused compat/strl.h include goes with it.
The wheel event already records mouse_wl/mouse_wr; hand them back under
RETRO_DEVICE_ID_MOUSE_HORIZ_WHEELUP/DOWN so cores and the menu see tilt
scrolling, with the same axis orientation sdl3_input.c uses.
Each unit targets a single SDL major version and exports its own vtable:
input_sdl1 (ident "sdl") and input_sdl2 (ident "sdl2"). Makefile.common
selects the object on HAVE_SDL2, griffin includes the matching unit in
each arm of its SDL chain, and the driver registry lists whichever one
the build defines. The SDL1 arm of griffin now carries the input driver
and joypad next to the audio driver so it links on its own.
sdl1_input.c omits the KMOD_SCROLL probe: 0x8000 is KMOD_RESERVED under
SDL 1.2, so scroll lock stays unreported there and remains SDL2-only.
A build with Vulkan, Qt, ffmpeg, HID and Bluetooth audits 59 worker
entries where a default Linux build audits about 42, and the wider call
graph reaches five gfx_display helpers from gl2_frame, gl3_frame and
vulkan_frame. They carry the same argument as the pairs already
sanctioned through the menu and widgets stack: past gfx_display_flush_as
they touch only blend_on and dispctx. Keyed on the reading function, so
one line each covers all three roots.
video_thread_hw_allowed() goes in with its reasoning written out. The
core's context type and the configured driver name are both settled
before the video thread exists, and a driver change reinits through a
join. The bounded transient between the menu's write and that reinit is
recorded rather than papered over, together with why latching is not the
cheaper answer: the macro calling it runs inside gl2 and gl3 init.
The count itself is worth a note, since an entry absent from the binary
is not a finding and a narrow build reporting none is not a clean tree.
Checked by injecting an unsanctioned reader into one of the five
functions: the pairs are per reader, so the audit still reports it.
The Device setting reached both init paths and neither used it, so the
list the menu offered could be browsed but not applied. Both now open
the device by name and fall back to the default with a warning when
the name has gone with the hardware it named, rather than failing init
on a stale setting.
Enumeration no longer assumes SDL_INIT_AUDIO is already up: the
frontend builds the list whether or not the driver came up, which is
how the user picks a different device after the current one failed.
The microphone driver gains the same list; it had none.
SDL 1.2 opens the device with its audio thread already running and
unpauses it by writing a plain flag, so the ring and the park built
between those two points reach that thread with nothing ordering them:
a period that observes the device unpaused may observe a ring that is
not yet built. Taking SDL's audio lock, which that thread holds around
every period, across the unpause is the edge. SDL2 locks and uses an
atomic for this itself.
The handhelds this driver serves are MIPS and ARM, where the stores
are free to arrive in the order the race allows.
samples/audio/sdl1_counters covers this and the driver's device
accounting: SANITIZER=thread reports the race when the lock is taken
back out. Run by hand, as the sdl_lost_wakeup suite is.
Both drivers run on an SDL callback that is the device asking for one
period, so counting what it asks for counts device time, and the short
read it zero-fills is a period of silence for want of audio. Bytes and
periods are counted on that path with one atomic add each and turned
into the frontend's frames_consumed() and underruns() off it.
frames_consumed() gives the frontend the device's real rate against
the host clock, so the resampling ratio can be trimmed by the
difference between two crystals; underruns() feeds the statistics
overlay and the one line the frontend logs at teardown.
sthread_raise_current_priority and sthread_set_affinity return false
both from the guard ahead of the call they make and from the #else arm
for a platform with neither. Inverting each guard lets the arm fall
through to a single return after #endif.
The 30 Hz lanes sat one period inside their own tolerance. A delivery
needs RING_FREE >= its own size, so at the 2560-frame floor a
1600-frame delivery waits until held <= 959; draining a period at a
time from the previous refill, the values the writer is released at
cycle through 448, 512 ... 896, and the first of those eight is below a
period. One refill in eight has to let the ring dip under a period
before it can be refilled, which is 24 of 600 periods - the 4% the
floor was chosen against, and exact rather than noisy. The bound was
5%, so ordinary jitter on top of a fixed 24 reached 29 and 30 against a
limit of 30. It is now twice the geometric figure, which still fails a
four-period floor and a worker that holds a period back as a reserve;
both were re-checked by injection.
The writer's usleep pacing goes with it. Rate-matched to the device by
construction - 1600 frames per 3125 us against 512 per 1000 us - and
usleep only ever oversleeps, so the ring level could only walk
downwards. It made no difference to the counts either way, because the
refill point is set by the ring geometry and not by the writer's rate,
and the writer already self-paces by blocking for room.
A six-period floor would not dip at all; the released values start at
960. That is a latency decision rather than this lane's to make.
Nothing in the tree includes it, and every type, extern and macro it
declares is absent everywhere else, including in psp1_gfx.c next to it.
Its "pspgu.h" include is unresolvable from gfx/drivers, so it cannot
compile where it sits.
Both pacing suites generate wasapi_sim.c from audio/drivers/wasapi.c
with sed. The committed copies had drifted from the driver, and since
make regenerates only when the driver is newer, a fresh clone could
build either text depending on checkout timestamps. Generating it
always is what the suites mean by the driver's text.
RA_IPHONEOS_DEPLOYMENT_TARGET and RA_TVOS_DEPLOYMENT_TARGET set the floor
for every target in RetroArch_iOS13.xcodeproj, so a runner on an Xcode that
has raised its minimum can build the store packages without a tree edit.
Unset, the targets keep the values they build with today.
Xcode 27 accepts macOS deployment targets in the 12.0 - 27.0.x range, so
the store package targets 12.0. RA_MACOSX_DEPLOYMENT_TARGET overrides it
for builds on machines carrying an older Xcode.
The Windows job builds this with cl /W3 /WX, where sprintf raises C4996
and C2220 turns it into an error, so that lane has never been green.
snprintf takes the buffer size and MSVC does not deprecate it; the
lane's other translation unit is already clean.
Two things the three drivers took on trust.
A worker that fails to start is not an initialisation failure: running
is set before the create, so the driver comes back reporting itself
alive with nothing consuming its ring. Writes then wait out their laps
and return zero, and free() skips the stop because it is nested under
the thread pointer. init now releases the port, the ring and the
eventcount and reports none, and start() puts running back and answers
false, which is the same hole on resume.
write() derived its frame count as uint16_t from a byte length. Above
65535 frames the count wrapped small, the room check waved it through
on the space the wrapped value asked for, and the non-wrapping memcpy
then copied the original byte length - 262144 bytes into a 12 KiB ring
under the harness. The frontend's output scratch is sized at
max_buffer_samples * AUDIO_MAX_RATIO * slowmotion_ratio and written in
one call, so the length is reachable from an extreme resampler ratio.
The count is a size_t, a length that is not whole frames carries the
frames it has, and anything past what the ring holds is taken short
rather than waiting out 800 ms for room that cannot come.
psp_ring grows a lane for each, over all three drivers: device_mock
supplies the drivers' sthread_create through a rename so one can be
refused, and the write lane walks 0, 1, 65535, 65536, 65537 and 131072
frames plus a six-byte write. Against the previous code the first lane
reports FAIL with alive=1 on all three and the second is an ASan
heap-buffer-overflow of 262144 bytes in psp_audio_write.
The contract says @at lies within the reservation, and neither function
checked it. On POSIX the map is MAP_FIXED, so an address outside the
area does not fail - it replaces whatever is mapped there. The offset
was also taken by subtracting base from an address not yet known to
belong to the area, the legacy slot check could overflow in both the
rounding and the addition, and the placeholder path confirmed only that
the first byte fell in a free range, so a length reaching past the end
recorded a range running backwards.
memshm_area_offset carries both arguments as uintptr_t and orders the
comparisons so neither the addition nor the subtraction can wrap; the
POSIX paths round the length up to a page first, because that is what
mmap replaces. unmap refuses an unbalanced call rather than wrapping the
mapping count.
On Windows the range table was also edited before the OS calls that
make the split, and a failure returned without putting it back, leaving
the software map and the VADs describing different things. Each step now
calls the OS first and records the split only once it succeeds, and a
table too full to record the result is refused before anything is
touched.
The test grows negative lanes for each of these: an address below the
base, the first address past the end, a length crossing it, a length
that would wrap, and an unbalanced unmap - plus a mapping made outside
the area, checked byte-wise to still be there afterwards. Against the
previous code the first two report FAIL and the third takes a bus error.
One file per SDL generation, each with its own vtable: audio_sdl1
("sdl") in sdl1_audio.c and audio_sdl2 ("sdl2") in sdl2_audio.c. The
SDL2 file keeps the microphone driver, which is SDL2-only. The SDL1
file drops the SDL2 device-API stubs it no longer needs and checks
SDL_OpenAudio()'s return properly - the shared file compared an
SDL_AudioDeviceID (unsigned) against < 0, so a failed open was taken
for a good one.
Makefile.common builds each object from its own SDL branch instead of
from HAVE_SDL_COMMON, griffin picks the file matching the SDL version,
and the samples/audio suites follow the renamed symbols.
A samples/audio suite that ships tsan.supp does so because its reports
come from an uninstrumented dependency; latency_contract's covers
OpenAL Soft, whose own threads and atomics TSan cannot see into. The
workflow passes the file, so the hand check reports a race the CI job
does not.
The SceUID member in the three console audio drivers was never
assigned or read. On Vita it was the only thing those drivers wanted
from psp2/kernel/processmgr.h, threadmgr.h and sysmem.h, and
psp2/audioout.h includes psp2/types.h itself, so those three go with
it. PSP1 keeps pspkernel.h: pspaudio.h has no includes of its own and
relies on it for the types in its prototypes.
The Vita compile-only stubs each declared their own SceUID, which
collided under -pedantic in any translation unit taking more than one
of them - features_cpu.c takes processmgr.h and, through
retro_timers.h, threadmgr.h. The typedef moves to a psp2/types.h stub
where the real headers keep it, and the two that need it include it.
The sysmem.h stub held nothing else and no lane includes it now.
The ring has to hold a delivery beside the period the worker is
playing, and a delivery is a video frame of audio: 800 frames for
60 Hz content at 48 kHz, but 1600 for 30 Hz. The four-period floor
fitted the first and not the second, so at settings below the
default a 30 Hz core starved - 14% of periods handed silence through
the threaded pipeline, 93% through the inline one, where the whole
delivery goes in a single write.
Five periods is the floor, which is the ring the default 64 ms
setting already asks for. So no setting at or above the default
moves, 60 Hz content is unchanged at either floor, and 30 Hz content
at the low end goes to 0.3% and 4%. Settings below 64 ms now report
64 ms rather than 53 ms, which is the whole of the visible change.
Content slower than 30 Hz is not covered: 20 Hz wants a sixth
period, which would push the default itself to 75 ms. Left alone
rather than moved for content that rare.
samples/audio/psp_ring drives the starvation check at both delivery
sizes; the floor at four fails the 30 Hz one on all three and leaves
60 Hz green.
sdl_audio_list_new returns NULL both from the guard on the list it
allocates and from the SDL1 arm. Wrapping the body in that guard lets
both paths fall through to a single return after #endif.
The worker handed the device silence unless two periods were held,
so at the low end of the latency range - where the ring is at its
four-period floor and the frontend delivers 800 frames at a time -
it spent a quarter of every period playing silence with audio
sitting in the ring. Measured against a device taking a period every
10.67 ms and a writer delivering a video frame of audio at 60 Hz:
27% of periods silent at the floor, against 0.3% when a single held
period is enough. At 64 ms and above the two are the same, so
nothing above the floor changes.
PSP1 needed the window it hands over to change with it. It was
giving the device the window after read_pos rather than the one
read_pos names, which is whole only when two periods are held; it
now takes the window read_pos names and publishes the index after
the call returns, as the other two already did. That also retires
the period it used to discard unplayed at startup.
samples/audio/psp_ring gains a starvation check at the lowest
setting, with the writer paced to the device rather than run flat
out. Putting the reserve back fails it on all three; leaving the
reserve out but giving PSP1 its old window back fails PSP1 alone on
a discontinuous window, which is the coupling between the two.
A failed check in the suite now releases the driver, so one failure
cannot leave a worker running against the next.
The GE's block transfer takes each side at 1023 at most and both
strides as a multiple of 8, no wider than 1024. The frame and menu
paths pass the core's width and pitch straight through, so a core
outside those limits gets a malformed transfer: vecx hands over
330-pixel rows and o2em 340-wide ones.
Round the destination stride, which is ours, up to a multiple of 8 and
carry it into sceGuTexImage; crop either side at 1023. A source stride
the transfer cannot express is copied a row at a time, where the
stride is not read, through a call list grown to the geometry - the
CPU writes 8 command words a row instead of repacking the frame
itself, which at 340x240 is 1920 uncached stores against 40800.
The harness holds all four limits on every blit and checks the frame
arrives in the texture for both stride paths.
The ring was a fixed 8192 frames on all three, which at 48 kHz is
170 ms of buffering whatever the user asked for, and half of that is
what rate control steers to. Size it from the latency setting at the
rate the device opened at, less the period the device itself holds,
so buffer_size() and the latency it implies are what was asked for.
The size stops being a power of two, so the wrap stops being a mask:
indices stay inside the ring and RING_HELD/RING_FREE compare instead.
The ring is a whole number of periods, which is what keeps every
window the worker hands over inside it, and the floor is four - the
worker wants two periods held before it hands one over, and the
writer needs room past that. At 48 kHz that floor is 53 ms reported,
so settings under it get the floor; above it the figure is exact.
The rate the device took is now kept, since the sizing needs it.
samples/audio/psp_ring gains a check that buffer_size() tracks the
setting across 8..256 ms rather than returning a constant: pinning
the ring back to 8192 fails it on all three. The ring's own checks
hold across the rewritten wrap - giving the reserved frame back still
fails all three with a discontinuous window.
Each audio_mixer_load_* function returns NULL both from the guard on its
allocation and from the #else arm for a codec that is not built. Wrapping
the body in that guard lets both paths fall through to a single return
after #endif.
The three drivers had no suite, so the two races fixed in ba5339c4
and d36404a1 had nothing holding them down.
samples/audio/psp_ring builds all three drivers for their own
platforms against the compile-only SDK stubs and supplies the bodies:
a device that reads the window it is handed and takes a period over
it. The writer's frames carry a running count, so a window whose
frames do not continue the last one means the writer reached data the
device had not finished with - a lapped ring, or a period published
free before the device took it - and the check holds without a
sanitizer.
Reintroducing either fix's absence fails it, and each fails its own
platforms: the room check that cannot tell a full ring from an empty
one fails all three, publishing read_pos before the output call fails
Vita and PS4 while PSP1 stays green, which is right - it hands over
the window after read_pos and frees one only on the iteration after
it played. Clean under ASan+UBSan+LSan and TSan.
tools/platform_stubs/orbis carries the sceAudioOut declarations, the
set the psp and vita stubs already had.
Not wired into Linux-samples-audio.yml; run it by hand, or through
tools/audio-suite-check.sh, which now finds it for all three drivers.
write_pos and read_pos are both masked into the ring, so the writer
having filled every frame and the writer having written nothing both
read as a distance of zero, and the free-space computation answered
with the whole ring. A writer that caught up exactly lapped: it wrote
from read_pos forward, over the window the device was reading, while
the worker - reading the same distance - took the full ring for an
empty one and played silence.
Hold a frame back, so the two states differ.
Found with a threaded model of the mainloop against the write path
using the tree's own retro_atomic ops, under TSan: the race reproduces
with the frame returned and does not with it held back, and it is
independent of the read_pos publication order - each fix is needed and
neither covers the other.
The worker advanced read_pos and released it before handing the same
window to sceAudioOutOutput, so the period the device was reading was
already the writer's to fill. A write landing in it while the call
was in flight put the writer's samples into audio being played.
The output call returns once the device has taken the window, which
is where the period becomes free: move the release store there, and
the notify with it. PSP1 does not have this - it outputs the window
after read_pos, which is still-unread data, and frees a window only
on the iteration after it played.
The 4x16 split makes each slice tall and narrow, which is the wrong
shape for an unswizzled texture's cache lines; what another split
costs can only be measured on the device. Take both counts from the
build when it sets them, and add a check-grids target that holds the
coordinate contracts for each candidate so only speed separates them.
Red sits in the low bits of every PSP display format, which is what
the CLUT passes psp_init() builds convert the core's frame into: the
pass indexed at red_shift writes the low channel, the pass indexed at
0 writes the high one. So read_viewport's 565 and 5551 arms take blue
from the top of the pixel, as its 4444 and 8888 arms already do.
Record the layout next to both, and in the harness, whose expected
bytes for the two 16-bit cases disagreed with its own 4444 and 8888
cases.
These three reported neither, so the sink rate estimate had nothing
to trim the resampling ratio with and the statistics overlay showed
no starvation for any of them.
Neither SDK offers an underrun signal or a queued-sample query the
drivers can use - the PSP SRC's output call returns the sample count
it was reserved with, and sceAudioOut has no rest-sample equivalent
on PS4 - but both are available from the worker's own cadence. The
output call returns once the device has taken the period before it,
so a completed call is a period consumed; the branch that hands the
device silence is the starved one. One atomic add each, where it
happens, read by the frontend.
Vita is the one that could do better: sceAudioOutGetRestSample()
reports a main port's unplayed samples, so frames_consumed() there
could be what was released less what the device still holds rather
than the cadence. Left for someone who can watch it against the
overlay on hardware, since that number steers rate control.
buffer_size() left out the period the output call holds while the
device plays it, which is buffering the driver controls; the
reported size and the latency it implies now agree.
sceKernelRegisterSubIntrHandler() invokes the handler as
void (*)(int, void *), so declaring it otherwise leaves it called
through an incompatible type. Take the driver from the void* argument,
and move the psp_set_texture_enable() declaration ahead of the
statement before it; the compile lane now rejects both.
Links gfx/drivers/psp1_gfx.c against a stand-in for the machine it
depends on: main RAM and VRAM each mapped twice so the cached and
uncached aliases alias, a no-access page after VRAM, display lists
that stay the GE's until waited on, and a vblank sub-interrupt that
can fire inside free().
Holds the read_viewport byte order and buffer fill, the teardown
order, GE ownership of lists and framebuffers, both blit bounds,
cache-line ownership of what the GE writes, the coordinate grid
across all four rotations, vsync pacing, and init under allocation
failure.
The message goes into the framebuffer through the CPU. With a
hardware-rendering core the driver deliberately leaves the GE alone,
so it has to be waited on here, where the two would otherwise be
writing the same buffer.
The screenshot task sizes its buffer from the viewport and encodes all
of it, so a viewport reaching past the framebuffer has to come back
black there rather than leave the encoder reading uninitialised heap
with every row shifted. Clear the buffer, place each row at the offset
the viewport gives it, and read the framebuffer through the uncached
alias the GE wrote it through.
video_driver.h specifies read_viewport as 24bpp BGR, which the 4444
and 8888 arms answer with; the 565 and 5551 arms emit the channels the
other way round, and those are the modes the PSP runs in, so every
screenshot comes out with red and blue exchanged.
psp->frame_coords is written through the uncached alias, so rebuilding
it is 256 stores straight to memory every frame. Keep the source
geometry they were built for and skip the rebuild while it holds.
VRAM holds the display buffers and the colour tables before the core
frame, and the menu texture buffer holds one screen of 4444; a core or
menu framebuffer larger than what is left runs sceGuCopyImage() off
the end of either. Record what VRAM is left at init and clamp the
copied height in both paths.
psp_init() dereferences the coordinate buffers straight after
allocating them, and hands back a driver whose display lists the
frontend will pass to sceGuStart(). Check every allocation and tear
the partial driver down through psp_free() instead.
psp_on_vblank() clears it from interrupt context while psp_frame()
reads and sets it, so the compiler has to reload it rather than keep
it in a register. On PSP this is what retro_atomic's own backend
reduces to.
sceGuFinish() hands the list to the GE and returns, so the buffer is
still the GE's when the menu pushes a texture between frames.
Synchronise before refilling it, and ignore a call with no driver or
no frame.
The GE context save area and the menu texture are written by the GE,
so a cache line either of them shares with data the CPU writes is lost
to the next writeback. Give the context storage its own 64-byte
aligned allocation instead of a slot in the driver struct, and raise
the menu texture's alignment to a cache line.
psp_on_vblank() writes through the driver pointer it was registered
with, so the sub-interrupt has to be disabled and released before
psp_free() releases anything it reaches. Free the driver struct itself
by that pointer, and drop the main_dList guard so a driver whose first
allocation failed is still torn down.
gfx/drivers/psp1_gfx.c is reachable only through the griffin unity
build, so nothing in CI compiles it. Give it a platform_video lane,
and extend the PSP stub header set with the kernel, display and GU
shapes it names.
Three functions carry a #else arm whose only job is to return the same
constant the enabled arms already fall back to. Inverting the guard that
precedes each arm's final return lets the arm fall through, so the
constant is returned in one place after #endif.
Sixteen #if/#endif pairs across twelve files enclose nothing, so they
select no code in any configuration. Removing them leaves preprocessor
output identical.
The ring and the period of silence the worker outputs when the ring
runs short were two allocations of the same lifetime, neither ever
resized and the silence never written after init. Put them in one
uint32_t arena, the silence past the ring's end, so init has one
allocation to check and free one pointer to release.
The window handed to the output syscall stays inside the ring:
read_pos starts at zero and only ever advances by AUDIO_OUT_COUNT,
which divides AUDIO_BUFFER_SIZE, so it is always a multiple of the
period and the last window ends exactly at the ring's end. The
producer's wrap splits at that same boundary.
Each of these backends takes a fixed set of sample rates, and a
setting outside it was passed straight to the hardware: the port
open failed, init returned NULL and the session ran with no audio
at all. Report the rate actually opened through new_rate instead, as
gx_audio does, so the frontend resamples to it.
- PSP1: the SRC's rate list is fixed, so fall back to 48 kHz when
the requested rate is refused.
- Vita: the main port opens at 48 kHz only.
- PS4: sceAudioOut opens at 48 kHz only.
The ring and silence buffers were allocated without checking the
result, and neither allocation failure nor a failed eventcount init
released the port that had already been opened. Allocate with calloc,
check both, and release the port on every failure path.
PS4: sceAudioOutInit() ran after the port it initialises was already
open; it now comes first. stop() returned false unconditionally, so
CMD_EVENT_AUDIO_STOP always failed on Orbis - pause, sleep/wake and
the shutdown path with it. It stops the worker and reports the
resulting state, as the vtable's contract asks and the other two do.
The PSP1, Vita and PS4 backends share no SDK: each opens its port
with a different call, outputs through a different syscall and
releases differently. Give each one its own file and vtable.
audio/drivers/psp_audio.c - PSP1, audio_psp, "psp"
audio/drivers/psp2_audio.c - Vita, audio_psp2, "vita"
audio/drivers/ps4_audio.c - Orbis, audio_ps4, "orbis"
Driver idents are unchanged, so existing configs keep working.
Makefile.common, Makefile.orbis and griffin select the file for the
platform being built; the matrix's Vita lane now compiles
psp2_audio.c.
ps4_audio.c drops the unused verbosity.h include, which on ORBIS
pulls in debugnet.h, and declares malloc through stdlib.h.
tools/vulkan_teardown_order_check.py fails when vulkan_free() or vulkan_check_swapchain() retires the command pools, the descriptor pools and the pipelines in any order but innermost first, when a descriptor set is freed by hand rather than by the pool destroy that owns it, and when the pool asks for per-set free that nothing needs. The order is legal Vulkan and the validation layer passes it, so the lavapipe harness beside it cannot see this class; a driver that walks its own back references as a pool is destroyed faults at exit instead. Runs as its own job in the Vulkan workflow, self-test first, like the idle-wait check.
The sets in a block live and die with the pool, and VK_DESCRIPTOR_POOL_CREATE_FREE_DESCRIPTOR_SET_BIT is required only by vkFreeDescriptorSets, which nothing calls. Drivers that provide a per-set allocator for the flag no longer hand one out. The slang filter chain's pools are created this way already.
A command buffer references the descriptor sets and the pipelines it was recorded with, and a descriptor set references the layout it was allocated from, so teardown runs innermost first: command pools, then descriptor pools, then the pipelines that own the set layout. Destroying a descriptor pool releases every set allocated from it, so that one call frees them, and the per-frame command buffer and pool handles are cleared as they go, like the descriptor managers and buffer chains beside them.
A driver that keeps its own references along that chain walks them as each pool is destroyed - MoltenVK does, and the fault lands inside vkDestroyDescriptorPool on the video thread as RetroArch exits.
With a single staging buffer, the busy flag set by frame N's blit is
only cleared when the GPU finishes that blit's command buffer. With up
to MAX_INFLIGHT frames queued ahead of it, that has usually not
happened by the time frame N+1's update arrives, so a texture updated
every frame dropped most of its frames and streamed at a fraction of
its rate.
The staging buffer now holds MAX_INFLIGHT frame-sized slots, each with
its own busy flag, used round-robin - the same shape as Vulkan's
VK_STREAM_SLOTS. A frame is dropped only when the slot it would use
is still being read, which with one slot per frame in flight is rare.
The flags are plain ints behind __atomic stores and loads: each is set
only by the updating thread and cleared only by the completed handler,
so no read-modify-write is shared between them.
metal_update_texture now encodes a blit into Context.blitCommandBuffer,
but it still ran inline on the caller's thread whatever threaded said.
Under threaded video that is the main thread, while the video thread
encodes into and commits the same command buffer at frame end.
MTLCommandBuffer is single-thread-only, and a completed handler added
after the video thread has committed the buffer is an exception, not
just a race.
metal_load_texture already routes mipmapped loads to the video thread
for exactly this reason, and video_driver.h says update_texture is
marshalled onto the video thread; Vulkan, D3D11 and D3D12 all do it.
Metal now does the same through video_thread_texture_handle, with the
result coming back through cmd.handle as in metal_texture_load_wrap.
The blit reads the staging buffer on the GPU and the next frame's
memcpy writes it from the CPU: hazard tracking orders one GPU access
against another and says nothing about that, so the race moved from
the texture to the buffer rather than going away. The texture carries
a flag now, set when a blit that reads the buffer is committed and
cleared from that command buffer's completion handler, and a frame
arriving while it is set is dropped - the same policy Vulkan, D3D11
and D3D12 keep, with the buffer rather than the texture as the thing
being protected. The flag is atomic because Metal finishes on
whatever thread it likes, and the handler holds the texture so the
flag is still there to clear.
metal_update_texture wrote the new contents with replaceRegion:, which
lands from the CPU the moment it is called - while the command buffers
already committed may still be sampling that texture. The comment
above it justified this by saying a command buffer retains what it
samples, which is true of the object's lifetime and says nothing about
its contents. Every other backend refuses exactly this: Vulkan checks
its fences, D3D11 maps with DO_NOT_WAIT, D3D12 compares a fence tag,
and each drops the frame rather than race.
The pixels go through a staging buffer per texture and a blit on the
shared blit command buffer instead. It executes in commit order with
the draws around it and Metal's hazard tracking orders the write
against the reads, so a streaming frame is neither raced nor dropped -
better than the drop the others settle for. The buffer is made on the
first update and kept, since a texture that streams updates every
frame, and it is released with the texture.
The concurrency group was keyed on the ref with cancel-in-progress
set, so every push to master that touched one of the watched paths
cancelled the run still going for the previous commit ("Canceling
since a higher priority waiting request ... refs/heads/master
exists"). With several commits pushed close together, most of them
never got the test run to completion.
Key the group on the commit SHA for pushes and only cancel in
progress runs for pull requests, where a new push really does
supersede the old one.
dispserv_videocore gets get_edid, reading the display's EDID over
HDMI DDC with vc_tv_hdmi_ddc_read() the way tvservice -d does: the
base block, then the extension blocks it announces, as far as the
buffer goes. The edid CRT preset (mode 5) takes its ranges from it
instead of warning and falling back to generic_15, and the EDID
entry under System Information shows it. Composite and DPI outputs
have no DDC; the read fails there and the fallback stays. Behind an
HDMI-to-VGA adapter the block is whatever the adapter reports, which
may be its own rather than the monitor's.
bcm_host_init() already connects the gencmd and tvservice clients.
The server now calls it at init (it returns at once when the video
driver made the call first) and sends hdmi_timings through that
client. The per-call connect and vc_gencmd_stop() it replaces, carried
over from crt_rpi_switch, stopped the gencmd client bcm_host shares.
The legacy Raspberry Pi firmware stack had its own CRT switching path
inside video_crt_switch.c: a hardcoded timing heuristic
(crt_rpi_switch) that bypassed the modeline engine, with every engine
call site fenced off by #if !defined(HAVE_VIDEOCORE). Keeping those
fences in step with the call sites is how 94d23651 broke the link on
Lakka's Pi Zero builds (fixed in 4e057a5).
VideoCore now has a display server like KMS and X11 do.
dispserv_videocore implements the modeline ops: the engine generates
the mode, and set converts it to hdmi_timings (porches and pulse
widths, inverted-sync flags, frame rate for interlaced modes), sends
it through gencmd, selects it with tvservice -e "DMT 87", resizes the
framebuffer with fbset and reinits the video driver, the same sequence
crt_rpi_switch ran. The vc_egl context and the dispmanx video driver
report RARCH_DISPLAY_VIDEOCORE so the server is picked up.
With that, video_crt_switch.c has no HAVE_VIDEOCORE left:
crt_rpi_switch and the crt_switch_write_edid stub are gone, VideoCore
builds get switchres.ini overrides and the CRT menu entry, and the
guard on crt_switch_display_server_lost() in video_driver.c goes back
to HAVE_MODELINE alone.
Behaviour differences on VideoCore:
- Timings come from the engine and the monitor preset instead of the
old heuristic.
- Without a super width set by the user, the policy locks a 1920
super resolution, as the old path always widened the mode near 1920
to keep the HDMI pixel clock up. It scales fractionally where the
old path picked an integer multiple of the core width.
- Doublescan is disabled; hdmi_timings has no way to express it.
- The last mode still stays on the wire at exit; the firmware has no
desktop mode to restore.
crt_switch_display_server_lost() (video_crt_switch.c) is defined
inside #if !defined(HAVE_VIDEOCORE) - VideoCore keeps its own
tvservice path and doesn't use the modeline ops table. The call
added in 94d23651 (\"crt: rebind the modeline ops when the display
server is rebuilt under a live engine\") only checked HAVE_MODELINE,
so any build with both HAVE_MODELINE and HAVE_VIDEOCORE (e.g. Lakka
RPiZero/RPiZero2 with the bcm2835-driver GLES stack) fails to link:
video_driver.c:(.text+0x20e8): undefined reference to
\`crt_switch_display_server_lost'
Add the same !defined(HAVE_VIDEOCORE) guard at the call site.
The last patch folded version 3 into version 2 and renamed the test's
modes with it, which broke the run that has to fail:
== a core that ignores what lock_context tells it (must fail)
d3d11_hw_interface_v2: ok
FAIL: it passed; the frontend in the test no longer disturbs the
context
The frame bound everything under "if (rebind || v3)", because in version
3 the render target changes with the sync index and must be set every
frame whatever lock_context said. With version 3 now being version 2,
that condition is always true: the core rebinds even when it is
pretending not to, so the control could not fail.
The render target is now bound every frame on its own, outside the
rebind block where it belongs - it is the sync index's, not the
context's - and everything else is bound only when lock_context says the
frontend has had the context. A core that ignores lock_context draws
into the right texture and draws it wrong, which is the point.
run.sh went with it: the version 3 run is the version 2 run now, so it
is gone, and v3-nowait is v2-nowait. Three runs, one that must pass and
two that must fail.
Ran here under Wine on llvmpipe, the same way CI does: the contract run
passes, both controls fail.
cf3c678de5 (#18405) rewrote every AKEYCODE_DPAD_CENTER press to
AKEYCODE_ENTER before storing it in android_key_state[], regardless
of which row it was written to. That fixed Enter on external
keyboards (rarch_key_map_android can only map RETROK_RETURN to one
keycode), but pad rows are read by raw keycode against joypad binds,
so any device whose OK/Center button is autoconfigured as "23" - the
Amazon Fire TV Remote, and other remotes/pads - could never trigger
that bind again: the press always landed in bit 66.
Restrict the alias to ANDROID_KEYBOARD_PORT, the only row read
through rarch_keysym_lut. Scoping on the row rather than on
AINPUT_SOURCE_DPAD matters: the hotplug code classifies
KEYBOARD|DPAD (0x301) devices as keyboards too, and ordinary
keyboards with arrow keys report exactly that, so a source check
would reintroduce #18395 for them.
Fixes#19593
The D3D11 hardware render interface had two negotiated versions above 1:
version 2, where the frontend is finished with the frame's texture by
the time video_refresh returns, and version 3, which adds a sync index
so the texture is handed over without a copy. Version 2 was never used
by anything - no core shipped against it - and keeping it meant the
frontend carried a copy path for a case that does not arise.
So version 3 is version 2 now, and what version 2 was is deleted. There
is one negotiated version above 1 and it is the one with the sync index,
which is also how libretro_d3d12.h version 2 and the Vulkan interface
work. RETRO_HW_RENDER_INTERFACE_D3D11_VERSION_3 is gone; a core that
named 3 as its maximum gets 2, and behaves exactly as before.
What went with it:
- The copy branch in d3d11_gfx_hw_ring_capture, taken when the ring
passed D3D11_HW_RING_CAPTURE_TEXTURE, and the macro itself.
- The threaded wrapper's d3d11_v3 flag and the two paths behind it.
hw_ring_install is now required for version 2 at all, since handing
the texture over without a copy is what version 2 is; a frontend
without it gives the core version 1.
- The sample test's separate version 2 mode; "v3" and "v3-nowait"
become "v2" and "v2-nowait".
Not built here - this is Windows-only code and needs a Windows build
before it is trusted.
A 10-bit swapchain with HDR off renders nothing. The game's audio and
input run, the log is clean, and the screen is black.
d3d12_gfx_init_pipelines hardcoded RTVFormats[0] to R8G8B8A8_UNORM for
every pipeline that draws to the swapchain. That is right whenever the
swapchain is 8-bit, and it is not always 8-bit: with HDR disabled and
video_swapchain_bit_depth set to 10, chain.bit_depth becomes
DXGI_SWAPCHAIN_BIT_DEPTH_10 and the back buffer is R10G10B10A2_UNORM. A
draw whose pipeline disagrees with its render target is invalid, so
every frame is thrown away.
It only bites with HDR off, which is why it looks backwards: with HDR on
the frame is drawn with the HDR pipes, and those have always been built
from the real swapchain format and recreated when it changes. The SDR
ones never were. So they are now built the same way, from
chain.formats[chain.bit_depth], which is set well before the pipelines
are created.
The debug layer says it plainly once it is asked:
ID3D12CommandList::DrawInstanced: The render target format in slot 0
does not match that specified by the current pipeline state.
(pipeline state = R8G8B8A8_UNORM, render target format =
R10G10B10A2_UNORM)
repeated for every draw of every frame.
Not built here; this needs a Windows build before it is trusted.
Two fixes to the press-driven frame swap:
- A swap that could not happen - the surface still in flight on the
video thread, or the submit failing - was recorded as done anyway.
Under threaded video a quick tap submits the pressed frame, the
release lands while it is still in flight, and the button then stays
pressed-looking until the next tap. The swap now reports whether it
happened and the current frame only changes when it did, so the
next poll retries.
- Every press and release rewound the stream and inflated one or two
full frames on the main thread (the second frame is composed over
the first, so it cannot be decoded alone). Both frames are now
composed once at load into a back-to-back pair and a swap is a copy.
An APNG whose pair cannot be composed, or whose canvas does not
match its decoded image, stays an ordinary looping animation.
Also reset the current frame when the surface is re-created, since the
new texture starts on the unpressed frame.
desc->image_index is the desc's position in its page's image list:
every desc with an image gets its own entry there, even when several
share one file. The anim_2frame arrays are sized num_images, one entry
per unique image of the pack, deduplicated by path. Indexing them with
image_index read (and, when the stray byte was nonzero, wrote) past
their end on any pack whose page has more image entries than the pack
has unique files - e.g. a dozen buttons sharing one png - and on
smaller packs toggled the wrong image, since page index N is not
unique image N and the page background shifts everything by one.
Record the unique index at load time, where the loader already has it
from its dedup lookup, as desc->pack_image_index, and use that. Also
skip image-less descs, whose calloc'd index of 0 would otherwise press
whichever file is unique image 0, and keep the block under HAVE_RPNG
with the rest of the animation code.
video_thread_get_poke_interface() had the driver write thr->poke
directly, on the main thread, while the video thread was already
running and reading that field - the asynchronous upload list takes it
under thr->lock every time it drains, so the read was guarded and the
write was not. A reinit while uploads are in flight is where the two
meet, which is the shape the font-marshal lane drives and where TSan
caught it.
The driver fills a local now and the lock publishes it, so both sides
of the field are under the same lock. Nothing else writes it: the
driver_data assignments beside it happen in the command handler, on
the thread that reads them.
[Video] Threaded video: D3D11 hardware render interface version 3, ...
[Video] Threaded video: D3D11 hardware render interface version 3, ...
A core may ask for the hardware render interface more than once - when
it first needs the context, and again when it builds its device - and
the ring logged its answer every time. The answer is the same each
time, so it is logged once per ring, and again only if it ever differs.
A new ring, after a reinit, says it again, which is when it is worth
reading. Same for the D3D12 line.
Passes tools/mingw_syntax_check.sh.
As d804bcb did for D3D11. The driver copies every hardware frame into
frame.texture[0] and draws from that, so that shader passes, history and
feedback read one texture the driver owns; with no shader preset active
nothing needs it, and it is a full-frame copy every frame.
For a core on interface version 2 it goes. Such a core keeps a texture
per sync index and leaves one alone until the frontend is done with it,
so the texture stays the frame it was handed over as for as long as the
driver draws from it, repeats and the menu's redraws included.
What D3D12 needs that D3D11 did not:
- a descriptor for the core's resource in the driver's own heap. A
core rotates a few textures, so one is made per resource and found
again; four are kept, with their resources, until another wants the
place. That only happens when a core's textures change - a change of
resolution - and work recorded with the old descriptor may still be
on the queue, so the queue is drained before it is written over;
- the resource's state. The core leaves its texture in required_state,
COPY_SOURCE, and expects to find it there. It is made
PIXEL_SHADER_RESOURCE for the one draw that reads it and put back,
in the same command list, on every frame that draws from it.
That draw is now what reads the core's texture, so it is what arms the
fence wait_sync_index waits for, where the copy did. SDR and HDR draw
the frame through the same one binding, so both get this. A shader
preset that is active, a frame from interface version 1 or a software
core, or a descriptor that cannot be had, all take the copy as before;
only a core that negotiated version 2 reaches any of this.
With the core handing its merge target over without a copy (lrps2
000ca3eb5), a D3D12 frame now goes from the core to the screen with
none at all.
Passes tools/mingw_syntax_check.sh (win32 gnu99 and the C89 lane). Not
run.
The driver copies every hardware frame into frame.texture[0] and draws
from that, so that shader passes, history and feedback all read one
texture the driver owns. With no shader preset active nothing needs it:
it is a full-frame copy, every frame, to draw a picture that is already
in a texture.
For a core on interface version 3 it goes. Version 3 is what makes it
safe: such a core keeps a texture per sync index and leaves one alone
until the frontend is done with it, so the texture stays the frame it
was handed over as for as long as the driver draws from it - repeats
and the menu's redraws included. The driver makes a shader resource view
of the core's texture (a core rotates a few, so a view per texture is
made once and found again; four are kept, with their textures, until
another wants the place) and binds it where it bound frame.texture[0]'s.
SDR and HDR draw the frame through that same one binding, so both get
it. With version 3 already handing the texture over without a copy, a
frame now goes from the core to the screen with none at all.
Everything else is as it was: a shader preset that is active, a frame
from any other interface version or from a software core, or a view that
cannot be made, all take the copy. Only a core that negotiated version 3
reaches any of this.
Passes tools/mingw_syntax_check.sh (win32 gnu99 and the C89 lane). Not
run.
The driver's Metal paths have only ever been compiled: no job runs
them and no hardware here can. Two assumptions carry them, and this
asserts both on the macOS runner's own device.
A streaming surface replaces a texture's contents while the GPU may
still be sampling it. Vulkan checks a fence, D3D11 maps with
DO_NOT_WAIT, D3D12 compares a fence tag, and each drops the frame
instead of racing; Metal's replaceRegion: has nothing of the kind, so
the path rests on Apple's rule that a managed or shared texture may be
written from the CPU. The lane writes sixty-four frames with work in
flight against each and reads the texture back.
An overlay page holds textures the pack owns. The page keeps them in
an array, a switch replaces that array, and dropping a page must not
take the pack's textures with it - the case that broke gl2 on Android.
The lane builds two pages over one set, switches, drops, and uses the
textures afterwards.
The sample does not link metal.m, which would drag in the frontend; it
reproduces the two patterns. A runner without a Metal device reports
77 and the step says so rather than passing quietly.
Version 2 says the frontend is finished with the frame's texture when
video_refresh returns. A frontend that presents from another thread can
only honour that by copying the texture there, which RetroArch's
threaded ring does: a full-frame copy every frame, to move a picture the
core already has in a texture into one the ring owns.
Version 3 is version 2 with the sync index the Vulkan interface and
libretro_d3d12.h version 2 have: get_sync_index, get_sync_index_mask and
wait_sync_index. The core keeps one present texture per index; the
frontend reads the texture itself, when its own thread gets to it; and
wait_sync_index returns once the frontend is done with what the index
carried last time. No fence, unlike D3D12: the device has one context,
and what the frontend has issued on it is ahead of whatever the core
issues next, so "the frontend's frame call has returned" is all that
done has to mean. wait_sync_index is not called with the lock held, for
the reason video_refresh is not. It is a new version number and not new
members on version 2 because version 2 has shipped: a core built against
a longer struct must be able to tell a frontend that has it from one
that does not.
The driver, with no wrapper in front, reads the texture inside
video_refresh: one index, nothing to wait for.
The threaded ring holds a reference to the core's texture in the slot
where version 2 made a copy, and the video thread hands it to the driver
through hw_ring_install, which D3D11 now implements; the driver takes
its frame's reference under the lock, as it does for version 2's copies.
The wait that covers a frame queued but not yet claimed, or claimed and
still being recorded - written for D3D12 version 2, where a slot is the
core's own texture for the same reason - is lifted out of the D3D12
block and used by both.
samples/gfx/d3d11_hw_interface_v2 gains two runs: version 3 with a
frontend slower than the core, which laps it and has to wait, and a core
that skips wait_sync_index, which is caught ("index 0 should hold frame
0, holds frame 18"). All four runs behave; run.sh requires the two
controls to fail. d3d11.c and video_thread_hw.c pass
tools/mingw_syntax_check.sh (win32 gnu99 and the C89 lane). The driver
and ring code itself is not run: it needs the core's half.
Version 2 as first written cost a D3D11 core frame rate under threaded
video instead of giving it anything: about 50 fps from a core that runs
at 60, the wrapper's drains piling up, 20 ms and more to vblank. Two
things in the contract and the lock, both fixed here; the core's half of
the first follows in its own tree.
video_refresh was to be called with the lock held. video_refresh is
where the threaded wrapper paces the core: it waits, up to a refresh,
for the video thread to take the frame before. The video thread cannot
finish the frame it is on without the lock. So every paced wait ran to
its timeout with the core holding the one thing that would have ended
it. The contract now says what it should have: set_texture with the
lock held, release it, then video_refresh, and never hold it across
video_refresh. The frontend takes the lock itself for the copy it makes
there - it always did, the lock being recursive - and the core cannot
draw into the texture meanwhile because it is inside video_refresh. The
paragraph about carrying on after video_refresh within the same lock
goes, since there is no such thing any more.
A Windows critical section is not fair. A thread that releases it and
asks again at once usually gets it back, over one that has been waiting;
a core that locks around each piece of GS work, thousands of times a
frame, keeps the frontend's thread out for the whole frame that way. The
frontend now says when it is waiting (an interlocked count around its
EnterCriticalSection), and the core's lock_context lets it go first:
while the count is up it yields the processor instead of entering. A
nested lock_context does not do that - the core would be waiting for a
frontend it is itself blocking - which is what core_depth is for. With
no wrapper in front the frontend is the core's own thread and never
waits.
samples/gfx/d3d11_hw_interface_v2 follows the contract as changed and
still passes, control included. d3d11.c passes
tools/mingw_syntax_check.sh (win32 gnu99 and the C89 lane). Not run on
the driver itself.
A D3D11 core on interface version 2 under threaded video, 120 Hz display,
60 fps core: stuck near 48 fps, 8.33 ms reported for the driver's render,
20 ms and more to vblank.
cb758f6 stopped the frame holding the lock the core and the driver share
the context with through the frame's waits. It left the other entry
point that takes that lock as it was: d3d11_present_last, which puts the
retained frame on screen again when the core has nothing new by the
deadline. With a core slower than the display that is not a rare path -
at 120 Hz under a 60 fps core it runs every other refresh - and it waits
for the display exactly as the frame does: the waitable swapchain's
latency wait, then a Present with the swap interval. It held the lock
through both, so for one refresh in two the core could not touch the
context at all, and a core that needs about five milliseconds of it per
frame came out at two frames in five refreshes.
Both loops in it - the light frames and black frame insertion's dark ones
- now give the lock up around the latency wait, and around Present where
the runtime's multithread protection is on, with the helpers the frame
uses. Neither wait depends on pipeline state, and the copy or clear
between them is issued with the lock held as before.
Passes tools/mingw_syntax_check.sh (win32 gnu99 and the C89 lane). Not
run.
A core driving a hardware context calls the frame callback from whichever
thread it renders on, and the frame updates the window title. SetWindowText
sends WM_SETTEXT, and a send does not return until the window's own thread
takes it off its queue -- but that thread is inside retro_run, waiting for
the core, and is not pumping messages. Each waits for the other. A PS2 core
presenting from its EE thread hangs at boot this way, with the frontend
thread parked in the core's own wait and the EE blocked in SendMessageW.
The same message with a timeout: the title is cosmetic and a frame that
cannot set it loses nothing, where a frame that blocks on it loses the
session. ABORTIFHUNG returns at once when the target is already known to
be stuck rather than waiting the deadline out. Both flags and
SendMessageTimeout predate NT 4, so the oldest target still builds; checked
at _WIN32_WINNT 0x0400, 0x0501 and 0x0600.
The null driver's present_last() read a setting the main thread was
writing, and TSan found it rather than anyone noticing by reading. Two
more were doing the same from frame(): xvideo and sdl2 took the OSD
message position out of the live settings while the video thread was
drawing with it. xvideo takes it from the video_frame_info_t its
caller already holds; sdl2 has two callers, frame() and the OSD poke,
so the position arrives as a pair of floats - the snapshot's from the
frame, the setting's from the poke, where the wrapper holds the main
thread inside the command and nothing can be writing.
tools/video_thread_state_check.py looks for the rest of the class
instead of waiting for the next one: it seeds from frame() and
present_last(), walks what they call, and fails on config_get_ptr(),
video_state_get_ptr() or runloop_state_get_ptr() anywhere it reaches.
It cannot follow a call through a function pointer or into another
file, so a clean run means no path it can follow, not no path at all.
The drivers for platforms with no video thread are listed with that
reason - a read that is latent today and a finding the day the
platform runs threaded - and an entry whose driver has stopped
reading fails too, so the list shrinks. It carries a self-test for
both directions and runs beside the gfx samples.
A D3D11 core on interface version 2 under threaded video ran at 48 fps
where it runs at 60 otherwise, with 16 ms to vblank and 34 ms worst.
Under version 2 the driver's frame takes the lock the core and the
driver share the immediate context with, and it took it for the whole of
the frame. The frame blocks in three places: the waitable swapchain's
latency wait at the top, the wait for vertical blank, and Present. The
first two do not touch the context at all, and holding the lock through
them kept the core off the context for most of a refresh, every refresh.
The lock is given up around the two waits and taken again after, which
marks the context used by the frontend as any other turn does. What the
core does to the pipeline state meanwhile cannot matter at either point:
the latency wait comes before the frame has bound anything, and the wait
for vertical blank after it has issued its last draw and flushed.
Present does touch the context. With the runtime's own multithread
protection on - ID3D10Multithread, which a D3D11 immediate context has
answered for since Windows 7 - the runtime serialises Present against
the core's calls itself, for as long as it needs the context, so the
lock is given up around a waiting Present as well. Protection is turned
on when version 2 is handed out; where that is not possible, Present
stays under the lock as before.
With no wrapper in front the frame runs on the core's thread inside the
core's own lock; giving up one level of a recursive lock there changes
nothing, as it should. Version 1 cores never reach any of this.
Passes tools/mingw_syntax_check.sh (win32 gnu99 and the C89 lane). Not
run.
The contract went in with 357a049. This is the frontend's half of it.
Nothing changes for any core until one asks: version 2 is handed out
only to a core that negotiated it, and every existing core still gets
version 1 - under threaded video, the deferred context behind the proxy,
exactly as before.
Negotiation. GET_HW_RENDER_CONTEXT_NEGOTIATION_INTERFACE_SUPPORT answers
version 1 for the D3D11 type, and
d3d11_hw_interface_negotiated_version() (gfx/common/d3d11_hw_interface.h)
is where the driver and the ring both learn what the core asked for.
The lock. A critical section in the driver, recursive, which is what the
contract needs: with no wrapper in front the driver's frame runs inside
the video_refresh the core called with the lock held, and the frame
calls itself for black frame insertion. Only two of the driver's entry
points can run while the core is running - the frame, and the repeat of
the last one - so those two take it; everything else reaches the video
thread as a command the main thread waits for, and the main thread is
not inside the core while it waits. Taking it marks the context as used
by the frontend, which is what lock_context reports to the core and
clears.
The frame. With no wrapper, set_texture's texture is what the frame
reads, in place of whatever is bound at pixel shader slot 0. Under the
threaded ring the core gets the immediate context itself - no deferred
context, no proxy, no command list to replay - and at video_refresh the
driver copies the named texture into the slot's own: one call on the
context the core's thread holds the lock for, ahead of anything the core
does next. The slot's texture is replaced (a change of frame size) and
the video thread takes its reference to it under that same lock.
The header gains one paragraph: video_refresh is the frontend's use of
the context too, so a core that carries on after it within the same lock
treats the state as disturbed.
d3d11.c, video_thread_hw.c and runloop.c pass tools/mingw_syntax_check.sh
(win32 gnu99 and the C89 lane). Not run: it needs a core that asks for
version 2, which is the next change, and a Windows GPU.
samples/gfx/d3d11_hw_interface_v2 still runs the contract itself.
A D3D11 device has one immediate context, and it cannot be used from two
threads at once. Version 1 hands it to the core and takes the frame from
whatever is bound at pixel shader slot 0, which works only while the
frontend draws on the core's thread. A frontend that presents from
another thread cannot share it under those rules; all it can do is give
the core a deferred context, and a deferred context records but cannot
read back - no Map for reading, no query results. A core that reads its
render targets back loses that silently, and has to be written around a
context whose state and object lifetimes do not behave like the one it
was written for.
Version 2 shares the immediate context by taking turns:
lock_context / unlock_context
whoever holds the lock has the context to itself and everything
D3D11 can do with it, reading back included. lock_context returns
true when the frontend has used the context since the core last
released it, and the core then binds again whatever it relies on;
false means the context is exactly as it left it. Recursive. A
core may lock and unlock as often as it likes within a frame.
set_texture(texture)
the frame, named outright. The frontend is finished with it by
the time video_refresh returns - one context orders everything -
so there is no sync index, and the slot 0 convention is not needed.
context is always the immediate one in version 2. The struct is version
1's with members added at the end; RETRO_HW_RENDER_INTERFACE_D3D11_VERSION
stays 1 because existing cores reject anything else, and a core gets
version 2 only by asking through the negotiation interface libretro
already has (RETRO_HW_RENDER_CONTEXT_NEGOTIATION_INTERFACE_D3D11 is new
in libretro.h), exactly as for D3D12. How the lock is made is the
frontend's business: a lock of its own is enough, since both sides take
it for every use, so nothing here needs ID3D11Multithread or Windows 10.
This is the contract and a proof that it is sufficient; no driver
implements it yet and no core asks for it, so nothing changes for anyone
until they do.
samples/gfx/d3d11_hw_interface_v2 runs it on a real D3D11 device: a core
that draws through cached state, reads its own render target back every
frame and hands the texture over, against a frontend thread that takes
the lock and draws with its own target, shaders and viewport before
checking the frame it was given. 200 frames drawn, read back and seen.
A core that ignores lock_context's answer is the control and is caught
at once ("the core drew frame 1 and read back 0"); run.sh requires that.
Built with mingw-w64, run under Wine, added to the gfx samples lane,
which fails rather than skips if it finds no D3D11 device. The header
compiles as C89 and as C++.
ThreadSanitizer, threaded_video harness, present-repeat lane:
data race gfx/video_driver.c:451 in video_null_present_last
read by 'ra-video' video_null_present_last <- video_thread_loop
write by main thread lane_present_repeat (threaded_video_test.c:373)
The null driver's present_last asked config_get_ptr() for
video_black_frame_insertion. Under the threaded wrapper that is the
video thread reading a setting the main thread writes, which is the
read the wrapper exists to keep off that thread; the lane writes the
setting to test black frame insertion and the two met.
The count now comes from the frame: video_null_frame keeps
video_info->black_frame_insertion, the snapshot the main thread took
for that frame, and present_last returns it plus one. frame and
present_last run on the same thread with or without the wrapper, so
there is nothing left to race, and it is how the real drivers learn the
value. The lane still sees what it checks - it runs frames after
changing the setting, so the repeat that follows carries the new count.
Passes tools/mingw_syntax_check.sh. The harness was not rebuilt here
(a full sanitizer build of the frontend); the CI lane that reported
this is the check.
The contract went in with 75f55ed. This is the frontend's half of it.
Nothing changes for any core until one asks: version 2 is handed out
only to a core that negotiated it, and every existing core still gets
version 1 and the copy that goes with it.
Negotiation. GET_HW_RENDER_CONTEXT_NEGOTIATION_INTERFACE_SUPPORT answers
version 1 for the D3D12 type. d3d12_hw_interface_negotiated_version()
(gfx/common/d3d12_hw_interface.h) reads what the core then set and gives
min(what it asked for, 2), or 1; the driver and the ring both ask there,
so they cannot disagree.
The driver, with no wrapper in front. Its frame runs inside
video_refresh and reads the core's texture there, so there is one sync
index. set_texture_fenced queues a Wait on the core's fence ahead of the
frame's command list - a wait on the GPU's timeline, none on the CPU -
and a fence signalled behind that list is what wait_sync_index waits.
The threaded ring. A version 2 slot is the core's own texture: publish
records and submits nothing on the core's thread, which is where version
1's cost was (a command list, two barriers, a copy, a submit and a
signal, every frame). The video thread gives the driver the texture and
the core's fence through hw_ring_install, which D3D12 now implements;
the driver queues the Wait ahead of its frame and reads the texture
where it used to read the slot's copy. The slot holds a reference to the
texture and the fence until it is handed over again. A core that calls
version 1's set_texture on a version 2 interface gets version 1's path
for that frame.
One thing version 2 cannot share with the copy path. A slot's fence says
the video thread has driven the driver with the slot and the GPU is
done; it says nothing while a frame is queued and not yet claimed, or
claimed and still being recorded. With a copy per slot that window is
harmless - the copy is replaced by a newer one. With version 2 the slot
is the core's texture, and a core let back in then would draw into what
the frame about to be recorded reads. So for version 2 slots the wait
also covers the window: until no queued frame names the slot, on the
condition the video thread already broadcasts when it claims a frame and
when it finishes one. Other APIs' waits are untouched.
d3d12.c, video_thread_hw.c and runloop.c pass tools/mingw_syntax_check.sh
(win32 gnu99 and the C89 lane). Not run: it needs a core that asks for
version 2, which is the next change, and a Windows GPU to run it on.
samples/gfx/d3d12_hw_interface_v2 still runs the contract itself.
Version 1's whole handoff is set_texture(texture, format). It does not
say when the texture is complete or when the core may draw into it
again, so the only safe thing a frontend can do is copy it before
video_refresh returns - and one that presents from another thread has to
record and submit that copy on the core's thread. That is the cost the
threaded video statistics show for a D3D12 hardware core next to a
Vulkan one (about 23 us a frame against 5): not D3D12, whose queues are
free-threaded and whose fences do what Vulkan's semaphores do, but an
interface that exposes none of it.
Version 2 adds what the Vulkan interface has had all along:
get_sync_index / get_sync_index_mask
the core keeps one present texture per index and is told which to
draw into
wait_sync_index
blocks until the frontend has finished with the texture last
handed over for the current index; until then it is not the core's
set_texture_fenced(texture, format, fence, value)
the frame, complete once the fence reaches the value; the frontend
does not read it before then, on whatever queue or thread it reads
The struct is version 1's with members added at the end, so one object
serves both, and RETRO_HW_RENDER_INTERFACE_D3D12_VERSION stays 1 because
existing cores compare against it and reject anything else. A core gets
version 2 only by asking, through the negotiation interface libretro
already has: RETRO_HW_RENDER_CONTEXT_NEGOTIATION_INTERFACE_D3D12 is new
in libretro.h, and retro_hw_render_context_negotiation_interface_d3d12
carries the highest version the core can use. An older frontend answers
GET_..._NEGOTIATION_INTERFACE_SUPPORT with 0 for the type and the core
carries on with version 1, having asked for nothing.
This is the contract and a proof that it is sufficient; no driver
implements it yet and no core asks for it, so nothing changes for
anyone until they do.
samples/gfx/d3d12_hw_interface_v2 runs it: a core and a frontend that
presents from its own thread, sharing a device and a queue, with the
handoff above and no copy on the core's thread. The frontend is slower
than the core, so the core laps it and has to block in wait_sync_index;
every frame must reach the frontend holding the frame number it was
published with. 240 of 240 do. A core that skips wait_sync_index is
built in as the control and is caught on its first lap ("index 0 should
hold frame 0, holds frame 237"); run.sh requires that. Built with
mingw-w64, run under Wine with vkd3d over Mesa's software Vulkan driver,
and added to the gfx samples lane, which fails rather than skips if it
finds no D3D12 device. The header compiles as C89 and as C++.
.gitignore carried a bare run.sh, which git matches at every depth, so
the script the gfx samples job calls could never be added and the job
has been failing on a file that is not in the tree. The rule is for
the checkout's own run.sh and now says so.
run.sh builds the oracle and the deferred proxy it exercises with
mingw-w64 - dxguid and uuid carry the interface GUIDs the proxy names
- brings up an Xvfb display when the job has none, and runs the three
cases the test documents: the ring and the immediate context must
pass, and the ring as it used to be, resetting the context every
frame, must fail. A build where that last one passes has lost the
regression the sample exists for.
The PS2 core on D3D12 with its hardware renderer showed a black picture
with HDR on: audio, input, widgets, statistics and the overlay all fine.
With HDR off it showed its picture. With the core's software renderer it
showed its picture either way.
With HDR on, and a menu or an overlay up, the frame is drawn into
chain.back_buffer and then composited onto the swapchain image. That
composite used pipes[VIDEO_SHADER_STOCK_HDR], whose blend state is
SRC_ALPHA / INV_SRC_ALPHA, and the HDR shader returns the sampled alpha
with the colour. So the frame was blended over the freshly cleared
swapchain image at whatever alpha the core's texture holds.
For a software frame that is 1: XRGB8888 has no alpha to read. A
hardware core hands over its own render target. The PS2 core's holds the
guest's alpha - the GS keeps destination alpha in the frame buffer, and
for most of a frame it is zero or 0x80 - so its picture was composited at
zero or half over black. The software renderer's output is opaque, which
is why the same core showed a picture in that mode; and the overlay the
reporter runs is what put the frame through the back buffer at all.
The composite now uses VIDEO_SHADER_STOCK_NOBLEND_HDR: same shader, same
target format, blending off. There is nothing under the frame to blend
with - the swapchain image is cleared immediately before - and the SDR
path has always drawn the frame with blending off. The second use of
STOCK_HDR further down, which lays the back buffer under the menu and
widgets, is left as it is.
Found with the D3D12 debug layer forced on from dxcpl, which showed the
core's own draws to be clean, and the reporter's HDR on/off and
software/hardware results. Passes tools/mingw_syntax_check.sh. Not run:
no HDR output here.
With HDR on, the D3D12 debug layer reports, for every overlay image of
every frame:
ID3D12CommandList::DrawInstanced: The render target format in slot 0
does not match that specified by the current pipeline state.
(pipeline state = R8G8B8A8_UNORM, render target format =
R10G10B10A2_UNORM) [RENDER_TARGET_FORMAT_MISMATCH_PIPELINE_STATE]
Thousands in a ten-second capture with an input overlay loaded, and
nothing else.
d3d12_render_overlay() binds sprites.pipe_blend unconditionally. That
pipeline is built for an R8G8B8A8 target; the overlay is drawn onto the
swapchain's back buffer, which under HDR is R10G10B10A2 or
R16G16B16A16_FLOAT. A draw whose pipeline disagrees with the bound render
target is invalid, and what it does without the debug layer is up to the
driver. Every other sprite draw in the file - blend_begin, blend_end,
the font, the menu sprites on either side of the overlay call - already
picks the _hdr pipeline by testing chain.current_rt_format. The overlay
now does the same. It is drawn between the same root CBV and sprite
setup the menu sprites use, so PSMainHDR sees the state it expects.
Passes tools/mingw_syntax_check.sh. Not run: no HDR output here, and
Wine has none to offer.
A sample that builds, runs and asserts something, and that no
workflow names, is a test nobody runs. font_lifecycle sat that way
long enough for its expectation to drift away from the font driver it
was written against, and it is not alone: fifteen others are runnable
and unclaimed, from the audio pipeline's format and layout oracles to
the overlay alpha-flag and viewport seqlock ones.
This job takes whatever is left over, under the address and undefined
sanitizers with leak detection on. The list is computed from the
sample tree and the workflows rather than written down, so a new
sample is covered by existing rather than by being remembered, and a
sample that gets a workflow of its own drops out of here on its own.
Samples needing a library the runner does not carry are skipped by
name; their own workflows install what they need.
The sample held the older rule - that a font made without a threading
hint rebuilds on the calling thread - and nothing ran it, so it sat
there disagreeing with the code. A rebuild goes through the video
thread whenever the wrapper is up, whatever thread made the font: a
font size change rebuilds the OSD font from the settings path on the
main thread, and a GL backend making its context current there while
the video thread holds it is an X BadAccess. The hint decides where a
font is created, not where it is rebuilt, and the sample now says so
and runs beside the other gfx samples.
The sweep found it, but only after it stopped inventing failures:
"does this sample have a check target" was a grep for check:, which a
comment satisfies, so samples whose runnable target is called
something else were sent to a rule that does not exist. make -n
answers the question properly.
The console/mobile Graphics Widgets Scale Override row is a plain
S_FLOAT row, which has range and OK handler slots, but it left them
empty and relied on two builder lines after ADD_DESC to supply the
0.2-5.0 range and setting_action_ok_uint. That dependency is what
broke in fb4a17e562 (fixed in #19584): the builder lines moved into
the desktop branch and the console row was left with nothing.
Put the range and handler in the row itself, the same way the
windowed row already does, and move the two builder lines back into
the desktop branch where only the fullscreen LV row still needs them
(the LV float grammar has no range or handler slots). The comment
explaining that is restored.
No behavior change: SDESC_RANGE_MINMAX with 0.2/5.0/0.01 is the same
range, enforced at both ends, that add_range(..., true, true) applied,
and the handler is the same. Desktop rows are untouched.
fb4a17e562 moved the OK handler and the 0.2-5.0 range of the widget
scale row into the desktop branch, so on console and mobile builds
Graphics Widgets Scale Override was left with no range and no OK
action and could not be changed. Apply them after either row again.
Six d3d12_set_hdr_* setters declared their counter inside the for
statement (for (unsigned i = 0; ...)), which is C99 and does not build
as C89. Declare it at the top of the block instead. No functional
change.
read_viewport and read_viewport_hdr clamp the viewport width and height
to the full output size, but never check x + width or y + height. The
viewport origin is user-controlled (custom aspect ratio), so a viewport
pushed off the right or bottom edge made the CPU copy loops - and the
shared dxgi_hdr_readback_* decoders, and the GPU HDR readback loops -
read past the end of each source row and, on the last rows, past the
end of the mapped staging/readback memory.
Add dxgi_readback_clamp_window() and call it in all four readback
entry points, against the real source size (the staging desc on D3D11,
the swapchain size on D3D12). Width and height double as the stride and
row count of the caller's buffer, which task_screenshot clamps the same
way, so they are left alone and the window is slid back inside the
surface instead. Vulkan already clamps its copy extent; the D3D drivers
did not.
A D3D11 hardware core under threaded video showed one frame and then
black, with widgets, overlays and statistics still drawn over it. With
threaded video off the same core showed its picture. Reported on the PS2
core.
The hardware ring gives the core a deferred context and closes its
recording at every video_refresh with FinishCommandList(..., FALSE, ...).
FALSE resets the deferred context to the default state each time. A core
written against the immediate context does not expect that: what it
binds stays bound there, because the frontend's own frame does not touch
those slots, so it binds some things once and never again. The PS2 core
binds its vertex-expansion buffer to VS slot 0 once, at device creation,
and the state it rebinds after each present does not include it. From
the second frame on that slot was empty, every sprite, line and point
collapsed to nothing - the screen and depth clears games draw as sprites
among them - and the picture was black.
TRUE keeps the deferred context's state across lists, which is how the
immediate context the core was written for behaves. The value lives in
d3d11_deferred_proxy.h as D3D11_HW_RING_KEEP_CONTEXT_STATE so the test
below uses what the driver uses.
samples/gfx/d3d11_hw_ring runs the ring's sequence - record through the
real proxy, take the texture at PS slot 0, FinishCommandList, replay on
the immediate context, copy - against a core that binds through a cache,
rebinds a fixed set after every present and has one binding it makes
once. It draws a different colour each frame over a black clear and
reads the frontend's copy back. Built with mingw-w64 and run under Wine,
where wined3d implements deferred contexts over OpenGL: the ring and the
immediate context show all four frames; the ring with FALSE shows the
first and then black, and run.sh requires that it still does. Added to
the gfx samples lane, which already has mingw and Wine. Nothing on Linux
exercised the D3D11 ring before this.
d3d11.c passes tools/mingw_syntax_check.sh. Not run on real Windows.
d3d12_gfx_read_viewport() ignored the return value of
dxgi_hdr_readback_to_bgr24() on the CPU HDR fallback path and always
returned true, so a failed decode was reported as a successful
screenshot with an unwritten buffer. D3D11 already returns false here;
make D3D12 match.
Follow-up to #19585. d3d11_gfx_read_viewport_hdr() builds its staging
desc from the swapchain backbuffer desc the same way the SDR path did,
but never cleared MiscFlags. Backbuffer misc flags are not valid on a
D3D11_USAGE_STAGING texture, so strict drivers (seen on Qualcomm
Adreno X2-90 for the SDR path) reject CreateTexture2D. The HDR path
already NULL-checks the result so it did not crash, but the HDR
screenshot silently failed.
Also initialise BackBuffer to NULL so the !BackBuffer check after
GetBuffer is not testing an uninitialised pointer when GetBuffer
fails.
The dupe lane added with 8ae3e66882 failed the ASan lanes:
heap-use-after-free ... READ of size 64
#1 video_thread_frame gfx/video_thread_wrapper.c
#3 video_driver_cached_frame
#6 run_frames
#7 lane_dupe_under_wrapper
freed by thread T0 here:
#1 lane_dupe_under_wrapper
The test's fault, not the wrapper's. video_driver_frame() keeps the
pointer it is given as the cached frame - borrowed, exactly as it
keeps a core's framebuffer - and the menu re-presents it on the next
frame it draws. The lane malloc'd its test pattern, pushed it, freed
it, and then ran two more frames with the menu up, which went back
for it.
The pattern is now a static buffer, so what the cache borrows stays
valid the way a core's framebuffer does, and the lane invalidates the
cached frame before it hands back, so the lanes after it do not find a
test pattern there.
I had only run the lane on a plain build, which is how this got
through. Run this time the way CI runs it, on trees built for it:
SAN=address (ASan+UBSan) and SAN=thread, CYCLES=2, all lanes pass.
tools/settings_check.py table --check fails on master: two settings
were added without the reference being regenerated alongside, so the
"Settings table drift" CI job reports drift for every change that
follows, whatever it touches.
bool audio_fastforward_callback
uint menu_thumbnail_preview_threads
Regenerated with
python3 tools/settings_check.py table --write tools/settings_table.reference
Those two rows, in each of the three configurations, are the whole
difference. table --check, unity (desktop and mobile) and
settings_migrate_selftest.py all pass.
A Windows build configured with --disable-wasapi did not compile:
configuration.c: error: 'struct <anonymous>' has no member named
'audio_asio_output_channel'
The field was declared in settings_t inside the #ifdef HAVE_WASAPI
block, next to audio_wasapi_sh_buffer_length. Everything that uses it -
its row in settings_def_audio_asio_action.h, the descriptor and
configuration tables that row expands into, the menu representation in
menu_setting.c, and asio.c itself - is under HAVE_ASIO, and ASIO and
WASAPI are switched independently. With ASIO on and WASAPI off the row
was emitted and the field was not there.
It gets a HAVE_ASIO block of its own. Builds with both drivers, or
with neither, see no difference.
Checked with a mingw cross build's own compile lines, HAVE_ASIO
defined and HAVE_WASAPI not: configuration.c fails before and compiles
after; menu_setting.c and asio.c compile.
With threaded video on, the ffmpeg core ghosted on Vulkan and D3D12 -
every frame doubled with an older one - and on D3D11 faded to grey and
back. gl and glcore were clean, because there the core renders with
OpenGL itself and never sends pixels at all.
A core with nothing new calls video_refresh with NULL, and the driver
repeats the frame it has. The ffmpeg core does that for every display
refresh that falls between two video frames: every other call for a
30 fps clip on a 60 Hz display.
video_thread_frame() treated that push like any other. It picked a
free ring slot, copied nothing into it because there was nothing to
copy, and published it. The video thread then gave the driver that
slot's buffer as the frame. What the buffer held was whatever had last
been put there: the frame from two pushes before, or - when the real
frames all arrive zero-copy in the other slot, as the ffmpeg core's
now do - something older still, down to the 0x80 fill the slots are
allocated with. The driver uploaded it and showed it as new. So the
display went new, old, new, old at the refresh rate.
A slot now records that its push carried no pixels, and for such a
slot the video thread hands the driver NULL, which is what the core
sent and what every driver already gets without the wrapper. The
pixel-format conversion and the video filter are skipped for it, as
they have nothing to work on. A hardware frame the ring could not
publish takes the same route; the comment there already said it was
to be treated as a dupe.
Regression test: a new lane in the threaded_video harness pushes
pixels, NULL, pixels, NULL through the wrapper with the driver's
frame() interposed, and requires the driver to see exactly that. On
the old code it fails with the driver given stale pixels for every
dupe; with this change it passes.
Also run on a real build under Xvfb, ffmpeg core on the software path:
Vulkan and gl1, threaded video on and off.
Loading a hardware-rendered OpenGL core (the ffmpeg core, any GL core)
on Windows with the glcore driver and threaded video on crashed at
load: a call through NULL out of the core's context_reset(), with
[WGL] wglMakeCurrent(core context) failed: 0x000000aa.
just before it. 0xaa is ERROR_BUSY: the core's context was current on
another thread. It was - the video thread had taken it.
gl3_t::flags is a uint16_t, and two flags are defined past the end of
it:
GL3_FLAG_HW_RING_EXPECTED = (1 << 16)
GL3_FLAG_OVERLAY_BORROWED = (1 << 17)
`gl->flags |= GL3_FLAG_HW_RING_EXPECTED` therefore stores nothing, and
every test of either flag is false, with no warning from any compiler.
HW_RING_EXPECTED is what gl3_core_context_is_mains() reads between
init and the ring coming up, to keep the video thread from taking the
core's context while the main thread is about to. With the flag lost,
the bind at the end of gl3_init() ran, the video thread left init
holding the core's context, and the main thread's wglMakeCurrent() for
it failed. With no context current, every GL function the core then
looked up came back NULL, and it called the first.
OVERLAY_BORROWED is what stops glcore deleting overlay textures that
belong to the overlay pack (load_textures). With the flag lost, glcore
deleted textures it had only been lent.
The field is now a uint32_t, as gl2's already is. Nothing else
changes: every other flag keeps its bit.
Found with a mingw build under wine, which fails the same way as a
real Windows box: before, the 0xaa error and no frames; after, the
core's context goes current on the main thread, 117 frames are pushed
and the process exits 0. glcore on Linux unchanged, threaded video on
and off.
The Win64 fastjmp_set saves xmm6-xmm15 with movaps, which faults on an
address that is not a multiple of 16. The header says the buffer is
"16-byte aligned for the movaps stores" in a comment over a plain
unsigned char array, whose alignment is 1. A file-scope static gets away
with it because compilers put a 240-byte static on a 16-byte boundary
anyway; a fastjmp_buf that is a struct member after a char sits at +1,
and the first fastjmp_set dies at "movaps %xmm6, 0x50(%rcx)".
The member is declared aligned to 16 with the MSVC or GNU spelling, both
valid in C89, so alignof(fastjmp_buf) is 16 on Win64. The System V and
AArch64 forms store through general-purpose moves and stp and are
unchanged, as is the setjmp fallback.
fastjmp_test gains that case: a buffer placed after a char, its address
checked on Win64, and a set and jump through it everywhere. Linux has no
aligned stores, so the Linux samples job could never have seen this; the
MSYS2 job in Cross-libc-vfs-samples builds and runs fastjmp_test too
now, under a second, and triggers on the fastjmp paths. Built with
mingw-w64 and run under Wine: the new case fails on the old header at
the address check and then faults, and passes on this one.
Closing content with threaded video on, the glcore driver and an OpenGL
hardware core (the ffmpeg core is the easy one to hit it with) killed
the process on X11:
X Error of failed request: BadAccess
Major opcode of failed request: 150 (GLX)
Minor opcode of failed request: 26 (X_GLXMakeContextCurrent)
The hardware ring takes the core's GL context on the main thread at
init and has to give it up there at free - only the thread a context
is current on can release it. video_thread_hw_free() runs on the video
thread, inside CMD_FREE, so it hands that release to the main thread
through video_thread_call_on_waiter().
video_thread_call_on_waiter() finds the wrapper through
video_thread_thr_capture, and video_thread_free() clears the capture
before it sends CMD_FREE. So by the time the ring asked, there was no
wrapper to be found, and the function returned without calling
anything. The core's context stayed current on the main thread, and
gl3_deinit_hw_render(), a few lines later on the video thread, bound
it: BadAccess on GLX, which Xlib answers with exit(1). On WGL the same
bind is a failed wglMakeCurrent, the context cannot be deleted while
another thread holds it, and the main thread goes into the next
session still holding a dead context.
video_thread_free() now names the instance it is freeing for the
length of CMD_FREE, and video_thread_call_on_waiter() falls back to it
when the capture is gone. The capture itself still clears first, so
the font and texture entry points stop taking the instance exactly
when they did. And with no wrapper to be found at all the call is made
in place, as it is in a build without threads, instead of vanishing.
Verified on a real build under Xvfb (Mesa): glcore + threaded video +
ffmpeg core exits 1 with the X error before, exits 0 after.
Loading a video in the ffmpeg core with the gl driver and threaded
video on crashed at load. On Windows it is a call through NULL out of
the core's context_reset(), from drivers_init(); on Linux a segfault in
the same place, after
[Video] Threaded video: the core's GL context could not be taken on
the main thread; the core will have no context.
Under the wrapper an OpenGL core renders on the main thread while the
driver draws on the video thread, so the core cannot borrow the
driver's context as it does unthreaded. It needs one of its own that
shares objects with the driver's: the shared context, which the
hardware ring makes current on the main thread before context_reset()
runs. gl2_hw_ring_context_new() refuses when there is no shared
context to take.
gl2 only ever made a shared context when the setting was on or the
core had asked for one with SET_HW_SHARED_CONTEXT. A core that did
neither - the ffmpeg core is one - was therefore sent into
context_reset() with no context current at all. On WGL every function
it looks up without a context comes back NULL, and it called the
first one.
glcore has always forced the shared context for hardware cores. gl2
now does the same whenever the wrapper is running and the ring is
allowed for the session, which is exactly when it cannot work without
it. Unthreaded behaviour is unchanged.
Verified on a real build under Xvfb (Mesa), gl + threaded video +
ffmpeg core: crashes at load before, plays and exits 0 after; threaded
off unchanged.
* I'm pleased to release a new variant of my changes
to the FFMPEG core. I have, I believe, fixed the
issues with the current code as well as my old
version. The core should now play files containing
many more video codecs--including HEVC and AV1 files.
The issue stemmed from how avcodec_send_packet
and avcodec_receive_frame must be mutually looped
when using the modern FFMPEG interfaces; the current
code essentially disregards the returned value so
does the wrong thing in certain scenarios.
I've also changed the `seek_adjust` function slightly
so that the time adjustment is nearer to exactly five
minutes for the common input.
I've also cleared the SEEK_L2_STR and SEEK_R2_STR
variables in a few more places to fix a bug where
seeking could get stuck on the wrong adjustment.
I've also fixed the description of the controls;
they are currently misreporting a 60 second seek
instead of the implemented 30 seconds.
A few more bugs, unfortunately, remain: the audio
decoder has a bug not unlike the bug in the video
decoder; changing the audio to a stream having a
different rate will increase or decrease the speed
of the video being played; the implementation is
somewhat fragile in the face of broken files in
that stuttering can accumulate and even crash
the decoder.
A note on reporting bugs: please, before you report
a bug with a certan file formats, run your sample file
through FFMPEG to make sure the file itself contains
no errors. As I mentioned above, the implementation
is somewhat fragile; the core, as is, really needs
files that contain no errors for proper function.
* ffmpeg core: review fixes for the send/receive decode loop change
Four things found reading the change:
* A comment in seek_frame() picked up half of another line in an
edit ("taken us to thestrength without using"). Put back.
* select_decoder() cleared FF_THREAD_FRAME with ^=. It works there,
because the bit was set two lines up, but it is a toggle, not a
clear, and the next edit that moves the assignment turns it into
a bug. Use &= ~.
* sws_worker_thread() now builds its scaler from the decoded frame's
own size and format instead of the container's, which is the right
thing, but it means a frame from a damaged file can be one that
sws_getCachedContext() refuses - and the NULL went straight into
sws_setColorspaceDetails() and sws_scale(). Check it, log it, and
still finish the slot so the ring keeps its order and the main
thread is not left waiting for a frame that never completes.
* The rewrite left two things behind that now warn: decode_thread()
still computed a frame_size nothing takes any more, and
video_buffer_has_open_slot() lost its only caller. Both removed.
* ffmpeg core: software video path stops copying frames it does not have to
A frame on the software path (any driver that is not GL - Vulkan,
D3D, Metal) was copied twice after the scaler was done with it: row
by row out of its ring slot into a buffer of the core's own, and then
by the frontend out of that buffer into its texture. At 3840x2160
that is 33 MB a copy, 60 times a second.
The first copy bought nothing. The frame is already sitting in a slot
in the layout retro_video_refresh_t takes, so now the slot is what
the frontend is given.
The second is what RETRO_ENVIRONMENT_GET_CURRENT_SOFTWARE_FRAMEBUFFER
is for. The core asks for the frontend's framebuffer every frame, as
the API requires; a driver that has one (Vulkan, D3D12) hands out the
mapped texture it draws from, the frame is copied into that, and the
frontend's own copy does not happen. One copy from scaler to GPU
instead of two. Where the frontend has nothing to offer, or offers a
format that is not XRGB8888, the slot is passed directly and it is
one copy that way too.
Showing a frame out of its slot means the slot cannot go back to the
decoder the moment it has been shown: the frontend may come back for
the last frame it was given (a paused menu redraws it), and by then
the decoder would be halfway through writing the next picture into
it. So the slot is held until the frame after it is due. Holding is
simply not opening - the slot stays finished at the tail, where the
decoder cannot go.
video_buffer_clear() (a seek) takes every slot back whether it is
held or not, and the same index can be finished again with a frame
nobody has seen by the time the holder lets go. The ring now counts
its clears, and a held slot is given back only if none happened since
it was taken; otherwise releasing it would drop that frame and push
the tail past it.
The software path's ring gets a fifth slot, so the decoder still has
four to run ahead with. That slot is the memory the temporary buffer
used to be: the buffer is now only allocated where it is still used,
which is the GLES texture upload.
Also checks the slot it is handed is not NULL, which the old code
did not: a clear between the wait and the take left ctx unset and
the next line read ctx->pts.
---------
Co-authored-by: LibretroAdmin <[email protected]>
tools/sample_link_check.sh linked every sample a change touches and
called that a check. A sample that builds and passes plainly can
still be the one holding a leak or a use-after-free, and only the
sanitized run says so, so a sample with a check target is now built
and run the way its CI job runs it: address and undefined sanitizers,
leak detection on. Both modes go through one routine - the --all path
had a loop of its own, which is how the full sweep went on linking
while the selected one checked. A sample that cross-builds a Windows
target is built and not run, since this host cannot run it.
Two things it found on the first pass. pipeline_clocked builds a sinc
resampler per run through retro_resampler_realloc and never released
it, leaving seven behind and 232 KB with them; the teardown releases
it as audio_driver_deinit does. And input_overlay_textures matched a
page's entry to the pack's unique image by the pixel buffer they
share, which is NULL on both sides once the pixels are released - and
NULL matches the first image every time. A released buffer matches
nothing now, so the pack is declined and reloaded from its path
instead of drawing one image everywhere.
HAVE_XCB was enabled by the libxcb check alone, but the code it gates
(gfx/common/vulkan_common.c) includes X11/Xlib-xcb.h and calls
XGetXCBConnection, and Makefile.common links -lX11-xcb. Those come
from the separate x11-xcb package (libx11-xcb-dev on Debian/Ubuntu),
so a box with libx11 + libxcb but no x11-xcb failed with:
gfx/common/vulkan_common.c:30:10: fatal error: X11/Xlib-xcb.h:
No such file or directory
Check for x11-xcb in configure and turn HAVE_XCB off when it is
missing, and link via X11_XCB_LIBS instead of a hardcoded flag.
The X11 Vulkan context only ever creates an XCB surface, so build and
register gfx_ctx_vk_x only when HAVE_XCB is set rather than shipping a
context that cannot create a surface.
Every overlay test in the tree stubs the driver or builds the pack by
hand, and every one of them passed while an enabled overlay covered
the whole screen. A person with threaded video off saw it in a second.
This test does what that person did.
samples/input/overlay_on_screen/overlay_on_screen_test.py takes a real
retroarch binary, starts it under Xvfb on Mesa's software GL with the
menu up and an input overlay enabled, grabs the root window with xwd
and reads the pixels - once with video_threaded off, once with it on.
The overlay is made on the spot: two solid magenta squares, top left
and bottom right, on a full-screen page at full opacity; magenta is no
colour the menu draws. It requires that
- the middle of each square is magenta (the overlay is up, in place);
- the middle of the screen is not (no image is stretched over it);
- magenta covers about the area of the two squares (8% here).
Against the previous commit's parent it fails in both modes with
"overlay covers 100.0% of the screen, its two squares are 8.0%"; with
the fix it passes at exactly 8.0% in both.
New workflow CI Linux overlay on screen: a slim X11 + OpenGL build
(shader compilers, the big menu drivers, networking and codecs off)
behind ccache, then the test, some twenty seconds. It runs when the
overlay frontend, the loader, gfx_surface, the video driver core, the
threaded wrapper or gl2 change.
An enabled input overlay covered the whole screen: every image of the
page drawn full-screen, one over the other (sonninnos; reproduced here
on the gl driver under Xvfb, threaded video off AND on).
The loader does not give the pack private image structs. For the first
overlay::image or overlay_desc::image that names a file,
task_overlay_load_image_texture() registers that struct's ADDRESS as
the pack's unique image, so input_overlay::images[] points into the
pages. Since 86fbf34e01 the pack's pixels are released once the driver
has the textures, with image_texture_free(images[i]) - which therefore
cleared .pixels inside the descs themselves. And "does this desc have
an image" was asked of .pixels, in four places: the page's and the
descs' vertex_geom, the movable-desc update, and the pressed-button
alpha. The very descs that had just been given textures answered no,
their geometry was never sent to the driver, and the driver drew each
image where a page load leaves it: over the whole screen.
e9468dedf8 moved that release later and kept it, and said of these
structs that they were "copies ... only ever tested ... and are left
to say so". They are not copies; that comment was wrong, and no test
could tell, because overlay_textures_test built its pack by hand with
private image structs.
- OVERLAY_HAS_IMAGE() asks the width. A desc or page without an image
is calloc()ed and has none; one with an image keeps it for good
(input_overlay_drop_pixels puts the sizes back after the free). The
four tests use it.
- With nothing reading .pixels as a flag any more, every copy of a
released pointer is cleared - the pages' load_images as before, and
now the page and desc image structs that share the file - both when
the pixels are dropped and when a surface in flight adopts them.
Nothing is left pointing at freed memory.
- overlay_textures_test lays its pack out as the loader does: descs
with images, images[] pointing into them, later users holding
copies. A new "has image" lane requires every desc to still report
its image once the page has gone as textures, and to hold no stale
pointer. With the old question put back it fails for the four descs.
One audit, one commit. Every item below was found by reading the
overlay code of each driver after the gl2 blackout (78fda8c573); what
could be run here was, and what could not is said at the end.
Frontend
- input_overlay_load_active() always went on to set a page's per-image
alpha and geometry after loading it. Twice the page may not be in
the driver: load() can fail - its answer was thrown away - and a
pack with neither textures nor pixels left is never handed to load()
at all. The setters then ran with the new page's indices against
whatever the driver held before. input_overlay_load_page() now
answers none / pixels / textures, counts a load() that returned
false as none, and the geometry is skipped for none. Every driver's
load() ends in true on success and the threaded wrapper passes the
answer back.
gl1, gl2, gl3 (glcore), vulkan
- 7661ab06e7 bounded vertex_geom and tex_geom in the three GL drivers
and missed set_alpha: NULL overlay_color_coord with no page loaded,
and a write past the page's block for an index off its end.
- Vulkan's three setters had no check at all; vk->overlay.vertex is
NULL after vulkan_overlay_free().
All now return for a NULL array or an index at or past the count.
d3d9 (cg, hlsl)
- load() set overlays_size whether or not the calloc() behind it had
worked. It fails cleanly now; a page with no images is no page.
- set_alpha, vertex_geom, tex_geom: bounded, and NULL-checked.
- hlsl ignored LockRect()'s result and copied to whatever d3dlr held;
cg already checked it.
- enable() set its flag inside a loop over the page's images, so never
for a page with none. Set once.
- full_screen(): NULL driver or array is a return.
d3d10
- d3d10_free_overlays() released each texture and never freed the
array holding them, while every load() made a new one over it: a
leak per page switch and one at teardown. Freed; count and buffer
pointer cleared with it.
- load() called through an unchecked calloc(), CreateBuffer() and
Map(). A page that cannot be built is freed and reported.
- The setters mapped a buffer that might not exist and indexed it
without a bound. One helper refuses a NULL buffer, an index at or
past the count, and a Map() that failed.
d3d11
- The sprite buffer is "reused across pages while it is big enough" -
but d3d11_free_overlays(), run at the top of every page load, had
already released it, so every page switch was a CreateBuffer. The
page and the buffer now go separately: the page at a page load, the
buffer at teardown.
- A page whose sprite buffer could not be made or mapped kept its
count; it is freed and reported.
- The setters ignored Map()'s result and wrote through pData, which a
failed Map() leaves uninitialised, with no bound. Same helper as
d3d10's.
d3d12
- A load_textures page copied each of the pack's d3d12_texture_t. The
deferred upload's state (dirty, upload_fence, the upload buffer)
lives in that struct, so copy and original part company at the
first draw: the pack's struct stays dirty for good and every later
page re-records an upload of every image it shares; and an animated
image's next frame marks the pack's struct dirty while the driver
draws from a copy that is not, so the animation stands still. The
page now holds pointers (overlays.refs): into its own array for a
load() page, to the pack's structs for a load_textures page.
- load_textures released the previous page's own textures and the
sprite buffer without draining the queue. The comment's reason - the
queue is drained at the top of every frame, so nothing reads them
between frames - has it backwards: the last frame may still be in
flight. The page free drains when, and only when, it has textures
of its own to release.
- The sprite buffer was released at the top of every page load, so
its reuse had nothing to reuse and every page switch was a drain
and a new buffer. Kept across pages; released, behind a drain, at
teardown.
- A page that cannot be built is freed and reported; a zero handle is
refused rather than dereferenced; the setters go through the same
kind of helper as d3d10's and d3d11's.
metal
- loadImages: stored -newTexture:'s result straight into an
NSMutableArray; nil there is an exception. The page ends at that
image and reports failure.
- -_getForIndex: answered &contents[index * 4] whatever the index and
whether or not there was a buffer. NULL for both now, and the three
updates return on NULL.
- Ownership needed nothing: the array retains the pack's textures.
CI
- tools/overlay_setter_bounds.py, beside the ownership check in CI
Linux libretro-common tests: for each audited driver, set_alpha,
vertex_geom and tex_geom - or the one helper they map through - must
compare the index with the page's count before writing; for Metal,
-_getForIndex: must check the buffer's length and its callers take
NULL. Against the drivers as they were: 25 failures across all ten.
- overlay_textures_test asks load_page for its exact answer and gains
a lane where the driver's load() fails.
Tested: both CI scripts; mingw gnu99 + linux-c89 syntax gate on all
eleven C files; gl1, gl2, gl3, vulkan and the frontend compiled in a
configured Linux build; overlay_textures_test plain and THREADS=1
under ASan + UBSan with leak detection. Not tested: metal.m cannot be
compiled here, and none of the D3D or Metal code has been run - no
runner and no machine here has those drivers.
No runner can run a GL or D3D driver, so the gl2 overlay blackout got
through with four correct siblings beside it. tools/
overlay_texture_ownership.py is the check at source level: every file
in gfx/drivers that implements load_textures must set its "borrowed"
marker when it takes the pack's handles, test it (negated) where it
frees a page, and clear it so a later load() page is not leaked.
metal.m is exempt, with the reason in the script: its textures are ARC
references. vulkan.c clears its marker by wiping the overlay struct,
which the script is told about.
Against gl2.c as it was, it fails with "the marker is never tested"
and "never cleared"; d3d11, d3d12, gl3 and vulkan pass. Runs beside
the VFS backend parity check in CI Linux libretro-common tests.
On the GL driver, rotating an Android device turned the on-screen
overlay into semi-transparent black boxes that stayed until the core
was restarted (reported by parkerlreed, Galaxy Fold, Adreno 840,
GLES 2 context, threaded video off).
Since df4b5a7e0b a page of the overlay is handed to the driver as the
pack's own texture names (load_textures). gl2_overlay_alloc() marks
such a page GL2_FLAG_OVERLAY_BORROWED - and nothing ever read the
flag. gl2_free_overlay() runs at the top of every page load and called
glDeleteTextures() on whatever names the previous page held, so the
first page switch deleted the pack's textures. A rotation is a page
switch (landscape to portrait and back), most names are on both pages,
and the pack - still cached, still sure it has its textures - went on
handing out dead names: an unbound texture samples as black, the
overlay's alpha does the rest. gl3, d3d11, d3d12 and vulkan honour
their marker; gl2 alone did not.
gl2_free_overlay() now deletes only names the driver made itself
(load()), and clears the flag so the next such page is not leaked.
4552975f37 stopped flushing INFO/DEBUG lines to the log file and only
flushed [ERROR] and [WARN], to save the per-line fflush cost. In
practice that costs more than it saves:
- When RetroArch crashes, the lines that explain the crash are
usually the informational ones written just before it. Those were
exactly the lines left sitting in stdio's buffer and lost, which
makes crash reports much harder to read in the cases where gdb
does not help.
- Anything following the log live (tail -f, monitoring scripts) saw
nothing until the buffer happened to fill (#19571).
Go back to the previous behaviour: one write and one flush per line,
for every tag. rarch_log_tag_wants_flush() and the flush parameter on
rarch_log_line_emit() are removed. The strcmp tag matching for the
Android log priority from afd57d615e stays - that overread fix is
independent of the flush policy.
The log-atomicity harness's flush lane now asserts the promise that
matters: a RARCH_LOG (informational) line must be readable from the
file as soon as the call returns, before any deinit.
Fixes#19571
The emulated mailbox checked its fence wait only for VK_TIMEOUT, so
VK_ERROR_DEVICE_LOST or an out-of-memory error handed the main thread
the image as VK_SUCCESS. A lost device was still caught, by the next
frame's submit, but only after that frame had been recorded and
submitted against it.
Carry the wait's result instead. An error is handed over the same way
as an image, so the main thread's existing acquire error path takes it
from there: it drops the swapchain and, for a lost device, raises
VIDEO_FLAG_GPU_DEVICE_LOST.
The runner's libmbedtls-dev is 2.28, which has no TLS 1.3, so the mbed
leg could not see the session-ticket read failure (#19580) or the
missing psa_crypto_init(). Add a leg that builds Mbed TLS 3.6.0 - the
one release that shows both - and runs the harness against it. Library
only, about 20 s; a tarball download that still fails after retries
skips the leg with a warning instead of failing the job.
3.6.0 also ignores MBEDTLS_SSL_VERIFY_OPTIONAL under TLS 1.3, so the
harness can no longer lean on it: mbed_seed.c wraps the real backend's
ssl_socket_init and adds the test CA, as bear_seed.c does for BearSSL.
The handshake now verifies on every Mbed TLS version.
Makefile: MBEDTLS_DIR=<built tree> selects the library, and clean
removes both seed objects whichever SSL_BACKEND it is run with - a
stale mbed_seed.o built against other headers links and then fails
every TLS case.
Verified on 3.6.0: without the psa_crypto_init() change 4 cases fail,
without the ticket change case 6 fails, with both all pass (ASan+UBSan).
Mbed TLS 3.x runs TLS 1.3 on top of PSA and leaves psa_crypto_init()
to the application. The backend only called it for 4.x. 3.6 compiles
TLS 1.3 in by default, so with a system 3.6 the result was:
3.6.0 every handshake fails with MBEDTLS_ERR_SSL_INTERNAL_ERROR
(-0x6C00) while building the ClientHello key share - no
HTTPS at all, RetroAchievements login included
3.6.1+ the library quietly stops offering TLS 1.3 when PSA is not
initialised, so connections fell back to TLS 1.2
Call it on 3.x whenever TLS 1.3 or MBEDTLS_USE_PSA_CRYPTO is compiled
in. The bundled 2.x and the 4.x path are unchanged.
A TLS 1.3 server sends NewSessionTicket messages after the handshake.
Mbed TLS 3.6.0 and 4.x report each from mbedtls_ssl_read() with
MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET, asking to be called again;
3.6.1 and later 3.6 releases drop tickets by default. Neither receive
path knew the code. The nonblocking one only got past it when isagain()
happened to find EAGAIN left in errno, and otherwise failed the
transfer, so an HTTPS request to a TLS 1.3 server such as
retroachievements.org failed at random with receive_header_failed: the
cause of intermittent "Login failed: No response" with those versions
as the system Mbed TLS. Keep reading instead.
cheevos_login_test gains a case that reads with the tickets already
buffered and errno cleared, which fails without this change on Mbed TLS
3.6.0 or 4.x.
overlay_textures_test, THREADS=1:
- late handles now holds the whole sequence: both pages through load()
with live pixels while the uploads are pending, the uploads posted
exactly once, the page on screen promoted to textures when the
handles are in, the pixels gone after that and not before, a second
promote doing nothing, and the next page switch uploading nothing;
- late and declined / late and failed: the handles come in and the
driver declines them, or the thread could not make them - load()
from then on, pixels kept, no texture held for nothing;
- freed in flight: the pack is freed, and separately the driver torn
down, with the uploads still queued, and the video thread runs
afterwards. Before gfx_surface_free_adopt() this was a
heap-use-after-free on the video thread's read of the pixels; the
torn-down pack that lives on reports nothing to come back from and
is never handed to load().
Already in the CI step (ASan + UBSan with leak detection), so the
adopted pixels are checked for being freed as well as for not being
freed early.
Two things the threaded lanes of overlay_textures_test turned up.
Under threaded video the pack's textures were never used.
input_overlay_upload_textures() posts the uploads and polls for the
handles in the same breath; the video thread has not woken yet, so the
handles are not there, and the function released everything and had
the page shown through load(). The uploads still ran, and their
textures were unloaded as they completed. The next page switch did it
all again: three page loads of a three-image pack measured nine
texture uploads thrown away and three load() calls, which is more work
than before the pack's textures existed.
Late handles are now a state, not a failure. The surfaces are kept,
the page goes through load() this once, and
input_overlay_promote_textures() - one call per input poll, a few
pointer tests when nothing is pending - hands the page on screen to
the driver as textures when the last handle is in, after which the
pixels go as they do unthreaded. A page switch in the meantime posts
nothing. A pack that will not be shown as textures (upload failed,
driver declined) keeps its pixels and gives back what was uploaded.
And a use-after-free, narrow but real. A still image's upload borrows
the pack's pixels (gfx_surface_submit_external). Freeing the pack, or
tearing the driver down, with that upload still queued left the video
thread to read freed memory: the surface knew to outlive its flight
(dying), the pixels did not. gfx_surface_free_adopt() lets such a
surface take the pixels with it and free them at the completion;
input_overlay_release_textures() uses it, and the image is left with
its size and no pixels, exactly as after the pixels are dropped.
input_overlay_has_source() now asks every image for its pixels, not
the first, since a release in flight takes only some.
overlay_textures_test was built without HAVE_THREADS, so what happens
under the threaded wrapper was read off the code and never run. With
THREADS=1 the sample now links a stub of the wrapper's async upload
path: video_thread_async_post() queues the node as the real one does,
video_thread_async_poll() delivers completions on the caller, and the
video thread is a function the test calls at the point in the sequence
where it wants that thread to have got round to its queue. The race
between the posts and the poll is therefore the test's to decide, and
both outcomes are lanes:
- prompt handles: the uploads finish before the poll. Both pages go as
textures, each with its own images, and the pixels go afterwards.
Against 86fbf34e01's code this lane fails exactly as the unthreaded
one does - threaded video was never immune, only slow enough;
- late handles: the poll finds nothing, the page goes through load()
with live pixels, the pack keeps them, and when the thread runs it
reads pixels that are still there. Nothing outlives the pack.
Runs under ASan + UBSan in CI Linux samples/input beside the existing
two builds; well under a second.
vkAcquireNextImageKHR has already succeeded by the time the emulated
mailbox waits on the fence, so a VK_TIMEOUT there means only that the
presentation engine has not released the image yet. Treating it as a
failure left the thread idle without re-arming REQUEST_ACQUIRE, while
the main thread kept HAS_PENDING_REQUEST set, so nothing asked for
another acquire and video stayed frozen until fast-forward ended and
the mailbox was torn down.
Re-arming is not an option: the acquired image cannot be handed back,
and the next acquire would pass a fence that is still pending. Keep
waiting on the same fence, checking the DEAD flag between waits so
teardown stays prompt.
Drivers that let the fence rather than the acquire carry the wait reach
this. AMD's Windows driver holds the fence for tens of milliseconds
during a display stall, and minimizing a fast-forwarding exclusive
fullscreen window there froze video in 3 of 4 runs before this change
and in none after.
Leaving the wait on DEAD leaves the fence pending, and a pending fence
must not be destroyed, so wait on it once more first, bounded by the
same refresh-derived timeout as the mailbox's other waits so a driver
that never signals cannot hang the join.
samples/input/overlay_textures compiles input/input_overlay_textures.c
as it is against the real gfx_surface and a stub driver that reads
every pixel it is handed, so a stale pointer is an ASan report rather
than a picture nobody looks at. The pack is two pages over three
images, one on both pages. Lanes:
- textures: a driver with load_textures gets each page's own images'
textures and load() is never reached;
- page switch: the second page uploads nothing;
- pixels go: once the driver has the textures the pixels are
released and the sizes stay;
- declined / no such path / upload fails: a driver that answers
load_textures with false, has none, or cannot make a texture gets
live pixels through load() on every page, and the pack stays whole;
- spent pack: textures gone with the driver and pixels gone before
them - nothing is shown, and the pack says it cannot come back;
- no images: a pack of hitboxes alone is always reusable.
Against 86fbf34e01's code the first lane is a heap-use-after-free in
load(); with the pixels dropped before load_textures() has answered,
the declined lane fails. Runs plain and under ASan + UBSan in CI Linux
samples/input, well under a second each.
dac760d996 added anim_list to the payload a finished overlay load hands
over; the loader forgets it at the hand-off, as it does the image
list. Two takers did not free it:
- overlay_budget_test's consumer, which is why the ASan lane of CI
Linux samples/tasks reports 4752 bytes leaked in 12 allocations,
one string_list per successful load
(task_push_overlay_load_default, task_overlay.c:1500);
- input_overlay_loaded() when it cannot allocate the pack, which
freed the image list and leaked this one with its file buffers.
Both now free the entries' buffers and then the list.
With threaded video off, enabling an input overlay showed one
oversized blob across the screen or took RetroArch down (reported by
sonninnos, bisected to 86fbf34e01).
86fbf34e01 freed the pack's decoded pixels inside
input_overlay_upload_textures(), as soon as every image had a handle.
The very next loop builds each page's handle list by matching the
page's pixel pointers against the unique images' - which had just been
set to NULL. No entry matched, the upload released everything and
returned false, and input_overlay_load_page() fell back to load() with
the pages' own copies of those pointers: freed memory, handed to the
driver as images. With threaded video the handles are not there at the
poll, the function returns before the free, and the page goes through
load() with live pixels, which is why only the unthreaded case showed.
There was a second way in. The wrapper's overlay table answers
load_textures for any driver, and the one beneath it may have no such
path (d3d9, d3d10, gl1, ...): load_textures() returns false after the
upload succeeded, and the fallback is load() again - so freeing at the
upload leaves that route nothing to read either.
The pixels are now released by input_overlay_load_page(), and only
once load_textures() has answered true. The pages' copies of the
pointers are cleared with them, so nothing is left that looks like an
image and is not one. The sizes are kept, as 86fbf34e01 meant them to
be: image_texture_free() zeroes them. A pack left with neither
textures nor pixels shows nothing instead of reaching load(), and the
cache check in input_overlay_init() becomes input_overlay_has_source(),
which also stops a pack with no images at all from being reloaded from
its path on every toggle.
The upload of a pack's images, the page lists built from them, their
release, and the choice between showing a page as textures
(load_textures) or as pixels (load) move from input_driver.c to
input/input_overlay_textures.c, unchanged. input_overlay_load_active()
keeps the per-page alpha and geometry and calls the new
input_overlay_load_page() for the rest.
Nothing behaves differently. input_driver.c cannot be compiled outside
the frontend, so none of this could be put under a test; the new file
needs gfx_surface and four driver entry points, which a sample can
stub. The next commits fix what 86fbf34e01 broke here and then pin it.
A badge that is on disk reaches a widget a few frames after it is
asked for - decoded by a task, uploaded by the video thread, never
waited on - and the progress tracker and the challenge indicators drew
the placeholder trophy for those frames. The tracker is shared by
every achievement with measured progress, so a game advancing two of
them at once has it change hands every few frames, and it threw the
old badge away and loaded the new one each time: a flicker without
end.
rcheevos_get_badge_texture_ex() says why a 0 is a 0: pending while the
file is on disk and loading, not pending while it is being downloaded
or after a failure.
Progress tracker:
- it keeps the last four badges it has shown, so going back to one is
immediate and loads nothing
- an update whose badge is not there yet is not applied until the
badge arrives: what is on screen - the previous progress, or nothing
on a first appearance - stays exactly as it is meanwhile, text and
badge together. The wait is at most CHEEVO_BADGE_HOLD_FRAMES frames,
and none at all when the badge is not coming soon; then the
placeholder is drawn, as before
- the hide timer does not fire under a waiting update
- its textures are unloaded on context destroy and on free, which
used to leave the handle behind
Challenge indicators: one that has just asked for its badge is not
drawn while the answer is pending, under the same bound, and keeps its
place so its neighbours do not move. They now ask before they draw
instead of after, so the badge shows a frame sooner.
The holds count frames; no clock is read for them.
A surface with a submit in flight refuses every slot, not just the
one it is using, so the second slot an animated overlay image was
given could never be written: its frames are composed on the main
thread and submitted at once, which leaves no window for a slot to
fill while another is out. That is a frame of memory per animated
image - eight megabytes for a 1080p animated border - for something
unreachable. The slot count goes to one, and the branch that picked
between slots says what it always did: a frame arriving while the
last is still out is dropped, and the next poll tries again.
Two slots remain right where a worker thread decodes into one while
the other is with the video thread, which is the animated thumbnail's
case and not this one. The sample holds the contract and records the
reason.
An overlay image that carries an acTL animation chunk keeps its file
bytes and gets a stream over them; the pack gives it a surface with
slots rather than a static one, and its already-decoded first frame
goes into the first slot, so nothing decodes twice. Each poll composes
whatever frame is due into a free slot and submits it, which updates
the texture in place: the pages' handles stand, so a page switch is
still an upload of nothing and an animation costs no texture of its
own. A slot the video thread has not finished with means the frame is
skipped rather than waited for - an overlay is a control surface, and
a late button image is worse than a dropped one - and the end of a
pass rewinds, since a control surface that animates should keep doing
it. A hidden overlay does not advance: composing frames nobody can
see is work with no observer, and a resumed loop is invisible on
button art.
An animated image keeps its compressed bytes and its slots where a
still releases its pixels once the texture exists, which is what a
stream needs to decode from. samples/gfx/overlay_apng builds its own
APNG and holds the contract the pages depend on: frames that differ,
a pass that ends and replays, one texture across the submits, and a
free with a frame still in flight.
The still loader could never be asked for the extra precision, so a
16-bit PNG was flattened to eight bits a channel whatever the driver
could sample. texture_image::pix10 is an ask on the way in and an
answer on the way out now: set it and a decoder with a 10-bit path -
rpng for 16-bit PNG, the video stills - emits XRGB2101010 and reports
what it produced; leave it clear and the decode is the 8-bit one it
always was. A failed decode clears it, so nothing mistakes a stale
request for an answer.
The overlay loader asks when the surface layer says the driver
samples XRGB2101010, which is how a pack's artwork reaches an HDR
swapchain without a pass that throws its low bits away first. A
driver that later cannot sample it narrows the image as before.
A loaded pack held its decoded pixels for the whole session against
one event: a video reinit, where the textures die with the driver.
Measured against libretro/common-overlays that is 0.9 MiB for a
gamepad pack and up to 40 MiB for a keyboard, a 4K border or a 6x
landscape pad - held the entire time to save a decode at a moment
that tears down and rebuilds a graphics driver anyway, and one that
reloads the overlay from its path regardless
(video_driver_init_internal). So the pixels are freed as soon as
every image of the pack has its texture, and the width and height
stay, which is what the layout reads.
A cached pack is reusable only while it still has something to
upload from, so the cache check asks for live textures or live
pixels and a pack that has neither is reloaded from its path; an
upload with nothing to upload declines, which the existing
TEXTURES_DECLINED fallback already handles.
An overlay pack keeps its decoded pixels after the textures exist, in
case a video reinit has to upload them again. Whether that is worth
having is a question about how many megabytes it is, so the counters
answer it: overlay_pixel_kib rises with each image uploaded for a pack
and falls when the pack's textures are released, which also makes the
accounting checkable.
The Vulkan harness target filtered its log down to the pass and
failure lines, so the baseline lines the counters print - the loads,
updates, posts and copies a run actually made - never reached the job
meant to report them. It passes those through now, and the overlay
lane prints and checks the pack figure beside the rest.
The one-by-one white texture is built on the stack and handed to the
driver with its channel order never written, so the drivers read
whatever the stack held there. Four 0xff bytes come out the same
either way, which is why it has gone unnoticed, but the field is the
caller's to set and every other producer sets it: it comes from the
surface layer now, like the rest.
2026-09-18 18:22:09 +02:00
2112 changed files with 169948 additions and 279556 deletions
printf '::error title=Blocking wait on a menu path::task_queue_wait() must not be reachable from the menu. Use cancellation with a stale-completion guard, or a deferred continuation.\n'
exit 1
fi
printf 'No task_queue_wait on the guarded menu paths.\n'
# The close's blocking helpers wait inside task_save.c, where
# the grep above cannot see them: the menu and the tasks must
printf '::error title=Blocking close on a menu path::content_wait_for_*_state_task() blocks the frame loop. Close through retroarch_main_deinit_begin/pending/finish from a main-thread task.\n'
for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -property installationPath`) do set VSPATH=%%i
- NETWORK: New network commands SAVE_STATE_SLOT N and GET_CONFIG_PARAM
- NETWORK: HELP network command lists every command with what it does
- NETWORK: LIST_CORES, LIST_PLAYLISTS and GET_PLAYLIST network commands, to find content and the core to load it with
- NETWORK: MCP server: AI assistants can drive RetroArch through the Model Context Protocol, every network command a tool (off by default, see docs/mcp-server.md)
- NETWORK: Built-in TLS 1.2/1.3 client; mbedTLS and BearSSL are no longer bundled (--enable-mbedtls / --enable-bearssl use a system one)
- NETWORK: Built-in SMB2/3 client with signing, AES-CCM/GCM encryption and Kerberos; libsmb2 is no longer bundled (--enable-libsmb uses a system one)
- NETWORK: Built-in NFSv3 and NFSv4 (4.0, 4.1, 4.2) client for loading content from nfs:// exports
- NETWORK: NFS files stay usable across a server restart, a dropped connection or a long pause: the client reconnects and reopens what it needs
- 3DS/VITA/SWITCH/WIIU: nfs:// content through the built-in NFS client
- 3DS/VITA/SWITCH: smb:// content through the built-in SMB client
- WINDOWS/MSVC: the Visual Studio 2005 - 2017 projects build the TLS client, the keychain, SMB and NFS
- NETWORK: SMB files stay usable across a server restart: the connection is mended and the file opened again where it was
- NETWORK: NFSv4.2 reads use READ_PLUS: holes in sparse files no longer cross the network
- 3DS/SWITCH: nfs:// content through the built-in NFS client
- NETWORK: Optional SMB/NFS read-ahead with background prefetch, for large disc images over a slow link (off by default)
- NETWORK: Saved passwords are encrypted in retroarch-keychain.cfg; an optional passphrase lets the keychain move to another machine (see docs/network-stack.md)
- NETWORK: SMB 3.1.1 signs with AES-GMAC where the server offers it, several times faster than AES-CMAC on signed reads
- NETWORK: HTTPS downloads whose length is set by the server closing the connection now finish instead of hanging
- OVERLAY: Allow overlays to position the viewport
- OVERLAY: Hold button function
- OVERLAY: Allow interaction during next overlay swap
RARCH_LOG("[CoreAudio] The callback came up short %u time%s; at worst it wanted %u samples more than the %u it found, against a %u-sample ring and a %u-frame device period.\n",
Loaded 100 of 2112 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
