Add bounded XMA2 packet traversal and one-shot overlap draining. Honor speaker masks when folding WMA/XMA to stereo and expose .xma through the gated mixer and native player paths.
Compare startup, sample count and tail PCM to a pinned FATE reference in CI; retain allocation-free decoding and the existing raw-frame contract.
Add borrowed ASF and RIFF/xWMA streams with bounded fragment assembly. Drain decoder planes directly into audio_transfer output and reuse the mixer's streaming voices. Route music input through VFS-backed moving windows.
The builtin player uses source-rate integer intervals, float callbacks when supported and a legacy S16 fallback. Respect partial callback acceptance and reset without retaining a growing output queue. Gate the feature with HAVE_RWMA, disabled by default, and preserve FFmpeg/MPV selection.
Test analytic PCM, packet fragments and refill, loop reset, implicit xWMA setup, truncated objects, sample-rate cadence, callback backpressure and codec-disabled builds.
With OpenXR built, every RETRO_DEVICE_POINTER and LIGHTGUN read walked
the core port's users to gather their slots before asking the headset,
which answers false at once whenever its laser does not drive a core's
pointer - nearly always without a headset. The headset is now asked
first whether it does, and the walk is skipped when not.
openxr_input_test checks the answer with the laser off and on.
Adapt the MIT WaxFlow spectral decoders with immutable codebooks, reusable transform storage and borrowed PCM output. Keep compressed carry only for frames crossing packet boundaries and look up reused envelopes without copying a resampled curve. Preserve the upstream numeric-table provenance and MIT notices.
Add strict C89, direct-transform, malformed-input and packet-continuation regressions, plus an optional test-only FFmpeg PCM oracle. Gate frontend builds with HAVE_RWMA; playback integration and core replacement remain separate changes.
To tell a playlist, where the right stick cycles thumbnails, from any
other list, the menu's stick handling built a full menu_entry_t for the
list's first entry - six string buffers, over 4 KiB on the stack, and
every label and value callback run - every frame the right stick was
held, only to read its type. The type is read from the list instead.
input_driver_collect_system_input()'s frame goes from 4304 to 576
bytes, back under the 4 KiB budget.
frontend_overhead_test gains a lane: in a playlist the right stick is
its own bind, elsewhere the D-pad.
Every poll made the join - which user's controller plays as which
player - for every user, and compared it with the last one. It is
made from the Plays As entries, the controllers' claims and names,
Max Users and the users' Device Index, none of which a frame changes.
The poll now makes it only when one of them moved since it last did:
input_plays_as_gen, which already moved for most changes to entries,
also moves wherever a claim, an Auto player, a share or a
controller's name is written, and the two settings are compared with
what the poll last saw.
mapped_port_test checks after each poll that the join kept is the one
the poll would make; a poll that never makes it again fails there in
25 places, 52 with OpenXR.
Callgrind, frontend_overhead input lane: the join's 3.7M instructions
a run are gone.
With two users on the same pad (Device Index) and the same core port,
the rumble flush wrote that pad's motor once for each user: two driver
writes, an ioctl or a HID transfer each, for one frame's value. A mask
of the pads written skips the repeat.
mapped_port_test covers it.
Each core read tested Tap Delay's idle mask on its own before the
existing test for held-back buttons and the enabler's game buttons.
With Tap Delay idle or off that was one more load and branch on every
read. The check now sits inside that gate, which a flag set with the
idle mask opens.
Callgrind, frontend_overhead input lane: the core reads' wrapper back
to master's cost before Plays As; with the two patches after it the
lane goes from 485.7M to 469.5M instructions.
The translation sync failed converting msg_hash_tt.json: "packed
emitter: keys not in source: MSG_FIRST_RUN_DOWNLOAD_PROMPT". The
prompt's English text was three adjacent literals on three lines,
which h2json does not join, so the key never reached msg_hash_us.json
while the translations carry it. It is one literal now; h2json
exports it and json2h converts a translation holding it.
2c060e0 stopped d3d9_hlsl_views_hw_copy() from storing the view's size
as the slot's last_dims, which kept the first pass from making the
slot's vertices; d3d9_ex_test still asked for the old store and failed
"the slot holds the view's size" on both devices. It now checks the
copy leaves last_dims as it found it.
overlay_behind_menu, read only under HAVE_OVERLAY, is declared there
too: the test's build without it warned it unused.
Revert 7777f39f5c: libretro-common requires an MIT implementation. Remove the derived decoder sources, codebooks and optional build/CI wiring. A replacement must use an independent clean-room process; the LGPL port must not be relabeled or reused as MIT.
Adapt LGPL WMA spectral decoding and codebooks behind a C89 packet API without a libavcodec dependency. Decode raw XMA bit spans directly and lend planar float PCM from reusable context storage. Allocate only at creation, retain bounded split-packet history, and keep file I/O, playback and resampling outside the decoder.
Add optional HAVE_RWMA sources, analytical XMA/IMDCT and packet-history tests, malformed-input and allocation checks, and a test-only FFmpeg PCM oracle in CI. Linux ASan/UBSan, MinGW and MSVC focused tests pass. Core migration, history serialization and gameplay validation remain separate.
altool from Xcode 27.2 logs an ERROR line about
CDUploaderStateInfoResumeFilePath even when the upload succeeds, and
fastlane fails the upload on any ERROR line from altool. Skip that line
so upload_to_testflight continues to processing and tester distribution;
every other altool error still fails the upload.
* OpenXR: the controllers' slots are not read from the driver's view
A joypad driver that keeps its pads' state lends it to the snapshot
bridge through view(), and the bridge reads that in place of a copy
(48f9503c16). The headset's joypad wrapper is made from the driver it
wraps and kept that driver's view(), so under udev or winraw the
bridge read the controllers' slots from the driver's own pads there,
which are empty: the OpenXR controllers gave no input at all.
The wrapper of a driver that lends now has a view() of its own. It
lends none for the controllers' slots, which the bridge then copies
through the wrapper's button and axis calls, and the driver's for
every other slot.
openxr_input wraps a driver that lends all-zero views, and reads the
controllers' slot and one of the driver's the way the bridge does.
* Menu: bind capture goes to the bound user, not the controller slot
A bind capture knows the user whose binds it sets and the slot of the
controller it listens to, that user's Device Index. Three of its steps
used the user whose number is the slot instead: Bind All set every
bind after the first in that user's binds, each capture took the key
it replaced, whose mapping bit it clears, from that user's bind, and a
mouse bind listened to that user's mouse.
So with user 1's Device Index set by hand to the controller in slot 3,
Bind All for user 1 wrote all but its first bind over user 3's, which
were saved as such, and cleared the bit of a key user 3 still had
bound. A mouse bind for user 1 was never heard with one mouse, and
took user 3's click with a mouse for each user, though at run time
user 1's binds read user 1's mouse. All three now go to the bound
user.
frontend_overhead runs Bind All for user 1 on slot 3, and clicks each
user's mouse while binding user 1's B there.
* samples: hotkey_menu reads its sources as UTF-8
extract.py read input/input_driver.c in the locale's encoding, cp1252
for a native Windows Python, which fails on that file's UTF-8, so the
harness did not build there without PYTHONUTF8=1. It reads and writes
UTF-8 now; on Linux its output is unchanged.
* Input: a core port's rumble reaches every user mapped to it
A core port's rumble now goes to every user mapped to that port, the
users its input is read from, rather than to the user with the port's
number. Its motor answers true when any of theirs does; a port no user
is mapped to rumbles nothing and answers false. With every user on its
own port nothing changes.
* Input: Tap Delay holds Hotkey Enable back from its first frame
The hotkey pass reads the controllers as the last poll left them, and
the frame's own poll comes after it, under early, normal and late
polling alike. So with Hotkey Enable Tap Delay on, each press of the
Enable button reached the core for one frame before the tap saw it
and began holding it back.
While the tap is idle, a core read that finds the Enable's game button
down now asks whether the Enable is held in the frame's poll, and if
so holds the button back. The same button from another controller or
key, with the Enable up, is still the game's. Other reads cost one
test, and with Tap Delay at 0 nothing changes.
* Input: users sharing a core port drive the menu and fire profile hotkeys
With the menu up, user 1 and every user mapped to its core port drive
it. The other users on the hotkey user's core port fire hotkeys from
their own controllers in the one hotkey pass; with Hotkeys Follow
Player 1 those are core port 1's other users. With user 1 on None, the
users on core port 1 do both, and user 1 keeps them. With every user
on its own port nothing changes.
A sharing controller fires the hotkeys its profile binds, never the
hand-set ones, which stay the hotkey user's pad's: a hand-set button
names a button of one controller, and on a pad of another model it is
some other button. Its Hotkey Enable is its profile's too. While the
hotkey user has a Hotkey Enable, a sharer's hotkeys need the sharer's
own or the keyboard's, and a sharer holding its own opens only its own
hotkeys, not the hotkey user's; held on any of them, it holds game
input back. With none for the hotkey user, sharers fire freely. The
table of a sharer's profile hotkeys is made when a bind or its profile
changes, not each frame.
With Hotkey Enable Tap Delay each controller taps on its own: a tap
holds back and replays that controller's own Enable button. A hotkey
swallows the tap of the controller that fired it, and the keyboard's
tap when the keyboard's Enable opened it. Any controller's Enable held
short of the delay holds game input back, another controller's Enable
held past it included. Enable presses released while the delay is 0
are not played once it is turned on, user 1's included.
* Input: Plays As, a controller's own player, places it on connect
Sixteen entries, input_plays_as_device_N and input_plays_as_player_N,
name controllers as a reservation does, vid:pid then name, each with a
player (0 Auto). When they are read, each connected controller, lowest
slot first, takes the first entry naming it that no other has taken; a
0000:0000 vid:pid matches by the name after it, not every controller
without one. Giving a controller a player makes an entry for it,
refused when all sixteen are in use; set back to Auto, its entry is
freed.
The entries are read with the binds and written with the input
entries, a free entry's keys taken out. Two of a model are told apart
only by order, so a model's entries are written at their own places
in the order a start hands them out: lowest slot first, a gone one's
in its place, and one on Auto ahead of another of its model as an
Auto entry of its own. Once the main configuration is saved, the
entries in memory are laid out the same way, so reading it again in
the session gives each its own too. Saving an override puts them back
after its comparison load, as it does the binds. The reservations'
vid:pid parse is shared with them, unchanged.
On connect, a controller set to player N takes port N. Whoever else is
there moves to the first free port, unless it also plays as N, by its
own player or, on Auto, by N's reservation naming it: then the
newcomer shares, from the highest free port below Max Users. An Auto
controller that N's reservation names shares the same way when port N
holds a controller set to N. An explicit player beats a reservation
naming the controller. With any reservation set, Auto controllers are
otherwise left to the reservations; with none, and a player set by an
entry, they take the first free port. With no player set nothing
changes.
The moves are for this run only, through input_driver_exchange_ports(),
now in every build: the configuration keeps the ports the user set. A
reservation's own placement made while the ports are in a runtime
layout is runtime as well, so it is not saved either; reservations
place again at every connect.
A joypad driver restart lets go of every claim, since its pads can come
back in other slots with no disconnect: a pad reported again where it
is plugged in takes back its entry, the rest take theirs in slot order.
While a player is in effect, the registry's restore of the ports is
left to placement. A config reload keeps each controller's entry while
it still names it, so two of a model keep theirs. The entries count
among the settings whose change starts the joypad driver again at a
video restart, and so does a config reload that undid runtime moves.
* Input: a controller playing as another's player shares its core port
A user whose controller plays as another user's player feeds that
user's core port through the port map: that user's Mapped Port, from
remap files or identity. Its own Mapped Port is never changed or
saved. Everything that reads a user's core port reads this effective
port, and the Quick Menu's Mapped Port row reads "Shares Player 1 (core
port 1)" for a sharer.
An Auto controller placed to share with the player a reservation names
keeps sharing while it stays connected, also once that player's
controller leaves. The poll rebuilds the map when who shares changes,
and the core is told its controllers again between frames, not while
netplay or a replay owns the ports. Under First Press a press on any
of a player's controllers gives that player's user its port.
* Menu: a controller's player in Input > Controllers and Port N Controls
Settings > Input > Controllers, after RetroPad Binds, has a row per
connected controller in port order, then those past Max Users, then
the Plays As entries of controllers that are gone. A row's value is
the controller's player or, on Auto with a port, "Auto (1)", the player
Auto gives it. The sublabel names the player first and a controller
port only by its page name: "Player 1", "Player 1, shared · binds in
Port 3 Controls" for a controller sharing a player from another port,
"Not playing: more controllers than Max Users", or "Not connected".
On a Controllers row, Left and Right step through Auto, 1 to Max Users;
OK opens them as a list. RetroPad Binds > Port N Controls starts with a
row for each connected controller playing as the player port N feeds,
the same setting, then Add Controller: the other connected controllers
with a port, each with the player it plays as now, where a pick moves
that controller to this player. One controller on several players stays
Device Index's.
A change to a connected controller marks the configuration changed and
places it again as its connect did, with any controller it pushes off a
port, and no driver restarted. The entry point is in task_autodetect.c,
beside the reservation placement it reuses. A controller whose change
left no player in effect goes to the first free port. While Plays As is
in use, a change of Max Users places again the controllers left with no
port and those whose player comes within it. With every entry taken a
new one is refused with a notice.
A controller set to a player above Max Users reads "Above Max Users:
plays as Auto (Player 2)". On Port N Controls a row's sublabel reads
"This port" or "Shared · binds in Port 3 Controls". Device Index's note
for a controller another port reads becomes "used by Port 2".
After Add Controller, Port N Controls links to Port M Controls for each
controller playing as player N from port M, such as "OpenXR Controllers
(Port 4 Controls)": that controller's binds are port M's. Quick Menu >
Controls > Port N Controls starts with the same links to the pages that
hold its remaps. Both lists and the Quick Menu's Port N Controls are
made again on hotplug, and Port N Controls after a Device Index change,
with the cursor kept on its row.
The menu lanes are in stereo_headset_test, which already links the
shipping menu objects and runs in CI's OpenXR jobs; a build without
OpenXR skips it.
* Input: a controller's own hotkey buttons
A Plays As entry keeps a button and an axis of its own for any hotkey,
input_plays_as_<hotkey>_btn_N and _axis_N, written only where set; an
entry that has one is kept on Auto. It counts neither as a player nor
among the settings whose change starts the joypad driver again. A
sharer reads its own first, then its profile's, never the set's; the
hotkey port's own controller reads its own, then the set's, then its
profile's. Button and axis go each on their own.
The bind values, "8", "h0up", "+1", are read and written by helpers in
input_driver.c that the configuration's own binds now use too.
* Input: Hotkey Enable pairs with the controller it is on
A sharing controller's hotkeys go by its own Hotkey Enable, its bind or
its profile's, whatever the hotkey user has: one with an Enable fires
while it holds it or the keyboard's, one with none fires freely. With
Hotkey Device Type Merge on and a keyboard Hotkey Enable bound, a
sharer with none waits for the keyboard's, as master makes user 1's pad
wait. Holding a sharer's Enable holds game input back as any Hotkey
Enable does, and opens only that sharer's hotkeys.
Keyboard and mouse hotkeys keep master's Hotkey Enable rule whatever
sharers are connected: a sharer's Enable does not open them. Nor does
it open an input_combo_N entry with a key; an entry of RetroPad buttons
alone opens with it, as those buttons are every port's. A sharer with
an Enable of its own needs it for the menu toggle too, and whether a
pending menu tap is kept is read from the sharer's own binds, not the
set's.
The hotkey user's pad Enable plays no part in a sharer's hotkeys: each
controller's hotkeys are its own, and so is its Enable.
With Hotkey Enable Tap Delay, the hotkey of a sharer with no Enable of
its own leaves the keyboard's tap alone, as the keyboard's Enable did
not open it. Under Hotkey Device Type Merge, where such a sharer waits
for the keyboard's Enable, its hotkey swallows the keyboard's tap.
* Menu: hotkeys bound from a sharing controller are its own
Binding a hotkey also listens to the controllers that fire the hotkeys,
those on the hotkey user's core port, and its prompt gains "Listening
to Player 1's controllers", for the player the hotkey user plays as. A
press on the keyboard, the mouse, user 1's pad or the hotkey user's
goes to the set as before; a sharing controller's press is kept as that
controller's own, with an entry made for it at its player, refused with
the full notice when none is free. A press from a controller that has
its own bind for the hotkey replaces that bind, user 1's pad included.
Any bind, RetroPad binds too, also hears the port's own controller when
it is in a slot at or past Max Users, where Plays As placement can put
it, so that port's binds can be set from it.
A hotkey's row shows each connected controller's own after the set's,
"Select · OpenXR Controllers: L3", and clearing it clears theirs too.
* OpenXR: the headset shares player 1 on Auto and moves no one
For the headset's controllers, Combined or each hand in Separate, Auto
means sharing player 1, so a VR player's pad and headset both drive
player 1 with no setup. They take the highest free slot and the highest
free port below Max Users, so pads keep the low ones, and share their
player whether anyone is on its port or not, moving no one; a pad
turned on later lands on port 1 and shares with them. Set to a number,
they do the same with that player. Every other controller's Auto takes
the first free port.
A reservation naming the headset beats its Auto, as for pads, and a
number beats a reservation. A pad set to the player whose port a
headset on Auto holds takes that port, and the headset moves to the
highest free port, or gets none. Any change of Max Users places the
headset again at once, on the highest free port. Wherever Plays As asks
whether a player is in effect, a connected headset on Auto counts as
player 1, as one set to 1 does.
OpenXR looks up each controller's player only when the devices or the
entries change, and places a controller again when it starts or stops
playing as a player, or when a slot below Max Users frees up for one
placed past it. A player's controller that a pad pushed out of its slot
is placed by its player rather than taking the port with it. A headset
entry names the controller by the name OpenXR looks it up by.
The laser answers the core once, for the core port the headset's
controllers play on, so it no longer mixes with another user's mouse;
in Separate each hand's player gets it, the pointing hand aiming and
firing and the other reading offscreen. A headset controller with no
player drives no pointer, and the Monado e2e cases follow the new
placement.
mapped_port_test is built a second time with OpenXR, as
mapped_port_openxr_test, and CI runs both.
* Menu: OpenXR Controller Layout under Input > Controllers
Combined or Separate is how the headset's controllers appear, so it
moves from the headset's video rows to the bottom of
Settings > Input > Controllers, named for what it sets. The key and its
values are unchanged. Switching it reconnects the headset's
controllers, and the cursor stays on the row.
* Input: Plays As in core, content directory and game overrides
Save Override now saves the controllers' own players and hotkeys. Each
controller whose player or own hotkeys differ from the main
configuration's (with --appendconfig) has its whole entry written, all
of its entries for two of a model, in the order a start hands them
out, one on Auto ahead of another of its model as an Auto entry; one
the main configuration gives an entry and the session none is written
as Auto. An entry the same as the main configuration's is not written,
so a controller set back drops out of the override.
An override's entries are read on their own and go over the ones read
before by controller, as entries are matched to controllers (vid:pid,
else name), not by number: the last file naming a controller gives it
its whole entry, in the place of the one it had, two of a model taking
a file's entries in order, an Auto one included, the connected ones
lowest slot first; a controller no override names keeps the main
configuration's. Auto takes one back to Auto, and a controller new to
the entries takes a free one, if any of the sixteen is. Until now the
files' keys merged one by one, so an entry could pair one file's
controller with another's player, and no override could take an entry
back to Auto.
Loading or unloading an override from the menu, or loading the active
ones again, places the controllers again at once, as a Player change
does: those whose player changed, or all of them where the reload read
the configured ports back over the moves. Content load and close leave
it to the joypad driver's restart, as before.
mapped_port_test covers the merge, what is saved and the placing
again; video_screens_test saves, unloads and loads a game override
with a Plays As change.
* Input: a controller keeps its Plays As entry while its twin is unplugged
Two controllers of one model share the entries that name them, each
taking the first one no other connected controller holds. With one of
the two unplugged, anything that made the other take an entry again
could hand it the unplugged one's: a Player or Max Users change,
loading or unloading an override, any re-read of the main config, a
joypad driver restart, and Online Updater > Update Controller
Profiles, which reports every connected controller again. The
controller left then played the other one's player, and that one came
back to its twin's.
An unplugged controller's entry now stays its own for the rest of the
session, and only a connect takes it: that controller coming back, or
another of its model. A reload keeps it while the entry in its place
is still of its model, and an override's entries pair with it as if
the unplugged controller still held it from its slot. So an override
loaded with a twin unplugged gives the other twin the entry it adds
and leaves the unplugged one its own. Save Configuration and Save
Override count it as that controller's too, so what they write pairs
the same way when it is loaded again.
Placed again for a Player or Max Users change, or by Update Controller
Profiles, a controller keeps the entry it holds, or none. Reported
again by its driver while still connected, it keeps the one it holds.
A reload is judged as a whole once its last override is read: a
controller takes an entry it did not hold before only where the
reload added it for its model, so one an override newly names still
gets that entry.
A joypad driver restart still lets every claim go and hands the
entries out again in the order the controllers are reported, as a
start does, but its reports leave an unplugged controller's entry
alone. A driver's report queued behind a profile update's is not
dropped as a repeat of it, since the update takes no entry, and the
update it supersedes shows no toast, so one shows, as before.
* OpenXR e2e: FRAME_BINDS catches up with the grip poses already bound
Every profile binds pointer/grip to both hands' grip poses, and
input-frame-profile matches FRAME_BINDS exactly, so the table now lists
the poses as they are bound.
* Menu: OpenXR touch settings on Input > Controllers
Touch Hands, Touch Button and Touch Passthrough are added for the hands
that touch a screen they point at, the button that touches, and whether
that button's other binds fire too, with Touch Hands None to turn touch
off for the headset's controllers; a later commit puts them to use.
They follow OpenXR Controller Layout, Laser Pointer and Laser Beam,
which move here from Stereo 3D & Screens, at the end of Settings >
Input > Controllers, shown only for a video driver with a headset path.
A hotplug keeps the cursor on any of these rows, and overrides save
them per game.
* OpenXR: the headset's menu button is button 16, bound to Menu Toggle
The menu action of each profile presses button 16 of the pad, the left
hand's in Separate, instead of being a hotkey of its own, and the
built-in profiles bind Menu Toggle to it and label it "Menu", as pad
profiles name their Guide or Home button. It opens the menu from the
same button as before and can be bound elsewhere like a pad's Guide
button. Touch's one menu button presses button 16 once held a second.
In Edit Screens the menu button always opens the menu, whatever is
bound and whether or not Hotkey Enable is held.
Sharing player 1, the button goes by Hotkey Enable as a sharer's Guide
does where its profile has none: it opens the menu whether or not
player 1's Hotkey Enable is held, and the enabler alone opens nothing.
mapped_port_test covers both.
With the menu open, a press of the controllers' buttons hides the
laser's dots until a trigger is pulled, so hand drift cannot undo their
navigation. The menu button navigates nothing, and with no content Menu
Toggle leaves the menu open, so its press does not count.
* OpenXR: the Touch Button's buttons as pointer actions of each hand
Grip, stick click, bumper and A to Y each have an action per hand in the
laser's action set, bound from a table per profile in the same
suggestion as the pad's rows, so either controller layout can read them;
the trigger is the pad's own. The Frame's exact binding list in the e2e
gains them.
A runtime that refuses a suggestion loses its palm first, then these
rows, then edit mode's, a smaller loss tried before a larger, and keeps
the rest of the profile. INPUT_OPENXR_MAX_BINDS goes from 48 to 64 to
hold these rows with edit mode's (Index 52, Touch 51, the Frame 58); a
profile whose pad and Touch Button rows pass it is not suggested at
all, and the log names it.
* OpenXR: touch with the chosen button and hands
The Touch Button, read from its own action or, for the trigger, from the
pad's, touches a live screen the hand points at; the trigger still
clicks the menu, and is L2 or R2 on a screen once it is not the Touch
Button. A hand touches only if Touch Hands has it and its controller has
the button, as read from each hand's interaction profile; only such
hands aim at the screens, become the pointer or show a dot there, and
with none the screens stay the mouse's. The others aim at the menu
alone, which video_xr_pick() now takes as a flag. Touch Passthrough lets
the trigger's L2 or R2 through its touch.
A hand takes a changed setting at its next press. A button held through
a change of Touch Button, Touch Hands or the hand's controller profile
is no press: the trigger goes by its own last read, and any other button
counts as held until it is let go.
* OpenXR: a touch holds back its button's other binds
OpenXR gives an input to every action bound to it, so a Touch Button
other than the trigger also reaches the pad. While it touches, or a
press of it would, the pad's buttons bound to the same input of that
hand in the profile's rows are let go, in Combined and in Separate: the
Index grip's L or R, the Frame right stick's menu button. Touch
Passthrough leaves them pressed.
With the menu open, a press of the controllers takes the laser from the
dots until a trigger pull. Like the triggers' L2 and R2, what a Touch
Button's input feeds does not count as such a press, with Touch
Passthrough on or off, so the button can still touch a live screen
there. Another button on the same hand still counts.
* Menu: Touch Button offers the buttons the controllers have
The dropdown lists the buttons either hand's controller has, as its
interaction profile names them, and the one chosen; left and right step
through the same, skipping the rest. A choice the controllers lack, as
from an override made on another controller, reads "(not on this
controller)". With no controller known every button is offered. The poll
publishes those buttons through an atomic, since with threaded video XMB
draws the value on the video thread.
* docs: OpenXR controllers, touching and the menu button
Where the headset's input rows are now, what Touch Hands, Touch Button
and Touch Passthrough do, None included, and that the menu button is a
Menu Toggle bind that Hotkey Enable applies to as to a pad's Guide; on
Touch controllers in Combined it is Start, and button 16 once held a
second. Laser Pointer's description says the Touch Button touches a
screen and the trigger clicks the menu, true with Touch Passthrough on
too.
Frames compressed with a dictionary were refused. rzstd_dict_new()
takes one as the reference trainer writes it - ID, the literals'
Huffman tree, the three sequence tables and the recent offsets, then
content - or plain content, and rzstd_decode_dict(),
rzstd_decode_reuse_dict() and rzstd_encode_dict() use it. A frame
starts from the dictionary's tables and offsets, and its matches reach
back into the content through the path the block ring already takes
for a window that has come round. The encoder searches an index of the
content built once with the dictionary, starts from its offsets and
names it in the frame header; its blocks write tables of their own.
A dictionary is only read once built, so threads can share one.
The other entry points still refuse a frame that names a dictionary,
now in the frame's begin rather than its header reader, so
rzstd_frame_content_size() reads such a frame's size.
rzstd_dict_test decodes frames the reference compressed with a trained
dictionary, which need its tables, and refuses them without it or with
another; and round-trips the encoder's frames at every level with
either kind of dictionary.
A call that ran out of output while the trailer was being written came
back into the block loop on the next call, parsed the finished input
again and emitted another, empty, final block before the trailer: the
stream decoded wrong, and with one byte of output at a time it never
ended. The zlib and gzip headers were only written when the whole
header fit, so an output smaller than the header made no progress at
all. Both now resume where they stopped.
rinflate_window_test compresses through outputs of 1 to 7 bytes in
each container and checks the result is the one-shot stream.
Windows views under Wine failed every d3d9_hlsl exact-3ds case run
through a preset on the core's Direct3D 9 texture with a black screen,
since e2c75d8. The view copied into the chain's frame was right, read
back texel for texel, but d3d9_hlsl_views_hw_copy() then stored the
view's size as the slot's last_dims: the first pass took that to mean
the slot's vertex buffer was already made for that size and never
wrote it, so it drew from an empty buffer. The copy no longer stores
it; the first pass makes the vertices when the size differs and the
chain's end records it, as for a software frame.
Under Wine on 26.04 the five hardware preset cases pass, and the whole
d3d9_hlsl lane on Direct3D9Ex and on the legacy device.
On Ubuntu 26.04's llvmpipe (LLVM 21) the blur shader read in place
from a hardware-ordered frame came out 33 to 54 bytes apart from the
copy, each by one step, and glsl_in_place_test's three linear
comparisons did the same: its reads land on texel edges, where
glsl_rect.h already says a GPU compiling the other program may weigh
the texels one step apart. Linear comparisons now pass within one step;
nearest still has to match byte for byte. Both tests pass on 26.04.
Since 5bf0c36 the first-run download prompt is offered when the assets
folder does not exist yet, as in every fresh config these tests write,
and it comes up as a confirm message over the menu, which then holds
the menu's input. stdio_e2e's MENU down left the selection on 0, and
the views e2e's pointer-sbs-full cases on x11 and sdl3 found the menu's
cursor still at the first pointer position. Both configs now set
rgui_show_start_screen false, which the welcome screen and the prompt
both hang off.
On 26.04 stdio_e2e passes and the x11 pointer cases pass, both
pointer-sbs-full cases failing without it as on CI.
Discard pictures that expire before the next presentation deadline, then retain one covering that deadline. Keep speculative durations and ordinals private until presentation, freeze retry choices, and clear pending progress on seek, restore or renderer shutdown.
Account for cached and prefetched leases alongside reordered VA reads within the existing 32-surface bound. Expose a read-only upcoming-frame end on the stream pacing clock, with checked arithmetic and portable timing, catch-up and save-state regressions.
Let VA report its nonblocking writable mask and spare surface capacity before preparing a decode target. Preserve prepared retries and distinguish physical ownership from DPB pressure.
Opt the MP4 core into consumed-input progress and hold one future HEVC picture until its interval is due. Keep public clocks and WPOS state on the displayed picture, releasing speculative leases on seek, reset and renderer shutdown. Legacy stream callers retain their retry behavior.
WPOSv3 cannot encode a pending user seek target. Return false without writing until scanning and decoding finish, while allowing pending state restorations and already decoded targets to serialize normally.
Keep requested movie positions outside video intervals, clamp audio to its declared presentation edit, and stop emitting after shutdown. Check exact remaining PCM and the final picture on CPU and native hardware paths.
Select the presentation slot in movie coordinates, accounting for initial decoder PTS and leading edits. Keep raw decoder seeks unchanged and compare reordered H.264/HEVC core seeks with sequential picture references.
Clear the cached movie with an intentional blank frame during a leading video gap, including hardware state restores. Preserve the first decoded picture when the gap expires and keep required hardware decoding separate from blank presentation.
Retain the restored display ordinal and serialized clocks while a hardware decoder catches up. Retry without advancing playback, preserve pending restores across renderer replacement, and cancel them on reset or a newer seek.
Advance discarded hardware pictures without exporting output leases. Rotate free decode slots so an asynchronous native write does not immediately block reuse of the first free slot. Preserve normal output and busy retry behavior.
Keep movie-coordinate seeks pending until the requested picture is presented. Preserve the request across decoder and importer backpressure, cancel it on reset or a newer seek, and restart it after renderer replacement.
Keep the MP4 clock steady while a selected hardware decoder waits for its first picture, including codecs with a software fallback. Cover decoder and output backpressure in the HEVC core ownership test.
Keep the ready hardware picture available when the following packet cannot decode yet. Its lease remains valid until replacement or transfer to the caller; only decoder-owned planes expire on a decode attempt. Cover a held H.264/HEVC picture followed by decoder and output backpressure, including single transfer and balanced release.
Destroy stream and VA import resources that remain attached to frontend-held texture handles. Preserve the handle allocation through immediate or deferred retirement so reset neither leaks device children nor frees a handle still owned by the frontend.
Retain the submitted image until presentation resumes and advance offscreen frame slots only after their fences complete. Poll busy slots without adding a wait or reusing the submitted fence and binary semaphore.
Use the measured DMA-BUF size for import when it covers the VA descriptor and image memory requirements. Intel exports can report less storage than the kernel backing object; continue refusing undersized objects.
Intel rejects multiple elements outside slice-parameter buffers. Pack the entry offsets into a single subset buffer while preserving the validated coded offsets.
Submit bounded HEVC long-slice metadata through an owned VA session and export native surfaces to a matching Vulkan DRM device. Retain producer and renderer leases through asynchronous submission and device teardown, and refuse unsupported import combinations before publication.
Preserve sampled YUV chroma positioning in GPU conversion and cover decoder, import, retirement and threadless contracts.
A core rendering with the Direct3D 9 interface had its texture drawn
whole through a preset, its views ignored: each view's chain only took
a view of a software frame, written straight out of it into the
chain's own texture. Now each view of the core's texture is copied on
the GPU into its chain's frame, with StretchRect, point sampled, from
where it lies to the corner: the texels as they are, one copy a view,
with no trip through the CPU. The chain's frames are then render
targets in the core's texture's format, made with the chain and black
past the view, filled black again when a smaller view lands in a slot
that held a larger one. They stay its history, so a preset reading
earlier frames reads each view's own.
A chain made for one kind of frame is made again for the other. A
frame without a texture handed over draws no views.
samples/gfx/d3d9_ex checks the copy on a real Direct3D 9 device and a
Direct3D9Ex one under Wine, texel for texel and black past the view,
with a smaller view after a larger one. e2e/run.py now runs the d3d9
hardware case through D3D9's own presets too, earlier frames
included, matching the software frame's screen, and checks the log
says the views were copied out of the core's texture.
A core rendering with the Direct3D 9 interface (libretro_d3d9.h) had
its views ignored: its texture was drawn whole, every screen of it as
one picture, and the views fallback reported. Without a preset its
views are now drawn straight from the core's texture, where they lie,
as a software frame's are from the texture it is uploaded into: no
copy. The texture is unbound after, as the frame's own draw leaves it,
for the core to draw into next. Through a preset, whose chains each
take a view of a software frame written straight in, the core's
texture is still drawn whole.
The video_views test core gets a d3d9 mode for video_views_test_hw: it
writes its frame into a system memory surface, copies it with
UpdateSurface into a render target texture per sync index (the
regular maximum's size with video_views_test_max large) and hands it
over with set_texture, taking the device only between lock_context and
unlock_context. e2e/run.py runs the 3DS cases on d3d9_hlsl from that
texture - 2D, side by side, anaglyph, interlaced and threaded - and
the pixel-for-pixel 3DS case, which must match the software frame's;
the Windows views workflow under Wine runs them, plain and on a device
without Direct3D9Ex.
task_push_dbscan() takes a third argument, user data, and the
cancelled-scan case added with the arcade set fingerprints called it
with two, so the test no longer built.
A glcore or Vulkan preset reading earlier frames (OriginalHistory1 and
on) kept copying each view of a hardware frame out to a texture of its
own, then copied that texture again into the history at the frame's
end: two copies a view a frame. Its passes can read the view where it
lies like any other preset's; only the history needs a copy, and that
copy can come straight from the view's rectangle. So such a preset now
reads in place, and the history takes the one copy:
- Vulkan: vkCmdCopyImage from the view's origin in the frame, no longer
from the corner of a copy. Copies of the same texels, the same size.
- glcore: the history's quad copy samples the view's rectangle in the
frame, as the copy out of a larger texture already did, nearest at
texel centres.
gl3_framebuffer_copy_partial() made, filled and deleted a vertex buffer
on every call; the chain now makes that buffer once and the copy writes
its quad into it, so neither the history copy nor the copy out of a
larger texture allocates mid-game.
samples/gfx/gl3_shader_status and samples/gfx/vulkan_chain_alloc check
that a chain with history reads in place, that the frame is not copied
out first, and that the history copy comes from the view's rectangle
(and, on glcore, through the chain's own buffer); both fail on the old
code.
The null input driver had no keys_down, so the frontend took the slow
fallback for it: every frame it asked for every hotkey and every
user's turbo and hold bind one at a time, since it could not know that
none of them can be pressed. With keys_down saying no key is down, the
hotkey pass and the poll skip those reads, as they do for the drivers
that give their keys.
Callgrind over the frontend harness's input-cost lane, which runs on
this driver: 40% fewer instructions in all, 83% in the hotkey pass and
50% in input_driver_poll. Every lane gives the same results.
The harness mocks that copy the null driver to answer keys through
input_state now clear keys_down, so they still test that kind of
driver.
Four kinds of shader kept the copy of a hardware frame's view; each now
reads the frame where it lies.
textureOffset and textureLodOffset: scalefx and cubic read the frame a
whole number of texels away. The helper takes the offset in the view's
texels: nearest adds it to the texel picked before wrapping, as a
sampler does, linear moves the coordinate by it before clamping.
RARCH_frame(c) is RARCH_frame_at(c, vec2(0.0)), so other reads come out
as before; RARCH_FrameTexels.zw carries 1 / the view's size.
Overloaded functions given the frame: crt-royale's tex2D_linearize
comes in float2 and float3 forms. A call goes to one of them, so when
every call gives the frame at a place and every form takes a sampler
there, all of them carry it; one form that can't keeps the copy for
all. Which parameters carry the frame now grows from the calls that
pass it, so functions that only call each other carry nothing.
A function reading the frame ahead of its declaration: crt-royale
declares its input after its helpers. The declaration, when it is the
only one, moves up to just before the first function reading it, and
the helper with it.
An undef of a read's name: crt-royale undefines COMPAT_TEXTURE and
defines it again. Between the two it means nothing, so it is only kept
from the rewrite when code declares a function of that name.
A define's own parameters no longer count as uses of a function's
parameter of the same name, so COMPAT_TEXTURE_LOD(s, c, l) does not
keep a function taking a sampler s from carrying the frame.
glsl-shaders: 661 of 684 shader files rewrite (647 before), and 510 of
621 presets read in place (493 before); 100 of the rest read earlier
frames, which only the copy keeps. Every newly covered shader matches
the copy byte for byte on llvmpipe, desktop GL and GLES 3; nothing that
matched before changed. samples/gfx/glsl_rect adds offset and early-
declaration shaders to the byte-for-byte oracle, and overload, undef
and define cases to the rewrite checks.
Follow-up to the first-run download prompt (PR #18878).
- Downloads go through action_ok_download_generic (new public wrapper
action_ok_download_bundle) instead of a copied version of it. The
copy predated the stream-to-disk sink, so a first run held the 72MB
assets zip and the 52MB database zip in RAM, and skipped the check
that refuses to truncate an archive still being extracted.
- The free-space check asked about the target directory, which on a
real first run does not exist yet: statvfs failed, -1 came back, and
assets and databases were never offered. It now asks the nearest
existing parent, and an unknown answer no longer blocks the download.
- The prompt is an ordinary msg enum through menu_dialog_confirm_set.
The 512-byte confirm_text buffer in menu_dialog, the second confirm
setter and the extra branches in the iterate path are gone.
- Missing bundles are scanned once at menu init into a bitmask that the
confirm handler uses; directories are not walked a second time.
- Offered only with HAVE_CONFIGFILE, the only first-run signal there is;
dead have_bundled_assets logic removed.
- path_get_free_space: GetDiskFreeSpaceExA resolved at run time (not in
the first Windows 95), GetDiskFreeSpaceA fallback, no <windows.h> on
Xbox, statfs on Android (statvfs needs API 19), statvfs only on
platforms known to have it instead of every __unix__ target.
- path_is_empty_directory: dotfiles (.nomedia, .DS_Store) do not count,
the open call does not filter them outside Windows; documented.
- Ozone caches the Cancel label with the other footer labels instead of
measuring it every frame the dialog is up.
- Pointer: one close, then confirm or cancel.
- Prompt text no longer promises overlays on platforms that skip them.
A GLSL pass reads unit 0, its input, through any sampler RetroArch does
not bind by name: every sampler links with the value 0, and only names
ending in Texture (and look-up textures' ids) are ever set. Shaders
such as jinc2, crt-hyllian and xbr-lv2 read the frame that way, as
s_p or decal, and so kept the copy; worse, such a sampler in a later
pass kept the copy for the whole preset. Such a name is now the frame
in the first pass and the pass before's output after it, as GL has it.
With two names for the frame in one stage (Texture and s_p, or Texture
and OrigTexture in the first pass) the rewrite put its helper in twice
and the program would not compile; it is now guarded, so the first
declaration's helper serves both, which read the same unit.
A function given the frame as a parameter had its body's reads
rewritten, but a define naming that parameter and expanded in the body
(crt-aperture's TEX2D reading tex) read the whole texture instead of
the view. Such a parameter now keeps the copy, unless its name is the
frame's own everywhere, as adaptive-sharpen's Source is.
glsl-shaders: 647 of 684 shader files rewrite (630 before), and 493 of
621 presets read in place (439 before); on llvmpipe, desktop GL and
GLES 3, every newly covered shader matches the copy byte for byte.
samples/gfx/glsl_rect tests the unbound name, two names in one stage,
a look-up texture's name left alone, and both define cases.
The overlap-copy check flagged three same-base copies, all disjoint:
rinf_build doubles its fast and pfast tables in place, copying entries
[0, cur) to [cur, 2cur), and video_windows_test copies place_d[1].name
into place_d[2].name. memmove would only slow the table build the
in-place doubling was written to speed up, so they are allowlisted.
"states that never hash alike" failed once with the host seeing the
client's stick on none of 1513 frames, and did not fail again in 18
runs here. A failing lane now prints how often the client's own stick
moved, which shows whether its network gamepad was read at all, and
the device ids each core was given, which show whether the ports were
the analog joypads stick sync needs; the network gamepad's [Network]
and bind lines join the log lines it prints.
A preset whose passes read the original frame (OrigTexture, or the
PassPrev that names it) kept its views' copies. Every pass that reads
the frame - the first, and each of those - is now rewritten for it,
swapped in together for a view's draw and given back after. A preset
reading earlier frames still keeps the copies those need, as does one
whose passes name a sampler nothing binds.
Of libretro's glsl-shaders presets reading the original, 45 of 94 now
read in place.
A hardware frame's views went through a GLSL preset as copies, each
view drawn out of the frame into a texture of its own first. The
preset's first pass is now rewritten, once for its filter and wrap, to
read each view where it lies in the frame instead, as it would read
the copy: the same texels, filtering and edges, top row first.
The rewrite (glsl_rect.h) follows every read of the frame on the
source text - by its names, macros naming it or a read, and function
parameters every call gives it - into RARCH_frame(), which maps the
coordinate into the view and clamps or wraps it as the copy's sampler
would. Anything it can't follow, a pass reading the original frame or
earlier ones, a sampler it doesn't know, or linear filtering with
repeat, mirror or border keeps the copy. The rewritten program is
swapped in for each view's draw and given back after.
Of libretro's glsl-shaders, 633 of 684 files rewrite. On llvmpipe, 543
of the 612 cases that draw match the copy byte for byte; the rest
differ where a sample falls exactly on a texel edge or a filter weight
on a rounding step, by one step except in shaders that threshold such
values (hqx pass 1 linear, nnedi3 nearest).
samples/gfx/glsl_rect checks the rewrite and draws each filter and
wrap both ways on GLES 3 and 2; gl2_window_ui checks the views go
through it with no copy.
The 'Scanning content: "<dir>"' log line (and its CLI printf)
only ran on database-matched scans, so a plain manual scan with no
database logged nothing. Move it below both branches so every scan
that starts logs the folder it is walking.
input_state_wrap_slow asked input_pointer_frame_read() before falling
through to the keys and the driver, for every device. It answers only
for the mouse, pointer and lightgun, so for the joypad, analog and
keyboard reads that make up nearly every slow read it was a call that
always said no. Those devices no longer ask.
Measured with callgrind over the frontend harness's input-cost lane:
10% fewer instructions in all, 16% in input_driver_poll and 17% in the
hotkey pass. Every lane gives the same results.
A set with no clones has the same merged and split fingerprint. Both
keys indexed the record twice, so the lookup returned it twice; with
its size equal to the archive's that read as two sized records and the
set fell through to a whole read. The second key is now skipped when
it equals the first, which also keeps the index smaller.
The fingerprint's CRC list sits on the stack for sets of up to 64
members, so most archives no longer allocate for it.
rar names no member it cannot unpack (solid, encrypted, an older
packer), so a fingerprint taken over the members it does name would
claim a set that may not be all there. Such an archive now has none
and goes on to its other passes.
7z and rar also name no folder, which would look the same, so the
scan's directory walk asks them to name one as zip does, ending in
'/'. Nothing else sets the flag.
The arcade databases key each set on the CRC of the whole
TorrentZipped archive, so the same ROMs packed any other way - zstd
or 7z, other names or order, a merged parent - never match.
An optional set_fp1 key holds the CRC-32 of the set's ROM CRCs,
de-duplicated, sorted and taken as big-endian bytes; set_fpm1 holds
the same for a merged parent. The archive's one directory walk gives
its fingerprint, leaving out folders, empty members and cue or gdi
sheets; a member that stores no CRC leaves it without one. Each
database whose cores open the archive's type and that carries the
keys is looked up through an index built like the crc one. A hit
writes the record's entry, as a match on the archive's own CRC does,
and no archive is read whole for it.
An archive of several members is looked up by its fingerprint before
its size is. An archive of one member is looked up by it only after
its member's CRC missed, so a first-member hit held for a cartridge
database still wins.
Where several records of a database share the fingerprint, as a set's
split and non-merged packings can, the one whose size is the archive's
is taken; where several have that size the archive's own CRC decides,
and where none does, the one named like the archive, else the first.
The readers read a TorrentZip set whole with no grant only while no
database that opens zip carries fingerprints, so the fingerprint
indexes of those databases are built before the readers start. One
that cannot be built counts as carrying none: that database can match
the set only by its own CRC. A database without the keys costs one
walk per scan; older versions skip the keys.
libretro-database can give the arcade records a set fingerprint: the
CRC-32 of a set's ROM CRCs, de-duplicated, sorted and taken as
big-endian bytes, and the same for a merged parent. Map both keys in
c_converter and the Lua converter so the rdb keeps them. A converter
without these rows drops them, as it drops every key it does not map,
and readers that do not know them skip them.
The lexer matches punctuators by their first byte, skips whitespace,
comments and splices in one pass, and hashes identifiers as it scans them;
node chunks are no longer zeroed ahead of use. The same output in about
38% fewer instructions.
DXBC declares a uniform block only when the code reads it, as fxc does, so
a binding the code never names is not one the root signature or pipeline
layout must have.
These files are now the same as parallel-gamecubewii's copy.
* vulkan: add the required extensions a v1 core leaves out
A core on context negotiation interface v1 makes its own device in
create_device() from the required_device_extensions RetroArch passes.
Under XR_KHR_vulkan_enable those include the OpenXR runtime's, and a
core that ignores the list (Flycast) leaves them out, which crashes
SteamVR in xrCreateSession.
The core now gets a vkGetInstanceProcAddr whose vkCreateDevice adds
the required extensions its create info left out, and a warning names
them for the core's developers. A core that makes its device some
other way under XR_KHR_vulkan_enable can't be vouched for, so headset
output is off for it.
The Vulkan validation job runs the v1 harness core with the required
extensions left out and checks for the warning.
* vulkan: make a v1 core's device the way RetroArch makes its own
The vkCreateDevice a v1 core gets now goes through
vulkan_context_create_device(), so its device also gets the temporal
upscaler's features and, under XR_KHR_vulkan_enable2, is made by the
runtime; a core that makes its device another way loses headset output
under either OpenXR interface. Required extensions the core's GPU lacks
are no longer forced on, which matters for a core that picks a GPU
other than the one offered, and a core that looks up
vkGetInstanceProcAddr through the one it was given still reaches the
wrapper.
The validation job checks that a core passing the required extensions
draws no warning, and that one resolving vkCreateDevice through its
own vkGetInstanceProcAddr lookup still gets them added.
---------
Co-authored-by: libretroadmin <[email protected]>
A preset's views were read where they lie only from a core image of one
level; one with mipmaps had each view copied out of its top level. Read
those in place as well, through a view of that level alone, made once
a frame and destroyed when the frame's slot comes round, since the core
need not keep the image past the frame. A core view made with chained
structures still takes the copy.
Every Huffman symbol tops up the entropy bit buffer, and the refill
took bytes one at a time: a compare against 0xFF, a variable shift,
an OR and a bound check per byte, about five bytes per refill on the
64-bit buffer. On a 4K JPEG that was 12-15% of the decode.
The refill now loads the next eight bytes as one big-endian value
and tests all of them for 0xFF at once (a zero byte in its
complement). When there is none - nearly always, in entropy-coded
data - it inserts exactly the bytes the byte loop would have taken:
from code_bits that is k = (FILL - code_bits) / 8 + 1, and the k bytes
read as one number land with one shift at the positions the loop
would have put them byte by byte. The buffer, the bit count and the
read position come out identical, so everything downstream - markers,
restart intervals, the prefix decoder's checkpoints - sees the same
state. An 0xFF among the eight, fewer than eight bytes left, the
24-bit prefix-decoding target and 32-bit bit buffers all keep the
byte path. The value is assembled from bytes with shifts, which GCC
and Clang compile to one load and a byte swap, and which is correct
on either byte order.
Verified: decoded output identical to before - the 200-file corpus
(baseline and progressive, every sampling, gray, CMYK and Adobe RGB,
odd sizes), three 4K images (4:2:0, 4:4:4, progressive), restart
intervals of 1 MCU, 2, 7 and 1 rows in baseline, 4:4:4 and
progressive, and 500 fuzzed files - in both decode modes, on x86
under ASan/UBSan, on aarch64 and on big-endian ppc64 under qemu.
rjpeg, image_loader and image_avail sample tests pass; builds C89
-pedantic -Wextra without warnings, with clang, MinGW-w64, the NDK
(aarch64, armv7, i686) and PowerPC, and as C++.
3840x2160 decode: instructions -4.4% (4:2:0), -5.7% (4:4:4), -2.5%
(progressive); best time 26.4 -> 25.4 ms and 36.8 -> 36.3 ms.
Each view's texture took four glTexParameteri calls every frame. Like
the frame's own textures, it now takes them when made and when the
preset's filter or wrap changes, kept per slot in one packed word.
After every call the inflate backend copies the last 32 KB of what it
just produced into its own 32 KB history ring, so that a later call's
back-references still resolve once the caller binds a new output
slice (rpng unfilters rows in place inside its ring, so the bytes
cannot be read back from there). rpng fed it 32 KB slices, so every
decoded byte was copied a second time.
The slice is now 64 KB - the size the comment above it already argued
for ("must comfortably exceed the 32 KB deflate window and [be] small
enough to sit in L1/L2 alongside the window. 64 KB does both") - and
the history copy covers half the output. A slice is still bounded by
contiguous ring space and by what the filters have consumed, as
before; each rpng_process_image step simply decodes up to 64 KB. The
decoded bytes do not depend on how the output is sliced.
Verified: 428 decodes of 214 PNGs (4K screenshots, interlaced, RGBA,
8K, RGB/RGBA at 28 widths, and Paeth-on-every-row images for every
colour type and depth) give the same pixels as before in both output
layouts, under ASan/UBSan. rpng, image_loader, image_decode_slice,
overlay_apng and image_avail sample tests pass; no new C89 warnings.
Decoding a 3840x2160 PNG screenshot: the history copy goes from 25.8 M
to 14.7 M instructions; mixed content best 38.5 -> 36.3 ms, median
42.8 -> 37.1 ms; smooth gradient best 9.75 -> 9.26 ms. 128 KB slices
measured only marginally better and would leave the comment's cache
argument behind on small-cache ARM parts.
Corrects the previous commit ("SSE2 paths enabled on MSVC x64"),
which overstated what it changed. RetroArch's MSVC projects all
build through griffin, and griffin.c includes retro_environment.h
first - which defines __SSE2__ for MSVC's x64 and /arch:SSE2 targets
- so those builds already ran the PNG SSE2 paths. The explicit
_MSC_VER && _M_X64 test only affected rpng.c or rpng_encode.c
compiled on their own.
Both files now include retro_environment.h and test __SSE2__ alone,
as pixconv.c and the audio converters do, so a file built on its own
takes the same path as the unity build.
That includes 32-bit MSVC with /arch:SSE2 - the default for x86 since
VS2012, and what griffin's Win32 configurations already get. There,
only the first three __m128i arguments of a function can be passed by
value; the rest are rejected with C2719, and the decoder's Paeth step
took seven. It is now a macro, so no vector is passed by value at
all. Every input is read into a temporary before the result is
assigned, so the call sites that update a0/a1 in place are unchanged
in meaning; GCC and Clang inlined the function anyway and produce the
same code (decode instruction counts equal to within 14 per image).
This has not been compiled with MSVC itself.
Verified: 428 decodes of 214 PNGs - including 153 written with Paeth
on every row for gray 8/16, gray+alpha 8/16, RGB 8/16, RGBA 8/16 and
palette at widths 1-257 and Adam7 - give the same pixels as before in
both output layouts, under ASan/UBSan. Both files compile against a
union-typed __m128i as in MSVC's headers. rpng, image_loader,
image_decode_slice, overlay_apng and image_avail sample tests pass;
builds without new warnings with GCC (C89 -Wextra), clang, MinGW-w64,
the NDK (aarch64, armv7, x86_64, i686) and PowerPC, and as C++.
With a screen in its own window, a software frame's views through a
preset fell back to uploading the whole frame and copying each view out
of it on the GPU, because the window copied its views from the frame.
The main window's draw now writes the windows' views straight into
their textures beside its own, and a window draws a view written this
frame as it is: one upload per view, no frame upload, no copies.
The frame is still uploaded whole, and the windows still copy, when the
main window draws no views or draws them straight from the frame.
Both PNG files turned their SSE2 code on with #if defined(__SSE2__)
alone. MSVC never defines __SSE2__ - SSE2 is simply part of the x64
baseline there - so RetroArch's MSVC builds, desktop and UWP, have
been decoding and encoding every PNG with the scalar fallbacks: the
row filters, the pixel conversions, all of it. Most of formats/
(mp3, aac, opus, vorbis, h264, vp8, vp9, atrac3, the deflate encoder)
already names MSVC explicitly for this reason.
The gate is now defined(__SSE2__) || (defined(_MSC_VER) &&
defined(_M_X64)). 32-bit MSVC with /arch:SSE2 is deliberately left
out: it allows at most three by-value __m128i parameters (C2719) and
the decoder's Paeth kernels take more. The SSSE3 runtime path stays
off on MSVC - neither of its compiler predicates matches - and MSVC
ARM64 is unchanged. Nothing changes for GCC or Clang builds.
Not compiled with MSVC itself. What was checked instead: both files
compile cleanly against an emmintrin.h whose __m128i is a union, as
MSVC's is, with every intrinsic declared as a plain function - so any
GCC vector-extension use (operators, subscripts, casts on __m128i)
would fail, and a planted one does; every SIMD region in the two
files keys off this one gate; the gate evaluates on for _MSC_VER with
_M_X64, off with _M_IX86 or _M_ARM64; both files build C89 -pedantic
-Wextra and with MinGW-w64 without warnings.
What it is worth, measured with GCC with and without the SSE2 paths
(the closest stand-in for the MSVC build before and after): decoding
a 3840x2160 PNG screenshot 22.8 -> 10.3 ms for a smooth image and
61.4 -> 38.4 ms for mixed content, a large RGBA image 47.6 -> 36.1
ms; encoding a 4K XRGB8888 / RGB565 frame 6% / 13% faster. MSVC's
own auto-vectoriser may claw some of the scalar code back, so its
numbers will differ.
Placement runs every frame, and each named screen or menu setting built
every listed display's key, each in O(count), to compare. A key starts
with its display's name, cut at most to its size less three, so only
displays whose name prefixes the setting are built and compared:
about 44% less per placement with every screen named.
The test now picks the second of two same-named displays, short and cut.
The menu and notifications placed in another window went through an
offscreen RGBA image, cleared and drawn each frame, then blended over
the window. Laid out at the window's size, they are now drawn directly
on its back buffer while it is bound, with the main window's viewport
and projections left as they were.
The image is kept for a layout made at another size, and is made late,
for one frame, when the window turns out not to be drawable; the menu's
frame still runs once either way.
samples/gfx/gl2_window_ui covers both paths and the fallback.
The menu and notifications placed in another window were drawn into an
offscreen image, splitting the main render pass to do it, then blended
over the window as a full-screen quad. When they are laid out at the
window's size, record them straight into the window's pass instead: no
image, no store and reload of the main pass on tilers, no extra blend.
The image stays for a layout made at another size and for the fallback
over the main window while the window is not up, and is made only then.
A turn of an Android display reaches RetroArch as a new size on paths
these harnesses already run on the host: the main window's content
rectangle or EGL surface, the second display's surface, and the window
layouts and display list the frontend publishes. Pin what each must do
with it, so a regression fails here and not only on a device.
- android_content_rect: a 1920x1080 to 1080x1920 swap asks one resize
at the swapped size on the Vulkan context, and on the gl one, whose
check_window() is now extracted too, with egl_get_video_size()
stubbed.
- android_second_surface: the same surface at a new size is one
GEOMETRY at that size, the same generation, no CLOSE, still shown;
a window kept across a driver reinit meanwhile is taken at the size
last drawn, and the next driver's first poll brings the new size.
- video_screens: a window drawn again at swapped dims lays the menu out
at that size and maps touches through the new layout from the first
read; a display whose size changes under the same name keeps its
placement and does not ask again for a move that was given up; a
menu waiting for a turned display's window lays out at the size
listed now.
A touch or mouse button already down when the menu opened acted on the
menu. A finger held on the game as the menu toggle fired landed on
Resume once let go and shut the menu again, and a held right button
cancelled on the menu's first frame. This happened in every window, a
single window included.
Such a press is now dropped until it lifts, as one held across a change
of the menu's window is: no POINTER_DOWN, POINTER_UP, OK on release or
cancel comes of it, and the next press works as any. The cancel cover
takes a pointer's back, Android's Back key among them. A cancel already
down in the window the menu moves to is held back the same way. Only a
real open counts: retroarch_menu_running() with the menu already up, as
after a failed load, changes nothing. A press the menu held as it closed
no longer fires its release on the next open either.
With the menu drawn in the Presentation, it reads that window's first
touch, over the window at the menu layer's size, and nothing from the
main window; RGUI maps the touch through its framebuffer as it is fitted
to the layer. An input overlay, which stays in the main window, no
longer turns the menu's touches off there. The press and release bounds,
the drag and swipe thresholds, the wheel and the on-screen keyboard's
hit test use the layer's size too.
A press held while the menu changes window is dropped until it lifts: no
tap, release or cancel comes of it in either window. With the menu in
the main window nothing changes.
The touches on the main window and on the second display go to the
frontend's pointer publication each poll, raw, each with the window it
is in, and the publication answers the pointer, the screen pointer,
the second display's pointer and a lightgun aimed by touch. The mouse
stays the driver's: its motion and wheel are taken by each read, as
before.
Every answer is as it was; the pointer_frame test checks each one the
publication gives against the driver's own answers before this change.
Three things differ. Binds to the mouse's left, right and middle buttons
now work: they never fired on Android. And a touch is placed at the
first read after the poll instead of as its event came, so a finger held
still while the viewport changes follows the change at once.
And a tap on a main window that shows no screen is on nothing: it fires
no lightgun button there, while a tap in the letterbox still does.
Android's input driver is to publish its touches and mouse through the
frontend's pointer publication, so that each touch carries the window
it came from, without changing any answer a core or the menu gets
from it. Five flags let a driver ask for that, and change nothing for
the drivers that publish today: a touch pressed only inside what it
is read against; touches in a screen's own window as a second list;
the mouse answered by the driver, whose motion and wheel go with each
read; touch indices as slots that keep their place, with no mouse
standing in; and pointer ids the frontend does not hold left to the
driver.
The publication also holds 32 touches, and gives the menu a touch in
its own window over that window's client area.
When the menu's display cannot take a window of its own and the game
is shown elsewhere, RetroArch's own window has nothing to draw until
the menu opens there. It now shows the RetroArch logo from the
monochrome icon set, centred on black, drawn as one quad each frame.
Without that file or without widgets it stays black.
When placement puts the menu or the notifications in the Presentation,
they are drawn into one offscreen image at that window's size, between
two halves of the frame's pass, and composited over the window's
screens. While the image is drawn, the display backend places its quads
and meshes against the image's height, not the swapchain's.
A window that shows the UI takes an image on every frame it has UI to
draw, not only on a new frame with views: a paused hardware core repeats
its frame under the menu, and a core without a map has no views. On a
repeat its screens are drawn again from the last frame. A view that only
this window draws, and that the last new frame did not copy, is left out
rather than turning the main window's views off. A window with only UI
publishes an empty layout, so touches there miss the screens.
A window with nothing to draw this frame, or with no free image, keeps
its last image. Only a window that is gone sends the UI to the main
window, in the same pass, unless the layer is still waiting for its
window. With HDR the layers stay in the main window. With both layers in
the main window the frame is drawn as before; the menu's draw moves into
a helper both targets share.
When placement puts the menu or the notifications in the Presentation,
they are drawn into one offscreen image at that window's size and
composited over its screens. A window that shows them is drawn on every
frame it has UI to draw, with or without content and views, so the menu
keeps answering under a paused core; with nothing to draw it keeps its
last image.
If that window cannot be drawn in a frame, the image goes over the main
window in the same frame, unless the layer is still waiting for its
window to appear. Without the views' programs the layers stay in the
main window. With both layers in the main window the frame is drawn as
before.
The menu and the notification widgets each lay out at the size of the
window they are drawn in, which can now differ from the main window's:
the runloop asks for each layer's size, and under the threaded wrapper
the widgets' worker takes the notifications' size from the frame it
draws. With both in the main window nothing changes.
A driver teardown let the second display's Presentation go. When
content closed and the driver reinited, the hide could reach Android
before RetroArch's Activity arrived on that display, which then showed
neither.
A context's window ops can now keep and take a window. At teardown,
gl2 and vulkan keep each window their context can keep: its EGL or
Vulkan surface goes, and the Presentation stays up with its last image.
The next driver takes it at init and publishes it with no screens, so
it still counts as shown on its display and the close's keep holds it.
Placement lets it go as before. A window nobody took goes once the
next driver is up, or at shutdown.
gl2 no longer draws a kept window, so it keeps its last image as it
does on vulkan.
Under the threaded video wrapper the app thread runs ahead of the
video thread, so for about a frame after a snapshot lets the
Presentation go or moves it, the newest published layout still names
the old display. A make decided then completed at once and RetroArch
left before the new Presentation drew; a keep could be set on the
window being retired.
Each snapshot now carries a serial, and each published layout the
serial of the snapshot it was drawn from. A window counts as shown only
from a layout drawn at or after the last snapshot that asked for it
differently, so make before break holds. A snapshot the wrapper takes
back undrawn needs nothing special: the later one that is drawn counts.
A keep only holds a window that is up. A driver with no window no longer
makes one for it, which at content close covered RetroArch's display
with a Presentation that was never drawn.
The frame also reads shown, the clock and the menu state once, for
placement and the UI targets alike.
With Separate Screen Windows on, video_windows_place() decides each
frame, from the display settings, the published display list, the
content and the menu, where each screen, the menu and the notifications
go, and which display RetroArch's own window is on. It replaces the
route, which sent only screens 2 and up to a Presentation.
Auto no longer depends on where RetroArch was started: Game Screen 1
takes an external monitor, else the default display; Game Screen 2 the
built-in second panel, unless a monitor took Game Screen 1. A display is
in use only while it shows a screen of the running game or the open
menu. RetroArch's window goes to a display the game may use that refused
a Presentation, else to the lowest shown screen's, else, with no game,
to the menu's. Any other display in use gets the one Presentation.
RetroArch moves only when content starts or closes, a setting or the
display list changes, or a display refuses. A change of the core's map
within two seconds of a start is still the start; a later one only
re-routes the screens. Replacing content is a close and a start, decided
once the new content is up. Opening or closing the menu never moves
anything. Moves are made before they break: a display RetroArch leaves
that still shows something gets its Presentation first, and one it moves
onto keeps its Presentation, not drawn, until RetroArch has drawn there.
A move that cannot start or does not arrive within two seconds gives
that display up until the display names change.
Placement runs whenever windows are on, with or without content or
views. A UI layer whose Presentation is being made waits for it up to a
second, the frame the menu opens included, then falls back to the main
window. A main window that shows no screen reports touches off-screen,
and the menu maps them over the whole window.
On Android, Separate Screen Windows reads "Game Screens on Other
Displays". While it is on, Stereo 3D & Screens lists a Game Screen N
Display row for each screen, Menu Follows (Game Screen 1, Game Screen 2
and each further screen the core has, or Separate), Menu Display (only
when the menu follows no screen) and Notifications (With the Game or
With the Menu). A Menu Follows screen kept from a core with more
screens stays until it is changed. A display row offers Auto, each
display by name with its product and size, such as "Display: DP Screen
(DELL U2720Q 2560x1440)", and Off. OK opens a dropdown of the displays
listed now. "Game Screen" and "Display:" keep a game's second screen
apart from a panel Android calls Screen-2.
The name is what is saved, so the same port is found again whichever
monitor is on it; a display no longer listed shows as "Display: <name>"
and counts as Auto. video_screenN_display, video_menu_follows,
video_menu_display and video_notifications_window are saved in the
config and in core and content overrides. They replace
video_screenN_window_mode: an old Hidden loads as Off, Attached and
Separate as Auto, and the old key is no longer written. The mode setting
is gone, and the monitor row is listed on no platform;
video_screenN_window_monitor stays in the config for the desktop.
Until placement reads them, screen 1 stays in RetroArch's window and the
others go by Auto unless Off.
Each published display now says whether it is Android's
DEFAULT_DISPLAY. On a dual-screen handheld both panels report the
same unknown connection, so this is the only way to tell the main
panel from the second one when choosing a display automatically.
The second display's Presentation never has the focus now, so it never
receives a key or a stick move and never reports a focus change. Its key
and motion relay, the ring that carried them to native code, its focus
commands and its part in the focus count go: RetroArch's focus is its
Activity's, as before the second display existed.
When the focused Activity moves between displays, Android sends the new
display's focus gain and the old display's loss in either order. If the
loss comes last, the Activity is never told it has focus again, so
RetroArch stays paused and stops drawing.
After a move that leaves it unfocused, the window is made unfocusable
until Android has taken the focus, then focusable again, which makes
Android send the gain. Under load the window manager can apply a short
toggle's set and clear back to back, and the focus never moves, so the
flag is held until that focus event, or a second at most. A gain ends
the move only once it has outlasted a traversal, so a loss that follows
it still refocuses.
Native code can now ask Java to move RetroArch's Activity onto a
display of the published list. The request is posted to the UI thread
and never waited on. Java starts the running Activity again with
setLaunchDisplayId and a copy of its own intent, so the same content
is seen and nothing restarts; Android then moves the Activity in place.
When the move cannot start (before Android 8, a display that has gone,
or a launch Android refuses), Java logs it once and raises a flag the
main thread reads on its next frame.
A display whose Presentation failed is now published as refused instead
of invalid. It stays a valid target, so Auto and a saved name still
point at it and RetroArch can move onto it; only a Presentation is kept
off it. A list that changes only in which display is RetroArch's own
keeps it refused.
Each display also carries its DeviceProductInfo name (Android 12 and
later), which the menu shows next to the display's name.
A screen will name its display by the platform's name for it, which
says which port a monitor is on. video_windows_display_key() gives
that name, with " 2" or " 3" added for a second or third display
listed under the same name, so each one can be told apart and saved.
A display also carries its product name, for the menu, and whether a
Presentation failed on it.
video_views_test_map_change names a map the core changes to once
video_views_test_map_change_frame frames have run; none clears the map
with an empty one. A harness can then check how the frontend places a
game whose map changes early in a start or later on. It is off by
default.
The Vulkan modes keep their images at the first frame's size, so there
a change that would resize the frame is refused and logged.
The second display's Presentation could take the focus, so a tap there
gave its display the keys. When the content closed and the Presentation
went, the keys stayed on that display and reached the app under it: on
the AYN Thor the pad drove the launcher's bottom-screen activity instead
of RetroArch's menu. Taking the focus back with moveTaskToFront did not
help, since RetroArch's task was already in front on its own display.
The Presentation is now never focusable. Touches still reach it; keys
and sticks always go to RetroArch's Activity.
Changing Android's Display size setting, or RetroArch's window going to
a display of another density, relaunched the Activity and RetroArch
exited. The Activity now handles a density change itself.
From API 31 a Display obtained with the Activity's Resources reports
RetroArch's own window, so the size published for a display could be
RetroArch's window's, and it changed when that window went to another
display. A Display with no Resources attached reports the display's own
logical size.
A uint setting's list can leave values out: Mouse Index lists only the
mice there are, and Keyboard Index lists only the keyboards there are
and the one chosen now. The pick was worked out from the entry's
position, as the minimum plus whole steps, so a value left out shifted
every pick after it: with mice at #1 and #3, picking #3 set #2. Each
entry now carries the value it lists, and the pick sets that one.
The network settings harness passes each entry's entry_idx, which
carries its value, as the menu does, and picks a Mouse Index past an
index with no mouse and a Keyboard Index past an index with no keyboard.
Every decoded row is converted from PNG's byte order into the 32-bit
words the frontend uses. For 8-bit RGB - every screenshot RetroArch
writes - SSE2 has no byte shuffle, so four pixels cost four
whole-register shifts and masks to spread the triples into words,
plus a mask-and-shift R/B swap for ARGB output. SSSE3's pshufb picks
each output byte straight out of the load: one shuffle and the alpha
OR per four pixels. 8-bit RGBA to ARGB is likewise one shuffle in
place of the masked swap. The bounds and scalar tails are the SSE2
versions', so the output is identical.
A stock x86-64 build has no flag enabling SSSE3, so where the
toolchain can build a function for it without one - GCC 4.9+, or any
compiler with __has_attribute(target) and <tmmintrin.h>, the same
predicate encoding_deflate.c uses for its CRC32 hash - the choice is
made once per decode in rpng_process_init from cpu_features_get().
With -mssse3 it is made at compile time. Without SSE2, on ARM and on
other targets nothing changes.
rpng.c now calls cpu_features_get() on those x86 builds, so a build
compiling rpng.c must also link features/features_cpu.c. Every
RetroArch build (Makefile.common, which Makefile.win includes, and
griffin) and every libretro-common sample that builds rpng.c already
does.
Decoded pixels are identical to before - 5 large PNGs (4K screenshots,
interlaced, RGBA, 8K) and RGB and RGBA images at 28 widths from 1 to
1919, in both output layouts, 122 decodes - with the SSSE3 path chosen
at run time (also under ASan/UBSan), compiled in with -mssse3, and
forced off. rpng, image_loader, image_decode_slice, overlay_apng and
image_avail sample tests pass. Builds without new warnings with GCC
(C89 -Wextra), clang, MinGW-w64 and the Android NDK (x86_64 and
aarch64), and as C++.
Decoding a 3840x2160 PNG screenshot: smooth gradient 11.6 -> 10.0 ms.
The conversion alone goes from 4.0 to 1.7 ms per frame; on content
whose decode is bound by the filters or by streaming the output to
memory the difference is within noise.
Every dynamic block rebuilds three decode tables, and each has a
2048-entry fast[] and pfast[] pair. A code of length len owns every
entry whose low len bits are its bit-reversed code, so the build
stored it at each of 2048 / 2^len slots, one strided store pair per
slot - 4096 stores per table, three tables per block. Smooth image
data compresses into many short blocks, and on a 4K gradient
screenshot that was a tenth of the whole PNG decode.
The tables are now filled as libdeflate does it: codes are placed
length by length into a table exactly 2^len entries long, and before
each next length the table is doubled by copying its lower half up,
which lays down every repeat of every shorter code at once. Most of
the work becomes a handful of memcpy calls. The packed-entry
computation moves into rinf_pfast_entry(), unchanged.
The tables come out entry for entry as before. In a complete code
every final slot belongs to exactly one code - a short one, placed at
its own length and afterwards only copied, or a long one, whose
prefix slot is zeroed - so a slot's leftover from the previous block
is always overwritten; an incomplete code is still zeroed first, and
the copies keep its gaps zero.
Verified: 300000 builds into one never-cleared rinf_huff - random
complete, incomplete, single-symbol and junk length sets for all
three tables, 86% of them accepted - leave the struct and the return
value byte-identical to before after every build, also under
ASan/UBSan. zlib streams of six kinds at levels 1, 6 and 9 decode to
the original for every input/output chunk pairing (840 runs) with the
same hashes and status as before, and 6720 corrupted-stream decodes
match the old ones exactly. Decoded pixels of five PNGs are
unchanged. All rzip, archive, stream, rpng and encodings sample tests
pass; builds C89 -Wextra without new warnings, for aarch64, armv7 and
PowerPC.
Decoding a 3840x2160 PNG screenshot: gradient 13.8 -> 11.8 ms, mixed
content 41.2 -> 38.7 ms.
The careful (symbol-at-a-time) path exists for the moments when a
symbol or a copy straddles a buffer edge, and at the top of each of
its iterations it checks whether the margins are comfortable again
and, if so, jumps back into the fast loop. But after finishing a
match copy it fell straight through to decoding the next symbol
itself. If that symbol was another match, the length/distance steps
ran here, set copy_active, and the next pass through the hand-back
check saw an active copy and stayed - so the careful path only ever
gave control back after a literal.
Smooth image data is long runs of back-to-back matches with no
literal between them. One match straddling an inflate slice edge put
the rest of its run on the careful path, whose in-buffer copy for
short distances is a byte loop - and at distance 3, RGB's period,
each byte waits on the store of the byte three before it. Decoding a
3840x2160 gradient screenshot, 57,583 of the careful path's 58,363
iterations were that: nearly every match in the image.
After a completed copy the loop now goes back to the hand-back check
before decoding anything. That only re-runs a check the loop already
makes, so what is decoded is unchanged.
Verified: zlib-compressed noise, gradient, checker, short-period,
zero and mixed data at levels 1, 6 and 9 decode to the original for
every pairing of input chunk (1, 7, 64, 4096 bytes, whole) and output
chunk (1, 3, 258, 273-275, 1000, 32K, 64K, 4M), 840 runs, with the
same output hash and status as before; 6720 decodes of corrupted
copies of those streams give the same bytes, length and status as
before, also under ASan/UBSan. Decoded pixels of five PNGs (4K
screenshots, interlaced, RGBA, 8K) are unchanged in both output
formats. All rzip, archive, stream, rpng and encodings sample tests
pass; builds C89 -Wextra without new warnings, for aarch64 and armv7.
Decoding a 3840x2160 PNG screenshot: smooth gradient 39.1 -> 17.9 ms
(instructions -21%, the rest was the serial byte loop's latency);
mixed content about -7%.
Follow-up to "Scanner: Match Jaguar CD discs by CRC".
When a sheet had no data track, the fallback parsed the whole sheet
a second time with that track forced to data. That re-reads the cue
and reopens every track file - a Redump Jaguar CD has one file per
track, so ~20+ extra opens per disc during a scan.
Note the first track of session 2 (while no data track has been seen)
during the one parse instead: its start index, its end (next track's
index in the same file, or the file's end), its path and its open
stream, which the CRC pass then hashes without a fresh open, as a
data candidate's is. The force_track parameter goes away.
Same answers as before on 13 sheets (one file per track, single bin,
shared files, Enhanced CD, data before/after session 2, missing file,
CD-i, REM comments), ASan/UBSan clean. On a 24-track Redump-style
sheet: 48 -> 24 track opens, ~100 -> ~47 us per sheet.
"unsized TorrentZip sets: readers stop reading whole" failed twice on
the four-core runner with 16 of 24 zips read whole, four rounds of the
readers against three allowed: the sanitized scan ruled on the first
set later than the 150 ms the readers took over each read. Forced to
four readers here, the lane read 7 to 12, at the limit. A whole read
now takes 500 ms, so the same delay in the scan lets fewer rounds
through: 6 to 9 over six runs at four readers. The limit stays at
three rounds.
The lazy parse loop (levels 4-9, the screenshot encoder's level 6)
read and wrote pos, nsyms, the fixed-Huffman bit count and the
pending match through the struct at every position. Each literal it
records stores into the symbol buffer and the frequency table, and as
far as the compiler can tell any of those stores may have changed one
of the fields, so every one was reloaded after every symbol. On
noisy input nearly every position is a literal.
The loop now holds that state, and the tuning values and table
pointers it uses, in locals and writes them back when it ends.
Literals are recorded against the locals; matches - where the
per-symbol cost is not the issue - hand nsyms and the bit count to
rd_record_match() and take them back. Nothing the loop calls reads
the parse state: the match finder takes the position as an argument
and the insert paths touch only win, head and prev. The decisions
and their order are untouched.
Compressed output is byte-identical to before over the level 0-9 x
input chunking x six-input matrix, zlib-wrapped and raw, with the CRC
and the shift-xor hash, on x86 (also under ASan/UBSan) and aarch64
under qemu, and over a 3 GiB stream at level 6. rzip, rpng, archive
and encodings sample tests pass; builds C89 -Wextra without new
warnings, for aarch64, armv7 and PowerPC.
screenshot_io_bench, 3840x2160 BGR24 PNG: noise instructions -10.0%
(8.53 G -> 7.68 G); gradient unchanged. On the x86 box measured the
wall time moves within noise - the remaining cost there is cache
misses in the hash tables, and the reloads were L1 hits an
out-of-order core absorbed. In-order cores are where the cut should
show.
The encoder tries every filter on every row, and Paeth was the
costliest of them: SSE2 and NEON both widened each byte to 16 bits
to form pc = |a + b - 2c|, so a step covered only 8 bytes, with
unpacks going in and a pack coming out.
The predictor now stays in bytes. pa = |b - c| and pb = |a - c| are
byte absolute differences. pc needs nine bits only where b - c and
a - c have the same sign, and there it equals pa + pb, which is no
smaller than either; saturating it at 255 therefore cannot change how
it compares with pa or pb, which is all the predictor asks of it.
Where the signs differ pc is |pa - pb| and fits. A zero difference
makes both forms equal, so its sign does not matter. The selection
is unchanged: a unless pa > pb or pa > pc, then b unless pb > pc.
SSE2 builds the unsigned compares from saturating subtracts; NEON has
them, and vabd/vqadd, natively. Each step now covers 16 bytes.
Exact: the SSE2 and the NEON predictor (aarch64 under qemu) match
the scalar rpng_paeth_pred() for all 2^24 inputs. PNG output is
byte-identical to before for BGR24, XRGB8888 and RGB565 across widths
1-1920 and both pitch signs, on x86 under ASan/UBSan and on aarch64.
rpng sample tests pass; builds C89 -Wextra without warnings, as C++,
for aarch64, armv7 NEON and PowerPC.
screenshot_io_bench, 3840x2160 BGR24 PNG: gradient instructions
-7.3% (2.58 G -> 2.39 G), median 53.4 -> 49.8 ms; noise -2.2%.
0ca986d reads the CRC hash key with one 4-byte load and masks off the
fourth byte, which the window's 8-byte margin makes safe in the
compressor. The test still built its keys in a 3-byte array, so ASan
caught the load running one byte past it and the core samples group
failed. The key buffer is now 4 bytes, the fourth filled with noise
the mask has to drop.
Follow-up to "MCP: fix token length, SIGPIPE, stdio EOF, cleanup".
- input_driver: the generated token is saved at once, but
config_save_file refuses while overrides are active (and is skipped
with no config path). The modified flag was dropped with it, so a
token made with a core's overrides loaded was lost on exit even with
config_save_on_exit. Set the flag as before, then save now.
- mcp_server stdio: ignore SIGPIPE directly instead of calling
network_init(). stdio uses no sockets; network_init brings up
WSAStartup on Windows and the network stack on consoles for nothing.
- mcp_structured_record: take strlen(seps) once, not per column, and
guard an empty seps string, where n - 1 would index far out of range.
Every 32 KB of input the encoder slides its window down, and with it
every link in head[] and prev[] - 64K entries, 256 KB - was rewritten:
dropped by 32768, or zeroed when it fell out of the window. On a 4K
screenshot that is some 760 slides and 190 MB of table traffic per
capture, about an eighth of a smooth image's encode; on compilers
that do not vectorise the loop (MSVC, the older GCCs consoles build
with) it is worse.
The links are now stored as position + hoff instead of position + 1.
A slide raises hoff by the slide amount, which drops every stored
position by it without touching the tables. A link that slid out of
the window reads back as a negative position, below any search limit,
so a chain walk ends on it exactly where it ended on the zero the
rewrite used to put there; the zero calloc leaves still reads back as
"empty". rd_insert's "chain exists" answer is likewise unchanged, so
the parse is the same. So the stored values never leave int32 range,
the old rewrite still runs once hoff passes 1 GiB, and hoff restarts
from 1.
The chain walk keeps the position itself rather than the one-based
link, so the per-candidate "cur != 0" test goes away with the rewrite.
Compressed output is byte-identical to before: the level 0-9 x input
chunking x six-input matrix, zlib-wrapped and raw, with the CRC and
the shift-xor hash, and again with the rebase forced at every slide
and every third slide (RD_HOFF_REBASE overridable for that), also
under ASan/UBSan; and a 3 GiB stream at levels 1 and 6, which crosses
the real rebase twice. rzip, rpng, archive and encodings sample
tests pass; builds C89 -Wextra without new warnings, for aarch64,
armv7 and big-endian PowerPC.
screenshot_io_bench, 3840x2160 BGR24 PNG: gradient instructions
-11.5% (2.92 G -> 2.58 G), best 45.2 -> 40.7 ms, median about -10%;
noise -2.3%.
Every position the encoder inserts into a hash chain - on screenshot
content, nearly every position - first builds the 24-bit key for the
CRC32 hash from three byte loads, two shifts and two ors. That was
the single most expensive line in the encoder: 522 million
instructions for three 4K captures, on smooth and noisy images alike.
On little-endian targets the key is now one 32-bit load with the top
byte masked off, which is the same value. The fourth byte it reads
is always inside win[]: a key is only taken where three bytes remain
before win_len, and win[] already carries an 8-byte margin past its
largest win_len for the word compares in rd_longest_match().
Big-endian targets keep the byte-wise form, and builds without a CRC
hash (shift-xor only) are untouched.
Compressed output is byte-identical to before over the level 0-9 x
input chunking x six-input matrix, zlib-wrapped and raw, for each
hash build: runtime-detected SSE4.2, -msse4.2, and aarch64 +crc
under qemu; valgrind memcheck clean over the same matrix. rzip,
rpng, archive and encodings sample tests pass; builds C89 without
new warnings, for aarch64 with and without +crc, armv7, and
big-endian PowerPC.
screenshot_io_bench, 3840x2160 BGR24 PNG: gradient instructions
-11.3% (3.29 G -> 2.92 G), median about -10%; noise -4.0%, best
368 -> 351 ms.
rd_insert() hands back the chain head it just linked for a position,
and the greedy parser (levels 1-3) already used it to skip the match
search when there is no earlier occurrence. The lazy parser - levels
4-9, which includes the level-6 screenshot encoder - threw it away
and called rd_longest_match() at every position, which reloaded the
head from prev[] and returned 0 when it was empty. On noisy input
that is about half of all positions: a full call, register saves and
the struct loads of the prologue, to learn there was nothing to do.
Both parsers now pass the head in and only call when a chain exists;
rd_longest_match() starts its walk from it instead of re-reading
prev[]. An empty chain returned 0 with no side effects before, so
the parse - and the stream - are unchanged.
Compressed output is byte-identical to before over the level 0-9 x
input chunking x six-input matrix, zlib-wrapped and raw, also under
ASan/UBSan; rzip, rpng, archive and encodings sample tests pass;
builds C89 without new warnings and for aarch64.
screenshot_io_bench, 3840x2160 BGR24 PNG: noise instructions -10.3%
(10.37 G -> 9.30 G), best 393 -> 380 ms, median about -4%; gradient
unchanged.
A screenshot of a core's own frame - no GPU readback, no rescale -
hands the encoder XRGB8888 or RGB565 rows, which it turns into PNG's
R,G,B per row. Both conversions ran a pixel at a time, ahead of
filters that are all vectorised; the BGR24 readback path got the same
treatment in the previous commit.
SSE2 builds each pixel as a 32-bit lane 0x00BBGGRR (R,G,B,0 in
memory) and then closes the gaps without a byte shuffle: in each
64-bit half the odd lane moves down a byte onto the even lane's zero,
and the upper half moves down to follow the lower one. Four pixels
land in the first 12 bytes; the last four are rewritten by the next
store or the scalar tail. XRGB8888 gets its lane from a 16-bit
rotate and two masks, RGB565 from the same shifts and masks as
pixconv_rgb565_to_xbgr8888. NEON reads XRGB8888 as four planes (vld4,
little-endian only) and expands RGB565 with one vsri per channel,
which is exactly the top-bits replication; both store with vst3.
Other targets keep the scalar loops.
PNG output is byte-identical to before: 12 widths from 1 to 1920,
padded and negative pitches, both formats, ASan/UBSan clean. The
conversions match the old loops on SSE2, on NEON (aarch64 under qemu)
and on the scalar path at every width 0-300 and, for RGB565, for all
65536 input values, and never write past the row. rpng sample tests
pass; builds C89 -Wextra without warnings, as C++, and for aarch64
and armv7 NEON.
Per 3840x2160 frame, conversion alone: XRGB8888 3.75 -> 2.45 ms,
RGB565 9.77 -> 4.00 ms.
RetroFW's jz4760 OSS driver moves a node to the codec in 16-byte DMA
units: it drops a write's remainder, and a write under one unit starts
a transfer of none. The threaded pipeline writes any number of frames,
so most writes lost one to three frames - static at its rate of writes -
and a short one could leave the device playing nothing it was given.
On RetroFW a write now goes out in whole units, its remainder held to
lead the next one.
samples/audio/oss_report models that DMA; built with RETROFW=1, which
CI now does, it checks every node is whole units and plays in order.
Fixes#20006
A BGR24 screenshot - the GPU-readback path - has each row turned
into PNG's R,G,B order before filtering. That swap ran a byte at a
time, ahead of filters that are all vectorised, and was two thirds of
rpng_encode_row: about 12 instructions a pixel, 83 million per 4K
capture.
SSE2 has no byte shuffle, but this swap needs none. A 16-byte load
that starts on a pixel boundary holds every byte's partner two places
away in a pattern that repeats every three bytes, so two whole-register
byte shifts and three fixed masks build it; each step completes four
pixels and steps 12 bytes, its last four bytes rewritten by the next
store or the scalar tail. NEON loads the channels as planes (vld3)
and stores them back crossed. Other targets keep the scalar loop.
PNG output is byte-identical to before: 17 widths from 1 to 1920 x 4
heights x smooth and noisy content x both pitch signs, ASan/UBSan
clean. The swap matches the byte loop at every width 0-300 on SSE2,
on NEON (aarch64 under qemu) and on the scalar path, and never writes
past the row. rpng sample tests pass; builds C89 without warnings and
for aarch64 and armv7 NEON.
screenshot_io_bench, 3840x2160 BGR24 PNG: 50 million fewer
instructions per capture (gradient -4.3%, noise -1.4%); gradient
median 69.0 -> 65.9 ms.
Each dynamic block builds its codes by first sorting the used symbols
by frequency. That was an insertion sort, which is cheap for a few
symbols but costs about m*m/4 moves for a full alphabet - and a noisy
block, which every busy screenshot has plenty of, uses all 256
literals, so the sort ran to some 16000 moves per block.
Alphabets of more than 48 symbols now go through a stable byte-wise
radix sort on the frequency: one pass for each byte the largest
frequency uses, which is one or two since a block holds at most
RD_BLOCK_SYMS symbols. The symbols come in in symbol order, so a
stable sort gives the same order the insertion sort did - ties still
fall in symbol order - and the codes and the stream are unchanged.
Small alphabets (distances, code lengths, short blocks) keep the
insertion sort. The ping-pong half uses gl_wt and gl_left, which are
free until the tree build; nothing new is allocated.
Compressed output is byte-identical to before over the level 0-9 x
input chunking x six-input matrix, zlib-wrapped and raw; the sort
itself matches the insertion sort on 200000 random arrays of 49-288
symbols with frequencies up to 32 bits (ASan/UBSan clean); the rzip,
rpng, archive and encodings sample tests pass; builds C89 without new
warnings and for aarch64.
screenshot_io_bench, 3840x2160 PNG: noise instructions -5.8%
(11.17 G -> 10.52 G), best 427 ms -> 392 ms; gradient unchanged.
When a block does not compress - noise, video frames, already-
compressed archive members, the busy parts of a screenshot - it goes
out stored, and every byte of it was moved by a loop that tested for
output room, copied one byte and decremented a counter. On a noisy
4K screenshot that loop was a tenth of the whole encode.
The bytes now go out with memcpy, in runs as long as the remaining
block or the output room allows; a full output still suspends the
copy exactly where the byte loop did, so a caller handing in small
output buffers sees the same stream.
Compressed output is byte-identical to before: the earlier level 0-9
x chunking x six-input matrix (zlib-wrapped and raw, 1 MB output),
and a focused stored-block run - noise, half-noise and 4-bit noise at
levels 1, 6 and 9 into output buffers of 7, 8, 16, 17, 100, 4096,
65537 bytes and 1 MB, all suspending mid-block; ASan/UBSan clean; the
rzip, rpng, archive_zip_methods, archive_stream and encodings sample
tests pass; builds C89 without new warnings and for aarch64.
screenshot_io_bench, 3840x2160 PNG, noise: instructions -9.2%,
best 451.9 ms -> 403.7 ms, median 556.3 -> 492.3 ms; gradient (no
stored blocks) unchanged.
On the threaded pipeline audio_driver_pause_fade() armed the resume
ramp by writing fade_in_pending, a plain bool, on the main thread, and
the producer read and cleared it in audio_driver_submit_width() - on
the thread that submits, which for a core delivering from a thread of
its own is not the main thread. A data race; a resume could also lose
its ramp if the clear landed after a new arm.
fade_in_pending is now a retro_atomic_int_t. The main thread stores it;
whoever applies the ramp takes it with fetch-and against zero
(audio_driver_fade_take(); exchange is not on every retro_atomic.h
backend), in the producer and in audio_driver_arm_resume(). No lock,
and nothing waits.
ff_speedup gains a lane with the producer on a thread of its own while
the main thread ends frames, holds and releases fast-forward and
pauses, the consumer's share taken on the main thread. Under
ThreadSanitizer, which CI already runs ff_speedup with, it reported
this race and nothing else after the fast-forward measurement fix
(e24dafdc), and reports none now. The harness clock can run on real
time for a case with threads.
The CI step runs its lanes at once, in real time, so it takes as long
as the longest lane: rate_control, four settings at each of two device
latencies, 24 s; ff_exit, five transitions, 21 s; setpoint, three
latencies twice over, 18 s.
setpoint and rate_control now take a device latency (setpoint_16,
_32, _64; rate_control_32, _64) and ff_exit a transition (ff_exit_1
to _5), as audio_paced already did, and the step runs all fourteen at
once. The longest is now a single setting: 12 s of wall time for the
step here, three runs, no failures and no retries, on two CPUs. The
whole-case names still run the whole case, and a name that matches
nothing fails instead of passing on no run.
Three copies on the flush path that the next pass overwrote:
- Integer front pair at unity to an s16 device with a master gain:
the batch was copied into the output scratch, then the gain ran over
it in place. The gain now reads the batch and writes the scratch, one
pass (audio_driver_gain_s16(), the same Q16 multiply).
- In-process MIDI synth on the integer path: the game audio was copied
into the writable view, then the synth summed into it. The sum is now
written from the game audio, one pass, and a synth that rendered
nothing leaves the batch where it is, uncopied.
- In-process MIDI synth for a float core: copied in, then summed. The
sum is written in the pass that brings the audio in. The synth renders
before the path is chosen, so one that has nothing this batch no
longer forces an s16 core off its passthrough through two conversions.
Output is unchanged sample for sample. Measured on 800-frame stereo
batches: the float synth sum 167 -> 132 ns per batch; the s16 gain
saves the 23 ns copy, about 0.5% of that pass; a silent synth saves the
copy and, on the float path, an s16 -> float -> s16 round trip.
unity_identity gains lanes: the integer gain at an attenuation and at a
saturating boost against the Q16 oracle; the synth sum on the integer
pair and for a float core against what the copy and the add gave; and
a silent synth leaving the batch on its passthrough, which on the float
path failed before (one s16 -> float and one float -> s16 conversion).
The shared stubs' synth is replaced there by one the cases control.
The fast-forward speed measurement (last_flush_time and its two
averages) is taken by one thread: the submitter on the inline
pipeline, the main thread's frame end on the threaded pipeline and for
a callback core. Other threads wrote it too. Inline, the main thread's
frame end, pause and stop reset it while a core submitting from a
thread of its own was measuring. A callback core's audio thread reset
it beside the main thread's frame-end measurement. On the threaded
pipeline the producer read and reset it, and pipe_ff_frames was a
plain size_t added on the submitting thread and zeroed on the main one.
These are data races, and on 32-bit targets a 64-bit store can tear.
Now only the measuring thread writes the measurement. Any other thread
asks: one flag, stored only when not already up, which the measuring
thread takes before it next measures (audio_driver_ff_mult_sync(),
taken with fetch-and since exchange is not on every retro_atomic.h
backend). An idle frame costs one load. pipe_ff_frames is atomic: added
by the submitter, taken by what is there at the frame end, never stored
to zero. The producer knows the first frame of a hold from a flag of its
own instead of reading the main thread's measurement. A flush outside
fast-forward on the measuring thread still resets outright, which the
inline transport's source stamp overwrites as before.
This replaces the request-handler resets added for pause_fade in
0e35c138 with the same ask every other thread uses.
discrete_multichannel gains a lane with a core submitting from a
thread of its own while the main thread ends frames, holds and releases
fast-forward and pauses. Under ThreadSanitizer it reported nine races
on last_flush_time and pipe_ff_frames on the old code and none now; the
CI step builds it with TSan and runs that lane. Checks that read the
measurement straight after another thread's reset now accept the
pending ask.
check-ff-stall runs in real time, twice, and again under
ThreadSanitizer in CI. Each run spent 0.5 s settling into the hold,
1.5 s stalled and 1 s after it.
The stall is now 1.25 s, the shortest that still takes the old code's
estimate past the stretch's 4x duration bound with margin (from a bit
over 1 s); at 0.5 s the old code passes. A quarter of a second settles
on each side and half a second is judged, as before.
The lane still fails on the code before the gap fix (the stretch off
60 of 60 frames) and passes with it, three runs each variant, and under
ThreadSanitizer.
audio_driver_pause_fade() reset the fast-forward flush measurement on
the main thread. That is right where the measurement is taken there:
the threaded pipeline's producer and a callback core measure at the
frame end. Inline, the thread that submits measures, and a core may
submit from a thread of its own; the main thread's reset then wrote
last_flush_time beside that thread's read-modify-write of it, a 64-bit
store that can tear on 32-bit targets.
Inline, the reset now rides the pause and resume requests pause_fade
already posts, applied by the submitter under the inline claim before
its next flush measures. The main thread resets only for the threaded
pipeline and callback cores, and always without HAVE_THREADS.
The ff_speedup and discrete_multichannel pause lanes pass unchanged.
A stall inside a fast-forward hold was measured as one long flush
interval, and a long enough one pushed the speed estimate past the time
stretch's duration range. The stretch then stayed off for the rest of
the hold: the threaded transport suspends and resumes only from a
drained ring, which fast-forward keeps full; and with Speedup off the
inline flush put back the stamp from before the interval, so every
later flush measured from it.
An interval longer than the average's whole window at 1x
(AUDIO_FF_EXP_AVG_SAMPLES expected intervals) is now a gap, not a speed:
the estimate seeds again at the configured ratio, as after a pause.
Shorter intervals are still measured. One of them can take the estimate
of a hold running at 1x or faster down to about half speed at most,
inside the stretch's range, and any single interval that could push it
out is a gap. The inline, threaded and callback-core paths share the
measurement.
Inline, the stamp is put back only for the Speedup fallback, which may
measure the interval again. Put back otherwise, it stays stale until it
ages into a gap.
The ff_speedup and discrete_multichannel harnesses stall a 2x hold for
three seconds, threaded with and without Speedup, and inline. Inline, a
slow spell of intervals under a gap also takes the estimate out of
range, and the stretch has to be back within a few frames of the hold
returning to 2x. pipeline_wakeups gains a lane on the real wrapper
without Audio Sync, where the consumer primes again after the stall; it
checks that the hold's audio and the stretch come back. The native
transport CI job runs it, also under ThreadSanitizer.
A pause inside a fast-forward hold left the flush timestamp from before
it, so the first flush after the resume measured the whole pause as one
interval. The speed estimate then read the core several times slower
than real time, outside the time stretch's duration range, and the
stretch stayed off until fast-forward was released; inline, only with
Speedup off.
Reset the measurement in audio_driver_pause_fade(), on the main thread
as the frame end resets it. Every pause and resume goes through it, and
so do the jump fades of a state load or core reset, where the reseed is
harmless. The next flush seeds the estimate at the configured ratio, as
the start of a hold does.
The ff_speedup and discrete_multichannel harnesses pause a 2x hold for
125 ms, threaded and inline, and check that the speed estimate and the
stretch hold through it.
In Ozone, opening an image from the file browser with OK and pressing
Up/Down to step to the next image changed the picture but kept the
first image's name as the title.
The title was refreshed by each menu driver on a selection change, and
Ozone only did that when the fullscreen view had been asked for with
START. The viewer opened by OK sets the "shown" flag but not the
"wanted" one, so its title was never refreshed while stepping.
Every step through the viewer - the OK that opens it, Up/Down/Left/
Right, continuous playback - goes through
menu_media_open_selected_file, so the title now belongs there: it takes
the driver's title buffer and rewrites it for the file it opens. Ozone
and XMB pass theirs; no driver can open a file in the viewer and leave
the old name up. A row the viewer cannot show leaves the title alone.
samples/gfx/media_controls: viewer_title drives the real
menu_media_browser.c through OK, Down, a row the viewer cannot show,
Down past it and Up, checking the title each time, plus truncation to
the buffer and no buffer. It fails on the old code.
The frame header refused any JPEG whose component ids were not 1, 2,
3(, 4) or 0, 1, 2(, 3). Adobe's 4-component files - the CMYK and
YCCK the output path has a whole branch for - number their components
'C', 'M', 'Y', 'K', so every one of them failed to load: Photoshop
CMYK JPEGs, and Pillow's or libjpeg's CMYK output. libjpeg takes any
ids; the scans name components by id anyway. Now any ids are taken,
and only two components sharing one is refused.
That also lets through the 3-component files that hold R, G and B
rather than Y, Cb and Cr, which the same check used to refuse (their
ids are 'R', 'G', 'B') and which would otherwise come out with a
YCbCr transform applied to RGB. The colour space is decided as
libjpeg decides it: a JFIF APP0 means YCbCr; else an Adobe APP14's
transform 0 means RGB; else ids 'R', 'G', 'B' do. RGB rows skip the
colour transform and the fused YCbCr kernel and are only put in the
requested byte order. The JFIF segment is now recognised for this.
Decoded pixels are unchanged for every YCbCr and greyscale image in
the 176-file Pillow and 24-file cjpeg corpus (baseline and
progressive, all samplings, restart intervals), through the iterative
decode in both byte orders and the monolithic decode. The 66 Pillow
CMYK cases (baseline and progressive, 1x1 to 1920x1080), which all
failed before, now decode and match libjpeg's CMYK samples (C*K/255
etc. on the Adobe-inverted values) within one level; cjpeg -rgb
output (Adobe transform 0), its progressive form and the same file
with every APP segment stripped (ids alone) match libjpeg within one
level, where before they failed. Of 300 corrupted progressive files,
two that only failed for a mangled component id now decode, as they
do in libjpeg; ASan/UBSan and LeakSanitizer clean over 500 corrupted
files; the rjpeg and image_loader sample tests pass; builds C89, as
C++ and for aarch64.
A progressive JPEG decoded iteratively kept a full-size plane per
component alongside its coefficients, inverse-transformed every
component into its plane one after another, and only then - in
rjpeg_process_image - ran the resample and colour conversion over
the whole image, reading the planes back cold. At 4K 4:4:4 that is
24 MB of planes nobody needed whole and one more trip through DRAM.
The finish phase now goes one MCU row at a time across all
components: dequantise and inverse-transform that row's blocks, then
emit the output rows that are complete - one MCU row behind, exactly
as the baseline fusion does, since the vertical resampler's far row
for the tail of a span is the first plane row of the next. The fused
emitter is armed when the last scan ends; the planes, which the scans
never touch, are allocated ring-sized (three MCU-row spans) on the
iterative path, as baseline's already are, and rjpeg_process_image
finds the output made. The monolithic path keeps whole planes and its
own finish, unchanged.
Decoded pixels are bit-identical to before on 200 images - baseline
and progressive, 4:4:4, 4:2:2, 4:2:0 and cjpeg's 2x1, 1x2, 4x1, 4x2
and 1x4 sampling, restart intervals (every 3 MCU rows, every block),
an optimised progressive script, a jpegtran-progressive transcode,
greyscale, sizes 1x1 to 1920x1080 - through the iterative decode in
both byte orders and the monolithic decode, and on 300 corrupted or
truncated progressive files; ASan/UBSan and LeakSanitizer clean; the
rjpeg and image_loader sample tests pass; builds C89, as C++ and for
aarch64.
Progressive, iterative decode:
3840x2160 4:4:4 peak heap 110.8 MB -> 86.2 MB, LL misses -22%,
best 193.7 ms -> 178.5 ms
3840x2160 4:2:0 peak heap 72.5 MB -> 60.3 MB, LL misses -17%,
best 123.1 ms -> 116.4 ms
1920x1080 4:2:0 peak heap 18.2 MB -> 15.2 MB
GCC 15 warned that sv_ym may be used uninitialized in
rvp8_video_decode: the saved sets were filled under !kf and read back
in the else of a separate kf test, a pairing it does not follow. The
save now fills them for every frame - a key frame with the defaults it
used to reset to at the restore - and the restore copies them back
whenever refresh_entropy_probs is 0. The same entropy results, and
the warning is gone at -O2 and -O3.
Error-resilient (refresh_entropy_probs 0, a key frame every 20) and
normal 176x144 libvpx streams decode byte-exact against FFmpeg before
and after.
Every position a match covers goes into the hash chains so later
matches can reach it, and on the repetitive content screenshots are
made of - flat runs, gradients, upscaled pixel art - that is nearly
every byte. Each one was a call to rd_insert that re-chose the hash
(a run-time CPU-feature branch on a stock x86-64 build), loaded its
three bytes afresh, and was guarded by a per-position bounds test:
42% of a 4K screenshot's encode was spent there.
rd_insert_range() now does a whole interior at once: the bound is
worked out once (rd_insert_end, the same q + 3 <= end), the hash is
chosen once per run, the tables stay in registers, and the shift-xor
hash rolls from one position to the next - ((h << 5) ^ next) & mask is
exactly the hash of the next position, since what the mask drops from
h would be shifted out past it anyway. The chains it builds are the
ones the calls built, position for position, so the output does not
change. rd_insert itself, still used for the position being matched,
is inline.
Compressed output is byte-identical to before for levels 0-9, zlib-
wrapped and raw, fed 1, 7, 4096, 65536 bytes at a time or all at once,
on gradient, noise, flat-run, text, zero and mixed inputs - through the
run-time CRC32 hash, the shift-xor hash (CPU features masked) and the
compile-time SSE4.2 hash; ASan/UBSan clean on both run-time paths; the
deflate, rzip, archive_stream, archive_zip_methods, rpng and other
encodings sample tests pass; builds C89 without new warnings and for
aarch64 with and without +crc.
screenshot_io_bench, 3840x2160 PNG:
gradient instructions -25.6%, best 75.2 ms -> 58.8 ms
noise instructions -4.1%, time unchanged within noise
RSLC_EXTENDED takes the rest of the samplers - sampler1D, sampler3D,
samplerCube and samplerCubeArray of each base, and the 1D, 2D, 2D array,
cube and cube array shadow samplers - with every texture function GLSL
has on them, the shadow ones through the Dref sample and gather
instructions. It takes a stage redeclaring its gl_PerVertex block or
gl_FragCoord, a geometry stage's inputs declared as plain unsized
arrays, gl_PointCoord, gl_PrimitiveIDIn, and GL_EXT_spirv_intrinsics'
spirv_execution_mode with its capabilities and extensions; a float
controls mode declared without them gets the ones it needs. These go to
SPIR-V only; the other destinations refuse a stage that uses them.
A constant division by zero folded to zero; it folds to the infinity or
NaN IEEE gives, as glslang folds it. Cemu's decompiled shaders and the
community graphic packs written for them compile to SPIR-V that draws as
glslang's does, pixel for pixel on lavapipe.
A lookup finds directories as well as files, the root at the empty
path, and searches each directory's sorted entries by bisection instead
of scanning them. A listing's entries carry their place: a file's
offset and size, a directory's first entry and count. zar_index_offset()
tells a caller where the index begins, so it can keep that part of the
file in memory and answer lookups and listings without I/O.
zar_write.c builds an archive front to back through a write callback,
compressing blocks with rzstd where HAVE_RZSTD is defined and finishing
with the footer's SHA-256, so an emulator can make .zar/.wua files
without the reference ZArchive library. Its output reads back with the
reference reader, and the reference writer's output with this one.
Capping an oversized image box-averaged it by a power of two into a
second buffer of the reduced size, then either kept that (when it was
the target) or ran the sinc stage from it, freeing the original in
both cases. A 4K screenshot capped to 1920 held an 8.3 MB copy it did
not need: every averaged pixel lands below every source pixel a later
block still reads (the next block starts at y*f*sw + (x+1)*f, the store
is at y*dw + x, with f >= 2), so the reduction can be made in place.
Now it is. When the box result is the target the buffer is shrunk to
fit with realloc and kept - no allocation at all - except on GEKKO,
whose decoders memalign their output. When the sinc stage follows,
its output buffer and filter are made before the box runs, so a
failure still leaves the image exactly as it was; the box stage itself
cannot fail once its geometry has been checked.
Two things made the in-place loop slower until handled: the compiler
must assume the output aliases the source it is still reading, so a
row's averages are built in a 64-wide local run and copied out; and
the box being a power of two, the four divisions per pixel become
shifts, which give the same result for unsigned sums.
Output is bit-identical to before - pixels, dimensions and return value
- over 352 cases: 11 shapes from 2047x2 to 16385x3, eight caps, 8- and
10-bit, with and without the upscale stage; ASan/UBSan clean, no
leaks. The image_scale_transfer test now asserts the new ownership:
an exact box makes no allocation (it runs with an allocation budget of
0), a box + sinc makes one, and a failed sinc set-up leaves the source
untouched; image_loader passes too. Builds C89, as C++, for aarch64
and with GEKKO.
4K -> 1920x1080: additional image storage 8.3 MB -> 0, 17.9 ms instead
of 20.1 ms (8-bit) and 14.5 instead of 17.2 (10-bit).
An Adam7 image allocated a buffer for each of its seven passes,
converted the pass into it and scattered it into the image, and freed
it - and freed it again and reallocated it every time a pass found
its data not yet inflated, which on a streamed decode is most passes
several times over. Each pass also allocated and freed its own three
scanline buffers.
Now:
- pass 7 (x 0, stride 1) is the image's odd rows, whole, so it is
unfiltered and converted straight into them: the output cursor
steps two image rows a line (out_stride, which the regular path
sets to the width and passes 1-6 to the pass width). Half the
image is no longer written into a buffer and copied out again.
- passes 1-6 share one buffer, allocated once at the largest of them
and kept, through the waits too, until the decode ends.
- the three scanline buffers are one block, reserved once at the
image's full pitch, which no pass exceeds, and kept across passes.
All of it goes back through the buffer pool where one is set (the
pool's three line slots become one).
Decoded pixels are bit-identical to before on a new corpus of 150
interlaced images - grey 1/2/4/8/16-bit, RGB and RGBA 8/16, grey+alpha,
palette 1/2/4/8-bit with tRNS, tRNS on grey and RGB, sizes from 1x1
up through every empty-pass shape, single and many IDATs - and match
Pillow on the 8-bit ones; on the 546-image corpus, its multi-IDAT
re-chunking, the APNG set and 550 corrupted or truncated files;
ASan/UBSan clean, no leaks; the rpng sample tests pass; builds C89,
as C++, as one translation unit, for aarch64 and with GEKKO.
800x600 RGBA interlaced, decoded twice: 196 allocations (88 MB) ->
26 (10.4 MB), instructions -1.9%, D1 and LL data misses -8%.
The VP8L colour-indexing (palette) transform always expanded into a
second full-size buffer, w*h words, and then freed the indexed one.
That buffer is only needed when the indices are packed - 2, 4 or 8 to
a pixel for palettes of 16 colours or fewer - where the expansion runs
top to bottom and would overwrite packed rows not yet read. A palette
of 17 to 256 colours carries one index per pixel, so each index now
becomes its colour where it lies; the fused channel swap already
falls back to the stream's own buffer when there is no expansion
buffer.
Decoded pixels are bit-identical to before, and to libwebp's dwebp,
on palette images of 3, 12, 40, 100, 200 and 250 colours with and
without alpha (packed and unpacked), lossy images with palette-coded
alpha, a palette animation, and 160 corrupted palette files plus the
earlier corrupted sets; ASan/UBSan and LeakSanitizer clean; builds
C89, as C++ and for aarch64.
640x480 with a 40-colour palette: 1.2 MB less allocated,
instructions -7.7%, D1 misses -19%.
Each APNG frame still made a new inflate stream and freed it at the
end of the frame: for the built-in inflater a calloc of 73 KB - its
32 KB back-reference window and Huffman tables zeroed afresh - and for
zlib an inflateInit2 with its window allocation. On small frames that
zeroing cost more than decoding the frame.
trans_stream_backend gains an optional last member, reset: put a
stream back to the state stream_new gave it, keeping its allocations
and whatever define() set. The built-in inflater implements it with
rinflate_reset, which already restores a fresh instance without
re-zeroing the window or tables (a stale window byte is unreachable
once wprior is 0, and tables are rebuilt before use); zlib with
inflateReset, which keeps the window inflateInit2 allocated. Every
other backend lists NULL, and a backend that does not mention the
member gets NULL too.
rpng's buffer pool gains a slot for the stream. A decode takes it -
reset as it is taken - when its backend matches and can reset, and
otherwise makes one as always; a finished, failed or abandoned decode
gives it back, or frees it where there is no pool or no reset. So an
APNG stream inflates every frame with the one inflater, and stills
and every other trans_stream user are untouched.
Composited frames are bit-identical to before on the whole APNG set
and 250 corrupted or truncated files, with the built-in inflater and
with zlib; still PNGs decode identically on the 546-image corpus;
ASan/UBSan clean and no leaks under either backend; the rpng,
archive_file, archive_stream, archive_zip_methods and archive_zst
sample tests pass; builds C89 without new warnings, as one translation
unit and for aarch64.
Two passes of an animation, instructions / data refs / allocated:
128x128, 25 frames -50.6% / -69.5% / 4.1 MB -> 0.54 MB
77x51 Adam7, 10 frames -31.9% / -60.3% / 1.8 MB -> 0.38 MB
640x480, 20 frames -1.3% / -4.1% / 20.3 MB -> 17.5 MB
Every frame of a playing APNG allocated, and freed again afterwards,
the decoder's inflate ring (at least 64 KB), its three scanline
buffers, an Adam7 pass buffer per pass for interlaced frames, and the
frame's whole decoded image - mid-playback, every frame, every loop.
rpng now takes an optional buffer pool (rpng_internal.h): a slot per
role - ring, the three lines, the pass buffer, the output image - each
holding at most one buffer, the larger one when two meet. Every place
rpng allocated one of these takes it from the pool first and falls
back to the allocation it always made, and every place it freed one
gives it back instead; with no pool, giving is free(), so stills and
every other caller behave exactly as before. The output keeps its
32-byte memalign on GEKKO, a pooled image having been allocated there
too. The APNG stream owns a pool, hands each frame's image back once
composited, and releases it on close: once a stream has seen each size
it needs, frames stop allocating these.
Composited frames are bit-identical to before on every APNG in the
set (all colour types and depths, interlaced, palette with tRNS,
hidden default image, multi-fdAT frames, every dispose/blend mix) and
on 250 corrupted or truncated files; still PNGs decode identically on
the 546-image corpus, single and multi-IDAT; ASan/UBSan clean and no
leaks; the rpng sample tests pass; builds C89, as C++, as one
translation unit (griffin), for aarch64, and with GEKKO defined.
Heap allocations for two passes of an animation:
128x128, 25 frames 534 -> 294
77x51 interlaced, 10 frames 704 -> 412
640x480, 20 frames 470 -> 372, 5.6 MB less allocated
What remains per frame is the decoder's own state: the rpng and
process structs, the inflate stream and the IDAT span list.
Every APNG frame was decoded by building a standalone PNG in the
stream's scratch: the file's IHDR (with the frame's size), PLTE and
tRNS, then all of the frame's IDAT/fdAT payloads copied end to end
into one IDAT - the whole compressed frame, every frame, every loop.
The scratch PNG now carries an empty IDAT where the data would go, and
the payloads are handed to rpng as spans of the animation file itself
(rpng_add_idat_spans, internal through rpng_internal.h), read in place
exactly as a still PNG's IDAT chunks are. rpng keeps the base those
spans are relative to - buff_start unless a caller gives its own - and
refuses to mix the two. The APNG part list is the shared span type,
and the duplicate PNG signature goes in favour of rpng_internal.h's.
With the previous change a frame split over several fdAT chunks costs
no more at the seams than one in a single chunk.
Composited frames are bit-identical to before on RGBA, RGB and RGBA at
16 bits, grey and grey 16 interlaced, grey+alpha, palette with tRNS
(interlaced and not), Adam7 RGBA, a hidden default image, frames split
over up to five fdAT chunks, every dispose/blend mix, the earlier ten
animations, and 250 corrupted or truncated files (same frames, same
failures); ASan/UBSan clean; the rpng sample tests pass; builds C89, as
C++, as one translation unit (griffin) and for aarch64.
A 640x480 20-frame animation: instructions -1.4%, data refs -8.2%,
D1 misses -4.7%, 165 KB less allocated. Small frames gain little: the
decoder is still set up afresh per frame, which is the next step.
The compressed stream is fed one IDAT payload at a time. When a
chunk's payload ran out partway through an output slice, the step
returned, and the next one rebound the output to a fresh slice before
handing over the next chunk. To the inflater that rebind turns
everything produced so far into history, so the back-references that
reached across the seam were then served out of its 32 KB window a
byte at a time instead of copied within the buffer, and the window
was committed at every chunk end as well.
libpng and most other writers split the image data into 8 KB IDAT
chunks, so this hit nearly every PNG in the wild. Now, when a chunk
runs out with room left in the slice and another chunk follows, the
next payload is handed over and the inflate carries on into the same
output in the same step. Pacing is unchanged: a step still produces
at most one slice.
Decoded pixels are identical to before on the 546-image corpus (every
colour type and depth, interlaced and not, every filter) and on the
same corpus re-chunked into IDATs of 1, 7, 13, 100, 1000 and 8192
bytes with an empty IDAT mid-stream, which also decode identically to
the single-chunk originals; with the built-in inflater and with zlib;
ASan/UBSan and LeakSanitizer clean; the rpng sample tests pass.
A 3.1 MB RGBA PNG written with 8 KB IDATs now decodes at the cost of
the same image in one IDAT: instructions 664.3M -> 626.1M (-5.7%),
data refs -12.2%. Single-IDAT files are unchanged.
Wii U content did not scan. It now takes the serial lookup by its
product code:
- a WUD, or a Wii U disc by another name (.iso, probed only once the
other detectors find nothing, so other discs read nothing more), by
the "WUP-P-XXXX" its header starts with;
- a WUX, through the new formats/wux, the same octets of the disc it
stands for;
- a Wii U archive (.wua), by the product and company codes of its base
title's meta/meta.xml, read through formats/zar.
formats/wux reads WUX, the Wii U image format that keeps each distinct
sector once, as the WUD it stands for. Like the other readers it does
no file I/O: it asks for each range it needs. Opening reads the
32-octet header alone; the sector index is read a window at a time,
only where a read needs it; a read's data lands in its own destination,
one request per run of sectors the file keeps in order.
formats/zar gains a directory listing, zar_dir_begin() and
zar_dir_step(), each name read whole, the listing's end ZAR_END.
A Wii U release's serial in the database is the product code and the
maker's two characters, or the code alone for a title sold as a
download; a disc does not say its maker. The code is therefore looked
up as a stem, in the Wii U database alone - glob() in a libretrodb
query now matches binary fields, as a serial is kept - and the release
picked is the one of the maker an archive names, else a download for
an archive and a disc release for a disc. The playlist names the
release found.
wux_test is new, zar_test lists directories, and the scan test gains a
Wii U lane: a WUD, a WUX, a disc named .iso, two archives whose update
is listed before the game, and another database holding the codes.
A VP8 image or animation frame with an ALPH chunk built a w*h byte
plane for its alpha - a memcpy of the whole chunk for raw alpha, a
narrowing pass out of the VP8L decode for compressed alpha - then
unfiltered that plane top to bottom in a second pass and merged it
into the decoded pixels in a third, and freed it.
alph_apply() now takes the frame and does it a row at a time: the
row's filtered bytes go into a two-row buffer (the row, and the one
above that the vertical and gradient filters predict from), are
unfiltered there, and merged into the top byte of each pixel while
the row is still in cache. The w*h plane and its two cold passes are
gone; what remains is one 2*w byte buffer. Every check that can fail
(header, short raw data, a VP8L decode that fails) is made before the
first row is written, so a bad chunk leaves the frame opaque exactly
as before.
Output is bit-identical to before on stills with raw and compressed
alpha under every filter (none, horizontal/fast, gradient/best),
quantised alpha, lossy alpha animations from img2webp and ffmpeg, and
200 corrupted files; alpha matches libwebp's dwebp byte for byte;
ASan/UBSan and LeakSanitizer clean; builds C89, as C++ and for
aarch64. A 3655x2053 image with alpha allocates 7.5 MB less per
decode and takes 8-10% fewer data-cache misses; instructions are
unchanged.
An animation frame that covers the whole canvas and is copied rather
than blended - no_blend set, or a source whose alpha bytes are all
0xFF - was decoded into a buffer of its own and then memcpy'd row by
row over the canvas, every pixel of it, before that buffer was freed.
Such a frame replaces every canvas pixel and its buffer is exactly
canvas-sized, so the two now trade places: the decoded frame becomes
the canvas and the old canvas is freed in its place. Nothing the copy
would have overwritten (the key-frame clear, the previous frame's
dispose-to-background) is done first either. The stream's contract
already makes the returned canvas valid only until the next call, and
the channel-order switch converts whichever buffer is the canvas.
Composited frames are bit-identical to before on full-canvas lossy,
lossless and no_blend-with-alpha animations, sub-rectangle ones from
ffmpeg, img2webp (lossless, mixed, lossy) and gif2webp (lossless and
lossy, with disposal), and on the 300 corrupted files; ASan/UBSan and
LeakSanitizer clean; aarch64 builds. A 4 s 640x480 lossy animation
of full frames no longer copies 1.2 MB per frame: data-cache misses
-34%, data refs -0.7%, instructions -0.3% (decoding dominates).
Every AC-3 seek threw the decoder away and made a new one: a calloc of
the whole decoder (some 26 KB) and the 640 cos/sin evaluations of its
transform tables, only to land in the state the old one could be put
back into.
rac3_decoder_reset() now does that: it zeroes the stream state - the
overlap delay, coefficients, grouped-mantissa and E-AC-3 frame state,
scratch - and reseeds the dither, exactly as rac3_decoder_new leaves
them, while the tables and the caller's DRC setting stay. The struct
puts those first so the reset is one memset from the first stream
field to the end. The seek calls it instead of new + free, and can
no longer fail on an allocation.
Output is bit-identical to before for 1/0, 2/0 (AC-3 and E-AC-3),
3/0, 3/1, 4/0 and 5.1 streams, f32 and s16, reads of 1 to 10000
frames with six seeks each (to 0, to a block boundary, mid-block,
past the end); ASan/UBSan and LeakSanitizer clean; the rac3 and
audio_transfer_ac3 sample tests pass; builds C89, as C++ and for
aarch64. 500 seeks with a short read after each on a 5.1 stream:
490 allocations (13.3 MB) -> 7 (0.25 MB), instructions -1.1%.
Redump gives one serial to every release of an Xbox or Xbox 360 title,
and to the extras discs that came with them, so a serial lookup found
the first record - "Halo Movies (Japan)" for a North American Halo.
formats/xdvdfs now reads the regions a title is for: the game region of
an XBE's certificate, and the region of an XEX's security info, each
only where the executable is long enough to hold it. The scanner reads
the region field of the records a serial matches when it has regions to
go by, and takes the first record whose regions overlap them; when none
does, the first record stands in, as before.
The duplicate DB_EXTRACT_* definitions in database_info.c go: the
header carries them, with DB_EXTRACT_REGION added.
Pressing OK on an image in the file browser read and decoded the file
a second time even though its thumbnail was already on screen.
menu_media_open_selected_file only reuses a thumbnail when
gfx_thumbnail_has_file says it holds the file, and that answered yes
only for a live animation session. Every still (PNG, JPEG, BMP, TGA,
single-frame GIF/WEBP) has no session, and neither does an animation
the preview budget refuses - large ones land as a still - nor one that
finished its last loop. All of those were reset and loaded again.
Each thumbnail now records a 64-bit hash of the path it was requested
from (no per-thumbnail path buffer), cleared on reset.
gfx_thumbnail_has_file matches it while the still is shown, or while
its load is in flight and not superseded; a failed or cancelled load
still gets requested again.
samples/gfx/gfx_thumbnail_anim lane 16 drives the real
gfx_thumbnail.c: a still is recognised in flight and once shown, and
not for another path, after a reset, when superseded or when the load
failed. It fails on the old code.
A key frame freed the per-macroblock info array, and the next inter
frame calloc'd it again, so every key frame of a stream playing cost
a free and an allocation of mbw * mbh entries (8160 at 1080p) in the
middle of playback.
The video decoder now owns the array for its life and grows it only
when a key frame brings more macroblocks. A key frame still detaches
it from the decode state, and the first inter frame after one clears
the entries it uses, which is what the calloc gave it, so the decode
reads exactly what it did. A failed begin leaves the array with the
decoder instead of freeing it.
Output is bit-identical to before on every stream in the VP8 set
(altref, error-resilient, partitions, odd sizes, 1080p, noisy), a
keyframe-every-7 stream, one switching between three frame sizes
mid-stream, and 120 corrupted streams; ASan/UBSan and LeakSanitizer
clean; builds C89, as C++ and for aarch64. Heap allocations for the
4 s keyframe-every-7 clip: 23 -> 9.
Turning a NAL into its RBSP moved every byte through a four-way test
for an emulation_prevention_three_byte, one byte at a time, though
such a byte turns up only a few times per megabyte. The scan now
jumps from one 0x03 to the next with memchr, checks the neighbours of
each against the same rule on the same input bytes, and memcpys the
stretches between the ones that qualify.
SPS and PPS NALs were unescaped into a malloc of their own and freed
again. Streams repeat their parameter sets in band, often before
every IDR, so that ran during playback. They now unescape into the
current context's scratch, as slices already do; nothing of a slice is
in it between NALs, a queued slice having taken its buffer along.
The new unescape matches the old byte for byte on 3,000,000 random
buffers of 0 to 63 bytes drawn heavily from 0x00-0x03, under ASan with
exact-size buffers. Decoded pictures are identical to before on
baseline, main with in-band parameter sets every 10 frames, high with
8 slices, B-frames, CAVLC at qp 0, a flat high-bitrate clip and 4:4:4
lossless; the lossless (threaded and not), state and hw sample tests
pass; ASan/UBSan clean; aarch64 builds.
Instructions to decode a whole clip:
640x360 CAVLC qp 0, 6.9 MB 2831.5M -> 2750.8M (-2.9%)
640x360 4:4:4 lossless 2582.6M -> 2560.6M (-0.9%)
640x360 baseline crf 20 808.5M -> 802.8M (-0.7%)
Heap allocations, main with parameter sets every 10 frames: 100 -> 83.
Every Huffman table in a VP8L stream cost four heap operations: a
malloc and free of an 8 KiB symbol scratch and a calloc for the table
itself, then the same again for the code-length table read before it.
An image with a meta-Huffman image has five tables per group and can
have hundreds of groups, so a 3655x2053 lossless WebP made 770 heap
allocations to decode, nearly all of them this, and an animation paid
it again on every frame.
Now:
- the stream decode allocates the symbol scratch once, in the same
block as its code-length scratch, and hands it to every table build;
- the code-length table (three-bit lengths, 7-bit root, so exactly 129
entries and never a second level) lives on the stack, 516 bytes;
- every group's tables are carved in turn out of one arena per stream,
sized at start for the five root tables of every group and doubled
when a second-level table does not fit. A move would invalidate the
pointers, so each table records its offset and the pointers are set
once the last group is read. The arena is zeroed as it is handed
out, exactly as the calloc'd tables were, so an incomplete code reads
the same.
Heap allocations for a decode, old -> new:
3655x2053 lossless 770 -> 23
1024x768 lossless (-z 9) 304 -> 23
640x480 lossless (-z 9) 132 -> 23
10-frame 480x360 animation 896 -> 174
Total bytes allocated falls 8-24%; peak heap rises 0.3-1.6% with the
arena's slack. Instruction counts and data-cache misses are unchanged
within noise: pixel decode dominates the time.
Output is bit-identical to before on lossless (-z 0/6/9, -exact with
alpha), near-lossless, lossy with lossless alpha, still and animated
(lossless and mixed) images, and on 300 corrupted files (same pixels,
same failures); ASan/UBSan and LeakSanitizer clean. Builds C89
-pedantic warning-free, as C++, and for aarch64.
gfx_thumbnail_anim_test stopped linking when the savestate preview
started from the configured state folder: gfx_thumbnail.c now calls
dir_get_ptr(RARCH_DIR_SAVESTATE), which gfx_thumbnail_preview's stubs
already answer and this harness's did not. It answers with an empty
folder, as an unset one reads.
Every Opus packet was decoded into the context's pending buffer and
then copied into the caller's, whether or not the caller had room for
it as it stood.
A read with nothing pending now looks at the next packet's length in
its TOC byte and, when the whole packet fits in what is left of the
caller's buffer, decodes it there and counts it as produced; pending
is not touched. Packets that cannot go out exactly as decoded still
take the pending path: one with pre-skip left to drop off its front,
any packet on a movie timeline (which maps and may overlap or pad
it), and the tail of a read too short for the next packet. The end
bound (end granule, WebM padding) applies by count either way. A
malformed packet whose TOC undercounts it is refused by ropus for
room before anything is decoded, and goes to pending as before, so
errors land where they always did.
Output is bit-identical to before for mono, stereo and 5.1 (mapping
family 1), 2.5, 20, 40, 60 and 120 ms packets, Ogg and WebM, s16 and
f32, reads of 1 to 10000 frames with and without a seek, and for 240
corrupted streams; ASan/UBSan clean, the audio_transfer sample tests
pass.
Decoding a whole stream, 4096-frame reads, instructions / data refs:
stereo 20 ms f32 -0.2% / -0.4% s16 -0.1% / -0.2%
5.1 f32 -1.4% / -8.8% s16 -0.5% / -4.0%
The AAC arm decoded every access unit into a float buffer of its own
(raac interleaving and scaling its planar synthesis output into it),
then copied that buffer into the caller's on an f32 read, or, on an
s16 read, scaled each sample back up by 32768 and rounded it one at a
time. Every sample was written twice, and s16 paid an extra scalar
conversion pass that raac already has in SIMD.
raac now offers the decode in two halves: raac_decode() leaves the
frame inside the decoder, planar, where synthesis put it, and
raac_copy_s16()/raac_copy_f32() interleave any slice of it into a
caller's buffer, converted exactly as raac_decode_s16/raac_decode_f32
would. Those two entry points are now the decode plus a copy of the
whole frame, and the copies keep their SSE2/NEON paths at any offset
and length, with a scalar tail; mono f32 gets a vector path too.
The AAC arm decodes with raac_decode() and each read copies its slice
straight from raac into the caller's buffer, in whichever format it
asked for, so s16 and f32 still mix freely mid-packet. The per-stream
float buffer, its allocation at start and its free are gone.
Output is bit-identical to before for mono, stereo, 3- and 6-channel,
ADTS and M4A, and the doubled-rate (16/22/24 kHz) SBR path, in f32,
s16 and alternating reads of 1 to 10000 frames with a seek, on SSE2
and the scalar build; ASan/UBSan clean, NEON and armv7 build, the
raac, audio_transfer and rmp4 sample tests pass.
Instructions to decode a whole stream, 4096-frame reads:
44.1 kHz stereo M4A s16 168.1M -> 136.4M (-18.8%)
f32 131.4M -> 130.7M (-0.6%)
22 kHz stereo (SBR) s16 419.5M -> 389.8M (-7.1%)
f32 391.6M -> 384.4M (-1.8%)
formats/zar looks files up in a ZArchive and reads them, with no file
I/O of its own. A lookup asks for a directory's entries a run at a
time and for each name only as far as it can match; a read asks for a
block kept as it is straight into its destination, decodes a
compressed block (HAVE_RZSTD) into the destination when the read
wants all of it, and otherwise into one block kept for the reads that
follow.
The scanner takes a .zar by the title of the default.xex it holds.
Each APNG frame is rebuilt as a PNG and handed to rpng, and three CRCs
were computed for it - over IHDR, IEND and the whole of the frame's
compressed data - though that PNG goes nowhere but rpng, which does
not check chunk CRCs. They are left zero.
The region a DISPOSE_PREVIOUS frame saves was reallocated to that
frame's size on every such frame. It is now allocated once, at the
canvas size every frame is checked to fit when the frames are indexed;
and if that allocation fails the region is not saved, rather than the
old buffer being kept at a size that may not match the frame restored
from it.
Four APNGs from FFmpeg and six variants rewritten to dispose to
previous (all frames, and alternating with background and none)
composite identically before and after, nine of them byte-exact
against FFmpeg; the tenth differs from FFmpeg before and after, where
a frame blended over a restored region shows the restored pixels as
the APNG specification has it and FFmpeg does not.
rpng_apng_bounds_test and rpng_decode_corpus_test pass. 0.7% fewer
instructions over a 20-frame animation.
Each game's description, year and manufacturer went through a
PATH_MAX_LENGTH stack copy, a trim, five strstr scans and, for each
HTML code found, string_replace_substring's allocate, copy and copy
back, before a last copy into the field. The replacing was also wrong:
rxml has already decoded the character references, so decoding
&, <, >, ' and " a second time turned a title's
literal "&" (written &amp; in the DAT) into "&".
The text is now trimmed by pointer, with the same whitespace rules as
before, and copied into the field once, bounded, as rxml hands it
over; the exported logiqx_dat_html_code_list table, which nothing else
uses, is gone with the loop.
Fields with entities, surrounding whitespace, over-long text, empty
text and UTF-8 read the same as before; only a doubly-escaped title
differs, and now reads correctly. A 60000-game DAT reads identically
and takes 23% fewer instructions to load and walk.
scan_begin_budget_test passes.
Each of the three UPnP replies (device description, external address,
port mapping) was copied into a NUL-terminated heap buffer, then
copied again by rxml_load_document_string, which keeps its own. The
buffer is now handed over with rxml_load_document_owned, which keeps it
as the document's or frees it on failure, so the second copy and its
allocation are gone and the caller no longer frees it.
natt_desc_parse_test passes.
The normalise stage compacts the input as it turns CR and CRLF into
LF, copying every scalar from its read position to its write position
- onto itself, byte by byte, for the whole file, until the first CR,
which most files never have. While nothing has been compacted the
scalar is already where it belongs, so both positions just advance;
copying starts at the first CR. One scalar a work item, and its UTF-8
validation, as before.
The subtitle_srt tests pass. Loading a 4.4 MB, 40000-cue SRT counts
3.6% fewer instructions in cachegrind; wall time on this box is within
its noise.
rvp8_begin filled the Y, U and V planes with 127 before every frame -
about 3 MB at 1080p - and the decode then overwrote all of it: every
macroblock writes its whole 16x16 (8x8 chroma) area, the 127 / 129
edges intra prediction reads beyond the frame are applied per
macroblock, and motion compensation clamps to the reference. The fill
is gone. WebP lossy, which decodes through rvp8_begin too, loses it as
well.
Seven libvpx streams - odd and tiny sizes (203x117, 17x9), noisy
content, alt-ref, error-resilient, four token partitions and 1080p -
decode byte-identical before and after, six of them byte-exact against
FFmpeg's libvpx decode (the error-resilient one already differed);
clean under ASan and UBSan, and Valgrind memcheck finds no use of an
uninitialised byte on five of them, so nothing read the fill. On the
1080p stream cachegrind counts 4.8% fewer instructions, 25% fewer data
writes and 43% fewer L1 write misses; 6.42 -> 6.31 ms a frame.
A VP8 frame whose refresh_entropy_probs is 0 uses the probability
updates it carries for itself only; the next frame starts from what
this one started from. rvp8_begin recorded the flag, and its comment
left the save and restore to the persistent decoder, which never did
either, so the updates carried on into every later frame.
Error-resilient streams send the flag on every frame, and they decoded
wrong from the first inter frame on.
rvp8_video_decode now keeps the context a frame inherits - coefficient,
MV and intra mode probabilities - before it reads its updates, and
when the frame turns out not to refresh, puts that back afterwards; a
key frame's starting point is the defaults, which are set again (the
mode and MV sets are refilled with theirs when found cleared, as the
next inter frame already does).
Four error-resilient libvpx streams - 352x288, a noisy 320x180, 640x360
with alt-ref and 97x55 with two token partitions - decoded wrong before
and are byte-exact against FFmpeg's libvpx decode now; six other
streams stay byte-exact; clean under ASan and UBSan.
rmp3_stream moved every byte twice on the way in and once more on the
way out:
- Every input byte was copied into the 4 KB hold, and after each frame
the hold's remaining ~3.6 KB was shifted down - about nine times the
compressed stream moved, to reassemble frames that almost never
straddle a window.
- Each decoded frame went into the stream's own PCM buffer and was
copied out: 4.6 KB a frame for stereo s16, 9.2 KB for f32.
Now, with nothing held and the window holding as much as a full hold
would (or the rest of the stream once EOF is stated), the frame is
decoded where it lies, handed exactly the bytes the hold would have
been given, so the decoder sees nothing different; the hold is kept
only for a frame that straddles two windows, and a window that yields
no frame is left in the hold as before. With room in the caller's
buffer for any frame the stream can carry - 1152 frames of up to two
channels, a channel change fixed up in place there - the frame is
decoded straight into it; the synthesis stores sample by sample, so any
alignment will do. The first frame, which sets the stream's channel
count, and short outputs still go through the PCM buffer.
The output is bit-identical before and after on CBR and VBR, mono and
stereo, MPEG-1 and MPEG-2 and ID3-tagged streams, in s16 and f32, with
sliding windows of 700 bytes to the whole file and outputs of 100 to
5000 frames - 224 combinations; rmp3_frame_test passes; clean under
ASan and UBSan. Decoding a 128 kbps stereo file, memcpy and memmove
drop out of the profile and L1 data misses fall by 25%.
The title read takes the executable an Xbox 360 disc boots,
default.xex, ahead of an Xbox one's default.xbe, and reads one on its
own as well: an XEX's execution info, an XBE's certificate or an STFS
package's header, each only as far as the title needs, after a 24-octet
head that tells them apart. The game partition is also looked for where
development images keep it, and a padded root reads as empty.
The scanner takes .xex and .xcp files by their title, and Xbox 360
discs too; a disc of a set gets its place in it ("FO-2046-1"). A .xex
that is no Xbox executable, as an Atari 8-bit one, goes on to the CRC
lookup.
A FLAC's last frame is shorter than its block size, so the decoder
runs out of input partway through the frame it was sized for. rflac
reads that as the end of a span - roll back and wait for more - unless
rflac_set_eof says nothing more comes, and buffer mode, which hands the
decoder the whole file at once, never said so: the last frame was
waited for and lost. A 3.7 s stream at 44.1 kHz read 159744 of its
163170 frames with 4096-frame blocks, short by up to a block on every
file whose length is not a whole number of blocks.
Buffer mode now calls rflac_set_eof with the span it hands over, and
so does the seek table's jump, which hands over the rest of the file.
With the end stated, rflac no longer copies its decoder (4.4 KB) before
every frame for a rewind point that cannot be used: an underrun after
rflac_set_eof is the end of the stream, kept rather than rolled back,
so the point is taken only while more input may come.
Eight FLACs - mono, stereo, 24-bit and 5.1, blocks of 192 to 16384 and
one without a seek table - now read every frame, byte-exact against
FFmpeg's decode; what was read before is an exact prefix of what is
read now; seeks land exactly; audio_transfer_flac_test passes; clean
under ASan. Despite decoding the frames that were lost, cachegrind
counts 2.8% fewer instructions with 192-frame blocks and 0.8% with
1152.
A FLAC seek decodes forward from the nearest point it can reach into a
stack scratch of 1024 * 2 s16, asking for up to 1024 frames a step -
right for stereo, but a frame is a sample per channel, and FLAC goes up
to eight: a 5.1 stream wrote 6144 samples into 2048, past the end of
the stack buffer. Any seek in a multichannel FLAC does it, including
the seek to 0 that loops a sound.
Each step now asks for at most what the scratch holds at the stream's
channel count, assuming the most (eight) before the header has been
read. Both decode-forward loops, after the seek table's jump and after
a rewind, take the bound.
A 5.1 FLAC seeked at several points reads exactly what a straight
decode gives there, and ASan, which reported the overrun in
rflac_read_pcm_frames_s16 under audio_transfer_flac_seek_to, is clean;
mono and stereo streams read the same bytes as before.
Every hunk a read touched was decoded into the cache and its slice
copied out, so a read of whole hunks - every chdstream hunk load and
read-ahead, rchd_read_hunk_begin, and the inner hunks of a large byte
range - decoded each hunk and then copied all of it once more. A
parent reference copied twice: the parent's cache, the child's cache,
then the destination.
A hunk the read covers whole and that is not already cached is now
decoded, or fetched from the parent, straight into the caller's
buffer, and the v3/v4 CRC is checked there. The cache keeps holding
what it held, which is still a valid hunk; only a read of part of a
hunk decodes into the cache to copy its slice out. No codec uses the
cache as scratch.
Every read pattern - all hunks forwards and backwards, whole-image,
unaligned, aligned multi-hunk, repeated and cross-boundary byte ranges
- gives the same bytes as before on chdman images in every codec
(none, zlib, lzma, huff, flac, zstd, a mix, and cdzl, cdlz, cdfl, cdzs
CDs with data and audio tracks) and on raw and CD children with
parents; byte ranges match the source image; children match libchdr
hunk for hunk; clean under ASan and UBSan. Cachegrind counts 10%
fewer instructions over those reads on an uncompressed image, 5% on a
zlib one and 2.8% on a cdzl CD.
A child CHD's parent reference is a whole hunk of the parent at a unit
position, fetched with rchd_read_begin. When the image ends mid-hunk,
the child's last hunk then asks for bytes past the parent's logical
end, which a byte read rightly refuses, so reading that hunk failed
with RCHD_ERROR_PARAM - every read touching the last hunk of a child
image, the one a reader tries last.
When the reference starts on one of the parent's hunks it is that
whole hunk, padding included, which is what rchd_read_hunk_begin
gives; the fetch now uses it there and keeps the byte read otherwise.
A raw child CHD made by chdman, whose image is not a multiple of the
hunk size, now reads every hunk identical to libchdr reading the same
child and parent (hunk 150 of 151 failed before); a CD child still
matches on all 150, and byte ranges over the child match the source
image.
Each frame went three times through memory: rac3 decoded it into the
arm's raw buffer, a pass mapped it into pend in the mixer's channel
shape, and the read copied pend out (or converted it to s16). For
mono, 2.0, 3.0 and 5.1 the map is the identity - the decoder's slot
order is already WAV's - so the middle pass changed nothing at all.
- The map is marked identity when it is one.
- A read with room for a whole frame (an A/52 frame is at most 1536)
decodes it straight into the caller's buffer: an identity map in f32
directly, any other map or s16 from the raw buffer with the map,
gain and conversion in one pass. pend now only holds the frame a
short read or a seek leaves part of.
Bit-identical before and after on seven layouts (mono, 2.0, 3.0, 2.1,
3.1, 4.0, 5.1) in f32 and s16 at request sizes 1 to 10000 frames with
a seek mid-stream; audio_transfer_ac3_test passes; clean under ASan
and UBSan. Reading in 4096-frame requests: stereo 62.3 -> 60.3 ns a
frame in f32 and 65.8 -> 63.8 in s16, 5.1 118.8 -> 112.5 and 133.1 ->
123.7.
Every frame cleared the whole mode-info grid - 54 bytes a cell, about
1.75 MB at 1080p - and once decoded read it all again to copy each
cell's references and motion vectors into the next frame's predictor
array. Neither pass is needed:
- Every cell is covered by exactly one block, which writes its whole
footprint (rvp9_decode_block's replicate loop) before anything later
reads it, and the neighbours a block reads were decoded earlier in
the frame, so the clear only ever wrote values that were
overwritten.
- A block's references and MVs are final at the replicate loop, so
the predictor entries are written there, in the same pass, and the
end of the frame only swaps the two arrays. Tile workers write
disjoint cells, as they already do for the grid.
Byte-exact against FFmpeg/libvpx with 1 and 4 workers on
check_pixels.py's streams and five more: alt-ref with compound
prediction at 640x360 and 10-bit 512x288, a noisy 480x272, an
error-resilient 352x288 and a tiled 1080p one; clean under ASan and
UBSan. On the 1080p stream cachegrind counts 2.3% fewer instructions,
13.5% fewer data writes and 14.6% fewer L1 data misses; wall time on
this two-CPU box moves within its own noise.
Every process() call ended by copying its new output, up to 32 KB,
into the back-reference window, so the next call can resolve
references once the caller binds another buffer. A stream that has
reached its end decodes nothing more until a reset, which clears the
window's reach, so that copy was never read: a one-shot inflate - each
CHD zlib hunk, a whole zip or 7z member - paid for a window copy of
its last 32 KB. The commit is now skipped when process() returns END.
It cannot move to rinflate_set_out to cover the chunked case too:
rpng unfilters the inflate output in place before it rebinds, so a
late commit would capture the unfiltered bytes.
rinflate_window_test passes. A CD hunk (18816 bytes, reset between
hunks as rchd does) inflates in about 57.9 us against 58.6 us.
With R,G,B,A byte order requested on a little-endian host, an 8-bit
RGBA row's conversion is the identity, so each row was unfiltered in
the inflate ring and then copied whole into the output. Every
unfilter kernel reads the raw line and writes the line it is handed,
and their tails cover the out-of-place case, so the output row is now
that line and the row above it the prev: the copy is gone, the ring
releases each raw line as soon as it is read, and the wavefront pair
path writes its two rows into the output the same way. Interlaced
images and every other format keep the ring path; the scalar build
copies the raw line into the output row and unfilters there.
546 synthetic images - RGBA8, RGB8 and gray+alpha, widths 1 to 1023,
each filter alone, runs of one filter for the pair path and random
mixes - decode identically before and after with RGBA order on and
off, clean under ASan and UBSan. A 1080p RGBA8 image decodes in
27.5 ms against 27.8 ms; inflate is most of the rest.
formats/xdvdfs finds an Xbox disc's game partition, at the start of a
trimmed XISO or where XGD1, XGD2 and XGD3 images keep it, looks files
up in its directory trees and reads the title of its default.xbe, with
no file I/O of its own. A lookup asks for one entry at a time, only as
much of each name as decides the search.
The scanner takes .xiso files, and .iso and .chd files no other
detector finds a serial for, through the serial lookup by the XBE's
title ("MS-004"), without the ISO size hint, which would rule out
trimmed images. Other discs read nothing more for it.
Each ADPCM sample depends on the one before, so decoding is a chain of
dependent steps, and the decoder kept that chain's state in arrays
indexed by the runtime channel count: every sample's state went
through memory and back. The blocks are now decoded with the state in
locals for a run at a time - an IMA channel's eight-sample run, or a
whole MS mono or stereo block - with MS at other counts still on
arrays:
- IMA, any channel count: per channel run, state loaded and stored
once per eight samples. Stereo decodes its two runs together, two
independent chains the core overlaps. The step, its product and the
nibble's sign are one lookup in a 89x16 table built from the step
table, which it replaces.
- MS mono and stereo: their own loops, both channels' state in locals.
- The nibble's sign is applied without a branch; the clamp keeps its
branches, which real audio almost never takes, so they predict and
cost nothing, where a select sits on the chain.
- A window stops decoding where it ends instead of running to the end
of the block, and a run wholly inside it is stored without a test a
sample.
Output is identical on every fixture, on the round-trip test and on
music-like and random payloads. Decode time a sample at 2.1 GHz,
music-like IMA / MS, before -> after: mono 2.63 -> 2.58 / 5.15 ->
3.44 ns, stereo 2.44 -> 1.90 / 4.07 -> 2.25 ns; random payloads, which
clamp constantly, 16 to 53 percent faster.
An MS ADPCM block's nibbles are one stream, high nibble first, each
frame's channels in turn. The decoder took the high or low half by
channel parity and stepped to the next byte after each odd channel,
which is the stream for one, two or four channels; with three or five
a frame starts mid-byte, and the decoder read that byte's high nibble
again instead of its low one, so every frame after the first came out
wrong. The half now follows a running nibble count.
test_rwav encodes IMA and MS at one to five channels with a reference
encoder that keeps its own reconstruction, and checks rwav decodes
exactly that, whole and through windows that start and end inside
blocks. MS at three channels failed before.
rc_hash reads a .ciso file through formats/ciso as the disc image it
stands for, the way RVZ and WIA files are read, so a CISO identifies
as its ISO does. The file reader lives in cheevos/cheevos_ciso.c: it
reads on the descriptor path, straight into the buffer rc_hash gives,
and seeks only when the file is not already where a read starts. The
reader is now built with cheevos as well as with the database.
ciso_hash_test hashes a GameCube disc as an ISO and as a CISO with
blocks left out, and checks the two hashes match.
A block states its opening step, and a run of large nibbles grows it
by up to 768/256 a sample with no bound, so a crafted block overflowed
the adaptation's multiply within a few dozen samples: undefined
behaviour, in practice a step that wraps back to small values. The
step is now capped at INT_MAX / 768, the largest the next adaptation
and the nibble times step both still hold, as reference decoders cap
it. Past a step of 32767 every non-zero nibble already clamps, so
only streams that were saturated garbage decode differently.
The prediction sums two products of a 16-bit sample and a coefficient
the file states, which reach 2^31 at -32768 each; it is summed in 64
bits and divided as before.
The fixtures decode to the same samples. test_rwav adds a block of
the largest step and +7 nibbles, which UBSan flagged before and which
now stays pinned at the top.
A G.711 or ADPCM WAV was decoded to s16 in a scratch buffer of a
sample a channel a frame, then copied or converted into the
pipeline's own buffer and the scratch freed. For one or two channels
the scratch is not needed:
- s16, stereo: rwav decodes in the pipeline's layout already, so it
decodes into the buffer and the copy is gone.
- s16, mono: decoded into the first half, then spread to pairs from
the end, each pair written at or past the sample it reads.
- float: decoded as s16 into the tail of the float buffer (a half for
stereo, a quarter for mono), then converted front to back; every
float lands at or before the bytes of each sample still to be
read, and those are read through memcpy so no float store can be
ordered ahead of them.
More than two channels still fold from a scratch, which is larger
than the stereo output. Both pipelines' output is byte-identical to
before for G.711, IMA and MS ADPCM at one, two and three channels;
peak memory while loading drops by the scratch, 12 MB for a 1.5 MB
stereo IMA file.
An IMA or MS ADPCM header may state fewer frames a block than its
blocks hold but not more. Both fixtures state exactly what theirs
hold; the test parses each with that count, a smaller one, one more
and the 16-bit maximum. Without the bound in rwav_walk the larger
counts parse.
rwav now bounds a WAV's frame count by what its payload can hold, but
the mixer's pipelines expand that count before allocating: two output
samples a frame, as floats and padded to 16, and for more than two
channels a decode scratch of an s16 a channel a frame. 8-bit mono
into stereo float is eightfold, which a large enough file still wraps
on a 32-bit size_t, and the frame count is kept as an unsigned. Both
pipeline builders now refuse a count none of those fit in before
sizing anything.
Reject samples-per-block values larger than a coded block can hold.
Small payloads can otherwise inflate the decoded frame count.
On 32-bit builds, that count can wrap mixer allocation arithmetic and
make decoding write past the resulting buffer.
Also reject overflow when converting the block count to a frame count.
Both focus-lost lanes have failed since ubuntu-latest moved to 26.04 and
Wine 10, and waiting longer for the release did not help: RetroArch's
window never lost the focus. With no window manager, Wine 10 hands the
activation straight back to the window it holds as the foreground when
xdotool clicks another one, so RetroArch got no WM_KILLFOCUS and
GetForegroundWindow() still named it. A front.exe helper now calls
SetForegroundWindow() on the window being given the focus, alongside
the click.
On 26.04's Wine 10 the two lanes fail without it and pass with it, and
the whole script passes.
A reinit that keeps the widgets dropped every message still waiting in
the ring for its first frame. On the first start after an install or
upgrade, the asset extraction reinits video one frame after the first
context init, so the custom Vulkan driver's message never showed.
The ring holds only plain messages (task updates have their own queue),
and gfx_widgets_free() still empties it when the widgets go.
The playlist entry's savestate thumbnail path took the current
savestate folder as its base. That folder is whatever the last
content's states were redirected to (states/<dir>/<core> with sorting
on), or empty when no content has run since start, which sent the
lookup to the content's folder even with a state folder configured.
Every content load starts from the configured folder,
RARCH_DIR_SAVESTATE, so the preview does too.
As a load does, the preview leaves sorting alone when there is no
folder to sort under, and when no sorting applies and the folder is
missing it looks beside the content, where states are then written.
gfx_thumbnail_preview_test lays out a stale current folder to check
the configured one is used, and a missing one to check the fallback.
formats/ciso reads a CISO file as the disc image it stands for, with
no file I/O of its own: like formats/rvz, it describes each range of
the file it needs as a request and the caller supplies it. A read asks
for one range per run of stored blocks, straight into the read's own
destination, and blocks left out are zeroed in place; the head lands
in the block map's own storage and the map is built over it. Heads
that are not CISO, PSP CSO included, are refused.
The scanner takes .ciso files through the serial lookup, reading the
disc header through the reader on a descriptor-path stream.
ciso_test covers the reader and the database scan test a GameCube and
a Wii image, and one whose disc starts with a block left out.
video_vulkan_driver_path is an ordinary config key, and the config is
chosen by whoever launches RetroArch (the CONFIGFILE extra), so a config
could name a driver zip that RetroArch would extract and run.
A pick in the menu now records the zip's path and version in a trust
file in app-private storage. Context init uses the system driver for
any other driver and asks the user to pick it again. When no trust file
exists yet, drivers already extracted are trusted, so current users do
not have to pick them again.
runloop_core_dylib_load resolved the core's path to check that it
lies in the app's own folders, then handed the original, unresolved
path to dlopen. Any symlink on that path could be repointed between
the check and the open. android_core_path_allowed becomes
android_core_path_resolve, which hands back the resolved path it
checked, and that path is the one opened. Which cores are allowed is
unchanged.
A launch intent's LIBRETRO, DATADIR and CONFIGFILE extras are chosen by
whoever sends it, so another app could have RetroArch dlopen any .so.
Every core open now resolves the path and refuses it unless it lies
under the app's dataDir or nativeLibraryDir, or in one of the package's
own code folders in its install folder, all read from ApplicationInfo
rather than from the extras. A Play core install moves the package to a
new code folder while RetroArch runs, so a fixed nativeLibraryDir is
not enough.
free() returned when SNDCTL_DSP_RESET failed, which is what a device
that has gone away does, leaking the descriptor, the stage and the
driver. The reset is now best-effort and the device is closed and
freed either way.
samples/audio/oss_report wraps close and frees a heap driver whose
reset fails; LeakSanitizer in the CI lane catches the old early return.
net_http_connection_set_tls() hands a request its own
ssl_socket_retro_config: trust anchors, a client certificate, suites.
Such a request runs on a connection of its own, neither taken from
the pool nor returned to it, so settings never leak between requests
to the same host. With another TLS backend the request fails.
tls_http takes the settings and a sequence of GETs; the local server
matrix covers a client certificate over AES-256-CBC and pool
isolation in both directions against a keep-alive server.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LBQPGdk3hhQeqHc7WNDcYr
Both focus-lost lanes failed in one run with the core seeing B pressed
and never released. The lane closed RetroArch 1.5 s after giving the
other window the focus, and the release reaches the core only on its
next poll after Wine hands the focus over, which a loaded runner can
take past that. It now watches the log for the release, for at most
15 s, before closing.
The turns lane failed with 2 of 200 asks finding the core 12 turns
ahead against a limit of 4. The count is read just before the
frontend asks, so a frontend thread taken off its CPU between the two
lets the core run on however fairly it gives way. Such an ask is now
made a second time and counted by the second: the preemption does not
repeat, while a core that does not give way is far ahead again.
Under Wine, 8 runs of the fair core peak at 2 turns; 6 runs of the
unfair one are still caught, 4 to 18 asks over the limit at up to 110
turns ahead.
ssl_socket_retro_configure() gives one connection its own settings
before it connects: trust anchors in place of the bundle, whether to
verify the chain and the hostname, the suites to offer, and an RSA
client certificate chain with its key. A 1.2 CertificateRequest is
answered with that chain and a CertificateVerify signed with SHA-256,
SHA-384 or SHA-1 as the server allows, or with an empty Certificate
when there is none to give; a connection presenting a certificate runs
1.2. Such a connection neither resumes a cached session nor leaves one
behind, so no session crosses from one policy to another.
Offered only when a connection names them, the RSA key exchange
(premaster encrypted with rsa_pkcs1_encrypt(), the leaf checked for
keyEncipherment through x509_verify_chain_ex()) and the AES-CBC
suites with HMAC-SHA1/256/384 records, whose padding and MAC are
checked together so that a bad one fails the same way either way. The
default ClientHello is unchanged. These are what the servers of the
Wii U and other consoles of its time expect.
tls_config drives a configured connection in the local server matrix:
own anchors, no hostname check, every new suite over a 1 MB transfer,
a required client certificate from PKCS#1 and PKCS#8 keys under each
signature hash, and a CBC record changed in flight refused.
map_read sized chd->map as sizeof(map_entry) * totalhunks straight from
the header. With a 32-bit size_t the product wraps (0x0CCCCCCD hunks of
the 20-byte i386 entry allocate 4 bytes), and the extraction loop then
writes a full 512-entry chunk past the end of the buffer.
The map follows the header as one record per hunk plus the end-of-list
cookie, so the file size now bounds the count, and the count must also
keep the allocation inside size_t; either failure is CHDERR_INVALID_DATA
before anything is allocated. Regression lane chd_map_alloc_test opens
v3/v4 images from memory, including both wrapping counts with a full
map chunk present.
rsa_pkcs1_sign() signs through the CRT with a fixed-window
exponentiation that scans its whole table, and checks the result with
the public exponent before handing it out, so a bad CRT parameter or a
fault cannot leak the factors. SHA-1 and MD5 || SHA-1 (TLS 1.0 and
1.1) join the hashes PKCS#1 v1.5 signs and verifies with.
x509_rsa_private_key_parse() reads PKCS#1 and PKCS#8 keys as views
into the DER.
sect233r1 (NIST B-233), the curve of the DSi, Wii, Wii U and 3DS
certificates and tickets, gets key generation, ECDH and ECDSA signing
and verification: a fixed-length Lopez-Dahab ladder on X/Z, peer points
checked against the curve and the order-n subgroup, and RFC 6979 nonces
with HMAC-SHA256. Its field product runs on PCLMULQDQ or PMULL where
the CPU has them, chosen at run time as crypto.c chooses its GHASH.
The crypto suite checks the RFC 6979 B-233 vectors, an OpenSSL ECDH
vector and the refusal of an order-2n point, and RSA signatures that
match OpenSSL's byte for byte from PKCS#1 and PKCS#8 keys;
crypto_threads_test covers the new CPU check.
A call to a function other than main is expanded where it is made, as
fxc does, rather than emitted as a subroutine: Wine's DXBC
translator refuses call and label. The parameters, locals and result
take temporaries above the caller's; an indexed parameter or local
keeps one indexable temporary, the function never being active twice.
A return before the body's end leaves through a loop wrapped around
the body, and one from inside a loop or switch of the body sets a flag
that each enclosing loop and switch breaks out on.
rslc_ext_test builds a stage with early returns, one from inside a
loop, and an indexed parameter, and checks no bytecode it writes holds
call or label.
Under RSLC_EXTENDED a pixel stage reads the layer its primitive was sent
to through gl_Layer: SPIR-V's Layer builtin, flat, with the Geometry
capability; SV_RenderTargetArrayIndex in HLSL; [[render_target_array_index]]
in MSL; and in Direct3D bytecode an input after the position, where a
geometry stage writes it, the position declared even when unread so the
two signatures line up. Writing it there is refused.
rslc_ext_test builds a layered pixel stage to every output and checks the
write is refused.
A geometry stage now compiles to DXBC for shader model 4.0, 4.1 and 5.0:
the input primitive, output topology and vertex count, the stream (5.0),
instancing through gl_InvocationID (vGSInstanceID, 5.0 only), the input
vertices as two-dimensional v[vertex][register] operands copied into
temporaries where main starts, gl_in's position and clip and cull
distances after the block members, as the vertex stage places them,
EmitVertex as the outputs copied out and emit (emit_stream m0 on 5.0),
EndPrimitive as cut, and gl_Layer as SV_RenderTargetArrayIndex in a
register of its own after the position. A 5.0 geometry stage's outputs
are in an OSG5 chunk.
A geometry stage's interpolated position input is declared
dcl_input_siv and its clip distances plain inputs, as fxc does, and
every output signature element now records the components outside its
mask as never written, as fxc does, so the signatures match its bytes.
rslc_ext_test also builds the two geometry fixtures to bytecode.
RSLC_DST_DXBC compiles a vertex, pixel or compute stage to a DXBC
container of shader model 4.0, 4.1 or 5.0 (opts.version 40 / 41 / 50):
the input and output signatures and the program, checksummed, in the
result's spirv / spirv_len words. The interface is the HLSL printer's,
register for register: varyings are packed into signature registers the
way fxc packs that printer's stage structs, so a stage from here links
with one compiled from it, and blocks, samplers, storage buffers and
images take the same slots.
Values live in four-component temporaries, a variable indexed with a
non-constant in an indexable temporary, outputs in temporaries copied
out where the stage ends. Functions are subroutines owning their own
registers, one per binding of their sampler parameters. Workgroup
memory is raw thread-group storage; storage buffers are raw views.
Geometry stages, subpass inputs and subgroup operations have no form
here yet.
Under RSLC_EXTENDED a stage may reduce over its subgroup and a fragment
stage may ask whether it is a helper. SPIR-V takes the GroupNonUniform
instructions at subgroup scope, and a module that uses them is SPIR-V
1.3; every other module stays 1.0. MSL prints simd_is_first, simd_min,
simd_max and simd_is_helper_thread, the last where it is read. GLSL and
HLSL refuse the subgroup operations, HLSL the helper invocation too.
rslc_ext_test checks the bounding box's reduction, and its check target
validates that stage against Vulkan 1.1.
A fragment stage may declare a float input attachment with its
input_attachment_index and read it with subpassLoad. SPIR-V gets a
SubpassData image read at (0, 0) and the InputAttachment capability;
MSL takes it as the colour attachment's value, a [[color(N)]] entry
parameter handed to the functions that read it, as SPIRV-Cross prints
framebuffer fetch. GLSL and HLSL refuse it.
The core group of the libretro-common samples job failed with
"Target not classified": rslc_ext_test, added with RSLC_EXTENDED's
array, multisampled and buffer samplers, is built but was in neither
RUN_TARGETS nor NORUN_TARGETS. It runs standalone and passes, so it
joins rslc_test and rslc_build_test in RUN_TARGETS.
* Fix core info cache ignoring late .info files and dropping cached entries on rewrite
* core_info: write the cache straight from the core list
The cache writer takes the core list's entries as they are instead of
a deep copy of every installed core, and a core read from its info
file no longer gets copied into a cache list that is not written.
core_info_cache_test also checks that an unchanged core set leaves the
cache alone, and runs under ASan and UBSan in Linux-samples-tasks.
* core_info: read cached entries in place, take core paths from the listing
The cache reader parses each entry straight into the next slot of the
cache list instead of a separate allocation moved in afterwards, and a
firmware list grows geometrically rather than one entry per realloc.
A core's path is taken over from the directory listing, which is freed
at the end of the same pass, instead of being copied.
core_info_cache_test adds a start served from the cache alone, with
more cores than the list starts with and long firmware lists, and a
cache cut off inside an entry, which must be discarded without a leak.
* string_list: add packed splits, use them for core info lists
string_split_packed() splits into a single allocation holding the
list, its elements and the tokens, marked by cap 0 with elements
present. string_list_free() releases it in one free; growing it and
string_list_deinitialize() are refused, so a packed list stays
read-only.
Every pipe-separated core info field is split this way, from the
cache and from info files alike, which takes a warm start over the
full libretro-core-info set from 16661 allocations to 8851. The
string_split_packed sample checks it against string_split and the
refusals, under ASan and UBSan and as a mingw-w64 build under Wine.
---------
Co-authored-by: libretroadmin <[email protected]>
RSLC_EXTENDED only, so parallel-gamecubewii's Metal backend can take its
MSL from rslc instead of SPIRV-Cross. Matched against SPIRV-Cross's MSL
for every shader the core generates:
- sampler2DArray, sampler2DMSArray and samplerBuffer as texture2d_array,
texture2d_ms_array and texture_buffer; sampling, texelFetch and
textureSize on them; arrays of samplers as array<...> by reference.
- Every uniform block and storage buffer the stage uses, at the buffer
index of its binding; two the stage uses may not share one. Storage
buffers are device (const device when read only) with an unsized
last member of one element, and their atomics atomic_*_explicit. A
block's bool is a uint, turned back on read; a struct member is taken
when it lays out as the block does.
- gl_ClipDistance / gl_CullDistance in a [[clip_distance]] array and
each as a user(clipN) varying.
- A compute stage: kernel main0 with the thread position builtins,
workgroup memory as threadgroup C arrays, images as texture2d(_array)
with their access, imageLoad / imageStore / imageSize and the barriers.
- Only the inputs, blocks and samplers the stage uses are declared, as
Metal wants a vertex descriptor entry for every attribute named; both
outputs of a dual-source pair name their index.
In every build, a case section of a switch that declares something and
has another case after it is a block in MSL, as C++ forbids jumping past
the declaration; outside fragment stages texture() samples level 0 in
MSL and HLSL. The slang-shaders corpus compiles byte-identical to before
in the stock build.
The VFS reads the configured server whatever host a URL names, so a
scan or refresh no longer compares the host: a playlist saved under an
older server address refreshes after the SMB server setting changes,
and the check takes no name lookup. A share other than the configured
one is still refused.
vfs_test checks URLs of other hosts, unresolvable ones included, pass
on their share alone.
From the menu, the main thread stat'd an smb:// content folder up to
three times before its scan task ran: for the playlist name, for
whether it is a file and for the scan's folder flag. Manual Scan, Scan
Directory, Scan File and a lone Refresh Playlist each did, and against
a server that does not answer each stat waits out a connect with the
menu frozen.
The manual scan settings now keep what the caller knows the content
folder to be. The menu's folder pick, Scan File and Scan Directory say
whether it is a folder or a file, a refresh says its saved folder is a
folder, and --scan and the chain's refreshes take it from the worker's
read of the path. For an smb:// path of known kind those three places
use the kind instead of a stat: the menu paths stat it none at all on
the main thread, and a --scan or refresh of a reachable share is not
stat'd again when its scan is pushed. The kind is kept with the
settings a scan chain saves and puts back. A local path, and an
smb:// one of unknown kind, stats as before.
A scan or refresh of an smb:// folder that could not be read said
"not found" or "skipped: content folder not found", whatever the
cause, and logged nothing; in the menu, a scan scanned the folder path
as a single file. A URL naming another host or share than the SMB
settings was read from the configured server and share all the same.
The scan task, the --scan path task and the playlist load of a
refresh now check such a folder on the task worker before the scan
starts: no SMB server set, a URL host or share other than the
settings', or, when the folder cannot be read, the share's recorded
connect failure (name lookup, connect, login, share). A URL refused
for its host or share is never stat'd by them. The reason goes to the
menu notification, one log line and the --scan line or the refresh's
skip line. The item still fails, or the playlist is still skipped and
left untouched, as before. A folder the server reports missing keeps
the message it had.
The man page states the host and share rule.
A failed connect kept only the client's error text, the last of any
thread's, so a caller of stat could not tell a wrong password from a
missing folder.
Each pool now keeps the outcome of its last connect: connected, or the
step that failed (name lookup, connect, login, share), from the
built-in client's phase or libsmb2's NT status. smb_url_failure()
reads it for a URL without I/O or taking a use.
The VFS also connects to the configured server whatever host a URL
names, and a configured share overrides the URL's, so a scan of
smb://otherhost/... reads the configured server and saves the other
host into the playlist. smb_url_check() says whether a URL names the
configured server and share: hosts compare as text, ignoring case and
without a user@ prefix, a port or IPv6 brackets, and only on a
mismatch are both names looked up; the share is compared only when
one is configured. It may block on that lookup, so it is for worker
threads.
vfs_test checks each failure code against a local smbd, frozen server
included, and the URL checks without a server, the URLs the menu
builds from the settings included. vfs_threads_test reads the code
beside the pool evictions under TSan.
rsmb_connect() kept only an error string and the last reply's status,
so a caller could not tell a wrong password from a missing share or a
server that is not there. The context now records the step a failed
connect was on (lookup, TCP connect and negotiate, session, tree
connect; a send or receive that fails counts as the connect), read
with rsmb_get_phase(), and smb2_get_phase() for the VFS's compat
layer. smb_test checks each step against a local smbd.
Without RSLC_EXTENDED a stage holds only the uniform and push blocks,
so pr_block_instance's index test is a compile-time true and GCC no
longer threads an out-of-range path into prog.blocks[] (-Warray-bounds).
The rslc sample builds at -O2 with -Werror=array-bounds so the bounds
checker sees the default build's fixed-size block arrays.
* Add optional composite sync support to CRT SwitchRes
* CRT SwitchRes: fall back to separate sync where CSync cannot apply
An invalid csync value, or a display server without composite sync,
now falls back to separate sync with one warning instead of aborting
every mode switch, which left the desktop's separate sync on the wire
anyway and broke the EDID writer whenever csync was set. With csync
off, listed modes are selectable as before, polarity included; only
the modes generated here are held to the requested polarity.
XRandR takes over a same-named RA mode only when its timing is
identical, so a previous session's leftovers are reused and cleaned
up, and suffixes the name otherwise; the name scan uses one resources
fetch. The CRT sample and the live Xorg sample cover both.
---------
Co-authored-by: libretroadmin <[email protected]>
RSLC_EXTENDED only, so parallel-gamecubewii's D3D backends can take
their HLSL from rslc instead of SPIRV-Cross. Each piece is laid out the
way SPIRV-Cross lays it out, so the backends bind what they bound:
- An in / out block is a static of its struct; its members travel as
Block_member fields, one TEXCOORD location each from the block's.
- gl_ClipDistance / gl_CullDistance travel in one floatN with an
SV_ClipDistance0 / SV_CullDistance0 semantic (four at most).
- A storage block is a ByteAddressBuffer at t<binding> when readonly and
an RWByteAddressBuffer at u<binding> otherwise, named after the block;
reads Load at the member's byte offset, an element that is a struct
through a loader made for it, and the atomics through Interlocked
helpers. A plain store has no form yet and is refused.
- A compute stage: [numthreads], SV_DispatchThreadID, SV_GroupThreadID,
SV_GroupID and SV_GroupIndex, groupshared memory, the barriers, and
four-component storage images as RWTexture2D(Array) at u<binding>.
- A cbuffer may hold a struct whose members are all four-wide vectors,
which packs alike in HLSL and std140.
- A vertex input takes no interpolation modifier, and a buffer or
multisampled texture no sampler state.
Every shader parallel-gamecubewii generates for D3D, the ubershaders, the
utility shaders, the texture decoders and a sample of specialized ones,
prints HLSL that glslang's HLSL front end accepts. The slang-shaders
corpus compiles byte-identical to before in the stock build.
RSLC_EXTENDED only, for the texture decoders parallel-gamecubewii runs
on the GPU:
- RSLC_STAGE_COMPUTE, its layout(local_size_x/y/z) in; (1 when unset)
and gl_GlobalInvocationID, gl_LocalInvocationID, gl_WorkGroupID,
gl_NumWorkGroups and gl_LocalInvocationIndex.
- image2D and image2DArray in float, int and uint, with their format
in the layout and readonly / writeonly; imageStore, imageLoad and
imageSize.
- shared globals, in Workgroup memory.
- barrier, memoryBarrier, memoryBarrierShared, memoryBarrierBuffer,
memoryBarrierImage and groupMemoryBarrier, with glslang's scopes and
semantics.
SPIR-V and GLSL; HLSL and MSL refuse the stage. Every decoder the core
generates compiles, passes spirv-val, and prints GLSL glslang accepts.
The slang-shaders corpus compiles byte-identical to before in the stock
build.
What parallel-gamecubewii's shared utility shaders need, every one of
which it compiles when its Vulkan backend starts:
- An in / out block may go without an instance name; its members are
then named on their own (RSLC_EXTENDED).
- A geometry stage's input instance[] takes its length from the input
primitive, and gl_in[] is there with gl_Position, gl_PointSize and the
clip and cull distances, the arrays as long as the highest element
read (RSLC_EXTENDED).
- A bool, bvec in a uniform or storage block is held as a uint, uvec in
SPIR-V, as Vulkan has no bool in memory; reads compare against zero,
writes select 1 or 0.
- A fragment stage writing gl_FragDepth declares DepthReplacing, as
Vulkan requires.
The slang-shaders corpus compiles byte-identical to before in the stock
build.
Behind RSLC_EXTENDED, for parallel-gamecubewii's bounding box and its
dynamic vertex loader:
- buffer Name { members } [instance]; with readonly, writeonly,
coherent, restrict and volatile: std430 unless it says std140, at
its own set and binding, its last member possibly an array of
unstated length. SPIR-V 1.0 has it as a BufferBlock in the Uniform
class, an unsized member as a runtime array, and readonly and
coherent as NonWritable and Coherent on the members, as glslang
writes them. A member of a writable buffer is an lvalue.
- atomicAdd, atomicMin, atomicMax, atomicAnd, atomicOr, atomicXor,
atomicExchange and atomicCompSwap on an int or uint member of a
writable buffer, at device scope with no ordering.
- a further uniform block is std140 whether or not it says so.
GLSL prints the buffers; HLSL and MSL refuse them yet.
samples/gfx/rslc_ext adds the core's bounding box and vertex-fetch
shapes; spirv-val takes both.
Without the define the slang-shaders corpus compiles byte for byte as
before to every destination; the instructions it takes differ by
0.06%.
* fix(ps3): overlap core rendering with RSX flips while preserving VSync
Wait for the pending flip before frontend composition so the core can
render while the flip is in progress. Keep flip completion synchronized
when menu or overlay uploads are active.
With VSync enabled, present the cached image for duplicate frames even
when no UI is visible. This prevents emulation from running too fast
when audio sync is disabled.
Cover flip timing and software and hardware frame duplicates with and
without visible UI in the RSX regression tests.
* rsx: keep the core behind the flip with two swapchain images
Max Swapchain Images now chooses the RSX queue depth: two keep the
CPU behind each vsync flip for the lowest input latency, three let
the core run into the pending flip. The setting is shown for RSX.
A vsync dupe redraws the last frame that loaded; a failed upload no
longer becomes the cached frame, and with nothing cached the dupe
leaves the screen as it is instead of presenting a cleared buffer.
rsx_alloc_test models the GCM flip status, so a wait on a flip that
was never reset or queued shows up, and gains flip_latency and
dupe_keeps_image lanes.
* PS3: default to two swapchain images
RSX keeps the core behind each vsync flip with two images, the lowest
input latency; three, letting the core run into the pending flip,
stays available as a setting.
---------
Co-authored-by: LibretroAdmin <[email protected]>
resolute-updates published linux-image-generic 7.0.0-43 before the
linux-image-unsigned and linux-modules packages it names, so run_vm.sh
took 7.0.0-43 and could not download it. hwe_kernel() now walks every
version of the metapackage, newest first, and takes the first whose
unsigned image and modules both have a candidate: 7.0.0-42 today, 43
once it is published. The CI cache key comes from the same function.
smb_cfg_race_test failed under TSan: two reader threads wrote the same
smb_err_ring slot. The slot came from a counter taken modulo the ring
size, so once the counter wrapped, two writers still filling their
messages could land on one slot. Each slot now has a busy flag a writer
takes with a compare-and-swap, moving on to the next slot when it is
taken, and the slot last published is skipped, since a reader may be
holding it. With every other slot in use the message goes unrecorded.
NFS fills its ring the same way and gets the same change.
smb_cfg_race_test failed 7 of 8 runs under TSan before and 0 of 20
after; nfs_cfg_race_test and nfs_error_test pass.
RetroFW's jz4760 OSS driver has no poll method, so select() reports the
descriptor ready at once whether or not a fragment is free. Eight wakes
without room now latch that: from then on wait_writable() claims the
room and the blocking write sleeps in the driver on its fragment-done
interrupt, rather than handing back 0 for the audio thread to ask again
at once - at real-time priority on a single core, nothing else ran.
A non-blocking descriptor claims nothing. Logged once at teardown.
samples/audio/oss_report models that driver with wrapped ioctl, select
and write: a full poll-less device hands every pass to the write, and a
device whose select() does wait still skips the pass.
Fixes#20006
Behind RSLC_EXTENDED, for parallel-gamecubewii, which runs a geometry
stage for wide lines, points and stereo, and passes its varyings
through interface blocks whenever one is in use.
- in / out Name { members } instance[N]; in every stage: a struct
variable at the block's location, decorated Block in SPIR-V, each
member keeping its own flat / noperspective / centroid / sample
(an integer member of a fragment input is flat). GLSL prints the
block as written; HLSL and MSL refuse one yet.
- RSLC_STAGE_GEOMETRY: layout(points | lines | lines_adjacency |
triangles | triangles_adjacency, invocations = N) in;,
layout(points | line_strip | triangle_strip, max_vertices = N) out;,
an input block as an array with one element per vertex,
gl_Position, gl_PointSize, gl_ClipDistance, gl_CullDistance through
gl_PerVertex, gl_InvocationID, gl_Layer, EmitVertex() and
EndPrimitive(). SPIR-V and GLSL; HLSL and MSL refuse the stage.
samples/gfx/rslc_ext compiles a wide-line geometry stage shaped as
the core generates it, and blocks between a vertex and a fragment
stage; spirv-val takes every module and glslang the printed GLSL.
Without the define the slang-shaders corpus compiles byte for byte as
before to every destination; the instructions it takes differ by
0.05%.
Behind RSLC_EXTENDED, for parallel-gamecubewii, whose vertex shaders
write two clip distances whenever depth clamping is in use.
Each array is as long as the highest element the stage writes, as
glslang sizes an implicitly sized one; the index must be a constant
below 8, the least maxClipDistances Vulkan guarantees. The arrays sit
in gl_PerVertex at those lengths, with the ClipDistance and
CullDistance capabilities only when written. GLSL prints them as
written; HLSL and MSL have no form of them yet and refuse the stage.
Without the define the slang-shaders corpus compiles byte for byte as
before to every destination, in the same time.
More of what parallel-gamecubewii's shaders need, behind RSLC_EXTENDED
like the samplers before it:
- more than one uniform block in a stage: each further block takes a
slot past the push block and keeps its own set and binding (HLSL
register(bN) at the binding). The reflection tables still describe
only the first uniform block and the push block.
- sized arrays of samplers at one binding, indexed by constants or by
any expression; HLSL indexes the texture and its sampler state
alike.
- layout(index = 1) on a fragment output, for dual-source blending:
SPIR-V Index, GLSL index =, HLSL the next SV_Target, MSL index(1).
- layout(early_fragment_tests) in;: SPIR-V EarlyFragmentTests, HLSL
[earlydepthstencil], MSL [[early_fragment_tests]].
MSL refuses a third block. The block count stays at two without the
define, and the slang-shaders corpus compiles byte for byte as before
to every destination, and no slower.
A build with RSLC_EXTENDED defined takes what an emulator's own
shaders need and the frontend's never do. This is the first part,
for parallel-gamecubewii, whose generated shaders go to rslc in place
of glslang and SPIRV-Cross:
- sampler2DArray, sampler2DMSArray and samplerBuffer, each with its
i and u forms
- texture, textureLod, texelFetch and textureSize on them: an array's
level and a multisampled image's sample go where SPIR-V, GLSL and
HLSL want them, and a multisampled or buffer image's size is
queried without a level
- gl_SampleID, which makes a stage that reads it run per sample
(SampleRateShading, flat)
SPIR-V, GLSL and HLSL print all of it. MSL has no form of the new
samplers yet and refuses them with an error.
Without the define those names are not keywords and the new
overloads are not in the builtin table. The 3346 stages of
libretro/slang-shaders compile to SPIR-V, GLSL 450, HLSL 50 and MSL
byte for byte as before, and no slower.
samples/gfx/rslc_ext tests every form; its check target also runs
spirv-val on the modules when it is installed.
The macOS metal-harness failed "frame memory lane, copied frames" with
17 of 40 frames copied by the CPU, on a commit that touched only
.gitlab-ci.yml; the run before it passed. The lane pushed its 60 frames
back to back. The video thread hands a frame buffer back on the pass
after the GPU has finished with it, so on the runners' paravirtual GPU
every one of the driver's buffers could still be in flight at the next
push, which then took a host buffer: the GPU's latency, not the path
the lane checks. The frames now go a display period apart, as a core's
do.
* android: add quest build
* android: quest builds through the existing Android lane
Android.yml's assembleDebug already builds, alignment-checks and uploads the questDebug APK, so the separate workflow is dropped. The GitLab release job stays.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Since ubuntu-latest moved to 26.04, CI Linux (i686) stops at checkout:
git fetch cannot resolve github.com from the 32-bit xenial image, ten
tries in a row, and the public resolvers the job passes with --dns do
not change that. The 64-bit container jobs on the same runners check
out and build. The job goes back to ubuntu-24.04, where it last
passed, like the mipsel lanes.
menu-harness failed its hot-function budget on Ubuntu 26.04:
audio_driver_sample_batch came out at 623 bytes against a 512 ceiling,
with runloop_iterate and video_thread_frame a few bytes short of
theirs. Built from the same tree, GCC 13 emits 511, 6911 and 4319
bytes for the three and GCC 15 623, 8026 and 4825, so the step is the
compiler and not the source. The reference now holds what GCC 15
measures on the runner, frames included; the quarter of slack above it
is for the next real change.
webm_av1_hw_test and preview_av1_hw_test still called make_mp4, which
the last change removed, and took the MP4 writer through
rav1_webm_fixture.h, which now asks for the byte helpers alone. Both
include rav1_mp4_fixture.h first for the whole writer and call
make_mp4_clip(1, 0).
FFmpeg 7.1 and later take a frame's primaries and transfer into the
encoder over the -color_primaries and -color_trc output options, so
libaom, libvpx and libx265 without its own colour parameters write
untagged streams. mp4_mux's HDR sources lost their transfer and run.py
stopped on the missing key; the frames are now tagged with setparams.
The same goes for the hand-run generators that tag through those
options alone: gen_av1_decode.py's colour stream, the D3D11 HDR AV1
fixtures and mkhdrfixtures.py's VP9 movies.
FFmpeg 8's concat also converts every segment to the first one's
colour space, so the ffmpeg_live colour-metadata lane could no longer
switch from BT.709 to BT.601 inside one lavfi graph: the second half
arrived converted and labelled BT.709. The lane now plays one BT.709
segment and its receive_frame wrapper relabels the frames after the
third as BT.601, which is the transition the driver has to follow.
The webm_player timeline-gap fixtures cut their silent second with
aselect before the encoder, and FFmpeg 8 closes that gap: the next
frame takes the previous one's end, so the packets ran on unbroken and
the seek into the gap found signal. The packets are now dropped after
the encoder with the noise bitstream filter, which keeps every
timestamp and gives the same packets on FFmpeg 6.1 and 8.
gcc 15 threads a path into the reference-preset check on which it sees
config_dir at address zero and warns that strlen reads past it in every
build. The length is taken once, right after the directory is filled.
Built with ffmpeg 8, libaom 3.13, x265 4.1, glibc 2.43 and clang 21,
five lanes failed on the 26.04 runners:
- AV1 fixtures: a monochrome sequence comes out full range with no
matrix whatever is asked for; the oracle takes the colour ffprobe
reads instead of assuming it.
- Preview audio oracle: ffmpeg 8 leaves a timestamp gap at every seam
of a -stream_loop copy, which the mixer honours and ffmpeg's raw
decode does not. trailing_large loops its picture and encodes its
sound as one track.
- rh265 multi-slice oracle: x265 4.1 writes unreadable slice headers
when two slices are encoded on several threads; those cases encode
on one worker.
- vkms: the kernel follows the runner's own release, its modules may
sit under /usr/lib/modules, and modules-extra is fetched only where
it exists.
- clang -std=c89 -pedantic: glibc's const-generic strchr/strstr under
_GNU_SOURCE are _Generic; -Wno-c11-extensions goes wherever
-Wno-c99-extensions already does.
ci_paths filters a file at the top of a samples tree by its own path.
rh264 and rh265 lose the parameters no instantiation reads, the 8-bit
H.264 kernels mark the depth they ignore, and the AV1 WebM fixtures
leave out the MP4 box writer they never use.
A program that generates its shaders today prints GLSL and has rslc
parse it again. rslc_build.h lets it make the same stage by calls:
types, inputs and outputs, the uniform and push blocks, samplers,
functions, expressions and statements, then rslc_b_end emits any
rslc destination. Nothing is printed or parsed on the way.
The builder sits in rslc_front.c, under RSLC_BUILD, and reuses the
parser's own typing helpers: implicit conversions, builtin overload
resolution and constant folding are the same code, so a stage built by
calls compiles to the same words as the same stage written as GLSL.
Statement makers return nodes, so an if, loop or switch takes a single
statement or a block alike. An error poisons the build: every later
call returns 0 and rslc_b_end reports it, so a caller checks once.
Without RSLC_BUILD nothing changes: rslc_front.o's text is the same
size and the griffin build and size budget are untouched.
samples/gfx/rslc_build writes a vertex and a fragment stage both ways,
covering blocks, matrices, swizzles, structs, const arrays, out
parameters, every loop kind, switch, discard, ternaries and texel
fetches, and checks the SPIR-V, GLSL and HLSL are identical; it also
checks the error paths. Claimed in the samples workflow.
On 26.04 qemu-user-static is a virtual package with two providers, so
apt refuses it and the cross jobs fail before building. qemu-user ships
the static emulators there, as qemu-aarch64, qemu-arm and qemu-ppc, the
names 24.04's qemu-user uses too; the cross jobs install it and run
those. libfreetype6-dev is asked for by its real name, libfreetype-dev.
s3_backup_then held six path buffers on the stack, 12.8 KiB, and
s3_read and s3_delete_object two each, a little over 4 KiB. The backup
names now go in one allocation, and the URL and canonical URI of an
object come from s3_object_urls, one allocation holding both, which
s3_update_upload uses too. The largest frame in the driver is now
s3_build_request_url's 2.1 KiB. The hand-copied empty payload hash in
the GET and DELETE goes to S3_EMPTY_PAYLOAD_SHA256.
s3_backup_test fails those allocations for a backup before an upload
or a delete, a plain upload, a delete and a read: nothing starts,
nothing is reported, nothing leaks.
The test never linked under MinGW: net_http.c and its sockets need
ws2_32, which the Makefile left out, as the other network samples do
not. It also wrote its scratch files under /tmp, which a native Windows
build resolves to \tmp on the current drive, usually missing. They go
in the current directory now, and clean removes the .exe too.
An upload read the whole file into one malloc before it started: up to
the 64 MiB multipart threshold for a single PUT, and the whole file,
however large, for a multipart upload, held until its last part was
acknowledged. Out of memory the upload failed, and on the consoles the
threshold is sized for, that is not far off.
The body is now pulled from the file by net_http as the socket takes
it, through net_http_connection_set_content_source, the way WebDAV
already uploads: one send buffer instead of the file. Each multipart
part is a byte range of the file, seeked to on its first run so it
reads its own bytes whatever was read before it, and a replay on a
fresh connection rewinds to the range's start. The payload hash for the
signature is taken by reading the range in 64 KiB runs into
sha256_stream_update. The fallback to a single PUT when the multipart
start fails in transport streams the whole file the same way.
task_push_http_transfer_with_source is task_push_http_transfer_with_
content with a source in place of the buffer. The upload's URL buffers
move off the stack (4.2 KiB to 112 bytes).
s3_backup_test's stub drains every streamed body as net_http would,
with a replay from the start, and checks it is all there and is what
was signed. New lanes: an 11 MiB multipart upload (three parts, each
its own range of the file), its single-PUT fallback, and an empty file.
The multipart threshold can be set by the test for that.
The SigV4 signer assembled the canonical request and the string to
sign in 4 KiB stack buffers and MACed them with its own HMAC. A URI
or copy source past 4 KiB was signed cut short, and so was a secret
key over 123 octets (the setting takes 185), so those requests came
back 403. The two buffers also put the signer's frame at 9 KiB.
The canonical request now goes straight into sha256_stream_update as
it is made, and the string to sign into hmac_sha256_update, both from
crypto/kdf.c: nothing is assembled, so nothing is cut short. The query
is sorted as an index of spans over the caller's string, on the stack
up to 16 parameters, so the signer allocates only its result. The
hand-written HMAC, the query copy and the old signature helper go.
HAVE_S3 now needs HAVE_CRYPTO, which every build that defines it
(Apple) already has; griffin says so if it is missing.
s3_backup_test checks seven signatures against an independent SigV4
computation, including a 5001-character URI and a 150-octet secret,
which master signs wrong. The query and out-of-memory tests follow the
new shape, and the test rebuilds when the driver changes.
A READ reply is read as it is parsed: the RPC and READ headers into
c->rx, the data itself straight to where the octets go, for v3 READ,
v4 READ and each data run of READ_PLUS, across fragment marks. A reply
the fast path does not take - an error, GRACE or DELAY, READ_PLUS
refused, a denied call - is read whole into c->rx and parsed as
before. Nothing is asked of the socket that the reply has not promised,
so a short reply fails rather than waits.
Lanes: local_read_plus_test.sh reads the sparse file again from
fake_nfs42.py with every reply cut into 333-octet fragments and an
empty last one, as READ_PLUS and, with READ_PLUS refused, as READ.
* Android: load custom Vulkan driver packages on Adreno
Sideloaded arm64 builds can load a Vulkan driver package (a zip with
meta.json and one .so, such as Mesa's Turnip) on Adreno GPUs with
Android 9 or later. As libadrenotools does, a private copy of the
system Vulkan loader is opened in a linker namespace of its own, where
a small hook library hands it the custom driver as its HAL. The new
video_vulkan_driver_path setting names the zip; empty is the system
driver, and core, content-directory and game overrides carry it. The
package is checked and extracted to internal storage once per version.
Packages from any vendor are accepted. Only Mesa ones have been
tested, so any other vendor logs that it is untested and that its load
cannot be confirmed. A Mesa package that leaves the stock Qualcomm
driver running, as the hook does when the custom library fails to
load, counts as failed.
When the custom driver fails to create an instance, a device or the
first swapchain, the context is redone on the system driver in the same
process. A driver that takes the process down during its init is turned
off for that core until it is picked again, and a "Safe start" launcher
shortcut starts RetroArch once on the system driver. Play Store, Quest
and API 16 builds do not carry any of it.
Co-authored-by: lim2222 <[email protected]>
* Menu: Custom Vulkan Driver setting
Settings > Video > Output gains a Custom Vulkan Driver row on builds
and devices that can load one. OK opens a browser for driver zips in
<system>/vulkan_drivers, Start goes back to the system driver, and the
sublabel says when the chosen driver crashed during its last start and
is off for the running core. While RetroArch runs from the Safe start
shortcut, the row reads "System (Safe start)" and the sublabel names
the driver the next normal start uses. The choice applies at the next
Vulkan start; picking a driver never restarts video.
Co-authored-by: lim2222 <[email protected]>
* archive: a streamed Deflate member writes no more than its size
A Deflate member streamed to its file a window at a time is now
refused as soon as it inflates past the size its headers give, so no
more than that size reaches the disk, as for Deflate64, LZMA, zstd and
bzip2. Both the zlib and the built-in inflate paths.
Lane: archive_stream_test "inflate bound", a member of stored blocks
that holds 1 MiB under a 300 KiB size, counting what the VFS writes.
* Vulkan custom driver: merge onto master, ask the GPU once
The menu row moves one entry up its table, clear of the manual-scan
row master renamed. Whether the device can take a custom driver is
asked once and kept, rather than each time the Video settings list is
built.
* Vulkan custom driver: extract a picked package in a task
Picking a driver pushes a task that checks the package and extracts
it on a worker, into the dir the next Vulkan start looks in, so that
start only maps the library rather than inflating up to 64 MiB while
the main thread waits. The task reads only what it was handed.
It writes through a .prep dir of its own; a context init that gets
there first extracts through its .tmp as before, and whichever
finishes second keeps the other's. The once-per-process prune leaves
.prep dirs alone while a task runs. A driver set by an override,
never picked, is still extracted at its first start.
---------
Co-authored-by: lim2222 <[email protected]>
Co-authored-by: LibretroAdmin <[email protected]>
Ubuntu 26.04 carries no MIPS cross toolchain, so on ubuntu-latest
apt-install.sh cannot resolve gcc-mipsel-linux-gnu and the compile
matrix and libretro-common test jobs fail before building anything.
Both jobs run on ubuntu-24.04, where the dingux mips lane and the
crypto suite under qemu-mipsel keep running.
gl2 published whether it can draw views only from gl2_frame(), so
until its first frame it reported none. At content start and after
each video driver reinit, a core asking for the views status on its
first retro_run() was told views are neither presented nor stereo,
and that frame was not laid out as views: one frame of the core's
own layout, or of its packed frame shown whole.
gl2 now publishes it at init, as vulkan does.
The test built its fixture under $TMPDIR or /tmp and removed it with
system("rm -rf ..."): a native Windows build has neither /tmp nor a
POSIX shell behind system(), so make check stopped at "could not build
fixture under /tmp/scan_begin_fixture". The fixture goes in the
temporary folder (TEMP or TMP on Windows, TMPDIR elsewhere, else /tmp)
and is removed with rmdir /s /q on Windows.
make check runs database_scan_test twice in one root. The "fs off
main" lane deleted the playlists it wrote in its playlist folder but
not the one outside it, so the second run's write of that playlist
pushed an entry already on top, the push refused it, and the lane
failed at its fixture. It deletes both now.
playlist_parity_test's cleanup let cmd.exe print what rmdir could not
find or remove; the cleanup is best effort, so it says nothing now.
make check handed the tests the POSIX path mktemp gives, and none to
the index test, which then used /tmp. Under MSYS2 the tests are
native Windows programs and could open neither, so the run stopped at
the index test's "test databases could not be written". Each test now
gets its own fresh root, put through cygpath -m where there is one;
elsewhere the path is unchanged.
The autofix lane and the two failed-copy lanes compared paths with the
POSIX strings they pushed. playlist.c stores what playlist_resolve_path()
makes of them, which on Windows is "C:\games\game.bin" for
"/games/game.bin", and autofix writes the rest of a rebased path in the
local form, "/mnt/newgames\dir000\...". So 15 checks failed there
while the copies they test behaved.
The failed-copy lanes now compare with the path as a push stores it,
through playlist_resolve_path() itself; the autofix lane takes either
slash after the new base on Windows.
The test made its fixture with system("mkdir -p /tmp/...") and removed
it with "rm -rf": a native Windows build has no /tmp and its system()
runs cmd.exe, so it stopped at "fixture mkdir failed". The fixture now
goes in the temporary folder (TEMP or TMP on Windows, TMPDIR
elsewhere, else /tmp), made with path_mkdir(), and is removed with
rmdir /s /q on Windows.
* file_path: stop at the end of the path after '~'
fill_pathname_expand_special() skipped two characters after a leading
'~' or ':', so a bare "~" read past its NUL and "~x" lost the x. It
now skips the prefix and a slash after it, if there is one.
* scan: hand the callback user data and what the scan did
task_push_dbscan() and task_push_manual_content_scan() take user data
for their callback, and the callback's task_data is now a
task_dbscan_result_t: how many entries the scan added, how many its
playlists already held, and how many files matched nothing. A file is
counted by what became of it, so an archive that is expanded is not
counted as unmatched, and a loose scan's unmatched files count as
added. Callers pass NULL for now.
* scan: say in the result when a scan failed
A scan that ends before it runs (bad setup, no valid content, a DAT
that will not load) or ends with a task error now sets the result's
failed flag. A database folder that is missing or unset is treated as
an empty one, so it reports the same "no database" error instead of
finishing as if it had scanned.
* scan: count the files a scan could not read
A file whose scan fails (an unreadable file, a failed lookup) was in
no count. The result now carries an errors count for them. They do
not fail the scan.
* scan: print --scan errors on stderr and totals on stdout
A command-line scan now ends with a totals line on stdout, or with the
reason it failed on stderr. A file it could not read is listed on
stderr. The ++/==/?? marks are coloured only when their stream is a
terminal, so pipes and redirected output get plain text; on Windows
redirected output keeps its marks instead of losing them to
WriteConsole.
* scan: the manual scan finder skips a finished task
A finished scan stays in the task queue while its callback runs, and
the finder that refuses a second scan for the same playlist still
matched it, so a scan pushed from that callback was refused.
* retroarch: run every --scan, then exit with what the scans did
Each --scan is collected while the command line is parsed and run in
turn once it is done, so later options and further --scan flags are
no longer dropped. A path is expanded and made absolute first; one
that does not exist, or a scan that cannot start, is reported on
stderr and the next path still runs. --scan with --cloudsync or
--command is refused.
Without --menu the exit status is 1 if any scan failed, else 0 if any
added an entry, else 2, and a content path given with the scans is
ignored, with a line on stderr. With --menu each finished scan starts
the next, as a second scan is refused while one runs.
* manual_content_scan: clear the database name a refresh did not save
A playlist with no saved database name left the name buffer
uninitialised, and the refresh compared it against every installed
database, so whatever was on the stack could pick a database the
playlist never named.
* scan: Skip Database References names the scan's one playlist
The option was meant to put the playlist's name in each entry's
db_name, even for a database match (#18641), for users who build
their own thumbnail sets. When the scan picks the playlists, db_name
also picks the playlist an entry goes to, so with the option on every
match went to one playlist named after the content folder.
It now applies only when the scan writes one playlist, and gives
that playlist's name, a loose scan's unmatched entries included. With
the target on <Automatic>, or Fully Automatic with the option left on
from a Custom scan, matches go to their databases' playlists as with
it off; the entries of a custom playlist look their thumbnails up
under its name, not the folder's.
* scan: name the scanned folder in the start line
The "Scanning content" line, in the log and in --scan's output, has
named the database folder since the manual scan and the database scan
were merged (2fdf736d51); before, it named the path being scanned. It
names the content folder again.
* manual_content_scan: reset, scan overrides and their apply
The command line's scan options need each scan to start from the
defaults and to lay their values over them, or over a playlist's saved
settings. Add a reset, a record of those options and a function that
applies it through the existing setters: a core named by path, file
name or display name, a database, a playlist, the database check or a
DAT, extensions and the switches, stopping with a reason at the first
that does not resolve.
A core path is normalised before it is compared. A core's file name
matches in the case given first, then ignoring case, on every
platform; a display name matches exactly. Given another database, a
playlist named after its database keeps that name as a custom one, so
a refresh still writes the same playlist.
The core setter's copy step becomes its own function, so a core found
by path or file name can be set. Nothing calls the new functions yet.
* scan: run --scan items through one chain
The --scan paths ran through a loop in retroarch.c, and with --menu
through a second chain of their own. The scan options and playlist
refreshes to come need one runner for every item, from the command
line and from the menu.
task_push_dbscan_chain() runs a list of items one task at a time: a
prep task lists the databases, then each step is pushed from the last
task's callback on the main thread, so nothing waits and nothing
locks. An item that fails is reported and the run goes on; a
cancelled task ends it. retroarch.c builds the items in the first
argument pass and hands them to the chain; without --menu it waits
for the chain's end and exits as before.
Running out of memory prints "out of memory" on stderr: for an item's
path the item fails, and for the chain itself nothing runs, which
without --menu exits 1.
The chain also prints a scan's override errors, and the warnings for
an option that needs one playlist or does nothing for a single file,
which the scan options of the next commit can trigger.
* retroarch: Manual Scan options for --scan
--scan-playlist, --scan-database, --scan-match, --scan-dat, --scan-core,
--scan-exts and the five switches set the Manual Scan settings of the
--scan before them; a --scan without any stays Fully Automatic.
Values are checked in the first pass, so an option before any --scan,
an empty or bad value, or one too long for its setting ends the run
with its reason on stderr and status 1, before the config loads and
before anything is scanned. Cores, databases and DAT files are
resolved when their scan runs, and one that does not resolve fails
only that scan.
* scan: refresh playlists in the chain
A refresh item rescans one playlist with its saved scan settings, as
the menu's Refresh Playlist does, with the item's scan options laid
over them and written back by the scan. A refresh-all item does the
same for every playlist in the playlist folder that has saved
settings, in the order Manage Playlists lists them, histories and
favorites left out.
A refresh item names its playlist by name or by a path in the playlist
folder. A bare "name.lpl" is the playlist folder's file, and an
absolute path is normalised before it is checked against the folder.
The name is looked up in the folder's listing on every platform, the
exact name first, then ignoring case, since macOS and FAT/exFAT cards
ignore case; the refresh then takes the file's own name, so a
database's playlist stays one. Two or more playlists that match only
in other cases fail the item.
The playlist folder walk and each playlist load run in prep tasks: on
the worker with a threaded queue, under the shared I/O window without
one, each pushed from the previous step's callback. A playlist whose
saved content folder is gone is skipped rather than failed. A listing
that runs out of memory fails the refresh-all; a playlist folder that
is missing or cannot be read lists nothing, and the refresh-all ends
with "no playlists with saved scan settings". A refresh's start line
names its content folder; its totals and failure lines name the
playlist.
* retroarch: --scan-refresh and --scan-refresh-all
--scan-refresh=NAME|PATH rescans a playlist with its saved scan
settings, as the menu's Refresh Playlist does, and --scan-refresh-all
rescans every playlist that has them. Both are items of the same run
as --scan: the scan options after one are laid over the saved
settings and written back, and the exit status follows the same rule.
--scan-playlist after a refresh is refused, so a refresh always
updates its own playlist, and an empty --scan-refresh is refused as
the other empty values are. Like --scan, the refresh options cannot be
combined with --cloudsync or --command; the error names whichever of
the three comes first on the command line.
* menu: Refresh All Playlists
Manage Playlists gets Refresh All Playlists as its first entry when it
lists any playlist, so an empty list still says there are no entries.
It rescans every playlist that has saved scan settings, with those
settings, through the same chain as --scan-refresh-all, and ends with
one notification saying how many playlists were refreshed, skipped
(content folder gone) and failed.
While a content scan runs it is refused with a new message; a refused
scan had none. Other languages come through Crowdin.
While a scan chain runs (Refresh All, or a command-line scan with
--menu), Start Scan, a playlist's Refresh, Scan Directory and Scan
File are refused with the same message, as they would run with the
settings the chain last loaded. task_push_dbscan() returns false then
too, which covers the companion and the iOS caller.
* docs: --scan, refresh and the Manual Scan options in the man page
--help lists the options but not what a script needs: the order the
scans run in, the exit status and what the options set. The man page
now has them, next to --cloudsync, and CHANGES.md has the entry.
* scan chain: writes from workers reach the playlist cache by note
A playlist written by a task - the chain's prep parse, a scan, a
playlist-manager task, whatever thread it runs on - leaves a note on a
lock-free stack instead of reading the cached playlist; the main
thread applies the notes when it next decides whether the cache may
stand in, so no worker reads playlist_cached while the main thread
frees or installs it. Without threads or pointer atomics a write
applies at once.
The chain's prep tasks set their handlers by name, and
thread_read_audit.py fails on a handler set from a
retro_task_handler_t parameter, which would hide every handler passed
in. The chain puts the Manual Scan settings it found back when it
ends, and is no longer running when its done callback runs, so that
callback may start another. task_push_dbscan() says why it refuses,
so the menu and the companions show the same reason, and the
companions' status line takes it from companion_core. --scan and
--scan-refresh set the verbosity override, so a config with logging
off cannot detach the console the totals go to, and refuse a path too
long for the chain's buffers. The Refresh All notification passes the
length written, and fill_pathname_expand_special() stops at the end of
a buffer shorter than the home or application directory.
Lanes: playlist_parity_test writes the cached file from a worker,
once at the same size and then while the main thread reuses and frees
the cache; database_scan_test checks the refusal message, the restored
settings and a done callback that starts the next chain;
fill_pathname_test expands into a buffer shorter than the home
directory.
* playlist: header-only parse for the scan chain's prep
The prep task needs a playlist's saved scan settings and nothing else,
yet parsed every entry and, through autofix, could write the file back
from a worker. playlist_parse_begin_header() stops at "items", rebases
the scan record in memory when the base content directory moved, and
marks the playlist so playlist_write_file() refuses it.
Lane: playlist_parity_test header_only (entries skipped, scan dir
rebased, file bytes untouched, old format).
* scan chain: ask the file system nothing on the main thread
The chain's steps ran on the main thread between its tasks and stat'd
and realpath'd there: a --scan path's resolve and stat, a
--scan-refresh path's two realpaths, the content folder's stat, the
saved and override DATs' stat and size, an override core path's
realpath. On a network share each of those can stall a frame.
- manual_content_scan_fs_read() reads them on any thread, touching
nothing but its arguments; the _fs variants of the content-folder,
playlist and override setters take its answers instead of asking.
The plain setters pass NULL and behave as before, so the menu is
unchanged.
- A --scan path gets its own prep task (dbscan_path_handler); a
--scan-refresh resolves its path inside the load task, and the load
reads the folder, the saved DAT and the overrides' paths once the
header is parsed. The worker gets copies of the item's strings.
- The prep's state is freed in one place.
Lane: database_scan_test "fs off main": six items that each stop
before their scan, on a threaded queue, with path_stat, path_is_valid,
path_is_directory, path_get_size and path_resolve_realpath wrapped:
no call on the main thread.
Not moved: the scan task's own push still stats the content folder
and the DAT (manual_content_scan_get_task_config), shared with the
menu.
* samples: the scan chain's lanes hold on Windows and without realpath
From a native Windows run of both tests:
- database_scan_test "fs off main" refreshed f_gone.lpl through
"pl/../pl/", which only a realpath folds; the sample builds without
RARCH_INTERNAL, where path_resolve_realpath() folds nothing, so the
item was rightly not in the playlist folder. It names the playlist
by its plain path now, which the check still realpaths on the
worker.
- "another case, a database's playlist" read the system name type
after the chain ended, and the chain puts the Manual Scan settings
back then. The item callback records each scan's type instead.
- playlist_parity_test's header lane expected "/moved/snes"; autofix
converts the suffix to the local form, "/moved\snes" on Windows.
- The worker lane's stress read required a core name of every read.
Windows replaces the file with MoveFileExW, and an open that meets
the replace fails: that read is an empty playlist, which has none.
---------
Co-authored-by: LibretroAdmin <[email protected]>
verbosity.c mirrors each log line into the desktop companion under
HAVE_COMPANION_WIMP, which ui_companion_driver.h turns on for every
desktop Windows build, so each sample that links verbosity.c failed
to link there: undefined ui_companion_driver_log_active and
ui_companion_driver_log_msg. samples/ui_companion_log_stub.c gives
the six of them a companion that is never open.
The database sample's main.c used strlcpy() without <compat/strl.h>,
an error with GCC 14 and later.
s3_canonicalize_query_string splits the query on '&', sorts the
parameters by name and then by value in byte order, and joins them
again; names are compared on their own, so a name that extends
another sorts after it. Callers keep URL-encoding each name and value.
When the canonical headers or query cannot be allocated the signer
returns no signature, which s3_build_auth_header_ex passes on as NULL
and every caller treats as a failure.
s3_backup_test checks the sort order, including names that extend
one another, and fails each of the signer's allocations in turn.
Closes#19956
The cnonce is written only when a challenge is accepted and cleared
with the rest of the digest, so webdav_create_digest_auth_header()
builds nothing while it is empty: after a challenge the parser
rejected, the next request goes out with no Authorization header and
the server's 401 is reported as usual.
webdav_reauth_test sends rejected challenges (an open-ended nonce or
realm, no nonce, qop="auth-int") after an accepted one and checks
that no header is built from them.
Closes#19954
Since 0e7a56f171 each GPU index remembers the device it was chosen as,
but a change made in the menu left the old name saved. On the next
driver init the resolver found that name at its old position and went
back to it, so the new choice was ignored, and on EGL contexts the menu
value reverted too. Left, Right, the dropdown and reset now save the
name of the device at the chosen index, and index 0 clears it. An index
past the end of the list leaves the saved name alone. Left no longer
sets index -1 on an empty device list.
* nfs, smb: apply menu setting changes without a restart
The clients took their settings at startup and after config_load
only. Their numbers (NFS version, ports, timeout, read-ahead,
connection count, SMB auth mode) are copies, so a menu change waited
for a restart. Each change now hands the clients the current values.
SMB keys its pools on the whole configuration, so its next connection
uses the new values. The NFS pool matched a free connection by export
alone, so a changed server, version or port could ride an old one.
New settings now close the idle connections and advance a generation;
a slot whose generation or server differs connects again. The pool is
kept rather than replaced, so nfs_init_cfg() cannot free it under a
thread that has just taken a slot.
On version 4 an NFS port of 0 means 2049 and the mount port is
unused; the sublabels say so.
vfs_threads_test changes the ports while a prefetcher holds a slot.
* nfs, smb: log why a network directory listing failed
A failed file browser listing of an nfs:// or smb:// directory showed
"Directory not found" and logged nothing. It now logs one error line
with the URL, the settings the client ran with and the client's
reason.
The NFS client's reasons say what failed: the step and the server's
status (v4.2 mount: LOOKUP roms: NFS4ERR_NOENT), the address a connect
was refused at or timed out on, and RPC rejects such as PROG_MISMATCH
or AUTH_TOOWEAK. A refusal (PERM, ACCESS, WRONGSEC, or a v3 MOUNT
ACCES) adds that a Linux export needs 'insecure' for a client port
above 1023, as this client's always is. A failed walk or redial keeps
its own reason.
Failures that never reach a server record one too: no export, no
server in the URL, no SMB server set. SMB gets smb_get_last_error() to
match nfs_get_last_error().
nfs_error_test checks the reasons without a server and against
fake_nfs42.py.
* nfs: open files keep the settings they were opened with
An open nfs:// file or directory stored only its server and export, so
after a settings change its next read reconnected with the new version
and ports: a v3 handle sent over v4, or the stream's server asked on
another server's port.
Open files and directories now keep the version, ports and timeout
they were opened with and reconnect with those. Each pool slot carries
the parameters it connected with and is reused only on an exact match,
which replaces the settings generation. Read-ahead is client-side and
still follows the settings on every take.
nfs_error_test reads an open file on across a port and version change.
* smb: list a server's shares when no share is set
Browsing smb://server with no share configured never listed anything.
retro_vfs_opendir_impl rejected any SMB handle whose dir was NULL,
which is exactly the handle retro_vfs_opendir_smb returns for a share
list, and leaked it. The backend already reads and closes that handle,
so the dispatcher now accepts it. The UWP dispatcher does the same and
takes an entry's directory flag from the listing instead of a stat.
vfs_test checks the share list through the dispatcher as well.
* nfs: a file opened over a private connection keeps its export
With no NFS Export set, the export comes from the URL and is found when
connecting. When every pooled connection is busy, as it is once a few
read-only streams' prefetchers hold one each, the open goes over a
private connection and the file or directory handle recorded an empty
export. Every later borrow then mounted "": refused on v3; on v4 the
pseudo root, where the path under the export is not found, so such a
stream never got its prefetcher.
Record the export the connection was made with, through a new
rnfs_get_export().
nfs_error_test opens a file by URL while a prefetcher holds the only
slot.
* nfs, smb: settings as immutable copies; pools keyed by the copy
nfs_init_cfg() and smb_init_cfg() copy the settings they are given,
strings included, and publish the copy through vfs_cfg_snapshot.h:
a reader holds the copy it got between enter and leave, and a copy
that was replaced is freed once nothing holds it - retro_retire.h's
epochs where the pointer atomics are lock-free, a reference count
under a short flag elsewhere. Menu edits may rewrite the settings in
place while pool, prefetch and task threads read them.
NFS resolves a URL against the held copy's export without copying it,
and applies the timeout and read-ahead on every take: neither is
part of what a connection must match. SMB pools remember the copy
they were last matched to, so a take on that copy and share finds its
pool without building or comparing a key. A READDIR failure names the
result's own status.
nfs_cfg_race_test and smb_cfg_race_test free the caller's struct right
after handing it over, then resolve against it, and hand one struct
over repeatedly while four threads stat and list; they need no server
and run ahead of the server matrices in local_unfs3_test.sh and
local_samba_test.sh.
---------
Co-authored-by: LibretroAdmin <[email protected]>
A glyph cell looked up in the current frame is not handed to another
codepoint until the frame is over. Where the CPU writes cells into
memory the GPU samples - the RSX's atlas in texture memory, the
textures GXM and GX2 copy cells into - the frames before may still be
drawing from them, and a cell reused for a new glyph shows up in their
place. A consumer now says how many earlier frames that can be
(font_atlas.inflight): the RSX one, GXM as many as swaps may be
pending, GX2 the frame swapped last, and a cell any of them looked up
waits until they are over. The default, zero, is the rule as it was.
font_frame_test fills the cache in one frame with two in flight: the
next two frames take none of its cells, and the third finds every cell
free and the first frame's glyph where it was.
Container jobs resolve through the runner host's Docker network, and
when that upstream resolver flakes the checkout step fails with
"Could not resolve host: github.com" regardless of the image. Pass
--dns 1.1.1.1 --dns 8.8.8.8 so the job network's embedded resolver
forwards to public resolvers instead.
* file_path: path_parent_dir stops at a scheme:// root
The slashes of "scheme://" were taken for directory separators, so
going up from "nfs://host/" gave "nfs://", then "nfs:/", "./" and the
local working directory. The menu's Parent Directory fell off a
network share into the local cwd, and path_mkdir() on a missing share
path recursed down to a local mkdir("nfs:/").
A scheme root now returns "", as "/" and "C:\" do. path_basedir() and
paths without a trailing slash are unchanged, so "cdrom://drive1.cue"
still has the parent "cdrom://".
* menu_displaylist: offer the configured NFS server as a file browser root
The root of every file and directory browser lists the local volumes and
the configured SMB share, but not the NFS server. Scan Directory, Scan
File, Content Scan's Content Location and the directory settings could
not reach nfs:// at all; only Load Content's Browse NFS Export could, and
it offers no way to scan or pick a folder.
The root now gains the nfs:// entry from menu_displaylist_build_nfs_root()
when an NFS server is set, the same way the SMB share appears.
* menu: Parent Directory stops at the configured NFS or SMB root
Going up from the configured network root could still reach levels that
cannot be browsed. With the NFS export given in the server address
(nfs://host/mnt/user/roms), every level above the export shows "Directory
Not Found". With an SMB share set, smb://srv/ lists the share's root
again, and every folder opened from there lists that root again.
Parent Directory now goes to the drive list from the root the settings
describe, without the subdir: nfs:// plus the server address, or smb://
plus the server and, when a share is set, any one folder after it, since
the SMB VFS opens the configured share whatever that folder is called.
The subdir remains a start folder that can be left. Other paths keep
stopping at scheme://authority/.
* menu: Content Location offers the same places as Load Content
Content Scan's Content Location opened its folder picker at the start
directory, so a configured NFS export or SMB share was only reachable
by climbing Parent Directory up to the drive list.
It now opens Load Content's list of places first, built by the same
code: Start Directory, Downloaded Content, the drives, Browse SMB Share
and Browse NFS Export, without the two settings shortcuts. Each opens
the folder picker at its own root. When a location is stored, the list
starts with it, and that entry reopens the picker in the folder that
holds it, with it selected, as Content Location did before. A stored
NFS or SMB root reopens at the root itself, since the level above it
cannot be browsed.
* menu: show the NFS export in file browser titles
With an export set in the NFS settings, the browse root is nfs:// plus
the server and the VFS resolves paths against the export, so the title
showed the server and folders but never the export. Titles of paths
under the configured server now show it after the server:
nfs://server/export/folder.
Only the title text changes. Stored paths, list entries, other hosts
and an export given in the server address stay as they were.
* file_path: a scheme is only a path's first component; NFS root lanes
path_parent_dir takes "://" for a scheme only when no separator comes
before it, so a local folder whose name holds "://" keeps its parent.
path_parent_dir_test pins that next to the scheme roots.
load_content_filter_test gains the configured NFS server: its root
with the export in the server address and set apart, the levels
Parent Directory skips, titles with the export after the server, and
the root's entry in a file browser's drive list.
---------
Co-authored-by: LibretroAdmin <[email protected]>
* net_smb2: bound directory entry lengths without overflow
A server-controlled FileNameLength could wrap the addition used to
validate a QUERY_DIRECTORY entry. The UTF-16 decoder would then read
past the directory reply buffer.
Compare the length by subtraction and validate nonzero next-entry
offsets against the complete current entry and remaining reply.
* net_smb2: wrap-proof NTLM target info span, U+FFFD for lone surrogates
The NTLM challenge's target-info offset and length are checked by
rsmb_span_ok, which cannot wrap a 32-bit size_t. rsmb_utf8 takes a
low surrogate only when one follows its high surrogate and writes
U+FFFD for either half on its own, so names from the server are
always valid UTF-8.
samples/network/smb_retro/smb_dir_bounds_test hands rsmb_readdir
hostile directory pages placed at the end of their allocations
(wrapping and oversized name lengths, next offsets inside an entry
or past the page) alongside a well-formed one, and checks the name
decoder and span check; local_samba_test.sh runs it ahead of the
server matrix.
---------
Co-authored-by: LibretroAdmin <[email protected]>
* net_nfs3: stop bitmap walks after truncated replies
NFSv4 bitmap word counts come from the server. Once an XDR cursor
failed, two loops kept decrementing the declared count, allowing a
short reply to consume billions of iterations.
Stop both walks on cursor failure, matching the existing reclaim path.
* net_nfs3: bound every reply read to the bytes received
Fixed-size fields in replies (stateids, verifiers, the session id)
are read through xdr_get_fixed and xdr_skip, so the cursor never
passes the end of a reply; nfs4_res reports a cut-short result as
NFS4ERR_BADXDR rather than NFS4_OK. OPEN results share one parser
for create and reclaim and READDIR pages one taker for v3 and v4,
each leaving the caller's state alone for a truncated reply, and
xdr_get_opaque checks a length and its pad apart so neither wraps
on 32-bit targets.
samples/network/nfs_retro/nfs_reply_bounds_test drives these
decoders on short and hostile replies placed at the end of their
allocations; local_unfs3_test.sh runs it ahead of the server
matrix.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Where a new step goes and when it gets a job of its own, claiming a
new sample and regenerating the paths filters, the hand-picked source
lists harnesses link, and keeping tests to what the code does rather
than what the current runner image happens to do.
Every push ran every workflow. tools/ci_paths.py now writes each
workflow's push and pull_request paths filter: everything, less the
samples trees and the other workflow files, plus the sample
directories its jobs build or run (through the scripts they call and
the paths those directories name in turn) and the workflow itself. A
push to one sample runs the full-tree builds no longer, only the
workflows that run that sample; a push to one workflow runs it alone;
any other change runs everything as before. Jobs that call a tool or
CI script, walk the samples trees or read the workflow files keep
those whole, and workflows with a filter of their own keep it.
The CI local runner workflow runs tools/ci_paths.py --check, which
fails when a filter no longer matches what its workflow names, and
tools/ci_local.py which reads past a generated filter to the coverage.
samples-gfx ran about 150 steps one after another, and the thumbnail
preview job built and ran ASan and then TSan. The steps now run in five
jobs (drivers and widgets; Win32, X11 and display servers; D3D under
Wine; video and recording under Wine; KMS, Wayland, fonts and media),
each keeping its steps in their order, and the TSan half of the
thumbnail preview runs in thumbnail-preview-tsan, making the fixtures
it reads that 'make fixtures' does not.
Wine 10's Ubuntu packages keep wineserver in Wine's library directory
with nothing on the PATH, and the jobs start and stop it by name: the
network clients' Windows builds stopped at 'wineserver: command not
found', and every 'wineserver -k' and 'wineserver -w' elsewhere
quietly did nothing. apt-install.sh links it into /usr/local/bin when
it finds it there and not on the PATH.
Wii U get_arguments() prints through the argc/argv pointers it was
handed, and argv[1] only when there is one. The in-place URL decode
passes the string's real size to string_percent_decode(), and Basic
auth decodes its own buffer with the length it already has, so LTO
no longer sees a SIZE_MAX bound wrapping around.
Helpers only some configurations call are compiled for those only:
the overlay-only input and menu helpers under HAVE_OVERLAY, the
container-only audio_transfer time and movie-header helpers under
their container, and the scalar xxh3 long path off NEON. hidraw
hands autoconfig the stored vid/pid, rcheevos copies its operand by
assignment, and the Qt companion asks QScreen from Qt 5.10 on.
The gap and overlap fixtures cut timestamps with aselect and asetpts.
ffmpeg 8 regroups the filter's audio into encoder frames and gives each
the timestamp of its first sample, so on ubuntu-latest the gap fixture
came out with no gap and the timeline test found the sine where the
seek expects silence. Those fixtures' source frames are now exactly
the encoder's (960 samples for Opus, 64 for Vorbis), which ffmpeg
hands on as they are, timestamps included.
The log console attaches to the parent's console through AttachConsole,
which is Windows 2000 and later: mingw-w64 13 declares it only for
_WIN32_WINNT 0x0500 and up, so the NT 4.0 build (the Visual C++ 6
project's) no longer compiled, and on older Windows the import kept
the executable from loading. It is resolved from kernel32 the way
GetConsoleWindow is, and without it a console of our own is opened.
Built with the ffmpeg on ubuntu-latest, the 8-bit 4:2:0 stream's
sequence header carried BT.709 matrix coefficients but unspecified
primaries and transfer (S 192 128 8 2 2 1 0), where ffmpeg 6.1 and 7.0
write all three from the output options. testsrc2's frames carry no
colour, and the newer ffmpeg appears to take what the frame says over
those options. setparams tags the frames with what the options say, so
the encoder is told BT.709 either way. Unchanged with ffmpeg 6.1.
embedded() asserted a load it had just queued was still pending, which
a worker thread can already have finished: one run in three failed with
clang. run_mode() makes the same check only without threads; embedded()
does too.
Unoptimised under ASan and UBSan, vtt_lines_step()'s frame was 2040
bytes with clang 18 and 2056 with the clang on ubuntu-latest, past the
sample's 2048 limit. A whole line's handling is its own function now;
the larger of the two is about 1500 bytes.
The SPA headers on ubuntu-latest now declare in for loops
(spa/pod/compare.h, spa/param/audio/layout-types.h), which -std=c89
rejects outright; reading them as system headers only lifts -pedantic.
The three PipeWire objects take -std=gnu99 after the lane's -std=c89,
and keep -Werror=declaration-after-statement for their own code.
Validation layers before 1.4 began each message with its ID,
"Validation Error: [ VUID-... ] ...". Later ones (1.4.341 on
ubuntu-latest) hand the ID over only as pMessageIdName, so the debug
callback logged an error with no name, and every harness keyed on the
ID or on "Validation Error" stopped seeing it: vulkan_validation's
acquire test, expecting MissingAcquireWait, failed; threaded_video,
vulkan_hdr_ui_pass and vulkan_resize, counting "Validation Error",
would have passed with errors. The ID is put back in front, in the old
shape, when the message does not already carry it.
rsx_font_init() keeps the atlas in texture memory through the font
driver's font_atlas_place() since b0353a6439, and the harness, which
compiles rsx_gfx.c without font_driver.c, no longer linked. It now has
the placement the font driver makes, and its renderer hands the buffer
back the way the font driver's release does, so the in-place path is
the one font_sweep fails allocation by allocation.
Use aligned native-word loads and stores with compiler barriers on modern MSVC x86/x64, avoiding locked read-modify-write operations for acquire/release access. Evaluate store arguments before the barrier. Keep 64-bit accesses on 32-bit x86 and other architectures on their existing paths, and retain the full sequentially consistent fence.
Exercise all available widths and single evaluation. Checked MSVC x64/x86 and volatile:iso, MinGW, Linux ASan/UBSan, native eventcount stress and full parallel-xbox360 builds/lifecycle fixtures. MSVC assembly uses native moves; the pinned synthetic load loop measures about 0.185 ns/op versus 3.66 ns/op for compare-exchange, without implying dependent-load latency or application speedup.
Vulkan-Hpp undefines MemoryBarrier, so including retro_atomic.h afterwards fails in the sequentially consistent fallback load, and later fence use can also fail. Express the full fence with InterlockedExchange on a local word and retain ARM ordering barriers. Test Windows macro removal before and after header inclusion.
Checked native MSVC x64/x86, MinGW, Linux ASan/UBSan and emitted AArch64 seqlock barriers.
net_http hands the body to callers as sent and decodes no content
coding, so every request now says so with Accept-Encoding: identity;
without the header a server is free to pick any coding. The resume
harness checks both the first request and the resumed one carry it.
A device that samples no A8_UNORM - some of feature level 9 - is given
an RGBA8 font texture rather than whatever the format list offers past
A8, none of which keeps the coverage in the alpha the font shader
reads; each texel staged or handed to UpdateSubresource is white with
the coverage as its alpha, the latter through a scratch the font keeps.
The font_atlas_upload lane makes the texture on a device that samples
no A8 and sends two cells far apart into RGBA8 by map and while the
staging texture is the GPU's, and covers D3D10's rectangles the same
way.
The font texture is linear B8 with its pitch its width, the atlas's
own layout, so the atlas is placed in texture memory and is the
texture: the glyph cache rasterizes into it and nothing is copied. A
grown atlas is the texture as it is; the memory it left is retired
until the font goes. Without texture memory for it the atlas stays on
the heap and is copied in as before.
font_atlas_place() moves an atlas's samples into storage the consumer
makes - texture memory it samples them from as they are - and the
glyph cache keeps them there as the atlas grows, each buffer it moves
out of handed to the consumer's storage_free, the last when the font
goes. The font_lifecycle frame test refuses the storage, then places
the atlas, grows it there and frees the font.
The LUMINANCE_ALPHA texels an upload is converted into go in one
scratch buffer per font, grown to the largest upload asked for and
kept, which a grown atlas's respecify is made in before the texture
takes the new size. The gl2_pbo_readback lane respecifies and uploads
a region with every allocation refused once the scratch is made.
Each of the font texture's two upload buffers is mapped once and stays
mapped while the texture has it (d3d12_font_map), let go of before the
texture is (d3d12_font_unmap). The atlas's dirty rectangles are staged
on their own and copied in with a boxed copy each, between one pair of
barriers. The d3d12_frame_ring lane counts the buffer's maps across a
frame's strings and stages two cells far apart as two regions.
The glyph cache keeps the cells drawn into since the last upload as up
to FONT_ATLAS_DIRTY_MAX rectangles. Vulkan, D3D10, D3D11, Metal, GX2
and GXM now send each one rather than their bounding box: Vulkan as
one copy with a region and a flush range per rectangle, D3D10 and
D3D11 through one map of the staging texture and a boxed copy each,
Metal as a blit each, GX2 and GXM as rows of each, GX2 flushing just
those. Vulkan keeps what it has staged since its last upload with
font_dirty_rects_add(), which the cache's own list now uses.
The font_atlas_upload lane sends two cells in opposite corners on GX2,
GXM and D3D11 (by map and while the staging texture is the GPU's)
into textures holding a fill value everywhere else.
A left-aligned line is drawn without being measured first, so a glyph
it caches lands in the atlas after the line has begun. As sdl3 does,
the SDL2 raster font sends what it has batched and uploads the atlas
when a lookup dirties it, and GDI updates its atlas mirror then,
deselected while it may be made again. The SDL2 raster font's unused
atlas_dirty flag is removed. The font_lifecycle sdl2 test draws a line
that caches a glyph partway through.
The raster font's texture takes the grown atlas's size when the buffer
of texels for the respecify has been made, and keeps the size it has
while it cannot be (gl1/gl2_raster_font_follow_atlas); a region past
the texture goes in clamped to it, and the atlas stays dirty while a
region cannot be sent. gl2 sets a zero row length for the upload where
it has one. The gl2_pbo_readback harness grows an atlas with the
allocation refused, then allowed.
The atlas texture is its own, filled from the CPU only while no command
buffer names it. Cells the glyph cache rewrites are staged in the
frame's buffer ring and copied in by the blit command buffer the frame
commits ahead of its drawing, which orders the copy after the frames
still in flight that sample the texture, clamped to the texture as
well as the atlas. A grown atlas whose texture cannot be made keeps
the old texture, and font init fails without one.
The font texture is made again at the atlas's grown size beside the
old one, which is let go only once the new one is whole; when it
cannot be made the old one stays, and the atlas region is clamped to
the texture as well as the atlas. The font_atlas_upload lane makes the
texture again on a device whose texture creation fails, and then on one
where it succeeds.
An atlas whose texture could not be made again at its grown size is
larger than the texture it has: the dirty region goes into it clamped
to the texture's own width and height, row by row, so the cells the
texture holds keep being uploaded and nothing lands past it
(gx2_font_upload_dirty, gxm_font_upload_dirty, rsx_font_copy_dirty).
GX2 flushes a whole sprite from the CPU's cache for each glyph drawn
(gx2_font_invalidate_sprites).
The font_atlas_upload lane runs the three uploads, extracted from the
drivers, into textures allocated exactly their size, under ASan and
under mingw-w64 with Wine, and checks the size GX2 flushes.
A glyph wholly off the page's top or left edge is clipped to no rows
or columns and is not blended, and the room left on the page is taken
after the clip. The font_lifecycle omap lane blends FreeType's glyphs,
whose boxes are their ink, from the shipping driver into a page exactly
its pixels in size, with the message at the top, where '-' lies wholly
above the page.
A font copy recorded in the frame being recorded reads the upload
buffer only once that frame is submitted, and no cell drawn in a frame
is staged again in it, so a later string bringing new glyphs in the
same frame stages over that copy's source in place, waiting for
nothing (d3d12_font_stage_dirty).
The atlas region is clamped to the texture as well as the atlas, and a
grown atlas whose texture cannot be made keeps the old texture. The
d3d12_frame_ring lane stages three strings into one frame with one and
two frames in flight, where a wait for a fence not yet signalled
fails, and an atlas larger than its texture.
* vulkan: map staging memory whole for whole-size flushes
Texture and staging memory is flushed or invalidated with VK_WHOLE_SIZE
at the initial upload in vulkan_create_texture, the font atlas, the
frame and streamed textures and the readback buffers, but was mapped
only up to the texture's size. A whole-size range must end on a
nonCoherentAtomSize boundary or at the end of the allocation
(VUID-VkMappedMemoryRange-size-01389), which a size such as a 1x1 RGBA
image's 4 bytes in a 16-byte allocation does not. Only memory that is
not host-coherent is flushed, so it shows mostly on mobile drivers.
Mapping to VK_WHOLE_SIZE makes the mapping end where the allocation
does, as it already ends in the menu texture and recording readback
paths.
* vulkan: skip the font atlas flush when no copy follows
When the atlas grows and the textures cannot be made again at its
size, needs_update stays set while the dirty box is empty, its start
at the atlas's size. Every frame the flush was then aimed at the end of
the staging memory or past it (VUID-VkMappedMemoryRange-size-00686),
though the copy that follows, bounded by the staging texture, sent
nothing.
Flush only when the dirty box starts inside the staging texture, the
same bound the copy uses.
* vulkan: map texture memory from the allocation's start
A linear image's texels may start past offset 0 of its memory, while
the whole-size flushes and invalidates on that memory are issued from
offset 0, which has to lie inside the mapping. VULKAN_MAP_TEXTURE maps
the whole allocation and points past the subresource offset, as the
menu upload path already does; the frame, software framebuffer and
initial-upload textures use it. The initial upload skips its copy when
the map fails.
* vulkan: bound the font atlas flush and copy by one box
The dirty box is clamped to the staging texture once, and both the
flush and the copy use it: an empty box, including one left past the
textures after the atlas grew and they could not follow, does neither,
and the flushed range always covers what the copy reads.
* vulkan: drop the old texture memory reuse in vulkan_create_texture
The old texture is parked on the deferred list, which clears it, before
memory is chosen, so it never has memory to hand over or to free here.
The reuse branch and both cleanups that followed it are removed.
* samples/gfx/threaded_video: run the Vulkan lanes on non-coherent memory
fake_rebar_layer gains a unified-memory layout whose cached
host-visible type is not coherent and whose allocations are whole
pages (FAKE_REBAR_NONCOHERENT), and binds linear images past offset 0
of their memory (FAKE_REBAR_IMAGE_PAD), so the driver's hand-written
flushes and invalidates run under the validation layer.
check-vulkan-noncoherent runs the lanes with both, for RGB565 and
XRGB8888 cores, and holds that CPU-written textures and linear frame
images land there; the Vulkan validation workflow runs it.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Preserve AMD64 floating-point controls and Windows stack metadata, and
AArch64 callee-saved SIMD registers and floating-point state. Reject
allocation-size overflow and enter new AArch64 contexts through a return
trap.
Keep Windows switch code in read-only executable image sections. Publish
Unix fallback initialization atomically, reduce context storage and use a
direct resume branch on SysV AMD64. Private allocation hooks allow cores
to retain guarded-stack ownership without changing the public API.
Add ABI register and independent FP-mode regressions for Linux AMD64,
Windows AMD64 and AArch64, and run the native sample in common CI.
A failed accumulator or frame allocation can leave an uninitialized document allocated. Free the independent allocations directly before tree initialization, rather than traversing its uninitialized members. Exercise owned one-shot and incremental parsing with poisoned allocations and each allocation failure.
Load little-endian bytes explicitly on architectures that previously dereferenced unaligned uint32_t pointers. Cover unaligned whole-block and fragmented updates against a known digest and verify that input is unchanged.
Borrow complete input blocks, dispatch once per batch and use a 16-word scalar schedule. Keep modern MSVC SHA instructions behind the existing runtime feature probe. Cover streaming boundaries, unaligned input, SHA-224 and raw block processing.
The test took 49 s, nearly all of it in two places that test nothing:
- each of the hundreds of audio re-inits built the sinc resampler's
tables from scratch;
- eight scheduler cases call a consumer pass with nothing queued,
which waits out the frontend's one-second idle bound each time.
The driver's resamplers now come from a cache in the test: one parent
per configuration, kept for the run, and the driver gets a sibling of
it -- its own fresh stream over the parent's tables, freed as usual. A
backend without siblings is made the usual way. And the test sets the
idle bound to 20 ms, far past anything a case waits for; the driver
keeps its one second unless a build defines AUDIO_PIPE_WAIT_MAX_US.
49 -> 12 s, 20 s under ASan and UBSan, every case passing.
Under ASan and TSan the default run was doubled to 32 s, so the sink
estimate would still settle with more of its windows left out. All
eight runs of the CI step go at once, so these two set its length;
the rest take 12 to 16 s.
At 16 s under both sanitizers, eight runs at once on two cores, three
rounds: the estimate applies two or three times, settled every time,
and the mean is within 22 ppm of the 500 allowed. With two CPU hogs
on top both lengths drift alike, 387 ppm at 16 s and 401 at 32, so
the longer run bought no margin. The doubling is gone.
The CI step 36 -> 23 s with its two builds.
A flush into a lent span fills it straight from the resampler only as
far as fit() says is sure to fit; without one it sizes from the ratio's
bound, which leaves a few frames of the span to the copy at the wrap.
Only sinc had a fit.
Both now run process()'s own clock without the samples: nearest steps
its fraction, CC its distance, in the same float operations, so the
answer is exact. Every CC arm, C, SSE and NEON, moves distance the
same way, so one model covers them; a CC stream made for downsampling
and run above a ratio of one is moved to the upsampler from its start,
as process() does. Builds whose float arithmetic is not repeatable --
x87 extended precision, fast-math -- and the PSP's VFPU arm have no
fit and keep the bound.
lend_write's fit case now runs all three backends, nearest and CC made
at one ratio and run at eight others, and also asks before any input
has been processed: 2296 fits exact. Mutations of each fit's step,
threshold and the CC switch are each caught.
Linux-audio-transport took twelve minutes a matrix entry, most of it
waiting: the wide formats (32 runs, 220 s), the tempo extremes and live
controls (20 runs, 150 s), the speed-linked low-pass (8 runs, 84 s) and
the lend through the real wrapper (3 runs of a second each, 53 s). Each
run is a stretch of audio against a clocked device on next to no CPU,
run one after another in shell loops and recipes.
Each run is its own make target now, and a check runs its runs under a
sub-make with PAR (-j16, each run's output kept together); PAR= gives
the old order. The live set is one list the layout, speed-lpf, runloop
and auto checks all run with their own environment, check-wide runs
every format through both layouts and the live set on each at once, and
the lend, pause, menu, consumer-exit and kept-device checks run their
runs the same way.
On two cores: check-wide 220 -> 16 s, check-tempo 150 -> 8 s,
check-speed-lpf 84 -> 11 s, check-lend 53 -> 18 s, the same under
ASan within a few seconds; the whole job 1459 -> 234 s for both matrix
entries here, every check run twice without a failure.
A match copied with slack went to the wide copy once the source was
two vector widths back. The wide copy's short case is one fixed
32-byte memcpy, and on builds with no SSE2 or NEON a vector is eight
bytes, so a match 16 to 31 bytes back and 32 or fewer long was copied
over itself: undefined, aborted by the sanitizer and by fortified
libcs, and liable to repeat stale bytes. The wide copy now also needs
the source 32 bytes back; below that the eight-byte path, which takes
any offset of eight or more, does it. On SSE2 and NEON both limits are
32, so nothing changes there.
The raw block test is also built with the vector path switched off
(rzstd_raw_block_test_novec, run in the samples job); it fails without
this change and passes with it.
The overlap-copy check flagged rzstd_match_copy's call sites since the
helper copies within one buffer; it is now audited and on the allow
list.
gl2.c now tells video_driver.c whether a hardware core's context is
shared (video_driver_hw_context_shared_note()), and the samples that
build gl2.c against the stand-ins in gl2_pbo_readback/stubs.c -
gl2_pbo_readback, gl_last_pass_fbo and gl2_kms_frames - stopped
linking on the undefined reference. The stand-in takes the note and
does nothing with it, as the tests run no hardware core.
Grow short periods before word copies and keep the final tail narrow. Cover offsets 1 through 32 and exact output boundaries in the existing regression.
Provide a single-owner decoder for callers processing independent frames into their own output buffers. Keep the one-shot entry point, and test frame resets and recovery after truncated or undersized output.
The flush in parts left one copy at the driver's wrap: the input given
to the first span was sized from the output bound, which keeps a margin
of sixteen frames and one more over the ratio, so the resampler's
output fell some twenty-five frames short of the span and that sliver
was filled from the frontend's buffer.
The sinc resampler can say exactly what the next process() makes of
each input frame: a frame is pushed while the phase clock is a phase
or more, a frame comes out at each step of the integer ratio until it
is again, and nothing else enters into it on the nominal path. The
resampler interface gains an optional fit(): the most input frames
whose process() is sure to produce no more than the frames asked for,
exact for the stream's state where the backend can tell, 0 where it
cannot (the decimated fast-forward path, or a backend without one, and
the caller sizes from the bound as before). The flush sizes the first
span's input from it, so the span is filled to within the frames one
input frame makes - a frame or two at 1.5x, often none at all - and the
copy left at the wrap is that or nothing. The bypass fills its span
exactly as it is, a frame in being a frame out.
retro_resampler_t's new member is last, so a backend that does not set
it is NULL there; libretro-common is RetroArch's, for the cores to
sync.
lend_write_test checks the fit is exact: for every quality, with and
without HQ oversampling, at ratios above and below one, after runs
that leave the stream at any state, the input fit() names makes no
more than asked and one frame more makes more. Off by one either way
it fails; a fit that overshoots also publishes past the span, which
the device stand-in refuses.
When a restart keeps the WGL contexts the core is told so and not
reset, but a core-profile or debug context - glcore always, gl when the
core asks for 3.1 or later - was then created again over the kept one.
The core went on with objects the new context does not have, and the
kept shared context leaked. A kept context is now used as it is.
wgl_context_keep.py runs create_gl_context() against stubbed WGL calls.
With Driver Pacing on and nothing else holding the loop, the backend's
release keeps the content's frame period. On a driver without a backend
- gl, glcore, or a GPU without the extension - the setting did nothing,
and with vsync and audio sync off the loop ran unthrottled. The frame
limiter now holds the content's period in exactly the iterations a
backend would have owned: a content frame at normal speed, focused, the
menu down, and nothing else pacing. Driver Pacing off is unchanged.
A core that asks for its context to be kept is not reset across a video
restart when the context driver kept its contexts. The gl driver gives a
hardware core either its own context or a second one shared with it -
always the shared one under the threaded wrapper - and framebuffer and
vertex array objects are not shared between the two. Toggling threaded
video moved the core from one to the other without a context_reset, and
its next draw used objects that do not exist there: an access violation
inside the GL driver on WGL, which keeps both contexts.
The gl driver now reports which context the core renders in, and when
that changes across a kept-context restart the acknowledgement is
withdrawn, so the core is reset into the context it now has.
gl_context_move.py checks the decision and its wiring.
Where the driver's ring wrapped inside a flush, a float stereo device
still took that flush the old way: the lend came back short and was
abandoned, the resampler wrote the frontend's buffer, and write()
copied every frame into the ring - about one flush in sixteen on a
typical ring, and the last copy the lend left on that path.
The flush now runs its output stage in parts. The resampler is given
as much input as is sure to fit the short span (audio_driver_input_
bound), writes it there carrying its state, and the span is published.
Its output falls a little short of the bound, so a sliver of a few
frames is left before the wrap: enough input to fill it is produced in
the frontend's buffer, the sliver copied from there - the one small
copy left - and the frames over are carried to the head of the next
span, where the rest of the flush is produced past the wrap. The
mixer, the clamp, the resume ramp and the pause mute run on each part
in turn, each carrying its state, so the stream is the one the whole
flush made: the lend test compares it bit for bit. The bypass's copy
splits the same way. An int16 device keeps narrowing into its spans
on the way out; a device that lends nothing, or no room after a
bounded wait, takes the staged write as before.
The produce, the finish and the sink accounting are their own
functions now, run once per part; audio_driver_input_bound() is built
for every configuration.
lend_write_test's rings wrap 1000 frames into a 1200-frame flush, and
one wraps twice: a float stereo device resampled or at unity, threaded
and inline, must give the written stream bit for bit with write()
never called. A ring that cuts a frame at its end still sends what is
left through write(). Dropping the carry, not shifting it, or letting
the fit overshoot the span each fail it.
A lend asks the driver for one span to hold a whole flush. Where the
driver's ring wraps inside it the span comes back short, the lend is
abandoned, and the flush is produced in the frontend's buffers and
copied in by write() - about one flush in sixteen on a typical ring.
The resampler's output cannot be split there, but every stage after it
makes the device's frames one at a time, its state carried from call
to call: the narrowing of a float mix for an int16 stereo device, the
upmix for a wider one (int16, float, or float narrowed for an int16
device, the core's own surround channels merged over it), and the
binaural render on headphones. audio_driver_stage_spans() runs such a
stage into as many spans as the driver lends - both sides of the wrap
- in whole frames; audio_driver_stage_write() takes the rest, a frame
the wrap would cut or what has no room, into the stage's own buffer at
the offset the whole would have had, so a short threaded write's tail
is still where audio_driver_retain_output() looks for it, and writes
it. The upmix and binaural lend, which took one span or none, is this
now; a float stereo device's resampler output still goes through
write() at the wrap. The extras merge takes the first frame of the
span it works on.
lend_write_test gains a device whose ring wraps inside the span, of
whole frames and of a size that cuts a frame at its end: int16 and
float stereo, 5.1 float and int16, headphones float and int16, a core
sending its own 5.1, threaded and inline. Each stream must equal the
written one bit for bit, with write() never called where the ring
holds whole frames. On the previous audio_driver.c those cases call
write(); a merge that ignores the span's first frame fails the 5.1
core's.
Every microphone flush ended the same way: its last stage - the
narrowing of float samples, or the left channel of the integer
resampler's stereo - produced the samples in final_frames, and
retro_spsc_write() then copied them into the ring the core reads.
microphone_enqueue() takes the ring's span at the head, in the one
span or two its wrap leaves, and has the stage write the samples
there: the float fast path, the integer resampler's left channel and
the float resampler's narrowing all reach the ring without the stop in
between. final_frames is gone from the arena and from
microphone_driver_state_t; the resampler's output bound is the smaller
of the two buffers still on the way. An int16 device at its rate keeps
its single copy, from what the driver read.
The copy it saves on an 800-sample flush, a 60 Hz frame at 48 kHz:
0.03 us, and an intermediate buffer of the largest flush.
mic_handshake_test gains a lane that puts the ring's head at every
place it can be and enqueues runs of every length up to the whole
ring, with both fills: the ring must hold exactly what the stage used
to produce on its own, and report the samples it took. Writing only
the first span, or taking the right channel, fails it.
The lend of the driver's buffer stopped at stereo: on a wider device
the upmix produced the device frames in upmix_buf (upmix_i16 for an
int16 device) and write() copied them into the driver's buffer, and
with Headphone Virtual Surround on the binaural render did the same
into its own two-channel buffer.
audio_driver_write_frames() now borrows the span for the frames the
widening stage is about to produce - the upmix, its int16 form, the
narrowing of a float mix for an int16 device, or the binaural render -
and has the stage write them there, the core's own surround channels
merged in place as before, then publishes them. Inline and blocking it
waits for the room first, as write() would have; a span cut short, or
no room after the wait, leaves the stage writing its own buffer and
write() taking it as it always has.
The copy it saves, each 60 Hz flush into a 64 ms device: 0.28 us for
5.1 float, 0.13 us for 5.1 int16, 0.09 / 0.05 us on headphones - and
the pass over the stage's own buffer before it.
lend_write_test runs a 5.1 device and Headphone Virtual Surround,
float and int16, resampled and rate-matched, threaded and inline,
checks the stage under test is the one running, and compares the lent
stream bit for bit with the written one: write() is never called,
and a driver cutting spans short still gets the same stream through
write(). On the previous audio_driver.c the wider cases each call
write().
The slowest steps left in Linux-samples-audio:
- sinc fast-forward, 40 s: an ASan build and run, an AVX one, then a
MinGW one under wine, one after another, each run on one thread.
Each is now built to its own binary and the three run at once.
- sinc sibling, 20 s: its ASan and MinGW runs, likewise.
- the fill cap lane, 36 s: its audio_paced lane runs three device
latencies of four settings each in real time. PACED_ONLY takes
audio_paced_16, _32 or _64 for one latency, so the step runs the
three beside the other four lanes; an unknown one is refused.
Measured on two cores: sinc fast-forward 40 -> 21-24 s, sinc sibling
20 -> 15 s, the fill cap lane 36 -> 24-27 s, each three runs in a row
or more without a failure.
wasapi_exclusive_pacing_test was the slowest step left in
Linux-samples-audio, 87 s: twelve scenarios of a fake device on the
wall clock, three seconds each and again with retries, then the clock
and bitstream cases - once under ASan and once under TSan, one after
another, on next to no CPU.
PACING_SCENARIO=<n> now runs scenario n alone, PACING_SCENARIO=rest the
cases after the scenarios, and PACING_SCENARIO=count prints how many
scenarios there are, so the step runs every one under both sanitizers
at once through run-parallel.sh and picks up a scenario added later on
its own. A number past the last one is refused rather than passing on
nothing. Unset, the test runs everything as before.
On two cores the runs take 8 s instead of 86 s, with no scenario
needing a retry in three runs in a row.
xaudio_lend_test failed on GitHub's runner with "a lend is within the
free room and the allocation". The test read write_avail() and then
called write_begin(), and checked the lend against the earlier
reading - but XAudio2's thread frees buffers whenever it finishes one,
so a lend taken a moment later can rightly find more room than was
read before it. A runner slow enough between the two calls failed
the check with the driver doing nothing wrong.
The room is now read right after the lend. Only the test's thread
takes room, so it can only have grown since the lend measured it, and
a lend past it is still a lend past the room. With a 3 ms sleep put
between the two calls, the old check fails on every run and the new
one passes.
Three steps of Linux-samples-audio took six minutes between them, and
two of them spent it waiting: pipeline_rate_control's eight runs and
pipeline_clocked's fill cap lanes are seconds of audio each against a
device that drains in real time, on almost no CPU, run one after
another.
.github/workflows/scripts/run-parallel.sh starts commands at once and,
when all are done, prints each one's output in order with a pass or
FAIL line, failing the step if any failed. The rate control step builds
its ASan and TSan binaries in turn and runs all eight at once; the fill
cap step runs its five lanes at once, pipeline_clocked_test taking
PACED_ONLY=<lane> to run one (PACED_ONLY=1 still runs them all).
discrete_multichannel_test is CPU-bound - most of it building Kaiser
tables - so it is now built at -O1 instead of -O0, and its plain and
ASan runs share the runner's cores.
Measured on two cores: pipeline_rate_control 162 -> 33 s of runs, the
fill cap lanes 116 -> 36 s, discrete_multichannel 87 -> 40 s. Each new
arrangement passed three times in a row there.
The XAudio2 driver had no write_begin / write_end, so every flush's
output was produced in the frontend's own buffer and then copied by
xa_write() into the buffers XAudio2 plays - the copy WASAPI, DirectSound
and the other lending drivers no longer make.
The driver's sixteen buffers are cut from one allocation, so the free
ones after the head are one span up to the allocation's end.
xa_write_begin() lends that span, never more than the free room, and
xa_write_end() moves the head over what was published, submitting each
buffer it fills. The span stops where the free room does, so every
buffer it fills has a slot and write_end never waits. A span cut short
at the allocation's end leaves the frontend's staged write to wrap, as
for the other drivers. The submit is shared with xa_write().
samples/audio/xaudio_lend runs the shipping driver on wine's XAudio2
(Windows' own on Windows): lends stay within the free room and the
allocation, lends and writes make one stream, write_end publishes all
it is given at once even filling the whole room, an abandoned lend
takes nothing, and lent audio plays at the pace written audio does.
Lending past the room hangs it; a write_end that does not move the
head fails it. Linux-samples-audio runs it.
A replay records or plays back a checkpoint every few seconds, in the
middle of the game, and each took fresh state-sized buffers: the
statestream encoding (half again the state), the compressed copy -
calloc'd, zeroed only to be overwritten by the compressor - and on
playback the compressed copy read from the file and a calloc'd
buffer to decode into, then copied into cur_save for a raw state.
The handle now keeps the encoding and compression buffers
(bsv_movie_scratch()), grown only when a state outgrows them, never
zeroed, freed with the movie. A raw state is inflated straight into
cur_save, so its copy is gone too. A zlib checkpoint whose buffer
could not be taken now fails instead of compressing into NULL, and a
checkpoint that fails part way marks cur_save as holding no state.
The buffers a zstd checkpoint of a raw state takes and touches, per
checkpoint (the codec's own work aside): 4 MiB 0.81 -> 0.41 ms,
16 MiB 2.66 -> 1.35 ms, 64 MiB 45.5 -> 9.0 ms.
bsv_statestream_test gains a lane that records and plays back eight
checkpoints, raw and statestream, plain and zstd, each decoding to the
bytes written, with nothing state-sized of the replay's own allocated
after the first two (what zstd allocates inside a call is measured
through rzstd directly and allowed for). It fails on the previous
bsvmovie.c. bsv_checkpoint_bounds_test's copy of
bsv_movie_load_checkpoint() follows the shipping one again, with
bsv_movie_set_end() and the size checks it had fallen behind on.
A save over a slot first reads the file there into the undo buffer,
so Undo Save State can put it back. That read went through rzip: on
every save over a compressed slot the old state was inflated in full,
and kept inflated in memory for the rest of the session. Undo then
wrote it back through rzip, deflated afresh, compressed or not as the
setting had since become rather than as the file was.
The backup is now the file's own bytes, read plainly, and undo writes
them back as they are. Nothing is inflated on a save, nothing deflated
on undo, the buffer held for undo is the file's size rather than the
state's, and the slot comes back byte for byte.
An 8 MiB state that compresses to 4 MiB: 2.1 -> 0.6 ms for the backup
on every save over the slot, 12.4 -> 3.3 ms for the undo, and 4 MiB
less held for it. At 32 MiB: 9.9 -> 3.8 ms and 41.6 -> 25.3 ms.
save_state_io_test gains an RZIP=1 build with HAVE_COMPRESSION, as the
frontend ships it, where every lane also passes, and a lane that saves
over a slot with the compression setting turned in between, both ways,
then undoes: the slot must come back byte for byte. It fails on the
previous tree. Linux-samples-tasks runs that build under ASan.
A streamed IPS patch built its target as the source arrived: a second
buffer the size of the ROM, taken at open, every byte of the source
copied into it, then the records laid over it, and the source freed.
The target of an IPS patch is the source with the records on it, and
the source is the content buffer the load already owns.
patch_stream_finish_into() finishes a stream whose source the caller
owns (malloc'd, given to patch_stream_set_source()): an IPS patch puts
its records on that buffer in place, growing it with realloc() only
when the target is longer, and any other format hands back its target
in the buffer's place. On failure the buffer is still the unpatched
source, as before. Fed in place, IPS copies nothing while the source
arrives; a chunk fed from elsewhere copies what came before through to
the target and carries on as before. The target is no longer taken at
open but by the first copy-through, so the in-place path never takes
one. The content load finishes its patch with finish_into().
A 16 MiB ROM with a 2000-record IPS patch, first touch: 6.7 ms ->
0.05 ms to apply, and 16 MiB less heap during the load.
The patch_stream test finishes every in-place and fallback comparison
with finish_into() on a buffer of its own, and checks a 4 MiB IPS
source is patched where it lies without being moved or copied. Leaving
the grown tail unzeroed, skipping the records in place or in the
catch-up, copying nothing in the catch-up, or copying through while fed
in place each fail it.
The content load streams a soft patch alongside the read: each span the
transfer fills is fed to the applier as it arrives. BPS and xdelta may
copy from anywhere in the source, so both kept their own growing copy of
every byte fed (BPS in src_buf, xdelta in own_src), and UPS kept the
part it had not consumed yet in its carry. The transfer buffer already
holds the whole source in one allocation that never moves, and stays
alive, detached as the content, until the patch is finished - so that
copy is a second ROM in memory, written once and read back.
patch_stream_set_source() (and vcdiff_stream_set_source() under it)
tells the stream where the source it is about to be fed lies. Chunks
fed from there are read in place: no copy, no realloc as it grows, no
memmove of the UPS carry. A chunk fed from anywhere else falls back to
the old copy, starting with what came before it, so a caller that does
not keep the source whole still gets the right answer. IPS keeps no
source and is unchanged. content_file_pump_source() sets the transfer
buffer as the source.
A 32 MiB BPS hack patch: 32 MiB less heap during the load, and
143 -> 137 ms to apply.
The patch_stream test feeds each comparison as a copy, in place, or in
place then from another buffer (freed at once, so ASan catches a stream
still reading it), and checks with glibc's heap counters that a 4 MiB
BPS source and the xdelta source are no longer copied. Dropping the
bytes the fallback copies, restarting the UPS carry at 0, reading the
wrong byte in place, or ignoring the setter each fail it.
The inline pipeline's lend of the driver's buffer sizes what it borrows
with audio_driver_output_bound(), which was defined only with
HAVE_THREADS (or the harness's AUDIO_OUTPUT_BOUND_CHECK): a build
without threads stopped at audio_driver.c on an implicit declaration,
caught by the compile matrix's no-threads lanes. The bound is defined
for every build now; its inverse, which only the threaded pipe uses,
stays behind the guard.
The save and load tasks moved a state in 100 KiB quanta, and rzip cuts
its stream into 128 KiB chunks. A write shorter than a chunk never met
rzipstream_write()'s direct path, so every byte was first copied into
the stream's buffer and compressed one chunk at a time, and the
parallel chunk compressor (two or more whole chunks in one write)
never ran for a savestate at all. A read always inflated each chunk
into the stream's own buffer and copied it out.
A compressed save now writes two whole chunks per quantum: they are
compressed straight from the state, across the worker pool. A
compressed load reads two whole chunks per quantum, and rzipstream_read()
now inflates a whole chunk wanted by the caller straight into the
caller's buffer, the read-side mirror of the write's direct path; a
chunk that would inflate past chunk_size is corrupt and fails as it
did. What a quantum moves to or from the device is the compressed
size, about half the state, so the slowest storage the quantum is
sized for is not asked for more per tick. Uncompressed states keep
the 100 KiB quantum, and Emscripten its 16 MiB one, already a multiple
of the chunk. RZIP_DEFAULT_CHUNK_SIZE moves to rzip_stream.h for
writers that want to meet it.
For a 16 MiB state on two cores, a compressed save goes from about
217 ms to 118 ms and the output is byte for byte the same; a load in
whole chunks goes from about 31.5 ms to 29.5 ms (best of six).
The rzip codec sample reads both codecs back in quanta from one byte
to past several chunks, around a rewind and a short last chunk, and
checks that 100 KiB and two-chunk writes make the same file.
Each failed on a loaded CI host while the code it checks was right.
asio_teardown judged whether the teardown wait ended as the last
callback left by a 100 ms wall-clock limit, and a busy host was late
to wake the waiter (170 ms seen). It now watches the driver's bounded
wait instead: a bound that runs out with no callback left inside is a
notify the wait slept through. The negative build, whose last callback
out does not notify, fails that way on every run however fast the host
is; a waiter that is merely woken late passes.
pipeline_clocked retried its shallow setpoint run when the host ran
the device dry or held the core, but not when the host took the
consumer off its core during the first second and drained it below the
setpoint, which then failed the startup check. That dip is retried as
well. Priming short of the setpoint does it in every run, and still
fails.
d3d11_turns' unfair core must be caught keeping the frontend waiting,
which only shows when the core is running on another CPU as the
frontend asks; on a host that had it off the CPU, the unfair core let
the frontend in at once and the must-fail run passed. An ask now
counts only when the core was seen taking turns just before it; a run
with too few such asks reports INCONCLUSIVE and is run again, and the
unfair core gets up to five runs to be caught.
D3D11 had no get_current_software_framebuffer, so every frame from a
software core was copied on the CPU, by dxgi_copy into the dynamic
sw_direct texture or into history[0]'s staging texture, while GL,
glcore, Vulkan, D3D12 and D3D9 all let the core render into memory
the GPU reads from.
The core is now lent one of three staging textures, mapped READ_WRITE
(host-cached memory, which cores may read back) with DO_NOT_WAIT, so a
slot the GPU is still copying from is passed over rather than waited
for; when none is free the core renders into its own buffer that one
frame. A frame pushed from a slot is unmapped and copied into
chain.history[0] with CopySubresourceRegion, and drawn from there as
any staged frame is, by the stock chain, a preset or views. The copy
on the GPU replaces the one on the CPU. The slot the last pushed frame
came from is not lent straight back, and each push retires the cached
frame, which points into memory the unmap takes away; a redraw draws
history[0] as it stands.
Lent only for unthreaded video (the threaded wrapper lends its own
slots), for XRGB8888 and RGB565 frames, while history[0] is at the
frame's size and in its format, and when the slot's row pitch is the
frame's tight pitch.
The new d3d_failure_paths sw_lend_gpu sample runs the shipping
functions on a real D3D11 device: 120 frames rendered into lent slots
reach history[0] through the GPU copy, the cached frame's slot is never
lent straight back, every push retires the cached frame, other sizes
and hardware cores are declined, and every slot goes unmapped. It runs
under Wine in Windows-wine-menu.
With the threaded pipeline off, the flush always resampled into
output_samples_buf (or narrowed into output_samples_int16) and write()
then copied the whole chunk into the driver's ring, even though every
driver with a ring in front of the device - pulse, pipewire, alsa,
coreaudio, wasapi, sdl2, aaudio, opensl, jack, dsound, wdmks - lends
that ring through write_begin/write_end. Only the threaded consumer
took the lend.
The inline flush now takes it as well. A blocking device is first
waited on for the chunk's room through wait_writable(), the wait
write() would otherwise have made piece by piece. When the driver
cannot lend the whole chunk - the span is cut short by the ring's
wrap, or the bounded wait found too little room - the lend is
abandoned and the staged write runs and blocks as before. That saves
one copy of each flush's output, about 6.4 KB a frame for float
stereo at 48 kHz and 60 fps; the resampler still dominates the
flush's cost.
The lend write sample runs its four cases inline as well: the lent
stream is bit for bit the written one, write() is not called for a
resampled chunk, and every begin is balanced by an end.
In thread_video_t the main thread's per-push writes shared cache lines
with fields the video thread writes every frame: core_time with
miss_run, drain_cooldown and drain_pending; last_time with the lent
slots' busy flags; the handoff counts and the published pacing values
with repeat_request, repeat_group and margin_extra_us. In
audio_driver_state_t the pipe's consumer (pipe_gen, pipe_stalled, the
underrun and priming state, the source wait counts) and producer
(pipe_data_gen, the runloop snapshot, pipe_ctrl_avail, writer_held,
the fast-forward multiplier) were interleaved over the same four
lines. Each write by one thread took the line back from the other.
The main thread's fields now sit together behind a line of padding,
with the values it publishes on a line of their own, the way the
struct already separates its other shared fields. The audio pipe's
consumer and producer fields form two groups, each between full lines
of padding, under a new AUDIO_PIPE_LINE. A full line on each side
keeps the separation wherever the fields land, without relying on
the allocation's alignment. It costs 184 bytes in thread_video_t and
192 in audio_driver_state_t.
The threaded video sample's line separation lane and the audio
pipeline wakeups sample assert the new separations by offset, so a
field added above a pad cannot quietly put them back on one line.
Every frame each running task with a title went to the widgets with a
fresh copy of its title (and error) and a new command, only for the
widgets' owner to compare the title with the one it already had and
free it: a few allocations a frame per task, on the main thread,
during gameplay.
The task system now counts publications of a task's title and error
(task_get_text_gen()). The widgets record on the task what they last
sent it: the key, that count, and progress, flags, style and category
packed into one word. While the text is unchanged it is not copied,
and an update that changes nothing is not sent at all. A finished
task is always sent whole.
When the widgets' owner loses what a task sent it - a task widget
freed, an update dropped for lack of room, widgets cut loose for a
reinit - it bumps a static epoch, and every task sends its title
again. A widget handed on to another task is under a key that task
never sent, so its first update carries its title too. Where the
strings are read without the setters' publication (the runner without
lock-free atomics) the count never repeats and every update copies,
as before.
An unchanged push drops from about 30 ns to 11 ns, with no allocation.
The task bar sample checks that nothing is sent for an unchanged
task, that a progress step is sent once, and that a new title reaches
the widget; its title churn test now republishes the titles it
rewrites in place.
The retained-backbuffer copy and the OpenXR UI copy each moved two
images with two separate vkCmdPipelineBarrier calls before the copy
and two more after it. Each pair now goes in one call with both image
barriers and the union of the stage masks, halving the barrier calls
on those paths.
The pairs sit back to back with no work between them, so the union
waits on nothing the separate calls did not: an execution dependency
already covers every earlier command in its source stages, whichever
image it names.
The driver's VkPipelineCache was created empty on every init and
thrown away on deinit, so the stock pipelines and every slang preset
pass were compiled from scratch by the driver on each launch, preset
load and driver reinit; only the SPIR-V was cached.
The cache blob is now stored through the shared shader cache in
<cache dir>/vkpipeline, one file per vendor/device ID. It is mapped
and handed to vkCreatePipelineCache where it lies, after the header
(version, vendor, device, pipelineCacheUUID) is checked so a blob from
another GPU or driver never reaches the driver; a rejected blob falls
back to an empty cache and is replaced on the next deinit.
Saving happens on deinit only, never while content runs, and is
skipped when the cache data is the size last read or written, so a
fullscreen toggle or driver reinit does not rewrite the file.
The Vulkan 1.1 probe and android_apply_device_defaults() sat outside
any ANDROID guard, so every non-Android Unix build failed on JNIEnv,
jobject and g_android. Both callers and the prototype are already
Android-only.
When ro.sf.lcd_density was empty, the display server ran "wm density"
through popen(), a shell child from inside the app. That command
prints "Physical density: <n>", which atoi() read as 0, so the
fallback never produced a density and every such device got the
90 dpi guess; from an app the command is often refused anyway.
The fallback is now getResources().getDisplayMetrics().densityDpi, the
density Android itself scales by. ro.sf.lcd_density is still read
first, so a device that reports it gets the same DPI as before, and the
90 dpi guess stays for a device that reports neither.
With this nothing calls system_property_get() any more, and it is
removed: the Android frontend no longer runs a shell to read anything.
A fresh install picks Vulkan, AAudio and 64 ms when the device has
them, but "Reset to Default Configuration" went back to the built-in
OpenGL ES, OpenSL ES and 128 ms, the defaults the oldest devices need.
Reset now sets the same device-dependent values a fresh install gets.
The built-in defaults are still not changed, so a config saved with
"save minimal" keeps meaning what it did; the values are set as the
config's own and saved whatever the save mode. A device whose Vulkan
driver fails is reset to Vulkan like any other, and the next start
falls back to OpenGL ES again.
android_apply_first_run_defaults() is now android_apply_device_defaults(),
declared in configuration.h instead of locally in configuration.c.
When the Vulkan driver failed to start, RetroArch exited. On Android
the config still named Vulkan, so the app closed on every start until
its data was cleared. With fresh installs now starting on Vulkan
whenever the device declares Vulkan 1.1, a declared driver that still
fails would do that from the first start.
On Android the frontend now switches to the "gl" driver in the same
launch, threaded or not, and saves the config so the next start does
not try Vulkan again. A failed threaded start already puts the driver
back and leaves no thread, so the retry starts clean. Not done for a
core that renders with Vulkan itself, which OpenGL ES cannot stand in
for; that still fails as before.
A new Android install starts on OpenGL ES, OpenSL ES and 128 ms of
audio latency, which suits the oldest devices RetroArch runs on and
leaves current ones on the slower path.
On the first start, when no config file exists yet, the frontend now
picks:
- the Vulkan driver, when the device declares Vulkan 1.1 or later
(PackageManager.hasSystemFeature("android.hardware.vulkan.version",
0x00401000), what its driver passed conformance at). This is read
from the system, not by loading the driver. A device with only a
1.0 driver, or none, keeps OpenGL ES.
- the AAudio driver from Android 8.1 (API 27), with 64 ms of latency.
8.0 has AAudio but with the bugs that make Oboe use OpenSL ES there;
older releases keep OpenSL ES. A device whose latency comes from a
quirk (Xperia Play, GAMEMID_BT) keeps that latency.
Existing installs are not touched. This runs only when there is no
config at all; a config that exists but will not open keeps the
built-in defaults. And the built-in defaults stay what they were:
with "save minimal" a config leaves out every key that matches them,
so changing them would move those installs to new drivers on update.
The choices are set as the install's own values instead and saved
with the rest of its config, so they last whatever the save mode.
Each driver is only picked when the build has it.
frontend_android_get_version_sdk(), frontend_android_get_name(), the
device-model reads in frontend_unix_get_env() and frontend_unix_init(),
and the display server's ro.sf.lcd_density read all went through
system_property_get(), which popen()s the getprop binary: a fork and
exec of a shell to read one property. Several of these run on every
startup, and the SDK level is asked again by the input driver.
They now use __system_property_get(), which the frontend already uses
for the manufacturer and model and which every supported API level
has. The "wm density" fallback is a command, not a property, and
still goes through system_property_get().
With Increment Save State Index on, Save State moves the slot up
before the save is pushed. A save the queue turned away - the saves
and loads held behind the one in flight already at their bound -
wrote nothing there, yet the slot stayed advanced, so the next Load
State named a slot with no file in it. command_event_main_state()
also called that save a success, and pruned old saves for it.
It returns false for a save turned away now, and prunes nothing for
it, and the slot goes back to where it was. A save that is pushed and
fails later on (a full disk) still leaves the slot advanced: later
saves may have moved it again by then.
The state-hold-full lane gains the case: with the hold full and
Increment Save State Index on, Save State must be refused and leave
the slot where it was. It fails on the old code.
The content-reply lane checks that a line-based command interface is
not answered for LOAD_CONTENT. It asked through rr_ask(), which pumps
until a reply comes or 5000 frames pass - and as no reply should
come, it always waited the full 5000: 83 seconds at 60 Hz, two thirds
of the harness's run time, for a load that is through in a few
frames.
rr_ask_unanswered() waits for the load the command started to be
through, then a grace period of 30 frames for a reply that should not
come: one is sent from the frame the load finishes in. The lane goes
from 87 s to 4 s, and the harness from about 125 s to 40 s. The check
still bites: with the interface made structured, the reply is caught.
A save or load pressed while another runs is held behind it, up to
four; past that bound it is turned away. Save State then logged
queued=NO and Load State returned false, and neither put anything on
screen: the hotkey did nothing, silently.
Both say so now - "Failed to save state to <file>." and "Failed to
load state." - through the message command_event_main_state() already
shows for its other outcomes.
A state-hold-full lane in the content_load harness fires six Load
States in the frame after a Save State: some are held, one past the
bound must say so on screen, and the held ones still finish. It fails
on the old code.
Two backups or restores of the same core may not run at once, and a
second one was refused without a word. It was also refused while the
first had finished but not yet retired - its callback still to run -
so Backup pressed again just after one completed did nothing either,
and the finder read the finished task's handle to decide.
A finished backup or restore is passed over now, before its handle is
read: its file work is done, and a second one is taken. One refused
because the same core's is still running puts "A backup or restore of
this core is already running" on screen. Sideload Core flushed the
menu to the top even when its restore was refused, leaving the user
somewhere else with nothing done; it flushes only for a restore that
started, as Restore Backup already did.
core_backup_io_test's findable lane now expects the second backup to
be taken while the first is finished but unretired, and a new lane
checks that a backup of a core whose backup is queued is refused with
a message on screen. Both fail on the old code.
The throughput guard asked for 256KiB per net_http_update() averaged
over the body. That is exactly net_http.c's drain budget, so it only
held when the test's server thread had a full budget waiting before
every call; on a loaded two-CPU runner it sometimes had not, and the
check failed about one run in four with the drain intact.
What the guard is for is that a call reads until the socket is empty
rather than taking one bite and returning. It checks that now, per
call, from the test's recv() hook: every body update must end on
EAGAIN, at the byte budget (within the minimum read window net_http.c
will not spend on a short read), or at the iteration cap. The average
is still printed.
Passed eight runs in a row, and six with three busy loops on the two
CPUs. With net_http.c cut to one recv() per call, every body update
is reported.
When the menu goes, its Explore build is cancelled and left to wind
down; its callback then installs nothing. The finder still took it
for a build in progress, so Explore opened again before it had wound
down started no build of its own and stayed on "Initialising list"
until it was left and opened again.
A cancelled build is passed over now. The cancel flag tells, not the
task's handle, which may already be gone.
explore_build_test gains a reopen lane: a build pushed and abandoned
as the menu's teardown does, then Explore opened again, must end with
an index installed. It fails on the old code.
Play Feature Delivery installs one core at a time, and a core picked
from the Core Downloader while another installed was turned away
without a word: the pick did nothing, however long the install took.
A pick from the menu is held now, in order, and started from the
running install's callback once it is through; the core it replaces
is unloaded then, as the install may delete its file. The same core
picked twice waits once, and past a bound of four a pick is refused
as before. The core switch keeps pushing its own installs one at a
time and moving on from one refused, and the callback leaves the held
picks waiting if the switch has pushed one meanwhile. Frame thread
only, where the menu pushes these and the callback runs: no lock.
pfd_switch_cores_test gains a lane, threaded and unthreaded: picks
made while the first install downloads are taken, run one at a time
and all install once it is through, and a repeated pick waits once.
It fails on the old code.
There is one cloud sync at a time, and a sync asked for while another
ran was turned away with a log line and nothing on screen. Content
loads and closes start network-bound syncs, so Sync Now, Resolve:
Keep Local and Resolve: Keep Server chosen during one did nothing.
A sync asked for without a pusher waiting on it is owed now, and
starts from the running sync's callback once it is through. Asked
for more than once, it runs once, with the conflict resolution asked
for last; a resolution wins over a plain sync. A pusher that waits on
its own sync is still told it was not run. The finder passes over a
finished sync, which stays in the queue while its callback runs, so
the follow-up started from there is not taken for a running one.
Frame thread only, where every push and the callback run: no lock.
cloudsync_pacing_test gains a follow-up lane: Keep Local and then
Sync Now during a sync are taken, keep the resolution, and one more
sync runs once the first is through. It fails on the old code.
A finished task stays in the queue until its callback has run, and
the duplicate-download guard found it there: the same URL pushed from
the failed transfer's callback was refused as already in flight. The
achievement badge downloader retries that way after a network error,
so every retry was dropped, the badge's callback data leaked and the
badge queue stalled.
A finished transfer is no longer in flight, and the guard passes it
over now, in task_http.c and task_http_emscripten.c alike.
http_transport_failure_test gains a lane that fails a GET and pushes
the same URL from its callback: the retry must be taken and reach its
own callback. It fails on the old code.
One Wi-Fi task at a time, and any other pushed while one ran was
turned away without a word. The networks list pushes a scan every 30
seconds while it is shown, and a scan can take seconds: Connect,
Disconnect and the Wi-Fi Enabled toggle chosen during it did nothing,
the toggle leaving the setting and the driver apart.
They are held now, in order, and pushed as the task ahead of them
finishes, from a callback every Wi-Fi task is given that runs the
caller's own first. Only a scan behind a scan already running or
held is refused: it would find the same. Past a bound of four a task
is refused as before. The task in flight is tracked by its handler
rather than found in the queue, where a finished task is still found
while its callback runs. Frame thread only, where the menu pushes
these and their callbacks run: no lock.
wifi_tasks_test runs task_wifi.c on the real threaded queue with a
driver whose scan takes 300 ms: everything pushed during it is taken
and runs after it in order, one on the driver at a time, each with
its callback, and a second scan is refused. It fails on the old code.
CI runs it under ASan + UBSan and TSan.
One screenshot that writes a file runs at a time, and one asked for
while another encoded was turned away without a word: the Screenshot
hotkey did nothing while a save's thumbnail or an achievement's
capture encoded, a second press straight after the first was lost,
and a save's thumbnail was skipped behind a screenshot.
It is held now, in order, and pushed as the screenshot ahead of it
finishes: its callback hands the claim on. Each held screenshot keeps
its own copy of the frame, taken when it was asked for, so it shows
what was on screen then. Past a bound of four it is refused as
before. Frame thread only, where take_screenshot() and the callback
run: no lock, and nothing per frame.
A screenshots-held lane in the content_load harness takes a second
screenshot while the first encodes and one behind a save's thumbnail,
and checks every file is written. It fails on the old code.
State tasks are TASK_TYPE_BLOCKING, so the queue takes one at a time
and refused the rest, which were then thrown away without a word.
Load State pressed while a save was still writing did nothing, nor
Save State during a load, an undo or an automatic save: the hotkey
had to be pressed again once the other was through.
A state task the queue turns away is now held, in order, and pushed
as the one ahead of it finishes; past a bound of four it is refused
as before. The save a backup load was taken for goes ahead of
anything held, so a Load State pressed straight after Save State
loads what that save wrote, not the file it replaced. The pending
flags become counts, so a content close, a quit and a movie recording
wait for held requests too. Anything still held when the savestate
backups are reset is discarded and its caller told.
All of it is on the frame thread, where every state push and every
state callback runs: no lock, and nothing per frame.
save_state_io_test's exclusion test now expects the hold and its
bound, and a new test checks that a load pressed during a save is
taken, counted as pending, and applies what the save wrote. The
state-load-latency lane gains Load State in the same frame as Save
State. Both fail on the old code.
Screenshots, Bluetooth scans and Wi-Fi tasks were TASK_TYPE_BLOCKING,
and the queue refuses any blocking task while another is running. A
state save or load is one, so Load State pressed while any of them ran
was dropped without a word: nothing happened until it was pressed
again after the other task was through. A save's thumbnail, a
full-size capture of a 4K output or a Bluetooth scan (up to ten
seconds) held it off, which looked like states taking a very long
time to load, at random and with every core.
Each keeps one at a time among its own kind instead and no longer
takes the blocking slot, so only another state task holds a save or
load back:
- a screenshot that writes a file takes a lock-free claim at the push
and gives it back as it finishes;
- a Bluetooth scan is refused while another is live, and its window's
end moves into the scan from a file static;
- a Wi-Fi task is refused while another Wi-Fi task is live.
The content_load harness gains a state-load-latency lane: with
threaded tasks and compressed states, Load State fired as the hotkey
does must reach the core within 30 frames after a save, beside the
save's thumbnail, beside a screenshot and after a video reinit. It
fails on the old code. The screenshot lanes find their screenshot by
its handler, not by its type. bluetooth_scan_test checks that a
blocking task is taken during a scan and a second scan is not.
The reap closes the device after deinitialising the pad, so the lane
waits for the close as well before checking it, rather than reading the
close count while the worker is between the two.
raac_state_size/save_state/load_state: what the frames decoded so far
leave the next ones - each channel's stream state and overlap, the
coupling channels, the SBR elements, the QMF histories at the doubled
rate and the PNS generator - for a decoder opened with the same
configuration, which then decodes on exactly as the saved one. Any
other size or configuration is refused.
raac_state_test saves before every unit of mono, stereo, 5.1 and
22.05 kHz (QMF) streams and loads into a new decoder.
rh264_video_state_size/save_state/load_state: the decoder between
access units - parameter sets, picture order, reference marking, the
DPB and output queue with their samples and motion. A loaded decoder
decodes on exactly as the saved one, and saves the same bytes again.
Pointers are not stored; a state from another layout is refused, and
a refused load leaves the decoder as after a seek.
A threaded decoder's last picture, open until the next one opens, is
closed by a save and shown when that opening would have shown it.
rh264_state_test splits seven clips at every access unit on one
context, four, and a pool; it runs in the samples and TSan lanes.
A frame's output depends on what the frames before it left behind, so
a decoder recreated after a savestate load, run-ahead or rewind gets
the next frame wrong. state_size/save_state/load_state copy that
history out as bytes and back into a decoder made with the same
parameters.
The tests decode each stream once straight through and once loading
the state into a new decoder halfway, and the two must match exactly.
A core with an audio callback runs behind the threaded wrapper, which
keeps the device blocking whatever the menu asks. At Menu Frame Rate
'Display Rate' the menu feeds silence by the clock, so it now writes
only what the device takes without waiting, keeping half its buffer,
and does not make up time the device could not take. The recording
still receives every frame. The menu runs at the display's rate behind
such cores again.
samples/audio/menu_callback_feed holds the feed against a device that
counts every write that would have blocked.
Fixes#19971
A kept device that does not match the next core's settings - a core
override of the audio latency, say - is closed before the pipeline is
rebuilt and before the next device opens, so it is released when the
next one may want the same device.
The threaded wrapper freed while parked now leaves from the park: it
no longer starts the device and runs one consumer pass on whatever
pipeline is current, nor reports a departed consumer or wakes it.
alive is published before the release from the park so the thread
sees it on the way out. Lane: pipeline_wakeups KEPT_DROP.
A callback may still be inside after ASIOStop, so teardown and the
reclaim of a parked instance waited for the in-callback count to reach
zero by yielding and then sleeping a millisecond at a time. The last
callback out now notifies an eventcount and the wait parks on it; the
200 ms deadline remains only for a driver whose callback never returns.
The eventcount is built for Win32 ASIO builds without threads too, as
it already is for DirectSound, XAudio and waveOut: the callback runs on
the driver's thread either way, so the write's park no longer falls
back to Sleep(1).
samples/audio/asio_teardown runs the driver's count and wait, pulled
out of asio.c, under TSan, ASan and wine; a build whose last callback
does not notify fails.
The P-256 and P-384 constants are computed on first use. The thread
that claims a curve computes them into it; one that finds another
thread computing them makes its own copy on its stack for that call
instead of yielding until the other is done.
test/crypto/ec_first_use_test makes eight first uses at once under TSan
and ASan, and holds one thread in the middle of the set-up while
another's first use must finish with the same key; built to wait for
the held set-up, that must fail.
The core info list read by the content scanner's task threads, and the
renderer's device handed to Apple decoder setup, each carried their own
copy of the last-reader-frees scheme. Both now use retro_retire.h,
whose handshake is sequentially consistent operations only; core_info's
copy relied on fences, which ThreadSanitizer cannot follow.
retro_retire_init() brings up one that is not zero-filled.
The worker publishes each display snapshot by swapping a pointer, and
waited for every caller to leave the one it replaced before freeing it.
It now hands the old snapshot to retro_retire.h: the last caller out
frees it, and the worker carries straight on with Mutter.
The threads case of samples/gfx/mutter_displayconfig reads the list
from four threads while the mode is switched; freeing a snapshot with a
caller still inside it is a TSan data race there.
OK on an entry of the Images (image history) list opens the image in
the menu viewer at once, in all four menu drivers, instead of offering
Run and loading it as content - a load that the menu then turned into
a file browser at the image's folder, so Back went to that browser.
Back now closes the viewer and leaves the Images list as it was, the
same entry selected; on Ozone's Images tab that also needed the
depth-one Back to stop moving the selection while the viewer is open.
An entry whose file is gone, or with the built-in image viewer off,
keeps its Run action. The menu-media workflow runs a new history-image
lane through each driver's real input path.
The run job checks out RetroArch beside Play!, so the shared apt
scripts are there, and lists its build packages in APT_PKGS for
apt-install.sh with the job's apt cache.
samples/frontend/ps2_whole_file reads, parses and writes whole files
through the shipping file_stream.c, vfs_implementation.c and
config_file.c - the path core info files and retroarch.cfg load
through. Built with PS2=1 it runs on the console's own runtime
(libcglue and fileXio over host:); a plain 'make check' runs the same
checks on Linux and Windows.
PS2-runtime.yml builds the ELF in the PS2 release image and runs it
under AutoTest from a pinned Play! commit, failing on any FAIL line or
a missing PASS verdict.
The screenshot task pushed to the image history only under
HAVE_IMAGEVIEWER, which normal builds no longer set now that images
open in the menu's own viewer. The image history and the Images tab
it feeds live under HAVE_IMAGEVIEWER || HAVE_MENU, so screenshots
were saved but never listed there. The push now takes that guard.
The content_load harness gains a lane that takes a screenshot through
the shipping task and checks it heads the image history.
ps2sdk's close() passes iomanX's result through, which is the closed
descriptor number on success, so the descriptor path treats only a
negative return as a failed close. Whole-file reads on PS2 - core info
files and retroarch.cfg - depend on it.
vfs_close_result_test builds the VFS for PS2 with a close() that has
those semantics and runs in the Linux, Wine, glibc/macOS and Windows
sample lanes.
Fixes#19952
Touch and fill the source frame before starting the clock so the benchmark reads resident pixel data rather than untouched calloc zero pages. Keep snapshot allocation and rotation inside the timed callback.
Rotate 90/270-degree snapshots in 32-pixel tiles so source columns and destination rows stay in cache. Select rotation outside the pixel loop and use a row-wise reverse copy for 180 degrees, retaining the one-buffer snapshot and packed output.
Exercise tile edges and non-square frames in the snapshot regression.
Use the pixel format's byte width and the source row pitch instead of inferring pixel size from pitch/width. Build the packed, rotated snapshot while the cached-frame borrow is valid, avoiding a second full-frame allocation and the copy back into the first. Read only active pixels, including on the last padded row.
Cover rotations, padded and narrow frames, source lifetime, allocation failures and deferred screenshot ownership on Linux and Windows.
Cores request rotation from retro_load_game, which runs before
drivers_init(), so video_driver_set_rotation() sees the vtable but no
instance yet. The rotation is latched in the system info and applied
by video_driver_init_internal(), so the request does succeed; report
it as such and only skip the driver call. Returning false sent cores
such as mame2003-plus to their internal rotation fallback (#19957).
A driver without set_rotation, or no driver at all, still reports
failure. Covered by set_rotation_deferred_test in the Linux gfx
samples lane, which runs the function extracted from video_driver.c.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LBQPGgGpgC8aYP7saK5N72
The session cache is shared by every thread making TLS connections, and
every lookup and store took a flag spun on with sthread_yield. Where the
pointer atomics are, each slot now points at an entry that is never
changed once published: a handshake that learns a ticket, a master
secret or a TLS 1.3 PSK builds the host's entry anew, swaps it in with a
compare-and-swap, and the entry it replaced is freed once no reader is
in. Builds without the pointer atomics keep the flag.
queues/retro_retire.h holds that last-reader-frees scheme for both this
and the x509 trust store, which used its own copy.
test/net/tls_cache_threads_test stores and reads entries for more hosts
than there are slots from several threads, and holds a read on its
entry while a store for another host must finish; built on the flag,
that must fail.
TLS handshakes on several threads walk the process-wide trust store
while a reload may replace it, and every walk and reload took a flag
spun on with sthread_yield. Where the pointer atomics are, a walk now
counts itself in and reads the published store, and a reload swaps its
store in with a compare-and-swap; the store it replaced waits on a list
and is freed by whoever is last out, so no walk or reload waits on
another. Builds without the pointer atomics keep the flag.
x509_store_threads_test holds a walk just as it takes the store and
requires a load of another store to finish meanwhile; built on the
flag, where the held walk keeps it, that must fail.
Reuse decoded PNG storage for a matching RGB565 software frame and narrow its pixels in place. This removes the output allocation and scaler context while preserving the decoder buffer's existing lifetime and the conversion path for resized replies.
Verify pointer identity, colors, zero additional image-task allocations, and allocation failures in both reuse and resized RGB565 paths.
Define the positive, pixel-aligned stride contract for ARGB8888-to-RGB565 conversion in the same buffer. Use byte accesses in the scalar path so overlapping 32-bit input and 16-bit output do not violate C aliasing rules; retain the existing alias-safe SIMD loads and stores.
Check all RGB565 colors, write bounds, short widths, vector tails and packed or padded output rows.
The overlap checker flags the row copies because they share a base buffer. Use memmove to keep row duplication compatible with the check without an allowlist entry.
A resume fills the drained device with silence before the core's first
audio. With a blocking writer it now fills only to rate control's
setpoint, half the buffer, instead of to full, where the writes after it
waited for the surplus to play out: a hitch of a frame or more on every
return from the menu, a state load or any other jump, which a VRR display
shows. A non-blocking writer keeps the full top-up as its cushion.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LBQPGeAnyXTjoifK18NSfm
Move decoded PNG pixels into the output slot when their geometry and native format already match the software frame. This avoids a full-frame output allocation and copy, as well as scaler setup, while retaining opaque XRGB alpha and the existing cleanup lifetime.
Check pointer identity, zero additional task allocations, guarded hardware/HDR/invalid frames, and allocation failures in both reuse and conversion paths.
Borrow validated BMP pixels from the response for the synchronous scaler call. Feed decoded PNG ARGB pixels directly to the scaler instead of allocating and flipping a BGR copy, preserving opaque software-frame output and keeping decoder cleanup separate from borrowed data.
Check colored PNG orientation and alpha, RGB565 output, unaligned BMP payloads, allocation failures and borrowed-buffer lifetime. Run the reply harness on Linux and Windows with and without PNG support.
Retiring a pad's slot closes its gate and waits for the readers still
inside; it waited by yielding in a loop. It now parks on an eventcount
that the last reader out of a closed slot notifies, so the thread
retiring a pad (a HID driver's worker, or the main thread at teardown)
sleeps until the slot is empty. A live slot's readers never notify.
samples/input/connection_gate holds a reader inside while the slot is
retired and requires the retiring thread to park; a build without the
eventcount must fail.
Fuse BGR24 point sampling with ARGB8888 or RGB565 output conversion for reductions of at least four source pixels per output pixel. This avoids full-frame input and output conversion buffers while preserving the existing near-unity SIMD path.
Extend pixel, signed-stride, workspace and benchmark coverage to both BGR24 input paths.
Bind the missing ARGB8888-to-RGB565 converter for direct conversion and resized output. This lets translated BGR24 images return to RGB565 cores when their dimensions differ from the frame. Use truncating channel reduction with SSE2, NEON and scalar implementations.
Check RGB565 roundtrips, SIMD tails, signed strides and padding, plus point/bilinear/sinc scaling from ARGB8888, BGR24 and RGB565.
Convert selected RGB565 samples directly to ARGB8888 or BGR24, and ARGB8888 samples directly to BGR24, without full-frame conversion buffers. Keep filter validation and use the existing SIMD conversion path for enlargements and reductions below four input pixels per output pixel.
Cover sampling, signed strides, allocation behavior and all RGB565 values against the existing converters.
The GameCube and Wii on os/gekko (GEKKO_NATIVE) have a futex, the
address wait the Linux, Switch and 3DS backends park on, but took the
condition variable and its lock on every notify. They now park on the
epoch word with gk_futex_wait and wake with gk_futex_wake; a timed wait
is turned into time base ticks and clamped rather than wrapped. The
libogc builds keep the condition variable.
retro_eventcount_test builds the backend on the host with GEKKO=1,
against a slice of os/gekko and its futex stood in for over Linux's at
the Wii's time base rate; check also requires a build that loses every
wake to fail.
A 4096-square packed 10-bit white image reduced to one pixel produced
cff3fcff rather than ffffffff: channel sums overflowed unsigned int.
The 8-bit path has the same problem at an 8192-square box.
Use wide accumulators only above the largest safe power-of-two factors:
2048 for 10-bit and 4096 for 8-bit. Retain the existing 32-bit loop for
normal thumbnail reductions, including its truncating division.
Check both safe boundaries and large-factor colored output through the
image scaling API, with ownership cleanup and allocation counts.
Expand each source row once and copy it vertically for power-of-two
thumbnail enlargement. These factors are exact in the existing 16.16
point mapping, so 8-bit and packed 10-bit pixels remain identical while
filter-table setup and repeated point sampling are avoided. Keep the
generic scaler for other factors.
Check dimension multiplication and the pixel limit before allocating,
including on 32-bit size_t targets. Exercise 2/4/8/16x output against the
existing sampling reference, 3/5x fallback, metadata and allocation retry.
When box decimation already produces the target dimensions, the scaler's
same-size sinc stage is just a copy. Transfer the decimated buffer into
the image instead of allocating and copying another destination.
Preserve pixel averaging, metadata and the sinc fallback for nonmatching
dimensions. Test 8/10-bit pixels, allocation-budget success, failure/retry
and ownership cleanup on Linux and Windows.
Point sampling copies from the canonical input to the canonical output
without using the horizontal-pass frame. Keep its geometry validation,
but omit its allocation, clearing and metadata for point contexts.
Native ARGB8888 and packed 10-bit point scaling need no frame arena;
format conversions retain their input/output frames. Bilinear and sinc
keep their workspace and filter setup is unchanged.
Test native, converted and 10-bit pixels with padded/signed strides,
mode reuse, frame-allocation failures and oversized geometry. Run the
sample under Linux sanitizers and Windows/Wine.
The shared sample job builds the all target and requires every executable
to be classified. Give file_list_oom both so it can run there as well as
in the dedicated Linux/Windows allocation checks.
Recent libsmb2 only adds accepted server contexts to its active list, so smb2_context_active() rejects connected clients. This makes the external backend fail seek/read/tell immediately after opening a file. Check the client's socket instead, retaining creation/destruction serialization for older libsmb2 releases.
Nothing reports an SD card or USB drive coming or going, so the thread
looks once a second. It waited out that second on a condition variable
under a mutex it held the whole time; it now waits on an eventcount,
and stopping it is an atomic flag and a notify that wakes it at once.
samples/frontend/gx_devthread runs the thread against libogc and libfat
stand-ins: a drive put in is mounted, a card taken out unmounted, and a
stop returns at once; a stop that does not wake the thread must fail.
Run the new list and conversion tests under ASan/UBSan, then cross-build
and execute them under Wine. Exercise the production-source Vulkan, menu
and translation guards with both native and MinGW compilers.
Sample and convert XRGB8888, RGB565 and BGR24 directly into the BGR24
destination when downscaling. Match the existing point scaler's fixed-point
sampling and RGB565 expansion without full-frame ARGB intermediates or
filter allocations. Keep the generic scaler for enlargement.
Return conversion setup failures to screenshot and translation callers.
Reject changed software snapshot dimensions before writing to a destination
allocated from the earlier snapshot. Translation readbacks already at the
requested size transfer buffer ownership instead of allocating and copying
a second image; resized readbacks use the fused helper.
The regression compares pixels and padding against the generic scaler for
three formats, six size pairs and four stride orientations. It also checks
failure recovery, small output, and translation ownership and bounds.
Failure propagation was motivated by PS5_RetroArch port change 0086.
The fused downscaler and translation ownership transfer are new work.
Optionally enable VK_KHR_get_physical_device_properties2 wherever the
implementation advertises it, rather than only on desktop Windows.
Vulkan 1.0 cores may need these queries during context negotiation.
Move the existing entry so Windows does not request it twice.
Inspired by PS5_RetroArch port change 0089.
A cached-frame render may change viewport dimensions after a screenshot
caller allocated its destination. Reject that capture before copying,
rather than clipping into an output with a different row layout. Apply
the guard to SDR/BGRX and native HDR captures.
Both HDR uniform buffers are allocated regardless of runtime HDR support.
Release them outside the support conditional using the existing null-safe
destructor; retain conditional cleanup for HDR-only image resources.
Includes production-source guard and cleanup probes on Linux and Windows.
Inspired by PS5_RetroArch port changes 0090 and 0093; rejects dimension
changes instead of silently changing capture geometry.
Resolve the literal "default" through the frontend's DEFAULT_DIR_REMAP,
instead of allowing it to become a relative directory named default.
Explicit paths are unchanged; a frontend with no remap default resolves
to the same empty value used during configuration initialization.
Inspired by PS5_RetroArch port change 0113.
Check both string allocations before publishing a list entry. Construct
inserted entries before moving existing rows, reject out-of-range indices
and overflowing growth, and avoid reallocating an already reserved capacity.
Acquire menu callbacks before inserting a row and return them to the pool
if insertion fails. A rejected prepend must not reinitialize an existing row.
Includes deterministic allocation failures and immediate retry checks.
Inspired by PS5_RetroArch port change 0080; retains upstream's shared empty
strings and callback pools without adding a persistent failure latch.
Poll the epoch before publishing a Windows waiter or allocating its per-thread handles. Detach the wake list with a single CAS while respecting the unlink bit.
Preserve the alert, keyed-event and event tiers and high-resolution timeout handling. Add deterministic publication and unlink race checks to the Windows wait matrix. Separate benchmark control words from eventcount metadata and report ping-pong roundtrip time accurately.
Readers, control messages and teardown all bump the worker's futex word
from their own threads; with a plain increment two of them could land
as one and the worker would sleep through a wake until its next scan.
The count is an atomic, and the futex waits on its word.
samples/input/gekko_hid now runs in CI under ASan and TSan, which
caught the plain increment.
Use WaitOnAddress without a library waiter-list lock on Windows 8 and newer. Retain legacy backends by default and add an opt-in strict mode that refuses lock-based backends. Native Windows timeouts round up to milliseconds.
Publish Linux spin calibration with CAS so another initializer can make progress while the first is suspended. Cover concurrent initialization, nested calibration, and strict backend selection.
libretro_d3d11.h version 2's lock on the immediate context lets a core's
outermost lock_context give way to a frontend waiting for it, since a
critical section is not fair. The core used to spin on SwitchToThread
until the frontend was in; it now parks on an event the frontend sets
once it is. The park and the frontend leaving the queue form a Dekker
pair on seq_cst atomics, so a parked core is always woken and the
frontend sets the event only when one is parked.
samples/gfx/d3d11_turns runs a core locking in a tight loop against a
frontend asking for the context, under Wine; a core that does not give
way must fail.
A pad that leaves is asked to stop and its reader cancelled; the worker
asks again on every pass, since a cancel only returns a transfer
already waiting, and reaps the pad once the reader says it is done. No
thread sleeps for it.
A driver freed while a reader still waits leaves the pad to that
reader, which frees it when its transfer returns. The driver itself is
freed by the last of it and the readers still running, so a late
reader no longer touches a freed driver.
samples/input/gekko_hid runs the driver against os/gekko stand-ins:
unplugging while the reader waits, with its cancel lost, and freeing
the driver under a waiting reader; a build that asks only once must
fail.
The renderer's device for VideoToolbox HEVC and VP9 decoder setup is
held in a decoder_device_slot: the lifecycle thread replaces it with
one exchange, and setup on any thread takes a retained copy. A replaced
device is released once no thread is between loading and retaining it,
by whichever thread is last out. The Metal snapshot's device,
generation and HDR flag travel in the same node, so a view is always
one consistent publication. The setup lock is gone.
samples/gfx/videotoolbox_vp9/snapshot_test.c replaces the device in
the middle of an acquire and races replacements against readers; a
build that releases on replacement is run as the negative lane.
The recorder's av_log callback builds each line in a slot of the
calling thread's own, claimed with a compare-and-swap and released once
the line is out, so lines from the encoder threads never splice and no
call waits on another.
The ffmpeg core hands each subtitle track's newest change to the thread
that renders through a one-slot mailbox swapped with a single exchange;
only that thread calls into libass, seeks included. A change supersedes
the one before it, so the held memory stays bounded when nothing
renders. Targets without pointer atomics keep a lock around the swap.
samples/cores/ffmpeg_ass_events checks the hand-off against a libass
stand-in, and samples/record/ffmpeg_pipeline gains a lane for whole log
lines from several threads.
Jobs posted while the ring is full go on a lock-free stack, counted
before they are pushed. Whichever thread finds the ring empty and holds
the drain token takes the oldest for itself, moves as many more as fit
into the ring and wakes a worker for each; one turned away leaves word,
so the holder looks again before it lets go. Overflow nodes come from a
pool of spares on a free list tagged against reuse.
A worker that sees a job counted and cannot take one sleeps instead of
yielding in a loop, and passes on a wake that found it still marked
parked.
tpool_add_work joins the lock-acquire budget at zero.
Leaving fast-forward drops what it left in the threaded pipeline's ring
behind a jump fade, and the device's writer drops core audio until the
device has played down to rate control's setpoint, so a blocking writer
is no longer held for as long as the queue takes to play out. That held
the core in device-period steps for seconds after every fast-forward,
which a VRR display shows as judder until the menu is toggled (#19940),
and cost every other configuration a hitch on exit. The five-frame
non-blocking window Sync to Exact Content Framerate used for this is
removed; it never drained the queue.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LBQPGeAnyXTjoifK18NSfm
The driver stage of a staged content load leaves the audio driver out
when its init would open the same device the same way for the next
core - same driver, device, output rate, latency, resampler and
threading, no DSP filter, time stretch or core audio callback, and the
core's float and multi-channel entries as they were. The device,
pipeline and mixer keep running through the rebuild of the rest; once
the video driver has settled the rates, the new core's rate is set in
place and the stream reset a core swap needs is done. When the rate
cannot be taken in place, the driver is opened again, the device kept
across as before. The driver lookup of the stage leaves a carried
driver's table alone.
The content_load harness's close keeps its tone playing with the
device never stopped, and a load with time stretch on sets the device
aside and takes it back.
The system sounds are loaded again after every driver stage. A slot
that already holds the sound from the same file now keeps it, and a
callback the load would have run - the menu music's play - runs on the
kept one; a slot the mixer lost loads again. A content load or a
driver reinit no longer cuts a menu sound that is playing over it, nor
restarts the menu music.
The content_load harness finds its tone where an install keeps its
sounds and loads it through the system sounds, so the reload at the
end of a confirmed Close Content runs against the sound playing over
it.
The core title the menu shows stays the one captured when a staged
load starts - the core it is for, when its library is already open -
until the load is through, so it changes once instead of following
the libraries closed and opened between the stages.
The content_load harness reads the title every frame of a playlist
Run and checks it changes at most once, to the title the load ends
on.
Auto, picked from a playlist entry's Load State list, loads the auto
state whatever Auto Load State is set to, through the same path as an
automatic load, deferred for a core that must run first.
The content_load harness's playlist lane picks Auto with Auto Load
State off.
Requests wait in two lanes of slots the UI thread fills; a worker takes
a job by swapping its slot to NULL, and the UI thread takes one back
the same way, so each goes to exactly one owner. A job is one record
from request to delivery, back through a lock-free list. animate()
hands the animation thread a whole request, holding the still it waits
for; the worker parks that still's session in an atomic slot and marks
the job landed. Prefetch is served oldest first, as the header says.
An eviction keeps an entry a decode in flight still points at.
GCD builds take the lock-free runner, with GCD's queues in place of its
workers. Each push of a task due now makes a call that takes tasks
from its ring until it finds none: plain tasks on a serial queue, so
they never overlap, parallel ones on a concurrent queue. A serial
keeper queue holds the tasks waiting for their time, with one timer
source, and every timed push and wake nudges it. Each init is a
generation its calls hold; deinit waits only for the calls past their
check, and leaves a detachable handler behind after the bound as the
threaded runner does. The locked GCD runner and task->sched are gone.
task_queue_deinit() waits for every thread still inside the wake gate
before the runner goes.
The GCD-capable samples also build against a libdispatch stand-in on
pthreads (GCD=standin), and task_queue_overlap_test checks which tasks
run at once on either runner, with pushes stopped halfway into the ring
through a test hook.
Unloading the game inside a staged content switch ends the core's
audio with the pause tail instead of stopping the device: the device
keeps running until the driver stage sets it aside or closes it, so
the menu's feed and a menu sound playing over the close - the OK of a
confirmed Close Content - play on without being written into a
stopped device. The core stays gated until the drivers are rebuilt.
The content_load harness plays a tone through a confirmed close and
checks the device is never stopped while the menu feeds it and that
the tone it is handed is whole on both sides of the driver stage.
Picking a slot from a playlist entry's Load State list records the
pick on the menu handle, and the content load it starts takes it in
event_init_content(), however many frames later that runs. The pick
loads that slot ahead of any auto state, and is dropped with a load
that fails before the content comes up.
The content_load harness runs an entry through the real Load State
list and checks which state the core unserialized, with an auto state
present and without one.
gfx_widgets_visible() answers false for inactive widgets itself, so
the replay path assigns its answer directly. The replay test lane
drops the widget pointer and fps_show it set again after the loop.
A waker reads the parked block, sets WOKEN and then signals the
semaphore. A park woken through the semaphore now takes WOKEN with an
acquire before its frame goes, so the order the semaphore gives is one
ThreadSanitizer sees too; a Mach semaphore is not one it intercepts,
and on Darwin it reported the next park's write to the block as a race
with the waker's read.
The eventcount test's semaphore builds on Linux reach the C library's
semaphore past the interceptors, so they see no more order than Darwin
gives and the race is caught there.
Populate widgets_visible on the video thread during cached-frame replay,
which bypasses the normal worker loop. Keep UI composition and HDR
decisions consistent with the widgets being drawn without advancing
their animations again.
Extend the replay tests to cover visible widgets and stale visibility
after widgets are disabled.
For the parallel-xbox core, which is moving QEMU's own atomics onto
retro_atomic.h: QEMU keeps 64-bit counters and flag words it adds to
and masks atomically, and a 16-bit halfword (the vCPU's exit request)
that other threads store to while generated code reads the 32-bit word
it is part of, so it cannot be widened.
retro_atomic_fetch_{add,sub,or,and}_64 are acq_rel and return the old
value, wherever RETRO_ATOMIC_HAS_64 is set; a backend with no 64-bit
read-modify-write of its own (32-bit MSVC, Apple OSAtomic) loops on
retro_atomic_cas_64. retro_atomic_u16_t has relaxed and acquire/release
loads and stores on every backend.
With the lock-free runner a task's title and error are also published
as atomic pointers beside the handler's own fields. Progress reports
and snapshots read the published copies inside a read epoch, and a
string replaced while a reader is inside goes through the epoch's
limbo instead of being freed at once, so a worker replacing its title
never waits on the frame that is formatting it. The getters return the
published copies; a title or error assigned to the field directly is
published when the task is pushed.
The locked runner keeps property_lock.
The wide-frame renders count the extra positions while filling their
slot table, from the bits the frame carries, so the copy loops never
read a slot the table does not hold. This clears GCC's
-Wmaybe-uninitialized on the pipeline render; the inline transport
takes the same shape.
GCC builds of the AVX-512 folds spell VPCLMULQDQ out in EVEX bytes with
a gas macro that reads the register numbers off the operands GCC
picked, so binutils older than 2.30 assemble them; the code is the same
instruction stream as the intrinsic form. The 128-bit tails stay out
of those functions so none of their PCLMULQDQ lands on XMM16-31.
<cpuid.h> is included once per unity build, since clang 12 and older
ship it without an include guard.
libretro-common/samples/encodings/crc32_gas checks every register
triple of the macro against the assembler and the SDM, that GCC's
assembly of the file at -O1 to -Os names no VPCLMULQDQ an old assembler
rejects, and that encoding_rzstd.c and encoding_crc32.c share a unity
build under an unguarded <cpuid.h>.
play_feature_delivery_enabled() tested the isMetaQuestBuild method ID
rather than the flag read through it. The ID resolves on every build,
so feature delivery was off everywhere and Play Store builds offered
the whole buildbot core list through the core downloader.
The answer now comes from the build flags the frontend reads from Java
at init, and is not cached until it has read them. The
play_feature_delivery sample checks Play Store, Meta Quest and other
builds with both method IDs resolved, as on a device.
Where the atomics have compare-and-swap and pointer operations, the
threaded runner takes no lock and waits on no condition variable.
Every task is on a live list from its push until it is retired: any
thread links it at the head, only the checking thread unlinks it, and
find(), retrieve() and the lookups behind wake, cancel and reset walk
it from any thread inside a read epoch. A retired task is cleaned up
and freed once no finder can still hold it, at once when none is
inside, since finders read task->state.
Workers take ready tasks from two bounded MPMC rings: the plain one,
which only the one worker takes from, and the parallel one, which
every worker does; what overflows waits behind them in order. A task
with a 'when' to wait for goes to the timer keeper, whichever idle
worker holds the token, which sleeps until the earliest is due and
hands the token on before it runs anything. Main-thread tasks and
finished ones reach the checking thread through lock-free stacks.
Workers, task_queue_wait() and deinit's bounded wait for the workers
park on eventcounts. Each gather with work in flight and each pass's
end advance one counter that both read, so a handler still sees what
the frame loop wrote before a check, as the frame count a load waits
on, and has read it before the loop writes again.
The volatile and PS2 backends keep the locked runner, as does GCD.
task_queue.c now uses retro_eventcount.c, which the sample builds
compiling it with threads now link.
task_queue_parallel_test floods both rings past what they hold with
every worker held; task_queue_switch_test holds a waker in
retro_eventcount_notify() where the runner takes no lock.
audio_driver_set_core_float() decides a source format change at once
and hands it to the threaded pipeline's consumer as a request word, to
install before its next pass; the producer writes nothing into the ring
until it has. A populated ring keeps its native format, as before, and
the producer converts its writes meanwhile. The stage a running or
suspended transport needs in the new format is made by the producer and
the old one handed back to it to free; one that cannot be made drops
the transport, keeping the new source native.
That was the last control to park the audio thread, so
audio_thread_apply_control() goes. Harnesses that parked the consumer
to look at its state stop and start the wrapper instead, and
thread_handshake checks audio_thread_parked() across its round trips.
pipeline_format hands its format change to a consumer pass and checks
nothing takes effect before it.
Suspending, resuming, preparing and releasing the pitch-preserving
transport are decided on the producer's side, which keeps the transport
as it has decided it in pipe_transport_mode, and handed to the
consumer as a request word to install before its next pass. Nothing
waits and the device keeps running, so the speed changes that switch
the transport on and off no longer stop and start it.
The producer keeps writing meanwhile: the source in the ring is the
same whichever consumer takes it. A resume or a prepare still wants the
source drained, and puts its processing request into the metadata queue
behind what is already there rather than resetting the queue; the
consumer leaves it for a later pass while a short device write is
still to finish. A prepared stage is made by the producer, and the
replaced or released ones are handed back to it to free. With the
consumer parked the change is carried out in place, as before.
pipeline_wakeups' automatic transport lane checks the device is not
stopped across the suspension and the recovery.
audio_driver_set_input_rate() decides the change on the calling thread
and hands it to the threaded pipeline's consumer as a request word, as
the pause requests are, to install before its next pass. Until it has,
the producer writes nothing into the ring, so the ring, the slice and
the ratio are the consumer's alone and neither thread waits on the
other; the device is no longer stopped and started for it. With the
consumer parked, or gone, the caller installs the change itself.
A ring that has to grow is made by the caller and moved into by the
consumer with retro_spsc_grow_into(), which allocates nothing; the old
buffer and resamplers come back to the caller to free with the next
change or at teardown. The extra lanes of a multichannel core, which
used to refuse the change, are dropped with the front pair they share
tables with and remade at the new ratio by the next pass.
pipeline_rate_change hands each threaded change to a consumer pass and
checks nothing takes effect before it; pipeline_wakeups' kept-device
lane changes the rate under the real wrapper with the device running;
discrete_multichannel changes it with the extra lanes up; retro_spsc
moves a ring into a caller's buffer.
Without the threaded pipeline, the thread that submits the core's audio
holds the inline state: a submit takes inline_busy with a compare-and-
swap and records itself as the owner. The main thread's pause, resume,
format change and stop reset are published as request words, as the
threaded pipeline's already are, and carried out by that thread at its
next submit - before its gates, so a silenced core's thread writes its
pause tail - or at once by the main thread when it is the submitter, or
nothing has submitted yet. A callback core's thread settles them in its
loop. A submit that finds the main thread carrying a request out drops
its batch rather than wait. The core format the mixer follows is
published by the flush. audio_driver_sample_batch grows by the call
that settles requests on the submitting thread, and its hot-function
reference moves with it.
With that, every holder of the state lock is a pass that no other runs
beside, and the lock is only made where the atomics have no pointer
operations; elsewhere audio_driver_state_lock() counts the pass and
nothing more.
ff_speedup pauses a core that submits from its own thread and checks
the main thread writes no tail and that thread writes it at its next
submit.
* vulkan: end the HDR UI layer's render pass when a shader is loaded
With HDR on and a shader preset loaded, the game pass ends the main
render pass and clears end_main_pass. When the menu, an overlay, a
message or a widget is drawn on top, the UI layer then begins
sdr_render_pass on the off-screen buffer, and with end_main_pass still
false nothing ends it. The composite that follows records its barriers
and begins the next render pass inside it, which Vulkan does not allow
(VUID-vkCmdBeginRenderPass-renderpass,
VUID-vkCmdPipelineBarrier-None-07889).
RADV draws these frames correctly anyway. On Android the touch overlay
is on by default, so every frame takes this path, and it is the likely
cause of the lost device in #19699 (HDR plus any shader on an Adreno
tablet).
* vulkan: test the HDR10 frame's render passes under the validation layer
No CI lane ran the Vulkan driver with an HDR swapchain: lavapipe on X11
has no HDR10 surface format. So nothing saw the UI layer's render pass
left open under the composite whenever a shader preset and any UI met,
which the previous commit fixes.
samples/gfx/vulkan_hdr_ui_pass runs the shipping frontend from the
VULKAN_DEBUG=1 build on lavapipe under an Xvfb of depth 30, which gives
the surface a real 10-bit format. A layer, fake_hdr_layer.c, reports
that format a second time paired with HDR10_ST2084 and passes an HDR10
swapchain on in sRGB with the same format and images. The driver then
selects HDR10 and takes its whole HDR path; what it records is
unchanged.
The cases: a two-pass preset with the menu open, with the statistics,
and with an OSD message; the menu case again on the video thread; and
the controls, the preset with no UI and the menu with no preset. Each
must create an HDR10 swapchain and log the HDR game pass and composite
pass it is meant to take, so a run that fell back to SDR, or did not
draw the UI, fails rather than passing vacuously. Any validation error
fails a case.
Without the previous commit, the four preset-plus-UI cases fail with
VUID-vkCmdPipelineBarrier-None-07889 and
VUID-vkCmdBeginRenderPass-renderpass, the two errors in the report; the
controls pass either way. It runs in the Vulkan validation workflow's
harness job, which already builds the tree VULKAN_DEBUG=1.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LBQPGctE3xjHw1i9kE5EzK
---------
Co-authored-by: libretroadmin <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
* fix(ps3) RSX: compose frontend layers correctly on duplicated frames
Redraw the cached core image before frontend layers when a duplicated
frame needs presentation. This keeps widget animations visible without
covering overlays or statistics.
Use widget visibility instead of subsystem activity to avoid unnecessary
redraws. Present one cleanup frame when widgets disappear or are disabled,
and ignore empty OSD messages when deciding whether to redraw.
Add regression coverage for layer ordering, visibility transitions and
hardware, software and CPU fallback frames.
* RSX: take any removed frontend layer off a duped frame
One bit records whether the last presented image carries frontend
layers; a duped frame without any is presented once more over the
bare core image, so a closed menu, an expired message, statistics or
an overlay turned off no longer stay on screen until the next real
frame. Replaces the widgets-only bit; the ui_cleanup lane covers it.
* video: answer widget visibility once per frame in video_frame_info_t
widgets_visible is filled by the thread that draws the frame - in
video_driver_frame without the wrapper, and on the video thread after
gfx_widgets_worker_step with it - so drivers read one bool instead of
calling gfx_widgets_visible() at each decision. Every driver's views
UI layer, the Vulkan/D3D12 HDR UI routing and RSX's dupe presentation
read it; the widget-less variants of those calls collapse into one.
* gl1: present duped frames that carry frontend layers
A duped frame with the menu, an overlay, an OSD message, statistics or
visible widgets - or one that takes the last of them off - is drawn
again over the core image the texture still holds and presented, so
widget animations no longer freeze and their draws no longer pile up
in an unpresented backbuffer. A bare dupe stays unpresented.
gl1_draw_tex draws the texture as it stands when given no source.
* gl1/gl2/gl3: scRGB menu brightness only while widgets are on screen
The encode takes Menu HDR Brightness instead of paper white while any
UI is up; widgets now count when they are visible, as on Vulkan and
D3D12, rather than whenever the widget subsystem is enabled - which
kept SDR content at menu brightness all the time.
---------
Co-authored-by: LibretroAdmin <[email protected]>
* video: per-screen geometry for the headset's floating screens
Each floating screen, by the core's logical screen id, gets its own
offsets, yaw, pitch, roll, scale and visibility (video_screen_xr_t),
applied on top of where the layout puts it. video_views_xr_reset gives
the defaults and video_views_xr_normalize clamps each value to its
range: offsets to 10 m either way, angles to 180 degrees, scale from 0.1
to 10. A planner given no screens uses the defaults, so nothing moves
until a setting does.
* configuration: save per-screen headset geometry
video_screen<N>_xr_x, _y, _z, _yaw, _pitch, _roll, _scale and _visible
keep each floating screen's adjustments for screen ids 1 to 8, and
survive configuration overrides like the other video settings. A minimal
save writes only the values that differ from the defaults.
The video_screens settings sample checks the round trip and the override
cases, and runs in the OpenXR input and Wine views jobs. The defaults
fixture now links video_views.c, which the reset uses.
* menu: per-screen controls for the headset's floating screens
Stereo 3D & Screens gains Screen, which picks a floating screen by its
logical id among those in the current frame's map, and for it
Horizontal, Vertical and Depth Offset, Yaw, Pitch, Roll, Scale and
Visible, with Reset Screen and Reset All Screens. The rows apply live
and are hidden where they cannot apply, such as a tracked projection's
eyes. Stepping between screens reads one snapshot of the map and is
bounded, so a tracking change mid-step cannot leave it walking an empty
map.
docs/openxr.md describes the controls; the video_screens sample drives
them through the shipping menu actions.
* video: apply per-screen headset geometry to the frame snapshots
Each XR planner, Vulkan's and GL's, takes the normalized screen settings
with the frame snapshot, so a screen moves, turns, scales or hides as
its rows change, while paused too; recentering and tracked projections
are unaffected. Screen images are kept between frames, so shader
feedback survives a placement change.
A retained XR target that cannot be allocated is skipped quietly,
remembered by its size so it is not retried every frame; only the
desktop UI and canvas targets log the failure.
Covered by openxr_core_test, the video_screens sample, the e2e runner's
feedback shader case and the new vulkan_views_target sample.
* android: keep the OpenXR EGL context C89
gl_xr_egl_init declared its extension string after statements.
* video: pose inverse, layout spots and grab maths for headset edit mode
Each item's layout spot comes from one helper that placement uses and
the quad set records per slot, so a shown pose turns back into the
settings that placed it. Grab maths: the item follows the aim, level
unless tilting, offsets clamped to the settings' range; the stick pushes
along the ray and resizes. Edit picking sees every item and takes the
menu by a handle bar under it.
* video: a frame without a view map is screen 1 in the headset
A core without a views map (SNES, say) had no screen to select, so its
frame could not be moved. Its whole frame now takes screen 1's offsets,
turn, scale and visibility, and the selector lists Screen 1. gl's window
stays as laid out while the menu is open over it.
* video: the headset's menu in front of screen 1, or where it was put
The menu sat at a fixed spot, so moving screen 1 closer left the menu
behind it. It now floats 10 cm in front of screen 1, facing as it does,
at its default angular size and never nearer than 0.4 m, while screen 1
floats and shows, and is still submitted after the screens. A menu place
(video_openxr_menu_xr_* with video_openxr_menu_placed) overrides that;
it is saved like the per-screen keys, and Reset All Screens clears it.
* input: Edit Headset Screens mode for the OpenXR controllers
Edit Screens starts from a hotkey, a headset action, the
HEADSET_EDIT_SCREENS command or a Stereo 3D & Screens row, and stops on
the same, on opening the menu, on a video reinit or when the session
ends. It syncs its own action set in place of the game's, points the
lasers at every item whatever the Laser Pointer setting, keeps the
menu's quad up while editing, and with While Editing Screens set to
Pause pauses only content that was running. A runtime that refuses its
bindings keeps the game's and leaves edit mode unavailable.
The setting reaches the headset's poll with the others, and the pause
and resume are asked of the frontend as platform requests, so the input
driver issues no command and takes no settings itself. A Laser Pointer
value past its choices, which a hand-edited config can hold, reads as
Auto rather than as edit mode's.
* input: grab, tilt, push and resize headset screens in edit mode
A grip on an item takes it in the screens' frame: it keeps its pose
relative to the hand, stays level unless the trigger is held, and the
stick pushes it along the laser and resizes it, past a deadzone. Each
poll writes the item's offsets, turn and scale where the menu's rows
write them, through named frontend calls, clamped so it waits at a limit
and resumes from there. The menu is taken by its handle; a menu that
follows screen 1 is placed as large as it shows. The main button resets
a free item as Reset Screen does, or puts the menu back in front of
screen 1. A recenter, a screen leaving the map and leaving edit mode let
go, counted for the frame snapshot; an untracked hand holds its item
still.
* openxr: held screens follow the hand at the headset's rate
The main thread publishes what each hand holds through a seqlock; each
headset frame re-places a held item, both eyes of a stereo screen, from
that frame's own aim, so it no longer waits for the next video frame. An
item let go stays where the hand left it until a quad set planned at or
after its release (an edit count carried in the frame snapshot) shows
it, even once the hand holds another item, so nothing steps back. Lazy
Follow holds its glide while anything is held.
Edit mode outlines every item, brightly the one under a laser, and draws
the menu's handle bar under it, from the laser's image, now an atlas of
the dot, two outlines and two handle bars. They take only the layers the
screens leave after the dots.
* openxr: edit mode end-to-end cases and documentation
The test layer's aim can roll, and the e2e runner drives edit mode
against Monado: grabs that follow the hand on every headset frame and
stay where they are let go, level and tilt, the stick, resets of a
screen and of the menu, the menu taken by its handle and following
Screen 1 at its size, outlines and the handle, a no-map core, a views
core's fallback frame, Follow Head, Lazy Follow, recentering, two hands,
pausing, and what starts and ends edit mode, including the headset's
hotkey bits with no command interface. The cases whose work is a host's
or driver's own run on glcore and gl too. docs/openxr.md explains
editing screens in the headset and where the menu goes.
* openxr: a laser beam from each controller to its dot
A thin strip runs from each hand's aim to its dot, or 1.5 m into
nothing, turned about its own axis each headset frame to face the head
so it never shows edge on. It comes from the laser's image, so it needs
no new swapchain, and uses layers left after the dots and edit mode's
outlines, so it is the first to go when the runtime has few. The new
Laser Beam setting shows it in edit mode only (the default), always, or
never; outside edit mode, Laser Pointer and its hotkey hide it with the
dot.
* openxr: edit mode outlines in three bands, outside the item
A light band between two dark ones shows on white, black and busy
borders alike. The rim now sits wholly outside its item, 5 mm clear of
it even after filtering, so it never covers the picture; its region
grows to keep two texels a band at about the old thickness. Screens'
outlines are submitted before the screens, so a neighbour placed flush
covers the part of a rim over its picture, while the menu's outline and
handle bar stay on top. The handle bar moves 5 mm below the menu's
outline, which it would otherwise cover, and gets the same bands, faint
and bright.
* input: choose a headset edit item with Select Next and Previous
Edit mode steps from the laser through the screens it can move and the
menu. A chosen item is the one lit, either hand's grip takes it from
wherever the hand is, and Reset resets it, so a screen hidden behind
another can be moved. Whatever a hand holds stays lit.
A ray on the menu's handle bar takes the menu before any screen in front
of it. Within 3 cm of the bar, a ray that came from the bar keeps the
menu even over a screen, and one that came from elsewhere takes it only
where no screen is hit.
* openxr: snap a held headset screen to the screens near it
Snap to Screens, on by default, turns a held screen parallel or square
to a nearby screen and lines up its plane, edges or centre. Each pull is
exact within 3 cm or 5 degrees and fades out over as far again, so the
screen catches up with the hand rather than jumping, and snapping fades
in over the first 5 cm or 10 degrees after a grab, so taking a screen
never moves it. With several screens near, the snap that leaves the
least error wins, so a near screen's exact pull beats a far one's fading
pull whatever their order.
One stateless function inside the grab serves the main thread and the
headset's thread, so a screen is saved where it was shown. The holding
hand buzzes as a snap takes hold, once, and not again until the screen
has left that pull.
* input: turn a held headset item with the free hand's stick
A quarter turn of a held screen needed the controller turned a quarter
turn, which swung the screen round the hand on its long reach, so a
square snap could not be reached. Now the stick of the hand holding
nothing turns what the other hand holds about the vertical through its
middle, faster the further it is pushed, and snapping still catches it
parallel or square.
A carried item keeps its pitch and roll and its angle to the way the
hand points across the floor, so raising, lowering or twisting the hand
no longer tips or swivels it; the trigger lets it ride the hand rigidly.
* openxr: smooth the controller aim in edit mode
A screen held about 2 m out on the laser magnified every tremor of the
hand. In edit mode each hand's aim now goes through a One Euro filter,
which smooths a slow hand hard and a fast one little, with its rate
taken from the headset frame's own time step. The headset's thread
filters once a frame and hands the result to the main thread, which
grabs, moves and lets go from it, so a screen is saved where it was
shown. The new Edit Smoothing setting picks Off, Low, Medium (the
default) or High; outside edit mode the lasers are never smoothed.
* openxr: name the chosen edit item beside the controller
Edit mode now shows a faint tag just above the hand that last pressed
Select Next or Previous, reading Laser, Screen N or Menu, so what the
grip will take is plain without a notice. The words are drawn once into
the laser's image in the default font and the interface language, light
over a dark halo like the outlines so they read on any background, and
the tag is one more headset layer; when the runtime is short of layers
the beams are dropped before it.
On a runtime whose swapchains are too narrow for the words, the laser's
image keeps its old width and there is no tag.
* input: tilt a held headset item with the free hand's stick
In edit mode the free hand's stick already turns the item the other hand
holds. Its up and down now tilt that item too, about the item's own
left-right line through its middle, at the turn's rate and with no
limit: up tips the top back, as when pushing a monitor's top back. It
works whether the item is carried or rides the hand with the trigger
held, and snaps as the turn does.
Within about 25 degrees of left or right the stick only turns, and
within about 25 degrees of up or down it only tilts, so a push meant for
one does not drift into the other. Edit mode's starting notice lists the
turn and tilt and stays about 8 seconds.
* input: snap a turned or tilted headset item to angle steps
While the free hand's stick turns or tilts a held item in edit mode, the
item clicks into steps of the new Snap to Angles setting (15 degrees by
default; 30, 45, 90 or Off), so it is easy to set straight. The turn
counts from facing the user's head across the floor, the tilt from
upright. Within about 2 degrees of a step the item holds there and the
holding hand buzzes once; the stick's own angle keeps adding up, so
pushing on leaves the step.
Only the stick steps: carrying an item or tilting it with the trigger
never clicks, and a snap to another screen that would engage wins over a
step.
* openxr: show the back of a headset screen turned away
Runtimes draw only the front of a quad layer, so a screen or the menu
turned away from the head vanished. When the head is behind a quad's
plane, the frame now submits a plain dark panel in that quad's place: a
solid grey cell of the laser's image, at the quad's size, turned to
face the head. It takes the quad's own layer, so the layer count and
cap are unchanged. Edit mode picks either face, so an item turned away
stays grabbable, and its outline, handle and dot turn with its back.
* input: keep a screen snap's angle once the free stick lets go
A snap to another screen shows a held screen square or parallel to it,
but the item's own angle stays where the stick or the hand took it, up
to several degrees off. The next turn with the free hand's stick started
from that hidden angle, so the screen left the snap early on one side,
late on the other, and caught up with a lurch.
Now, as the free stick returns to its deadzone and as a push starts, a
snap engaged on the turn becomes the item's own angle: the carried turn,
or the orientation the trigger tilts. A snap still fading in never does,
since that would hop the screen into it. The place is left to the hand,
and carrying alone never keeps the angle, so pointing still pulls a
screen out of a snap. Letting go of the grip leaves the item where it
shows, as before.
* openxr: hinge a held headset screen to a neighbour's edge
Snap to Screens now also hinges a held screen to another along a
shared edge at any angle between them, so a DS or 3DS pair can be
opened at a slant: an edge within 5 degrees of parallel to one of the
other's, within 3 cm of its line and sharing some length with it,
turns exactly parallel and moves onto it, centres or ends level, its
angle left free. The pull fades in over the first 3 cm of shared
length, so a screen slides off the end of the other's edge without a
jump, and a diagonal neighbour never hinges. Screens that would
overlap do not hinge; the pull is whole from flat to square and fades
out folded shut.
Flat and square snap as before and report a hinge at 0 and 90. A turn
to the same screen pivots on the shared edge, so the edge stays on the
line as the screen leaves flat or square. The snap still keeps nothing
between calls; its result carries a hinge bit, and video_xr_snap_hinge
also says which edge and line, for the stick to swing about.
* input: swing a hinged headset screen about the shared edge
While a held screen is hinged to another, the free hand's stick axis
that turns nearest the shared edge, the tilt for a top or bottom edge
and the turn for a side one, now swings it about that edge, so it stays
attached; the other axis still turns or tilts it and can pull it off.
Where the swing leaves it, angle and place, it stays as the stick lets
go. Snap to Angles counts a hinged screen's steps from flat with the
other screen.
A hinge counts only while the screen's own edge is within its range,
not when other snaps alone hold it on the line, so a swing never
carries a screen round a line it is far from.
A square corner is a side hinge at 90 degrees, so the turn stick there
now swings the screen about the corner's edge. Main thread only;
nothing new reaches the frame loop, which still shows the grab through
video_xr_grab_pose.
* openxr: leave a gap between hinged headset screens
Edges that meet in edit mode, side by side, at a corner or hinged at any
angle, now click to a set gap from within 1 cm and otherwise keep the
gap the hand gives them, still hinged, up to 10 cm (letting go by 13).
Screens no longer have to touch, which made them overlap. Each grab
carries the gap, so the edit poll and the frame loop snap alike; until a
setting sets it, it is 0.
Pushed past touching, a hinged edge stops there for the first centimetre
of the push, the stop fading out by 3 cm, and ticks once per push.
A swing pivots on the held screen's own edge, so the gap keeps through
it, and a commit keeps the hand's gap unless the click holds it. After a
swing lets go past square, that stick axis and its steps keep pivoting
on the same edge until the edge leaves the hinge's reach, so a flick
past square and back does not creep.
Of two neighbours within the 5 degree turn snap, the nearer edge wins
the turn and the hinge, by one measure for every screen, so a screen
stacked behind, one off the ends of the edge or a side wall at a corner
never outranks a hinge the turn holds.
* menu: add Hinge Gap for headset screens
Hinge Gap (0, 1, 2, 3, 5 or 10 cm, default 2) sets the gap edit mode's
Snap to Screens leaves between screens that meet edge to edge, side by
side, at a corner or hinged. Edit mode takes it as it starts and gives
it to each grab; the click to it buzzes like the other snaps.
* menu: put Edit Screens under Headset Output
Stereo 3D & Screens now lists Edit Screens and its snaps right under the
switch that reveals them, then the headset's own setup, then where the
screens go, with the per-screen rows last and Visible first among them.
* openxr: track each held screen's hinge on the frame loop
The frame loop already re-snaps each held screen every frame. It now
also keeps, per hand, the hinge that snap holds whole, until the edge
leaves the hinge's 4.5 cm reach, where the frame draws it, so edit mode
can mark it. A kept hinge whose target no longer matches is dropped.
video_xr gains the hinge line's ends, along the overlap of the two real
edges and midway across the gap, an outline cut that drops one edge's
rim, and a green strip in the laser's image, before the tags.
* openxr: mark a hinged headset screen with a green line
While a held screen's hinge holds, the two edges that meet drop their
outline rim and a 6 mm green line runs along the hinge where they
overlap, midway across the gap, so it is clear when the hinge has taken
hold.
The line is drawn as the beam is, a centimetre nearer the eye than the
edges, since SteamVR does not keep the order of overlapping quad layers,
and comes after the outlines in the layer budget; hidden rims cost no
layer.
* input: buzz longer as a headset screen hinges
A hinge taking hold now gets one 150 ms buzz at 0.8 on the hand carrying
the screen, against the 20 ms snap tick, so the attach can be felt. It
fires again only after the edge leaves the hinge's reach, hinges
elsewhere, or the other hand takes the screen it hinges on, so a swing
past square and back does not repeat it. Ticks due while it plays are
skipped rather than cutting it short.
* openxr: find headset screens resting in a hinge
Two screens no hand holds rest in a hinge when their poses pass the
test that engages a carried screen's hinge, with either one carried,
and the hinge, held whole with nothing else pulling, leaves the carried
one where it is. So a hinge found at rest stays hinged when its screen
is taken. The engaged test moves into its own function so both use the
same one, and one target can pull whole whatever the grab's fade.
* openxr: keep a headset screen's hinge shown at rest
In Edit Screens, two screens no hand holds that rest in a hinge keep
the green line and their hidden rims, whether let go hinged or loaded
that way. The main thread counts them only when a resting screen
changes and sends them with the hands' record, so a release and its
resting pair arrive together; the frame loop places each line from
the screens it shows. A screen hinged at rest on two adjacent edges
still keeps one of those rims.
The hands' record now outgrows the quads, so the seqlock's copy is
sized to the larger of the two.
* openxr: pick up a hinged headset screen still hinged
Taking up a screen that rests in a hinge starts the carry on that hinge,
whole from the first poll, with no buzz and no jump, until it leaves the
hinge's reach, another hinge takes hold, or the other hand takes the
screen it hinges on. Until the grab's own snapping has faded in, the
kept hinge's target still pulls whole, so a straight pull out does not
hop.
The grab carries the hinge so both threads hold it the same; of several,
the one the snap ranks best carries over, and only where holding it
moves nothing. While the other screens fade in, the hinge's screen also
leads the turn, so a nearer one still fading does not take it.
* openxr: end a headset screen's hinge where a push passes the stop
Pushing a hinged screen into its neighbour holds the edge at touching
for 3 cm of push, fading out by 5.5 cm. Where it stops holding, the
hinge ends: the line goes, the rims come back, and the lid and buzz are
cleared, so screens pushed into each other show no hinge that a let-go
there would not rest in. Pulled back, the hinge takes hold again 5 mm
short of where it ended, with the long buzz, so a tremor there does not
flicker it.
The grab carries that through flag, and the frame loop keeps its line
until the main thread sets it, so the two threads never disagree for
more than a poll.
At Hinge Gap 1 a pair let go touching rests hinged and is taken up
hinged, as at every other gap. A screen swung into a square corner gets
the long buzz on arrival, and the other hand taking the target does not
hop the carried screen by the push's depth.
* input: let go of a headset screen where it was before the grip opened
Opening the grip twitches the hand, and the poll that sees the grip
open would move the screen by that twitched aim. Half a degree of pitch
moves a screen 1.8 m out by 1.6 cm, enough to knock it off its hinge's
gap or out of level, so it would not rest hinged.
Each hand keeps a short trail of where, turned and as large as its
screen showed, entries at least 10 ms apart, and a grip let go puts the
screen as it showed 80 ms before (INPUT_OPENXR_EDIT_REWIND_US), or as it
was taken if held for less. Edit mode's end and a recenter let go where
the screen shows.
* openxr: find where a let-go headset screen settles in its hinge
A screen let go while its hinge shows should land exactly in that hinge,
so the hinge still shows at rest and a pick-up is hinged.
video_xr_hinge_settle holds the hinge whole from the hand's pose, then
moves the edge to the set gap from within 3 cm of it (touching stays
where the stop holds it) and levels it along the edge within the level
pull's reach. Its result must pass the rest test, and it turns no
further than a grab's hinge engages, which shares that test.
A pair resting inside a settle band but off its point is no longer
listed at rest, since a pick-up and let-go would move it. A touching
rest is taken 5 mm deeper into the push stop, at every Hinge Gap, so a
hand that drifts less than that apart before letting go settles touching
rather than at the gap.
* openxr: settle a let-go headset screen into its hinge
A screen let go while its hinge shows now goes exactly into that hinge:
to the set gap from within 3 cm of it, touching from the stop's hold,
and level along the edge from within the level pull's reach. It then
rests there, listed and green, in the same publish, and is saved
there. Elsewhere it stays where it shows, as before.
Each trail entry also records where the hand put the screen with no
snap and whether a let-go there settles. At the let-go the rewound
entry is settled again against the screens as they are now, at that
entry's size, and put in one go.
* openxr: let a hinged headset screen rest at any fold
Snap to Screens' turn to flat, and its turn to square at a corner, is
whole within 5 degrees and fades out by 10, so a hinged screen held
there shows at twice its angle off less 10. No fold shown in that fade
read as a rest: the pair was not listed, a let-go there did not settle
into its hinge, and taking it up would have moved it.
Where a hand takes a rest, it now holds the screen at the angle the
turn's fade shows it from, the inverse of the turn's pull, so every
fold rests, is listed and is taken up unmoved. Held snapping is
unchanged.
* input: keep a headset screen's hinge through a shaking hand
Edit Screens' hinge buzz, lid and push stop followed the snap's own
engage and reach on every poll. A hand shaking half a degree (1.6 cm at
the screen) about a hinge's edge flipped them many times a second, each
flip a long buzz.
Each hold now keeps one mark: the hinge a let-go would settle into. It
takes hold at once, with the long buzz, and lets go only once polls have
seen the hand out of where a let-go would settle for 150 ms, each poll
counting at most 50 ms, so a stalled main thread or a spell untracked
never lets it go alone. The other hand taking its target, hinged
elsewhere or not, the target leaving and the let-go end it at once. The
lid waits for the mark to go, the push stop follows it, and a let-go
while it shows settles into it from where the hand last would have, edit
mode's end and a recenter included.
* openxr: draw a held screen's hinge as the main thread marks it
The grab carries the main thread's hinge mark, and the frame loop draws
a held screen's green line and cuts its rims from that mark alone,
through video_xr_grab_marked, while it snaps the screen every frame. A
frame loop taking the hinge by itself between polls would flicker the
line for a hand shaking at the edge of the hinge's range while the mark
held. A mark whose target the frame does not show cuts no rim.
The other hand taking the mark's target ends it, and the lid, on any
poll, tracked or not, so a lost hand's line does not come back when that
hand lets go.
video_xr_grab_near_hinge loses its past argument, which only the frame
loop's keep used.
* openxr: keep edit mode's hinge layers in place
SteamVR does not keep submission order for overlapping quad layers, so
a layer added, dropped or moved to another index can pop the edit
outlines in front of the screens, as a wobble at a hinge would do to
its line's layer.
The hinge lines are fixed slots: one for each hand holding a
screen, hand 0 first, then one for each pair at rest. A slot with no
line to show is drawn in place from its screen's middle, at rows of
the laser image that are empty either way up. The layer count and
every index change only at a grab, a let-go or a re-count of the pairs
at rest; a mark only changes crops and which rows a slot shows.
* input: date a let-go's trail by frame and stop it at a stall
Each trail entry is dated by the frame its aims are from, one a frame
and per hand, and the release poll records a new frame's aims first.
The rewind stops at a stall, entries more than 2.5 frames apart, a
frame counted as at least 20 ms so a fast headset's close entries never
read as one. Under load a stall of 0.1 to 0.3 s can come just before
the release; the entry from 80 ms back would then hold the pose from
before it and step the let-go back across all the stall's motion.
With Edit Smoothing on, which damps the twitch, a let-go that does not
settle into a hinge stays where and as large as it shows: a rewind there
would step back by what the smoothed aim still lags. The hinge decision
and the settle use the trail either way.
* openxr: cut edit outlines from cells instead of turning them
Cutting a top or bottom rim from the one outline image turns both
outline quads a quarter turn and swaps their width and height, and on
SteamVR that can blink every edit outline as the hinge takes hold or
lets go; side cuts, which never turn, do not.
The laser's image carries twelve pre-cut outline cells after the
tags, faint and bright, for a cut top, bottom, left, right, top and
bottom, or left and right, so a cut only picks a cell and the quads keep
their pose and size. The image is then 5582 texels wide. A runtime whose
swapchains are narrower, or that does not say how wide they can be,
keeps the turned and cropped outline at 2498 or 1208 wide. One log line
at the laser's creation gives the width chosen and the runtime's limit.
GL writes the top and bottom cells swapped for its bottom-up rows, and
Android's copy of the image is allocated rather than static.
The edit-hinge-3ds-narrow e2e case runs the turned cut at
RA_XR_LAYER_MAX_WIDTH=4096, on Vulkan and the GL EGL lanes.
* video: draw headset screens straight into the runtime's images
Each headset screen was drawn into an image of the driver's own and
copied from it into the runtime's on every new frame, Vulkan with a
render pass and GL3 with a blit. Screens are now drawn into the
runtime's image itself, as before per-screen placement, with no image
of the driver's and no copy.
A screen drawn again with no new frame - resized, made again or moved
while paused - replays the last frame's final pass from the chain:
build_viewport_pass_replay draws it at the new size from what it read
then, the passes' outputs, their feedback and the history that frame
saw, with that frame's count. Nothing else runs, feedback does not
swap and history stays, so a feedback or history preset holds still
while paused. A chain made to replay keeps one more history image, so
the frame's end writes where the last frame did not read; the window's
chains keep what they did. This also has movie frames reach the
headset, which the copy held at the first one, and draws a screen
resized while paused at its new size rather than stretching the old
image.
* openxr: leave what users who never edit see as it was
Screens seen from behind turn to their grey backs only while Edit
Screens is on, where screens are turned around; outside it nothing
changes and no head is located for them. The grabs carry whether
editing is on.
The menu opens at its own spot, straight ahead of the anchor, as it
did. Menu in Front of Screen 1, off by default, keeps it just in front
of screen 1 and moving with it.
The frame loop copies the main thread's grabs only once their stamp
moves, so outside edit mode no frame copies them. A new session clears
the frame loop's own copy and passes over what was published before
it, rather than publishing itself: the main thread stays the grabs'
one writer.
On Touch, the left menu button that opens the menu from edit mode is
spent until let go: held on as edit mode ends, it is no Start and does
not toggle the menu again after a second.
---------
Co-authored-by: libretroadmin <[email protected]>
* scan: read an archive's directory once for all its passes
The scan fetched an archive's first member's CRC, size and extension
through a helper that stops at the first entry and reports how many
the header counts, and its readers did the same. One walk now reads
the whole directory, in the readers as in the scan: every member's
stored CRC, size and name, kept for each of the archive's passes.
Its transfer state is on the heap, out of the readers' frames. The
walk replaces the helper in the archive passes; the helper stays for
its other users.
The walk decodes the first member for a CRC where the archive stores
none, a zstd stream or a RAR 5 member hashed with BLAKE2, so it is
looked up as before. Later members are not decoded and keep a CRC of
0. A .bz2 is decoded to be listed at all.
Folders are not members: an archive of a folder and one file has one
member, that file, and is looked up by its CRC first.
* zip: report a TorrentZip end comment
A zip whose end-of-central-directory comment is TORRENTZIPPED- and
eight hex digits was written by TorrentZip, as arcade sets are. The
zip reader already reads that record to find the directory, so it now
says so in the transfer state, for a caller that wants the hint. The
CRC in the comment is not checked: nothing depends on it.
* database_info, libretro-db: archive record sizes beside the crc index
The crc index build walks each database once, off the main thread.
In the same walk it now also collects the size of every record whose
rom_name ends in an archive extension (zip, 7z, apk, rar, zst), kept
in one sorted array keyed by size and type, so a caller can ask
whether a database may hold an archive of a given type and size. A
record of that type with no size, or an index built without the
sizes, answers yes. The sizes count against the index budget.
The stepped database scan reports one string field for this:
libretrodb_scan_begin() and its callback gain a name field. A
record's rom_name can also be extracted on request.
* scan: look every archive member up from the directory
An archive whose first member missed was read whole and, when that
missed too, expanded: each member went through the databases again,
by the stored CRC the directory already held. An archive of several
members was read whole before any lookup, even where no database
could hold it, so disc zips and console sets were read whole again.
One member: its stored CRC is looked up first, as before. A hit in a
database claimed only for the member's type is held, and every
member's CRC is looked up next. On a miss the archive is read whole
only when a database whose cores open its type may hold it by its
exact size; its own CRC is then looked up there, and where that is
not asked or misses, every member's.
Several members: the same size question comes first. Each database
whose cores open the type is then asked for the first member's CRC,
and for the archive's own CRC only when the size let it be read
whole, so an arcade record still wins and no database claimed only
for a member's type is walked before it. Where neither matches,
every member's stored CRC is looked up.
A database may hold the archive when it has a record whose rom_name
is of the archive's type and size, one of that type with no size, or
no sizes at all. Every member is looked up with the gates and size
checks the expansion uses. A hit names the archive itself only when
it is the one record hit, that record is named for this archive and
its database's cores open the type; other hits are written as the
expansion wrote them, archive#member. The expansion stays for scans
that keep unmatched files, for archives whose directory cannot be
read, and where nothing hit and a member stores no CRC, which only
the expansion unpacks for one.
Readers read an archive whole ahead only after a whole match, twice
as many after each such match up to the cap; any other verdict takes
the rest back. A TorrentZip set of several members is still read
whole by a reader with no grant while some database opens zip. An
archive over the readers' cutoff is handed back before a grant is
spent on it.
This narrows "several members are read whole first" to "read whole
when a database may hold it by its size". A record whose crc is an
archive's own CRC but whose rom_name is not of the archive's type, or
whose size is not the file's, is no longer matched; each of FBNeo's
596 sets carries its zip's size. A zip of a folder and one ROM takes
the one-member order.
* database_scan_test: rar archives through every pass
A RAR 4 archive of stored entries, written by hand as the zips are,
takes the passes zip and 7z take: a first member's hit where a core
opens rar names the archive, a later member's elsewhere is held as
archive#member, and the archive is read whole only while a database
whose cores open rar holds a rar record of its size.
* database_scan_test: define the fixture byte writers once
* archive_file: clear the TorrentZip flag for every archive
The flag was cleared only by the zip reader, so a transfer reused for
a 7z, a rar or an archive with no backend kept what the last zip said.
It is now cleared before any backend opens the archive.
* scan: readers read TorrentZip sets whole while the scan asks for that
Readers read every TorrentZip set of several members whole while some
database opens zip, but where no database may hold a set by its size
the scan drops that CRC, so a folder of sets no database holds was
read whole for nothing. Each TorrentZip set the scan rules on now
says whether its own CRC was asked, and the readers read the next ones
whole only while it was.
---------
Co-authored-by: libretroadmin <[email protected]>
* vfs: return an SMB read-ahead connection with its pool's window
A read-ahead stream turns off the client read-ahead on its
prefetcher's pooled connection while the prefetcher runs. It did so
before opening the file and left it off when the open or the
prefetcher failed, so the slot went back to its pool with no window.
On close it set the window from the current settings, but a pool
keeps the settings it was built with and is used again when the
settings return to them, so after a read-ahead change a connection
could go back to its pool with the wrong window. The window now goes
off only after a successful open and is restored from the pool's key
whenever it was turned off.
* smb_retro: switch the SMB settings under vfs_threads_test's workers
A new phase has a thread switch the settings between two connection
keys while the workers open, read and list, so slots are taken from
and given back to two pools as a settings edit would. The two-shares
phase covers a second share; this covers the key changing under them.
* smb_retro: test that a slot goes back with its pool's window
A stream with a prefetcher open while read-ahead is turned off now
checks, once it closes, that every idle pooled connection still has
its pool's window. This fails without the read-ahead fix before it.
smb_pool_windows_ok() makes the check, with a getter for the window
the built-in client holds.
The switching phase is driven by the workers, each moving the key on
after a round, instead of a thread spinning on smb_init_cfg(). It
switches across five keys, more than there are pools, half of them
with read-ahead off, so idle pools are evicted beside busy ones, and
it checks the windows once the workers are done.
---------
Co-authored-by: libretroadmin <[email protected]>
With the threaded pipeline a pause or resume no longer takes the state
lock or writes to the device from the main thread. It publishes its
discard as before and a request word, and the consumer, the only thread
that writes to the device, carries the requests out before its next
pass, oldest first: the period-matched tail and its mute for a pause,
the mute reset and top-up for a resume. A consumer parked for data
returns to do it, and a stop settles what is left, writing no tail into
the stopped device. Without the threaded pipeline the tail is still
written where the call is made, under the lock.
pipeline_wakeups' pause boundary checks the pause writes nothing from
the main thread, and waits for the consumer's tail before counting stale
audio.
Orbis libc exports localtime_s(), in the C11 Annex K order, where other
libcs have localtime_r(), so threaded PS4 builds take that rather than
the shared static localtime() fills. It writes only the nine int fields
of a struct tm, so the copy it fills starts zeroed.
Both are published with pointer atomics, and a holder of the state lock
- a pass - counts itself in and out of pass_seq. The main thread swaps
the DSP filter in or out without waiting for a pass: what it takes out
is freed at once when no pass runs, and otherwise once pass_seq has
moved on, at the next swap or the next frame. Readers outside a pass
see what the filter allows through dsp_caps. The record tap is
installed with a store after its begin and comes out with a
compare-and-swap; removal ends it once no pass that could have seen it
remains, and both still answer "not yet" while a pass runs. A backend
without pointer atomics changes them under the lock.
mixer_lock builds with the DSP filter, swaps and frees it while a pass
holds the state lock, and checks nothing is freed before the pass ends.
The main and task threads hold mixer_streams[] with a compare-and-swap
owner word and park on an eventcount only behind one another; the mix
never takes it. A play's stop callback, on the mixing thread, only
reports the finish - its sound's tag into a per-slot word - and the
table's next holder, or the main thread once a frame, settles it:
retiring the sound, or starting the next stream of a sequential run.
That callback used to start the next stream itself, inside the mix and
under the state lock the mix holds, which deadlocks against itself on a
non-recursive mutex. A stop recognises its play by the voice's serial,
so a voice the mix has finished and handed on is left alone. The
volatile backend, whose compare-and-swap is not atomic, holds the table
with the state lock.
A WAV is loaded straight from the caller's bytes: its load copies the
samples it keeps, so the copy made before it is gone.
audio_mixer gains a per-sound tag for the callback to read. mixer_lock
finishes a sequential play from a mix holding the state lock, plays,
stops, adds and removes from two threads while a third finishes voices,
and checks a WAV is built from the caller's bytes.
The Orbis SDK declares localtime_r, but its libc import library does not
export it. Use the existing localtime fallback on Orbis so threaded PS4
builds link again.
nvda_controller_win32.c carries the ARM64 procedure format strings of
NVDA's own controller client, alongside the x86 and x64 ones, and the
stub descriptor its current (2024.4) build uses; the strings for all
three are that client's byte for byte.
samples/frontend/win32_nvda takes NVDA_DLL64 and NVDA_DLL32, NVDA's own
x64 and x86 clients from its controllerClient zip, to make the same
calls through them against the stand-in first.
A blocking driver's calls reach the worker on a many-producer stack and
its results come back on another; the worker parks on an eventcount
when the sync is still open and the queue is empty, and leaves once it
ends. A call queued as the worker leaves either starts a new worker or
is taken by the leaving one. cloud_sync_deinit() waits for the worker
on an eventcount, still bounded by its timeout.
Without pointer atomics the calls run directly, as without threads.
The async test queues a sync from the worker's leaving point.
frontend/drivers/nvda_controller_win32.c talks to NVDA's controller
interface over local RPC itself, as NVDA's controller client does:
bound to ncalrpc:[NvdaCtlr.<session>.<desktop>], its four procedures
called through NdrClientCall2 with MIDL's format strings - the x64 ones
NVDA's client carries, byte for byte, and the same procedures for the
x86 stack - so 32-bit builds reach NVDA too. rpcrt4 and the session and
desktop calls are resolved at run time; where any is missing,
PowerShell speaks as it does without NVDA.
samples/frontend/win32_nvda runs it, x64 and x86, against a stand-in
NVDA in the test: an RPC server for the interface as widl compiles it.
Every procedure arrives in order with its string as sent, non-ASCII,
empty and long ones among them; with no NVDA listening or after deinit
nothing arrives and the calls fail.
- rtime_localtime() uses localtime_r(), or the MS CRT's per-thread
localtime(); no lock, no spin, nothing for rtime_init() to make.
- faulthandler: install and remove claim one CAS state word; one that
finds it taken fails instead of spinning or taking a lock.
- core_info: the scanner's core_info_database_claim() reads a published
list under a reader count. A list taken down goes on a retired stack,
freed by whoever finds the count at zero: the main thread or the last
reader out. The mutex stays for backends without pointer atomics.
- cheevos badge queue and mutter's no-pointer-atomics fallback: their
locks guarded nothing, and go.
core_info_claim_test races claims against list rebuilds; the
faulthandler test races installs.
The widget and autosave harnesses become samples of their own -
samples/gfx/widget_snapshot_ownership, samples/gfx/widget_task_lifetime
and samples/runloop/autosave_snapshot - each built and run by make
check like the rest, the task lifetime test with the widget defines its
stubs build against. The autoconf suite, its .ratst scripts and test
input configs move to samples/input/test_input; the workflows that ran
them follow. The SPIRV-Cross MSL binding testbench, which nothing built
or ran, goes.
lakka.h and lakka-switch.h join the rest of Lakka's OS layer in
os/lakka; switch_performance_profiles.h, which only HAVE_LIBNX builds
read, starts os/libnx. Includes and the iOS Xcode projects follow.
Pause mock read completions and wait for both live pads to queue a read before deleting the HID client. Otherwise the deliberately broken IOS can have no reads to lose and spuriously pass the negative test.
Free discarded mock requests so the negative test detects missing adapter teardown rather than leaking its own request records.
CoreMIDI's notification of a removed object moves one atomic count
on, and touches no instance; each instance's read and write look for
its endpoint again only when the count has moved, on the thread that
calls them, so a read or write lists no devices while none has gone.
The instance registry and its lock go, and with them the cap of eight
instances. The receive callback's shutdown flag is an atomic.
samples/midi/coremidi_driver builds the driver against a stand-in for
CoreMIDI: reads and writes list no devices until one is removed, a
removed input drops the notes queued from it, every instance finds a
removal, and removals notified from another thread run beside reads.
OpenXR has no way to wait for an event. While the session is down, or
xrWaitFrame cannot wait, the XR thread parks on an eventcount, and the
drawing thread makes the non-blocking xrPollEvent for it once a frame:
the events reach it decoded through an SPSC queue, and the first one
hands polling back. One word says whose xrPollEvent it is, so the two
never poll at once, and a stop wakes a parked thread at once, or after
the drawing thread's poll in progress.
openxr_core_test parks the thread, checks it polls nothing, starts the
session from the drawing thread's poll, and stops it parked and beside
a poll; a stop that never wakes it fails the run after 60 s.
The Vita driver copies the core's frame into its texture a row at a
time, the frame's width of each row, or in one copy when the core's
rows are the texture's; textures are allocated at their row stride,
the width rounded to 8. Both drivers copy with the C library's memcpy,
NEON-tuned on their targets, and memory/neon/memcpy-neon.S goes.
samples/gfx/gxm_frame_upload runs the driver's frame upload against
the psp2 stubs with guarded GPU memory: core rows longer, as long and
shorter than the texture's, widths not of 8, rows copied once each,
nothing written past the texture or read past the frame.
A swap publishes the new track space as a record - space, origin,
number and carry range - with one pointer exchange and parks the old
one on a lock-free retired list; the frame loop's thread takes the
current record at each frame and destroys the retired spaces then, as
their only other user. A projection drawn before a recenter's remake
shows in the new space by the record's carry range, so nothing
rewrites it and the projection keeps one writer.
With that, every runtime call of the desktop GL host is on its XR
thread, so the host has no lock. Vulkan keeps its queue lock, which
the core's lock_queue shares.
openxr_core_test swaps from a core thread while the frame loop
submits: every frame shows the carried projection in a live space,
and each space is destroyed once.
The Play Feature Delivery glue is Android's alone - JNI calls into
RetroActivityCommon - and the last platform directory of its kind at
the tree's root. It joins the other OS layers under os/; includes, the
Makefile, griffin, the compile matrix and the two samples that build
it follow.
The polling thread and the frontend shared one adapter list under an
OSFastMutex, which the frontend took on every poll, and the HID attach
callbacks queued events by swapping the list head out and back in. That
swap pair is not atomic as a whole: two callbacks at once (the HID
stack calls them from more than one thread) can each take the list and
put theirs back over the other's, losing an attach or a detach. A lost
detach leaves the pad registered for good.
Each thread now keeps its own list and hands adapters across through
the lock-free stacks of mpsc_stack.h:
attach callbacks -> hid_events -> polling thread
polling thread -> hid_new -> frontend: adapters to set up
frontend -> hid_retired -> polling thread: to free
An adapter's state is a retro_atomic_int_t, and each value is moved on
by one thread only, so no two ever write it from the same state. The
read completion runs the packet handler before it says READY, since
READY is when the polling thread may start winding the adapter down;
it used to say READY first. The frontend deregisters and deinitialises
a pad as it retires the adapter, the polling thread frees the adapter,
and at shutdown the polling thread tears down what is left, as before.
No lock remains, and the frontend's poll no longer takes one.
samples/input/wiiu_hid_threads runs the driver against stand-ins for
Cafe OS on pthreads: two threads attach and detach devices while the
frontend polls and an "IOS" thread completes the reads, and every pad
must be registered, fed, deregistered and deinitialised in order, on
the right threads, with nothing left at shutdown. Its negative build
loses the reads in flight at shutdown and must fail. Against the old
driver (32-bit PowerPC under qemu) it fails: a few hundred pads of
detached devices are never torn down.
Expose reusable net_rtmp and rflv_mux interfaces without frontend dependencies, including bounded TCP socket service. Use the SDK MIT license for these newly authored modules.
Move the portable tests, register optional RTMP/FLV build gates and exercise standalone SDK and frontend builds. Recording-specific capture and encoder integration stays in record/.
A voice's owner word marks a caller parked on it, so letting a voice
go notifies only when someone waits: the mix takes and releases each
voice with one compare-and-swap and one exchange, and never reaches
the eventcount - nor, where its bookkeeping is locked, its lock. With
that the PS2's interrupt-masked atomics take the owner word too; only
the volatile backend, whose compare-and-swap is not atomic, keeps a
lock per voice.
audio_mixer_voice_threads_test holds one voice from two callers while
a mix takes it between them, and fails rather than hangs when a
caller is never woken.
The Wii U's Espresso needs a cache flush inside a lwarx/stwcx. pair for
the reservation to hold across its cores, which no compiler emits and
Cafe OS's own atomics carry. retro_atomic.h gains a Wii U backend,
chosen ahead of the C11, C++11 and GCC builtins there: compare-and-swap,
exchange and fetch-add/sub/or/and call coreinit's OS*Atomic between the
fences their ordering asks for, while loads, stores and fences stay
inline. Across the Wii U build that takes every lwarx/stwcx. out of
RetroArch's own code. The newlib lock hooks, the exception handler's
first-fault flag and the SwapAtomic helpers the HID driver uses go
through it too, in place of the builtins they called directly.
libretro-common/samples/atomic/retro_atomic_wiiu_backend checks the
backend against stand-in coreinit exports: one call per
read-modify-write, the right one, with the previous value returned,
none for loads, stores and fences, and no update lost under threads.
The LiveArea bubble launcher lives under pkg/vita/launcher. The two
host tools its build.bat runs, vita-mksfoex and vita-pack-vpk, are C
sources there in place of prebuilt Windows executables, built by the
builder's Makefile with the host compiler. vita-mksfoex writes the
param.sfo vitasdk's 2016 tool writes, byte for byte, and refuses a
title too long for STITLE instead of writing a broken one;
vita-pack-vpk deflates with libretro-common, storing what deflate
cannot shrink, and fails on two members under one name.
samples/pkg/vita_launcher_tools checks both against the reference
param.sfo outputs and reads each VPK back with libretro-common's zip
reader.
The UWP entry point and helpers live under os/uwp. The UWP projects add
os to their include paths, so <uwp/...> as libretro-common names it
keeps resolving; RetroArch's own sources include them by their path.
The libogc 2 linker scripts go to os/gekko/libogc, the PSP kernel
functions PRX to os/psp1 and the Vita heap's sbrk to os/vita, beside
each platform's other system code.
Add bounded FLV muxing and worker-owned RTMP framing, handshake, control and AMF publish negotiation. Retain borrowed output across short writes and cancellation without frame-path I/O or waits.
Exercise standard and legacy publish sessions against an independent receiver, including name-only onFCPublish callbacks. Socket transport and native encoder delivery remain separate integration work.
glslang's process state is kept for the life of the process once
started: a shader task can still be compiling when its video driver
goes, or when the task queue leaves it running at exit, so it is never
finished under one. With that, a detached compile - the deferred
load's precompile tasks - compiles with glslang as the chain's thread
does, falling back from rslc to it the same way, and hands back the
cache key of the compiler that made the stages.
The detached entry no longer takes the HLSL/MSL flag it ignored.
Track the PBO extent on GLES3 so viewport reads do not recreate the ring and discard every pending frame. Exercise the production initializer for ring retention and resizing on Linux and Windows.
The loader stub the libogc builds embed to hand the console to the next
program lives under os/gekko/rvl with the rest of the Wii's own code;
its binary object's symbols are named after the new path.
* vfs: close a private SMB connection when an open, stat or opendir fails
When every pool slot is held, smb_take() connects privately for the
caller. If the open, stat or opendir then fails, smb_heal() returns
NULL when the connection is alive (a missing path, the usual case) or
cannot be replaced, and the caller stored that NULL over its context
and gave back nothing, so the connection and its socket were never
freed.
Keep the caller's context when smb_heal() returns NULL, so smb_give()
closes it. A pooled slot is given back as before.
vfs_test holds both pool slots with open files and then stats, opens
and lists a missing path; LSan reported the three connections before.
* vfs: release an SMB stream's connection when close follows a failed mend
When a stream's connection cannot be mended (the server is gone and
the reconnect is refused), or its file cannot be opened again, the
mend clears the stream's handle. Close returned early on the missing
handle, so a private stream leaked its connection, a pooled stream
kept its slot busy for the pool's life (smb_shutdown then left the
pool unfreed), and the reopen record leaked either way.
Close now checks for the connection rather than the handle and closes
the handle only when there is one. After a failed mend a private
connection is freed without a goodbye, since the server may be gone.
smb_idle_test gains a mode in which the server is stopped for good
during the pause, and local_samba_test.sh runs it, so the sanitizer
lane's LSan reports these leaks.
* cloud sync: make and drop SMB contexts under the VFS's list lock
libsmb2 keeps every context on one process-wide list with no lock of
its own, so the SMB VFS makes and drops its contexts under a lock
(4e6885d5a3). The SMB cloud sync driver called smb2_init_context() and
smb2_destroy_context() directly. A sync that began or ended while the
VFS connected or disconnected on another thread, as during a scan,
could lose a list entry, after which a live connection reads as dead,
or break the list.
The VFS exports its two wrappers and cloud sync uses them. The
built-in client has no such list; there they call straight through.
cloud_sync_test runs cloud sync's connect and disconnect beside private
VFS connections; the libsmb2 TSan lane reported the race before.
* scan: readers take smb:// paths when an SMB client is built
The scan's parallel readers were kept off every "://" path. smb://
paths go through the SMB VFS, whose connection pool serves several
streams at once, and the commits before this one fix the connection
leaks and the libsmb2 list race that readers would otherwise hit. So
readers now take smb:// paths in builds with an SMB client, built-in
or libsmb2; other schemes stay with the scan.
database_scan_test adds smb://, nfs:// and ftp:// archives to a
threaded scan's list and counts the readers' reads of each. Its TSan
build gets a stub SMB backend, so CI runs both halves.
* vfs: keep an SMB pool per share, never freed under its users
Pools live in a static table, one per connection key. Each pool's state
and user count share one atomic word: a lookup or a held slot is a use,
only an idle pool is evicted, and a retired pool closes its connections
when its last user leaves. Readers on two shares each keep their pool's
connections, and no thread touches a freed pool.
A failed reconnect after a dead transport frees the connection without
the round trips of a logoff; a pooled slot reconnects on its next take.
vfs_threads_test runs threads alternating between two shares of one
directory with no share configured.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The salamander launcher has no C++ left in it, and the C++ in libctru,
its allocators, needs nothing from libstdc++, so it links with gcc;
the C++ build rules it no longer uses, and the shader rules it never
did, are gone.
The tree's 3DS layer (the custom crt0 and link spec, system and memory
setup, svchax, the old GPU command layer and the 3dsx/CIA launchers)
lives under os/ctr, beside os/gekko and os/wiiu.
The linear heap allocator is C: the same first-fit pool, so buffers
land where they did, with live allocations found by address in an
open-addressing table instead of a per-allocation tree node, and list
nodes taken from cache-line-aligned slabs and reused. On the same
trace it runs about half the instructions and practically no
malloc/free calls. With the whole heap allocated, a further allocation
now fails instead of starting the heap over and handing out its first
address again, and the free-space queries no longer fault.
samples/frontend/ctr_linear checks placement, alignment, coalescing,
the full-heap case and pages handed back from the heap's end.
Use the format-ID specifier required by AudioToolbox when selecting AAC sample rates. Validate requested audio settings and retry with the encoder default bitrate when the preset is rejected.
Report refused or failed audio/video appends instead of returning success. Cover the property ABI, buffer release and writer states with portable regressions, and add a native AudioToolbox contract check.
glslang's process state is started in video_driver_init_internal(),
before the video thread exists, and finished in
video_driver_free_internal(), after it is gone; a compile only reads
one state word. Compiles share glslang's built-in symbol tables
instead of rebuilding them each time, and no global mutex or guarded
static stands in the compile path. A caller with no video driver
around it, such as a tool, starts the process on its first compile.
Against glslang 15 a compile takes 1.1 ms instead of 73 ms, with
byte-identical SPIR-V.
The webOS ipk packaging - the icon, its build script, the aarch64
repack script and the notes - lives under pkg/webos with the other
platforms' packaging. Makefile.webos stages and packages the ipk there,
the webOS workflow collects its artifacts from there, and the Homebrew
Channel manifest takes its icon from pkg/webos on master.
elf2rpl, the host tool that turns the linked Wii U ELF into an RPL, is
now a single C89 file, os/wiiu/wut/elf2rpl/main.c, in place of the C++
sources, their own ELF header, CRC-32 and endian helpers, and the
Visual Studio project files.
Big-endian fields go through retro_endianness.h; the byte swap the old
be_val.h picked by PLATFORM_WINDOWS/PLATFORM_APPLE/PLATFORM_LINUX is
gone with it, along with those defines - it did not build on any other
host and swapped wrongly on a big-endian one. Section CRCs and import
signatures come from libretro-common's encoding_crc32 instead of the
bundled bytewise table.
The input is read once and its sections used where they lie; symbol
and import lookups by address are hash tables; the output is laid out
in one buffer and written with one call. On a 7 MB core ELF with 41k
relocations the RPL comes out byte for byte as before, and so does the
log at every -v level, in about a quarter of the time (15 ms against
52 ms; 47 ms against 104 ms at -v -v -v -v).
Malformed input now ends in an error instead of a crash: header,
section, symbol and relocation offsets are checked against the file,
a stub with no symbol behind it is reported, and an unknown option is
refused rather than taken for the input file.
The tool is built with the host C compiler (HOST_CC, gcc by default),
which an exported devkitPPC CC no longer reaches.
samples/frontend/wiiu_elf2rpl builds the C tool and adds checks that
every section's SHT_RPL_CRCS entry is the zlib CRC-32 of its contents,
and that truncated ELFs and an unknown option fail cleanly; it no
longer needs the leak and alignment sanitizer exemptions the C++ tool
did.
devkitPPC's newer libc and libstdc++ reach the OS through weak hooks
(__syscall_exit, the __gthread_* ones) that the RetroArch runtime does
not define. A -pie link leaves each reference with its final value
written - zero, or a branch to itself behind a check that the hook
exists - and a relocation against the still-undefined symbol, which
elf2rpl handed the loader against $UNDEF: a symbol it cannot resolve
and a 24-bit branch to address zero, out of reach from the code area.
Every RPX linked this way fails to load with error 160-2203 before any
of it runs. Such a value does not depend on where the module loads, so
elf2rpl drops the relocation. Makefile.wiiu rebuilds elf2rpl when its
source changes.
samples/frontend/wiiu_elf2rpl writes a linked ELF in that shape, runs
elf2rpl on it and checks the RPL carries no relocation against an
undefined symbol while those against resolved targets survive.
The UWP solution lives in pkg/msvc beside the desktop ones, split per
Visual Studio release: RetroArch-msvc2022-UWP (v143) and
RetroArch-msvc2026-UWP (v145). Both projects share pkg/msvc/msvc-uwp
(manifest, assets, ANGLE/MESA libraries, core placeholders) and the
CppWinRT package in pkg/msvc/packages, with intermediates and generated
files kept per project. The MSVC workflow builds 2022-UWP on the
windows-2022 runner and 2026-UWP on windows-2025.
Closes#19893
The decode tables are built in one pass: each symbol's packed entry
is made as its code is reversed (through a byte table, not a bit
loop) and splatted beside the fast entry, instead of two 2048-entry
passes after the build; and nothing is cleared beforehand, since a
complete code set writes every index it can reach (an incomplete one,
which zlib accepts for a distance table, still clears). The fast
loop's refill is branchless: eight bytes shifted in above the count
on every iteration, the whole ones counted, the rest left above the
count where the next refill puts the same bits again; the loop's
limits are read once rather than reloaded after every output store.
A stored block takes the whole bytes the bit buffer holds first and
then copies the rest straight from the input, instead of going byte
by byte for the whole block whenever the buffer was not empty on
arrival, which after a Huffman block it never is.
On one machine, best of many runs: a 4 MiB binary at 43% is decoded
at 375 MB/s (338 before; zlib-ng 440, zlib 1.3.1 190), a literal-
heavy table at 272 MB/s (226 before; zlib-ng 304), and stored blocks
at 1.7 GB/s (1.1 before; zlib-ng 1.7). Checked byte for byte against
zlib's output over 1500 random streams of every level, wrapper and
window with the input and output handed over in random slices, under
the address and undefined-behaviour sanitizers.
The gzip header was read in three pieces with a suspend possible
between them, and a resume started over at the first: input that
ended inside the ten fixed bytes (a slice boundary there) failed the
stream with an ID check on the wrong bytes. The identification and
flags are read in one step and the six bytes after them counted down
in the next, so a resume carries on where the input ended.
The compressor always matched up to 32 KiB back whatever window_bits
asked for. zlib's convention gives the window's size in the same
argument (|bits| for raw and zlib, bits - 16 for gzip), and callers
rely on it: qcow2 compressed clusters are written with -12 so that a
4 KiB window decodes them. The match distance is now capped at that
window, and the zlib header's CINFO says which window was used.
Without UPower, the Game Mode helper re-read /sys/class/power_supply
every 5 seconds to notice a charger being plugged in or pulled. It now
listens on the kernel's uevent broadcast (NETLINK_KOBJECT_UEVENT, group
1, the one udev itself reads; unprivileged processes may receive it)
and re-reads sysfs only when a SUBSYSTEM=power_supply event arrives, or
when the socket reports dropped events (ENOBUFS). The helper now never
wakes unless something actually changed.
UPower is also asked only once per Game Mode session: when it is
absent, later checks go straight to sysfs instead of making a D-Bus
call that is bound to fail.
The 5 s re-check remains only where the uevent socket cannot be opened
at all (some containers).
samples/frontend/gamemode gains a "nowake" lane: run under strace
without UPower, it checks that a uevent socket is opened and bound and
that no poll() ever uses the 5 s timeout. Skipped without strace.
* scan: look archives up by their first member's CRC first
A zip, 7z, apk, rar, zst or bz2 was hashed whole before its first
member's CRC was fetched. Cartridge databases key on ROM CRCs, so a
one-ROM-per-zip set always matches by the member: the whole read, the
main cost of scanning zips on a slow share, was wasted.
The databases are now walked first for the member's CRC alone. Where
it is stored (zip, 7z, RAR with CRC32) that costs one directory read;
a zst, bz2 or BLAKE2 RAR member is decoded for it, as before. Only
when that misses is the archive read whole, or a reader's whole result
taken, and the databases walked again for its own CRC, which the
arcade databases key on. The archive is expanded only after both
miss. Each walk's size check uses its own size: the member's, then
the archive's.
By default the core gate looks in a database only when a core claims
the file's extension. Stock cartridge cores do not list zip, so a
cartridge zip was read whole and matched only once expanded. The
first walk now also takes the databases claimed for the first member's
extension. A hit there is held while the walk finishes the databases
claimed for the archive's name: a match in one of those is written for
the archive; otherwise the archive is expanded without the whole
read. With "Scan Inside Archives" off nothing expands, so the first
walk keeps to the archive's name.
The parallel readers likewise start an archive with its first member's
CRC, size and extension, and hand them to the scan. They hash
archives whole only as far as the scan grants: a member-CRC miss
grants one whole read ahead, each further miss in a row twice the
last, up to 16 at a time, and a member-CRC match takes back what is
left. A lone unknown cartridge then costs one wasted read, while an
arcade folder soon has the readers 16 ahead. While the scan first
walks a database's index in a member pass mid streak, the readers may
keep 16 ahead, so the walk does not leave them idle. A grant goes
first to the nearest archive ahead that has only its member CRC.
Without a reader's whole result the scan hashes an archive it needs
whole itself, in steps, like a file over the cutoff.
An archive with no CRC for its first entry, such as a zip or 7z with
a directory first, skipped every database, so its own CRC was never
compared. It now goes straight to the whole walk, and readers always
hash it whole. In first-match mode the walk's early exit leads to the
whole walk too, while that has not run.
The CRCs, sizes and walk are cleared at each file's start. A file
that errored left them to the next, and an archive after it was
looked up by the stale CRC. The serial is cleared with them: an
archive matched after a file that errored in its serial lookup was
written under that file's serial.
Every database's member CRC is now checked before any whole CRC;
each database used to check the whole CRC first. That differs only
when a zip's first member matches a cartridge record and the whole
zip an arcade record. The cartridge record now wins, written for the
zip itself where the core gate lets the zip's name into the cartridge
database, otherwise through the expansion. With "Scan Inside
Archives" off nothing expands, so in that second case the arcade
record still wins.
* samples/tasks/database: write an rdb count over 127 as a uint
A record count from 128 up is written as a msgpack uint32. The
read-ahead lanes size their sets from the reader window, 64 files per
reader, so they write 131 to 259 records; a fixint byte past 127
decodes as a map or a bool and the database is rejected on two- and
three-core hosts.
* scan: keep a refused crc index build refused
A database whose crc index build is refused is flagged, and neither
the size probe nor the CRC lookup starts another build for it. The
index budget only shrinks, so a retry could only walk the database
again to the same refusal, once per content file.
The database scan test gains a cap on index builds and a lane that
scans plain files with every build refused: one query walk each.
* scan: look archives of several members up by either CRC
An archive whose header lists more than one entry, an arcade set as a
rule, is read whole first, and each database takes its own CRC or its
first member's. A set whose first member is a cartridge ROM is
written for its arcade record rather than expanded into the cartridge
playlist, and an arcade folder walks no database claimed only for a
member's extension. Readers read such an archive whole without a
grant. The member pass stays for one-member archives, where a
cartridge zip needs no whole read.
file_archive_get_file_crc32_size_and_ext() reports the entry count
the backends already take from the archive's header. A database
claimed only for the member's extension joins the member pass only
while it has or may get a crc index; unindexed, the lookup is a full
walk the expansion repeats. A reader's member CRC survives a whole
read that fails or is over the cutoff, so the scan reads only the
archive itself.
Lanes: an arcade set with a cartridge ROM first, an arcade folder
beside a cartridge database, sets whose reader reads fail, and
cartridge zips against an unindexed database.
* archive: read a mapped zip's directory in place
When the archive is mapped, zip_parse_file_init() finds the end of
central directory record, reads the ZIP64 records and walks the
central directory where they lie; only the context is allocated.
Unmapped archives read them through the stream. The scan's member
pass opens every zip for its directory, so this is its hot path.
archive_zip64_test counts reads through the stream and checks that,
mapped, a plain zip with a comment and a ZIP64 one give their first
member's CRC without any.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Two things beetle-saturn carried in a private copy of rchd, done here
instead so every core reads CHDs with the same file.
Version 3 and 4 images store no unit size, and rchd inferred one only
from hard-disk metadata, so a v3/v4 CD image reported its hunk size
as its unit. Its tracks then did not fit and the open failed. Follow
MAME's guess_unitbytes(): hard-disk metadata first, then any CD or
GD-ROM track metadata (CHCD, CHTR, CHT2, CHGT, CHGD) means a 2448-byte
CD frame, else the hunk size. beetle-saturn papered over this with a
caller-supplied unit hint, which is no longer needed.
The sector ECC tables were filled on first use behind a plain flag,
which two decoders on two threads could race on; beetle-saturn added a
call to fill them before starting its workers. They are now constant
arrays, so there is nothing to set up and nothing shared is written.
Checked: a version 4 CD image built from a chdman image's raw data and
track metadata failed to open before and now opens with a 2448-byte
unit, two tracks, and every frame identical to the version 5 image it
came from. Images whose sectors had valid ECC (so chdman stripped it
and rchd regenerates it) read back byte-identical with cdlz, cdzl,
cdzs, cdfl and the default codecs, before and after, under ASan/UBSan.
Use null-driver callbacks for the synthetic OpenGL restart and restore the cached video selection before leaving the test. Assert that the null driver is restored so later lanes cannot inherit a forced GPU backend.
Line-buffer harness output so a crash does not hide the last completed lane.
Require the 509-character limit after literal concatenation, including macro-expanded message text, and explain how to handle longer text without suppressing diagnostics.
Keep the Game Mode explanation under 509 characters so the MSVC6 stand-in compiles with pedantic errors enabled. Align the registry harness deadzone refresh with its loop scope.
rchd_feed_borrow said lent bytes had to outlive every rchd_read_step()
that returned RCHD_PENDING. The decoder is done with them sooner: once
it has decoded the hunk, which is before it asks for the next range.
Spell that out, so a caller without a mapping can lend one staging
buffer per request and refill it for the next instead of paying for
rchd_feed_at()'s copy, and say that lent bytes are used once.
Documentation only; the decoder already behaves this way.
Reading a CHD allocated on the hot path. Every hunk fetched from the
file malloc'd a fresh staging buffer for its compressed bytes and freed
the last one; every CD FLAC hunk built a FLAC decoder (four
allocations) and tore it down; and the decoded-hunk cache, the CD
codecs' sector scratch and rflac's rewind point were all allocated on
the first read rather than at open. For an emulator core each of these
lands in the middle of a game.
- The staging buffer is kept and reused. It is sized to one hunk when
the open finishes (a compressed blob never exceeds a hunk) and only
grows for a blob that does. A borrowed supply points at the
caller's memory without freeing it.
- A borrowed range is handed out once: a later request for the same
range collects it again instead of reusing a pointer the caller only
promised for one read.
- One CD FLAC decoder per image, made at open and rewound with
rflac_reset() per hunk.
- The hunk cache and the CD sector scratch are allocated at open. Each
falls back to allocating on first use if that fails.
- rflac: a headerless decoder allocates its rewind point with itself,
not on its first frame.
Checked with a malloc counter over every sector of CD LZMA, CD FLAC
and mixed images at the default and 1 MiB hunk sizes: 299 and 25
allocations during reads before, 0 after. rchd_supply_test and
rchd_read_test pass under ASan/UBSan with leak detection on 23 chdman
images (four CD codecs plus the default set, hunks of 19584, 523872,
526320 and 1047744 bytes), exercising both the borrow and the copy
paths.
Like the Windows Low-Latency Power Plan, the performance power profile
now only applies on AC power. The Game Mode helper stays alive while
Game Mode is on and follows the power source:
- UPower's OnBattery property, re-read on each PropertiesChanged signal
UPower sends when the charger is plugged in or pulled;
- without UPower, /sys/class/power_supply (a mains supply offline, or a
discharging battery with no mains supply listed), looked at again
every 5 seconds since nothing announces the change.
Pulling the charger releases the hold, plugging back in takes it again.
The I/O priority part costs no power and stays on either way.
The helper waits in poll() on the bus socket and a wake pipe, so it
costs nothing between changes; turning Game Mode off writes one byte to
the pipe and the helper releases, closes and exits. Hand-over is still
the one atomic word (want + busy, CAS only), no locks. Leaving on exit
still starts nothing: the hold goes with the process's bus connection.
samples/frontend/gamemode gains fake_upower (answers OnBattery, flips
it and signals on SIGUSR1) and two lanes: "plug" (hold on AC, release
when pulled, hold again when plugged back in) and "unplugged" (no hold
until plugged in).
Game Mode no longer dlopens Feral's libgamemode and asks gamemoded to
act for us. Of what gamemoded does with its stock configuration, two
things matter to RetroArch, and both can be done from inside the
process without root:
- I/O priority best-effort level 0 on every thread (gamemoded's
ioprio=0 default). ioprio_set() acts on one thread, so every thread
in /proc/self/task gets it; threads created later inherit it. The
previous priority is put back on leaving.
- The CPU in its performance mode (gamemoded's governor and platform
profile). This is now a "performance" HoldProfile on
power-profiles-daemon (or tuned-ppd, same interface) over the system
bus, which desktop polkit policy grants an active session without a
password; the pre-0.20 net.hadess name is tried as a fallback. ppd
drops the hold by itself when our connection goes away, so a crash
never leaves the machine stuck in performance mode.
Screensaver inhibit, gamemoded's other default, RetroArch already does
with Suspend Screensaver. The rest of gamemoded (GPU clocks, split-lock
sysctl, X3D mode, core parking, renice, custom scripts) is opt-in there
or needs root, so it is not carried over.
The D-Bus exchange runs on a short-lived detached helper through the
existing libdbus run-time loader, so entering and leaving return at
once. Hand-over between helpers is one atomic word (want + busy, CAS
only), no locks. On exit only the I/O priority is restored and no
helper is started: the hold goes with the process's bus connection.
The code is plain C89, so C89 builds now get Game Mode too
(C89_GAMEMODE=no is gone). --disable-gamemode still turns it off.
samples/frontend/gamemode is rewritten: it builds gamemode_linux.c on
its own, runs a private dbus-daemon as the system bus with a fake ppd,
and checks the hold/release, the legacy name, a refusal, no daemon at
all, the I/O priority on existing and new threads, and that the exit
path sends nothing.
The lane counts on a hog task spending the per-frame I/O window ahead
of the screenshot, so each check encodes one row. When the harness is
descheduled for a whole window period (a loaded 2-CPU runner), or a
signal cuts the hog's sleep short, the screenshot opens a fresh window
and finishes the remaining rows at once, failing the lane about one
run in six.
The hog now waits out the period by the clock, and a check that ran
past the period counts as late; a late run proves nothing either way
and is taken again, up to three times.
The version 5 hunk ceiling was 512 KiB, but chdman writes hunks of up
to 1 MiB (its HUNK_SIZE_MAX). Every CD image made with a hunk above
524288 bytes -- 526320 (215 frames) and up -- was refused at open.
libchdr, which rchd replaced, had no such cap. Reported against
beetle-saturn (libretro/beetle-saturn-libretro#98).
Raise the version 5 ceiling to 1 MiB. Checked byte-exact against the
source image for CHDs made by chdman 0.264 at hunk sizes 19584,
523872, 526320 and 1047744 with cdlz, cdzl, cdzs, cdfl and the default
codec set.
Match RGUI scratch-line storage and exact nearest-neighbor sampling in the framebuffer harness, and release the scratch allocation. Stub command_frame_start in the hardcore input fixture so input polling links.
Each comes from a source of its own - a packed stream, or a coder of
one from one - into a buffer that keeps what the converter left of the
last piece; one that cannot stream is decoded whole, as all three were.
r7z_bcj2_decode_in() takes all four inputs a piece at a time, and a
branch candidate whose bit or target is not there yet waits for it
unread. It replaces r7z_bcj2_decode_piece(), which streamed main alone.
r7z_stream_test and r7z_bcj2_test take them 16 octets at a time.
One chain of CRC32X waits on each instruction's two or three cycles;
three lanes of 1 KiB keep the unit busy, and are put back together by
moving a lane's register on past 1024 zero octets, a linear map done a
nibble at a time from a 512-octet table: crc = on(on(c0) ^ c1) ^ c2.
No PMULL needed.
tools/encodings/crc32_bench.c times it against one chain on the
hardware; crc32_test's seams take in the lanes' sizes, and pass under
qemu-aarch64 with and without the instructions in the baseline.
Neither is reflected, so each 16 octets byte-swapped is 128 of the
message's coefficients as they come, and the register's seed is added
to the front. One fold, of x^n mod P for either polynomial, keeps an
accumulator equal to every octet so far mod P; it and what is left are
then the table path's from a seed of 0, with no reduction of their own.
PCLMULQDQ with SSSE3 from 64 octets, VPCLMULQDQ from 256.
From 1 KiB on, 26-27 octets a cycle against the tables' 0.75 and 1.15:
Vorbis and Opus pages 25-37x, CHD hunks 16-23x. crc32_test checks both
against bit-by-bit references at every length to 1200 and alignment,
the seams, and split anywhere.
32 octets a turn with AVX2 and 64 with AVX-512, chosen at runtime, the
weighted sum by pmaddubsw then pmaddwd; what is left goes through the
16-octet SSE2 blocks. Against those, 2.1-2.9x with AVX2 and 2.5-3.3x
with AVX-512 from 1 KiB on, 1.4-1.9x at 128-256 octets: every zlib
stream inflated or deflated, PNG's among them.
adler32_test runs every check through each width the CPU takes, and
its longest bound no longer reads past its buffer.
The ring BZip2 is read into is R7Z_BZIP2_RING, which r7z_stream_test
builds at 2560 octets: fx_big_bzip2's three blocks go round it. PPMd
over 6000 octets of noise takes about as much input as it makes, so
each step runs to where it stops short of the end of what is fetched.
The frontend passes a core's sensor calls for a sub-device
(RETRO_SENSOR_SUBDEVICE) on to the drivers with the index kept. It
remembers which sub-device sensors the core enabled, as it does the
controller's own, so one not enabled, or whose enable no driver took,
reads 0 without asking a driver. The controller's own sensors are
handled as before, and sensitivity and orientation apply to both; the
profile's axis remap stays the controller's.
The SDL2 (2.26 and up) and SDL3 joypad drivers answer sub-device 1 from
SDL's _L sensors: a Wii Remote's Nunchuk accelerometer, the left
Joy-Con of a pair. SDL3 restores it on a reconnect as it does the
others. Every other driver already refuses an index it does not know.
samples/input/sensor_subdevice runs the wrappers as they ship.
Index 1 is the part held in the left hand - a Wii Remote's Nunchuk, the
left Joy-Con of a pair - as SDL reports those (its _L sensors), so
cores and frontends number a controller's parts alike. A core finds out
whether the frontend has the part from the enable action's result,
since a frontend without it reads 0.
Some controllers carry more than one sensor of the same kind on a single
port. A Wii Remote with a Nunchuk attached is one player with two
accelerometers, and adding MotionPlus makes it two accelerometers and a
gyroscope, but retro_sensor_interface addresses exactly one sensor of
each kind per port. A second port is not an alternative, because ports
denote players.
Carry a sub-device index in the upper bits of the existing ID, the same
way RETRO_DEVICE_SUBCLASS encodes a subclass into a device ID. Index 0
denotes the controller itself and encodes to exactly the values already
in use, so every existing core and frontend is unaffected.
No new environment call or capability flag is needed, because the
existing contract already describes the fallback: a frontend that does
not implement this returns false from retro_set_sensor_state_t and 0
from retro_sensor_get_input_t, both of which are already documented, and
a core that receives either answer behaves as it does today. Cores and
frontends can therefore adopt this independently of one another.
Definitions only; no behaviour changes.
RetroArch used libiosuhax for three things: opening the IOSU handle
and an FSA client in os/wiiu/main.c, the storage_usb devoptab, and the
raw SD/USB sector access libfat reads sd: and usb: through. Those now
live in os/wiiu/fs/iosuhax.c, with everything but what main.c and
libfat call made static. deps/libiosuhax is gone.
Never used, so not taken: IOSU memory and kernel read/write, the SVC
call, volume flush, FSA mount/unmount (main.c always registered an
already-mounted path), and chmod, which newlib never reached - the
devoptab table had the chmod handler in the deviceData slot, which
mount_fs then overwrote. The table now has newlib's order, as libfat's
does.
No locks: every FSA call is one IPC request the iosuhax server carries
out on its own, and nothing is shared between threads once a device is
mounted, so the per-device OS mutex is gone. The one shared resource
left is a data buffer, claimed with a retro_atomic exchange; a thread
that finds it held stages through the stack instead.
No allocation per call: every call used to memalign one or two buffers,
every path was malloc'd, and every directory entry too. Requests are
now built in aligned stack buffers and paths written straight into
them. File and sector data go through a stack buffer up to 1 KB, the
shared 128 KB buffer made at mount up to that, and beyond it straight
into the caller's buffer: the server puts its reply header in front of
the data, so the 0x40 bytes in front of the stretch being read are
kept aside and put back. A large ROM read no longer allocates a buffer
its own size and copies it; the data is never copied.
Bugs fixed:
- statvfs had the device info call write 0x64 bytes into an 8-byte
stack variable
- stat read the path after freeing it
- fstat passed the file state's address as the FSA file handle
- open checked the wrong pointer for allocation failure
- seek with an unknown origin returned with the mutex held
- the device lookup used strtok, which is not thread safe
- read reported an error as end of file; it now returns -1
- seeking from the end after writing past it used the old length
Requests are byte for byte what libiosuhax sent: both were run on
32-bit big-endian PowerPC against a recorder and compared. The data
paths were run against a fake server for every size and alignment,
with the shared buffer free and held, short reads and reads past the
end, under AddressSanitizer, with guard bytes around the caller's
buffer. A compile-matrix lane now builds the file against a stand-in
for newlib's sys/iosupport.h.
The video filter golden-hash suite (34 test programs, each built twice
in CI, plain and with ASan/UBSan) built and ran everything one at a
time. On a 2-core box it took 21 s plain and 55 s with the sanitizers.
- Each test is now its own make target (run_golden_<filter>), so
make -j builds and runs them side by side; the workflow passes the
runner's CPU count to all three golden suites.
- Every run cleared the whole 5 MB output buffer before filtering; it
now clears only the rows the hash and the comparison read, which is
all they ever looked at.
- The test programs (golden_<filter>) go in .gitignore, beside the
golden_<filter>.h tables that stay tracked.
On the same 2-core box: 10 s plain and 28 s with the sanitizers. With
the sanitizers, most of what is left is Blargg's NTSC table, which the
filter builds at every create (about 40 million instructions each, 200
creates per variant, 5 variants).
Follow-up to the SDL3 vsync/device reopen/input series.
sdl3_next_window_is_sdl3(), which lets the SDL3 input driver survive a
video restart, decided from the video_driver setting alone:
- When a core asks for gl, glcore or vulkan, the frontend swaps the
setting and keeps the user's driver in cached_driver_id. On the
SDL3 video driver that is the common restart - loading any GL or
Vulkan core - and sdl3_ctx_enabled() keeps it in an SDL3 window
because of cached_driver_id, but the helper saw "glcore" and said
no, so the input driver was torn down and rebuilt for nothing.
- For any driver that has no context driver (d3d*, metal, ...) it
still looked at video_context_driver, so a gl_sdl3 left over in
that setting said yes. The SDL3 input driver then survived into a
window that is not SDL's, with no keyboard events.
Take the driver in front and ask sdl3_ctx_enabled() about the context
it would use: vk_sdl3 for vulkan, gl_sdl3 for gl, glcore and gl1, no
for everything else.
The SDL3 GL context now reports GFX_CTX_FLAGS_FULLSCREEN_IN_PLACE, so
a fullscreen toggle calls set_video_mode on the window it has. That
still went through the cached-context path every time: a needless
SDL_GL_MakeCurrent, a "Using cached GL context" log line, a
cached-context ack, and two swap-interval calls (the adaptive probe
and the restore), which some drivers make wait on the compositor.
When the window already existed and the context is this instance's
own, return right after the restyle.
The int16 paths round every fixed-point product half away from zero,
written as (v >= 0) ? ((v + h) >> s) : -(((-v) + h) >> s). Compilers
kept the branch in the busier loops, where it is taken on the sign of
the signal and mispredicts. The new dsp_filters/dsp_round.h computes
the same number without it, floor((v + h - (v < 0)) / 2^s), and
bitcrusher, chorus, echo, overdrive, panning, phaser, reverb and
vibrato use it.
It takes >> of a negative value as the arithmetic shift it is with
every compiler RetroArch is built with, as the equalizer's transform
and the video filters' lane code already do. A version that avoids that
too (shifting in unsigned after moving the value up by 2^63) cost an
extra add and was 8-12% slower in the light filters, so it was not
used. iir, wahwah, early reverb, tremolo and WSOLA keep their rounding:
there the new form was slower on 32-bit or 64-bit ARM.
Output is unchanged bit for bit: 3000 calls per filter of 1 to 3000
frames, with random, full-scale, square and quiet input, at 32, 44.1
and 48 kHz and five settings each (defaults, scaled down, scaled up,
zero, offset; reverb also at room size 1 and freeze), on x86-64,
aarch64 and armhf (qemu).
Time per 1024 frames against before:
x86-64 aarch64 (qemu) armhf (qemu)
reverb 0.24 0.93 0.58
vibrato 0.44 0.94 0.86
chorus 0.51 0.95 0.82
phaser 0.68 0.77 0.76
overdrive 0.88 0.76 0.86
bitcrusher 0.93 0.81 0.87
echo 0.96 0.67 0.81
panning 0.97 0.90 0.92
Reverb was the second most expensive filter on that path: 0.26 ms per
1024 frames on x86-64, now 0.06 ms.
The equalizer's int16 path (a fixed-point FFT convolution) took 0.6 ms
per 1024 frames, against 0.06 ms for the float path; it was the most
expensive DSP filter by far. In its transform:
- The butterflies' rounding (half away from zero) branched on the sign
of every product, and the halving between stages did the same for
every value. Both are now branch-free, with the same results, and
the halving vectorises.
- The overflow check took the absolute value of every result and ORed
them together; it now tests each against the limit with one compare.
- The twiddle pointer steps instead of being multiplied out, and the
first butterfly of each group, whose twiddle is exactly (1, 0), skips
its multiplies.
- The imaginary part's rounding is spelled differently from the real
part's, so GCC on AArch64 does not pair the two into a vector (moving
both products to vector registers and back for one add and shift).
Output is unchanged bit for bit: 1600 calls of 1 to 4000 frames with
random, full-scale, square and quiet input, at block sizes 2^6, 2^8,
2^10 and 2^13, on x86-64 (GCC, Clang), aarch64 and armhf (qemu).
Per 1024 frames: GCC x86-64 0.35 of the time (0.62 ms -> 0.22 ms),
Clang x86-64 0.74, armhf 0.61, aarch64 0.91 (qemu, relative only).
The stream is a source and what comes after it. The source is the
folder's first coder - LZMA and LZMA2 through their windows, PPMd,
Deflate, Deflate64, BZip2 a block at a time, or Copy - read a chunk at
a time with what the last left kept in front. Up to three branch
converters or Delta follow it, each passing on what it converts; and a
BCJ2 folder takes its main input from its source a piece at a time,
the call, jump and range coder inputs decoded whole first.
r7z_bcj2_decode_piece() is the converter over pieces of main.
What is made and not yet handed out waits for the next entry, so
in-order entries still decode their folder once.
r7z_filter_fixtures.h gains 7-Zip 23.01's archives through PPMd, BZip2,
Deflate, Deflate64, BCJ2 and Copy; r7z_stream_test streams them, built
to make 4 KiB and read 256 octets a step. r7z_methods_test and
r7z_bcj2_test stream their 7-Zip archives too.
Four ZMM accumulators, 256 octets a turn, then one 64 at a time, its
four lanes folded into the 128-bit tail the PCLMULQDQ path ends with.
Chosen at runtime where AVX-512 is and CPUID leaf 7 has VPCLMULQDQ,
built with GCC 8, clang 6 and Visual Studio 2019 on x86-64.
From 512 octets on, 26-37 octets a cycle against PCLMULQDQ's 10.5-11:
2.5x at 512, 3.4x at 4 KiB, 2.8x past the caches. crc32_test's fold
boundaries take in the new seams.
A job on the ubuntu-24.04-arm runner builds RetroArch with OpenXR and
uploads the stripped binary, so aarch64 testers, such as Steam Frame
users, can download a build of any commit or pull request.
The apt cache key now carries the runner's architecture off x86-64, so
an arm job never restores an amd64 entry; x86-64 keys are unchanged and
their caches stay warm. When a download stalls, apt-install.sh retries
architectures other than amd64 and i386 from ports.ubuntu.com, since
archive.ubuntu.com carries no arm64 packages.
Follow-up to "SMB/NFS: shut the connection pools down after the core
is unloaded".
That change moves smb_shutdown() and nfs_shutdown() after the core's
deinit, so a core closing its share files on unload no longer closes
them on a freed connection. Both functions still free the pool
whatever holds it, though, and two holders can outlive that point:
- An SMB stream holds its slot for its whole lifetime, so a file the
core (or anything else) never closed still points into the pool.
- The exit drains tasks for at most three seconds before the core is
deinited. A task blocked on a dead share is still running when the
pool goes, and frees it out from under its in-flight call.
Unpublish the pool first, so nothing new can take a slot from it, and
free it only when no slot is held, the same rule the pools already
follow when they are replaced. If a slot is still held the pool is
left alone; the process is exiting and the OS reclaims it.
The shutdown stays where that change put it, before
frontend_driver_deinit(): the Switch tears its sockets down there,
and freeing the pool sends the SMB disconnect over them.
retroarch_main_quit_finish() freed the SMB and NFS pools before
main_exit() deinitialised the core, so a core that closes files it has
open on a share in its unload or deinit (Vita3K, through FILE* streams
over the VFS) closed them on a freed connection: SIGSEGV in
retro_vfs_file_close_smb() on quit.
The integer CC resampler (the s16 audio path's "cc") worked out each
output frame's four kernel taps from eight evaluations of the
fixed-point cosine integral, each four rounded multiplies.
- Neighbouring taps share an end (tap i is int(x + 0.5) - int(x - 0.5)
at x = distance + 1 - i), so five evaluations now make the four
taps.
- The integral's polynomial is clamped to +-0.5, and past |u| = 1 the
clamp decides: from 1 + 2048 ulp up to 10.6 every Q24 value comes out
exactly +-0.5 (checked value by value), so those return at once. The
outer taps land there, so most evaluations skip the polynomial.
Output is unchanged bit for bit: 36000 calls at ratios 0.3 to 1.8,
varied per call as rate control does, with random and full-scale
input, on x86-64, aarch64 and armhf (qemu).
Beyond 10.6 the polynomial's products overflowed int64 (undefined
behaviour), which happened for a while after a large ratio jump (slow
motion on, then off, at a downsampling rate). Those now return the
clamped +-0.5 the polynomial is meant to reach; that is the only case
whose output can differ.
800 frames per call, GCC -O2:
ratio x86-64 aarch64 (qemu) armhf (qemu)
1.0 0.45 0.25 0.38
1.088 0.57 0.23 0.36
0.919 0.59 0.24 0.39
0.5 0.69 0.93 0.89
audio_mix_volume (mix a buffer into another at a volume) is not called
in RetroArch but is part of libretro-common, which cores use.
- SSE2 is now used wherever the whole build has it, MSVC included (it
does not define __SSE2__, so it got the C loop), as the other
libretro-common files already do.
- ARM builds with NEON get a NEON version; the C loop was not
vectorised there at -O2, as it may alias.
- The SSE2 version kept its 16 samples in arrays of vectors, which
GCC at -O2 put through memory; it now works on 8 samples a step
in registers, then 4, then one by one.
At 2048 samples (GCC 13, -O2, x86-64): SSE2 0.40 us -> 0.18 us; the C
loop is 0.68 us. Sums match the C loop's bit for bit on x86-64,
aarch64 and armhf (qemu), for lengths 0 to 70 at odd offsets.
audio_mix_volume_C and audio_mix_volume_SSE2 keep their names;
AUDIO_MIX_NO_SIMD builds the C loop alone.
Each frame used to be encoded into the NTSC signal, and the noise added,
on the calling thread before the workers started: about 0.47 ms of a
1.1 ms frame at 256x224 with AVX2 on two threads, so more threads
gained little. Now only the setup is done there (the input conversion,
each line's sync and burst, the frame's parameters: 0.06 ms). The rest
is a list of tasks the workers take in order as each comes free:
- 32-line bands, each encoding its video lines into their rows of
the signal and adding the noise to those rows (the noise generator
is jumped to each band's first sample, as the lane code already
did, so every sample gets the same number);
- the line-ordered start of decoding (sync search, colour carrier),
once every band is encoded;
- 32-line bands of decoding, each copying its output rows.
A task waits (lock-free, on retro_atomic) only for tasks already taken,
which their workers are running, so the work cannot stall with any
number of workers, nor when the packets are run one after another: a
single worker just does every task. A worker that wakes late also no
longer holds up the frame, since the others take its share. Each
worker has its own encoder lane buffers, made at create. Without
lock-free atomics, the calling thread does the encoding as before.
Race fix: when the picture has more lines than output rows (224 rows
from 240 lines, say), a line with no rows of its own still draws one,
the next line's first, and that line draws over it or blends with it.
Where a band boundary fell after such a line, two workers wrote the
same row at once, and the output depended on timing. It happened in
the existing code at some thread counts (at 256x224 with 9 workers, for
one). Bands now never end after such a line.
Output is unchanged byte for byte from the code before this change run
on one thread: checked over 7 sizes, XRGB8888 and RGB565, plain C, AVX2
and AVX-512, 8 frames each (field, noise and carrier state carry
across them), 1 to 16 workers, with the packets run in order, in
reverse order and on concurrent threads; also with the fallback
without atomics, on aarch64 under qemu, and under ThreadSanitizer.
At 256x224, 2 workers (2-core box): 0.81-0.94 of the time with plain
C, 0.86-0.88 with AVX2, 0.77-0.81 with AVX-512; the gain grows with
the worker count, as the part left on the calling thread went from
0.47 ms to 0.06 ms.
The upscale mapped output pixels to source pixels through a 16.16
ratio rounded down, (fb_width << 16) / out_width. At any factor that
is not a power of two the ratio comes out short, so the first source
column and row are repeated once too often and the last ones once too
few: at 3x, column 0 is three pixels wide plus one and the last column
is two, and the drift moves every column boundary between them. It
shows as uneven text and uneven borders at Internal Upscale 3x, 5x, 6x
and 7x, and at the Auto level whenever it lands on one of those.
The source of each output pixel is now floor(dst * fb / out), stepped
along each row and column with a remainder, so there is no division
and nothing can overflow. At a whole factor each source pixel is
repeated exactly that many times.
Output at 2x, 4x and 8x is unchanged byte for byte; at other factors
it now matches an exact reference (checked over 1500 random sizes,
factors and pitches, under ASan and UBSan).
Text: every glyph pixel was a bool to test, so each character cost 50
(5x10) to 100 (10x10) branches, set or not. The font LUTs now also
hold each glyph row as a bit mask (bitmapfont_lut_t.rows, built by the
loaders from the bool LUT, which stays for the other users), and the
blits walk only the set pixels: each mask is taken 5 bits at a time
through a 32-entry table of set columns, in ascending order, so pixels
and shadows are written in the same order as before. No compiler
builtins. The eight blit variants share two helpers for the glyph.
Upscale (Menu > User Interface > Internal Upscale): the source column
of each output column is worked out once per redraw instead of per
pixel, and each output row is built once into a line buffer and copied
to every output row that repeats it. The output can be a buffer loaned
by the driver (upload heap, dumb buffer), so it is written row by row
and never read back. The line buffer is allocated before the loan is
taken, grown only when the output gets wider, and freed with the
upscale buffer.
Output is unchanged byte for byte. Text, 22 lines of 40 characters:
x86-64 armhf (qemu) aarch64 (qemu)
5x10 0.47 0.54 0.67
5x10 shadow 0.56 0.49 0.63
10x10 0.37 0.61 0.51
10x10 shadow 0.41 0.44 0.46
6x10 0.37 0.57 0.57
6x10 shadow 0.57 0.52 0.57
Upscale of 320x240, 2x to 6x: 0.27-0.39 of the time on x86-64 (at 4x,
460 us down to 145 us), 0.26-0.58 on armhf and 0.22-0.51 on aarch64
(qemu, relative only).
Tested against the old code with standalone harnesses: all ten blit
functions on random glyphs (blank, full and mixed rows, so shadows
overlap the next glyph) and random UTF-8 text across every font range,
40000 blits; the upscale loop over 600 random sizes, integer and
non-integer factors and padded pitches, under ASan and UBSan. Changing
the walk order, run step or line reuse makes them fail. Builds clean
with GCC (gnu99 and gnu89 with -Wdeclaration-after-statement, with and
without HAVE_LANGEXTRA) and Clang.
Super 2xSaI and SuperEagle now copy blocks of one colour a vector at a
time, as 2xSaI does: a run writes a vector of pixels doubled on both
output rows, and the full rule runs on just the pixels whose block is
not one colour; after vectors that copy nothing, the plain loop takes
the next stretch, so detail costs what it did. For Super 2xSaI, as for
2xSaI, the pixel after and the one below already settle it (its rules
then only produce those pixels and blends of each with itself); for
SuperEagle all four pixels of the block are tested, since one of its
outputs blends in the fourth. The runs (SSE2, NEON; AVX2 picked from
the SIMD mask) and the row loop around them move into
gfx/video_filters/softfilter_copyrun.h, which 2xSaI now uses as well.
Scale2x: each output corner is the pixel above or below or the pixel
itself, chosen by equality tests, so whole vectors of pixels are now
worked out branch-free, with the first and last pixel of a row one by
one. It also takes the workers it is given: rows are split between
them, and the rows above and below are clamped against the frame, not
the band.
Grid2x, Grid3x, Gameboy3x, Dot Matrix 3x, Normal2x and its width- and
height-only variants, Scanline2x, Upscale 1.5x and Upscale 1.66x Fast
take the workers they are given too: each output row (for the
upscalers, each group of 3 or 5) comes from its own source rows alone,
so the rows or groups are split between the workers. They were held
to one on a note that they were not thread-safe. The fixed-size
upscalers to 320x240 stay as they are.
Output is unchanged byte for byte. On one thread at 256x224 (GCC):
Super 2xSaI game 0.85-0.93 of the time, flat 0.21-0.44, RGB565
gradients 0.87; noise and XRGB8888 gradients within 2%
SuperEagle game 0.67-0.73, flat 0.14-0.27, RGB565 gradients 0.88;
noise and XRGB8888 gradients within 5%
Scale2x 0.25-0.59 with SSE2, 0.17-0.37 with AVX2, every frame
Tests: samples/gfx/filter_golden gains tables for the eleven filters
gaining threads, recorded from the code before this change (each is now
also run with two to four workers against one), and builds of Super
2xSaI, SuperEagle and Scale2x without their vector code. NEON was run
under qemu on aarch64 and armhf.
Follow-up to "fix(ps3): RSX - restore frontend state for CPU fallback
frames".
That change restores the frontend's RSX state for both hardware and CPU
core frames, but still skips it on every duped (NULL) frame. A dupe
is not always empty: the menu, the overlay, the OSD/statistics text
and the widgets are all drawn on top of it, and with the menu open the
frame is presented. Those draws then run with whatever surface,
depth, blend and vertex attribute state the core left behind. A
core that runs at 30 fps and dupes every other frame hits this every
second frame.
Restore whenever the frontend will draw anything this frame: a core
frame, the menu, the overlay, statistics, a message or the widgets.
A bare dupe that draws nothing still skips the reset, so it stays free.
Add a check to rsx_alloc_test for a dupe under the menu (it fails
without this change), and make the test rebuild when rsx_gfx.c
changes: the test #includes the driver, but nothing told make, so
editing the driver silently re-ran the old binary.
Hardware cores share RSX state with the frontend and may submit either
hardware frames or CPU buffers. Restore frontend state for both paths
to prevent CPU fallback frames from inheriting the core's render state.
Keep the reset skipped for NULL duplicate frames.
Follow-up to "rslc: initialize local constants at their use sites".
That change stores the constant right before each dynamic access,
which fixes the Adreno scratch-memory blowup, but every access to the
same constant reuses one Function variable. With two or more access
sites the variable is stored again after it has already been read,
and from different blocks. Mesa's nir_opt_large_constants only
promotes a variable to a constant table when all of its stores are
constant, come before any read and sit in one block, so the shared
variable is no longer promoted on RADV, ANV, Turnip, NVK or lavapipe
and falls back to registers or scratch -- the same class of slowdown
the change set out to fix, moved to Mesa drivers.
Emit a fresh Function variable for each access instead, stored right
before it is read. This is exactly what glslang does with its
"indexable" temporaries: the store stays inside the branch that reads
it, and each variable is written once before its only read, so Mesa
promotes every one of them again.
Global constants are unchanged: one Private variable initialised with
the constant.
Keep constant-array stores inside the branches that use them instead of initializing every array at function entry. This avoids excessive scratch-memory traffic in Hyllian shaders on Adreno GPUs.
The surface a core's views are laid out on was created the first time
a views frame came through, an allocation in the middle of a game, and
cleared to black with the RGA on every views frame.
It is now made at init with the other surfaces; without it, views are
drawn whole as before. The views overwrite the same rectangles every
frame, so the rectangles of the last frame are kept (at most
VIDEO_VIEWS_MAX_PLACEMENTS of them, in the driver's own struct) and
the surface is cleared only when they change.
The ODROID-GO Advance video driver did a full drmModeSetCrtc modeset
for every frame and cleared the whole page with the RGA before every
blit. It also wrote into pages without knowing whether the panel was
still scanning them out.
Presenting: the first frame sets the mode and every frame after is a
drmModePageFlip, which the kernel carries out at the next vblank on
its own. The only wait is for the previous flip to land before the
next is queued, and the RGA work for the new frame runs before that
wait. A page is never drawn while it is on screen or queued. A driver
that refuses page flips falls back to drmModeSetCrtc.
Fast-forward and vsync off: instead of showing every fourth frame,
a frame whose flip would have to wait is skipped before any RGA work
is done for it. The panel shows one frame per refresh and the core
runs as fast as it can, threaded or not.
Clearing: the blit writes every pixel inside the picture, so only the
borders can hold anything stale. A page is cleared once after the
picture moves, the menu/game/views switch or the rotation changes,
after a message was drawn over it, or while paused (the dimming blend
has to land on black) - never as a matter of course.
16-bit pages for 16-bit cores: the scanout pages take the core's
depth, so an RGB565 core's frames go out at 16 bits. That halves the
bytes the RGA writes and the display controller reads every frame.
Menu, messages and views are still converted by the RGA on the way.
Menu: RGUI's frame is kept at its own depth and copied by rows; the
per-pixel RGB565 to XRGB8888 loop on the CPU is gone, and with it its
colour error (low bits of one channel spilled into the next). The menu
surface is made at init and only made again if a frame outgrows it, so
opening the menu allocates nothing.
Bugs fixed:
- surfaces recorded their pitch as width * bpp, but Rockchip pads dumb
buffer rows to 64 bytes. A core whose rows do not fall on that
boundary (a 700 pixel wide PSX frame is 1400 bytes, padded to 1408)
was lent a framebuffer with the wrong pitch, and the RGA read each
row from the wrong place. The kernel's pitch is used now, for the
core's loan and for every RGA stride.
- mappings and prime fds were never released, and a failed init leaked
everything it had made
- pages and the views surface are no longer mapped for the CPU, which
never touches them
- the console's CRTC is put back on exit
oga_blit no longer uses designated initializers, so the file is C89.
2xSaI: a pixel equal to the pixel after it and the one below it comes
out as four copies of itself, whatever the fourth pixel of its 2x2
block is (the full rule's second and third cases then only produce
it). Those pixels are now found and written a vector at a time (SSE2
and NEON, 4 or 8 pixels; AVX2, 8 or 16), and the full rule runs on just
the pixels that failed, as 2xBR does. After vectors that copy nothing
the next 1, 3, 7 and up to 63 vectors' worth of pixels go through the
plain loop, and those stretches run in the plain loop's own form, so
detail costs what it did; builds without a vector tier keep the plain
loop alone.
Pixel Art AA: each of the four output corners is the pixel or a blend
of it with the neighbour above or below, chosen by equality tests, so
whole vectors of pixels are now worked out branch-free, with the first
and last pixel of a row (whose side neighbours clamp) one by one. It
also takes the workers it is given instead of one: rows are split
between them, and the rows above and below are clamped against the
frame, not the band.
Normal4x, Grid4x, Gameboy4x and Dot Matrix 4x take the workers they are
given too. They were held to one on a note that they were not
thread-safe, but each output row comes from its own source row alone.
On one thread they are bound by writing 4x the pixels out; copying
repeated output rows, tried as well, was slower than writing them.
2xBR: its backoff carried the vectors to skip from row to row, but
restarted each row at that distance, so once it reached its cap on a
frame narrower than the skip (504 pixels), no later row of the frame
tried a vector again and none could reset it. The skip is now carried
in pixels, running on into the next row. 2xSaI shares the same loop.
The SSE2, AVX2 and NEON gating and SOFTFILTER_FORCE_INLINE move into
gfx/video_filters/softfilter_simd.h, used by 2xBR, 2xSaI, De-Dither and
Pixel Art AA.
Output is unchanged byte for byte. On one thread at 256x224 (GCC):
2xSaI game 0.61-0.63 of the time with SSE2, 0.60-0.62 with AVX2;
flat 0.12-0.20; RGB565 gradients 0.75; noise and XRGB8888
gradients within 3%
Pixel Art 0.16-0.30 with SSE2, 0.13-0.19 with AVX2, every frame type;
AA 0.85-0.97 without a vector tier
4x unchanged on one thread; two workers about 0.55-0.7 of the
time, where they could only use one
Tests: samples/gfx/filter_golden gains tables for Pixel Art AA and the
four 4x filters, recorded from the code before this change, and an
output buffer large enough for 4x; Pixel Art AA and 2xSaI are also
built without their vector code. NEON was run under qemu on aarch64
and armhf.
Only one file in RetroArch ever used libgo2: the ODROID-GO Advance GL
context, gfx/drivers_context/drm_go2_ctx.c. The oga video driver
already talks to librga and DRM itself, and libgo2's audio, input,
backlight, PNG and queue code was never called. What the context used
- open the panel, make the EGL/GBM surface, and have the RGA scale and
rotate each frame into a portrait buffer for the CRTC - now lives in
the context itself, and deps/libgo2 is gone.
No thread, queue, mutex or semaphore remains. libgo2 ran a render
thread, a mutex-guarded queue and two semaphores only to hide a full
drmModeSetCrtc modeset every frame. Frames are now page-flipped with
drmModePageFlip, which the kernel already does asynchronously, so the
only wait left is for the previous flip to land before the next is
queued - the vsync throttle itself - and the RGA work for the new
frame runs before that wait instead of after it. That is also one
frame less latency than the old three-deep queue. A driver that
refuses page flips falls back to drmModeSetCrtc.
Overhead taken out of the frame:
- the whole-screen RGA colour fill before every blit is gone; the
borders beside the picture are cleared once per buffer when the
picture's size or position changes, and never when it fills the panel
- each GBM buffer's prime fd is exported once and kept on the buffer
(gbm_bo user data) instead of a per-frame search of a malloc'd list
- the scanout buffers' fds, strides and formats are worked out at init,
not per blit
- nothing is allocated per frame
With vsync off (swap interval 0, e.g. fast-forward) a frame whose
flip would have to wait is not shown rather than waited for, so the
core runs uncapped without tearing; before, the interval was ignored.
The EGL context now comes from egl_common like drm_ctx's, so
egl_bind_hw_render works on it (a hardware core gets its shared
context) instead of being handed an empty egl_ctx_data_t. It is still
GLES 3. Fixed on the way: the context data was never freed, prime fds
were leaked, and the console's CRTC is put back on exit.
-lpng and -lz are no longer linked for HAVE_ODROIDGO2; only libgo2's
unused PNG writer needed them. A compile-matrix lane now builds the
context against the librga stand-in; nothing compiled it before.
A folder of LZMA or LZMA2 and a converter after it - BCJ, ARM, ARMT,
ARM64, PPC, SPARC, IA64 or Delta - streams through its window as one of
LZMA alone does. Each piece is copied out of the window, which matches
still read unconverted, and converted there; the octets of an
instruction not yet seen to its end wait for the next piece, and the
folder's last pass as they are. Two converters in a row are decoded
whole, as before.
r7z_filter_fixtures.h holds 7-Zip 23.01's archive of each, over a
dictionary of 4 KiB; r7z_stream_test streams their members in and out of
order, and one of x86 calls close together across every piece's end.
GET_VARIABLE and SET_CORE_OPTIONS_DISPLAY looked through every option
for the key. Past 64 options the keys are put in slots by their hash
when the options are made, and probed; up to 64 the walk in order is
quicker and stays. A lookup among 400 options takes 87 cycles, not 239;
among 200, 53 for 151.
Most of a game frame is pixels 2xBR only copies: a pixel equal to
enough of its four direct neighbours that none of the corner rules can
fire. The flat-block test already skipped the rest of their
neighbourhood, but still cost five loads, four comparisons and four
scalar stores a pixel, about 6 cycles. Now that test runs on 4 or 8
pixels at once (SSE2, NEON; 8 or 16 with AVX2), every pixel of the
vector is written doubled on both output rows, and the full rule runs
on just the pixels that failed it, whose own output then replaces the
copy: the same result the per-pixel path gives.
On detail, where no pixel in a vector is copied, the vector work would
be wasted, so after such a vector the next 1, 3, 7 and up to 63 vectors'
worth go one by one, carried from row to row. The full rule is forced
inline in this path; as a call, the loop around it cost detailed frames
several percent. Builds with no vector tier keep the previous per-pixel
loop exactly, as the reworked one is slower there.
SSE2 is used where the build has it (all x86-64), NEON on ARMv8 and ARM
builds with NEON, and AVX2 is picked at create from the SIMD mask, gated
like the other filters' tiers.
Output is unchanged byte for byte. On one thread at 256x224, under GCC
with SSE2 a game frame takes 0.71 (RGB565) and 0.80 (XRGB8888) of the
time it did and a flat one 0.35 and 0.56; with AVX2 0.74, 0.78, 0.33
and 0.49. Noise and smooth gradients, where nearly nothing is copied,
take the same time as before (within 2%), under Clang too.
Tests: 2xBR moves into samples/gfx/filter_golden, recorded from the
code before this change, where it is also run with the host's SIMD
mask and built without its vector code; samples/gfx/xbr_filter_golden,
which only did the first, is retired. SOFTFILTER_NO_SIMD now leaves out
the vector code of every filter that has any (2xBR, De-Dither, and SNES
NTSC as well as its own SNES_NTSC_NO_SIMD), and the harness builds each
of them that way too. NEON was run under qemu on aarch64 and armhf.
De-Dither forced itself onto one worker whatever RetroArch asked for,
and spent about 28 cycles a pixel in a chain of short-circuited colour
comparisons whose branches game content mispredicts.
It now takes the workers it is given: each gets a band of rows, and the
6x6 pattern is bounded by the frame's height, not the band's, so the
output does not depend on how many there are. The interior of each row
goes through a vector run that works out all ten comparisons of 4
(SSE2, NEON) or 8 (AVX2) XRGB8888 pixels, or 8 or 16 RGB565 ones, at
once and branch-free, with the plain per-pixel code for what is left of
a row; the edges are copied as before. SSE2 is used where the build has
it (all x86-64), NEON on ARMv8 and ARM builds with NEON, and AVX2 is
picked at create from the SIMD mask, gated like the other filters'
tiers (target attribute on GCC 5+ and Clang 4+, MSVC 2017 15.3+, the
baseline only under PlayStation clang and clang-cl).
Output is unchanged byte for byte. On one thread at 256x224 a game
frame takes 0.17 (RGB565) and 0.30 (XRGB8888) of the time it did with
SSE2, and 0.11 and 0.16 with AVX2; noise in XRGB8888, where the old
code bailed out early, 0.77 and 0.40. More workers divide that
further.
The golden harness gains De-Dither, and its game frames a band of rows
alternating between two colours: until now nothing in them was dither
that runs down but not across, so De-Dither's RGB565 vertical path (and
any filter's handling of it) went untested. Every table is re-recorded
for the new frames, from the code before this change for De-Dither and
from current master, which matches the earlier recordings byte for
byte, for the rest. NEON was run under qemu on aarch64 and armhf.
A Phosphor2x line went through five sweeps of the output: the source
splatted to every other pixel, the pixels between blended in, then the
red, green and blue phosphor bleeds each read-modify-writing the line in
place, then the scanline below read back from it. Every output pixel
only depends on the stretched pixel just before it, so each is now built
in a register with the same set_ macros, the scanline pixel from it, and
both lines are written once.
The work is done in 32 bits: RGB565 values stay within their 16 there,
and GCC's 16-bit register forms for uint16_t cost a merge on each
operation, which is why the RGB565 path took twice as long as XRGB8888
under GCC (Clang already avoided it).
Output is unchanged byte for byte (32-bit x87 builds, whose float
tables differ from SSE builds, give the same bytes as before too). At
256x224 on one thread under GCC, a frame takes 0.53-0.82 of the time it
did in RGB565 and about 0.9 in XRGB8888.
The standalone translation unit selects AVX2 blitters using the softfilter capability mask. Include its defining header directly so PS5 compilation does not rely on the wrapper inclusion order.
Each output pixel of the SNES NTSC filter is the sum of six kernel
entries (twelve in hires) out of a table indexed by input colour, then
clamped and packed: 42 loads and 35 adds for every 7 pixels. Read the
other way round, each input pixel adds a run of 14 consecutive table
entries into 14 consecutive output pixels, so a chunk of 7 output pixels
is a handful of whole-vector loads and lane blends. The new blitters
add up the chunk that way, 4 lanes at a time with SSE2 or NEON and all 8
at once with AVX2, and clamp and pack lane-wise with the same operations
as the scalar macros.
That needs one lane per table entry, so where a vector blitter is built
the table keeps the low 32 bits of each entry instead of an unsigned
long: entries are still built at full width at create, and the
blitters only add, mask and shift them, with nothing above bit 30
reaching a pixel, so the sums give the same output. Where unsigned long
is 8 bytes (64-bit Linux and macOS) that also halves the table, from 8
MB to 4 MB. Builds without a vector blitter keep the old table and the
old scalar blitters unchanged.
SSE2 is used where the build has it (all x86-64; 32-bit x86 built with
SSE2) and NEON on ARMv8 and ARM builds with NEON. AVX2 is picked at
create from the SIMD mask, built with the target attribute on GCC 5+ and
Clang 4+, as is on MSVC 2017 15.3+, and only where the baseline has it
under PlayStation clang and clang-cl. The table gains a few readable
entries either side for the blended-away lanes that fall just outside
it.
Output is unchanged byte for byte, in every tvtype. At 256x224 on one
thread a frame takes 0.27-0.32 of the time it did with AVX2 and
0.37-0.47 with SSE2; hires (512 wide) 0.34-0.42 and 0.45-0.56.
The golden harness gains tables for Blargg NTSC SNES (composite, and RF,
RGB and S-Video, which build different tables, RF without the field
merge), a build of it with the vector blitters left out, and a check
that a filter created with the host's SIMD mask gives the same bytes as
one created with none. NEON was run under qemu on aarch64 and armhf:
it matches the recorded tables, and on aarch64 builds, where GCC fuses
multiply-adds while building the table, gives the same bytes as the old
code there.
Pieces of up to 256 octets are held, in fixed copies that overlap, until
1 KiB is; longer ones go where they lie. AVX-512 builds the stripe the
held octets end inside in a register and keeps what is left a vector at
a time, so the next call's loads come from one store each. AVX-512 now
needs BW, from GCC 5 and Visual Studio 2017 15.3.
Against XXH3_64bits_update() with AVX-512, in pieces of 1 to 512 octets
1.13-1.86x, 700 to 1500 1.03-1.12x, 2 KiB even, 3 KiB on 1.05-1.30x.
Labels of eight bytes and up hash with XXH64 and spacers with FNV-1a;
entries match on length and memcmp. A smooth ticker frame takes about
12% fewer cycles.
A member out of a folder of others over 32 MiB is streamed into a
buffer of its own, and the folder never held whole. A stream that has
put out one member of a folder goes on from there for a later one, so
a folder's members streamed in order decode it once, whether to their
files or into memory.
Makefile.griffin builds Linux with _GNU_SOURCE, as Makefile does, so
rthreads.c's affinity calls are declared from the first system header.
The Vulkan X context's window hints and the udev joypad's sensor
functions have names of their own, which the X context and udev input
driver's no longer clash with, and the X11 keyboard LED driver is in
the unity build where X11 is.
rar5_write.h writes a RAR 5 archive of one member, LZ symbols in codes
of fixed lengths with delta and E8 filters among them, and works out
what it decodes to. Members of 10 and 40 MB in dictionaries of 4 and
32 MiB, with matches back across the dictionary and filters' blocks
across where the ring comes round, must come out whole and streamed,
from memory and read.
Every output pixel of Phosphor2x went through float maths twice: the
phosphor bleed scaled each channel by a gamma curve, and the scanline
pass scaled each channel by a level picked from the pixel's brightest
channel, each followed by a float-to-integer conversion. Both results
depend only on small integer inputs (a channel value for the bleed; the
brightest channel and the channel being scaled for the scanline), so
they are now worked out once at create into tables, with the same
expressions and operand types the lines used, and the lines look them
up. The tables are about 70 KB and live in the filter's existing
allocation; nothing is allocated per frame.
Output is unchanged byte for byte. At 256x224 on one thread a frame
takes about half the time it did, in both pixel formats and for game,
flat and noise content.
The golden harness that held the 2xSaI family is generalised so any
filter can use it: it skips input formats a filter does not take, picks
the output format and size the filter reports, and checks that 2, 3 and
4 workers give the same output as one. It moves from
samples/gfx/sai_filter_golden to samples/gfx/filter_golden, and gains a
table for Phosphor2x recorded from the code before this change.
Remove the extra closing directive after the UDEV input sources. It has no opening conditional and prevents every Griffin platform configuration from preprocessing.
libShake was used by one driver, sdl_dingux, for its vibration motor,
and linked on webOS for nothing. What it did there is a few evdev calls:
find the node that plays a periodic sine, upload a sine per motor, play
and stop it, set the gain. input/common/linux_ff.h does those with no
allocation (libShake scanned /dev/input with scandir and malloc'd a node
per entry) and none of its bugs (an open that failed was taken as
success).
The uploads and plays now go out from the shared output writer, as
udev's and XInput's rumble do, so a motor's driver never holds the
frame; the frontend's thread notes the strengths, both in one word, and
returns. A core asking the same strength each frame costs no call at
all. On MIPS (-O2, mips32) that path is 15 to 19 instructions with no
stack frame, where it was about 32 with a frame and four spills.
Every SDL Dingux target gets the motor now, not only the three that
built libShake; one without a motor finds none and has no rumble, as
before.
The output writer makes no syscall for a wake while one is still on its
way, which every driver on it gains from. xinput_rumble_test says a call
did not wait by the write not being done yet, not by a time a loaded
runner can overrun.
samples/input/dingux_rumble runs the real driver against a stand-in
/dev/input and SDL, with threads and without.
A member streamed out is unpacked round a ring of its window or a
filter's largest block, and two steps: once the next step would not
fit before the end, the decoder goes on from the start, and a match
reaching back before it is copied from the end and on from the start.
What is held back for a filter is read from where it lies, in two where
the ring comes round. Nothing is moved, the ring is the dictionary's
size whatever that is, and the 241 MB member of the tests streams in
two thirds of the time.
The window a frame is decoded through a block at a time is a ring of
the window and two blocks, at least a megabyte: a block goes where it
fits whole, at the start again once it would not, and a match reaching
back before the start is copied from the end and on from the start.
Nothing is moved, and the buffer is half what it was.
Listing decodes the stream a block at a time for its size and CRC,
where the file stream lends it whole or through a ring of twice the
most a block takes. A member up to 8 MiB is kept for the extraction
that follows; a bigger one is decoded again, to its file a block at a
time. A member read to a file goes there a block at a time, and none
of one that does not decode is left behind.
The statestream block index keys its blocks with xxh3_64(), against
XXH32 a tenth of the time on a 16 KiB block and under half on 64 and
128 octets. The PS4 shader cache keeps its XXH64 keys, now from
xxh64_update() on the stack, so cached binaries still match. Nothing
else used deps/xxHash, which goes.
xxh3_reset(), xxh3_update() and xxh3_digest() give xxh3_64() of the
pieces handed over one after another, as XXH3_64bits_update() does.
Pieces are taken a stripe at a time where they lie, whole blocks
through the one-call loops, on the path the CPU has; under 256 octets
are held between calls, and holding a piece is a copy and no more.
The lanes are bound by the multiplier however the stripe loop is
unrolled, so it takes one stripe a turn, and an input of one stripe
skips the loop: with lengths that vary call to call, 5-12% fewer cycles
than the reference inlined under 256 octets, and the same above.
When a pixel and its right, lower and lower-right neighbours are one
colour, both filters give all four output pixels that colour: the
block lands in the rules' third branch, whose four tie-break terms
each compare two equal colours and cancel to zero, and a colour
blended with itself is that colour, in both formats. Most of a game
frame is such blocks, yet each one paid for the tie-break, and for
Super 2xSaI the corner rules after it.
- SuperEagle checks for the flat block inside that branch and sets
the four pixels without the tie-break. Nothing else reaches the
check, so other content pays nothing for it.
- Super 2xSaI checks the four pixels before loading the rest of the
4x4 neighbourhood and writes the block at once. Placed in the
third branch instead, it saved half as much and still cost as
much elsewhere.
- 2xSaI is left as it is: its rules already settle a flat block on
the first test of that branch, and a check of its own gained
little on game frames while costing up to 30% on noise.
Output is unchanged byte for byte. One thread, 256x224, time against
before (lower is faster; ranges over repeated paired runs and over
code alignment):
game-like flat noise
SuperEagle 0.55-0.60 0.42-0.44 0.98-1.00
Super 2xSaI 0.43-0.50 0.21-0.29 1.00-1.21
Noise, where no block is ever flat, is the worst case: there Super
2xSaI pays for the extra test on every pixel, by an amount that moves
with code layout.
samples/gfx/sai_filter_golden passes for all three filters, plain and
under ASan/UBSan, and fails if the flat path writes any other colour.
PlayStation clang declares in immintrin.h only the intrinsics of the
ISAs a whole build enables, so a kernel under a per-function AVX2 or
AVX-512 target compiles on every host and fails on Orbis. The matrix
now finds each unit carrying a per-function target or 256/512-bit
intrinsics by content and compiles it for x86_64-scei-ps4 (AVX) and
x86_64-sie-ps5 (AVX2, no AVX-512), so a new one is checked the day it
lands. A clang without the targets reports the lane skipped.
2xSaI, Super 2xSaI and SuperEagle read a 4x4 neighbourhood, two rows
below the current one. Each worker stopped those reads at the bottom
only when its slice was the frame's last. A worker whose slice ends
one row above the bottom, ahead of a one-row slice, read a row past
the end of the input: past the core's frame, into whatever follows
it. Rows divide among workers as height * i / threads, so this happens
when the frame has fewer than about two rows per worker - small
frames, or many filter threads.
Each worker now gets the number of frame rows below its slice instead
of a last-slice flag, and clamps against the frame. For the last
worker that is what it did before; for the others, the output changes
only where they read out of bounds. Over the golden matrix below,
one case changes (SuperEagle, a 3x3 frame in RGB565, with three and
four workers);
every other output is the same.
samples/gfx/sai_filter_golden runs each of the three filters over
game-like frames and noise, both formats, sizes from 1x1 to 320x240,
frames smaller than the created size, input rows at a wider stride,
and one to four workers. It checks output hashes recorded from this
implementation, and that two, three and four workers give one
worker's output byte for byte. It runs plain and under ASan/UBSan in
the filter plugins workflow; ASan reports the out-of-bounds read
against the previous code.
A client compares the host's CRC for a frame either when it arrives or
once the client reaches that frame, and both now go through one check.
The first check at or after a state loaded from the host decides
whether this core's CRCs can be compared: that state is the host's, so
a mismatch there turns desync checks off with a warning, as a core
whose states carry a timestamp needs. Any later mismatch asks for the
host's state. A client logs each state it loads from the host.
samples/network/netplay_e2e gains a rollback desync lane with input
latency, so some checks wait for the client to reach their frame, and
a lane whose core states never hash alike between peers, which must
stop at one resync past the join with its checks turned off.
NETPLAY_E2E_JUNK puts a per-peer serialize count in the test core's
state.
demodulate_begin adds noise to all 238420 samples of the signal before
the workers start, one sample at a time: each step of the linear
congruential generator (rn = 214019 rn + 140327895, mod 2^32) waits
on the one before, a multiply and an add per sample, about 0.45 ms of
every frame.
Applying the generator k times is again one multiply and add, with
A = a^k and C = c (a^(k-1) + ... + 1), mod 2^32. So lanes can each
start at their own sample and step by the lane count, and they see
exactly the numbers the one-at-a-time loop does. With AVX2, 32
samples a step in four vectors; with AVX-512, 64 in four; four
independent vectors keep the multiply's latency hidden. The noise,
the clamp and the narrowing to bytes are the scalar loop's integer
operations. The samples left over finish in the scalar loop from the
state crt_lcg_jump computes, so v->rn, which carries to the next
frame, is unchanged too.
The tier is the one the lanes elsewhere use, from the SIMD mask given
to create; this pass needs no buffers. demodulate_begin comes before
the lane code in the file, so the tier macros and the generator's
helpers now sit in front of it.
256x224 on one thread, AVX-512 Xeon:
demodulate_begin 0.45 -> 0.10 ms (both tiers)
frame 3.1 -> 2.8 ms
The golden test's noise scenarios, run over four consecutive frames,
cover the carried state; it passes at all three tiers, and fails at
the wide tiers alone if the vector arithmetic is changed, or at every
tier if the jump is.
The modulator runs on the calling thread before the workers start, so
it adds to every frame whatever the worker count. For RGB input, the
format the wrapper feeds it, each line runs three one-pole filters
along its samples from zero and turns them into a signal level, and
it converted the source pixel from RGB to YIQ again for every sample
- about three times per pixel at 256 wide. That was about 1.3 ms at
256x224.
Lines are independent here, so they now run in lanes, as the decoder's
do: 8 with AVX2, 16 with AVX-512, each lane doing the scalar loop's
integer operations in its order, so the output is unchanged byte for
byte. Each batch's source rows are converted to YIQ once per pixel,
into lane-interleaved planes, a column at a time. Then the lanes run
the filters and levels per sample, and each lane's levels are copied
to its signal row. Lines left over (236 lines do not divide by 8 or
16) take the loop as before.
The planes are sized at create from the largest frame create is told
of; a wider frame, which the softfilter contract rules out, is
modulated line by line instead. The tier is the decoder's, from the
SIMD mask given to create, built per function and, under PlayStation
clang or clang-cl, only where the baseline enables it. The lane code
sits before the modulator's EXP_P definition, so it carries its own
copy, with an #error should the two ever differ.
256x224 on one thread, AVX-512 Xeon:
modulate 1.28 -> 0.53 ms (AVX2), 0.54 ms (AVX-512)
frame 3.8 -> 3.05 ms with either
What is left of the modulator is mostly the per-pixel conversion and
the copy back to the rows, both scalar.
samples/gfx/crt_filter_golden now names its tiers SIMD tiers, since
they cover the modulator as well; its table and matrix are unchanged
and it passes at all three.
After demodulate_begin, each active line is decoded on its own: three
equalisers, each a chain of IIR stages, run over every sample, every
step waiting on the one before. That was about 4.7 ms of a 256x224
frame on one thread.
Lines are independent, and without bloom (as built) every line covers
the same samples, so they can run in lockstep: one line per lane, 8
with AVX2 or 16 with AVX-512. Each lane does exactly the scalar
decoder's integer operations in the same order - the multiplies keep
the low 32 bits and the shifts are arithmetic, as int arithmetic is on
every target this builds for - so the output is unchanged byte for
byte.
- The lanes' signal is transposed to one byte per lane per sample, and
Y, I and Q are decoded in separate passes so each pass's state stays
in registers.
- The decoded samples stay interleaved, and the pixel pass, now
crt_render_line, reads them through a stride. The line-by-line
decoder passes its struct crt_yiq array the same way, so both share
one pixel pass.
- Lines are taken in order, a batch at a time; what is left over (fewer
lines than lanes) is decoded line by line, so rows are drawn in the
same order as before.
- The tier comes from the SIMD mask RetroArch passes to create - the
plugin calls nothing of its own to detect the CPU - with
SOFTFILTER_SIMD_AVX512 added to softfilter.h as RetroArch's
RETRO_SIMD_AVX512 bit. The lane buffers for every worker are one
block made at create. As elsewhere, the tiers are built per function
and, under PlayStation clang or clang-cl, only where the baseline
enables them.
Per frame at 256x224 on one thread, AVX-512 Xeon:
original 10.0 ms
previous commit 7.5 ms
AVX2 3.85 ms
AVX-512 3.67 ms
(AVX-512 adds less than its width: the passes are bound by how many
operations issue, and 512-bit ones have two ports to AVX2's three.)
samples/gfx/crt_filter_golden now runs its table at every decode tier
the CPU has, with one and seven workers (seven leaves lines over at
both lane counts). Its matrix is smaller, since every CRT frame costs
about the same whatever its size: the input formats alternate between
scenarios. The table is recorded again from the original
implementation, from before the previous commit. 6 s plain, 18 s
under ASan/UBSan with all three tiers.
samples/gfx/filter_plugins creates the filters with the SIMD mask
RetroArch would pass on the runner, so its concurrent 1-against-2-3-4
workers check, and its ThreadSanitizer and ASan runs, cover a filter's
wide paths too.
Mock CPU capabilities alongside the Switch harness frontend stubs. Check three horizon-driven handler passes within the lane deadline, with the scheduler timing allowance, so sanitizer scheduling delays do not fail a fixed-rate assertion.
Link the sinc dispatch harness with the string compatibility helper required by ARM CPU detection. Include scalar and NEON output comparisons so ARM runs exercise the tap-shape check.
Supply scalar CPU capability mocks in the isolated GL and recording harnesses alongside their existing frontend mocks.
At 256x224 the NTSC-CRT filter took about 7 ms a frame on one thread:
1.7 ms modulating the composite signal and 0.4 ms starting the decode,
both on the calling thread before the workers, then 5 ms decoding
lines on the workers.
Most of that was the compiler fetching and storing things it could
have held. The signal is signed char and the decoded output is stored
through int and unsigned char pointers, any of which may alias the
filters' state and the settings, so every store forced them back to
memory:
- The decoder runs the three-band equaliser for Y, I and Q on every
sample through struct EQF pointers. EQF_STEP now runs it on locals,
the same updates in the same order. The original eqf() is left for
the configurations that still call it.
- The decoder's pixel loop read contrast, blend and the output format
from the CRT struct for every pixel; they are read once a call.
- The modulator divided to find each sample's source pixel, switched
on the pixel format, ran its three IIR filters through globals and
recomputed the white level, which divides by 100, for every sample.
For RGB input, the format the wrapper feeds it, the source columns
are a table built once a frame, the filter history is in locals for
the line (it is reset at every line), and the levels are hoisted.
Other formats keep the general loop.
Everything is integer arithmetic and the operations are unchanged, so
the output is the same byte for byte. Per frame at 256x224, one
thread:
modulate 1.7-1.8 -> 1.0-1.1 ms (serial, before the workers)
decode lines 5.1-5.6 -> 3.7-4.2 ms
total 7.1-7.9 -> 5.1-5.9 ms
samples/gfx/crt_filter_golden holds the filter to output hashes
recorded from the previous implementation: noise off, default and
heavy, scanlines, blend, monochrome, hue, contrast, brightness,
saturation, black and white point, both input formats, several sizes
including frames smaller than the created maximum, one to four
workers and four frames in a row (field, frame parity and the noise
generator carry across frames); it runs plain and under ASan/UBSan
in the filter plugins workflow, beside the NTSC and 2xBR ones.
A memory search visited every item of the core's memory in turn: a
region lookup, a branch on item size, a loop over sub-items that at
byte size and wider runs once, and a branch on the search type, even
for items an earlier search had already ruled out - after a search or
two, nearly all of them. Each failing item also cost a memset call,
and the match count was reloaded and stored every item, since stores
through the uint8_t match bytes may alias it.
Now:
- Regions are walked in order, items at multiples of their size lying
whole inside one, as before, without a lookup per item.
- Runs of clear match bytes are stepped over 16 bytes at a time with
SSE2, 32 with AVX2 or 64 with AVX-512BW (chosen at run time from
cpu_features_get(); built per function, and under PlayStation clang
or clang-cl only where the baseline enables them), or a word at a
time elsewhere. An item is still judged by its first match byte: a
set byte part-way through one, which a change of item size between
searches can leave, rounds back to the item's start.
- At byte, 16- and 32-bit sizes the search type is chosen once and
each type gets its own loop, with direct stores for a failing item
and the count held in a local. The 1-, 2- and 4-bit sizes keep the
general path.
32-bit values were assembled as byte * 256 * 256 * 256 in int, which
overflows (undefined) for a top byte of 0x80 or more; they use
unsigned shifts now, the same value.
33 MB of memory on an AVX-512 Xeon, before -> after:
8-bit items: first search 270-344 -> 45-48 ms,
after one search 146-188 -> 91-126 ms,
after two 128-162 -> 9-12 ms
32-bit items: first search 75-125 -> 18-80 ms,
later searches 49-66 -> 5-11 ms
samples/cheats/cheat_search keeps the previous algorithm as its
reference and checks the match bytes, the match count and the saved
memory after each of 14400 chained searches: random region layouts,
every item size, both byte orders, every search type, memory changed
between searches in runs or scattered, the item size changed without
starting over, and the scan capped at each SIMD tier in turn.
samples/cheats/cheat_regions reports no SIMD, so it runs the baseline
scan.
A bzip2 member decoded to a file and neither mapped nor lent whole is
read into a ring of twice the most a block can take, kept that far
ahead of the decoder, instead of being read whole.
A stream can be handed over in two pieces, as a ring buffer wraps, and
again before each block from where rbzip2_taken() says it was taken
to. The bit reader moves onto the second piece in the slow path it
takes at the end of the input.
The bit buffer is filled when under 24 bits are left, with one load of
as many octets as it holds: 4-6% fewer instructions decoding.
rzstd_blocks_open(), _want() and _give() decode a frame handed over
in the pieces it asks for: the next block and the next block's header,
each decoded where it was read. A .zst file and a zip member stored
with Zstandard that are neither mapped nor lent whole are read a block
at a time into a buffer of one, rather than whole into memory.
Past a single segment the buffer a frame is decoded through holds
twice the window, so the window is moved down once per window's worth
decoded rather than once a block.
rzstd_raw_block_test decodes its frames in pieces too, and refuses one
a piece short; archive_zst_test and archive_zip_methods_test extract
their windowed frames read a block at a time where nothing is mapped.
A stored member extracted to a file goes out from the mapping, or what
the stream lends, however small, without being copied into a buffer
first. A member read to a file for a core that wants a path is written
out as it is decoded, as an extracted one is.
archive_zip_test reads every member of its sequence to a file, and with
the CRCs wrong gets none.
Every buffer a folder's output may end up in has an octet to spare
after it, so a member read whole that is its folder's entire output,
every member of a non-solid archive, is the folder's buffer itself,
handed over with a NUL after it rather than copied. Other members are
copied out of the folder once, as before.
r7z_archive_detach_test writes the spare octet of everything it is
handed.
The buffer a member streams through reaches as far past the window
again as the window, up to 64 MiB, and moves down once that much more
is final, so each octet is moved about once, and at every dictionary
size the point it moves at is one it reaches.
rrar_archive_extract() leaves an octet to spare after the member, and
the RAR backend hands that buffer to the caller with a NUL in it rather
than copying it.
rrar_archive_test streams with the buffer reaching 32 KiB past the
window, so it moves many times a member, and writes the spare octet of
every member it extracts.
A frame that carries one has its XXH64 taken over each block as it is
decoded, where the block lies, and checked against the low four octets
at its end, whole or a block at a time. A frame without one costs
nothing more. Everything that builds encoding_rzstd.c builds
encoding_xxh64.c beside it.
archive_zst_test reads and extracts a frame from libzstd whose checksum
is a bit out, and gets nothing either way.
xxh64() and xxh64_reset()/update()/digest() give XXH64, and xxh3_64()
the 64-bit XXH3, seeded or not, as xxHash specifies them. Past 240
octets XXH3 runs on AVX-512, AVX2 or SSE2, whichever the CPU has, or
NEON, or scalar code.
XXH3 adds each stripe's octets to a sum of their own and swaps that sum
into the accumulators once a block, so a stripe is five operations and
each accumulator one add long. A seed does not make a secret of its own
on AVX-512: every key is made in a register once per input. Each stripe
asks for the data eight and sixty-four stripes ahead.
xxhash_test checks both against xxHash's own values, XXH64 in pieces of
every size, and each XXH3 path the CPU runs against the others at every
length up to three blocks and every alignment.
features_cpu.c copies the CPU's model name with strlcpy() on the
architectures that read it from /proc/cpuinfo, so every sample that
links it links compat_strl.c too; fast_cores_test includes it beside
features_cpu.c. They link on PowerPC and ARM as well as x86.
Each frame the packet setup filled a shared buffer with one YUV key
per source pixel on the calling thread before the workers started,
and every pixel then clamped all of its 25 key lookups and 9 pixel
loads to the frame edge and loaded the whole 5x5 neighbourhood, even
where none of its four corner rules could fire.
Now:
- Each worker fills its own key rows: its slice of the frame plus the
two rows either side, padded by two keys at both ends, with rows
past the frame and the padding repeating the nearest edge key. That
is the clamp the neighbourhood reads through, so the key lookups
need none, and the key pass runs on the workers in parallel instead
of serially beforehand. The buffers are sized at create from the
largest frame and thread count; a frame beyond that size (which
the softfilter contract rules out) is pixel-doubled as before
rather than growing a buffer mid-frame.
- A corner rule needs the centre pixel to differ from both pixels
beside that corner. When no corner can fire, the four output pixels
are the centre pixel and are written without loading the diagonal
pixels or any key. Flat areas, the bulk of a game frame, take this
path.
The corner rules and blends themselves are untouched, so the output
is unchanged byte for byte. Per frame at 256x224 on one thread:
game-like frame RGB565 870 -> 546 us
XRGB8888 844 -> 373 us
noise RGB565 4260 -> 3620 us
XRGB8888 3090 -> 2414 us
samples/gfx/xbr_filter_golden holds the filter to output hashes
recorded from the previous implementation: game-like frames and
noise, both formats, sizes from 1x1 to 320x240, frames smaller than
the created maximum, input rows at a wider stride, and one to four
workers; it runs plain and under ASan/UBSan in the filter plugins
workflow, beside the NTSC one.
The NTSC-Multi-System filter decoded each output sample by summing 12
(16 for Atari, 8 for C64) products of the composite line and the
carrier, taking an integer modulo for the carrier phase and clamping
the position for every product, then dividing twice by a run-time
value. At 256x224 that cost about 5 ms a frame on one thread.
Each product depends only on the input position, not on which output
sample reads it, so it is now computed once per position into a line
padded with the edge sample (the clamp the window and the notch read
through), and each output takes a running window sum: two adds and
two subtracts in place of the products, modulos and clamps. The
carrier phase advances by a step and wraps with a compare instead of
a modulo, in the encode as well. The window pass is instantiated per
system so the divisor is a constant the compiler can turn into a
multiply or shift; C's truncating division is kept.
Everything is integer arithmetic, so the output is unchanged byte for
byte: compared against the previous code in every system mode, with
sharpness, hue/saturation and both artifact extremes, both formats,
widths 1 to past the 1024 cap, one and three workers and consecutive
frames (1560 cases).
256x224 RGB565: 4.8-5.2 ms -> 1.0 ms per frame
256x224 XRGB8888: 4.8-5.0 ms -> 1.0 ms per frame
The YIQ cache the old encode filled and never read, and the separate
composite buffer, are gone from the per-worker scratch.
The sharpness pass shifted a negative value left (Y << 1), which is
undefined; it is Y * 2 now, the same value on every target.
samples/gfx/ntsc_filter_golden holds the filter to output hashes
recorded from the previous implementation over the same matrix, plain
and under ASan/UBSan, in the filter plugins workflow.
A client behind a lockstep host can be handed a state labeled with a
frame it has not reached yet, and when that state is read after the
frame has advanced the forced rewind loads it one frame past run. The
rewind now makes the state's frame the one run next, so the client
never runs an earlier frame's input on top of the host's state and
the resync actually mends the desync.
samples/network/netplay_e2e gains a lane where the client runs slower
than the host and has its RAM upset, and NETPLAY_E2E_PACE_MS to set a
peer's pace.
As the other run-time SIMD units: Clang's immintrin.h for PlayStation
targets and clang-cl declares only the intrinsics the whole build
enables, so the AVX2 and AVX-512 conversions are built there only when
the baseline carries them.
Clang's immintrin.h for PlayStation targets (and clang-cl) declares
only the intrinsics of ISAs the whole build enables, so the AVX2 and
AVX-512 kernels that sinc_resampler, pixconv, float_to_s16 and
state_manager compile under per-function targets do not exist there.
On those compilers a tier is built only when the baseline carries it:
the PS4 keeps sinc's AVX kernel, the PS5 keeps the AVX2 tiers, and the
run-time choice never picks a tier that was left out.
convert_s16_to_float widens every s16 core's audio before the
resampler, on every flush. It gains AVX2 and AVX-512 forms of the SSE2
loop, picked by convert_s16_to_float_init_simd() from
cpu_features_get(), built per function so older CPUs keep SSE2 and
compilers without per-function targets (before GCC 5, Clang 4, MSVC
2017 15.3) leave them out.
Each sample is an exact sign-extend and convert, then one multiply by
gain / 32768: the same single rounding as the SSE2 and scalar paths,
so every target gives the same float for the same sample. AVX2 takes
16 samples a step; AVX-512 masks its last step, so it has no scalar
tail.
One 534-frame stereo batch (1068 samples) on an AVX-512 Xeon:
SSE2 0.14 us, AVX2 0.07-0.09 us, AVX-512 0.08-0.10 us.
New libretro-common/samples/audio/s16_to_float pins each tier the CPU
has and compares it bit for bit with the scalar rule: every s16 value
at six gains, then every length up to 100 at odd offsets, with a guard
word after each buffer.
pixconv.c, scaler_int.c, float_to_s16.c, float_to_s32.c and
s16_to_float.c test __SSE2__, which GCC and Clang define but MSVC does
not. retro_environment.h translates MSVC's _M_X64 / _M_IX86_FP into
__SSE__ / __SSE2__, but none of the five included it, so outside a
unity build that happens to pull it in first (griffin), an MSVC build
of any of them - a core built for UWP or Xbox, say - compiled the
scalar paths only.
Include it at the top of each. Preprocessed with MSVC's macros in
place of GCC's, all five now take their SSE2 paths. Those paths use
only SSE2 intrinsics MSVC has had since 2010; the 64-bit-only
_mm_cvtsi128_si64 in scaler_int.c is behind __x86_64__, so MSVC x64
takes the portable branch beside it. pixconv.c keeps its existing
SCALER_NO_SIMD for MSVC 2013 and older. GCC and Clang builds are
unchanged.
convert_float_to_s16 narrows the resampled mix for every s16 audio
driver on every flush. It gains AVX2 and AVX-512F forms of the SSE2
loop, picked by convert_float_to_s16_init_simd() from
cpu_features_get(), built per function so older CPUs keep SSE2 and
compilers without per-function targets (before GCC 5, Clang 4, MSVC
2017 15.3) leave them out.
Both run the SSE2 arithmetic step for step - scale, NaN to zero,
copysign(0.5) bias, clamp, truncate - so every target still gives the
same s16 for the same float. AVX2 converts 16 samples a step and fixes
the per-lane pack order with one permute. AVX-512 masks its loads and
uses the masked saturating narrow-and-store for the last step, so it
needs no scalar tail.
One 800-frame stereo flush (1600 samples) on an AVX-512 Xeon:
SSE2 0.45 us, AVX2 0.24 us, AVX-512 0.22 us.
New libretro-common/samples/audio/float_to_s16 pins each tier the CPU
has in turn and checks it against the scalar rule on exact .5 points,
full scale and past it, infinities, NaNs of both signs, denormals and
random bit patterns, at every length up to 100 and at odd offsets,
with a guard word after each buffer.
conv_rgb565_argb8888, conv_rgb565_abgr8888 and conv_0rgb1555_argb8888
are what the SDL2, SDL3, GL1 and DXGI upload fallbacks and the scaler
run every frame for 16-bit cores. They gain AVX2 and AVX-512BW forms
of the SSE2 arithmetic: 16 and 32 pixels a step, with a permute to put
the per-lane unpacks back in pixel order. The AVX-512 form also takes
each row's tail through masked loads and stores instead of a scalar
loop.
They are built per function and picked once at run time from
cpu_features_get(), so older CPUs keep the SSE2 path and compilers
without per-function targets (before GCC 5, Clang 4, MSVC 2017 15.3)
leave them out. Only RetroArch's own build (RARCH_INTERNAL) or one
defining PIXCONV_RUNTIME_SIMD compiles them, so cores that carry
pixconv.c without features_cpu.c build as before. The cached tier is
published with a release store, as converters run on several threads.
Microseconds per frame, RGB565 -> ARGB8888, on an AVX-512 Xeon:
size sse2 avx2 avx512
256x224 14.2 8.2 8.3 -42%
320x240 18.8 10.4 10.4 -45%
640x480 75.7 44.1 41.3 -45%
1280x720 221.0 195.2 198.3 -10% (bound by memory writes)
The ABGR and 1555 forms gain 28-42% at the same small sizes.
pixconv_test_wide builds the converters in with a test hook that caps
the tier, runs every 16-bit input through each tier the CPU has, and
every row width from 1 to 100 with a guard word after each row, so
every tail length is checked for writing past the row.
Every input frame was split into the planar L/R rings, each sample
stored twice so a window stays contiguous: four scalar stores per
frame, about a quarter of the resampler's time at Lowest to Normal.
New SSE, AVX and AVX-512 kernels read the window straight out of
data_in, interleaved as the core gave it. Only windows reaching back
past the start of a batch read from a seam: the last 'taps' frames of
history plus the first 'taps - 1' of the batch, copied once per batch.
The seam fits in the 4 * taps floats the rings use, so the shadow
copy, reset and stage restart work unchanged.
The window is read oldest first, so for these kernels the phase
tables are reversed at init and laid out in blocks where one unpack
of a block with itself gives the coefficients doubled up per L/R
pair, in frame order, with no cross-lane shuffle. A trailing block of
8 taps under AVX-512 is doubled with one permute instead.
That unpack competes with the multiply-adds for a port on long
filters, so the ring kernels stay for those. Each instance picks at
init by tier and tap count; the crossovers were measured: SSE up to 16
taps, AVX up to 128, AVX-512 up to 192. C and NEON are unchanged.
ns per output frame at 32040 -> 48000 Hz, 534-frame batches, paired
against master on an AVX-512 Xeon:
quality tier taps before after
lowest sse 8 5.89 3.83 -35%
normal sse 16 8.29 7.00 -15% (the default)
higher avx 64 19.10 14.59 -23%
higher avx512 64 19.45 14.72 -28% to -50%
highest any 256 unchanged (ring kernels)
sinc_dispatch now checks each SIMD tier separately against the scalar
kernel, both with one batch and with a stream of uneven batches (some
shorter than the filter), so history is carried across the seam. With
the seam's carry-over disabled the test fails, as it should.
Rollback keeps a savestate for every buffered frame and serializes every
frame, which a state in the tens of megabytes (Dolphin) cannot afford,
and needs states that restore deterministically enough to rewind on.
Lockstep runs a frame only once every player's real input for it has
arrived and never rewinds, so all delta frames share one state, used
only to hand a joiner the game or to resync. Desync checks hash the
core's system RAM, or the RAM in its memory map, rather than a whole
state; a state is serialized for them only when the core shows neither.
Input latency rises by a frame whenever lockstep has to wait and falls
again once input keeps arriving early, so it holds full speed over a
slow link. A host keeps sending while it waits on a joiner, grows its
buffers if the core's state has grown, and starts on the running game
instead of reloading it. A client takes a state labeled with a frame
whose input it already has, and loads it there.
Lockstep is a local choice and plays with rollback peers of this
protocol, and it is on by itself for cores past 32 MB of state and for
cores whose savestates load but cannot be rewound on, which netplay
used to refuse. The Lockstep setting asks for it everywhere. Older
clients cannot join a lockstep host.
samples/network/netplay_e2e runs a host and a client over the loopback
on a test core: sticks synced or not, lockstep, a lockstep host with a
rollback client, a desync mended and a 40 MB state.
Rewind diffing (find_change for the next dirty word, find_same for the
end of a dirty run) gains an AVX-512BW tier next to SSE2 and AVX2, and
find_same gets the AVX2 form it lacked. Tiers are picked once from
cpu_features_get(); RETRO_SIMD_AVX512 promises BW.
Three things make the wide scans pay off rather than lose to SSE2:
- The two blocks are now 64-byte aligned and a whole number of lines
apart, so a scan position has the same offset within a line in both.
The wide loops step back to a line boundary inside bytes already
proven equal, and from there no load straddles a cache line on
either block. Before this, AVX-512 lost to SSE2 on dirty states.
- Each scan starts narrow (16 bytes, then 32) and only then goes to
full width, since most scans in a dirty state end within the first
few words.
- raw_compress is compiled once per tier with the scans inlined. A
target function cannot inline into baseline code, and a call per
scan cost more than the scan on dirty states.
All tiers write byte-identical patches. Median time against SSE2,
paired runs on an Emerald Rapids Xeon (lower is faster):
state pattern avx2 avx512
16K clean 0.49 0.28
64K 1 dirty in 4096 0.60 0.45
256K 16-word runs 0.61 0.49
256K 1 dirty in 256 0.70 0.61
1M clean 0.85 0.84
any 1 dirty in 64 ~1.0 ~1.0-1.05
4M+ any ~1.0 ~1.0 (memory bound)
samples/state_rewind allocates its blocks the same way and checks that
every tier the CPU has writes the same patch over 300 seeded pairs,
with the sentinel and scan-pad cases re-run under each tier.
The AVX kernels were only built when the whole build had __AVX__, and
no RetroArch build passes -mavx, so shipped x86 builds never ran them
and Higher/Highest/HQ always used SSE.
Build the kernels whenever the compiler can target a single function
(GCC 4.9+, Clang 4+, MSVC 2010+ for AVX and 2017 15.3+ for AVX-512)
and select them at init from the mask cpu_features_get() fills in.
The rest of the file stays at the build's baseline, so CPUs without
AVX never execute an AVX instruction, and older compilers leave the
kernels out and keep the SSE/C paths. MSVC clears the upper vector
halves on return explicitly; GCC and Clang do it for target functions.
Add AVX-512 kernels: 16 taps per step with fused multiply-add, a
masked load for a leftover block of 8 taps instead of a second loop,
and both channels reduced and stored as one 64-bit write. They take
over where AVX is used today (Higher, Highest, HQ); Lowest, Lower and
Normal stay on SSE, which is still faster at their tap counts.
Measured on a 2.1 GHz Xeon with AVX-512, ns per stereo output frame:
quality taps sse avx avx512
higher 64 26.6 23.1 18.2
highest 256 77.3 60.3 41.8
hq 384 147.1 107.4 86.6
sinc_dispatch now runs every kernel the CPU supports against the
scalar kernel at three ratios and two bandwidths, one of which gives
a tap count that exercises the masked tail.
AVX2 was reported from the CPUID bit alone. AVX2 runs on YMM
registers, so on an OS or VM that does not save YMM state (no
OSXSAVE, or XCR0 without the AVX bit) code dispatched on
RETRO_SIMD_AVX2 would fault. Require the same ymm_state check AVX
and FMA3 already use.
The AVX-512 check called xgetbv without first proving OSXSAVE is
set. xgetbv is an invalid instruction until the OS enables XSAVE, so
a CPU with AVX-512F under such an OS crashed during feature
detection. Test ymm_state first.
RETRO_SIMD_AVX512 now means the x86-64-v4 baseline: F, CD, BW, DQ
and VL. Code dispatched on it can use byte/word ops and 128/256-bit
masked forms, which is what pixel conversion, rewind diffing and
audio work need; F alone is useless for those. Every AVX-512 CPU
except Xeon Phi has all five. The Darwin path checks the matching
hw.optional keys, since macOS enables ZMM state lazily and xgetbv
cannot decide it there.
Netplay sent a joypad port's buttons only, so a core reading the sticks
of a joypad port (N64, GameCube and others) desynced or saw them at
rest on every remote player. The host now sends joypad ports as analog,
three words instead of one, with bit 30 marking them; each side hands
the core the joypad device it set, so nothing changes for the core.
This is protocol 8. A host with Sync Analog Sticks on, the default,
turns away older clients with the out-of-date message rather than
letting them desync; with it off, any version still joins.
The scanner treats .rar as an archive, and .bz2 is a compressed file
the way .zst is, so a .bz2 is opened even when no core lists it; ".7Z"
is an archive as ".7z" is.
database_scan_test scans a 7z, a RAR, a RAR 5 with BLAKE2, a .zst, a
.bz2 and a zip with a Zstandard member, each matched through its
member, on both queues.
file_archive_get_file_crc32() asks for a member's CRC with crc_wanted.
The .zst backend decodes the frame a block at a time for it, and the
RAR backend unpacks a RAR 5 member that carries a BLAKE2 hash in place
of a CRC.
archive_zst_test and archive_rar_test check the CRC of a .zst member
and of a RAR 5 member with BLAKE2.
A touchscreen reports each finger as an ellipse, not a point; Android
gives its axes and orientation (AMotionEvent_getTouchMajor/Minor and
getOrientation). A thumb laid across up and right, or resting between
two face buttons, covers both - and read by its centre alone presses
one. After RetroArch-Lite's contact-area input.
"D-Pad Input Method" and "ABXY Input Method" choose how each reads a
touch: "Direction" - the way the centre lies, as always, and the
default - "Contact Area", every way the contact covers, or "Both".
Each tip of the ellipse's two axes presses the way it lies from the
area's middle, past a small dead zone there. "Contact Area Size"
scales how large a contact counts. Measured in pixels per axis against
the pointer's own space; RetroArch-Lite measured both against the
screen's height, which skews a contact on a wide screen.
Android's reader keeps each pointer's contact beside its position, and
the driver hands them to the frontend each poll
(input_driver_publish_touch_contacts). A finger with no contact size,
and every other driver, read by the centre's way. The rows show on
Android, where an overlay has the area they apply to.
overlay_contact_test holds the geometry: fingertips, thumbs across two
ways, upright and level contacts, magnification, wide screens.
Give the RSX context and render interface proper IDs in libretro.h
(RETRO_HW_CONTEXT_RSX = 11, RETRO_HW_RENDER_INTERFACE_RSX = 6) and ship
one interface, version 1, with retire_memory in it, instead of two
versions picked through the context's version_major.
Retired memory goes on a ring and is covered by one label write per
frame, with no wait-for-idle and no flush per call.
Font vertex pages are all allocated with the font, and batches bigger
than a page are drawn in page-sized pieces, so nothing is allocated
while drawing.
When the menu shows over a duped frame from a hardware core, the core's
image is drawn behind it rather than the software texture. The frontend
state reset runs only on frames the core rendered.
The host mapping default stays 32 MB; RSX_HOST_SIZE still overrides it.
The PSL1GHT guards around the new code are dropped: the driver only
builds with HAVE_GCM, which only Makefile.psl1ght sets.
Expose an experimental libretro RSX context and versioned rendering
interface for PSL1GHT cores. Present core-owned RSX textures without CPU
uploads, share the command FIFO and unused host mapping, and defer
allocation retirement until queued GPU work completes.
Integrate context negotiation and enforce main-thread rendering. Restore
frontend GPU state before compositing and use fenced per-font vertex
pages to prevent oversized Ozone batches from corrupting queued text.
Invalidate the vertex cache before menu and font draws.
Make the RSX host allocation configurable at build time. Extend regression
coverage for hardware image validation, deferred resource retirement,
oversized font batches and allocation failures.
Run codecs, other formats and core samples in separate jobs, retaining every directory and assigning follow-up checks to one group. Bound parallel builds and keep per-group compiler caches.
Register the MP4 VP9 hardware-stream test for execution rather than leaving its built target unclassified.
Use a private prefix and the existing persistent-server helper inside one X display. Exercise both cold and warm test runs, preserving their exit status and bounding the complete Wine invocation.
rzstd_blocks_begin()/step()/free() decode a frame through a buffer of
its window and two blocks, which slides down keeping the window. The
frame stays whole in memory, as with bzip2.
Zip members stored with Zstandard and .zst files bigger than 256 KiB
are written out a block at a time. archive_zip_methods_test and
archive_zst_test extract a frame from libzstd with a 128 KiB window
whose matches reach nearly that far back.
The five symbols a stream gives per round are gathered into a word in
the order they go into memory, whichever the host's byte order.
archive_zst_test, run on a big-endian host, extracts frames whose
literals are in four streams.
rrar_archive_extract_sink() hands a member out in pieces: stored ones
straight from the archive, compressed ones from a buffer of the window
and room for a filter block, which slides down as output goes. A
filter block goes out filtered once it is all there. The CRC is checked
at the end.
The RAR backend extracts members bigger than 256 KiB this way.
rrar_archive_test streams every fixture through a small step and
compares it with whole extraction; archive_rar_test extracts the big
member and leaves no file when a byte of it is changed.
A member bigger than 256 KiB of a folder with one LZMA or LZMA2 coder
over one pack stream is decoded through a ring of the dictionary's
size, at most 256 KiB a step, and written out as it goes. Its CRC, or
the folder's when the member is all of it, is checked at the end.
Other folders are decoded whole.
archive_7z_test extracts LZMA, LZMA2 and Deflate members bigger than
that, and refuses one with a byte of its stream changed.
An LZMA2 dictionary reset, which 7-Zip writes at every block when it
compresses on several threads, forgets what came before without moving
where output goes. Literal and position contexts count from the reset,
and the byte before the window's start is its end.
r7z_methods_test extracts a two-block LZMA2 archive whole and in
slices, and decodes LZMA through a ring smaller than the stream.
The keypress haptic was held to the top of the next runloop iteration,
up to a frame after the press: the press can come on a core's libco
stack, where Java is not to be entered. A thread of its own - the shared
output writer, attached to the JVM by rjni as it first asks - now makes
the buzz the moment it is woken; waking it is no Java. The pending flag
is an atomic, cleared by the lifecycle handlers as before, and the
runloop still flushes it where there are no threads.
A movable desc's geometry went to the video driver every poll, moved or
not. On D3D11 each call marks the overlay's vertex buffer dirty, so it
was mapped and rewritten every frame; under threaded video each was a
packet through the ring. The last delta sent is kept per desc and the
geometry is sent only when it differs - 23 times across an 8 second
on-screen run where it was about 500 - and reset wherever the whole
page's geometry is set.
The poll copied the last state and cleared the new one, about 150 bytes
each, every poll: the overlay now keeps both and flips between them
(INPUT_OVERLAY_ST()), clearing only the new one's keys, sticks and
buttons. The state each touch is tested into is cleared only there too,
its touches being unused.
Each finger was read with three calls into the pointer frame; the frame
store answers down-or-not and where in one look
(input_pointer_frame_finger), with the driver asked as before when the
frontend does not hold the pointer.
With "Show Inputs" off, the overlay took each control's state from the
physical controller, as for "Physical": a movable stick's image was set
to the controller's stick - at rest - over where the finger had put it,
every poll, and never moved. Only "Physical" reads the controller now;
off reads the touches as "Touched" does, and shows nothing.
overlay_stick_on_screen_test runs the real binary under Xvfb with a
movable stick, drags it right and up, holds it and lets go, and looks
at where it is drawn each time, threaded video off and on.
Keep VP9 decoding and native frame leases on the existing asynchronous Metal owner path. Add optional decoder restart and MP4 hardware submission with bounded backpressure, terminal drain and retained-frame ownership.
Cover configuration, lifecycle, superseded renderer ownership, seek restart and EOF with portable contract tests. Add native Metal pixel/pipeline acceptance and a Vulkan MP4 reference oracle.
A member of these bound for a file goes to it a window at a time, as
stored and deflated ones do, rather than whole through memory.
rinflate64_process() decodes Deflate64 in pieces from input in parts,
keeping the last 64 KiB for matches that reach back past its output;
rinflate64_decode() is one call of it, and takes eight octets in a
load where it can, which makes it a fifth faster. An LZMA member
decodes through r7z_lzma_stream into a window of its dictionary's
size, written out from where it lies. rbzip2_begin_blocks() and
rbzip2_step_block() hand a stream's blocks over one at a time from the
decoder's own buffer; a bzip2 member's stream is still read whole.
archive_zip_methods_test extracts members of each bigger than the
window, the LZMA one's dictionary wrapping, and leaves no file for a
wrong CRC or a changed byte; r7z_methods_test decodes Deflate64 in
pieces through windows of 1 to 70000 octets, and a match to the first
octet and one before it; rbzip2_test takes a stream's blocks one at a
time.
The playlist it checks is not written is removed before that scan and
after the block, so a root an earlier run left it in does not fail
the check. make check runs the scan test twice in one root to keep it
so, and gives the budget test its sanitizer allowance when SANITIZER
is set, as CI does.
Some duplicates nothing can tell apart: a remapper (DS4Windows,
reWASD, x360ce) reads a controller and shows a virtual one of it, and
nothing in the virtual pad's identity says so. Only the user knows,
so the user chooses.
Information > Input Information has a row under each controller,
"Ignore This Controller": that model is left out by every joypad
driver that knows a controller's ids - raw input, DirectInput, the
hybrid driver, udev and the HID drivers - the way the controllers
Steam Input stands in for already are. The controller driver restarts
so it goes at once. Each model left out is listed after the
controllers, a row to read it again. The list is the
input_ignore_devices setting, written as SDL's
SDL_GAMECONTROLLER_IGNORE_DEVICES is, and read beside that variable.
input_device_ignored() moves into input_driver.c, as it now reads the
settings; the check itself stays in input_ignore_devices.h with the
list's helpers, which ignore_devices_test covers. The driver tests
that link a driver are given the frontend's function.
A DualShock 4 or DualSense on Bluetooth and plugged in to charge is
two devices to DirectInput, and both the DirectInput driver and the
hybrid driver's DirectInput pads gave it two ports. Raw input already
told the links apart by the pad's Bluetooth address; DirectInput gives
the device's HID path too (DIPROP_GUIDANDPATH), so the enumeration
reads the address the same way and leaves a pad's second link out.
The address read moves to win_pad_ident.h, shared by raw input and
DirectInput. It finds HID's two calls in hid.dll, so a build that
links no HID library has it, and it runs where a pad connects - on the
enumeration's task for DirectInput - never on a frame.
win_pad_ident_test now holds a pad on both links to one port under
every Windows joypad driver. The raw input rumble test counted every
library load as XInput's; it counts XInput's.
The static-core, Apple and Android jobs publish retroarch-precompiled/
and retroarch-repo/, which core pipelines and App Store packaging fetch
from the latest successful master job through cross-project needs.
Keep them for a week so they survive pipelines that end failed or
cancelled; the server-side size-based prune bounds the storage.
rbzip2_begin() binds a stream and where it decodes to, and each
rbzip2_step() decodes a block of it, its CRC checked, until the end and
the stream's CRC; rbzip2_decode() is the same steps run through. The
7z BZip2 stage pauses on them like the others, with a decoder of its
own, and its folder fills from the front for r7z_archive_decode_peek().
rbzip2_test decodes a stream a block at a time, with a buffer one
short and with a block's CRC wrong; r7z_methods_test steps a folder of
three BZip2 blocks.
7-Zip applies it to arm64 code: BL's word offset, and ADRP's page
offset within 512 MiB either way, made absolute, with the stream's
start from the coder's properties when it has them.
r7z_methods_test extracts 7-Zip's archive of an arm64 shared object,
undoes XZ Utils' filter of the same from 4096, and takes XZ Utils'
words that go back as well as forward and reach the ends of ADRP's
range and past them.
Members' sizes reach the archive callbacks as 64 bits, so 7z, RAR and
.zst members of 4 GiB or more are listed and extracted where the host
can hold them. The zip backend reads ZIP64: the end of the directory
through its locator, and members' sizes and offsets from the 0x0001
extra field, the offset kept in the walk itself where a pointer cannot
hold it. A stored member bound for a file goes through the window a
piece at a time, as a deflated one does, so a zipped disc image is
extracted without the whole of it in memory.
archive_zip64_test reads a small archive with every ZIP64 field forced
and a sparse one past 4 GiB every way, and refuses a short extra field,
a missing locator and a record past the end.
Connect the MediaCodec recording frontend to worker-owned encoders and
Vulkan capture without frame-path readbacks or new waits. Keep capture
leases through native retirement and defer cold queue work on contention.
Calibrate AAC presentation edits, omit an empty audio source, and drain
healthy recordings before video resets. Retain the teardown path for
failed or ambiguous GPU work. Include portable lifecycle, audio, file
and capture regressions.
On Windows a mapped file's prefetch hint is handed to the file's
read-ahead thread, which calls PrefetchVirtualMemory (or, before
Windows 8, touches the pages) while the caller returns at once.
PrefetchVirtualMemory is not documented to return before its reads
complete, and the reader is usually an emulation thread, so a cold
image must not stall it on the disk. Each hint starts past the read that
made it, and a range that goes on from the last continues where that one
got to, so no byte is requested twice.
vfs_prefetch_offthread_test stands a slow PrefetchVirtualMemory in
through VFS_WIN32_PREFETCH_HOOK and checks that no read waits on it,
that it never runs on the reader, and that the file is requested once.
fill_pathname_abbreviated_or_relative, filestream_copy_loop,
retro_vfs_copy_begin_impl, retro_vfs_dirent_is_dir_stat,
retro_vfs_dirent_stat_slow and cpu_class_by_value take their
PATH_MAX_LENGTH and per-CPU scratch from the heap, so each frame stays
within the 2 KiB libretro-common thread-stack budget; the largest of
them is now 400 bytes on x86-64 GCC.
The udev joypad driver lit nothing: it had no set_player, unlike the
other desktop drivers. It now finds the LED class devices under each
pad (its HID device, or an Xbox 360 pad's USB device) and lights them
as each console does: a DualSense's five player lights, a DualShock
4's light bar in the player's colour, a DualShock 3's four, a Switch
pad's four and an Xbox 360 pad's ring. A DualSense's light bar is left
alone, as are lights it never lit; ones it lit go out when no port
reads the pad. Four rings under one wireless receiver are not told
apart, and are left.
Writing a light can wait on the pad (a Switch pad answers a request),
so the rumble writer's thread writes them: the lights are opened where
a pad arrives and handed over through atomic slots, with the player
wanted an atomic of its own. Without threads the caller writes them,
as with rumble. A driver that adds a pad's lights just after its input
nodes is looked at again a few times.
The kernel makes these files root's.
pkg/linux/70-retroarch-player-lights.rules makes them writable by the
user; without it the pad is left as it is, and the log says so.
samples/input/udev_player_lights runs the real driver against a
directory laid out as /sys is, under ASan and TSan.
The DualShock 3 pressure and udev rumble tests zero a pad and set its
sensor descriptor to -1, but not the touchpad's, added since: it stayed
0, and every poll read stdin as the pad's touchpad. With stdin an open
pipe, as a CI step can have it, the read blocked until the 60 second
timeout.
Which of Windows' views reads a controller is now decided in one place
(input/common/win_pad_ident.h) instead of four copies that disagreed.
The XInput-hybrid driver matched "IG_" in capitals only, so an Xbox pad
whose raw input path came lower case ("ig_") was read through XInput
and DirectInput both: two ports. The check is now case-insensitive
everywhere, and Bluetooth paths are told the same way, classic and LE.
A DualShock 4 or DualSense on Bluetooth and plugged in to charge is two
devices to Windows. Raw input now reads the pad's Bluetooth address
from each link at connect (the serial string over Bluetooth, a feature
report over USB, on a handle opened with no access): the second link of
a pad already there is kept aside and not read; when the first goes,
it carries on in the same port with no disconnect. DirectInput gives
no path to tell them by, and is left as it was.
samples/input/win_pad_ident holds a table of real controllers - Xbox
pads on USB, Bluetooth and Bluetooth LE with paths in either case,
8BitDo X and D modes, Sony pads on one or both links, Switch Pro,
WinUSB pads, Steam Input's - to one port each under every Windows
joypad driver.
With vsync off and no present timestamps, the hold's "next vblank" is
only the video thread's last completion plus a period. Each frame's
due time was clamped to it, so whenever that thread ran late on a busy
machine the schedule slipped by the same amount and never caught up:
content ran 3-6% slow (57 fps for 60). The clamp now applies only to a
real display grid; the clock grid keeps exact content periods, as the
timestamp path already did.
The threaded video pacing-rate lane, which caught this as a "flaky"
failure on a loaded CI box, now measures from the gaps between frames
and leaves out gaps of two content periods or more - the machine
taking the CPU away, which no pacing can make up - and reports how
many. Every hold pattern a case has stays under that, so a wrong
period still fails. With both CPUs saturated, every case is now
within 1.1%.
Where Hotkey Enable is also bound to a game button (Select, most
often), the game saw it for the first frames of every hotkey use.
With "Hotkey Enable Tap Delay" set, the game buttons sharing the
enabler's button or key are kept from the core until it is known
whether a hotkey follows:
- let go with no hotkey used, the press is given to the core then,
for as many frames as it was held (two at least);
- held past the delay (real milliseconds) with no hotkey, it is the
game's from then on, along with the other buttons;
- used with a hotkey, the core never sees it, or stops seeing it the
moment the hotkey fires.
Off by default; when off nothing runs. The core's read tests it with
the core hold's existing branch. A frontend harness lane covers taps,
holds and hotkeys.
r7z_archive_decode_step() pauses PPMd and Deflate stages the way it
pauses LZMA's, and r7z_archive_decode_peek() shows their folders
filling from the front. PPMd decodes a slice of symbols per step;
Deflate binds the folder's buffer once and rinflate_extend_out() moves
the end of the output on each step, so matches reach back into the
buffer itself and a step costs what the one-shot decode does.
r7z_methods_test steps 300 KB folders of each, reading them part way,
and refuses one with a byte changed.
Zip members stored with Deflate64 (method 9), as Windows and 7-Zip
write large files, decode whole through rinflate64_decode, and LZMA
members (method 14) through r7z_lzma when the 7z backend is built,
alongside the zstd and bzip2 modes: read whole, extracted, and through
an entry source.
archive_zip_methods_test reads 7-Zip's archives of both every way, with
Deflate64 matches over 32 KiB back, and refuses a wrong CRC and an LZMA
header without 5 octets of properties.
Wake the cold cleanup worker when exported frames return and wait on returned fence readiness instead of polling every 10ms. Retain the notifier through slot publication so cleanup cannot free it during the final callback.
When video becomes unthreaded, retire the retained window before creating its replacement. Activating the replacement first can synchronously wait for the sleeping window owner. Keep same-thread window reuse and the existing cleanup wait unchanged.
Fall back to the Ubuntu HTTPS archive using package URLs and strong checksums from apt metadata. Bound both transfer paths and retain download errors. Exercise mirror success, timeout recovery, invalid metadata and failed verification before the VM test.
7-Zip writes all four. PPMd runs the rrar var.H model behind 7z's
range decoder, BZip2 goes to rbzip2, Deflate to the built-in inflate,
and Deflate64 to rinflate64_decode, a one-shot decoder with the 64 KiB
window and the extended length and distance codes.
r7z_methods_test extracts 7-Zip archives of each, whole and in slices,
and runs a Deflate64 stream with matches past 32 KiB and lengths past
258 octets.
The on-screen overlay is hidden while a stylus is in use and comes back
two seconds after it was last used. Any pen event counted, a pen merely
hovering over the screen too, so the gamepad overlay went away as soon
as an S-Pen came near it, and "Hide Overlay When Controller Is
Connected" did not keep it (#19824).
A hovering pen now counts only where hovering moves the pointer ("Stylus
Hover Moves Cursor"); otherwise only a pen touching the screen does. And
"Hide Overlay While Using Stylus", on by default, can be turned off to
keep the overlay on screen while the pen is used, as melonDS does.
A member is decoded a step at a time into the folder cache and taken
from there, where a stored folder of a mapped archive is the archive
itself: extracted to a file it is written from where it is held, with
nothing copied, and read whole it is copied out once, into the buffer
the caller keeps.
archive_7z_test lists, reads and extracts a solid, a stored and a BCJ2
archive 7-Zip wrote through the archive layer, mapped and not.
An entry source opens for a zip member stored with Zstandard or bzip2
too. Such a member does not stop part way, so the first read decodes
all of it, from the mapping or what the stream lends where it can,
straight into the caller's buffer, which has room to the member's end,
and checks it against its CRC. The extract path decodes whole members
through the same two helpers.
archive_zip_test and archive_bz2_test read every member through an
entry source, and refuse one whose CRC is wrong.
An entry source takes its member's header offset and sizes from the
central directory, and read them from the mapping unchecked. They are
now bounded by the archive's size at open, as the extract init bounds
them: a member that does not lie inside the archive opens no source.
archive_zip_test opens sources for members stored too long, packed past
the end and with their header past it.
A DualShock 4's or DualSense's touchpad clicks as one button, where a
PlayStation game tells its left half from its right, often as Select
and Start (#19369). Controller Touchpad has a fourth setting, "Halves as
Buttons": pressed in, the pad presses the RetroPad button set for the
half its finger is on - "Touchpad Left Half" and "Touchpad Right Half",
Select and Start by default - and both with a finger on each.
It needs no new button numbers, so no profile or core changes, and
works for every driver that reads a touchpad (hid, sdl2, sdl3, udev).
Once a poll, where macros and remapped buttons are pressed, the click
and the fingers are read where the driver keeps them; nothing is
copied that was not before, and with no port in the mode a frame has
nothing more to do. The frontend harness's touchpad lane holds it to
the halves, both at once, no click, a button set by hand, and nothing
in another mode.
Include the directory-walk declaration used by the shared wallpaper helper so GCC 16 can build the texture-menu tests. Capture Windows compiler output explicitly before reporting a failed build.
Save the current movie path, position and pause state before the viewer releases its player. Restore only the matching movie once its transport is ready, and discard saved state on media changes or cancellation. Cover delayed readiness and refusals in the shared controls tests, plus paused-position recovery through a real driver reset in all four menu harnesses.
From an archive in memory, a folder that is stored, or only Copy, is
held as the folder cache in place, whole or decoded a step at a time:
a member borrowed is the archive's own octets, and one extracted is
copied out of them once. Copy hands its input on as its output in any
chain. A member detached from such a folder is a copy, being the
caller's to free.
r7z_stored_test borrows, extracts and detaches the members of a stored
archive 7-Zip wrote, from memory and through positioned reads.
Zip members stored with method 12 are decoded with rbzip2, whole, from
the mapping, from what the stream lends or else read, as Zstandard
members are. A .bz2 file stands for the file named as it is without the
extension: a stream does not say how long it decodes to, so the member
is decoded when it is listed, its size and CRC then known, and kept for
the extraction that follows. "#" after ".bz2" names its member.
HAVE_RBZIP2, on unless configured off, builds both and the decoder;
RVZ's bzip2 groups take the decoder alone where it is off.
file_archive_whole() is the whole-file read the .zst and .bz2 backends
share.
archive_bz2_test lists, reads and extracts bzip2 zip members and a .bz2
file, and refuses members whose CRC is wrong.
file_archive_compressed_read() of a zip member whose data is corrupt
or fails its CRC returns failure, not success with no octets.
archive_zip_test reads every member of its wrong-CRC archive whole.
A block's inverse transform is a chain of loads from anywhere in it.
Each entry is now linked to the one before it as well, and the block
walked from both ends at once, so one walk's loads wait alongside the
other's; the runs are undone in a pass after. A block that is mostly
long runs of rank 0, whose walk stays near itself, keeps the single
walk with the runs undone as it goes.
rbzip2_decode_alloc() decodes a stream whose size is not known into a
buffer grown as the blocks come.
Steam Input, starting a game with a controller it translates, makes a
virtual pad of it and puts the real one's USB ids in
SDL_GAMECONTROLLER_IGNORE_DEVICES ("0x054c/0x09cc,..."), so that the
game reads only the virtual pad. SDL's games honour it; RetroArch's own
drivers did not, and read the controller twice: once as itself, once
through Steam.
The drivers that know a controller's ids at connect now pass over one
that is listed: winraw, DirectInput, the XInput and DirectInput driver,
udev, and the hid layer under every HID backend. A user can set the
variable too, for a controller another program mirrors. It is read
when a controller connects, never on a frame. ignore_devices_test holds
the list to how Steam writes it and SDL reads it.
From an archive in memory, the packed streams feeding a BCJ2 folder's
ports, whether decoded whole or a port at a time, are read in place
rather than copied out first; only an archive that is read has them
read into buffers of its own.
r7z_bcj2_test extracts two BCJ2 archives 7-Zip wrote, from memory and
through positioned reads, whole and in slices, and counts what
extracting from memory allocates against reading.
Object lists name libretro-common through $(LIBRETRO_COMM_DIR), so the
crypto check matched nothing; the variable is expanded before matching.
GameCube, Wii and PS2 may build crypto.o alone, the ciphers the
RetroAchievements hashes use, and are refused the KDFs, public-key
code and X.509.
Groups compressed with bzip2, in RVZ and WIA files alike, are decoded
with rbzip2; the decoder and its block buffer are made once for the
file. rvz_test writes RVZ and WIA images with bzip2 groups.
rbzip2_decode() decodes one whole bzip2 stream held in memory into a
buffer sized for it, checking every block's CRC and the stream's. Codes
up to ten bits decode in one table look, the CRC is taken eight octets
at a time, and the block buffer is kept from one stream to the next.
No file I/O.
rbzip2_test decodes streams bzip2 wrote and streams of every shape a
small writer in the sample makes, and refuses truncated, corrupted and
randomised ones.
The 3DS hashes decrypt CIA titles with CBC and NCCH ExeFS sections with
CTR. Wherever crypto.o is built, cheevos/cheevos_aes.c serves rcheevos'
AES interface on it in place of the bundled software AES: the same
contexts and the same 128-bit counter, on AES instructions where the
CPU has them.
aes_cbc_decrypt() has no chain to wait on, so on AES-NI and the ARMv8
AES instructions eight blocks go through the inverse cipher at once,
with the InvMixColumns round keys made once per call. @in and @out may
alias, and the IV is left holding the last ciphertext block so a caller
can continue the chain.
Limit the optimizer workaround to MSVC 2005 and restore optimization before the next function. Check both mixer configurations and adjacent compiler versions in CI, including a missing-guard failure control. The native VC8 build is still required to confirm the reported C1001 is resolved.
A macro was not started with netplay on: hosting with nobody connected,
one that was started never reached the core. It does now. A macro's
steps go in where a remapped button does, and netplay takes the local
player's input from there, as it is given to the core; so the host's
or a client's macro is sent and replayed like buttons pressed by hand,
for every player alike.
The frontend harness's macro lane hosts netplay and holds the core to
being given the macro as it is written, frame for frame; with the
macro kept from the core under netplay it fails.
Unpaced frontend iterations can exhaust sixteen polls before a WARP upload returns its slot. Keep processing frames under a two-second watchdog outside submit timing, and require all sixty submits. Preserve the descriptor handoff and zero-copy budgets, and stop the lane on a real recovery timeout.
The menu's hotkeys list edits the controller combinations
(input_combo_N) and macros (input_macro_N); the desktop companions had
no way to. View > Controller Shortcuts now lists them in Qt, Win32 and
Cocoa: each as the configuration writes it, by its number, then an
empty row to add the next under the lowest number free. A row's text
is set from what is typed; an empty text takes the number out; text
that does not read is refused and the row left as it was. How a row is
written, and what a combination can do, is shown beside it.
The rows come from the shared core (companion_core_shortcut_*), so the
three windows are thin. companion_core_test, and the Win32 and Cocoa
harnesses, add, refuse, and take out a combination through them.
A .zst archive is opened with the hints every archive is, so it is
mapped where the host can afford it, and its frame is decoded from
filestream_view() when the stream lends it whole: the mapping, a
read-ahead window or a frontend's mapping. Only otherwise is it read
into a buffer. Extraction decodes from the transfer's own stream.
Deflate slices and whole zstd frames are taken from filestream_view()
where the stream can lend them, an smb:// or nfs:// read-ahead window
or a frontend's mapping, and read into the slice buffer only where it
cannot. A zstd member's buffer is allocated only when it is read.
rmsgpack_read() takes each field from a window the stream lends with
the new intfstream_view(), and reads only what the window does not
hold. Buffered streams lend their window and file streams lend what
filestream_view() lends. The stream is left where the value ends.
Reading every entry of a 150,000-entry database through a cursor
goes from 1.31 to 1.61 million entries a second, with the same
values. The cursor is what the Explore menu and database lookups
read through.
rmsgpack_window_test writes values of every shape: integers of every
width and sign, and strings and binaries up to past the reader's
window. It reads them back through buffered streams with 4 KiB and
64-octet windows, a file stream mapped and not, and a memory stream,
which lends nothing. Every value must match and the position must be
right after each. It passes under ASan and UBSan.
The reader takes RVZ groups compressed with LZMA or LZMA2, and WIA
images stored as they are, with PURGE, LZMA or LZMA2. LZMA and LZMA2
decode through r7z_lzma_stream and r7z_lzma2 with the group's output
as the window. An LZMA stream marks no end, so it is decoded as far as
the group's exception lists say, then its data. Builds without the 7z
backend link the two decoders for it alone. cheevos hashes .wia files
through the same reader. bzip2 groups still fail to read.
rvz_test writes LZMA streams with an encoder of its own that codes
every octet as a literal, through a real range coder and adaptive
model. LZMA2 streams mix stored chunks with chunks that keep the
model, reset it, or reset it with new props. It reads 27 images,
under ASan and UBSan, under qemu on big-endian PowerPC, and on 64-bit
Windows under Wine.
Comments in crypto.c, lrc_hash.c and cheevos.c now describe only what the
code does.
A pad's motion sensors or touchpad can come after the pad. Two hotplug
paths took them, one per tag, and each made a udev device for every
connected pad to find whose HID device it shared. The pad keeps its HID
device's path now, and one path takes a node that comes or goes: it is
matched by that path and given its role as at connect
(linux_pad_members.h). A Wii Remote's accelerometer, which udev does not
tag, is taken when it comes after the pad too; it was missed.
The barriers around a DLSS, FSR or XeSS evaluate were scoped to the
compute stage: the output's last read ordered before a compute write,
its result made visible from compute, the core's inputs left to the
core's own fragment-and-compute barrier. Which stages a library uses
inside its evaluate is not ours to know, and on a GPU that overlaps
stages a write of the output from any other left the frame's display
sampling it part-written: now and then a frame shown without the
history, every edge aliased.
A full memory barrier now precedes each evaluate, the output's
barriers span all commands, and the output may be written by a
transfer. The Vulkan harness runs under the layer's synchronization
validation, with the stand-in library clearing its output through the
transfer stage, and records two frames in one command buffer: the
old barriers raise write-after-write hazards there, these do not.
A DualShock 4, a DualSense or a Wii Remote comes out of the kernel as
several nodes beside the pad's: motion sensors, a touchpad, raw reports.
At each connect the driver searched every input or hidraw device on the
system for each of them in turn - up to four scans - and kept those
whose HID parent was the pad's.
It now lists the pad's HID device's own nodes once, and a table
(linux_pad_members.h) gives each its role: the node tagged as an
accelerometer for the sensors, else the untagged ones in order, as for
a Wii Remote; the one tagged as a touchpad for the touchpad, never tried
as the sensors; the hidraw node for a DualShock 3's pressures. The
pad's own node takes none. linux_pad_members_test holds the choice to
that.
Decode extended Android primary/matrix pairs while retaining coded/native conflict checks. Convert SMPTE 170M SDR primaries to Rec.709 in the existing compute pass, writing sRGB or linear FP16 output. This admits the mixed color standard reported for untagged AV1 video without adding a pass or readback. Test metadata conflicts, unsupported HDR combinations and GPU pixels against a double-precision XYZ reference.
A format in libretro-common reads no file. formats/rvz opened its file
through filestream and read groups from it. Like rchd, the reader now
describes each range it needs as a request, and the caller supplies
it however it likes and says how much came:
while ((err = rvz_open_step(rvz, &req)) == RVZ_PENDING)
rvz_feed(rvz, pull(req.offset, req.dst, req.length));
rvz_read_begin(rvz, offset, dst, len);
while ((err = rvz_read_step(rvz, &req)) == RVZ_PENDING)
rvz_feed(rvz, pull(req.offset, req.dst, req.length));
A request names where its octets go. Where a read can be served from
the file as it is - a raw group stored uncompressed and unpacked -
that is the read's own destination, so nothing is copied twice, as
before. A short supply is asked for again from where it stopped. A
supply of nothing fails the operation, and the next read goes on as
before. An operation that asks returns, and runs again from where it
stood: work done before the request (groups already decoded into
their slots, the open stages behind it) is kept. The file's size comes
from its head, which is hashed, for the bound on group ranges.
rvz_new()/rvz_free() replace rvz_open()/rvz_close(). cheevos.c, the
only user, reads the requests through the VFS, straight into the
buffers they name.
Also moves the padding generator's 2 KiB state and a hash block's
per-sector tables off the stack into the reader. Frames were over the
2048-octet budget the PSP and GX threads allow (rvz_block_load 2288,
rvz_junk 2160); the largest is now rvz_block_load at 752 (x86-64 gcc).
rvz_test makes and reads its files in memory, supplying every request
from there and now and then short. It checks:
- that requests are made into the read's own buffer (1798 of them)
- that a supply of nothing fails the read, and the read after it is
whole
- that nothing is read before the image is open
It passes under ASan and UBSan, on 64-bit Windows under Wine, and
under qemu on 32-bit big-endian PowerPC. 900 runs on damaged files
through the memory supplier were clean.
rc_hash through cheevos' adapter, on the same images as before, gives
their ISOs' hashes. Wii is 8.4-9.3x faster than the old reader and
GameCube 1.03-1.13x, measured with the machine otherwise idle.
A decoder can finish or fail before presenting any picture, including after hardware recovery finds no software decoder. End these sessions without waiting for the movie-end deadline. Retain hardware recovery and the audio tail of movies that have shown a picture. Cover empty EOF and decode failure in inline and threaded playback.
DLSS and FSR weigh their history by the frame time delta they are handed.
A core that leaves it at 0 had the NGX backends pass 0 on, and FSR a
fixed 16.667 ms. With no time between frames the upscaler's weighting of
motion against time is undefined, and on a real display it showed as
single frames now and then presented without their history: the raw
jittered sample with every edge aliased.
The shared layer now supplies the period of the core's declared frame
rate when the core gave no frame time, which is what a presented frame
advances the content by whatever the clock did. The Vulkan and D3D12
harnesses check it reaches the evaluate.
Some controllers reach the OS as more than one input device: the Xperia
Play's pad is two, the SHIELD keeps its NVIDIA button apart, the GPD XD
its back button, the ARCHOS GamePad its sticks. The Android driver
merged them with two global ids, pad_id1 and pad_id2, in a chain of
special cases: one such device at a time, two members at most, and
nothing done when one went away.
A device group (input/input_group.h) is one pad made of several OS
devices. The first member to arrive has the slot, later ones are read
as it, in either order. A member leaving keeps the pad; the lead leaving
hands the slot to the next member; the last leaving ends the group.
Several groups stand at once. A rule names the model, the members'
names and each one's role: part of the pad, the controller proper (a
second controller is a pad of its own), or a part that only joins, like
the NVIDIA button.
The Android driver's special cases are now a rule table
(android_groups.h) and the two globals are gone. input_group_test holds
the table to the old chain on every model and name it tries, and the
groups to join orders, members leaving, two groups at once and roles.
Wait for the mock fetch callback without running the task consumer, so a completed download cannot commit before the fixture marks the save live. Keep the download pacing assertion under the same ordering. Use a local temporary directory and native path separators so the fixture also runs on Windows.
Pending recording startup uses CMD_EVENT_RECORD_DEINIT. Include command.h instead of relying on unrelated headers to expose the event enum; Orbis has no such transitive include.
cheevos_rvz.c is replaced by a small adapter in cheevos.c that hands
rc_hash the new reader's seek, tell and read. The console comes, as
before, from the disc header's Wii or GameCube magic word.
Against the old reader, on images whose ISOs are known:
- Hashes it got right come out the same. On Wii images rc_hash is
7.2 to 7.9 times faster, with rcheevos' own MD5 now most of what is
left. GameCube is about even, the time there being MD5's too.
- The old reader dropped a group's hash exceptions, so a Wii image
with any came out with the wrong sectors. With chunks over 2 MiB it
read one exception list too few from a short group, shifting that
group's data by two octets, and the hash came out wrong. Both now
come out as their ISOs do.
- It returned reads shorter than asked inside an image; rc_hash does
not loop on a short read.
rc_hash keeps the file reader it was given from one load to the next.
After an RVZ, a disc image of another kind was opened with the RVZ
reader and its hash failed. The reader is now set on every load: the
RVZ one for a file it takes, the default otherwise.
The reader needs AES to encrypt Wii partitions again. A build without
the crypto library, Wii and Wii U among them, links crypto.o for it
alone, without HAVE_CRYPTO or the rest of the library. HAVE_CHEEVOS_RVZ
still follows HAVE_RZSTD, and configure now turns it off with cheevos
or RZSTD.
RVZ is Dolphin's compressed GameCube and Wii image format. rvz_open(),
rvz_read() and rvz_close() read a file as the disc image it stands
for: every read returns the octets an ISO of the same disc has at that
offset. The code is C89, and was written from the format description
(WiaAndRvz.md), not from an existing reader.
- The disc is a sorted list of regions: raw data, rounded down to its
sector as the format requires, and each Wii partition's two data
segments. The header area comes from the disc struct, and gaps read
as zeros.
- Groups are stored as they are or with Zstandard (HAVE_RZSTD). Groups
of zeros are taken as zeros. RVZ packing is undone, padding included:
the generator's state is kept mapped to the octets it stands for, so
its output is copied out whole.
- Wii partition data is put back together a 2 MiB hash block at a
time. H0 runs over each group's run of sectors with
SHA1DigestMany(), a sector of zeros takes the zero KiB's hash, then
H1 and H2 are computed and the hash exceptions applied. The hash
areas and then the data are encrypted eight sectors at a time with
aes_cbc_encrypt_lanes(). A disc whose header says it is not
encrypted gets its hashes and no encryption.
- A raw group stored as it is, is read where it is asked for. A raw
group read whole is decoded straight into the caller's buffer.
- The heads, the disc struct and the partition entries are checked
against their SHA-1s, and nothing is read from past the end of the
file.
samples/formats/rvz writes RVZ files from made-up discs and reads them
back, octet for octet, in order, at random and across the end. The
writer follows the description too: its padding generator is written
out as the description gives it, and it hashes and encrypts a sector
at a time. The discs cover:
- GameCube and Wii, chunks of 32 KiB to 4 MiB, NONE and Zstandard,
stored and compressed groups mixed, groups of zeros
- padding cut at group ends and padding over many words
- one and two partition segments, the first a whole hash block and
not
- encrypted and not, exceptions in the hashes and in their padding,
and a short last hash block
A damaged head, a damaged disc struct and a cut-off file must not
open.
Each of these mutations of the reader fails the test: no exceptions,
the wrong padding shift, no NONE padding, the wrong IV, padding offsets
counted per segment, a short H2, unrounded raw data, one exception list
per group, and encryption ignoring the header. It passes under ASan
and UBSan, and on x86-64, 64-bit Windows (Wine), aarch64, armhf,
mipsel and 32-bit big-endian PowerPC. 3,600 runs on randomly damaged
files, the hashes made to agree, were clean under ASan and UBSan.
Every 64-octet block went through the context one at a time. Each was
copied into context->block, the state was loaded and stored around
it, and cpu_features_get() was asked again. The block functions now
take a run of whole blocks straight from the caller's buffer and keep
the state in registers across it. This applies on SHA-NI, the ARMv8
SHA1 instructions and the portable rounds alike. SHA1Input() hands
them whole blocks where they lie, and SHA1Digest() no longer goes
through the stream context at all.
SHA1DigestMany() hashes n messages of one length at a stride, the
form of a Wii sector's 31 H0 hashes, and of its H1 and H2 tables.
Where the hardware does the rounds, two messages run side by side,
because each round waits several cycles on the one before. Elsewhere
it is SHA1Digest() in a loop.
On SHA-NI (best of 7):
gcc -O2 clang -O2
1 KiB messages 722 -> 949 735 -> 1006 MB/s
one 2 MiB message 936 -> 1043 983 -> 1171 MB/s
620 octets 1118 -> 1354 770 -> 1387 MB/s
test_hash checks SHA1Digest() and SHA1DigestMany() against the stream
fed in small pieces, at every padding boundary and with unaligned
strides, plus the million-'a' vector. It passes on SHA-NI and, under
qemu, on x86 without SHA-NI, aarch64 with and without the
instructions (built in and dispatched), armhf, mipsel and 32-bit
big-endian PowerPC.
aes_cbc_encrypt() encrypts a CBC chain and leaves the IV holding the
last ciphertext block, so the chain can be continued.
aes_cbc_encrypt_lanes() encrypts several independent chains at once.
A CBC chain waits on the cipher block by block. On AES-NI and the
ARMv8 crypto extensions, eight chains, four or one are interleaved
with each held in a register of its own, so the waits overlap. The
portable path encrypts one block at a time.
The RVZ reader is the first user: it encrypts a Wii partition's 64
sectors a hash block at a time.
The register form matters. With the chains in an array indexed by a
count known only at run time, gcc kept them on the stack, and every
round of every chain waited on a store and a load besides the cipher.
That ran at 2.4-2.7 GB/s on eight 32 KiB chains. Held in registers
they run at 6.3-7.5 GB/s, and clang, which promoted the array itself,
gains about 2%.
test_crypto checks SP 800-38A F.2.1, then 1 to 13 chains with 128 and
256 bit keys, in place and not, against a block-by-block reference.
It passes natively under ASan and UBSan, and under qemu on aarch64
(with and without the crypto extensions), armv7 with NEON, armv6k and
mipsel.
The encoder searches a block for matches, then stores it raw instead
when compressing it does not come out smaller. The search moves the
three recent offsets the decoder keeps, and a raw block leaves the
decoder's as they were, but the encoder's stayed moved. The next
compressed block could then name an offset by a repeat code that the
decoder resolved to a different offset. The frame decoded, here and
with the reference decoder, to the wrong octets and with no error.
That takes a block of noise containing a match the search finds, too
short to pay for the block, followed by a compressed block that
repeats at that match's distance. RVZ groups of Wii partition data
built that way came back wrong. The bsv replay writer, rzip and
trans_stream compress through the same encoder.
The offsets are now saved before the search and restored when the
block goes out raw. samples/encodings/rzstd has the three blocks that
showed it. Every level failed there before the fix and passes after.
The cleanup killed the domain controller and slept half a second
before removing its directory. Samba's children can still be writing
under it then, and the removal failed with "Directory not empty". That
failure became the script's exit status, so a matrix that had passed
was reported as failing.
The cleanup now waits, up to five seconds, until no process names the
directory, killing whatever is left, and then removes it. The script
exits with the matrix's own status, not the removal's.
A pipelined fetch reads each reply's header and READ response first
and, for a successful READ of one of its requests, receives the data
where that request wants it in the caller's buffer. A signed reply's
signature is checked over the pieces; a sealed one decrypts its
header aside to route by, and its data where it landed - after the
tag for AES-GCM, with the MAC for AES-CCM. Every other message is
received whole into the connection's buffer, as before.
The SMB fuzz target opens and reads a file too, from a seed taken
from a guest session that reaches the READ.
The listener copies the input for each connection it accepts, but a
connection accepted after the iteration that made it returned - a
client that gave up, its connect still queued - copied an input the
fuzzer had freed. The TLS, SMB and NFS targets now clear the input
at every exit, so such a connection is answered with nothing.
aes_cmac_init/update/final, aes_ccm_stream_* and aes_gcm_stream_*
take a message in pieces that need not be blocks. The GCM stream
hashes the AAD and ciphertext piece by piece and runs the counter
mode at any octet offset, so pieces can be decrypted where they lie
after the tag is checked; GMAC is its AAD alone. CCM decrypts or
encrypts in order with the MAC alongside. They keep their chaining
state in locals for the duration of a call, and run as fast as the
one-shot forms or faster under gcc -O2, clang -O2 and gcc -Os.
test_streams checks each against its one-shot form over 400 random
messages cut into up to three pieces anywhere, empty ones included.
An original Xbox pad, or an adapter for one, is an XID device (USB
class 0x58), not a HID one, and no driver of Windows' runs it. A new
handler reads its 20-byte report: the D-pad, Start, Back and the stick
clicks as bits, the six face buttons and both triggers as a pressure
each, and the sticks. A face button or trigger counts as pressed past a
third of its travel; its pressure is an axis, which the built-in
profile binds, so a core that reads pressure has it. Rumble goes out as
output report 0 from the packet handler, when it changes.
On Windows the WinUSB scan takes such a pad by its interface class,
whoever made it, except the DVD remote's receiver and Steel
Battalion's controller. One with no driver at all is said once on
screen, and Information > Input Information offers to move it to
WinUSB, as for the Wii U GameCube adapter; one on XBCD or libusb is
left where it is.
Run driver and media acceptance directly and replay failures under GDB for diagnostics, retaining the original exit code and captures. Keep GDB=1 for explicit debugger runs. Link the media harness as a console program so its output reaches CI without a debugger. Both the unchanged baseline and candidate pass natively but can fail under GDB; do not substitute the diagnostic replay result for the primary run.
Defer recording until the first core frame after pending startup state loading. This lets the core publish its resulting AV information before the output is opened, preserving the requested filename instead of creating a header-only file and restarting into a dated recording. Cancel pending startup on stop, content close and quit. Exercise the actual startup functions and retain the auto-state-load harness link.
Quest sets PLAY_STORE_BUILD, which excluded it from the explicit 16 KiB linker settings. NDK 29 aligned its LOAD segments but left RELRO ending at a 4 KiB boundary. Apply both maximum and common page sizes by ABI for every flavor, including Quest and Play Store builds. Keep the APK alignment checks unchanged.
Independent Vulkan decoding reproduces displaced reference-picture blocks on this device and driver, while software and D3D11 decoding match the reference. Use the existing decoder fallback without changing the Vulkan renderer. Keep the earlier 610.47 AV1 workaround separate and test neighboring device and driver admission.
HAVE_NGX gated the whole temporal upscaler - the interface handed to
cores, the settings, the threaded wrapper's carry and the DLSS, FSR
and XeSS backends alike - under the name of one vendor's library. A
MetalFX backend on the Metal driver would have to be built under a
define that reads as NVIDIA's. The subsystem is now HAVE_TEMPORAL_
UPSCALER wherever HAVE_NGX stood, in the build files, the drivers, the
menu, the samples and the compile matrix; nothing changes in what is
built.
A core sets the temporal upscaler's inputs, then pushes the frame, and
the driver takes them inside its frame call. Under the threaded video
wrapper that call runs on the video thread, later: by then the core
has set the inputs of a newer frame, or they went to the frame before.
A frame was upscaled with another frame's jitter and motion vectors,
and the next shown without upscaling at all, raw and aliased: the
black and white outlines flashing in Ocarina of Time under DLSS.
The wrapper copies the inputs into its hardware ring slot at the push,
on the core's thread, and carries the copy to the driver on the video
thread before the frame call; the driver's takes then come from the
carry alone. The threaded video harness gains a lane that pushes
hardware frames with inputs behind a slow present and checks that
every frame drawn took its own.
Android batches motion events and hands them to the window at the next
display frame, so a stick move or a finger drag reached the input
queue up to a frame late. Ask for unbuffered dispatch of joystick and
pointer sources (API 30+), posted to the decor view since the request
only takes on an attached view.
Claude-Session: https://claude.ai/code/session_01YTBjv1trXRxAXvNwCt89Ka
Co-authored-by: Claude Opus 5.5 <[email protected]>
android_joypad_state() read the hats and axes of the pad mapped to the
port (joy_idx), but the buttons of the pad that has the port's own
number. With a port's device index set to another pad, the stick and
d-pad hat followed the setting and the buttons did not.
Claude-Session: https://claude.ai/code/session_01YTBjv1trXRxAXvNwCt89Ka
Co-authored-by: Claude Opus 5.5 <[email protected]>
* Android: read the input queue on a thread of its own
Android sends a window its next key only once the window has finished
the last one, and a NativeActivity gets every key twice: before the
IME, marked AKEY_EVENT_FLAG_PREDISPATCH, and after it. Read on the
main thread once a frame, with the first copy finished unhandled for
the IME, each key took two polls, about 30 keys a second at 60 Hz. A
pad mashed faster than that queued up in the system, and its presses
kept arriving seconds after it was let go.
A thread with its own looper now reads the queue. It copies each event
into an SPSC ring and finishes it at once, so Android sends the next
one straight away. The copy before the IME is the one read, unless the
system keyboard is open, when keys still go by the IME first. Volume
keys are left to the system and read once, after the IME. Until the
first input driver starts, events are finished unread. With the ring
full, the reader leaves events in the queue until the poll makes room.
The poll applies the copies in order through the same handlers. A key,
a finger or a mouse button can now go down and back up between two
polls, so the poll stops at the first event that would undo what an
earlier one did in the same poll and leaves the rest for the next
(android_poll_changes.h): a press shorter than a frame is still seen
for one. Mouse moves in one poll are summed.
The reader answers APP_CMD_INPUT_CHANGED once it has let go of the old
queue. The AMotionEvent calls looked up with dlsym are looked up once,
before the reader starts, since it reads them too.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YTBjv1trXRxAXvNwCt89Ka
* Android input reader: records of their own size, read where they lie
Each event went through the input reader's ring as a 312-byte record,
whatever it was, built on the reader's stack, copied into the ring and
copied out again by the poll. A key needs 44 bytes of it and a stick's
move 88.
A record is now as long as its event: a key ends before the stick
axes, a stick's move before the touch fields, and a touch, mouse or
pen carries only the pointers it has. The reader builds each where the
poll will read it, and the poll applies it there
(android_input_ring.h); nothing is copied after the event leaves the
input queue. A record never crosses the ring's end: the rest of a lap
is a pad the poll steps over. The ring is 64 KiB, some 1500 keys.
A stick's moves are its state: the poll keeps only the last of a frame.
The reader publishes what it writes once the queue is empty, and a
stick's move written over its unpublished move before it, as long as
the hat stays where it was, so a fast pad no longer fills the ring and
holds back keys behind it.
A hat, the D-pad of many pads, tapped between two polls was lost: the
poll's hold-back covered keys, fingers and mouse buttons only. Its four
directions are kept back the same way now.
android_input_ring_test holds the ring to reading in place, records
whole across laps, merging only unpublished moves, and two threads;
android_poll_changes_test has a hat lane.
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: LibretroAdmin <[email protected]>
Handle a terminal hardware job before scheduling its timestamp when no picture has been presented. A failed first job can carry a future timestamp and must not postpone software recovery or end-of-stream reporting.
Cover recovery and missing-fallback paths with a failed job timestamp thirty seconds ahead of the polling clock.
The UWP backend replaces vfs_implementation.c, so filestream_view()
needs its own definition there. It lends from an smb:// stream's
read-ahead window; local files are never mapped on UWP, so everything
else answers -1 and the caller reads.
filestream_view() points at up to len octets at the stream position
and moves past them, as filestream_read() would after copying them:
from the gets/getc lookahead, from the file's mapping, from a
frontend VFS's mapping, or from an smb:// or nfs:// stream's
read-ahead window. -1 lends nothing and moves nothing, and the
caller reads as before.
The scan CRC step, sha1_calculate(), the cloud sync MD5 and
filestream_matches_buf() take what can be lent where it lies and
read the rest, so a hash over a share with read-ahead on no longer
copies what the prefetcher already holds; on a mapped file they
consume the mapping as before. A mapping over 4 GiB is hashed in
blocks rather than passed as one length the hashers truncate.
vfs_view_test covers the lookahead, mapped, unmapped, update-stream
and frontend-mapping cases and sha1_calculate(); the smb:// and
nfs:// thread tests read a file through lends against its contents
with read-ahead off and on.
Use 16 KiB maximum and common linker page sizes on arm64 and x86_64 so sideloaded libraries do not require the Android compatibility mode. Check LOAD and RELRO segments in the media-enabled frontend and packaged 64-bit APKs, with ZIP alignment validation.
Include the AV1 backend registry when HAVE_RAV1 is enabled. Media-enabled Android unity builds otherwise leave its registration and session entry points unresolved.
Add a complete Android media link to CI so missing unity dependencies are caught alongside the individual source checks.
A core's transparency mask, the pixels to take from this frame over the
history (particles, translucent effects), went to DLSS as its
TransparencyMask, a parameter DLSS reserves and never reads, so such
pixels kept their history and left trails behind them. It now goes to
DLSS as the current color's bias mask, which DLSS weighs them by, on
Vulkan and Direct3D 12 alike.
filestream_vprintf() formats into a buffer the stream owns, grown to
the longest line printed on it and freed at close, so threads printing
to their own files share nothing and a line of any length is written
whole. A line past the buffer is formatted once more into room for it.
filestream_printf_test writes lines around and past the first buffer
and prints from two threads into two files, each checked for exactly
its own text.
A stream read on sequentially - one read call spanning two hunks, or
32 hunk loads in a row a sector at a time - starts a vfs_prefetch
thread with a file and an rchd decoder of its own. The thread decodes
the odd hunks, one window each, and the reader the even ones, so the
two split the work rather than race for it: a whole-track hash or CRC
decodes on two cores. A ready odd hunk is read from its window where
it lies; a miss decodes it on the reader as before and retargets the
thread. Audio tracks are swapped on whichever side decodes. Nothing
starts on one core, or for the few sectors an identification reads.
chd_stream_ahead_test reads data and audio tracks long enough to
start the thread - in bulk, a sector at a time and across seeks -
against their sector contents, and closes streams mid-decode.
The buffer rchd's requests are read into before they are fed belongs
to the stream, allocated at open: streams open and read on whichever
thread holds them, so two CHDs read at once - a scan and an
achievement hash, say - no longer share one. chdstream_getc() reads
the octet it returns and reports EOF at the end of the track.
chd_stream_threads_test opens and reads two images on two threads,
each checked against its own sector contents.
Recheck cancellation after capture retirement before publishing the final
state, preserving earlier errors. Gate capture release in the dispatcher
regression so cancellation at this boundary cannot report success.
The layer's window is where the frontend lays the frame out - aspect,
integer scale, custom viewport, cut where it leaves the screen - set
only when it moves. A rotation is turned by the CPU into the page. A
frame is drawn only into the page not on the screen, as the vsync
thread now only reports the flip that landed; without vsync a frame
meeting a flip in flight is dropped, and a framebuffer that cannot wait
for a vsync ends the thread instead of spinning. An XRGB8888 core is
lent the page to be shown next. Messages and statistics are blended
into the page; screenshots read it back.
Pages are XRGB8888 past the console whatever its depth, laid out only
where the framebuffer has room for two; the menu converts at the end
of the framebuffer, past them, and shows before any core. Init fails
cleanly off sunxi hardware or without a free scaler, and the layer
ioctls pass their arguments as the kernel reads them, unsigned long.
Lanes in samples/gfx/sunxi_display; docs/video-driver-sunxi.md.
The CPU drivers composing a core's views give the compositor the
rotation the layout was made for, so each view is turned in its own
place: drm from its own rotation (its views surface is never turned),
the others from the frame path's rotation snapshot. omap_init no longer
reads settings it never used. Lane in samples/gfx/drm_surface_lifetime.
video_views_cpu_t carries the quarter turns its views are drawn at:
each view is turned before it is scaled into the placement the layout
gave it for the core's rotation. dispmanx sets it; the other CPU
drivers leave it at zero. Lane in samples/gfx/video_views.
The message and the statistics are each drawn at the screen's
resolution on a DispmanX layer at the top, as large as the text: drawn
and uploaded only when the text, its place or its colour changes, into
the one of two resources not on the screen, which the next flip shows.
A layer that outgrows its resources gets new ones; the old ones go once
the flip that took them off has landed.
Each overlay image is an element of its own over a resource uploaded
once at load; the HVS crops, scales, mirrors and blends it, and a move,
fade or toggle changes the elements in the next flip's update. An image
switched off, transparent or off the screen has no element.
video_monitor_index picks the firmware display (default, HDMI 0, HDMI 1,
LCD, composite), falling back to the default when it is not there; the
screen size is asked of that display. Screenshots snapshot the screen as
composed and cut it to the viewport.
The frame's element goes where the frontend lays the frame out -
aspect, integer scale, custom viewport, cut where it leaves the screen -
and moves there with the next flip. A core's rotation is a quarter
turned by the CPU and a half by the HVS. video_smooth is taken at run
time and a nearest kernel is put back on exit; a smooth start leaves
config.txt's kernel alone. Without vsync, a frame that meets a pending
flip is dropped rather than waited for.
Rows reach the firmware at its own pitch: a lent framebuffer and the
threaded wrapper's frame memory are laid out at it, other frames once
into the surface. RGUI blends by its own alpha on a layer above the
frame. Lanes in samples/gfx/dispmanx_display; docs/video-driver-dispmanx.md.
DLSS's feature libraries, Super Resolution's and Neural Rendering's,
stay loaded once NGX has loaded them, so NGX set up again on the next
content's device or after a driver restart finds them in instead of
loading and checking them again. NGX's start-up, and the making of each
feature, is timed in the log.
Turning Neural Rendering on or off no longer restarts the video driver:
its device extensions are asked for (once a run) whenever its library is
in the DLSS directory, and the core asked about it then too. Turned off,
its model is kept, and turning it on again takes the next frame.
vfs_prefetch_view() is vfs_prefetch_read() without the copy: it points
at the octets in a ready window and returns how many follow there. Only
the reader recycles windows, so the pointer stays valid until its next
read, view or seek. A reader that decodes ahead into windows (a core's
CHD hunks, one window each) can then hand out sectors where they lie.
The prefetcher is built in every configuration, griffin included,
rather than only with the network backends: statically linked cores
read ahead through it too. Without HAVE_THREADS it is stubs.
A core that renders in steps, such as a multiple of the console's
resolution, can land outside the one DLSS mode's range and was shown
without upscaling, saying nothing. The request now names the range DLSS
takes over all its modes, and a frame outside its mode's runs in the
mode whose range it is in, the nearest to its size; a frame no mode
takes is said once in the log.
NGX's extension names are kept for the life of the process instead of
asked again, with its feature libraries loaded and checked, every time
a context is made. Neural Rendering's library is asked about only while
it is turned on; turning it on or off now restarts the video driver.
The harness core plays through the MIDI interface it is handed, a
note-on every retro_run while output_enabled() says so and a note-off
from retro_deinit, and a counting driver installed in place of "null"
records device opens, frees and the notes that reach it.
midi boot: a core asking in its load at boot (-L, before the first
drivers_init) plays through the device. midi requested: a core asking
during a staged load, on the old session's drivers, plays through the
rebuilt ones and keeps playing across a CMD_EVENT_REINIT (the
DRIVER_LIFETIME_RESET path SET_SYSTEM_AV_INFO takes); its note-off at
unload reaches the device; after closing content and after starting a
core that does not ask the output is shut with every open freed.
A staged content load runs the new core's init and load on the
previous session's drivers, then rebuilds them. The core's
GET_MIDI_INTERFACE request now ends in runloop_event_deinit_core after
retro_deinit rather than in driver_uninit, so drivers_init reopens the
device on the rebuilt drivers for a core that asked during its load.
The content_load harness gains a lane for it: a core asking at load
has the output open after the load, the next core that does not ask
has it shut.
* samples/database: include compat/strl.h in the scan test
The test calls strlcpy(), which mingw-w64 does not declare, so it did
not build for Windows.
* PS1 scan: PSX.EXE means no serial only as the BOOT target
A root directory can list a PSX.EXE ahead of the executable SYSTEM.CNF
boots: Tomb Raider (USA) lists one before SLUS_001.52. Stopping at the
first PSX.EXE in the probe window sent such discs to a whole-track CRC,
eight seconds for a CHD. Look for the BOOT target first, matched without
regard to case so cdrom:\psx.exe counts. A disc that boots PSX.EXE, or
has no SYSTEM.CNF so the BIOS boots PSX.EXE, has no serial and still
falls back to its CRC (#15241); any other disc is searched for its
serial with PSX.EXE directory records ignored.
* 3DS scan: identify .3ds, .cci and .cxi by the NCCH product code
These images took the default whole-file CRC, 0.5-4 GB a game. The
product code in the NCCH header (partition 0 of a cart image, or the
start of a bare NCCH) is the serial the 3DS database is keyed on, and
the header is plaintext in encrypted dumps too. A code held by one
record matches it outright. Revisions and regions that share a code fall
back to the file's CRC to choose, and to the first record when no CRC
matches; with scan_serial_and_crc on every match is confirmed by CRC, as
for PSP and Dreamcast, and a file none of whose records carry its CRC
goes on to the plain CRC lookup with the CRC already computed. A code no
record carries, or a header without valid magic such as a CIA, keeps the
CRC path, in first-match mode too. Like the CRC lookup, the serial pass
visits only databases an installed core claims the extension for, unless
Scan Without Core Match is on.
* scan: run ITERATE steps while the shared I/O window lasts
A file takes about twenty ITERATE steps, and the handler ran one per
call. On the regular task queue that is one step per frame: 1000 NES
zips took 335 s of pacing, 400 arcade sets 1639 s. The ITERATE states
now run in a loop under the same window the BEGIN states use, which
times the steps rather than counting them and grants every call at least
one. The scan test's unclaimed-database lane spends the window before
each check, so that it still counts steps.
* scan: hash file CRCs a quantum at a time across steps
A step that needed a CRC computed all of it in one handler call: a whole
CHD track with LZMA decode, or a multi-gigabyte 3DS image. On the
regular queue that froze a frame, and on the threaded queue it held the
worker for seconds. Each CRC site now opens its span and returns, and
later steps hash 256 KiB quanta while the shared I/O window lasts, at
least one per call. The lookup that wanted the CRC runs when it is in,
at the same entry. The one-shot CRC helpers, which nothing else called,
are gone.
* scan: walk each database once for its crc and serial indexes
The first file to reach the serial lookup built a crc index and then a
serial index for every database, a full walk each: 47 MB of Mobile -
J2ME twice in one handler call, the second for an index that came out
empty. The crc walk now reads serials as well. It notes whether a
database has any, so the serial lookup passes over one with none, and
for the serial lookup it builds the serial index in the same pass. Index
contents, matches and the split of the index budget are unchanged.
* scan: let a database's indexes be built a run of records at a time
A database's crc and serial indexes were built by one walk and a qsort
in a single call: 50-70 ms for Mobile - J2ME. libretrodb_scan_begin(),
_step() and _end() walk a run of records per call, and
database_info_index_job_*() builds both indexes over as many calls as it
takes, each walking a run of records or merging a run of entries in a
resumable merge sort. libretrodb_scan_field2() and
database_info_index_new() are built on them and answer as before.
* scan: build each database's indexes over steps
The first file to reach a database built its indexes in one handler
call: 52-68 ms on the regular queue for Mobile - J2ME, 13-17 frames'
worth of the 4 ms window. The build now runs over the steps that follow
the lookup that began it, a run of records walked or entries merged at a
time while the window lasts, and the lookup takes the indexes when they
are done. A database counts as size-checked only then. Cancelling frees
a build in flight. The one-shot builders, database_info_crc_index_new(),
database_info_serial_index_new(), database_info_index_new() and
libretrodb_scan_field() and _field2(), lose their last caller here and
go; the tests build through the job and the stepped scan.
* task_queue samples: build on macOS, run on GCD in a macOS job
No sample ran the GCD runner and nothing in CI built HAVE_GCD. Apple's
ld has no --wrap, so on Darwin the switch and init-failure tests
redirect the wrapped calls from task_queue.c by macro instead, and
libdispatch comes from libSystem there. The switch test gets a GCD=1
build, and a macOS job runs every task_queue sample plain, under
ASan+UBSan and TSan, threaded and, where it has a GCD=1 build, on GCD.
* task_queue: a switch waits out a wake from another thread
task_queue_wake() is documented as safe from any thread, but a switch
between the threaded and the unthreaded queue frees the threaded
runner's lock and condition variable while a waker on another thread can
still be inside it. Toggling Threaded Video makes that switch at the
next frame. Two seq_cst counters now keep them apart without a lock: a
switch that finds a waker inside is put off to the next
task_queue_check(), and a waker that finds a switch under way leaves its
task to it. A wake on the thread that drives the queue cannot meet a
switch and skips the counters.
A switch now ends every park, and the new runner takes wakes again
before it runs any task, so a parked task loses nothing to a wake
dropped during the switch. A wake dropped while a switch is put off and
then taken back, or for a task whose park a switch keeps because it
leaves a stuck handler behind, waits for that park's horizon.
The switch test holds a waker inside the queue across a switch, wakes
from four threads through 1000 switches, and holds the main thread where
the new runner first runs a task while that task's first pass wakes it.
GCD builds leave out the threaded runner's own code, which they never
used and which warned there. The switch test also checks that its
realloc() wrapper ran, since on macOS that wrap is a macro rather than a
link option.
* scan: a scan of one file only adds to its playlist
A Content Location that is a file is scanned as one file, which the Scan
button, --scan=FILE and iOS "open in" already rely on. Such a scan
cleared the playlist when Overwrite was on, and recorded the file, and
its core, as what the playlist refreshes with, so Refresh Playlist later
rescanned that file alone. Overwrite is now ignored for a file, and the
playlist's scan record and default core are left as they were. Validate
Existing Entries is skipped too: a file scan's extensions can differ
from its folder's, so validating with them could drop entries the folder
scan added. A later folder scan or Refresh still prunes. With no
database and an automatic target playlist, a file is filed under its
folder's name, as a scan of that folder files it, rather than in a
playlist named after the file. A Content Location that is missing, as on
a drive that is offline, counts as a file, so Overwrite no longer
empties its playlist while the drive is away.
* menu: Content Location takes a file or a folder
Content Scan's Content Location now opens one browser that lists folders
and files in every scan method: OK on a file picks it, <Use This
Directory> picks the folder. In Automatic it lists every file, as an
Automatic scan reads every file; Custom keeps listing the scan's
extensions. The Scan Single File toggle, which switched the browser to
files and was shown only in Custom with Search Recursively off, goes
with its setting and strings, and a scan of a file no longer turns
Search Recursively off behind the user's back. Import Content drops Scan
Directory and Scan File, as the Scan Directory list already had, leaving
Content Scan as the one way in; Scan File no longer started a scan. On
Apple, File > Open and the iOS document picker opened from Content
Location set the location as OK on a file does.
* task_queue: hold pushes from other threads for the main thread
task_queue_push() from another thread could meet a switch between the
threaded and the unthreaded queue, which frees the lock and condition
variable the push takes. On the unthreaded queue, with no switch at all,
it put the task on a list the main thread walks without a lock. Windows
pushes from the video thread under threaded video, and HID builds from
libusb's thread. Such a push now goes through the switch's gate: when
the gate is shut, or the unthreaded queue is up, the task goes onto a
lock-free list that the next gather pushes on the main thread, and the
push returns true. A push is also held while earlier ones still wait, so
a thread's pushes start in the order it made them. A BLOCKING task
refused there is cancelled, and its callback still runs. The switch test
pushes from four threads on the unthreaded queue and through 1000
switches, checks a held BLOCKING push, and holds the main thread inside
the gather that reopens the queue to show that a thread's next push does
not pass its held one.
* task_queue: cancel by ident and find safely from other threads
task_queue_cancel_task() and task_queue_find() from another thread met
the same switch and the same unlocked lists as a push did.
task_queue_cancel_ident() cancels by ident from any thread: a locked
walk when the threaded queue is up with nothing held, otherwise an ident
left for the next gather, which applies it after its pushes, to a held
push as well. A held push cancelled so retires cancelled without
running, and its callback runs. A find from another thread goes through
the switch's gate and reports false when it cannot walk the lists. The
queue's own workers and GCD threads skip the gate, and a find on the
main thread also sees held pushes. Each worker keeps its thread id in a
slot of its own and clears it before it reports that it left, so a
thread that reuses a joined worker's id is not taken for one. The switch
test cancels and finds from other threads on both queues and across a
put-off switch, and holds the main thread in the gather to show that a
thread's cancel of its own held push is not lost. Once extra workers
start, finds from handlers on the one worker and on an extra still skip
the gate, and a find with a joined extra worker's id from inside deinit
is refused.
* autoconfig: make connects reported on other threads on the main thread
libusb's hotplug thread and the Wii's HID poll thread report pads from
their own threads, where task_queue_find() can no longer answer exactly:
at a switch, or on the unthreaded queue, it reports false. Off the main
thread, input_autoconfigure_connect_ex() and
input_autoconfigure_disconnect() now copy the request into a main-thread
task and return true. Its callback makes the call on the main thread, in
the order the requests were made, so the duplicate check, the port's
sequence number and the settings it reads are the main thread's. A new
sample drives task_autodetect.c on the task queue with the rest stubbed,
reports pads from another thread, and checks that every find runs on the
main thread, that a pad reported twice queues one connect, that a
disconnect handed off is made and that a cancelled request is not.
* task_queue: wait, reset and deinit see pushes held for the main thread
On the unthreaded queue every push from another thread is held for the
next gather. A wait with nothing running returned without running it,
and a reset, the exit drain's included, left it to run normally
afterwards. Cancels held for the gather were never freed at deinit.
The unthreaded wait now drains held pushes first, and a reset drains
them into the runner before it cancels, so a held task meets a queued
one's fate. Deinit applies and frees held cancels; held pushes stay held
for the next runner's first gather, so no callback runs inside deinit.
* task_queue: make the Threaded Tasks flag atomic
Task handlers on the worker read it through task_queue_is_threaded(),
and the main thread writes it when the setting changes, so changing the
setting while a task ran was a data race. ThreadSanitizer shows it on a
database scan whose queue is switched under it.
* task_queue: a parallel task names the extra workers it needs
retro_task_t gains parallel_workers, which task_init() clears. A
parallel push starts the extra workers it finds missing, up to the
larger of task_queue_set_parallel_workers() and the task's number, at
most 8. They serve every parallel task and stay until deinit, as before,
so the pool only grows; 0 keeps the frontend's number. The started count
is atomic, so a push that finds enough takes no lock, and one that needs
more claims the next entries under running_lock.
The parallel test has a task asking for three, a later one asking for
fewer, one asking for more than eight, a plain task whose number is
ignored, the frontend's number holding for a task asking for fewer, and
a task that leaves the number alone.
* task_queue: let deinit leave extra workers behind in detachable handlers
A parallel task may now also be detachable, and then runs on the extra
workers like any other parallel task. Deinit gives every worker the one
bound it gave the one worker, the extra workers first. One still inside
a detachable handler past it is left behind to finish alone, as the one
worker was, and one inside any other handler is still waited for. The
primitives of an init are held by deinit and by each worker it leaves
behind, and the last of them to let go frees them, so the one worker and
several extras can be left behind together.
A worker left behind, the one or an extra, now runs the task's cleanup
once the handler returns, then frees the task, so a detachable task
releases what it holds however it ends. Its callback still never runs;
its cleanup, like its handler, may touch only what the task owns.
The parallel test runs a detachable parallel task on an extra worker,
leaves one stuck past the bound, waits for a slow one that is not
detachable, and leaves the one worker and two extras behind together
with a new queue running beside them. Each task left behind is cleaned
up once, by its own worker, after its handler returns, and one not left
behind at its retirement as before. The switch test switches to the
unthreaded queue and back with extra workers live.
* task_queue: on Emscripten, sleep after a pass on the one worker only
On Emscripten every worker sleeps 1 ms after each handler call, a
pthreads workaround from when the one worker ran every task. The extra
workers run parallel tasks, which work in many short passes, and the
sleep cost each of those passes 1 ms, more than the pass itself. The
extra workers no longer sleep; the one worker still does.
* emscripten: ask for at most 4 extra workers, and start 5 threads
In the browser a pthread the pool has not started is made on demand by
the main thread, and with the JS file system, where every thread's file
calls go through the main thread, 8 extra workers made each of those
calls about 40% dearer than 4 did. The frontend asked for cores - 3
extra workers, up to 8; on Emscripten it now asks for at most 4, and the
build starts 5 pthreads up front, enough for the one worker and those 4.
* scan: read small plain files on parallel reader tasks, in list order
A database scan on the threaded queue pushes min(4, cores) reader tasks,
none below 2, as parallel tasks that ask for that many extra workers, in
HAVE_THREADS builds with lock-free atomics, except on Wii U, whose C
library and FAT layer take no locks, and on Vita, where Vita3K's sceIo
tables take none. With Threaded Tasks off none starts and the scan
reads every file itself, stepped as before.
The scan hands the plain files and archives it would CRC whole to the
readers, up to 64 per reader ahead of it, each with its own copy of the
path. A reader claims them in list order, opens and sizes each, leaves
one over 128 MiB to the scan unread and CRCs the rest a quantum per
call. The scan takes each result at its own place in the content list,
so the databases see files in the same order as before, parks while the
next one is still being read, and reads itself whatever a reader could
not.
The scan never waits for its readers. At its end, on cancel and once
the queue is unthreaded, each reader leaves at its next call and hands
back the file in hand. Each reader drops its reference to what they
share in its cleanup, and the last reference frees it. A queue deinit
leaves a reader stuck in a read behind after its 1 s bound and never
calls it again; the worker left with it runs its cleanup once the read
returns, which hands back the file in hand and drops the reference.
The scan test covers list order, files over the cutoff, failed pushes,
cancels and quits with a reader stuck in a read, and queue switches
under a scan; CI runs it under TSan, without threads and with no
readers.
* scan: run the command-line scan with no menu on the threaded queue
A command-line scan with no menu waits for the task queue to run dry,
and only a scan on the threaded queue gets readers. In threaded builds
that run now uses the threaded queue whatever Threaded Tasks says; the
setting is not changed.
* menu: Import Content offers Scan Directory and Scan File
Both entries sit beside Content Scan when the database is built in.
OK on a file in Scan File's browser scans that file, as the Scan
action does. The load_content_filter harness checks that the
entries are listed and that OK on a file starts its scan.
* manual scan: Content Location holds a full file path
The location can now be a file, so it is kept in a PATH_MAX_LENGTH
buffer, as is the scan task's copy. A file whose path does not fit
is refused in the browser instead of being joined short. The
load_content_filter harness sets a location deeper than a folder
buffer and checks that it is kept whole.
---------
Co-authored-by: LibretroAdmin <[email protected]>
The helpers rslc prints for GLSL builtins MSL lacks (mod, inverse,
matrixCompMult, textureSize, findLSB/findMSB) are printed static inline
always_inline like the shader's own functions, so under Metal's fast
math they round as the same code inlined into main0 would.
* rslc: print pow as powr in MSL unless the base is a constant >= 0
The Metal driver compiles with fast math, where pow() of a negative
base returns pow(|x|, y). GLSL leaves a negative base undefined and
desktop GPUs return NaN, which is written as 0; SPIRV-Cross prints
powr() for that reason. The handheld colour-mod presets push a
channel slightly below zero, so on metal rslc turned it into colour:
a red bar came out 255,62,45 where glslang gives 255,0,0.
A constant base known to be >= 0 keeps pow(): the result is the same
and fast-math powr() is less exact there, which garbles
misc/print-resolution's text under SPIRV-Cross.
* rslc: print MSL shader functions as always_inline
The Metal driver compiles with fast math, where a function main0
calls can round a few ulp apart from the same code inlined into it.
SPIRV-Cross prints the shader's functions static inline
always_inline and rslc did not, so wherever a value sat on an 8-bit
rounding edge the pictures differed: xbr-lv3, misc/geom, hq2x and
others, and after the powr change blurs/sharpsmoother and
crt-royale-pal-r57shell.
Printed the same way, they match: on metal, 1128 of the 1166
non-bezel presets now give glslang's picture byte for byte, up from
989, and none that did stops.
DLSS 5's Neural Rendering model runs over DLSS Super Resolution's
output when its library, nvngx_dlssnr.dll, is in the DLSS directory.
It goes through the same NGX session as Super Resolution: the driver's
NGX core loads the library from the directory it is given at init,
creates the feature (18) on the frame's command buffer and evaluates
it on Super Resolution's output with the core's depth and motion
vectors at render size. Cores before its driver (616.64) have no such
capability; the session then marks it unavailable and Super
Resolution goes on alone.
Settings > Video > Scaling > DLSS Neural Rendering holds the switch,
off by default, and the model's tuning: preset, style, intensity,
local structure, local tone, skin structure and auto skin mask. The
entry is shown only with DLSS selected, the library found, NGX not
OptiScaler's, and no session having found the driver without it.
The settings reach the backends as one word. The model reads its
tuning only at creation, so a changed setting rebuilds the feature
once it has held for 30 frames; turning it off retires the feature
through the same ring as Super Resolution's. System Information and
the log name the library's version, read from its file without
loading it, or say the driver needs updating.
The upscaler samples drive both backends against stand-in NGX cores
that know the feature: the tuning at creation, the evaluate's inputs
and guide sizes, the rebuild after a change holds, the release when
off, and a core without it; the loader sample finds the library and
reads its version.
With one GL context handed between the core's thread and the video
thread, the owner word now also carries the waiting thread's ask and a
grant: a release goes to the thread that asked rather than back to the
free pool, so a take waits out exactly one hold of the other thread.
The push hands the context to the video thread together with the frame
it queues, whose pass takes the grant, and takes it back with a plain
take once that pass is done; there is no turn count or deadline.
gl_context_drain.py now runs the shipping take, release, drain and
command round trip as two coroutines over every schedule within three
preemptions - push, drain, command, free with and without the context
held, a video thread with back-to-back work - and fails on a deadlock,
on a bind while the context is current on the other thread, on GL work
without it, and on a take from the free pool past the other's ask.
Extract the shipping helper alongside the import functions that call it. Track its source as a dependency so the harness rebuilds when the helper changes.
Keep the terminal acknowledgement distinct from a parked stop and clear only the latter. A late stop cannot erase the final notification and wait on a worker that has exited. Cover that interleaving deterministically in the handshake harness.
The lane set wake_inside after task_queue_push() had returned, so a
worker that ran the first pass before then parked the task for its
whole horizon with no wake, and the lane timed out. Since workers are
woken by broadcast they take a task sooner and it failed about one run
in seven.
OptiScaler had to stand beside retroarch.exe under a system DLL's name
(winmm.dll) to hook NGX, which put it in every session whatever the
core. As OptiScaler.dll in the DLSS directory it is now loaded only
when a core asks for the upscaler with DLSS set, just before the NGX
core library it hooks; the status names it without loading it.
Next to retroarch.exe as nvngx.dll OptiScaler proxies NGX to itself; as winmm.dll it loads at start and proxies NVIDIA's own NGX library, which is how it runs DLSS on Vulkan.
NVSDK_NGX_VULKAN_RequiredExtensions was required for Vulkan, but it is
deprecated and OptiScaler has only the feature requirement calls that
replaced it, so with OptiScaler the Vulkan driver had no NGX library
and never made the session. Either is enough now: the extensions come
from the old call where the library has it, else from
NVSDK_NGX_VULKAN_GetFeatureInstanceExtensionRequirements and
GetFeatureDeviceExtensionRequirements for DLSS.
OptiScaler itself is not asked: it forwards those calls to the NGX
library it proxies, which as nvngx.dll beside the executable is itself,
until the stack overflows. It adds NGX's extensions in its own
vkCreateInstance and vkCreateDevice hooks. The test's stand-in NGX has
OptiScaler's exports.
A fence-only vkQueueSubmit with a NULL pSubmits is valid, but
OptiScaler, loaded as nvngx.dll for the temporal upscaler, hooks
vkQueueSubmit and refuses a NULL pSubmits without submitting. The
fence never signals, and the hardware ring's wait for it hung every
core on its first frame. vulkan_queue_signal_fence() submits one empty
batch instead, and tools/vulkan_null_submit_check.c keeps the NULL
form out of gfx/.
Decode container language codes and take an owned span before input-window retirement. Share ISO639 lookup with Matroska and cover extended-tag precedence and legacy numeric codes through the task loader.
Retain mdhd language fields and supported elng values without changing public track layout. Keep precedence and malformed-input behavior covered by a portable sample.
Check the effective compiler environment and exact version before running SDK jobs locally. Cover missing, failed and mismatched tools in the runner self-test.
Expose borrowed language spans without changing the public track layout. Prefer BCP47 metadata regardless of field order and retain the Matroska legacy default when the field is absent. Cover empty overrides, raw padding, long values, partial input and cursor lifetime.
Normalize each distinct language annotation while sealing the cue tree. Retain raw annotations and inherited-language indices, share changed comparison keys, and release preparation state before publishing the track. Cover cancellation, allocation failures, repeated keys and chunk boundaries.
Normalize aliases and extended ranges incrementally while preserving raw annotations and repeated constraints. Transfer changed result buffers into prepared owners without another copy.
Generate compact typed aliases from a pinned IANA snapshot, retaining grandfathered tags without replacements. Verify pooled data in the standalone subtitle suite.
Prepare bounded language-range filtering and enclosing-language indices during loading. Preserve raw annotations and borrowed fallbacks for immutable queries.
Retain owned cue trees, regions, STYLE components and bounded selector predicates for the builtin subtitle pipeline. Selector grammar, cascade, layout and styled rendering remain unfinished; this is an unpublished integration checkpoint.
The upscaler's libraries were loaded, and the Vulkan instance and
device made for them, whenever the setting was on, for every core.
OptiScaler standing in for nvngx.dll hooks the whole process once
loaded, and its Vulkan hooks hang or crash cores that never use the
upscaler.
A core now asks with RETRO_ENVIRONMENT_GET_HW_RENDER_UPSCALER_INTERFACE
during retro_load_game, beside SET_HW_RENDER; only then are the
libraries loaded and the context made for them, until the core
unloads. The System Information status finds the NGX library and
tells OptiScaler from its version resource without loading it.
NVSDK_NGX_GetSnippetVersion packs the patch and build into a nibble
each, so 310.9.1 read as 310.9.16. Take the file version on Windows
and the version in the file name on Linux, and decode the snippet word
properly where neither says.
Keyboard, mouse and controllers are always read in bulk by the poll,
on a window the first poll makes on the polling thread. The switch
only kept the earlier path, with its window made by init and its
reports taken by that thread's pump, and the =2 sink registration, for
comparison; both go, with the pump's staleness measurement that only
that path used.
Reject missed display deadlines in either baseline before comparing queue depth. Keep the reduction and underrun checks, with bounded retries and an explicit failure when no valid schedule is observed.
Both only selected the previous behaviour for comparison and no test
or workflow sets either: the snapshot bridge is always used for the
drivers that provide it, and Vulkan always applies a pending resize
before drawing the frame.
vulkan_init reads RETROARCH_VULKAN_RESIZE_LATE once into
VK_FLAG_RESIZE_LATE, so vulkan_frame tests a bit of vk->flags.
qnx_ctx takes an unset ORIENTATION as 0 instead of passing NULL to
atoi.
The DLSS feature library says its version through its own export
(NVSDK_NGX_GetSnippetVersion), which the loader now reads from the DLSS
directory before NGX loads the file; System Information and the log
name it. OptiScaler standing in for NGX is known two ways: its module's
version resource names it at the probe, and every parameter table it
hands out carries its key, which the session checks after init. The
status says "via OptiScaler".
The preset setting carries every letter NGX numbers, A to O, in NGX's
own numbering, so a newer runtime's presets can be asked for without a
new build; a letter older than the runtime found is marked as not in
it, and one the runtime does not know falls back to its default. A
sharpness setting (percent) rides in the mode word: FSR sharpens with
its own pass, DLSS 2.5.0 and older and OptiScaler's RCAS take the NGX
Sharpness parameter, XeSS and newer DLSS ignore it. A sharpness change
alone makes no new feature.
The loader samples read the version from a stand-in feature library
and see OptiScaler's key and, under Wine, its product name; the
upscaler samples check the status, the sharpness reaching each
backend and the feature kept across a sharpness change.
Built with HAVE_WIIU_DESCRIPTOR_IO=1 (Makefile.wiiu, off by default),
Wii U takes the descriptor path for whole-file reads, and since its SD
reads land by DMA in whole cache lines, vfs.h gives it
VFS_DESCRIPTOR_READ_ALIGN 64. filestream_read_file() then allocates its
buffer on a line with a line spare for the terminator and asks for whole
lines, so the file lands in place in one read() and one FSReadFile,
with no bounce copy.
samples/frontend/wiiu_descriptor_read runs filestream_read_file() with
that alignment over a stand-in read() that checks each request and
loses the rest of every line it covers, as a DMA read does.
Windows starts no read-ahead on a mapped view, so every first touch of
a page stalls the reading thread on the disk; with a disc image read
from the core's frame this shows up as a periodic core-time spike that
a warm file cache makes go away. Mapped reads that follow on from the
last one now hint the next 4 MiB each time half of it is used.
The hint is PrefetchVirtualMemory where it exists (Windows 8 on). Below
that a per-stream thread touches the view ahead of the reader, which
brings the pages in without copying anything; it is lock-free (atomics
and an eventcount) and only exists under HAVE_THREADS.
RETRO_VFS_PREFETCH=touch|none selects it or neither, for tests. Layers
that keep local handles behind an installed VFS (vfs_hybrid) now pass
filestream_prefetch() on through filestream_set_prefetch_cb().
The tree's Wii U layer (the coreinit, GX2, AX and friends headers, the
RPX import stubs, startup and linker scripts, the newlib hooks, the
exception handler, the SD devoptab, the HBL loader and the elf2rpl and
slang-cafe host tools) lives under os/wiiu, beside os/gekko. Makefile.wiiu
builds from there with -Ios/wiiu -Ios/wiiu/include, so <wiiu/...> headers
resolve as before; the few includes that reached files through the
tree root name os/wiiu.
newlib before 4.3 calls four lock hooks, with init told whether the lock
is recursive and every hook returning an int, and the Wii U toolchain's
newlib is 4.2. newlib_locks.c provides that set there and 4.3's separate
recursive hooks after it, chosen from __NEWLIB__/__NEWLIB_MINOR__ as
os/gekko/kernel/newlib.c does. samples/frontend/wiiu_newlib_locks runs
its checks against both.
Whole-file reads on Wii U went to open()/read() on the SD devoptab
without that path having run on hardware, and they are the first thing
a boot does: the salamander config and retroarch.cfg. Wii U leaves the
descriptor-I/O list until its descriptor reads have been exercised, so
it reads through the aligned stdio buffer as it did before.
FS reads land by DMA and the cache lines they cover are invalidated
whole, so a read into the caller's buffer that ends partway through a
line loses whatever the CPU last wrote to the rest of it, such as the
next heap block's header. sd_fat_read_r now reads in place only the
whole 64-byte lines of a line-aligned buffer and takes the rest through
its aligned bounce buffer, copying only the bytes that arrived.
Read, write and seek failures return -1 with errno set rather than 0 or
a raw FS status; a seek outside the file or with a bad origin leaves the
position alone and releases the device mutex; stat decides whether the
path is the mount root before freeing it.
samples/frontend/wiiu_sd_devoptab runs the devoptab against a stand-in
FS that invalidates whole lines, with stand-ins for devkitPPC's
<sys/iosupport.h>.
The frontend driver state keeps the fork mode the platform last
accepted (frontend_driver_get_fork()), and config_save_file_salamander()
marks the core it saves as pending when the process is leaving for it,
as salamander does before it launches a core. A core that a running
RetroArch switches to and that never comes up is passed over at the
first boot after; a plain quit or "Save Configuration" leaves no mark.
samples/frontend/salamander_pending takes the frontend's salamander
config functions out of configuration.c and checks both the switch and
the plain save.
No launcher calls into lrc_hash.c, and on x86 its SHA dispatch reaches
cpu_features_get(), which no launcher links: the DOS salamander, whose
link keeps every section, cannot resolve it. lrc_hash.o leaves every
salamander object list. The Wii launcher's logger build links
net_socket.c, which reads cpu_features_get_time_usec(), so that build
links features_cpu.o as well.
compile-matrix.sh now links each launcher's libretro-common list, read
from its Makefile, with frontend_salamander.c and a host frontend
driver (tools/platform_stubs/salamander_frontend.c), so a unit in a
list that needs one outside it fails there.
b6296db2 was the standalone libretro-common repository's copy of the
FidelityFX API and XeSS loaders, applied here instead: it put a second
copy of gfx/amd, gfx/intel, their headers and samples at the root of the
tree, beside the real ones under libretro-common/, which nothing builds.
Remove them; the patch belongs to libretro/libretro-common.
Run-time loaders for AMD FSR through the FidelityFX API and Intel XeSS,
C89, beside the NGX one: the SDKs' descriptor and parameter layouts and
their type numbers declared here so no SDK header is needed, the
library looked for under the SDKs' file names in a directory of the
caller's and kept once found. The samples pin the layouts to the SDKs'
sizes and offsets on x86-64 and drive the loaders against stand-in
libraries that export the SDKs' entry points, on Linux and under
mingw/Wine.
The setting gains AMD FSR and Intel XeSS, on the Vulkan and Direct3D 12
drivers as DLSS is. Each runs on its vendor's library, loaded at run
time from a directory of the user's and never bundled: FSR through the
FidelityFX API (amd_fidelityfx_vk.dll for Vulkan, FSR 3.1; SDK 2's
amd_fidelityfx_loader_dx12.dll or SDK 1.1's amd_fidelityfx_dx12.dll for
Direct3D 12, where the newest version the library offers the device is
taken, FSR 4 on the GPUs that have it), XeSS through libxess.dll on
either driver. The FSR and XeSS directories are settings beside the
DLSS one, upscalers/fsr and upscalers/xess in the RetroArch directory
by default.
libretro-common gains the loaders (gfx/amd/ffx_loader, gfx/intel/
xess_loader), C89, with the SDKs' descriptor layouts and type numbers
declared so no SDK header is needed, and samples that pin those layouts
to the SDKs' sizes and offsets and drive the loaders against stand-in
libraries. The drivers go through one dispatch (vulkan_upscaler,
d3d12_upscaler) over the three backends, which share the output image,
the layout moves around a dispatch and the ring that retires an image
and context after the frames in flight. XeSS builds its pipelines in the
library's background thread and the frames are shown as they are until
they are ready; nothing on the frame path waits. The frontend's mode
word carries the quality and preset as set and each backend translates
it, so the DLSS preset keeps its meaning and the others ignore it.
The upscaler samples run FSR and XeSS behind the dispatch on lavapipe
and on Wine's D3D12 against the stand-ins, with the inputs' flags,
formats, states, scales and reset read back from the descriptors the
libraries were given.
Salamander marks the core it launches in its config
(libretro_path_pending) and the frontend clears the mark once its
drivers are first up. A boot that still finds the mark on the
configured core launches the first other core in the cores directory
instead, so one core that dies on startup no longer takes every later
boot down with it; with no other core, nothing is launched and both
entries are dropped so the next boot looks again. A passed-over core
never falls back to salamander itself.
samples/frontend/salamander_pending boots salamander against a cores
directory with a stand-in frontend driver and checks the mark, the
pass-over, and the empty fallback.
devkitPPC's newlib serialises stdio, the environment, the time zone
and the atexit lists only through the weak __syscall_lock_* hooks,
which nothing on Wii U defined, so with threads on, libc state shared
between the frontend's threads went unlocked. wiiu/system/newlib_locks.c
provides them on OSMutex: newlib's lock word starts at 0, a mutex is
created on first use and published into it by compare-and-swap, and
OSMutex being recursive serves the recursive locks as well.
samples/frontend/wiiu_newlib_locks runs the hooks against stand-in
mutexes: threads racing first use leave one mutex published and the
counter it guards exact, plus the recursive, try and close paths.
The DSI, ISI and program handlers are installed with
OSSetExceptionCallbackEx in global all-cores mode, so a fault on a
thread running on core 0 or 2 prints the register dump and stack trace
instead of falling through to the system's fatal error screen. The
first core to fault owns the report; a second one parks.
samples/frontend/wiiu_exception_cores checks the three handlers are
installed for all cores and that a second fault parks instead of
reporting into the same buffer.
The pointer-array growth in the rslc frontend takes what its function
returns on failure, so a bool function returns false rather than NULL;
a matrix-compmult fold compares unsigned with unsigned; d3d10's frame
loses an unused counter.
The fullscreen viewer opens a file-list row only when it is not a
FILE_TYPE_IMAGE pick, so choosing a Background image in XMB sets the
wallpaper again instead of opening the image. The same rule gates OK in
the viewer on XMB, Ozone, MaterialUI and RGUI.
The menu media harness gains --wallpaper-pick, which goes through the
Background setting and its browser with the real driver and callbacks;
it runs for XMB and MaterialUI, the menus that offer the setting.
Closes#19911
Restore the macro include overwritten by the Crowdin round trip. Exclude both internal label headers from export, import and repacking, including path-qualified inputs and stale JSON files. Check this boundary in the packed-table workflow.
Resume direct hosting after an empty relay response or timeout instead of deferring another lookup. Cover failure, retry, success, deadline and cancellation through the shipping decision functions.
A preset's LUT images are decoded once every pass is built, and only
those some pass samples or takes the size of. The deferred load no
longer decodes them up front: the install loads them for the chain.
WebM playback uses H.264 and HEVC independently of the MP4 demuxer. Keep their decoder objects in the common player harness sources so the MP4=0 playback and AV1 handoff checks link.
Require Openbox to manage a mapped client before launching SDL cases. Check that the Xvfb and Openbox processes remain alive so a different server or manager cannot satisfy their readiness checks.
Exercise pending management, cleanup and dead-process rejection with portable mocked tests.
Keep the previous codec-owned reference until admission succeeds, then release it before installing the new context. Cover repeated replacement and rejected admission with a portable lifecycle regression.
Support capture-free native audio checks, require the requested audio driver, and compare underruns through modal exit and geometry restoration. Restore threaded windows on their owning thread and retain lifecycle diagnostics for failed image continuity runs.
The ten-minute job deadline can expire during the AV1 parameter checks after dependency installation and earlier format tests. Allow thirty minutes while keeping every test and the required aggregate result.
Supply menu filler from the threaded audio consumer so modal frame stalls do not starve the mixer. Preserve core handover cutoffs and partial writes, and use the existing idle path when the sink cannot progress.
Cover producer stalls, core resumption, suspension, partial writes and failed-sink recovery in the legacy and transport regression lanes.
A Temporal Upscaler Preset setting picks the DLSS model: the runtime's
default, or the transformer presets K (the best image, at more GPU
time) and J (less ghosting, some flicker), the two the SDK still offers
an application. The mode is one word on the way to the backends, the
quality and the preset, and a change of either makes the feature
again; the preset is set as the SDK's hint for every quality. The log
names the mode and the preset when the feature is made.
The upscaler samples' fakes record the preset hint and the tests check
it; they also clear the runtime they plant, so a run after another
starts without it.
Under the rolling scanline simulation a subframe draws one band of a
pass's target; the render area is now that band and the target keeps
its contents around it, cleared once when it is new, where it was
discarded every subframe. A pass that can discard clears only its
band.
A pass whose fragment stage can discard draws to a target cleared to
transparent black, as glcore and D3D clear theirs, instead of one
whose contents are undefined. crt-royale's mask-resize passes, which
discard past their tiles while the next pass filters across the edge,
stop picking up whatever memory the target was given.
The copy now also sets USB 3 Link Power Management to off on AC, so
xHCI links stay out of U1/U2 and connected controllers answer without
a link wake. Like the other device settings it is best effort: a
system without the setting applies the plan without it.
The win32_power_plan harness checks the value on the copy, that the
user's own plan keeps its setting, and the missing-setting case.
packed, row_major, attribute and varying are free in Vulkan GLSL and
reserved in the GLSL glcore is given, so the printers rename them as
they do the other reserved words; struct and block member names go
through the same renaming. scanline-classic's famicom-rgb and
color-base, which name a local and a parameter packed, compile on
glcore again.
The core library's NVSDK_NGX_*_Shutdown1 takes a second argument the
SDK's prototype leaves out: an int the library writes, whether it is
still in use, which the SDK's static library hands it. Called with
one argument, the library wrote through a register, faulting the
close of every hardware core. The loader's prototype and every call
now pass it, and the sample fakes write it as the library does.
The feature info and path list NGX is initialised with were the init
function's locals, the wide path freed on return; the core library
reads them again at shutdown, where the close of a hardware core
faulted inside it. The session owns them until after Shutdown1, on
Vulkan and Direct3D 12 alike. The sample fakes read the path list
again at shutdown, as the library does, and the tests check it.
The close of a hardware core faulted inside the NGX core library in
NVSDK_NGX_VULKAN_Shutdown1 with the device passed. The SDK asks for
Shutdown1(NULL), every device's resources; the capability parameter
map is destroyed before it too, as the SDK asks. Each teardown step
is logged so a fault there names the call.
Both upscaler samples plant the DLSS runtime's file in their test
directory before the session is expected to come up, and first check
that without it the driver library is never called.
The NGX core library's NVSDK_NGX_VULKAN_Init_ProjectID takes no
proc-address getters; the form with them is Init_ProjectID_Ext.
Calling the short one with the long argument list made the driver
read the SDK version as a pointer and fault on any Vulkan hardware
core, DLSS runtime present or not. Resolve _Ext and use it, with the
short form as the fallback on libraries without it.
Neither backend touches the driver library now unless the DLSS
runtime is in the DLSS directory; without it the upscaler logs and
stays off.
Use the Cg header switch to reuse the GL declarations already supplied by gl2_common. This prevents a desktop function declaration from colliding with the GLES3 symbol wrapper.
Cover GLES2 and GLES3 Cg combinations with Linux and MinGW compilers in the compile matrix.
NVSDK_NGX_Parameter is a C++ interface: the NGX core library exports
no NVSDK_NGX_Parameter_Set*/Get*, those are the SDK's static library
calling the object's virtual methods. Looking them up as exports found
nothing on a real driver (the log said so), so no NGX was ever usable.
The loader now calls the vtable, in the slot order the SDK's own static
libraries use: each overload group reversed on MSVC, declaration order
on the Itanium ABI, read from the libraries' disassembly. The stand-in
libraries of the tests give their parameters that vtable, in the
platform's order, so the tests call through the same path.
With the setting on, the log said what the probe found only once a
hardware-rendered core attached; in the menu alone it said nothing.
Logged when the Vulkan and Direct3D 12 drivers start as well.
The Temporal Upscaler setting was listed only once the NGX core library
and the DLSS runtime were both found, which left nothing to see and
nothing to read when either was missing. It is listed on the Vulkan and
Direct3D 12 drivers as the other scaling settings are, and when the
driver comes up with it on, the log says what the probe found: where
the NGX core library was looked for and loaded from, which entry points
it lacked, and whether the DLSS runtime is in the DLSS directory.
ngx_lib_status() carries that report out of the loader.
In the MSVC griffin build the backend is the first to include d3d12.h,
without COBJMACROS, so the C method macros never existed for the files
after it and the link failed.
The driver's NGX core library exports NVSDK_NGX_*_Init_ProjectID, with
the SDK version before the feature info, and NVSDK_NGX_*_EvaluateFeature;
Init_with_ProjectID and EvaluateFeature_C are the SDK's static
library's names for them. Resolving the latter found nothing, so no
NGX was ever found on a real driver and the Temporal Upscaler setting
never appeared. The library is also looked for by name on the library
search path, where the driver installs it. The stand-in libraries of
the tests take the library's shape.
A preset's LUTs are named up front for the semantic map and their
images loaded once every pass is reflected, only for those a pass
samples or takes the size of. A Mega Bezel screen-only preset stops
decoding and uploading 15 to 19 of its 23 images, about 90 MB.
Use core geometry for the minimum size in focus polling, matching WM_GETMINMAXINFO. Enforcing the initial window scale during a permitted resize can repeatedly enlarge the window and recreate presentation resources.
Check that focus polling preserves a window above the core minimum and still enlarges it when core geometry grows.
The combined formats job exceeded its ten-minute limit before finishing codec checks. Run the VP9 and Matroska checks in their own job with a twenty-minute budget, preserving every test and sanitizer setting.
Keep the original required check as an aggregate that succeeds only when both suites pass.
The DLSS Directory's default moves from the System directory's dlss to
upscalers/dlss in the application directory, beside shaders and system,
where other upscalers' runtimes will sit too.
The Direct3D 12 driver gets the temporal upscaler the Vulkan driver
has: a core that brought motion vectors and depth through the upscaler
interface has its frame evaluated by DLSS on the frame's command list,
and the output takes the core's texture's place for the copy into the
filter chain. gfx/common/d3d12_ngx.c mirrors vulkan_ngx.c: one session
per device, a feature per output size and mode made for the mode's
full dynamic range, inputs moved to shader-readable states and back,
the output a copy source, retired features and outputs released after
the frames in flight.
samples/gfx/d3d12_upscaler drives the real backend on Wine's D3D12
(vkd3d over lavapipe) against a stand-in NGX core library whose
evaluate copies the colour into the output, so the feature and evaluate
parameters, the states and the readable output are pinned on every
runner. Compile-matrix lanes build the backend and the driver with
HAVE_NGX, with and without threads.
Separate audio preparation and final picture publication from video seek completion so independent expensive stages do not share one continuation. Preserve prepared PCM and cancellation ownership across the additional yields.
Exercise a second player through EOF while a seek is waiting for input or continuing on the shared worker. Check replacement output and presentation slot retirement in inline and threaded modes.
Keep mixer voices while rebuilding the core audio pipeline. Reuse the initialized mixer at the same output rate; full teardown and output-rate changes still release it. Cover playing and stopped streams across repeated initialization in both pipeline modes.
A last pass without scale_type that reads its own feedback, as Mega
Bezel's combine pass does, gets the render target and copy that a
viewport-scaled one gets, on Vulkan and glcore. Its feedback sampler
was left unwritten, which lavapipe crashes on and other drivers read
as garbage.
The output, render and range sizes the upscaler passes around are
VIDEO_SCALE_PACK words, as the rest of the video code carries them:
one field or argument per size in the Vulkan NGX session, the frontend
module's backend and the driver, in place of width and height pairs.
A hardware-rendered core now has a way to feed a temporal upscaler:
RETRO_ENVIRONMENT_GET_HW_RENDER_UPSCALER_INTERFACE hands it an
API-neutral interface with get_request, which says whether an upscaler
will take the frame and what render size it wants for the mode (so the
core renders smaller, jittered and without its own temporal AA), and
set_inputs, which brings the frame's depth, motion vectors, jitter and
optional exposure and transparency masks beside the color image the
existing render interfaces hand over, under the same lifetime and sync
rules. Images are the context API's own handles, so the same contract
serves Vulkan and Direct3D 12, and later FSR and XeSS. No existing
interface version changes; a core that never asks is unaffected.
The Vulkan driver runs DLSS Super Resolution over it through NGX
(gfx/common/vulkan_ngx.c): a session on the device, the mode's render
sizes from NGX's optimal-settings callback, a feature made for the
mode's full dynamic range and the output size and re-made only when
those change, the frame's size as the render subrect, the output image
handed to the filter chain in the core's place, inputs in a depth
layout moved for the evaluate and back. Retired features and images go
after the frames in flight; nothing waits. The context enables the
device and instance extensions NGX asks for, on a device the core makes
as well, only while the setting is on, which now reinitializes the
driver so they can be. A Quality setting picks the mode (Auto, Native,
Quality, Balanced, Performance, Ultra Performance).
samples/gfx/vulkan_upscaler pins the glue on lavapipe under the
validation layer against a stand-in NGX core library, built with gcc
and mingw in the samples-gfx workflow.
Retain container timing and hidden-picture state through incremental reads, reordered output and native frame leases. Route supported tracks through the existing decoders and GPU imports, with software, mock and native regression coverage.
* threaded_video_test: freeze the core across the game-only lane's pause
On some runs the harness core dupes an oversize frame, which drops the
cached frame. If the game-only lane pauses right after one, the paused
redraws have no core image and gl1's game-only read fails. Where the
lane pauses depends on how many frames the lanes before it ran, so
adding a lane can move it onto one of those runs.
Freeze the core across the pause, as pixlane_once and lane_frame_mem
already do.
* video: keep the Android stub frame out of the shared driver vtable
On focus loss Android swapped video_null.frame into the current
driver's static video_driver_t, and on focus regain wrote the saved
pointer back into whichever driver was current by then. When a core
load switched drivers in between, e.g. a core restart asking for the
other HW context, video_gl2.frame became vulkan_frame: the next frame
ran vulkan_frame on a gl2_t and crashed, and video_vulkan was left
drawing nothing.
The stub is now a main_flags bit. video_driver_frame calls
video_null.frame while it is set, and GET_AUDIO_VIDEO_ENABLE still
reports video off. It holds across a driver switch or reinit until
focus returns. A focus loss before any driver exists still does
nothing.
threaded_video_test gains a lane that stubs under one driver, switches
to another and unstubs.
* video: test the stub frame on Android only
Only Android sets the stub, so other platforms take the driver's
frame without testing the flag. The flag is set whether or not a
driver is up yet, so a focus loss before drivers_init() also holds
until focus returns.
The stub-frame lane keeps its vtable and AV-enable checks everywhere
and checks for dropped frames on Android only.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Keep SPS/PPS snapshots with each decode context and retain its destination slot while the picture is in flight. Repeated parameter sets and hidden pictures can then overlap worker decoding safely. Exercise replacement, hidden output and slot reuse with delayed row publication under sanitizers.
A read-only open asking for a mapping takes the descriptor branch only
for plain files and SAF. cdrom://, smb:// and nfs:// streams stay on
their own backends and are read unmapped, so archives on a network
share open and list.
Lane: the smb_retro and nfs_retro vfs_threads_test open with
RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS, built with file mapping as
the shipping build has it.
The groundwork for DLSS Super Resolution on the Vulkan and Direct3D 12
drivers, with nothing of NVIDIA's built in or shipped. libretro-common
gains two run-time loaders under gfx/nvidia, shared by frontend and
cores alike: nvapi_loader.c, the NVAPI probe the Direct3D drivers
carried themselves, now behind one nvapi_query(); and ngx_loader.c,
which finds the driver's NGX core library (beside the program, else
where the driver's registry keys point, libnvidia-ngx.so.1 on Linux),
resolves the Super Resolution entry points for Direct3D 12 and Vulkan
with the parameter accessors, and restates only the SDK interface it
takes. A sample drives both against stand-in libraries on Linux and
under Wine, and is wired into the libretro-common samples workflow.
The frontend side (HAVE_NGX, Windows and Linux builds with Vulkan or
D3D12) adds the Temporal Upscaler setting under Video > Scaling, hidden
until both the NGX core library and the user-supplied DLSS feature
library are found, and the DLSS Directory setting it is looked for in
(<system>/dlss by default), plus a System Information line that says
what was found. The setting selects nothing yet: the drivers' evaluate
path and the core-side buffer exports follow.
Honor prior-output discard without dropping already-final output. Tag decoded pictures with their input timestamps and use bounded header lookahead to give MP4 visible pictures their correct timestamps and durations. Add stream, backpressure, seek and partial-input regressions.
rslc's tables mark each sampler active when code main reaches reads
it, as SPIRV-Cross's active interface variables do (blob v4). Both
reflection paths skip the rest, so a declared but unread history
frame, feedback buffer or texture costs nothing: the Mega Bezel
presets stop keeping three frames of history and copying into them
every frame, and an unread feedback sampler no longer gives the last
pass a render target.
rslc is the slang compiler everywhere. ./configure --enable-glslang
--enable-spirv_cross links the system's glslang and SPIRV-Cross next to
it when both are found, and only then does Settings > Drivers offer a
Shader Compiler choice between the two: one setting read per stage
picks the path, with nothing copied.
The MSVC 2010 project builds Vulkan and rslc. Samples that used the
vendored copies run on rslc or the system's SPIRV-Cross.
Retain the current media preview across settings navigation and restore the fullscreen viewer on Back without changing playback or pause state. Cover all four menus, real browser rebuilds and visible preview pixels.
The MSVC projects (2012 to 2026, UWP and the legacy Android one) build
rslc through griffin_rslc.c in place of glslang and SPIRV-Cross, as do
RetroArch_OSX107 and RetroArch_iOS11_Metal through griffin.c, and the
unix griffin build in Makefile.griffin. WANT_GLSLANG, which no
platform makefile set any more, is gone from Makefile.griffin, ngc and
wii.
rslc's MSL compiles under Apple's Metal compiler where SPIRV-Cross's
does: array, matrix and struct varyings go in one stage member per
element, column or field; length, distance, dot and normalize of
scalars, x *= M, and constructors mixing scalars and vectors or
running past a vector are spelled out; program-scope constants are
built from columns of literals, so Metal needs no global constructor;
decay and quad no longer clash with Metal's names.
Every non-bezel preset of slang-shaders gives glslang's picture on
metal, or one within the rounding where Metal's fast math orders the
two texts differently, so auto compiles with rslc on every driver.
configure builds rslc alone on macOS too, and the Xcode projects take
rslc through griffin.c in place of glslang and SPIRV-Cross.
The image gate runs metal (-d metal, pattern_core.dylib), and without
timeout(1) where macOS has none.
Use ReferenceBlock entries before or after Block to classify random-access packets, including laces. Retain the state across incremental reads and clear it on seek and rewind.
Discover S_TEXT/UTF8 tracks through the shared subtitle catalog and load
bounded cue spans asynchronously without reading unrelated media payloads.
Retain block durations and filter packets returned by cursor fallbacks.
Recognize Matroska in the internal viewer and browser. Cover sparse files,
malformed text and timing, cancellation, and all four menu adapters.
configure builds rslc by default, and outside Apple builds glslang and
SPIRV-Cross only when asked for (--enable-builtinglslang
--enable-builtinspirv_cross); Metal keeps them on macOS. Android
builds rslc alone (HAVE_RSLC_ONLY=0 brings the two back), and Wii U
takes rslc in place of SPIRV-Cross. The vendored glslang and
SPIRV-Cross stay in deps/.
rslc reads the literal strings of SPIR-V lowest byte first on every
host, as the words store them, so names and the GLSL.std.450 import
read right on big-endian Wii U.
d3d11 and d3d12 have met the image gate through Microsoft's
d3dcompiler_47: every non-bezel preset of slang-shaders gives glslang's
picture, or one within the last bit where D3DCompile orders the float
math of the two texts differently. Metal stays with glslang until its
xcrun run is in.
The image gate stops at once on a RetroArch that is missing or does
not start, instead of failing every preset without a log.
Expose unlaced BlockDuration without inferring duration for laces. Skip unrelated BlockGroups after their packet prefix so audio consumers avoid video payload reads. Keep refill tests meaningful by adding a large selected audio group while retaining the original video group.
Expose tx3g tracks through the shared player subtitle catalog and decode text incrementally. Preserve edit-adjusted cue timing, reject malformed UTF encodings and read metadata/sample ranges without ingesting movie payloads. Cover cancellation, sparse containers and all four menu adapters.
Capture native initialization thread stacks when the views core never becomes ready. Keep the existing readiness deadline and screenshot checks so a stalled SDL window still fails with its fault site preserved.
A ?: statement with assignments for operands is printed as if / else,
a local whose initializer reads the global of its name is renamed, a
local declared without a value starts at 0, and a function that can
run off its end returns 0 there. d3dcompiler_47 refused all four, so
crt-royale, crt-pocket, the sharp gaussian blurs and lottesRVM fell
back to the stock chain on d3d11 and d3d12.
Use the existing bounded core-readiness wait for native launches as well as Wine. Slow initialization otherwise consumes the settling deadline and loses the one-shot screenshot command before its endpoint exists. A delayed native launch reproduces the failure before this change and passes afterward.
Filter SimpleBlock payload admission by audio track and bound unpublished WebM preparation to the requested span. Preserve exact PCM timing and continue across clusters with no selected packets. Cover sparse headers, metadata yields, cluster transitions and preparation footprint.
Windows Vulkan omitted VP9 from its packed decoder bridge and the VP9
session rejected packed GPU leases. Admit the existing GPU export path,
preserve native H.264 ownership, and retain the validated VP9 header so
reopening after a seek cannot parse released file pages.
Feed seek lookahead from its actual frontier while pinning borrowed
packets. Load optional, bounded SeekHead/Cues metadata for the video
track instead of walking a multi-gigabyte movie to locate its keyframe.
Keep sequential fallback and endpoint-clamping behavior when the index
is absent, malformed or contains no shown picture at the selected tail.
Drive menu audio in the native media fixture so its PCM clock advances.
Validation: VP9 and thumbnail-preview ASan/UBSan suites; D3D11 decoder
mocks; strict C89 threadless frontend link. The new feeder regression
fails before the correction. Exact 8K file GPU playback and repeated
30/90/180/30-second native Vulkan/WASAPI seeks reach captured frames.
The 30-second jump falls from a timeout, then 6.6 seconds, to about
0.9 seconds with matching capture hashes. Audio scanning remains a
performance limitation; this does not claim sub-half-second seeks.
retroarch.exe sends --verbose to the console it attaches to rather
than to a redirected stdout, so on Windows each capture logs to a file
which the script appends to the capture's log.
Do not render the CC status label after discovery finds no subtitle tracks. The shared formatter covers Ozone, XMB, MaterialUI and RGUI while retaining loading, selected-track and disabled states.
Validated the production formatter with a strict C89 native check for absent tracks, loading, selected track and disabled subtitles.
Include the settings declaration in shared decoder stubs and supply the software-fallback stubs required by the GPU format transport fixture. Unexpected calls still fail the transport test. Run builtin-player CI when its shared decoder fixtures change.
Video-only reinitialization destroys menu GPU resources without the preview-worker drain used by full menu teardown. A native decoder constructor can still own that device after the preview is retired. Use the existing teardown helper before freeing video.
Add an optional native held-constructor video-only reset regression. It records premature device teardown and drains safely for baseline cleanup. The baseline fails this ordering check; corrected D3D11, D3D12 and Vulkan runs complete construction before free and resume hardware playback. The native frontend build, full threadless C89 link, compile matrix and MSVC6 stand-in pass. No frame-path synchronization or new wait primitive is introduced.
Defer native codec construction until the decode worker owns the stream, including hardware-only AV1 metadata opens. Keep pending seeks and pause state through software recovery when the first hardware frame cannot be decoded or imported. Retain unshown inline leases on temporary drops and distinguish permanent import refusal from retryable backpressure. Cancellation retains the existing nonblocking retirement path.
Cover cold admission, queued cancellation, paused and seeked recovery, allocation and worker-creation failures, and permanent versus temporary refusal. Native D3D11, D3D12 and Vulkan tests exercise hardware playback, asynchronous first-import refusal with software recovery, and closure/reopening while construction is held on the worker. GCC/Clang sanitizers, TSan, the thumbnail-preview workflow, compile matrix, MSVC6 stand-in and a full threadless C89 frontend link pass. No new production frame-path synchronization or core HDR ABI changes.
Vulkan and D3D11 native constructors read the mutable frontend hardware-decode setting. Keep that policy on the polling owner: snapshot it in the surface-player sink and gate hardware-only container admission at the menu entry point. Native backend opens now validate published capabilities without querying configuration. Existing players keep their captured policy across seeks.
Exercise the real surface-player constructor and early thumbnail admission with Off, On, explicit software-only flags, missing settings and changes after construction. The prior constructor and entry point fail these regressions. Read extracted sources as UTF-8 for the native Windows test runner.
Validation: GCC/Clang ASan+UBSan and GCC TSan handoff tests, native Windows handoff, complete thumbnail-preview workflow, compile matrix, MSVC6 stand-in and native frontend build pass. Six real D3D11/D3D12/Vulkan playback cases cover initially On and Off, change the setting during playback, then verify policy retention across captured seeks. Cold setup scheduling, first-frame fallback and intermittent AV12 latency remain separate unfinished work. No frame-path synchronization, readback or core HDR ABI change.
The 4K surface lane writes a lent slot before asking whether the GPU has returned it. A descriptor release is not that permission. Busy retries also consume the fixed sample count, failing native D3D11 with only 23 or 24 of 60 submits accepted.
Advance the existing frame pump until the slot is writable before touching it, with a bounded failure assertion outside submit timing. Keep the copy, submit-time, release and texture-reuse checks. Add a focused selector for the same lane. Preserve failed check-driver raw logs and upload them from Windows CI so a normal failing exit retains its diagnostic context.
Validation: native D3D11/D3D12 with RGB565 and XRGB8888 accept all 60 focused submissions; full native D3D11 RGB565 and D3D12 XRGB8888 harnesses pass. The prior full D3D11 cases fail only the sample-count assertion. Actual make recipes retain failure logs and remove success logs. No production code changes.
Predict the next packet from sample-table cursors without reading media or advancing the demuxer. Forward that range through the video stream and shared image-transfer feeder. Admit an MP4 packet larger than lookahead over bounded reads rather than an unpaced burst.
Cover partial input, repeated and nullable queries, interleaved tracks, selection, rewind, EOF and malformed ranges. A real 1.7 MB packet beyond a16 MiB header gap fails the unpaced negative and succeeds over four512 KiB feed polls, then decodes. The CI fixture runs inline and threaded paths.
Validation: full thumbnail-preview and libretro-common workflows, GCC/Clang ASan+UBSan parser checks, native Windows parser/feeder, compile matrix and MSVC6 stand-in pass. Actual Vulkan/DXGI AV12 seeks retain matching picture hashes with active WASAPI. Phase timings separate target-ready state from test capture overhead; intermittent reported long seeks remain unresolved.
Register the built SubRip sample targets alongside the other format tests so the sample workflow can run the parser instead of rejecting its target classification.
Validation: the complete libretro-common samples workflow builds 100 directories and runs 143 targets with zero failures, including the subtitle parser and the later sanitizer checks.
The retry probe tests only the lock bit. Use a relaxed pointer load there; the successful existing compare-and-swap still acquires the list. This avoids redundant ordered polling contending with the owner during a dense waiter storm. No list ownership or wakeup protocol changes.
The unchanged unrestricted 96-thread semaphore regression times out under Clang TSan and trips the GCC watchdog before this correction; both pass in about four seconds afterward. All backend checks pass Clang TSan and GCC ASan+UBSan; eight native Windows tier/timer combinations and the MSVC6/NT4 stand-in pass. The full libretro-common workflow also passes.
Feed the next contiguous local window when a seek reaches its buffered frontier instead of yielding after one decode step. Keep the four-millisecond and 1024-step budgets, limit contiguous feeds to eight, and retain backward-boundary and sparse-input behavior.
Validation: boundary regression fails on the prior implementation and passes on native Windows, GCC/Clang ASan+UBSan and GCC TSan. Full thumbnail-preview workflow and MSVC6 stand-in pass. Active WASAPI AV12 timeline checks complete, but remain about 0.55-0.63 seconds on both versions; this does not claim to resolve the intermittent AV12 latency report.
Record continuous playback and the opt-in CRT and refresh options in all three platform profiles. The emitted settings tables are unchanged; their committed reference now includes these intentional additions.
Validation: complete Linux samples/settings workflow passes, including table and unity checks.
Add opt-in desktop refresh matching and CRT SwitchRes source timing to the shared fullscreen viewer. Windows and X11 capture a temporary output lease and restore only when their applied mode is still current. Preserve core AV timing and reuse the owned mode across queued episode admission. Clear unstable cadence without treating an audio-only repeat tail as variable picture timing.
Validation: strict controls with GCC/Clang ASan+UBSan, GCC TSan and threadless; player loop regression and private sanitizer lanes; compile matrix, MSVC6 stand-in, complete Linux graphics and threadless menu jobs; native four-menu source tests and clean Vulkan/D3D11/D3D12 moving-pixel checks with a static negative. X11 live fractional refresh and conditional restoration pass. Windows physical refresh switching remains unverified: this session refuses CDS_TEST even for its current mode. No VRR policy or physical HDR claim.
Measure cadence from accepted movie timestamps, preserving fractional timing through inline and worker presentation. Clear evidence after seeks and reject variable or invalid intervals.
Expose cadence and source dimensions through the shared media state. Cover timestamp precision, hardware retries, threading, and real menu playback at integer and fractional rates. This prepares display integration without changing mode-switch or VRR policy.
The audio driver now exposes preview repeat control. Complete the standalone audio harness mixer API so the TSan and other sample builds link without the real mixer.
Advance naturally sorted folder movies only after clean playback completion. Retain the final frame, stop on decode errors, and expose previous/next and settings controls in all four menu drivers.
Drain terminal AV1 output and update preview audio repeat policy without reopening the player. Cover completion, queue lifetime, invalid next files, toolbar actions and native menu presentation.
The pitch lane replaces the menu GPU callback, so clear the software menu texture before capturing its replacement pattern. Otherwise RGUI pixels correctly composited above the callback obscure all eight D3D12 captures after the caption-layering fix. Keep the pattern, padding, slot, filter and pixel checks intact.
Pass the sequence OBU and resolved color to configuration-aware backends. Keep an owned bounded sequence header when initial admission or an in-band correction learns it, so reopening after a seek does not depend on the initial packet remaining resident. Reject level or tier changes during provisional admission.
Exercise configuration-only backend admission and MP4/WebM reopening with the original packet unavailable, alongside retention refusal and balanced decoder/lease cleanup.
On Linux a DualShock 4's or DualSense's touchpad is an input device of
its own beside the pad. The driver finds the one with ID_INPUT_TOUCHPAD
under the pad's HID parent, when the pad connects or when the node comes
after it, and reads its first two multitouch slots and its click into
the pad's view, which it lends. RETRO_DEVICE_TOUCHPAD and the Controller
Touchpad setting work on it; the node goes with the pad or by itself.
udev_checked_test feeds it a DualSense's events.
A Controller Touchpad option (off by default) has the core read port
1's touchpad itself: each finger down is a dot where it is, the screen
standing for the touchpad, green for the first and magenta for the
second. A core author has a working reader to look at, and the device
can be tried on a pad. Read by the core, the touchpad leaves the port's
Controller Touchpad mapping, which the Pointer setting shows otherwise.
It is not sent to the receiver.
The sdl2 driver hands a pad's touchpad over through get_touchpad(), as
sdl3 does: up to two fingers of the first touchpad and its press, read
from SDL once a poll and only when asked. RETRO_DEVICE_TOUCHPAD and the
Controller Touchpad setting work on it with SDL 2.0.14 and later.
The SDL3 bridge test gives SDL a virtual pad with a touchpad and holds
get_touchpad() to where SDL has the fingers and the press.
Windows CreateProcess searches System32 before PATH for a bare bash name, selecting the WSL launcher instead of MSYS2. Resolve the requested shell through PATH before compiling the menu media harness.
Draw the software menu texture after the menu GPU callback so RGUI captions remain visible over its movie. Restore alpha blending and the sprite pipeline around that pass. Exercise real GPU subtitle output in the existing Windows D3D harness.
Discover a bounded catalog asynchronously, cycle tracks and Off through the shared controls, and retain the selected CPU subtitle owner across video context resets. Apply the same behavior to Ozone, XMB, MaterialUI and RGUI, with cancellation and native rendering coverage.
Each program run under Wine started a server that left after it, and a
program started as the last server was leaving was cut off before
main() ("wine client error:0: recvmsg: Connection reset by peer"),
failing the job with nothing wrong in the tree. Steps that run Wine
programs and need no display now source scripts/wine-server.sh, which
keeps one server up for the step and starts Wine's services at once
with no output, so they cannot hold a step's pipe open. Steps that
open windows under Xvfb, start the server themselves or use a prefix
of their own are left as they are: Wine's desktop process, started by
their first window, would hold their output.
With one GL context handed between the core's thread and the video
thread, the video thread takes the context after claiming a frame and
only clears the ring's busy bit once it has drawn. A drain on the core's
thread that still held the context - the menu-texture drain behind a
dupe or software frame the core pushes from retro_run with the menu up,
or any other drain reached while the core runs - waited for a frame that
could not complete, and both threads waited forever.
video_thread_ring_drain() now releases a context the calling thread holds
for the length of the wait and takes it back after, as a command round
trip already does. gl_context_drain.py runs the drain against a modelled
video thread and fails on the deadlock; it runs in the samples-gfx job on
gcc, clang and mingw under Wine.
A stage rslc refuses is logged once, with the stage, the pass's file
and cache key, and rslc's reason - a warning where glslang compiles it
instead, an error in a build on rslc alone - so a tester's log says
which shader to look at. shader_chain_test holds the line.
A server kept for the whole step keeps the Wine services the step's
first program starts, and they hold that program's output: a step that
pipes a Wine program into tee or grep waits for an end of output that
does not come while the server lives. Linux-msvc6-standin's Win32 input
step passed its test and then hung for an hour. Back to a server for
each program until the race is fixed without that.
A core reads the touchpad of a port's controller with the pointer's
IDs: X/Y across the surface, PRESSED and COUNT, the index the touchpad
times 16 plus the finger down. It is not the screen, so a core can
read both - a handheld's front touchscreen and rear touchpad, say
(issue #19902).
The frontend answers it from the pad's state as the HID handlers lend
it, and the SDL3 driver hands its touchpads over through a new
get_touchpad op, taken once a poll. The capability bit is set where the
joypad driver can say. A core that reads the device has the touchpad
to itself: the port's Controller Touchpad mapping steps aside until the
core is unloaded.
Keep cursor movement stable during readback and require repeated live sizing commands. A uniform-colour fixture can compare central colour across different window geometries without treating letterbox changes as playback progress.
Keep resolution-dependent UI scale stable while the native move/size loop runs. Window and media geometry continue updating, and the final scale is applied after release. This avoids repeated font atlas construction during a drag.
Exercise initial and final widget scale transitions and sustained native resize playback with modal-state cleanup checks.
rslc_image_check.sh runs retroarch.exe directly from an MSYS2 MINGW64
shell, with Windows paths and pattern_core.dll, so the D3D captures go
through Microsoft's d3dcompiler_47. rslc_image_all.sh runs it over
every preset of a slang-shaders checkout outside bezel/, for each
driver named, keeping each verdict in out/gate_<driver>.log so a run
cut short resumes, and lists what differs or fails.
Android's Vulkan slang drivers and the MSVC 2015-2026 projects now
build rslc (griffin/griffin_rslc.c) beside glslang and SPIRV-Cross, so
rslc compiles their stages with those two behind it for one it
refuses. On Android HAVE_RSLC_ONLY=1 builds without them: 13.8 MB
for arm64-v8a against 18.9 MB with them.
rslc's GLSL printer indexes its swizzle strings rather than adding to
them, which clang warns about.
Request shared stereo float output explicitly. If WASAPI is inactive after frontend initialization, reinitialize it once before the preview owns an audio voice. A persistently unavailable or substituted driver still fails the fixture.
Allow the native regression to exercise cold and permanently unavailable setup without retrying the measured playback or resize operation.
webm_av1_hw_test compiled all of the player's sources again in one
command, for two defines that only rwebm_video.c and rmp4_video.c read.
Those two are compiled with them; the rest are the objects the build
already made. The four sanitized builds of the WebM player job's first
step took 25 minutes and take 18 now, under load.
./configure --enable-rslc --disable-glslang --disable-spirv_cross builds
the slang drivers with neither dependency (SLANG_RSLC_ONLY). Passes
reflect from rslc's tables - a module without them, from the tables
rslc's ingest reads back - and take rslc's text; a frame-rectangle pass
only the SPIR-V rewrite could serve copies the frame out instead.
glcore builds its own pipelines and chains from the GLSL their SPIR-V
was compiled from (gfx/drivers/vulkan_shaders/*.src.inc), printed with
rslc's flattened blocks: each block one vec4 array, members read out at
their layout offsets, as glcore uploads them.
samples/gfx/rslc_image takes -b to hold such a build to the glslang
one; rslc_test covers the flattened blocks.
Each program run under Wine started a server that left after it, and a
program started as the last server was leaving was cut off before
main() ("wine client error:0: recvmsg: Connection reset by peer"),
failing the job with nothing wrong in the tree. Steps that run Wine
programs now source scripts/wine-server.sh, which keeps one server up
for the step and ends it with the step. Steps that already manage the
server, or set their own trap, are left as they are.
Match the existing Clang C89 harness flags so boolean.h does not fail on its _Bool expansion. Keep the strict declaration, warning and stack checks, and leave GCC flags unchanged.
Check image visibility, presentations at the resized output dimensions and restored pixels through the real frontend loop. Ignore physical menu input while the fixture owns the window; native sizing commands still pass through Windows. Reset movie playback before measuring live A/V progress.
Adopt an already-converted caller only after the floating-point isolation probe succeeds. Pump cleanup leaves that caller owned by its original user. Exercise protected and unprotected foreign callers through modal resize, cancellation and destruction.
Run strict C89 native lifecycle and floating-point-state checks in MSYS2. Wine checks the supported path or explicitly reports unavailable continuity and validates fallback. Keep integrated harness window restoration asynchronous across video-owner threads.
Use the normal frontend message pump and external media launch, observe the actual sizing interval, move the window geometry, and require PCM and movie pixels to advance before releasing the loop.
Keep message dispatch on an owner-thread fiber so sizing and moving can return to normal frontend work without recursive runloop entry or redraw timers. Preserve floating-point state, defer menu commands until normal dispatch returns, and retire message fibers on their owning threads.
Guard WASAPI pump-only state and make the ring-room helper available to its single-threaded writer. Link the eventcount helpers used by native Windows audio workers even when frontend threading is disabled, including the griffin build. Compile the real WASAPI driver without HAVE_THREADS alongside its existing pacing tests.
The threaded stress gave each side 30 seconds in all for 200000
frames, which a loaded runner under ASan or TSan does not always
manage with one slot: the handoff costs a context switch a frame. The
30 seconds now run from the last frame either side moved, so a queue
that stops still fails and one that is only slow finishes.
The reference chain filled a pair's output buffer, 180 KB, before each
resample and read it back a byte at a time after. With one frame a call
that was most of the run: stretch_pipeline_test took 14 minutes under
ASan and ran past its 10-minute timeout. A pair writes at most 128
frames; the 512 after its output are watched, with one memcmp. The
stretch suite with its build now takes 2 minutes.
The playlist reader took the whole first line as the disc's path, so an
m3u starting with a UTF-8 byte order mark, a quoted path, or a
RetroArch "path|label" entry (which RetroArch's own m3u parser and the
disk control interface accept) gave no hash, and achievements did not
load for it. The mark, the label and the quotes are now left out.
The "potential consoles" verbose message printed the extension from the
m3u path after the m3u handler had freed it; it prints the copy taken
for the handler search instead.
* MCP: schema check on all tools, readable MENU
* MCP: MENU marks choices, answers when closed
* MCP: MENU types, shows dialogs; drop menu option
* MCP: fold INPUT, OPTION, CORE_OPTION; trim list
* MCP: MENU path, more, tab names; clearer errors
* MCP: menu title as a text line; trim the list
* MCP: trim STEPS' descriptions
* MCP: INPUT says the core gets what is held
* MCP: MENU review fixes
The float rows are added whatever the build, as the bool, int and uint
rows are, so a build without the menu loads and saves the aspect ratio,
refresh rate, fast-forward and slow-motion ratios and the other float
settings too. samples/settings/float_rows checks those are listed and
that no setting of any kind is listed twice.
Each of the three tests started and ended a Wine server of its own. One
that started as the last server was leaving was cut off before main()
("wine client error:0: recvmsg: Connection reset by peer") and failed
the step with nothing wrong in the tree. Keep one server up for the
step, as the network-clients job does.
The worker lives as long as the process, so a line it logged could
meet the log being torn down at exit. It now notes what it found in an
atomic word, and the callers - dbus_ensure_connection, _close_connection,
_suspend_screensaver, _screensaver_state - log those notes on their own
thread. The worker is built only where retro_atomic has pointer ops, as
it is published through one. dbus_screensaver_test checks the worker
logs nothing itself and that its findings are logged once.
The pacing-wait check gave the fake compositor a 2 ms refresh. Under
TSan on a busy machine the roundtrip took longer, the vblank was already
gone, the wait rightly did not sleep, and the check and the discard
check after it failed. A 250 ms refresh is still ahead after the
roundtrip; the wait sleeps out the rest.
Default the builtin player on when a supported menu and internal MP4 or WebM decoding are available. Exercise the shipping menu workflows with the optional WebM core disabled, so external-file launch cannot depend on that core enabling the setting.
A pass whose stages carry rslc's tables (vulkan, glcore) or its printed
text (slang_process, for d3d and metal) is remapped by rslc from the
pass's own source: slang_compile_remapped() places the members where
the SPIR-V rewrite would (slang_rect_place_pass, now exported), and the
rewritten stages keep their tables and the compiler's own GLSL, HLSL or
MSL, so no SPIRV-Cross crossing is needed for them. Stages from
glslang, or ones rslc will not take, are rewritten on the SPIR-V as
before. The slang_rect and rslc_remap harnesses take their placement
from slang_rect_place_pass.
A bounded seek can finish its slice while waiting for a local metadata span. Requeue that work directly instead of waiting for the next UI poll. Keep external input waits yielding to their producer.
The test gave tshark and the fake server two seconds to start. On a
busy machine tshark was not capturing yet, saw no replies, and the
decode check failed. Wait until tshark says it is capturing and the
server accepts a connection.
Parse UTF-8 SubRip tracks in bounded task slices, retain active-session ownership and publish cue text without reading files on the render thread. Draw cues in texture menus and the RGUI framebuffer, including while transport controls are hidden or playback is paused. Add subtitle enable/delay settings and parser, ownership, drawing and four-menu sidecar coverage.
* sdl3: full screen at the configured monitor's size
* sdl3: drop the display server's unreachable idle_wait
* sdl3: views fixes and the swap interval cap
* sdl3: one display helper, one readback path
* sdl3: sticky views fallback, cap through wrapper
* sdl3: clip turned views, vsync falls back to cap
* sdl3: hotplug, text input, event queue, fixes
* sdl3: keep audio stream across device unplug
* sdl3: release view textures, keep a row fallback, run the SDL3 samples
The per-view textures are released once the core stops sending views,
and kept through a fallback to drawing the frame whole, whose next
frame may be a dupe that draws them again. When the interlaced
placement's batch can't be allocated, its odd rows are drawn one at a
time as before rather than not at all.
sdl3_counters now checks that a reopen keeps the stream other threads
read, and it and sdl3_keyboard run in Linux-samples-gfx against the
SDL3 the job builds, now with audio, since no workflow ran either.
---------
Co-authored-by: libretroadmin <[email protected]>
A 64-bit host asks the VFS to map an archive whatever its size, since
pages are only faulted in as they are touched; a 32-bit host keeps the
256 MiB ceiling so a huge archive cannot take the address space the
core needs, and reads it through the stream.
Lane: archive_stream_test checks the mapping hint for an archive past
the ceiling on both word sizes.
* android: drain the sensor queue with both sensors off
Losing focus disables the accelerometer and gyroscope, but an event
already in the sensor queue stays there, and android_input_poll_user()
only read the queue while a sensor was on. The queue's fd stays
readable, so ALooper_pollOnce() keeps returning LOOPER_ID_USER and
android_input_poll() spins at full CPU. The pause and stop
acknowledgements wait for the next runloop iteration, which never
comes: RetroArch logs "App thread did not acknowledge activity state",
the launcher shows late and the app hangs until it gets focus back.
A swipe home hits it often, since focus is lost before onPause.
Always drain the queue and apply the events only while a sensor is on.
* gl2: make a regrown pass texture as the pass's first one
gl2_recreate_fbo() gave the larger texture no filter, so it kept GL's
default GL_NEAREST_MIPMAP_LINEAR with a single level. On GLES that
texture is incomplete, and once a viewport-scaled pass grew, the next
pass sampled black; desktop GL with texture storage gets a complete
one-level texture. Make it with gl2_create_fbo_texture(), as at init, so
it also keeps the pass's filter, wrap mode and float or sRGB format. The
feedback texture is regrown the same way.
* video: per-screen window settings and the screens' window plan
Each screen gets window settings in the config,
video_screenN_window_mode, _x, _y, _width, _height, _monitor and
_fullscreen: Attached, Separate or Hidden, and where a window of its own
goes and how big it is. Screen 1 is always Attached; the others default
to Separate. video_windows_enable, off by default, is the switch that
lets screens leave the main window at all.
video_windows_plan() turns those settings into a frame's plan: the
screens the main window draws, the ones that go to windows of their own
and the hidden ones. A screen stays in the main window while the switch
is off, where the driver has no windows, or once the renderer has sent
it back. The views layout takes exclude_screens, so the main window lays
out only the screens it keeps, and a window only its own.
This is the base the desktop secondary windows and the Android second
display share. The video_windows harness, run in the Linux gfx samples
workflow, checks the plan and a layout with screens left out.
* video: contract for secondary screens' native windows
A context can offer windows of their own for secondary core screens
through an optional gfx_ctx_window_driver_t, the last member of
gfx_ctx_driver_t: create, update, poll and destroy, and bind and
present for GL contexts. Requests, native handles and events carry a
generation from video_driver_window_generation(), a frontend counter
that a driver or context restart does not reset, so an event that
outlives its window cannot be taken for its replacement's.
GFX_CTX_FLAGS_VIDEO_WINDOWS is what a video driver will report once it
draws to such windows; nothing sets it yet.
* egl: window surfaces for secondary screens
egl_window_surface_new/bind/swap/free give a secondary screen's window
a surface on the main context's display and config, with the main
window's attributes. Bind makes the context current on it at swap
interval 0, or on the main surface again; free puts the main surface
back first where the freed one is current. None of them log.
* video: share the secondary windows' event bookkeeping
What a native layer keeps of each secondary window for the frontend -
its owner, record, request, position, the events waiting one bit per
kind, and whether it closed - is a video_window_common_t at the start
of its own state, and the rules on it live once in gfx/video_windows.c:
marking a window closed (CLOSE, with FOCUS_LOSS where it had the focus,
a waiting gain or resume dropped), poll's drain in its fixed order with
whatever has no room left waiting, and the sweep of an owner's windows
at context teardown.
* video: one placement clamp for secondary windows
video_window_place_rect() puts saved bounds on a monitor or its work
area: bounds partly on it are moved onto it, automatic ones and those
not on it at all are centred, and the size is clamped. Whether bounds
are on it is judged as saved, before the size is clamped, so an
oversized window partly on it keeps its position on the other axis.
* video: frame snapshot for secondary windows
A frame presented with views now carries what a driver needs to make,
place and lay out the screens' own windows without reading settings:
each screen's window settings, the window plan, the core's own map and
frame size, the layout parameters and a restore count, from which
video_driver_window_layout() lays out one screen's window. The snapshot
lives in video_driver_state_t and video_frame_info_t points at it, NULL
while windows are off or the driver cannot make them; the threaded
wrapper copies it into the frame's slot, as it does the statistics text.
While the driver can make windows, the main window's layout leaves out
the screens that have one of their own or are hidden.
video_driver_restore_windows() clears the screens a close attached and
counts the request, and window generations can be taken on any thread.
* video: published layouts and translate for secondary windows
A renderer publishes each secondary window's layout as drawn, with the
core map it was made from and the window's client size, through
video_driver_publish_window_layout() from the thread that runs the
context. Latest wins with no lock: each window has three slots handed
over by index exchange, as the window title is, and NULL clears it.
video_driver_translate_window_point() maps a point in a window to the
core's frame: window 0 through the viewport, as before, and a secondary
through its published layout, scaled from client to drawn pixels. A
point in a window that has no layout, was cleared or has another
generation is not mapped.
* input: window-aware pointer and touch publication
input_driver_publish_window_pointers() and _touches() take each device's
and each touch's window: 0 for the main window, else a screen's own with
its generation. A device in a screen's window is placed through that
window's layout for the pointer and the lightgun, and has no place on
the main window's screen; one whose window is gone reads off-screen.
Relative motion, buttons and the ports' devices are as they were.
input_driver_publish_pointers() and _touches() publish into the main
window, so the input drivers that call them need no change.
* samples: video_screens harness for the screens' own windows
The shared windows base is tested against the shipping configuration,
video and input objects: per-screen window settings across saves and
overrides, a screen's own layout, the frame snapshot and its copy
under the threaded wrapper, published layouts and point translation,
window-aware pointers and touches, and window generations. build.py
links the test with a configured non-Qt build's objects; the OpenXR
desktop jobs, which make one, run it.
* video: the window plan can keep a screen other than 0 attached
video_windows_plan() takes the screen that stays in the main window
whatever its mode, and screen 0 takes that screen's mode. Callers pass
0, which plans as before. Android's external monitor rule needs it:
screen 0 goes to the monitor and screen 1 stays on the device, where
the touch screen is.
* video: route the separate screens to a platform's displays
video_windows_route() chooses the display a platform's Separate
screens go to, from a list the platform publishes. A screen's monitor
setting N names the display of ordinal N; 0 is Auto, which takes an
on, valid and physical display that is neither private nor the main
window's: an external one (DIRECT or TRANSITIVE) first, then the
lowest ordinal. On an external monitor Auto puts screen 0 there and
keeps screen 1, so the touch screen stays on the device. One display
takes screens at a time; a screen bound for another stays attached.
Each display also carries its size, for the menu's display picker.
* samples: a touch through a second output's layout
A touch on a second display's surface, mapped through the layout drawn
there, lands in the touched screen's part of the packed frame: one
screen on a 1240x1080 panel, two screens sharing it, and screen 0 on a
monitor; the letterbox reads off-screen when asked.
* samples: pin the attached screen, picker and pillar cases
The plan keeps a screen other than 1 attached with screen 0 in its
mode, and falls back to Attached with no settings. The picker refuses a
private display but takes one that is off, and Auto never takes an
entry without an ordinal. A touch in a pillar beside two screens, not
reported off-screen, is held to the nearer one.
* video: route the screens' windows through a platform's display list
A platform publishes its displays from its UI thread, latest wins,
through three slots as the window layouts are handed over. Each frame
takes the newest list, routes the Separate screens with
video_windows_route() and plans with the screen it keeps attached; a
screen with nowhere to go stays in the main window. All screens bound
for one display share one window, led by the lowest one, and the frame
says which screens each window shows. Taking a list restores the screens
a failed window sent back, and video_driver_attach_windows() lets a
renderer send them back from any thread. A request names its display,
and Android gets a native window kind.
A screenshot or GPU recording through the threaded wrapper draws the
cached frame again on the video thread. That redraw takes no list: it
keeps the app thread's last route, as it reads the forced and restore
state, so a renderer keeps the second display's window and the capture
shows the right screens. The route is written only when it changes, so
Win32, which routes nothing and whose modal redraw runs beside the app
thread, never writes it while it is read.
The video_screens harness redraws the cached frame through the real
threaded wrapper and checks that it keeps the frame's route and plan.
* video: let a platform take the secondary windows back
video_driver_windows_retire() has the renderer let go of every
secondary window, through a new optional poke hook. It is for a
platform command such as Android's second surface going away, never a
frame. With the threaded wrapper up it is queued to the video thread
behind the frame being drawn, and nothing waits for it: a hook the
caller passes runs there just before and just after, for the platform
to hold its own requests back and to hear when it is done. Without the
wrapper all of it runs before the call returns. One is queued at a
time; false says the video thread cannot take it yet.
video_screens checks that a retire queued behind a frame being drawn
returns at once, and runs after the frame, between the hook's calls.
* video: one slot hand-over for the window layouts and displays
The window layouts and the display list are handed over the same way,
through three slots. Two small helpers publish and take a slot for both,
with the same memory orders. No behaviour change.
The harness adds two display list checks: a longer list keeps its first
VIDEO_WINDOW_DISPLAYS_MAX, and a get of capacity 1 fills only the first
entry.
* android: native side of the second display
The Java controller hands native code the display list and the second
display's surface through three natives. The list goes to
video_driver_publish_window_displays(). A surface becomes a node holding
one ANativeWindow reference, exchanged in for the video thread to take,
with a generation the touches carry. A size change of the same window
keeps the generation; a different window first gives up the old one.
Giving one up uses the main window's ticket handoff: the UI thread waits
while the app thread drops the node and queues the renderer's letting go
of its windows. The thread that lets go of them acknowledges once it is
done, so the UI thread is not left waiting for the app thread's next
poll; a retire the video thread cannot take yet is tried again from the
input poll. Native code asks Java to show or hide the display through
setSecondDisplay(), which posts and never waits, and asks nothing while
the renderer is letting go of a surface. The app state gets a 64-record
ring for the display's touches.
The natives run on the UI thread and read its own app state,
g_android_early, which android_app_free() clears unless a re-created
activity has already replaced it, so a callback after onDestroy finds
none. A display name GetStringUTFChars cannot read leaves no exception
pending for the next JNI call.
* android: second display windows for the gl and vulkan contexts
Both Android contexts give the renderer the secondary-window table.
Creating a window asks Java to show the display it is routed to and
returns unconfigured. Until Java publishes its first display list a
Separate screen resolves to display 0, which names no display, so its
window is refused and the renderer keeps the screen attached until a
list arrives. An update takes only the request's settings; another
screen or request generation is a new window, the renderer's to make.
Its poll hands it the newest surface native code received, raising
GEOMETRY and publishing which window and surface the input poll maps
touches through. The EGL context makes the window's surface on the main
context the first time it is bound after a surface arrives, at swap
interval 0, with the native visual id queried once at init, since the
query logs on failure and binding sits on the present path; the Vulkan
context gives the renderer the ANativeWindow and leaves the surface to
it.
Each request native code sends Java carries a number, and each surface
comes back with the number of the request it was made for. Destroying
the window drops the surface it held, then asks Java to hide the
display. When it was the last window, that hide gives up every surface
made for an earlier request, including one that reaches native code
after it: no window takes one, and its surfaceDestroyed frees what is
left and returns without the ticket handoff. A hide native code starts,
for closing or switching content or reinitialising the video driver,
never holds the UI thread.
* android: touches on the second display reach the core
Each touch on the second display crosses from the UI thread in the ring
with the generation of the surface it was on. nativeTouch sits with the
other second display natives and reads the UI thread's own app state,
which android_app_free() clears. A down or a move goes in only when it
leaves room for an up of every finger the input driver tracks, and an up
only when its down went in, so a full ring never drops a lift and leaves
a finger pressed for the core. MAX_TOUCH moves to platform_unix.h for
the native and the input driver to share.
The input poll drops the touches of any surface but the one the renderer
shows, maps the rest through that window's published layout into the
core's frame, and keeps the fingers after the main display's in the
core's pointer list. They never move the mouse, count as a quick tap or
reach the frontend's own screen pointer, so the menu and overlay stay on
the main display.
* input: the second display's fingers on a device of their own
With an overlay's pointer on, or one of its buttons held, the frontend
answers RETRO_DEVICE_POINTER from the overlay or not at all, and a
windowed overlay polls that device as its own touches. So the second
display's fingers cannot share it: the Android driver answers them on a
frontend-only RARCH_DEVICE_POINTER_SECOND and keeps RETRO_DEVICE_POINTER
to the main display. The frontend lists them for the core after the main
list's pointers, whoever answers for those: the overlay, none while the
pointer is blocked, or the driver. A driver with no second list answers
with its main list alone.
The pointer_second harness checks those answers against a stand-in
driver with both lists, through input_state_device() and the overlay's
pointer answer extracted from input_driver.c.
* android: a Presentation for the second display
SecondDisplayController lists the displays for native code - whether
each is the Activity's own, valid, on, private, a real panel or monitor
(product info, API 31+), its connection, name and size - and shows a
Presentation with a SurfaceView on the display native code asks for. The
window is not focusable and not touch modal and the view is not
focusable, so touching it neither pauses the core nor takes the gamepad;
it keeps the screen on and, on API 29+, keeps system gestures off it. It
follows the Activity from onStart to onStop, holding a request made
while stopped, and a frame after a display or configuration change
publishes the list again when it changed. A display it cannot show on
stays invalid until the list changes, so the display events a
Presentation coming or going may fire do not retry the failing show in a
loop. Each Presentation carries the number of the show request it was
made for and hands its surfaces to native code with it; touches go with
the surface's generation.
* android: second display failures and Presentations Android dismisses
Every display a show failed on stays invalid until the list changes or
the Activity starts again. A single failed id would not do: of two
displays that both refuse a Presentation, marking one failed makes the
other valid again, native code routes back to it, and the two take turns
failing forever. Starting again retries a one-off failure when the app
returns.
Android cancels a Presentation itself when its display's metrics change
(API 17-29). A dismiss we did not make clears it, so the dead
Presentation does not leave its display counted as shown. While native
code still wants that display, it is shown on again if the dismissed
Presentation's view had a surface. One cancelled before that cancels
itself, for example on stale display metrics, and rebuilding it would
loop at message rate; it counts as a failed show, so its display is
published invalid until the list changes or the Activity starts again.
* video: which screen windows a renderer makes and lets go each frame
video_windows_steps() tells a renderer, from one frame's snapshot,
which of its windows to let go of (no screens now, or another
display) and which to make. A window that closed or failed is not
made again until the restore count moves, so frames already queued
do not bring it straight back. Renderers share the rule instead of
each keeping its own.
* gl2: views draw into a given output; the preset's targets can grow
gl2_views_render() takes the layout, size and canvas it draws with
instead of reading the main window's, and the loop that grows the
preset's targets on a resize becomes a function of its own. Nothing
changes for the main window; a screen's own window will draw through
both.
* gl2: draw the screens' own windows
With a context that can make windows of their own for screens, gl2
reports GFX_CTX_FLAGS_VIDEO_WINDOWS while it may draw views. Each frame
it makes and lets go of the windows the frame's snapshot asks for and
draws each one's screens through the views compositor, with the preset's
passes where one is loaded. A window's pass targets grow only once its
surface is bound, so a failed bind has nothing to undo. A window is
drawn after the main window's frame and swapped at interval 0 before the
main swap, which stays the only one that can wait. The menu, OSD,
widgets and overlay stay in the main window.
A window that cannot be made or drawn hands its screens back to the main
window until the next restore and logs once. A window its context closes
hands them back too, and a restore that lands in the same frame does not
make it again; one closed with no screens left is not marked failed, so
it can be made again when its screens come back before the next restore.
The poke's windows_retire lets go of every window at once, for a
platform that takes a surface back.
The gl2_pbo_readback and gl_last_pass_fbo samples include gl2.c; neither
draws windows, so their shared stubs answer the frontend's window calls
with no-ops.
* samples: views test core logs a second finger, and asks for GLES 3
With a finger on each of two displays the second display's finger is
pointer 1. The core reads, maps and logs pointers 0 and 1 through one
helper, each with its own last state; pointer 0's lines and crosshair
are unchanged. Android's gl driver only gives GLES contexts, so a gles3
value of video_views_test_hw lets the core render through hardware there
too. gles3 with the checker or noise pattern logs an error at load:
those patterns upload a BGRA texture, which GLES 3.0 does not take.
* vulkan: the window-system surface switch as a helper
vulkan_surface_create() made the VkSurfaceKHR for each window system
in line. The switch moves into vulkan_wsi_surface_create(), which
takes the instance and returns the surface, so a second output on the
same device can make its own surface the same way. VULKAN_WSI_DISPLAY
stays where it was, since it needs the context's GPU. No behaviour
change.
* vulkan: a second output on the context's device
A vulkan_output_t is a surface of its own with a swapchain on the
context's device and queue, for a screen shown on another display. It
never waits in a frame. The acquire takes no time and uses a fence,
never a semaphore, so nothing the second display's presentation engine
signals ever enters the queue the main output's frames share. The fence
is polled, and an image not yet free is kept for the next frame. Out of
date remakes the swapchain at once; suboptimal remakes it once and then
keeps it, as the main chain does. On Android the swapchain takes the
size the window reports, since the surface's currentExtent can lag an
in-place resize and Android scales any extent to the window. A lost
surface is dropped. A swapchain let go of is destroyed once a fence
queued behind its last present has signalled. Only teardown waits, on
the output's own fences.
The harness runs the real code against a fake device and checks that no
frame-path call waits, that only completed objects are destroyed, and
that everything made is freed.
* vulkan: draw screens in windows of their own
Where the context has a windows table (Android's second display), a
screen the plan sends to a window of its own is drawn there with its
views' chains, sized for the window, and the frame's own view copies.
The window is drawn in the frame's command buffer, after the main pass
and before the chains end their frame. Its image comes from a fence
already signalled, so the frame's submission waits on nothing new; it
signals the window's semaphore, and the window presents after the
submission and before the main present, which stays the only one that
waits. The menu, OSD, widgets and overlay stay in the main window.
A window that can't be made, drawn or kept puts its screens back in
the main window until the next restore. One whose output is not done
yet is let go of on a later frame; the poke's windows_retire and the
driver's teardown wait for it, on the output's own fences.
* video: separate screen windows on by default on Android
On Android a Separate screen goes to a second display, which is what a
device with two panels is for, so the switch starts on there. Desktop
windows stay opt-in. The settings table records the Android default;
the harness checks that an override carries the switch and a screen's
mode.
* menu: a window mode and a display for each screen from 2 on
Each screen past the first gets two settings rows: Attached, Separate or
Hidden, and the display it goes to when it is Separate. The display
picker offers Auto and the displays the platform has published, shown as
the monitor index is ("2 (Screen-2 1240x1080)"), and keeps a display set
elsewhere until it is changed. It widens only up to the most displays
that can be published, so a hand-edited value in the thousands cannot
build a list that long on the main thread. The row's sublabel says that
the display RetroArch is on keeps the screen attached. Each screen's
rows have their own enum, as the per-port input rows do, so the dropdown
and the sublabel find the right one. A change to any of the rows gives a
screen the renderer sent back to the main window another try.
The settings harness checks the rows, the left key's clamp, and that a
change to any of them restores closed windows.
* menu: show the screens' window rows where the driver has windows
Stereo 3D & Screens shows Separate Screen Windows when the video
driver reports secondary windows. While it is on, it also shows the
mode and display of screen 2 and of each further screen the core has.
On Android the switch reads "Show secondary screens on another
display". A screen row's dropdown takes that row's label as its
title.
* android: show the second display again after its metrics change
On Android 10, a density or size change on the second display left it
empty until the app restarted. Android cancels a Presentation when its
display's metrics change, and the one made to replace it got the
display's resources from a cache keyed by display id, still holding the
old metrics. Presentation.onStart() found them stale and cancelled it
before it drew, so the display was marked failed and both screens stayed
on the main display.
Bring the new Presentation's resources up to the display's current
metrics before showing it. The refresh runs only before Android 12; from
then on a Presentation has a window context of its own.
* android: read input queue keys and sticks through their fields
The key and stick helpers of the input queue take the few fields
they read instead of the event, and the default loop's slot lookup
and key path become functions of their own. Nothing changes for the
queue; the second display's window can then hand its keys and stick
moves to the same code.
* android: a focus rule for the Activity and the second display's window
Once the second display's window can take focus, RetroArch's focus
comes from two windows. Where each display has its own focus, the
second window keeps its display's focus while the keys go elsewhere,
so it counts only while in use: after a key, stick move or touch from
it, a focus it gained while the Activity had none, or a hand-off of
the one focus from the Activity. Any other change of the Activity's
focus ends that, so a notification shade still pauses as before.
The rule is a header without NDK types; a host test replays the
sequences the AYN Thor's focus modes and a one-focus system produce.
* android: keys and sticks from the second display's window
Keys and joystick moves the second display's window receives cross
from the UI thread in a ring of their fields; the input poll handles
them with the input queue's own code, so mapping, device ids, axes,
hats and hotplug behave the same. The window's focus comes in as two
commands, and the focus of both windows is acted on once per poll by
the focus rule, only when it changes.
A paused app thread sleeps in its looper, so writing to either of the
second display's rings wakes it, and the poll reads both rings after
the looper so the woken poll acts on what woke it. The app thread
keeps a reference on its looper for those wakes until the activity is
freed.
* android: the second display's window takes focus and passes keys on
The second display's Presentation can take focus, so a tap on it or a
focus mode that locks the gamepad to its display (the AYN Thor's Bottom)
gives it the pad. Its keys and joystick moves go to native input as the
Activity's do. Each key gets native code's answer, so Android makes no
fallback key (BUTTON_B as BACK, BUTTON_MODE as HOME) of a key RetroArch
uses. It is not cancelable, so BACK stays with native code, and it never
becomes the soft keyboard's target. The IME refuses the Activity's text
field while the second display's window holds the focus, so the Activity
asks for the soft keyboard again when it regains focus.
When RetroArch dismisses the Presentation while it holds the focus, it
first brings its own task to the front, which takes REORDER_TASKS
(granted at install). Android keeps one focus for all displays, so it
would otherwise pass to that display's launcher and leave RetroArch
without focus until the next tap on the main display.
* android: hand the second display's surface back without waiting
surfaceDestroyed and a window change on the second display return at
once. The renderer's own ANativeWindow references keep the window
alive; a frame still drawing into it gets an error from the abandoned
queue, and the next frame's poll finds the surface gone, drops it and
lets the EGL surface or Vulkan output go, the output on later frames
while the presentation engine still holds an image. Nothing waits on
the video thread or the GPU, so the UI thread can't stall into an ANR
behind a slow frame, and the window keeps its display for the next
surface. The going surface also gives the windows another try, so a
failure it caused is not held against the next.
vulkan_output_finish() is the driver's teardown only. An acquire it
gave up on keeps its fence, which is never destroyed while it may still
be signalled.
* video: build the screens' own windows only where a platform has them
HAVE_VIDEO_WINDOWS gates the secondary windows: their state in the
video driver and the threaded wrapper's slots, the drivers' window code
and its per-frame calls, the second output and EGL window surfaces,
the input layer's second pointer list and window places, and the
per-screen settings with their menu rows and config keys. Android
defines it; elsewhere ./configure --enable-video_windows does, and the
CI job that runs video_screens_test builds that way. A build without it
compiles none of this and carries none of its state, which is about
15 KB of the video driver's state alone.
* video: a window failure counts only for the restore it was drawn under
A renderer reports a window it could not make or keep with the restore
count of the frame it was drawing. Failures travel in one word, the
screens beside that count, and the main thread takes them only when
the count is still its own. Under threaded video a frame laid out
before a display or surface change can fail after that change; its
failure no longer keeps the screen on the main display until some
later event.
* android: the jelly-bean package may reorder tasks too
It builds the same second display controller, which brings RetroArch's
task back to the front when it dismisses the second display's window on
a one-focus system; without REORDER_TASKS that call throws and the focus
stays on that display's launcher.
---------
Co-authored-by: libretroadmin <[email protected]>
Distinguish useful seek progress from an input wait and requeue bounded worker steps until positioning completes or a newer request interrupts them. Coalesce nearby metadata reads into the existing 32 KiB seek window while avoiding payload read-ahead across distant headers.
Add coverage for worker continuation without UI polls, replacement targets, zero-read warm windows and nearby versus distant header I/O. On AV12.mp4 a fresh 40-minute jump measured 268 ms with preview audio enabled and a 60 Hz poll cadence; rapid seek requests still resume playback.
Record proven runtime expressions so constant evaluation does not repeatedly recurse through their left spine. Emit arithmetic chains iteratively using compile-arena storage, preserving operand order and short-circuit handling. This keeps the existing 2600-term allocation-failure stress test within the default Windows stack.
opts.remap rewrites a stage ahead of every destination, so the GLSL,
HLSL and MSL a driver prints carry the frame rectangle as the SPIR-V
does. Each read of a named sampler becomes a call of a helper whose
GLSL the remap writes for the pass's sampling and the front end
parses after the stage: slang_rect.c's arithmetic, operation for
operation. A function the samplers are passed to is copied for each
way they are. The rectangle's members go after the block's own at
the caller's offset, or into a block of their own. The remap lives
with the printers; a build without them refuses every stage.
samples/gfx/rslc_remap holds it to slang_rect.c: the verdicts on
rslc's own module of every test shader, sampled each way, the
members' offsets, the printed destinations, clean failure out of
memory, and Mesa's draws of rslc's GLSL bit for bit against the
stage reading a texture of its own. With --corpus it checks a whole
corpus the same way.
Without a mapping (smb:// and nfs:// streams, archives over the
mapping cap) the 7z and RAR backends parse through the stream the
archive layer already opened, via the parsers' read-callback APIs and
file_archive_read_at(): listing reads only the headers, and a member's
packed bytes are read when it is decoded. SMB and NFS reads continue
from the network past the read-ahead windows rather than returning
short, and a prefetch seek issued while an earlier retarget is still
pending is judged against that target.
Lanes: libretro-common/samples/file/archive_stream (unmapped VFS
answering in pieces) and samples/file/vfs_prefetch (seek behind a
pending retarget); the smb_retro and nfs_retro vfs_threads_test
read past the windows.
* PS3: Store personal data in per-user directories
Resolve the active console user before loading configuration and store
personal data under /dev_hdd0/home/<id>/retroarch. Keep installation
resources shared and stop initialization if the user directory
is inaccessible.
Import external configurations into the user's directory, rebasing
personal paths while preserving the source. Support both
Sony SDK and PSL1GHT builds and add portable regression tests.
* fix (CI): missing file
* PS3: carry shared USRDIR data over to each user
A user's first boot copies the personal data of the shared USRDIR
layout into their own directory: saves, savestates, remaps, configs,
playlists, autoconfig profiles, histories, core options and the
keychain, then writes their configuration with the paths that pointed
into USRDIR moved to their directories. USRDIR itself is left as it
was, files the user already has are kept, and each file is copied
whole or not at all; the configuration is written last, so a boot
that fails part way copies again on the next one.
When the user's directory cannot be used, RetroArch runs on the
shared layout as before and logs why, instead of exiting. An imported
configuration has one copy per source, which later imports reuse as
the user left it.
---------
Co-authored-by: Jhonathan Corrêa <[email protected]>
Co-authored-by: libretroadmin <[email protected]>
Remember when a pointer press starts outside the menu bounds and discard its release even if it lands inside. Use framebuffer coordinates for RGUI and output coordinates for texture menus. Cover an outside press released over the image viewer in all four menus.
The line saying which Bluetooth address a controller connects to was
added as one of the port's controller entries, so the controller's
display name, profile and VID/PID were shown under it a second time.
It is an entry of its own now, with nothing under it.
A controller paired with no computer holds an address of zeros, shown
as 00:00:00:00:00:00; it now says it connects to nothing.
Most of the tests here link the MP4 demuxer and the H.264 decoder, and
each compiled both again in its own one-command build, which ccache
cannot keep: the slowest step of samples-gfx. They are compiled once a
build now, into a directory of the compiler's and the sanitizer's own,
and linked by every test that needs them. Neither depends on a flag
only some of the tests pass. clean also removes the three tests it
used to leave behind.
Measured on two CPUs, with the same results: the Linux ASan + UBSan
check 232 s before, 157 s after; mingw-w64 under Wine 185 s, 147 s.
On GitHub each job restores a ccache of its own; here they share one.
Each job's "ccache -M 250M" therefore capped the one they all share,
and the jobs evicted each other's objects; and a clean copy is a new
directory each run, which ccache puts in a debug build's hash, so
nothing a run before had compiled matched.
ci_local now gives each step CCACHE_MAXSIZE=1G, CCACHE_BASEDIR set to
the tree, and CCACHE_NOHASHDIR, where they are not set already.
samples-tasks run twice on two CPUs: 756 s, then 419 s, every compile
of the second a hit (2030 of 2030, against 1201 before).
Every trial fed its source a byte, two and three at a time, sources of
up to 8 KiB among them: some two hundred million feed calls under ASan,
most of the test's time, reaching no record boundary the seven-byte
feed does not. Those three feed sizes now go over sources of up to
1025 bytes, past the 1024-byte edges; larger ones keep the 7, 64, 1024
and whole-buffer feeds.
Measured on two CPUs: 103 s before, 54 s after; 238510 comparisons of
the 280000, all matching.
The test ran under ASan + UBSan at -O0, the slowest step of
samples-tasks. -O1, as the other samples build, keeps every check and
runs faster; and the 2 MiB buffer each of the 20000 trials built its
patches in is allocated once, as each comparison reads its own
exact-size copy of the patch, where an overread is caught.
Measured on two CPUs: 138 s before, 103 s after, the same report.
Every suite was built with gcov's instrumentation and followed by an
lcov pass over the whole tree, which reads more each time as suites
add their coverage files. Nothing in CI reads the result: no report
is made or uploaded.
Makefile.test takes COVERAGE=0 to build and run the suites without
it; the default is unchanged. CI passes it, and no longer installs
lcov.
Measured on two CPUs, the unit-tests target: 201 s with coverage,
54 s without, the same fourteen suites passing the same checks.
The check compiles libretro-common three times a run - the host's
compiler and two mingw-w64 ones - and every run compiled all of it,
unchanged or not. Through ccache, which keeps each .su with its object,
a file unchanged since the last run is not compiled again.
Measured on two CPUs: 397 s without ccache, 469 s with an empty cache,
116 s with a warm one, with the same report each time. CI restores the
job's cache from master's, so a run is usually warm. The 19 MB it adds
fits the job's 250 MB cache.
The three slowest steps of a job say where most of its time goes, not
where all of it does: a job of fifty short steps hides its cost in
them. --times lists every step, slowest first.
Size the sample limit from a 64 MiB per-track index budget instead of silently cutting every track at 262144 samples. Reject oversized tables and failed fragment growth instead of publishing a partial movie. Cover tail seeks across the old limit, fragmented tracks and allocation failure.
RGUI reports pointer positions in its menu framebuffer. Compare those positions against the framebuffer dimensions so valid clicks are not rejected by the output viewport bounds.
* fix(ps3): PSL1GHT menu flickering and RSX vertex addressing
Preserve the menu vertex cursor across frames to avoid overwriting
vertices that the RSX may still be reading.
Fix duplicate offsets in menu and font vertex buffers, and present
native menu frames when no core frame or software menu texture exists.
Add regression tests for vertex addressing, buffer wrapping, queued
frame preservation, and menu presentation.
* rsx: keep queued text vertices and duped frames behind the menu
The font vertex cursor now runs on across messages, raster blocks and
frames like the menu's, instead of rewinding to the start of the buffer
after each one and overwriting text the RSX has not drawn yet.
When the menu presents a frame with no new core frame, the last core
frame is drawn behind it again, since the surface it was drawn to was
cleared at the previous flip.
---------
Co-authored-by: libretroadmin <[email protected]>
While a deferred load holds it, the slang compiles take their rslc
arena from a pool of blocks reused stage after stage - one per compile
in flight, taken with a CAS from whichever thread compiles - so a
stage costs the compiler one allocation, its blob. A stage a block
cannot hold is compiled again on the allocator and the block grows for
the stages to come; the last holder frees the blocks.
rslc: a caller's arena survives the scratch arena's releases, and a
compile that runs out of memory at any point, in a caller's arena or
on the allocator, fails with its message instead of reading a node or
list it could not make. rslc_test drives a long stage through every
exhaustion point of a caller's block and every refusal of an
allocator.
* fix(psl1ght,retroachievements): RetroAchievements build
Use native recursive mutexes for rcheevos on PSL1GHT.
* PS3: build RetroAchievements and drop the mock mutex test
PSL1GHT builds now define HAVE_CHEEVOS, so the PS3 CI lane compiles
the lv2 lightweight-mutex rc_mutex backend against the real SDK and
fails if rc_compat ever falls back to pthreads there again. That
replaces the host-side mock sample, which could only check the
backend against its own copy of the SDK constants.
rc_compat.c no longer forces _XOPEN_SOURCE on PSL1GHT, matching the
other native mutex backends.
---------
Co-authored-by: libretroadmin <[email protected]>
The standalone thumbnail harness links the real fade path, which now queries the frontend startup token. Supply the inactive token at that frontend boundary so the regular and threaded sanitizer targets link again.
The webOS tools and NDKs came from four release-downloader calls to the
GitHub releases API, and any one HTTP 500 failed the job. They are now
fetched by webos-deps.sh, which retries each call and keeps the assets
in actions/cache keyed by the latest release tags: a warm cache needs no
download, and an outage that outlasts the retries builds with the cached
releases. webos-deps-test.sh checks both paths against a stub gh.
A deferred directory refresh can retain the previous page's driver metadata. Populate the completed browser page before activating its selected media, so the action opens the internal viewer instead of reloading a content core and rebuilding the video drivers. Assert that startup never queues a content-core load.
Title-bar mouse releases must not dismiss a fullscreen image preview. Reject out-of-window releases before dispatching them to menu drivers. Cover still-image retention in all four menus, including native Windows maximize and restore.
Cover the full output viewport until the viewer is ready and suppress startup widgets. Skip hidden thumbnail fades on external launches and keep the active filename in the viewer header. Exercise widescreen startup, widget suppression and filenames across all menu drivers, with standalone fixture stubs for the new query.
In the sidebar, Scan picks one of the other playlist tabs with equal
chance and does nothing when there is none. The scroll sound plays
only when the tab changed.
Test: samples/menu/playlists_scan's `ozone sidebar<N>` runs cover zero,
one and two playlist tabs.
Closes#19904
Scan on the Playlists tab picks one of the other playlists through
menu_playlists_tab_random_selection(), shared by all four menu
drivers, and does nothing when there is no other playlist.
random_range() now stays inside its inclusive range for every rand()
value.
Tests: samples/menu/playlists_scan presses Scan with zero, one and two
playlists on each driver; samples/menu/random_range holds
random_range() to its range and an even spread across it.
Keep resident timing walks within four milliseconds per preparation step instead of yielding after 256 packets. This avoids hundreds of worker wakeups on long seeks while preserving timeline mapping and decoder preroll. Add a generated two-hour AAC seek regression and PCM comparison.
The step prepares a pass in pass order - its stages, reflection,
framebuffer, pipeline layout and descriptor sets - and hands the
pass's shader modules and pipeline to a RETRO_TASK_FLG_PARALLEL task,
so the pipeline compiles run several at once beside the frame; with
an unthreaded task queue the step builds the pipeline in place.
shader_load_deferred_t.builds counts a driver's build tasks. A
cancel waits for none to be left before the driver's cancel step
frees the chain, and the finalizing step installs the chain, or frees
it after a pass failed, once none are.
menu-resize and menu-resize-running walk RGUI to Screen Distance and
Width, step both with the menu open, and step Width once more just
after the menu reopens. Each step makes every screen's swapchains and
the menu's again at another size. No frame may leave out a screen or
the open menu, every quad shows all of its swapchain, every screen
image is the core's, and nothing logs an error. The walk counts rows,
so the check first confirms that the first press moved the screens.
The test layer's busy new keeps a new swapchain's images busy until
the next xrEndFrame, so every step has a frame with only the old
swapchains to show, whatever the timing. Before the previous commit
both cases fail at every step. The glcore lanes run them too; gl's
screens are cut from its window and do not resize.
Closes#19899
A slot made again at another size destroyed its swapchains before the
new ones had an image, so every xrEndFrame until the release left the
slot out. Each step of Screen Distance or Width resizes every screen
and the menu, so the whole view blinked.
The old swapchains now stay on screen until the new ones are released
and are destroyed after the next xrEndFrame, under the graphics lock.
A forgotten slot drops them too, so a menu that comes back still never
shows its last image. Vulkan no longer destroys the swapchain itself
on a resize.
Windows' own driver cannot run the Wii U GameCube adapter: it fails to
start, so no driver RetroArch has can read it, and it reaches no port.
When the WinUSB scan finds an adapter that nothing could open, it says
so on screen once, and Information > Input Information has a row to
enable it. That moves it to Windows' own WinUSB driver, as the
DualShock 3's row does: one administrator prompt, for a copy of
RetroArch that makes the change and leaves, nothing downloaded and no
driver of RetroArch's own. The adapter then comes back on WinUSB, where
it is read. A DualShock 3 plugged in at the same time is left alone.
Only the DualShock 3 was taken through WinUSB: the scan skipped every
other pad on it, the Wii U GameCube adapter among them, without a word.
The adapter is taken now, and a pad on WinUSB has its interrupt pipes
looked up rather than assumed.
The adapter takes its start (0x13) and rumble (0x11) on its interrupt
OUT pipe, as libusb sends them, not as SET_REPORT on the control pipe,
which the DualShock 3 keeps. They go the way a pad on Windows' HID
driver is written to: one write in flight, the newest waiting behind it
for a later poll, so the frontend's thread never waits on one.
Keep thumbnail uploads running while suppressing the parent browser during external image and movie launches. Release presentation when the activated fullscreen view is ready, or recover on cancellation and decoding failure. Cover all four menus with captured startup frames and lifecycle checks.
* glslang: stop exporting the builtin copy to cores
A Unix build links the builtin glslang into the executable with default
visibility. When a library in the link also defines or references
glslang (on Arch, libavdevice pulls in libavfilter -> libplacebo ->
shaderc), ld exports those definitions, and a core that bundles and
exports its own glslang binds its calls to RetroArch's older copy
instead. flycast's Vulkan renderer then crashes in GlslangToSpv on its
first shader compile.
Compile the builtin glslang with -fvisibility=hidden so it never leaves
the executable.
* spirv-cross: keep the builtin copy out of cores' reach
The builtin SPIRV-Cross objects are compiled with -fvisibility=hidden,
like the builtin glslang, so a core bundling its own SPIRV-Cross always
binds to that copy.
tools/builtin_deps_export_check.sh reads the built glslang and
SPIRV-Cross objects and fails if any of them defines a symbol ld could
export; only libstdc++ header instantiations are allowed.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Shader settings gain Simulated Input Resolution with a width and
height, off by default. While it is on, a slang preset's first pass is
told its source has that size, so resolution-sensitive shaders such as
CRT and edge-smoothing ones treat an upscaled frame as the small image
they were written for. Later passes keep their real SourceSize.
The size is recorded in the preset when it is loaded and every slang
backend reads it from there: glcore, Vulkan, D3D10/11/12, Metal and
gx2. Changing a setting reapplies the preset. The rows appear in the
shader quick menu only when the video driver takes slang.
slang_hlsl_backend checks that the size reaches the first pass alone
and survives the copy the D3D drivers make of a pass's semantics, and
runs in the gfx samples workflow.
Co-authored-by: libretroadmin <[email protected]>
Co-authored-by: hunterk <[email protected]>
Cancel unaccepted input and reset the explicit reference chain without reopening a healthy raw backend. Keep exported GPU leases independent. Opaque packet decoders and failed sessions retain the close/reopen fallback. Restore the cached initial HDR state before replay and cover busy restart, retained output and fallback admission.
Read bounded OBU headers and complete metadata records without requiring whole compressed samples. Refill sparse input waits within the existing worker budget and skip file gaps without reading page prefixes. Preserve the HDR checkpoint timeline and ordinary decoder waits. Cover range bounds, skipped payloads, HDR changes and sparse worker progress on Windows and Linux.
A deferred slang load on d3d11 or d3d12 compiled one pass after
another: the step processed a pass, ran D3DCompile on both stages
and made its objects before the next pass began, so a long preset
spent its load in fxc one stage at a time.
The step now processes the pass (slang_process, which needs the
chain's semantics) and hands its HLSL to a build, which compiles the
stages and makes the shader objects or the pipeline and the constant
buffers. The builds are RETRO_TASK_FLG_PARALLEL tasks, so with extra
workers several passes compile at once beside the frame; with an
unthreaded task queue the build runs in the step as before.
- The load state is reference counted: the load holds one, each
build one, and the last to let go frees what the passes made. A
cancelled or failed load frees its LUTs and preset at once; builds
still running finish into the state and drop it.
- Each build holds a reference to the device (and on d3d12 to the
root signature) for its run.
- A build that fails marks the state; the next step ends the load as
a failure in the step did.
- The finalizing step waits, one frame at a time, for the builds
still running before it installs the chain.
- d3dcompiler_common: D3DCompile is resolved once, by
d3d_compiler_load() at the load's begin, before any build runs;
the builds only read it. A load whose compiler cannot be loaded
steps on the chain thread as before.
* libretro-common: stop exporting MD5 and krb5 symbols
libretro-common's MD5_Init/Update/Final and krb5_encrypt/decrypt/
string_to_key share their names with OpenSSL's libcrypto and MIT
krb5's libk5crypto. When either library is in the link, ld exports
the executable's definitions and their callers bind to them instead:
libssl's MD5_Init lands in RetroArch's copy, whose MD5_CTX is 152
bytes against OpenSSL's 92, and so do the MD5 calls of a core that
bundles LibreSSL (citra).
Compile md5.c and net_krb5.c with -fvisibility=hidden so these
definitions stay inside the executable.
* libretro-common: hide MD5 and krb5 in the sources, check it in CI
Hide md5.c and net_krb5.c's definitions with a visibility pragma in
the sources instead of a Makefile.common flag, so griffin builds and
cores carrying libretro-common get it too. net_krb5.c's includes move
above the pragma, which would otherwise hide libc and socket
declarations. The pragma is popped at the end of each file, so
nothing after them in griffin.c is affected.
The Linux CI job checks that neither object exports a function.
---------
Co-authored-by: libretroadmin <[email protected]>
* replay: accept captured normal states for BSV checkpoints
Let netplay supply a frame's saved state after confirmation without serializing the current speculative core state. Preserve the existing on-disk format and local recording path.
* netplay: commit BSV input frames after rollback confirmation
Stage the input queries seen by the core and replace them during rollback. Commit each confirmed frame once, using portable initial states and checkpoints captured at its emulated boundary. Keep local input sampling and stalled iterations out of the movie. Reject unsynchronized starts and stop at state discontinuities.
* test: cover confirmed netplay BSV recording and offline replay
Exercise the shipping recorder and rollback loop with delayed input, ring wrap, mid-session starts, stalls, absent ports and checkpoints. Compare host, client and spectator recordings, then replay through the ordinary BSV reader. Add sanitizer and Windows CI coverage.
* netplay: keep replay state access in the input frontend
Keep recording flags, checkpoint requests and notifications in the input frontend. Exercise the same frontend calls in the netplay BSV harness.
* netplay: free BSV staging and keep netplay movies out of savestates
Free each staged frame's states once it is committed, and drop all
staging when a netplay recording stops, so a session no longer holds a
normal savestate per ring slot. Skip the speculative initial state at
record start, keep netplay movies out of savestates since they lag the
core by unconfirmed frames, and keep the recorder's state in flag words.
The netplay BSV tests check that committed and stopped recordings keep
no states, and the MinGW lane links statically like the other wine lanes.
---------
Co-authored-by: libretroadmin <[email protected]>
A per-port "Controller Touchpad" setting says what the touchpad on the
port's pad is to its core, where the pad has one:
- Touchscreen, the default: each finger on the pad is a touch on the
pointer device at that place on the screen, two fingers two touches;
while no finger is on the pad, the screen's own touches are read.
- Mouse: moving a finger moves the port's mouse, added to the mouse's
own moves, and pressing the pad in clicks - a right click with two
fingers on it.
- Off.
The pad's state is read through the joypad driver's view, as the poll
left it, so the touch is as new as the buttons and nothing is copied.
A mouse's moves are worked out once a poll, at the first ask.
Only the DualSense's touchpad click was read, as a button. Both pads
send two fingers in every full report: each one's place, and whether it
is down. The handlers now keep them in the pad's view, where the
pointer device puts a place, with the pad's press beside them, so the
frontend reads them with the buttons at the poll, through the view the
joypad driver already lends, with nothing copied. A finger's x and y
are packed into one word.
The two pads share the layout, read by sony_pad_touch.h, with the reach
SDL gives each. Other makes' pads are not known to lay theirs out the
same and are taken to have none.
RETRO_TASK_FLG_PARALLEL lets a task run beside any other on extra
workers that take only tasks marked so; every other task keeps to the
one worker, one at a time as before, and no task is in two handlers at
once. The frontend sets how many extra workers start (cores less
three, from retroarch.c); none by default. Detachable and main-thread
tasks ignore the flag, and the GCD runner, whose tasks already run on
the system's concurrent queue, is unchanged.
A deferred shader load pushes up to four parallel compile tasks, each
taking the next pass and compiling a file once however many passes
name it; the stepper lists what they finished before each step and
parks until the pass it needs lands. The blob table's claim moves with
the stepper, whose calls may each run on a different worker or GCD
thread. task_queue_parallel_test covers the runner.
Selecting a device showed nothing: whether it paired went only to the
log. A pairing now says on screen when it starts and whether it worked,
and a Wii controller that could not be paired gets a reminder to press
its sync button and select it again while its lights blink.
The scan task calls the driver from the task thread while the frontend
can stop the driver from its own, so quitting during a scan freed the
driver under the task, which then ended the scan on freed memory and
corrupted the heap.
The driver started is now held by a count: the frontend's hold, and one
taken for each scan, on the frontend's thread, when the scan task is
pushed. Stopping the driver lets go of the frontend's; whichever lets go
last frees it, so a scan running finishes on a live driver. No lock and
nothing waited for.
A fixed startup sleep can capture a cold Wine prefix before the application has started its core. Require a PLAYING or PAUSED GET_STATUS response before settling and comparing frames. Keep the existing retry for a loader that exits before logging.
Use the existing frame-time helper when a paused driver reports no refresh rate. Dividing by zero produced an invalid timer margin and a signed overflow in the imageviewer sanitizer smoke run.
Restart parser, reference and output ordering state while retaining a healthy backend session and its slot ownership. Keep the existing reopen fallback on software, broken sessions or allocation failure. Exercise repeated restarts against the checking backend.
Continue from reconstructed references when the target is ahead and no later cue offers a shorter path. Preserve held packets, honor lowered residency bounds and restart for backward or unsafe replacements. Keep a warm preview file window when its new cursor is still resident. Generated pixel, cancellation, timing-prefix and counted-I/O tests guard both paths.
Use unsigned arithmetic when filling the inflate bit buffer. Supply a fail-fast context release stub for the D3D9 harness, whose device path does not release a GL context.
An unset bus address invokes dbus-launch during SDL startup. Give each headless case an unreachable address under its private runtime directory, so startup cannot wait for an autolaunched desktop session. Four blocked-launch SDL cases fail before this change and render successfully with it.
The hid driver's DualSense handler gives its PS button as button 16 and
already opens the menu with it, but its built-in profiles bound nothing
there, so Menu Toggle showed None. They now carry it, so it shows as
the profile's and a saved profile keeps it.
Bind last, golden and altref to completed frame arenas and decode into an unreferenced fourth arena. Preserve reference-copy ordering and borrowed-plane lifetime, and release retained macroblock state if an arena allocation fails. Generated independent pixel references cover hidden frames, cropped geometry and resolution changes; counted large copies and injected allocation failures guard the ownership path.
Windows asked the user for a Wii controller's PIN, which is the radio's
own address in binary and cannot be typed. The driver now registers for
the device's authentication and answers its PIN request itself, falling
back to the older call without Vista's functions. A device whose name
the search had not heard yet is looked up again before pairing, so a
Wii controller is not taken for another device.
Quitting while a pairing waited on Windows hung, as the driver joined
its thread. A thread still running is now let go, and the last of it
and the driver to finish frees the driver.
A profile's hotkeys live with the controller's profile, not the port's
binds, so a profile saved under the device's name - a new file when it
came from a downloaded profile or a built-in one - lost every hotkey it
had, such as a Menu Toggle on the pad. They are written to the saved
profile now, with their labels.
Ozone and XMB must reopen the selected fullscreen file even when ordinary thumbnails are disabled. Exercise application fullscreen transitions with real movie pixels and seeking across all four menus, with direct and threaded GL presentation. Windows tests can require the DXGI bridge.
The Windows driver corrupts H.264 reference pictures through Vulkan Video, including in independent FFmpeg decoding. Reuse the matched-adapter packed GPU export path without CPU readback. Keep other drivers on Vulkan Video and leave correct software fallback when GPU interop is unavailable. Check generated H.264 exports against software, including cropping.
Avoid restoring a cursor that no prefix fill consumes. Consecutive window reads no longer seek; metadata probes still leave the residency frontier untouched. Keep cursor restoration for prefix fills and whole-file fallback. Count reads and seeks in the generated window regression.
A pass is slang_process, two D3DCompiles and the device's object
creation, all free-threaded; the context and the queue are touched by
begin and by the finalizing step only, which stay on the video thread.
The begin says so, and reads into the load's state the frame-rectangle
flags the step used to read of the driver. fxc's time on a heavy pass
then runs beside the frame instead of stopping it.
Abort if the scripted fence queue unexpectedly reaches single-context handoff. Keep the production publication path linked after it began calling the context-release helper.
Fail fast if deferred shader ownership reaches the synchronous cycle harness. Normalize fixture and expected path separators for native Windows, preserving preset identity and order checks.
Create the prepared sound while the initial header is resident, before seeking can decommit earlier cluster headers. Retain its window owner immediately so cancelled preparations release borrowed data safely. Exercise later WebM seeks and cancellation with float and s16 output against resident reference audio.
The Edge sends its Fn buttons and its two back buttons in the high
bits of the third button byte, which the handler dropped, so they could
not be bound. They are now buttons 19 and 20 (Fn) and 21 and 22 (back
buttons, left first), as SDL reads them. The Edge also gets the
DualSense's built-in autoconfig under its own name.
task_queue_push() refuses a task without running or freeing it, so
the shader precompile's task was leaked on that path, with a comment
saying the queue had freed it. The pusher frees it.
A driver whose pass step may run on any thread says so from its begin
(the chain is not on screen before the finalizing step, and a pass's
objects are made at device level), and the load's task then runs the
steps themselves, one per call, marking the load busy around each so
the main thread's tick waits; the finalizing step stays on the chain's
own thread. vulkan opts in. The blob table serves one thread at a
time: the task claims it for the load, the other threads' lookups
answer nothing meanwhile, a listed blob whose last reference goes on
any thread is parked on a dead list the table drains at its next
operation, and the counts are atomic. The debug object counters in
vulkan_common are atomic for the same reason.
On the single-context ring the push held the context across its
waits on the next slot's fence, which the video thread signals under
the same token, and the video thread took the token before claiming
a frame, which a push waiting for ring space depends on. The push now
gives the context up as soon as the frame is captured; one context
runs its commands in order, so the cross-thread fence waits are not
needed on it and are skipped. The video thread takes the token after
its claim.
egl_hw_core's shim gains a swap slower than the push's wait for the
video thread's turn, and the lane runs the one-context case through
it.
2026-10-24 01:18:12 +02:00
6820 changed files with 406115 additions and 439413 deletions
make clean check THREADS=$threads SANITIZER=address,undefined
make clean THREADS=$threads CC=x86_64-w64-mingw32-gcc LDFLAGS=-static && make -j$(nproc) all THREADS=$threads CC=x86_64-w64-mingw32-gcc LDFLAGS=-static
Loaded 100 of 6820 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Codex