A download from these servers was gathered whole in memory, up to the
256 MB body limit, then written to the local file in one piece; the
WebDAV driver carried a TODO asking for exactly this. They now use
task_push_http_download_file_with_headers: the body goes to the file
in pieces as it arrives, and the callback opens the finished file to
hand on. The HTTP error text these drivers log still reaches them.
A body with no Content-Length or chunked framing is still refused, and
its file is removed with it. A 404 (or S3's 400) still means the
server does not have the file; the download removed the file already.
A download was gathered whole in a buffer of the size the server
reported, then written to the local file in one piece. The local
file is now opened first and each SMB read goes to it as it arrives,
through one buffer the size of a read: no copy of the whole file in
memory, and no allocation sized by the server. A failed local write
fails the download; it was not checked before.
A downloaded file is written in one piece (WebDAV, S3, Google Drive,
SMB, iCloud, iCloud Drive) or in 64 KB pieces (NFS), and the sync
task reads it back 256 KB at a time to hash it. These streams are
opened read-write, which takes the descriptor path on the desktops
and Android now, so they ask for it: the body goes to the file in one
write() with no copy through a stdio buffer, and reads back without
one. Where descriptor writes are not available the hint is ignored.
The updated manifest keeps its buffer: the JSON writer hands it over
1 KB at a time.
Each input query copied every event it looked at out of the array and,
on big-endian, byte-swapped its id again. Compare in place against an id
converted once, with the id checked first as it differs most between a
frame's events. The netplay recorder's own lookup does the same.
Key and input events were read and written one record at a time, though
the arrays already have the file's layout: move each frame's events with
one stream call straight to or from the array, and none for an empty
list, which is most frames. The push functions build the event in its
slot, after the capacity check, and playback passes key events on from
the array.
The Win64 build's -Wshorten-64-to-32 narrowings are made explicit where
the value is bounded: socket()/accept() handles kept as RetroArch's int
descriptors, ai_addrlen passed as socklen_t, UDP and pipe lengths,
COM/D3D counts and shader frame counters, WASAPI/XAudio2 buffer sizes.
net_socket clamps each send/recv to Winsock's int count on _WIN32 and
lets the existing loops move the rest; the remote-input descriptors are
plain ints on every platform, as socket_init returns them.
xaudio29.h declares its interfaces through a macro that, in C, leaves
out DECLARE_INTERFACE's second typedef of the forward-declared names.
Follow-up to "MCP: fix token length, SIGPIPE, stdio EOF, cleanup".
- input_driver: the generated token is saved at once, but
config_save_file refuses while overrides are active (and is skipped
with no config path). The modified flag was dropped with it, so a
token made with a core's overrides loaded was lost on exit even with
config_save_on_exit. Set the flag as before, then save now.
- mcp_server stdio: ignore SIGPIPE directly instead of calling
network_init(). stdio uses no sockets; network_init brings up
WSAStartup on Windows and the network stack on consoles for nothing.
- mcp_structured_record: take strlen(seps) once, not per column, and
guard an empty seps string, where n - 1 would index far out of range.
Each of the three UPnP replies (device description, external address,
port mapping) was copied into a NUL-terminated heap buffer, then
copied again by rxml_load_document_string, which keeps its own. The
buffer is now handed over with rxml_load_document_owned, which keeps it
as the document's or frees it on failure, so the second copy and its
allocation are gone and the caller no longer frees it.
natt_desc_parse_test passes.
s3_backup_then held six path buffers on the stack, 12.8 KiB, and
s3_read and s3_delete_object two each, a little over 4 KiB. The backup
names now go in one allocation, and the URL and canonical URI of an
object come from s3_object_urls, one allocation holding both, which
s3_update_upload uses too. The largest frame in the driver is now
s3_build_request_url's 2.1 KiB. The hand-copied empty payload hash in
the GET and DELETE goes to S3_EMPTY_PAYLOAD_SHA256.
s3_backup_test fails those allocations for a backup before an upload
or a delete, a plain upload, a delete and a read: nothing starts,
nothing is reported, nothing leaks.
An upload read the whole file into one malloc before it started: up to
the 64 MiB multipart threshold for a single PUT, and the whole file,
however large, for a multipart upload, held until its last part was
acknowledged. Out of memory the upload failed, and on the consoles the
threshold is sized for, that is not far off.
The body is now pulled from the file by net_http as the socket takes
it, through net_http_connection_set_content_source, the way WebDAV
already uploads: one send buffer instead of the file. Each multipart
part is a byte range of the file, seeked to on its first run so it
reads its own bytes whatever was read before it, and a replay on a
fresh connection rewinds to the range's start. The payload hash for the
signature is taken by reading the range in 64 KiB runs into
sha256_stream_update. The fallback to a single PUT when the multipart
start fails in transport streams the whole file the same way.
task_push_http_transfer_with_source is task_push_http_transfer_with_
content with a source in place of the buffer. The upload's URL buffers
move off the stack (4.2 KiB to 112 bytes).
s3_backup_test's stub drains every streamed body as net_http would,
with a replay from the start, and checks it is all there and is what
was signed. New lanes: an 11 MiB multipart upload (three parts, each
its own range of the file), its single-PUT fallback, and an empty file.
The multipart threshold can be set by the test for that.
The SigV4 signer assembled the canonical request and the string to
sign in 4 KiB stack buffers and MACed them with its own HMAC. A URI
or copy source past 4 KiB was signed cut short, and so was a secret
key over 123 octets (the setting takes 185), so those requests came
back 403. The two buffers also put the signer's frame at 9 KiB.
The canonical request now goes straight into sha256_stream_update as
it is made, and the string to sign into hmac_sha256_update, both from
crypto/kdf.c: nothing is assembled, so nothing is cut short. The query
is sorted as an index of spans over the caller's string, on the stack
up to 16 parameters, so the signer allocates only its result. The
hand-written HMAC, the query copy and the old signature helper go.
HAVE_S3 now needs HAVE_CRYPTO, which every build that defines it
(Apple) already has; griffin says so if it is missing.
s3_backup_test checks seven signatures against an independent SigV4
computation, including a 5001-character URI and a 150-octet secret,
which master signs wrong. The query and out-of-memory tests follow the
new shape, and the test rebuilds when the driver changes.
s3_canonicalize_query_string splits the query on '&', sorts the
parameters by name and then by value in byte order, and joins them
again; names are compared on their own, so a name that extends
another sorts after it. Callers keep URL-encoding each name and value.
When the canonical headers or query cannot be allocated the signer
returns no signature, which s3_build_auth_header_ex passes on as NULL
and every caller treats as a failure.
s3_backup_test checks the sort order, including names that extend
one another, and fails each of the signer's allocations in turn.
Closes#19956
The cnonce is written only when a challenge is accepted and cleared
with the rest of the digest, so webdav_create_digest_auth_header()
builds nothing while it is empty: after a challenge the parser
rejected, the next request goes out with no Authorization header and
the server's 401 is reported as usual.
webdav_reauth_test sends rejected challenges (an open-ended nonce or
realm, no nonce, qop="auth-int") after an accepted one and checks
that no header is built from them.
Closes#19954
A blocking driver's calls reach the worker on a many-producer stack and
its results come back on another; the worker parks on an eventcount
when the sync is still open and the queue is empty, and leaves once it
ends. A call queued as the worker leaves either starts a new worker or
is taken by the leaving one. cloud_sync_deinit() waits for the worker
on an eventcount, still bounded by its timeout.
Without pointer atomics the calls run directly, as without threads.
The async test queues a sync from the worker's leaving point.
lakka.h and lakka-switch.h join the rest of Lakka's OS layer in
os/lakka; switch_performance_profiles.h, which only HAVE_LIBNX builds
read, starts os/libnx. Includes and the iOS Xcode projects follow.
* vfs: close a private SMB connection when an open, stat or opendir fails
When every pool slot is held, smb_take() connects privately for the
caller. If the open, stat or opendir then fails, smb_heal() returns
NULL when the connection is alive (a missing path, the usual case) or
cannot be replaced, and the caller stored that NULL over its context
and gave back nothing, so the connection and its socket were never
freed.
Keep the caller's context when smb_heal() returns NULL, so smb_give()
closes it. A pooled slot is given back as before.
vfs_test holds both pool slots with open files and then stats, opens
and lists a missing path; LSan reported the three connections before.
* vfs: release an SMB stream's connection when close follows a failed mend
When a stream's connection cannot be mended (the server is gone and
the reconnect is refused), or its file cannot be opened again, the
mend clears the stream's handle. Close returned early on the missing
handle, so a private stream leaked its connection, a pooled stream
kept its slot busy for the pool's life (smb_shutdown then left the
pool unfreed), and the reopen record leaked either way.
Close now checks for the connection rather than the handle and closes
the handle only when there is one. After a failed mend a private
connection is freed without a goodbye, since the server may be gone.
smb_idle_test gains a mode in which the server is stopped for good
during the pause, and local_samba_test.sh runs it, so the sanitizer
lane's LSan reports these leaks.
* cloud sync: make and drop SMB contexts under the VFS's list lock
libsmb2 keeps every context on one process-wide list with no lock of
its own, so the SMB VFS makes and drops its contexts under a lock
(4e6885d5a3). The SMB cloud sync driver called smb2_init_context() and
smb2_destroy_context() directly. A sync that began or ended while the
VFS connected or disconnected on another thread, as during a scan,
could lose a list entry, after which a live connection reads as dead,
or break the list.
The VFS exports its two wrappers and cloud sync uses them. The
built-in client has no such list; there they call straight through.
cloud_sync_test runs cloud sync's connect and disconnect beside private
VFS connections; the libsmb2 TSan lane reported the race before.
* scan: readers take smb:// paths when an SMB client is built
The scan's parallel readers were kept off every "://" path. smb://
paths go through the SMB VFS, whose connection pool serves several
streams at once, and the commits before this one fix the connection
leaks and the libsmb2 list race that readers would otherwise hit. So
readers now take smb:// paths in builds with an SMB client, built-in
or libsmb2; other schemes stay with the scan.
database_scan_test adds smb://, nfs:// and ftp:// archives to a
threaded scan's list and counts the readers' reads of each. Its TSan
build gets a stub SMB backend, so CI runs both halves.
* vfs: keep an SMB pool per share, never freed under its users
Pools live in a static table, one per connection key. Each pool's state
and user count share one atomic word: a lookup or a held slot is a use,
only an idle pool is evicted, and a retired pool closes its connections
when its last user leaves. Readers on two shares each keep their pool's
connections, and no thread touches a freed pool.
A failed reconnect after a dead transport frees the connection without
the round trips of a logoff; a pooled slot reconnects on its next take.
vfs_threads_test runs threads alternating between two shares of one
directory with no share configured.
---------
Co-authored-by: LibretroAdmin <[email protected]>
A client compares the host's CRC for a frame either when it arrives or
once the client reaches that frame, and both now go through one check.
The first check at or after a state loaded from the host decides
whether this core's CRCs can be compared: that state is the host's, so
a mismatch there turns desync checks off with a warning, as a core
whose states carry a timestamp needs. Any later mismatch asks for the
host's state. A client logs each state it loads from the host.
samples/network/netplay_e2e gains a rollback desync lane with input
latency, so some checks wait for the client to reach their frame, and
a lane whose core states never hash alike between peers, which must
stop at one resync past the join with its checks turned off.
NETPLAY_E2E_JUNK puts a per-peer serialize count in the test core's
state.
A client behind a lockstep host can be handed a state labeled with a
frame it has not reached yet, and when that state is read after the
frame has advanced the forced rewind loads it one frame past run. The
rewind now makes the state's frame the one run next, so the client
never runs an earlier frame's input on top of the host's state and
the resync actually mends the desync.
samples/network/netplay_e2e gains a lane where the client runs slower
than the host and has its RAM upset, and NETPLAY_E2E_PACE_MS to set a
peer's pace.
Rollback keeps a savestate for every buffered frame and serializes every
frame, which a state in the tens of megabytes (Dolphin) cannot afford,
and needs states that restore deterministically enough to rewind on.
Lockstep runs a frame only once every player's real input for it has
arrived and never rewinds, so all delta frames share one state, used
only to hand a joiner the game or to resync. Desync checks hash the
core's system RAM, or the RAM in its memory map, rather than a whole
state; a state is serialized for them only when the core shows neither.
Input latency rises by a frame whenever lockstep has to wait and falls
again once input keeps arriving early, so it holds full speed over a
slow link. A host keeps sending while it waits on a joiner, grows its
buffers if the core's state has grown, and starts on the running game
instead of reloading it. A client takes a state labeled with a frame
whose input it already has, and loads it there.
Lockstep is a local choice and plays with rollback peers of this
protocol, and it is on by itself for cores past 32 MB of state and for
cores whose savestates load but cannot be rewound on, which netplay
used to refuse. The Lockstep setting asks for it everywhere. Older
clients cannot join a lockstep host.
samples/network/netplay_e2e runs a host and a client over the loopback
on a test core: sticks synced or not, lockstep, a lockstep host with a
rollback client, a desync mended and a 40 MB state.
Netplay sent a joypad port's buttons only, so a core reading the sticks
of a joypad port (N64, GameCube and others) desynced or saw them at
rest on every remote player. The host now sends joypad ports as analog,
three words instead of one, with bit 30 marking them; each side hands
the core the joypad device it set, so nothing changes for the core.
This is protocol 8. A host with Sync Analog Sticks on, the default,
turns away older clients with the out-of-date message rather than
letting them desync; with it off, any version still joins.
Resume direct hosting after an empty relay response or timeout instead of deferring another lookup. Cover failure, retry, success, deadline and cancellation through the shipping decision functions.
* MCP: schema check on all tools, readable MENU
* MCP: MENU marks choices, answers when closed
* MCP: MENU types, shows dialogs; drop menu option
* MCP: fold INPUT, OPTION, CORE_OPTION; trim list
* MCP: MENU path, more, tab names; clearer errors
* MCP: menu title as a text line; trim the list
* MCP: trim STEPS' descriptions
* MCP: INPUT says the core gets what is held
* MCP: MENU review fixes
* replay: accept captured normal states for BSV checkpoints
Let netplay supply a frame's saved state after confirmation without serializing the current speculative core state. Preserve the existing on-disk format and local recording path.
* netplay: commit BSV input frames after rollback confirmation
Stage the input queries seen by the core and replace them during rollback. Commit each confirmed frame once, using portable initial states and checkpoints captured at its emulated boundary. Keep local input sampling and stalled iterations out of the movie. Reject unsynchronized starts and stop at state discontinuities.
* test: cover confirmed netplay BSV recording and offline replay
Exercise the shipping recorder and rollback loop with delayed input, ring wrap, mid-session starts, stalls, absent ports and checkpoints. Compare host, client and spectator recordings, then replay through the ordinary BSV reader. Add sanitizer and Windows CI coverage.
* netplay: keep replay state access in the input frontend
Keep recording flags, checkpoint requests and notifications in the input frontend. Exercise the same frontend calls in the netplay BSV harness.
* netplay: free BSV staging and keep netplay movies out of savestates
Free each staged frame's states once it is committed, and drop all
staging when a netplay recording stops, so a session no longer holds a
normal savestate per ring slot. Skip the speculative initial state at
record start, keep netplay movies out of savestates since they lag the
core by unconfirmed frames, and keep the recorder's state in flag words.
The netplay BSV tests check that committed and stopped recordings keep
no states, and the MinGW lane links statically like the other wine lanes.
---------
Co-authored-by: libretroadmin <[email protected]>
* MCP: cold paused, held input, stacks, arg types
* MCP: checkpoints, prompt, completions, records
* MCP: completion errors, sized prompt, CI shaders
* MCP: rollback via loader, STOP n, completion fixes
* MCP: ask input_driver whether a replay records
* mcp: Clarify steps, and ensure MCP standards
* MCP: queued STEPS, checkpoint step, 32 steps
* MCP: queued STEPS say how long they sat idle
* MCP: queue ends on load/reset; simplify
* MCP docs: queue stops, idle, list caching
* MCP: shorter STEPS wording
* MCP: notice when port taken, --command, taps
* MCP: drop the play_game prompt and completions
* MCP: queue, tap and reset fixes from a review
* MCP: queue outcomes, hex values, field fixes
* command: --command takes an empty host or port as the default
"CMD;;PORT" and "CMD;HOST;" fill in the default host or port, so an
empty segment after a ";" counts as a host or port field again. A STEPS
list still keeps its own ";" steps, which carry a space, a "|" or are
"checkpoint".
---------
Co-authored-by: LibretroAdmin <[email protected]>
* MCP: answer once when reply_to answered already
* Commands: refuse slots past INT_MAX
* MCP: follow-ups to WATCH and the folded tools
* Commands: take CRLF lines; keep long family names
* MCP: fixes from a review of the server
* stdio_e2e: tool list size after the combined fixes
* MCP: lone CR, over-long stdio lines, slot cap
* MCP: stdio keeps its agreed revision
* docs: MCP server doc matches the code again
* Leave CHANGES.md out of this change
* MCP: STEPS, PRESS until, 16-bit waits, 16 watches
* MCP: 4 kB off tools/list; CORE; LOAD_CONTENT paused
* MCP: refuse unknown params; taps; review fixes
* MCP: GET_LOG, READ, RUN_UNTIL say what they mean
* MCP: LOAD_CONTENT's frame back under budget; STEPS counts its taps
LOAD_CONTENT copies the content path, to the heap, only where a pause
setting follows it, which keeps its stack frame where it was. STEPS
counts the frame let go between two presses of a button against the
frames one request runs, so a list it accepts can finish in them.
Lane: the taps case in samples/network/command_frames.
---------
Co-authored-by: LibretroAdmin <[email protected]>
Real-client masks are built by one helper that skips the slave's own
input, filed under MAX_CLIENTS, so no shift reaches 32. The remaining
client-number masks shift unsigned.
A core's input query reads only the words its device's state carries:
analog sticks by index 0..1 and axis 0..1, analog buttons as their
digital bit at full pressure, joypad and pointer bits below 32, and
keyboard words below the state's size; anything else reads 0.
samples/network/netplay_input covers both helpers.
1 << n on an int is undefined when n is 31. Netplay's loops over
MAX_CLIENTS (32) and its input-share loops over 32 bits reach it; the
masks they build are uint32_t, so its variable shifts become 1U << n.
turbo_pressed kept its 'turbo held' flag in the sign bit of an int32_t,
set with 1 << 31 and tested with >= 0. It is now uint32_t, the flag is
1U << 31, and the test reads that bit.
Closes#19837
Three things probing master turned up, and a trim that pays for two of
them.
GET_INPUT [player] answers what a player holds without changing it.
There was no way to ask: SET_INPUT answers in that shape but has to set
something first, and RELEASE_INPUT lets go before it answers, so it
always reads none. A client could not ask what it was holding.
LOAD_CORE answered "The command failed." for a missing path and for a
library that is not a core alike - the generic fallback, because it
returned false without ever saying why. It says which now.
Neither LOAD_CORE nor LOAD_CONTENT unloads the running core for a path
that turns out not to be one. The library is probed first, with
libretro_get_system_info(), which is what core scanning already uses
and which keeps what it finds, so asking twice is free. Checking the
paths stopped a typo costing the game; a stale library or something
renamed still did.
tools/list pays for every tool's description on every session, so the
two new ones are paid for where the list repeated itself:
readOnlyHint is false by default and 37 tools said so anyway, which was
770 bytes, and the memory map named hexadecimal in four columns that
its own description already covers. 48 tools in 22861 bytes, from 46 in
21449.
RUN_UNTIL's conditions move into command.c: a table in the header is a
copy in every translation unit that includes it, and nothing outside
command.c compares them.
While a token poll is in flight the polling task parks; the poll's
callback publishes the outcome through atomics, clearing poll_pending
last, and wakes the task by ident when the outcome is final. A pending
reply leaves the next poll to its interval. A cancel ends the task once
no poll is in flight, since the callback writes into the context the
task's cleanup frees. gdrive_backup_test drives the handler and the
callback through a poll, a cancel and a final reply.
While the device has a transfer in flight the task parks, and the
transfer's callback wakes it; busy is an atomic, cleared by the callback
on the main thread and read by the task on the worker. Before a UPnP
discovery reply arrives the task looks again every 10 ms.
nat_task_serialisation_test has a threaded lane with the fake device made
asynchronous, as natt.c is.
* LOAD_CONTENT: check paths before unloading
* mcp: results carry structuredContent
* LIST_CORES filters by content path
* Structured rows for core options and memory map
* MCP: do not advertise a cacheable tool list
* SET_INPUT holds several controls at once
* A row opening with MORE is a row, not a cursor
* MCP: fewer tools, and rows a client can rely on
structuredContent and outputSchema never reached a client over stdin
and stdout. The gate read the MCP-Protocol-Version header, or the
version the stateless revision sends with every request; stdio has
neither, so the revision agreed at initialize was parsed, used for the
handshake reply and then dropped. It now holds for the session, which
over stdio is the connection.
A command answers with structuredContent exactly when it declared an
outputSchema, so a client reads the schema it was handed rather than
probing each reply. SET_INPUT, RELEASE_INPUT and GET_LOG declare one
now.
The "MORE [next]" line was read by a literal in the server. The shape
of a reply names its own trailers instead - the lines that carry a
number about the reply rather than a row of it - so GET_LOG's NEXT and
SKIPPED are the same mechanism rather than two more special cases, and
a reply says nothing about being cut short where it cannot be. A reply
whose lines are prose declares no columns: GET_LOG's page runs to 60k,
and a row apiece would answer with it twice.
Thirty-four commands are no longer offered as tools. A hotkey that
steps a selection - the next state slot, the previous shader, the next
cheat - is a hotkey because a person presses it while watching the
screen. Where a command takes the value outright that one is the tool,
and stepping towards it is only a way to get the value wrong; where
nothing reports the selection either, there is nothing worth offering
yet. READ_CORE_RAM and WRITE_CORE_RAM go too: an achievement address
looks like READ_CORE_MEMORY's system address and is not one. The disc
and the menu are kept, because stepping them blind is all there is and
a disc swap is part of playing a game. All of them are still commands
on every line interface.
The tool list is 45 tools and 20k, from 79 and 23k, and the end-to-end
test holds it to a budget: every description is context a client pays
for before it does anything.
* PRESS: hold, run and let go in one request
A tap was four requests: SET_INPUT, RUN_FRAMES, SET_INPUT to release,
RUN_FRAMES again. Each is its own HTTP connection against four slots,
so a field test spent about 25 of them crossing a menu and about 10 on
one jump.
PRESS <player> <control>[+<control>...] <frames> holds the buttons,
runs that many frames and lets them go, answered once the last frame
has run. It is RUN_FRAMES' job with buttons held through it: the same
deferred reply, the same one-at-a-time rule, the same pause the runloop
applies to frame advance, so nothing new blocks and no second state
machine exists. Whatever ends the frames lets the buttons go, including
the error that ends them early.
Axes stay SET_INPUT's. They are held at a value rather than pressed,
and SET_INPUT with RUN_FRAMES is still the way to hold a control across
several steps.
SET_INPUT and PRESS read the same request - a player, controls joined
by "+", and a number - so that parse is one routine now, and both
resolve every control before either acts on any.
* CHANGES: the MCP and command work in this series
* GET_STATUS: say what it actually answers with
The description promised the content's CRC32 and the reply has never
carried one: it is the system and the content's file name, separated by
a comma.
HELP prints that text on the line interfaces and tools/list hands it to
every MCP client, where it is the prose a client reads to decide how to
take the reply apart. One told to expect three comma-separated fields
gets two.
Adding the CRC instead would be worse. Working it out means hashing the
content, which for a core using need_fullpath reads a whole disc image
from disk - task_movie.c already carries that note, having hit it - and
GET_STATUS is the command a client polls. It would also change the
reply for every client that reads the line today.
* command: descriptions that match the replies
tools/list hands a command's description to every MCP client, and HELP
prints it on the line interfaces: it is the prose a client reads to
decide how to take a reply apart. Five of them described a reply that
was not the one sent.
SET_INPUT and RELEASE_INPUT answer with three fields - the player, the
buttons held and the axes held - and the description named two. A
client splitting on what it was told got the player where it expected
the buttons. RELEASE_INPUT did not say it answered at all, and with no
player it lets go for everyone while answering for the first of them,
which is worth saying rather than leaving to be discovered.
GET_LOG can open with SKIPPED and a count where the lines asked for had
already been overwritten, which the description did not mention: a
client reads it as a log line.
READ_CORE_MEMORY and WRITE_CORE_MEMORY both answer - with the bytes,
and with how many were written - and neither said so.
* READ_CORE_MEMORY: format bytes without snprintf
Both memory reads wrote their reply with an snprintf per byte, to turn
one byte into two hexadecimal digits. At COMMAND_READ_NBYTES_MAX that
is 16384 calls, which costs 1.5ms on an aarch64 Pi - a tenth of a frame
spent formatting, on the main thread, in the command an assistant reads
memory with most. A nibble table is 0.02ms for the same bytes, 68 times
faster, and the output is identical for all 256 values of a byte in
both of the formats the two sites used.
The harness core now maps its memory, because nothing in the tree did:
READ_CORE_MEMORY, WRITE_CORE_MEMORY and GET_MEMORY_MAP had no test that
reached past "no memory map defined", so the formatting this changes
was uncovered. The test asserts the replies byte for byte, including
every digit of a byte, a read clamped at the end of its region, and one
outside every region. Breaking the table deliberately fails three of
them.
* PRESS: leave SET_INPUT's holds in place
PRESS lets go only of the buttons it pressed itself, so a button a
client already holds with SET_INPUT is still held once the press ends.
* command, mcp: trim comments and order new struct members
Drop the superseded LIST_CORES and reply-splitting comments, shorten
the new comments to what stays true, and order the new struct members
by alignment.
---------
Co-authored-by: LibretroAdmin <[email protected]>
--mcp-stdio serves MCP on stdin and stdout as newline-delimited
JSON-RPC, for a client that starts RetroArch itself; no token is asked.
It takes the stdout descriptor for the protocol at option parsing and
points stdout at stderr, so RetroArch's log and a core's prints never
reach the client's stream. Reads and writes never wait: a line is
handled once whole, answers are queued and written as the pipe takes
them, and both outlive the server across a content load. It offers
logging: after logging/setLevel, new lines from the GET_LOG ring at
that level and above go out as notifications/message from the poll, so
the threads that log do no more than before. Closing stdin quits.
RARCH_LOG_V copies each formatted line into a ring of the last 256
lines once it is armed, which the MCP server's start or the first
GET_LOG does; unarmed, a log line costs one relaxed load more. Any
thread may log: a writer claims a line number, writes the slot as
atomic words between a busy mark and its stamp, and the reader keeps a
line only when the stamp matches before and after the copy, so a
reader never waits and never sees a torn line. GET_LOG answers the
lines since the one asked for, then NEXT; lines overwritten before they
were read are counted as SKIPPED. Desktop builds with the MCP server
only; samples/runloop/log_ring writes from four threads while reading,
under ASan and TSan.
* mcp: tools take named parameters, not one string
* mcp: preserve explicitly supplied empty parameters
Apply defaults only to absent parameters and retain separators after empty operands. This lets SET_SHADER accept an empty preset to disable shaders.
* mcp: normalize named integer parameters before dispatch
Convert integral JSON decimal and exponent forms to plain decimal digits without floating-point rounding. Reject fractional values, numeric strings and expansions beyond the request limit before running handlers. Keep the legacy opaque argument path unchanged.
---------
Co-authored-by: Codex <[email protected]>
* tests: cover MCP option state and speed changes
Exercise the shipping option handlers and runloop setters with counted driver events. Cover recorder conflicts, hardcore activity, rejected speed changes and repeated requests without extra work. Run the harness under Linux sanitizers and on Windows in the network CI workflow.
* mcp: SET_OPTION/GET_OPTION, real errors
* mcp: apply option changes to runtime state
Reject recording and streaming conflicts before initializing a recorder. Read hardcore activity rather than its configured preference. Apply speed targets synchronously through runloop setters, sharing the fast-forward side effects with hotkey handling and respecting runtime restrictions.
---------
Co-authored-by: Codex <[email protected]>
A refused certificate or a connection made with verification disabled
now shows an on-screen notice alongside the log line: the refusal names
the 'TLS Certificate Verification' setting, the disabled notice says
connections are not being checked. Background tasks can connect many
times a minute, so each notice shows once a session, guarded by an
atomic since the hooks run on connection threads. Optional mode stays
log-only.
The BearSSL backend now reports a handshake that ended on the
certificate through ssl_socket_log_verify_fail, as the built-in client
and mbedTLS do, and ships the weak no-op hooks.
samples/network/tls_retro/verify_hook_test.sh checks that a refused
certificate reaches the hook on each backend; it runs in the built-in
client's local server matrix and in the mbedTLS and BearSSL lanes.
The cloud_sync_capture hook copies the settings a driver's calls read,
on the main thread as the sync's task is pushed, where the task takes
its own copy too. SMB and NFS read the share, the account, the
timeouts and the destructive flag from that copy rather than from the
live settings: their calls run on a worker, read some of them after a
network call returns, and one still blocked past the exit wait would
otherwise read settings the frontend has freed.
cloudsync_async_test checks the hook runs once on the calling thread
and that a driver without one is left alone.
cloud_sync_deinit() moves the calls on to a new generation. A call made
before it that is still running when the wait gives up is disposed of by
the worker once it returns - its file closed, nothing handed back - so
its handler never runs for a task that is gone, and a sync begun later
never sees it.
cloud_sync_poll() takes the caller's budget callback and runs the oldest
finished result unconditionally, then more only while the budget allows;
the cloud sync task polls inside its shared I/O window, so a burst of
results is spread over as many runs as the window asks. NULL runs them
all. cloudsync_async_test covers the paced poll and a sync begun while an
abandoned call is still stuck.
A tool whose work runs over later frames answers when that work is
through, with what came of it, or with its name, ERROR and the reason:
LOAD_CONTENT, START_CORE, CLOSE_CONTENT and UNLOAD_CORE once the load or
close ends; LOAD_STATE_SLOT and SAVE_STATE_SLOT once the state is
applied or written; PLAY_REPLAY_SLOT, RECORD_REPLAY, SEEK_REPLAY and the
replay checkpoint tools once the replay has started or the seek or
checkpoint has run; SET_SHADER once the preset has compiled; AI_SERVICE
with the service's text. A line-based interface keeps the answers it
had.
The waiting is one shape, task_notify_t (tasks/task_notify.h), held by
the work's owner: the content load, the save and load tasks, the movie
tasks, the deferred shader load and the translation service. State and
replay work a command starts runs from a main-thread task, outside the
core's own input poll. The command interfaces outlive a content load,
remade only when their settings change, and what arrives during a load
waits for it.
samples/tasks/content_load gains lanes for the state-slot, replay, seek and
content replies; samples/network/mcp_server follows.
Closes#19702
Outside the frontend's top level and the replay code, five files
still took the input state for a field or two. Each asks by name now:
- netplay switches non-blocking input for catching up
(input_driver_set_nonblocking()) and asks whether the core's input
is blocked (input_driver_libretro_input_blocked(), which was there);
- configuration.c marks the remap cache active
(input_driver_set_remapping_cache_active()), saves and restores the
devices' names around a configuration swap
(input_driver_device_info_save(), _restore()), and resets and fills
a port's sensor axis map through input_driver_set_sensor_map() -
the map is built whole and set in one call. The getter,
input_config_get_sensor_map(), moves to input_driver.c;
- the autoconfig task asks a port's display name
(input_config_get_device_display_name(), which was there);
- the overlay task asks where the eight-way slopes are kept
(input_driver_overlay_eightway_slopes());
- the AI service presses and reads its buttons
(input_driver_ai_gamepad_press(), _pressed()).
What is left outside input/ is the frontend's top level (retroarch.c,
runloop.c) and the replay code being handed the state by the commands,
the save-state and movie tasks and rewind.
tools/input_state_grab_check.py now counts tasks/, network/,
command.c, configuration.c and state_manager.c too, with that list.
The overlay task's sample stubs the slopes call instead of the input
state. The AI service and netplay paths are syntax-checked with their
features on, as the Linux build leaves them out; the harness passes.
SMB and NFS are flagged CLOUD_SYNC_DRIVER_FLG_BLOCKING: their calls
block on the network until done. With threads, cloud_sync_*() queues
them for one worker per sync, which runs them in call order, and
returns at once; cloud_sync_poll(), called first by the task handler,
runs each completion handler on the handler's own thread. A slow or
unreachable server no longer holds up the thread driving the sync -
the frame loop itself with Threaded Tasks off. Each call is one
allocation with its names copied; the worker leaves once the sync ends
or fails to begin. Without threads the calls are made directly.
On exit, after the task drain, cloud_sync_deinit() drops the calls not
started (closing an upload's file) and waits at most a second for the
one under way, so no driver code runs into the teardown.
samples/tasks/cloudsync/cloudsync_async_test covers ordering, threads,
copying, refusals, worker turnover and the exit path in CI.
A download is written to <file>.rafetching and renamed over the local
file only once complete, so a failed open or a read failing part way
leaves the save as it was; the task skips leftover .rafetching files.
Only the server answering NOENT counts as "not there": an unanswered
open, stat or unlink fails rather than reading as a missing file or a
finished delete. An upload is one RENAME over the old file; with
destructive sync off the old file is first kept as <path>-<time>, then
-2 to -9 within one second, and a backup that cannot be made or a stat
with no answer fails the upload instead of replacing unguarded.
Names that do not fit are refused rather than truncated, empty
components in nfs_subdir are dropped, and a directory another client
makes at the same moment is accepted. Path buffers and the transfer
buffer sit in one scratch block allocated per session, keeping every
frame small. samples/network/nfs_backup runs the driver against an
in-memory export in the samples CI workflow.
Use the built-in NFS client for cloud synchronization, including atomic uploads and non-destructive backups. Enable the backend in regular and Griffin builds so it is available on webOS, iOS, and Android.
Statics rely on implicit zero initialisation instead of {0}, and local
aggregates whose first member is an array get the matching braces, so
GCC 4.x -Wmissing-braces stays quiet. Locals the compiler cannot prove
written are initialised (rh265 keep[], rflac block header, query.c
lengths), loops older VRP misreads are reshaped, the unused SDL2
geometry forward declaration is gone, and task_queue no longer names a
variable main.
rzstd probes BMI2 with __get_cpuid_max/__cpuid_count, which every
cpuid.h the branch accepts provides. Hole punching needs uClibc-ng 1.0
with its Linux-specific API on uClibc. The sinc resampler's pop_options
does not warn on GCC 4.x MIPS. generate_wayland_protos.sh strips the
event type and deprecated-since attributes for scanners whose DTD
predates them, which ignore them anyway.
The MCP SCREENSHOT tool answers with the screenshot itself as image
content, PNG up to 4 MiB, followed by the file's path as text, so a
client on another machine, or without access to the screenshot
directory, can see what is on the screen. Past 4 MiB, or for another
format, it answers with the path alone as before.
The screenshot task reads the PNG it has just written back on its own
thread and base64-encodes it there, as the encoder streams rows into
the file and keeps no copy; the main thread only hands it on. A command
interface takes it through the new optional reply_image_to, which only
the MCP server sets; every other interface is sent the text.