Commit Graph
105 Commits
Author SHA1 Message Date
LibretroAdmin 66376b18c5 cloud_sync: WebDAV, S3 and Google Drive downloads go straight to disk
A download from these servers was gathered whole in memory, up to the
256 MB body limit, then written to the local file in one piece; the
WebDAV driver carried a TODO asking for exactly this.  They now use
task_push_http_download_file_with_headers: the body goes to the file
in pieces as it arrives, and the callback opens the finished file to
hand on.  The HTTP error text these drivers log still reaches them.

A body with no Content-Length or chunked framing is still refused, and
its file is removed with it.  A 404 (or S3's 400) still means the
server does not have the file; the download removed the file already.
2026-11-11 07:42:20 +01:00
LibretroAdmin 8a9d7eece3 cloud_sync/smb: write a download as it arrives
A download was gathered whole in a buffer of the size the server
reported, then written to the local file in one piece.  The local
file is now opened first and each SMB read goes to it as it arrives,
through one buffer the size of a read: no copy of the whole file in
memory, and no allocation sized by the server.  A failed local write
fails the download; it was not checked before.
2026-11-11 07:04:21 +01:00
LibretroAdmin a5668b1a9a cloud_sync: bulk writes for downloaded files
A downloaded file is written in one piece (WebDAV, S3, Google Drive,
SMB, iCloud, iCloud Drive) or in 64 KB pieces (NFS), and the sync
task reads it back 256 KB at a time to hash it.  These streams are
opened read-write, which takes the descriptor path on the desktops
and Android now, so they ask for it: the body goes to the file in one
write() with no copy through a stdio buffer, and reads back without
one.  Where descriptor writes are not available the hint is ignored.

The updated manifest keeps its buffer: the JSON writer hands it over
1 KB at a time.
2026-11-11 06:38:01 +01:00
LibretroAdmin 24457eef48 S3: keep the path buffers off the stack
s3_backup_then held six path buffers on the stack, 12.8 KiB, and
s3_read and s3_delete_object two each, a little over 4 KiB. The backup
names now go in one allocation, and the URL and canonical URI of an
object come from s3_object_urls, one allocation holding both, which
s3_update_upload uses too. The largest frame in the driver is now
s3_build_request_url's 2.1 KiB. The hand-copied empty payload hash in
the GET and DELETE goes to S3_EMPTY_PAYLOAD_SHA256.

s3_backup_test fails those allocations for a backup before an upload
or a delete, a plain upload, a delete and a read: nothing starts,
nothing is reported, nothing leaks.
2026-11-04 22:46:27 +01:00
LibretroAdmin 9d85133572 S3: stream uploads from the file, not a whole-file copy
An upload read the whole file into one malloc before it started: up to
the 64 MiB multipart threshold for a single PUT, and the whole file,
however large, for a multipart upload, held until its last part was
acknowledged. Out of memory the upload failed, and on the consoles the
threshold is sized for, that is not far off.

The body is now pulled from the file by net_http as the socket takes
it, through net_http_connection_set_content_source, the way WebDAV
already uploads: one send buffer instead of the file. Each multipart
part is a byte range of the file, seeked to on its first run so it
reads its own bytes whatever was read before it, and a replay on a
fresh connection rewinds to the range's start. The payload hash for the
signature is taken by reading the range in 64 KiB runs into
sha256_stream_update. The fallback to a single PUT when the multipart
start fails in transport streams the whole file the same way.

task_push_http_transfer_with_source is task_push_http_transfer_with_
content with a source in place of the buffer. The upload's URL buffers
move off the stack (4.2 KiB to 112 bytes).

s3_backup_test's stub drains every streamed body as net_http would,
with a replay from the start, and checks it is all there and is what
was signed. New lanes: an 11 MiB multipart upload (three parts, each
its own range of the file), its single-PUT fallback, and an empty file.
The multipart threshold can be set by the test for that.
2026-11-04 22:36:51 +01:00
LibretroAdmin 44090f5ea9 S3: sign with the shared SHA-256 and HMAC, streamed
The SigV4 signer assembled the canonical request and the string to
sign in 4 KiB stack buffers and MACed them with its own HMAC. A URI
or copy source past 4 KiB was signed cut short, and so was a secret
key over 123 octets (the setting takes 185), so those requests came
back 403. The two buffers also put the signer's frame at 9 KiB.

The canonical request now goes straight into sha256_stream_update as
it is made, and the string to sign into hmac_sha256_update, both from
crypto/kdf.c: nothing is assembled, so nothing is cut short. The query
is sorted as an index of spans over the caller's string, on the stack
up to 16 parameters, so the signer allocates only its result. The
hand-written HMAC, the query copy and the old signature helper go.

HAVE_S3 now needs HAVE_CRYPTO, which every build that defines it
(Apple) already has; griffin says so if it is missing.

s3_backup_test checks seven signatures against an independent SigV4
computation, including a 5001-character URI and a 150-octet secret,
which master signs wrong. The query and out-of-memory tests follow the
new shape, and the test rebuilds when the driver changes.
2026-11-04 22:36:51 +01:00
Anthony Schemel fd880c6ac0 s3: sort query parameters as SigV4 requires; sign nothing out of memory
s3_canonicalize_query_string splits the query on '&', sorts the
parameters by name and then by value in byte order, and joins them
again; names are compared on their own, so a name that extends
another sorts after it. Callers keep URL-encoding each name and value.

When the canonical headers or query cannot be allocated the signer
returns no signature, which s3_build_auth_header_ex passes on as NULL
and every caller treats as a failure.

s3_backup_test checks the sort order, including names that extend
one another, and fails each of the signer's allocations in turn.

Closes #19956
2026-11-04 17:51:42 +01:00
Anthony Schemel b82437f038 webdav: no Digest header without an accepted challenge
The cnonce is written only when a challenge is accepted and cleared
with the rest of the digest, so webdav_create_digest_auth_header()
builds nothing while it is empty: after a challenge the parser
rejected, the next request goes out with no Authorization header and
the server's 401 is reported as usual.

webdav_reauth_test sends rejected challenges (an open-ended nonce or
realm, no nonce, qop="auth-int") after an accepted one and checks
that no header is built from them.

Closes #19954
2026-11-04 17:44:31 +01:00
Davey HughesandLibretroAdmin 9d38efa2d6 scan: let the parallel readers open smb:// files (#19939)
* vfs: close a private SMB connection when an open, stat or opendir fails

When every pool slot is held, smb_take() connects privately for the
caller. If the open, stat or opendir then fails, smb_heal() returns
NULL when the connection is alive (a missing path, the usual case) or
cannot be replaced, and the caller stored that NULL over its context
and gave back nothing, so the connection and its socket were never
freed.

Keep the caller's context when smb_heal() returns NULL, so smb_give()
closes it. A pooled slot is given back as before.

vfs_test holds both pool slots with open files and then stats, opens
and lists a missing path; LSan reported the three connections before.

* vfs: release an SMB stream's connection when close follows a failed mend

When a stream's connection cannot be mended (the server is gone and
the reconnect is refused), or its file cannot be opened again, the
mend clears the stream's handle. Close returned early on the missing
handle, so a private stream leaked its connection, a pooled stream
kept its slot busy for the pool's life (smb_shutdown then left the
pool unfreed), and the reopen record leaked either way.

Close now checks for the connection rather than the handle and closes
the handle only when there is one. After a failed mend a private
connection is freed without a goodbye, since the server may be gone.

smb_idle_test gains a mode in which the server is stopped for good
during the pause, and local_samba_test.sh runs it, so the sanitizer
lane's LSan reports these leaks.

* cloud sync: make and drop SMB contexts under the VFS's list lock

libsmb2 keeps every context on one process-wide list with no lock of
its own, so the SMB VFS makes and drops its contexts under a lock
(4e6885d5a3). The SMB cloud sync driver called smb2_init_context() and
smb2_destroy_context() directly. A sync that began or ended while the
VFS connected or disconnected on another thread, as during a scan,
could lose a list entry, after which a live connection reads as dead,
or break the list.

The VFS exports its two wrappers and cloud sync uses them. The
built-in client has no such list; there they call straight through.

cloud_sync_test runs cloud sync's connect and disconnect beside private
VFS connections; the libsmb2 TSan lane reported the race before.

* scan: readers take smb:// paths when an SMB client is built

The scan's parallel readers were kept off every "://" path. smb://
paths go through the SMB VFS, whose connection pool serves several
streams at once, and the commits before this one fix the connection
leaks and the libsmb2 list race that readers would otherwise hit. So
readers now take smb:// paths in builds with an SMB client, built-in
or libsmb2; other schemes stay with the scan.

database_scan_test adds smb://, nfs:// and ftp:// archives to a
threaded scan's list and counts the readers' reads of each. Its TSan
build gets a stub SMB backend, so CI runs both halves.

* vfs: keep an SMB pool per share, never freed under its users

Pools live in a static table, one per connection key. Each pool's state
and user count share one atomic word: a lookup or a held slot is a use,
only an idle pool is evicted, and a retired pool closes its connections
when its last user leaves. Readers on two shares each keep their pool's
connections, and no thread touches a freed pool.

A failed reconnect after a dead transport frees the connection without
the round trips of a logoff; a pooled slot reconnects on its next take.

vfs_threads_test runs threads alternating between two shares of one
directory with no share configured.

---------

Co-authored-by: LibretroAdmin <[email protected]>
2026-10-29 18:17:24 +01:00
LibretroAdmin b4c26c9716 google_drive: the OAuth poll task parks on its poll, woken by an outcome
While a token poll is in flight the polling task parks; the poll's
callback publishes the outcome through atomics, clearing poll_pending
last, and wakes the task by ident when the outcome is final.  A pending
reply leaves the next poll to its interval.  A cancel ends the task once
no poll is in flight, since the callback writes into the context the
task's cleanup frees.  gdrive_backup_test drives the handler and the
callback through a poll, a cancel and a final reply.
2026-10-14 21:39:52 +02:00
Roman Vasilyev cfbc895b78 Add a separate NFS cloud sync subdirectory (#19722) 2026-10-09 09:18:05 +02:00
libretroadmin 0e464bdf41 cloudsync: a blocking driver's calls read settings copied as the sync is pushed
The cloud_sync_capture hook copies the settings a driver's calls read,
on the main thread as the sync's task is pushed, where the task takes
its own copy too. SMB and NFS read the share, the account, the
timeouts and the destructive flag from that copy rather than from the
live settings: their calls run on a worker, read some of them after a
network call returns, and one still blocked past the exit wait would
otherwise read settings the frontend has freed.

cloudsync_async_test checks the hook runs once on the calling thread
and that a driver without one is left alone.
2026-10-05 12:51:48 +02:00
libretroadmin 5e968f8b99 cloudsync: blocking drivers run on a worker, results come back by poll
SMB and NFS are flagged CLOUD_SYNC_DRIVER_FLG_BLOCKING: their calls
block on the network until done. With threads, cloud_sync_*() queues
them for one worker per sync, which runs them in call order, and
returns at once; cloud_sync_poll(), called first by the task handler,
runs each completion handler on the handler's own thread. A slow or
unreachable server no longer holds up the thread driving the sync -
the frame loop itself with Threaded Tasks off. Each call is one
allocation with its names copied; the worker leaves once the sync ends
or fails to begin. Without threads the calls are made directly.

On exit, after the task drain, cloud_sync_deinit() drops the calls not
started (closing an upload's file) and waits at most a second for the
one under way, so no driver code runs into the teardown.
samples/tasks/cloudsync/cloudsync_async_test covers ordering, threads,
copying, refusals, worker turnover and the exit path in CI.
2026-10-05 04:38:55 +02:00
libretroadmin 9274caf034 cloudsync/nfs: never lose a save to a failed transfer
A download is written to <file>.rafetching and renamed over the local
file only once complete, so a failed open or a read failing part way
leaves the save as it was; the task skips leftover .rafetching files.
Only the server answering NOENT counts as "not there": an unanswered
open, stat or unlink fails rather than reading as a missing file or a
finished delete. An upload is one RENAME over the old file; with
destructive sync off the old file is first kept as <path>-<time>, then
-2 to -9 within one second, and a backup that cannot be made or a stat
with no answer fails the upload instead of replacing unguarded.

Names that do not fit are refused rather than truncated, empty
components in nfs_subdir are dropped, and a directory another client
makes at the same moment is accepted. Path buffers and the transfer
buffer sit in one scratch block allocated per session, keeping every
frame small. samples/network/nfs_backup runs the driver against an
in-memory export in the samples CI workflow.
2026-10-05 03:39:20 +02:00
Roman Vasilyev a7c4f1e7c1 cloudsync: add NFS backend
Use the built-in NFS client for cloud synchronization, including atomic uploads and non-destructive backups. Enable the backend in regular and Griffin builds so it is available on webOS, iOS, and Android.
2026-10-05 03:39:20 +02:00
LibretroAdmin 71318ef1ec Fix build warnings on old GCC, uClibc and old wayland-scanner
Statics rely on implicit zero initialisation instead of {0}, and local
aggregates whose first member is an array get the matching braces, so
GCC 4.x -Wmissing-braces stays quiet. Locals the compiler cannot prove
written are initialised (rh265 keep[], rflac block header, query.c
lengths), loops older VRP misreads are reshaped, the unused SDL2
geometry forward declaration is gone, and task_queue no longer names a
variable main.

rzstd probes BMI2 with __get_cpuid_max/__cpuid_count, which every
cpuid.h the branch accepts provides. Hole punching needs uClibc-ng 1.0
with its Linux-specific API on uClibc. The sinc resampler's pop_options
does not warn on GCC 4.x MIPS. generate_wayland_protos.sh strips the
event type and deprecated-since attributes for scanners whose DTD
predates them, which ignore them anyway.
2026-10-04 04:23:50 +02:00
libretroadmin 39ebd0399d s3: HMAC hex pairs are assigned, not initialised, for C89
The three hex_byte buffers in the HMAC and signature code take their
two digits by assignment; C89 only allows constant initialisers in an
aggregate. The compile matrix holds s3.c to the full C89 lane, since
no build compiles it in C89 otherwise.
2026-10-03 01:07:00 +02:00
Anthony Schemel 5c34f32f08 s3: encode '&', '=' and '?' in the object key
s3_url_encode is called only from s3_build_request_url, for the object
key, yet it left '&', '=' and '?' unencoded as query delimiters. A '?'
in a key then ends the path where s3_build_canonical_uri_from_url looks
for the query (strpbrk(path_start, "?#")), so the request goes to the
wrong key and is signed over the wrong URI. '&' and '=' are common in
save names ("Rocky & Bullwinkle") and S3 percent-encodes them in its
own canonical URI, so the SigV4 signature most likely mismatches.
Encode all three; '/' stays literal.
2026-10-02 23:45:36 +02:00
Anthony SchemelandClaude Opus 5.5 9a64f3fa3c webdav: a fresh client nonce for each digest challenge
webdav_create_digest_auth set cnonce to the constant "1a2b3c4f" for
every login. The cnonce is the client's own input to the digest
response; with it fixed, a hostile server (or anyone in the middle of
plain HTTP) chooses every other input and can test password guesses
against precomputed tables.

webdav_create_cnonce fills a 33-byte buffer in the state with 16 bytes
from crypto_random_bytes as hex, or, without HAVE_CRYPTO or when no
entropy source answers, an MD5 of the microsecond clock and a counter.
The buffer replaces the const char pointer, so nothing is allocated.

Tested against a local server that checks the digest response: before,
every login sent cnonce=1a2b3c4f; after, each run sends a new 32-digit
cnonce, and the server accepts the response both times and the sync
completes. Builds clean with make; webdav.c compiles as gnu89 with and
without HAVE_CRYPTO.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-01 21:04:31 +02:00
LibretroAdmin 3ef1a02a5d webdav: bound every scan in the Digest challenge parser
The WWW-Authenticate challenge is whatever the server sends with a
401, and the parser trusted its shape:

- realm, nonce and opaque took strchr(ptr, '"') + 1 - ptr as a length.
  With no closing quote strchr() returns NULL, the length is a huge
  value, and malloc() of it fails or the strlcpy() that follows writes
  through NULL.
- An unknown unquoted parameter was skipped with
  while (*ptr != ',' && *ptr != ','), which never checks for the end
  of the string, so one at the end of the line ("stale=FALSE") read on
  past it until it happened to meet a comma.
- qop and unknown quoted values had the same missing-quote hole.

Every value is now found with a bounded scan, and a challenge that
cannot be parsed is refused - which the caller already handles as
"cannot re-authenticate". Quoted values go through one helper that
copies them out and replaces any earlier copy, so a repeated parameter
no longer leaks the first; the HA1 hash is recomputed rather than
leaked when realm repeats. qop is read as a list whose members may be
spaced ("auth-int, auth" was refused), and a quoted algorithm is
accepted.

webdav_reauth_test parses well-formed and malformed challenges from
exactly-sized buffers under ASan. The previous parser aborts on the
first malformed one.
2026-10-01 02:56:44 +02:00
LibretroAdmin 1924d2a5dc webdav: take the Digest challenge as const
webdav_needs_reauth() now walks the response's header block with
net_http_header_next(), which hands out const char *, and passed each
line to webdav_create_digest_auth(char *) - a -Wdiscarded-qualifiers
warning. The parser only ever reads the challenge (every field it
keeps is copied out with strlcpy/strdup), so its parameter and cursors
are const now.
2026-10-01 02:48:25 +02:00
LibretroAdmin dba5458fb5 net_http: keep response headers in one block instead of a string_list
Every transfer allocated a string_list for its response headers: the
struct, a 32-slot array, and a strdup per header line, then freed all
of it when the task finished - on the task thread, or on the video
thread in an unthreaded build. A typical response has 10 to 20 lines;
a 64-line one cost 67 allocations before a byte of body arrived.

The headers now go into one block as they are parsed: each line
NUL-terminated, an empty line at the end, grown by doubling from 1 KiB
and bounded by the existing 64 KiB header cap. It is allocated with
the first header line (a failed connect allocates nothing) and handed
over whole. In the harness, 64 header lines cost one allocation more
than one line does. The Location offset replaces the pointer into the
list, since the block may move as it grows.

API:
  net_http_headers_take()  the block, ownership to the caller (free());
                           same NULL rules as net_http_headers_ex()
  net_http_header_next()   walk the lines
  net_http_header_value()  first field by name, case-insensitive
  net_http_body_is_framed() takes the block

net_http_headers() / net_http_headers_ex() remain for code outside
RetroArch, building a string_list from the block on demand; nothing
in-tree calls them.

http_transfer_data_t.headers is the block. task_http hands it over
and frees it; task_core_updater frees it; the emscripten backend
compacts the browser's header text into the same layout in the buffer
it already had, so it also drops to one allocation. WebDAV, S3 and
Google Drive walk it with net_http_header_next(), and S3's ETag lookup
uses net_http_header_value(). The WebDAV, S3, transport-failure and
cheevos harnesses build or take blocks; the S3 harness links net_http.c
for the helpers.
2026-10-01 02:28:04 +02:00
libretroadmin 7e4ca8f2c6 Use the built-in SMB2/3 client for smb:// and SMB cloud sync
HAVE_RETROSMB puts net_smb2.c behind the VFS SMB backend and the SMB
cloud-sync driver in place of libsmb2. configure turns it on whenever
HAVE_CRYPTO is on and no libsmb2 was asked for, so main builds stop
carrying the bundled library; --enable-smbclient / --enable-libsmb
still select a libsmb2. Both consumers keep their libsmb2-shaped source
through net/net_smb2_compat.h, which presents the calls they make -
context and credential setup, connect, open / read / write / lseek /
ftruncate / close, stat / unlink / mkdir / rename, opendir / readdir /
closedir, echo, the I/O size limits, error text - over the built-in
client with the same return conventions (lseek's -EINVAL, libsmb2's
type and stat structs). net_smb2.c gains unlink, mkdir, rename (SET_INFO
FileRenameInformation) and ftruncate (FileEndOfFileInformation) for
them. A Kerberos request fails at connect so the backend's NTLMSSP
fallback runs; share enumeration (SRVSVC over IPC$) is not carried yet,
so browsing starts at a named share.

The samba matrix now also drives the real vfs_implementation_smb.c
(stat, open, seek, read, tell, opendir / readdir) over each dialect,
and the compile matrix gains the griffin HAVE_RETROSMB shape. A full
RetroArch build links with the client selected and no libsmb2 symbol.
2026-09-28 20:17:59 +02:00
LibretroAdmin 7ec3c021a5 net_http/webdav: stream the upload body from the file instead of reading it whole
A WebDAV upload read the complete file into a buffer, and
net_http_connection_set_content() then copied that buffer again, so
two copies of the file sat in memory for the whole transfer. Saves are
small, but a save state is not, and several uploads run at once.

net_http gains a request-body source: net_http_connection_set_content_source()
takes a Content-Length, a read callback and an optional rewind
callback. net_http_send_request() pulls the body from the source in
64 KiB runs and sends each as it arrives, so one run is all that is
ever held; a source that delivers fewer bytes than the advertised
length fails the request as a transport error would, since the header
has already promised them. When a request has to be replayed on a
fresh connection the source is rewound first; a source with no rewind,
or one whose rewind fails, is not replayed.

task_push_webdav_put_stream() wraps that for the task queue, and the
cloud-sync WebDAV driver uploads straight from the open RFILE through
filestream_read, rewinding with filestream_seek. The existing seek to
the start before the upload is kept, as the Digest-challenge retry
still reaches this path with the file at its end.

Checked with a 3 MB body PUT to a local server: the received body is
byte-identical (SHA-256) and the request completes with 201.
2026-09-27 17:31:15 +02:00
LibretroAdmin 3c1b3a5f5e net_http: move the body-framing check out of cloud_sync_driver
cloud_sync_http_body_is_framed lived in network/cloud_sync_driver.c,
which the webdav_options, webdav_reauth and gdrive_backup samples do
not link, so all three failed with an undefined reference once
webdav.c and google_drive.c started calling it (#19631).

The check restates the two tests net_http itself uses to pick the body
framing, so it belongs beside them: it is now net_http_body_is_framed
in libretro-common/net/net_http.c, which every one of those samples
already links. cloud_sync_driver.c/h lose the copy and its includes.

webdav_reauth's stale-nonce test answered the retried GET with a 200
that carried no headers, which the read callback now correctly refuses
as unframed; the test's response gets a Content-Length like a real one.
All three samples build and pass under ASan/UBSan again.
2026-09-26 18:50:15 +02:00
Anthony SchemelandClaude Opus 5.5 891d09385c cloud_sync: refuse unframed download bodies, handle allocation failure (#19631)
net_http fails a transfer that ends before its Content-Length or its
final chunk. A response with neither is delimited by the connection
closing, and net_http documents that it cannot tell a complete body
from one cut off by a dropped connection. The WebDAV and Google Drive
read callbacks accepted such a body as a successful download and wrote
it over the local save. A connection that drops mid-download could
therefore replace a save with a truncated copy, which the next sync
then uploads.

The read callbacks now treat a 2xx response without Content-Length or
"Transfer-Encoding: chunked" as a failed download and leave the local
file alone. The check is shared as cloud_sync_http_body_is_framed and
matches the headers net_http itself uses to pick the body framing.

Also NULL-check allocations that were used unchecked: the WebDAV
digest-auth chain (HA1, HA2, response, header), webdav_delete's
callback state, and the Google Drive context in
gdrive_begin_with_token. Each now fails the operation instead of
writing through NULL.

Tested on Linux: full build; touched files also compile with
C89_BUILD=1 without warnings. A small program linked against the
built cloud_sync_driver.o returned true for Content-Length and for
chunked (any case), false for no framing, "gzip, chunked" and a NULL
list. Not tested against a live WebDAV or Google Drive server.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-09-26 17:18:17 +02:00
LibretroAdmin 978e8e0589 compile-matrix: two more files no job compiles, and errors first
The S3 cloud-sync backend only griffin includes, and nothing defines
HAVE_S3 for it at all; the Lakka wifi driver needs HAVE_LAKKA. Neither
is in the C89 build's object list, so neither was checked, and both had
a mixed declaration - s3_parse_url declaring a size_t after a log call,
connmanctl_tether_start_stop an int after a popen.

A failing lane also used to print the first twelve lines of the
compiler's output, which for a file with a page of warnings ahead of
its one error showed nothing to act on. The reporting moves into one
helper the four call sites share, and it prints the errors when there
are any.

Verified: 112 lanes, 0 failures, from a clean tree; C89_BUILD clean;
putting either declaration back fails its own lane.
2026-09-21 19:39:18 +02:00
LibretroAdmin 6702926dd9 icloud: report failures as failures, and keep copies under non-destructive sync
The CloudKit driver:

- reported success for a read whose local file could not be written
  (with no file, which cloud sync takes as "not on the server" and
  drops from the server manifest) and for a record with no readable
  data (with an empty file, which cloud sync takes as the save);
- saved a new record when the lookup before an upload failed, leaving
  a duplicate of a record that may well exist;
- reported a delete as done before the deletes were sent, and as a
  success whatever became of them;
- ignored non-destructive sync: deletes removed records for good and
  uploads replaced their data.

Reads now fail unless the data was loaded and written in full. Uploads
fail on a failed lookup. Deletes report once every record has been
dealt with, with the result. With destructive sync off, a delete
renames the record's path to <path>-<yymmdd-hhmmss> and an upload that
replaces a record first saves one under that name holding the old
data, not replacing anything if that fails, as the other drivers keep
their copies. The server manifest is not kept. A duplicate-record log
line also passed an NSString to %s.

samples/network/icloud_cloudkit includes icloud.m and replaces
CloudKit with an in-memory database by overriding +[CKContainer
defaultContainer] in a category, using real CKRecord, CKAsset and
CKQuery objects. It checks each case above and that every request
reports once. Against the old driver 8 checks fail. Runs in CI macOS
under ASan+UBSan.
2026-09-21 15:22:18 +02:00
LibretroAdmin 5bb3e201a5 google_drive: keep the replaced copy when an upload overwrites a file
With destructive sync off, a Google Drive delete keeps the server's
copy, renamed in place to <name>-<yymmdd-hhmmss>, but an upload
replaced the file's content outright, so the setting protected against
deletes and not against overwrites.

An upload to an existing file now asks Drive to copy it (files.copy,
server-side) to <name>-<yymmdd-hhmmss> in the same folder, the name a
delete uses, before the PATCH. A failed copy stops the upload rather
than overwrite without a backup. A file that does not exist yet has
nothing to keep, and the server manifest, rewritten every sync, is not
backed up.

A 401 on the PATCH now retries only the PATCH. It used to go back
through the whole update, search included, which with the backup would
take it a second time; the file ID is known by then.

samples/network/gdrive_backup includes google_drive.c and plays the
Drive API request by request: copy before PATCH, from the file being
replaced, named and placed as a delete names it; a failed copy stopping
the upload with one failure report; a 401 on the copy and on the PATCH
each refreshing the token once; destructive sync, a new file and the
manifest not copied. Against the old driver 5 checks fail. Runs in CI
Linux samples/network under ASan+UBSan and TSan, as does the SMB
harness.
2026-09-21 15:11:28 +02:00
LibretroAdmin ff1d6571b3 smb: upload through a temporary file and keep the replaced copy
smb_update() opened the server's file, truncated it and wrote. A write
that failed part way - a dropped connection, a full share - left the
server holding a truncated save, in either sync mode. And with
destructive sync off, a delete kept the old file as
<path>-<yymmdd-hhmmss> while an upload overwrote it in place, so the
setting protected against deletes and not against overwrites.

Uploads now write <path>.rauploading and rename it into place once it
is complete. Before that rename, the file being replaced is renamed to
the backup name a delete uses (non-destructive) or removed
(destructive); if the final rename fails, it is put back. A failed
write, a backup that cannot be made or a failed final rename all leave
the old file exactly as it was and no temporary file behind, reported
as one failure. The server manifest is never backed up.

Also: the local file is read from its start rather than wherever the
stream was left, a short read fails the upload, and a zero-byte write
from the server no longer loops forever.

samples/network/smb_backup runs smb.c against an in-memory libsmb2
share and checks what the share holds after each case: new content in
place and the old kept beside it; no backup for destructive sync, a new
file or the manifest; the old file intact after a failed write, backup
or final rename; an empty upload; the configured subdirectory. Against
the old driver 13 checks fail.
2026-09-21 15:11:23 +02:00
LibretroAdmin 05f19857db s3: honour non-destructive sync by backing up before delete or overwrite
The S3 driver never read cloud_sync_destructive: a delete removed the
object for good and an upload overwrote it, so turning destructive
sync off did nothing for S3 users. WebDAV, Google Drive and SMB keep
the server's copy in deleted/ on delete, and WebDAV now on overwrite.

With the setting off, a delete or upload first copies the object to
deleted/<path>-<yymmdd-hhmmss>, the name WebDAV uses, with CopyObject
(a PUT on the new key naming the source in x-amz-copy-source), which
is server-side. The source is /<bucket>/<key> for both addressing
styles: a path-style endpoint's canonical URI already is that, a
virtual-hosted one gets the bucket in front. A 404 means there is
nothing to keep; any other failure, including a 200 carrying an
<Error> body, which CopyObject can answer, stops the delete or upload
rather than lose the copy. The server manifest is not backed up.

S3 requires every x-amz-* header to be signed, so the SigV4 code takes
an optional copy source and signs it in its sorted place. The
signature for a CopyObject request and for a plain GET was checked
against botocore's SigV4 for the same inputs and matches.

samples/network/s3_backup includes s3.c and plays the server request
by request: copy first, into deleted/, with the right source for
virtual-hosted and path-style endpoints and x-amz-copy-source signed;
404 not an error; a failed or <Error> copy stopping the operation with
one failure report; destructive sync and the manifest not copied.
Against the old driver 6 checks fail. Runs in CI Linux samples/network
under ASan+UBSan and TSan.
2026-09-21 14:52:37 +02:00
LibretroAdmin 67e3aef27d webdav: back up the server's copy before an upload replaces it
With destructive sync off, a WebDAV delete moves the server's file
into deleted/, but an upload overwrote it in place: the setting
protected against the server's copy being removed and not against it
being replaced, which is the more likely way to lose a save (an older
or broken save from another device overwriting a good one). The code
carried a TODO/FIXME for exactly this. The local side already keeps
cloud_backups/ before a fetch replaces a local file.

An upload now copies the server's file to deleted/<path>-<yymmdd-
hhmmss>, the name a delete uses, before the PUT. COPY rather than
MOVE, so the server keeps its file until the PUT replaces it, and an
upload that fails after the backup leaves the server as it was. A 404
means there is nothing to back up yet; any other failure stops the
upload rather than overwrite without a backup. The COPY and the PUT
each get their own Digest retry. The server manifest is rewritten
every sync and is not backed up; its name moves to
cloud_sync_driver.h as CLOUD_SYNC_SERVER_MANIFEST so the driver can
recognise it.

Adds task_push_webdav_copy() beside task_push_webdav_move() in both
HTTP backends, sharing the Destination-header code.

samples/network/webdav_backup includes webdav.c and plays the server
request by request: backup before PUT, into deleted/ named as a delete
names it, 404 not an error, a failed backup stopping the upload with
one failure report, destructive sync and the manifest not backed up,
and a Digest retry each for COPY and PUT. Against the old driver 5
checks fail. Runs in CI Linux samples/network under ASan+UBSan and
TSan; the other WebDAV harnesses gain the new stub.
2026-09-21 14:40:01 +02:00
LibretroAdmin 6aa1b1b779 icloud_drive: report every request exactly once
Cloud sync counts its outstanding requests and waits for each one to
report, so a driver must call the completion handler exactly once per
request. The iCloud Drive driver broke that both ways:

- a read whose local destination could not be created never called
  it, so the sync waited forever;
- a read whose cloud file existed but could not be loaded, and an
  upload whose local file could not be read, reported failure and then
  carried on and reported again, decrementing the outstanding count a
  second time.

It also treated an unavailable iCloud container as "file not
present": a read reported success with no file, which cloud sync now
takes as the server not having the file and drops from the server
manifest, and a delete reported success. Nothing was checked, so all
three now report failure. A short write of the downloaded data is a
failure too.

samples/network/icloud_drive includes icloud_drive.m and redirects
the container to a temporary directory by overriding
URLForUbiquityContainerIdentifier: in a category, so the driver runs
unchanged; a nil answer stands for iCloud being unavailable. It checks
that every request reports once with the right result, for the paths
above and the ordinary read, missing file, upload and delete. Against
the old driver 8 checks fail. Runs in CI macOS under ASan+UBSan.
2026-09-21 14:18:12 +02:00
LibretroAdmin 9867add40b webdav: retry a Digest challenge once, and rewind a retried upload
Every WebDAV request callback retried when the server answered 401
with a Digest challenge, and nothing limited how often. That is right
for a stale nonce, but with a wrong password the server answers every
retry with a new challenge, so the first request of a sync - the
OPTIONS probe - looped forever and the sync never reported anything.
Allow one retry per request (webdav_retry_auth()); a second challenge
now fails the request with an error naming the credentials. Each
MKCOL in a directory walk counts as its own request.

A retried PUT also sent garbage. webdav_do_update() reads the upload
into memory, leaving the file at its end, and the retry read it again
from there: it got nothing and sent a buffer of the right length full
of uninitialised memory, replacing the save on the server whenever a
nonce expired mid-upload. Rewind before reading, and fail the upload
on a short read.

samples/network/webdav_reauth includes webdav.c and plays the server
for OPTIONS, GET, PUT, DELETE, MOVE and MKCOL: a server that keeps
rejecting the credentials must get exactly one retry and one reported
failure, a single stale-nonce challenge must still be retried and
succeed, and every PUT must carry the file's bytes. Against the old
code every request type loops and every retried PUT sends the wrong
bytes. Runs in CI Linux samples/network under ASan+UBSan and TSan.
2026-09-21 14:18:10 +02:00
LibretroAdmin 2bb0215ddd webdav: check callback state before using it
webdav_read_cb(), webdav_update_cb(), webdav_delete_cb() and
webdav_backup_cb() dereferenced webdav_cb_st->path to log a failure,
and read and delete again on the reauth path, before the NULL check
that came further down. Check once at the top and return, as
webdav_stat_cb() already does, and drop the checks that can no longer
fail.
2026-09-21 13:39:05 +02:00
LibretroAdmin d997fe013b cloudsync: stop refetching files the server does not have
A file listed in the server manifest but missing on the server (such
as one uploaded under a name cut short at '#' before the WebDAV path
encoding fix) came back from the driver as success with no file.
task_cloud_sync_fetch_cb() logged that as "Failed to write file from
server", kept the entry in the server manifest and failed the sync,
so the same fetch failed on every sync after.

That result was ambiguous in the drivers too: WebDAV and Google Drive
also returned success with no file when the download could not be
written locally, and when the file on the server was empty. Both now
hand back an open RFILE for any found file, empty included, and
report a local open or short write as a failure. Success with no file
now means only "not on the server", which is what SMB, S3 and
iCloud Drive already meant by it.

task_cloud_sync_fetch_cb() then drops such an entry from the server
manifest and marks it for upload, leaving the local file and local
manifest alone: if this device has the file, the next sync sees it
as new and uploads it, which repairs the server.

Since an empty server manifest now arrives as an empty RFILE rather
than none, treat an empty one as no manifest, as an empty body was
before; rjson rejects empty input.
2026-09-21 13:39:01 +02:00
LibretroAdmin 7d158390ae s3: log the transport failure reason
task_http passes the failure reason (stage plus code, such as
"ssl_connect_failed (-0x7780)") to every transfer callback as err,
but every S3 callback discarded it with (void)err, so a transport
failure was logged only as "HTTP -1". Pass err through to
s3_log_http_failure() and s3_multipart_fail() and print it when there
is no status, the way webdav_log_http_failure() already does; the
"Did not get HTTP data" warnings include it too.
2026-09-21 13:30:59 +02:00
Chris BradelandLibretroAdmin 4ae32cee45 Fix WebDAV cloud sync crashes and encoded save paths (#19598)
* cloudsync: reject invalid manifests

Check the JSON parser result before sorting a manifest. Discard partial manifests and stop the sync with an error instead of dereferencing an incomplete entry.

* webdav: handle missing files and encode sync paths

Treat HTTP 404 as a missing file without writing the server's error page to disk. Encode sync paths separately from the WebDAV base URL so filenames containing '#' are sent as filenames, not fragments.

* webdav: keep '/' as separator in deleted/ backup path

fill_pathname_join_special(dest, "deleted", path) appends
PATH_DEFAULT_SLASH when the directory has no slash of its own. On
Windows that is a backslash, which net_http_urlencode then sends as
%5C, so every backup MOVE targeted "deleted%5C<path>" instead of the
deleted/ collection. Pass "deleted/" so the existing '/' is kept.

* cloudsync: recover from an invalid local manifest

manifest.local is rewritten in place, so a crash, full disk or power
loss during the write can leave it empty or truncated. rjson rejects
empty input, so with the new manifest validation every later sync
stopped with "Invalid manifest" and the only way out was switching
sync drivers, which is what deletes the file.

Log a warning, delete the bad local manifest and continue as a first
sync. With no local manifest, files that differ between local and
server go through conflict resolution instead of being overwritten.
An invalid server manifest is still fatal.

---------

Co-authored-by: LibretroAdmin <[email protected]>
2026-09-21 13:22:18 +02:00
libretroadmin 9f5bcf6b3a webdav: probe the endpoint even when it wants no credentials
Every sync opens with OPTIONS, and the response to it is the only thing
that establishes the server speaks WebDAV. MKCOL answers 405 on a
collection that is already there (RFC 4918 9.3.1), and that 405 is
forgiven only for a verified endpoint, so the probe now runs whether or
not a username and password are configured: an endpoint reached
anonymously uploads into directories it created on an earlier sync
instead of failing on the first MKCOL and taking every PUT with it.

A 401 carrying a digest challenge is only worth retrying when there are
credentials to answer it with; without them the retry would meet the
same challenge for as long as the server kept issuing it, so cloud sync
reports the endpoint as needing authentication and stops. The
webdav_options sample drives webdav_sync_begin() through both cases and
asserts the probe is dispatched, with an Authorization header only where
one exists.
2026-09-14 10:43:57 +02:00
Suk Eunho fb6c49fd7b webdav: don't walk past a NULL when the URL has no path (#19516) 2026-09-08 06:11:19 +02:00
LibretroAdmin 99f381f4ac webdav: judge an endpoint by its DAV methods, not by PUT
The OPTIONS probe reads the DAV header (RFC 4918 10.1) and, failing
that, looks for PROPFIND, MKCOL or PROPPATCH in Allow; either one
means WebDAV. PUT is no longer consulted. Allow describes the target
resource (RFC 9110 5.6.1.2), the target cloud sync probes is always a
collection, and PUT is not defined on a collection (RFC 4918 9.7.1),
so a correct server omits it - Apache mod_dav does - and its absence
says nothing about the endpoint. Only a response carrying an Allow
header with no WebDAV method in it now fails the sync, which is the
plain HTTP server the probe is there to catch; dav_verified, and so
the MKCOL 405 rule that rides on it, follows the same signal.

Fixes libretro/RetroArch#19486, where Koofr was rejected for a
collection Allow list without PUT while working in other WebDAV
clients.

samples/network/webdav_options drives the probe as shipped over the
mod_dav, Allow-only, DAV-only, HTTP/2-lowercase, plain-HTTP and
plain-HTTP-with-PUT header sets plus the token-matching edges, and
runs clean under ASan/UBSan/LSan and TSan. C89 checked with the
linux-c89 job's flags.
2026-09-02 07:11:11 +02:00
libretroadmin c289215f6e net: say which transport stage failed, and the TLS library's code for it
A transfer that never reaches a status now reports where it stopped.
net_http keeps the first failing stage ("dns_lookup_failed",
"ssl_connect_failed", ...) and, for the TLS stages, the library's own
error through the new ssl_socket_last_error(), which both backends
implement: the last mbedtls return code from config, setup, hostname
or handshake, and BearSSL's engine error. net_http_failure() hands
them to the task layer, whose error string becomes
"Download failed: ssl_connect_failed (-0x7200)." rather than
"Download failed.", and the webdav failure line appends it whenever
the status is -1.

This is what a log from a console can carry: the transport logger
only exists in DEBUG builds and writes to stderr, so a Vita that could
not complete a handshake had nothing to show but "HTTP -1". The
mbedtls code is printed in the form its error table uses, so -0x7200
reads as MBEDTLS_ERR_SSL_INVALID_RECORD without a debug build
(libretro/RetroArch#19475).
2026-08-31 04:57:09 +02:00
LibretroAdmin e1a842ca8e webdav: ask the server for each collection once per sync
Track the collections a sync has created and skip the MKCOL for any the
walk has already been through, so the request count follows the number of
distinct directories rather than the number of files. A 143-file sync
over the directory layout in libretro/RetroArch#19475 issues 4 MKCOLs
instead of 357.

The list is per-sync and dropped in webdav_sync_begin, so a directory
removed on the server between syncs is created again. Lookups compare
exactly rather than through string_list_find_elem(), which folds case
where WebDAV does not. Task callbacks are retired on the main thread, so
the list needs no lock.

Covered by a harness driving the walk against a simulated server,
including out-of-order and interleaved completion, MKCOL failure, and 405
from a server that never advertised DAV; clean under ASan+UBSan+LSan and
TSan. C89 checked with gcc -std=c89 -pedantic; cross-compiled for Vita.
2026-08-30 16:04:46 +02:00
LibretroAdmin 820b3bc6cb webdav: detect endpoints that do not speak WebDAV
Cloud sync opens with an OPTIONS request, so inspect that response for
the DAV header (RFC 4918 10.1) and for PUT in Allow (RFC 9110 5.6.1.2)
before doing any work.  A server that answers OPTIONS but rejects PUT is
reported at once with a hint that WebDAV endpoints are usually https on a
dedicated host; a missing DAV header alone only warns, since not every
server is strict about advertising it.

MKCOL returning 405 counts as "collection exists" only once the server
has been seen to implement DAV, and an https URL in a build without TLS
now says so.  Failure diagnostics are assembled into one buffer and
emitted in a single write, so concurrent transfers no longer splice their
log lines into each other.

Helpers covered by a standalone harness under ASan+UBSan+LSan and TSan,
including the header sets from libretro/RetroArch#19475; C89 checked with
gcc -std=c89 -pedantic.
2026-08-30 15:52:46 +02:00
LibretroAdmin b12d1d0684 strl: include compat/strl.h where strlcpy_lit() is used
The literal conversion left 39 files calling strlcpy_lit() without
including the header that declares it. They built anyway: the files were
already calling strlcpy() without including compat/strl.h either, and on
glibc 2.38 and later that resolves to the C library's own strlcpy(), so
nothing ever forced the include. strlcpy_lit() has no such fallback --
it is an implicit declaration that compiles and then fails to link.

RetroArch itself did not notice, because its translation units pull the
header in through something else. The standalone samples do not, and
samples/tasks/database is where it surfaced:

  scan_begin_budget_test.c: undefined reference to `strlcpy_lit'

Adding the include to each of them fixes that and removes the reliance
on the accidental libc fallback for strlcpy() at the same time.

Checked by building every sample rather than the one that failed: 42
under samples/, 38 under libretro-common/samples and libretro-db/samples,
with libxrandr, libdrm and libvulkan headers installed so the X11, KMS
and Vulkan ones actually compile instead of being skipped. All build
except samples/gfx/font_lifecycle, which fails identically on a clean
tree (undefined reference to video_driver_thread_wrapper_active in its
HAVE_THREADS variant) and is unrelated.

The sample binaries were run as their workflows run them, including
"make fixtures" for gfx_thumbnail_preview and a long enough timeout for
the 1000000-iteration SPSC stress in gfx_thumbnail_status_atomic: all
pass. samples/tasks/database builds, passes "test -x database_task" and
exits 1 with its usage line, which is what the smoke step expects.
2026-08-30 02:53:03 +02:00
LibretroAdmin b9be38fd55 strl: copy string literals without scanning them at runtime
strlcpy() begins with strlen() of its source. It lives in another
translation unit, so with no LTO that scan happens at runtime even when
the source is a literal whose length the compiler already measured, and
the call itself cannot be folded away either.

strlcpy_lit() passes the length over as sizeof(). When the destination
size is also a constant -- sizeof(buf), the usual case -- nothing is
left but the stores memcpy() compiles to:

  strlcpy      "core_options"  sizeof(buf)   7.01 ns
  strlcpy_lit  "core_options"  sizeof(buf)   0.42 ns
  strlcpy      41-char literal sizeof(buf)   7.74 ns
  strlcpy_lit  41-char literal sizeof(buf)   0.71 ns
  strlcpy      "core_options"  runtime len   7.11 ns
  strlcpy_lit  "core_options"  runtime len   0.70 ns

The destination bound is kept. When the literal does not fit it defers
to strlcpy(), so the copy truncates as before rather than overrunning;
a form that took only the literal would have turned every one of these
sites into an overflow waiting for a small buffer. The literal is
concatenated with "" so a non-literal source is a compile error rather
than a sizeof() that silently measures a pointer.

818 sites across 104 files were converted by parsing arguments rather
than by pattern-matching text: only a three-argument call whose second
argument is a bare string literal is rewritten, so a macro that happens
to expand to a literal is left alone. Line endings are preserved, so
the CRLF files carry only their own changed lines.

test/compat/test_strl.c compares the two against each other over every
destination size from 0 past the literal's length, for literals that
are empty, single-character, path-shaped, adjacently concatenated and
escape-bearing, checking the return, the terminator and the whole
buffer so an overrun past the terminator is caught. ASan+UBSan+LSan and
TSan both 4/4. The full menu_displaylist_build_list walk over every
displaylist type is byte-identical to before the change
(md5 a2a081bcac09cdc8d72d67efd14ac40e).
2026-08-30 02:10:20 +02:00
LibretroAdmin ec56b24dc7 (cloud sync) google_drive: never return without invoking the completion handler
task_cloudsync.c holds an in-flight count that only the driver's
completion handler releases, so a true return from read/update/delete
is a promise that the handler runs exactly once; any path that skips
it stalls the sync task forever at 100% (#19435). gdrive_do_patch()
had two such paths - a zero-byte local file and a failed malloc - and
all fourteen task_push_http_*() call sites had another, since those
helpers return NULL without ever calling back. Every failure exit now
reports through the handler, via gdrive_cb_state_finish() /
gdrive_walk_fail(), and a zero-byte file is uploaded normally so a
non-empty server copy gets truncated. The begin path and the entry
points' calloc()s instead return false, which task_cloudsync already
fails cleanly.

Also: drop the manual Content-Length on PATCH uploads
(net_http_send_request() emits it for every P* method, so it went out
twice), and clear poll_pending when the OAuth poll request cannot be
dispatched so device authorization retries instead of stalling.

Sanitizer sweep: ASan+UBSan+LSan clean, TSan clean over 31 runs, on a
15-scenario harness driving the driver against a scripted HTTP stub
(success, empty file, create-then-upload, folder walk, 401->refresh->
retry, and every dispatch-failure path); exactly one completion per
operation, zero leaks. gcc -std=gnu89 -Wall -Wdeclaration-after-statement
clean.
2026-08-19 09:50:08 +02:00
LibretroAdmin 5fc46bcb1b Revert the SMB2 context pool series
Reverts, newest first:

- 05db1e076 "smb2: rebuild the pool when settings change, move cloud
  sync onto it, fix signing negotiation"
- cea330ebe "smb2: allocate nothing at all when no SMB server is
  configured"
- 3c7447d9d "smb2: serialize context use, replace cached context
  pointers with generation handles"
- 36766173e "smb2: publish loaded settings to the SMB backend, harden
  the pool paths"

05db1e076 was not part of the original series but depends on it
(smb_cfg publication, smb_shutdown(), generation handles), so it has
to go with the other three; a three-commit revert does not apply
cleanly and would not compile.

All touched files are restored byte-exact to their state at 558b24a68,
the parent of the series. Unrelated later work in configuration.c
(282ca43c0, gl_direct_spirv setting rows) and vfs_hybrid.c (c46b14a79)
is untouched. No in-tree references to the removed pool API
(smb_pool_acquire/release/heal_if_dead) remain; retroarch_smb_init and
struct smb_settings predate the series (e9f9dd443) and stay.

Note this reintroduces the issues the series addressed, including the
signing-mode negotiation fix and the late publication of the numeric
SMB settings fields.
2026-08-10 20:40:18 +02:00
LibretroAdmin 05db1e0766 smb2: rebuild the pool when settings change, move cloud sync onto it, fix signing negotiation
Three related changes to how SMB connections are owned:

1. Settings changes now take effect without a restart. The pool
   snapshots the parameters it was built with (server, share,
   credentials, workgroup, timeout, context count, auth mode - subdir
   excluded, it only affects path building and reads the live
   configuration). A publication whose parameters differ tears the
   pool down; the next smb:// access rebuilds it lazily with the new
   values, and an identical republication leaves a live pool
   untouched. Publications run on the main thread and smb_shutdown()
   takes every lock, so in-flight operations finish first and their
   generation handles fail closed - the rebuild rides entirely on the
   machinery the pool-locking commit introduced. Combined with
   publish-on-save, the full menu flow now works end to end: change a
   setting, save, and the next SMB access uses it.

2. The cloud sync SMB driver used to connect a private context per
   sync session and hold it in unlocked driver state. It now draws
   from the shared pool: smb_pool_acquire()/smb_pool_release() hand
   out a connected context under its slot lock per operation, and
   smb_pool_heal_if_dead() gives cloud sync the same
   heal-once-and-retry policy the VFS operations use at each
   operation's first server contact - previously a transport drop
   simply failed the whole sync cycle. Per-session connection setup,
   the driver's duplicate parameter plumbing, and its unsynchronized
   context field are all gone; connection ownership has exactly one
   home. Acquisition also prefers idle slots now (slock_try_lock
   across the pool, blocking on the round-robin slot only when every
   slot is busy), so a long cloud sync transfer holding one slot does
   not stall unrelated VFS operations that round-robin onto it.

3. smb2_set_security_mode() takes the SMB2_NEGOTIATE_SIGNING_* flags,
   but the VFS backend passed the SMB2_SEC_* authentication constants
   into it, so selecting Kerberos (2) silently demanded
   signing-required (2) and the auth probe's first attempt did the
   same. Signing negotiation and authentication are independent:
   signing is now always ENABLED (the cloud sync driver already did
   this correctly) and the authentication call carries the auth mode.

Verified offline before commit:

- Sanitizer harness extended with: identical republication leaving a
  live pool untouched (zero reconnects), a changed parameter tearing
  it down and the next access rebuilding with the new values, clearing
  the server shutting SMB down entirely, signing always negotiated
  ENABLED with authentication carrying the configured constant
  (including KRB5, where the old coupling requested REQUIRED), and the
  public acquire/heal/release API single-threaded and inside the
  8-thread hammer with heal chaos. ASan+UBSan+LSan clean and TSan
  clean across all of it; the negative test (locks compiled out) still
  fails immediately under TSan.
- Full Makefile build with --enable-builtinsmbclient, threads,
  networking and cloud sync compiled in; clean link, no new warnings
  outside vendored deps. Settings table drift check, griffin-order
  unity compile, and the migration tool self-test all clean.

The stale-handle behavior on rebuild is the generation design working
as intended: streams opened before a settings change fail closed on
their next operation and callers reopen against the new pool, which is
the correct semantic for "the server I was talking to changed".
2026-08-10 02:57:09 +02:00
libretroadmin 51c3c8dca0 task_http_emscripten: move to emscripten_fetch, restoring headers, binary bodies and response status
The emscripten backend sat on emscripten_async_wget2_data(), and that
API is the root cause of how far this file had drifted from
tasks/task_http.c.  It takes the request body as a single
NUL-terminated `const char*` and offers nothing for request headers, a
user agent, or reading response headers back, so everything the native
path provides beyond "GET a URL into memory" was silently dropped or
stubbed out.  emscripten_fetch() covers all of it; switching over
closes six defects at once.

- Request headers were accepted and discarded on every entry point.
  task_push_http_transfer_with_headers() sent none, which meant no
  Authorization header, so cloud sync could not authenticate at all.
  All five webdav helpers logged "Response headers not supported,
  webdav won't work" and then did exactly that.  RetroArch passes
  headers as a raw CRLF blob and fetch wants a NULL-terminated
  key/value array, so http_req_headers_parse() converts between them.

- task_push_webdav_put() passed put_data as a NUL-terminated string
  and dropped the `len` argument entirely, truncating any binary
  payload at its first zero byte.  Length now travels explicitly.

- Response headers are recovered via
  emscripten_fetch_get_response_headers() and split into the same
  string_list of "Name: Value" lines net_http.c produces.

- resp->status was hardcoded to 200, and the error callback recorded
  no status at all, so callers branching on data->status -- webdav's
  401 digest challenge, the thumbnail downloader's 404 check -- saw a
  NULL task_data and could not distinguish "not found" from "the
  network died".  Both paths now settle with the real status, and
  error bodies are kept since 4xx/5xx routinely carry one.

- wget_onload_cb() returned early when its malloc failed, leaking the
  whole response buffer.  The fetch is now closed unconditionally.

- Cancelling a task never aborted the transfer.  http_handle_free()
  closes the fetch, which emscripten guarantees suppresses any further
  callback.

Two lifetime fixes carried over from task_http.c:
task_push_http_transfer_file() assigned t->title after
task_queue_push(), the exact use-after-free already fixed and
commented in the native file -- harmless single-threaded, a genuine
race under -pthread/PROXY_TO_PTHREAD.  And emscripten_fetch does not
copy requestData or requestHeaders, so http_handle now owns both for
the fetch's lifetime rather than retaining a caller's buffer.

Also drops the local http_transfer_progress_cb(), which called
video_display_server_set_window_progress() unconditionally while its
header was behind #ifdef RARCH_INTERNAL, in favour of the shared
task_window_progress_cb() the native path uses; and sets
RETRO_TASK_FLG_ALTERNATIVE_LOOK, which was missing here.

network/cloud_sync/webdav.c matches the digest challenge
case-insensitively now.  Browsers lower-case response header names, so
"WWW-Authenticate: Digest " never matched on this backend.  Header
names are case-insensitive per RFC 9110 5.1, so the consumer was what
needed fixing; webdav_create_digest_auth() skips a fixed byte count
past the prefix, so the offset is unaffected.

Makefile.emscripten passes -sFETCH=1, which the fetch API requires.

Adds samples/tasks/http/http_emscripten_headers_test.c covering both
conversions against hand-built blobs of the kind
task_push_webdav_move() and task_push_http_transfer_with_content()
produce: bare LF, missing trailing CRLF, empty values, embedded
colons, malformed lines, and growth past the initial array capacity.
A header dropped here is an auth failure rather than a crash, so the
degenerate cases are worth pinning.  33 checks, clean under ASan and
UBSan.  Verified with emcc 6.0.5: the file compiles clean at -Wall and
-sFETCH=1 links.
2026-08-02 08:13:55 +02:00