Commit Graph
2 Commits
Author SHA1 Message Date
libretroadmin 03245b669e rthreads: fold the console pthread wrapper headers into rthreads.c
rthreads.c is the only user of the GEKKO and 3DS pthread wrappers, so
they now live inline in its platform include section rather than as
separate headers. psp_pthread.h is referenced by nothing - PSP threading
comes from the toolchain's pthreads - and is removed outright. Comments
across the tree that point at thread stack sizes now name rthreads.c.
2026-08-31 22:58:35 +02:00
LibretroAdminandlibretroadmin 0b96149251 File I/O: mapped-view accessor, allocation-free file/buffer comparison, and the stack-safety work that fell out of it (#19330)
* file_stream/vfs: read files that report no size, and expose the mapping

Two file-I/O defects found by auditing the read paths, plus the
construct the second one was missing.

filestream_read_file() sized its allocation from
filestream_get_size() and read exactly that many bytes, so a file
reporting zero came back empty - with a *success* return,
indistinguishable from a genuinely empty file.  Every procfs entry
stats as zero, so this was not a corner case: the /proc/apm battery
probe in platform_unix.c, the /proc/modules sg check in cdrom.c and
the dingux /proc/jz/battery reader all silently read nothing and
reported success.  mem_stats.c reads /proc/meminfo through a raw
open() and read() into a stack buffer, which is what routing around
this looks like when the helper is not fixed.

A stat size is a hint, not a contract - synthetic filesystems have
no length until read, and a growing file's is stale the moment it
is taken - so with no usable hint, read to EOF.  Verified against
real procfs: /proc/cpuinfo and /proc/meminfo now deliver their
contents where they previously delivered an empty string.  A
genuinely empty file still reads empty and a missing file still
fails, which are the two cases the fix could plausibly have broken.

RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS has always mmap'd the
file, but the mapping was reachable only from inside
retro_vfs_file_read_impl(), which memcpy's out of it - so asking for
a map bought a copy of data that was already addressable, and the
point of the hint was lost.  Only two call sites in the tree use it
at all.  Add filestream_get_mapped_ptr() (and the _impl behind it):
a borrowed read-only view of the whole file, valid until close, NULL
whenever there is no mapping - no HAVE_MMAP, no hint, a write mode,
or a frontend-supplied VFS, which is driven entirely through the
callbacks and has no mapping this side of them.  NULL is a normal
answer, so callers keep their copying path and treat this as a fast
lane.

On top of it, filestream_matches_buf(): does this file hold exactly
these bytes?  It settles a size mismatch without reading, compares
in place against the map where there is one, falls back to a 64 KiB
window otherwise, stops at the first differing byte, and allocates
nothing.  The window is sized to the VFS's own stdio buffer because
below that each read is staged through it - reintroducing the copy
the mapped path exists to avoid, which cost 41% in the first version
of this.

save.c's SRAM dirty-check is the first consumer.  It read the whole
save file into a fresh allocation purely to memcmp and free it, and
checked the size only afterwards - so a save that had changed size,
the one case where the answer is free, still paid a full-size malloc
and a full-file read.  Cores with megabytes of save RAM paid that on
every save, against the memory budget that is scarcest on the
handheld targets.  Measured, unchanged/changed and peak extra heap:

  128 KiB  -50% / -66%    128 KiB -> 0
    2 MiB  -47% / -94%      2 MiB -> 0
    8 MiB  -49% / -99%      8 MiB -> 0
   32 MiB  -80% / -100%    32 MiB -> 0

The compressed lane is unchanged: rzipstream has a real header, so
it never had the size problem and has no map to compare against.

Oracle: samples/file/vfs/vfs_mapped_ptr_test, added to the samples
run allowlist, plus an explicit MMAP=0 CI lane.  Both lanes are
needed and the Makefile now makes HAVE_MMAP a variable to get them:
with mappings, matches_buf compares in one memcmp and its windowed
fallback never executes, so the window-boundary cases in the default
run assert on code that did not run.  The test says which lane it
took.  Verified to fail with the size-0 fix reverted (procfs reads
empty), with the window advance off by one (MMAP=0 only - the mapped
lane does not notice), and with the size gate removed.  The dir's
Makefile also gained the conventional SANITIZER opt-in block, absent
until now, so the workflow's SANITIZER= was being silently ignored
for both tests here.

Verified: full build and link clean; save.c compiles clean in both
the HAVE_COMPRESSION and -UHAVE_COMPRESSION lanes; griffin-shape
single-TU compile of the touched libretro-common files clean;
ASan/UBSan/LeakSan and TSan clean on both MMAP lanes; C89 -pedantic
-Wdeclaration-after-statement clean on file_stream.c (the eight
findings in vfs_implementation.c are pre-existing, identical before
and after); workflow YAML parses; the data_transfer and image-decode
oracles added earlier still pass.

Noted and not claimed: the hint gate in
retro_vfs_file_get_mapped_ptr_impl() is redundant by construction -
'mapped' is only ever set under the hint - so removing it changes
nothing observable and the oracle does not discriminate it.  It is
kept to match the read and seek paths, which gate the same way.

* vfs: stop zeroing the per-open stdio buffer

Every buffered open allocated its stdio buffer with calloc, so each
one cleared 64 KiB (128 KiB on WiiU, which was already exempt by
using memalign) before handing it to setvbuf.  Nothing ever reads
those bytes: stream->buf is allocated, passed to setvbuf, and freed,
and stdio fills it before it is read from.  The zeroing was pure
cost.

It was also the single dearest part of opening a small file.  3000
opens of 256-byte files, the shape a library scan or a menu full of
thumbnails produces - thousands of opens, few bytes each:

  filestream_read_file   3.2 us -> 1.8 us per file
  open + 64-byte header  3.1 us -> 1.8 us per file

44% off, from not clearing a buffer the read never touches, and 187
MiB less allocator traffic per 3000 files.  The buffer size itself
is unchanged and stays justified by the write measurements recorded
above it; this is only about how the allocation is obtained.

The regression this could introduce is stale bytes surfacing through
a read, so vfs_mapped_ptr_test grew a lane for it: alternating opens
of two files with different lengths and distinct fills, read to EOF
in 4093-byte steps so the final read of each is short and leaves the
buffer partly filled, verifying every byte and that reads past EOF
deliver nothing rather than residue.

That lane is a guard rather than a discriminator - it cannot fail on
this change, because the change is safe by construction - so it was
checked the other way round: a build that deliberately memsets the
buffer to 0xDE before setvbuf still passes every content check,
which is the positive evidence that stdio never surfaces these bytes
to a reader.

Verified: full build and link clean; griffin-shape single-TU compile
clean; ASan/UBSan/LeakSan and TSan clean on both MMAP lanes of the
oracle.

* task_cloudsync: hash through filestream, and hash the whole file

task_cloud_sync_md5_rfile() was the only place in the tree that read
a memory mapping straight out of the VFS handle - filestream_get_vfs
_handle() and then hfile->mapped - which is a task reaching past the
layer that owns the mapping into a struct it does not, and which
answers nothing at all for a frontend-supplied VFS.
filestream_get_mapped_ptr() asks the same question through
filestream, and NULL stays a normal answer, so the read loop remains
the general path.

Two things fall out of doing it that way.

The 256 KB read buffer was allocated before the branch, so every
hash taken from a mapping malloc'd and freed a quarter of a megabyte
without a byte of it being touched.  It now belongs to the branch
that uses it.

More importantly the two branches did not hash the same bytes.  The
mapped branch always covered the whole file; the read branch started
wherever the stream happened to be.  Two of the three callers hand
this a freshly opened stream, but the third takes one from a fetch
callback, so on a platform with mappings and a non-zero position the
same content produced two different manifest entries depending on
which branch ran.  A partial hash is not a useful answer to "what is
this file", so the read path is squared up with the mapped one - seek
to the start first - rather than the other way round.

The oracle's map-versus-read agreement check covered only the first
4096 bytes, which is exactly the prefix a whole-file consumer like
this would pass while still producing a different digest.  It now
compares the entire file, and is verified to fail both with the
reported mapping length off by one and with the mapping base off by
one.

Verified: full build and link clean; task_cloudsync.c compiles clean
with HAVE_CLOUDSYNC and HAVE_NETWORKING; ASan/UBSan/LeakSan and TSan
clean on both MMAP lanes of the oracle; the eight C89 findings in
vfs_implementation.c are pre-existing and unchanged.

* rzip_stream/save: compare compressed saves in place too

The SRAM dirty-check got a size-first, allocation-free compare in a
recent commit, but only on the uncompressed lane - and
HAVE_COMPRESSION is the default build, so the lane that actually
ships kept decompressing the entire save into a fresh buffer to
memcmp and free it, and kept checking the size only afterwards.  The
measured win applied to the minority configuration.

rzipstream_matches_buf() is the counterpart: the uncompressed size
comes from the rzip header, so a save that changed size is settled
before anything is decompressed; otherwise it decompresses a chunk
at a time into a fixed window and stops at the first difference.
There is no mapped fast path here and there cannot be - the bytes
being compared do not exist on disk in that form - so what this
avoids is the whole-file allocation rather than the copy.

save.c now uses the matching construct in each lane, and the two
lanes are one block rather than two divergent copies.

One deliberate asymmetry, found by the test rather than by
inspection: a zero-byte file never matches here, not even against
zero bytes, where filestream_matches_buf() does match it.  An empty
file has no rzip header, so rzipstream_open() cannot open it and
there is nothing to compare - and false is the answer that serves
the caller, since writing then replaces it with a valid file.  My
first version of the oracle asserted the filestream behaviour and
failed; the construct was right and the expectation was wrong.  It
is now asserted explicitly, and documented in the header, so the
difference stays deliberate rather than surprising anyone moving a
call site between the two.

Oracle: samples/streams/rzip/rzip_matches_buf_test, added to the
samples run allowlist.  rzip reads are decompressions, so the chunk
loop re-enters an inflate stream rather than walking a buffer, and
its boundaries do not line up with the compressor's chunking - a
difference on a read boundary, on an rzip chunk boundary, and
between them are three different offsets, and all three are checked,
on both compressed and uncompressed input since rzip accepts both
through the same entry point.  Verified to fail with the chunk
advance off by one (2 failures) and with the size gate removed (4).

The dir's Makefile also gained the conventional SANITIZER opt-in
block, absent until now, so the workflow's SANITIZER= was being
silently ignored for rzip_chunk_size_test as well.

Verified: full build and link clean; save.c compiles clean in both
the HAVE_COMPRESSION and -UHAVE_COMPRESSION lanes; ASan/UBSan/
LeakSan and TSan clean; C89 -pedantic -Wdeclaration-after-statement
clean on rzip_stream.c; workflow YAML parses; rzip_chunk_size_test
still passes.

* rjson: release the grown buffers in rjson_parse_quick, and let CI see it

Fifteen sample Makefiles had no SANITIZER opt-in block.  The samples
workflow passes SANITIZER=address,undefined to every sample
directory, so the variable was accepted and ignored and those
targets built unsanitized - while the workflow's own comment names
rbmp_test and cdrom_cuesheet_overflow_test among the tests that
"use ASan as the regression discriminator for the bound-check they
verify".  Two of the tests it named were in that set: their bound
checks had nothing behind them.

Adding the block to all fifteen turned up a real leak.

rjson_parse_quick() keeps its rjson_t on the stack, but the two
buffers inside it are not on the stack: the string buffer grows onto
the heap for a long string token, and the nesting stack grows for
deep nesting.  Neither was released on either return path.
rjson_free() cannot be used there because it frees the handle too,
so the buffer release is split out into _rjson_free_buffers() and
called from both.

This is not only a leak.  The one caller in the tree is
netplay_room_parse.c, parsing lobby responses, so the input is a
remote server's and both growths are its choice - string tokens up
to _RJSON_MAX_SIZE, nesting up to the 50-level ceiling - leaking
per refresh at whatever size the other end picked.

Coverage for both halves, since the fix has two:
test_parser_long_string() already forced string growth and is what
LeakSanitizer caught; nothing forced stack growth, because the
inline stack holds 10 levels and every existing case is shallower.
Both of parse_quick's returns are now exercised, because both
leaked: 40 levels grows the stack and parses, 200 grows it to the
ceiling and then fails, which is the path a hostile input takes.
Verified against the reverted fix, which reports three distinct
leaks - the string and one per nesting lane.

Verified: all fifteen newly-sanitized directories build and their
tests pass under ASan/UBSan/LeakSan (28 sample binaries run clean,
including every previously-unsanitized one); full RetroArch build
and link clean; griffin-shape compile of rjson.c clean; C89
-pedantic -Wdeclaration-after-statement clean on rjson.c.

* CI: run the seven sample tests that were only ever compiled

Nine sample targets were built by the samples workflow and never
executed.  Two of them should not be: rzip is a CLI tool that prints
usage and exits non-zero without arguments, and http_test performs
real HTTP requests against an external host, so running it would
make the job depend on a third party being up.  The other seven are
self-contained regression tests that exit 0 on success -
data_transfer_source/prefix/window_test, audio_transfer_window_test,
rjpeg_test, r7z_lzma_props_test and vcdiff_test - and all seven pass
under ASan/UBSan/LeakSan.  They are now in the run allowlist.

The reason they were lost is worth fixing separately from the fact
that they were.  A target absent from RUN_TARGETS printed
"[skip-run] not in run allowlist" and carried on, which reads
identically whether the omission was deliberate or an oversight - so
adding a test to a directory that already had one, and forgetting
the allowlist, produced a test that compiled forever and ran never,
silently.  That is the same failure shape as the SANITIZER blocks in
the previous commit: CI reporting coverage it did not have.

So the skip is now explicit.  NORUN_TARGETS lists the two that are
deliberately not run, each with its reason, and a target in neither
list is an error naming the target and both lists.  Verified by
removing vcdiff_test from the allowlist, which fails the job with
exactly one error identifying it.

Verified: the workflow's own script run against the tree with
MAKE_ARGS as CI sets it - 33 directories built, 41 targets run, zero
unclassified.  The one failure in that run is retro_spsc_test timing
out at 60s, which is this single-core container rather than the
change: it passes with a longer timeout, and CI runners are not
single-core.

* vfs: define the mapped-ptr accessor on UWP, and check backend parity

The UWP build fails to link:

  griffin.obj : error LNK2001: unresolved external symbol
                retro_vfs_file_get_mapped_ptr_impl

vfs_implementation_uwp.cpp does not extend vfs_implementation.c, it
replaces it - the UWP project compiles the .cpp and not the .c - so
adding the accessor to the C backend left UWP without it, while
file_stream.c, which both builds share, calls it unconditionally.
Every other target linked fine.

That backend never maps a file: retro_vfs_file_open_impl() there
nulls 'mapped' and clears RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS
unconditionally.  So the definition is just NULL and a zero length,
which is the documented "no mapping here" answer that every caller
already has to handle.  It still has to be written out: a missing
definition is a link error, not a fallback.

Checked the rest of the surface rather than only the symbol that
broke - the two backends now agree on all 22 retro_vfs_*_impl
functions, with no other gaps.

The failure mode is worth a guard.  It compiles everywhere, links
nowhere but UWP, and surfaces at the end of a long MSVC build on a
runner most contributors do not have - so the next function added to
the C backend breaks it again the same way.  tools/vfs_backend_parity
.py compares the two backends by symbol and fails on anything the C
backend defines and the UWP one does not, as a source-level check
costing nothing on Linux, wired into the libretro-common tests
workflow.  Verified by deleting the definition again, which reports
exactly the missing symbol.

Verified: C linkage confirmed empirically rather than assumed - the
definition compiled in a C++ TU emits an unmangled
retro_vfs_file_get_mapped_ptr_impl matching the undefined symbol
file_stream.c asks for, and a harness that mimics the UWP
arrangement (real VFS with its C definition removed, the symbol
supplied from a C++ TU) links and runs, having first reproduced the
LNK2001 without it.  Full build and link clean; the
vfs_mapped_ptr_test oracle passes on both MMAP lanes; workflow YAML
parses.

* streams: size the compare windows by the stack, not by the stdio buffer

filestream_matches_buf() and rzipstream_matches_buf() each held a
64 KiB buffer on the stack.  sthread_create() gives Vita threads a
0x10000 stack - 65536 bytes - and -fstack-usage puts the frame at
65648, so the function overflows the whole thread stack on its own,
before any caller frame.  Both are libretro-common API, so a caller
on a spawned thread is not hypothetical, and an overflow there is
not a slow path.

4 KiB instead: 65648 -> 4208 bytes measured.

What that costs is small and lands in the right place.  A window at
or above the VFS's own 64 KiB stdio buffer does read better, because
those reads go straight to the destination instead of being staged
through it - on the unchanged case, which reads everything, 4 KiB
against 64 KiB is 30.9 us vs 27.0 us on a 512 KiB file and 831 us vs
774 us on 8 MiB.  Seven to fourteen per cent, and only on this
branch: where the platform maps the file the comparison does not
copy at all, so the targets that pay are the ones without mappings,
which are the same ones with the small stacks.

The comment justifying 64 KiB was also wrong to state it as
established.  It generalised from an earlier whole-file-slurp
against streaming measurement, where the 41% gap was syscall count
rather than window size; window sizes were never measured until now,
and the real figure is the seven to fourteen above.  The numbers in
both comments are now the measured ones.

Both oracles had their boundary cases aimed at 64 KiB, so with a
4 KiB window they were no longer testing a boundary at all - just
more mid-file checks under a boundary's name, which is the same
shape as asserting on a code path that did not run.  They now target
the real window and its neighbours, with the tie to the
implementation written down so the next resize does not silently
detach them again.

Verified: full build and link clean; both oracles pass under
ASan/UBSan/LeakSan and TSan, on both MMAP lanes for the filestream
one; C89 -pedantic -Wdeclaration-after-statement clean on both TUs;
VFS backend parity check passes.

* streams: size the compare buffers for the smallest thread stack

The previous commit fixed these against the Vita stack, which is
0x10000, and stopped there.  It is not the smallest: PSP and GX
threads get 8 KiB (STACKSIZE in psp_pthread.h and gx_pthread.h), and
3DS 32 KiB.  A 4 KiB local is 51% of an 8 KiB stack before the
caller's frame, which is not a margin.

Both buffers are 1 KiB now.  Measured with -fstack-usage, the frames
are 1136 and 1120 bytes - 14% of 8 KiB, against 51% at the previous
size and 801% at the original one.

The cost is real and lands where it should.  Larger reads are
faster, and at or above the VFS's own 64 KiB stdio buffer they skip
it entirely: on the unchanged case, which reads every byte, 1 KiB
against 4 KiB against 64 KiB is 1753 / 1432 / 1153 us on an 8 MiB
file.  The platforms that pay it are the ones with no memory mapping
- where there is a mapping, neither function copies at all - and
those are the same platforms with the 8 KiB stacks.  It runs when a
save is written, not in a frame.

Both sizes are now named constants in the headers, and both oracles
take their boundary offsets from those names rather than repeating
the number.  The boundary cases have already been aimed at a stale
value once, in the previous commit, and a boundary case pointed at
the wrong offset is not a boundary case - it is another mid-file
check under a boundary's name, which passes while testing nothing.

Verified: full build and link clean; both oracles pass under
ASan/UBSan/LeakSan and TSan, on both MMAP lanes for the filestream
one; C89 -pedantic -Wdeclaration-after-statement clean on both TUs;
VFS backend parity check passes.

Noted, not changed: the same -fstack-usage sweep across
libretro-common finds config_file_io.c's config_file_write() at
16432 bytes, twice an 8 KiB thread stack, from a fixed
char buf[0x4000] handed to setvbuf.  Unlike the archive functions
that also looked large, it does not shrink on those targets - their
frames are dominated by PATH_MAX_LENGTH, which is 512 there rather
than 2048, and drop under 4 KiB once rebuilt with it.  Whether that
matters depends on whether config writes ever run off the main
thread on PSP or GX, which I have not traced, so it is flagged
rather than guessed at.

* config_file: move the write buffer off the stack, and gate frame sizes

config_file_write() held its 16 KiB stdio buffer as a local, giving
it a 16432-byte frame by -fstack-usage.  PSP and GX threads get 8 KiB
of stack (STACKSIZE in rthreads/psp_pthread.h and gx_pthread.h), so
that is twice the whole stack before the caller's frame.

The previous commit flagged this without tracing it.  Traced now, and
it does reach a thread: input_autoconfigure_connect_handler is a task
handler, so it runs on the task thread whenever the queue is
threaded, and it calls ..._scan_config_files_external ->
..._index_write -> config_file_write.  That is the path taken when a
gamepad is plugged in.

The buffer is heap now, freed after fclose because it has to outlive
every write through the stream, and skipped entirely if the
allocation fails - the C library default is used instead, as in
retro_vfs_file_open_impl(), so memory pressure costs write speed
rather than the config.  Frame: 16432 -> 48 bytes.

tools/stack_budget.py measures every libretro-common translation unit
with -fstack-usage and fails on frames over 2 KiB, which leaves most
of an 8 KiB stack for callers.  It builds with -DPSP deliberately:
PATH_MAX_LENGTH is 512 on those targets rather than 2048, and a
host-shaped measurement overstates anything holding a path buffer -
several archive functions look like 6 KiB on a desktop build and are
under 4 KiB there.  Wired into the libretro-common tests workflow.

The 37 frames already over budget are recorded in an allowlist rather
than fixed.  Almost all are codec and compression inner loops (rvp9,
rh264, raac, ropus, rflac, rzstd, rvorbis) and chd, whose working
sets are large by nature, whose call graphs have not been traced to a
thread on those targets, and several of which may not be built for
them at all.  The list is unaudited and says so: this is a ratchet,
not a clean bill of health.  Each entry pins the size measured today,
so an allowlisted frame that grows is still caught.

Verified against three deliberate regressions, because the first
version of this tool passed them all while measuring nothing: gcc
writes the .su file alongside the object named by -o, not alongside
the source, so every translation unit silently yielded no frames and
the check reported a clean tree.  It now fails outright if nothing
was measured.  With that fixed it catches a new oversized frame
(the 64 KiB compare window, 65648 bytes), this commit's own fix
reverted (16448), and an allowlisted frame grown past its recorded
size (config_file_dump at 12400 against 4208).

Verified: full build and link clean; config_file_test passes under
ASan/LeakSan, which exercises the write path; C89 -pedantic
-Wdeclaration-after-statement clean on config_file_io.c; VFS backend
parity check passes; workflow YAML parses.

* tools: correct the 8 KiB stack citation, and add call-chain analysis

The stack-budget work cited "PSP and GX threads get 8 KiB, see
STACKSIZE in psp_pthread.h and gx_pthread.h" in five places.  Half of
that is wrong: nothing includes psp_pthread.h.  rthreads.c reaches
for gx_pthread.h under GEKKO and ctr_pthread.h under _3DS, and PSP
falls through to plain pthreads with no stack attribute set, so the
8 KiB figure comes from GEKKO alone - with 32 KiB on 3DS and 0x10000
on Vita.  The sizing decisions that figure drove are unaffected; the
attribution was not, and a comment that sends a reader to an
unreferenced header is worse than none.

The larger point is that per-frame checking cannot see the shape of
the problem.  tools/stack_budget.py measures one frame at a time,
which is necessary and not sufficient: a frame runs on top of
everything that called it.  config_file_write() was caught only
because its own frame held the 16 KiB; its caller,
input_autoconfigure_connect_handler, measures 1488 bytes and looks
fine.  Ten 1.5 KiB frames in a row pass every per-frame check and
still overflow 8 KiB.

tools/stack_chain.py sums frames along the call graph GCC emits with
-fcallgraph-info=su and reports the deepest chain from each root,
task handlers being the roots that matter - a threaded task queue
runs them on a worker rather than the main thread.

It is deliberately not wired into CI.  It is a screening tool, not a
verdict: the deepest path may be infeasible, indirect calls are
invisible so a real chain can be deeper than anything it reports, and
it only measures what the given configuration compiles.  That last
one is not academic - the same roots measure 73 KiB with a host
config.h and 20 KiB with a console-shaped feature set, and neither
figure describes a platform anybody built.  A check reporting numbers
that need a real target configuration to mean anything earns an
allowlist and then gets ignored.

Run against a psp1-shaped feature set, two task handlers exceed
GEKKO's 8 KiB by a wide margin, both through image decode:
task_overlay_handler at 20008 bytes and task_file_load_handler at
18568, each ending in rwebp_process_image -> alph_decode.  GEKKO sets
HAVE_THREADS, so those handlers do run on the 8 KiB worker there.
Not acted on here: confirming it needs a real Wii or GameCube build
configuration and a look at whether those chains are feasible, which
is a different piece of work from this branch.

Verified: full build and link clean; stack_budget and vfs parity
checks pass; both oracles pass under ASan/UBSan/LeakSan, on both MMAP
lanes for the filestream one; C89 -pedantic
-Wdeclaration-after-statement clean on the three touched TUs;
stack_chain.py exercised against a populated call graph and against
both its error paths.

* rwebp: move the 16 KiB Huffman scratch off the stack

vh_build() held int sorted[4096] as a local - 16 KiB in one frame,
twice the 8 KiB a GEKKO thread gets (STACKSIZE in
rthreads/gx_pthread.h).  It is reachable from two task handlers,
which run on a worker rather than the main thread whenever the task
queue is threaded:

  task_file_load_handler -> task_image_load_handler ->
    task_image_process -> image_transfer_process ->
    rwebp_process_image -> ... -> vh_build
  task_overlay_handler -> ... -> image_texture_load_internal ->
    image_transfer_process -> rwebp_process_image -> ... -> vh_build

Wii and GameCube set HAVE_RWEBP and HAVE_THREADS both, so a WebP
thumbnail or overlay decoded on the task thread overflows the stack
there.  BIG_STACK does not help: it enlarges the initial stack
through bootstrap/gx/rvl.ld, while spawned threads still come from
OSCreateThread with STACKSIZE.

The array is heap now, and uint16_t rather than int - every value in
it is a symbol index, bounded by the ns > 4096 rejection at the top,
and every read of it was already masked with 0xFFFF.  The early
returns become goto fail so the allocation is released on every exit.

Measured with the real wii feature set, taken from
'make -f Makefile.griffin platform=wii -n' rather than guessed at:

  vh_build frame          16784 -> 400 bytes
  task_overlay_handler    19976 -> 7064 bytes worst-case chain
  task_file_load_handler  18568 -> 5656 bytes worst-case chain

Every task handler chain is now under 8 KiB, but task_overlay_handler
at 7064 is 86% of it and that is not a margin.  The next thing added
anywhere under image decode puts it over, and nothing in CI will say
so - tools/stack_chain.py is a diagnostic, deliberately not a gate,
for the reasons in its docstring.

Found with that tool.  It is exactly the case tools/stack_budget.py
cannot see: vh_build is in libretro-common, so the per-frame check
covers it, but at 16784 it sat under the 'recorded rather than fixed'
list of codec frames that were already over budget when that check
was written - large working sets, call graphs not traced to a task
thread.  This one traces.

Verified: full build and link clean; the image_decode_fuzz WebP lane
decodes identically before and after (259 attempts, 69 decoded) under
ASan/UBSan/LeakSan and TSan; image_decode_slice_test passes;
stack_budget and vfs parity checks pass; C89 -pedantic
-Wdeclaration-after-statement clean.

* stack_budget: drop the stale vh_build entry, and test the allowlist's premise

Two problems with the allowlist, one urgent.

vh_build was recorded at 16784 bytes. The previous commit moved its
scratch array to the heap and the frame is 400 now, so the entry was
silently licensing a forty-twofold regression back to where it
started. Removed - the check catches a return to a 4096-entry local
again, verified by putting one back, which reports vh_build at 8592
and exits non-zero.

The rest of the list was exempted on an assumption written down as
one: that those codec and compression frames do not reach a task
thread on the small-stack targets, and that several are not built
there at all. There is now a tool for testing that, so it has been
tested - against the feature sets those targets really use, taken
from 'make -f Makefile.griffin platform=wii -n' and Makefile.ctr
rather than guessed at:

  wii (8 KiB threads, GEKKO)   2 of 37 entries reachable from a task
                               handler: config_file_dump and vh_build.
                               The other 35 are not compiled there.
  ctr (32 KiB threads)         3 of 37: the same two plus
                               mem_stats_proc_meminfo.

So the premise holds for the codec entries on those two targets, and
the comment now says that with its basis rather than asserting it.
It is still not a general claim, and the text says so too: indirect
calls are invisible to the call-graph walk, and a target built with a
different feature set can reach further than either of these.

Worth noting alongside it: config_file_dump is reachable on both, at
4208 bytes, inside the input_autoconfigure_connect_handler chain that
totals 5800 of GEKKO's 8192. Under budget, but 71% of the stack on a
path that a gamepad being plugged in takes.

Verified: full build and link clean; the ratchet's failure path
exercised in both directions; vfs parity check passes; the WebP fuzz
lane still decodes identically (259 attempts, 69 decoded) under
ASan/UBSan/LeakSan.

* stack: fix oversized frames on principle, not on current reachability

The allowlist exempted frames on the reasoning that nothing reaches
them from a task thread, or that the codec is not built on a target
with a small stack.  Both are statements about today, and both have
already been wrong once: config_file_write() was exempt on exactly
that basis until its caller turned out to be a task handler.
Platforms gain features and call graphs gain edges, so the question
for a large frame is not "can this overflow today" but "does this
need to be on the stack at all".

Two more that did not.

chd_read_header_core_file() held a whole chd_file as a local - the
struct embeds the working state of every codec it can use, huff,
zlib, cdzl and lzma - and zeroed it in full, to read a header that
touches one member.  57504 bytes -> 32.  No small-stack target builds
CHD today, which is exactly the reasoning being retired here.

rd_gen_lengths() held the Huffman length generator's scratch as
locals: 2*288 nodes of weight, left, right and depth for the
two-queue tree, plus three 288-entry sort arrays.  12768 -> 352, by
moving them into struct rdeflate, which is calloc'd once per stream,
so they cost nothing extra to keep and are no longer re-established
per call.

Both entries are dropped from the allowlist, along with the stale
one this pass also turned up.

The check itself was under-measuring.  Its define set was narrower
than any real build, and HAVE_7ZIP is the case that proved it:
without it, libchdr_chd.c's largest frame is 8320 bytes; with it,
57504.  A seven-fold difference in shipped code, under a define that
wii, ctr and every desktop build set - so the check was measuring
code no target compiles and missing code every target does.  The
define set now includes 7ZIP, CHD and the image decoders.

The allowlist comment states the rule, and records what is left as
debt rather than as exemption: mostly codec inner loops whose working
sets are large by nature, which is not the same as
necessarily-on-the-stack - most could hold their scratch in the
decoder state, which is allocated once per stream.  They are recorded
because that is per-codec work with real regression risk, not because
they are safe.

Verified: full build and link clean; 12 sample tests covering
deflate, zip, zstd, PNG round-trip and decode corpus, CHD, rzip, cdfs
and 7z all pass under ASan/UBSan/LeakSan, plus the image decode fuzz
lane; stack_budget and vfs parity checks pass; the C89 findings in
both touched TUs are pre-existing and unchanged in count.

* rzstd: move the FSE compression tables into the encoder's scratch

rzstd_emit_block() held its three FSE compression tables as locals.
Each carries a 1<<accuracy_log entry table plus two 256-entry delta
arrays, so three of them are most of a 9440-byte frame - and
rzstd_encode() already owns heap scratch for the match finder, so
they belong beside it, allocated once per call rather than once per
block.

The frame is gone entirely: with the tables out of it the function is
small enough that GCC inlines it, and the largest frame left in the
translation unit is 2368 bytes.

This is the codec debt from the previous commit, taken in the order
it can be proved rather than the order of size. The vp9 entries are
larger - rvp9_build_inter_pred_hbd is 51456 - but nothing in the tree
decodes VP9 video in a test, and changing a decoder inner loop with
no way to check the output is how a silent corruption ships. zstd
has a round trip, so zstd goes first.

Proved rather than smoke-tested: the same 40 inputs - five content
shapes (ramp, random, runs, text, mixed) across eight lengths from 0
to 300000 bytes - compressed with the encoder before and after, and
the compressed bytes are byte-identical in every case, digest for
digest. Every case also decodes back to its source. That is a
stronger statement than "the tests still pass": the encoder's output
did not change at all.

Worth recording because it nearly went unnoticed: archive_zstd_test
does pass, but it covers content_size header guards and never
compresses anything, so it would not have caught an encoder
regression. Picking an entry "because it has a test" is not the same
as the test covering the thing being changed.

Verified: compressed output byte-identical across all 40 cases;
round trip correct for each; harness clean under ASan/UBSan/LeakSan;
full build and link clean; archive_zstd_test and both rzip tests
pass; C89 -pedantic -Wdeclaration-after-statement clean, unchanged
from before; stack_budget passes with the entry removed.

* lrc_hash: hash through the mapping, and get the block buffer off the stack

sha1_calculate() read the file into unsigned char buff[4096], a
local, which made it a 4304-byte frame - over half of the 8 KiB a
GEKKO thread gets (STACKSIZE in rthreads/gx_pthread.h) - to hash a
file.

It now asks for RETRO_VFS_FILE_ACCESS_HINT_FREQUENT_ACCESS and takes
filestream_get_mapped_ptr(): where the platform maps the file the
whole thing is already addressable and there is nothing to copy, so
it is one SHA1Input() call over the mapping. Where there is no
mapping - NULL is a normal answer - it falls back to the same block
loop with the buffer on the heap. Hashing a whole file dwarfs one
allocation either way. 4304 -> 224 bytes.

This is the second consumer of that accessor, after task_cloudsync's
MD5, and the same shape: mapped fast path, copying path kept as the
general case.

Proved rather than smoke-tested. Both lanes were built before and
after and their output diffed: 25 cases, three content shapes across
eight lengths chosen to straddle the 4096 block - empty, sub-block,
exact multiples, and off-by-one either side - plus a missing file
that must still be rejected.

  HAVE_MMAP on   digests identical before and after
  HAVE_MMAP off  digests identical before and after
  the two lanes also agree with each other

That last line is the one worth having. The mapped path is new code
that the default build takes every time, so the copying path it
replaces would otherwise stop being exercised - and "the tests pass"
would mean the tests pass on one of the two paths.

Verified: both lanes byte-identical to the previous implementation
and to each other; both clean under ASan/UBSan/LeakSan; the existing
libcheck hash suite passes (3 checks); full build and link clean;
C89 -pedantic -Wdeclaration-after-statement unchanged at 26
pre-existing findings; stack_budget passes with the entry removed.

* file_stream: move filestream_vscanf's scan window to the heap

char buf[4096] as a local made this a 4368-byte frame, over half of
the 8 KiB a GEKKO thread gets (STACKSIZE in rthreads/gx_pthread.h).
4368 -> 272.

Shrinking the buffer would have been the cheaper change and the wrong
one: the window is how far a single conversion may reach, so a
smaller one fails differently on long input rather than merely more
slowly. The harness below pins that - with 4094 bytes of padding
before a number, the value is truncated at the window edge and only
its first digit is read, and at 4200 the conversion fails outright.
Both behaviours are preserved exactly; either would have changed
silently under a smaller buffer.

The size is now FILESTREAM_SCANF_WINDOW in the header, so a test can
drive input either side of it rather than assume a value - the same
reason the compare window below it is named.

rfscanf() has no in-tree callers; it is an fscanf() shim for cores,
which may well call it in a parse loop, so one allocation per call
rather than a per-call malloc of something larger. Beside the read
this function already performs, it is nothing.

Verified by diffing behaviour before and after across conversion
kinds and input shapes: integers, negatives, strings, literal
prefixes, field widths, hex, float, partial matches, empty input,
three successive calls on one stream, and padding either side of the
window. Return codes, parsed values and - the part that matters -
the resulting stream position are identical in every case. The
position is the thing a bug would hide in: this function seeks back
to just past what it consumed, so an off-by-one there is invisible
in the parsed output and breaks the next call.

Caught during that: the first version of this leaked the buffer on
every call. The final block has a filestream_seek() between va_end()
and return, so the pattern the free was added to did not match, and
it landed only on the early-out path. LeakSanitizer reported 9
allocations for 9 calls. The free now sits after the seek, which
needs bufiter - buf and so has to run first.

Verified: behaviour identical to the previous implementation across
every case; clean under ASan/UBSan/LeakSan; full build and link
clean; C89 -pedantic -Wdeclaration-after-statement clean, unchanged;
stack_budget, vfs parity, both MMAP lanes of vfs_mapped_ptr_test and
data_transfer_pool_scrub_test all pass.

---------

Co-authored-by: libretroadmin <>
2026-08-03 04:50:10 +02:00