gfx_thumbnail: do not hand a windowed container to the preview mixer

gfx_thumbnail_preview_audio_start copies container_size bytes out of
anim_buf to give the mixer bytes that outlive the thumbnail.  On a
windowed handle anim_buf is a reservation and anim_buf_len is the
file's full length, so that copy reads the whole file out of a mapping
where only the resident head plus the sliding window is committed: it
walks off the head into unmapped pages and faults.

  Thread 1 received signal SIGSEGV
  #0  msvcrt!memmove
  #1  gfx_thumbnail_anim_audio_begin
  #2  gfx_thumbnail_anim_install
  #3  gfx_thumbnail_anim_open
  #4  gfx_thumbnail_request
  rbx/r12 = 0x1d1f0acbd (7817178301) - the file length, verbatim

The malloc succeeds first, which is what makes this a fault rather
than a graceful bail: a multi-gigabyte commit charge is satisfiable
from the pagefile, so the allocator hands back a destination and the
read side is what dies.  Reproduced identically under ASan on Linux
with overcommit enabled - SEGV in memcpy at the first uncommitted page
past the head.

anim_install calls audio_begin whenever anim_read_pending is clear,
and a window never sets it (a window has no completion to wait for),
so every windowed video reached this.  It was unreachable before the
previous commits only because gfx_thumbnail_request_file - then the
sole windowed entry point - is the savestate path and never sees
anything but .png, whose type the audio gate rejects.

Skip the hand-off for windowed handles.  Streaming the mixer from a
window needs the feeder the video decoder already has, driven from the
audio thread, which is the cross-thread change anim_install documents
as out of scope; until then a windowed preview is silent rather than
fatal.  The fully-resident path is untouched and keeps its audio.

Sweeps, on a 7.28 GB 3840x2160 H.264 + AAC file with a trailing moov
past the 4 GiB boundary: ASan+UBSan+LSan and standalone LSan and UBSan
(with pointer-overflow and signed-integer-overflow) clean over open,
30 frames of decode, and close; TSan clean over a worker/feeder pair
modelling the decode-vs-commit race.
This commit is contained in:
LibretroAdmin authored and libretroadmin committed 2026-08-14 03:21:57 +02:00
1 parent 3389c38a32
commit 9650f045ce
1 file changed
+19
+19
View File
@@ -738,6 +738,25 @@ static void gfx_thumbnail_anim_close(gfx_thumbnail_t *thumbnail)
static void gfx_thumbnail_anim_audio_begin(gfx_thumbnail_t *thumbnail)
{
enum image_type_enum type = (enum image_type_enum)thumbnail->anim_type;
/* A windowed handle has no container to hand over. anim_buf_len is
* the file's full length, but only the permanently resident head
* plus the sliding window is committed - the rest is reserved
* address space, and reading it faults. The mixer hand-off below
* copies container_size bytes out of anim_buf, so on a window it
* walks off the head into unmapped pages: SIGSEGV inside memcpy,
* with a copy destination the allocator was happy to provide
* because a multi-gigabyte commit charge is satisfiable from the
* pagefile.
*
* Streaming the mixer a windowed source needs the same feeder the
* video decoder gets, driven from the audio thread - the
* cross-thread change anim_install already notes as out of scope.
* Until then a windowed preview is silent rather than fatal; the
* fully-resident path is unaffected and keeps its audio. */
if (thumbnail->anim_windowed)
return;
if ( ( (type == IMAGE_TYPE_WEBM)
|| (type == IMAGE_TYPE_MP4))
&& config_get_ptr()->bools.menu_thumbnail_preview_audio)