Optimization pass: no polling sleeps or locks in the hot handoffs

The recompiler thread looked for work every 10 ms, so a function the
interpreter had just run waited 5 ms on average before its translation
began, and every queued address could allocate inside std::queue. Its
queue is now a fixed retro_spsc ring with an eventcount
(PPCRecompilerWorkQueue): a push wakes the thread at once, the thread pops
without the recompiler's spinlock, and a full queue refuses the push so
the address stays unvisited and is offered again later.

Loading the Vulkan pipeline cache napped 10 ms whenever 50 pipelines were
queued and again while the last ones compiled, which held a warm driver
cache to 50 pipelines per 10 ms. The GPU thread now sleeps on an
eventcount (LatteGpuWake) that the compile threads notify after each
pipeline and that a pause request or a stop also notify, so it refills
the queue as soon as there is room and still answers a context loss at
once. The pipeline cache's save thread keeps its 15 s debounce but sleeps
on an eventcount that the renderer's destruction notifies, instead of
looking every 250 ms.

The CPU timer's TSC calibration measured for three seconds and the title
start waited for it, which in RetroArch is every boot. Both clocks are
read back to back, so 25 ms already gives about one part in 10^5: the
first estimate is published then and refined at 200 ms and 3 s by the
same measurement. The rate only divides elapsed ticks, so a refinement
changes no time already counted. The title start waits on an eventcount
instead of a 10 ms poll.

Audio goes to the frontend straight out of the ring buffer (Peek/Consume)
and AX's blocks are converted straight into it (Reserve/Commit), so the
per-frame copy into a 192 KiB flush buffer, the pull buffer and the
conversion vector are gone. The ring moves to LibretroAudioRing.h.

The frame gate and the frame handoff between the GPU thread and retro_run
are atomics and eventcounts in place of two mutexes and condition
variables, with the same predicates and the same timeout.

retro_spsc is synced from RetroArch's libretro-common. test/handoff checks
the work queue (every address once and in order from three serialised
pushers, wake-up from a parked consumer, the bound, the stop), the GPU
wake, and the audio ring (whole-frame spans across the wrap for 2 and 6
channels, a producer and a consumer thread streaming), plain, under TSan
and under ASan/UBSan, in both CIs.

.github/workflows/libretro.yml

@@ -91,6 +91,16 @@ jobs:
           make -C test/gpu_cothread clean
           SANITIZER=address,undefined ./test/gpu_cothread/run_tests.sh
 
+      - name: "Thread handoffs"
+        # the recompiler's work queue, the GPU thread's wake, the audio
+        # ring: plain, under ThreadSanitizer, under ASan + UBSan
+        run: |
+          ./test/handoff/run_tests.sh
+          make -C test/handoff clean
+          SANITIZER=thread ./test/handoff/run_tests.sh
+          make -C test/handoff clean
+          SANITIZER=address,undefined ./test/handoff/run_tests.sh
+
   macos:
     if: github.event_name != 'workflow_dispatch' || inputs.platforms == 'all' || contains(inputs.platforms, 'macos')
     name: macos-${{ matrix.arch }}

.gitlab-ci.yml

@@ -187,3 +187,20 @@ test-gpu-cothread:
     - ./test/gpu_cothread/run_tests.sh
     - make -C test/gpu_cothread clean
     - SANITIZER=address,undefined ./test/gpu_cothread/run_tests.sh
+
+# The lock-free handoffs between the core's threads: the recompiler's work
+# queue, the GPU thread's wake, the audio ring.
+test-handoff:
+  stage: test
+  image: $CI_SERVER_HOST:5050/libretro-infrastructure/libretro-build-amd64-ubuntu:backports
+  needs: []
+  variables:
+    GIT_SUBMODULE_STRATEGY: none
+    CC: gcc-12
+    CXX: g++-12
+  script:
+    - ./test/handoff/run_tests.sh
+    - make -C test/handoff clean
+    - SANITIZER=thread ./test/handoff/run_tests.sh
+    - make -C test/handoff clean
+    - SANITIZER=address,undefined ./test/handoff/run_tests.sh

CHANGELOG.md

@@ -4,6 +4,11 @@ What changed in the libretro core of this fork, newest first. Changes that come
 
 ## Unreleased
 
+- Boot no longer waits up to three seconds for the CPU timer's calibration: a first estimate is ready 25 ms after the core loads and is refined in the background.
+- The recompiler starts on a newly run function at once instead of at its next 10 ms look, so hot code spends less time interpreted, and its work queue no longer allocates.
+- Vulkan: loading the pipeline cache no longer pauses 10 ms whenever 50 pipelines are queued, which held a warm cache to 5,000 pipelines a second; closing content no longer waits up to 250 ms for the pipeline cache's save thread.
+- Audio goes to RetroArch straight out of the ring buffer, and AX's channels are converted straight into it: no copies on the way, and no 192 KiB flush buffer.
+- The frame gate and the frame handoff between the GPU thread and `retro_run` take no locks.
 - OpenGL goes through the frontend alone: the core no longer makes GL contexts of its own (WGL on Windows, GLX on X11, EGL on Wayland), links no GL, EGL or X11 library, and loads every GL function from RetroArch. Cemu's GPU thread runs as a cothread on RetroArch's thread, where its context is current, and the finished frame is blitted into RetroArch's framebuffer instead of read back into memory and uploaded again each frame. Wayland no longer has an OpenGL path of its own.
 - Android: the fibers the emulated cores run on keep libco's state per thread (`LIBCO_MP`); without it a switch on one core could save its registers into another core's fiber.
 - The core is now ParaLLEl GX2, `parallel_gx2_libretro`. RetroArch keeps core options, remaps and overrides under the core's name, so they start from the defaults: to keep them, copy RetroArch's `config/Cemu/` to `config/ParaLLEl GX2/` and rename the files in it from `Cemu.*` to `ParaLLEl GX2.*`. The core's own files stay in `<system>/Cemu/` and `<save>/Cemu/`.

Makefile.common

@@ -116,6 +116,7 @@ CORE_SOURCES_CXX += \
 	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/IML/IMLRegisterAllocatorRanges.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/IML/IMLSegment.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/PPCRecompiler.cpp \
+	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/PPCRecompilerWorkQueue.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/PPCRecompilerImlGen.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/PPCRecompilerImlGenFPU.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Espresso/Recompiler/PPCRecompilerIntermediate.cpp \
@@ -128,6 +129,7 @@ CORE_SOURCES_CXX += \
 	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LatteCommandProcessor.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LatteGSCopyShaderParser.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LatteGpuCothread.cpp \
+	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LatteGpuWake.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LatteIndices.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LatteOverlay.cpp \
 	$(CORE_DIR)/src/Cafe/HW/Latte/Core/LattePerformanceMonitor.cpp \
@@ -585,6 +587,7 @@ LRC_SOURCES_C := \
 	$(LRC_DIR)/net/net_http.c \
 	$(LRC_DIR)/net/net_socket.c \
 	$(LRC_DIR)/net/net_socket_ssl_retro.c \
+	$(LRC_DIR)/queues/retro_spsc.c \
 	$(LRC_DIR)/rthreads/retro_eventcount.c \
 	$(LRC_DIR)/rthreads/rthreads.c \
 	$(LRC_DIR)/streams/file_stream.c \

dependencies/libretro-common/include/retro_spsc.h

@@ -0,0 +1,462 @@
+/* Copyright  (C) 2010-2026 The RetroArch team
+ *
+ * ---------------------------------------------------------------------------------------
+ * The following license statement only applies to this file (retro_spsc.h).
+ * ---------------------------------------------------------------------------------------
+ *
+ * Permission is hereby granted, free of charge,
+ * to any person obtaining a copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation the rights to
+ * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+ * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+ * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __LIBRETRO_SDK_SPSC_H
+#define __LIBRETRO_SDK_SPSC_H
+
+/*
+ * retro_spsc.h - portable single-producer / single-consumer byte queue
+ *
+ * Lock-free byte-stream queue with one writer thread and one reader
+ * thread.  Uses release-store / acquire-load on two cursors (the
+ * Lamport / Disruptor design) rather than a single shared count, so
+ * neither thread issues atomic RMW operations on the hot path.
+ *
+ * Constraints:
+ *   - Exactly ONE producer and ONE consumer.  No locking is performed;
+ *     two producers or two consumers will corrupt the queue.
+ *   - Capacity must be power-of-2.  Init rounds up.
+ *   - Maximum capacity is SIZE_MAX/2 (so head - tail never overflows
+ *     the well-defined unsigned modular range).
+ *   - The buffer is byte-addressed.  Callers pushing structured records
+ *     are responsible for framing.
+ *
+ * Memory model:
+ *   - Producer writes data into the buffer (non-atomic stores), then
+ *     publishes the new head via release-store.
+ *   - Consumer acquire-loads head (sees data writes that preceded the
+ *     release), reads data (non-atomic loads), then publishes the new
+ *     tail via release-store.
+ *   - Producer acquire-loads tail to know how much space is free.
+ *   - This pairing gives the SPSC invariant on every backend
+ *     retro_atomic.h supports lock-freely.  On the volatile fallback
+ *     (no real backend), correctness reduces to single-core or x86 TSO,
+ *     same as every other retro_atomic.h caller.
+ *
+ * Cache behaviour:
+ *   - head and tail are placed on separate cache lines via explicit
+ *     padding to RETRO_SPSC_CACHE_LINE.  Without this, the producer's
+ *     publish would invalidate the consumer's tail line and vice versa,
+ *     halving throughput on contended SMP.  The padding is a
+ *     performance hint; correctness does not depend on it.
+ *   - The padding runs past tail as well, so the isolation survives
+ *     embedding retro_spsc_t by value in a larger struct.  Without a
+ *     trailing pad, whatever the embedder declares next lands within
+ *     tail's line and the consumer's publish starts invalidating it -
+ *     which defeats the point for exactly the callers this primitive
+ *     exists for.
+ *   - The struct carries no alignment of its own: an embedder on the
+ *     heap gets whatever malloc gives, so the cursors can start at any
+ *     offset within a line.  The pads are therefore sized so that each
+ *     cursor and its private copy sit a full line past the end of the
+ *     previous group, which keeps them on different lines wherever the
+ *     struct lands.
+ *
+ * Lifetime:
+ *   - retro_spsc_init allocates an internal buffer; retro_spsc_free
+ *     releases it.  The caller owns the retro_spsc_t struct itself.
+ *   - The struct is NOT itself thread-safe to construct or destroy
+ *     while in use.  Build it on one thread, hand the producer pointer
+ *     to one thread and the consumer pointer to another, then tear
+ *     down on one thread after both producer and consumer have stopped.
+ *
+ * Example:
+ *
+ *   retro_spsc_t q;
+ *   retro_spsc_init(&q, 4096);
+ *
+ *   // Producer thread:
+ *   uint8_t msg[64] = ...;
+ *   if (retro_spsc_write_avail(&q) >= sizeof(msg))
+ *      retro_spsc_write(&q, msg, sizeof(msg));
+ *
+ *   // Consumer thread:
+ *   uint8_t msg[64];
+ *   if (retro_spsc_read_avail(&q) >= sizeof(msg))
+ *      retro_spsc_read(&q, msg, sizeof(msg));
+ *
+ *   retro_spsc_free(&q);
+ *
+ * Comparison with libretro-common's existing fifo_queue_t:
+ *   - fifo_queue uses a slock_t internally; safe with multiple producers
+ *     and multiple consumers, but every push/pop takes a mutex.
+ *   - retro_spsc is lock-free but limited to one producer and one
+ *     consumer.  Use this when (a) the producer/consumer count is fixed
+ *     at one each (audio driver -> backend, video -> task system, etc.)
+ *     and (b) the lock contention in fifo_queue measurably matters.
+ *   - For most code paths, fifo_queue is the better default.  This
+ *     primitive is for hot paths where lock-free is a measured win.
+ */
+
+#include <stddef.h>
+#include <stdint.h>
+#include <boolean.h>
+
+#include <retro_common_api.h>
+#include <retro_atomic.h>
+
+RETRO_BEGIN_DECLS
+
+/* Cache-line padding size.  64 covers x86-64, AArch64, most modern
+ * ARMs, and modern PowerPC.  Apple Silicon's effective coherency line
+ * is 128 due to its M-series cluster topology; over-padding to 64
+ * still avoids false sharing, just slightly less efficiently.  Older
+ * 32-bit ARMs (ARMv6, Cortex-A8) used 32-byte lines but tolerate the
+ * larger pad without correctness impact. */
+#ifndef RETRO_SPSC_CACHE_LINE
+#define RETRO_SPSC_CACHE_LINE 64
+#endif
+
+/* Padding helper.  C89 has no _Static_assert; the array-size trick is
+ * portable.  We pad to the next multiple of cache-line size after the
+ * pointer-and-size_t prefix and after each cursor. */
+
+/* Effective cache line for padding.  We pad each cursor to a full
+ * cache line.  If the pre-cursor fields exceed RETRO_SPSC_CACHE_LINE
+ * on some hypothetical small-cache target, the underflow on the
+ * subtraction would produce a giant array; guard with a max() so
+ * the pad is always at least 1 byte and never underflows. */
+#define RETRO_SPSC_PAD0_BYTES \
+   ((RETRO_SPSC_CACHE_LINE > (sizeof(uint8_t*) + 3 * sizeof(size_t))) \
+      ? (RETRO_SPSC_CACHE_LINE - (sizeof(uint8_t*) + 3 * sizeof(size_t))) \
+      : 1)
+/* Each cursor shares its line with that side's private copy of the
+ * other cursor (see cached_tail / cached_head below).  A full line of
+ * pad after the pair puts the next group a line past the pair's last
+ * byte, so the two cannot share a line at any placement of the struct;
+ * a line less the pair would leave them sharing one whenever the
+ * struct starts late enough in a line. */
+#define RETRO_SPSC_PAD1_BYTES RETRO_SPSC_CACHE_LINE
+
+typedef struct retro_spsc
+{
+   uint8_t            *buffer;
+   size_t              capacity;   /* power of 2: the most the ring holds */
+   /* The buffer's size less one, also a power of 2: positions wrap
+    * here. Equal to capacity - 1 unless the buffer is mirrored. */
+   size_t              mask;
+   /* The buffer's size when its pages are mapped a second time right
+    * behind it, else 0: a span then runs past the end into the
+    * mirror, and no reservation is cut short at the wrap. */
+   size_t              mirror;
+   /* Pad so head sits on a fresh cache line, isolating it from
+    * the buffer/capacity fields that init may touch. */
+   uint8_t             _pad0[RETRO_SPSC_PAD0_BYTES];
+   retro_atomic_size_t head;       /* producer publishes; consumer reads */
+   /* Producer-private copy of tail.  The producer computes room from
+    * this and only re-reads the real tail - the consumer's cache line
+    * - when the copy says there is not enough, i.e. once per fill
+    * rather than once per write.  Always <= tail, so room computed
+    * from it is never overstated.  Plain size_t: only the producer
+    * ever touches it. */
+   size_t              cached_tail;
+   /* Pad so tail sits on its own cache line, isolating it from head. */
+   uint8_t             _pad1[RETRO_SPSC_PAD1_BYTES];
+   retro_atomic_size_t tail;       /* consumer publishes; producer reads */
+   /* Consumer-private copy of head, the mirror of cached_tail: always
+    * <= head, re-read only when it says the ring is short. */
+   size_t              cached_head;
+   /* Pad past tail so an embedding struct's own fields cannot land on
+    * tail's line.  Same width as _pad1. */
+   uint8_t             _pad2[RETRO_SPSC_PAD1_BYTES];
+} retro_spsc_t;
+
+/**
+ * retro_spsc_init:
+ * @q             : The queue.
+ * @min_capacity  : Requested capacity in bytes.  Rounded up to the
+ *                  next power of 2.  Must be > 0 and <= SIZE_MAX/2.
+ *
+ * Allocates the internal buffer and zero-initialises both cursors.
+ * Both cursors begin at 0; the queue is empty.
+ *
+ * Returns: true on success, false on allocation failure or invalid
+ * @min_capacity.
+ *
+ * After this returns true, the producer thread can call
+ * retro_spsc_write / retro_spsc_write_avail and the consumer thread can
+ * call retro_spsc_read / retro_spsc_read_avail.
+ */
+bool retro_spsc_init(retro_spsc_t *q, size_t min_capacity);
+
+/**
+ * retro_spsc_init_mirrored:
+ * @q             : The queue.
+ * @min_capacity  : As retro_spsc_init; capacity is the same.
+ *
+ * As retro_spsc_init, but the buffer's pages are mapped twice, back
+ * to back, where the platform allows it, so write_begin and read_begin
+ * return every free or filled byte contiguously, across the wrap. The
+ * buffer is at least a page (an allocation granule on Windows); a ring
+ * smaller than that still holds only @min_capacity rounded up, so
+ * capacity, room and fill read exactly as they would from
+ * retro_spsc_init. Where mirroring is unavailable the queue is a plain
+ * one and q->mirror is 0.
+ */
+bool retro_spsc_init_mirrored(retro_spsc_t *q, size_t min_capacity);
+
+/**
+ * retro_spsc_free:
+ * @q : The queue.
+ *
+ * Releases the internal buffer.  Caller must ensure both producer
+ * and consumer have stopped using @q before calling.  Safe to call
+ * on a queue that retro_spsc_init failed on (no-op).
+ */
+void retro_spsc_free(retro_spsc_t *q);
+
+/**
+ * retro_spsc_clear:
+ * @q : The queue.
+ *
+ * Resets head and tail to 0, discarding any unread data.  The
+ * underlying buffer is preserved (no reallocation).
+ *
+ * SAFETY: callable only when both the producer and consumer are
+ * quiesced -- e.g. before either has started, or after both have
+ * stopped.  Concurrent calls with a live producer or consumer are
+ * a data race.  This is the same lifetime constraint as
+ * retro_spsc_init / retro_spsc_free.
+ *
+ * Typical use is when a stream is stopped and restarted (e.g. an
+ * audio driver pausing and resuming, or switching device formats),
+ * and stale buffered data should be discarded before the new
+ * stream begins.
+ */
+void retro_spsc_clear(retro_spsc_t *q);
+
+/**
+ * retro_spsc_grow:
+ * @q             : The queue.
+ * @min_capacity  : Requested capacity in bytes, rounded up as for
+ *                  retro_spsc_init.
+ *
+ * Moves the queue to a larger buffer of the same kind, mirrored or
+ * not.  The unread data and both cursors carry over unchanged, so a
+ * position the caller recorded in the ring still names the same byte.
+ * A queue already holding @min_capacity is left as it is.
+ *
+ * SAFETY: as retro_spsc_clear, both the producer and the consumer must
+ * be quiesced, and any span either holds is invalid afterwards.
+ *
+ * Returns: true on success; false on allocation failure, with the
+ * queue untouched.
+ */
+bool retro_spsc_grow(retro_spsc_t *q, size_t min_capacity);
+
+/**
+ * retro_spsc_grow_into:
+ * @q : The queue.
+ * @n : A queue from retro_spsc_init or retro_spsc_init_mirrored, empty,
+ *      at least as large as @q.
+ *
+ * retro_spsc_grow with the buffer allocated by the caller, so the move
+ * itself allocates and frees nothing: @q takes @n's buffer with its
+ * unread data and cursors, and @n is left holding @q's old buffer for
+ * the caller to retro_spsc_free.  The allocation and the free can then
+ * be on a thread other than the one that moves.
+ *
+ * SAFETY: as retro_spsc_grow.
+ *
+ * Returns: false, with neither queue touched, if @n is the smaller.
+ */
+bool retro_spsc_grow_into(retro_spsc_t *q, retro_spsc_t *n);
+
+/**
+ * retro_spsc_write_avail:
+ * @q : The queue.
+ *
+ * Producer-side query: how many bytes can be written before the
+ * queue is full.
+ *
+ * Returns: byte count, in [0, capacity].
+ *
+ * SAFETY: callable only from the producer thread.
+ */
+size_t retro_spsc_write_avail(const retro_spsc_t *q);
+
+/**
+ * retro_spsc_read_avail:
+ * @q : The queue.
+ *
+ * Consumer-side query: how many bytes are available to read.
+ *
+ * Returns: byte count, in [0, capacity].
+ *
+ * SAFETY: callable only from the consumer thread.
+ */
+size_t retro_spsc_read_avail(const retro_spsc_t *q);
+
+/**
+ * retro_spsc_write:
+ * @q     : The queue.
+ * @data  : Source bytes.
+ * @bytes : Number of bytes to attempt to write.
+ *
+ * Writes up to @bytes from @data into the queue.  If the queue has
+ * less than @bytes free, writes only what fits.
+ *
+ * Returns: number of bytes actually written.
+ *
+ * SAFETY: callable only from the producer thread.  Concurrent calls
+ * with another producer corrupt the queue.
+ */
+size_t retro_spsc_write(retro_spsc_t *q, const void *data, size_t bytes);
+
+/* Producer-only whole-frame write; returns frames, not bytes. Free bytes
+ * shorter than one frame remain unused. The byte queue capacity need not be
+ * divisible by frame_bytes, and frames may wrap across its physical end.
+ * Zero frames or frame_bytes writes nothing. As with write(), q must be
+ * initialized and data must cover the frames that can be accepted.
+ * Consumers must read/skip whole frames to preserve stream boundaries. */
+size_t retro_spsc_write_frames(retro_spsc_t *q, const void *data,
+      size_t frames, size_t frame_bytes);
+
+/**
+ * retro_spsc_read:
+ * @q     : The queue.
+ * @data  : Destination bytes.
+ * @bytes : Number of bytes to attempt to read.
+ *
+ * Reads up to @bytes from the queue into @data.  If the queue has
+ * less than @bytes available, reads only what is present.
+ *
+ * Returns: number of bytes actually read.
+ *
+ * SAFETY: callable only from the consumer thread.  Concurrent calls
+ * with another consumer corrupt the queue.
+ */
+size_t retro_spsc_read(retro_spsc_t *q, void *data, size_t bytes);
+
+/**
+ * retro_spsc_peek:
+ * @q     : The queue.
+ * @data  : Destination bytes.
+ * @bytes : Number of bytes to peek.
+ *
+ * Like retro_spsc_read but does not advance the read cursor.  The
+ * peeked bytes remain available for the next read.
+ *
+ * Returns: number of bytes peeked.
+ *
+ * SAFETY: callable only from the consumer thread.
+ */
+size_t retro_spsc_peek(const retro_spsc_t *q, void *data, size_t bytes);
+
+/**
+ * retro_spsc_write_begin:
+ * @q   : The queue.
+ * @ptr : Receives a pointer to the contiguous writable region at the
+ *        current head.
+ *
+ * Producer-side zero-copy reservation.  Returns the number of bytes
+ * that can be written CONTIGUOUSLY at *@ptr - this is min(total free
+ * space, distance to the physical end of the ring; on a mirrored ring
+ * the total free space alone), so it can be
+ * smaller than retro_spsc_write_avail() when the head is near the
+ * wrap point.  No cursor moves; call retro_spsc_write_end() with the
+ * number of bytes actually written (which may be 0, and must not
+ * exceed the returned span).  A caller needing more than the span can
+ * fall back to retro_spsc_write(), which handles the split copy.
+ *
+ * The pointer stays valid until write_end; the consumer cannot enter
+ * this region because head has not moved.  Alignment of *@ptr is
+ * whatever the caller's own write granularity guarantees: the ring
+ * is byte-addressed, so producers that always write N-byte records
+ * keep every span N-aligned.
+ *
+ * Returns: contiguous writable byte count (0 when full or when the
+ * head sits exactly at the wrap point of a full lap).
+ *
+ * SAFETY: producer thread only.  At most one begin outstanding.
+ */
+size_t retro_spsc_write_begin(retro_spsc_t *q, void **ptr);
+
+/**
+ * retro_spsc_write_end:
+ * @q     : The queue.
+ * @bytes : Bytes actually written into the span from write_begin.
+ *
+ * Publishes @bytes with a release-store on head, making them visible
+ * to the consumer.  Passing 0 abandons the reservation.
+ *
+ * SAFETY: producer thread only; @bytes must not exceed the span
+ * returned by the matching retro_spsc_write_begin.
+ */
+void retro_spsc_write_end(retro_spsc_t *q, size_t bytes);
+
+/**
+ * retro_spsc_read_begin:
+ * @q   : The queue.
+ * @ptr : Receives a pointer to the contiguous readable region at the
+ *        current tail.
+ *
+ * Consumer-side zero-copy drain.  Returns the number of bytes
+ * readable CONTIGUOUSLY at *@ptr - min(total buffered, distance to
+ * the physical end of the ring; on a mirrored ring all of it).
+ * Otherwise wrapped data needs a second begin/end round after
+ * consuming the first span.  No cursor moves;
+ * call retro_spsc_read_end() with the bytes actually consumed.
+ *
+ * The region stays stable until read_end: the producer cannot write
+ * into it because tail has not moved.
+ *
+ * Returns: contiguous readable byte count (0 when empty).
+ *
+ * SAFETY: consumer thread only.  At most one begin outstanding.
+ */
+size_t retro_spsc_read_begin(retro_spsc_t *q, const void **ptr);
+
+/**
+ * retro_spsc_read_end:
+ * @q     : The queue.
+ * @bytes : Bytes actually consumed from the span from read_begin.
+ *
+ * Frees @bytes back to the producer with a release-store on tail.
+ * Passing 0 abandons the read.
+ *
+ * SAFETY: consumer thread only; @bytes must not exceed the span
+ * returned by the matching retro_spsc_read_begin.
+ */
+void retro_spsc_read_end(retro_spsc_t *q, size_t bytes);
+
+/**
+ * retro_spsc_skip:
+ * @q     : The queue.
+ * @bytes : Number of bytes to discard.
+ *
+ * Consumer-side discard: frees up to @bytes back to the producer
+ * without copying them anywhere, clamped to what is actually
+ * buffered.  This is what a consumer that has already inspected a
+ * record through retro_spsc_peek or retro_spsc_read_begin needs in
+ * order to commit the read later, without re-deriving a span it
+ * has already seen.
+ *
+ * Returns: number of bytes actually discarded.
+ *
+ * SAFETY: consumer thread only.
+ */
+size_t retro_spsc_skip(retro_spsc_t *q, size_t bytes);
+
+RETRO_END_DECLS
+
+#endif /* __LIBRETRO_SDK_SPSC_H */

dependencies/libretro-common/queues/retro_spsc.c

@@ -0,0 +1,639 @@
+/* Copyright  (C) 2010-2026 The RetroArch team
+ *
+ * ---------------------------------------------------------------------------------------
+ * The following license statement only applies to this file (retro_spsc.c).
+ * ---------------------------------------------------------------------------------------
+ *
+ * Permission is hereby granted, free of charge,
+ * to any person obtaining a copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation the rights to
+ * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+ * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+ * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/* memfd_create goes through syscall(), and ftruncate and shm_open
+ * are POSIX: strict C89 builds hide all three without these. */
+#if defined(__linux__) && !defined(_GNU_SOURCE)
+#define _GNU_SOURCE
+#endif
+/* Tiger must keep the legacy symbols rather than UNIX2003 variants. */
+#if defined(__APPLE__) && !defined(_DARWIN_C_SOURCE) \
+      && !(defined(__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__) \
+         && (__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 1050))
+#define _DARWIN_C_SOURCE
+#endif
+
+#include <stdlib.h>
+#include <string.h>
+#include <retro_inline.h>
+#include <retro_spsc.h>
+
+/* Where a buffer can be mapped twice, back to back. RETRO_SPSC_MIRROR_SHM
+ * selects the POSIX shm path on any POSIX host, for its tests. */
+#if defined(_WIN32) && !defined(_XBOX) && !defined(__WINRT__)
+#define SPSC_MIRROR_WIN32
+#elif defined(RETRO_SPSC_MIRROR_SHM)
+#define SPSC_MIRROR_SHM
+#elif defined(__linux__) && !defined(__EMSCRIPTEN__)
+#include <sys/syscall.h>
+#ifdef __NR_memfd_create
+#define SPSC_MIRROR_MEMFD
+#endif
+#elif defined(__APPLE__)
+#include <TargetConditionals.h>
+#if !TARGET_OS_IPHONE
+#define SPSC_MIRROR_SHM
+#endif
+#elif (defined(__FreeBSD__) && !defined(__ORBIS__)) || defined(__NetBSD__) \
+      || defined(__OpenBSD__)
+#define SPSC_MIRROR_SHM
+#endif
+
+#if defined(SPSC_MIRROR_WIN32)
+#include <windows.h>
+#elif defined(SPSC_MIRROR_MEMFD) || defined(SPSC_MIRROR_SHM)
+#include <unistd.h>
+#include <fcntl.h>
+#include <sys/mman.h>
+#endif
+
+#if defined(SPSC_MIRROR_WIN32) || defined(SPSC_MIRROR_MEMFD) \
+      || defined(SPSC_MIRROR_SHM)
+#define SPSC_MIRROR
+#endif
+
+#ifdef SPSC_MIRROR_SHM
+#include <stdio.h>
+#endif
+
+/* Bytes contiguous at @idx: to the buffer's end, or past it into the
+ * mirror. */
+static INLINE size_t spsc_contig(const retro_spsc_t *q, size_t idx)
+{
+   return q->mirror ? (size_t)-1 : q->mask + 1 - idx;
+}
+
+#ifdef SPSC_MIRROR
+#if !defined(SPSC_MIRROR_WIN32) && !defined(MAP_ANONYMOUS)
+#define MAP_ANONYMOUS MAP_ANON
+#endif
+
+/* What a mapping is made of: a page, or Windows' allocation granule. */
+static size_t spsc_mirror_granule(void)
+{
+#ifdef SPSC_MIRROR_WIN32
+   SYSTEM_INFO si;
+   GetSystemInfo(&si);
+   return (size_t)si.dwAllocationGranularity;
+#else
+   long page = sysconf(_SC_PAGESIZE);
+   return page > 0 ? (size_t)page : 4096;
+#endif
+}
+
+static void spsc_mirror_free(uint8_t *base, size_t size)
+{
+#ifdef SPSC_MIRROR_WIN32
+   UnmapViewOfFile(base);
+   UnmapViewOfFile(base + size);
+#else
+   munmap(base, size * 2);
+#endif
+}
+
+#ifndef SPSC_MIRROR_WIN32
+/* Both views of a descriptor's pages, inside one reservation so nothing
+ * can land between them. Takes the descriptor. */
+static uint8_t *spsc_mirror_map(int fd, size_t size)
+{
+   uint8_t *base = NULL;
+   void    *res;
+   if (ftruncate(fd, (off_t)size) == 0)
+   {
+      res = mmap(NULL, size * 2, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS,
+            -1, 0);
+      if (res != MAP_FAILED)
+      {
+         base = (uint8_t*)res;
+         if (     mmap(base, size, PROT_READ | PROT_WRITE,
+                     MAP_SHARED | MAP_FIXED, fd, 0) == MAP_FAILED
+               || mmap(base + size, size, PROT_READ | PROT_WRITE,
+                     MAP_SHARED | MAP_FIXED, fd, 0) == MAP_FAILED)
+         {
+            munmap(base, size * 2);
+            base = NULL;
+         }
+      }
+   }
+   close(fd);
+   return base;
+}
+#endif
+
+static uint8_t *spsc_mirror_alloc(size_t size)
+{
+   uint8_t *base = NULL;
+#if defined(SPSC_MIRROR_WIN32)
+   unsigned tries;
+   HANDLE   h = CreateFileMappingA(INVALID_HANDLE_VALUE, NULL,
+         PAGE_READWRITE, (DWORD)((uint64_t)size >> 32), (DWORD)size, NULL);
+   if (!h)
+      return NULL;
+   /* The reservation is released before the views are placed in it,
+    * so another thread can take the address meanwhile: try again. */
+   for (tries = 0; tries < 16 && !base; tries++)
+   {
+      uint8_t *at = (uint8_t*)VirtualAlloc(NULL, size * 2, MEM_RESERVE,
+            PAGE_NOACCESS);
+      void    *lo, *hi;
+      if (!at)
+         break;
+      VirtualFree(at, 0, MEM_RELEASE);
+      if (!(lo = MapViewOfFileEx(h, FILE_MAP_ALL_ACCESS, 0, 0, size, at)))
+         continue;
+      hi = MapViewOfFileEx(h, FILE_MAP_ALL_ACCESS, 0, 0, size, at + size);
+      if (hi == (void*)(at + size))
+         base = at;
+      else
+      {
+         if (hi)
+            UnmapViewOfFile(hi);
+         UnmapViewOfFile(lo);
+      }
+   }
+   CloseHandle(h);
+#elif defined(SPSC_MIRROR_MEMFD)
+   int fd = (int)syscall(__NR_memfd_create, "retro_spsc", 1u /* MFD_CLOEXEC */);
+   if (fd >= 0)
+      base = spsc_mirror_map(fd, size);
+#else
+   static unsigned serial;
+   unsigned tries;
+   for (tries = 0; tries < 8 && !base; tries++)
+   {
+      char name[48];
+      int  fd, length;
+      length = snprintf(name, sizeof(name), "/retro_spsc.%lu.%u",
+            (unsigned long)getpid(), serial++);
+      if (length < 0 || (size_t)length >= sizeof(name))
+         continue;
+      if ((fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL, 0600)) < 0)
+         continue;
+      shm_unlink(name);
+      base = spsc_mirror_map(fd, size);
+      break;
+   }
+#endif
+   /* A write through one view must read back through the other. */
+   if (base)
+   {
+      base[0] = 0x5a;
+      if (base[size] != 0x5a)
+      {
+         spsc_mirror_free(base, size);
+         return NULL;
+      }
+      base[0] = 0;
+   }
+   return base;
+}
+#endif
+
+/* Round @v up to the next power of 2.  Returns 0 if @v == 0 or if
+ * the next power of 2 would overflow (caller checks @v <= SIZE_MAX/2). */
+static size_t spsc_round_up_pow2(size_t v)
+{
+   size_t r;
+   if (v == 0)
+      return 0;
+   /* If already a power of 2, return as-is. */
+   if ((v & (v - 1)) == 0)
+      return v;
+   /* Round up.  Loop terminates because v <= SIZE_MAX/2 means r
+    * cannot overflow. */
+   r = 1;
+   while (r < v)
+      r <<= 1;
+   return r;
+}
+
+bool retro_spsc_init(retro_spsc_t *q, size_t min_capacity)
+{
+   size_t cap;
+
+   if (!q || min_capacity == 0 || min_capacity > (SIZE_MAX / 2))
+      return false;
+
+   cap = spsc_round_up_pow2(min_capacity);
+   if (cap == 0 || cap > (SIZE_MAX / 2))
+      return false;
+
+   q->buffer = (uint8_t*)malloc(cap);
+   if (!q->buffer)
+      return false;
+
+   q->capacity    = cap;
+   q->mask        = cap - 1;
+   q->mirror      = 0;
+   q->cached_tail = 0;
+   q->cached_head = 0;
+   retro_atomic_size_init(&q->head, 0);
+   retro_atomic_size_init(&q->tail, 0);
+   return true;
+}
+
+bool retro_spsc_init_mirrored(retro_spsc_t *q, size_t min_capacity)
+{
+#ifdef SPSC_MIRROR
+   size_t   cap, size, granule;
+   uint8_t *buf;
+
+   if (!q || min_capacity == 0 || min_capacity > (SIZE_MAX / 2))
+      return false;
+   cap     = spsc_round_up_pow2(min_capacity);
+   granule = spsc_mirror_granule();
+   /* Positions wrap at the buffer's size, so it has to be a power of
+    * 2 as well: the capacity, or the granule where that is larger. */
+   size    = cap > granule ? cap : granule;
+   if (     cap && !(granule & (granule - 1))
+         && size <= (SIZE_MAX / 4)
+         && (buf = spsc_mirror_alloc(size)))
+   {
+      q->buffer      = buf;
+      q->capacity    = cap;
+      q->mask        = size - 1;
+      q->mirror      = size;
+      q->cached_tail = 0;
+      q->cached_head = 0;
+      retro_atomic_size_init(&q->head, 0);
+      retro_atomic_size_init(&q->tail, 0);
+      return true;
+   }
+#endif
+   return retro_spsc_init(q, min_capacity);
+}
+
+void retro_spsc_free(retro_spsc_t *q)
+{
+   if (!q)
+      return;
+   if (q->buffer)
+   {
+#ifdef SPSC_MIRROR
+      if (q->mirror)
+         spsc_mirror_free(q->buffer, q->mirror);
+      else
+#endif
+         free(q->buffer);
+      q->buffer = NULL;
+   }
+   q->capacity = 0;
+   q->mirror   = 0;
+}
+
+void retro_spsc_clear(retro_spsc_t *q)
+{
+   if (!q)
+      return;
+   /* Quiescence is the caller's responsibility (documented).
+    * Under that assumption, no other thread is touching head or
+    * tail, so plain init is correct here -- and necessary, because
+    * plain assignment to a retro_atomic_size_t is illegal under
+    * the C11 stdatomic backend. */
+   q->cached_tail = 0;
+   q->cached_head = 0;
+   retro_atomic_size_init(&q->head, 0);
+   retro_atomic_size_init(&q->tail, 0);
+}
+
+bool retro_spsc_grow(retro_spsc_t *q, size_t min_capacity)
+{
+   retro_spsc_t n;
+
+   if (!q || !q->buffer)
+      return false;
+   if (min_capacity <= q->capacity)
+      return true;
+   if (!(q->mirror
+         ? retro_spsc_init_mirrored(&n, min_capacity)
+         : retro_spsc_init(&n, min_capacity)))
+      return false;
+   retro_spsc_grow_into(q, &n);
+   retro_spsc_free(&n);
+   return true;
+}
+
+bool retro_spsc_grow_into(retro_spsc_t *q, retro_spsc_t *n)
+{
+   uint8_t *old_buffer;
+   size_t   old_capacity, old_mask, old_mirror;
+   size_t   head, tail, pos;
+
+   if (!q || !n || !q->buffer || !n->buffer || n->capacity < q->capacity)
+      return false;
+
+   head = retro_atomic_load_acquire_size(&q->head);
+   tail = retro_atomic_load_acquire_size(&q->tail);
+   for (pos = tail; pos != head; )
+   {
+      size_t from = pos & q->mask;
+      size_t to   = pos & n->mask;
+      size_t run  = head - pos;
+      if (!q->mirror && run > q->mask + 1 - from)
+         run      = q->mask + 1 - from;
+      if (!n->mirror && run > n->mask + 1 - to)
+         run      = n->mask + 1 - to;
+      memcpy(n->buffer + to, q->buffer + from, run);
+      pos        += run;
+   }
+
+   old_buffer     = q->buffer;
+   old_capacity   = q->capacity;
+   old_mask       = q->mask;
+   old_mirror     = q->mirror;
+   q->buffer      = n->buffer;
+   q->capacity    = n->capacity;
+   q->mask        = n->mask;
+   q->mirror      = n->mirror;
+   q->cached_tail = tail;
+   q->cached_head = head;
+   retro_atomic_size_init(&q->head, head);
+   retro_atomic_size_init(&q->tail, tail);
+   n->buffer      = old_buffer;
+   n->capacity    = old_capacity;
+   n->mask        = old_mask;
+   n->mirror      = old_mirror;
+   n->cached_tail = n->cached_head = 0;
+   retro_atomic_size_init(&n->head, 0);
+   retro_atomic_size_init(&n->tail, 0);
+   return true;
+}
+
+size_t retro_spsc_write_avail(const retro_spsc_t *q)
+{
+   /* Producer query.  We read our own head (which we wrote) and the
+    * consumer's tail (which they wrote with a release-store).
+    * acquire-load on tail so any reads we do based on the freed
+    * space are ordered after the consumer's tail publication.
+    *
+    * The header says producer-only, but audio_driver.c also calls
+    * read_avail from the producer side as a fill-level query, and the
+    * same could happen here.  Keep BOTH loads acquire so a cross-side
+    * caller still gets a happens-before with the other thread's
+    * release; the query functions are not hot enough to justify the
+    * relaxed own-counter load used in the data-path functions below. */
+   size_t head = retro_atomic_load_acquire_size(
+         (retro_atomic_size_t*)&q->head);
+   size_t tail = retro_atomic_load_acquire_size(
+         (retro_atomic_size_t*)&q->tail);
+   /* head - tail is well-defined modular arithmetic on size_t.
+    * Invariant: 0 <= head - tail <= capacity. */
+   return q->capacity - (head - tail);
+}
+
+size_t retro_spsc_read_avail(const retro_spsc_t *q)
+{
+   /* Consumer query.  acquire-load on head pairs with the producer's
+    * release-store on head, so the data writes that preceded the
+    * release are visible by the time we read them. */
+   size_t head = retro_atomic_load_acquire_size(
+         (retro_atomic_size_t*)&q->head);
+   /* acquire on tail too: see retro_spsc_write_avail - this is called
+    * from the producer side in audio_driver.c as a fill query. */
+   size_t tail = retro_atomic_load_acquire_size(
+         (retro_atomic_size_t*)&q->tail);
+   return head - tail;
+}
+
+size_t retro_spsc_write(retro_spsc_t *q, const void *data, size_t bytes)
+{
+   size_t head_idx, first;
+   const uint8_t *src = (const uint8_t*)data;
+   /* head is ours (relaxed).  Room is first computed from our private
+    * copy of tail, which is never ahead of the real one; only when it
+    * says there is not enough do we acquire-load the consumer's tail
+    * (their cache line), which pairs with their release-store so the
+    * freed space is really free. */
+   size_t head  = retro_atomic_load_relaxed_size(&q->head);
+   size_t avail = q->capacity - (head - q->cached_tail);
+   if (avail < bytes)
+   {
+      q->cached_tail = retro_atomic_load_acquire_size(&q->tail);
+      avail          = q->capacity - (head - q->cached_tail);
+   }
+   if (bytes > avail)
+      bytes = avail;
+   if (bytes == 0)
+      return 0;
+
+   head_idx = head & q->mask;
+
+   /* first = bytes from head_idx to end-of-buffer */
+   first = spsc_contig(q, head_idx);
+   if (first > bytes)
+      first = bytes;
+
+   memcpy(q->buffer + head_idx, src, first);
+   memcpy(q->buffer, src + first, bytes - first);
+
+   /* Publish: release-store ensures the memcpys above are globally
+    * visible before the consumer observes the new head. */
+   retro_atomic_store_release_size(&q->head, head + bytes);
+   return bytes;
+}
+
+size_t retro_spsc_write_frames(retro_spsc_t *q, const void *data,
+      size_t frames, size_t frame_bytes)
+{
+   size_t head, avail, bytes, index, first;
+   const uint8_t *src = (const uint8_t*)data;
+   if (!frames || !frame_bytes) return 0;
+   head = retro_atomic_load_relaxed_size(&q->head);
+   avail = (q->capacity - (head - q->cached_tail)) / frame_bytes;
+   if (avail < frames)
+   {
+      q->cached_tail = retro_atomic_load_acquire_size(&q->tail);
+      avail = (q->capacity - (head - q->cached_tail)) / frame_bytes;
+   }
+   if (frames > avail) frames = avail;
+   if (!frames) return 0;
+   /* Multiplication is bounded by capacity, even for an oversized request. */
+   bytes = frames * frame_bytes;
+   index = head & q->mask;
+   first = spsc_contig(q, index);
+   if (first > bytes) first = bytes;
+   memcpy(q->buffer + index, src, first);
+   memcpy(q->buffer, src + first, bytes - first);
+   retro_atomic_store_release_size(&q->head, head + bytes);
+   return frames;
+}
+
+size_t retro_spsc_read(retro_spsc_t *q, void *data, size_t bytes)
+{
+   size_t tail_idx, first;
+   uint8_t *dst = (uint8_t*)data;
+   /* acquire on head pairs with producer's release-store; this is
+    * what makes the subsequent memcpys safe to read. */
+   /* tail is ours (relaxed).  Available from our private copy of head
+    * first, never ahead of the real one; only when it says too little
+    * do we acquire-load the producer's head, which pairs with their
+    * release-store so the bytes are really there. */
+   size_t tail  = retro_atomic_load_relaxed_size(&q->tail);
+   size_t avail = q->cached_head - tail;
+   if (avail < bytes)
+   {
+      q->cached_head = retro_atomic_load_acquire_size(&q->head);
+      avail          = q->cached_head - tail;
+   }
+   if (bytes > avail)
+      bytes = avail;
+   if (bytes == 0)
+      return 0;
+
+   tail_idx = tail & q->mask;
+
+   first = spsc_contig(q, tail_idx);
+   if (first > bytes)
+      first = bytes;
+
+   memcpy(dst, q->buffer + tail_idx, first);
+   memcpy(dst + first, q->buffer, bytes - first);
+
+   /* Publish: release-store so the producer can re-use this space. */
+   retro_atomic_store_release_size(&q->tail, tail + bytes);
+   return bytes;
+}
+
+size_t retro_spsc_peek(const retro_spsc_t *q, void *data, size_t bytes)
+{
+   size_t tail_idx, first;
+   uint8_t *dst = (uint8_t*)data;
+   /* Consumer side, same scheme as retro_spsc_read; the cast is only
+    * because peek takes a const queue and the cached copy is state. */
+   retro_spsc_t *w  = (retro_spsc_t*)q;
+   size_t tail  = retro_atomic_load_relaxed_size(&w->tail);
+   size_t avail = w->cached_head - tail;
+   if (avail < bytes)
+   {
+      w->cached_head = retro_atomic_load_acquire_size(&w->head);
+      avail          = w->cached_head - tail;
+   }
+   if (bytes > avail)
+      bytes = avail;
+   if (bytes == 0)
+      return 0;
+
+   tail_idx = tail & q->mask;
+
+   first    = spsc_contig(q, tail_idx);
+   if (first > bytes)
+      first = bytes;
+
+   memcpy(dst, q->buffer + tail_idx, first);
+   memcpy(dst + first, q->buffer, bytes - first);
+   /* No tail update: peek does not consume. */
+   return bytes;
+}
+
+size_t retro_spsc_write_begin(retro_spsc_t *q, void **ptr)
+{
+   size_t head_idx, span;
+   /* head is ours (relaxed).  Room from the private copy of tail
+    * first; re-read the consumer's tail only when that copy, rather
+    * than the wrap, is what limits the span - a stale copy would hand
+    * back less than is free. See retro_spsc_write. */
+   size_t head  = retro_atomic_load_relaxed_size(&q->head);
+   size_t avail = q->capacity - (head - q->cached_tail);
+   head_idx     = head & q->mask;
+   span         = spsc_contig(q, head_idx);
+   if (avail < span && avail < q->capacity)
+   {
+      q->cached_tail = retro_atomic_load_acquire_size(&q->tail);
+      avail          = q->capacity - (head - q->cached_tail);
+   }
+   if (span > avail)
+      span = avail;
+   *ptr = q->buffer + head_idx;
+   return span;
+}
+
+void retro_spsc_write_end(retro_spsc_t *q, size_t bytes)
+{
+   if (bytes == 0)
+      return;
+   /* head is written only by the producer, i.e. by this thread, so
+    * reading our own last store needs no ordering: a relaxed load is
+    * enough (and on the MSVC/Apple/__sync backends the acquire load is
+    * a locked RMW, so this also removes a lock-prefixed instruction
+    * from every span commit).
+    * Release: the caller's stores into the span happen-before the
+    * consumer's acquire-load of head, same pairing as retro_spsc_write. */
+   retro_atomic_store_release_size(&q->head,
+         retro_atomic_load_relaxed_size(&q->head) + bytes);
+}
+
+size_t retro_spsc_read_begin(retro_spsc_t *q, const void **ptr)
+{
+   size_t tail_idx, span;
+   /* tail is ours (relaxed).  Available from the private copy of
+    * head first; re-read the producer's head only when that copy,
+    * rather than the wrap, is what limits the span.  See
+    * retro_spsc_read. */
+   size_t tail  = retro_atomic_load_relaxed_size(&q->tail);
+   size_t avail = q->cached_head - tail;
+   tail_idx     = tail & q->mask;
+   span         = spsc_contig(q, tail_idx);
+   if (avail < span && avail < q->capacity)
+   {
+      q->cached_head = retro_atomic_load_acquire_size(&q->head);
+      avail          = q->cached_head - tail;
+   }
+   if (span > avail)
+      span = avail;
+   *ptr = q->buffer + tail_idx;
+   return span;
+}
+
+void retro_spsc_read_end(retro_spsc_t *q, size_t bytes)
+{
+   if (bytes == 0)
+      return;
+   /* tail is written only by the consumer, i.e. by this thread, so a
+    * relaxed load of our own counter suffices; see retro_spsc_write_end.
+    * Release: our reads of the span happen-before the producer's
+    * acquire-load of tail sees the space as free. */
+   retro_atomic_store_release_size(&q->tail,
+         retro_atomic_load_relaxed_size(&q->tail) + bytes);
+}
+
+size_t retro_spsc_skip(retro_spsc_t *q, size_t bytes)
+{
+   /* tail is ours (relaxed).  Available from our private copy of head
+    * first, never ahead of the real one; only when it says too little
+    * do we acquire-load the producer's head, which pairs with their
+    * release-store so the bytes are really there. */
+   size_t tail  = retro_atomic_load_relaxed_size(&q->tail);
+   size_t avail = q->cached_head - tail;
+   if (avail < bytes)
+   {
+      q->cached_head = retro_atomic_load_acquire_size(&q->head);
+      avail          = q->cached_head - tail;
+   }
+
+   if (bytes > avail)
+      bytes = avail;
+   if (bytes == 0)
+      return 0;
+
+   retro_atomic_store_release_size(&q->tail, tail + bytes);
+   return bytes;
+}

src/Cafe/HW/Espresso/PPCTimer.cpp

@@ -4,13 +4,22 @@
 #include "util/highresolutiontimer/HighResolutionTimer.h"
 #include "Common/cpu_features.h"
 
+#include <atomic>
+#include <thread>
+#include <rthreads/retro_eventcount.h>
+
 #if defined(ARCH_X86_64)
 #include <immintrin.h>
 #pragma intrinsic(__rdtsc)
 #endif
 
 uint64 _rdtscLastMeasure = 0;
-uint64 _rdtscFrequency = 0;
+// Published by the calibration thread, first after 25 ms and then twice more
+// as it gets better (PPCTimer_initThread); read under sTimerSpinlock
+static std::atomic<uint64> _rdtscFrequency{0};
+// The title start waits on this for the first estimate (PPCTimer_waitForInit)
+static retro_eventcount_t sTimerReady{};
+static bool sTimerReadyOk = false;
 
 struct uint128_t
 {
@@ -31,7 +40,18 @@ uint64 muldiv64(uint64 a, uint64 b, uint64 d)
 	return diva * b + moda * divb + moda * modb / d;
 }
 
-uint64 PPCTimer_estimateRDTSCFrequency()
+// The TSC's rate, measured against the high resolution timer. Upstream
+// measured once, for three seconds, and the title start waited for it - three
+// seconds off every boot that came sooner than that after the core was loaded,
+// which in RetroArch is every boot. The two clocks are read back to back, so
+// the error is the gap between those reads (well under a microsecond) over the
+// time measured: 25 ms is already good to about one part in 10^5. So the first
+// estimate is published after 25 ms, and the same measurement, still running,
+// replaces it at 200 ms and at 3 s with the one upstream would have had. The
+// rate is only ever a divisor of elapsed TSC ticks (PPCTimer_getFromRDTSC), so
+// a refined one changes how fast guest time runs by that one part in 10^5 from
+// then on, and moves no time already counted.
+int PPCTimer_initThread()
 {
     #if defined(ARCH_X86_64)
 	if (!g_CPUFeatures.x86.invariant_tsc)
@@ -39,44 +59,33 @@ uint64 PPCTimer_estimateRDTSCFrequency()
     #endif
 
 	_mm_mfence();
-	uint64 tscStart = __rdtsc();
-	unsigned int startTime = GetTickCount();
-	HRTick startTick = HighResolutionTimer::now().getTick();
-	// wait roughly 3 seconds
-	while (true)
+	const uint64 tscStart = __rdtsc();
+	const HRTick startTick = HighResolutionTimer::now().getTick();
+	const auto start = std::chrono::steady_clock::now();
+	for (const int ms : { 25, 200, 3000 })
 	{
-		if ((GetTickCount() - startTime) >= 3000)
-			break;
-		std::this_thread::sleep_for(std::chrono::milliseconds(10));
+		std::this_thread::sleep_until(start + std::chrono::milliseconds(ms));
+		_mm_mfence();
+		const HRTick stopTick = HighResolutionTimer::now().getTick();
+		const uint64 tscEnd = __rdtsc();
+		uint64 hrtFreq = 0;
+		const uint64 hrtDiff = HighResolutionTimer::getTimeDiffEx(startTick, stopTick, hrtFreq);
+		if (hrtDiff == 0)
+			continue;
+		const uint64 tscFreq = muldiv64(tscEnd - tscStart, hrtFreq, hrtDiff);
+		if (tscFreq == 0)
+			continue;
+		_rdtscFrequency.store(tscFreq, std::memory_order_release);
+		if (sTimerReadyOk)
+			retro_eventcount_notify(&sTimerReady);
 	}
-	_mm_mfence();
-	HRTick stopTick = HighResolutionTimer::now().getTick();
-	uint64 tscEnd = __rdtsc();
-	// derive frequency approximation from measured time difference
-	uint64 tsc_diff = tscEnd - tscStart;
-	uint64 hrtFreq = 0;
-	uint64 hrtDiff = HighResolutionTimer::getTimeDiffEx(startTick, stopTick, hrtFreq);
-	uint64 tsc_freq = muldiv64(tsc_diff, hrtFreq, hrtDiff);
-
-	// uint64 freqMultiplier = tsc_freq / hrtFreq;
-	//cemuLog_log(LogType::Force, "RDTSC measurement test:");
-	//cemuLog_log(LogType::Force, "TSC-diff:   0x{:016x}", tsc_diff);
-	//cemuLog_log(LogType::Force, "TSC-freq:   0x{:016x}", tsc_freq);
-	//cemuLog_log(LogType::Force, "HPC-diff:   0x{:016x}", qpc_diff);
-	//cemuLog_log(LogType::Force, "HPC-freq:   0x{:016x}", (uint64)qpc_freq.QuadPart);
-	//cemuLog_log(LogType::Force, "Multiplier: 0x{:016x}", freqMultiplier);
-
-	return tsc_freq;
-}
-
-int PPCTimer_initThread()
-{
-	_rdtscFrequency = PPCTimer_estimateRDTSCFrequency();
 	return 0;
 }
 
 void PPCTimer_init()
 {
+	if (!sTimerReadyOk)
+		sTimerReadyOk = retro_eventcount_init(&sTimerReady);
 	std::thread t(PPCTimer_initThread);
 	t.detach();
 	_rdtscLastMeasure = __rdtsc();
@@ -97,7 +106,7 @@ uint64 PPCTimer_getRawTsc()
 
 uint64 PPCTimer_microsecondsToTsc(uint64 us)
 {
-	return (us * _rdtscFrequency) / 1000000ULL;
+	return (us * _rdtscFrequency.load(std::memory_order_relaxed)) / 1000000ULL;
 }
 
 uint64 PPCTimer_tscToMicroseconds(uint64 us)
@@ -106,19 +115,34 @@ uint64 PPCTimer_tscToMicroseconds(uint64 us)
 	r.low = _umul128(us, 1000000ULL, &r.high);
 
 	uint64 remainder;
-	const uint64 microseconds = _udiv128(r.high, r.low, _rdtscFrequency, &remainder);
+	const uint64 microseconds = _udiv128(r.high, r.low, _rdtscFrequency.load(std::memory_order_relaxed), &remainder);
 
 	return microseconds;
 }
 
 bool PPCTimer_isReady()
 {
-	return _rdtscFrequency != 0;
+	return _rdtscFrequency.load(std::memory_order_acquire) != 0;
 }
 
 void PPCTimer_waitForInit()
 {
-	while (!PPCTimer_isReady()) std::this_thread::sleep_for(std::chrono::milliseconds(10));
+	// Asleep until the first estimate, 25 ms after PPCTimer_init at most
+	while (!PPCTimer_isReady())
+	{
+		if (!sTimerReadyOk)
+		{
+			std::this_thread::sleep_for(std::chrono::milliseconds(1));
+			continue;
+		}
+		const int key = retro_eventcount_prepare_wait(&sTimerReady);
+		if (PPCTimer_isReady())
+		{
+			retro_eventcount_cancel_wait(&sTimerReady);
+			break;
+		}
+		retro_eventcount_commit_wait(&sTimerReady, key);
+	}
 }
 
 FSpinlock sTimerSpinlock;
@@ -150,7 +174,7 @@ uint64 PPCTimer_getFromRDTSC()
 	#endif
 
 	uint64 remainder;
-	uint64 elapsedTick = _udiv128(_rdtscAcc.high, _rdtscAcc.low, _rdtscFrequency, &remainder);
+	uint64 elapsedTick = _udiv128(_rdtscAcc.high, _rdtscAcc.low, _rdtscFrequency.load(std::memory_order_relaxed), &remainder);
 
 	_rdtscAcc.low = remainder;
 	_rdtscAcc.high = 0;

src/Cafe/HW/Espresso/Recompiler/PPCRecompiler.cpp

@@ -3,6 +3,7 @@
 #include "PPCFunctionBoundaryTracker.h"
 #include "PPCRecompiler.h"
 #include "PPCRecompilerIml.h"
+#include "PPCRecompilerWorkQueue.h"
 #include "util/containers/RangeStore.h"
 #include "Cafe/OS/libs/coreinit/coreinit_CodeGen.h"
 #include "config/ActiveSettings.h"
@@ -44,7 +45,7 @@ struct
 {
 	std::atomic_bool initialized{false};
 	FSpinlock recompilerSpinlock;
-	std::queue<MPTR> targetQueue;
+	PPCRecompilerWorkQueue targetQueue; // pushed under recompilerSpinlock, popped without it
 	std::vector<ppcInvalidationRange> invalidationRanges;
 	std::atomic_int_fast32_t recompilerEnableCount{0};
 	// recompiler thread
@@ -113,8 +114,13 @@ void PPCRecompiler_visitAddressNoBlock(uint32 enterAddress)
 		s_ppcRecompilerState.recompilerSpinlock.unlock();
 		return;
 	}
-	// add to recompilation queue and flag as visited
-	s_ppcRecompilerState.targetQueue.emplace(enterAddress);
+	// add to recompilation queue and flag as visited - unless the queue is
+	// full, in which case it stays unvisited and is offered again later
+	if (!s_ppcRecompilerState.targetQueue.Push(enterAddress))
+	{
+		s_ppcRecompilerState.recompilerSpinlock.unlock();
+		return;
+	}
 	ppcRecompilerInstanceData->ppcRecompilerDirectJumpTable[enterAddress / 4] = PPCRecompiler_leaveRecompilerCode_visited;
 
 	s_ppcRecompilerState.recompilerSpinlock.unlock();
@@ -476,33 +482,24 @@ void PPCRecompiler_thread()
 
 	while (true)
 	{
-        if(s_ppcRecompilerState.workerThreadStopSignal)
-            return;
-		std::this_thread::sleep_for(std::chrono::milliseconds(10));
+		// Asleep until an address is pushed or the thread is stopped: a push
+		// is answered at once instead of at the next 10 ms poll.
+		s_ppcRecompilerState.targetQueue.WaitForWork(s_ppcRecompilerState.workerThreadStopSignal);
+		if (s_ppcRecompilerState.workerThreadStopSignal)
+			return;
 		// asynchronous recompilation:
 		// 1) take address from queue
 		// 2) check if address is still marked as visited
 		// 3) if yes -> calculate size, gather all entry points, recompile and update jump table
-		while (true)
+		uint32 enterAddress;
+		while (s_ppcRecompilerState.targetQueue.Pop(enterAddress))
 		{
-			s_ppcRecompilerState.recompilerSpinlock.lock();
-			if (s_ppcRecompilerState.targetQueue.empty())
-			{
-				s_ppcRecompilerState.recompilerSpinlock.unlock();
-				break;
-			}
-			auto enterAddress = s_ppcRecompilerState.targetQueue.front();
-			s_ppcRecompilerState.targetQueue.pop();
-
-			auto funcPtr = ppcRecompilerInstanceData->ppcRecompilerDirectJumpTable[enterAddress / 4];
-			if (funcPtr != PPCRecompiler_leaveRecompilerCode_visited)
-			{
-				// only recompile functions if marked as visited
-				s_ppcRecompilerState.recompilerSpinlock.unlock();
-				continue;
-			}
-			s_ppcRecompilerState.recompilerSpinlock.unlock();
-
+			// Without the lock, as the pushers check it before taking it: a
+			// stale answer either skips an address the interpreter will offer
+			// again, or translates one that makeRecompiledFunctionActive then
+			// refuses under the lock.
+			if (ppcRecompilerInstanceData->ppcRecompilerDirectJumpTable[enterAddress / 4] != PPCRecompiler_leaveRecompilerCode_visited)
+				continue; // only recompile functions if marked as visited
 			PPCRecompiler_recompileAtAddress(enterAddress);
 			if(s_ppcRecompilerState.workerThreadStopSignal)
 				return;
@@ -751,6 +748,8 @@ void PPCRecompiler_init()
 
 	// launch recompilation thread
     s_ppcRecompilerState.workerThreadStopSignal = false;
+    if (!s_ppcRecompilerState.targetQueue.Init())
+        cemuLog_log(LogType::Force, "Recompiler: no work queue, everything stays interpreted");
     s_ppcRecompilerState.workerThread = std::thread(PPCRecompiler_thread);
 }
 
@@ -763,11 +762,12 @@ void PPCRecompiler_Shutdown()
         return;
     // shut down recompiler thread
     s_ppcRecompilerState.workerThreadStopSignal = true;
+    s_ppcRecompilerState.targetQueue.Wake();
     if(s_ppcRecompilerState.workerThread.joinable())
         s_ppcRecompilerState.workerThread.join();
     // clean up queues
-    while(!s_ppcRecompilerState.targetQueue.empty())
-        s_ppcRecompilerState.targetQueue.pop();
+    s_ppcRecompilerState.targetQueue.Clear();
+    s_ppcRecompilerState.targetQueue.Free();
     s_ppcRecompilerState.invalidationRanges.clear();
     // clean range store
     s_ppcRecompilerState.functionStorage.clear();

src/Cafe/HW/Espresso/Recompiler/PPCRecompilerWorkQueue.cpp

@@ -0,0 +1,72 @@
+#include "Cafe/HW/Espresso/Recompiler/PPCRecompilerWorkQueue.h"
+
+bool PPCRecompilerWorkQueue::Init()
+{
+	if (m_initialized)
+		return true;
+	if (!retro_spsc_init(&m_ring, kCapacity * sizeof(uint32_t)))
+		return false;
+	if (!retro_eventcount_init(&m_wake))
+	{
+		retro_eventcount_free(&m_wake);
+		retro_spsc_free(&m_ring);
+		return false;
+	}
+	m_initialized = true;
+	return true;
+}
+
+void PPCRecompilerWorkQueue::Free()
+{
+	if (!m_initialized)
+		return;
+	retro_eventcount_free(&m_wake);
+	retro_spsc_free(&m_ring);
+	m_initialized = false;
+}
+
+bool PPCRecompilerWorkQueue::Push(uint32_t address)
+{
+	if (!m_initialized || retro_spsc_write_avail(&m_ring) < sizeof(address))
+		return false;
+	retro_spsc_write(&m_ring, &address, sizeof(address));
+	retro_eventcount_notify(&m_wake);
+	return true;
+}
+
+bool PPCRecompilerWorkQueue::Pop(uint32_t& address)
+{
+	if (!m_initialized)
+		return false;
+	return retro_spsc_read(&m_ring, &address, sizeof(address)) == sizeof(address);
+}
+
+void PPCRecompilerWorkQueue::WaitForWork(const std::atomic_bool& stop)
+{
+	if (!m_initialized)
+		return;
+	for (;;)
+	{
+		if (stop.load(std::memory_order_acquire) || retro_spsc_read_avail(&m_ring) >= sizeof(uint32_t))
+			return;
+		const int key = retro_eventcount_prepare_wait(&m_wake);
+		if (stop.load(std::memory_order_acquire) || retro_spsc_read_avail(&m_ring) >= sizeof(uint32_t))
+		{
+			retro_eventcount_cancel_wait(&m_wake);
+			return;
+		}
+		retro_eventcount_commit_wait(&m_wake, key);
+	}
+}
+
+void PPCRecompilerWorkQueue::Wake()
+{
+	if (m_initialized)
+		retro_eventcount_notify(&m_wake);
+}
+
+void PPCRecompilerWorkQueue::Clear()
+{
+	if (m_initialized)
+		retro_spsc_clear(&m_ring);
+}

src/Cafe/HW/Espresso/Recompiler/PPCRecompilerWorkQueue.h

@@ -0,0 +1,48 @@
+#pragma once
+
+#include <atomic>
+#include <cstdint>
+
+#include <retro_spsc.h>
+#include <rthreads/retro_eventcount.h>
+
+// The recompiler's to-do list: guest addresses the interpreter has run once and
+// the recompiler thread should translate. Upstream kept them in a std::queue
+// under the recompiler's spinlock and had the thread look every 10 ms, so a hot
+// function ran interpreted for 5 ms on average before its translation even
+// began, and each push could allocate.
+//
+// Now a fixed ring and an eventcount: a push wakes the thread at once, the
+// thread pops without the spinlock, and nothing allocates after Init.
+//
+// One producer at a time: the pushers are the emulated cores, and every push is
+// made inside the recompiler's spinlock, which serialises them (and gives the
+// ring's producer side the ordering it needs from one pusher to the next). One
+// consumer: the recompiler thread.
+class PPCRecompilerWorkQueue
+{
+public:
+	// Enough for every function a title touches in its first seconds; when it
+	// is full a push is refused and the address stays unvisited, so the
+	// interpreter offers it again later rather than anything being lost.
+	static constexpr size_t kCapacity = 16384;
+
+	bool Init();
+	void Free();
+	// Producer: false when full
+	bool Push(uint32_t address);
+	// Consumer: false when empty
+	bool Pop(uint32_t& address);
+	// Consumer: returns when there is work or stop is set. No timeout: the
+	// only things that end the wait are a push and Wake().
+	void WaitForWork(const std::atomic_bool& stop);
+	// Anyone: lets a waiting consumer out (to see a stop)
+	void Wake();
+	// Neither side running: drop whatever is queued
+	void Clear();
+
+private:
+	retro_spsc_t m_ring{};
+	retro_eventcount_t m_wake{};
+	bool m_initialized = false;
+};

src/Cafe/HW/Latte/Core/LatteGpuWake.cpp

@@ -0,0 +1,30 @@
+#include "Cafe/HW/Latte/Core/LatteGpuWake.h"
+
+#include <cstdlib>
+
+namespace
+{
+	// Brought up on first use and kept for the life of the process: the
+	// threads that notify it come and go with each title, and freeing it
+	// under one of them would be the only way this could fail.
+	struct Wake
+	{
+		retro_eventcount_t ec{};
+		Wake()
+		{
+			if (!retro_eventcount_init(&ec))
+				std::abort();
+		}
+	};
+}
+
+retro_eventcount_t* LatteGpuWake_Get()
+{
+	static Wake s_wake;
+	return &s_wake.ec;
+}
+
+void LatteGpuWake_Notify()
+{
+	retro_eventcount_notify(LatteGpuWake_Get());
+}

src/Cafe/HW/Latte/Core/LatteGpuWake.h

@@ -0,0 +1,33 @@
+#pragma once
+
+#include <rthreads/retro_eventcount.h>
+
+// What the GPU thread sleeps on while it waits for its own helpers - the
+// pipeline cache's compile threads during a load - instead of sleeping by the
+// clock. Notified by those helpers as they make progress, and by anything that
+// asks the GPU thread to stop doing what it is doing (a pause request, a stop),
+// so a wait here never outlasts the reason for it.
+//
+// Usage, on the waiting thread, with a predicate of its own:
+//   LatteGpuWake_WaitUntil([&] { return done() || Latte_GetStopSignal() ||
+//                                       Latte_IsGpuPauseRequested(); });
+retro_eventcount_t* LatteGpuWake_Get();
+void LatteGpuWake_Notify();
+
+template<typename TPred>
+void LatteGpuWake_WaitUntil(TPred pred)
+{
+	retro_eventcount_t* ec = LatteGpuWake_Get();
+	for (;;)
+	{
+		if (pred())
+			return;
+		const int key = retro_eventcount_prepare_wait(ec);
+		if (pred())
+		{
+			retro_eventcount_cancel_wait(ec);
+			return;
+		}
+		retro_eventcount_commit_wait(ec, key);
+	}
+}

src/Cafe/HW/Latte/Core/LatteThread.cpp

@@ -1,6 +1,7 @@
 #include <condition_variable>
 #include <mutex>
 #include <thread>
+#include "Cafe/HW/Latte/Core/LatteGpuWake.h"
 #ifdef _WIN32
 #include <process.h> // _endthreadex
 #endif
@@ -158,6 +159,7 @@ static std::condition_variable sGpuPauseCv;
 void Latte_RequestGpuPause()
 {
 	sGpuPauseRequested.store(true, std::memory_order_release);
+	LatteGpuWake_Notify(); // out of any LatteGpuWake_WaitUntil, to the gate
 }
 
 void Latte_ReleaseGpuPause()
@@ -664,6 +666,7 @@ void Latte_Stop()
 	}
 	sLatteThreadRunning = false;
 	_lock.unlock();
+	LatteGpuWake_Notify();
 	// The one thing upstream's stop does not cover. A thread parked at the
 	// pause gate is asleep on a condition variable whose predicate does check
 	// the stop signal - but only when something wakes it, and nothing did. A

src/Cafe/HW/Latte/Renderer/Vulkan/VulkanPipelineStableCache.cpp

@@ -12,6 +12,8 @@
 #include "Cafe/HW/Latte/Core/LatteShaderCache.h"
 #include "util/helpers/helpers.h"
 #include "util/crypto/sha.h"
+#include "Cafe/HW/Latte/Core/Latte.h"
+#include "Cafe/HW/Latte/Core/LatteGpuWake.h"
 
 struct
 {
@@ -99,8 +101,13 @@ bool VulkanPipelineStableCache::UpdateLoading(uint32& pipelinesLoadedTotal, uint
 	{
 		if (m_compilationQueue.size() >= 50)
 		{
-			std::this_thread::sleep_for(std::chrono::milliseconds(10));
-			return true; // queue up to 50 entries at a time
+			// Queue up to 50 entries at a time, and refill as soon as one is
+			// taken: upstream napped 10 ms here, which held the compile
+			// threads to 50 pipelines per 10 ms whenever the driver's own
+			// cache made them faster than that - the common case on a
+			// second boot.
+			WaitForCompileProgress([this] { return m_compilationQueue.size() < 50; });
+			return true;
 		}
 
 		uint64 fileNameA, fileNameB;
@@ -117,7 +124,7 @@ bool VulkanPipelineStableCache::UpdateLoading(uint32& pipelinesLoadedTotal, uint
 	}
 	if (g_vkCacheState.pipelinesLoaded != g_vkCacheState.pipelinesQueued)
 	{
-		std::this_thread::sleep_for(std::chrono::milliseconds(10));
+		WaitForCompileProgress([] { return g_vkCacheState.pipelinesLoaded == g_vkCacheState.pipelinesQueued; });
 		return true; // pipelines still compiling
 	}
 	return false; // done
@@ -492,6 +499,17 @@ bool VulkanPipelineStableCache::DeserializePipeline(MemStreamReader& memReader,
 	return true;
 }
 
+// Runs on: the GPU thread, in UpdateLoading. Until the compile threads have
+// moved on, or the GPU thread is asked to do something else - the loading
+// screen's own loop asks for those right after this returns.
+template<typename TPred>
+void VulkanPipelineStableCache::WaitForCompileProgress(TPred progressed)
+{
+	LatteGpuWake_WaitUntil([&] {
+		return progressed() || Latte_GetStopSignal() || Latte_IsGpuPauseRequested();
+	});
+}
+
 int VulkanPipelineStableCache::CompilerThread()
 {
 	SetThreadName("plCacheCompiler");
@@ -502,6 +520,8 @@ int VulkanPipelineStableCache::CompilerThread()
 			continue;
 		LoadPipelineFromCache(pipelineData);
 		++g_vkCacheState.pipelinesLoaded;
+		LatteGpuWake_Notify(); // the loader is waiting for this, or for room in the queue
+
 	}
 	return 0;
 }

src/Cafe/HW/Latte/Renderer/Vulkan/VulkanPipelineStableCache.h

@@ -55,6 +55,8 @@ public:
 
 private:
 	int CompilerThread(); // runs on: this cache's own plCacheCompiler threads
+	template<typename TPred>
+	void WaitForCompileProgress(TPred progressed);
 	void WorkerThread();  // runs on: this cache's own plCacheWriter thread
 
 	std::thread* m_pipelineCacheStoreThread;

src/Cafe/HW/Latte/Renderer/Vulkan/VulkanRenderer.cpp

@@ -2778,7 +2778,38 @@ void VulkanRenderer::StopPipelineCacheSaveThread()
 		return;
 	m_destructionRequested = true;
 	m_pipeline_cache_semaphore.notify();
+	if (m_pipeline_cache_wake_ok)
+		retro_eventcount_notify(&m_pipeline_cache_wake);
 	m_pipeline_cache_save_thread.join();
+	retro_eventcount_free(&m_pipeline_cache_wake); // safe after a failed init too
+	m_pipeline_cache_wake_ok = false;
+}
+
+// Runs on: the save thread. Sleeps for the duration unless the renderer is
+// being destroyed, which ends the sleep at once; true if it is.
+bool VulkanRenderer::PipelineCacheSaveWait(std::chrono::milliseconds duration)
+{
+	const auto deadline = std::chrono::steady_clock::now() + duration;
+	for (;;)
+	{
+		if (m_destructionRequested)
+			return true;
+		const auto left = std::chrono::duration_cast<std::chrono::microseconds>(deadline - std::chrono::steady_clock::now()).count();
+		if (left <= 0)
+			return false;
+		if (!m_pipeline_cache_wake_ok)
+		{
+			std::this_thread::sleep_for(std::chrono::microseconds(std::min<int64_t>(left, 250000)));
+			continue;
+		}
+		const int key = retro_eventcount_prepare_wait(&m_pipeline_cache_wake);
+		if (m_destructionRequested)
+		{
+			retro_eventcount_cancel_wait(&m_pipeline_cache_wake);
+			return true;
+		}
+		retro_eventcount_commit_wait_timeout(&m_pipeline_cache_wake, key, left);
+	}
 }
 
 void VulkanRenderer::PipelineCacheSaveThread(size_t cache_size)
@@ -2807,17 +2838,19 @@ void VulkanRenderer::PipelineCacheSaveThread(size_t cache_size)
 		m_pipeline_cache_semaphore.wait();
 		if (m_destructionRequested)
 			return;
-		for (sint32 i = 0; i < 15 * 4; i++)
-		{
-			if (m_destructionRequested)
-				return;
-			std::this_thread::sleep_for(std::chrono::milliseconds(250));
-		}
+		// Saved 15 s after the last new pipeline asked for it, as upstream
+		// does; a renderer going away ends the wait instead of being noticed
+		// up to 250 ms later
+		if (PipelineCacheSaveWait(std::chrono::seconds(15)))
+			return;
 
 		// always prioritize the compiler threads over this thread
 		// avoid calling stalling lock() since it will block other threads from entering even when the lock is currently held in shared mode
 		while (!m_pipeline_cache_save_mutex.try_lock())
-			std::this_thread::sleep_for(std::chrono::milliseconds(250));
+		{
+			if (PipelineCacheSaveWait(std::chrono::milliseconds(250)))
+				return;
+		}
 
 		size_t size = 0;
 		VkResult res = vkGetPipelineCacheData(m_logicalDevice, m_pipeline_cache, &size, nullptr);
@@ -2896,6 +2929,7 @@ void VulkanRenderer::CreatePipelineCache()
 	size_t cache_size = 0;
 	vkGetPipelineCacheData(m_logicalDevice, m_pipeline_cache, &cache_size, nullptr);
 
+	m_pipeline_cache_wake_ok = retro_eventcount_init(&m_pipeline_cache_wake);
 	m_pipeline_cache_save_thread = std::thread(&VulkanRenderer::PipelineCacheSaveThread, this, cache_size);
 }
 

src/Cafe/HW/Latte/Renderer/Vulkan/VulkanRenderer.h

@@ -1,5 +1,7 @@
 #pragma once
 
+#include <rthreads/retro_eventcount.h>
+
 #include "Cafe/HW/Latte/Renderer/Renderer.h"
 #include "Cafe/HW/Latte/Renderer/Vulkan/VulkanAPI.h"
 #include "Cafe/HW/Latte/Renderer/Vulkan/RendererShaderVk.h"
@@ -623,6 +625,11 @@ private:
 	Semaphore m_pipeline_cache_semaphore;
 	std::shared_mutex m_pipeline_cache_save_mutex;
 	std::thread m_pipeline_cache_save_thread;
+	// Wakes the save thread out of its 15 s wait when the renderer goes away,
+	// instead of it noticing at its next 250 ms look
+	retro_eventcount_t m_pipeline_cache_wake{};
+	bool m_pipeline_cache_wake_ok = false;
+	bool PipelineCacheSaveWait(std::chrono::milliseconds duration);
 	VkPipelineCache m_pipeline_cache{ nullptr };
 	std::unordered_map<uint64, VkPipeline> m_backbufferBlitPipelineCache;
 	std::unordered_map<uint64, VkDescriptorSet> m_backbufferBlitDescriptorSetCache;

src/libretro/CemuLibretro.cpp

@@ -94,6 +94,7 @@
 #include "input/motion/MotionHandler.h"
 #include <condition_variable>
 #include <atomic>
+#include <rthreads/retro_eventcount.h>
 
 // GL entry points come from the frontend, which owns the context. No wgl, glX,
 // egl or CGL anywhere in the core: making a context current, sharing one,
@@ -393,9 +394,14 @@ static bool s_render_ahead = false;
 // the core back in charge of its own speed, which is the thing being fixed. What
 // keeps that from hanging a shutdown is libretro_frame_gate_release, called
 // before anything waits for the GPU thread to stop.
-static std::mutex s_gate_mutex;
-static std::condition_variable s_gate_cv;
-static unsigned s_gate_tokens = 1;    // one, so the first frame does not wait
+//
+// No lock either. The state is atomics; a waiter parks on s_gate_wake, an
+// eventcount, and whoever changes the state notifies it after the change - so
+// a waiter that saw the old state is either woken or sees the new one when it
+// checks again before sleeping.
+static retro_eventcount_t s_gate_wake;
+extern retro_eventcount_t s_frame_wake;
+static std::atomic<unsigned> s_gate_tokens{1};    // one, so the first frame does not wait
 static std::atomic_bool s_gate_released{false};  // shutting down: nothing waits any more
 
 // True between "the frontend asked for a frame" and "the frame was handed
@@ -420,6 +426,36 @@ static std::atomic_bool s_frame_permit{false};
 static std::atomic<int64_t> s_last_grant_ns{0};
 static constexpr auto kFrontendStoppedAfter = std::chrono::milliseconds(100);
 
+// The eventcounts the gate and the frame handoff park on, brought up once for
+// the life of the process: the threads that wait on them come and go with each
+// title, and freeing one under such a thread is the only way they could fail.
+static bool libretro_wake_init()
+{
+	static const bool s_ok = [] {
+		return retro_eventcount_init(&s_gate_wake) && retro_eventcount_init(&s_frame_wake);
+	}();
+	return s_ok;
+}
+
+// Parks the calling thread until pred() holds. Every change that can make it
+// hold is followed by a notify of the same eventcount.
+template<typename TPred>
+static void libretro_wait_until(retro_eventcount_t* ec, TPred pred)
+{
+	for (;;)
+	{
+		if (pred())
+			return;
+		const int key = retro_eventcount_prepare_wait(ec);
+		if (pred())
+		{
+			retro_eventcount_cancel_wait(ec);
+			return;
+		}
+		retro_eventcount_commit_wait(ec, key);
+	}
+}
+
 // Nonzero while something outside the frame loop needs the emulator to keep
 // moving: the frame gate parks threads, and a handshake that waits for one of
 // those threads to arrive somewhere would otherwise wait forever (see
@@ -458,15 +494,21 @@ void libretro_frame_gate_wait()
 				std::chrono::steady_clock::now() - start).count(), std::memory_order_relaxed);
 		}
 	} addWait{waitStart};
-	std::unique_lock lock(s_gate_mutex);
 	// The frame is delivered, so the window closes here rather than when the
 	// next one opens. The emulated cores keep going until the frontend has
 	// stopped asking altogether (s_last_grant_ns); this thread waits for the next
 	// token.
 	s_frame_permit = false;
-	s_gate_cv.wait(lock, [] { return s_gate_tokens > 0 || s_gate_released || s_gate_hold_open; });
-	if (s_gate_tokens > 0)
-		s_gate_tokens--;
+	libretro_wait_until(&s_gate_wake, [] {
+		return s_gate_tokens.load(std::memory_order_acquire) > 0 ||
+			s_gate_released.load(std::memory_order_acquire) ||
+			s_gate_hold_open.load(std::memory_order_acquire) != 0;
+	});
+	// Take the token if there is one. Only this thread takes; a grant racing
+	// with this sets it back to one, which is the cap anyway.
+	unsigned tokens = s_gate_tokens.load(std::memory_order_acquire);
+	while (tokens > 0 && !s_gate_tokens.compare_exchange_weak(tokens, tokens - 1, std::memory_order_acq_rel))
+		;
 }
 
 // Runs on: the emulated CPU threads and the GPU thread - everything that is
@@ -482,13 +524,12 @@ void libretro_frame_window_wait()
 	// window to keep it out of.
 	if (!s_game_loaded || Latte_IsOnGpuCothread())
 		return;
-	// The state is written under s_gate_mutex but read here without it. This
-	// runs on every pass of the scheduler's idle loop - millions a second on
-	// the main core now that the cores are not parked between frames - and the
-	// GPU command processor takes it at every command. With the mutex on each
-	// of those, the idle loop held it nearly all the time and the GPU thread
-	// queued behind it at every command: 60 fps fell to 15. The lock is for
-	// waiting only.
+	// This runs on every pass of the scheduler's idle loop - millions a
+	// second on the main core now that the cores are not parked between
+	// frames - and the GPU command processor takes it at every command. When
+	// it took a mutex on each of those, the idle loop held it nearly all the
+	// time and the GPU thread queued behind it at every command: 60 fps fell to
+	// 15. Open, it costs a few atomic loads and a clock read.
 	auto open = [] {
 		const int64_t now = std::chrono::steady_clock::now().time_since_epoch().count();
 		return s_frame_permit.load(std::memory_order_acquire) ||
@@ -499,10 +540,9 @@ void libretro_frame_window_wait()
 	};
 	if (open())
 		return;
-	std::unique_lock lock(s_gate_mutex);
 	// Time only ever turns this from true to false; the way back is a grant,
-	// which notifies under the mutex, so a predicate wait cannot miss it.
-	s_gate_cv.wait(lock, open);
+	// a hold or a release, each of which notifies after it changes the state.
+	libretro_wait_until(&s_gate_wake, open);
 }
 
 // Lets the emulator run outside a frame for as long as the hold is held. For
@@ -512,42 +552,42 @@ void libretro_frame_window_wait()
 // Runs on: the frontend's thread, from the libretro callbacks.
 void libretro_frame_gate_hold_open(bool hold)
 {
-	std::lock_guard lock(s_gate_mutex);
 	if (hold)
-		s_gate_hold_open++;
-	else if (s_gate_hold_open > 0)
-		s_gate_hold_open--;
-	s_gate_cv.notify_all();
+		s_gate_hold_open.fetch_add(1, std::memory_order_acq_rel);
+	else
+	{
+		unsigned held = s_gate_hold_open.load(std::memory_order_acquire);
+		while (held > 0 && !s_gate_hold_open.compare_exchange_weak(held, held - 1, std::memory_order_acq_rel))
+			;
+	}
+	retro_eventcount_notify(&s_gate_wake);
 }
 
 // retro_run. Capped at one: a frontend that ran ahead must not build up credit
 // it can spend later, or the emulator would sprint after every menu visit.
 static void libretro_frame_gate_grant()
 {
-	std::lock_guard lock(s_gate_mutex);
-	s_gate_tokens = 1;
-	s_frame_permit = true;
+	s_gate_tokens.store(1, std::memory_order_release);
+	s_frame_permit.store(true, std::memory_order_release);
 	s_last_grant_ns.store(std::chrono::steady_clock::now().time_since_epoch().count(), std::memory_order_release);
-	s_gate_cv.notify_all();
+	retro_eventcount_notify(&s_gate_wake);
 }
 
 // Shutdown, and any other point where waiting for a token would be waiting for
 // a frontend that is not coming back.
 void libretro_frame_gate_release()
 {
-	std::lock_guard lock(s_gate_mutex);
-	s_gate_released = true;
-	s_frame_permit = true;
-	s_gate_cv.notify_all();
+	s_gate_released.store(true, std::memory_order_release);
+	s_frame_permit.store(true, std::memory_order_release);
+	retro_eventcount_notify(&s_gate_wake);
 }
 
-// Frame synchronization
-static std::mutex s_frame_mutex;
-static std::condition_variable s_frame_cv;
+// Frame synchronization: a flag and an eventcount retro_run parks on
+retro_eventcount_t s_frame_wake;
 std::atomic_bool s_frame_ready{false};
 
-// The GPU thread telling retro_run that the frame is done. Under the mutex and
-// with a notify, both of which matter: retro_run waits on s_frame_cv with a
+// The GPU thread telling retro_run that the frame is done. The flag and then
+// the notify, both of which matter: retro_run waits on s_frame_wake with a
 // 33 ms timeout, so a flag set without waking it is a frame that arrives when
 // the timeout expires rather than when it is ready - 30 fps out of a 60 fps
 // title. The Vulkan renderer used to store the flag on its own and did exactly
@@ -593,11 +633,8 @@ void libretro_signal_frame_ready()
 {
 	s_prof_frames_ready.fetch_add(1, std::memory_order_relaxed);
 	s_game_frames.fetch_add(1, std::memory_order_relaxed);
-	{
-		std::lock_guard lock(s_frame_mutex);
-		s_frame_ready.store(true, std::memory_order_release);
-	}
-	s_frame_cv.notify_one();
+	s_frame_ready.store(true, std::memory_order_release);
+	retro_eventcount_notify(&s_frame_wake);
 }
 static std::atomic_bool s_shutting_down{false};
 // Set by the first retro_run that runs the startup handshake. A launch that
@@ -3808,6 +3845,13 @@ RETRO_API void retro_init()
 {
 	if (s_initialized)
 		return;
+	// The frame gate's and the frame handoff's eventcounts, before any thread
+	// that could wait on them. Without them nothing can wait, so nothing runs.
+	if (!libretro_wake_init())
+	{
+		libretro_log(RETRO_LOG_ERROR, "could not set up the frame gate's eventcounts\n");
+		return;
+	}
 
 	// Breadcrumbs through the frontend's log rather than Cemu's. Cemu's log.txt
 	// does not exist until the paths below are set up, so a core that dies in
@@ -4024,11 +4068,8 @@ RETRO_API void retro_set_controller_port_device(unsigned port, unsigned device)
 static void libretro_wake_frame_waiters()
 {
 	s_shutting_down = true;
-	{
-		std::lock_guard lock(s_frame_mutex);
-		s_frame_ready = true;
-		s_frame_cv.notify_all();
-	}
+	s_frame_ready.store(true, std::memory_order_release);
+	retro_eventcount_notify(&s_frame_wake);
 	// Before ShutdownTitle, which stops the GPU thread: a thread parked at the
 	// frame gate is a thread that never gets there.
 	libretro_frame_gate_release();
@@ -5135,12 +5176,11 @@ static void libretro_start_install(std::vector<TitleInfo> found, bool removeSour
 // emulator standing still.
 static void libretro_frame_gate_rearm()
 {
-	std::lock_guard lock(s_gate_mutex);
-	s_gate_released = false;
-	s_gate_tokens = 1;
-	s_frame_permit = true;
-	s_gate_hold_open = 0;
-	s_gate_cv.notify_all();
+	s_gate_released.store(false, std::memory_order_release);
+	s_gate_tokens.store(1, std::memory_order_release);
+	s_frame_permit.store(true, std::memory_order_release);
+	s_gate_hold_open.store(0, std::memory_order_release);
+	retro_eventcount_notify(&s_gate_wake);
 }
 
 static void libretro_prepare_and_launch_title()
@@ -6124,13 +6164,10 @@ RETRO_API void retro_unload_game()
 	s_reset_requested.store(false, std::memory_order_release);
 	s_shutting_down = false;
 	s_frame_ready = false;
-	{
-		std::lock_guard lock(s_gate_mutex);
-		s_gate_released = false;
-		s_gate_tokens = 1;
-		s_frame_permit = true;
-		s_gate_hold_open = 0;
-	}
+	s_gate_released.store(false, std::memory_order_release);
+	s_gate_tokens.store(1, std::memory_order_release);
+	s_frame_permit.store(true, std::memory_order_release);
+	s_gate_hold_open.store(0, std::memory_order_release);
 	s_ppc_process_exited = false;
 	// An install loaded as content has nothing else to wait for it
 	if (s_install_content && s_convert_thread.joinable())
@@ -7450,10 +7487,27 @@ RETRO_API void retro_run()
 		else
 #endif
 		{
-			std::unique_lock lock(s_frame_mutex);
-			profTimedOut = !s_frame_cv.wait_for(lock, frameTime, [] {
-				return s_frame_ready.load();
-			});
+			// Until the frame is ready or its time is up, parked on the
+			// eventcount the GPU thread notifies after setting the flag
+			const auto deadline = std::chrono::steady_clock::now() + frameTime;
+			profTimedOut = false;
+			while (!s_frame_ready.load(std::memory_order_acquire))
+			{
+				const int64_t left = std::chrono::duration_cast<std::chrono::microseconds>(
+					deadline - std::chrono::steady_clock::now()).count();
+				if (left <= 0)
+				{
+					profTimedOut = true;
+					break;
+				}
+				const int key = retro_eventcount_prepare_wait(&s_frame_wake);
+				if (s_frame_ready.load(std::memory_order_acquire))
+				{
+					retro_eventcount_cancel_wait(&s_frame_wake);
+					break;
+				}
+				retro_eventcount_commit_wait_timeout(&s_frame_wake, key, left);
+			}
 			s_frame_ready = false;
 		}
 	}

src/libretro/LibretroAudioAPI.cpp

@@ -28,12 +28,10 @@ LibretroAudioRingBuffer LibretroAudioAPI::s_ring_buffer;
 std::atomic_bool LibretroAudioAPI::s_thread_mode{false};
 LibretroAudioAPI::MultiAudioCallback LibretroAudioAPI::s_multi_callback = nullptr;
 unsigned LibretroAudioAPI::s_out_channels = 2;
-std::vector<int16_t> LibretroAudioAPI::s_flush_buffer;
 
 LibretroAudioAPI::LibretroAudioAPI(uint32 samplerate, uint32 channels, uint32 samples_per_block, uint32 bits_per_sample)
 	: IAudioAPI(samplerate, channels, samples_per_block, bits_per_sample)
 {
-	s_flush_buffer.resize(LibretroAudioRingBuffer::kBufferSamples);
 	s_ring_buffer.Reset();
 }
 
@@ -41,8 +39,6 @@ LibretroAudioAPI::~LibretroAudioAPI()
 {
 	m_playing = false;
 	s_ring_buffer.Reset();
-	s_flush_buffer.clear();
-	s_flush_buffer.shrink_to_fit();
 }
 
 bool LibretroAudioAPI::NeedAdditionalBlocks() const
@@ -61,41 +57,53 @@ bool LibretroAudioAPI::FeedBlock(sint16* data)
 		return false;
 
 	// AX's channels (the TV mode: 1, 2 or 6) to the output's (2 or 6). 5.1
-	// is FL FR C LFE SL SR, as standalone hands it to cubeb.
+	// is FL FR C LFE SL SR, as standalone hands it to cubeb. Converted
+	// straight into the ring, in as many pieces as it lies in there.
 	const size_t frames = m_samplesPerBlock;
 	const unsigned in = m_channels, out = s_out_channels;
-	const int16_t* block = data;
-	if (in != out)
-	{
-		m_convert.resize(frames * out);
-		for (size_t f = 0; f < frames; f++)
-		{
-			const int16_t* s = data + f * in;
-			int16_t* d = m_convert.data() + f * out;
-			if (in == 1)
-			{
-				for (unsigned c = 0; c < out; c++)
-					d[c] = c < 2 ? s[0] : 0;
-			}
-			else if (in == 6 && out == 2)
-			{
-				// Centre and surrounds at -3 dB, LFE left out
-				const float c = s[2] * 0.7071f;
-				const float l = s[0] + c + s[4] * 0.7071f;
-				const float r = s[1] + c + s[5] * 0.7071f;
-				d[0] = (int16_t)std::clamp(l, -32768.0f, 32767.0f);
-				d[1] = (int16_t)std::clamp(r, -32768.0f, 32767.0f);
-			}
-			else
-			{
-				for (unsigned ch = 0; ch < out; ch++)
-					d[ch] = ch < in ? s[ch] : 0;
-			}
-		}
-		block = m_convert.data();
-	}
 	const size_t sampleCount = frames * out;
-	const size_t written = s_ring_buffer.Write(block, sampleCount);
+	size_t written = 0;
+	if (in == out)
+		written = s_ring_buffer.Write(data, sampleCount);
+	else
+	{
+		size_t f = 0;
+		while (f < frames)
+		{
+			int16_t* dst;
+			size_t n = s_ring_buffer.Reserve(&dst, (frames - f) * out);
+			n -= n % out;
+			if (n == 0)
+				break;
+			const size_t pieceFrames = n / out;
+			for (size_t i = 0; i < pieceFrames; i++, f++)
+			{
+				const int16_t* s = data + f * in;
+				int16_t* d = dst + i * out;
+				if (in == 1)
+				{
+					for (unsigned c = 0; c < out; c++)
+						d[c] = c < 2 ? s[0] : 0;
+				}
+				else if (in == 6 && out == 2)
+				{
+					// Centre and surrounds at -3 dB, LFE left out
+					const float c = s[2] * 0.7071f;
+					const float l = s[0] + c + s[4] * 0.7071f;
+					const float r = s[1] + c + s[5] * 0.7071f;
+					d[0] = (int16_t)std::clamp(l, -32768.0f, 32767.0f);
+					d[1] = (int16_t)std::clamp(r, -32768.0f, 32767.0f);
+				}
+				else
+				{
+					for (unsigned ch = 0; ch < out; ch++)
+						d[ch] = ch < in ? s[ch] : 0;
+				}
+			}
+			s_ring_buffer.Commit(n);
+			written += n;
+		}
+	}
 	AccountWrite(sampleCount, written);
 	return written > 0;
 }
@@ -155,33 +163,51 @@ void LibretroAudioAPI::FlushAudio(size_t frames)
 	if (availableSamples > 3 * frameSamples)
 		wanted = availableSamples - 2 * frameSamples;
 	wanted -= wanted % channels;
-	const size_t bufferCap = s_flush_buffer.size();
-	const size_t samplesToRead = std::min({availableSamples, wanted, bufferCap});
-	const size_t samplesRead = s_ring_buffer.Read(s_flush_buffer.data(), samplesToRead);
-
+	const size_t samplesToRead = std::min(availableSamples, wanted);
+	size_t samplesSent = 0;
+	const size_t samplesRead = SendFromRing(samplesToRead, samplesSent);
 	if (samplesRead == 0)
 		return;
+	AccountFlush(samplesRead, samplesSent);
+}
 
-	const size_t framesRead = samplesRead / channels;
-
-	// retro_audio_sample_batch_t may consume fewer frames than offered; loop so
-	// a partial accept doesn't lose audio.
-	size_t framesSent = 0;
-	const int16_t* cursor = s_flush_buffer.data();
-	while (framesSent < framesRead)
+// Hands the frontend up to sampleCount buffered samples straight out of the
+// ring - no copy on the way - and consumes all of them, as many as it took or
+// not: what it refuses is dropped, as it always was. Returns how many were
+// taken out of the ring; sent is how many the frontend accepted.
+size_t LibretroAudioAPI::SendFromRing(size_t sampleCount, size_t& sent)
+{
+	// 5.1: FL FR C LFE SL SR, as RetroArch's speaker bits run
+	constexpr unsigned k51 = 0x001 | 0x002 | 0x004 | 0x008 | 0x200 | 0x400;
+	const size_t channels = s_out_channels;
+	size_t taken = 0;
+	bool refused = false;
+	sent = 0;
+	while (taken < sampleCount)
 	{
-		// 5.1: FL FR C LFE SL SR, as RetroArch's speaker bits run
-		constexpr unsigned k51 = 0x001 | 0x002 | 0x004 | 0x008 | 0x200 | 0x400;
-		const size_t accepted = channels == 6
-			? s_multi_callback(cursor, framesRead - framesSent, 6, k51)
-			: s_audio_callback(cursor, framesRead - framesSent);
-		if (accepted == 0)
+		const int16_t* span;
+		const size_t n = s_ring_buffer.Peek(&span, sampleCount - taken);
+		if (n == 0)
 			break;
-		framesSent += accepted;
-		cursor += accepted * channels;
+		const size_t frames = n / channels;
+		size_t framesSent = 0;
+		// retro_audio_sample_batch_t may consume fewer frames than offered;
+		// loop so a partial accept doesn't lose audio
+		while (!refused && framesSent < frames)
+		{
+			const int16_t* cursor = span + framesSent * channels;
+			const size_t accepted = channels == 6
+				? s_multi_callback(cursor, frames - framesSent, 6, k51)
+				: s_audio_callback(cursor, frames - framesSent);
+			if (accepted == 0)
+				refused = true;
+			framesSent += accepted;
+		}
+		sent += framesSent * channels;
+		s_ring_buffer.Consume(n);
+		taken += n;
 	}
-
-	AccountFlush(samplesRead, framesSent * channels);
+	return taken;
 }
 
 size_t LibretroAudioAPI::PullAudio(size_t maxFrames)
@@ -202,27 +228,12 @@ size_t LibretroAudioAPI::PullAudio(size_t maxFrames)
 		s_ring_buffer.Discard(excess - excess % channels);
 	}
 
-	// Its own buffer: s_flush_buffer belongs to retro_run's FlushAudio
-	static std::array<int16_t, 1024 * LibretroAudioRingBuffer::kChannels> s_pull_buffer;
 	const size_t wanted = std::min<size_t>(maxFrames, 1024) * channels;
-	const size_t samplesRead = s_ring_buffer.Read(s_pull_buffer.data(), wanted);
-	const size_t framesRead = samplesRead / channels;
-
-	size_t framesSent = 0;
-	const int16_t* cursor = s_pull_buffer.data();
-	while (framesSent < framesRead)
-	{
-		constexpr unsigned k51 = 0x001 | 0x002 | 0x004 | 0x008 | 0x200 | 0x400;
-		const size_t accepted = channels == 6
-			? s_multi_callback(cursor, framesRead - framesSent, 6, k51)
-			: s_audio_callback(cursor, framesRead - framesSent);
-		if (accepted == 0)
-			break;
-		framesSent += accepted;
-		cursor += accepted * channels;
-	}
+	size_t samplesSent = 0;
+	const size_t samplesRead = SendFromRing(wanted, samplesSent);
 	if (samplesRead)
-		AccountFlush(samplesRead, framesSent * channels);
+		AccountFlush(samplesRead, samplesSent);
+	const size_t framesSent = samplesSent / channels;
 	return framesSent;
 }
 

src/libretro/LibretroAudioAPI.h

@@ -1,101 +1,9 @@
 #pragma once
 
 #include "audio/IAudioAPI.h"
+#include "libretro/LibretroAudioRing.h"
 
-#include <array>
 #include <atomic>
-#include <cstring>
-#include <vector>
-
-// Lock-free ring buffer for audio samples.
-// Single producer (Cemu AX audio thread), single consumer (retro_run thread).
-class LibretroAudioRingBuffer
-{
-public:
-	// ~341 ms at 48 kHz 5.1 (a second in stereo). It has to hold everything AX may make between
-	// two retro_runs, and a retro_run grants up to 250 ms (see
-	// AXOut_LibretroGrantSamples). At 128 ms a slow device's frames - 8 or 9
-	// a second in sco8487's heavier Deus Ex scenes - got grants the ring could
-	// not take, and the rest was dropped. Drained every retro_run, so the
-	// size adds no latency.
-	static constexpr size_t kBufferFrames = 16384;
-	// Up to 5.1; the ring holds whole frames of the output's channels, 2 or
-	// 6, both of which divide its size
-	static constexpr size_t kChannels = 6;
-	static constexpr size_t kBufferSamples = kBufferFrames * kChannels;
-
-	LibretroAudioRingBuffer() = default;
-
-	size_t GetReadAvailableSamples() const
-	{
-		const size_t w = m_write_pos.load(std::memory_order_acquire);
-		const size_t r = m_read_pos.load(std::memory_order_relaxed);
-		return (w >= r) ? (w - r) : (kBufferSamples - r + w);
-	}
-
-	size_t Write(const int16_t* data, size_t sampleCount)
-	{
-		size_t w = m_write_pos.load(std::memory_order_relaxed);
-		const size_t r = m_read_pos.load(std::memory_order_acquire);
-
-		const size_t used = (w >= r) ? (w - r) : (kBufferSamples - r + w);
-		const size_t available = kBufferSamples - used - kChannels;
-		const size_t toWrite = (sampleCount <= available) ? sampleCount : available;
-
-		if (toWrite == 0)
-			return 0;
-
-		const size_t firstChunk = (w + toWrite <= kBufferSamples) ? toWrite : (kBufferSamples - w);
-		std::memcpy(m_buffer.data() + w, data, firstChunk * sizeof(int16_t));
-		if (firstChunk < toWrite)
-			std::memcpy(m_buffer.data(), data + firstChunk, (toWrite - firstChunk) * sizeof(int16_t));
-
-		m_write_pos.store((w + toWrite) % kBufferSamples, std::memory_order_release);
-		return toWrite;
-	}
-
-	size_t Read(int16_t* dest, size_t maxSamples)
-	{
-		const size_t w = m_write_pos.load(std::memory_order_acquire);
-		size_t r = m_read_pos.load(std::memory_order_relaxed);
-
-		const size_t available = (w >= r) ? (w - r) : (kBufferSamples - r + w);
-		const size_t toRead = (maxSamples <= available) ? maxSamples : available;
-
-		if (toRead == 0)
-			return 0;
-
-		const size_t firstChunk = (r + toRead <= kBufferSamples) ? toRead : (kBufferSamples - r);
-		std::memcpy(dest, m_buffer.data() + r, firstChunk * sizeof(int16_t));
-		if (firstChunk < toRead)
-			std::memcpy(dest + firstChunk, m_buffer.data(), (toRead - firstChunk) * sizeof(int16_t));
-
-		m_read_pos.store((r + toRead) % kBufferSamples, std::memory_order_release);
-		return toRead;
-	}
-
-	void Reset()
-	{
-		m_write_pos.store(0, std::memory_order_relaxed);
-		m_read_pos.store(0, std::memory_order_relaxed);
-	}
-
-	// Drops the oldest samples, from the reading side, so it is safe while
-	// the writer goes on (Reset() is not)
-	void Discard(size_t sampleCount)
-	{
-		const size_t w = m_write_pos.load(std::memory_order_acquire);
-		const size_t r = m_read_pos.load(std::memory_order_relaxed);
-		const size_t available = (w >= r) ? (w - r) : (kBufferSamples - r + w);
-		const size_t toSkip = (sampleCount <= available) ? sampleCount : available;
-		m_read_pos.store((r + toSkip) % kBufferSamples, std::memory_order_release);
-	}
-
-private:
-	std::array<int16_t, kBufferSamples> m_buffer{};
-	std::atomic<size_t> m_write_pos{0};
-	std::atomic<size_t> m_read_pos{0};
-};
 
 class LibretroAudioAPI : public IAudioAPI
 {
@@ -108,7 +16,7 @@ public:
 		unsigned channels, unsigned layout);
 
 	LibretroAudioAPI(uint32 samplerate, uint32 channels, uint32 samples_per_block, uint32 bits_per_sample);
-	// The ring and the flush buffer are statics that outlive any one instance,
+	// The ring is a static that outlive any one instance,
 	// and until this they were only ever put back by the next run's
 	// constructor - so a closed title left its last few milliseconds of audio
 	// sitting there for the next one to start with. Emptied here instead,
@@ -157,6 +65,7 @@ public:
 	static bool IsStatsLogging() { return s_log_stats; }
 
 private:
+	static size_t SendFromRing(size_t sampleCount, size_t& sent);
 	static void AccountWrite(size_t offered, size_t written);
 	static void AccountFlush(size_t read, size_t sent);
 	static void ReportStats();
@@ -164,11 +73,8 @@ private:
 	static AudioCallback s_audio_callback;
 	static MultiAudioCallback s_multi_callback;
 	static unsigned s_out_channels;
-	// One block converted to the output's channels
-	std::vector<int16_t> m_convert;
 	static LibretroAudioRingBuffer s_ring_buffer;
 	static std::atomic_bool s_thread_mode;
-	static std::vector<int16_t> s_flush_buffer;
 
 	static bool s_log_stats;
 	static uint64_t s_stat_offered;

src/libretro/LibretroAudioRing.h

@@ -0,0 +1,136 @@
+#pragma once
+
+#include <array>
+#include <atomic>
+#include <cstddef>
+#include <cstdint>
+#include <cstring>
+
+// Lock-free ring buffer for audio samples.
+// Single producer (Cemu AX audio thread), single consumer (retro_run thread,
+// or RetroArch's audio thread in Audio Thread mode).
+//
+// Both sides work in place: the producer converts AX's block straight into the
+// ring (Reserve/Commit) and the consumer hands the frontend a pointer into it
+// (Peek/Consume), so a sample is written once and read once on its way out.
+// Positions only ever move by whole frames of 2 or 6 samples and the size is a
+// multiple of both, so a frame never straddles the end and every span handed
+// out is whole frames.
+class LibretroAudioRingBuffer
+{
+public:
+	// ~341 ms at 48 kHz 5.1 (a second in stereo). It has to hold everything AX may make between
+	// two retro_runs, and a retro_run grants up to 250 ms (see
+	// AXOut_LibretroGrantSamples). At 128 ms a slow device's frames - 8 or 9
+	// a second in sco8487's heavier Deus Ex scenes - got grants the ring could
+	// not take, and the rest was dropped. Drained every retro_run, so the
+	// size adds no latency.
+	static constexpr size_t kBufferFrames = 16384;
+	// Up to 5.1; the ring holds whole frames of the output's channels, 2 or
+	// 6, both of which divide its size
+	static constexpr size_t kChannels = 6;
+	static constexpr size_t kBufferSamples = kBufferFrames * kChannels;
+
+	LibretroAudioRingBuffer() = default;
+
+	size_t GetReadAvailableSamples() const
+	{
+		const size_t w = m_write_pos.load(std::memory_order_acquire);
+		const size_t r = m_read_pos.load(std::memory_order_relaxed);
+		return (w >= r) ? (w - r) : (kBufferSamples - r + w);
+	}
+
+	// Producer: room for up to maxSamples in one piece, at *ptr. What is
+	// written there becomes visible with Commit.
+	size_t Reserve(int16_t** ptr, size_t maxSamples)
+	{
+		const size_t w = m_write_pos.load(std::memory_order_relaxed);
+		const size_t r = m_read_pos.load(std::memory_order_acquire);
+		const size_t used = (w >= r) ? (w - r) : (kBufferSamples - r + w);
+		// One frame kept free, so full and empty are told apart
+		const size_t room = kBufferSamples - used - kChannels;
+		size_t n = room < maxSamples ? room : maxSamples;
+		if (n > kBufferSamples - w)
+			n = kBufferSamples - w;
+		*ptr = m_buffer.data() + w;
+		return n;
+	}
+
+	void Commit(size_t sampleCount)
+	{
+		const size_t w = m_write_pos.load(std::memory_order_relaxed);
+		m_write_pos.store((w + sampleCount) % kBufferSamples, std::memory_order_release);
+	}
+
+	size_t Write(const int16_t* data, size_t sampleCount)
+	{
+		size_t written = 0;
+		while (written < sampleCount)
+		{
+			int16_t* dst;
+			const size_t n = Reserve(&dst, sampleCount - written);
+			if (n == 0)
+				break;
+			std::memcpy(dst, data + written, n * sizeof(int16_t));
+			Commit(n);
+			written += n;
+		}
+		return written;
+	}
+
+	// Consumer: the oldest buffered samples that lie in one piece, at most
+	// maxSamples, at *ptr. They stay valid until Consume.
+	size_t Peek(const int16_t** ptr, size_t maxSamples) const
+	{
+		const size_t w = m_write_pos.load(std::memory_order_acquire);
+		const size_t r = m_read_pos.load(std::memory_order_relaxed);
+		size_t n = (w >= r) ? (w - r) : (kBufferSamples - r);
+		if (n > maxSamples)
+			n = maxSamples;
+		*ptr = m_buffer.data() + r;
+		return n;
+	}
+
+	// Drops the oldest samples, from the reading side, so it is safe while
+	// the writer goes on (Reset() is not)
+	void Consume(size_t sampleCount)
+	{
+		const size_t w = m_write_pos.load(std::memory_order_acquire);
+		const size_t r = m_read_pos.load(std::memory_order_relaxed);
+		const size_t available = (w >= r) ? (w - r) : (kBufferSamples - r + w);
+		const size_t toSkip = (sampleCount <= available) ? sampleCount : available;
+		m_read_pos.store((r + toSkip) % kBufferSamples, std::memory_order_release);
+	}
+
+	void Discard(size_t sampleCount)
+	{
+		Consume(sampleCount);
+	}
+
+	size_t Read(int16_t* dest, size_t maxSamples)
+	{
+		size_t done = 0;
+		while (done < maxSamples)
+		{
+			const int16_t* src;
+			const size_t n = Peek(&src, maxSamples - done);
+			if (n == 0)
+				break;
+			std::memcpy(dest + done, src, n * sizeof(int16_t));
+			Consume(n);
+			done += n;
+		}
+		return done;
+	}
+
+	void Reset()
+	{
+		m_write_pos.store(0, std::memory_order_relaxed);
+		m_read_pos.store(0, std::memory_order_relaxed);
+	}
+
+private:
+	std::array<int16_t, kBufferSamples> m_buffer{};
+	std::atomic<size_t> m_write_pos{0};
+	std::atomic<size_t> m_read_pos{0};
+};

test/handoff/.gitignore

@@ -0,0 +1,3 @@
+handoff_test
+handoff_test.exe
+*.o

test/handoff/Makefile

@@ -0,0 +1,30 @@
+# The lock-free handoffs the core's threads use (run_tests.sh): the
+# recompiler's work queue, the GPU thread's wake, the audio ring.
+# SANITIZER=thread or SANITIZER=address,undefined for the sanitizer runs.
+LRC := ../../dependencies/libretro-common
+SRC := ../../src
+CXX ?= g++
+CC ?= gcc
+FLAGS := -O1 -g -I$(SRC) -I$(LRC)/include -DHAVE_THREADS
+ifneq ($(SANITIZER),)
+   FLAGS += -fsanitize=$(SANITIZER) -fno-omit-frame-pointer
+endif
+LIBS := -lpthread
+ifneq (,$(findstring mingw,$(CC)))
+   LIBS += -static
+   EXE := .exe
+endif
+C_SRCS := $(LRC)/queues/retro_spsc.c $(LRC)/rthreads/retro_eventcount.c $(LRC)/rthreads/rthreads.c
+CXX_SRCS := handoff_test.cpp \
+	$(SRC)/Cafe/HW/Espresso/Recompiler/PPCRecompilerWorkQueue.cpp \
+	$(SRC)/Cafe/HW/Latte/Core/LatteGpuWake.cpp
+
+handoff_test$(EXE): $(CXX_SRCS) $(C_SRCS) $(SRC)/libretro/LibretroAudioRing.h
+	$(CC) $(FLAGS) -std=gnu99 -c $(LRC)/queues/retro_spsc.c -o retro_spsc.o
+	$(CC) $(FLAGS) -std=gnu99 -c $(LRC)/rthreads/retro_eventcount.c -o retro_eventcount.o
+	$(CC) $(FLAGS) -std=gnu99 -c $(LRC)/rthreads/rthreads.c -o rthreads.o
+	$(CXX) $(FLAGS) -std=gnu++20 $(CXX_SRCS) retro_spsc.o retro_eventcount.o rthreads.o $(LIBS) -o $@
+	rm -f *.o
+
+clean:
+	rm -f handoff_test handoff_test.exe *.o

test/handoff/handoff_test.cpp

@@ -0,0 +1,303 @@
+// The lock-free handoffs between the core's threads, each tested for what its
+// users rely on:
+//  - the recompiler's work queue delivers every address exactly once and in
+//    each pusher's order, wakes its consumer on a push (no poll - a lost wake
+//    hangs this test), refuses rather than grows when full, and lets a waiting
+//    consumer out on a stop;
+//  - the GPU thread's wake ends a wait as soon as the predicate is made true;
+//  - the audio ring hands out whole frames in place, across the wrap, for 2 and
+//    6 channels, and streams intact between a producer and a consumer thread.
+#include "Cafe/HW/Espresso/Recompiler/PPCRecompilerWorkQueue.h"
+#include "Cafe/HW/Latte/Core/LatteGpuWake.h"
+#include "libretro/LibretroAudioRing.h"
+
+#include <atomic>
+#include <chrono>
+#include <cstdio>
+#include <memory>
+#include <thread>
+#include <vector>
+
+static int s_failures = 0;
+#define CHECK(cond) \
+	do { if (!(cond)) { std::fprintf(stderr, "FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); s_failures++; } } while (0)
+
+// --- recompiler work queue ----------------------------------------------
+
+// The core's pushers are serialised by the recompiler's spinlock; this is that
+static std::atomic_flag s_pushLock = ATOMIC_FLAG_INIT;
+
+static void TestWorkQueueStream()
+{
+	auto q = std::make_unique<PPCRecompilerWorkQueue>();
+	CHECK(q->Init());
+	constexpr uint32_t kProducers = 3, kPerProducer = 60000;
+	std::atomic_bool stop{false};
+	std::vector<uint32_t> lastSeen(kProducers, 0);
+	std::vector<uint8_t> seen(kProducers * kPerProducer + 1, 0);
+	std::atomic<uint32_t> received{0};
+	bool ordered = true, unique = true;
+
+	std::thread consumer([&] {
+		for (;;)
+		{
+			q->WaitForWork(stop);
+			uint32_t a;
+			while (q->Pop(a))
+			{
+				const uint32_t producer = a / (kPerProducer + 1);
+				const uint32_t seq = a % (kPerProducer + 1);
+				if (seq <= lastSeen[producer])
+					ordered = false;
+				lastSeen[producer] = seq;
+				if (seen[producer * kPerProducer + seq - 1]++)
+					unique = false;
+				received.fetch_add(1, std::memory_order_relaxed);
+			}
+			if (stop.load())
+				return;
+		}
+	});
+	std::vector<std::thread> producers;
+	for (uint32_t p = 0; p < kProducers; p++)
+	{
+		producers.emplace_back([&, p] {
+			for (uint32_t seq = 1; seq <= kPerProducer;)
+			{
+				while (s_pushLock.test_and_set(std::memory_order_acquire))
+					std::this_thread::yield();
+				const bool pushed = q->Push(p * (kPerProducer + 1) + seq);
+				s_pushLock.clear(std::memory_order_release);
+				if (pushed)
+					seq++;
+				else
+					std::this_thread::yield(); // full: the interpreter would offer it again later
+			}
+		});
+	}
+	for (auto& t : producers)
+		t.join();
+	while (received.load() < kProducers * kPerProducer)
+		std::this_thread::yield();
+	stop = true;
+	q->Wake();
+	consumer.join();
+	CHECK(received.load() == kProducers * kPerProducer);
+	CHECK(ordered);
+	CHECK(unique);
+	q->Free();
+}
+
+static void TestWorkQueueFullAndStop()
+{
+	auto q = std::make_unique<PPCRecompilerWorkQueue>();
+	CHECK(!q->Push(1)); // not initialised: refused, not a crash
+	CHECK(q->Init());
+	size_t pushed = 0;
+	while (q->Push((uint32_t)pushed + 1))
+		pushed++;
+	CHECK(pushed == PPCRecompilerWorkQueue::kCapacity); // bounded, and that bound
+	uint32_t a = 0;
+	CHECK(q->Pop(a) && a == 1);
+	CHECK(q->Push(0xABCD)); // room again
+	q->Clear();
+	CHECK(!q->Pop(a));
+
+	// A consumer asleep on an empty queue is woken by a push: it has been
+	// asleep long enough to be past any spin when the push comes, and a lost
+	// wake shows as the push never being taken
+	for (int round = 0; round < 5; round++)
+	{
+		std::atomic_bool never{false};
+		std::atomic_bool took{false};
+		std::thread sleeper([&] {
+			q->WaitForWork(never);
+			uint32_t v;
+			took = q->Pop(v) && v == 77;
+		});
+		std::this_thread::sleep_for(std::chrono::milliseconds(30));
+		CHECK(!took.load());
+		CHECK(q->Push(77));
+		const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(5);
+		while (!took.load() && std::chrono::steady_clock::now() < deadline)
+			std::this_thread::sleep_for(std::chrono::milliseconds(1));
+		CHECK(took.load());
+		if (!took.load())
+		{
+			// Let it out so the test can end and say so
+			never = true;
+			q->Wake();
+		}
+		sleeper.join();
+	}
+
+	// A consumer asleep on an empty queue is let out by a stop
+	std::atomic_bool stop{false};
+	std::atomic_bool out{false};
+	std::thread consumer([&] {
+		q->WaitForWork(stop);
+		out = true;
+	});
+	std::this_thread::sleep_for(std::chrono::milliseconds(20));
+	CHECK(!out.load());
+	stop = true;
+	q->Wake();
+	consumer.join();
+	CHECK(out.load());
+	q->Free();
+}
+
+// --- GPU thread wake ----------------------------------------------------
+
+static void TestGpuWake()
+{
+	for (int round = 0; round < 2000; round++)
+	{
+		std::atomic_bool ready{false};
+		std::thread helper([&] {
+			ready.store(true, std::memory_order_release);
+			LatteGpuWake_Notify();
+		});
+		LatteGpuWake_WaitUntil([&] { return ready.load(std::memory_order_acquire); });
+		helper.join();
+		CHECK(ready.load());
+	}
+	// Asleep past any spin when the predicate turns true
+	{
+		std::atomic_bool ready{false};
+		std::atomic_bool done{false};
+		std::thread waiter([&] {
+			LatteGpuWake_WaitUntil([&] { return ready.load(std::memory_order_acquire); });
+			done = true;
+		});
+		std::this_thread::sleep_for(std::chrono::milliseconds(30));
+		ready.store(true, std::memory_order_release);
+		LatteGpuWake_Notify();
+		waiter.join(); // a lost wake hangs here, and the runner's timeout says so
+		CHECK(done.load());
+	}
+	// Already true: returns without waiting
+	LatteGpuWake_WaitUntil([] { return true; });
+}
+
+// --- audio ring ---------------------------------------------------------
+
+static void TestAudioRingSpans(unsigned channels)
+{
+	auto ring = std::make_unique<LibretroAudioRingBuffer>();
+	int16_t next = 0, expect = 0;
+	// Many laps of the ring in uneven pieces: every span is whole frames,
+	// and what comes out is what went in
+	for (int round = 0; round < 4000; round++)
+	{
+		const size_t frames = 1 + (round * 7919) % 3000;
+		size_t want = frames * channels;
+		while (want)
+		{
+			int16_t* dst;
+			const size_t n = ring->Reserve(&dst, want);
+			if (n == 0)
+				break;
+			CHECK(n % channels == 0);
+			for (size_t i = 0; i < n; i++)
+				dst[i] = next++;
+			ring->Commit(n);
+			want -= n;
+		}
+		size_t drain = ((round % 3) == 0) ? ring->GetReadAvailableSamples() : frames * channels / 2;
+		drain -= drain % channels;
+		while (drain)
+		{
+			const int16_t* src;
+			const size_t n = ring->Peek(&src, drain);
+			if (n == 0)
+				break;
+			CHECK(n % channels == 0);
+			for (size_t i = 0; i < n; i++)
+				if (src[i] != expect++)
+				{
+					s_failures++;
+					std::fprintf(stderr, "FAIL audio ring: sample out of order (%u channels)\n", channels);
+					return;
+				}
+			ring->Consume(n);
+			drain -= n;
+		}
+	}
+	// Never more than the ring less one frame
+	int16_t* dst;
+	ring->Reset();
+	size_t total = 0;
+	for (;;)
+	{
+		const size_t n = ring->Reserve(&dst, LibretroAudioRingBuffer::kBufferSamples);
+		if (!n)
+			break;
+		ring->Commit(n);
+		total += n;
+	}
+	CHECK(total == LibretroAudioRingBuffer::kBufferSamples - LibretroAudioRingBuffer::kChannels);
+}
+
+static void TestAudioRingThreads()
+{
+	auto ring = std::make_unique<LibretroAudioRingBuffer>();
+	constexpr size_t kTotal = 8u * 1024 * 1024;
+	std::atomic_bool bad{false};
+	std::thread producer([&] {
+		int16_t v = 0;
+		size_t done = 0;
+		while (done < kTotal)
+		{
+			int16_t* dst;
+			size_t n = ring->Reserve(&dst, std::min<size_t>(kTotal - done, 960 * 2));
+			n -= n % 2;
+			if (!n)
+			{
+				std::this_thread::yield();
+				continue;
+			}
+			for (size_t i = 0; i < n; i++)
+				dst[i] = v++;
+			ring->Commit(n);
+			done += n;
+		}
+	});
+	int16_t expect = 0;
+	size_t got = 0;
+	while (got < kTotal)
+	{
+		const int16_t* src;
+		const size_t n = ring->Peek(&src, 1600);
+		if (!n)
+		{
+			std::this_thread::yield();
+			continue;
+		}
+		for (size_t i = 0; i < n; i++)
+			if (src[i] != expect++)
+				bad = true;
+		ring->Consume(n);
+		got += n;
+	}
+	producer.join();
+	CHECK(!bad.load());
+	CHECK(got == kTotal);
+}
+
+int main()
+{
+	TestWorkQueueStream();
+	TestWorkQueueFullAndStop();
+	TestGpuWake();
+	TestAudioRingSpans(2);
+	TestAudioRingSpans(6);
+	TestAudioRingThreads();
+	if (s_failures)
+	{
+		std::fprintf(stderr, "%d check(s) failed\n", s_failures);
+		return 1;
+	}
+	std::printf("handoff: all passed\n");
+	return 0;
+}

test/handoff/run_tests.sh

@@ -0,0 +1,9 @@
+#!/bin/sh
+# The lock-free handoffs between the core's threads. RUNNER prefixes the test
+# (wine for a MinGW build).
+set -e
+cd "$(dirname "$0")"
+make -s
+RUN=${RUNNER:-}
+command -v timeout >/dev/null 2>&1 && RUN="timeout 300 $RUN"
+$RUN ./handoff_test${EXE:-}