mirror of
https://github.com/libretro/parallel-gx2.git
synced 2026-11-07 04:25:58 -10:00
The recompiler thread looked for work every 10 ms, so a function the interpreter had just run waited 5 ms on average before its translation began, and every queued address could allocate inside std::queue. Its queue is now a fixed retro_spsc ring with an eventcount (PPCRecompilerWorkQueue): a push wakes the thread at once, the thread pops without the recompiler's spinlock, and a full queue refuses the push so the address stays unvisited and is offered again later. Loading the Vulkan pipeline cache napped 10 ms whenever 50 pipelines were queued and again while the last ones compiled, which held a warm driver cache to 50 pipelines per 10 ms. The GPU thread now sleeps on an eventcount (LatteGpuWake) that the compile threads notify after each pipeline and that a pause request or a stop also notify, so it refills the queue as soon as there is room and still answers a context loss at once. The pipeline cache's save thread keeps its 15 s debounce but sleeps on an eventcount that the renderer's destruction notifies, instead of looking every 250 ms. The CPU timer's TSC calibration measured for three seconds and the title start waited for it, which in RetroArch is every boot. Both clocks are read back to back, so 25 ms already gives about one part in 10^5: the first estimate is published then and refined at 200 ms and 3 s by the same measurement. The rate only divides elapsed ticks, so a refinement changes no time already counted. The title start waits on an eventcount instead of a 10 ms poll. Audio goes to the frontend straight out of the ring buffer (Peek/Consume) and AX's blocks are converted straight into it (Reserve/Commit), so the per-frame copy into a 192 KiB flush buffer, the pull buffer and the conversion vector are gone. The ring moves to LibretroAudioRing.h. The frame gate and the frame handoff between the GPU thread and retro_run are atomics and eventcounts in place of two mutexes and condition variables, with the same predicates and the same timeout. retro_spsc is synced from RetroArch's libretro-common. test/handoff checks the work queue (every address once and in order from three serialised pushers, wake-up from a parked consumer, the bound, the stop), the GPU wake, and the audio ring (whole-frame spans across the wrap for 2 and 6 channels, a producer and a consumer thread streaming), plain, under TSan and under ASan/UBSan, in both CIs.
463 lines
18 KiB
C
463 lines
18 KiB
C
/* Copyright (C) 2010-2026 The RetroArch team
|
|
*
|
|
* ---------------------------------------------------------------------------------------
|
|
* The following license statement only applies to this file (retro_spsc.h).
|
|
* ---------------------------------------------------------------------------------------
|
|
*
|
|
* Permission is hereby granted, free of charge,
|
|
* to any person obtaining a copy of this software and associated documentation files (the "Software"),
|
|
* to deal in the Software without restriction, including without limitation the rights to
|
|
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
|
|
* and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
|
*
|
|
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
|
*
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
|
|
* INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
|
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
|
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
|
*/
|
|
|
|
#ifndef __LIBRETRO_SDK_SPSC_H
|
|
#define __LIBRETRO_SDK_SPSC_H
|
|
|
|
/*
|
|
* retro_spsc.h - portable single-producer / single-consumer byte queue
|
|
*
|
|
* Lock-free byte-stream queue with one writer thread and one reader
|
|
* thread. Uses release-store / acquire-load on two cursors (the
|
|
* Lamport / Disruptor design) rather than a single shared count, so
|
|
* neither thread issues atomic RMW operations on the hot path.
|
|
*
|
|
* Constraints:
|
|
* - Exactly ONE producer and ONE consumer. No locking is performed;
|
|
* two producers or two consumers will corrupt the queue.
|
|
* - Capacity must be power-of-2. Init rounds up.
|
|
* - Maximum capacity is SIZE_MAX/2 (so head - tail never overflows
|
|
* the well-defined unsigned modular range).
|
|
* - The buffer is byte-addressed. Callers pushing structured records
|
|
* are responsible for framing.
|
|
*
|
|
* Memory model:
|
|
* - Producer writes data into the buffer (non-atomic stores), then
|
|
* publishes the new head via release-store.
|
|
* - Consumer acquire-loads head (sees data writes that preceded the
|
|
* release), reads data (non-atomic loads), then publishes the new
|
|
* tail via release-store.
|
|
* - Producer acquire-loads tail to know how much space is free.
|
|
* - This pairing gives the SPSC invariant on every backend
|
|
* retro_atomic.h supports lock-freely. On the volatile fallback
|
|
* (no real backend), correctness reduces to single-core or x86 TSO,
|
|
* same as every other retro_atomic.h caller.
|
|
*
|
|
* Cache behaviour:
|
|
* - head and tail are placed on separate cache lines via explicit
|
|
* padding to RETRO_SPSC_CACHE_LINE. Without this, the producer's
|
|
* publish would invalidate the consumer's tail line and vice versa,
|
|
* halving throughput on contended SMP. The padding is a
|
|
* performance hint; correctness does not depend on it.
|
|
* - The padding runs past tail as well, so the isolation survives
|
|
* embedding retro_spsc_t by value in a larger struct. Without a
|
|
* trailing pad, whatever the embedder declares next lands within
|
|
* tail's line and the consumer's publish starts invalidating it -
|
|
* which defeats the point for exactly the callers this primitive
|
|
* exists for.
|
|
* - The struct carries no alignment of its own: an embedder on the
|
|
* heap gets whatever malloc gives, so the cursors can start at any
|
|
* offset within a line. The pads are therefore sized so that each
|
|
* cursor and its private copy sit a full line past the end of the
|
|
* previous group, which keeps them on different lines wherever the
|
|
* struct lands.
|
|
*
|
|
* Lifetime:
|
|
* - retro_spsc_init allocates an internal buffer; retro_spsc_free
|
|
* releases it. The caller owns the retro_spsc_t struct itself.
|
|
* - The struct is NOT itself thread-safe to construct or destroy
|
|
* while in use. Build it on one thread, hand the producer pointer
|
|
* to one thread and the consumer pointer to another, then tear
|
|
* down on one thread after both producer and consumer have stopped.
|
|
*
|
|
* Example:
|
|
*
|
|
* retro_spsc_t q;
|
|
* retro_spsc_init(&q, 4096);
|
|
*
|
|
* // Producer thread:
|
|
* uint8_t msg[64] = ...;
|
|
* if (retro_spsc_write_avail(&q) >= sizeof(msg))
|
|
* retro_spsc_write(&q, msg, sizeof(msg));
|
|
*
|
|
* // Consumer thread:
|
|
* uint8_t msg[64];
|
|
* if (retro_spsc_read_avail(&q) >= sizeof(msg))
|
|
* retro_spsc_read(&q, msg, sizeof(msg));
|
|
*
|
|
* retro_spsc_free(&q);
|
|
*
|
|
* Comparison with libretro-common's existing fifo_queue_t:
|
|
* - fifo_queue uses a slock_t internally; safe with multiple producers
|
|
* and multiple consumers, but every push/pop takes a mutex.
|
|
* - retro_spsc is lock-free but limited to one producer and one
|
|
* consumer. Use this when (a) the producer/consumer count is fixed
|
|
* at one each (audio driver -> backend, video -> task system, etc.)
|
|
* and (b) the lock contention in fifo_queue measurably matters.
|
|
* - For most code paths, fifo_queue is the better default. This
|
|
* primitive is for hot paths where lock-free is a measured win.
|
|
*/
|
|
|
|
#include <stddef.h>
|
|
#include <stdint.h>
|
|
#include <boolean.h>
|
|
|
|
#include <retro_common_api.h>
|
|
#include <retro_atomic.h>
|
|
|
|
RETRO_BEGIN_DECLS
|
|
|
|
/* Cache-line padding size. 64 covers x86-64, AArch64, most modern
|
|
* ARMs, and modern PowerPC. Apple Silicon's effective coherency line
|
|
* is 128 due to its M-series cluster topology; over-padding to 64
|
|
* still avoids false sharing, just slightly less efficiently. Older
|
|
* 32-bit ARMs (ARMv6, Cortex-A8) used 32-byte lines but tolerate the
|
|
* larger pad without correctness impact. */
|
|
#ifndef RETRO_SPSC_CACHE_LINE
|
|
#define RETRO_SPSC_CACHE_LINE 64
|
|
#endif
|
|
|
|
/* Padding helper. C89 has no _Static_assert; the array-size trick is
|
|
* portable. We pad to the next multiple of cache-line size after the
|
|
* pointer-and-size_t prefix and after each cursor. */
|
|
|
|
/* Effective cache line for padding. We pad each cursor to a full
|
|
* cache line. If the pre-cursor fields exceed RETRO_SPSC_CACHE_LINE
|
|
* on some hypothetical small-cache target, the underflow on the
|
|
* subtraction would produce a giant array; guard with a max() so
|
|
* the pad is always at least 1 byte and never underflows. */
|
|
#define RETRO_SPSC_PAD0_BYTES \
|
|
((RETRO_SPSC_CACHE_LINE > (sizeof(uint8_t*) + 3 * sizeof(size_t))) \
|
|
? (RETRO_SPSC_CACHE_LINE - (sizeof(uint8_t*) + 3 * sizeof(size_t))) \
|
|
: 1)
|
|
/* Each cursor shares its line with that side's private copy of the
|
|
* other cursor (see cached_tail / cached_head below). A full line of
|
|
* pad after the pair puts the next group a line past the pair's last
|
|
* byte, so the two cannot share a line at any placement of the struct;
|
|
* a line less the pair would leave them sharing one whenever the
|
|
* struct starts late enough in a line. */
|
|
#define RETRO_SPSC_PAD1_BYTES RETRO_SPSC_CACHE_LINE
|
|
|
|
typedef struct retro_spsc
|
|
{
|
|
uint8_t *buffer;
|
|
size_t capacity; /* power of 2: the most the ring holds */
|
|
/* The buffer's size less one, also a power of 2: positions wrap
|
|
* here. Equal to capacity - 1 unless the buffer is mirrored. */
|
|
size_t mask;
|
|
/* The buffer's size when its pages are mapped a second time right
|
|
* behind it, else 0: a span then runs past the end into the
|
|
* mirror, and no reservation is cut short at the wrap. */
|
|
size_t mirror;
|
|
/* Pad so head sits on a fresh cache line, isolating it from
|
|
* the buffer/capacity fields that init may touch. */
|
|
uint8_t _pad0[RETRO_SPSC_PAD0_BYTES];
|
|
retro_atomic_size_t head; /* producer publishes; consumer reads */
|
|
/* Producer-private copy of tail. The producer computes room from
|
|
* this and only re-reads the real tail - the consumer's cache line
|
|
* - when the copy says there is not enough, i.e. once per fill
|
|
* rather than once per write. Always <= tail, so room computed
|
|
* from it is never overstated. Plain size_t: only the producer
|
|
* ever touches it. */
|
|
size_t cached_tail;
|
|
/* Pad so tail sits on its own cache line, isolating it from head. */
|
|
uint8_t _pad1[RETRO_SPSC_PAD1_BYTES];
|
|
retro_atomic_size_t tail; /* consumer publishes; producer reads */
|
|
/* Consumer-private copy of head, the mirror of cached_tail: always
|
|
* <= head, re-read only when it says the ring is short. */
|
|
size_t cached_head;
|
|
/* Pad past tail so an embedding struct's own fields cannot land on
|
|
* tail's line. Same width as _pad1. */
|
|
uint8_t _pad2[RETRO_SPSC_PAD1_BYTES];
|
|
} retro_spsc_t;
|
|
|
|
/**
|
|
* retro_spsc_init:
|
|
* @q : The queue.
|
|
* @min_capacity : Requested capacity in bytes. Rounded up to the
|
|
* next power of 2. Must be > 0 and <= SIZE_MAX/2.
|
|
*
|
|
* Allocates the internal buffer and zero-initialises both cursors.
|
|
* Both cursors begin at 0; the queue is empty.
|
|
*
|
|
* Returns: true on success, false on allocation failure or invalid
|
|
* @min_capacity.
|
|
*
|
|
* After this returns true, the producer thread can call
|
|
* retro_spsc_write / retro_spsc_write_avail and the consumer thread can
|
|
* call retro_spsc_read / retro_spsc_read_avail.
|
|
*/
|
|
bool retro_spsc_init(retro_spsc_t *q, size_t min_capacity);
|
|
|
|
/**
|
|
* retro_spsc_init_mirrored:
|
|
* @q : The queue.
|
|
* @min_capacity : As retro_spsc_init; capacity is the same.
|
|
*
|
|
* As retro_spsc_init, but the buffer's pages are mapped twice, back
|
|
* to back, where the platform allows it, so write_begin and read_begin
|
|
* return every free or filled byte contiguously, across the wrap. The
|
|
* buffer is at least a page (an allocation granule on Windows); a ring
|
|
* smaller than that still holds only @min_capacity rounded up, so
|
|
* capacity, room and fill read exactly as they would from
|
|
* retro_spsc_init. Where mirroring is unavailable the queue is a plain
|
|
* one and q->mirror is 0.
|
|
*/
|
|
bool retro_spsc_init_mirrored(retro_spsc_t *q, size_t min_capacity);
|
|
|
|
/**
|
|
* retro_spsc_free:
|
|
* @q : The queue.
|
|
*
|
|
* Releases the internal buffer. Caller must ensure both producer
|
|
* and consumer have stopped using @q before calling. Safe to call
|
|
* on a queue that retro_spsc_init failed on (no-op).
|
|
*/
|
|
void retro_spsc_free(retro_spsc_t *q);
|
|
|
|
/**
|
|
* retro_spsc_clear:
|
|
* @q : The queue.
|
|
*
|
|
* Resets head and tail to 0, discarding any unread data. The
|
|
* underlying buffer is preserved (no reallocation).
|
|
*
|
|
* SAFETY: callable only when both the producer and consumer are
|
|
* quiesced -- e.g. before either has started, or after both have
|
|
* stopped. Concurrent calls with a live producer or consumer are
|
|
* a data race. This is the same lifetime constraint as
|
|
* retro_spsc_init / retro_spsc_free.
|
|
*
|
|
* Typical use is when a stream is stopped and restarted (e.g. an
|
|
* audio driver pausing and resuming, or switching device formats),
|
|
* and stale buffered data should be discarded before the new
|
|
* stream begins.
|
|
*/
|
|
void retro_spsc_clear(retro_spsc_t *q);
|
|
|
|
/**
|
|
* retro_spsc_grow:
|
|
* @q : The queue.
|
|
* @min_capacity : Requested capacity in bytes, rounded up as for
|
|
* retro_spsc_init.
|
|
*
|
|
* Moves the queue to a larger buffer of the same kind, mirrored or
|
|
* not. The unread data and both cursors carry over unchanged, so a
|
|
* position the caller recorded in the ring still names the same byte.
|
|
* A queue already holding @min_capacity is left as it is.
|
|
*
|
|
* SAFETY: as retro_spsc_clear, both the producer and the consumer must
|
|
* be quiesced, and any span either holds is invalid afterwards.
|
|
*
|
|
* Returns: true on success; false on allocation failure, with the
|
|
* queue untouched.
|
|
*/
|
|
bool retro_spsc_grow(retro_spsc_t *q, size_t min_capacity);
|
|
|
|
/**
|
|
* retro_spsc_grow_into:
|
|
* @q : The queue.
|
|
* @n : A queue from retro_spsc_init or retro_spsc_init_mirrored, empty,
|
|
* at least as large as @q.
|
|
*
|
|
* retro_spsc_grow with the buffer allocated by the caller, so the move
|
|
* itself allocates and frees nothing: @q takes @n's buffer with its
|
|
* unread data and cursors, and @n is left holding @q's old buffer for
|
|
* the caller to retro_spsc_free. The allocation and the free can then
|
|
* be on a thread other than the one that moves.
|
|
*
|
|
* SAFETY: as retro_spsc_grow.
|
|
*
|
|
* Returns: false, with neither queue touched, if @n is the smaller.
|
|
*/
|
|
bool retro_spsc_grow_into(retro_spsc_t *q, retro_spsc_t *n);
|
|
|
|
/**
|
|
* retro_spsc_write_avail:
|
|
* @q : The queue.
|
|
*
|
|
* Producer-side query: how many bytes can be written before the
|
|
* queue is full.
|
|
*
|
|
* Returns: byte count, in [0, capacity].
|
|
*
|
|
* SAFETY: callable only from the producer thread.
|
|
*/
|
|
size_t retro_spsc_write_avail(const retro_spsc_t *q);
|
|
|
|
/**
|
|
* retro_spsc_read_avail:
|
|
* @q : The queue.
|
|
*
|
|
* Consumer-side query: how many bytes are available to read.
|
|
*
|
|
* Returns: byte count, in [0, capacity].
|
|
*
|
|
* SAFETY: callable only from the consumer thread.
|
|
*/
|
|
size_t retro_spsc_read_avail(const retro_spsc_t *q);
|
|
|
|
/**
|
|
* retro_spsc_write:
|
|
* @q : The queue.
|
|
* @data : Source bytes.
|
|
* @bytes : Number of bytes to attempt to write.
|
|
*
|
|
* Writes up to @bytes from @data into the queue. If the queue has
|
|
* less than @bytes free, writes only what fits.
|
|
*
|
|
* Returns: number of bytes actually written.
|
|
*
|
|
* SAFETY: callable only from the producer thread. Concurrent calls
|
|
* with another producer corrupt the queue.
|
|
*/
|
|
size_t retro_spsc_write(retro_spsc_t *q, const void *data, size_t bytes);
|
|
|
|
/* Producer-only whole-frame write; returns frames, not bytes. Free bytes
|
|
* shorter than one frame remain unused. The byte queue capacity need not be
|
|
* divisible by frame_bytes, and frames may wrap across its physical end.
|
|
* Zero frames or frame_bytes writes nothing. As with write(), q must be
|
|
* initialized and data must cover the frames that can be accepted.
|
|
* Consumers must read/skip whole frames to preserve stream boundaries. */
|
|
size_t retro_spsc_write_frames(retro_spsc_t *q, const void *data,
|
|
size_t frames, size_t frame_bytes);
|
|
|
|
/**
|
|
* retro_spsc_read:
|
|
* @q : The queue.
|
|
* @data : Destination bytes.
|
|
* @bytes : Number of bytes to attempt to read.
|
|
*
|
|
* Reads up to @bytes from the queue into @data. If the queue has
|
|
* less than @bytes available, reads only what is present.
|
|
*
|
|
* Returns: number of bytes actually read.
|
|
*
|
|
* SAFETY: callable only from the consumer thread. Concurrent calls
|
|
* with another consumer corrupt the queue.
|
|
*/
|
|
size_t retro_spsc_read(retro_spsc_t *q, void *data, size_t bytes);
|
|
|
|
/**
|
|
* retro_spsc_peek:
|
|
* @q : The queue.
|
|
* @data : Destination bytes.
|
|
* @bytes : Number of bytes to peek.
|
|
*
|
|
* Like retro_spsc_read but does not advance the read cursor. The
|
|
* peeked bytes remain available for the next read.
|
|
*
|
|
* Returns: number of bytes peeked.
|
|
*
|
|
* SAFETY: callable only from the consumer thread.
|
|
*/
|
|
size_t retro_spsc_peek(const retro_spsc_t *q, void *data, size_t bytes);
|
|
|
|
/**
|
|
* retro_spsc_write_begin:
|
|
* @q : The queue.
|
|
* @ptr : Receives a pointer to the contiguous writable region at the
|
|
* current head.
|
|
*
|
|
* Producer-side zero-copy reservation. Returns the number of bytes
|
|
* that can be written CONTIGUOUSLY at *@ptr - this is min(total free
|
|
* space, distance to the physical end of the ring; on a mirrored ring
|
|
* the total free space alone), so it can be
|
|
* smaller than retro_spsc_write_avail() when the head is near the
|
|
* wrap point. No cursor moves; call retro_spsc_write_end() with the
|
|
* number of bytes actually written (which may be 0, and must not
|
|
* exceed the returned span). A caller needing more than the span can
|
|
* fall back to retro_spsc_write(), which handles the split copy.
|
|
*
|
|
* The pointer stays valid until write_end; the consumer cannot enter
|
|
* this region because head has not moved. Alignment of *@ptr is
|
|
* whatever the caller's own write granularity guarantees: the ring
|
|
* is byte-addressed, so producers that always write N-byte records
|
|
* keep every span N-aligned.
|
|
*
|
|
* Returns: contiguous writable byte count (0 when full or when the
|
|
* head sits exactly at the wrap point of a full lap).
|
|
*
|
|
* SAFETY: producer thread only. At most one begin outstanding.
|
|
*/
|
|
size_t retro_spsc_write_begin(retro_spsc_t *q, void **ptr);
|
|
|
|
/**
|
|
* retro_spsc_write_end:
|
|
* @q : The queue.
|
|
* @bytes : Bytes actually written into the span from write_begin.
|
|
*
|
|
* Publishes @bytes with a release-store on head, making them visible
|
|
* to the consumer. Passing 0 abandons the reservation.
|
|
*
|
|
* SAFETY: producer thread only; @bytes must not exceed the span
|
|
* returned by the matching retro_spsc_write_begin.
|
|
*/
|
|
void retro_spsc_write_end(retro_spsc_t *q, size_t bytes);
|
|
|
|
/**
|
|
* retro_spsc_read_begin:
|
|
* @q : The queue.
|
|
* @ptr : Receives a pointer to the contiguous readable region at the
|
|
* current tail.
|
|
*
|
|
* Consumer-side zero-copy drain. Returns the number of bytes
|
|
* readable CONTIGUOUSLY at *@ptr - min(total buffered, distance to
|
|
* the physical end of the ring; on a mirrored ring all of it).
|
|
* Otherwise wrapped data needs a second begin/end round after
|
|
* consuming the first span. No cursor moves;
|
|
* call retro_spsc_read_end() with the bytes actually consumed.
|
|
*
|
|
* The region stays stable until read_end: the producer cannot write
|
|
* into it because tail has not moved.
|
|
*
|
|
* Returns: contiguous readable byte count (0 when empty).
|
|
*
|
|
* SAFETY: consumer thread only. At most one begin outstanding.
|
|
*/
|
|
size_t retro_spsc_read_begin(retro_spsc_t *q, const void **ptr);
|
|
|
|
/**
|
|
* retro_spsc_read_end:
|
|
* @q : The queue.
|
|
* @bytes : Bytes actually consumed from the span from read_begin.
|
|
*
|
|
* Frees @bytes back to the producer with a release-store on tail.
|
|
* Passing 0 abandons the read.
|
|
*
|
|
* SAFETY: consumer thread only; @bytes must not exceed the span
|
|
* returned by the matching retro_spsc_read_begin.
|
|
*/
|
|
void retro_spsc_read_end(retro_spsc_t *q, size_t bytes);
|
|
|
|
/**
|
|
* retro_spsc_skip:
|
|
* @q : The queue.
|
|
* @bytes : Number of bytes to discard.
|
|
*
|
|
* Consumer-side discard: frees up to @bytes back to the producer
|
|
* without copying them anywhere, clamped to what is actually
|
|
* buffered. This is what a consumer that has already inspected a
|
|
* record through retro_spsc_peek or retro_spsc_read_begin needs in
|
|
* order to commit the read later, without re-deriving a span it
|
|
* has already seen.
|
|
*
|
|
* Returns: number of bytes actually discarded.
|
|
*
|
|
* SAFETY: consumer thread only.
|
|
*/
|
|
size_t retro_spsc_skip(retro_spsc_t *q, size_t bytes);
|
|
|
|
RETRO_END_DECLS
|
|
|
|
#endif /* __LIBRETRO_SDK_SPSC_H */
|