mirror of
https://github.com/libretro/RetroArch.git
synced 2026-11-09 03:29:43 -10:00
libretro-common: add retro_spsc.h portable lock-free SPSC byte queue
A lock-free single-producer / single-consumer byte queue built on
retro_atomic.h. Wired into the standard libretro-common build
(Makefile.common, griffin) so it ships with every RetroArch
configuration, but no production callers yet -- this commit lands
the primitive only. Subsequent commits can convert hand-rolled
SPSC patterns (audio/drivers/coreaudio.c's atomic_size_t ring,
audio/drivers/opensl.c's __sync_*-guarded buffered_blocks counter)
to use it.
Design
------
Two-cursor (head + tail) Lamport / Disruptor style, NOT the single
shared count. Each cursor is written by exactly one thread and
acquire-loaded by the other. The producer publishes new data with
a release-store on head; the consumer publishes consumed bytes
with a release-store on tail. Neither thread issues atomic RMW
operations on the hot path -- only retro_atomic_load_acquire_size
and retro_atomic_store_release_size, which retro_atomic.h provides
across all 7 backends.
Capacity is power-of-2 (rounded up at init), masked with
capacity - 1. Wraparound uses the standard two-memcpy pattern.
size_t modular arithmetic on (head - tail) is well-defined as long
as the difference stays within capacity, which the producer
enforces by checking write_avail before each push.
head and tail are placed on separate cache lines via explicit
char-array padding (RETRO_SPSC_CACHE_LINE, default 64). Without
this, the producer's release-store on head would invalidate the
consumer's tail cache line and vice versa, halving throughput on
contended SMP. Padding is a performance hint; correctness does
not depend on it. The padding macros are guarded against
underflow if RETRO_SPSC_CACHE_LINE is misconfigured to be smaller
than the prefix fields.
Comparison with the existing fifo_queue_t:
- fifo_queue takes an slock_t internally; it's MPMC-safe but
every push/pop costs a mutex.
- retro_spsc is lock-free but limited to one producer / one
consumer. Use it in code paths where (a) producer/consumer
counts are fixed at one each (audio render thread <-> audio
submission, video thread <-> task queue, etc.) and (b) the
fifo_queue lock contention measurably matters.
Comparison with audio/drivers/coreaudio.c's hand-rolled ring:
- coreaudio uses a single shared atomic_size_t `filled` count
rather than two cursors; producer fetch_add's it, consumer
fetch_sub's it. Correct, but optimises for "give me
write_avail" being a single atomic load (audio drivers do
that query every fill) at the cost of RMW on every push and
pop.
- retro_spsc inverts that trade-off: write_avail / read_avail
each cost two atomic loads, but push and pop only do
load_acquire + store_release. Better for the general case
where push/pop count vastly exceeds availability checks.
- retro_spsc also pads head and tail; coreaudio doesn't (its
single shared atomic doesn't suffer false sharing).
Build wiring
------------
Build requirements:
- retro_atomic.h (header-only, always available)
- <stddef.h>, <stdint.h>, <stdbool.h>, <stdlib.h>, <string.h>
- That's it. No HAVE_THREADS gate is needed for compilation:
retro_spsc.c builds on every target retro_atomic.h supports,
including pre-thread console targets (PSP, original Wii).
On a single-threaded build it just sits there as dead code.
Wired in through:
- Makefile.common adds retro_spsc.o next to fifo_queue.o under
the unconditional libretro-common object list.
- griffin.c #includes retro_spsc.c next to fifo_queue.c (Apple
builds and console unity builds pick it up via griffin).
Test
----
libretro-common/samples/queues/retro_spsc_test/ exercises the
queue under producer/consumer concurrency:
- Property checks (single-threaded): power-of-2 round-up at init,
fresh-queue avails, push/pop content round-trip, peek does not
advance, wraparound across the buffer end.
- SPSC stress (HAVE_THREADS): producer pushes 10M sequential
32-bit tokens through a 4 KB buffer; consumer reads and
verifies each token matches the expected sequence. The
small buffer relative to the message volume forces heavy
interleaving between producer and consumer, exercising the
wraparound path on every iteration.
The stress harness has no per-iteration handshake -- both threads
spin on avail() in tight loops -- so the test is sensitive to
real synchronisation bugs. Verified: deliberately moving the
producer's release-store on head BEFORE the buffer memcpys (so a
consumer observing the new head can read uninitialised bytes)
makes the stress fail with hundreds of mismatched tokens out of
10M, even on x86 TSO without TSan. The same bug under TSan with
halt_on_error=1 exits 66 with "data race in retro_spsc_read_avail".
CI
--
.github/workflows/Linux-libretro-common-samples.yml:
- retro_spsc_test added to RUN_TARGETS so the auto-discovery
job builds and runs it under SANITIZER=address,undefined
(the workflow default).
- A new step builds and runs the test under Clang +
SANITIZER=thread with TSAN_OPTIONS=halt_on_error=1, mirroring
the retro_atomic_test TSan lane. TSan is the strict
validator for this primitive specifically: missing-barrier
regressions show up as data races even on x86 TSO, where the
hardware would otherwise hide them at runtime.
Verified locally:
- gcc -O0/-O2/-O3 -Wall -Werror, x86_64
- clang -O2, x86_64
- g++ -xc++ -std=c++11 (CXX_BUILD-style)
- aarch64-linux-gnu-gcc + qemu-user, 10M tokens clean,
objdump shows real ldar/stlr at the cursor accesses
- arm-linux-gnueabihf-gcc compile-clean
- Forced backends: C11 stdatomic, GCC __sync_*, volatile
fallback (volatile fallback correct only on x86 TSO, same
contract as every other retro_atomic.h caller)
- TSan halt_on_error=1: clean run, exit 0
- TSan halt_on_error=1 on bug-injected build: exit 66
- ASan/UBSan: clean
- Bug-injected without sanitizer: 815 mismatches / 10M tokens
Not verified on real hardware:
- MSVC ARM64 (inherits retro_atomic.h's untested-on-hardware
state for that backend)
- Real PowerPC SMP (Wii U, Xbox 360)
This commit is contained in:
1 parent
b84cdace9c
commit
b894479eeb
7 files changed
+758
No files matched your search
@@ -80,6 +80,7 @@ jobs:
|
||||
task_queue_title_error_test
|
||||
tpool_wait_test
|
||||
retro_atomic_test
|
||||
retro_spsc_test
|
||||
)
|
||||
|
||||
# Per-binary run command (overrides ./<binary> if present).
|
||||
@@ -289,6 +290,31 @@ jobs:
|
||||
|
||||
TSAN_OPTIONS=halt_on_error=1 ./retro_atomic_test
|
||||
|
||||
- name: Run retro_spsc_test under Clang + ThreadSanitizer
|
||||
shell: bash
|
||||
working-directory: libretro-common/samples/queues/retro_spsc_test
|
||||
run: |
|
||||
# retro_spsc.c is a lock-free SPSC byte queue built on
|
||||
# retro_atomic.h. Its correctness contract is acquire-load
|
||||
# / release-store on the head and tail cursors, with the
|
||||
# buffer reads/writes between them ordered by those barriers.
|
||||
# Missing or weakened barriers produce torn data on the
|
||||
# consumer side, observable as content mismatches in the
|
||||
# stress harness AND as TSan-reported races. The default
|
||||
# ASan/UBSan pass above catches the content mismatches but
|
||||
# not the races; this lane catches both.
|
||||
#
|
||||
# halt_on_error=1 makes TSan exit non-zero on the first race
|
||||
# rather than continuing -- which is what we want for CI:
|
||||
# any race here means the SPSC contract is broken.
|
||||
set -u
|
||||
set -o pipefail
|
||||
|
||||
make clean
|
||||
CC=clang make all SANITIZER=thread
|
||||
|
||||
TSAN_OPTIONS=halt_on_error=1 ./retro_spsc_test
|
||||
|
||||
# Cross-architecture validation lane for retro_atomic_test.
|
||||
#
|
||||
# The samples job above runs on x86_64, which is a strongly-ordered
|
||||
|
||||
@@ -364,6 +364,7 @@ OBJ += \
|
||||
input/input_autodetect_builtin.o \
|
||||
input/input_keymaps.o \
|
||||
$(LIBRETRO_COMM_DIR)/queues/fifo_queue.o \
|
||||
$(LIBRETRO_COMM_DIR)/queues/retro_spsc.o \
|
||||
$(LIBRETRO_COMM_DIR)/compat/compat_fnmatch.o \
|
||||
$(LIBRETRO_COMM_DIR)/compat/compat_posix_string.o
|
||||
|
||||
|
||||
@@ -815,6 +815,7 @@ INPUT (HID)
|
||||
FIFO BUFFER
|
||||
============================================================ */
|
||||
#include "../libretro-common/queues/fifo_queue.c"
|
||||
#include "../libretro-common/queues/retro_spsc.c"
|
||||
|
||||
/*============================================================
|
||||
AUDIO RESAMPLER
|
||||
|
||||
@@ -0,0 +1,250 @@
|
||||
/* Copyright (C) 2010-2026 The RetroArch team
|
||||
*
|
||||
* ---------------------------------------------------------------------------------------
|
||||
* The following license statement only applies to this file (retro_spsc.h).
|
||||
* ---------------------------------------------------------------------------------------
|
||||
*
|
||||
* Permission is hereby granted, free of charge,
|
||||
* to any person obtaining a copy of this software and associated documentation files (the "Software"),
|
||||
* to deal in the Software without restriction, including without limitation the rights to
|
||||
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
|
||||
* and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
|
||||
* INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
||||
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#ifndef __LIBRETRO_SDK_SPSC_H
|
||||
#define __LIBRETRO_SDK_SPSC_H
|
||||
|
||||
/*
|
||||
* retro_spsc.h - portable single-producer / single-consumer byte queue
|
||||
*
|
||||
* Lock-free byte-stream queue with one writer thread and one reader
|
||||
* thread. Uses release-store / acquire-load on two cursors (the
|
||||
* Lamport / Disruptor design) rather than a single shared count, so
|
||||
* neither thread issues atomic RMW operations on the hot path.
|
||||
*
|
||||
* Constraints:
|
||||
* - Exactly ONE producer and ONE consumer. No locking is performed;
|
||||
* two producers or two consumers will corrupt the queue.
|
||||
* - Capacity must be power-of-2. Init rounds up.
|
||||
* - Maximum capacity is SIZE_MAX/2 (so head - tail never overflows
|
||||
* the well-defined unsigned modular range).
|
||||
* - The buffer is byte-addressed. Callers pushing structured records
|
||||
* are responsible for framing.
|
||||
*
|
||||
* Memory model:
|
||||
* - Producer writes data into the buffer (non-atomic stores), then
|
||||
* publishes the new head via release-store.
|
||||
* - Consumer acquire-loads head (sees data writes that preceded the
|
||||
* release), reads data (non-atomic loads), then publishes the new
|
||||
* tail via release-store.
|
||||
* - Producer acquire-loads tail to know how much space is free.
|
||||
* - This pairing gives the SPSC invariant on every backend
|
||||
* retro_atomic.h supports lock-freely. On the volatile fallback
|
||||
* (no real backend), correctness reduces to single-core or x86 TSO,
|
||||
* same as every other retro_atomic.h caller.
|
||||
*
|
||||
* Cache behaviour:
|
||||
* - head and tail are placed on separate cache lines via explicit
|
||||
* padding to RETRO_SPSC_CACHE_LINE. Without this, the producer's
|
||||
* publish would invalidate the consumer's tail line and vice versa,
|
||||
* halving throughput on contended SMP. The padding is a
|
||||
* performance hint; correctness does not depend on it.
|
||||
*
|
||||
* Lifetime:
|
||||
* - retro_spsc_init allocates an internal buffer; retro_spsc_free
|
||||
* releases it. The caller owns the retro_spsc_t struct itself.
|
||||
* - The struct is NOT itself thread-safe to construct or destroy
|
||||
* while in use. Build it on one thread, hand the producer pointer
|
||||
* to one thread and the consumer pointer to another, then tear
|
||||
* down on one thread after both producer and consumer have stopped.
|
||||
*
|
||||
* Example:
|
||||
*
|
||||
* retro_spsc_t q;
|
||||
* retro_spsc_init(&q, 4096);
|
||||
*
|
||||
* // Producer thread:
|
||||
* uint8_t msg[64] = ...;
|
||||
* if (retro_spsc_write_avail(&q) >= sizeof(msg))
|
||||
* retro_spsc_write(&q, msg, sizeof(msg));
|
||||
*
|
||||
* // Consumer thread:
|
||||
* uint8_t msg[64];
|
||||
* if (retro_spsc_read_avail(&q) >= sizeof(msg))
|
||||
* retro_spsc_read(&q, msg, sizeof(msg));
|
||||
*
|
||||
* retro_spsc_free(&q);
|
||||
*
|
||||
* Comparison with libretro-common's existing fifo_queue_t:
|
||||
* - fifo_queue uses a slock_t internally; safe with multiple producers
|
||||
* and multiple consumers, but every push/pop takes a mutex.
|
||||
* - retro_spsc is lock-free but limited to one producer and one
|
||||
* consumer. Use this when (a) the producer/consumer count is fixed
|
||||
* at one each (audio driver -> backend, video -> task system, etc.)
|
||||
* and (b) the lock contention in fifo_queue measurably matters.
|
||||
* - For most code paths, fifo_queue is the better default. This
|
||||
* primitive is for hot paths where lock-free is a measured win.
|
||||
*/
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
#include <stdbool.h>
|
||||
|
||||
#include <retro_common_api.h>
|
||||
#include <retro_atomic.h>
|
||||
|
||||
RETRO_BEGIN_DECLS
|
||||
|
||||
/* Cache-line padding size. 64 covers x86-64, AArch64, most modern
|
||||
* ARMs, and modern PowerPC. Apple Silicon's effective coherency line
|
||||
* is 128 due to its M-series cluster topology; over-padding to 64
|
||||
* still avoids false sharing, just slightly less efficiently. Older
|
||||
* 32-bit ARMs (ARMv6, Cortex-A8) used 32-byte lines but tolerate the
|
||||
* larger pad without correctness impact. */
|
||||
#ifndef RETRO_SPSC_CACHE_LINE
|
||||
#define RETRO_SPSC_CACHE_LINE 64
|
||||
#endif
|
||||
|
||||
/* Padding helper. C89 has no _Static_assert; the array-size trick is
|
||||
* portable. We pad to the next multiple of cache-line size after the
|
||||
* pointer-and-size_t prefix and after each cursor. */
|
||||
|
||||
/* Effective cache line for padding. We pad each cursor to a full
|
||||
* cache line. If the pre-cursor fields exceed RETRO_SPSC_CACHE_LINE
|
||||
* on some hypothetical small-cache target, the underflow on the
|
||||
* subtraction would produce a giant array; guard with a max() so
|
||||
* the pad is always at least 1 byte and never underflows. */
|
||||
#define RETRO_SPSC_PAD0_BYTES \
|
||||
((RETRO_SPSC_CACHE_LINE > (sizeof(uint8_t*) + sizeof(size_t))) \
|
||||
? (RETRO_SPSC_CACHE_LINE - (sizeof(uint8_t*) + sizeof(size_t))) \
|
||||
: 1)
|
||||
#define RETRO_SPSC_PAD1_BYTES \
|
||||
((RETRO_SPSC_CACHE_LINE > sizeof(retro_atomic_size_t)) \
|
||||
? (RETRO_SPSC_CACHE_LINE - sizeof(retro_atomic_size_t)) \
|
||||
: 1)
|
||||
|
||||
typedef struct retro_spsc
|
||||
{
|
||||
uint8_t *buffer;
|
||||
size_t capacity; /* power of 2; mask = capacity - 1 */
|
||||
/* Pad so head sits on a fresh cache line, isolating it from
|
||||
* the buffer/capacity fields that init may touch. */
|
||||
uint8_t _pad0[RETRO_SPSC_PAD0_BYTES];
|
||||
retro_atomic_size_t head; /* producer publishes; consumer reads */
|
||||
/* Pad so tail sits on its own cache line, isolating it from head. */
|
||||
uint8_t _pad1[RETRO_SPSC_PAD1_BYTES];
|
||||
retro_atomic_size_t tail; /* consumer publishes; producer reads */
|
||||
} retro_spsc_t;
|
||||
|
||||
/**
|
||||
* retro_spsc_init:
|
||||
* @q : The queue.
|
||||
* @min_capacity : Requested capacity in bytes. Rounded up to the
|
||||
* next power of 2. Must be > 0 and <= SIZE_MAX/2.
|
||||
*
|
||||
* Allocates the internal buffer and zero-initialises both cursors.
|
||||
* Both cursors begin at 0; the queue is empty.
|
||||
*
|
||||
* Returns: true on success, false on allocation failure or invalid
|
||||
* @min_capacity.
|
||||
*
|
||||
* After this returns true, the producer thread can call
|
||||
* retro_spsc_write / retro_spsc_write_avail and the consumer thread can
|
||||
* call retro_spsc_read / retro_spsc_read_avail.
|
||||
*/
|
||||
bool retro_spsc_init(retro_spsc_t *q, size_t min_capacity);
|
||||
|
||||
/**
|
||||
* retro_spsc_free:
|
||||
* @q : The queue.
|
||||
*
|
||||
* Releases the internal buffer. Caller must ensure both producer
|
||||
* and consumer have stopped using @q before calling. Safe to call
|
||||
* on a queue that retro_spsc_init failed on (no-op).
|
||||
*/
|
||||
void retro_spsc_free(retro_spsc_t *q);
|
||||
|
||||
/**
|
||||
* retro_spsc_write_avail:
|
||||
* @q : The queue.
|
||||
*
|
||||
* Producer-side query: how many bytes can be written before the
|
||||
* queue is full.
|
||||
*
|
||||
* Returns: byte count, in [0, capacity].
|
||||
*
|
||||
* SAFETY: callable only from the producer thread.
|
||||
*/
|
||||
size_t retro_spsc_write_avail(const retro_spsc_t *q);
|
||||
|
||||
/**
|
||||
* retro_spsc_read_avail:
|
||||
* @q : The queue.
|
||||
*
|
||||
* Consumer-side query: how many bytes are available to read.
|
||||
*
|
||||
* Returns: byte count, in [0, capacity].
|
||||
*
|
||||
* SAFETY: callable only from the consumer thread.
|
||||
*/
|
||||
size_t retro_spsc_read_avail(const retro_spsc_t *q);
|
||||
|
||||
/**
|
||||
* retro_spsc_write:
|
||||
* @q : The queue.
|
||||
* @data : Source bytes.
|
||||
* @bytes : Number of bytes to attempt to write.
|
||||
*
|
||||
* Writes up to @bytes from @data into the queue. If the queue has
|
||||
* less than @bytes free, writes only what fits.
|
||||
*
|
||||
* Returns: number of bytes actually written.
|
||||
*
|
||||
* SAFETY: callable only from the producer thread. Concurrent calls
|
||||
* with another producer corrupt the queue.
|
||||
*/
|
||||
size_t retro_spsc_write(retro_spsc_t *q, const void *data, size_t bytes);
|
||||
|
||||
/**
|
||||
* retro_spsc_read:
|
||||
* @q : The queue.
|
||||
* @data : Destination bytes.
|
||||
* @bytes : Number of bytes to attempt to read.
|
||||
*
|
||||
* Reads up to @bytes from the queue into @data. If the queue has
|
||||
* less than @bytes available, reads only what is present.
|
||||
*
|
||||
* Returns: number of bytes actually read.
|
||||
*
|
||||
* SAFETY: callable only from the consumer thread. Concurrent calls
|
||||
* with another consumer corrupt the queue.
|
||||
*/
|
||||
size_t retro_spsc_read(retro_spsc_t *q, void *data, size_t bytes);
|
||||
|
||||
/**
|
||||
* retro_spsc_peek:
|
||||
* @q : The queue.
|
||||
* @data : Destination bytes.
|
||||
* @bytes : Number of bytes to peek.
|
||||
*
|
||||
* Like retro_spsc_read but does not advance the read cursor. The
|
||||
* peeked bytes remain available for the next read.
|
||||
*
|
||||
* Returns: number of bytes peeked.
|
||||
*
|
||||
* SAFETY: callable only from the consumer thread.
|
||||
*/
|
||||
size_t retro_spsc_peek(const retro_spsc_t *q, void *data, size_t bytes);
|
||||
|
||||
RETRO_END_DECLS
|
||||
|
||||
#endif /* __LIBRETRO_SDK_SPSC_H */
|
||||
@@ -0,0 +1,198 @@
|
||||
/* Copyright (C) 2010-2026 The RetroArch team
|
||||
*
|
||||
* ---------------------------------------------------------------------------------------
|
||||
* The following license statement only applies to this file (retro_spsc.c).
|
||||
* ---------------------------------------------------------------------------------------
|
||||
*
|
||||
* Permission is hereby granted, free of charge,
|
||||
* to any person obtaining a copy of this software and associated documentation files (the "Software"),
|
||||
* to deal in the Software without restriction, including without limitation the rights to
|
||||
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
|
||||
* and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
|
||||
* INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
||||
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <retro_spsc.h>
|
||||
|
||||
/* Round @v up to the next power of 2. Returns 0 if @v == 0 or if
|
||||
* the next power of 2 would overflow (caller checks @v <= SIZE_MAX/2). */
|
||||
static size_t spsc_round_up_pow2(size_t v)
|
||||
{
|
||||
size_t r;
|
||||
if (v == 0)
|
||||
return 0;
|
||||
/* If already a power of 2, return as-is. */
|
||||
if ((v & (v - 1)) == 0)
|
||||
return v;
|
||||
/* Round up. Loop terminates because v <= SIZE_MAX/2 means r
|
||||
* cannot overflow. */
|
||||
r = 1;
|
||||
while (r < v)
|
||||
r <<= 1;
|
||||
return r;
|
||||
}
|
||||
|
||||
bool retro_spsc_init(retro_spsc_t *q, size_t min_capacity)
|
||||
{
|
||||
size_t cap;
|
||||
|
||||
if (!q || min_capacity == 0 || min_capacity > (SIZE_MAX / 2))
|
||||
return false;
|
||||
|
||||
cap = spsc_round_up_pow2(min_capacity);
|
||||
if (cap == 0 || cap > (SIZE_MAX / 2))
|
||||
return false;
|
||||
|
||||
q->buffer = (uint8_t*)malloc(cap);
|
||||
if (!q->buffer)
|
||||
return false;
|
||||
|
||||
q->capacity = cap;
|
||||
retro_atomic_size_init(&q->head, 0);
|
||||
retro_atomic_size_init(&q->tail, 0);
|
||||
return true;
|
||||
}
|
||||
|
||||
void retro_spsc_free(retro_spsc_t *q)
|
||||
{
|
||||
if (!q)
|
||||
return;
|
||||
if (q->buffer)
|
||||
{
|
||||
free(q->buffer);
|
||||
q->buffer = NULL;
|
||||
}
|
||||
q->capacity = 0;
|
||||
}
|
||||
|
||||
size_t retro_spsc_write_avail(const retro_spsc_t *q)
|
||||
{
|
||||
/* Producer query. We read our own head (which we wrote) and the
|
||||
* consumer's tail (which they wrote with a release-store).
|
||||
* acquire-load on tail so any reads we do based on the freed
|
||||
* space are ordered after the consumer's tail publication.
|
||||
*
|
||||
* We can read our own head without an acquire because we are the
|
||||
* only writer to it; but retro_atomic.h does not expose relaxed
|
||||
* loads, so use acquire. Cost is one extra fence on weak-memory
|
||||
* targets, which is negligible compared to the actual work
|
||||
* surrounding this query. */
|
||||
size_t head = retro_atomic_load_acquire_size(
|
||||
(retro_atomic_size_t*)&q->head);
|
||||
size_t tail = retro_atomic_load_acquire_size(
|
||||
(retro_atomic_size_t*)&q->tail);
|
||||
/* head - tail is well-defined modular arithmetic on size_t.
|
||||
* Invariant: 0 <= head - tail <= capacity. */
|
||||
return q->capacity - (head - tail);
|
||||
}
|
||||
|
||||
size_t retro_spsc_read_avail(const retro_spsc_t *q)
|
||||
{
|
||||
/* Consumer query. acquire-load on head pairs with the producer's
|
||||
* release-store on head, so the data writes that preceded the
|
||||
* release are visible by the time we read them. */
|
||||
size_t head = retro_atomic_load_acquire_size(
|
||||
(retro_atomic_size_t*)&q->head);
|
||||
size_t tail = retro_atomic_load_acquire_size(
|
||||
(retro_atomic_size_t*)&q->tail);
|
||||
return head - tail;
|
||||
}
|
||||
|
||||
size_t retro_spsc_write(retro_spsc_t *q, const void *data, size_t bytes)
|
||||
{
|
||||
const uint8_t *src = (const uint8_t*)data;
|
||||
size_t avail, head, tail, mask, head_idx, first;
|
||||
|
||||
/* read tail first to know how much room there is */
|
||||
head = retro_atomic_load_acquire_size(&q->head);
|
||||
tail = retro_atomic_load_acquire_size(&q->tail);
|
||||
avail = q->capacity - (head - tail);
|
||||
if (bytes > avail)
|
||||
bytes = avail;
|
||||
if (bytes == 0)
|
||||
return 0;
|
||||
|
||||
mask = q->capacity - 1;
|
||||
head_idx = head & mask;
|
||||
|
||||
/* first = bytes from head_idx to end-of-buffer */
|
||||
first = q->capacity - head_idx;
|
||||
if (first > bytes)
|
||||
first = bytes;
|
||||
|
||||
memcpy(q->buffer + head_idx, src, first);
|
||||
memcpy(q->buffer, src + first, bytes - first);
|
||||
|
||||
/* Publish: release-store ensures the memcpys above are globally
|
||||
* visible before the consumer observes the new head. */
|
||||
retro_atomic_store_release_size(&q->head, head + bytes);
|
||||
return bytes;
|
||||
}
|
||||
|
||||
size_t retro_spsc_read(retro_spsc_t *q, void *data, size_t bytes)
|
||||
{
|
||||
uint8_t *dst = (uint8_t*)data;
|
||||
size_t avail, head, tail, mask, tail_idx, first;
|
||||
|
||||
/* acquire on head pairs with producer's release-store; this is
|
||||
* what makes the subsequent memcpys safe to read. */
|
||||
head = retro_atomic_load_acquire_size(&q->head);
|
||||
tail = retro_atomic_load_acquire_size(&q->tail);
|
||||
avail = head - tail;
|
||||
if (bytes > avail)
|
||||
bytes = avail;
|
||||
if (bytes == 0)
|
||||
return 0;
|
||||
|
||||
mask = q->capacity - 1;
|
||||
tail_idx = tail & mask;
|
||||
|
||||
first = q->capacity - tail_idx;
|
||||
if (first > bytes)
|
||||
first = bytes;
|
||||
|
||||
memcpy(dst, q->buffer + tail_idx, first);
|
||||
memcpy(dst + first, q->buffer, bytes - first);
|
||||
|
||||
/* Publish: release-store so the producer can re-use this space. */
|
||||
retro_atomic_store_release_size(&q->tail, tail + bytes);
|
||||
return bytes;
|
||||
}
|
||||
|
||||
size_t retro_spsc_peek(const retro_spsc_t *q, void *data, size_t bytes)
|
||||
{
|
||||
uint8_t *dst = (uint8_t*)data;
|
||||
size_t avail, head, tail, mask, tail_idx, first;
|
||||
|
||||
head = retro_atomic_load_acquire_size(
|
||||
(retro_atomic_size_t*)&q->head);
|
||||
tail = retro_atomic_load_acquire_size(
|
||||
(retro_atomic_size_t*)&q->tail);
|
||||
avail = head - tail;
|
||||
if (bytes > avail)
|
||||
bytes = avail;
|
||||
if (bytes == 0)
|
||||
return 0;
|
||||
|
||||
mask = q->capacity - 1;
|
||||
tail_idx = tail & mask;
|
||||
|
||||
first = q->capacity - tail_idx;
|
||||
if (first > bytes)
|
||||
first = bytes;
|
||||
|
||||
memcpy(dst, q->buffer + tail_idx, first);
|
||||
memcpy(dst + first, q->buffer, bytes - first);
|
||||
/* No tail update: peek does not consume. */
|
||||
return bytes;
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
TARGET := retro_spsc_test
|
||||
|
||||
LIBRETRO_COMM_DIR := ../../..
|
||||
|
||||
# retro_spsc.c is the lock-free SPSC byte queue under test; the test
|
||||
# harness drives a producer and consumer thread through rthreads.c
|
||||
# (HAVE_THREADS). retro_atomic.h is header-only and pulled in via
|
||||
# the queue's transitive includes.
|
||||
#
|
||||
# Build under SANITIZER=thread (TSan) for the regression-catching
|
||||
# CI run; SANITIZER=address,undefined as the default for the
|
||||
# auto-discovery workflow gives a coarser smoke test. Both pass on
|
||||
# correct code.
|
||||
SOURCES := \
|
||||
retro_spsc_test.c \
|
||||
$(LIBRETRO_COMM_DIR)/queues/retro_spsc.c \
|
||||
$(LIBRETRO_COMM_DIR)/rthreads/rthreads.c
|
||||
|
||||
OBJS := $(SOURCES:.c=.o)
|
||||
|
||||
CFLAGS += -Wall -pedantic -std=gnu99 -g -O0 \
|
||||
-DHAVE_THREADS -I$(LIBRETRO_COMM_DIR)/include
|
||||
LDFLAGS += -lpthread
|
||||
# rthreads.c uses clock_gettime + CLOCK_REALTIME on Linux glibc; on
|
||||
# older glibc those live in -lrt. Harmless on newer glibc.
|
||||
LDFLAGS += -lrt
|
||||
|
||||
ifneq ($(SANITIZER),)
|
||||
CFLAGS := -fsanitize=$(SANITIZER) -fno-omit-frame-pointer $(CFLAGS)
|
||||
LDFLAGS := -fsanitize=$(SANITIZER) $(LDFLAGS)
|
||||
endif
|
||||
|
||||
all: $(TARGET)
|
||||
|
||||
%.o: %.c
|
||||
$(CC) -c -o $@ $< $(CFLAGS)
|
||||
|
||||
$(TARGET): $(OBJS)
|
||||
$(CC) -o $@ $^ $(LDFLAGS)
|
||||
|
||||
clean:
|
||||
rm -f $(TARGET) $(OBJS)
|
||||
|
||||
.PHONY: clean
|
||||
@@ -0,0 +1,238 @@
|
||||
/* SPSC stress test: producer pushes N bytes total; consumer reads N
|
||||
* bytes; verify the byte stream is exactly an expected sequence.
|
||||
* Validates ordering, no torn reads, no duplicates, no drops.
|
||||
*
|
||||
* Design notes:
|
||||
* - Producer writes incrementing 32-bit "tokens" (i = 1, 2, 3, ...).
|
||||
* - Consumer reads the byte stream and reassembles tokens.
|
||||
* - Each token is checked against expected sequence; mismatch =>
|
||||
* reordering or torn write.
|
||||
* - Producer/consumer race in tight loops with no per-iteration
|
||||
* handshake; a real lock-free SPSC must handle this concurrency
|
||||
* without external synchronisation.
|
||||
* - Run under TSan for race detection. Run without sanitizer for
|
||||
* throughput sanity. */
|
||||
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
#include <retro_spsc.h>
|
||||
#include <rthreads/rthreads.h>
|
||||
|
||||
/* Total tokens to push. ~10M gives a reproducible test of <2s on
|
||||
* x86_64; under qemu-aarch64 and TSan it's ~30s. */
|
||||
#define TOTAL_TOKENS 10000000
|
||||
|
||||
/* Buffer capacity in bytes. Smaller -> more producer/consumer
|
||||
* interleaving (better race coverage); larger -> better throughput
|
||||
* but less torture. */
|
||||
#define BUF_BYTES 4096
|
||||
|
||||
typedef struct
|
||||
{
|
||||
retro_spsc_t q;
|
||||
/* Captured by the producer's loop sentinel and read by main after
|
||||
* join, so it doesn't need to be atomic. */
|
||||
unsigned long mismatches;
|
||||
unsigned long produced_tokens;
|
||||
unsigned long consumed_tokens;
|
||||
} test_state_t;
|
||||
|
||||
static void producer_thread(void *arg)
|
||||
{
|
||||
test_state_t *s = (test_state_t*)arg;
|
||||
uint32_t token;
|
||||
|
||||
for (token = 1; token <= TOTAL_TOKENS; token++)
|
||||
{
|
||||
/* Spin until there is room for a full token. */
|
||||
while (retro_spsc_write_avail(&s->q) < sizeof(token))
|
||||
; /* spin */
|
||||
retro_spsc_write(&s->q, &token, sizeof(token));
|
||||
s->produced_tokens++;
|
||||
}
|
||||
}
|
||||
|
||||
static void consumer_thread(void *arg)
|
||||
{
|
||||
test_state_t *s = (test_state_t*)arg;
|
||||
uint32_t expected_token = 1;
|
||||
|
||||
while (expected_token <= TOTAL_TOKENS)
|
||||
{
|
||||
uint32_t got;
|
||||
while (retro_spsc_read_avail(&s->q) < sizeof(got))
|
||||
; /* spin */
|
||||
retro_spsc_read(&s->q, &got, sizeof(got));
|
||||
if (got != expected_token)
|
||||
s->mismatches++;
|
||||
expected_token++;
|
||||
s->consumed_tokens++;
|
||||
}
|
||||
}
|
||||
|
||||
static int run_stress(void)
|
||||
{
|
||||
test_state_t s;
|
||||
sthread_t *prod;
|
||||
sthread_t *cons;
|
||||
|
||||
memset(&s, 0, sizeof(s));
|
||||
if (!retro_spsc_init(&s.q, BUF_BYTES))
|
||||
{
|
||||
fprintf(stderr, "FAIL: retro_spsc_init\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
prod = sthread_create(producer_thread, &s);
|
||||
if (!prod)
|
||||
{
|
||||
fprintf(stderr, "FAIL: sthread_create(producer)\n");
|
||||
retro_spsc_free(&s.q);
|
||||
return 1;
|
||||
}
|
||||
cons = sthread_create(consumer_thread, &s);
|
||||
if (!cons)
|
||||
{
|
||||
fprintf(stderr, "FAIL: sthread_create(consumer)\n");
|
||||
sthread_join(prod);
|
||||
retro_spsc_free(&s.q);
|
||||
return 1;
|
||||
}
|
||||
|
||||
sthread_join(prod);
|
||||
sthread_join(cons);
|
||||
retro_spsc_free(&s.q);
|
||||
|
||||
if (s.mismatches != 0)
|
||||
{
|
||||
fprintf(stderr,
|
||||
"FAIL: %lu mismatched tokens out of %lu\n",
|
||||
s.mismatches, s.consumed_tokens);
|
||||
return 1;
|
||||
}
|
||||
if (s.produced_tokens != TOTAL_TOKENS
|
||||
|| s.consumed_tokens != TOTAL_TOKENS)
|
||||
{
|
||||
fprintf(stderr,
|
||||
"FAIL: produced=%lu consumed=%lu (expected %d each)\n",
|
||||
s.produced_tokens, s.consumed_tokens, TOTAL_TOKENS);
|
||||
return 1;
|
||||
}
|
||||
|
||||
printf("[pass] stress: %d tokens through %d-byte buffer, "
|
||||
"0 mismatches\n",
|
||||
TOTAL_TOKENS, BUF_BYTES);
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Single-threaded property checks: verify the API contracts that
|
||||
* don't require concurrency. */
|
||||
static int run_property_checks(void)
|
||||
{
|
||||
retro_spsc_t q;
|
||||
uint8_t buf[64];
|
||||
uint8_t readback[64];
|
||||
size_t i, n;
|
||||
|
||||
/* init with non-power-of-2 should round up */
|
||||
if (!retro_spsc_init(&q, 100))
|
||||
{
|
||||
fprintf(stderr, "FAIL: init(100)\n");
|
||||
return 1;
|
||||
}
|
||||
if (q.capacity != 128)
|
||||
{
|
||||
fprintf(stderr, "FAIL: capacity %zu != 128 (round up)\n",
|
||||
q.capacity);
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* fresh queue: read_avail = 0, write_avail = capacity */
|
||||
if (retro_spsc_read_avail(&q) != 0
|
||||
|| retro_spsc_write_avail(&q) != 128)
|
||||
{
|
||||
fprintf(stderr, "FAIL: fresh-queue avails\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* write 64, read 64, verify */
|
||||
for (i = 0; i < sizeof(buf); i++)
|
||||
buf[i] = (uint8_t)(i + 1);
|
||||
n = retro_spsc_write(&q, buf, sizeof(buf));
|
||||
if (n != sizeof(buf))
|
||||
{
|
||||
fprintf(stderr, "FAIL: write returned %zu\n", n);
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
if (retro_spsc_read_avail(&q) != sizeof(buf))
|
||||
{
|
||||
fprintf(stderr, "FAIL: read_avail after write\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
memset(readback, 0, sizeof(readback));
|
||||
n = retro_spsc_read(&q, readback, sizeof(readback));
|
||||
if (n != sizeof(buf) || memcmp(buf, readback, sizeof(buf)) != 0)
|
||||
{
|
||||
fprintf(stderr, "FAIL: read content mismatch\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* peek does not advance */
|
||||
retro_spsc_write(&q, buf, 32);
|
||||
if (retro_spsc_peek(&q, readback, 32) != 32)
|
||||
{
|
||||
fprintf(stderr, "FAIL: peek returned wrong size\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
if (retro_spsc_read_avail(&q) != 32)
|
||||
{
|
||||
fprintf(stderr, "FAIL: peek advanced read cursor\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
/* drain so wraparound test starts from a known state */
|
||||
retro_spsc_read(&q, readback, 32);
|
||||
|
||||
/* wraparound: write 100 (forces wrap given capacity 128 + 32-byte
|
||||
* peek state above) */
|
||||
for (i = 0; i < sizeof(buf); i++)
|
||||
buf[i] = (uint8_t)(0x80 + i);
|
||||
n = retro_spsc_write(&q, buf, sizeof(buf));
|
||||
if (n != sizeof(buf))
|
||||
{
|
||||
fprintf(stderr, "FAIL: wraparound write\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
memset(readback, 0, sizeof(readback));
|
||||
n = retro_spsc_read(&q, readback, sizeof(buf));
|
||||
if (n != sizeof(buf) || memcmp(buf, readback, sizeof(buf)) != 0)
|
||||
{
|
||||
fprintf(stderr, "FAIL: wraparound read content mismatch\n");
|
||||
retro_spsc_free(&q);
|
||||
return 1;
|
||||
}
|
||||
|
||||
retro_spsc_free(&q);
|
||||
printf("[pass] property checks\n");
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(void)
|
||||
{
|
||||
if (run_property_checks() != 0)
|
||||
return 1;
|
||||
if (run_stress() != 0)
|
||||
return 1;
|
||||
puts("ALL OK");
|
||||
return 0;
|
||||
}
|
||||
Reference in new issue
Block a user