Files
RetroArch/samples/tasks/wait_timeout/wait_timeout_test.c
LibretroAdminandLibretroAdmin 334e8b7baf UI non-blocking, part 3: netplay host init and content close (#19417)
* netplay: bound the wait for tunnel information

Stage 1 of making netplay host init non-blocking: the hang becomes an
error, with no new states and no restructuring.

Hosting through a relay waits for the lobby server to answer with a
tunnel address.  The wait was already narrowed to that one query, but
it was still unbounded - and it is a round trip to a machine we do
not control.  A lobby server that is down, blackholed or merely slow
left the frontend hung with no way out but killing it.  It now gives
up after ten seconds and reports failure like any other query, so the
caller falls back to direct mode with the warning it already has.
Generous on purpose: timing out a query that would have succeeded
costs the user their session, while waiting a few seconds longer
costs only patience.

task_queue_wait_timeout() is the bound.  It wraps the caller's
condition in one that also goes false at a deadline and hands that to
the existing wait, so every queue implementation - regular, threaded
and GCD - keeps its own waiting behaviour and no platform-specific
code is touched.  Adding a variant to the implementation vtable would
have meant editing the GCD path blind, which is the shape of a recent
break and not worth repeating for this.

A timeout alone would have been a bug, not a fix.  The query is still
in flight when we give up, and its callback writes
host_room->mitm_address and mitm_port unconditionally - so it would
land after the fallback and publish a tunnel address for a session
running direct.  Each query now carries a generation token and the
callback ignores results that are no longer current; abandoning a
query bumps the generation.  That also closes the pre-existing case
of two queries overlapping.

The oracle covers the primitive rather than netplay's HTTP, because
that is where the risk is: the netplay side is a few lines routing
into an else branch that already existed.  Five lanes over both queue
implementations assert that a satisfiable wait returns as soon as it
is satisfied - not after sitting out the timeout, which would pass a
naive check - and that an unsatisfiable one gives up near the
deadline rather than early or never.  Removing the deadline hangs the
lane until CI kills it, which is the behaviour being removed.

Verified: builds with networking ENABLED and DISABLED, the latter
being the configuration class that broke recently; task_queue.c links
for ARM32 without NEON; all nine task oracles still pass, since that
file is shared; the new lanes pass plain, under ASan+UBSan+LSan and
twice under TSan.

Not verified: behaviour against a real unresponsive lobby server.

* netplay: split the tunnel query into begin / await / result

Stage 2 of making netplay host init non-blocking: a refactor that
changes no behaviour, so it can be reviewed on its own before
anything moves.

netplay_mitm_query() welded "ask the lobby server" to "wait for the
answer".  Those are now netplay_mitm_query_begin(),
netplay_mitm_query_ready(), netplay_mitm_query_await() and
netplay_mitm_query_result(), with netplay_mitm_query() calling them
back to back at the same call site.  Nothing is deferred and nothing
moves; the point is that the next step - issuing the query when the
user opens the host UI, so the answer is already there when host
setup runs - becomes a change of WHERE begin() is called rather than
a restructuring of host init.

Every path was walked against the original:

  handle NULL/empty       false                 begin() false
  custom, decode fails    false                 begin() false
  custom, decode ok       address set, result   begin(), await(), result()
  http push fails         pending cleared,false begin() false
  http ok, reply in time  wait, then result     await() waits, result()
  http ok, timeout        warn, bump gen, false await() identical

One deliberate difference, because the split exposed an edge the old
shape hid.  The custom-relay path answers synchronously and the old
code returned without ever consulting the pending flag - so an
earlier query left outstanding could not make it wait.  Routed
through await() it would have, so the custom path now supersedes any
query in flight: it bumps the generation and clears the flag.  That
also stops a late callback from overwriting the address it just set,
which the old code would have allowed.

Verified: builds with networking enabled AND disabled; the
wait_timeout CI step replayed verbatim; the netplay NAT, task
ordering and watchdog oracles still pass.

Not verified: behaviour against a real lobby server.

* netplay: start the tunnel query when the user commits to hosting

Stage 3, and the one that removes the stall the user actually feels.

Host setup needs the relay address, and the query for it was issued
at the moment of need - so the wait was the full round trip, every
time.  It is now issued at CMD_EVENT_NETPLAY_ENABLE_HOST, where the
user commits to hosting, which is well before the address is wanted:
on the content-reload path, before content has even started loading.
The round trip overlaps work that was going to happen anyway, so by
the time host setup asks, the answer is usually already there and the
wait costs nothing.

Nothing is deferred and host init is not restructured.  The wait
stays exactly where it was, as the backstop for the case where the
answer has not arrived yet - which is what makes this cheap to
reason about and cheap to revert.

Prefetching only matters if asking twice is avoided, so the query
records the handle it was issued for, and host setup reuses an
answer that is already in hand or on its way.  It deliberately does
NOT reuse a prefetch that came back empty: a transient lobby failure
at prefetch time would otherwise doom a session that asking again
would have got, and one extra request is the better trade.  Every
state was walked:

  no prefetch                query issued as before
  prefetch in flight, same   reused, one query, overlapped
  prefetch answered          reused, no wait at all
  prefetch came back empty   re-issued
  relay changed since        re-issued, picks the new server
  custom relay               answered synchronously, reused

The prefetch is best effort in the strictest sense: skipped when the
relay is off or unconfigured, and if it fails for any reason host
setup asks exactly as it did before.  It is not speculative - the
user has already chosen to host, so no request is made for anyone who
merely opens a menu.

Verified: the call site sits inside #ifdef HAVE_NETWORKING, computed
from the preprocessor nesting rather than eyeballed, and both builds
were made - networking enabled AND disabled, the configuration class
that broke recently.  The wait_timeout CI step was replayed verbatim
from the workflow, and the netplay NAT and task ordering oracles
pass.

Not verified: behaviour against a real lobby server, including
whether the overlap is long enough in practice to hide the round
trip on the content-reload path.

* content close: pin the save wait, and say what it is waiting for

Stages 0 and 1 of unblocking content close.  Nothing about the wait
changes; this establishes what it guarantees and stops it looking
like a hang.

Stage 0, the oracle lane.  Closing content waits for in-flight save
and load tasks because their handlers call into core function
pointers - retro_serialize via content_get_serialized_data,
retro_unserialize via core_unserialize - while
runloop_event_deinit_core() unloads the dylib underneath them.  A
worker mid-call when the library goes away dispatches into freed
code.  That is why this wait, unlike the others removed in this
series, cannot simply be deleted.

The lane pins the property the unload depends on: the file is
checked incomplete before content_wait_for_save_state_task() and
complete after, so a wait that returned early - on a budget
boundary, say - fails here rather than becoming a crash on a device.
Neutering the wait fails it, as it should.

Stage 1, legibility.  The wait cannot be shortened without deferring
the teardown itself, which is later work, but an unexplained frozen
frame and a frame that says "saving state" are very different
experiences - and on slow storage this is seconds, not milliseconds.
The message is pushed and one frame forced out before blocking,
since nothing draws once the wait starts.

It fires only when there is something to wait for, using the same
condition the wait itself uses, so an ordinary close with no save in
flight is untouched and shows nothing.  That predicate had to become
public to be asked from here; its load counterpart already was.

Verified: builds with networking enabled and disabled; the
save_state oracle passes, including the new lane, and fails when the
wait is neutered.

Not verified on a device: whether the message reads well in the case
that matters, which is the slow-storage close it exists for.

* runloop: introduce the content-closing state, observed by nothing

Stage 2 of unblocking content close.  A no-op by construction: the
state is set and cleared around the teardown that already exists, so
while it is true the main thread is inside that teardown and no
frame runs to ask about it.

It is introduced on its own precisely because the work that makes it
observable - returning to the frame loop instead of blocking on the
save wait - is a lifecycle change, and this is the piece the rest
will key off.  Reviewing the two separately means a problem with the
lifecycle change is one revert, not an unpicking.

A plain bool in runloop_state_t rather than a RUNLOOP_FLAG bit.
Bits 0-30 of that word are taken and bit 31 was deliberately vacated
to avoid a cross-thread race with the AppIntents entity query;
reusing it would undo that reasoning for no gain, and this is
main-thread only.

The clear is placed at the block's single exit, which was checked
rather than assumed: there is no early return or goto between the
set and the break, so one clear covers every path.  A leaked true
would be harmless today and would wedge the frame loop at stage 3,
which is exactly the kind of thing worth getting right while it
still cannot bite.

Verified: nothing reads the state except its own accessor, which
nothing calls - grepped rather than asserted, since "no-op" is the
entire claim here; builds with networking enabled and disabled; the
save_state oracle passes and the wait_timeout CI step was replayed
verbatim from the workflow.

* task_queue: fix two data races TSan found in CI

Two separate races, one mine and one older.

Mine, in the oracle: work_pending was cleared by the task HANDLER,
which runs on the worker thread, and read by the waiting main
thread.  A plain bool across two threads - a race on inspection,
with no tooling needed to see it.  It was also a misrepresentation
of the thing under test: the code being modelled,
netplay_mitm_query_cb(), clears its flag in the CALLBACK, which runs
on the thread pumping the queue.  Clearing it there fixes the race
and makes the test match what it claims to model.

The older one is in the queue itself.  retro_task_threaded_reset()
set RETRO_TASK_FLG_CANCELLED under running_lock, while the worker
reads task->flags under property_lock - two locks for one field.
It needed a reset to land while a worker was mid-task, which is why
it sat undisturbed until a lane that cancels an unfinished task
turned up.  The flags are now set under property_lock as well.
Nesting is safe: property_lock is never held while any other lock is
taken anywhere in this file, checked by walking every lock/unlock
pair, so it can only be an inner lock and no cycle exists.

Both are load-bearing, measured rather than assumed: 40 TSan runs
report 0 races with both fixes, and reverting the queue fix alone
brings it back at 6/40.

On the reporting of this: the commit that introduced the oracle said
it passed twice under TSan.  That was true - I ran it, three times
including the CI replay, and it passed each time.  The race
reproduces about once in twelve runs here, so three clean runs were
about a 78% chance of missing it.  The claim was accurate and the
verification was not adequate, which is the more useful way to say
it.  Repeat counts for anything threaded are now sized against how
rare the failure can be, not against how many runs feel like
enough - and a shared mutable flag written from a handler should
have been caught by reading it, which is where this should have
stopped.

Verified: 40 TSan runs clean on the changed lane; 8 each on the
other threaded oracles, since task_queue.c is shared by all of them;
every task oracle passes on a plain build; desktop build clean; the
CI step replayed verbatim from the workflow.

* runloop: don't run the core while content is closing

Stage 3 of unblocking content close.  Still inert, and deliberately
so: it establishes what the frame loop must do while the closing
state is set, before anything can set it with frames running.

retro_run() must not be entered once closing has begun, because the
teardown unloads the library that function lives in.  The guard
therefore has to exist before the wait moves out of the close path,
not with it - otherwise the change that lets frames run would be
introducing both the opportunity and the protection at once.

That it cannot fire today was checked rather than assumed: both
core_run() call sites are inside runloop_iterate(), and the content
close block in CMD_EVENT_CORE_DEINIT reaches neither - it calls no
iterate, no core_run and no run_ahead, so the main thread cannot be
in the frame loop while the flag is set.

It polls input and presents a cached frame rather than returning
bare, matching the netplay-paused case immediately below, and for
the same reason: a frame that simply stops being produced reads as a
hang, which is the whole complaint being worked on here.

Verified: builds with networking enabled and disabled; the
save_state oracle passes, including the lane that pins the close
wait; the wait_timeout CI step replayed verbatim from the workflow.

Worth stating before stage 4: at that point this guard stops being
inert, and video_driver_cached_frame() will be running while a core
is being torn down.  That is the first thing to look at if the frame
loop misbehaves during a close, and it is why the harness lane for
content close belongs before that change rather than after it.

* runloop: fix C89 violation in core_run

The content-closing guard was placed before netplay_preframe's
declaration, so that declaration followed a statement:

  runloop.c:8808: error: ISO C90 forbids mixed declarations and code

netplay_preframe is now declared with the other locals and assigned
where it was, so the guard still returns before netplay's pre-frame
call and the declarations stay at the top of the block.

My own standing rule is that every C file gets the strict C89 gate
before commit, and I did not run it on this one - the ordinary
desktop build does not use those flags, so it compiled clean and I
took that as sufficient.  It is not, and the gate exists precisely
because it is not.

Verified with the flags the C89 build actually uses, on runloop.c
with networking both enabled and disabled, and with a full
C89_BUILD=1 compile of the tree; reinstating the original ordering
reproduces the reported error, so the check is real rather than
vacuous.

* samples: drive the content-closing guard against the real frame loop

Stage 4 will stop blocking on the save wait and let frames run while
a close finishes.  The guard that makes that safe - core_run()
refusing to enter retro_run() while content_closing is set - is
already in place, and is at present completely unexercised: closing
is synchronous, so no frame runs during it, and the guard would go
live in the same change that first creates the situation it protects
against.  If it were wrong, the symptom would be a call into an
unloaded dylib.

So it is driven here first, directly: the real core_run(), in a real
frontend brought up headless, with the flag set.  Links the shipping
objects with only main() replaced; nothing is stubbed.

Three assertions, and the first exists to stop the others passing
vacuously: with the flag CLEAR the core must actually be entered, or
this harness would be proving nothing about a core_run() that never
reaches retro_run() in this environment anyway.  Then with the flag
set it must not be entered, and once cleared it must resume - a
sticky guard would leave the frame loop dead after a close.  A
second lane runs 240 frames with the flag set, the shape of a close
on slow storage, checking core_run() returns cleanly every time
rather than faulting or wedging.

Removing the guard fails both lanes with exactly the message they
exist to produce.

It joins the existing menu-harness workflow, which already makes the
non-Qt build these harnesses need - with Qt enabled ui_qt.o defines
main and the link collides - so it adds about two seconds rather
than another full build.  The step was replayed verbatim from the
workflow before commit, and the menu harness alongside it still
passes.

Verified: strict C89 gate on runloop.c with networking on and off,
which is the check I skipped last time and should not have.

* retroarch: extract the content-close teardown into its own function

Preparation for removing the close freeze, and nothing else: the
sixty lines that run after the save/load wait move into
command_event_finish_content_deinit(), still called from exactly
where they used to run.

They have to become resumable.  The wait above them is the freeze
this work is about, and removing it means this half runs later -
from the frame loop, once the tasks have finished - so it needs to
be callable from somewhere other than the middle of a switch case.
Doing the extraction on its own makes the next change a question of
WHEN it is called rather than what it does, which is the difference
between a revert and an unpicking if that goes wrong.

Every local is re-derived from its accessor rather than passed in,
so the function carries no dependency on the caller's frame - which
is what lets it be called from the frame loop later.

The extracted body was compared statement by statement against the
original block: 53 statements, zero differences.  That check was
worth writing, because the first attempt at this anchored on a
comment that appears twice in the file and silently swallowed an
unrelated case - it failed to compile, but a subtler overshoot would
not have.

Verified: strict C89 gate on retroarch.c with networking on and off;
builds with Qt and headless; both full-program harnesses pass, and
the save_state oracle with them.

---------

Co-authored-by: LibretroAdmin <[email protected]>
2026-08-16 22:11:06 +02:00

231 lines
7.4 KiB
C

/* Oracle for task_queue_wait_timeout().
*
* task_queue_wait() waits indefinitely, which is correct for work
* that is certain to finish and wrong for anything depending on
* something outside the machine. The netplay MITM query is the
* motivating case: it waits on an HTTP round trip to the lobby
* server, and a server that never answers hangs the frontend with no
* way out.
*
* The bound is implemented by wrapping the caller's condition in one
* that also goes false at a deadline, so each queue implementation -
* regular, threaded, GCD - keeps its own waiting behaviour and no
* platform-specific code is involved. What has to hold:
*
* completes - a wait whose work finishes returns as soon as it
* does, reports success, and does NOT sit out the
* remaining timeout.
* times out - a wait whose condition never clears returns anyway,
* reports failure, and takes about the timeout rather
* than forever.
* no-op - a condition already false returns immediately.
* threaded - the same holds on the threaded queue, where the
* waiting is a different implementation.
*
* The timing assertions are deliberately loose at the top end: this
* runs on shared CI machines and the point is "bounded, not
* indefinite", not a precise duration. The lower bounds are tight,
* because returning EARLY would silently break the caller.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <boolean.h>
#include <queues/task_queue.h>
#include <features/features_cpu.h>
#include <retro_timers.h>
static unsigned failures = 0;
#define CHECK(cond, ...) \
do { \
if (!(cond)) \
{ \
fprintf(stderr, "FAIL %s:%d: ", __FILE__, __LINE__); \
fprintf(stderr, __VA_ARGS__); \
fprintf(stderr, "\n"); \
failures++; \
} \
} while (0)
#define TIMEOUT_USEC 300000 /* 0.3s - long enough to measure */
/* ------------------------------------------------------------------ */
/* A task that finishes after N ticks, and one that never finishes */
/* ------------------------------------------------------------------ */
static int ticks_remaining;
static bool work_pending;
/* Handlers run on the WORKER thread under the threaded queue, so
* nothing here touches state the waiting thread reads. The flag is
* cleared in the callback below - which is where the code being
* modelled clears it, netplay_mitm_query_cb(), and callbacks run on
* the thread pumping the queue.
*
* Clearing it here was both a data race and a misrepresentation of
* the thing under test. */
static void finite_handler(retro_task_t *task)
{
if (ticks_remaining > 0)
{
ticks_remaining--;
return;
}
task_set_flags(task, RETRO_TASK_FLG_FINISHED, true);
}
static void endless_handler(retro_task_t *task)
{
/* Models a network wait: the task stays in the queue, and the
* thing the caller is waiting for never arrives. */
}
/* Runs on the thread pumping the queue - the waiting thread - just
* as the netplay query callback does. */
static void cb(retro_task_t *task, void *task_data, void *user_data,
const char *error)
{
work_pending = false;
}
/* The caller's own condition, as a netplay-style "is it still
* pending" flag rather than a queue lookup. */
static bool still_pending(void *data)
{
return work_pending;
}
static bool push(void (*handler)(retro_task_t*))
{
retro_task_t *task = task_init();
if (!task)
return false;
task->handler = handler;
task->callback = cb;
return task_queue_push(task);
}
/* ------------------------------------------------------------------ */
/* Lanes */
/* ------------------------------------------------------------------ */
static void lane_completes_before_timeout(bool threaded)
{
unsigned had = failures;
retro_time_t started, elapsed;
bool ok;
task_queue_init(threaded, NULL);
ticks_remaining = 3;
work_pending = true;
CHECK(push(finite_handler), "push failed");
started = cpu_features_get_time_usec();
ok = task_queue_wait_timeout(still_pending, NULL, TIMEOUT_USEC);
elapsed = cpu_features_get_time_usec() - started;
task_queue_deinit();
CHECK(ok, "%s: reported failure for work that finished",
threaded ? "threaded" : "regular");
CHECK(!work_pending, "%s: work did not actually finish",
threaded ? "threaded" : "regular");
/* The point of the bound is that it does not delay the ordinary
* case; sitting out the full timeout would be a regression that
* still "passes" a pass/fail check. */
CHECK(elapsed < (TIMEOUT_USEC / 2),
"%s: took %lldus for work that finished immediately - the "
"wait is sitting out its timeout instead of returning",
threaded ? "threaded" : "regular", (long long)elapsed);
if (failures == had)
fprintf(stderr, "[pass] completes-before-timeout (%s, %lldus)\n",
threaded ? "threaded" : "regular", (long long)elapsed);
}
static void lane_times_out(bool threaded)
{
unsigned had = failures;
retro_time_t started, elapsed;
bool ok;
task_queue_init(threaded, NULL);
work_pending = true;
CHECK(push(endless_handler), "push failed");
started = cpu_features_get_time_usec();
ok = task_queue_wait_timeout(still_pending, NULL, TIMEOUT_USEC);
elapsed = cpu_features_get_time_usec() - started;
/* Abandon the endless task so deinit is not left holding it. */
task_queue_reset();
task_queue_check();
task_queue_deinit();
CHECK(!ok, "%s: reported success though the work never finished",
threaded ? "threaded" : "regular");
/* Tight lower bound: giving up early would cut short a round trip
* that was about to succeed. */
CHECK(elapsed >= (TIMEOUT_USEC - (TIMEOUT_USEC / 10)),
"%s: gave up after %lldus, well before the %dus timeout",
threaded ? "threaded" : "regular", (long long)elapsed,
TIMEOUT_USEC);
/* Loose upper bound: the machine is shared, the claim is only
* that this terminates rather than hangs. */
CHECK(elapsed < (TIMEOUT_USEC * 10),
"%s: took %lldus against a %dus timeout - not bounded",
threaded ? "threaded" : "regular", (long long)elapsed,
TIMEOUT_USEC);
if (failures == had)
fprintf(stderr, "[pass] times-out (%s, %lldus)\n",
threaded ? "threaded" : "regular", (long long)elapsed);
}
static void lane_already_done(void)
{
unsigned had = failures;
retro_time_t started, elapsed;
bool ok;
task_queue_init(false, NULL);
work_pending = false;
started = cpu_features_get_time_usec();
ok = task_queue_wait_timeout(still_pending, NULL, TIMEOUT_USEC);
elapsed = cpu_features_get_time_usec() - started;
task_queue_deinit();
CHECK(ok, "reported failure for a condition already false");
CHECK(elapsed < (TIMEOUT_USEC / 2),
"waited %lldus for something already finished",
(long long)elapsed);
if (failures == had)
fprintf(stderr, "[pass] already-done lane\n");
}
int main(void)
{
lane_completes_before_timeout(false);
lane_times_out(false);
lane_already_done();
lane_completes_before_timeout(true);
lane_times_out(true);
if (failures)
{
fprintf(stderr, "FAIL wait_timeout_test: %u failures\n", failures);
return 1;
}
fprintf(stderr, "PASS wait_timeout_test\n");
return 0;
}