Squash of SuperDistance's three commits of PR #2050 (decode between the pieces of a read on the pipelined groups, the
piece lines counting the rows the slots decoded meanwhile, the stall watchdog staying on for a read in pieces),
applied after #2326 here: the BYIELD give-way is #2326's give_way lambda, the concurrent block runs first and the
piece path serves a read that STRATA_PIPED_CONCURRENT leaves to it (pictures, configurations where it is off).
Co-authored-by: SuperDistance <[email protected]>
From a review of the concurrent read:
- With external serving batch_poll still finds a window that will not finish (its graph ended or failed with a layer
to ring, or no layer rang for 20 s): the served count is read before the stream query and the ring count after it,
at most one query a millisecond; the last ring's time is an atomic. batch_serve reads a window's row count after its
served count, so a window launched while the doorbell thread was descheduled is not served with the last one's rows.
- #1996's pinned upload rings (0.1.43.1) grew a slot at its first larger upload, a pinned free and allocation on the
read's thread: the drafter's records and the prompt path's routing tables reserve their slots before the read
(PinnedRing::reserve, Prefill::reserve_uploads).
- Off, said once, where serving a doorbell makes a CUDA call (--pcie-mode dma, STRATA_FS_SLOTS, the helper caches) or
the expert source recycles the buffers it hands out (the GGUF in place, unbuffered, STRATA_IO_PREFETCH); a prompt
with pictures is read the usual way. The expert sources' read counters are atomics.
The check is a pure function, concurrent_read_off (batch_read.hpp), with a test in batch_read_test.
- pump's all-idle branch turned the stall watchdog off when the last slot finished mid-read; it stays on while a prompt
is read until the request ends.
- Between windows the doorbell thread slept 50 us, a whole timer tick on Windows (up to 15.6 ms): it now waits on a
condition each launch notifies (STRATA_TEST_BELL_SLEEP_US=N, a test, restores the sleep for an A/B). With nothing
decoding the main thread waits for the read's end or a line instead of sleeping. A pool failure the doorbell thread
set is reported before pump clears the flag for the next window.
- The read's log line ends with the slots' rate and, per stage, its windows and the host time that set each up. The
priority stream's cudaGetLastError runs only under the opt-in.
A prompt read on a layer split with no slot decoding was one run, so the BYIELD of the batch_interleave_test step 4 was
dropped at the end of the read; the long prompt kept its slot and the next BGEN failed with 'ERR BGEN: no such free slot'
(pre-existing since 0.1.40.4). That read now goes in pieces of STRATA_BATCH_YIELD_CHUNKS chunks (default 8, 0 = one run), the
same chunks a single run reads, with the yield check between pieces. batch_read_piece holds the arithmetic; batch_read_test covers it.
