Decode reads missed expert blobs from host RAM three ways; the PCIe share's
blobs cross into VRAM anyway and were dropped after the window. With the
switch set, the share takes each layer's most-routed misses and a fetched
expert routed enough more than the layer's least-routed resident expert that
the window does not route is copied straight into that expert's slot
(fetch_blobs gets an optional dst2 target per blob). =2 leaves the tier's own
swaps out of a layer where an admission ran. STRATA_ADAPT_FETCH_VERIFY=1 reads
every admitted slot back. Off, with a log line saying why, where it cannot run.
Unset changes nothing.
Ported from PR 2964 (sergqwer) onto f-0216-core; the strata generate verify
loop was merged by hand (main's speculative-q lines kept) and the DETAILS.md
note became a row of the opt-in switch table.
l620 (RTX PRO 5000, IQ3_S, 3,896-slot cache, --pcie-frac 0.25): decode =2 +10.2%,
=1 +5.0% (4 rounds); token forcing over 14,167 tokens: mismatch 2.73% vs 2.53% /
2.53% for two unset arms (+0.19 pt, 90% CI -0.02..+0.42), PASS.
(cherry picked from commit b2bfaef906acee4f86bdf9f052aee6a4d295fb4e)
SET_DEFAULTS is the one per-(backend, set) table (Metal only; IQ3_S stays on the MTP). Setup, strata launch and strata drafting fetch fetch the tuned pack through the signed feed
(pinned size and SHA-256, resumable, silent offline, the standard drafter and then the MTP as fallbacks). Apache-2.0 licence and the modified-by-Strata notice ship in
data/dflash2-tuned and are copied beside the file. The Metal 'not tested' line is replaced for these sets.
Coder: expert-pruned 256-expert Coder, M5 Max, 5 interleaved pairs, identical greedy tokens in all 40 runs per class, over --spec 2: story -2.5%
(0/5), code +7.1%, JSON +9.3%, agent turn +12.5% (5/5 each). It has no Mac setup entry yet; the name is what a config for it would pass.
fp16 decode (PR 4807): setup writes STRATA_METAL_F16=1 into the config's env for a resident IQ3_S or Q2_0 pack (STRATA_METAL_F16=0 there turns
it off; streaming and the other sets are untouched). The owner's rule was an option unless it passes the 61-task quality suite: off / on / off
again on IQ3_S 54/61 each, on Q2_0 54/61 each, the same 7 tasks failing in all arms, no family drop, no new hallucination flag, 0 pass/fail
changes between two runs of the same arm; decode +14% (IQ3_S) and +15% (Q2_0) in the suite runs, top-1 96-97% and mean KL 0.009-0.016 against fp32
(phaseA-tests/mac-q/RESULTS.md sections 20, 22, 30).
Router: ds4_route and ds4_route_publish left an all-NaN gate row's pick at 0x7fffffff and read sc[0x7fffffff] (threadgroup memory) and
vram[0x7fffffff]; the test process dies with SIGBUS on the old kernel. ds4_route_fallback_pick takes the lowest expert not picked yet
(the CUDA/HIP guard's rule) and raises a flag in a buffer of the Metal host; router_nonfinite_take synchronises and reads it
(metal_host::router_bad_take), so ds4_main ends the request with the error it already has. The warning names the GPU on Metal; the
CUDA/HIP text is unchanged. Test: metal/tests/ds4_route_guard_parity.cpp (finite, all NaN, +-inf, half NaN, fewer than k finite).
Dense Q4_K: ds4_gguf.cu rows() sends a dense matrix of a type metal_host::dequant_rows does not know to iq::dequant_rows, whose CUDA
launch has no Metal dispatch and aborts the engine. ds4_dq_q4k (ggml's dequantize_row_q4_K, the Q5_K kernel without the fifth bit) is
added; any other dense type now fails with a plain message naming it. Test: metal/tests/ds4_dq_parity.cpp (Q4_K, Q5_K, Q6_K, Q8_0, BF16
against ref_quant, padded row stride).
The engine's #879 check only sees logits that come back to the host; the Metal engine picks tokens on the GPU, so a half-precision overflow (STRATA_METAL_F16) would show as wrong text, not as an event. With the variable set, each head window's logits are scanned: a line for the first five windows that held a non-finite value, a count every 500 head windows. Off by default, no cost unset.
Unset, the output is unchanged (the build with the change, unset, gives the same greedy ids as the build without it in 12 of 12 runs, gemv_parity / gr_parity / expert_parity pass). With STRATA_METAL_F16=1 the decode windows of a resident model use half-precision GEMVs (Q2_0, Q3_K-Q6_K, IQ4_XS) and the Q2_0 routed experts.
M5 Max, spec 3 + min-p 0.5, 3 interleaved pairs, decode tok/s unset -> on: Q2_0 story 105.0 -> 114.9 (+9.4%), code 125.6 -> 143.7 (+15.5%), json 130.8 -> 135.1 (+3.3%, one noisy pair), agent 116.1 -> 131.6 (+13.4%); IQ3_S (dense GEMVs only) story 93.1 -> 104.3 (+8.8%), code 103.6 -> 118.5 (+12.5%), json 122.5 -> 135.4 (+10.6%), agent 103.8 -> 117.8 (+12.3%).
It changes arithmetic: teacher-forced over 4,995 positions against the same build unset, Q2_0 top-1 97.12%, mean KL 0.0094 (p99 0.119), perplexity 4.456 against 4.442; IQ3_S top-1 96.16%, mean KL 0.0160 (p99 0.211), perplexity 4.119 against 4.135. The half range is not guarded (see the PR). Off by default.
Window::init accepts 512 or 256 experts a layer; the decode router (router_route / moe_route), the prompt router (route_topk) and the plan buffers use the model's count, the SSD stream reads it from the GGUF. The route kernels are templated on the logits a lane holds (16 for 512, 8 for 256); the 512 path is the old code token for token, the logits rows keep their 512-float stride, the predictor's unused columns are -inf. The MTP layer keeps its own 512 experts.
Tests: prompt_parity route_topk and fuse_parity router_route run both counts (all pass on the M5 Max). IQ3_S 512 experts: greedy ids identical to the build before in every class and prompt (3 interleaved pairs), speed equal (decode +/-0.5% median, prompts -0.5..+5.9%, inside the spread). Not run: an end-to-end 256-expert model (58 GB download).
Niko1221