docs: STRATA_GRAPH_PRIO and STRATA_MMVQ_IL_PF (sm_80, opt-in decode switches)

DETAILS.md gets a section with the two switches, what each does, that both are bit-exact, and their decode gain on
the CMP 170HX (+1.8 % with STRATA_QIL=1, +1.4 %). MMVQ_IL_TABLE.md gets a paragraph on the kIlPf_80 / kIlPf58_80
tables (variant << 4 | rows, Q6_K only) and on measuring a card with mmvq_il_parity --bench --emit-table --pf 0,3,4.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
linxuhaoandClaude Opus 5.5 committed 2026-10-22 20:53:14 +00:00
1 parent 116992ab8a
commit 6d6f0f39fd
2 files changed
+26

No files matched your search

+11
View File
@@ -2563,3 +2563,14 @@ CUDA card, where it measured +1..3 % prompt speed on a Tesla P100 at 30-60K toke
Options that change answers (other summation order) are separate and explicit: `STRATA_SELECT_SIMT=1|2` (FP32 tiled block scorer,
KL 0.01 against the warp scorer per #2071), `STRATA_PA_V2T=1` (v2 prompt attention, #2092/#2146), `STRATA_PF_FUSED=1
STRATA_PF_FUSED_SM75=1` (fused int8 prompt experts for the Q2_0 pack, #2145; only on a 7.5 card, sm_80+ is unchanged), `STRATA_BF16_TC=1` with `STRATA_XN_F16` (#2091).
## CUDA opt-in decode switches measured on a CMP 170HX (sm_80; off unless you set them)
Measured on one CMP 170HX (GA100, 70 SMs) with the Qwen3.8-Flash-Next IQ3_S native pack, `--spec 3`, decode after a ~3K-word
prompt, a fresh engine per run, reference and switch runs interleaved. Both switches keep every output bit (the greedy answers to
11 prompts are the same with the switch on and off). Neither does anything on HIP, gfx906 or Metal.
| Switch | What | Answers | CMP 170HX |
|---|---|---|---|
| `STRATA_GRAPH_PRIO=1` | the verify window graph's kernel nodes run at their capturing stream's priority (`cudaGraphInstantiateFlagUseNodePriority`): the GDN gates / QSA indexer branch at the highest, the shared expert at the lowest. Nothing changes on a card whose priority range is 0..0 | bit-exact | decode +1.8% with `STRATA_QIL=1` (both runs above every reference run) |
| `STRATA_MMVQ_IL_PF=1` | hoisted-load, exact-integer kernels for the interleaved small-batch matvec where the sm_80 table picks them (Q6_K only so far, [MMVQ_IL_TABLE.md](MMVQ_IL_TABLE.md)); `=2` to `=6` force one variant everywhere (A/B only) | bit-exact | decode +1.4% (two ref / on / ref / on sets) |
+15
View File
@@ -34,3 +34,18 @@ Which table a card gets:
The emit rule: per cell, the fastest rows count if it takes at least 3% off the plain kernel's time, else 0; shapes of one
class that disagree give 0; a class with no measured shape copies its neighbour below.
## The hoisted-load kernels (STRATA_MMVQ_IL_PF, sm_80 only so far)
`STRATA_MMVQ_IL_PF=1` lets a second table pick, per cell, a variant of `native_mmvq_il` that issues every weight load of a
group before its dots and forms the dots as exact integers (`kIlPf_80` for 2-4 tokens and `kIlPf58_80` for 5-8, in
`src/kernels/cuda/native_mmvq.cu`; a cell is `variant << 4 | rows`, 0 = the rows table above with the plain kernel). Every
variant gives the same bits as the plain kernels. Only Q6_K has cells so far: on the CMP 170HX the variants tie the plain
kernel for Q4_K / Q5_K / IQ4_XS within the run-to-run noise. Unset or 0 keeps today's kernels; `=2` to `=6` run one variant
for every type and shape (A/B only).
To measure a card: `build/mmvq_il_parity --bench --emit-table --pf 0,3,4` (`--emit-table` must come right after `--bench`).
Every listed variant is bit-checked against native_mmvq first, then timed interleaved per case (`--rounds N` rounds of 100
calls, default 5; the median per variant on the `PFBENCH` lines). The last block (`variant << 4 | rows over --pf`) is in the
source form of `kIlPf_<cc>`. `il_pf_table` reads only sm_80's table today, so a new card's block also needs its lookup
there. Post the whole output on the issue.