mirror of
https://github.com/Niko1221/Strata.git
synced 2026-11-07 18:35:04 -10:00
docs: STRATA_GRAPH_PRIO and STRATA_MMVQ_IL_PF (sm_80, opt-in decode switches)
DETAILS.md gets a section with the two switches, what each does, that both are bit-exact, and their decode gain on the CMP 170HX (+1.8 % with STRATA_QIL=1, +1.4 %). MMVQ_IL_TABLE.md gets a paragraph on the kIlPf_80 / kIlPf58_80 tables (variant << 4 | rows, Q6_K only) and on measuring a card with mmvq_il_parity --bench --emit-table --pf 0,3,4. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
116992ab8a
commit
6d6f0f39fd
2 files changed
+26
No files matched your search
@@ -2563,3 +2563,14 @@ CUDA card, where it measured +1..3 % prompt speed on a Tesla P100 at 30-60K toke
|
||||
Options that change answers (other summation order) are separate and explicit: `STRATA_SELECT_SIMT=1|2` (FP32 tiled block scorer,
|
||||
KL 0.01 against the warp scorer per #2071), `STRATA_PA_V2T=1` (v2 prompt attention, #2092/#2146), `STRATA_PF_FUSED=1
|
||||
STRATA_PF_FUSED_SM75=1` (fused int8 prompt experts for the Q2_0 pack, #2145; only on a 7.5 card, sm_80+ is unchanged), `STRATA_BF16_TC=1` with `STRATA_XN_F16` (#2091).
|
||||
|
||||
## CUDA opt-in decode switches measured on a CMP 170HX (sm_80; off unless you set them)
|
||||
|
||||
Measured on one CMP 170HX (GA100, 70 SMs) with the Qwen3.8-Flash-Next IQ3_S native pack, `--spec 3`, decode after a ~3K-word
|
||||
prompt, a fresh engine per run, reference and switch runs interleaved. Both switches keep every output bit (the greedy answers to
|
||||
11 prompts are the same with the switch on and off). Neither does anything on HIP, gfx906 or Metal.
|
||||
|
||||
| Switch | What | Answers | CMP 170HX |
|
||||
|---|---|---|---|
|
||||
| `STRATA_GRAPH_PRIO=1` | the verify window graph's kernel nodes run at their capturing stream's priority (`cudaGraphInstantiateFlagUseNodePriority`): the GDN gates / QSA indexer branch at the highest, the shared expert at the lowest. Nothing changes on a card whose priority range is 0..0 | bit-exact | decode +1.8% with `STRATA_QIL=1` (both runs above every reference run) |
|
||||
| `STRATA_MMVQ_IL_PF=1` | hoisted-load, exact-integer kernels for the interleaved small-batch matvec where the sm_80 table picks them (Q6_K only so far, [MMVQ_IL_TABLE.md](MMVQ_IL_TABLE.md)); `=2` to `=6` force one variant everywhere (A/B only) | bit-exact | decode +1.4% (two ref / on / ref / on sets) |
|
||||
@@ -34,3 +34,18 @@ Which table a card gets:
|
||||
|
||||
The emit rule: per cell, the fastest rows count if it takes at least 3% off the plain kernel's time, else 0; shapes of one
|
||||
class that disagree give 0; a class with no measured shape copies its neighbour below.
|
||||
|
||||
## The hoisted-load kernels (STRATA_MMVQ_IL_PF, sm_80 only so far)
|
||||
|
||||
`STRATA_MMVQ_IL_PF=1` lets a second table pick, per cell, a variant of `native_mmvq_il` that issues every weight load of a
|
||||
group before its dots and forms the dots as exact integers (`kIlPf_80` for 2-4 tokens and `kIlPf58_80` for 5-8, in
|
||||
`src/kernels/cuda/native_mmvq.cu`; a cell is `variant << 4 | rows`, 0 = the rows table above with the plain kernel). Every
|
||||
variant gives the same bits as the plain kernels. Only Q6_K has cells so far: on the CMP 170HX the variants tie the plain
|
||||
kernel for Q4_K / Q5_K / IQ4_XS within the run-to-run noise. Unset or 0 keeps today's kernels; `=2` to `=6` run one variant
|
||||
for every type and shape (A/B only).
|
||||
|
||||
To measure a card: `build/mmvq_il_parity --bench --emit-table --pf 0,3,4` (`--emit-table` must come right after `--bench`).
|
||||
Every listed variant is bit-checked against native_mmvq first, then timed interleaved per case (`--rounds N` rounds of 100
|
||||
calls, default 5; the median per variant on the `PFBENCH` lines). The last block (`variant << 4 | rows over --pf`) is in the
|
||||
source form of `kIlPf_<cc>`. `il_pf_table` reads only sm_80's table today, so a new card's block also needs its lookup
|
||||
there. Post the whole output on the issue.
|
||||
Reference in new issue
Block a user