docs: the cluster decode kernels run on sm_90+ (RTX 50), not RTX 40

This commit is contained in:
Niko1221 committed 2026-10-02 18:24:22 +02:00
1 parent 2aa6ace036
commit 82cf9b83c8
1 file changed
+1 -1
+1 -1
View File
@@ -22,7 +22,7 @@ measured with Swift 1.5's IQ2_XS, which runs at the original's speed.
**Engine 0.1.36 (#136), the same PC:** Q2_0's prompt experts run on fused int8 tensor-core kernels (RTX 30 and newer):
4K 1,294 -> 1,570, 32K 2,170 -> 2,653, 128K 2,123 -> 2,468 tokens/s (+16-22%), as close to an FP16 reference as the
previous kernels (closer at 32K: teacher-forced KL 0.009 vs 0.012). The decode path's block selection and greedy
argmax run on thread-block clusters (RTX 40/50 class, sm_90+; the same tokens): Q2_0 output at 4K 89 -> 93.5, at 128K
argmax run on thread-block clusters (RTX 50, sm_90+; other cards keep the previous kernels; the same tokens): Q2_0 output at 4K 89 -> 93.5, at 128K
64.5 -> 76.4 tokens/s. `STRATA_PF_FUSED=0` keeps the previous prompt kernels (byte-identical answers to 0.1.35);
`STRATA_PF_FUSED=1` also runs the native IQ packs' fused kernels (opt-in: IQ2_XS prompts +12% at 4K, +3% at 32K, the
IQ3 packs about even); `STRATA_QSA_CLUSTER=0` / `STRATA_ARGMAX_MULTI=0` turn the decode kernels off. The tables