Niko1221 fc21e54214 Merge the 0.1.36 speed work: fused Q2_0 prompt experts on by default (STRATA_PF_FUSED=0 off), the native IQ packs' fused kernels opt-in (=1), cluster decode kernels; the fused test waits for its uploads ...

The ring rule: 1024 slots for the Q2_0 pack's fused experts (P3), 512 for native packs with fused layers.
Quality (teacher-forced vs the FP16 path, Q2_0): fused as close as MMQ at 8K, closer at 32K (phaseA-tests s18-tf).

2026-10-02 16:31:42 +02:00

..

prefill_fused_iq_test.cpp

#136: the fused int8 prompt experts for the native IQ packs (gate/up IQ2_XXS IQ2_XS IQ2_S IQ3_XXS IQ3_S IQ4_XS, down Q2_0 IQ4_NL; opt-in STRATA_PF_FUSED=1)

2026-10-02 16:25:57 +02:00

prefill_fused_moe_test.cpp

Merge the 0.1.36 speed work: fused Q2_0 prompt experts on by default (STRATA_PF_FUSED=0 off), the native IQ packs' fused kernels opt-in (=1), cluster decode kernels; the fused test waits for its uploads

2026-10-02 16:31:42 +02:00

prefill_mmq_kquant_test.cpp

prefill: UD-Q4_K_XL prompts 2.8x faster (16K: 57 -> 160 tok/s on an RTX 5070) - 32 stager threads and a 128-deep ring when the experts are read from the GGUF in place; MMQ for its Q4_K / Q5_K / Q5_1 experts

2026-10-01 10:30:41 +02:00