John SmithandClaude c291aa74ab fix(scripts): make 1M YaRN context actually work on Qwen 3.8 27B (qwen35) ...

The server silently capped the slot at the model's native 262K training
context because --override-kv targeted qwen2/qwen2vl/qwen3 arch keys while
the GGUF architecture is qwen35 (context_length=262144). Fix the override
to qwen35.context_length and align the launcher with the upstream
(q38rocm / ROCmFPX, pinned engine build 244) validated recipe:

- --override-kv qwen35.context_length=int:1048576 (removes the cap)
- TurboQuant KV (-ctk q8_0 -ctv turbo4) + matching draft KV
- MTP speculative decode (--spec-type draft-mtp, n_max 6 / p_min 0.6)
- RAM context checkpoints sized for 1M on 128 GB (64 @ 32768 MiB)
- Strix Halo env inlined, -dev ROCm0, HIP_VISIBLE_DEVICES=0
- --no-context-shift, --no-mmap, -fit off, -t 16 -tb 32, --metrics

Co-Authored-By: Claude <[email protected]>

2026-08-31 00:24:57 -04:00

..

run_yarn_1m.sh

fix(scripts): make 1M YaRN context actually work on Qwen 3.8 27B (qwen35)

2026-08-31 00:24:57 -04:00