Niko1221 47a83696e9 The GSQ-RCO Coder (256 of 512 experts): setup, profile, speeds (PR #54) ...

- setup: a third family, "coder" (IQ1_M), with its own expert profile; its
  shard 2 and vision encoder are the original's files and are reused; 32 GB of
  RAM is now enough to install (the Coder's experts are 23 GB).
- data/expert-profile-coder.bin: the shipped ranking mapped onto the kept
  experts through the release's rco-allocation.txt (72% GPU hits, 12 GB card);
  tools/make_profile.py takes --n-expert.
- README/DETAILS: the Coder's speeds (prompts 599-1,350 tok/s, output
  43-53 tok/s, 1K-262K) and what it is; bench/results/2026-09-28-coder.
- The experimental speed projection can be turned on for the Coder too.

2026-09-28 14:20:58 +02:00

96 KiB

Raw History

View Raw