French answers drafted poorly with the shipped subsets: 23.6% of the token occurrences of French text fall outside
draft_vocab_en.bin, and the draft head cannot propose those (no subset adds the accented words and word pieces).
- tools/draft_vocab.py: a coverage mode. --corpus FILE... adds to the base the most frequent tokens of the corpus that
together cover --coverage (default 0.99) of its occurrences, and prints the share outside the base and the new list.
- data/draft_vocab_fr.bin: draft_vocab_en.bin plus 5,686 tokens (46,211 ids), from 10 French Wikipedia articles
(France, Paris, Napoleon Ier, Langue francaise, Union europeenne, Litterature francaise, Cuisine francaise,
Economie de la France, Revolution francaise, Seconde Guerre mondiale; 1.24 M characters, 326,926 tokens) at 0.99:
python tools/draft_vocab.py --gguf <IQ3_XXS shard 1> --base data/draft_vocab_en.bin --corpus <the articles> \
--coverage 0.99 --out data/draft_vocab_fr.bin
Outside the list on that corpus: 0.74% (23.65% with en).
- setup.py: "fr" in DRAFT_VOCABS / --draft-vocab (~151 MiB head on IQ3_S, between en and cyrillic). The sha256 check
knows it as a shipped subset, so a later --draft-vocab cjk/en replaces it and a subset made by hand stays.
- tests: tools/test_draft_vocab.py (the coverage mode), tools/test_setup_draft_vocab.py (fr replaces a shipped subset
and back, keeps the en ids first in their order, its MiB estimate).
The default subset (cjk) is unchanged.
The shipped draft subsets hold 142 of the vocabulary's 18,580 Cyrillic tokens, so an answer in Ukrainian, Russian,
Bulgarian or Serbian drafted almost nothing. data/draft_vocab_cyrillic.bin is draft_vocab_en.bin plus every token
whose text is Cyrillic (58,963 ids), built with the repository's own tool:
python tools/draft_vocab.py --gguf <IQ2_XS shard 1> --base data/draft_vocab_en.bin --add cyrillic
- tools/draft_vocab.py: a "cyrillic" script (U+0400-052F, 1C80-1C8F, 2DE0-2DFF, A640-A69F).
- setup.py: --draft-vocab cyrillic; docs/DETAILS.md.
On the NVFP4 fork: 1.4 -> 2.1 tokens per round, 83 -> 109 tok/s in Ukrainian answers, English unchanged.
- setup: a third family, "coder" (IQ1_M), with its own expert profile; its
shard 2 and vision encoder are the original's files and are reused; 32 GB of
RAM is now enough to install (the Coder's experts are 23 GB).
- data/expert-profile-coder.bin: the shipped ranking mapped onto the kept
experts through the release's rco-allocation.txt (72% GPU hits, 12 GB card);
tools/make_profile.py takes --n-expert.
- README/DETAILS: the Coder's speeds (prompts 599-1,350 tok/s, output
43-53 tok/s, 1K-262K) and what it is; bench/results/2026-09-28-coder.
- The experimental speed projection can be turned on for the Coder too.
- --pcie-mode auto is the copy kernel for native packs too. Issue #31's thread
dumps show the host stuck for good in the DMA path's cudaMemcpyAsync (a driver
lock) inside a verify window while the GPU spins on the flag those copies
raise. The copy kernel needs no host CUDA call there; IQ3_S decode 45.3 -> 44.8
tok/s. --pcie-mode dma keeps the old path.
- setup: a locally compiled engine records its version; one kept after a failed
compile is recognised by the version in its binary, and a config it cannot
read (--prefill auto on 0.1.12) goes back to 2048 (#49).
- setup: nvcc in /opt/cuda (Arch), a clearer message off Ubuntu/Debian (#46);
a failed build is tried once more (#45: ptxas's intermittent PTX parse error).
- data/expert-profile.bin ranks all 24,576 experts (the first 8,000 unchanged):
the cache stopped at 8,000 slots on 24-32 GB cards (#46). tools/make_profile.py
builds or extends a profile from --dump-routing traces; --expert-cache 0 with a
profile now means auto.
The engine takes llama.cpp's control-vector flags (--control-vector-scaled
FILE:SCALE, --control-vector-layer-range A B, --cvec-mode add|project,
--cvec-dir per-layer|single:L) and applies the vector after each steered
layer's FFN write on every hyper-connection stream: project h -= s (h.v) v,
or add h += d - where the experimental-speed-projection package's patched
llama.cpp applies it (build_cvec after the layer's second combine).
- strata/kernels/cvec: one kernel per (stream, token); with a pending fused
write it applies it first with the fused read's own arithmetic, so a
loaded-but-off vector is bitwise the stock engine. cvec_parity tests it.
- Per request: cvec=0|1 on the GEN/GENI line, the API field
"experimental_speed_projection", a switch in the web app's Sampling
drawer; a switch drops the conversation cache once. About and the
Monitor say which way each request ran.
- setup asks (default off; --experimental-speed-projection on|off|GGUF),
original Qwen3.8-Flash-Next only. The vector ships in
data/experimental-speed-projection with a README that says what it is:
its package describes it as a refusal-direction projection.
Measured (bench/results/2026-09-27-esp): off = stock token for token; on
costs 0.2-0.4% per token; mean KL 0.063 on 2,557 teacher-forced tokens.
Inference engine, OpenAI/Anthropic API on localhost, optional image input, one-file setup
(START-HERE.bat / setup.sh). Model files are downloaded from Hugging Face by the setup.
Niko1221