INTEL_ARC.md: "What has been run" rows for 2x and 1x Arc Pro B60, plus notes for the card: PCI id 8086:e211 (bmg-g21), setvars.sh
stopping at OCL_ICD_FILENAMES under `set -u`, running natively without Docker, and the layer-split startup warning that counts the
other card's layers.
sycl/setup_intel.py: the B60 table entry was e221, which the kernel's xe id list files under the BMG-G31 cards (0xE220-0xE223). Both
Arc Pro B60 cards measured here are 8086:e211, so setup named them "Intel GPU e211 (xe)" with a guessed VRAM size. e211 is added;
e221 stays, since there is no card here to say it is wrong. tools/test_setup_sycl.py checks the table.
COMMUNITY_BENCHMARKS.md and bench/results/2026-10-04-community-2x-arc-pro-b60/: the report, in the template's format, with the
server configs, engine logs and per-request timings.
With an engine that keeps --resident-experts on a layer split, setup no longer
steers a low-RAM PC with two cards to one of them:
- at install, the low-RAM mode on several GPUs takes the resident variant by
the same rule as on one card (what no card holds fits the RAM with
LOW_RAM_HEADROOM_GB beside it), without the one-card question;
- at a start, a resident config on several GPUs keeps --resident-experts
(split_mmap leaves it alone);
- a resident config started on a PC with two cards is offered both, "y" by
default, as any other config.
It is gated on MIN_ENGINE >= RESIDENT_SPLIT_ENGINE (0.1.40 here, a guess at
the release that would carry the engine change), so until then setup behaves
as before and the golden configs are unchanged. Two tests run the new paths
with MIN_ENGINE patched; tools/test_setup_*.py pass (260 tests).
- "experimental" is per model now: MODELS["UD-Q4_K_XL"] keeps it, the unsloth family and UD-IQ4_XS drop it. The
first menu lists the family without [experimental], with a one-line about (UD-IQ4_XS: a 94 GB download; under
~80 GB of RAM part of its experts are read from the SSD). Its size menu lists UD-IQ4_XS first (the default) and
UD-Q4_K_XL second; an experimental size always sorts last. --model names are unchanged.
- UD-IQ4_XS: no EXPERIMENTAL warning, --check verdict without it. Images are asked (off by default), per model: the
engine's image path has no restriction for this pack, which uses the original model's image encoder; not yet run
with images. UD-Q4_K_XL keeps images off.
- MCP: the fallback tables mirror setup's (per-model experimental/vision), sizes in setup's order, per-size
"experimental" and "images", the vision check per model.
- Docs: README and its six translations, MODELS, UNSLOTH_Q4, AI_SETUP, MCP_SERVER.
- Tests: the menus, the default, --model by name, images, --check, the pack/tokenizer paths; the golden configs
are unchanged.
Added where a config goes to several GPUs (setup --gpus / a multi-GPU install, start --gpus, "use both" at
start). Single-GPU configs are not touched. Opt-out: setup --no-remote-expert-opt, or "remote_expert_opt": false
in the config (a user key, kept on re-runs). The engine uses it only with a helper cache (--expert-cache-device1..3),
so a layer split runs as before. tools/test_setup_remote_opt.py; the 2x16 GB golden configs gain the flag.
On a card whose experts mostly run on the CPU the slots cost speed per request (RTX 5070, Q2_0: 11-24 %), so setup
now recommends slots only when the expert cache (every card of a split counted) still holds half of the model's
experts beside them (Q2_0: 24 GB and up, or a split); elsewhere it stays at one at a time and says "parallel N
reduces waiting for several users but costs about 10-25% speed per request on this card". --parallel N is
honoured as asked, as before. BATCHING.md: the table with the change against one at a time.
server: a request decoding in a slot whose neighbours finished (no other slot busy, nobody waiting, at least 32
tokens allowed) is BSTOPped and continued on the solo path with its MTP drafts - the engine copies the slot's
sessions back (INFO slot_cache=1; at most twice per request; STRATA_PARALLEL_SOLO=0 keeps it in the slot).
engine: INFO batch_slots=N slot_cache=0|1.
tools/batch_interleave_test.py: the back-to-solo path, and a next turn from a slot's turn checkpoint (a client
that drops the reply's thinking).
docs: "parallel": N, what a slot costs and what setup recommends, the server's handling (interleaved prompt reads,
BYIELD, slot conversation cache, back to solo), exactness and its settings, the RTX 5070 measurements (the slots
buy waiting time, not speed, on a 12 GB card), the protocol; DETAILS / README / AI_SETUP mention the option.
batch_test.py: the engine log in the temp folder, the CUDA DLLs on PATH, ASCII output, --mt-min (empty: the
engine's default kernels, for the invariance check without the exact settings). server: /metrics live.waiting
and live.outside_slots instead of a guessed count.
The slots' sessions may take a fifth of the expert cache (~VRAM less 5 GB), up to 4: 2 on 12 GB, 3 on 16 GB, 4
from 24 GB, none below. Without --parallel setup only says it once; with it the config gets "parallel": N and a
count above the recommendation is kept with a note.
CUDA 13 dropped Pascal and Volta. A model whose GPUs include one (its oldest card decides: one engine runs per model)
now runs the experimental CUDA 12 engine in engine-cuda12/ - the ready-made strata-windows-x64-cuda12.zip with NVIDIA's
CUDA 12 pip libraries, or compiled here with a CUDA 12.x toolkit (Linux; STRATA_NVCC picks one, #601) - and setup says
why. engine/ and every model on it stay as they were.
- the cards are an opt-in: naming one (--gpu / --gpus), --cuda 12, STRATA_EXPERIMENTAL_SM60=1, or a PC with no newer
NVIDIA card and no supported AMD card; a PC with a newer card keeps recommending it
- --cuda 12|13 (or STRATA_CUDA) overrides the choice and is kept, with a warning where it cannot work (recommend,
never force); --cuda 12 also runs with a driver older than 580 (525+), which the stop now mentions
- at a start, a Pascal / Volta card added to a CUDA 13 model moves that model's config to the CUDA 12 engine
- updates look at both engine folders; a compiled CUDA 12 engine builds in build-cuda12/
- an RTX 50 card in a CUDA 12 engine is a warning (CUDA 12.8+ builds it; #220 crashed with 12.8), no longer a stop
The setup golden test (every recommended config on five PCs) is unchanged; tools/test_setup_older_gpus.py covers the rest.
- CMake: STRATA_ISA_FLOOR (empty by default) builds ggml-cpu for AVX (Sandy/Ivy Bridge) or SSE4.2
(Nehalem/Westmere) instead of the build host or AVX2; every other ggml ISA option off. Strata's own
AVX2/AVX-512 kernels stay in the binary behind their CPU checks. Excludes STRATA_PORTABLE. The ggml
block follows the Strata_Dirigo fork's (rwkeyes).
- STRATA_FORCE_ISA=avx2|avx|sse caps the engine's CPU probes, so the older-CPU paths run on any PC
(how this was tested here: no old CPU). cpu_sse42_ok(), isa_floor_build().
- The engine: a normal build still refuses a CPU without AVX2, now pointing at the experimental build;
a floor build checks its own floor (AVX, or SSE4.2 + POPCNT) and warns that it is experimental and
slow (or, on an AVX2 CPU, that the normal build is faster).
- setup: a CPU without AVX2 warns ("experimental, slow") and compiles the engine on the PC with the
floor it has, instead of failing (#595's idea); the prebuilt AVX2 engine is not used there. Recorded
in BUILD.json; a build folder for another floor is configured afresh. STRATA_ISA_FLOOR=avx|none asks
for it on any PC. Windows + AMD has no source build: a clear stop. The MCP server's recommendation
notes it instead of refusing.
- tools/test_setup_oldcpu.py.
Default build: nothing changes (no new CMake definitions, the probes answer as before without the env).
Conflicts with 0.1.39: --trim-stage-weights is now the same switch as STRATA_STAGE_TRIM=1 (PR #639's
implementation, already in 0.1.39); the #533 VRAM command is taken between batch windows only when no slot
is decoding.
Two private engines, the same conversation: the baseline keeps it live, the candidate
parks it (an unrelated request in between, a re-rendering client, or two long
conversations taking turns) and restores it. Each turn's greedy tokens must match and
its decode speed must be within --tolerance (15%) of the live turn's. Pure logic
tested without a GPU (tools/test_conversation_restore_speed.py).
#528 did not reproduce on the RTX 5070 with it: Q2_0 37K tokens (--kv-resident 16384)
and IQ3_XXS 50-75K tokens (--kv-resident 32768) decoded at the live turns' speed after
a restore, with the same tokens.
docs/AMD_HIP.md's gfx1030 report measured an i7-13700KF (8P + 8E) at 38-42 tok/s with
all 15 workers against 36 with 8, so a CPU with no more E-cores than P-cores keeps the
engine's own count; the E-heavy chips the fork measured (i9-14900KF: 15 of 23 faster)
get the recommendation.
UD-IQ4_XS sits between IQ3_S and UD-Q4_K_XL: IQ3_S gate/up experts (IQ4_XS in one layer) with IQ4_NL downs (Q8_0 in
five layers), 59.5 GB of them, and UD-Q4_K_XL's dense side. The engine's kernels already take these formats; it was
packed (iq_pack.py --compat-bf16) and run on the Strix Halo, not yet on NVIDIA.
- MODELS["UD-IQ4_XS"] in the unsloth family: three shards pinned at the same revision 38bb39e with their sizes and
SHA-256 (the Hub's LFS pointers), the RAM budget like UD-Q4_K_XL (budget, one GPU, no images, --compat-bf16 pack,
no experts.bin), engine 0.1.38 or newer (the fused-layout fix for packs not every layer of which is covered).
- A model can have its own shard pattern and count (model_file / model_shards); budget_model() tells which Unsloth
model a config runs, for the #498 multi-GPU messages and RAM need.
- The AMD question (#429) stays for UD-Q4_K_XL only (nvidia_only): UD-IQ4_XS's formats have AMD prompt kernels.
- --gguf-dir accepts UD-IQ4_XS files (#444's list says so).
- UNSLOTH_Q4.md: its files, what setup does, what is known (Strix Halo), and the BF16 rounding of the hyper-connection
projections that every --compat-bf16 pack has (not changed here). MODELS.md: a paragraph.
- test_setup_unsloth: pins, budget, a mocked install, the engine check, no AMD question, the config read back.
UD-Q4_K_XL's install, its tests and every other model's config are unchanged.
find_nvcc takes the newest toolkit it finds, so with CUDA 12.8 and 12.9 installed, 12.9 was always used - and 12.9's
math headers do not compile with glibc 2.43 (Ubuntu 26.04). STRATA_NVCC=<nvcc> is now the only one considered (a
warning when the file is missing, or when it is too new for the Pascal/Volta build). CUDA_HOME is a candidate like
CUDA_PATH. TROUBLESHOOTING.md: the two build errors and what to do (g++-14 for gcc > 14; CUDA 12.8 or 13.x for glibc
2.43). Without STRATA_NVCC nothing changes.
On a 7200 RPM disk the default --ple-io direct's random reads of the n-gram table stalled prompts past the 60 s
watchdog, and the stall report looked like any other engine stall.
- Engine (Linux): at start with --ple-io direct, the table's disk is looked up in sysfs (queue/rotational); on a
rotational one it warns and recommends --ple-io ram or an SSD (not on WSL, whose virtual disk says rotational).
- Stall report (Linux): the engine's threads in state D and their wchan ("threads waiting on the disk (state D): 16
of 35 (blk_io_schedule x16)"), with what that means.
- Setup (Linux): a model on a rotational disk gets --ple-io ram when the RAM holds the table beside the model,
otherwise a warning that recommends an SSD. Nothing changes on an SSD.
- DETAILS.md: --ple-io direct is for SSDs (it reads unbuffered, not through the OS cache), and a troubleshooting row.
The tokenizer matches <think> and </think> as their special tokens even
in plain text (type 4, as llama.cpp does), so a user quoting "</think>"
handed the model a real end-of-reasoning token and the answer came back
empty. Before the template is rendered the tags in message text (user,
tool, system, assistant history, tool call arguments, tool descriptions)
are swapped for private-use marks; after it, those spans are encoded as
ordinary text (Tokenizer.encode's new plain= spans), the template's own
markers stay special. An assistant turn that opens with its own
<think>...</think> block keeps that block. A prompt without the tags is
encoded exactly as before; count_tokens uses the same path.
hardware_key, the settings file's key for a saved calibration, read the cards with nvidia-smi's list: on an AMD PC
every card was "?|0GB" (two different cards shared one calibration), and on a PC with an NVIDIA card too a HIP index
named that NVIDIA card or stopped setup ("there is no GPU N"). A HIP config's cards now come from amd_gpus, in HIP's
numbering, with the architecture in the name (an RX 7900 XTX and an RX 7900 XT are both gfx1100; a card without a
product name is known by its arch). Setup reuses a saved calibration on Linux HIP as on NVIDIA (setup_calibration);
Windows HIP keeps the defaults for now. The tuning is still offered on NVIDIA only - a `./setup.sh --calibrate` on
HIP runs it by hand, and each control is verified on HIP before setup offers it.
The NVIDIA key and the settings file's format are unchanged. Tests: tools/test_setup_amd.py CalibrationKey (7).
French answers drafted poorly with the shipped subsets: 23.6% of the token occurrences of French text fall outside
draft_vocab_en.bin, and the draft head cannot propose those (no subset adds the accented words and word pieces).
- tools/draft_vocab.py: a coverage mode. --corpus FILE... adds to the base the most frequent tokens of the corpus that
together cover --coverage (default 0.99) of its occurrences, and prints the share outside the base and the new list.
- data/draft_vocab_fr.bin: draft_vocab_en.bin plus 5,686 tokens (46,211 ids), from 10 French Wikipedia articles
(France, Paris, Napoleon Ier, Langue francaise, Union europeenne, Litterature francaise, Cuisine francaise,
Economie de la France, Revolution francaise, Seconde Guerre mondiale; 1.24 M characters, 326,926 tokens) at 0.99:
python tools/draft_vocab.py --gguf <IQ3_XXS shard 1> --base data/draft_vocab_en.bin --corpus <the articles> \
--coverage 0.99 --out data/draft_vocab_fr.bin
Outside the list on that corpus: 0.74% (23.65% with en).
- setup.py: "fr" in DRAFT_VOCABS / --draft-vocab (~151 MiB head on IQ3_S, between en and cyrillic). The sha256 check
knows it as a shipped subset, so a later --draft-vocab cjk/en replaces it and a subset made by hand stays.
- tests: tools/test_draft_vocab.py (the coverage mode), tools/test_setup_draft_vocab.py (fr replaces a shipped subset
and back, keeps the en ids first in their order, its MiB estimate).
The default subset (cjk) is unchanged.
On a hybrid CPU (Intel 12th gen+, AMD Zen 5 + Zen 5c) setup writes --pool-workers
N into the run config: the performance cores but the host loop's one, plus half
of the efficiency cores. An E-core runs the expert kernels ~2.2x slower and each
layer waits for its slowest part; Hardin22 measured 15 workers at 165 / 116 tok/s
against 106 / 84 with all 23 on an i9-14900KF (#642). A recommendation only: the
line is in the config, a saved calibration's measured count wins, and a CPU whose
cores are all alike gets the same config as before.
The engine's pool, with every core allowed (the default affinity), now orders the
P-cores first on a hybrid CPU, so a smaller pool runs on the P-cores and the first
E-cores rather than on whatever the OS numbered first (from Hardin22's fork).
Non-hybrid CPUs: unchanged.
Co-Authored-By: Hardin22 <[email protected]>
Kept in the run config's vision.max_tokens (the server already passes it
to the encoder); a setup run again keeps it for the same encoder device.
More than the default is allowed with a note on the encode time. Without
the flag the config is as before. DETAILS.md: vision.mmproj can point to
a Q8_0 mmproj of this model.
- The draft layer's K/V lives on the last stage's GPU (the drafter is loaded there); its save, ring restore and
verification now run with that stage's image under that device, instead of with stage 0's image on GPU 0, where
the ring restore kernel ran against another GPU's memory (seen in Niko1221/Strata#653, which places it the same way).
- Parking with a layer split on one GPU (--split-device 0) is refused at start: untested.
- The whole-session validate refuses an image with stage images; clearer message for stage parts in live.
- --trim-stage-weights says which layers each GPU loads, and fails when index.txt cannot be read.
- conversation_snapshot_test: the whole-session refusal; parking_test.py runs with STRATA_SNAPSHOT_VERIFY=1.
tools/make_profile.py builds the ranking as the base's order, then the
trace's pairs, then the fill. The base is only appended to, so --base can
never move a pair the base already ranks - and the shipped
data/expert-profile.bin ranks all 24,576 pairs, which makes a trace a
no-op:
wrote ...: 24576 ranked pairs (24576 from the base, 0 from the traces, ...)
So the documented way to seed a profile from --dump-routing (rank the
pairs your workload routes most) silently did nothing with the shipped
base; only --no-base changed anything, and that drops the base's ranking
entirely.
Add --reorder: rank the trace's pairs first (most frequent first), then
let the base fill the rest, then the fill. The default is unchanged. The
ranking moves into a rank_profile() so it can be tested without a model;
tools/test_make_profile.py covers the no-op, --reorder, --no-base, the
frequency/tie order and an end-to-end run over a complete base. On this
host's coding trace the same command now puts 5,808 trace pairs first.
--no-browser (on a start or at setup) saves "open_browser": false in the
model's run config; start() and run-<model>.bat/.sh then leave --open
out, and the server ignores --open when the config says so (a start
script written earlier still has it). --browser undoes it. Without the
flag the config, the command and the script are as before.
A re-run for an installed model (another --context, say) rewrote
strata-<model>.json from scratch and dropped the user's sampling,
mcp_servers, mcp and other blocks. Setup now carries over every key it
does not write (and env/vision entries of the user's own, such as a Q8_0
mmproj), keeps the earlier file as strata-<model>.json.bak and says what
it kept and which hand-added engine options it did not. A copy set up
like an earlier install carries them over from that install's config.
A first install writes the same config as before.
- conversation_snapshot_*(..., const QsaState* draft): nullptr saves, validates and restores an image without the
draft layer's K/V. The later stages park that way; the draft ring is saved once, with the first stage's image.
- ConversationKvReuse::stages / ConversationCache::retain(.., stage_kv): every stage's restored K/V is retained, so
its next park copies only what changed, as the first stage's already did.
- conversation_checkpoints_split / _merge: the checkpoints are moved into their stage parts for the capture and
back afterwards (and on restore), instead of copied.
- Tests: conversation_cache_test (split/merge round trip without copies, refused misalignments, per-stage reuse),
conversation_snapshot_test (draft-less images: estimate, validation both ways, A/B/A exactness, incremental reuse).
- tools/parking_test.py: a follow-up decodes the same tokens whether its conversation stayed live or came back
from the parking cache (every stage's image with a layer split). 4-GPU split, IQ3_S: 3,786 and 9,276-token
conversations restored in 42 / 51 ms, follow-ups identical; without parking the same follow-ups took 3.4 / 6.8 s.
- tools/batch_test.py: --keys to send sampling keys (sampled rows in batch windows).
- docs/BATCHING.md: parking together with batching, the Testing section; docs/DETAILS.md: parking now works with
--layer-split.
With --batch, a consumer that left mid-answer (a client disconnect, a tool call or a stop string ending the
response) left the engine writing the abandoned answer's tokens, and the next request read them as its own:
it answered the previous question again. Now the solo request is STOPped and read to its DONE, an admission
to its BADM, and a slot is BSTOPped and freed only at its BDONE. STRATA_BATCH_TRACE=1 logs the control lines.
tools/early_close_test.py reproduces it (solo and with a second request running).
The engine's --serve can now keep up to N conversations open (--batch N) and decode them together, one token
each per verify window; with a layer split, --batch-groups G pipelines the slot groups through the GPUs, and
--trim-stage-weights makes every GPU load only its own layers' dense weights (VRAM back to the expert cache).
The server runs concurrent requests in the slots. See docs/BATCHING.md.
One line under the start banner: the config's engine options without the model's file paths, and the
server's fields (host:port, an API key set or not, GPUs, draft subset, fit_max_tokens, thinking budget).
The key itself is never printed.
With the default 700 MiB reserve the expert cache fills an AMD card that also drives a KDE/Wayland desktop. When the
desktop then needs VRAM (the browser --open starts, a new app), amdgpu moves GPU memory to system RAM (GTT), which
the experts already fill: the OOM killer ends kwin_wayland / plasmashell (#560, R9700), or the compositor fails to pin
its framebuffer (#516, 7900 XTX). --vram-reserve-mib 3072 / 4000 fixed it in both reports.
A recommendation, nothing is changed: setup prints the tip on a Linux desktop session (WAYLAND_DISPLAY or DISPLAY) with
the HIP backend and no reserve given, and the server prints it after the start when the engine reports less than
2048 MiB of VRAM free (INFO vram_free_mib) and the reserve is under 3072 - before --open starts the browser.
INSTALL.md and AMD_HIP.md describe the symptom and the fix. No engine code.
installed_configs() and update_install() keep only JSON objects with "exe" and "args"; any other
strata-*.json in the folder gets a one-line warning naming it.
Setup no longer writes a fixed 300 MiB reserve for cards under 8 GB: the engine's own rule (lower the reserve only as
far as the smallest working cache needs, down to 300) chooses better, and a fixed 300 left an emulated small card with
5 MiB free. Setup keeps the tip (an 8K context, --draft-vocab en). When the engine lowered the reserve and the card ends
below the 256 MiB LOW line, it prints one warning (requests may stall; close other programs that use the GPU, or lower
--max-context) instead of advising a larger reserve, which would leave the cache no room.
On an RTX 3060 Laptop (6 GB) the weights, the KV cache and the draft layer left 0.29 GiB, the default 700 MiB reserve
and the draft head took it, the expert cache got 0 slots and the verify window refused to start.
- setup: a single card under 8 GB (nvidia-smi below 7.5 GB; an 8 GB card lists 7.99) gets --vram-reserve-mib 300
unless one is given (images on the GPU keep theirs), marked in the config as setup's own choice so a later setup
decides again. It prints what frees more VRAM if the start still stops: an 8K context, --draft-vocab en, other
programs closed. The draft layer stays (the server needs --mtp and --spec). 8 GB and larger cards: the same config
(golden test).
- engine: a cache too small to lend the prompt path a 256-token chunk (its buffers plus the 128-slot floor; one slot
without --prefill) makes the prompt path allocate its own on top. When the default reserve leaves less than that,
the reserve shrinks to what leaves exactly that cache, down to 300 MiB (what comes after the cache is paid from
the reserve: at 300 an emulated small card ended with 5 MiB free, LOW by the serve check). With no slot at all the start stops with how many
MiB are short, what the session and the draft head take, and the options. A start whose cache could lend that
chunk is sized as before. Checked with a VRAM ballast on the 5070 (IQ3_XXS, 32K): 258 slots needed, 0.71 GiB free
-> "about 333 MiB more", and with --vram-reserve-mib 700 given, "733 MiB more" plus the reserve tip.
- verify's refusal points at that line.
Calibrated on a Radeon AI PRO R9700 with tune_hipblaslt (24 of the gfx1100
table's 26 shapes). Setup only picks it when the card is gfx1201 and the
installed hipBLASLt is exactly 100202.
With that hipBLASLt the plain hipBLAS path is far off: on the full GSQ-RCO
IQ3_XXS (KV streaming, engine 0.1.31) a fresh 32K prompt reads at 1,177 tok/s
with the table against 638 without it, and a 7K prompt at 1,164 against 656
(median of 4 warm runs per arm, arms alternated); decode is unchanged.
hip_prefill_hipblaslt_gemm passes with it. docs/AMD_HIP.md notes that the
1.4.1 result (0-3%) does not carry over to 1.2.2.
The engine already leaves --vram-reserve-mib MiB of VRAM free (default 700). INSTALL.md now says what it does, its
default, and how to set it (the flag, or "--vram-reserve-mib", "2048" in the config's args). Setup's new flag writes
it into the config only when given (one value: it replaces the images' own 700, with a note when it is less than the
GPU image encoder needs); on a start of an installed model it is saved into that model's args, and a new copy set up
from an earlier install keeps it. The default config is unchanged (golden test). Tests in test_setup_choices.py.
The engine refuses --resident-budget-gib with a layer split, so setup always dropped UD-Q4_K_XL to one GPU. With RAM
>= the GGUF size + UNSLOTH_RAM_LEFT_GB (~135 GB) and 2+ usable cards, setup now asks (one GPU stays the default and
--yes keeps it; an explicit --gpus is honoured; an explicit --resident-budget-gib keeps one GPU) and the split writes
"gpu": [..], "layer_split": "auto" without the budget: all the experts loaded into RAM at start, the files through the
OS file cache (the config #498 measured: 2x RTX 3090, 31 -> 64-78 tok/s). Latent bug: START-HERE --gpus 0,1 (or the
start-time offer, or a hand-edited config) on a UD-Q4 config kept the budget and the engine exited with code 2; the
split now drops it, or setup stops before saving when the RAM is too small. docs/UNSLOTH_Q4.md says so. Tests in
tools/test_setup_unsloth.py; the golden configs are unchanged.
hf() takes the host from HF_ENDPOINT (default huggingface.co, a trailing slash dropped); the pinned revisions, the
fallback to main and the SHA-256 checks are unchanged. tools/mtp_fetch.py reads the MTP tensors from the same host.
Step 5 prints the folder setup expects the model files in and how to use files downloaded by hand (that folder with
their original names, or --gguf-dir), and the mirror when one is set. INSTALL.md says the same. Tests: the endpoint
in hf()/hf_unpinned()/mtp_fetch, and step 5's words over a mocked install. The golden configs are unchanged.
LocalXPU