setup: a tip (not a change) for images on a 12 GB card: --vram-reserve-mib 1000 if a request stalls

With the image encoder on the GPU, the 700 MiB reserve can leave ~200 MiB free on a 12 GB card (UD-IQ4_XS
end-to-end check on an RTX 5070). Setup prints a one-line recommendation; the written config is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01QJJwUv9g3cD2ztX6KWS8rk
This commit is contained in:
Niko1221andClaude Opus 5.5 committed 2026-10-04 14:16:36 +02:00
1 parent c2d1f19a43
commit 6f32ec070f
1 file changed
+5
+5
View File
@@ -220,6 +220,7 @@ MMPROJ = "mmproj-Qwen3.8-Flash-Next-BF16.gguf"
ESP_VECTOR = ROOT / "data" / "experimental-speed-projection" / "Qwen3.8-Flash-Next-experimental-speed-projection.gguf"
# the image encoder on the GPU (~1.2 GB at 1024 image tokens) warms up before the engine starts, so the engine
# sizes its expert slots around it and the default reserve (700 MiB) is enough; engines before 0.1.2 need more
VISION_GPU_SMALL_RESERVE_MIB = 1000 # the tip for images on a <= 12 GB card (the engine's LOW line asked ~1003)
VISION = {"gpu": {"max_tokens": 1024, "reserve_mib": 700},
"cpu": {"max_tokens": 300, "reserve_mib": 700}}
EXE = "strata.exe" if WIN else "strata"
@@ -4293,6 +4294,10 @@ def main() -> int:
args += ["--resident-budget-gib", f"{budget:g}"] # GiB kept in RAM (#498: a layer split has no budget)
if vision != "none":
args += ["--vision", "--vram-reserve-mib", str(VISION[vision]["reserve_mib"])]
if vision == "gpu" and a.vram_reserve_mib is None and 0 < gpu.get("vram_gb", 0.0) <= 12.5:
# a tip only (recommend, never force): on a 12 GB card the encoder's 700 MiB can leave ~200 MiB free
print(f" tip: images on a {gpu['vram_gb']:.0f} GB card can leave little VRAM free; if a request stalls, "
f"run setup again with --vram-reserve-mib {VISION_GPU_SMALL_RESERVE_MIB}")
if a.vram_reserve_mib is not None: # #493: VRAM left free for other programs (only when given)
if "--vram-reserve-mib" in args:
i = args.index("--vram-reserve-mib") + 1