Niko1221/Strata
1
0 Fork 0
mirror of https://github.com/Niko1221/Strata.git synced 2026-11-07 14:21:06 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity

server: note that the FP16 weight cache and the no-loan prompt path are the engine's own automatic defaults

Browse Source
This commit is contained in:
Niko1221 committed 2026-10-31 12:33:01 +01:00
1 parent cda6463e8e
commit b19b38aff1
1 file changed
+3
Download Patch File Download Diff File Expand all files Collapse all files
serve/server.py
+3
View File
@@ -3677,6 +3677,9 @@ PARALLEL_STACK_ENV = {
"STRATA_SLOT_COPY_BATCHSYNC": "1", "STRATA_BATCH_GDN_GROUPS": "1", "STRATA_BATCH_ATTN_ROWS": "1",
"STRATA_BATCH_SELF_COMMIT": "1", "STRATA_QSA_SLOTS": "1",
}
# 0.2.9.1: the two parts that need the card's free VRAM are the ENGINE's own, never an environment variable of ours: the FP16 weight
# cache (bit-exact; takes only VRAM beyond 3 GiB kept free) and no loan of cache slots (only when every expert is resident and the
# prompt path's own buffers fit beside 3 GiB; KL 0.006, QUAL-PAR2). STRATA_PF_W16_CACHE_MIB=0 and STRATA_AUTO_NOBORROW=0 turn them off.
# Only where the stack was run alone: Linux CUDA, all nine. HIP got no measurement without replicas (its only run was inside the
# replica config), Windows none at all (its host-arena / pinned-memory path differs): nothing there.