Only the engine / Control number: the app, the Terminal and the Metal engine keep their versions here; the release bump of the other files
is the release's own commit.
nvidia-smi gives a GB10's memory.total as [N/A]. tools/strata_mcp.py's own NVIDIA probe (the fallback of the launch hardware page)
dropped the card; it now counts it as setup.py's gpus() does (arm64 only). strata doctor --live accepted no card with that figure; on arm64
it lists the card with no memory figure (x86 still refuses it). The DeepSeek fit plan for a unified-memory GPU with no BIOS carve-out
(GB10) no longer speaks of Windows, a BIOS and a Radeon, and quotes the numbers measured on the Spark. The Windows Strix Halo plan text
is unchanged (tests).
tools/dgx_bench.py, dgx_warm.py, dgx_compare.py, dgx_quality.py, dgx_quality24.py, dgx_needle.py, dgx_turns.py, dgx_longctx.py, dgx_128k.py,
dgx_kvresident*.py, dgx_concurrent.py and dgx_smoke.py (the acceptance gate in one command; its data folder is now an option and its scratch
files live in a folder of their own) and docs/DGX_SPARK_LINUX.md.
- strata_control/targets.py: host_platform() names Linux ARM64 'linux-aarch64' (as uname and Tauri do; a Mac stays macos-arm64) and the
cuda backend is committed to there (nothing else is: no HIP, SYCL or CPU engine for arm64 Linux)
- strata_control/feed.py: linux-aarch64 is a feed platform for engine, dsv41 and control packages (and a Tauri target name, no app is built)
- packaging/package_terminal.py: the Terminal for linux-aarch64 with python-build-standalone 3.12.15 aarch64-unknown-linux-gnu (SHA-256 from
the release's SHA256SUMS); packaging/lock_wheels.py + locks/runtime-linux-aarch64-cp312.txt: its hash-pinned wheels (the same 23 packages
as the x86_64 lock; platformdirs resolved to 4.13.6 against 4.13.0 there)
- packaging/make_local_feed.py: --dsv41 for the DeepSeek packages, linux-aarch64 terminals; packaging/install.sh installs on Linux ARM64
and says the engine is for the DGX Spark (GB10) only, instead of refusing
- packaging/build_cuda_linux_aarch64.sh: the release recipe (strata + strata-dsv41 + the reader, the engine package and the separate
DeepSeek package, sm_121 + compute_121, driver 580.65, build id naming the commit)
- tests: test_linux_aarch64.py, the Spark rows in test_release_matrix.py
Nothing changes for another platform: every edit is a new table row, a new branch for aarch64 Linux or a new file.
- gpus(): a GPU whose memory nvidia-smi does not report (GB10) on arm64 is counted as Strix Halo's is: the RAM less the OS share
(UMA_OS_LEFT_GB), uma True, dedicated_gb 0, so the low-RAM and fit rules do not count the same memory twice. #3622 listed the whole RAM
as VRAM beside the RAM.
- cpu_info(): an arm64 CPU is named from lscpu (Cortex-X925 + Cortex-A725), not 'unknown CPU'. The PCIe probe is skipped for a
unified-memory GPU (nvidia-smi's 1.0 x1 there means nothing).
- get_prebuilt(): on arm64 Linux the ready-made engine is the linux-aarch64 package of the signed feed (engine_from_feed); without one,
setup compiles as #3622 had it do always. setup no longer forces --build on arm64.
- Step 7 on a GB10 (is_gb10: uma, sm_121, arm64 Linux, nothing else): --mmap-experts (one copy of the experts, all 24,576 cache
slots), --prefill auto:32768, --mtp-q4 all, --mtp-hnorm stream, each measured on one GB10 with the answers unchanged; --draft-vocab
en is printed as a tip, not set.
- The CUDA repository key URL is sbsa on arm64; the DeepSeek-engine-missing message names the Spark build script on arm64.
Tests: test_setup_arm64.py (unified-memory shape, is_gb10, lscpu name); test_setup_pins.py skips its x86-zip Engine tests on arm64 Linux.
nv_defaults.cpp gets an sm_121 table beside sm_86 and sm_60: STRATA_PF_FUSED=1 (fused IQ prompt kernels), STRATA_PREFILL_NVFP4=w4a4
(NVFP4 prompt MMQ with FP4 activations), STRATA_HC_UPMIX=1 (fused hyper-connection read), STRATA_SPEC_COST=1 (cost-aware MTP window).
Measured on one GB10 with paired A/Bs (docs/DGX_SPARK_LINUX.md); a switch the user set is kept. STRATA_SM121_PERF=0 turns the table
off, and so does the 0.2.7 master switch STRATA_DEFAULTS=026. Unlike the sm_86 / sm_60 tables these are not bit-exact (the fused prompt
path and the FP4 activations round differently; 24 of 24 greedy answers were identical), so the start line does not call them exact.
moe_fused_iq.cu: on a GB10 the work-item rule never reached its 128-row tile (the launch average is above 112 rows an expert at every
prompt size), so pick_ww takes the 128-row tile from 65,536 routed rows a launch up (+3% prompt at 16K and 32K; 4K keeps 64).
STRATA_PF_TILE_DEBUG=1 prints the choice. Other parts take the line they took before.
- ds4_gg_cpu.cpp: amd_family() on a non-x86 CPU read no variable the rest of the function uses; it now reports no AMD family
- exl3_cpu.cpp: kGemmK moved above the AVX-512 section, as exl3_cpu_gemm_scratch() sizes by it on every CPU
- ds4_model.cu: uma_free(): on an integrated NVIDIA GPU on Linux cudaMemGetInfo counts only MemFree, while the model files' page cache
holds most of what the kernel can still give back. The placement, the CPU-lane warm-up and the vision tower size their room from
MemAvailable less DS4_UMA_HEADROOM_GIB (default 6) when that is larger. A discrete GPU, HIP and Windows are not touched.
On a GB10: 12,163-12,226 of 15,744 experts in graphics memory where cudaMemGetInfo's free placed 11,961.
#3622 made the strata target compile on aarch64. This adds what a Spark build needs beyond that:
- the trellis dispatcher (tq_cpu.cpp, its scalar rows) is part of the arm64 kernel library, as it is in the Metal build, so the
DeepSeek engine's CPU lane links; ds4_tq_cpu.cpp names the AVX units on x86-64 only
- a STRATA_PORTABLE build on arm64 sets ggml-cpu's -march to armv8.2-a+dotprod+fp16+i8mm (STRATA_ARM64_ARCH) instead of the build
host's; GGML_NATIVE OFF alone would leave plain armv8-a and no NEON dot products
- the x86 kernels' parity programs and benches are not configured on arm64; the test suite defaults off there (-DSTRATA_BUILD_TESTS=ON
still builds what builds)
- spark/build_engine.sh: the reproducible build of strata, strata-dsv41 and the picture reader into an engine folder, with the CUDA
libraries beside them (RUNPATH $ORIGIN)
Every change is behind STRATA_CPU_X86 being OFF or inside a new file; the x86 and Metal builds do not take it.
The 0.2.7 DeepSeek changes (persistent-grid IQ kernels, int8 rounding batches, hc_mix_split with a block counter, head columns,
chunk-path knobs) are kept; the Metal engine's hooks sit on top of them: every kernel launch is strata_ds4_launch(), the rounding
batch ends in the Metal multi-quantiser dispatch, the head stays at 4 columns a pass on Metal (8 moves the KL), and the macOS
parallel expert prefetch is macOS-only.
Same 15 files as the 0.2.6 bump (issue template, AGENTS.md, CMakeLists.txt 0.1.52, desktop and host package files, Cargo.toml and the strata-desktop entry in Cargo.lock, SettingsUpdates comment, qa1-storage test, metal 0.1.52-metal, build_hip_linux.sh PKGVER 0.1.52, strata_cli PRODUCT_VERSION, the two version tests), plus the Strata-0.2.7-<file> names in the md files.
The engine's own tower (319 MiB) through strata launch and setup (--vision off still turns it off); setup fetches
the pinned 972 MB mmproj; the server sends pictures to the tower. Checked on 4x R9700, 1x R9700 and a real
launch on 2x A6000 (download, pack, picture answer over HTTP); 0/10 stalls, KL unchanged. Also: a newer shard 1
from --model-path (ISTA's 2026-10-26 chat template) is accepted after its own size and SHA-256 check.
Pipelined groups beat the serial default at 2 and 4 streams on 2x A6000 (+17/+22%) and 2x A4000 (+7..12/+17..20%);
one stream equal, -4% on A4000 IQ3_XXS. STRATA_BATCH_GROUPS_NV_SERIAL=1 or STRATA_DEFAULTS=026 gives serial.
ISTA re-published (and patch_chat_template.py rewrites) the first shard of each set with a larger chat template, so a shard 1 on disk can be 11 KB
larger than the pinned revision's. setup refused it (size mismatch; the GGUF preflight does not read DeepSeek tensors, so it was not taken as a
whole own file). The pinned revision stays the download; the newer shard 1 is accepted by its own size and SHA-256 with --gguf-dir.
native_arithmetic_parity failed on an Intel i9-13900K (AVX2 + AVX-VNNI, ggml built native)
in five of its eight configurations: "gu=16 down=20 ff=96 nt=2 token=0". Not the VNNI row
path: STRATA_NO_AVXVNNI=1 changes nothing, STRATA_NO_IQ4NL=1 passes. The IQ4_NL down
reference (iq4nl_reference_rows) differs from that ggml's one-token dot on the scalar tail
block of a row with an odd number of blocks (ff=96 is three): that ggml does not contract
`sumf += d * sumi` into an FMA, the reference lets the compiler do it. On the other
machines tested the two agree, so it is a property of the ggml build, not of the CPU.
With an unfused tail in the reference all eight configurations pass on that machine.
The invariant path now checks each reference once against the linked ggml-cpu on synthetic
blocks before using it (iq4nl_reference_ok for IQ4_NL with 2, 3, 5 and 20 blocks;
iq512_reference_ok for types 18, 21 and 22 with SwiGLU, on AVX-512 CPUs). On a mismatch
it prints one line and the invariant path uses ggml's own one-token dot per token: slower
on such a build, never different. The Default path is not touched; only the opt-in
STRATA_SPEC_PROB=1 decode reaches this code.
Tests: native_arithmetic_parity, eight configurations, on the i9-13900K (4,866,048
comparisons, with the fallback line), z440 and the owner PC (see the series notes).
0.2.1 made a layer split over exactly two NVIDIA cards with --batch and no --batch-groups run one serial group (measured
then on 2x RTX A4000, IQ3_XXS: +12..16%). With the 0.2.7 code the pipelined groups are faster at 2 and 4 streams on
every box and quant measured, so the default goes back to the pipeline.
Measured (tools/serve_load.py, --batch 4, 256 tokens per request, interleaved pairs, total tok/s, pipelined / serial):
- 2x RTX A6000, IQ3_S, all experts in VRAM: 1 stream 101.7 / 101.7; 2 streams 125.7 / 107.4 (+17%); 4 streams 171.4 / 141.0
(+22%). The v0.2.6 binary on the same box: serial default 104.9 and 137.2, --batch-groups auto 124.6 and 169.6.
- 2x RTX A4000, Q2_0 pack (17037 of 24576 pairs cached, 4 clean pairs): 2 streams 80.2-82.5 / 75.0-75.5 (+7..9%, 4 of 4);
4 streams 110.8-111.5 / 94.8-95.3 (+17%, 4 of 4); 1 stream 63.6-68.7 / 68.2-70.3.
- 2x RTX A4000, IQ3_XXS (13056 of 24576 pairs cached, 3 pairs): 2 streams 73.2-73.7 / 65.8-66.5 (+12%, 3 of 3); 4 streams
100.5-100.7 / 82.9-83.6 (+20%, 3 of 3); 1 stream 58.4-60.5 / 60.7-62.6 (serial about 4% faster, 3 of 3).
The single stream runs on the solo path in both; only the A4000s show the 4%.
What changed: batch_groups.hpp has a pure rule nv_serial_wanted(env value, STRATA_DEFAULTS=026). STRATA_BATCH_GROUPS_NV_SERIAL
set to anything but 0 asks for the serial group, an explicit 0 wins over STRATA_DEFAULTS=026, and with the variable unset or
empty STRATA_DEFAULTS=026 gives v0.2.6's serial group. An explicit --batch-groups (a number or auto) still wins. Three or more
cards, AMD builds and --batch-mtp/--adapt-async handling are unchanged. The serial message names the opt-in.
Tests: batch_groups_test (the rule over the env values and 026, two cards pipeline, the existing serial cases). Real engine on
2x A6000 (IQ3_S), no --batch-groups: the engine says "2 groups of 2 slots" / "2 groups of 1 slot"; with 4 and 2 slots, greedy,
greedy+seeded sampled, slots finishing at different lengths and a cancel in the middle of a draft, every slot's tokens equal its
solo decode and the pipelined build's. A long-prompt slot differs from the pre-merge base build in the same way with the flag
and without it (it is the parent's prompt path, not this change).
Not covered: no run on AMD or on three cards; the 4% single-stream cost on the A4000s is measured on one box.
THIS COMMIT ALONE MAKES THE WALK A DEFAULT. Drop it and the layer-major prefill stays
an opt-in (STRATA_PREFILL_LAYER_MAJOR=1) with the chunk-major path as the default.
Rule (include/strata/core/sm120_series.hpp, pure, tested in src/core/arch_defaults_test.cpp):
- prefill_layer_major_default(hip, stages, v026): on for two or more stages on CUDA,
off on one GPU, in a HIP build, and under STRATA_DEFAULTS=026 (qwen_defaults_026());
- prefill_layer_major_resolve(env, ...): STRATA_PREFILL_LAYER_MAJOR=0 / 1 (any text
atoi reads) wins in both directions, over the rule and over STRATA_DEFAULTS=026.
Every decline of the earlier commits stays and runs in lm_prepare as before (HIP,
image prompts, DFlash2 taps, KVA, stop_at_chunk, a pending force_chunk, a peer GPU,
APR / NVFP4 / WMMA / fused layers, STRATA_EXPERT_NIBBLE, no VRAM, no pinned RAM):
such a prompt reads chunk-major with nothing but a log line. When the walk is on by
default (nothing set) the VRAM margin is 512 MiB instead of 96 and the super-chunk
target 32768 tokens (the size measured) instead of 8192; "a short prompt" is not logged.
STRATA_PREFILL_LM_TOKENS and STRATA_PREFILL_LM_MARGIN_MIB still win.
Measured (z440, 2x RTX A4000 16 GB, Qwen3.8-Flash-Next IQ3_XXS, experts streamed, 5
interleaved pairs, time to first token, medians): 8K 2.8x, 20K 4.8-5.7x, 32K 5.1-5.5x
faster (141 s to 26-28 s); decode 38.6 vs 38.4 tok/s; needles 12 of 12 found in both
arms. The speed-up is larger than the report's 2x because --prefill auto picks 512-token
chunks there. The first-token KL numbers are in the gate notes (3976-kl.sh).
Not covered: one GPU (stays opt-in), HIP, SYCL, Metal, three or more stages, a card
with more VRAM than the pack needs (where the experts are VRAM-resident and the
chunk-major path has little to re-read).
Problem. #3948: with STRATA_QFUSE=1 the verified text changes and MTP acceptance drops, on CUDA. The QFUSE kernels are bit-exact
(gr_parity, gdn_parity, 10,500 random windows each on sm_86 and sm_120, after 91a4b8a67), yet the engine's greedy text still differed
from QFUSE off on every prompt tried. Model run, l620 (RTX PRO 5000, sm_120), Qwen3.8-Flash-Next IQ3_S, MTP --spec 6, identity-gate
flags, 4 prompts of 4K and 12K tokens, 96 greedy tokens: QFUSE on differed from off on 4 of 4 (the off-vs-off control was identical).
Cause. A window of one token advances the GDN state and the indexer tail itself ("self-commit") so that commit() needs no commit
graph. Verifier::record_window decides this with
`T == 1 && !always_publish_ && !batch_rec_ && !g_qfuse() && one_token_self_commit()`: with STRATA_QFUSE the fused GDN output
kernel takes the q8_1 destination and the window does not self-commit. Verifier::commit() skipped the commit graph with
`last_t_ == 1 && !always_publish_ && one_token_self_commit()`, without the QFUSE term. So with QFUSE on, every accepted one-token
window left the recurrent state (and the indexer tail) one token behind. Windows of one token are the ordinary case when the drafter
offers nothing, which matches the report: fewer drafts offered, lower acceptance, different text, and an unchanged text where
no window of one token occurs. On HIP (where QFUSE is the default on gfx1151) one_token_self_commit() is off unless
STRATA_ONE_TOKEN_COMMIT=1, so the two sides agreed there.
Change. commit() takes the same condition: `... && one_token_self_commit() && !g_qfuse()`. Without STRATA_QFUSE nothing changes.
Result (same l620 run). QFUSE on with this change against QFUSE off, same build: logits md5 and the 96 greedy tokens byte-identical
on all 4 prompts (IDENTITY PASS); the build without the change differs on all 4. The default path (QFUSE unset) of the merged head
against the f-027-int2 head (2a2264c21): first-window logits and greedy tokens byte-identical on the same 4 prompts.
Not covered. Serve answers and draft counters (3948-qfuse.sh step 2, SERVE=1), sm_86, and the reporter's 40-prompt set.
The report asks for the first data mismatch between the GGUF row bytes and what the direct reader delivered, and for an early
check of the gathered values; neither exists. Its trace stops at "the embedding has 96 non-finite values" without saying whether
the reader gave a wrong row or the table holds one.
With STRATA_PLE_CHECK=1 (a tester switch, off by default; one static flag read per gather call when off) every direct-mode decode (read_row, collect,
gather_batch, also from the prefetched slots) checks the row's 160 values. For a value that is not finite or beyond 1e5 (an IQ4_NL
row cannot decode beyond 8.3e6; real PLE values are orders smaller) it re-reads that row from the file with a buffered read and prints
one line to stderr (at most 8 per table): the row, its file offset and page offset, the value, and either "the delivered bytes ARE the
file's bytes" or "THE DELIVERED BYTES DIFFER FROM THE FILE from byte N" with the first 8 bytes of each. That separates a reader
fault (the bug) from a damaged table. gates/3602-ple-direct.sh sets it in its direct arms.
Checked: ple_reader_test --selftest on z440 with the switch on prints the "ARE the file's bytes" line for the synthetic table's random
rows (which decode to huge values); with it off the output and the bytes are unchanged (ple_reader_selftest passes).
Not checked: the "DIFFER" branch has no natural trigger (that is the point); it was read, not run.
Problem. #3948: with STRATA_QFUSE=1 the verified text changes and MTP acceptance drops (RTX 3080, sm_86). The report suspected the
group counter or the nat_xq_ offsets of fused_gr.cu; the audit and the new two-group test (previous commit of this stream)
rule those out. What the new test found instead, on an RTX 5070 (sm_120, CUDA 13.0, MSVC), over 10,500 random windows of two groups: in
about 1 run of 400 one q8_1 code of the fused image differs by exactly one from native_quantize_q8_1's, in every mode (group-relative,
window-indexed, direct, graph replay). The scale/sum pair and the other codes agree.
Cause. native_mmvq.cu, which holds native_quantize_q8_1 (the quantiser every consumer of a q8_1 image was written for), is
compiled with --use_fast_math (CMakeLists.txt, the native_* list), so its `amax / 127` and `x / d` are the fast full-range division
(div.full.f32, up to 2 ulp off). fused_gr.cu (the hyper-connection read's tail, gr_up_q32_kernel) and verify_kernels.cu (the GDN
output norm, gdn_q8_1_store) are not in that list and divide with IEEE rounding. roundf(x / d) then lands on the other side of a .5
at the rare element whose quotient is next to a boundary. One changed activation code per ~10^6 values is about one per few
hundred tokens at 80 hyper-connection reads of 2560 values per token: enough to move a near-tie in the verify, and the
draft (which quantises with the separate kernel) disagrees with the verifier a little more often, which lowers acceptance. That
matches the report (acceptance 0.755 against 0.79, different text, and an unchanged text where no draft is made).
Change. include/strata/kernels/q8_1_finite.hpp: q81_div(a, b) is `a / b`, except in a translation unit that defines
STRATA_Q81_FASTMATH_DIV (CUDA device code only, never HIP), where it is the same PTX division the fast-math file gets (div.full.f32).
q8_1_quant and the scale use it. fused_gr.cu and verify_kernels.cu define the macro; nothing else does, so every other kernel,
the separate quantiser and every HIP build compile to what they did. STRATA_QFUSE is off by default on CUDA (it is on by default on
gfx1151, which is HIP and unchanged), so the default path is bit-identical.
Result (RTX 5070, sm_120): gr_parity's two-group window test: 10,500 runs with 1,500 random inputs for each T, all modes, zero
differences (before: dozens); the whole of gr_parity, gdn_parity and gdn_parity with STRATA_GDN_SPLIT=1 pass.
Tests. gr_parity: the two-group test prints the first differing bytes (block, field), and STRATA_GRP_REPS=N / STRATA_GRP_KEEP_GOING=1
repeat it longer and count every failure. gdn_parity: the q8_1 destination check uses its own stream (native_quantize_q8_1 refuses the
null stream, which crashed the test on Windows), unbuffered stdout and STRATA_GDN_PARITY_TRACE=1.
Not covered. sm_86 (the report's card) has not run the new test yet (z440 was reserved for other jobs); the cause is not
architecture specific. The model run (gates/3948-qfuse.sh) is still owed: QFUSE on against off must now be byte-identical, and the
draft counters equal.
Problem. #3844: --spec 5 measured +6.1% decode on an RTX 5060 Ti, but a saved preset that carries it was left out entirely
("outside the safe values [3, 4]"), and an imported one was refused. The tuner's BOUNDS are the values it tries; they are
not a finding that a value is harmful. The never list (--spec 6 and wider: 4% to 16% slower, --expert-cache 0, ...) is.
Change (Terminal / Control; touches strata_control/tune_knobs.py, strata_control/presets.py, strata_cli/tune_commands.py and
serve/server.py).
- tune_knobs.check_setting_parts(args, env, ctx) returns (never, bounds). check_setting is the two lists together, in the
order it had, so the tuner still skips --spec 5 and its tests are unchanged.
- presets.overlay applies a preset whose only problems are outside BOUNDS and returns info["warnings"], one short line each
("--spec 5: not tried by the tuner, kept because it is your setting"). A never-list setting still means "not applied".
presets.create and import_preset refuse only never-list settings.
- serve/server.py prints each warning as "[strata] warning: ..." when the preset is applied; strata_cli tune_commands
_effective adds "warnings" to its payload.
Tests. tests/tuning/test_presets.py: a preset with --spec 5 is saved, applied (the flag becomes 5, the input config is not
modified), warns once, and an in-range preset has no warnings; the never list still refuses --spec 6; import of --spec 5 works.
tests/tuning/test_autotune.py: --spec 5 stays in the tuner's refused list. python -m pytest tests/tuning: 274 passed;
tests/terminal -k "tun or preset or overlay": passed.
Not covered. The app's Details text does not show the warning yet (it reads the payload key above).
Problem. #2408 / PR #2409: restoring a large parked conversation on a layer split ended the server with a CUDA out of memory
on a later card. The PR checks the free VRAM of every card against --vram-reserve-mib before a restore. Its own review
points out that a loaded card is always below that reserve, so the check would skip nearly every restore on a split (and
on one GPU). It is not ported.
What a restore allocates. conversation_snapshot_restore writes into the session's existing buffers (pools(st), the GDN
state, the indexer tails); refresh_residency only launches kv_stream_reset or copies into the existing ring. The elastic K/V
growth (kvg_ensure) allocates, but only on one GPU. The one thing a restore can create on a card is the calling thread's
private copy stream for it (conversation_copy.hpp private_stream), which it takes from the pool of finished threads when
there is one (#3350). A failed restore is fatal by contract (the session may be half written). So the need is one stream
and one event, and only for a card whose stream does not exist yet (a snapshot loaded from disk, or a restore on a thread
that never copied on that card).
Change. conv_cache.hpp: kRestoreFirstUseMib (64) and restore_fits(free, stream_ready, n, need): a restore is refused only
for a card that has no copy stream yet and less than 64 MiB free. conversation_copy_stream_ready(device) says whether a
card has one. generate.cpp: before the restore, each card of the run is checked; a snapshot that would not fit goes back to
the cache and the prompt is read again, with a log line. A card that has its stream is never refused, however full.
No sample (a failed cudaMemGetInfo) means the restore goes ahead as it did.
Tests. conv_cache_test: the predicate (cards with a stream are admitted at any free size; the edge at the need; no
sample / no cards / no need). conversation_copy_streams_test: copy_stream_ready before and after the first copy, in the pool
after a thread ends, and the measured cost of a card's first copy stream and event (printed; the test requires under
16 MiB, so 64 MiB leaves a wide margin). conversation_snapshot_test: after the first restore, six more restores of two
images leave the free VRAM where it was.
Not covered. The pure predicate and the generate.cpp code build on z440 (CUDA 13.0); the GPU tests are listed in the stream
notes. No model run: gates/2409-restore-guard.sh (a held-VRAM start on a layer split) shows the skip line and an
unchanged answer on the second turn.
Problem. On a layer split the later cards are left with very little VRAM after loading (#3702: 68 MiB on the last card; #3638:
268 and 666 MiB, then every batched prompt read of more than ~400 tokens ends non-finite after hours; #3797: the weight arena
fails with 320 MiB free and the message does not say who holds the rest). The window-graph eviction (12b0bce95) and the copy
stream pool (#3350) are in already; this is the rest of the code audit of what is allocated after load.
Audit. The table of every lazy allocation after load (what, when, size, how a failure is handled) is in
phaseA-tests/notes-027-gh-eng-d.md. Bugs found and fixed here, each small:
- verify.cpp capture / capture_batch and mtp.cpp finish_capture: the cudaGraphUpload and its sync were only printed. The
upload is a warm-up, but a failure (no VRAM) stayed as the thread's pending error and the next kernel wrapper's
cudaGetLastError ended the engine. The error is now cleared and the first launch uploads the graph.
- verify.cpp capture_commit and mtp.cpp finish_capture: the commit graph called cudaGraphInstantiate directly (no
eviction of older layouts, unlike the window graphs) and reported the status of the end-of-capture call, which is success,
so the message read "no error". capture_commit now goes through instantiate_evicting and reports the instantiate's status.
- instantiate_evicting instantiates into a local and sets the member only on success (what a failed instantiate leaves in
its output is not specified, and a non-null member means "captured").
- prefill.cpp carve: after a failed take of the shared region the sub-allocators had no base and each cudaMalloc'd its
buffer (hundreds of MiB in all) on the card that had just run out. carve now returns at once; the caller reports the
failure as before.
- generate.cpp: a failed set_stage_helper / set_peer is reported and the run goes on, but its CUDA error stayed pending;
cleared.
Found and not changed: relend() re-plans only the first participant (#3638 comment). Read: it runs only from the VRAM
command, which is off on a layer split, so it cannot touch a later stage's loan. Other candidates (ignored cudaEventCreate for
the pipelined fences, unchecked expert copies, cudaFuncSetAttribute flags set before success) are in the notes.
Changes for the report of #3797 and the reserve (#3702).
- include/strata/program/gpu_report.hpp: on a failed weight-arena allocation (first card and each stage) the engine now
also prints, for every card of the run, the free and total memory and the largest other programs that hold memory on it
(NVML, loaded at run time; where the driver does not list them it says so). Text only; the failure is the same.
tests/program/gpu_report_test.cpp pins the text.
- include/strata/program/stage_reserve.hpp: the arithmetic of stage_room (what the split keeps free on a card) moved out of
generate.cpp unchanged; tests/program/stage_reserve_test.cpp compares it with the old expression over a grid. The file
also holds a PROPOSAL for a larger window term on a later card (96 MiB + 6 MiB per further window size + 48 MiB, 174 MiB
for --spec 4) which nothing calls: the default placement is not changed. gates/3702-later-reserve.sh measures it.
- The low-VRAM warning of a later card now adds how many MiB the plan meant to keep free there and how far off that was.
Tests: gpu_report_test, stage_reserve_test (host only, built on z440).
Not covered: no model was run. What the ~700 MiB the plan does not count on the last card is (window graph sets, kernel code
loads) needs the run in gates/3702-later-reserve.sh.
Problem. #3948 reports that STRATA_QFUSE=1 changes the verified text and lowers MTP acceptance on an RTX 3080 (sm_86); the
control (QFUSE off against off) was noisy too. The reporter read the code and suspects the per-group counter in
fused_gr.cu gr_q8_tail (not reset when a graph replays with another T) or the nat_xq_ offsets of a window of two groups.
Audit (no model run). The counter is reset by the block that finishes the group (atomicExch before it reads the 32 values),
so a launch leaves it at zero whatever T is; two blocks own a 32-column group in every kernel that calls gr_q8_tail, the
32-column kernel (gr_up_q32_kernel) quantises on its own without a counter; the fence pattern is the usual
threadFenceReduction one. The two destinations are consistent with their consumers in verify.cpp: the attention half's
xq_ is indexed from the group's first token (the projections read it from row 0), the expert half's nat_xq_ by the window's
token (the experts read it from tb). The GDN side (verify_kernels.cu gdn_q8_1_store, rows from t_out_begin) matches its
consumer too. I found no defect by reading. One thing the tests below will show if it matters: nvcc ignores the
`#pragma clang fp contract(off)` in gdn_q8_1_store, so on CUDA the ds.sum field could differ by one ulp from the separate
quantiser (the q8 bytes would not).
Tests (synthetic data, no model).
- gr_parity: the engine's G = 2 call pattern - the window split at (T + 1) / 2, two fused_gr_read_multi calls in a row on
one stream with the same counters, group-relative destinations (xq_) and window-indexed ones (nat_xq_), direct and as a
graph replay; fresh random inputs for every one of 200 repetitions for each T = 2..8; after every run both groups' q8_1
bytes equal native_quantize_q8_1 of `mixed`, nothing outside the written ranges changed, and all counters are back at zero.
- gdn_parity: gdn_step_norm_multi with a q8_1 destination for every T and t_out_begin (verify and commit calls): the bytes
equal native_quantize_q8_1 of the output rows, the outputs and the state are the ones of the call without it, and
nothing is written past the rows. A second ctest, gdn_parity_split, runs all of gdn_parity with STRATA_GDN_SPLIT=1 (the
4-blocks-a-head step, which has its own output-norm kernel).
Not covered. The reporter's model run (QFUSE on/off, MTP, 40 prompts): gates/3948-qfuse.sh. The tests were built on z440
(CUDA 13.0, sm_86); their GPU run is listed in the stream notes.
Problem. #3602 reports a deterministic all-logits NaN with --ple-io direct on IQ3_S (Windows, two GGUF shards) where
--ple-io mmap passes; a diagnostic build found non-finite values already in the gathered PLE embedding of one token. The
bytes the model got were not the file's (the largest finite value, 7.9e6, is what a random fp16 scale gives an IQ4_NL
row). This commit audits the reader and builds the test the report asks for; it does not claim to have found that
fault.
Audit (no model run). src/ngram/ple_reader.cpp, src/platform/direct_file.cpp and the consumers (PleTable gather_batch /
prefetch_rows / collect, prefill's double-buffered ple_gather, verify's windows): rows are copied and cached under the
reader's mutex after a complete read, short reads are checked per row, straddling rows get a two-page read, the offset
arithmetic is 64-bit, the set-associative cache and the slot buffers are only touched under the lock, and prefill's two
host buffers are ordered by the ple_copied events. I found no way to hand out a row before its read completed or from a
stale buffer.
One real bug, in the error path. After a failed read the I/O worker parks for good (worker_loop), but issue() went on
queueing jobs, so the next collect() waited for them for ever. issue() now returns a ticket with no jobs once the reader
has failed, and collect() returns the error at once (and drops the ticket). Same change in the SYCL copy
(sycl/src/ngram/ple_reader.cpp).
Tests.
- src/ngram/ple_reader_stress_test.cpp (ctest ple_reader_stress): 7 table shapes (header 0/192/4000/4051, rows of 1, 90,
110, 160 and 4096 bytes, with and without bytes after the table) x 15 reader configurations (io thread or caller,
in-flight 1/2/3/64, row cache of 0, 16 and 100000 rows, injected completion delays, keep-alive), four client threads
with one to three tickets outstanding collected in any order; every delivered row is compared byte for byte with a
buffered read of the file; collected buffers are compared again later (a late write would show); rows on every page
phase, the last rows, out-of-range rows; the reader after a failed read (the file is cut short under it); with --big a
5 GiB sparse table with rows around 2^31 and 2^32 and at its end.
- table_is_consistent_under_concurrent_use in src/ngram/ple_reader_test.cpp: one PleTable used by a prompt thread
(big gather_batch), a decode window (prefetch_rows, gather_batch) and single gathers at once, against the mapped reader.
Results (z440, Linux, g++ 13, reader + direct_file + test built alone): plain build with --big OK; ASan+UBSan OK. The
error-path test against the unchanged reader hangs (the test exits 3 after 20 s); with the change it passes. ThreadSanitizer
flagged a data race that was a mistake in the test (the delay changed from a second thread in the caller-thread mode,
which has no lock); fixed in the test.
Not covered. The NaN itself: it needs a run on the reporter's configuration (gate script gates/3602-ple-direct.sh). The
Windows run of the new test is listed in the stream notes.
Gyro-L (agentionai's 50 GiB file) stores its experts as a pair trellis: 128
values per block, a fp16 scale and 64 steps of K bits (K = 5, 6, 7), two
weights per step taken from the first pair of the TQ2_T codebook entry the
step's state selects. The engine refused the file because it did not know the
three type ids. This adds them next to the existing trellis types 144-146
without changing those.
What changed:
- gguf_reader.hpp / tools/gguf_reader.py: names and block sizes (42 / 50 / 58 bytes).
- tq_cpu.*, tq_states.hpp, tq_avx2.cpp: tq_supported / tq_kbits / tq_block_bytes
for the new ids (the bytes per block are 2 + 8K for the pair layout, 2 + 4K
for the others), a scalar pair kernel (the reference) and an AVX2 pair kernel
that is bit for bit equal to it; AVX-512 uses the AVX2 pair kernel.
- iq_kernels.cu: TqInfo, the lane decode (tqp_states8, a byte window), the
dot product, the decode-once Split, the fp16 dequantizer and the format lists
for 147-149. They are reached only by a pack that holds these types.
- moe_mmq.cu, strata_ggml_tq_overlay.cmake, third_party/ggml-tq/tq.cuh,
ggml-common-tq.h: the MMQ prompt path for the three types (overlay
enumerators, traits, tile loader branch, instances).
- native_expert_parity: names tqp5-7 and skips its synthetic check, with a
message, on a ggml that has no trellis traits.
- docs/GYRO.md: the "not supported" sentence now says how Gyro-L is read.
Changes to the contributor's patch:
- tq_cpu.cpp called the AVX2 pair kernel without the x86 guard the rest of the
file has, which breaks the Metal build (it compiles tq_cpu.cpp without
tq_avx2.cpp): the pair rows go through one helper that falls back to the scalar kernel.
- moe_mmq.cu named GGML_TYPE_TQP5-7 under STRATA_MMQ_TQ, but the public
agentionai llama.cpp has the older trellis types and not these: a new
STRATA_MMQ_TQP (set for the overlay and for a tree that has the tqp instances)
guards them, so that tree still builds. The HIP overlay lists get the three
instances too, and ggml_tq_overlay_test checks them (and sets the CMake
policy its IN_LIST needs).
- native_expert_parity: the skip test compared a type-name pointer to null, but
ggml_type_name of an id past the table reads past it (the test crashed on
an upstream ggml); it compares against GGML_TYPE_COUNT.
- docs/GYRO.md keeps the plain description and leaves out the contributor's
measured speeds (we have not repeated them).
Tests added: src/kernels/cpu/tqp_cpu_test.cpp (no ggml, no model: the scalar
kernel against a bit-by-bit reference from the format definition, every ISA
equal to scalar for 1-8 tokens and for gate/up rows, the type predicates for ids
0-299 against the old formulas), the pair types in tq_pascal_bench (fp16
dequantizer bit for bit against the definition, decode kernels within 3e-2,
decode-once == per-column), tools/test_gguf_tqp.py, and native_expert_parity
tqp5/6/7 registered when the ggml tree has the tqp instances.
Checked (no model): tqp_cpu_test passes on x86 (AVX2 on z440; AVX2 and
AVX-512 on the owner PC with MSVC); tq_pascal_bench on an RTX 5070 (sm_120,
MSVC build): 0 failures, the TQP dequantizer matches the definition in all
bits, and the checksum lines of TQ2_T/TQK6/TQK7 are byte-identical to the
build without this change. Builds: CUDA sm_86 (gcc), sm_60 (CUDA 12.9),
sm_120 (MSVC), SYCL, and the CPU files for aarch64.
Not covered: the contributor's own parity runs need a ggml tree that carries the
types (not public here), and the 50 GiB model run (load check, coherent
answer, 32K prompt, MTP, Gyro-S/M identity) is still to be done. Metal, SYCL and
HIP have no kernels for the types; HIP was read, not built.
Problem (KeyanHu-git): the gated proposal needs the choice between the MTP window and a
lookup window to be fixed by the history, not by the draws of the rounds it will verify.
Freezing the policy for the whole request (the first commit) does that, but also freezes
the cost table, so a repeat-heavy request kept the window sizes it inherited. In the
author's diagnostic workload the frozen policy chose mean lookup windows of 5.87 and 3.38
on two repeat requests; with this change 4.31 and 4.05 (from the PR text; not
reproduced here).
Change: the request's copy of the DraftPolicy (source_policy) gets three calls, used only
when gated_proposal is on:
- observe_committed(): the round's accept and probe counts, applied at once. A lookup
window counts as a probe of its size (tried_), so a size whose cost is only guessed is
tried kProbes times, not every round.
- defer_cost(end, t, ms): the round's time is held. A time covers the window and the next
MTP draft, so it depends on every Philox counter up to
max(p + T - 1, p + a + 1 + S_mtp).
- apply_costs(committed): applies, in order, the held times whose end is behind the
committed position, with the same update observe() makes (including the STRATA_SPEC_COST
clamp), in the context bucket the round ran in; delayed_n_ keeps a held time from
counting as a second probe.
choose() counts the probes as cost_n_ - delayed_n_ + tried_.
Port note: the DraftPolicy on the current tree keeps its costs per context bucket
(STRATA_SPEC_COST), so tried_, delayed_n_ and the held times carry the bucket. A gated
request does not run with STRATA_SPEC_COST (it falls back to the older rule), so its
copy stays in bucket 0.
Default path: unchanged. choose() reads two new terms; on any path that never calls the new
functions tried_ and delayed_n_ are +0.0 and cost_n_ - 0.0 + 0.0 == cost_n_ exactly.
Checked by driving the old DraftPolicy (namespace-renamed copy of f-027-gh-eng) and the
new one with the same random call sequences (observe, observe_chain, observe_probs,
set_context, seed_costs, set_robust, choose, chain, keep_draft; 400 sequences of 3000
steps): 25,983,538 bitwise comparisons of picks, costs and rates, 0 mismatches; one
observe_committed call makes the same harness report 60,528 mismatches. generate.cpp:
nothing calls the new functions when gated_proposal is false.
Tests: draft_policy_test (the PR's added assertions, on the bucketed class) and
spec_admission_test pass on z440 and on the owner PC.
Not covered: the sampling argument for the delay boundary (that a held time cannot steer
a window through a counter that is drawn again) is the author's; a unit test cannot show
it, only the distribution run (gate script 3416-spec-prob-dist.sh, arm NEWLK) can.
Problem (KeyanHu-git): the rejection rule of the previous commit judges each row against
the target's probabilities p. For a native (IQ) pack the CPU experts round a token's rows
differently depending on how many tokens share its expert in the window (the grouped
kernels against ggml's one-token dot, #152). So p of one row moved with the window width,
and the same sampled text could depend on the draft length. On random blocks a grouped
Default call differs from the token alone in 700 to 12000 of about 18000 output floats
for the IQ gate/up and IQ4_NL down formats.
Change: NativeArithmetic { Default, WindowInvariant }, fixed for one pool batch.
native_gu_rows / native_down_rows become thin wrappers of one template with the arithmetic
as a parameter. The Default instance keeps every old branch (the new conditions are
compile-time true there). WindowInvariant gives each token the bits of the one-token call
on this build and CPU: kernels that already match it stay grouped (NVFP4, Trellis and
Hadamard rows, KQ256 when ggml runs its AVX2/FMA dot); the IQ rows that differ use
iq512_reference_gu_rows (types 18, 21, 22: the same decode, ggml AVX2's accumulation
order, one weight decode per group) or iq4nl_reference_rows (IQ4_NL down), and anything
else falls to the one-token dot per token. ExpertPool::run_split_multi_native takes the
arithmetic and expert_pool_dispatch_multi passes it; generate.cpp uses the invariant
pool only when the MTP drafts probabilistically (mtp.prob(), i.e. only with
STRATA_SPEC_PROB=1). Prefill, greedy decode and every other decode run Default.
The pool picks the entry point with one compare and a direct call per chunk.
Default path (STRATA_SPEC_PROB unset): unchanged, shown by running the old and the new
native_gu_rows / native_down_rows (Default) on the same random blocks: 61 format, width
and amplitude lines (IQ2_XXS/XS/S, IQ3_XXS/S, IQ4_XS, NVFP4, Q4_K, Trellis with and
without folded Hadamard, IQ4_NL / Q2_0 / Q5_1 / Q8_0 down; widths 1 to 8; row sub-ranges),
output hashes identical in 10 settings of the environment switches on z440 (AVX2) and
in 13 on the owner PC (Zen4, AVX-512, MSVC), including STRATA_NO_IQ512, STRATA_IQ_GATHER
and STRATA_IQ_MT_MIN=1. g++ codegen of the AVX-512 row kernels is not byte-identical for
5 of 87 functions (inlining and register allocation of the same source); the AVX-512
kernels were run only under MSVC.
Tests: native_arithmetic_parity, eight ctest configurations (default, AVX, KQ256,
IQ_MT_MIN=1, no512, ggml only, NVFP4 AVX2, NVFP4 ggml), registered in CMakeLists.txt and
sycl/CMakeLists.txt. It compares the invariant entry points at every group width 1..8
with the one-token calls, bitwise, for the IQ, NVFP4, Q4_K/Q5_1/Q8_0, Trellis and
Hadamard formats. All eight pass on z440 and on the owner PC; the existing CPU expert
tests (iq_avx2_parity, iq_avx1_parity, iq512_gather_parity, pool_tasks_test,
nvfp4_cpu_parity, router_dot_parity, q2_bitplane_parity) pass.
Not covered: an ARM CPU (not built here), an AVX-512 machine with fast gathers (Sapphire
Rapids), the layer-split call site, and any model run.
Problem (KeyanHu-git, refs #2028, #2591): with STRATA_SPEC_PROB=1 the default
--spec-min-p gate tests the token drawn from the draft head's distribution q, but the
verifier still judged the draft against the unfiltered q and treated a failed gate as a
plain sample from p. That shifts the output distribution: with p = (0.5, 0.5),
q = (0.875, 0.125) and threshold 0.5 it emits (0.5625, 0.4375).
Change: the gate is part of the proposal. Keep q's mass on the tokens that pass the gate
without renormalising (mu); a draw that fails the gate becomes NULL. An offered token D
is accepted with min(1, p(D)/mu(D)); a rejection and NULL draw from the normalised
positive part of p - mu; a position with no attempted proposal is a plain sample from p.
Serve and the CLI build the same packet (core/gated_proposal.hpp) and give it to the
verifier explicitly (Verifier::set_spec_proposal, sample_tokens_proposal). The explicit
call has no plain-p fallback: an invalid packet row is reported, not sampled around.
STRATA_SPEC_PROB_GATE=top keeps its normalised-q rule.
Default path (STRATA_SPEC_PROB unset): unchanged. mtp.prob() is false, so gated_proposal
is false, the new draft-result argument is null, GatedDraftLimit does nothing,
Verifier::run takes the old branch (set_spec_q clears the measure flag), and the
sampler kernel's old instance is the old code with the new branches compiled out.
Read hunk by hunk; no hunk runs a new branch or a new allocation per token. The
default-identity model run is written as a gate script and not run yet.
Port notes against the PR as submitted:
- Unsupported combinations (parallel or multi-device decode, --lookup-chain,
--adapt-swaps, PCIe share, --spec-min-p not finite, STRATA_SPEC_COST, DFlash2,
STRATA_SPEC_T_CYCLE, --spec-corrupt, Metal) used to be an error that ended the
request or the server. They now warn once and keep the older rule, so setting the
variable never stops a run. --adapt-swaps 96 and the PCIe share are on by default, so
the exact rule needs --lookup-chain 0 --adapt-swaps 0 --pcie-frac 0.
- The lookup source of a probabilistic window is chosen by a copy of the policy fixed at
the request start, without asking whether the MTP's first guess agrees. Learning from
committed rounds inside a request is the policy commit of this series.
- The CLI loop on the current tree has the out-of-memory window retry, DFlash2 and
STRATA_SPEC_T_CYCLE; the gated path does not use the retry (a narrower window would
change the room the packet was built for). Serve keeps merged first windows
(STRATA_ADMIT_MERGE_FIRST) at their length.
- docs/DETAILS.md: two short bullets instead of the PR's long paragraph (that paragraph
no longer exists on this tree). bench/results and the PNGs are not ported.
Tests: gated_proposal_test and spec_admission_test (default, pick, top, exact-top-only)
pass on z440 (CPU, g++ 13) and on the owner PC (MSVC). On the owner PC's RTX 5070
(sm_120, MSVC build, synthetic data, no model) spec_proposal_parity, spec_verify_parity and
sampler_parity (default, one-block and old-kernel configurations) pass. Also built:
CUDA sm_86 on z440 and boston, SYCL on P1006 (engine and host tests). The Metal path is
not built here.
Not covered: a model-level distribution test (chi-square of the first tokens against
plain sampling) and the identity run with the variable unset. Both are written as gate
scripts (3416-spec-prob-dist.sh, 3416-default-identity.sh) and not run.
Prefill on one card, decode on another, session handed over through RAM; exact token for token (state hash match at 4K
and 16K on 2x A4000 and 2x A6000). Nothing changes unless split_serve / --split-serve is set. On two alike cards it is
slower than the layer split on every workload measured (docs say so); it can only pay on dissimilar cards. Idea and
manual setup by MarcoTool (#4000).
Measured on 2x RTX A6000 (IQ3_S): slower than the layer split on every workload;
exactness checks pass (Q2_0 on 2x A4000, IQ3_S on 2x A6000).
Idea and manual setup by @MarcoTool in #4000.
No copy, code-constant, JSON-spelling, retrieval or unanswerable defects (retrieval 12/12, JSON 3/3, unanswerable 9/9, extraction equal to off); two long-winded answers, which the runs with the prior off also show two or three of per run. Story +15%, code +37% against off.
f-027-gh-eng gained STRATA_EXPERT_NIBBLE (4-bit expert slots, the sm_60 experimental
build): only run_impl gathers such a layer for MMQ. The layer-major walk would read
the slot as GGUF blocks, so lm_prepare now declines when any layer is converted and
the prompt takes the chunk-major path. Compiled out of every build without
STRATA_EXPERT_NIBBLE_BUILT.
The engine did not build on aarch64: the CPU expert path assumed the x86 AVX
kernels, the CPUID probes and the x86 spin/fence intrinsics. After this change
the strata target compiles on aarch64 (CUDA host code, native GGUF packs).
On x86 nothing is selected differently.
What changed:
- CMake: STRATA_CPU_X86 is decided from CMAKE_SYSTEM_PROCESSOR (an unset
processor stays the x86 build). On aarch64 src/kernels/cpu/portable.cpp
stands in for the AVX kernel sources; native experts run on ggml-cpu's NEON
dot products. -ffp-contract=off on aarch64. STRATA_ISA_FLOOR is ignored with
a warning there.
- include/strata/platform/cpu_relax.hpp: strata_cpu_pause / strata_store_fence
map to _mm_pause / _mm_sfence on x86 and to yield / dmb oshst on aarch64.
verify.cpp, pool.cpp, session.cpp, mtp.cpp and graph.cpp use them.
- expert_layout.cpp: the CPUID/XCR0 probes compile on x86 only and report no
AVX on other CPUs.
- generate.cpp: --cpu-oracle-q8-0 is rejected on a non-x86 CPU.
- setup.py: nvidia-smi's "[N/A]" memory on GB10 (unified memory) takes the
pool size from /proc/meminfo on ARM64 only; the CPU probe no longer needs
the x86 "flags" line; setup warns that the ARM64 engine is compiled on the
PC and which packs work.
Changes to the contributor's patch, found while moving it onto the 0.2.7 line:
- portable.cpp had no iq512x_rows / iq512x_supported / iq512_gather_on (added
to the x86 kernels after the PR was written): the aarch64 link would fail.
- The Metal build (Apple silicon) shares these files. It keeps its barrier
(dmb ishst), its plain router loop (its bf16_rows_dot_multi is a stub), its
Metal start-up block in generate.cpp and its "unknown" CPU name.
- The CPU side of the GPU doorbell: strata_load_fence() after the spins that
wait for a flag the GPU raised (dmb ishld on ARM, nothing on x86 and on
Apple silicon), so the data written before the flag is not read early.
- A card whose memory nvidia-smi does not report stays unlisted on x86 (the
new fallback is for ARM64 only).
- tests/terminal/test_setup_arm64.py.
Checked: with the x86 CUDA build (sm_86, gcc 13 -O3, z440) the objects of
verify, session, mtp, graph, expert_source, pool, expert_layout and generate
are byte-identical before and after (the x86 wrappers are macros for the same
intrinsics). Every C++ file of the strata target compiles for aarch64 with
clang, and the symbols of the CPU kernel library link up (syntax and symbols
only: no ARM machine, no ARM run). The Metal-shared files compile for
aarch64-macos with STRATA_GPU_METAL.
Not covered: no ARM hardware here. The tests that include x86 headers
(iq_avx2_parity and the other AVX parity tools) and the DeepSeek target do
not build on aarch64; the strata target does not need them. Memory ordering
on ARM is reasoned, not run. The Q2_0 and Gyro packs need the x86 kernels.
DETAILS.md gets two sections: the decode switches (GRAPH_PRIO, MMVQ_IL_PF) with the
two sm_80-only defaults named, and the prompt switches (PF_FUSED_NT / BATCH / F16 /
WS, GDN_TC, PLE_COMPACT / PLE_COPY_STREAM, EMBED_VRAM, DEQUANT_SIDE) with what is
bitwise and what is rounding-level. MMVQ_IL_TABLE.md gets the sm_80 row of the table
list and a paragraph on the hoisted-load tables (kIlPf_80 / kIlPf58_80, Q6_K only) and
on measuring a card with mmvq_il_parity --bench --emit-table --pf 0,3,4.
Rewritten from the PR's text: every figure is labelled as the reporter's (one CMP
170HX; we have no sm_80 card and measured none of them), and the prompt-switch
section is new (the PR documented only the decode switches and the conversation
cache).
Original docs by linxuhao (PR #3779, commit 6d6f0f39).
Problem: the interleaved small-batch matvec (native_mmvq_il) has one weight block
in flight per warp: its four `if (kbx < bpr)` sections each sit behind their own
branch, so on a GA100 (ncu: 432 GB/s on the qkv Q6_K matrix) the loads of a group
are not issued together.
Change, opt-in (every variant is bitwise native_mmvq): native_mmvq_il_pf_kernel
issues every weight load of a group (4 virtual warps x R rows) before any of the
group's dots, with the addresses clamped to the row's last block for a tail, and
with the ring variants the next group's loads before this group's dots. The
per-lane accumulation order is the plain kernel's. P2's forms replace
conversions by exact ones where the value is a small integer: (float)p becomes
p + 1.5 * 2^23 minus 1.5 * 2^23 for |p| < 2^22, and the saturating byte subtract
of Q6_K's codes (0..63, so nothing saturates) becomes two instructions. Q4_K,
Q5_K, IQ4_XS and Q6_K have the variants; Q3_K (the Q2_0 pack's type) keeps the
plain kernel.
- STRATA_MMVQ_IL_PF=1: the measured table (kIlPf_80 / kIlPf58_80: Q6_K only, read
only when the card is sm_80; on every other card 1 does nothing);
=2..6: one variant for every type and shape (A/B only, any CUDA card).
- Unset or 0: the plain kernels through the same launch code (the variant argument
is 0, and the added code is one cached environment read and a switch).
- CUDA only; HIP has no interleaved path.
Tests: mmvq_il_parity --pf a,b,.. bit-checks each listed variant against
native_mmvq for every type and shape, T = 1..8 and rows 0/1/2/4, and with --bench
times them interleaved (PFBENCH lines) and prints the joint (variant, rows) table
in the source form of kIlPf_<cc>. New ctest entry mmvq_il_parity_pf runs it with
--pf 0,2,3,4,5,6 on whatever card is present.
Reporter's numbers (linxuhao, CMP 170HX): decode +1.4 % (two reference / on / reference /
on sets) over the plain kernels; with STRATA_GRAPH_PRIO decode +1.3 % in their
per-feature pair (4K prompts +3.9 %, 16K +1.2 %, 52K +0.2 % in the same pair, which
the reporter reads as noise). Q4_K, Q5_K and IQ4_XS variants tie the plain kernels
there. Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 116992ab).
Harness only; no engine code. The bench part of mmvq_il_parity times the
interleaved matvec cold (24 copies of the weights rotated past the L2), hot (one
copy), hot with other weights' reads in between (--gap MB, what a window's GEMVs do
between a prefetch and its consumer), and with the timed weights in a persisting L2
set-aside (--persist); --ab runs the five arms per case in one process, because
the clocks ramp over the first ~100 ms of a run. A 150 ms warm-up of launches runs
before the first timed case, and MMVQ_IL_TMIN bounds T from below.
The parity part is unchanged: without --bench the same checks run in the same
order (every rows choice against native_mmvq, bitwise). The bench code is CUDA only
(the test is built with STRATA_ENABLE_CUDA); the persisting-L2 calls are left out
under STRATA_HIP_GFX906.
Original change by linxuhao (PR #3779, commit 1f4b8139).
Problem: the decode window graph runs every kernel node at the launch stream's
priority, so the one-block indexer kernels of the side branch wait behind the
queued blocks of the big q GEMV, and the shared expert competes with the routed
experts.
Change, opt-in (bitwise by construction: no kernel changes, the same launches on
the same inputs; only the dispatch order moves): with STRATA_GRAPH_PRIO=1 the
side mixer's branch stream (GDN gates, QSA indexer chain) is created at the
device's highest priority, cs_ and side2_ in the middle, the shared expert's
stream at the lowest, and the window graph is instantiated with
cudaGraphInstantiateFlagUseNodePriority. If a driver did not copy the stream
priority into the nodes, the capture-time notes (the branch forks and the
priority streams' tails) set the attribute per node. One log line per captured
window prints the node priority histogram. A card whose priority range is 0..0
changes nothing.
CUDA only: the members and the three methods sit under !STRATA_USE_HIP &&
!STRATA_HIP_GFX906 && !STRATA_GPU_METAL; graph_prio_on() is false there.
Merged by hand with #3702 (the window-graph eviction): instantiate_evicting now
takes both keep_t and the flags, and the retry after freeing other window
sizes keeps the flags (the merged code would otherwise have dropped the node
priorities on that retry).
With the variable unset: every stream is created as before and the capture-time
notes return at once.
Tests: none of its own (a priority is a scheduling hint with no value to compare).
The window graphs are exercised by the engine; see the checklist for the
reporter's greedy-answer identity check.
Reporter's numbers (linxuhao, CMP 170HX, IQ3_S, --spec 3, 3K context, interleaved in
one session, each arm with STRATA_QIL=1): reference 125.3 / 126.6, GRAPH_PRIO
127.8 / 129.0, reference 125.0 / 126.8 tok/s, +1.8 %. Greedy 11 of 11 identical. In
the later per-feature pair together with STRATA_MMVQ_IL_PF: decode +1.3 %. Theirs, not
measured here.
Original change by linxuhao (PR #3779, commit 68c487c3).
One section in DETAILS.md for the switches the four conversation-cache commits
add: what each copies where, its VRAM and host memory cost, that they stay off
on HIP, gfx906 and Metal, and the measured times.
Rewritten from the PR's section: the times are labelled as the reporter's
(linxuhao, one CMP 170HX on PCIe Gen2 x4; we have no sm_80 card and measured
none of them), the independent RTX 3090 data of bitaccesscomau is added, and the
pool row describes STRATA_CKPT_POOL_READY=N instead of the PR's change to the
meaning of STRATA_CKPT_POOL=N (STRATA_CKPT_POOL=1 is unchanged).
Original section by linxuhao (PR #3779, commit a04f4399).
Problem: restoring a parked conversation copies its K/V from the host over the
link: 27-107 ms over a PCIe Gen2 x4 link.
Change, opt-in and only with STRATA_PARK_ASYNC=1 (CUDA only): a park also keeps
the whole K/V of its image in a VRAM slot of its own (64 MiB granules, up to N MiB
in all, only while 1.5 GB of VRAM stays free beside it); the background copy to
the host reads from the slot, and a restore of that image copies device to device
from the slot (about 1 ms) without waiting for the background copy. Each K/V
layer carries its own join-and-release token, so any order of destruction is safe
(the layers outlive their image when they are retained for the next park). Under
VRAM pressure the elastic K/V's growth releases every slot and the stage, then
retries once (both growth sites in generate.cpp).
With the variable unset the slot code is not reached: the budget is 0, a restore
finds dev_tag 0 and takes the host path, and the growth sites see
conversation_park_vram_release_all() return 0 and behave as before.
Tests: none of its own (it needs a card with the VRAM for the slot and a parked
conversation); see the checklist for the reporters' A/B/A conversation check with
STRATA_PARK_VRAM_MIB set.
Reporter's numbers (linxuhao, CMP 170HX): restore of a 3K conversation 28-47 ->
7-12 ms; outputs identical. The first restore of a run is slower (41.6 / 56.9 ms)
than later ones (6.5-12.6 ms) and they have not found why. Theirs, not measured
here.
Original change by linxuhao (PR #3779, commit bafca410).
Problem (reporter, CMP 170HX, PCIe Gen2 x4): a conversation switch's park still
paid 38-52 ms for the live state, because the STRATA_CKPT_POOL pool was empty at
every park (a switch takes two checkpoints), so the 118 MB GDN vector was
allocated fresh and page-faulted; and a checkpoint's 37 small synchronous
device-to-host copies queue behind the 4 MiB DMA pieces of STRATA_CKPT_ASYNC.
Change:
- STRATA_CKPT_POOL_READY=N (opt-in, 1..16, with STRATA_CKPT_POOL on): the pool
keeps N ready checkpoints, pre-sized to the session's state sizes with their
pages in, topped up on a background thread after any take (not with a layer
split).
- STRATA_CKPT_GATHER=1 (CUDA, opt-in): the small running-state vectors (ple, and
each layer's indexer tail, dead row and block position) are gathered device to
device into one buffer and fetched with one copy into a pinned buffer.
- The park stage is allocated rounded up to 64 MiB (a growing conversation does not
cudaFree/cudaMalloc it every time) and a VRAM-resident K/V skips the pointer
attribute query.
- The background helpers (pool top-up, the checkpoint DMA, the park DMA) leave the
host's pinned core (aux_cpus.hpp); no output depends on where a thread runs.
- A parked image whose live GDN state is still being copied joins before its
buffers are freed (SavedConversation::live_pending).
- STRATA_PARK_TRACE prints the joins, the live-state vectors' readiness and the
K/V sub-timers.
Differs from the PR as submitted: the PR made STRATA_CKPT_POOL=1 allocate two
ready checkpoints at startup (+236 MB of host RSS) and read STRATA_CKPT_POOL=N as
a count. That changes what an existing switch does, so here STRATA_CKPT_POOL=1
keeps its meaning (the pool holds dropped checkpoints only, nothing is allocated
ahead of time) and the ready pool has its own variable, STRATA_CKPT_POOL_READY.
A user of the PR's STRATA_CKPT_POOL=2 sets STRATA_CKPT_POOL=1
STRATA_CKPT_POOL_READY=2. The PR's one-checkpoint refill after a park with
STRATA_PARK_ASYNC is replaced by the top-up, as in the PR, and so is only active
with the new variable. Merged by hand into #2526's park_source.
With none of the new variables set: the stage size is rounded up only when the
park stage is on, the extra code is trace output behind STRATA_PARK_TRACE and
the join calls.
Reporter's numbers (linxuhao, CMP 170HX): park at a 3K context 64-75 -> 21-40 ms
(all park switches together, A/B/A conversation check identical, greedy 11 of 11
identical). Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 56088e71).
Problem: parking the outgoing conversation copies its K/V device to host and
deep-copies its checkpoints before the next prompt can start: 213 ms at a 3K
context, 334 ms at 13K on a PCIe Gen2 x4 card (the reporter's CMP 170HX).
Change, opt-in and CUDA only (off on HIP, gfx906 and Metal):
- STRATA_PARK_ASYNC=1: conversation_kv_save copies the device K/V device to
device into a VRAM stage (only while 1.5 GB stay free beside it; otherwise the
save copies as before), and a background thread moves the stage into the host
buffers in 4 MiB pieces through two pinned buffers. conversation_park_join
waits for it: every reader of a parked image's K/V, the cache before it frees an
entry, and the image itself (SavedConversation::pending) before its buffers go.
ConversationBuffer::resize_for_overwrite leaves the added bytes unfilled for
that copy; the buffer's segments use an allocator that value-initializes only
when a fill value is given, and resize(n, value) is unchanged for every other
caller.
- With it, the checkpoints this request drops right after the park (all of them
when another state comes in, else the ones off this prompt's path) move into
the parked image instead of being copied, and a background allocation refills
STRATA_CKPT_POOL for the next save. Only the main session's checkpoints move,
not an idle batch slot's (the park_source of #2526 serves both).
- conversation_snapshot_save takes dropped checkpoints' storage (the existing
STRATA_CKPT_POOL) for its copies; with the pool off, nothing is taken.
- Unset: park_drops is null, no storage is taken, the K/V copies are the
synchronous ones; the added work is the K/V size sum for the stage request and
a few join calls.
Differs from the PR as submitted: the PR's join in conversation_file.cpp
(put_payload) is not ported; the SAVE/RESTORE handler of generate.cpp joins the
park first (see the previous commit, which carries the PR's 33a8146a fix).
Hand-merged into the park_source / ParkSource refactor of #2526.
Not covered: if the background copy fails after resize_for_overwrite, the
parked image keeps unfilled bytes; the failure is logged. A restore of such an
image would read them. No test of the park path with a failing copy.
Tests: the PARK_ASYNC path has no unit test in this commit (it needs a card with 1.5 GB free for the
stage); the reporter's A/B/A conversation check covers it, see the checklist.
Reporter's numbers (linxuhao, CMP 170HX): park 213 -> 64-75 ms (3K), 334 -> 68 ms
(13K); greedy answers identical on their 11 prompts and the A/B/A conversation
check. Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 3e0c141c).
Problem: on a slow host link a turn checkpoint copies the whole GDN state to
the host on the request path (118 MB at 1.64 GB/s on a PCIe Gen2 x4 card, 72 ms;
an independent RTX 3090 on PCIe 3.0 x16 showed 73 ms to 62 ms with the switch).
Restoring a parked conversation copies it back.
Change, both opt-in and CUDA only (off on HIP, gfx906 and Metal):
- STRATA_CKPT_ASYNC=1: the GDN state is copied device to device into a VRAM stage
(one buffer the size of the state, allocated at the first checkpoint save, plus
a pinned host buffer of the same size; if either allocation fails the save falls
back to the synchronous copy), and a background thread moves it in 4 MiB pieces
into the checkpoint's host vector. Every reader joins first and only for the copy
it needs (restore, snapshot save and restore, the parked copies, the serve loop at
each request start, the SAVE/RESTORE session handler).
- STRATA_CKPT_VRAM_SLOTS=N (with ASYNC): the last N saved GDN states stay in VRAM,
tag-checked, only allocated while 1.5 GB stays free; a restore of a checkpoint
whose tag a slot holds copies device to device.
- STRATA_TRACE lines carry a steady-clock timestamp; STRATA_PARK_TRACE prints the
timing of the save steps.
- Unset: the code adds a mutex-guarded no-op join per checkpoint save, restore and
snapshot, a tag field that stays 0 and a few clock reads; the copies are the
synchronous ones as before.
Differs from the PR as submitted: the PR put a join into conversation_file.cpp
(put_checkpoint), which broke the conversation_file_test and
session_save_reclaim_test builds without CUDA and matched only the copied vector,
not the one the copy in flight writes, so a SAVE within one checkpoint copy after a
turn could write an incomplete GDN state (found by bitaccesscomau; fixed in the
PR's last commit 33a8146a). This commit carries that fix: conversation_file.cpp
stays as it is and the join runs first in the SAVE/RESTORE handler of
generate.cpp. The commit 33a8146a is therefore not a separate commit here.
One more join, found by running the new ctest entries: conversation_snapshot_sources (the SAVE
path of a session file) saves the running state with the background copy and the caller writes
it to the file at once, after the handler's join has already run. It now joins the copy of
that checkpoint before it returns; without it a session file written with
STRATA_CKPT_ASYNC=1 could hold an incomplete GDN state (conversation_snapshot_test failed
at "both files read" before the join).
Tests: conversation_snapshot_test now joins after each save and, with
STRATA_TEST_BIG_GDN=1 (a GDN state over 1 MiB, so the async path is taken), runs as
the new ctest entries conversation_snapshot_test_ckpt_async and
conversation_snapshot_test_ckpt_vram_slots: every checkpoint, capture and restore
must give the same bytes as the plain run.
Not covered: no test destroys a checkpoint while its copy is in flight (readers
join; the cache's eviction paths do not); the reporter saw 3 of about 35 engine
starts stop answering after a park with evictions, on the paths with the
switches off, and could not reproduce it in 15 reruns. The VRAM stage has no
headroom check of its own (an RTX 3090 with 147 MiB minimum free showed no failure
in 42 of 42 saves; the allocation failure falls back silently).
Reporter's numbers (linxuhao, CMP 170HX): turn checkpoint 71.4 -> 0.9 ms,
restore 114-126 -> 28-47 ms. Independent data (bitaccesscomau, RTX 3090 24 GB,
PCIe 3.0 x16, release defaults, 4 pairs): checkpoint save 73.2 -> 62.4 ms, tiny-prompt
time to first token median ratio 1.038, outputs byte-identical. Theirs, not measured
here.
Original change by linxuhao (PR #3779, commits 0d512925 and 33a8146a).
Problem: the prompt path's DeltaNet recurrence is a large part of the prompt time
of a GA100 card.
Change, opt-in and rounding-level (the output is not the FP32 kernels' bits):
src/prefill/gdn_tc.cuh holds the chunked WY-form recurrence with every product
on the tensor cores (f16x3 splits with power-of-two scaling, one block per value
head) and its prep kernel; kernels.cu dispatches it before the STRATA_GDN_CHUNKED
branch for prompt chunks of 128+ tokens when STRATA_GDN_TC is set.
Where it runs and where it does not:
- gdn_rec_tc returns "not supported" and the existing kernels run unchanged on a
card below sm_80, with fewer than 48 SMs, or with less than 122 KB of
shared memory a block (the kernel needs 124,944 B). That excludes sm_86, sm_89
and sm_120 (99 KB opt-in): the code is compiled into those builds and never
launched there. It runs on sm_80 and sm_90. A failed scratch allocation
(60.3 MB a device) also falls back.
- Device code for cards below sm_80 compiles to trapping stubs behind
STRATA_GDN_TC_SM80; the whole header sits under !__HIPCC__.
- Unset, the dispatch costs one cached flag test per call.
Tests: gdn_rec_parity gains the tc variant (T = 1..4099, 8192, 32768, six stress
inputs against gdn_rec_kh_kernel, gate 1e-4; "not on this card" where it cannot
run), --selftest-frag (each tensor-core helper on one warp against a double
reference) and --bench-tc. The new ctest entry gdn_rec_parity_selftest_frag runs
the helper self-test, which any sm_80+ card can run (77 below sm_80). The full
tc variant can only run on an sm_80 or sm_90 card.
Reporter's numbers (linxuhao, CMP 170HX): recurrence 105 -> 34 ms at 3.3K tokens
and 1,705 -> 547 ms at 52K; prompts +3.9 % (4K), +6.0 % (16K), +6.7 % (52K) over
the same base; within 2.3e-6 of the FP32 kernel in gdn_rec_parity; teacher-forced
KL against the bitwise base mean 0.0157, median 0.0011, max 0.098, top-1 100 %;
1 of 5 conversation checks identical and decode -2.6 % in their pair (they
think draft acceptance moves with the rounding; not checked). Theirs, not
measured here.
Original change by linxuhao (PR #3779, commit f1cabc99).
Problem: the NT = 256 kernels still decode every 32 values to int8 and scale
the int32 products in FP32.
Change, both opt-in and both rounding-level (the output is not bit-identical to the
int8 kernels; they are the PR's "rounding-level" switches):
- STRATA_PF_FUSED_F16=1 (implies the NT = 256 set): the decoded weights times
their block scales and the activations times theirs as F16, m16n8k16 F16
products into FP32 accumulators. The k order inside a 16-value group is
permuted the same way for both operands.
- STRATA_PF_FUSED_WS=1: native_ws_kernel, producer warps decode into an F16 ring
in shared memory and consumer warps run the same F16 products (named barriers),
so it has the F16 path's rounding. One block per multiprocessor; about 101 KB of
shared memory at its defaults, so it is set up on sm_80 / sm_90 cards and
says "not available" and keeps the 512-thread kernels on an sm_86 / sm_89 /
sm_120 card (99 KB). Compile-time knobs STRATA_WS_PROD / _SLOTS / _NB / _KS / _HTAB.
- Both take a WideBatch like the NT = 256 kernels, so STRATA_PF_BATCH above 128
reaches them. CUDA only (under !__HIPCC__); the Turing pass compiles the F16
instances against __trap() stubs and never runs them (dev_info stops before the
NT = 256 setup on Turing).
- With neither variable set the kernels, their launches and their shared-memory
sizes are as before (the F16 template parameter defaults to false).
Differs from the PR as submitted: when STRATA_PF_FUSED_Q2N=1 is on as well
(#3809, after the PR was written) the WS setup also sets up the Q2_0 gate/up
instance.
Tests: prefill_fused_iq_test_f16 and prefill_fused_iq_test_ws (ctest) run the
fused-experts test (double-precision reference and MMQ) with the switch set. The
test now skips (77) when a requested NT / F16 / WS kernel set is not set up on the
device, so a card that cannot run WS does not report a pass for the 512-thread
kernels.
Limits of the test for these two arms only: with STRATA_PF_FUSED_F16=1 or _WS=1 the
test uses the limits it already has for STRATA_PF_F16 (F16 products): fused against the
int8-rounded model rel RMS 5e-3 (2e-3 otherwise) and against MMQ 5e-3 (1e-3 otherwise).
Measured with F16 on an RTX A4000 (sm_86, z440), over the 11 pairs: against the int8-rounded
model rel RMS 2.85e-3 to 2.90e-3 and worst row 5.9e-3 to 8.1e-3; against MMQ the same;
against the FP32 reference 1.438e-2 to 1.480e-2 with MMQ at 1.435e-2 to 1.478e-2 (the
fused path stays within 1.003x of MMQ's rel RMS and 1.03x of its worst row). The rounding is the F16 rounding of the
weights times their scales and of the activations times theirs (an 11-bit mantissa, about
2^-11 relative per operand), which the int8-rounded model does not have; it is not a
different summation order. The same arm on an RTX PRO 5000 (sm_120): up to 2.90e-3 (rel RMS), worst row up to 8.1e-3.
The default path and the NT=256 arm keep the old limits and their printed errors are
unchanged: with no switch set the test prints identical output (52 lines) from the base
tree and this tree, on an RTX A4000 (sm_86) and on an RTX PRO 5000 (sm_120).
Reporter's numbers (linxuhao, CMP 170HX): one MoE layer of 3,256 tokens 8.27 ms
(NT = 256) to 7.90 ms (F16) and 7.65 ms (WS); prompts +2..+6 % over the
256-thread kernels, 1 of 11 greedy answers changes. Teacher-forced KL against the
bitwise base on their 20-prompt set: F16 / WS mean 0.0062, median 0.00006, max
0.076, top-1 100 %. Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 579cbfbb).
Problem: a layer split with --batch pipelines the slots in groups, and the pipelined groups run no MTP drafts.
--batch-mtp with serial windows leaves all but one card waiting. A multi-stream deployment wants both.
What changed: STRATA_PIPED_MTP=N (with --batch-mtp) keeps the pipeline and drafts in it. A group's window carries
each of its slots' rows: the slot's token and up to N MTP proposals (at most --spec - 1; the first always, each
further one while the draft layer was at least --spec-min-p sure of the one before). The last stage judges each
slot's proposals on their own. The number of rows to keep is known only then, and by then the earlier stages run
other groups' windows, which reuse the scratch the commit replays from. So a small save graph copies those rows
right behind each window (Verifier::batch_launch_rows), and every stage queues the commit of the kept rows after
the verdict (Verifier::batch_commit_rows). The slot drafter catches up on its own rows and drafts the next
proposals as an asynchronous chain (MtpDrafter::chain_launch). Unset, nothing changes.
Changes on top of the PR as sent (review):
- The verifier window size is worked out where the verifiers are built, from the slots that fit and the final
groups, not earlier from --batch as asked. Unset it is the old expression.
- STRATA_PIPED_MTP is decided again once the slots that fit are known. It says it is off, and why, when fewer than
2 groups are left, when STRATA_BATCH_SELF_COMMIT is on, or when a group's window would hold more than
kVerifyMaxT rows. The start line names groups and slots correctly.
- batch_groups::resolve keeps its old mtp_dropped rule; only mtp_piped is new, and it also applies with
--batch-groups given (the PR's fix 1). The "--batch-groups 1" test case now expects "not dropped".
- --remote-expert-opt alone no longer blocks --batch-mtp on a layer split: the object it guards is only made with
a helper expert cache (the PR's fix 2).
- docs/BATCHING.md rewritten with the numbers measured here.
Tests: batch_groups_test (the PR's cases, four "unset is as before" cases); builds on CUDA sm_86 (Linux), SYCL
(the sycl tree has its own verify.cpp and generate.cpp and does not include this code) and MSVC. Model runs:
see docs/BATCHING.md and the gate scripts 4121-*.sh.
Not covered: AMD (HIP) was not built or run; the draft layer on the first stage; helper expert caches.
2.0 showed copy and code defects on the RTX 3060 (IQ3_XXS); 1.5 showed none in 19 tasks and keeps the speed
(story +15%, code +37% vs off). The 61-task check decides 1.5 vs a 1.0 fallback.
+6.9% Qwen IQ3_S decode on Strix Halo Windows, prompts unchanged; token ids identical in 4 starts x 5 prompts.
Linux gfx1151 and other cards unchanged; STRATA_DEFAULTS=026 restores it. Docs label the DeepSeek Strix numbers by build.
The Windows guide says what 0.2.7 does and how to turn the switch back on, backed by five fixed greedy prompts
(895 ids) run four times with the switch on, off, on, off: identical ids. STRIX_HALO.md and the 0.2.7 release notes
get a line. The DeepSeek 4K prompt figures of 2026-10-25 (78.7, 87.2) stay and are labelled with their build; a table
of the 0.2.7 candidate changes one thing at a time (chunk size, DS4_DEFAULTS=026).
Interleaved A/B on a Strix Halo Windows PC (Qwen IQ3_S, engine 0.1.51): chat decode 43.6 tok/s with the fork,
46.6 without (+6.9%), prompts unchanged. Linux gfx1151 and every other architecture keep their table.
STRATA_SH_STREAM=1 turns it back on, STRATA_DEFAULTS=026 keeps v0.2.6's table.
arch_default_env(arch, defaults026, windows) is a pure rule; arch_defaults_test checks both platforms.
Ported PRs (authors kept) and our fixes, plus the owner's part-2 items: the restored TROUBLESHOOTING entries with a
link check, #3895 (no shared-expert fork calibration on Windows HIP with --kv-resident; a warning on a hand-set
STRATA_SH_STREAM=1), #3930 (docs), #3421 (Docker stops when models are present and MODEL_PATH is unset), #3570
(launch --show-key reads setup's config), #3311 (engine paragraph only) and #3029 (Strix Halo Windows guide,
re-measured on 0.2.7).
Structure from the guide in PR #3029 by HaiYunChuang (what the PC reaches, memory and the BIOS, the configuration, what is on by default, traps),
rewritten for the 0.2.7 install and with every number measured again: the Terminal and the gfx1151 engines built from f-027-int2 (3e468f5de), a
local signed test feed, strata launch as a standard test user on the NucBox EVO-X2 (Adrenalin 26.10.1), tools/strata_bench.py over a tunnel.
Qwen IQ3_S: 44 tok/s chat, prompts 755 / 821 / 792 tok/s at 4K / 16K / 32K. DeepSeek 2.00 bpw: 12 to 14 tok/s chat, prompts 214 to 342 tok/s.
An interleaved A/B of STRATA_SH_STREAM on Windows gfx1151 (three pairs): off is 6.9% faster in chat decode, prompts unchanged.
What launch chose and logged is quoted instead of a hand-made config. The first guide's tuning rows (engines 0.1.41 to 0.1.47) are not repeated.
Co-authored-by: HaiYunChuang <[email protected]>
Part 1 fixes and 12 ported PRs (authors kept), and the owner's part-2 items: the Monitor readings strip with
local currency (#3628), approval notification + opt-in minimise to tray (#3477), opt-in done sound (#3586),
app port setting with a busy warning (#3402) and interface size up to 200% (#3970). Host tests: only the two
known base failures on Linux; connector-setup and the new tests pass on Windows.
Problem: Gemm::native dequantizes each dense weight to FP16 in front of its
GEMM on the compute stream, so the dequant never overlaps the kernels queued
ahead of it.
Change, opt-in (CUDA only, always off on HIP): with STRATA_DEQUANT_SIDE=1 the
prompt path takes a second GEMM scratch (64 MiB more of its region, counted in
bytes_needed and carried through relayout) and Gemm::set_dequant_side hands it
to the Gemm. native() then dequantizes the next weight on a non-blocking side
stream into the scratch the compute stream read last but one. Four events order
every write and read of each scratch (free -> dequant -> ready -> GEMM -> free),
so each product sees the same bytes as in the serial form. The row-sliced path
(a weight larger than the scratch) stays serial. Metal answers false and keeps
the serial form.
Differs from the PR as submitted: if the side stream, its events or the
setup fail, the PR failed the prompt-path init. Here one line is printed and the
dense weights are dequantized in front of their GEMMs as before. Unset, nothing
changes (gemm_dequant_side() is one cached flag; the scratch, stream and events
are not created).
Test: new gemm_dequant_side_parity (ctest, GPU, synthetic, 77 without a device):
eight Q4_K / Q5_K / Q6_K weights of the engine's shapes (one larger than the
scratch, so it takes the sliced path) run three rounds through Gemm::native
serially and with the side stream, with and without a spin kernel queued ahead of
each GEMM; every output must be identical bit for bit.
Not covered: dense weights read by the side stream are the resident ones; a weight
that arrived by an upload on another stream would need its own event (the prompt
path has none that do).
Reporter's numbers (linxuhao, CMP 170HX): measured with STRATA_PLE_COPY_STREAM and
STRATA_EMBED_VRAM, the four PLE / embedding / dequant switches together give
+2.2..+2.6 % prompts. Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 741fc6ab).
DS4_DEFAULTS=026 behind the DeepSeek speed-ups Off toggle, the 12 GB NVIDIA route-prior row, the Advanced-only
Qwen '0.2.6 behaviour' row (STRATA_DEFAULTS=026), and the app-ui Quit test fix. Terminal suites pass; host suite
failures on Linux also fail on the base.
Owner decision for 0.2.7: the tables of PR #3453 (the cells where an RTX 4070 and an RTX 4080 community run gave the same rows count, and
the one-column Q3_K row) ship on for sm_89 cards, untested on Ada by us. The output is bitwise the same with them off (a table only
changes how many rows a warp takes); no end-to-end gain was seen on a 4090 or 4080 in the report, so the reporter tests 0.2.7 and reports.
The rule is a pure function in sm120_series.hpp: mmvq_il_89_default(hip, cc, v026) is true for cc 89 only, false in HIP builds and under
STRATA_DEFAULTS=026 (defaults_026.hpp, as 38c9080db did for the other defaults); mmvq_il_89_resolve() adds the variable: "1" and "0" win,
unset or other text is the rule. native_mmvq.cu reads it once. arch_defaults_test checks cc 89 on, every other cc off, HIP off, 026 off,
"0" off, "1" wins over 026, the empty text. mmvq_il_parity expects the on-tables by default and the old ones with =0 or 026.
Docs: MMVQ_IL_TABLE.md, DETAILS.md and the 0.2.7 release notes (a line that says it is not measured on an RTX 40 card, with the off switch).
French catalogs for the app (3,528 strings + fix/tune/tools catalogs) and the served web page (247 keys),
README.pl.md from the current README (#3355, Jakub Luwierski), Polski in every README language row, and the
README.md language row's double-encoded names fixed. Tests: 21 host files 180 pass on p1006, Terminal and web
i18n checks on l620.
serve/web/locales/fr.json (247 keys, plurals one/other, same terms as the app catalog) and the French line in serve/web/i18n.js. tools/web_i18n.py and serve/test_web_i18n.py (node and python) pass.
Ports the Polish translation from #3355 and the Polski entry in the language row of every README. The pull request translated the older long README, so README.pl.md is rewritten from the current README.md (logo, centred language row, DeepSeek speed tables including Strix Halo, Gyro, coffee button). README.md's own language row had garbled names (double-encoded text); it is written out correctly.
Co-authored-by: Jakub Luwierski <[email protected]>
fr.json, fix-fr.json, tune-fr.json and tools-fr.json (3,654 strings, vous, plain short words), the Francais entry in the language picker, every locale test and gap check now covers fr, a french-catalog test (spacing before ? ! : ;, apostrophes, short labels, no leftover English), scripts/fr_sync.py to list and merge strings added later, and scripts/language-shots.mjs for screenshots of the app in one language.
Wide/narrow CSS (#3176), page translations with a language choice, opt-in KaTeX math (#2972, #2974),
sandboxed svg/html preview (#1628), Load/Unload in the header (#3550), and one chat-history design
(#1787, #2615, #3823, #4004). Tests: browser suite on the owner PC, node + pytest serve suites on boston
(457 passed, 2 skipped).
Problem: on a GP100 die the verify window's gate/up kernels for the IQ2_S,
IQ3_XXS and IQ3_S expert formats are not held by memory bandwidth but by each
block's serial chain (unpack the codebook indices, the emulated dp4a, the
16-lane reduction); the same kernel with every dot product removed still takes
about two thirds of its time.
Change: with STRATA_EXPERT_NIBBLE=1 the engine, after the expert cache is
filled and every expert is resident, transcodes the gate/up blocks of every
fully resident IQ2_S / IQ3_XXS / IQ3_S layer (n_embd 2560, n_ff 640) into slots
of 4-bit codes, 16 bytes per 32 weights, with the integer scales and fp16 d kept
as they are; the down rows stay GGUF. The decode gate/up reads those slots with
kernels that run the production integer chain and float tail in the production
order (R4S / R4P / R4SV / R4PX by format and group size); the prompt path's MMQ
gather rebuilds the GGUF blocks from a slot on the fly. At load every item is
round-tripped and every slot's rebuilt GGUF blocks are compared with the staged
blob before the cache is sealed (ExpertCache:🦭 a later fill_slot stops the
engine). New: include/strata/core/expert_nibble.hpp (slot contract, switches,
per-layer table), nib_gu_proto.cuh (STRATA_NIB_GU_PROTO=1 variant), the
transcoder, hash and rebuild kernels and the 4-bit gate/up kernels in
iq_kernels.cu, generate.cpp (the fill, the VRAM accounting, STRATA_EXPERT_NIBBLE_
RECHECK), iq_nibble_parity and native_expert_bench --nibble.
Review changes against the PR:
- An option the switch cannot work with (--peer-device, --kv-grow, --vram-elastic,
remote caches, STRATA_MMQ_DIRECT, ..., a card that is not cc 6.0, a build
without the kernels) now prints a warning and the start goes on with the usual
GGUF slots, where the PR refused the start. What is known only later (an expert
out of VRAM, a loan, the prompt chunk) still refuses, since slots are
converted by then.
- The checks this switch adds to expert_source.cpp and prefill.cpp are compiled
out of every build that has no 4-bit kernels, so the default builds add nothing
to the hit split or the prompt loop.
- iq_nibble_parity gets --synthetic-only (every gate/up format x both down
types on random experts, no model file), skips with 77 on a card that is not a
GP100, and the synthetic and --header runs are ctests.
- The PR's docs section is rewritten for these changes and its measurements are
labeled as the author's.
Default path (switch unset): the table is empty, nibble::slot_bytes(l) is
expert_layout().blob_bytes(l) and nibble_max_slot_bytes() is max_blob, every
sizing the PR touched returns what it returned before, and the new branches in
generate.cpp (guard_raw before the adaptive tier's copies, the profile fill's
test), expert_cache.cpp (one bool in the fill calls and admit) and verify.cpp
(a capture-time layout choice, compiled out without the kernels) are not taken.
Proof on the kernels: in the sm_60 build (CUDA 12.9) the SASS of all 446
kernels iq_kernels.cu had before is identical with and without this change,
as is that of the 603 in native_mmvq.cu, 29 in qsa_select.cu and 10 in
moe_mmq.cu; 21 kernels are new.
Tests: iq_nibble_parity --header passes on z440 and boston (the host half of the
contract; also a ctest). The GPU parts (--synthetic-only, a model shard) refuse a
card that is not a GP100, so they were only compiled here (sm_60, CUDA 12.9,
P1004: iq_nibble_parity and native_expert_bench build; the whole tree builds).
sm_86 build without the SM60 option (z440): exit 0, the 4-bit paths compiled out.
Not covered: no GP100 run. The model gate is
phaseA-tests/gh027/gates/3455-expert-nibble.sh: iq_nibble_parity --synthetic-only
and on a model's layers on a P100, then identical replies with the switch on and
off (a layer is converted only when every expert of every layer is resident, so
the rig needs the pack to fit whole), then the decode A/B. The contributor's
numbers (four GP100 dies, Swift 1.5 IQ3_S, 47 of 48 layers converted, +12.7 GiB
VRAM, decode 47.3 -> 51.9 tok/s, +9.7%; prompt reading 2.3-3.0% slower; replies
of 15 requests equal) were not re-measured, and the enabled path was written
against an older base than this branch's.
Problem: on a GP100 die (compute capability 6.0: no dp4a, no tensor cores) the
prompt pass's routed experts run as MMQ launches per 16-expert group: a
gate/up, a swiglu, a quantize and a down launch per group, each group's experts
gathered and padded to the group's largest row count, every MMQ tile decoding
its weight blocks again. With every expert resident the per-group overhead, not
the weight bytes, is a large share of a prompt read on this card.
Change: with STRATA_MMQ_WS=1, a layer whose routed experts are all resident on
this GPU runs quantize_scatter, then gate/up as ONE weight-stationary launch
over all its experts (read where they lie in the expert cache), ONE
swiglu_quant over all its rows, then the down as ONE launch. New
src/prefill/moe_ws*.cu(h): a CTA holds 128 (or 256) weight rows of one expert
and runs them against up to 64 of that expert's activation rows, a thread per
weight row, so each weight block is decoded once per pass; gate/up for
IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS and IQ3_S, down for Q2_0 and IQ4_NL. Every
product keeps MMQ's int8 values, scales and per-32-value epilogue in its K
order, so GU, H and the down rows are MMQ's bits. ws::faster carries the
author's measured ws / MMQ time ratios per format and picks a layer only where
both products win at its rows per routed expert. prefill.cpp decides per layer
after quantize_scatter (all experts resident, this GPU alone, no peer or CPU
share, not the fused layout, not STRATA_MMQ_DIRECT, no Hadamard, no NVFP4 tail,
no FP4 activations, aligned, not STRATA_DBG_NAN); every other layer keeps its
path. STRATA_WS_FAST=1 (second opt-in, same bits) selects a down kernel for the
IQ4_NL 32-token tier with fewer instructions per pass.
Gating: the kernels and STRATA_PREFILL_WS exist only in the CUDA build with
STRATA_EXPERIMENTAL_SM60 (not HIP, not Metal; prefill.cpp links stubs that
answer mode 0 elsewhere). At run time ws::mode() is 1 only with STRATA_MMQ_WS=1
and every visible GPU at compute capability 6.0. Unset, nothing is allocated
(the offset and chunk buffers, under 1 MiB per die, exist only when the mode is
not 0), the arena layout and the mapped table are the ones before, and the
per-layer test short-circuits on the cached mode.
Not ported: moe_ws_down_bench (needs a model shard) and its CMake target.
Added: ws::set_mode (tests only) and tests/cuda/prefill_ws_parity_test.cpp, a
synthetic layer-level check: random experts of all 5 gate/up x 2 down formats,
random routing, the per-group MMQ (mmq::Context per expert) against ws::gate_up
+ swiglu_quant + ws::down, GU, Hq and Dm compared byte for byte, the down also
alone on the reference H and with STRATA_WS_FAST on. It gives its verdict on a
compute capability 6.0 card only (the kernels follow MMQ's sm_60 SASS); other
cards run it for information with --any.
Tests: prefill_ws_parity_test on boston (2x RTX A6000, sm_86, CUDA 13.1, gpu lock,
SM60 build, --any because it is not a GP100): 40 cases (all 5 gate/up x 2 down
formats, four routings each), 0 declined; the largest GU and down-alone error
against that card's MMQ is 5.2e-07 of the largest value (float rounding: MMQ's
sm_86 epilogue is not the sm_60 one the kernels follow), the full chain's Dm
differs up to 1e-3 where int8 rounding of H flips on those rounding
differences, and STRATA_WS_FAST output equals the default kernel's in all
cases, 0 differing bytes. The byte-for-byte verdict against MMQ is the P100's.
sm_60 compile of the whole tree (CUDA 12.9, P1004): exit 0; sm_86 build without
the SM60 option (z440): exit 0. SASS of the 10 kernels moe_mmq.cu had before
this change is identical in the sm_60 build.
Not covered: no GP100 run here. The contributor's numbers (prompt reads 31-37%
shorter on four GP100 dies with every expert resident: 240 tokens 2,156 -> 1,364
ms, a fresh 26K read 41,231 -> 28,555 ms; one run per arm) were not re-measured.
The model gate is phaseA-tests/gh027/gates/3752-mmq-ws.sh.
Problem: the token-embedding table of a native pack lives in mapped pinned host
memory, so every prompt's scattered row gather crosses the PCIe link. On a Gen2
x4 link that costs more than the table's size in VRAM is worth.
Change, opt-in: with STRATA_EMBED_VRAM=1 the table is copied into the current
device's VRAM when the table plus 512 MiB stay free (322 MiB for an IQ4_XS
table). The gather kernels read the same bytes through dev_ as before. If the
VRAM is not there or the copy fails, the table stays in mapped host memory as
before and one line says so. The start-up line now says where the table is
("in VRAM" also for the two existing VRAM fallbacks, which it called mapped
host memory).
With the variable unset nothing changes (one getenv at load).
Not covered: a layer split across several devices places the table on the
device that is current when it loads, as the existing VRAM fallback does. No
unit test (it moves the same bytes to a different memory; the gather kernel is
unchanged).
Reporter's numbers (linxuhao, CMP 170HX): measured together with
STRATA_PLE_COPY_STREAM and STRATA_DEQUANT_SIDE on a 51.8K-token prompt, the
embedding step went from 191-201 to 70 ms. Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 952dd784).
Problem: with STRATA_PLE_COMPACT the raw-row upload sat on the compute stream
before layer 1, which idles for the DMA (4.7 MB per 3K-token chunk, 2.9 ms on a
Gen2 x4 link).
Change, opt-in, only with STRATA_PLE_COMPACT=1: the raw rows go up on a
non-blocking stream of their own once the previous chunk's dequant has read the
device buffer (ple_raw_free event); the compute stream waits for the upload
(ple_copied) only at the dequant. Same bytes, same kernel. The stream is drained
when a run ends so no upload is left in flight from a buffer the next run
refills.
Differs from the PR as submitted: if the stream or the event cannot be created,
the PR failed Prefill::init. Here it prints one line and the rows go up on the
compute stream as with STRATA_PLE_COMPACT alone. With the variable unset
nothing changes (the stream and the events stay null).
Reporter's numbers (linxuhao, CMP 170HX): PLE time of a 51.8K-token prompt 95-103
to 47 ms, measured together with STRATA_EMBED_VRAM and STRATA_DEQUANT_SIDE
(+2.2..+2.6 % prompts for the four PLE/embedding/dequant switches together).
Theirs, not measured here.
Original change by linxuhao (PR #3779, commit 0ba47e61).
Problem: the prompt path reads 16 PLE rows a token, dequantizes them on the
host and uploads 10,240 floats a token. On a slow host link (PCIe Gen2 x4) that
upload is on the critical path of the first layer.
Change, opt-in with STRATA_PLE_COMPACT=1: for an IQ4_NL table held in memory
(--ple-io ram / mmap) the prompt path copies the raw 90-byte rows (1,440 bytes
a token) into the same pinned buffer, uploads them and runs
ple_iq4nl_dequant on the device (d times the code's value, one multiply, the
split-half nibble order of iq4nl_dequant_row). Any other table (Direct I/O,
pageable fallback, another format) and a failed allocation take the host path
as before. With the variable unset nothing changes: the new code is behind one
cached flag, and the extra members of the Prefill state stay null.
Two things differ from the PR as submitted:
- PleTable::gather_raw_iq4nl writes a row past the table as scale 0 and code 8
in every nibble. The PR wrote zero bytes, which the device decodes to
-0.0f (0 times code 0's value -127), while the host path writes +0.0f for such
a row.
- New test ple_iq4nl_dequant_test (ctest, synthetic, no model, 77 without a
device): the device decode against iq4nl_dequant_row, bit for bit, over every
fp16 scale pattern with random codes, random rows, an odd row count and the
out-of-table row.
Reporter's numbers (linxuhao, CMP 170HX, PCIe Gen2 x4): prompts +1.0..+1.7 %
(4K, 16K) with the switch on. Theirs, not measured here. On a Gen4 x16 card
the gain will be smaller.
Metal gets a stub (the Metal prompt runs through its window and never calls
it).
Original change by linxuhao (PR #3779, commit da818261).
Problem: the prompt path stores H (swiglu of the gate/up rows) as floats and
then quantizes it to the q8_1 rows the down product reads: two kernels and an
H round trip per MMQ group.
Change: swiglu_quant_kernel, a block per row (128 threads, 4 values each,
blockIdx.y the row's 512-value half), computes swiglu_kernel's expression for
each value and quantize_mmq_q8_1<D4>'s result for each 32-value block, so hq
holds the same bytes and H is never stored. swiglu_quant_ok(type) says where
it applies (a q8_1 D4 down type, no FP4 activations). Built only in the CUDA
build with STRATA_EXPERIMENTAL_SM60; HIP and the other builds get
swiglu_quant_ok() == false and a no-op. Nothing calls it in this commit; the
weight-stationary path of the next commit does. prefill.cpp gets the matching
stubs for builds without the MMQ sources.
Tests: prefill_swiglu_quant_test (new, registered under STRATA_BUILD_TESTS in
the SM60 build, skips with 77 without a device) compares the fused kernel with
swiglu + quantize byte for byte for Q2_0, IQ4_NL and Q8_0 down types, split and
interleaved gate/up rows, a zero row and a row with one huge value.
Run on z440 (RTX A4000, sm_86, CUDA 13.0, SM60 build, gpu lock): all 18 cases print
"0 of N bytes differ" (1, 37 and 4113 rows) and the test ends "prefill_swiglu_quant_test:
ok". sm_60 compile of the tree (CUDA 12.9, P1004): exit 0.
This is a partial port of another contributor's prompt-path fusion change;
only the kernel is taken, as in PR 3752.
Problem: on a GA100 card (70 SMs) the 512-thread fused prompt-expert kernels
leave the card idle at every barrier and at every grid tail.
Change, all opt-in:
- STRATA_PF_FUSED_NT=256 sets up a second set of native_kernel instances with
256 threads and two blocks a multiprocessor (WW 2 only), with two-pass weight
loads (the loads of the next k step are issued before the current step's
decode; load_unit_l gives the same words as load_unit). Every output is the
same thread's sum in the same order.
- STRATA_PF_BATCH=N launches up to N experts at once (WideBatch holds 512 blob
pointers, 4 KB of kernel parameters: CUDA 12.1+, Volta and newer; HIP, gfx906
and the Pascal build keep kMaxBatch). Above 128 it needs the NT = 256 kernels.
Below 128 it only changes the launch grouping.
- With neither variable set the 512-thread kernels, their launches and their
load sequence are unchanged: tile_rows(ww, 512) equals the old 256 / ww, the
two-pass code is behind a constant-false TWO_PASS, and the one added test in
experts_native is occ_nt > 0, which only STRATA_PF_FUSED_NT=256 sets.
- If the 256-thread instances cannot be set up (shared memory, occupancy) the
engine prints one line and keeps the 512-thread kernels.
Cost: the NT = 256 instances are new device code in every CUDA build (the
reporter measured 365,632 bytes for sm_80).
Tests: new ctest entry prefill_fused_iq_test_nt256 runs prefill_fused_iq_test
(the fused kernels against the double-precision reference and the MMQ path)
with STRATA_PF_FUSED_NT=256; it exits 77 below sm_80.
Reporter's numbers (linxuhao, CMP 170HX): one MoE layer of 3,256 tokens
8.27 ms with NT = 256; prompts +7.6 % (4K), +4.9 % (16K), +4.3 % (52K) for
NT=256 with BATCH=512 on the PR's own build. Greedy answers identical on their
11 prompts. These are theirs.
Original change by linxuhao (PR #3779, commit 056d36a9). Merged onto the Q2_0
fused path of #3809 (STRATA_PF_FUSED_Q2N=1) without textual conflicts; the PR
predates it, so one line is mine: the NT = 256 setup also sets up the Q2_0
gate/up instance when Q2N is on (without it that instance would have launched
without its shared-memory attribute).
moe_shared_init and moe_scatter_add came with the layer-major walk and had no test.
lm_kernels_parity runs them on synthetic data of the model's width (N = 2560, no
model, a few MiB) against a host reference:
- moe_shared_init: acc = shared * sigmoid(sg), within 1e-5 relative (the device
sigmoid uses __expf);
- moe_scatter_add: the per-token fma chain in row order, bitwise equal to the host
chain and bitwise equal between two launches (no atomics), over groups of one
row, one sub-chunk, several sub-chunks, one row per token and eight per token.
Registered in CMakeLists.txt next to gr_upmix_parity (CUDA builds, not gfx906),
SKIP_RETURN_CODE 77 when no device is visible.
Not covered: the walk itself needs a model (see the 3976 gate scripts).
Problem: an sm_80 card took the shared interleaved-MMVQ row tables, which
were measured on other cards (#3252 reports A100 slowdowns).
Change: kIlRows_80 (2-4 columns), kIlRows58_80 (5-8 columns) and kIl1Rows_80
(one column), looked up under the key 80 only. They were read off
mmvq_il_parity --bench --emit-table with the same 3 % rule as the other tables.
STRATA_MMVQ_IL_SM80=0 maps an sm_80 card to the key 81, which has no table, so it
gets the shared table again (and no one-column table, as before).
A table only picks the rows a warp (0, 1, 2 or 4); 0 means native_mmvq's kernels.
mmvq_il_parity already checks every rows choice against native_mmvq bitwise for
every type and shape, T = 1..8, so any entry of the new tables is covered.
Reporter's numbers (linxuhao, one CMP 170HX): decode +2.7..+3.2 % over the shared
table, Qwen3.8-Flash-Next IQ3_S. The measured card is not an A100; the rows may
want re-measuring there.
Original change by linxuhao (PR #3779, commit eaedca84); the commit message here
is rewritten. The sm_89 table of #3453 is a separate change in the same list.
Problem: sm_80 (GA100: A100, A30, CMP 170HX) ran the per-group dequant kernels
that sm_61 and sm_86 had already left for the wide ones, because
dequant_wide_on() had no sm_80 entry.
Change: dequant_wide_on() answers yes for compute capability 8.0 as well. The
test is the raw (major, minor) of the device, so it is false on every other
card (sm_60, 70, 75, 86 keep their answer, sm_89, sm_90 and sm_120 stay off as
before). STRATA_DEQUANT_WIDE=0 still turns the wide kernels off.
Reporter's numbers (linxuhao, one CMP 170HX, GA100 sm_80 with 70 SMs): the GPU
time of a 3,259-token prompt went from 857 to 792 ms, prompts +6.3..+7.2 % at
3.3K tokens and +3.9 % at 52K. We have no sm_80 card; these are theirs.
Bit identity: the wide kernels are the ones sm_61 and sm_86 already run by
default. dequant_wide_parity compares them with the per-group kernels byte for
byte for FP16, BF16, FP32, the padded stride and a row slice. The kernels have
no sm_80-only code, so the result does not depend on the card; what only an sm_80
card can show is that ptxas for sm_80 keeps the same arithmetic (see the
checklist in phaseA-tests/gh027/gates/3779-sm80-checklist.md).
Original change by linxuhao (PR #3779, commit 48dc535e); the commit message
here is rewritten.
Problem: a request declares tools; the model writes a complete call for a name that is not among them (bash for shell, a dropped
prefix, an MCP name with every "__" written as "_"). Since #3075 such a call stays text, which is the rule we keep, but the client
saw an ordinary answer with finish_reason stop and nothing else, so the agent turn ended with raw XML and the model never learned
of its mistake. Reporters on OpenCode, Pithagoras, OpenClaw, LibreChat, Open WebUI and Pi hit it.
What changed (declared-only stays: nothing undeclared is ever returned as a call, finish_reason is unchanged):
- serve/frontend.py OutputParser: a declared name that contains "__" is also read when written with every "__" as "_"
(mcp_playwright_browser_navigate for mcp__playwright__browser_navigate); the call goes to the client under the declared name.
Same rules as the other aliases: only when unambiguous, and a tool declared under that spelling wins. A reordered or renamed
tool (server1_label_values for label_values_mcp_server1, bash for shell) is not guessed.
- serve/server.py: when a block went back as text for an undeclared name, the reply carries
"strata_rejected_tool_calls": [{"name": ..., "reason": "undeclared"}] (the names the parser already collected, up to five): at the
top level of a Chat Completions reply and in its last streamed chunk, and in the message_delta event and the message of
/v1/messages. It is absent when nothing was rejected, so every other reply is unchanged. The text itself is untouched.
- The server window line says the response carries the names; the count goes to totals["tool_calls_unread"] and to
/metrics/prometheus as strata:tool_calls_unread_total.
- docs/DETAILS.md: the alias, the marker, the counter.
Tests: serve/test_tool_name_alias.py (collapsed spelling at several chunk widths, a declared spelling and a plain tool unchanged,
a tool declared in the collapsed spelling wins, two tools with one collapsed spelling are not guessed, reordered/renamed names stay
text and are named), serve/test_server.py ForcedToolChoice (the field in Chat Completions whole and streamed and in /v1/messages,
absent for an ordinary answer and a declared call, the collapsed name is a call; the counter). The whole serve/ suite: 1055 tests,
two failures that also fail on the base tree (test_fatal_recovery on this PC, RestartWindow.test_not_alive_until_ready under load).
Not covered: returning an undeclared call as a real tool call (the owner keeps declared-only); a block for a DECLARED name that came
back as text because it was malformed (counted, no name to list); the /v1/messages field with a real client; a stderr or log-file copy
of the warning.
Problem: a decode window's QSA select runs one CTA per query for the top-k
(block_topk_wide_kernel past ~135K cells) and the warp-shuffle block scorer
(block_scores_multi_kernel), in each QSA layer. Both grow linearly with the
context: on one GP100 die at 300K cells a top-k call takes 0.39 ms with 1-5 SMs
busy and a scores call 0.22 ms (two queries). At short contexts the same work is
small and an extra kernel is pure overhead.
Change: qsa_select.cu (inside !__HIPCC__ && STRATA_EXPERIMENTAL_SM60) gets
block_topk_multi_* (six kernels over several CTAs per query, one per 2,048
blocks of capacity, at most 32, exchanging through caller scratch: no clusters,
no spin waits, capturable) and block_scores_dx_kernel (the multi scorer's
per-head sums built in registers as the same 32-leaf xor-butterfly tree, no
__shfl_xor_sync, the tail block stays block_scores_tail_kernel's). New public
qsa_topk_multi_scratch_ints, qsa_block_topk_long and qsa_block_scores_long
forward to qsa_block_topk / qsa_block_scores outside the build gate or their
conditions. verify.cpp keeps a second set of window graphs (exec_[sel_long_][T])
for windows that reach STRATA_SELECT_LONG_AT cells (default 65536); the choice
is made before a capture and handed to the next stage of an overlapped layer
split, capture_all captures both, pl_launch picks by the window's position.
The ids and the score bits are the replaced kernels'. STRATA_TOPK_MULTI=0 and
STRATA_SCORES_DX=0 turn either half off inside the long graphs.
Review changes against the PR: in a build without the multi-CTA kernels
STRATA_SELECT_LONG=1 now says so and is ignored (the PR would have kept two
sets of graphs for nothing there); Metal gets stubs for the three new calls
(the SYCL tree has its own verify.cpp and qsa_select copy).
Default path: unset, select_long_on() is a cached false, topk_scratch_ is not
allocated, sel_long_ stays 0, and every window uses exec_[0][T], the graph it
used before; the index change is the only difference in verify.cpp.
Merge note for f-027-gh-eng (#3702 evict_window_graph, not in this branch's
base): its exec_ / exec_nr_ / launched_ accesses need the same [sel_long_]
first index.
Tests: src/kernels/qsa_select_long_parity.cpp (new; registered with
STRATA_SCORES_DX=1 STRATA_TOPK_MULTI=1 in the SM60 test block, and a second
case with STRATA_TOPK_MULTI_N=3; skips with 77 without the kernels) compares
qsa_block_scores_long with qsa_block_scores (nq 1-8, n_bid 8 .. 131072, tail
blocks of 0-3 cells, every block 0..n_bid memcmp) and qsa_block_topk_long with
qsa_block_topk (nq 1-16, contexts 1,000 .. 524,288, random scores, all-equal
scores and 7 distinct values for ties across CTAs, selected ids of every query;
the multi scratch is junk-filled and must have been written). z440 (RTX A4000,
sm_86, CUDA 13.0, gpu lock, build with -DSTRATA_EXPERIMENTAL_SM60=ON): "172
cases, 0 differ -> PASS" with one CTA per 2,048 blocks and with N=3. sm_60
compile of the whole tree (CUDA 12.9, P1004): exit 0.
Not covered: no GP100 run here (the kernels' last word is the P100, and the
speed claim). The contributor's numbers (four GP100 dies, every expert resident,
--pipeline-windows 3: 400 tokens at 115K context 51.0 -> 52.5 tok/s, +2.9%;
prompts below 64K unchanged; replies and state hashes equal on 15 requests) were
not re-measured. The model gate is phaseA-tests/gh027/gates/3820-select-long.sh.
Everything the Chat view shows (header, tabs, message box, settings, toasts,
tool blocks, the sidebar, the history and Load/Unload texts) now goes through the
translation catalog: serve/web/locales/{de,es,ja,ko,pt,ru,zh}.json beside en.json
(written from the English text in the code by tools/web_i18n.py). Plurals follow
the language (Russian has few and many, Japanese and Korean only one form).
serve/test_web_i18n.py fails when a catalog lacks a key, has a stale one or
changes a {placeholder}. A ninth language is one file plus one line in
serve/web/i18n.js. The Monitor, Images and About views are still English (the
language setting says so); they are the next step.
Browser tests: a language pick translates and is kept, /?lang= is for one visit,
an unknown code stays English, all seven catalogs load without an error.
docs/DETAILS.md: a section "The web page" (every switch, what the preview can and
cannot reach, the languages, the tests), the Sampling drawer's new name, and the
credits (KaTeX and the contributors of the PRs ported into the page).
Five overlapping proposals, one set of controls (Chat settings > Chats, all off
by default; off, the page is the page from before: one chat, New chat clears it
with an Undo, Send is off while an answer is written, no new request field).
- Saved chats list (#1787): the chats kept in this browser (IndexedDB, one record
per chat, names of attachments only), a list with day groups, search over
titles and text, rename, delete with Undo, collapse; a private window or full
store keeps the list in memory and says so once. Nothing goes to the server.
The list has the New chat button, so the composer's one is hidden while it is on.
- Edit and rewind messages (#2615 + #3823): under each message, Regenerate (a new
answer to the same question), Edit answer (its text only: thinking and tool
calls go, no request) and Rewind to here. Rewind asks, as #3823 proposes: "Edit
and send again here" hides this message and the ones after it, puts the
question and its attachments in the box and keeps one Undo bar ("2 messages
hidden"); "Branch from here" starts a new saved chat with the messages before
it and leaves this one alone. A branch keeps a title that says where it came
from ("Branch of ...", also at the very first message) and links back. One
control does what #2615 had as Edit on a question and as Remove from here.
- Send while answering (#4004, for the web page): "Wait" (default, as before),
"Send after" (the message waits and goes when the answer is done) or
"Interrupt" (the answer stops where it is, what is written stays and is marked
interrupted, the message goes at once; a running tool finishes first). One
message waits; Take back returns it to the box; Stop keeps it for you to send.
The pure logic is serve/web/history.js, the store serve/web/chatstore.js, the
page's part serve/web/chats-ui.js. Tests: serve/test_web_history.js (Node, with
the IndexedDB stub and cases of #1787), serve/test_web_history.py, and 17 browser
tests (tools/webtest/browser_test.mjs). Screenshots: tools/webtest/shots.mjs.
Also: the Sampling drawer is now Chat settings (its button says so), a wide
window lets a message use the column (#3176 widened only the column).
Ported by hand and merged: #1787 by Scotty Hudson, #2615 by Joyer Huang (uppet),
issues #3823 and #4004 by xvcezrqpbxtpzblrt (their proposals, for the desktop app,
carried to the web page).
Co-authored-by: Scotty Hudson <[email protected]>
Co-authored-by: Joyer Huang <[email protected]>
Co-authored-by: xvcezrqpbxtpzblrt <[email protected]>
The layer-major walk is a copy of run_impl's attention half with the experts moved
out. It does not carry four things run_impl does, and when one of them is asked for
the walk now declines (the prompt is read chunk-major, and the one-time "layer-major
off for this prompt" line names the reason):
- the DFlash2 drafter's prompt taps (tap_layers_): the walk would leave the taps
stale and the drafter would read old rows;
- STRATA_KVA (approximate late layers): the walk would silently read exactly;
- a stop_at_chunk callback (the pipelined batch reader that yields at chunk
boundaries): the walk reads every token, and stopped_at() must stay -1;
- a pending force_chunk (a --prefill-pipe plan): it is left set for run_impl.
The walk also resets stopped_at_ like run_impl does at the start of a run.
With STRATA_PREFILL_LAYER_MAJOR unset none of this is reached.
Not covered: no model run in this commit.
Only NVIDIA CUDA has run the layer-major walk. On a HIP build (wave32 or the
gfx906 compat layer) lm_prepare now declines with a reason, so a prompt takes the
chunk-major path and the one-time "layer-major off for this prompt" line names it.
The code still compiles everywhere the shared prefill.cpp does; nothing else
changes. With STRATA_PREFILL_LAYER_MAJOR unset nothing is reached.
Not covered: HIP and SYCL were not built or run (SYCL has its own prefill.cpp).
run_impl never clears a later stage's hand_in_, so after one short prompt (read
chunk-major) every later prompt was declined with "a layer split's chunk hand-off
is in use". lm_prepare_chain now declines only when a hand-off is really in flight
(a stage's next_run_ is valid) and otherwise clears the stale hand-in pointers.
A prompt longer than the common super-chunk is split into the same number of
super-chunks of near-equal size (32K at a 28K fit: 2 x 16K, not 28K + 4K). That
keeps a stage's residual rows in VRAM and lets a split's stages overlap on every
super-chunk. STRATA_PREFILL_LM_EVEN=0 gives the old largest-first split. It only
acts when STRATA_PREFILL_LAYER_MAJOR is set.
The split was declined whenever the model has PLE layers. The PLE block (layer 1)
reads the token history, which only the first stage has, and a later stage never
reaches it. So the split is declined only when the first stage does not hold both
layer 0 and layer 1.
Problem: layer-major prefill declined every layer split, so a split model kept
re-reading each layer's streamed experts once per 8192-token chunk.
Changed:
- lm_prepare_chain sizes every stage with the same super-chunk (the smallest any
stage fits, up to four rounds until they agree) and gives them back together;
- a stage runs only its layers [LB, LE); a later stage's rows arrive in its pinned
R_host;
- a stage before the last writes its final rows into the next stage's R_host on
the copy stream, then runs the next stage on a thread (std::async) while it reads
its own next super-chunk. There is one hand-off buffer: a stage's final pass waits
until the next stage has finished the previous super-chunk;
- layer_major_mid() asks the last stage, the one that reports the rows;
- run() drains the pipeline after the walk and clears the mid flag on every stage.
Not covered: only two stages were tried by the reporter (RTX 3060 + RTX 5060 Ti);
the peer-GPU tier and the stage helper have no layer-major form and decline.
run_layer_major called mmq::iota on ids_identity, which is null when no layer is on
MMQ; it now skips that. The QSA shapes come from core::qsa_shapes(g) as run_impl
does, instead of being filled in by hand.
lm_prepare required ids_identity and tok_dev even when no layer is on MMQ. Only a
model with an MMQ layer needs the identity table, so the check now asks for it only
then. Without this a model whose layers all take the FP16 path was declined.
Problem: lm_prepare declined every model whose experts are not all on MMQ. All 48
layers of UD-Q4_K_XL take run_impl's FP16 path, so layer-major never ran on them.
Changed: a layer outside the MMQ plan dequantizes each routed expert to FP16 once
per super-chunk and runs two FP16 GEMMs per entry on that entry's FP16 input rows
(a new per-super-chunk buffer mixed_h, the bits run_impl's gather reads), then
adds the group's rows per token with moe_scatter_add as the MMQ path does. The
buffers each path needs are checked separately, and a missing one gives a decline
reason that names it. The summary line counts the groups on the FP16 path.
Not covered: no model run in this commit.
Problem: the layer-major walk (#1670) was written for an older base. It read the
hyper-connection matrices as bf16 only, accepted expert formats its MMQ pass does
not carry, and called the QSA indexer with the older argument lists.
Changed (Norhiro's port of vizdatom's walk, kept as one step):
- the HC projections are dequantized per half when STRATA_HC_Q8 holds them as Q8_0
with no bf16 copy, and released after the GEMMs that read them are enqueued;
- lm_prepare declines (the prompt then runs chunk-major) for a layer that needs the
WMMA tiles, an APR (Hadamard-folded) layer, and an NVFP4 layer;
- STRATA_PREFILL_DIRECT is not used by the walk.
Not covered: no model run in this commit.
A super-chunk of S tokens is read layer by layer: each 512-token sub-chunk runs the attention half,
the FFN half hyper-connection read, the router and the shared expert in order (GDN / QSA / PLE state
advance as in the default path); then the layer experts run once for the whole super-chunk - each
routed expert staged once, its rows of every sub-chunk in one MMQ group, the results added per token.
Expert traffic drops by about S / 512 (a 12.9K prompt: ~509 GB -> ~79 GB on ryzen).
- STRATA_PREFILL_LM_TOKENS: the super-chunk target (default 8192), shrunk to fit free VRAM minus
STRATA_PREFILL_LM_MARGIN_MIB (default 96).
- STRATA_PREFILL_LM_R=vram|host|auto: the residual rows of the super-chunk in VRAM, or in pinned RAM
with the copies overlapped on the copy stream (auto: VRAM only when the whole target fits there).
- STRATA_LM_PREFETCH=0: no prefetch (default on: every streamed expert of a layer goes to the stager
when its attention halves start; the ones the routing skips are skipped if not started).
- Not for a layer split, a peer GPU, image prompts, the fused layout or a prompt under two chunks.
- on_chunk still reports every sub-chunk in order; layer_major_mid() tells serve not to checkpoint
before a super-chunk end (the state is already at that end).
Port to 0.1.43.1: the copied run_impl body follows the new base. Every blob the stager or the copy stream reads later
comes from blob_stable() (blob() may hand out a recycled STRATA_IO_PREFETCH buffer); the token ids go up through tok_up,
the QSA step records and the PLE rows' pageable fallback through up_ring (#1712, #1996); the PLE gather threads pin to
--aux-cpus; bf16x2 takes m.f16_io; the K8V4 append is upstream's, which writes the host copy and the staging pool by
halves. The indexer passes the INT8 pooled keys as the original does, in the QSA port's order (pooled_i8 and
pooled_scale after block_pos, the int8 qsa_block_scores(_tc) overloads); STRATA_IDX_INT8 stays opt-in and unsupported on
the prompt path, as in the original. stage_one calls the stager's pass() before wait(): with batched claims a routed job
after about a ring's depth of skipped prefetch jobs waited forever. lm_prepare reads the fused layout from m.fused_bufs
and declines while STRATA_DBG_FEAT, STRATA_DBG_ROUTE_OVERRIDE or STRATA_PREFILL_DUMP_R_ALL is set. serve's near-tail
checkpoint (--prompt-cache-tail) is not taken mid-super-chunk, so under layer-major it is taken only when the last
super-chunk is a single sub-chunk.
From vizdatom's #1670, original commit .
moe_shared_init writes the shared expert part of moe_combine before the routed experts are known;
moe_scatter_add adds one expert group rows into a per-token accumulator, per token in a fixed order
(deterministic, no atomics).
From vizdatom's #1670, original commit f0173382d0646907917edf99e59617c1412d898e.
Chunks below stream_all_min() stage each layer's unpinned experts on the stager's few threads, and with a mapped
file tier each of those is a chain of 128 KiB page faults, a shallow queue for the drive. With STRATA_PREFILL_WARM=1
(and a source that warms: the GGUF in place, or experts.bin with STRATA_BIN_WARM=1) a background thread hands the
source the layer's whole list at once before the stager starts, and the next layer's list from the previous chunk
(less what is on the GPU or pinned) while this layer computes. It only warms pages: the stager copies the same
bytes. At most four lists queued; a thread that falls behind drops the oldest. Off by default: the A/B arm.
Port to 0.1.43.1: the warmer thread pins itself with aux_cpus::pin_current_thread() first, like the other helper threads since --aux-cpus (it would otherwise inherit the host thread's CPU).
From vizdatom's #1670, original commit cabe33c4f585d4c9dd0af1fc9f014d00e2920042.
issued is a watermark, so a job nobody waits for could be passed while its thread was still writing
the pinned buffer the next job on that buffer was filling - a corrupt expert blob. A job now also
waits until the job kRing before it has marked itself ready. skip(j) lets a job the routing turned
out not to need be marked ready without reading it (the layer-major prefetch). The default walk
consumes every job in order, so neither changes it.
Port to 0.1.43.1: the stager now claims batched runs and sleeps in issued.wait, so the ready[i - kRing] wait
sits in the run's per-job wait loop after the issued wait, and a skipped job is left out of the run's gather
(no memcpy, prefill_copy or copy_blobs entry); its ready flag and `active` stay with the run's single store and
decrement. New pass(j): skipped jobs are never issued, so the layer-major walk moves the watermark to the
routed job it is about to wait for (store plus notify_all, which wakes sleeping threads); without it a routed
job after about a ring's depth of skipped ones waited forever. The default walks never call it.
From vizdatom's #1670, original commit 149b899548fbc98977219e6fd4ee7866384e61f6.
One button in the header of every tab: Load while the model is unloaded, Unload
otherwise. It calls POST /load and POST /unload (the server already had both,
for its own page only). Unload asks first and never interrupts a turn: the
button is off while an answer is being written, an image is made or a request is
queued. The pill and the Monitor say Unloaded / Loading / Unloading, a message or
an image on an unloaded model shows Loading, and a poll that started before the
POST cannot flip the button back (the POST's state is held until a later poll
agrees). On a phone the pill's text is the first thing the header row gives up
(it measures the row, not a fixed width), so Load/Unload and the theme button
stay on the screen from 320 px.
Browser tests (tools/webtest/browser_test.mjs): unload with its question, load
back, declining, not while answering, a message that loads, the header at 320 to
640 px.
Ported by hand from PR #3550 by Sakurai Momoka (sakuumomo); the texts now go
through the page's translation catalog.
Co-authored-by: Sakurai Momoka <[email protected]>
A complete ```svg or ```html block in an answer gets a Preview button. The
block's own code is drawn in an iframe (srcdoc) that is cut off from the page:
sandbox="" (an opaque origin: no access to the page's DOM, storage, cookies or
server, no scripts, forms, popups, downloads or navigation of the page), a CSP
in the frame's document (default-src 'none': no network, fonts, frames or media;
styles inline, pictures as data: only; no <base>, no form target), no referrer
and no permissions. A well-formed svg is shown as an <img> with a data: URL: an
image runs no script, loads nothing and animates in every browser. The frame
takes the page's light or dark colours and is built again when the theme
changes. A page (html) can also be run WITH scripts, one block at a time, after
a warning that is shown every time: sandbox="allow-scripts" only (still an
opaque origin), the CSP allows the block's inline script and no address; closing
the preview ends it.
What the sandbox flags alone do not stop is a link or <meta refresh> inside the
model's html loading a web page into the frame. The browser test shows it does
(tools/webtest/browser_test.mjs, with and without the header). So the page is
served with Content-Security-Policy: frame-src 'none' (nothing else: a srcdoc
frame inherits the page's policy, and the script preview needs its inline
script), and a <meta http-equiv=refresh> in a previewed page is dropped.
tools/webtest holds the mock server (the page's own files, scripted answers,
no model) and the browser tests; serve/test_web_preview.js checks the documents,
flags and frames, serve/test_web_preview.py the header and that the page never
listens for a frame's postMessage.
Ported by hand from PR #1628 by thoives.
Co-authored-by: Thomas Ivesdal-Tronstad <[email protected]>
A PC whose CPU lacks the AVX-512 set that setup checks for gets the native Q2_0 pack. On a CUDA card of sm_80 or newer its
Q2_0 / Q2_0 prompt experts run through MMQ, although the fused int8 kernel for that format (#2145) is in the binary and set
up only on a Turing card (b6443f3ae).
STRATA_PF_FUSED_Q2N=1 sets that kernel up on any sm_80+ card and takes it for the Q2_0 / Q2_0 layers, under its own variable
so that STRATA_PF_FUSED=1 keeps its bits. STRATA_PF_FUSED=0 and STRATA_PF_FUSED_NATIVE=0 still turn it off. A failed setup
leaves the occupancy as it was and the layers on MMQ.
- moe_fused_iq.cu: DevInfo.q2n; dev_info() sets up Q2_0 gate/up when the switch is set; gu_covered() and native_supported()
take the pair only when q2n is set (a Q2_0 gate with another down type is refused, as on Turing).
- tests/cuda/prefill_fused_iq_test.cpp: --unset-pf-fused leaves STRATA_PF_FUSED unset (the engine's default) and skips below
sm_80; CMakeLists.txt: the ctest entry prefill_fused_iq_test_q2n runs the Q2_0 / Q2_0 pair with the switch alone (exit 77 =
skipped without an sm_80+ card; CUDA branch only).
- docs/DETAILS.md: one section in the author's measured numbers (RTX 5080) and the A4000 caveat below.
Unset (the default), nothing changes: dev_info() reads the variable once and sets nothing up, gu_covered() and
native_supported() return what they returned. Read line by line against base: the Turing path (t75) and the HIP path are
untouched. Set, the prompt arithmetic changes (first-token KL against FP16 0.0110, MMQ 0.0112, the author's figure) and the
fused path's smaller buffers let --prefill auto pick a larger chunk, which is most of the author's 16K gain; the generated
text can change.
Not verified: an RTX A4000 (sm_86). The kernel was taken on every CUDA card in #2145 and then kept to Turing after an A4000
no longer matched the old output and read 4K prompts 4.5% slower; the PR does not repeat that run. Merge note: #3277 and #3779
change dev_info() and native_supported() in the same file.
Tests: prefill_fused_iq_test_q2n and prefill_fused_iq_test pass on an RTX A4000 (sm_86, synthetic data); on the Q2_0 / Q2_0 layer the
fused kernel is 1.76x / 1.30x / 1.24x the MMQ path at 2048 / 4864 / 8192 tokens with the same relative RMS against FP32. Builds on CUDA
sm_86 (Linux) and sm_120 (MSVC). Not covered: no model run; the engine-level A4000 check is the gate.
Chat settings > Render math (off by default). When it is on, \( \), \[ \], $x$
and $$ $$ in an answer are drawn as formulas by KaTeX 0.19.0, which is vendored
in serve/web/vendor/katex (script, style, 20 woff2 fonts, MIT licence and a
PROVENANCE.json with the npm integrity and the sha256 of every file, so the page
needs no network). Nothing of KaTeX loads until an answer holds a formula.
The shared core (math.js, from #2972) keeps the text of every message exactly as
the model wrote it: formulas are swapped for placeholders before the Markdown
runs and drawn over them afterwards, so history, copying, the Markdown export
and the requests never see the drawing. Code, link targets, prices ($5 and $10)
and a formula still being streamed stay text. A formula that fails, is over
16 KiB or times out stays as its source text.
Safety: KaTeX runs with trust off (no \href, \url, \includegraphics, \htmlClass:
they are drawn as their own name), maxExpand 1000, maxSize 20 and a fresh macro
table per formula; it builds DOM nodes itself, so no HTML string from a model
reaches the page. A formula already drawn is placed again at once while an
answer streams, so it does not flicker back to source.
serve/test_web_math.js runs the real vendored KaTeX against hostile formulas;
serve/test_web_math.py checks the vendored files against PROVENANCE.json and
that nothing outside this server is addressed. serve/web/vendor is -text in
.gitattributes so the hashes hold on a Windows checkout.
Ported by hand from PRs #2972 (core) and #2974 (KaTeX) by CC-David-CC.
Co-authored-by: David <[email protected]>
The served page had no way to translate its text: every string was English in
the HTML and the script. Now the English text stays in the code next to a key
(tr("key", "text", vars), data-i18n="key" in the HTML), a language is a catalog
serve/web/locales/<lang>.json, and a catalog that is missing or has no entry for
a key leaves the English text. The language is the user's choice (About >
Settings), kept in this browser; nothing is chosen from the browser's language,
because a page translated in part would mix two languages. /?lang=de shows one
for a visit.
The server hands out /web/locales/<lang>.json and /web/vendor/katex/** from
serve/webfiles.py: whitelisted names and types, no backslash, colon, percent or
dot-dot, and the resolved path must stay inside the folder (also against a link
out of it). tools/web_i18n.py reads the English text out of the code, writes
en.json and checks every catalog for the same keys and {placeholders}; the
check runs once the catalogs exist.
Only the part of PR #3311 by gopinath87607 that is about this engine: a periodic mark is saved at a chunk boundary of the batched prefill and the read carries on
(src/program/generate.cpp, periodic_checkpoint), so the interval is not a piece-size dial; what cuts a read into runs is the re-read, the root, a message
boundary and the start of the last turn; --prompt-cache-every 0 skips the periodic and the near-tail marks. The wall-time numbers and the comparison with the Maya
checkout belong to another engine and are left out. New section in docs/PROMPT_CACHE_TAIL.md, linked from the option's row in docs/DETAILS.md.
--show-key only knew a running launch or a preset. A model set up by setup.py (START-HERE.bat, ./setup.sh) keeps its generated key in strata-<model>.json, so the command
said there was no key and people read the JSON by hand. With no launch and no --preset, it now looks at the run configs of the installed models: one model with a key
prints that key (stdout is the key alone; a hint line with the model and the file goes to stderr), several models ask for --model NAME (any case, the size or the full
id), a name that matches nothing says so. --preset never falls back to a setup config, and no key is printed while the choice is open. Tests:
tests/terminal/test_launch_show_key_setup.py.
A volume or bind mount from the 0.1.x image keeps its models in /data/models. The entry point moves a folder that is part of the volume, but a folder the person
mounted there is left alone, and a failed move leaves the files too; with no MODEL_PATH the start then did not see them and downloaded the model again
(tens of GB) without a word. Now, when .gguf files are still in /data/models and MODEL_PATH is not set, the container stops with code 2 and names the folder,
says what to do (MODEL_PATH to the folder with the shards, or the files into /data/Strata-data/models) and that nothing was downloaded.
ALLOW_DOWNLOAD=1 goes on anyway with the same text as a warning. docs/DOCKER.md lists the setting. The other points of the issue (the key, the removed
variables, the status loop) are the owner's design questions and are not changed here. Tested by running the guard function with sh (tests/terminal/test_docker_files.py);
the whole container was not run (no docker on this PC).
Prefill on one card, decode on another, the session handed over in RAM through the engine's
own session file (SAVE/RESTORE). The engine's new request form GEN 0 prefill=1 reads a prompt
as a request does, up to where the first verify window starts, and stops with the session at
n - 1 tokens, which is the place a continuing request resumes from. Opt-in; without
split_serve nothing changes.
Idea and manual setup by @MarcoTool in #4000.
Problem: the strata-dsv41 packages were built for sm_86 and sm_120 (+ PTX 120). An RTX 4090 user was stopped at setup on 0.2.4.1; since
b750d757f an sm_86 package is accepted on an sm_89 card (cubin same-major rule), but the engine had no sm_89 image of its own, and the docs said
nothing about RTX 40 or about building the engine for a card the packages do not cover. The reporter built strata-dsv41 for 86-real;89-real
themselves and ran the 2.00 set at 40.2 tok/s on an RTX 4090 48 GB (theirs, --dspark on).
What changed:
- packaging/build_dsv41_cuda.sh and build_dsv41_windows.bat: the default architectures are now 86-real;89-real;120, so the DeepSeek CUDA 13
packages list sm_86, sm_89, sm_120, compute_120 (Windows and Linux alike). The CUDA 12.9 line in the script header says ARCH="86-real;89".
- CMakeLists.txt: a build of strata-dsv41 with no -DCMAKE_CUDA_ARCHITECTURES defaults to 86-real;89-real;120 (the Qwen engine's default
stays 120, and the warning now names the architectures it picked).
- strata_control/engine_component.py arch_native(): true when a package holds machine code for the card itself. resolve() and
feed.select_engine() use it as the last tie-breaker, so with otherwise equal packages the one with the sm_89 image is taken on an RTX 40;
an sm_86-only package still fits (no refusal).
- setup.py: the "engine not available" stop now points at the build recipe (docs/DEEPSEEK41.md); the developer line names the
architecture to pass. The card list already said RTX 30, 40 and 50.
- docs/DEEPSEEK41.md, docs/MODELS.md: RTX 40 coverage, the reporter's measurement quoted as theirs, no claim of our own (we have no Ada card),
and a section "Building the engine for your card".
Tests: tests/terminal/test_engine_manifest.py (arch_native; the build scripts and the CMake default name sm_89 and both OSes agree; the
package list they give is sm_86 sm_89 sm_120 compute_120; an sm_89 card is accepted and prefers the package with sm_89). tools/test_setup_dsv41.py
unchanged and passing.
Not covered: no run on an Ada card (gate script in phaseA-tests/gh027/gates/3935-ada-dsv41.sh).
One bullet from PR #3930 by NorwinYu: on two identical RTX 2080 Ti over a dual-socket Xeon E5-2696 v4 a plain split moved decode from 35.2 to 51.0 tok/s and an
11.6K prompt from 672 to 908 tok/s, because the CPU pool per window fell from 9.52 to 0.79 ms (STRATA_DECODE_TIMING=1). The page's own 5080 + 3090
figures, where the second card is on par, came from a fast CPU; the bullet says so and that it is one measurement. No code changes: setup already splits
across two cards. Shipped without a hardware run of ours; the reply asks owners of two older cards on slow CPUs to compare and report.
The chat, About and Monitor columns grow with the window (the chat and the
composer to 1100 px, About to 1100 px, Monitor to 1920 px), the metric cards
wrap with auto-fit, and under 640 / 560 / 480 px the controls get bigger
touch targets, the composer bar wraps, and long words, code and tables stay
inside the viewport.
Ported by hand from PR #3176 by Matthias Nott (mnott); the CSS is his.
Two paragraphs from PR #3895 by Vop3p, put under the table row they explain: the fork pays only when it has a CPU-pool wait to hide (4,859 slots on one
RX 6900 XT, GPU-reach wait 20.7 of 39.1 ms; with every expert resident it is 43.4 against 62.9 tok/s on an R9700), and the Windows RX 6900 XT fault with
the fork plus --kv-resident, its bisection and its limits (one PC, kernel not isolated). The text now says what the code does about it and asks RX 6000 owners on Windows to test.
On one Windows RX 6900 XT STRATA_SH_STREAM=1 together with --kv-resident failed the engine on its first request (verify: unspecified launch failure).
setup --calibrate (tools/calibrate.py) no longer measures the fork on Windows when KV streaming is on (the fork is off by default on AMD, so
nothing changes unless the person sets it), strata tune's never-pick list holds the combination (strata_control/tune_knobs.py), and a config that
sets STRATA_SH_STREAM=1 with --kv-resident on Windows HIP gets one warning line when the engine starts (serve/server.py sh_stream_warning). Nothing is refused or
removed: the variable still reaches the engine. Shipped without a hardware run, from one report; the reply to #3895 asks RX 6000 owners on Windows to test
and report. Tests: tests/terminal/test_sh_stream_windows.py.
The 0.2.5 rewrite dropped about 25 entries: the Mac quarantine steps, strata doctor, sending logs, the first-start freeze, the graphics memory the app keeps free,
Smart App Control, UPDATE.bat and git pull (#1276), the build tools and the gcc/glibc compile problems, the engine checksum, the engine stopped unexpectedly,
slower than the tables, the empty-reply and Turing switches, --expert-cache and --resident-budget-gib, the page-locked OOM, the NVIDIA 0x141 reset, CUDA 13.2 on
RTX 50, the AMD section (KFD, the build list, 0xC0000005, the driver reset, pinned allocations, verify timeouts) and the body of the verify-timeout entry, which
the rewrite had left empty. Each is back with the 0.2.7 facts: the app's reserve defaults (native-resources.mjs), the Drafting setting, --vision on|off|gpu|cpu
in the Terminal and --vision no in a source checkout, Intel Arc's CPU reader, and the Terminal route next to the source-checkout one.
Every pointer resolves: setup.py's driver-reset pointer, the server's Smart App Control message, DETAILS.md's STRATA_KEEP_EMPTY_TURNS / STRATA_TOPK_STREAM line,
OLDER_GPUS.md and MAC.md. tests/terminal/test_troubleshooting_page.py checks the entries, those pointers and the page's own links and anchors.
Problem: below sm_70 the prompt reads each QSA layer's attention through the
decode kernels (qsa_prompt_attn_batch returns false without tensor cores): 32
queries a call, every chunk's partial results (817 KB a query) written to
scratch and merged by a second kernel.
Change: src/kernels/cuda/qsa_pattn.cu keeps the contributor's default variant
only, ps_score_x_kernel (q in registers, the chunk's K staged with 16-byte
loads, two cells at a time through a select-free reduce-scatter) and
ps_value_d4_kernel (four dims per thread, merged into the output as the merge
kernel does). Every float operation of attn_chunk_kernel_pre75 and
attn_merge_kernel is kept in its order, so the output is the decode kernels'
bit for bit. The PR's other variants (per-chunk, fused persistent, bench split
codes), pattn_bench and the variant argument are not ported; pattn_bench needs
a STRATA_QSA_DUMP file from a model run.
Gating: compiled only in the CUDA build with STRATA_EXPERIMENTAL_SM60 (HIP and
Metal link a stub that answers false; prefill.cpp keeps the call out of the
Metal build). At run time only with STRATA_PATTN_SM60=1, on compute capability
6.x, int8 K/V, head_dim 256, 12 query heads per KV head, at most 33 chunks.
Unset, the only cost on the default path is one cached int test per QSA layer,
and only when qsa_prompt_attn_batch has already declined. Any refusal or a
failed launch returns false and the usual 32-query loop runs (one warning, no
exit; the PR called exit(1) on a launch failure).
Tests: src/kernels/qsa_pattn_parity.cpp (new, registered in CMakeLists.txt in
the experimental sm_60 test block, skips with 77 when the kernels are absent)
compares the kernels with the Daily's chunk kernel plus attn_merge_kernel on
random int8 K/V: a shuffled page table with ~3% unresident pages and a run of
64 unresident pages (whole chunks masked, some queries empty), three selection
shapes, caps 64 / 130 / 2051 / 2112, widths 0..cap, n_q up to 40 with a
scratch that holds only 2 decode-layout queries (several batches), junk in the
scratch and output. On a Pascal card it also requires the checked entry to take
the shapes and qsa_decode_attn_batch to match. The kernel entry that skips the
compute-capability test (qsa_pattn_batch_any_arch) exists for this test only.
Results: z440 (RTX A4000, sm_86, CUDA 13.0, build with
-DSTRATA_EXPERIMENTAL_SM60=ON, gpu lock): "qsa_pattn_parity: 468 cases, 0
differ, 29724792 nonzero output values in all -> PASS". sm_60 compile of the
whole tree (CUDA 12.9, P1004): exit 0.
Not covered: no P100 run here (the Pascal-only checks of the test and the
speed claims wait for the gate, phaseA-tests/gh027/gates/3943-pattn.sh). The
contributor's numbers (43.3 -> 30.8 ms a 1,536-query chunk per QSA layer at
26K context, 2-3% less time on 8K-150K token prompts on four dies) were not
re-measured; compute capability 6.1 is covered by the same switch and was not
measured by anyone.
The Speed group gets a row for Qwen on a discrete CUDA card of 13 GiB or less: "Prefers the parts of the model already on the graphics card, which makes answers
much faster on 12 GB cards. Turn off if answers look worse than before." On (recommended) by default; Off writes STRATA_ROUTE_PRIOR=0, On removes it. The
DeepSeek speed-ups row, when Off, writes DS4_DEFAULTS=026 and the six DS4_*=0; On removes all of them, and a file that holds only the six (an earlier test
build) shows as Off. The Qwen "0.2.6 behaviour" switch is Terminal-only and is not in the app. New strings fx82 and fx83 in the eight fix catalogs.
- DeepSeek speed-ups Off now writes DS4_DEFAULTS=026 (the engine's master switch back to the 0.2.6 defaults) plus the six DS4_*=0 for older engines; On removes
all seven. A file with only the six (an earlier test build) still reads as Off and On removes them (`legacy` entries).
- New switch "Faster answers on 12 GB NVIDIA cards" (the residency-biased routing default): on by default, Off writes STRATA_ROUTE_PRIOR=0. Applies to Qwen
Flash-Next (no arch) on a CUDA build with a discrete NVIDIA card of 13 GiB or less (not a chip that shares system memory), the engine's own rule.
- New Terminal-only switch "Qwen: 0.2.6 behaviour" (STRATA_DEFAULTS=026, off by default), a row of the Advanced table for every Qwen model, for
troubleshooting; the app never shows it. DeepSeek has DS4_DEFAULTS instead.
tests/terminal/test_speed_switches.py covers the new defaults, applicability, the seven DeepSeek entries and the legacy six, the Advanced rows and the
comparison with the app's table (the Terminal-only switch is left out of it).
/v1/chat/completions on the CUDA and HIP engine accepted logprobs and
returned an empty choices[0].logprobs.content. Now "logprobs": true with
"top_logprobs" 0 to 20 (and the legacy "logprobs": N) returns one entry
per token with its alternatives, streamed or not, and "allowed_token_ids"
returns a distribution renormalised over exactly the ids the caller named.
Engine (src/program/generate.cpp): the GEN line takes logprobs=K and
allowed=a,b,c. After each "T id" the engine writes one "LP <logprob>
[id:logprob ...]" line from the verify window's head logits before
sampling, penalties and temperature. The SYCL engine already wrote LP
lines for logprobs=K; it now reads allowed= as well, with the same line
format. A request without logprobs reaches none of this: both call sites
are behind "req_logprobs >= 0", a local int of the request, so the default
path prints what it printed before and does no extra work per token.
Changes from the PR: the call sites are guarded instead of calling the
function for every token, the function is static, the token index is
checked against the row, and the Chinese comment is in English.
Server (serve/server.py): LP lines are read before the token is yielded
and belong to the request that asked (tagged with its cancel Event), the
entries spell the reply, json_schema replies return no logprobs rather
than a mismatching set, an allowed id past the vocabulary is a 400, and
logprobs on a --parallel server is answered for max_tokens 1 only (a
slot's BT lines carry no LP line). serve/structured.py: _extract_json is
now extract_json (the logprobs alignment uses it); the two uses added by
the #2893 retry follow the rename.
Left out: the PR's bench and result files (none in the diff). #3332 (/v1/completions) and #4056
(logprobs on it) build on this change; this one needs nothing from them.
Tests: serve.test_logprobs and serve.test_logprobs_parallel (30 tests with
the PR's), serve.test_structured, serve.test_responses: 94 tests OK.
Not covered: no model ran here. The PR author reports 28 of 28 on an RTX
5080 with MTP and on a GTX 1080 Ti; the layer-split pipelined decode path
(the first call site) was not run by anyone, and the LP values were not
compared with a CPU softmax of dumped logits.
Co-authored-by: Leonardo Murillo <[email protected]>
Review of the first port: it registered kIlRows_89 / kIl1Rows_89 for every
compute capability 8.9 card by default. Bit-identity of an interleaved-kernel
choice to native_mmvq is a property of the compiled code, and the release
engine builds its own sm_89 cubin that no card of ours can run; the issue also
saw no end-to-end decode change on a 4090 or a 4080, and the 4070 and 4080
runs agree in 53 of the 75 cells of the 2-4 column table (33 of them non-zero). So the default
change has no measured benefit and an unproven bitwise claim on the card it
changes.
Change: sm_89 answers the shared table, with no one-column row, exactly as
before, unless STRATA_MMVQ_IL_89=1 is set (read once, like the neighbouring
switches). With it set the cells from the two community runs apply. Nothing
else in the lookup changes; sm_86, sm_120 and the others are untouched.
Why it is bitwise: native_mmvq_il_kernel keeps a separate accumulator set per
row (acc[v][i][c]) and sums each row over the same chunk order and the same
warp_sum, so the rows-a-warp count R only changes how many rows a warp walks,
not the order of any row's float additions; the plain-versus-interleaved
equality is what mmvq_il_parity measures (memcmp against native_mmvq for rows
1/2/4, T 1-8, odd row counts). That test passes on sm_86; it has not run on an
Ada card.
Tests: mmvq_il_parity now checks the unset behaviour (sm_89 equals the shared
table, Q3_K and the one-column row answer 0) or, with STRATA_MMVQ_IL_89=1, the
4070/4080 agreement. A second ctest, mmvq_il_parity_89table, runs the table
check with the switch set and no kernel case. On z440 (2x RTX A4000, sm_86,
CUDA 13.0, gpu lock): mmvq_il_parity and mmvq_il_parity_89table both pass, and
the binary prints "mmvq_il_parity: OK (every case, T 1-8, rows 1/2/4 and the
table, bitwise)" with the switch unset and set.
Model gate owed on an sm_89 card: phaseA-tests/gh027/gates/3453-sm89-il.sh
(mmvq_il_parity on the card, then identity of the answers with the switch on,
then a speed A/B that decides whether the default flips).
Co-authored-by: rodi4eg <[email protected]>
With --batch-groups above 1 on a layer split the pipelined batch loop never starts a round of the adaptive VRAM tier (the
serial batch windows start one every --adapt-every windows). While the slots are busy the tier only moves when they go back
to the solo path, so a fresh engine keeps its start-up profile for minutes.
STRATA_PIPE_ADAPT_EVERY=N (opt-in; unset, 0 or not a number is off): after N group windows pump starts no new group on the
first stage, lets the groups in flight finish, then runs one round as batch_step does between serial windows, adapt() and
apply_pending(true), so nothing in flight reads a cache slot while experts swap and the new experts have landed before the
next launch. Not while a prompt is read beside the slots (in_prompt_read, its loan of cache slots), not with --adapt-async,
not while the last round's copies are in flight, not with the tier off. The rule is two pure functions in
include/strata/program/batch_groups.hpp (pipe_adapt_every, pipe_adapt_due) with 12 new cases in batch_groups_test. With the
switch set the engine says so once at start and the "strata batch (pipelined ...)" line adds the rounds, the experts
swapped in and the time spent. docs/BATCHING.md documents the switch with the author's table (2x RTX 5090, six streams,
309 to 410 rows/s over eight bursts after a fresh start, two engines per setting).
Reviewed: unset costs one counter increment per group window and one false check where a group starts, and prints no new
line. A due round cannot wait forever: no new group starts, the groups in flight finish, then the round runs. A prompt read
sets in_prompt_read before the reader thread starts (including STRATA_PIPED_CONCURRENT) and clears it after the reader
joined; the drain after the read runs the round if it is due.
Tests: batch_groups_test (the new cases). Not covered and still the author's to show (the PR is a draft): a soak with
prompts read beside the slots, a Linux run, another GPU pair, more than two stages, and exactness settings with the switch
set. The model gate is written down.
STRATA_ENGINE_BACKEND=hip-rdna2 worked for strata update (updater.detect_hardware), but strata update --backend hip-rdna2 offered no package and
strata launch stopped with "STRATA_ENGINE_BACKEND='hip-rdna2' is not one of cpu, cuda, hip, metal, sycl". engine_component.split_backend reads
hip-<variant> as the hip family and its package; detect_backend (the start path), detect_hardware and update --backend all use it. Any other name is
checked as before. Tests in tests/terminal/test_gfx_selection.py.
STRATA_RAM_ADAPT (opt-in, --serve, one GPU, needs --adapt-every) lets the resident RAM copy follow the conversation:
experts the windows keep routing to take the RAM places of unused experts of their layer. Each move re-read the incoming
expert from the file and a layer moved one pair per window, although the incoming expert was usually read for the window
before and its blob still sits in that read's stage buffer.
What changed:
- FileExpertSource::replace_resident_staged(layer, in, out): the same move as replace_resident, with the bytes copied out
of the finished stage buffer of `in` (no file read). The copy runs under stage_mu_, so claim_stage cannot give the buffer
to another blob meanwhile; it needs a non-busy buffer whose key is `in`, and returns false (nothing changed) otherwise,
and the pair then goes the old way. The Foresight hold/unhold around the copy is the one install_resident uses.
- generate.cpp: a layer pairs up to 4 of its most routed outside experts with up to 4 of its least routed ones in the copy
(outgoing count below 0.5, incoming 2 or more), largest difference first. A pair whose incoming expert is staged moves by
copy; `n` now caps only the pairs that need a file read. The log line counts the moves and how many came out of a stage
buffer (ram_staged()).
- docs/DETAILS.md: the RAM_ADAPT bullet in the low-RAM list in plain words and one row of the measured table (the author's
numbers: RTX 3060, IQ2_XS and IQ3_XXS, 120 ShareGPT requests, 3 interleaved runs each).
Unset (the default) nothing runs: the block is guarded by ram_adapt_n > 0 as before. The weights moved are byte copies of the
same blob; which experts sit in RAM changes, as it already does with RAM_ADAPT, so greedy token ids may differ from run to
run once it is on. Changed from the PR: the DETAILS.md hunk was rewritten against the current, shorter bullet.
Tests: file_expert_source_test (new staged cases; the positive case needs unbuffered load and is skipped without it).
Not covered: no model run. Model gate written down: RTX 3060 ShareGPT replay =8 against =1.
The Build section said nothing about it. gemm.cu (create_hipblaslt_state, own_tuning_dir) takes STRATA_HIPBLASLT_TUNING, else the file
<engine folder>/tuning/<arch>-hipblaslt-<version>.txt, else plain hipBLAS. The report measured 826 against 1,043 tok/s at 8K prompts on a
gfx1151 (one run per arm, stated as such) and the log line that tells whether a table was taken. The extra log line the issue suggests for the
no-table case is a small engine change and is not part of this commit.
The first #3218 commit made setup.py admit a Maxwell (sm_52) card only
when a tester set STRATA_EXPERIMENTAL_SM52=1, and then routed it to the
CUDA 12 engine. That engine holds no sm_52 code, and users never set
environment variables to get a fix.
sm60_card() is back to Pascal and Volta (6.0 to 7.0). A PC whose only
NVIDIA cards are Maxwell now gets a warning (maxwell_note) that says no
ready-made engine holds code for it, gives the source-build recipe
(-DSTRATA_EXPERIMENTAL_SM60=ON -DCMAKE_CUDA_ARCHITECTURES=52, CUDA 12.x)
and the one reporter's result, and points to docs/OLDER_GPUS.md. The card
table line says Maxwell runs only from a source build. A Maxwell card
beside a newer card prints nothing new. Naming the card (--gpu, --cuda
12) or any env var does not admit it. The CMake floor from the first
commit is unchanged (opt-in build only).
Tests: tools.test_setup_older_gpus 23 tests OK (a Maxwell-only PC through
setup.main() with and without the old switches installs nothing and
prints the recipe; a Maxwell card beside an RTX prints no note).
Not covered: no engine was built for sm_52 by this commit; the P1004
compile is recorded in the notes.
The report written by BENCH.bat and bench.sh had fields missing or wrong: nvidia-smi's 'CUDA UMD Version:' (current drivers) was not read, the value
was the newest CUDA the driver supports but was labelled like the engine's toolkit, storage always said 'not measured', and Windows 11 showed as
Windows 10. Now the report says 'CUDA (driver supports up to): X; engine built for CUDA Y' (BUILD.json beside the engine), names the storage device
(Get-PhysicalDisk, system_profiler, lsblk -e 7 to skip snap loop devices), adds a 'GPU detail (after the run, near idle)' line with power limit and draw,
PCIe generation and width, pstate and temperature, and recognises Windows 11. The standard report (strata bench run, strata_cli/bench_facts.py) already
read the UMD version, storage and power limit; this brings the legacy tool to the same. Ported from PR #3250 by JoShMiQueL with its tests.
Two 0.1.44.1 changes no page told. TROUBLESHOOTING.md gets the start error 'the engine process ended: ...' (serve/server.py start_exit_note: the signal or
Windows exit code, and for an illegal instruction the AVX, AVX2, FMA and F16C flags the CPU lacks). SOURCE_INSTALL.md says a setup run again
pre-selects the installed size, marked '(installed)', and --yes takes it (setup.py size_tag, installed_size). Written by hand from PR #3243 by
Buddychek, whose hunks no longer fit the rewritten pages; each sentence checked against the two functions.
Review fix on the #4005 port. STRATA_PREFILL_COPY_WAIT_DEDUPE=1 skipped the copy stream's wait on a slot's release
event when the previous entry had waited on the same event. Two cases break that:
- At idx == ring the previous entry (ring - 1) is still in the first round, so its wait was on the event of an earlier
walk. The same slot id can be this walk's first flush, which re-records the event after that wait; skipping the wait for
slot 0 then lets the next copy overwrite a slot that the flush's gather has not finished reading.
- With a ring of one MMQ group (16, the smallest ring) a flush of the whole ring re-records the event right after the
previous slot's wait.
The rule is now copy_wait_dedupe_ok() in include/strata/prefill/copy_events.hpp: idx > ring, ring > group, same event.
copy_events_test models two to three walks with random group sizes and random issuer/consumer interleaving and checks that
a refill always waits on a record at or after the flush that released its slot. The same model with the PR's original rule
(idx >= ring) finds the stale-event case, so the test can fail. The default path is unchanged (dedupe off).
Tests: copy_events_test passes (z440, g++ -std=c++20). Not covered: a GPU run.
Review fix on the #2526 port. At an admission from a slot the port copies every checkpoint of the slot that is not
longer than the copied-back length into the main session's chain. Such a checkpoint is valid there only when its tokens
are a prefix of what was copied back (its K/V cells are then the ones the main session holds) and it has no pictures
(the main session's pictures are cleared at a give-back). The PR did not check either; a checkpoint left over from a
branch the slot no longer holds would sit in the main chain with cells that are not its own.
Both trees (CUDA/HIP and SYCL) now test the prefix and the empty picture list per checkpoint. The checkpoint the
request continues from is copied as before. Nothing changes when every checkpoint is a prefix, which is the normal case.
Tests: compiles on CUDA and SYCL (see the build lines of the branch). Not covered: no model run; the gate is written down.
docs/INTEL.md and docs/INTEL_ARC.md described an SSD expert tier that strata-sycl.sh's default (STRATA_VERIFY_NO_HOST=1) does not have: every
expert must be in VRAM or in the pinned RAM mirror, and the engine refuses to start otherwise (sycl/src/program/generate.cpp, 'REFUSED: ... are
neither in VRAM nor mirrored'). A native (IQ) pack refuses --spec 0 and 1 (sycl/src/program/generate.cpp and src/program/generate.cpp). Ported by hand
from PR #2153 by nekomario28 onto the rewritten pages, each sentence checked against those two messages. The hunk for docs/TROUBLESHOOTING.md is left
out: the entry it changed is no longer on that page.
Adds a minimal handoff configuration (--prompt-cache 1, --prompt-cache-every 0, --prompt-cache-root 0, --conversation-cache-mib 0), what each
option does, that it costs RAM, that it does not make cold prompts prefill in parallel, and how to look for the handoff in the engine log, /metrics
and /v1/status. Ported from PR #3334 by oscarlius (continuing #3035). The four options exist in src/program/generate.cpp and the o.prompt_cache > 0
gate matches the text; the log evidence was not reproduced here (no model run).
metal/src/mtl_rt.mm names MTLGPUFamilyMetal4 (macOS 26 SDK) and MTLCompileOptions.mathMode (macOS 15 SDK) behind @available, a run-time check;
the names must still exist in the SDK the engine is compiled against, so the metal-link job failed on the macOS 14 SDK on every pull request that
touched src/, include/ or metal/. The job now runs on macos-26 (the runner-images repository lists a macos-26 image). Ported from PR #3870
by stuchapin909. A GitHub Actions run cannot be made here: the author ran this workflow on his fork; the first run after the merge proves it.
#3824 changes the same line with macos-15 and an Xcode 26.0.1 path pin, and is not taken.
make_blob quantized its random test rows with ggml-cpu's type traits from_float, which the i-quants do not have: an IQ3_S pair called a null
pointer (an access violation on Windows, SIGSEGV on Linux) before the first timing. It now uses ggml_quantize_chunk, with a flat importance
matrix where the type needs one. A Q2_0 gate/up or down is timed as pool.cpp runs it (activations as an ActQ, Strata's own Q2_0 rows, SwiGLU
in between) instead of with ggml-cpu's Q2_0 dot, and the pair's header line says so. Other formats are timed as before. Ported from PR #3871 by
Licenser.
Built on this PC (MSVC, CPU only) and run: iq3_s/q2_0 0.531, iq2_xs/q2_0 0.336 and q2_0/q2_0 0.214 ms per expert per thread (--mb 512, 1 thread).
Checkpoint.raw mapped every shard for the life of the process. On Windows each faulted page of the ~64 GB of expert bytes is charged to the
process working set, so the pack step looked like a 64 GB RAM demand and was killed by free-RAM watchers on a PC with a model loaded (nothing
leaked: it all came back at once). raw() now reads the tensor's byte range into a fresh buffer through one cached file handle per shard; the
pages go to the system file cache, which counts as available. The author measured the same bytes, 1.2x faster, flat working set, and a
byte-identical pack. raw() is no longer thread-safe (this file uses no threads). Ported from PR #3378 by jerry78424 (the fix for his issue #3377)
with its two tests; I ran the tests (13 pass); the full 102 GB pack was not run.
A GGUF header with an unknown metadata type or an impossible length raised KeyError or OverflowError inside the reader, and setup, which expects
a ValueError, printed a traceback instead of its diagnostic naming the broken shard. Strings, scalars, arrays and tensor dimensions now share
one bounded read; counts are checked against the bytes left in the file before anything is constructed; type ids are checked before their values
are decoded; nested arrays are reported as unsupported; general.alignment must be a positive multiple of 8. The file name is added once to the
message. Ported from PR #2288 by KeyanHu-git with its tests (tools/test_gguf_reader_malformed.py).
Checked against the old reader on the 18 GGUF v3 vocabulary files of the pinned llama.cpp: same metadata, tensors and data start.
When one assistant message made several tool calls and the client returned the results in another order, the call ids were dropped before the
prompt was rendered, so the model saw anonymous results in the wrong order. The ids are now kept while the request is normalized; a run of
results is reordered to the call order only when the call ids and result ids are non-empty unique strings with matching sets. Anything else
(missing, duplicate, unknown or incomplete ids) keeps the received order, and the ids never reach the template. Ported from PR #3395 by
j-luwierski with its seven tests (serve/test_chat_tool_results.py); the hunk was merged by hand beside require_messages.
Problem: an Ada card took the shared MMVQ-IL table, which has no Q3_K row, so
Q3_K shapes ran the plain kernel and the other types used a table measured on
sm_86 and sm_120.
Change: kIlRows_89 (2-4 columns) and kIl1Rows_89 (one column) are registered
for compute capability 8.9. The 2-4 column table keeps only the cells where the
RTX 4070 run (rodi4eg) and the RTX 4080 run (kzhv) gave the same rows count and
sets the rest to 0, the plain kernel; the 4070 and 4080 disagree in 13 of 15
cells of the unmerged tables. The one-column row is Q3_K only, which both cards
gave. The issue's 5-8 column table is not taken: the 4080's changed in 10-14 of
20 entries between two runs of the same card, so there is no kIlRows58_89. The
bench output files of the PR are not ported.
Every choice in a table is bitwise native_mmvq's output (kernel row split only),
so answers do not change; this is speed only. No end-to-end decode change was
seen on a 4090 or a 4080 in the issue, so the gain is the microbenchmark's.
This does change which kernel an sm_89 card runs by default, so the model gate
(identity on sm_89) is written down in phaseA-tests/gh027/gates/3453-sm89-il.sh.
Tests: mmvq_il_parity (registered in CMakeLists.txt) now checks sm_89's table
against the two source runs (check_tables) instead of expecting the shared
table, and still checks bitwise parity of every case and rows choice on the GPU
it runs on (sm_86 on z440). docs/MMVQ_IL_TABLE.md updated.
Not covered: no sm_89 card here, so the table speed is the contributors'
numbers, not ours.
Co-authored-by: kzhv <[email protected]>
When the elastic image encoder failed to restart, its exception left request preparation and the client got a closed connection. It now goes
through the EngineDied path: HTTP 503 with the reason, a later picture tries again, text requests and a later good start are unaffected. An idle
encoder stop is logged even when the cache shrink was refused, and a layer split gets a line saying its encoder VRAM has to be kept free with
--vram-reserve-mib. Ported from PR #2435 by i-Vladimir-i with its tests (chat, messages and responses, streamed and not, plus a real encoder
that exits with code 71 on Linux) and the docs/VRAM_ELASTIC.md text.
On a layer split the later card is left with a few dozen MiB after loading, and each single-request window size is
captured the first time it is used (68, 64, 58, 52 and then 10 MiB free in the report); when one capture's
cudaGraphInstantiate ran out of memory, instantiate_evicting could only free graphs of batch slot layouts, a
single-request run has none, and "verify: instantiate: out of memory" ended the engine.
Now, after the batch layouts, a single-request window's capture frees the graph of the largest other window size (never the
one being captured or a prelaunched one; pick_window_victim) and instantiates again, and says so once: the freed size is
captured again when it is next used. capture_all keeps every size at once and is excluded. Only a failed instantiate does
anything new; with enough VRAM nothing changes.
window_evict_test (CPU, header-only) covers the choice. The real gate needs a model on a 2-GPU split with a thin later card
(listed in the triage file); not changed here: the later-stage reserve of the split search (kWindowMib), which is a default
placement change.
Under WDDM every cudaEventRecord and cross-stream wait is a submission of its own (the author measured ~80 / ~45 us each
on an RX 6900 XT under Windows 11), and the streamed walk paid one record and one wait per expert although a group
gathers after one wait on its last copy. Two tester switches, threaded issuer only:
- STRATA_PREFILL_COPY_EVENT_EVERY=N (2..64): the copied event is recorded every N-th entry, before the issuer waits and on
the last; the compute stream waits on the first recorded entry at or after the one it needs (the copy stream is in order).
- STRATA_PREFILL_COPY_WAIT_DEDUPE=1: the copy stream skips its wait on a slot's release event when the previous slot was
released by the same event.
The same copies into the same slots: the same bytes.
Default path (both unset): read line by line. The issuer records copied[slot] and waits on used[used_of[slot]] exactly as
before (the counters are bookkeeping, cover() is a no-op). Added to the PR so this holds by construction: wait_copied takes
the slot the old code used and, without the sparse switch, waits on copied[slot] at all three consumer sites (the PR derived
the slot from the entry number). The record/wait selection is now three pure functions in
include/strata/prefill/copy_events.hpp, with a host test (copy_events_test, registered in CMakeLists.txt) that models the
issuer and consumer with a full ring and checks that every wait names a live recorded entry at or after the one needed.
Tests: copy_events_test passes (z440, g++ -std=c++20). Not yet: a CUDA build of prefill.cpp in this tree (queued), no GPU
run. Not covered: the Windows HIP / WDDM speed claim and NVIDIA (the gate is written down). Needs the author's numbers
repeated before any default talk.
--host :: failed before the model loaded because Server always opened an IPv4 socket. Server now selects AF_INET6 when the bind address contains a
colon; the ready line, the 'from other devices' lines and the browser address bracket an IPv6 address. The bind is still the person's choice and
no default changes. Ported from PR #3212 by purrsec with its tests (serve/test_security.py, including a real :: bind). setup.py's --host help and
docs/DETAILS.md say :: is all IPv6 interfaces; the README hunks of the PR (they name strata launch --host, not checked) are left out.
A reporter built Strata for two Quadro M6000 (sm_52) by lowering two
floors and saw about 22 tokens a second with Qwen3.8-Flash-Next IQ1.
CMakeLists.txt: the experimental build (-DSTRATA_EXPERIMENTAL_SM60=ON)
now accepts sm_52 and prints a warning for it (Maxwell, one reporter, no
native half precision, CUDA 12.x). Without the flag the build still
stops at sm_75 as before, and the architectures a build uses are not
changed.
setup.py: sm60_card() admits a 5.2 card only when the tester sets
STRATA_EXPERIMENTAL_SM52=1, because the ready-made CUDA 12 engine holds
no sm_52 code and a Maxwell card must not be routed to it by default.
With the variable set, setup warns about the Maxwell card instead of
refusing. Default: a Maxwell card is "not supported" exactly as before.
docs/OLDER_GPUS.md: a Maxwell row and a short section, with the build
command and a request for an answer-text check.
Tests: tools/test_setup_older_gpus.py (20 tests, new Maxwell cases for
default, opt-in and a newer card present). Not covered: no model run, and
the answer text on a Maxwell card is unchecked.
When HEAD and the one-byte range probe both gave no length, download() took an early EOF as a finished file even though the repository pin (or
ModelScope's published size) names the size. The published size now stands in for the length, so the existing resume, retry and parallel code
publishes the file only after it has all its bytes; a size that disagrees with the pin is still refused before the transfer and the SHA-256
check is unchanged. Ported from PR #2287 by KeyanHu-git with its tests (tools/test_setup_download_length.py).
install_own_path treated a file as present only when <file>.done existed, and that mark is written by setup's own download or by verify_sha256,
which a file from another tool (LM Studio) never reached: setup said it was missing and asked for the full download size in free disk. With
--gguf-dir an existing file and its mmproj go to verify_sha256(own=True), which checks size and SHA-256, leaves a mismatching file alone and
writes the mark. A file that is not in the folder is still reported missing. Ported from PR #3606 by ai-bits with its two tests.
Under WDDM cudaMemGetInfo answers for the calling process: memory another process holds on the card (the image encoder,
a browser, another engine) is not in its free figure. The reporter measured, on an RTX 5090, 30,994 MiB free from CUDA
where NVML read 24,348 with 6 GiB held elsewhere, so --expert-cache auto sized the cache too large and shrank it by
guesses.
device_free_bytes() now takes min(CUDA's figure, NVML's figure for the whole card) when STRATA_VRAM_FREE=device is set.
nvml.dll is loaded from the system directory at first use (LoadLibraryExW with LOAD_LIBRARY_SEARCH_SYSTEM32), the card
is matched by its PCI bus id, and a missing nvml.dll, a missing export or a failed nvmlInit leaves CUDA's figure with
one line on stderr. The serve check of "VRAM free with everything loaded" uses the same function. The value can only get
smaller, so the switch can only shrink the expert cache.
Unset (the default), nothing changes: the NVML code is compiled only for _WIN32 and not for HIP, Metal or the no-VMM
build, and the env variable is read once. The serve check's min() is the same value elsewhere (on a unified-memory
system device_free_bytes() is never below cudaMemGetInfo's figure). Added to the PR: the NVML block is also left out
of a SYCL build (SYCL_LANGUAGE_VERSION), whose CUDA names are a compatibility layer. docs/DETAILS.md: one row in the
measured-and-left-off table instead of the PR's long paragraph.
Tests: builds on Linux CUDA (the Windows-only block is skipped), SYCL (P1006) and on Windows MSVC (owner PC build of
the final tree). Not covered: no run with a model; the Windows gate is written down.
resolve_rope accepted nan, inf, zero and values below 1 and handed them to the engine. It now stops with a message that names --rope-scale and the
valid range, before any configuration is written; valid factors and the automatic scaling are unchanged. Ported from PR #3393 by j-luwierski
(follow-up to #2950) with its tests; the --help text gets the range.
tools/check_cuda_spills.py reads the resource usage the compiler wrote into the kernel libraries (cuobjdump
--dump-resource-usage: STACK and LOCAL per entry function and per sm_) and fails on any Strata kernel outside its ALLOWED
table, which carries the measured allowance of the kernels that hold such an array on purpose; ggml's vendored kernels
(no namespace) are skipped and counted. ctest cuda_spill_guard runs it on eight libraries, skips (77) without cuobjdump or
a demangler, CUDA builds only (HIP has no cuobjdump); check_cuda_spills_unit runs tools/test_check_cuda_spills.py
(stand-in demanglers are sh scripts, so not on Windows).
Changes against the PR: (1) a demangler that exits 0 but prints a blank name used to make the kernel's key empty, which
reads as vendored, so a spilling Strata kernel was skipped; that is now "no verdict" (77) with a test. (2) The unit test is
registered. (3) The ALLOWED table was re-measured on f-027-int: sm_120 with CUDA 13.0 on Windows (MSVC) and sm_86 with the
CUDA 13.0 objects built on Linux (cuobjdump 12.9); allowances are only raised (new DFlash2 gemm8_kernel and
rope_table_kernel, native_gu_swiglu_kernel on sm_86, dequant_kernel / expdec_gu_kernel / prompt_attn_i8v3_kernel /
native_gu_swiglu_kernel on sm_120), none lowered or dropped. Result: 0 violations on both architectures; a test shows the
same object fails once one allowance is lowered.
Ported from PR #2683 (the tracking issue is #2682).
A json_object or json_schema answer that holds LaTeX such as \alpha is
rejected (502) because \a, \g and the like are not JSON escapes. New
opt-in: "repair_latex_escapes": true in response_format (and in
text.format on /v1/responses). Only after the strict parse failed, the
server doubles each backslash that does not start a valid JSON escape and
parses once more with the same duplicate-key and NaN rules, then checks
the schema as before, and prints one log line. Without the option, and
for any answer that parses on the first try, nothing changes.
Escapes that are valid JSON (\b \f \n \r \t \/ \" \ and a well-formed
\uXXXX) are left alone, because they cannot be told from LaTeX: a \beta
next to a \alpha comes back with a backspace. Truncated output, arrays,
NaN and duplicate keys still fail.
Tests: three new tests in serve/test_structured.py (default still 502,
opt-in 200 and stream, failures stay failures, helper cases, Responses
format passthrough). serve/test_structured.py 13 tests OK.
With STRATA_DECODE_TIMING=1 the serial line listed `verify + commit/emit + draft` after `ms/window =`, while the adaptive
wait before verify and the join after the draft were timed but printed after the sum, so the named parts did not add up
to ms/window. The serial line now reads `adapt wait + verify + commit/emit + draft + adapt join + other`, where `other` is
the decode time minus the named parts (not a timed step). The pipelined decode keeps its old line, byte for byte, and
nothing is printed with the timing off or with no windows. Diagnostic only: no decode path changes.
docs/DETAILS.md: a short paragraph on the new line (the hunk was rebased by hand onto the current file).
Tests: compiles on CUDA (the format string was counted against its 22 arguments); no parser in the repo reads
`ms/window = verify`. Not covered: no run of the line itself (it needs a decode).
The quick speed check has its own GPU probe, separate from strata_cli/bench_facts.py, and tried nvidia-smi then rocm-smi only. On a Radeon with
amd-smi alone it wrote the server's vendor stub and 'GPU driver: not read'. The same amd-smi probe as #3519 is added in the shape this file
uses, and a bare vendor name from the server no longer replaces a card the probe named. Ported from PR #3630 by l33tm4st3r.
The three new tests are POSIX-only (symlinks, /sys); run on z440 (Linux, Python 3.12): OK. tools/test_strata_bench.py on Windows: OK.
A batch slot holds the reply it decoded. The main session was parked with the conversation's prompt only (the
admission's copy), so when the next admission or a yielding read overwrote the slot, the reply lived nowhere: the
conversation's next turn found its prompt-read state and read the whole reply again (the author measured 255 of a
256-token reply re-read on four 200K chats over two slots).
What changed:
- park_current is split into park_source (one budget / RAM-admission / snapshot path) and two callers: park_current
(the main session, output line and byte accounting as before) and the new park_slot (an idle slot's ids, checkpoints,
sessions and draft K/V). The ParkSource passes what a park captures; the main session's retained K/V (#1503
take_reuse_for) is taken only for the main session, never for a slot.
- park_slot runs before copy_to_slot at a slot admission and before a BYIELD read overwrites a slot. It does nothing
when the live main session or a parked conversation already holds that state (ConversationCache::holds /
conversation_covers, new, with unit tests), for a slot with pictures, or without --batch-mtp when the drafter is on.
- When a slot is admitted into the main session its checkpoints come along (the shorter ones too), so the chain keeps its
branch points and the parked copies a turn back are superseded (#342) instead of taking an entry each.
- Same refactor in sycl/src/program/generate.cpp.
Changed from the PR: parking is an optimization like a checkpoint (#3350). The PR ended the request (ERR) or the batch
(batch_fatal) when park_slot's snapshot failed; here a failed snapshot, a failed device sync and a host allocation
that runs out are logged, the slot is overwritten as before and the request goes on. The slot's extra checkpoints that
come along at an admission are copied under a bad_alloc guard (the one the request continues from is copied as before).
Tests: conversation_cache_test (new holds() cases and the four-conversation replay at policy level) and
conversation_split_failure_test. Not covered: no model run (the batch path itself needs one); the gate is written down.
The Responses validator still refused anything below 1 after the chat
path made max_tokens of -1 and 0 mean no cap (documented in
docs/DETAILS.md). It now accepts -1 and 0 as no cap; values below -1,
floats, booleans and strings stay a 400. Nothing else changes: the
server already passes max_output_tokens or 0 to prepare(), which gives
the answer the room left in the context.
Tests: OverHttp.test_max_output_tokens_zero_and_minus_one_mean_no_cap
(stream and non-stream, status completed) and
test_max_output_tokens_bad_values_are_400. The first fails without the
change. serve/test_responses.py 43 tests OK.
Problem: with STRATA_Q8_SM60=1 the main model's Q8_0 matrices run on the GP100
Q8_0 kernel, but the MTP drafter's own Q8_0 draft head and projections kept the
generic kernels.
Change: with STRATA_MTP_Q8_SM60=1 (on top of STRATA_Q8_SM60=1) the drafter packs
its Q8_0 projections as packed copies, as the main model does (the prompt path's
draft_kv still reads the GGUF layout, so a copy is kept), and packs the draft
head's subset in place when every column count fits the kernel's shared-memory
planes and the data is 16-byte aligned (otherwise a packed copy). The in-place
pack writes the GGUF blocks back if the device write fails. Release unregisters
the entry without a cudaFree when the data was packed in place. New
native_q8_0_sm60_pack_quiet and native_q8_0_sm60_pack_in_place; HIP returns
false; metal/src/engine/stubs_0145.cpp gets matching stubs that answer
"nothing packed".
Replies do not depend on the drafts (they are verified); only the draft rounding
changes, because the Pascal kernel sums in another order.
Default path: mtp_q8_sm60() is false unless the env is set, so the packing calls
are skipped. The destructor now walks the tensor list once at teardown, not in a
hot loop. The SYCL tree has its own sycl/src/core/mtp.cpp, so it is not touched.
Tests: compile-only on sm_60 (CUDA 12.9) on P1004. Model gate (identical replies
with the flag on and off, decode speed on a P100) is written down in
phaseA-tests/gh027/gates/3751-mtp-q8-sm60.sh.
Not covered: no GPU run here; the contributor measured +4.4% on four GP100 dies
with all experts resident, one run per arm, not on a single streamed P100.
#1337 made a failed engine start in a later calibration step a candidate
that loses. An engine that started and then exited during its own rate
measurement still failed the whole run. measure_candidate() now treats
both the same: the candidate is dropped and recorded in
report["failed_starts"], the earlier results stay. It is used for the CPU
worker step, the adaptive expert tier step and the AMD shared-expert
fork. An operator cancel (CalibrationCancelled) is still raised. The first
start (steps 1 to 3) still fails the run.
src/core/verify.cpp: when adapt_gate_wait times out after 20 s, the
message now says which layer was reached (atomic and copy signal), that
the copy stream is not drained by the release, and dumps the
STRATA_VERIFY_TRACE breadcrumbs when that is on. Diagnostics only, #2188
stays open. The watchdog label is set only once a wait actually starts,
so a gate that is already open costs nothing extra (changed from the
PR, which set it on every call).
Tests: tools/test_calibrate.py 33 tests OK (3 new death tests).
Not covered: engine compile of verify.cpp is checked in the build step.
Problem: on a GP100 die (cc 6.0, no dp4a, no tensor cores) the verify window's
routed down projection for the native packs' IQ4_NL and Q2_0 rows (n_ff 640)
is latency bound. native_down_multi_kernel reads each weight as 2-byte loads,
one item at a time, straight from global memory.
Change: native_down_ws_kernel copies a block's rows (IQ4_NL 32, Q2_0 64, 11,520
bytes) into shared memory with 16-byte loads, then runs the same row dots on the
copy: same sub-warp, lane, items, float expressions and order. Only the integer
sum inside an item changes form (vmad byte selectors, prmt table lookups), the
same int8 x int8 products, so every output is native_down_multi_kernel's (expert
mode 0) bit for bit.
Gating: compiled only in the CUDA build with STRATA_EXPERIMENTAL_SM60 (not HIP,
not Metal). At run time only with STRATA_DOWN_WS=1 on a device of compute
capability exactly 6.0, IQ4_NL / Q2_0 down rows at n_ff 640, and not with
STRATA_OLD_IQ_MMVQ or STRATA_NO_SUB16_GU. Unset, the only change on the default
path is one cached boolean test per native_expert_grouped call.
docs/OLDER_GPUS.md lists the switch. The contributor's microbench numbers (one
PH402 die, Swift 1.5 IQ3_S, one-token window: IQ4_NL 72.6 us mode 8 and 54.5 us
mode 0 become 42.3 us; Q2_0 53.5 / 65.4 us become 38.6 us) were measured on a
four-GP100 rig, not here.
Tests: compile-only on sm_60 (CUDA 12.9) on P1004. The contributor's bitwise
check (native_expert_bench hash against modes 0 and 8) is not run here: it needs
a GP100 GPU run and the model's layers. The gate script is
phaseA-tests/gh027/gates/3867-down-ws.sh.
Not covered: no end-to-end measurement on a single P100 (16 GB cannot hold the
47 GB pack resident).
ROCm 7 and newer ship amd-smi and no rocm-smi, and the amdgpu product_name is empty on gfx1201, so a Radeon AI PRO R9700 run was filed
under 'unknown' with 'runtime: not measured'. bench_facts reads the market name from amd-smi static --asic, the VRAM from the amdgpu sysfs
file of the PCI address amd-smi list reports, the ROCm version from the install (or amd-smi -h), and keeps the probe's card name when the
server only says 'AMD Radeon'. Ported from PR #3519 by l33tm4st3r with its tests and the BENCHMARK_PROTOCOL.md text.
Changed while porting: the amd-smi branch also took macOS (os.name is posix there), which would have hidden the Apple chip; it now skips
darwin, and a test keeps the chip.
The "This computer" card said only "Fits fully in memory." for DeepSeek, with no word of the ~265 GB of disk. For a DeepSeek version it now says "Needs about {N} GB
of free disk space. What does not fit in memory is read from your SSD." (the spill line stays beside it when part of the model is read from the SSD), and the
selected version's line reads "{N} GB RAM + VRAM, plus about {disk} GB of disk". {disk} is not hard-coded: the catalog (tools/strata_mcp.py) gives each set
its `disk_gb`, the set's whole download (model files, n-gram table, drafter, metadata) plus the pack and room to work (dsv41_fit.DISK_HEADROOM_GB), so 2.00 and
2.25 differ. Same card and row markup, no design change; every other model keeps its card. Both sentences are fx80 and fx81 in the eight fix catalogs.
The Terminal's DeepSeek description makes no "fits fully in memory" claim (its lines already state the download and the free disk), so it is unchanged.
Older converters write the Engram scalars as engram.key_length and
engram.pad_id, and omit the sparse-attention geometry. The packer now
reads the old names only when the canonical engram.head_dim and
engram.pad_token_id are absent, and a new --config-json option fills the
omitted geometry from the source model's config.json. A value that
disagrees with metadata already in the GGUF stops the pack with a clear
message. The weights are never rewritten and a set that already has the
canonical names packs exactly as before.
Tests: LegacyModelMetadata (2 new, 5 in the file) in
tests/terminal/test_pack_gguf_engram.py.
Not covered: the author reports the older Q2_K set still needs dense
Q2/Q3 and Engram Q2_K tensor support in the engine; this does not add it.
The final response.completed / response.incomplete event and a
non-streaming /v1/responses reply now carry the same top-level timings
object that /v1/chat/completions has. The server already put timings in
the done dict; the Responses assembler dropped them.
Tests: UsageAndStatus.test_responses (stream and non-stream).
Unset timings leave the event unchanged; response.failed has none.
test_iq_pack imports without llama.cpp's gguf-py and skips the classes that build GGUF fixtures with a named reason; the MCP discovery tests use
a free port, not whatever Strata serves on 8080; the six conversation-cache tests add the repo root to sys.path so they run from the repo root
as the suite does. Ported from PR #3236 by Buddychek (tests only, no shipped code).
The 0.2.5 rewrite of the page dropped the 'Windows AMD: the driver resets' entry (v0.2.4.2 had it), although setup.py still tells a Windows gfx12
card owner to read it and tools/test_setup_amd.py (TdrPointer) checks for it, so that test failed on main. Restored from v0.2.4.2 with the
log and config locations worded for a source checkout.
Three 0.1.42 changes no page told. Ported by hand from PR #3244 by Buddychek (the PR conflicted with the rewritten pages), each sentence
checked against the code: /metrics/prometheus needs the API key like /metrics (serve/server.py _authorized), the prompt-speed row is
(prompt_tokens - reused) / prompt_ms (apps/host/server/monitor-dashboard.mjs), STRATA_VCVARS is read by setup.find_vcvars. The compiler
text now says Visual Studio 2022 (2019 is refused since 0.2.5, #3566) and went to docs/SOURCE_INSTALL.md.
conversation_detail::private_stream() and order_after_queued() kept one non-blocking stream and one event per device in
thread_local storage and never destroyed them. The layer split's later stages run each prompt chunk on a fresh
std::async thread (STRATA_HANDOFF_PIPE off, the default), and a checkpoint part is saved on that thread, so every
periodic checkpoint of every long cold prompt left a stream and an event behind on each later card: a few MiB of VRAM
per long prompt that never came back (#3470: the later card's free VRAM drops 2-6 MiB per long cold prompt, no plateau),
until cudaStreamCreate failed with "out of memory" (#3350: "conversation snapshot running-state copy: out of memory",
engine exit) on a card that was already nearly full.
A thread now hands its handles to a process-wide pool when it ends and the next thread takes them, so the number of
streams and events is the number of threads copying at once. Same stream, same flags, same contract; only the lifetime
changed. copy_streams_created() / copy_events_created() count what the process made.
conversation_copy_streams_test (CUDA, no model, skips without a device): 200 short-lived threads in a row create at most
one stream and one event; ten rounds of six concurrent threads at most six more; the same on a second device.
A conversation checkpoint is an optimization, like parking, but the writer treated a failed copy as fatal: "conversation
snapshot running-state copy: out of memory" (cudaErrorMemoryAllocation from the snapshot copy: the private copy stream's
first creation or the driver's staging on a nearly full card, not host RAM) made the prefill fail and the engine exit,
21 times in 4 h for one reporter and 16 in 34 h for another, each exit a reload and the in-flight requests lost.
Now, after a failed save, checkpoint_failure_is_survivable() clears the non-sticky allocation error and synchronizes the
device: when that is clean the checkpoint is skipped with a line on stderr and the request goes on (a layer split's stage
part is left empty, which already means "no checkpoint at this length"; the message-boundary / turn / root checkpoints
after a prompt chunk do the same). A device that is unwell (a sticky error, a GPU that stopped answering) still ends the
request exactly as before. Same edits in sycl/src/program/generate.cpp.
STRATA_TEST_COPY_FAIL_AT=N (tests only, like STRATA_TEST_SLOW_COPY_US): the N-th snapshot copy of the process fails with
cudaErrorMemoryAllocation, so the case can be made without a full card. Unset, nothing changes.
Not changed: "verify: instantiate: out of memory" (graph capture), which the reporter saw once.
CMakeLists.txt tested "NOT DEFINED CMAKE_CUDA_ARCHITECTURES" after enable_language(CUDA). Since CMake 3.18 that call
fills an unset variable with nvcc's default (sm_75 for CUDA 13), so the sm_120 default never applied: a hand build with
no option was an sm_75 build. The default is now set before enable_language, unless the user gave
-DCMAKE_CUDA_ARCHITECTURES or the CUDAARCHS environment variable, and the configure step prints a warning that says what
to pass for another card (a recommendation, not a refusal).
With -DCMAKE_CUDA_ARCHITECTURES=native, the one NVFP4 W4A4 unit (mmq_nvfp4_w4a4.cu) stayed on plain `native`, where
ptxas refuses the FP4 MMA on an RTX 50. `native` is now replaced by CMAKE_CUDA_ARCHITECTURES_NATIVE (the detected
architectures) before the 12x -> 12xa rewrite, as a list, so a mixed PC works too. docs/SOURCE_INSTALL.md says what
`native` and the other values build.
Release and setup builds pass their architectures explicitly and are unchanged. Configure matrix, CUDA 12.9 on Linux:
none -> 120 (+ the warning), CUDAARCHS=86 -> 86, 89, "75;86;120" (w4a4 120a only for the 12x entry), native -> detected.
Ported from PR #4029.
--adapt-decay took any number. Above 1 the adaptive tier's usage counts grow without end (inf after a few hundred
adaptations) and the swap gains turn NaN, which breaks the sort that orders the swaps; a negative value flips the counts.
The value is now checked where it is parsed (include/strata/program/adapt_decay.hpp, also in sycl/): a finite value in 0..1
is used exactly as before (0 and 1 included, the range the calibration profile already allows); anything else, NaN and inf
too, runs with the default 0.7 and says so on stderr. A warning instead of the PR's exit 2: a risky pick is reported,
the engine still starts.
adapt_decay_test (25 checks) covers the range, the fallback and finite counts after 100000 adaptations.
Ported from PR #1284 (the check; the refusal became a fallback).
The sandbox of a test start was deleted at once, so the PID records that show why a start failed were lost; only the last 64 KiB of the engine
log stayed. A failed start now also keeps run.json (without keys), the host, child and ready JSON records of the runtime (tokens cut to their
first 8 characters) and the host and server logs in .updates/logs/test-start-<version>-failed. The last three failed starts are kept.
Best effort, never hides the failure. Tests: tests/terminal/test_updater.py (FailedStartRecords).
The manual build in docs/INSTALL.md uses CMake's default generator (Unix Makefiles on Linux) in build/, and setup --build always asks for
Ninja, so cmake stopped with 'Does not match the generator used previously'. cmake_build now reads CMAKE_GENERATOR from the folder's
CMakeCache.txt; when it is not Ninja it says so in one line and removes the cache and CMakeFiles (no other file). A Ninja folder or a new
folder is untouched. Tests: tools/test_setup_build_dir.py.
A cmake -P script has no cmake_minimum_required, so CMake 3.x used the OLD behaviour and stopped at 'if given arguments ... IN_LIST: Unknown
arguments' (ctest ggml_tq_overlay with CMake 3.31). Checked with pip cmake 3.31.6 and 4.4.3 against the pinned llama.cpp: both print
'ggml_tq_overlay_test: ok'. A static test keeps the policy before the first IN_LIST in every cmake/*_test.cmake.
The script ended with a grep pipeline, so a failed compile exited 0 (and, before the ocloc check was added at the end, a good build exited 1).
A build.sh && next-step chain then ran after a failure. The status of cmake --build is kept and the script exits with it. The test runs the real
script with a stub cmake on PATH (good build: 0, failed compile: 1; the old script exits 0 on the failed compile).
tools/test_setup_golden normalized strata.exe but not strata-vision.exe, so all 25 baseline entries failed on Linux (48 subtests). VEXE
and HVEXE are mapped to <VEXE> and <HVEXE> like EXE, and the baseline records <VEXE>. test_nvfp4_local_pack imports pytest, which is not in
requirements.txt; under python -m unittest discover the module is now skipped with a reason instead of failing to import.
Substitutions were ordered by the length of their regular expression, and a user name wrapped in word bounds outranked the Strata or home
folder that contains it, so a report kept /home/<user>/Strata instead of <strata>. They are now ordered by the text they replace. Ported from
PR #3619 by ctr-137 with its test (Linux and Windows layouts).
The page said that starting a model never downloads and that nothing checks for updates in the background. Since 0.2.4.2 strata launch with a
DeepSeek model checks the signed feed (10 s at most, silent without a connection) and replaces an older installed DeepSeek engine package.
Ported from PR #4028 by pschaub (wording from issue #4022), checked against strata_cli/launch/runner.py upgrade_dsv41_engine.
engine_env put the ROCm library folders first on PATH, which only Windows' loader reads. On AMD Linux strata --families exited 127 on
libhipblas.so.3, so setup falsely said the engine does not run Qwen3.6. On Linux and elsewhere the folders now go first on LD_LIBRARY_PATH
(an existing value is kept); Windows keeps PATH. Ported from PR #3305 by xxxliqu1dxxx (resubmission of #2914), with its tests.
metal/setup_mac.py returned right after the engine step on --update while the model questions had been asked, so choosing a model the Mac did
not have ended with no download and no word. The update now names the missing model (or image encoder) and points at ./setup.sh --setup; an
update for a model that is installed prints what it printed before. Ported from PR #3848 by demetree, with its two tests (mocked; not run on a Mac).
adopt_user_defaults() set the token's default DACL with FA (FILE_ALL_ACCESS). That default also applies to new process objects, and the file
mask has no process query or set-information right, so a child could not be reopened by PID: the HighQoS helper timed out and
procs.start_token could not record the child. GA lets Windows map the mask to the object being created; the principals are unchanged.
Ported from PR #3428 by kossum (QoS reopen test). Test from PR #3399 by midhatn (a live child is identified and recorded) added; both fail
without the change and pass with it.
Co-authored-by: MIDHAT NASHAR <[email protected]>
_start() checked cfg['exe'] with Path.exists(), so a run config whose engine is component:strata-engine was refused as a missing file
before the server could resolve and verify it. Ported from PR #4027 by jubeater. A component reference is no longer tested as a file; model
files and literal engine paths are checked as before. ensure_engine_for, the Windows HIP runtime repair and the HIP architecture lookup
skip the source-build engine folder for a component. Eight tests (tools/test_setup_component_start.py).
Ported from PR #4011 by SuperDistance. project_version() read CMakeLists.txt with Path.read_text() and no encoding, which raised
UnicodeDecodeError on Windows code pages such as cp936. The test makes read_text default to each of 14 Windows code pages. The same crash was
reported in #3235 and #3414.
.git is not in the build context, so package_terminal.py had no commit to put in the archive and the documented source build stopped.
Ported from PR #3644 by robertoetcheverryr: ARG STRATA_COMMIT / ENV STRATA_BUILD_COMMIT as in docker/Dockerfile, and the two 'From source'
commands in docs/DOCKER.md pass --build-arg STRATA_COMMIT=$(git rev-parse HEAD).
Added: .dockerignore leaves out the folders docker/entrypoint.sh creates in a mounted data/ (install, state, home, Strata-data). A dangling
data/install/third_party/llama.cpp link in a clone that had been mounted made the Terminal stage fail with a copytree error (seen in the
review thread of #3644). Not built with docker here (no docker on this PC).
docker/build.sh staged only top-level tools/*.py, so the Terminal stage stopped at 'installer tool missing: tools/dsv41/tune.py'.
Ported from PR #3974 by hurui200320: the pathspec is tools/**/*.py, the no-.git fallback finds nested files, and the context test reads
INSTALLER_TOOLS and INSTALLER_SCRIPTS from packaging/package_terminal.py. Fixes the failure that images built from v0.2.4.1 and v0.2.4.2 had.
The portable lifeline watcher built a child's identity from `ps -o lstart=,stat=` and kept the whole line, so the token
changed with the process state (R while it starts, S asleep). The watcher compared its own reading with the token taken
at start, decided it was not the child it was told about and exited without ending it (test_lifeline "the child outlived
a killed server (watcher)", #3407). The identity is now the start time alone: field 22 of /proc/<pid>/stat on Linux
(a zombie has none), `ps -o lstart=` elsewhere; strata_control/procs.start_token drops the state column the same way.
A launch that is not a real Popen is not watched.
Telemetry._loop no longer ends on the first failing sample (a machine whose GPU or psutil source is unreadable): it
prints one line and keeps sampling, and NoLeakedSampler checks the thread itself instead of counting threads.
New test: the token is the same while the child runs (R) and sleeps (S); on the old code it fails on Linux
('...R' != '...S').
Ported from PR #3590.
A declared namespace.member tool (Codex MCP: mcp__filesystem.write_file) was accepted only as namespace__member (#3556).
Qwen3.8-Flash-Next also writes namespace=member (the <function= tag's equals sign ends up in the name), namespace_member
(one underscore dropped) and the bare member. Each is now an alias of the declared tool when exactly one declared tool
produces it and no declared name is the same; a name that matches nothing stays text (#3075), two namespaces that share
a member are not guessed, and a declared flat tool always wins. Requests without namespaced tools are unchanged.
serve/test_tool_name_alias.py (10 tests, 5 of the first 9 fail without the change; the tenth pins that a flat tool named
like the underscore alias wins), docs/DETAILS.md names the spellings.
Ported from PR #3972.
With --batch and conversation parking, a restore keeps the restored K/V (ConversationCache::retain) so the next park of
the main session copies only what changed. The reuse tracked a token count (unchanged_tokens), not whose tokens, and
copy_from_slot (a batch slot giving its conversation back to the main session) replaced the main session's K/V without
dropping the retained image. The next park reused the earlier conversation's K/V for the new one's prefix: a restore of
the new conversation then answered from the other one's history.
retain() now records the ids of the conversation the K/V belongs to; a park calls take_reuse_for(live), which reuses only
the prefix live shares with those ids (nothing when they share none or the ids are unknown), and one stderr line is
printed whenever a reuse is cut. copy_from_slot drops the retained image before it replaces the main session's K/V.
Same edits in sycl/src/program/generate.cpp. The guard only cuts reuse: at most one full copy of a parked conversation's
K/V, the saved bytes are otherwise identical; no flag, no default change.
Ported from PR #1503 onto the current tree (the first-stage ids sit after the stages field so the existing aggregate
initialisers keep their meaning). conversation_cache_test gains the cross-conversation cases of the PR and the logged
sequence (restore P, a slot hands Q to the main session, park Q): another conversation never gets the K/V, a diverging
one only the shared prefix on every stage, a slot hand-back drops it, unknown identity is never reused. Removing the
ids limit makes the first case fail.
A command on the allowlist used to run without a card with any option except -delete/-exec/-execdir/-fprint/-fls/-ok. Write modes (sort -o,
tree -o, uniq IN OUT), program-execution modes (sort --compress-program, rg --pre, git grep -O), date -s and tail -f got through, and so did
git config -e, git branch --set-upstream-to and git remote ... forms.
Each command now has a spec of the options and operands that only look (short and long, spelled in full); every other option, abbreviation
or attached form asks. git subcommands have the same specs, a branch/tag name is taken only with a list option, git config only reads with
--get. date takes only +FORMAT operands, uniq at most one operand, hostname none. less and more leave the list. Quoted texts are checked as
operands too. `du -sh` no longer trips the 'sh' word check.
Tests: apps/host/tests/gate-readonly.test.mjs (deny and allow cases for options, operands, spellings, quoting; every listed command asks for
an unknown option).
sample() applied the repeat / frequency / presence penalties to lg[id] for every token id of the last penalty_last_n
tokens of the history. A row shorter than the vocabulary (the MTP draft head picks among the first N token ids,
--draft-vocab N) made a history token with id >= N an out-of-bounds read and write on the host. A token that is not in
the row takes no penalty; the loop skips ids < 0 or >= lg.size(). A full-vocabulary row is bit-identical.
The loop and Sampling move to include/strata/qwen35/sampling.hpp (header-only) so a CPU unit test can run them:
src/qwen35/sampling_test.cpp (qwen35_sampling_test) checks nothing is written past a short row, that in-row tokens
take the penalty, and that a full row matches the unguarded loop bit for bit over 108 penalty/window mixes. Without the
guard the test dies with an access violation.
Co-authored-by: yehya <[email protected]>
cache_settle()'s store preload (DS4_SETTLE_STORE) read 2,243 hot experts into free store slots after every chunked prompt: 4,713 ms of a 4K prompt. Measured on the box (code fixture, prompt + 256 tokens, interleaved, 3 runs a side): with it on the decode is 13.9/14.4 s, with it off 14.2 s - no benefit - while the 4K prompt goes 31.3 -> 26.6 s (130.89 -> 153.78 tok/s). The swap loop above it is what the decode uses (it holds the CPU lane at ~5-7 k experts a run against ~14 k without), so that keeps its budget; DS4_SETTLE_STORE=1 restores the preload.
On the RTX 5070 (--agent-long, 22 tasks) split 36 kept agents at 6/10 (exact 4 and 7) and extraction at 585/606 facts (exact
585) for +11% prompt speed at 14K and +18% at 30K; the split-24 default passed 0/10 agent runs and found 556 facts, a tail of
8192 with split 24 found 574. KVA stays off unless STRATA_KVA=1. docs/KVA.md gives the numbers and the reasons; kva_test
checks the opt-in's split 36 / tail 2048 and the overrides. The geometry gate is unchanged.
The app toggle will write STRATA_KVA=1 for Qwen models in general; the split layer, the tail and the measured quality apply to
Qwen3.8-Flash-Next's shape only (48 layers, 512 experts). Any other shape (Qwen3.6-35B-A3B, ...) now logs "KVA configured but
NOT applied: this model's shape (L layers, E experts) is not one KVA was measured on" and reads exactly; the routine reasons
(short prompt, continuation) still come first. STRATA_KVA_ANY_MODEL=1 allows it, with one warning. kva_test and docs/KVA.md
cover both.
KVA was measured on the Flash-Next shape (48 layers, 512 experts) and the engine declines any other shape with a log reason, so the row would do nothing
for Qwen3.6-35B-A3B. Setup writes "arch" in a run config only for the families with an engine path of their own (Qwen3.6-35B-A3B: qwen35moe; DeepSeek:
deepseek41); every Qwen3.8-Flash-Next file (the original, Swift, Coder, Unsloth, Gyro, NVFP4) has none. The row is therefore shown for a model without an
"arch" on CUDA or HIP, in the app's Native engine settings and in the Terminal's Advanced table. The Radeon RX 9000 and DeepSeek rows are unchanged.
Tests: a Qwen3.6 config gets no KVA row (CUDA and HIP), the six Flash-Next family names do, and through the host fixture a model with an arch has no switch row.
The quality runs found lost records and invented rows in long extraction tasks and defects in agent loops with a ~10K
system prompt when most of the prompt was approximated. Opt-in now: unset (or 0) nothing runs, the draft layer included.
Switched on, the tail is 8192 tokens (STRATA_KVA_SUFFIX overrides), the safest configuration measured. A model other than
Qwen3.8-Flash-Next's geometry gets a one-line warning instead of a refusal. Kept: the untouched draft layer (cell floor
behind STRATA_KVA_DRAFT_FLOOR=1), the decline reasons, STRATA_KVA_SKIP=moe, the tests. docs/KVA.md says why it is off and
how to turn it on; kva_test checks unset = off and STRATA_KVA=1 = on with the 8192 tail.
The Speed row now follows the shared table's default: Faster long prompts reads "Off (recommended)" / "On" and its new sentence is "Reads very long prompts
faster by approximating part of the work. Off by default: answers about long documents can miss or mix up details." (replaces fx73; "Off (recommended)"
is fx79; translated in zh, ja, de, es, ru, pt, ko). The other two rows are unchanged ("On (recommended)" / "Off"). A STRATA_KVA=0 stored by an earlier
0.2.7 test build shows as Off. Still shown only where KVA applies (Qwen on CUDA or HIP). tests/speed-switches.test.mjs updated: the default launch has
no KVA entry, On writes STRATA_KVA=1, Off removes it, the old 0 shows as Off, and the other two switches are tested as before.
Owner decision (26 October, conditional on the last KVA variant results): KVA ships off. strata_control/speed_switches.py now gives every switch a default
and the environment it writes outside that default: KVA is default off and writes STRATA_KVA=1; the Radeon RX 9000 and DeepSeek switches stay on by
default and write their off entries as before. A launch in the default carries no entry. A file that holds STRATA_KVA=0 (the off switch of an earlier
0.2.7 test build) reads as off, which is what it is now; choosing Off cleans it up.
The Advanced table row says "off by default; set 1 to turn on" with the new sentence (`--set STRATA_KVA=1`; `off` or Del puts the default back).
tests/terminal/test_speed_switches.py covers the defaults, KVA on/off/old-zero, the rows and the comparison with the app's table.
The default stays the state-only form (byte-identical). With STRATA_KVA_SKIP=moe the GDN recurrence (gate projection, output
normalization, output projection) and the QSA attention (query side, selection, attention, output projection) run in full
for layers >= split, their outputs go into the residual, and the loop leaves before the MoE half; the expert transfers of
those layers are still skipped. The KVA line says "mode moe-skip". Motivation: the extraction losses of the state-only form
come from late-layer K/V and state built from a stale residual; this keeps the attention halves' contribution in it.
kva_test covers the switch.
fast_attn_enabled() was opt-in (DS4_FAST_ATTN=1); with the prompt chunk at 1024 it is worth far more than when
it was gated: measured on 4x R9700, 2.25 bpw, 8K code prompts, three warm prompts a load, the same ids
(5705e1164a) --
without it 1159 / 1180 / 1195 ppts (median 1180)
with it 1327 / 1354 / 1360 ppts (median 1354, +14.7%)
and after this commit the un-enved default reads 1314 / 1357 / 1369 (median 1357), i.e. identical to setting the
flag, so a deployment no longer needs any environment at all. The QK/PV sums are the matrix cores' order, which
the owner already approved for the prefill; the decode path keeps the scalar kernels. DS4_FAST_ATTN=0 turns it
off (it is a no-op on CUDA, whose dispatch is not compiled).
ring_mb_auto() returned a flat 400 MB on any box whose experts do not fit in VRAM, but on such a box the ring is exactly what lets a prompt queue its likely experts early. 400 -> 1200 MB measured on the box (settle off): the queue 3,200 -> 8,087 experts, the disk's effective rate 1.49 -> 2.44 GB/s, the compute's exposed read wait 11.4 -> 3.5 s, the 4K prompt 148.98 -> 190.98 tok/s; 16K 290.97 -> 315.01, 32K 316.92 -> 325.89. Decode unmoved in an interleaved A/B (17.54/16.39 against 17.61 tok/s). It is now a sixteenth of the VRAM room clamped to 400-1200 MB, so a smaller pool still loads; DS4_RING_MB still overrides.
c.A: the attention slab decides how many times each dense matrix is re-read a prompt. 8 -> 32 MB (STRATA_DS4_A_MB): 32K 325.89 -> 357.78 tok/s, 16K 315.01 -> 328.04, 8K +1.6%, 4K flat, decode unmoved (16.81 against 17.60 tok/s interleaved, noise-limited) for ~0.4 GB of VRAM.
Best measured prompt set with the two new defaults (DS4_SETTLE_MS=0;DS4_SETTLE_STORE=0, which is a prompt-only trade): 4K 195.18, 8K 258.47, 16K 328.04, 32K 357.78 tok/s. With the settle work at its default a 4K prompt is 31.3 s = 130.89 tok/s (was 88.30).
The dense GEMM was 11.1 s of a 4K prompt at hipBLAS's measured 4.97 TFLOPS (M 4096 N 2304 K 5120, the engine's call convention). A blocked WMMA kernel reaches 23.96 TFLOPS at that shape (128x128 tile, BK 32, 2-stage LDS, rocwmma; scripts/wmma2.cu) and the engine's own tuned kernel (BM 128 x BN 256, grouped tile order, raw wmma intrinsics) 20.8-20.9 at the wq_b shape and 38-41 at M 16384. gemm() now calls strata_pf_gemm_f16_ld with hipBLAS as the fallback (rows < 64, N < 512, K % 32, unaligned X: unchanged); STRATA_DENSE_PFGEMM=0 restores hipBLAS for an A/B. Same operands, the kernel's own k order (rounding-level).
Measured on the box (eng-028, 4E224E7F -> 467D7320, 4K prompt fixture p4k, chunk 16384): 46.4 -> 38.1 s, 88.30 -> 107.63 tok/s; with DS4_SETTLE_MS=0;DS4_SETTLE_STORE=0 27.5 s = 148.98 tok/s. 8K 134.66 -> 218.31, 16K 158.6 -> 290.97, 32K 178.9 -> 316.92 (settle off). Decode unchanged in an interleaved A/B (old 15.17/16.73, new 17.38/18.48 tok/s) with acceptance 151/164 = 92% (93/98/100/100/90 per position).
c.A (the attention slab, the reason each dense matrix is re-read 10-16x a prompt) takes STRATA_DS4_A_MB: 32 gives 107.63 -> 110.67, small because the expert reads now dominate.
The Models screen listed the shared sizes for DeepSeek but installed and loaded it at 8K whatever was chosen. Now the choice counts (8K stays the
default for a version that is not installed) and the list is DeepSeek's own: 8K to the limit the installed pack states in ds4.txt (max_position_embeddings,
read by the host as `contextMax`); a pack without it behaves as before (up to 256K). Past 256K the existing note slot under the model shows: "Not tested yet.
A context this long takes graphics memory from the model and reads long prompts slowly." (fx78 in the eight fix catalogs). Qwen's choices are unchanged.
src/lib/deepseek-context.ts is the app's copy of strata_control/dsv41_context.py; tests/terminal/test_dsv41_context.py compares the two.
The model reads 1,048,576 positions natively. Setup and `strata launch` offered DeepSeek the shared list that stops at 512K and dressed anything past
256K in Qwen's "rope scaling" words. Now:
- strata_control/dsv41_context.py: the choices are 8K, 32K, 64K, 128K, 256K, 512K and 1M (labelled "512K" and "1M"), limited to the pack's own limit
(max_position_embeddings in its ds4.txt) when it has one; a size past it (2M, or 1M on a pack that says 256K) is the one refusal; past 256K a plain
note says it is not tested yet and what it costs.
- setup.py: --context is checked before any download and again against the pack once it is built; the install tool refuses the same sizes.
- Terminal: the context step lists DeepSeek's own sizes (and the note beside them), a typed size is checked, `--context 512k` / `1m` and 524288 / 1048576
are accepted, plan and launch carry the note as a warning; Qwen's list and wording are unchanged.
- Engine: read, no change needed. strata-dsv41 takes --max-context as given (ds4_main.cpp) and sizes its buffers from it (ds4_model.cu); nothing clamps it
below the pack's limit, and the rope settings come from the pack. A test keeps it that way.
- docs/DEEPSEEK41.md: the cost (graphics memory per position taken from the expert cache, slow reading of very long prompts) and that 512K / 1M are untested.
Measured on a 5070 (IQ3_S) the cell floor cut decode after a 14K / 30K approximated prompt by 22 / 34 percent (drafts
accepted 100 -> 83 / 73 percent): a draft layer that reads only the exact tail loses its context. The approximated
chunks' draft cells are usable, so by default nothing about the draft layer changes: not the window, not the cells
built, not the attention. The floor machinery (MtpDrafter::set_cell_floor, window_ids_floor, KvaFloorRegistry) stays
behind STRATA_KVA_DRAFT_FLOOR=1 for research. kva_test now really covers window_start with floor 0, the registry and
wants_cell_floor.
The first default-on build set --mtp-window 2048 once per process whenever KVA was on, so short prompts (where KVA never
applies) lost draft context and decode speed. Now the draft layer keeps its window (32768 by default) and
MtpDrafter::set_cell_floor() names the first draft cell built from a real row: a KVA read raises it to the end of its
approximated chunks (they are neither built nor read), the draft attention starts at max(floor, pos - window)
(window_ids_floor, window_start.hpp), and KvaFloorRegistry gives every state a request continues from (live, checkpoint,
parked) the floor of its own tokens. A session KVA did not approximate has floor 0 and attends exactly as with
STRATA_KVA=0; with STRATA_KVA=0 or a process KVA is ruled out for the draft layer's code is 0.2.6's. The tail is 2048
whatever the window. Unit tests: window_start with floor 0 equals the plain window, the registry, wants_cell_floor.
- locale-gaps: the sentence added in 72c37ea2d (RTX 30/40/50, R9700, Strix Halo; "the line below says what to expect") had no key. fx77 in the eight fix catalogs.
- packaged-control: findControl was given an env without STRATA_COMPONENTS_DIR, so it fell back to the real per-user components folder and found the
updated Control installed on the PC (read only). The test now passes a scratch components folder.
Three rows under "Speed" in the native settings, in the existing row markup with the existing Select (On (recommended) / Off): Faster long prompts
(Qwen on CUDA or HIP; off writes STRATA_KVA=0), Faster answers on Radeon RX 9000 / AI PRO R9700 (Qwen on a gfx1200/gfx1201 card; off writes
STRATA_PCIE_FRAC_DEFAULT=old) and DeepSeek speed-ups (DeepSeek; off writes the six DS4_* = 0 of the contract). Each has a plain sentence saying what
turning it off does. A row is shown only where it applies, like the DeepSeek Speed row; the host decides from the model's file and the hardware report.
The switch is the model's file "env": off adds exactly its entries, on removes them, so a default launch is byte for byte what 0.2.6 started and the
Terminal, Control and the app reach the engine the same way (serve/server.py child_env). The host now shows and edits STRATA_* and DS4_* names (the
Strix Halo async entry is visible and kept); other names stay untouched. New strings are fx70..fx76 in the fix catalogs, translated in zh, ja, de, es,
ru, pt, ko. tests/speed-switches.test.mjs covers the default, each switch off and on again (same file), where each applies, and the catalogs.
strata_control/speed_switches.py holds the engine switch contract (Faster long prompts: STRATA_KVA=0; Radeon RX 9000 / R9700: STRATA_PCIE_FRAC_DEFAULT=old;
DeepSeek speed-ups: six DS4_* = 0), where each applies (Qwen on CUDA/HIP; Qwen on a gfx1200/gfx1201 card; DeepSeek) and the one-sentence effect text.
`strata launch` shows the environment names of the switches that apply as rows of the Advanced table (default "on", the sentence beside them); typing the
off value (`--set STRATA_KVA=0`) writes it into the run config's env, `off` takes it out again. A default launch carries none of them.
tests/terminal/test_speed_switches.py checks the names, the applicability, the config a launch writes, the rows, and that the app's table
(apps/host/server/native-settings.mjs) says the same.
The tail follows the draft layer: with KVA on and no --mtp-window given, the draft window becomes 2048 cells and the
fully computed tail 2112 (a given --mtp-window N sets the tail N + 64; a window of every cell, or STRATA_KVA_SUFFIX, is
handled and named). Layer splits, batch slots, --peer-device, pictures, the DFlash2 drafter, other model geometries and
the Metal build read exactly, with the existing "NOT applied: reason" line (quiet for continuation and short reads unless
STRATA_KVA=1). The reasons live in KvaConfig::decline and are unit tested (kva_test). Later chat turns continue from the
live session or a checkpoint and read their new tokens exactly over the cells the first read left; docs/KVA.md says so.
The slab rule (8 MB of q/o per device, 256 rows here) was set to keep the dense dequant's VRAM small; with a
1024-token chunk it dequantizes q_b/wo once per 256 rows - four times a chunk, 32 times an 8K prompt.
Measured on 4x R9700, 2.25 bpw, share 0.77, chunk 1024, 8K code prompt, ids 5705e1164a:
256-row rule 1204 / 1236 / 1250 ppts, decode 77.7 tok/s
chunk's 1024 1324 / 1353 / 1363 decode 78.1 (+9% prefill, decode unchanged)
The 1024 rows cost 268 MB a device (1% of the card). So high-share runs take min(chunk, 1024) rows and the
old rule stays elsewhere; DS4_ATTN_SLAB still overrides. The un-enved default verified at 1335 ppts /
77.8 tok/s with the ring at 253 MB.
With the 1024-token prompt chunk (the default where the cards hold most of the experts, abfe7d813) a big
staging ring is no longer the trade it was: every MB it takes is a tenth of an expert the decode keeps.
Measured on the same box, 8K code prompt + six 128-token decodes a load, ids 5705e1164a / 81d895366c:
2000 MB 1243 ppts / 73.2 tok/s
1000 MB 1287 / 77.5
500 MB 1314 / 78.5
256 MB 1324 / 78.4
128 MB 1338 / 78.3
So high-share runs take clamp(room/device * 0.008, 128, 512) MB (248 MB here) and the old deeper ring stays
for the cases it was measured for (share < 0.5, where the cards stream most of the experts). The un-enved
default verified at 1339 ppts / 78.1 tok/s, 42 s of summed GPU time in a 36 s pass.
prefill_chunk_default() returned 1024 only for a single-card run (own_default); a multi-card run whose cards
hold most of the experts fell through to 2048 (HIP skips the two 4096 branches) - and the rule's own datapoint
for 1024 was a 4 x R9700 run ('1024 370 tok/s, 8192 214'). Measured again here, 2.25 bpw, 4 cards, 8K code
prompt, --dspark auto, ids 5705e1164a, 4 prompts a load:
512 1110 / 1137 / 1142 ppts (median 1123)
1024 1229 / 1257 / 1270 (median 1243) <- this commit's default
2048 689 / 695 / 696 (median 692) <- what the engine picked before
8192 690 / 690 / 695 (median 690) <- what the 0.2.7 configs passed
Decode unchanged (73.2 against 73.2 tok/s medians, --dspark auto). So the owner's default 8K prompt goes from
692 to 1243 ppts (+80%) with no flag; --prefill-chunk still overrides.
DeepInfer's rule: verify a draft while the product of the confidences so far stays above the cut, instead of
only each draft's own floor. The 0.2.7 notes left it off ('not measured faster on code: H100 / owner PC'),
Project Maya runs 0.5 on a Radeon 8065S (+6.5% decode, the same tokens).
Measured on 4x R9700, DeepSeek 2.25 bpw, six 128-token decodes an arm in one load, same ids
(81d895366c): 71.7 -> 76.9 tok/s (medians) with STRATA_DS4_DSPARK_CUM=0.5, and a second pair 74.1/73.6;
the default with the rule on (no env) verified at 74.5 tok/s (6 decodes). CUDA keeps 0 (the rule was flat
there). STRATA_DS4_DSPARK_CUM still overrides either way.
The prompt path copies every streamed expert from the RAM copy into the pinned staging before its DMA (the
file's own note: 6.8 GB/s through the staging against 24-28 GB/s probes). Registering the copy lets it DMA
in place. Three interleaved pairs on 4x R9700, 2.25 bpw (share 0.72): the second 8K prompt of a process
12.0 / 11.9 / 11.9 s against 28.8 / 13.8 / 12.3 s unpinned; decode 35.42 / 35.50 / 34.9-36.0 against
35.43 / 35.45 / 34.9-36.0 tok/s, i.e. no cost; ids unchanged. Costs 23 s of load (registering a 50 GB
copy) - once a process, which for a server is the right trade. Auto only when the expert share is >= 0.5
(the CPU lane is then a minority of the decode, so the pinned pages' TLB cost cannot bite); DS4_COPY_PIN=0/1
forces it, and it stays off where the 12 GB cards' decode is CPU-lane-bound (the -15% measured there).
The RAM->staging memcpy runs on the IoQueue's threads; on a warm 8K prompt (stream stage 14.3 s, 12.8 GB a
device) they are busy 4.1 s, i.e. ~3 GB/s a device - a real share, and the thing to check in the 50-95 s
first-prompt runs (where the same counters are what to read first). DS4_PROF only.
The first prompt after a load is 2-9x slower than the second on the 4x R9700 2.25 bpw cell and it is not
the disk (0 MB read during it), the RAM store (removing it is worse), the dense GEMM shapes (q_b+rope
1.46 s) or the compute waiting (0 ms). The lead is the RAM->pinned-staging->DMA path (the file's own note:
6.8 GB/s through the staging against 24-28 GB/s probes); DS4_COPY_PIN=1 measured 47.9 -> 31.4 s on the
first prompt and 28.8 -> 12.0 s on the second, decode unchanged. This commit only adds the counters
(staging / DMA issue / slot wait) to the existing prompt stream line under DS4_PROF; no behaviour change.
No behaviour change on the 4x R9700 cell (126.8 GB of room / 4 = 31.7 GB a device, 8% -> 2000 MB, the value
measured in 76177ec34); on other card counts it is the per-device share that was meant.
The prompt path streams the experts the cards cannot hold. ring_mb_auto() gave 16 MB whenever the cards
held >= 1/1.3 of the experts, which is exactly this 2.25 bpw cell (126.8 GB of room, 156.3 GB of experts,
share 0.769): the stream then had no depth and the expert tiles waited 1.7 s inside a 5.2 s stage. A share
of the VRAM room (5%, 400-2000 MB) when any expert spills, 16 MB when they all fit (the 0.2.6 rule, kept
for the decode it was chosen for).
Measured, 4x R9700, 2.25 plain pack, 6 experts, --dspark off, 8192-token code prompt, ids 5705e1164a:
8K warm 632 -> 675-678 ppts (+6.8%, 5 prompts a run, 3 runs), agent decode 35.9 -> 35.3 tok/s (-1.7%,
4 runs). The profile of the warm pass: expert stage 5185 -> 4034 ms (kernels 3509 -> 2645 ms, the same
kernels now that the weights arrive in VRAM ahead of them, waits 1676 -> 1389 ms), everything else equal.
Also in this commit (opt-in, defaults byte-identical except the F16 activation path which is bit-identical):
- SetArgs::x16: gate/up take the layer's F16 copy of the activations (rows_f16 rounds exactly as the tile's
own smem store did) - half the activation bytes, 26% faster gate/up in the standalone bench
(box/expert_bench.cpp), +0.3% at the pass level (the stage is stream-bound).
- HIPK_CFG 6..11: tile variants for the expert GEMMs (64x128, 32x128, 128x256, 64x128/4 waves,
128/128+down 256, 64x256). 64x128 (cfg 6) is 21% faster than 128x128 in the bench and +0.9% in the pass.
- HIPK_DBG 1/2/3: switch the MMA / the decode off in the tiles (the breakdown in 09-marlin-kernel-diary.md).
- ds4_iq_hip.inc: a UTF-8 BOM on a source file removed.
It now ships on at 2.0 for a card of 13 GiB or less: +37.6% story / +35.2% code
decode over five interleaved pairs on an RTX 3060 12 GB, and it stays off on
bigger cards where it measured neutral to negative (l620 -1.9%/-0.5%, z440
-4.2%/+11.0%). It changes which experts run - the owner approved that trade
for 12 GB cards - and STRATA_ROUTE_PRIOR=0 turns it off.
The design was HIP-only: 80 resident blocks, a column slice of D each, two
grid barriers on an atomic counter, a barrier timeout so a card that cannot
hold all 80 blocks resident turns a hang into a refused start-up check. Ported
by giving grid_sync a CUDA branch (volatile generation counter + clock64() for
the timeout, scaled to SM clocks), guarding the gfx-arch refusal to HIP and
adding a masked-shuffle helper.
On the RTX PRO 5000 Blackwell 72 GB it passes its own check - '80 blocks on 110
multiprocessors, 0 token counts timed out, max relative difference 1.8e-07
against the default read' - and measurably does NOT win: story 201.4 -> 190.6
tok/s (-5.4%), code 245.6 -> 249.6 (+1.6%) over 3 interleaved pairs. That
matches the AMD note in the same file (an R9700 measured it neutral to 1%
slower), so it stays opt-in (STRATA_HC_PERSIST=1) and is not a default. The
port is kept as the starting point for a CUDA-tuned block mapping.
The router's top-10 is picked again with a logit bonus for the experts already
in VRAM (route_prior.cu); the weights stay the router's. Measured on an RTX
3060 12 GB, IQ3_XXS, 5 interleaved pairs of fresh engines: story 44.2 -> 60.9
tok/s (+37.6%), code 47.3 -> 64.0 (+35.2%); =1.0 gives +22.6% / +24.6%. Three
more pairs against the switch itself read +23.6% / +36.9%.
It changes which experts run, so it is a default only where the expert cache
cannot hold the model, and the owner approved that trade for 12 GB cards. It
is NOT a default on a bigger card: on an RTX PRO 5000 72 GB it measures
-1.9% / -0.5% and on 2x RTX A4000 -4.2% / +11.0% (3 pairs each); both were
re-run after this change and are unchanged (201.7 / 246.9 and 87.1 / 107.1
tok/s). STRATA_ROUTE_PRIOR=0 turns it off and any explicit value wins.
A draft for the owner: the one engine default that changed (the wide
hyper-connection up read on a 12.0 card, +1.9% story / +0.7% code decode on
the RTX PRO 5000, 5 pairs, bitwise), the four settings that are faster where
the PC supports them (PLE in RAM, the English draft vocabulary, --prefill-pipe
on a two-card split, q4_0 KV on 12 GB with its precision cost), and the list
of what was measured and rejected so it is not retried. No version number is
claimed: the owner decides what else 0.2.7 carries.
STRATA_GR_UP_WIDE (one warp a row, N/2 = 1,280 blocks instead of 160)
measured on the RTX PRO 5000 Blackwell 72 GB, IQ3_S, every expert in VRAM,
5 interleaved pairs of fresh engines: story 197.6 -> 201.3 tok/s (+1.9%),
code 243.5 -> 245.1 (+0.7%). A third run against the switch (default on vs
STRATA_GR_UP_WIDE=0) reads +1.7% / +0.8%. The produced token ids were
identical in all 6 pairs, and fused_gr_bench's per-T hashes are the plain
read's. The switch stays: =0 off, =1 on any card, STRATA_SM120_PERF=0 the
whole 12.0 set. Other cards are untouched.
setup writes DS4_ASYNC_T=4 into the config env for an integrated Radeon: the engine's mailbox path is sized for cards that hold
most of the experts and costs this box 31% on code (15.9-16.3 -> 20.1-21.3 tok/s), 16% on agent with identical accepted tokens,
9% on chat; the 2.25 set (no drafter) is unchanged. The same tokens come out. Discrete cards keep the engine default.
The engine's windowed decode uses the mailbox ("async") path for windows up to DS4_ASYNC_T (8 by default). That default was sized
for cards that hold most of the experts; on the 128 GB gfx1151 box the 6-row verify window is faster on the synchronous path.
Measured 2026-10-25 with the 2.00 bpw set and the shipped 0.1.51 engine (greedy, 256-token sets, three interleaved reps):
- code 15.87-16.33 -> 20.14-21.28 tok/s (+31%);
- agent, with identical accepted tokens a round (5.07): 15.93 -> 18.48 (+16%);
- chat 10.33 -> 11.30 (+9%);
- 2.25 code unchanged (11.2 -> 10.9): the drafter is unloaded there, so the extra syncs have nothing to hide.
The same tokens come out; only the scheduling changes, and the drafter's own single-token windows (T=1) stay on the async path.
dsv41_config_env(cards) writes DS4_ASYNC_T=4 into the generated config for an integrated Radeon only (a discrete card keeps the
engine default); STRATA_DSV41_ASYNC_T=n overrides the value and =8 restores the default. Tests:
tests/terminal/test_dsv41_async_t.py (5 cases). No engine rebuild; the signed dsv41 package is unchanged.
Docs: the STRIX_HALO and DEEPSEEK41 measured tables and the README row now read 19-21 tok/s code, 18 agent, 11 chat.
The trace origin was thread-local, and the prompt's later stages run on their
own thread, so their timestamps could not be compared with the first stage's.
One process-wide origin makes the hand-off visible.
Found with it on z440 (Q2_0, split 24, 16,383-token prompt, 8K chunks): stage
0 starts at 0 ms and 3,267 ms, stage 1 at 3,268 ms and 5,996 ms, read ends at
8,775 ms - the two stages do overlap chunk-for-chunk, and each takes ~2.7 s
per 8K chunk for its 24 layers. A split 30 run leaves stage 1 at 3.09 s a
chunk with 18 layers, so the per-chunk cost is not its layer count.
Same diagnostic as the request-step timestamps: with a layer split the chunk
line now says which stage is starting a chunk and when, so a prompt's stage
overlap can be read off the log. STRATA_TRACE only; no behaviour change.
Measured with it on z440 (Q2_0, split 24, 16,383-token prompt): serial reads
run stage 0's two 8,192-token chunks 3,332 ms apart and the whole read takes
8,788 ms; --prefill-pipe 1500 reads four 4,096-token chunks 1,582 ms apart and
the read takes 8,375 ms, so the stages overlap only partially at best.
The trace lines named the step a request reached but not how long it took to
get there, so a stall between two marks was invisible. Each line now carries
the time since the first traced step (thread-local). A diagnostic only: no
step, no order and no output changes without STRATA_TRACE set.
Found with it on z440 (Q2_0, layer split 24, 16,383-token prompt): the batched
read takes 8,758 ms while stage 0's own GPU timeline is 5,374 ms - stage 1 is
the slow one - and the expert-cache refill after the prompt costs a further
592 ms of the request.
Measured on the 128 GB box with the 2.00 bpw set: 4K prompt 78.7 -> 87.2 tok/s, 8K 108.9 -> 125.1, 32K 155.2 -> 176.9;
decode unchanged (16.53 -> 16.59). setup writes --prefill-chunk 8192 only for an integrated Radeon whose whole pool is at
least 64 GB; the escape hatch is STRATA_DSV41_NO_BIG_CHUNK=1. Also repairs UTF-8 damage in three docs from an earlier
PowerShell edit.
The engine's prompt-chunk rule returns 1,024 when a single card holds half or more of the experts; it was calibrated on
cards that hold most of them (4 x R9700). On the 128 GB Strix Halo (share 0.62, one shared pool) the bigger chunk wins:
measured with the 2.00 bpw set, engine 0.1.51, 2026-10-25: 4K prompt 78.7 -> 87.2 tok/s, 8K 108.9 -> 125.1, 32K 155.2 ->
176.9. Decode is unchanged (16.53 -> 16.59): the chunk buffers take ~5 GB from the GPU expert tier (81.9 -> 76.6 GB) and the
RAM store's hit rate rises (68.6% -> 76.8%) to compensate.
dsv41_prefill_chunk_args(cards, ram) writes --prefill-chunk 8192 for an integrated Radeon only, and only when the whole pool
(the RAM Windows sees plus the BIOS carve-out) is at least 64 GB; a discrete card or a smaller unified PC keeps the engine's
choice, and STRATA_DSV41_NO_BIG_CHUNK=1 turns it off. The chunk changes the prompt GEMMs' summation order, not the model: the
repo's chunk-size KL check reads 1024 0.1705 / 2048 0.1692 / 4096 0.1688 / 8192 0.1695. Test:
tests/terminal/test_dsv41_prefill_chunk.py (5 cases).
Also repairs UTF-8 damage in README.md, docs/STRIX_HALO.md and docs/MODELS.md from an earlier PowerShell edit (the middots,
em dashes and arrows had been double-encoded; no text changed). No engine rebuild is needed: the existing signed dsv41 package
accepts the flag.
The up projection of the hyper-connection read runs 160 blocks of 8 warps
(each warp takes 8 rows) on a 12.0 card, so 160 blocks of 192 SMs and ~6.7
warps an SM: the read is latency-bound at about 0.5 TB/s. This kernel takes
one row per warp and 2 columns x the 4 streams a block, so the grid is
N/2 = 1280 blocks (53 warps an SM). Every row's dot, xor tree, epilogue and
the cross-stream sum are gr_up_multi_kernel's, in its order, so the outputs
are bitwise the same; the token streams of both arms were identical in all
6 pairs measured.
Measured on l620 (RTX PRO 5000 72 GB, all experts in VRAM, IQ3_S, 3
interleaved pairs, fresh engine each): story 197.4 -> 201.3 tok/s (+1.9%),
code 243.0 -> 244.7 (+0.7%). Off by default (STRATA_GR_UP_WIDE unset); the
q8_1 tail of S26 needs the 16-column blocks, so the wide kernel is only used
when no q8_1 image is written.
The Windows AMD DeepSeek engine (feed member dsv41, windows-x86_64-hip-rdna3, gfx1151) ships with these
changes: the 0.2.6 copy-all rule is off on an integrated GPU (it copied 53 GB into RAM beside the GPU's 82 GB
in one 128 GB pool and an 8K prompt stalled; the in-place path is the 0.2.5 soak path), a Windows HIP package's
rocm/bin is on the engine's library path (setup, run config, app), the unified-memory fit plan counts the BIOS
carve-out, and --feed reaches the DeepSeek engine download. Docs, app text and tests included.
Measured on the Strix box (2.00 bpw, Faster, DSpark auto): 4K prompt 78.7 tok/s, 8K 108.9, code 17.6 tok/s,
chat 11.0. Clean-user end-to-end from a signed feed: install, download, pack, smoke, serve, chat, tool call,
agent round trip: 5/5.
strata launch --feed passed the feed to setup, but setup's parser had no --feed (getattr(a, 'feed', None)
was always None), so the flag was refused with 'Unrecognized arguments' and a local or non-default feed could
never serve the DeepSeek engine (found by the clean-user test on strix, 2026-10-25). It is now the same
option strata update has: a URL or a folder.
Found by the clean-user test on strix: 'strata launch --feed DIR' read the local feed for its own engine
check and then setup went to GitHub for the DeepSeek package, so a local (or any non-default) feed never
served the DeepSeek engine. setup's get_dsv41_engine/upgrade_dsv41_engine take the feed, install_dsv41
passes --feed, and runner.setup_args hands --feed to setup (a test in test_launch_dsv41).
The unified-memory plan counted only the RAM Windows sees (64 GB on the owner's box) and ignored the 64 GB
BIOS carve-out the GPU owns: 128 GB in one pool. uma_total_gb() adds the carve-out (or the GPU figure less
its shared part), so the plan now says about 109 GB of the 2.00 set's 132 GB stay in memory and 23 GB are read
from the SSD; the docs and the test fixtures use the 128 GB box. The 'measured' text says 128 GB unified too.
The Windows AMD DeepSeek engine package (feed member dsv41, windows-x86_64-hip-rdna3, arch gfx1151) is built from this
branch and installed as its own package; these are the code changes it needs.
- ds4_model.cu: on an integrated GPU the CPU lane's experts are no longer copied into RAM by the auto rule. The GPU's
expert tier and that copy are the same shared memory: on the 64 GB Strix Halo the GPU held 82 GB of experts, the rule
saw 62 GB 'free' and copied 53 GB, and an 8K prompt did not finish in 11 minutes. Found 2026-10-25 on strix; the
corrected build does the same prompt at 108.9 tok/s (4K 78.7), code answers 17.6 tok/s, chat 11.0. DS4_COPY_ALL=1
still forces the copy for a test.
- dsv41_engine.py: lib_dirs()/start_env() put the package's BUILD.json lib_dirs on PATH/LD_LIBRARY_PATH (a Windows HIP
package keeps hipBLAS/rocBLAS in rocm/bin, or a clean PC fails with 0xC0000135); the upgrade test-start uses it.
- setup.py: the dsv41 engine's library dirs come from the package (so the run config carries rocm/bin), the unsupported-card
text names Ryzen AI Max on Windows, and the manual package hint names the .tar.gz the release page actually has.
- runner.py: the same library dirs for the app/Terminal path.
- dsv41_fit.py: a unified-memory plan for the official IST-DAS sets (one pool, what stays in RAM, what is read from the SSD,
measured 14-16 tok/s on code, 10-11 on chat, 4K 78 / 8K 109), never a refusal; tools/strata_mcp.py uses it for the app
catalog's on-this-PC line.
- Docs: STRIX_HALO.md (DeepSeek section with speeds, driver 26.10.1, memory behaviour, limits), DEEPSEEK41.md, MODELS.md,
AI_REFERENCE.md, README.md, packaging/README.md (the Windows AMD package recipe).
- Tests: tests/terminal/test_dsv41_fit.py (7) and 4 new tests in test_dsv41_engine.py (lib dirs, feed selection for gfx1151).
Window 2's residual stream was window 1's stale tail: stream copies and mtl_rt's deferred command buffer had no
ordering at sync points (memcpy, stream sync). Now a full barrier over all streams and the context, and the context
takes a lock (helper threads flush too). engram_dequant_rows wrapped n*18/32*hd bytes but strides rows by hd: rows
past 192 of 288 read unmapped memory (inf scales, router picked INT_MAX). The CPU lane (ggml-cpu rows) was compiled
out, so every non-resident expert contributed zero. Debug: DS4_DBG_ALL (every layer), DS4_DBG_EMB, DS4_DBG_EG.
init_pre writes T*8 floats - 32 bytes for T=4 - and its wrapper wrapped the enclosing 16 384-byte page to bind them. That aliasing
MTLBuffer overlaps the arena pages that hold h, and with it present h read back as 'valid at both ends, NaN in the middle' with
nan_range starting at exactly 4 096 - one wrapped page of floats. The work is trivial, so it now runs on the CPU through host_view:
no wrap, no alias, no dispatch.
Result with the fix: h at layer 0 is clean (zeros=40960, ninf=0, max=0) instead of ninf=4096 max=7.6e+07. The output is still
0,0,0,... so embed's write still does not reach h - but the contamination that had been confusing every probe of that buffer is
gone, and the two remaining readings (host_view vs cudaMemcpy on h between embed and layer 0) can now be compared cleanly.
Fixes this session, all measured: the 1.32 GB all-zero embedding file (my converter read the sparse trial shards), my own
20000-dispatch flush cap (it blocked every prompt of 256+ tokens), and now the init_pre page alias.
Targets, unmet: decode 8.94 tok/s vs 100, prompt 4000 tokens ~27 tok/s vs 1000, and no valid token has ever been produced.
Measured this session, in order of how much they narrow the problem:
* the pack's embedding table is real, and the engine hands embed correct rows (rows_in: first=4759 4759 4759 4759)
* ds4_embed's kernel receives the right arguments (args probe: dim=5120 hc=4)
* one Dev, one h: DBG devh prints id=0 h=0x100011a8000, the same address layer() reads
* the memory model is sound: writing through host_view round-trips through cudaMemcpy at 4 KiB and at 64 MiB (42 -> 42)
* both dispatch routes work: a constant written via Arg::B reads back 3 3 3 3, via a device pointer passed by value 7 7 7 7
* the Metal embed path is load-bearing: breaking its pipeline changes the exit code 0 -> 3
What is still wrong: h is partly NaN at layer 0 (nan_range=4096..40703, max=7.6e+07) with valid data at both ends, and
wholly NaN from layer 1, so the logits never resolve and every run prints 0,0,0,... Nine hypotheses were tested and
disproved today: the zeroed table (a real bug, fixed, not the cause), two queues, host_view vs device pointer, memory
coherence, the binding, a dead Metal branch, a wrong dim, a short write, and two Dev records.
Targets, measured, unmet: decode 8.94 tok/s vs 100; prompt 4000 tokens ~27 tok/s vs 1000. The 51.8 and 176 tok/s reported
earlier were dispatch-ENCODING rates and are withdrawn here as they were in the notes.
Next: locate the real d.h allocation (several probes landed on paths the engine does not take) and read that buffer at
three points in one run - at allocation, after embed, and at layer 0.
embed writes h = 0x144f24000
layer 0 reads h = 0x100011a8000
Both report device 0, and forcing --gpus 0 does not change it, so it is not a tensor-parallel artefact: the engine holds two Dev
records with two h allocations, embed fills one and layer() reads the other, and the bounce in ds4_model.cu (if (l.dev != cur)) is
meant to reconcile them but does not on this path.
That single pair of addresses explains every symptom: layer 0's input buffer was never written (hence zeros=40960), the first block
then produces NaN, and the logits never resolve. It also retires the last four hypotheses: the embedding table (real), the embed
kernel (verified: writes as_type<float>(0x4759 << 16)), two-queue ordering (shared Context now), and stale host reads (the probe
flushes).
Correcting the record: the 51.8 tok/s decode and 176 tok/s prefill reported earlier were dispatch-ENCODING rates and are withdrawn.
The real measured rates are 8.76 tok/s decode and ~27 tok/s at 4000 tokens, and no valid token has ever been produced. Targets remain
100 tok/s and 1000 tok/s: both unmet.
Next: print cur, l.dev and the two h addresses inside layer() - whichever differs is the buffer the embedding never reached.
Measured this stretch:
* the pack's embedding table is real (embed_pinned = 3d6b 3e6e 3e65 3dd7)
* embed itself is correct: immediately after the call h[0..3] reads 55296, which is exactly as_type<float>(0x4759 << 16),
the BF16 embedding value converted the way both ds4_embed and embed_kernel convert it
* yet at the top of layer 0 h is all zeros (zeros=40960 max=0), and block 1 turns that zero into 100% NaN
Nothing between the two probes writes h except init_pre (which writes pre). The remaining suspect is which Dev's h the layer reads:
the engine has a cross-device bounce (cudaMemcpyAsync through a host staging buffer) that ds4_metal_flush() does not cover, and a
stale l.dev under 'tensor parallel 2 -> 1' would produce exactly this.
Three hypotheses about the embedding are now retired with evidence: the zeroed table (a real bug, fixed, did not change h), two-queue
ordering between init_pre/embed and the rest (they now share one Context, no change), and the kernel writing nowhere (it writes
correctly). Targets remain unmet: decode 8.76 tok/s vs 100, prompt 4000 ~27 tok/s vs 1000, and no valid token has been produced.
State of the port, honestly: the engine runs, 40+ kernels dispatch, the pack and the 110 GB Engram are real, and it has NEVER
produced a valid token. Both targets are unmet: decode 8.76 tok/s (target 100), prompt 4000 tokens ~27 tok/s (target 1000).
What this commit adds:
* the drafter path: ds4_mx_gu/ds4_mx_dn (MXFP4 prompt gate_up/down, one runtime-ne kernel), ds4_markov_add, ds4_confidence, with
hosts and guards. --dspark on now runs end to end, but the confidence is NaN (its taps come from the trunk, which is NaN), so no
drafts are offered and there is no drafter speedup.
* token-boundary commits: seven host-read sites in ds4_model.cu flush the deferred command buffer before reading a device result.
This is what exposed that the earlier 51.8 / 176 tok/s figures were dispatch-ENCODING rates, not GPU rates: with the work inside
the timed window the real numbers are 8.76 tok/s decode and ~27 tok/s at 4K. Those earlier numbers are withdrawn.
* embed/init_pre now dispatch through the shared Context (they had their own Metal queue, so no ordering against the copies that
feed them) - measured, and it did NOT fix the blocker.
* the embedding blocker, measured at both ends: the pack's table is real (embed_pinned reads 3d6b 3e6e 3e65 3dd7), ds4_embed is
dispatched at ds4_model.cu:7490 and returns, and h[0..3] is 0 0 0 0 immediately afterwards, so the kernel writes nothing into the
buffer the engine reads. Behind it, block 1 turns a zero input into 100% NaN (a 0/0 in the first norm or router).
* env-gated debug probes (DS4_DBG_H, DS4_DBG_EMB, DS4_DBG_L) left in for the next session; they are inert without the variables.
Known edit hazard, learned three times: patching by a generic statement pattern lands in the wrong function. Patch by unique
context or by function name.
Removed the per-dispatch histogram from encode() (it cost a mutex and a string key per dispatch; measurement showed no gain, so the
~10 us a dispatch is genuine encode + engine host work). Reverted the two experiments that did not hold up: the q8_quant dedupe
(the repeated calls all differ - 0 dispatches saved) and skipping host_view() in cpu_addr (breaks the engine, 15 dispatches).
Prefill chunk sweep, 4000 tokens, pack-2.25-real: --prefill-chunk 16 -> 165.90 tok/s, 32 -> 175.89, 64 -> 176.26. The engine runs
16-token windows regardless, so the curve flattens: 176 tok/s is the 4K number and the target is 1000.
Measured state: decode 51.8 tok/s (target 100), prompt 4K 176.3 tok/s (target 1000). Neither met; the remaining levers are kernel
fusion for decode and a key-tiled attention kernel for prefill, both rewrites.
Two measurements tonight that matter:
1. Every prompt of 256 tokens or more died at exactly '20073 dispatches, 2 command buffers'. That is the deferred command buffer's 20,000
dispatch cap - my own constant from when I introduced batching - flushing mid-token. Raising it to 2,000,000 makes the long prompts run:
64 tokens 535.8 tok/s
128 tokens 556.3 tok/s
256 tokens 542.6 tok/s (31,508 dispatches, 1 command buffer)
1024 tokens 451.0 tok/s (126,020 dispatches, 1 command buffer)
4000 tokens 170.9 tok/s (492,254 dispatches, 1 command buffer)
The curve falls with length, so the 4K figure is attention scaling, not fixed overhead.
2. The per-kernel dispatch histogram now works (Pipeline carries its name; the earlier '<unnamed>' was a lookup bug):
ds4_q8_quant 4122 (36%)
ds4_iq_mv_k 1218
ds4_finish 1038
ds4_hc_pre_norm 486 / hc_split 480 / hc_mix 480 / hc_post_v 480
ds4_gemv_f32 474
~1,890 dispatches a token at 8.8 us each = the 16.7 ms a token, so decode speed is 1/(launches x 8.8us). 100 tok/s needs
~1,130 launches: dedupe q8_quant per layer (-380) and fuse the four hc_* passes (-320) gets there arithmetically.
Decode 51.8 tok/s (target 100, measured on pack-2.25-real), prompt 4K 170.9 tok/s (target 1000). Neither target met.
Measured (real 110 GB Engram pack, engine's own lines):
prompt 64 tokens -> 535.76 tok/s
prompt 128 tokens -> 556.31 tok/s
decode -> 48.3-50.0 tok/s
prompt 256+ -> aborts, router sentinel 2147483647 (no pps figure)
Targets are 100 tok/s decode and 1000 tok/s prompt: both roughly 2x away, neither met.
Added:
* sparse_attn_packed<G> (G=8/4/2/1) and the BF16 row variant, with the host wrapper that mirrors AttnRows and picks G - the 4K
abort at ds4_kernels.cu:2161 is gone.
* ds4_iq_mv widened to 16 accumulators; host chunks any job with ncol > 16 into 16-column jobs (this prompt needs ncol=96), and
Q5_K/Q6_K dense matrices get ds4_iq_mv_k instead of being fed to the IQ2-only decoder.
* route, gather_rows, combine kernels.
* Wrap extents made exact across 7 wrappers (moe_sort, swiglu, hc_split scale/base, sparse_attn rows, unpack_sel, engram_dequant);
the old 'bytes + 65536' form could overrun an arena chunk's edge.
Known bug, not fixed: for prompts of 256 tokens or more the attention SINK logits read NaN (sink[57] at the first failing window),
which poisons the softmax denominator, then q, then the router. 64 and 128 tokens are clean, so it looks like a path switch at the
256 boundary (few = T <= kAttnScratchT in sparse_attn) or an overrun into the sink buffer. That is what stands between the current
556 tok/s and a 4K figure.
Context::run() committed a command buffer and blocked in waitUntilCompleted for EVERY dispatch. At ~60 dispatches a layer over 40
layers that is 3.3 us x 2400 plus a 133 us commit+wait each, and it showed up as exactly the 330 ms a token the engine reported.
Now a single dispatch is appended to a persistent command buffer and returned from without waiting; the buffer is committed when it
fills (20000 dispatches) or when the engine synchronises (strata::gpu's stream/device synchronise call ds4_metal_flush()).
Measured on the same fast harness (--vram-experts 0.05 --cpu-threads 2, ids 1,2, n=6):
before: prompt 5.43 tok/s, decode 3.00 tok/s, layers 330.1 ms/token
after: prompt 77.0 tok/s, decode 48.7 tok/s, layers 19.1 ms/token
with the experts resident in VRAM instead of streamed: decode 42.6 tok/s (layers 19.5 ms) - so residency is no longer the limit.
The remaining gap to 100 tok/s is the 1-token IQ2 matvec itself: with ncol == 1 the eight accumulators are idle and every weight is
decoded for a single dot, which is the 87.7 GB/s case measured earlier (versus 437 GB/s at an 8-token window). Fixing that is kernel
work, not runtime work.
Official harness completes with no abort: /tmp/r.out = '0,0,0,0,', zero 'no Metal dispatch', zero metal errors. Prompt 4 tokens in
5112 ms, decode 4 tokens in 18876 ms. The ids look degenerate and numerical parity is NOT established - the Engram in this pack is
zeros, and a wrong kernel would produce the same output - so what is proven is that the engine walks the entire path and emits ids,
nothing more.
Kernels added (each verified by the engine's next abort naming a later or different launch):
prompt expert GEMM: gate_up_kernel<T,NE>, down_kernel<T,NE> (ds4_iq.metal, with IQ3_XXS type 18 joining the decoders - this
pack mixes gate types 16/17 with down types 18/22)
mx_combine, engram_dequant (fmt 2), engram_gate, bf16_mv (engram wkv), compress, index_score, topk (radix with simd_ballot),
finish_kernel<true> (KV pack - the call site had no guard at all), finish rounding branch, gemv_f16_rows, block_max, hc_mean,
iota_rows, sampler_greedy (ds4_init.metal, 1521 lines)
mx_wait skipped under STRATA_METAL, like the pre-existing wait_seq no-op
Also fixed a real pre-existing bug: the 'payload + 65536 from the page base' wrap convention walked off the mapping when a small
buffer sat at an arena chunk's edge (flaky 'mtl: wrap 81920/98304 failed', exit 4, 2 runs in 4). ds4_finish.cpp now wraps exactly
the pages a buffer spans.
Not ported: sampler_kernel<0> (the sampled path; the harness is greedy), engram fmt 0/1 (this pack is fmt 2), bf16_mv's non-vector
variant. Expert types beyond 16/17/18/22 make the IQ kernels return false, so the CUDA launch names the line instead of a silent pass.
ds4_bounds_scan, ds4_moe_sort_count and ds4_moe_sort_fill (ds4_kernels.cu:1938-1943) with the host wrapper that launches them
in order on one command buffer (serial dispatch, so call order is execution order). kPicks = 8: idx is [T][8], q -> (q/8,
q%8); one threadgroup a virtual slot, a block-wide prefix over 8 warps, row_tok/pos written only by the filling lanes.
One Metal fact paid for: simd_ballot returns a simd_vote that does not cast to an integer here. The counts come instead from
simd_prefix_exclusive_sum (a lane's offset inside its simdgroup) and simd_sum (the group's total) - which is a cleaner
formulation than CUDA's ballot+popc anyway. AOT: 20,160-byte AIR, rc=0.
Correction worth recording: the engine's current gate is NOT moe_sort. It still stops at ds4_kernels.cu:1915, i.e. the Engram
row decoder (engram_dequant_tok) - I attributed :1915 to moe_sort two briefs ago and that was wrong. The routing trio is ported
and wired but not yet reached; its turn comes after the Engram decode.
ds4_hc_split is the SCALAR path of ds4_kernels.cu:268: lane 0 of a 32-thread group does the whole token in a local 8x8 matrix -
sigmoid pre/post, row softmax with max shift, +eps, column normalise, then iters-1 alternating row/column normalisations. That is
the same mathematics as the CUDA warp-lane version for hc == 4, without the shuffles: correctness first, the lane-parallel form can
follow if it ever shows in a profile.
The first attempt segfaulted (rc 139) and the cause is worth remembering: scale and base were memcpy'd as if they were plain host
pointers, but every operand the engine hands a kernel is a device buffer - reach it through strata::gpu::host_view() and pass it as
a wrapped Buffer with its offset. With that fixed the engine advanced from :1908 to :1915.
Queue, nine items, each verified by the engine advancing: init_pre, embed, cublasSgemm, finish, hc_post, hc_mix_scale, hc_mix,
hc_split -> next is engram_dequant_tok (:1915), the Engram row decoder whose maths was verified standalone earlier.
Three more kernels, each confirmed the only way that matters: the engine moved past the launch line it had been stopping at.
* hc_post_inplace (ds4_kernels.cu:1677) - h[t][k][e] = post[t][k]*y[t][e] + sum_j comb[t][j][k]*h_old[t][j][e]
* hc_mix_scale (:1692) - mixes[t][i] *= rsqrt(sum(h[t])^2 / hcdim + eps)
* hc_mix (:1901) - mixes[t][m] = (h[t] . fn[m]) * rsqrt(mean(h[t]^2) + eps)
all in ds4_init.metal with host glue in ds4_finish.cpp (which is now the home of the small kernels: it owns the Context, the
pipeline cache and the wrap/offset helpers). Queue so far, seven items, every one verified by the engine advancing:
init_pre, embed, cublasSgemm, finish, hc_post, hc_mix_scale, hc_mix -> next is hc_split (:1908).
hc_split is NOT a transcription: its softmax/sinkhorn is expressed in warp-lane geometry (shfl_xor masks 1,2 for rows and 4,8
across columns over lanes 0..15, for hc == 4) and it loops 'iters' times. A wrong lane mapping yields a plausible but skewed
combination matrix, so it needs a parity test against the CUDA version on synthetic mixes before it is trusted.
ds4_finish.cpp binds the engine's x as a wrapped buffer with an offset, passes Ds4Finish as a constant struct whose pointer fields
are the engine's gpuAddresses, and dispatches one launch with grid = n_vec (the kernel's blockIdx.x is the threadgroup index).
It declines the mrope (rd > 0) and rounding (quant != 0) branches and the KV pack, so those still abort with the launch line
rather than being skipped.
Verified the only way that matters: the engine no longer stops at :1894, it stops at ds4_kernels.cu:1921 (hc_post_inplace_kernel).
Queue so far, each step confirmed by the engine advancing: init_pre, embed, cublasSgemm, finish.
Two plumbing lessons: ds4_finish.cpp had to be added to STRATA_DS4_SOURCES (strata_ds4 does not use the engine-source glob), and
scp with a relative remote path put it nowhere - absolute paths from now on.
ds4_finish (ds4_kernels.cu:1894) with its Ds4Finish struct and the block reduction: warp_sum over the 256-thread group
(simd_shuffle_xor 16,8,4,2,1 into red[8], barrier, second warp_sum, barrier), the RMS path (inv = rsqrt(tot/len + eps), then
r *= inv * w[e]) and the store (BF16 round to nearest even into dst, or back into x).
AOT gate: xcrun metal -c -> 10,032-byte AIR, rc=0. Two Metal constraints cost a round each and are worth remembering:
'long long' is not supported (use 'long'), and a kernel parameter must be a buffer/constant reference, not a plain value.
Not wired yet: the host side still has to bind the Ds4Finish struct (pointers as 64-bit gpuAddresses, strides as long), and the
mrope (rd > 0) and rounding (quant != 0) branches are not written - the host wrapper will decline those until they are, so the
engine names the line rather than skipping a step.
ds4_model.cu:2737 is the engine's first dense call: cublasSgemm(OP_T, OP_N, n_mix, T, hcd, &one, fn, hcd, h, hcd, &zero,
mixes, 32) - mixes[t][m] = h[t] . fn[m], both operands F32, alpha 1, beta 0. ds4_sgemm_tn (one thread an output element) plus
metal_host::sgemm_tn wrap the engine's buffers and dispatch it; cublas_v2.h calls it for exactly that case and keeps aborting
for everything else, so an unported shape fails loudly rather than computing something wrong.
With that the engine passes the dense product and names its next need: finish_kernel<false> (the plain variant - RMS norm,
mrope, optional in-place rounding, no KV pack) at ds4_kernels.cu:1898.
Queue so far, each step verified by the engine advancing: init_pre, embed, cublasSgemm, then finish_kernel<false>.
The DeepSeek engine now loads a real 48,786-tensor pack on the Mac, plans the staging (102.26 GB of experts in unified memory,
5,604 on the CPU lane), and executes until it needs a kernel that has no dispatch. Two are ported:
* init_pre (ds4_kernels.cu:1865) - a three-line initialiser
* embed (ds4_kernels.cu:1873) - the BF16 gather with hc copies
each as MSL in metal/src/engine/msl/ds4_init.metal plus host glue in metal/src/engine/ds4_host.cpp (wrap the engine's own device
buffer through strata::gpu::host_view + mtl::Context::wrap, then Dispatch), wired into the shared call site under
#if defined(STRATA_METAL). The engine advanced past both and now asks for finish_kernel at :1896.
Two infrastructure fixes were needed and are in here:
* runtime.cpp: StreamImpl's encoder/command-buffer state is touched from the loader and the compute path, so two encoders could be
open on one command buffer and Metal asserted ('A command encoder is already encoding'). A std::recursive_mutex around
encoder()/flush()/encode_copy()/encode_fill()/encode_dispatch() fixes it - recursive because encode_copy calls flush (a plain
mutex deadlocked, which cost one round).
* ds4_host.cpp compiles its MSL relative to the context's kernel_dir (metal/kernels), so the engine's own kernels are
../src/engine/msl/<file>.
Known: this build has STRATA_NATIVE_EXPERTS=OFF, so the engine routes all experts to VRAM ('this build has no ggml-cpu'); the
CPU-lane path is untested on the Mac.
metal/src/engine/ds4_ctx.cpp owns one mtl::Context, allocates through Context::alloc and remembers gpuAddress -> Buffer, so the
engine's device memory can live in the same residency set as the dispatches. Off unless STRATA_DS4_METAL_CTX=1, which keeps the
current behaviour and makes the two paths comparable the moment a pack exists. Build rc=0.
It exists because strata::gpu (what metal_compat's cudaMalloc maps to) is a CUDA-shaped facade with its own device state: a dsv41
launch through mtl::Context would otherwise read a foreign residency set. Flipping cudaMalloc/cudaFree to this under the env var
is the two-line change, and it is deliberately left for a session that can test it against a loaded pack.
metal/src/engine/ds4_host.cpp compiles ds4_quant.metal once through mtl::Context::compile, wraps the engine's buffers with
cx.wrap(strata::gpu::host_view(...)), builds a Dispatch (args = buffer, inline ldx, q, d; grid = n/32; tg = 32) and runs it with
cx.run(). Build rc=0 - the port's integration path is now code, not a plan.
It also exposed the last architectural question, which the comments record: the Qwen engine allocates and dispatches entirely
inside mtl::Context (cx.alloc / cx.wrap / cx.run) and uses strata::gpu only for a purge helper, so a dsv41 kernel must be given
the gpuAddress of a Buffer allocated by the SAME Context that encodes its dispatch. Until metal_compat's allocations go through
that Context, the wrapper compiles but would dispatch into a foreign residency set - the plan for that is written up in
RIGS/m1002.md rather than guessed at here.
metal/tests/dsv41/iq2s_t8_parity.mm: block 82 B (f16 d, 32 index bytes, 32 sign bytes, 8 qh, 8 scale), grid index assembled
from qs[4ib+l] plus two bits of qh[ib] (<< (8-2l) & 0x300), signs read at qs+32 in the block, scale alternating db[l/2] -
transcribed from ggml's dequantize_row_iq2_s. Verified: 65,536 rows x 5,120 k x 8-token window, rel L2 2.86e-07, PASS.
Rate 515.71 GB/s effective (64.46 GB/s of raw weight traffic) - the fastest of the three expert formats and 89 % of the
579 GB/s streaming ceiling. With IQ2_XXS (437.5) and IQ2_XS (467.8) the 89 GB of IQ2 weight in the 2.25bpw set now has
verified Metal kernels, which is 165-195 tok/s of headroom on expert traffic alone.
metal/tests/dsv41/iq2xs_t8_parity.mm: block 74 B (f16 d, 32 uint16 qs, 8 scale bytes), grid = iq2xs_grid[qs & 511],
signs = ksigns_iq2xs[qs >> 9], two scales a group (sc & 0xf, sc >> 4) - transcribed from ggml's dequantize_row_iq2_xs and
verified against the CPU reference: rel L2 2.83e-07 at 24 MB and 2.88e-07 at 97 MB, PASS.
Rate: 117.4 GB/s effective on 24 MB (launch-dominated) and 467.8 GB/s effective on 97 MB, which is in line with IQ2_XXS's
437.5. IQ2_XS is 24 GB of the 2.25bpw set and IQ2_XXS 49 GB, so the two largest expert formats are now ported.
metal/tests/dsv41/iq2xxs_t8_parity.mm decodes the weights ONCE per 8-token window and applies them to eight accumulators, which is
how the engine runs decode (DSpark's draft/verify rounds are 4-5 tokens, the prompt window is 8). Same tables, same decode as
the verified one-token kernel, and the reference checks it: relative L2 2.91e-07 on sampled rows, PASS.
The number that matters: 437.52 GB/s effective (54.69 GB/s of raw weight traffic) against 87.71 GB/s for the warp-per-row form.
At 2.33 GB of expert reads a token that is ~5.3 ms of expert time, i.e. ~188 tok/s of headroom - the first measurement that says
the 50 tok/s target is reachable, and it says the expert path must be the T-window form from the start, as the repo's moe_mm is.
metal/tests/dsv41/iq2xxs_gemv_parity.mm decodes IQ2_XXS exactly as ggml does (block 66 B, db = d*(0.5 + (aux32[1] >> 28))*0.25,
grid = iq2xxs_grid[aux8[l]], signs = ksigns_iq2xs[(aux32[1] >> 7l) & 127]) and dots it with the activations. Measured against a
CPU reference that uses the same tables: 65,536 rows x 5,120 k, relative L2 2.88e-07, PASS, 87.71 GB/s.
Two findings that shape the expert path:
* the group headers advance 4 bytes and read 8, so an 8-byte uint2 load is misaligned on odd groups. Both vector-load variants
failed (L2 1.38 and 0.985) and were slower; byte assembly is the correct access pattern, as in CUDA's own decoder.
* 87.7 GB/s is 4.8x slower than the BF16 GEMV's 425 GB/s for the same traffic, because the grid decode costs per byte. At 2.33 GB
of expert reads a token that caps decode near 38 tok/s - below the 50 target - so the expert path must be tiled (one weight read
feeding several tokens, as the repo's moe_mm does) rather than one row per warp.
That is a measured design constraint now, not an assumption.
metal/src/engine/msl/ggml_grids.h carries kmask_iq2xs[8], ksigns_iq2xs[128], iq2xxs_grid[256], iq2xs_grid[512], iq2s_grid[1024],
iq3xxs_grid[256] and iq3s_grid[512] as MSL constant arrays. metal/tools/gen_grids.sh produces them from
third_party/ggml/ggml-common.h (3cf03257f, GGML_TABLE_BEGIN/END blocks), so the Metal expert kernels use bit-identical tables
to CUDA instead of a hand copy - the same discipline that caught the half-rounded-scale error in the Q8 kernel.
Every expert format in the 2.25bpw set (IQ2_XXS 49 GB, IQ2_XS 24 GB, IQ2_S 16 GB, IQ3_XXS) is grid-based, so nothing in the
expert path could be written without these.
metal/tests/dsv41/q8_gemv_parity.mm: Q8_0 blocks (32 int8 + one f16 scale) times F32 activations, one warp a row, lane-strided
blocks. 16,384 rows x 5,120 k, relative L2 8.11e-08, PASS, 173.52 GB/s and 326.62 GFLOP/s (0.514 ms for 89 MB).
The first run measured 1.85e-04 and looked like a kernel bug; it was the CPU reference using the exact float scale while the
kernel reads the half-rounded scale from the file. Matching the rounding took it to 8e-08 - the same class of drift that would
quietly cost quality if the port did not reproduce the file's arithmetic exactly.
Three kernels are now verified with reproducible parity tests: the Q8 activation quantiser, the BF16 GEMV and this one.
metal/tests/dsv41/bf16_mv_parity.mm compiles the MSL version of the BF16 matrix-vector product (ds4_gguf.cu:1043's
mx_mv_kernel, single job: one warp a row, lane-strided k, warp reduction), dispatches it on the GPU and compares against a
double-precision reference: 2048 rows x 5120 k, relative L2 1.80e-07, PASS.
It also prints throughput: 83.19 GB/s and 83.2 GFLOP/s, 0.252 ms for 21 MB of BF16 weights. That is a naive kernel (no
threadgroup tiling) against a GPU that streams 579 GB/s in rt_bench, so 7x of headroom remains - and the number matters:
2.33 GB of expert reads a token at 83 GB/s caps decode near 36 tok/s, which is exactly why the expert path has to use the
tiled multi-token kernels (the repo's gemv/moe_mm family) rather than a warp-per-row matvec.
A second measurement lesson: per-row relative error of ~1e-3 on a 5120-term float dot is expected cancellation, not a bug;
the standard measure is relative L2 over the result, which is what both parity tests now use.
metal/src/engine/msl/ds4_quant.metal carries the Q8 quantiser that metal/tests/dsv41/q8_quant_parity.mm proved bit-exact (4096
blocks, 131,072 values, 0 differences). It is the first dsv41 kernel in the engine's own MSL directory, next to the Qwen
engine's mtp/ple kernels, so the runtime loader will find it by name when the launch is wired.
The comment records the driver quirk that cost the debugging: threadgroup_position_in_grid arrived as 0 in every threadgroup
on this Mac, so the block index comes from thread_position_in_grid / 32.
metal/tests/dsv41/q8_quant_parity.mm compiles the MSL version of q8_quant_kernel (ds4_gguf.cu:201) at run time, dispatches it
on the GPU and compares every int8 against the same arithmetic on the CPU. 4096 blocks, 131,072 values: 0 differ, worst
scale error 4.6e-04 (half rounding). PASS.
Two bugs found on the way, both in the test harness rather than the kernel: indexing with thread_position_in_grid where the
threadgroup id was meant, and passing ldx = 1 where the stride between blocks of 32 values is 32 (which made block b read a
window starting at b instead of 32b - the giveaway was block 1 reproducing block 0's values shifted by one).
This is the pattern every remaining kernel follows: an MSL function, a pipeline from it, the args as buffers, one dispatch,
and a CPU reference in the same program.
strata_ds4_launch is a macro that passes __FILE__ and __LINE__ to strata_ds4_launch_at, so the abort message reads
'src/dsv41/ds4_iq.cu:1234' rather than a generic line. Each run of the engine is then an exact work item instead of a guess
about which of the 146 launches is next, and the message is machine-readable from a log. Build rc=0, strata_ds4 = 1,412,984 B.
Two things made this possible:
* the launch syntax. 146 kernel launches (119 in the sources, 27 in the vendor tree) are now
strata_ds4_launch(kernel, grid, block, shared, stream, args...), a call a C++ compiler can parse, with a macro that forces
the global name past argument-dependent lookup and a body that stops the run naming the kernel until a Metal dispatch exists.
Every source then compiles as C++ through include/metal_compat - .cu files included, since the Metal project has no CUDA
language and was silently skipping them.
* the last CUDA names the kernel files needed: __grid_constant__, cudaDataType, __umulhi/__mulhi, the bitwise atomics,
__halves2half2 / __half2half2 / __low2half / __high2half / __low2float / __high2float / __hadd2 / __hmul2 / __hfma2 /
__hneg2, the occupancy query and the stream-capture status.
The EXL3 legacy path (v1.1.8's kernels, NVIDIA before sm_80) is defined in metal/src/engine/exl3_legacy_stub.cpp: four of its
seven entry points abort with their name, three answer as the declarations require. The 2.25bpw text model does not use EXL3.
Result: build rc=0, 0 undefined symbols, strata_ds4 = 1,410,552 bytes, Mach-O arm64, and it runs (its own usage text). Next
is the run-time loop: a pack reports the first kernel it needs, implement it in MSL, repeat.
The DeepSeek host sources now build inside the Metal project as their own executable, with the .cu files compiled as C++ through
include/metal_compat. ninja reaches step 23/35 and ds4_model.cu (8,232 lines, host only) builds; the first failure is a <<<>>>
launch in ds4_gguf.cu, which is the port's real boundary.
The measured size of the remaining work: 101 kernel launches to convert to Metal dispatches and 78 kernel bodies to write in MSL
(text only; the vision tower adds 21 and is optional). ds4_model.cu has zero of both.
Two gotchas from building it: run_locked.sh returns exit code 69 for builds that are fine (run ninja directly), and the Xcode
licence line on stderr is noise - the toolchain compiles without accepting it. Run with -j8 directly, the host sources compile in
about a minute.
A full-list compile (-ferror-limit=0) found 350 errors behind the 20-error display cap, and they collapsed into twelve missing
families. With dim3, __threadfence_system, __funnelshift_r/l, rsqrtf, __expf/__logf/__sinf/__cosf, sincosf, __ldg, __ldcv,
__float2int_* and __shfl_up_sync in cuda_device_builtins.h, plus the C math names in scope, the counts are
ds4_iq 52, ds4_gguf 42, ds4_kernels 132, ds4_vision 27 (was 66/67/185/32) AND the distinct-names list is now EMPTY: what is
left is not a missing name but device construct - the <<<grid, block>>> launch syntax and shared-memory declarations that a
C++ compiler cannot parse as written.
That is the boundary of the host-compile audit, which was its purpose: the API surface is complete, and the remaining work is
the MSL implementation itself.
Twenty rounds of 'compile ds4_iq.cu as C++ and fix the first name it cannot find' produced the complete list, and it is small:
__byte_perm and the bit/int intrinsics, the SIMD video instructions with their exact per-lane semantics (__vsub4,
__vsubss4, __vadd4, __vaddss4, __vcmpeq4, __vcmpgtu4, __dp4a), the rn float helpers, integer min/max, __align__, the
vector types beyond float2/float4, the CUDA device built-ins for the host compile (threadIdx/blockIdx/__syncthreads/__shfl_*
- labelled as host-only, nothing there is a kernel), and wmma in CUDA 12's tag style whose every call aborts naming the
simdgroup-matrix equivalent.
ds4_model.cu stays at 0 errors. The four kernel sources are at the compiler's 20-error display cap; what is left behind it
is mma_sync's signature and single scalars (__float2int_rn), not structure. Next is the MSL work itself.
ds4_numerics.cuh names __NV_E4M3/__NV_SATFINITE and __half_raw, ds4_q6k.cuh names __fmul_rn/__ushort_as_half, and both kernel
files include mma.h: cuda_bf16.h now carries the fp8 enums and includes cuda_fp16.h (for __half_raw), cuda_fp16.h carries the
five rn helpers and __ushort_as_half, and mma.h says what it is (the wmma surface does not exist on Metal; the token path
does not use it, and an instantiation fails loudly).
State after this: ds4_model.cu 0 errors; the four kernel files are at the compiler's 20-error limit with a shorter list each -
the remaining class is the device code itself (block decoders, shuffles, shared memory), which is the MSL work.
The kernel sources' single remaining error was cuda_fp16.h not existing; with it, vector_types.h and cuda_fp8.h in place they
get past the wall and show their real errors - ds4_iq 7, ds4_gguf 7, ds4_kernels 20, ds4_vision 20 - which is now the work
queue for the Metal port. ds4_model.cu stays at 0.
Both copies (this branch and the box) carry the box's version of cuda_fp16.h; an earlier local edit had failed on a path and
this commit syncs them.
The last layer of the compat surface: the CUDA function-space qualifiers (__host__/__device__/__global__/__shared__/
__constant__/__forceinline__/__launch_bounds__), cpu_set_t's CPU_ZERO/CPU_SET for the macOS branch of ds4_os.hpp,
CUBLAS_GEMM_DEFAULT_TENSOR_OP, a cudaMemcpy2D that reports failure rather than copying nothing, and a cublasGemmEx that takes
any argument list (the engine passes cuBLAS's full 19-argument spelling) and aborts naming kernels/dense_mm.metal.
With that, src/dsv41/ds4_model.cu - 8,232 lines, host only - compiles against metal/include/metal_compat on the M5 Max. The
four kernel sources are one error each, and the kernel work itself (MSL bodies for 27 + 76 + 33 device functions, plus the
dense GEMM and a pitched copy at the runtime) is what remains before a first token.
The include had landed between '#ifndef NOMINMAX' and '#include <windows.h>', so on a Mac it was never reached - that is why
the audit still showed every Linux name undeclared. Moved after '#pragma once': ds4_model.cu goes from 20 to 15 errors.
The 15 left are four CPU_ZERO/CPU_SET uses (macOS has no cpu_set_t macros) and the __host__/__device__ qualifiers that
cuda_runtime.h does not define yet; both are a few lines and are the next step, then the Linux surface of the engine's
storage helpers is done.
ds4_os_apple.h maps O_DIRECT (-> 0, F_NOCACHE does the work), posix_fadvise (-> 0), mincore (-> ENOSYS, 'unknown' is the
answer the header's contract already allows), MADV_HUGEPAGE and SYS_mbind (no-ops) and the CPU-affinity calls (no-ops on a
kernel that schedules on its own), and ds4_os.hpp includes it at the top under __APPLE__.
State: included, but ds4_model.cu still reports the same 20 errors at the same uses, so something in the include order is
not taking effect yet - the next session's first job is one -E on ds4_os.hpp to see which definitions win. Not claiming it
works.
Compiling src/dsv41/*.cu against metal/include/metal_compat as C++ is how the DeepSeek-on-Metal port is scoped, and it is
much smaller than assumed: ds4_model.cu is 8,232 lines of *host* code with zero device markers, and the six non-kernel
sources already compile. What was missing at the compat layer:
* cuda_bf16.h - __nv_bfloat16/__nv_bfloat162/__nv_fp8_storage_t and the four conversions the engine uses
(__float2bfloat16_rn, __bfloat16_as_ushort, __nv_cvt_float_to_fp8, __nv_cvt_fp8_to_halfraw), E4M3 with round-to-nearest.
* cublas_v2.h - the nine GEMM calls (7 cublasGemmEx, 2 cublasSgemm) and their enums. The dense GEMM itself is NOT wired:
it aborts with a message naming kernels/dense_mm.metal (which exists and passes its parity test) rather than returning a
silently wrong result.
After these, ds4_model.cu's remaining 20 errors are all in ds4_os.hpp - the Linux-only OS surface (O_DIRECT,
posix_fadvise, mincore, MADV_HUGEPAGE, SYS_mbind, cpu_set_t/sched_setaffinity) - which needs a __APPLE__ branch.
strata_kernels_cpu compiled expert.cpp / q2_avx2.cpp / iq_avx*.cpp with hard-coded -mavx512f / -mavx2, so on an arm64 host the
build died with 'unsupported option -mavx512f for target arm64-apple-darwin27.2.0'. STRATA_CPU_KERNELS_X86 now decides:
on x86 the same sources and ISA flags as before, elsewhere the portable files plus src/kernels/cpu/iq256_portable.cpp, which
defines iq256_supported() = false and iq256_rows() = abort (callers check the first), so ds4_gg_cpu.cpp keeps ggml-cpu's
rows - which have their own NEON kernels on Apple Silicon.
Verified on the M5 Max: configure rc=0, build rc=0, ds4_gg_bench runs IQ3_XXS at 83.9 GB/s through ggml-cpu's rows (the
x86 EPYC does 60.5), 0 of 1,567,344 values differ in bits against ggml and 7.14e-09 against a double reference. The x86 path
is untouched: the guard only selects which sources join the library.
2026-10-23 17:53:59 +02:00
634 changed files with 65839 additions and 2050 deletions
description:"In the app: Settings > About. In the Terminal: `strata --version`. From source: the About tab of the web page, or `engine` in `GET /v1/status`, or the git commit."
Version 0.2.6, engine 0.1.51. The public model families are DeepSeekV4.1-Flash-1017,
Version 0.2.7, engine 0.1.52. The public model families are DeepSeekV4.1-Flash-1017,
Qwen3.8-Flash-Next and Qwen3.6-35B-A3B. DeepSeek V4.1-Flash runs in the app and in the Terminal
(`strata launch --family deepseek41 --model ist-das-2.00bpw`; sets in
[docs/MODELS.md](docs/MODELS.md#deepseek-v41-flash)). Read **[docs/AI_REFERENCE.md](docs/AI_REFERENCE.md)** for model availability, APIs, hardware limits and
@@ -15,7 +15,7 @@ Strata has three parts:
- **The app** (`apps/desktop`, the Tauri shell; `apps/host`, the Node host and the web UI). It is for people who do
not want a terminal. One installer per system ([latest release](https://github.com/Niko1221/Strata/releases/latest)):
# The AVX kernels exist for x86 only. On aarch64 (DGX Spark / GB10) portable.cpp stands in for them and
# native experts run on ggml-cpu's NEON dot products.
if(NOTCMAKE_SYSTEM_PROCESSORORCMAKE_SYSTEM_PROCESSORMATCHES"^(x86_64|AMD64|amd64|x64|X64|i[3-6]86|x86|X86)$")# an unset processor stays the x86 build, as before
set(STRATA_CPU_X86ON)
else()
set(STRATA_CPU_X86OFF)
endif()
if(STRATA_ISA_FLOORANDNOTSTRATA_CPU_X86)
message(WARNING"STRATA_ISA_FLOOR is an x86 older-CPU build and does not apply to ${CMAKE_SYSTEM_PROCESSOR}; ignored")
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Strata auf Buy Me a Coffee unterstützen"></a>
**Kostenlos, Open Source und ganz in deiner Hand.** Nutze Strata auf dem Desktop oder mit deinen liebsten Coding-Tools. Kein Konto nötig.
@@ -95,7 +95,7 @@ Unsicher, was passt? [Wähle ein Modell für deinen Computer](docs/MODELS.md#pic
Installiere die App, wähle ein Modell und leg mit dem Chat los. Es gibt einen Installer pro System. Beim ersten Start lädt die App die Engine für deine Grafikkarte herunter. [Hilfe zum Installer](docs/INSTALL.md#the-app).
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Apoya Strata en Buy Me a Coffee"></a>
**Gratis, de código abierto y tuyo para ejecutar.** Usa Strata en tu escritorio o con tus herramientas de programación favoritas. No necesitas cuenta.
@@ -95,7 +95,7 @@ Medido con el conjunto de 2.00 bpw en Strata 0.2.5. «Salida» es la velocidad d
Instala la aplicación, elige un modelo y empieza a chatear. Hay un instalador por sistema. En el primer arranque, la aplicación descarga el motor para tu tarjeta gráfica. [Ayuda con el instalador](docs/INSTALL.md#the-app).
<img src="docs/media/app-chat.png" width="900" alt="Aplicación de escritorio de Strata con un chat y una vista previa del código en vivo, una junto a otra">
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Soutenir Strata sur Buy Me a Coffee"></a>
**Gratuit, open source, et à vous de l'exécuter.** Utilisez Strata sur votre bureau ou avec vos outils de programmation préférés. Aucun compte nécessaire.
@@ -95,7 +95,7 @@ Vous ne savez pas ce qui tient ? [Choisir un modèle pour votre ordinateur](docs
Installez l'application, choisissez un modèle et commencez à discuter. Il y a un seul installateur par système. Au premier lancement, l'application télécharge le moteur de votre carte graphique. [Aide pour l'installateur](docs/INSTALL.md#the-app).
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Buy Me a Coffee で Strata を支援する"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Support Strata on Buy Me a Coffee"></a>
**Free, open source, and yours to run.** Use Strata on your desktop or with your favorite coding tools. No account needed.
@@ -63,14 +63,14 @@ disk space.
| 2.00 bpw | 257 GB | about 265 GB |
| 2.25 bpw | 274 GB | about 280 GB |
Runs on NVIDIA RTX 30, 40 and 50 series (Windows and Linux) and AMD Radeon AI PRO R9700 (Linux). Setup downloads the
files and the DeepSeek engine, then starts the model.
Runs on NVIDIA RTX 30, 40 and 50 series (Windows and Linux), AMD Radeon AI PRO R9700 (Linux), AMD Ryzen AI Max — Strix Halo
(Windows), and Apple Silicon Macs with 64 GB or more, text only ([Mac](docs/DEEPSEEK_MAC.md)). Setup downloads the files and the DeepSeek engine, then starts the model.
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Wesprzyj Stratę na Buy Me a Coffee"></a>
**Bezpłatna, z otwartym kodem źródłowym i do uruchomienia u siebie.** Używaj Straty na swoim komputerze albo razem z ulubionymi narzędziami do programowania. Konto nie jest potrzebne.
## Modele
Najpierw polecamy **ISTA-DASLab**, potem **Unsloth** i innych dostawców — o ile zgodny model mieści się w pamięci.
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Zużywa mniej pamięci, ale działa wolniej. |
W Windows otwórz potem nowe okno terminala. Następnie:
```sh
strata launch
```
Postępuj zgodnie z podpowiedziami, aby pobrać i uruchomić model. [Przewodnik po Terminalu](docs/TERMINAL_LAUNCH.md).
</details>
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplikacja Terminal Straty pokazująca aktywność modelu, szybkość, pamięć i ostatnie zapytania">
<br><sub>Obserwuj model i sprzęt w trakcie pracy.</sub>
</p>
<a id="build-from-source"></a>
[Budowanie ze źródeł](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Pełny przewodnik instalacji](docs/INSTALL.md)
| **AMD Ryzen AI Max · Strix Halo** | [Konfiguracja Strix Halo](docs/STRIX_HALO.md) |
<a id="using-it"></a>
## Narzędzia
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw** i inne — przez API zgodne z OpenAI i Anthropic.
Uruchom **`strata connect`**, aby zobaczyć instrukcję konfiguracji dla swojego narzędzia. [Przewodnik po połączeniach](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
[Testy wydajności](docs/COMMUNITY_BENCHMARKS.md) · [Jak to działa](docs/HOW_IT_WORKS.md) · [Współtworzenie](CONTRIBUTING.md)
## Wesprzyj Stratę
**Pomóż zachować lokalną sztuczną inteligencję otwartą dla wszystkich.** Kawa pomaga finansować rozwój i testy na większej liczbie sprzętu.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: wesprzyj Stratę" height="48"></a>
[Licencja MIT](LICENSE) · [Podziękowania i licencje modeli](docs/HOW_IT_WORKS.md#credits) · [Prywatność](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Dla narzędzi AI i automatycznych czytników: <a href="AGENTS.md">dokumentacja techniczna</a> · <a href="docs/AI_SETUP.md">instrukcja instalacji</a>.</sub>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Apoie o Strata no Buy Me a Coffee"></a>
**Gratuito, de código aberto e seu para executar.** Use o Strata no desktop ou com suas ferramentas de programação favoritas. Não precisa de conta.
@@ -95,7 +95,7 @@ Não sabe o que cabe? [Escolha um modelo para o seu computador](docs/MODELS.md#p
Instale o aplicativo, escolha um modelo e comece a conversar. Há um instalador por sistema. Na primeira vez que abre, o aplicativo baixa o motor da sua placa de vídeo. [Ajuda com o instalador](docs/INSTALL.md#the-app).
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="在 Buy Me a Coffee 上支持 Strata"></a>
{"core":{"default_permission":{"identifier":"default","description":"Default core plugins set.","permissions":["core:path:default","core:event:default","core:window:default","core:webview:default","core:app:default","core:image:default","core:resources:default","core:menu:default","core:tray:default"]},"permissions":{},"permission_sets":{},"global_scope_schema":null},"core:app":{"default_permission":{"identifier":"default","description":"Default permissions for the plugin.","permissions":["allow-version","allow-name","allow-tauri-version","allow-identifier","allow-bundle-type","allow-register-listener","allow-remove-listener","allow-supports-multiple-windows"]},"permissions":{"allow-app-hide":{"identifier":"allow-app-hide","description":"Enables the app_hide command without any pre-configured scope.","commands":{"allow":["app_hide"],"deny":[]}},"allow-app-show":{"identifier":"allow-app-show","description":"Enables the app_show command without any pre-configured scope.","commands":{"allow":["app_show"],"deny":[]}},"allow-bundle-type":{"identifier":"allow-bundle-type","description":"Enables the bundle_type command without any pre-configured scope.","commands":{"allow":["bundle_type"],"deny":[]}},"allow-default-window-icon":{"identifier":"allow-default-window-icon","description":"Enables the default_window_icon command without any pre-configured scope.","commands":{"allow":["default_window_icon"],"deny":[]}},"allow-exit":{"identifier":"allow-exit","description":"Enables the exit command without any pre-configured scope.","commands":{"allow":["exit"],"deny":[]}},"allow-fetch-data-store-identifiers":{"identifier":"allow-fetch-data-store-identifiers","description":"Enables the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":["fetch_data_store_identifiers"],"deny":[]}},"allow-identifier":{"identifier":"allow-identifier","description":"Enables the identifier command without any pre-configured scope.","commands":{"allow":["identifier"],"deny":[]}},"allow-name":{"identifier":"allow-name","description":"Enables the name command without any pre-configured scope.","commands":{"allow":["name"],"deny":[]}},"allow-register-listener":{"identifier":"allow-register-listener","description":"Enables the register_listener command without any pre-configured scope.","commands":{"allow":["register_listener"],"deny":[]}},"allow-remove-data-store":{"identifier":"allow-remove-data-store","description":"Enables the remove_data_store command without any pre-configured scope.","commands":{"allow":["remove_data_store"],"deny":[]}},"allow-remove-listener":{"identifier":"allow-remove-listener","description":"Enables the remove_listener command without any pre-configured scope.","commands":{"allow":["remove_listener"],"deny":[]}},"allow-set-app-theme":{"identifier":"allow-set-app-theme","description":"Enables the set_app_theme command without any pre-configured scope.","commands":{"allow":["set_app_theme"],"deny":[]}},"allow-set-dock-visibility":{"identifier":"allow-set-dock-visibility","description":"Enables the set_dock_visibility command without any pre-configured scope.","commands":{"allow":["set_dock_visibility"],"deny":[]}},"allow-supports-multiple-windows":{"identifier":"allow-supports-multiple-windows","description":"Enables the supports_multiple_windows command without any pre-configured scope.","commands":{"allow":["supports_multiple_windows"],"deny":[]}},"allow-tauri-version":{"identifier":"allow-tauri-version","description":"Enables the tauri_version command without any pre-configured scope.","commands":{"allow":["tauri_version"],"deny":[]}},"allow-version":{"identifier":"allow-version","description":"Enables the version command without any pre-configured scope.","commands":{"allow":["version"],"deny":[]}},"deny-app-hide":{"identifier":"deny-app-hide","description":"Denies the app_hide command without any pre-configured scope.","commands":{"allow":[],"deny":["app_hide"]}},"deny-app-show":{"identifier":"deny-app-show","description":"Denies the app_show command without any pre-configured scope.","commands":{"allow":[],"deny":["app_show"]}},"deny-bundle-type":{"identifier":"deny-bundle-type","description":"Denies the bundle_type command without any pre-configured scope.","commands":{"allow":[],"deny":["bundle_type"]}},"deny-default-window-icon":{"identifier":"deny-default-window-icon","description":"Denies the default_window_icon command without any pre-configured scope.","commands":{"allow":[],"deny":["default_window_icon"]}},"deny-exit":{"identifier":"deny-exit","description":"Denies the exit command without any pre-configured scope.","commands":{"allow":[],"deny":["exit"]}},"deny-fetch-data-store-identifiers":{"identifier":"deny-fetch-data-store-identifiers","description":"Denies the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":[],"deny":["fetch_data_store_identifiers"]}},"deny-identifier":{"identifier":"deny-identifier","description":"DenLine truncated
{"core":{"default_permission":{"identifier":"default","description":"Default core plugins set.","permissions":["core:path:default","core:event:default","core:window:default","core:webview:default","core:app:default","core:image:default","core:resources:default","core:menu:default","core:tray:default"]},"permissions":{},"permission_sets":{},"global_scope_schema":null},"core:app":{"default_permission":{"identifier":"default","description":"Default permissions for the plugin.","permissions":["allow-version","allow-name","allow-tauri-version","allow-identifier","allow-bundle-type","allow-register-listener","allow-remove-listener","allow-supports-multiple-windows"]},"permissions":{"allow-app-hide":{"identifier":"allow-app-hide","description":"Enables the app_hide command without any pre-configured scope.","commands":{"allow":["app_hide"],"deny":[]}},"allow-app-show":{"identifier":"allow-app-show","description":"Enables the app_show command without any pre-configured scope.","commands":{"allow":["app_show"],"deny":[]}},"allow-bundle-type":{"identifier":"allow-bundle-type","description":"Enables the bundle_type command without any pre-configured scope.","commands":{"allow":["bundle_type"],"deny":[]}},"allow-default-window-icon":{"identifier":"allow-default-window-icon","description":"Enables the default_window_icon command without any pre-configured scope.","commands":{"allow":["default_window_icon"],"deny":[]}},"allow-exit":{"identifier":"allow-exit","description":"Enables the exit command without any pre-configured scope.","commands":{"allow":["exit"],"deny":[]}},"allow-fetch-data-store-identifiers":{"identifier":"allow-fetch-data-store-identifiers","description":"Enables the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":["fetch_data_store_identifiers"],"deny":[]}},"allow-identifier":{"identifier":"allow-identifier","description":"Enables the identifier command without any pre-configured scope.","commands":{"allow":["identifier"],"deny":[]}},"allow-name":{"identifier":"allow-name","description":"Enables the name command without any pre-configured scope.","commands":{"allow":["name"],"deny":[]}},"allow-register-listener":{"identifier":"allow-register-listener","description":"Enables the register_listener command without any pre-configured scope.","commands":{"allow":["register_listener"],"deny":[]}},"allow-remove-data-store":{"identifier":"allow-remove-data-store","description":"Enables the remove_data_store command without any pre-configured scope.","commands":{"allow":["remove_data_store"],"deny":[]}},"allow-remove-listener":{"identifier":"allow-remove-listener","description":"Enables the remove_listener command without any pre-configured scope.","commands":{"allow":["remove_listener"],"deny":[]}},"allow-set-app-theme":{"identifier":"allow-set-app-theme","description":"Enables the set_app_theme command without any pre-configured scope.","commands":{"allow":["set_app_theme"],"deny":[]}},"allow-set-dock-visibility":{"identifier":"allow-set-dock-visibility","description":"Enables the set_dock_visibility command without any pre-configured scope.","commands":{"allow":["set_dock_visibility"],"deny":[]}},"allow-supports-multiple-windows":{"identifier":"allow-supports-multiple-windows","description":"Enables the supports_multiple_windows command without any pre-configured scope.","commands":{"allow":["supports_multiple_windows"],"deny":[]}},"allow-tauri-version":{"identifier":"allow-tauri-version","description":"Enables the tauri_version command without any pre-configured scope.","commands":{"allow":["tauri_version"],"deny":[]}},"allow-version":{"identifier":"allow-version","description":"Enables the version command without any pre-configured scope.","commands":{"allow":["version"],"deny":[]}},"deny-app-hide":{"identifier":"deny-app-hide","description":"Denies the app_hide command without any pre-configured scope.","commands":{"allow":[],"deny":["app_hide"]}},"deny-app-show":{"identifier":"deny-app-show","description":"Denies the app_show command without any pre-configured scope.","commands":{"allow":[],"deny":["app_show"]}},"deny-bundle-type":{"identifier":"deny-bundle-type","description":"Denies the bundle_type command without any pre-configured scope.","commands":{"allow":[],"deny":["bundle_type"]}},"deny-default-window-icon":{"identifier":"deny-default-window-icon","description":"Denies the default_window_icon command without any pre-configured scope.","commands":{"allow":[],"deny":["default_window_icon"]}},"deny-exit":{"identifier":"deny-exit","description":"Denies the exit command without any pre-configured scope.","commands":{"allow":[],"deny":["exit"]}},"deny-fetch-data-store-identifiers":{"identifier":"deny-fetch-data-store-identifiers","description":"Denies the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":[],"deny":["fetch_data_store_identifiers"]}},"deny-identifier":{"identifier":"deny-identifier","description":"DenLine truncated
"markdownDescription":"Denies the unminimize command without any pre-configured scope."
},
{
"description":"This permission set configures which\nnotification features are by default exposed.\n\n#### Granted Permissions\n\nIt allows all notification related features.\n\n\n#### This default permission set includes:\n\n- `allow-is-permission-granted`\n- `allow-request-permission`\n- `allow-notify`\n- `allow-register-action-types`\n- `allow-register-listener`\n- `allow-remove-listener`\n- `allow-cancel`\n- `allow-get-pending`\n- `allow-remove-active`\n- `allow-get-active`\n- `allow-check-permissions`\n- `allow-show`\n- `allow-batch`\n- `allow-list-channels`\n- `allow-delete-channel`\n- `allow-create-channel`\n- `allow-permission-state`",
"type":"string",
"const":"notification:default",
"markdownDescription":"This permission set configures which\nnotification features are by default exposed.\n\n#### Granted Permissions\n\nIt allows all notification related features.\n\n\n#### This default permission set includes:\n\n- `allow-is-permission-granted`\n- `allow-request-permission`\n- `allow-notify`\n- `allow-register-action-types`\n- `allow-register-listener`\n- `allow-remove-listener`\n- `allow-cancel`\n- `allow-get-pending`\n- `allow-remove-active`\n- `allow-get-active`\n- `allow-check-permissions`\n- `allow-show`\n- `allow-batch`\n- `allow-list-channels`\n- `allow-delete-channel`\n- `allow-create-channel`\n- `allow-permission-state`"
},
{
"description":"Enables the batch command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-batch",
"markdownDescription":"Enables the batch command without any pre-configured scope."
},
{
"description":"Enables the cancel command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-cancel",
"markdownDescription":"Enables the cancel command without any pre-configured scope."
},
{
"description":"Enables the check_permissions command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-check-permissions",
"markdownDescription":"Enables the check_permissions command without any pre-configured scope."
},
{
"description":"Enables the create_channel command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-create-channel",
"markdownDescription":"Enables the create_channel command without any pre-configured scope."
},
{
"description":"Enables the delete_channel command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-delete-channel",
"markdownDescription":"Enables the delete_channel command without any pre-configured scope."
},
{
"description":"Enables the get_active command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-get-active",
"markdownDescription":"Enables the get_active command without any pre-configured scope."
},
{
"description":"Enables the get_pending command without any pre-configured scope.",
"type":"string",
"const":"notification:allow-get-pending",
"markdownDescription":"Enables the get_pending command without any pre-configured scope."
},
{
"description":"Enables the is_permission_granted command without any pre-configured scope.",
"markdownDescription":"Denies the register_action_types command without any pre-configured scope."
},
{
"description":"Denies the register_listener command without any pre-configured scope.",
"type":"string",
"const":"notification:deny-register-listener",
"markdownDescription":"Denies the register_listener command without any pre-configured scope."
},
{
"description":"Denies the remove_active command without any pre-configured scope.",
"type":"string",
"const":"notification:deny-remove-active",
"markdownDescription":"Denies the remove_active command without any pre-configured scope."
},
{
"description":"Denies the remove_listener command without any pre-configured scope.",
"type":"string",
"const":"notification:deny-remove-listener",
"markdownDescription":"Denies the remove_listener command without any pre-configured scope."
},
{
"description":"Denies the request_permission command without any pre-configured scope.",
"type":"string",
"const":"notification:deny-request-permission",
"markdownDescription":"Denies the request_permission command without any pre-configured scope."
},
{
"description":"Denies the show command without any pre-configured scope.",
"type":"string",
"const":"notification:deny-show",
"markdownDescription":"Denies the show command without any pre-configured scope."
},
{
"description":"This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type":"string",
@@ -2491,6 +2701,60 @@
"type":"string",
"const":"shell:deny-stdin-write",
"markdownDescription":"Denies the stdin_write command without any pre-configured scope."
},
{
"description":"This permission set configures which kind of\nupdater functions are exposed to the frontend.\n\n#### Granted Permissions\n\nThe full workflow from checking for updates to installing them\nis enabled.\n\n\n#### This default permission set includes:\n\n- `allow-check`\n- `allow-download`\n- `allow-install`\n- `allow-download-and-install`",
"type":"string",
"const":"updater:default",
"markdownDescription":"This permission set configures which kind of\nupdater functions are exposed to the frontend.\n\n#### Granted Permissions\n\nThe full workflow from checking for updates to installing them\nis enabled.\n\n\n#### This default permission set includes:\n\n- `allow-check`\n- `allow-download`\n- `allow-install`\n- `allow-download-and-install`"
},
{
"description":"Enables the check command without any pre-configured scope.",
"type":"string",
"const":"updater:allow-check",
"markdownDescription":"Enables the check command without any pre-configured scope."
},
{
"description":"Enables the download command without any pre-configured scope.",
"type":"string",
"const":"updater:allow-download",
"markdownDescription":"Enables the download command without any pre-configured scope."
},
{
"description":"Enables the download_and_install command without any pre-configured scope.",
"type":"string",
"const":"updater:allow-download-and-install",
"markdownDescription":"Enables the download_and_install command without any pre-configured scope."
},
{
"description":"Enables the install command without any pre-configured scope.",
"type":"string",
"const":"updater:allow-install",
"markdownDescription":"Enables the install command without any pre-configured scope."
},
{
"description":"Denies the check command without any pre-configured scope.",
"type":"string",
"const":"updater:deny-check",
"markdownDescription":"Denies the check command without any pre-configured scope."
},
{
"description":"Denies the download command without any pre-configured scope.",
"type":"string",
"const":"updater:deny-download",
"markdownDescription":"Denies the download command without any pre-configured scope."
},
{
"description":"Denies the download_and_install command without any pre-configured scope.",
"type":"string",
"const":"updater:deny-download-and-install",
"markdownDescription":"Denies the download_and_install command without any pre-configured scope."
},
{
"description":"Denies the install command without any pre-configured scope.",
"type":"string",
"const":"updater:deny-install",
"markdownDescription":"Denies the install command without any pre-configured scope."
The built UI is included. To rebuild after source edits, run `npm run build`. `preview:ui` uses isolated simulated data; `npm run dev` connects the normal host.
The previous project README is retained in [docs/README-historical.md](docs/README-historical.md). It contains older implementation status and is historical reference.
## Languages
The app speaks English, Chinese, Japanese, German, French, Spanish, Russian, Portuguese and Korean. The English text is the key: `src/locales/en.json`
(with `fix-en.json`, `tune-en.json` and `tools-en.json`) holds every string, and each language has the same four files (`fr.json`, `fix-fr.json`, ...).
- **A new string.** Write it with `tr("...")`, then add it to the English catalog (`node scripts/append-localization.mjs`, or the next `fx` number in
`scripts/make-fix-locales.py`). The language tests fail until every language has it.
- **French.** `python scripts/fr_sync.py missing todo.tsv` lists the English strings that have no French yet, one per line (`id`, a tab, the text).
Translate the second column ("vous", short plain words, one next step; a lone `=` keeps the English), then `python scripts/fr_sync.py apply todo.tsv` merges it
(it also sets the French spacing before `: ; ? !` and the typographic apostrophe) and `python scripts/fr_sync.py check` confirms nothing is missing.
- **Screenshots in a language.** `npm run build`, then `node scripts/language-shots.mjs fr <folder> models settings setup` (hermetic preview host, scratch profile, headless Chrome).
"""Writes src/locales/gh-<lang>.json: the interface strings of the 0.2.7 GitHub items done in f-027-gh-app (the Monitor's Power card #3628,
approval notifications and the tray #3477, the finished-task sound #3586, the pinned port #3402, the interface size #3970).
Run from apps/host: python scripts/make-gh-locales.py
The keys are gh01.. in this order; gh-en.json is the English source. Placeholders ({0}, {1}) must be kept in every language.
A separate file keeps this work from colliding with the other catalogs (the same reason tune-, tools- and fix- exist)."""
importjson
importre
frompathlibimportPath
EN=[
"Energy since start",
"Cost since start",
"GPU fan",
"GPU clock",
"CPU temperature",
"SSD temperature",
"Electricity price",
"Price of one kilowatt-hour. The Monitor then shows what your graphics card has cost since Strata started. Leave it empty to hide the cost.",
"Price per kWh",
"Sound when work finishes",
"Plays a short sound when an answer or task that took a while is finished, or when Strata is in the background.",
"Play sound",
"Notify me when Strata needs an answer",
"In Manual mode, shows a desktop notification when the assistant waits for your approval and Strata is in the background.",
"Strata needs your answer",
"The assistant wants to use {0}. Open Strata to approve or deny.",
"a tool",
"Minimise to tray",
"Closing the window keeps Strata running in the tray, next to the clock. Choose Quit in the tray menu to stop it. Applies the next time Strata starts.",
"App port",
"The port Strata’s window and local address use. Leave it empty to let Strata choose (47831 when it is free). Applies the next time Strata starts.",
"Port (empty = automatic)",
"Using port {0} now.",
"Strata could not use its port",
"Port {0} is used by another program, so Strata started on port {1}. Chats and settings saved on port {0} are not shown until Strata can use it again. Close the other program and restart Strata.",
"Choose a port between 1024 and 65535, or leave it empty.",
"Makes all text, buttons and menus in Strata larger or smaller.",
"Preis einer Kilowattstunde. Der Monitor zeigt dann, was Ihre Grafikkarte seit dem Start von Strata gekostet hat. Leer lassen, um die Kosten auszublenden.",
"Preis pro kWh",
"Ton, wenn Arbeit fertig ist",
"Spielt einen kurzen Ton, wenn eine Antwort oder Aufgabe, die länger gedauert hat, fertig ist oder wenn Strata im Hintergrund läuft.",
"Ton abspielen",
"Benachrichtigen, wenn Strata eine Antwort braucht",
"Zeigt im manuellen Modus eine Desktop-Benachrichtigung, wenn der Assistent auf Ihre Freigabe wartet und Strata im Hintergrund läuft.",
"Strata braucht Ihre Antwort",
"Der Assistent möchte {0} verwenden. Öffnen Sie Strata, um zu erlauben oder abzulehnen.",
"ein Werkzeug",
"In den Infobereich minimieren",
"Beim Schließen des Fensters läuft Strata im Infobereich neben der Uhr weiter. Wählen Sie im Menü des Symbols „Beenden“, um es zu stoppen. Gilt ab dem nächsten Start von Strata.",
"App-Port",
"Der Port, den das Fenster und die lokale Adresse von Strata verwenden. Leer lassen, damit Strata ihn wählt (47831, wenn frei). Gilt ab dem nächsten Start von Strata.",
"Port (leer = automatisch)",
"Aktuell wird Port {0} verwendet.",
"Strata konnte seinen Port nicht verwenden",
"Port {0} wird von einem anderen Programm verwendet, daher wurde Strata auf Port {1} gestartet. Unter Port {0} gespeicherte Chats und Einstellungen werden nicht angezeigt, bis Strata ihn wieder verwenden kann. Schließen Sie das andere Programm und starten Sie Strata neu.",
"Wählen Sie einen Port zwischen 1024 und 65535 oder lassen Sie das Feld leer.",
"Macht alle Texte, Schaltflächen und Menüs in Strata größer oder kleiner.",
"Energie- und Temperaturwerte",
"pro kWh, in {0}",
"pro kWh",
"Währung ändern",
"Automatisch ({0})",
"Währung",
"weniger als {0}",
],
"es":[
"Energía desde el inicio",
"Coste desde el inicio",
"Ventilador de la GPU",
"Reloj de la GPU",
"Temperatura de la CPU",
"Temperatura del SSD",
"Precio de la electricidad",
"Precio de un kilovatio hora. El Monitor mostrará entonces lo que ha costado tu tarjeta gráfica desde que se inició Strata. Déjalo vacío para ocultar el coste.",
"Precio por kWh",
"Sonido al terminar el trabajo",
"Reproduce un sonido breve cuando termina una respuesta o tarea que tardó un rato, o cuando Strata está en segundo plano.",
"Reproducir sonido",
"Avisarme cuando Strata necesite una respuesta",
"En el modo manual, muestra una notificación de escritorio cuando el asistente espera tu aprobación y Strata está en segundo plano.",
"Strata necesita tu respuesta",
"El asistente quiere usar {0}. Abre Strata para aprobar o denegar.",
"una herramienta",
"Minimizar a la bandeja",
"Al cerrar la ventana, Strata sigue funcionando en la bandeja, junto al reloj. Elige Salir en el menú de la bandeja para detenerlo. Se aplica la próxima vez que se inicie Strata.",
"Puerto de la aplicación",
"El puerto que usan la ventana y la dirección local de Strata. Déjalo vacío para que Strata elija (47831 si está libre). Se aplica la próxima vez que se inicie Strata.",
"Puerto (vacío = automático)",
"Ahora se usa el puerto {0}.",
"Strata no pudo usar su puerto",
"El puerto {0} lo usa otro programa, así que Strata se inició en el puerto {1}. Los chats y ajustes guardados en el puerto {0} no se muestran hasta que Strata pueda volver a usarlo. Cierra el otro programa y reinicia Strata.",
"Elige un puerto entre 1024 y 65535, o déjalo vacío.",
"Hace más grandes o más pequeños todos los textos, botones y menús de Strata.",
"Lecturas de energía y temperatura",
"por kWh, en {0}",
"por kWh",
"Cambiar moneda",
"Automática ({0})",
"Moneda",
"menos de {0}",
],
"ru":[
"Энергия с запуска",
"Стоимость с запуска",
"Вентилятор GPU",
"Частота GPU",
"Температура CPU",
"Температура SSD",
"Цена электричества",
"Цена одного киловатт-часа. Тогда монитор покажет, во сколько видеокарта обошлась с запуска Strata. Оставьте поле пустым, чтобы скрыть стоимость.",
"Цена за кВт·ч",
"Звук по окончании работы",
"Проигрывает короткий звук, когда закончился долгий ответ или задача, а также когда Strata работает в фоне.",
"Проиграть звук",
"Уведомлять, когда Strata нужен ответ",
"В ручном режиме показывает уведомление на рабочем столе, когда помощник ждёт вашего подтверждения, а Strata работает в фоне.",
"Strata нужен ваш ответ",
"Помощник хочет использовать {0}. Откройте Strata, чтобы разрешить или отклонить.",
"инструмент",
"Сворачивать в трей",
"При закрытии окна Strata продолжает работать в трее рядом с часами. Чтобы остановить её, выберите «Выход» в меню значка. Действует со следующего запуска Strata.",
"Порт приложения",
"Порт, который используют окно и локальный адрес Strata. Оставьте поле пустым, чтобы Strata выбрала сама (47831, если свободен). Действует со следующего запуска Strata.",
"Порт (пусто = автоматически)",
"Сейчас используется порт {0}.",
"Strata не смогла использовать свой порт",
"Порт {0} занят другой программой, поэтому Strata запустилась на порту {1}. Чаты и настройки, сохранённые на порту {0}, не показываются, пока Strata не сможет снова его использовать. Закройте другую программу и перезапустите Strata.",
"Выберите порт от 1024 до 65535 или оставьте поле пустым.",
"Увеличивает или уменьшает весь текст, кнопки и меню в Strata.",
"Показания энергии и температуры",
"за кВт·ч, в валюте {0}",
"за кВт·ч",
"Сменить валюту",
"Автоматически ({0})",
"Валюта",
"меньше {0}",
],
"pt":[
"Energia desde o início",
"Custo desde o início",
"Ventoinha da GPU",
"Clock da GPU",
"Temperatura da CPU",
"Temperatura do SSD",
"Preço da eletricidade",
"Preço de um quilowatt-hora. O Monitor passa a mostrar quanto a sua placa de vídeo custou desde que o Strata iniciou. Deixe vazio para ocultar o custo.",
"Preço por kWh",
"Som ao terminar o trabalho",
"Toca um som curto quando uma resposta ou tarefa que demorou termina, ou quando o Strata está em segundo plano.",
"Tocar som",
"Avisar quando o Strata precisar de uma resposta",
"No modo manual, mostra uma notificação na área de trabalho quando o assistente espera sua aprovação e o Strata está em segundo plano.",
"O Strata precisa da sua resposta",
"O assistente quer usar {0}. Abra o Strata para aprovar ou negar.",
"uma ferramenta",
"Minimizar para a bandeja",
"Ao fechar a janela, o Strata continua em execução na bandeja, ao lado do relógio. Escolha Sair no menu da bandeja para encerrá-lo. Vale a partir da próxima vez que o Strata iniciar.",
"Porta do aplicativo",
"A porta usada pela janela e pelo endereço local do Strata. Deixe vazio para o Strata escolher (47831 quando estiver livre). Vale a partir da próxima vez que o Strata iniciar.",
"Porta (vazio = automático)",
"Usando a porta {0} agora.",
"O Strata não conseguiu usar a sua porta",
"A porta {0} está em uso por outro programa, então o Strata iniciou na porta {1}. Os chats e as configurações salvos na porta {0} não aparecem até o Strata poder usá-la de novo. Feche o outro programa e reinicie o Strata.",
"Escolha uma porta entre 1024 e 65535, ou deixe vazio.",
"Deixa todos os textos, botões e menus do Strata maiores ou menores.",
"Leituras de energia e temperatura",
"por kWh, em {0}",
"por kWh",
"Alterar moeda",
"Automática ({0})",
"Moeda",
"menos de {0}",
],
"ko":[
"시작 이후 에너지",
"시작 이후 비용",
"GPU 팬",
"GPU 클럭",
"CPU 온도",
"SSD 온도",
"전기 요금",
"1킬로와트시의 요금입니다. 모니터에 Strata를 시작한 뒤 그래픽 카드에 든 전기 요금이 표시됩니다. 비워 두면 비용을 표시하지 않습니다.",
"kWh당 요금",
"작업이 끝났을 때 소리",
"답변이나 오래 걸린 작업이 끝났을 때, 또는 Strata가 백그라운드에 있을 때 짧은 소리를 재생합니다.",
"소리 재생",
"Strata가 답을 필요로 할 때 알림",
"수동 모드에서 어시스턴트가 승인을 기다리고 Strata가 백그라운드에 있을 때 데스크톱 알림을 표시합니다.",
"Strata가 답을 기다리고 있습니다",
"어시스턴트가 {0}을(를) 사용하려고 합니다. Strata를 열어 승인하거나 거부하세요.",
"도구",
"트레이로 최소화",
"창을 닫아도 Strata는 시계 옆 트레이에서 계속 실행됩니다. 끝내려면 트레이 메뉴에서 '종료'를 선택하세요. 다음에 Strata를 시작할 때부터 적용됩니다.",
"앱 포트",
"Strata의 창과 로컬 주소가 사용하는 포트입니다. 비워 두면 Strata가 고릅니다(비어 있으면 47831). 다음에 Strata를 시작할 때부터 적용됩니다.",
"포트(비움 = 자동)",
"지금은 포트 {0}을(를) 사용 중입니다.",
"Strata가 지정한 포트를 사용할 수 없었습니다",
"포트 {0}을(를) 다른 프로그램이 사용 중이어서 Strata는 포트 {1}에서 시작했습니다. 포트 {0}에 저장된 채팅과 설정은 Strata가 다시 그 포트를 쓸 수 있을 때까지 표시되지 않습니다. 다른 프로그램을 닫고 Strata를 다시 시작하세요.",
"1024에서 65535 사이의 포트를 고르거나 비워 두세요.",
"Strata의 모든 글자, 버튼, 메뉴를 더 크게 또는 더 작게 만듭니다.",
"에너지 및 온도 측정값",
"kWh당, 통화 {0}",
"kWh당",
"통화 변경",
"자동({0})",
"통화",
"{0} 미만",
],
"fr":[
"Énergie depuis le démarrage",
"Coût depuis le démarrage",
"Ventilateur du GPU",
"Fréquence du GPU",
"Température du CPU",
"Température du SSD",
"Prix de l'électricité",
"Prix d'un kilowattheure. Le Moniteur affiche alors ce que votre carte graphique a coûté depuis le démarrage de Strata. Laissez vide pour masquer le coût.",
"Prix par kWh",
"Son à la fin du travail",
"Joue un court son quand une réponse ou une tâche qui a pris du temps est terminée, ou quand Strata est en arrière-plan.",
"Jouer le son",
"Me prévenir quand Strata a besoin d'une réponse",
"En mode manuel, affiche une notification sur le bureau quand l'assistant attend votre accord et que Strata est en arrière-plan.",
"Strata a besoin de votre réponse",
"L'assistant veut utiliser {0}. Ouvrez Strata pour accepter ou refuser.",
"un outil",
"Réduire dans la zone de notification",
"Fermer la fenêtre garde Strata actif dans la zone de notification, près de l'horloge. Choisissez Quitter dans le menu de l'icône pour l'arrêter. S'applique au prochain démarrage de Strata.",
"Port de l'application",
"Le port que la fenêtre et l'adresse locale de Strata utilisent. Laissez vide pour que Strata choisisse (47831 s'il est libre). S'applique au prochain démarrage de Strata.",
"Port (vide = automatique)",
"Port {0} utilisé en ce moment.",
"Strata n'a pas pu utiliser son port",
"Le port {0} est utilisé par un autre programme, donc Strata a démarré sur le port {1}. Les conversations et réglages enregistrés sur le port {0} ne s'affichent pas tant que Strata ne peut pas le réutiliser. Fermez l'autre programme et relancez Strata.",
"Choisissez un port entre 1024 et 65535, ou laissez vide.",
"Agrandit ou réduit tous les textes, boutons et menus de Strata.",
@@ -61,7 +61,7 @@ The chat's Mode menu is real Harness behaviour. Each turn the page sends the cha
| Bypass | `strata-bypass` | `danger-full-access` | `never` | Full access without prompts, as before. |
- Manual relies on the gate, so a Manual turn does not start unless the bridge has confirmed the gate is loaded (`/strata-gate` command): without it Manual would be an open sandbox that nobody asks about.
- The gate listens to `tools/pre-execute` (the seam the Harness's own Auto-review layer uses) and returns an `ask` decision; the Harness's approval service then asks, runs the tool only after `allowed-once`, and records the answer. Read-only tools and a short allowlist of look-only shell commands (Get-ChildItem, git status, ...) run unasked; MCP and unknown tools ask.
- The gate listens to `tools/pre-execute` (the seam the Harness's own Auto-review layer uses) and returns an `ask` decision; the Harness's approval service then asks, runs the tool only after `allowed-once`, and records the answer. Read-only tools and a short allowlist of look-only shell commands (Get-ChildItem, git status, ...) run unasked, each only with the options listed for it in `strata-gate.mjs` (an unknown option, or one that writes, runs a program, sets something or never ends, asks); MCP and unknown tools ask.
- Web search and web pages (`web_search`, `web_fetch`) ask once per chat in Manual, not once per call. The gate asks through the approval service itself (`webDecision`: "The assistant wants to search the web for: ... Search words go to <service>." / "... open the web page ..."), remembers a yes for the chat (a helper agent shares its chat's answer, and two searches started together share one card), and on a no gives the model `WEB_REFUSAL` ("The person did not allow web access ... Go on without it and say so."). Plan and Bypass are unchanged. Settings -> Tools has a switch, "Let the assistant search the web without asking" (off by default, `webWithoutAsking` in `tools-settings.mjs`, passed to the gate as `config.web.skipAsk`), that skips the question in Manual. A scheduled task has nobody at the card, so its web call is answered no unless that switch is on. Tests: `tests/web-ask.test.mjs`.
- The mode is read once per turn; an unknown or missing value is Manual.
@@ -93,6 +93,13 @@ What the person chose is saved by `tools-settings.mjs` (inside the host's settin
## Thinking and the output budget
The Harness staging script (`apps/desktop/scripts/stage-harness.mjs`) applies Strata's guarded pi-ai 1.1.0 patch to its staged copy
(`harness-budget.mjs`, #3684): below 64K the context safety room is at most `min(4096, floor(contextWindow / 16))` and never more
than half the estimated remaining room, so an 8K window keeps output room after a tool result instead of ending with a one-token
cap. The caller's output cap and the one-token floor remain; windows of 64K or more keep the previous calculation. `BUNDLE.json.patches`
records the patch identity, the dependency version and the before/after SHA-256. Unknown dependency bytes stop staging for review.
Source and already-installed Harness copies are not changed.
- A local route is reasoning-capable (`reasoning: true`) with `compat.thinkingFormat: 'chat-template'`: the request carries `chat_template_kwargs {enable_thinking, reasoning_effort}`, which the Strata server reads (a top-level `enable_thinking` is ignored by it, which is why pi-ai's `qwen` format left the model thinking). The chat's Reasoning menu travels as `thinking` in the stream body; the bridge sends it with `session/selectModel` (`reasoningEffort`) whenever the model or the level changed; a level a route does not have is dropped (never an error). API routes keep the provider default (the menu is disabled for them in the UI).
-`localOutputBudget(context)` is 30 percent of the window, at least 4096 and at most 32768 (9,830 tokens at 32K, 19,660 at 64K). The Harness's automatic compaction reads it as the room a reply reserves. pi-ai sends the window minus the prompt. App-loaded models get `fit_max_tokens: true` so the engine clamps instead of answering 400. Sampling is not set here: the Harness sends none, so the server's own defaults apply.
/** What `ls`, `dir`, `gci` and `Get-ChildItem` take. */
constLS={f:'aAbBcdDFgGhHiklLmnNopqQrRsStuUvxX1CZ',v:'ITw',ps:'path literalpath filter include exclude recurse depth name force file directory hidden readonly system attributes',
l:'all almost-all human-readable recursive directory classify reverse size inode dereference color group-directories-first sort time time-style block-size indicator-style si numeric-uid-gid no-group literal hide ignore quote-name escape author file-type full-time width tabsize'};
/** What `cat`, `type`, `gc` and `Get-Content` take. (-Wait never ends, so it is not here.) */
constCAT={f:'AbeEnstTuv',ps:'path literalpath totalcount tail head raw encoding readcount force filter include exclude delimiter stream asbytestream',
l:'show-all number-nonblank show-ends number squeeze-blank show-tabs show-nonprinting'};
constMEASURE={ps:'property sum average maximum minimum line word character ignorewhitespace inputobject standarddeviation allstats'};
constSELECT={ps:'first last skip skiplast property expandproperty unique index excludeproperty inputobject'};
constWHERE={ps:'property value inputobject eq ne gt lt ge le like notlike match notmatch contains notcontains in notin is isnot ceq cne cgt clt cge cle clike cnotlike cmatch cnotmatch ccontains cnotcontains cin cnotin'};
// POSIX. `sort` and `date` take no PowerShell names here: `sort -o FILE` writes, `date -s` sets the clock, and a PowerShell-looking `-Second` or `-Top`
// must not be read by a real sort or date as one of those. (Sort-Object and Get-Date above are the PowerShell forms.)
add('stat',{f:'LtxlnqrsFf',v:'c',l:'dereference file-system terse format printf'});
add('file',{f:'bhikLnNrvI',l:'brief dereference no-dereference mime mime-type mime-encoding keep-going no-pad no-buffer raw'});// no -C, -m, -f, -P (they write or read other files), no -z/-s/-p (they run decompressors, read devices, reset times)
add('du',{f:'abchHklmsxLPSD0',v:'tB',vn:'d',l:'all apparent-size bytes total dereference-args human-readable si summarize one-file-system dereference no-dereference separate-dirs count-links inodes null max-depth threshold block-size time time-style exclude'});
add('df',{f:'ahHiklmPTv',v:'txB',l:'all human-readable si inodes local portability print-type total sync no-sync output block-size type exclude-type'});
add('tree',{f:'adfilpsugDFqNQrtvxCnASUchJX',v:'LPIHT',vn:'',l:'help version filelimit charset sort timefmt noreport dirsfirst filesfirst prune du si gitignore matchdirs ignore-case inodes device nolinks'});// no -o, no -R
// `less` and `more` are not here: they wait for keys, and `less` can run commands.
/** The names that run without a card (for the tests). */
// `git config KEY` reads and `git config KEY VALUE` writes, so a key is only taken with --get; no --edit, --add, --unset, --file.
config:spec({f:'lz',l:'list get get-all get-regexp show-origin show-scope local global system worktree bool int bool-or-int path expiry-date type null name-only includes no-includes',lv:'default',
/** The PCI vendor ids of the display adapters Windows reports as present right now (PowerShell, Win32_VideoController); empty when that cannot be read. */
// DeepSeek-V4.1-Flash's own context limit, read from its pack (the app offers sizes up to it: src/lib/deepseek-context.ts).
// Project Maya v1.1.28 (#442, @needmorevram, MIT) did the same in its dashboard. strata_control/dsv41_context.py reads the same file for setup and the Terminal.
import{readFileSync}from'node:fs';
import{join}from'node:path';
/** The pack's own limit: `max_position_embeddings` in <pack>/ds4.txt. Null when there is no pack, no file or no such line (the choices then stay as before). */
/* The app's version has one source: the desktop shell passes it (STRATA_APP_VERSION, from apps/desktop/src-tauri/Cargo.toml). Outside the app it is read from the same file in a checkout, else 'dev'. */
/* The engine that ships inside the installer is put in the components folder when this PC has none (or an older one); no network, hashes checked. The model list waits for it. */
entry=serving.open({client:'Strata chat',endpoint:connection.protocol==='anthropic'?'/messages':connection.protocol==='openai-responses'?'/responses':'/chat/completions',model:`${connection.name} · ${model.name}`,cancel:()=>controller.abort(newDOMException('Cancelled by the user','AbortError'))});
serving.mark(entry.id,{state:'generating'});
constresult=awaitmodelProviders.stream({selection,message,history,images,signal:requestSignal,systemPrompt:['You are Strata, a helpful assistant. Answer clearly. Do not claim to have read files or performed actions unless their results are included in the conversation.',skillInstructions(appliedSkills)].filter(Boolean).join('\n\n'),onFrame:frame=>{
constresult=awaitmodelProviders.stream({selection,message,history,images,thinking,signal:requestSignal,systemPrompt:['You are Strata, a helpful assistant. Answer clearly. Do not claim to have read files or performed actions unless their results are included in the conversation.',skillInstructions(appliedSkills)].filter(Boolean).join('\n\n'),onFrame:frame=>{
constsafeId=value=>typeofvalue==='string'&&/^[a-z0-9](?:[a-z0-9_-]|\.(?!\.)){0,79}$/i.test(value);/* a dot inside the name: deepseek-v4.1-flash-ist-das-2.00bpw */
@@ -86,7 +87,7 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
if(JSON.stringify(next)!==JSON.stringify(config)){try{writeModelConfig(path,next);}catch{/* read-only folder: the load reports it */}}}}
functioninstalledInfo(info){if(!info)returninfo;return{...info,models:(info.models??[]).map(entry=>{constcfg=/^strata-[a-z0-9_.-]+\.json$/i.test(entry.config??'')?readJson(entry.configPath??join(repo(),entry.config)):{};constat=cfg.args?.indexOf('--native')??-1,nativeFile=at>=0?cfg.args[at+1]:null;constdataFolder=typeofnativeFile==='string'&&basename(dirname(dirname(nativeFile)))==='models'?dirname(dirname(dirname(nativeFile))):info.data_folder;/* A model the person pointed at their own GGUF folder (not <data>/models/<tag>): remembered, so a later load runs setup with that folder again instead of downloading the files (#3199). */constownGgufDir=typeofnativeFile==='string'&&basename(dirname(dirname(nativeFile)))!=='models'&&existsSync(nativeFile)?dirname(nativeFile):undefined;return{...entry,ownGgufDir,diskGb:entry.diskGb??entry.disk_gb,diskBytes:entry.diskBytes??entry.disk_bytes,sharedBytes:entry.sharedBytes??entry.shared_bytes,removableGb:entry.removableGb??entry.removable_gb,dataFolder,visionPrepared:!!cfg.vision,/* The picture files are prepared when the run config has a vision section: the Models screen then loads straight away instead of running setup again (it used to read 'off' for every model, which sent every load of a picture-capable model through "Apply settings and load", and setup writes the arguments, the draft layer and the reader back each time). */images:cfg.vision?'yes':'off',draftLayer:cfg.mtp!==false,drafting:cfg.drafting??'auto',/* Picture reading is ON unless the person turned it off: no saved choice and not loaded is undefined, not false (owner rule: pictures on by default). */visionEnabled:cfg.ui_vision_enabled??(()=>{construnning=[...instances.values()].find(i=>i.model===entry.model&&i.status==='ready');returnrunning?!!running.vision:undefined;})(),visionDevice:cfg.ui_vision_device,resourceProfile:cfg.ui_resource_profile,lowRamMode:cfg.args?.includes('--resident-experts')?'resident':cfg.args?.includes('--mmap-experts')?'mmap':'auto'};})};}
functioninstalledInfo(info){if(!info)returninfo;return{...info,models:(info.models??[]).map(entry=>{constcfg=/^strata-[a-z0-9_.-]+\.json$/i.test(entry.config??'')?readJson(entry.configPath??join(repo(),entry.config)):{};constat=cfg.args?.indexOf('--native')??-1,nativeFile=at>=0?cfg.args[at+1]:null;constdataFolder=typeofnativeFile==='string'&&basename(dirname(dirname(nativeFile)))==='models'?dirname(dirname(dirname(nativeFile))):info.data_folder;/* A model the person pointed at their own GGUF folder (not <data>/models/<tag>): remembered, so a later load runs setup with that folder again instead of downloading the files (#3199). */constownGgufDir=typeofnativeFile==='string'&&basename(dirname(dirname(nativeFile)))!=='models'&&existsSync(nativeFile)?dirname(nativeFile):undefined;return{...entry,ownGgufDir,contextMax:configContextLimit(cfg)??undefined,diskGb:entry.diskGb??entry.disk_gb,diskBytes:entry.diskBytes??entry.disk_bytes,sharedBytes:entry.sharedBytes??entry.shared_bytes,removableGb:entry.removableGb??entry.removable_gb,dataFolder,visionPrepared:!!cfg.vision,/* The picture files are prepared when the run config has a vision section: the Models screen then loads straight away instead of running setup again (it used to read 'off' for every model, which sent every load of a picture-capable model through "Apply settings and load", and setup writes the arguments, the draft layer and the reader back each time). */images:cfg.vision?'yes':'off',draftLayer:cfg.mtp!==false,drafting:cfg.drafting??'auto',/* Picture reading is ON unless the person turned it off: no saved choice and not loaded is undefined, not false (owner rule: pictures on by default). */visionEnabled:cfg.ui_vision_enabled??(()=>{construnning=[...instances.values()].find(i=>i.model===entry.model&&i.status==='ready');returnrunning?!!running.vision:undefined;})(),visionDevice:cfg.ui_vision_device,resourceProfile:cfg.ui_resource_profile,lowRamMode:cfg.args?.includes('--resident-experts')?'resident':cfg.args?.includes('--mmap-experts')?'mmap':'auto'};})};}
/* The settings file Strata Control and the installer read (control-runtime.mjs sets STRATA_SETTINGS_FILE to it): the app's data folder is written there. */
@@ -149,8 +150,8 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
functionsettingsFile(model){const{path}=recordFile(model);return{model,path,name:basename(path),...checkRunConfig(readFileSync(path,'utf8')),note:'Changes you make in the file apply the next time the model starts.'};}
/* A mistake in the file holds the start back with the line, instead of an engine that dies on it. Warnings (an unknown key) never do. */
functionassertSettingsFile(path){constcheck=checkRunConfig(readFileSync(path,'utf8'));if(!check.ok)thrownewError(`The settings file for this model has a mistake, so the model was not started. ${describeIssues(check.issues,'error').join(' ')}`);}
functioncheckedSettingsConfig(text){constcheck=checkAndParseRunConfig(text);if(!check.ok)thrownewError(`The settings file for this model has a mistake, so the model was not started. ${describeIssues(check.issues,'error').join(' ')}`);returncheck.config;}
functionsave(model,{values,mcp}){if([...jobs.values()].some(j=>j.state==='running'))thrownewError('Finish the current operation before saving native settings.');const{path,config}=record(model);if(busy(requests().filter(r=>instances.get(r.instanceId)?.model===model)))thrownewError('Finish this model’s requests before saving its settings.');constpatch=validateNativeSettings(values??{}),next={...config};if(patch.ds_experts!=null&&!dsExpertsSupported(config))thrownewError('The Speed choice needs the newer DeepSeek engine. Update the engine in Settings, then choose it.');for(constfieldofnativeFields())if(field.supported===false&&patch[field.id]!=null&&JSON.stringify(patch[field.id])!==JSON.stringify(config[field.id]))thrownewError(`${field.label} needs a newer native Strata installation. Update it in Settings → About.`);for(const[key,value]ofObject.entries(patch)){if(key==='env')next.env=restoreNativeEnv(value,config.env??{});elseif(value===null)deletenext[key];elsenext[key]=value;}if('vram_reserve_mib'inpatch){constbase=captureUserArgs(next);constmine=setUserFlag(base,'--vram-reserve-mib',patch.vram_reserve_mib);if(mine.length)next[USER_KEY]=mine;elsedeletenext[USER_KEY];next.args=mine.length&&patch.vram_reserve_mib!==null?mergeUserArgs(next.args??[],mine):setUserFlag(next.args??[],'--vram-reserve-mib',null);deletenext.ui_reserve_applied;if(Array.isArray(next.args_written))next.args_written=[...next.args];}if(mcp!==undefined){next.mcp_servers=validateMcpServers(mcp,config.mcp_servers??config.mcpServers??{});deletenext.mcpServers;}copyFileSync(path,path+'.bak');chmodSync(path+'.bak',0o600);consttemp=path+'.ui-tmp';writeFileSync(temp,JSON.stringify(next,null,2)+'\n',{mode:0o600});renameSync(temp,path);returnsettings(model);}
/* Which drafter runs, when it is not the one that was asked for (the engine's own words, "Drafting: MTP (DFlash2 didn't fit)"): under Details of the load. */
asyncfunctionnoteDrafting(job,instance,key){letnote='';try{constr=awaitfetcher(`http://127.0.0.1:${instance.port}/metrics`,{headers:{authorization:`Bearer ${key}`},signal:AbortSignal.timeout(2000)});if(r.ok)note=String((awaitr.json())?.engine?.drafting_note??'');}catch{/* a convenience */}constlines=[job.drafterNote,note].filter(Boolean);if(lines.length)job.details=[job.details,...lines].filter(Boolean).join('\n');}
if(externalRunning().length)thrownewError('A Strata server is already running outside this UI. Stop it before loading here, or connect its API in Settings → Models.');
if(args.multiple&&instances.size>=8)thrownewError('Unload an instance before adding another. Up to eight native instances can run here.');
try{awaitstartWith(retryConfig);append(id,'Strata Control reports the engine is ready (second try).');}
catch(second){constsaid2=humanStartError(second);instance.status='error';instance.error=said2;/* the first stop and the second try stay in job.details; the pop-up shows the last stop once */emit();thrownewError(instance.error);}}}
proc.on('error',error=>{instance.status='error';instance.error=error.message;emit();});proc.on('exit',code=>{if(instances.has(id)&&instance.status!=='unloading'){instance.status='error';instance.error=`Engine stopped${code===null?'':` (exit ${code})`}. Open diagnostics or reload.`;emit();}});emit();
{id:'mcp_max_result_chars',label:'Maximum tool result characters',group:'Tools',type:'number',min:100,max:1000000},
];
/** Engine environment names the app shows and edits: STRATA_* and the DeepSeek engine's DS4_*. Any other name in a model's file is kept untouched and not shown. */
{id:'faster_long_prompts',label:'Faster long prompts',group:'Speed',
hint:'Reads very long prompts faster by approximating part of the work. Off by default: answers about long documents can miss or mix up details.',
default:'off',env:{STRATA_KVA:'1'}},
{id:'faster_12gb_nvidia',label:'Faster answers on 12 GB NVIDIA cards',group:'Speed',
hint:'Prefers the parts of the model already on the graphics card, which makes answers much faster on 12 GB cards. Turn off if answers look worse than before.',
default:'on',env:{STRATA_ROUTE_PRIOR:'0'}},
{id:'faster_radeon_9000',label:'Faster answers on Radeon RX 9000 / AI PRO R9700',group:'Speed',
hint:'Shares the work between the graphics card and the processor in a way that makes answers faster on these cards. Turn off to go back to the previous setting, which is slower.',
hint:'Reads long prompts faster and loads the parts of the model it needs faster. Turn off if DeepSeek stops working or its answers look wrong; it then runs as in version 0.2.6.',
*experts)andtheenginedeclinesitonanyother,so"Faster long prompts"isshownforamodelwithoutan"arch"only.
*/
/** A discrete NVIDIA card of 13 GiB or less: the engine's rule for residency-biased routing (not an integrated chip, not a card that shares system memory). */
if(id==='faster_12gb_nvidia')return!arch&&backend==='cuda'&&smallNvidia(gpus);// the engine's routing needs the Flash-Next shape (512 experts, top-10) and a CUDA build
/** The switch is outside its default when every entry it writes is in the environment with its value; anything less or other (a hand-edited file, STRATA_KVA=0) is the default state and is left alone. */
/** What the settings answer carries: each switch that applies, with its words, its default, the entries it writes outside the default, and whether the feature is on. */
/** Settings the app itself applies when it starts a model (they are not read from the engine's source, so the engine's text never says it supports them). */
/** The engine arguments without the draft layer: --mtp <folder or count>, --spec-min-p <p> (its confidence threshold) and --batch-mtp. --spec stays: prompt lookup still uses its windows. Same rule as strata_control/draft_layer.py. */
@@ -40,7 +98,7 @@ export function validateNativeSettings(values){
for(const[key,value]ofObject.entries(values)){
if(key==='args'){if(!Array.isArray(value)||value.length>200||value.some(v=>typeofv!=='string'||v.length>4000||/[\n\r\0]/.test(v))||value.some(v=>['--host','--port','--api-key'].includes(v)))thrownewError('Engine arguments must be text values. Network access stays managed by Strata.');result.args=[...value];continue;}
if(key==='env'){if(!value||typeofvalue!=='object'||Array.isArray(value))thrownewError('Engine environment must be an object.');constenv={};for(const[name,text]ofObject.entries(value)){if(!/^STRATA_[A-Z0-9_]+$/.test(name)||typeoftext!=='string'||text.length>4000||text.includes('\0'))thrownewError('Use STRATA_ environment flags with text values.');env[name]=text;}result.env=env;continue;}
if(key==='env'){if(!value||typeofvalue!=='object'||Array.isArray(value))thrownewError('Engine environment must be an object.');constenv={};for(const[name,text]ofObject.entries(value)){if(!ENV_FLAG.test(name)||typeoftext!=='string'||text.length>4000||text.includes('\0'))thrownewError('Use STRATA_ environment flags with text values.');env[name]=text;}result.env=env;continue;}
if(key==='sampling'){if(!value||typeofvalue!=='object'||Array.isArray(value))thrownewError('Sampling must be an object.');constlimits={temperature:[0,10],top_p:[Number.MIN_VALUE,1],top_k:[1,64],min_p:[0,1],repetition_penalty:[Number.MIN_VALUE,10],presence_penalty:[0,10],frequency_penalty:[0,10],penalty_last_n:[0,1048576],seed:[1,2147483647]};for(const[name,n]ofObject.entries(value)){if(name==='experimental_speed_projection'){if(typeofn!=='boolean')thrownewError('Speed projection must be on or off.');continue;}if(!limits[name]||!Number.isFinite(n)||(['top_k','seed','penalty_last_n'].includes(name)&&!Number.isInteger(n))||n<limits[name][0]||n>limits[name][1])thrownewError(`Choose a valid sampling ${name}.`);}result.sampling={...value};continue;}
exportfunctionrestoreNativeEnv(env,previous={}){constresult=Object.fromEntries(Object.entries(previous).filter(([name])=>!/^STRATA_[A-Z0-9_]+$/.test(name)));for(const[name,value]ofObject.entries(env)){if(value===MASK){if(previous[name]===undefined)thrownewError(`Enter a new value for ${name}.`);result[name]=previous[name];}elseresult[name]=value;}returnresult;}
exportfunctionrestoreNativeEnv(env,previous={}){constresult=Object.fromEntries(Object.entries(previous).filter(([name])=>!ENV_FLAG.test(name)));for(const[name,value]ofObject.entries(env)){if(value===MASK){if(previous[name]===undefined)thrownewError(`Enter a new value for ${name}.`);result[name]=previous[name];}elseresult[name]=value;}returnresult;}
<inputclassName="st-input"inputMode="decimal"size={7}aria-label={tr("Price per kWh")}placeholder="0.30"value={text}onChange={e=>setText(e.target.value)}onBlur={commit}onKeyDown={e=>{if(e.key==='Enter')commit();}}/>
<spanclassName="small muted">{money.currency?tr("per kWh, in {0}",[currencyLabel(money.currency,money.locale)]):tr("per kWh")}</span>
<p>{tr("This is a very large model. The download is {0} GB and it needs about as much free disk space. It runs on one 12 GB graphics card with 46 to 64 GB of RAM, slowly: about 5 to 7 words a second. What does not fit in memory is read from your SSD, so an NVMe drive matters.",[Math.round(version.download_gb)])}</p>
<p>{tr("This is a very large model. The download is {0} GB and it needs about as much free disk space. It runs on NVIDIA RTX 30, 40 and 50 cards, on a Radeon AI PRO R9700 (Linux) or on a Ryzen AI Max / Strix Halo PC (Windows); how fast it is depends on the memory and card (the line below says what to expect on this computer). What does not fit in memory is read from your SSD, so an NVMe drive matters.",[Math.round(version.download_gb)])}</p>
<pclassName="small muted">{tr("You can stop the download and continue later; it picks up where it stopped. Strata warns you here but never blocks you.")}</p>
constnoFit=selectedSize?.on_this_pc==='does not fit';
// Graphics cards (strata_control/gpu_fit.py: the one rule of the app, the Terminal and setup): the recommended option is pre-selected, the person may pick another
// A card a loaded model already uses (a second model on it usually does not fit): said on its row, and not the pre-selected one while another is free.
{tr(noFit?<pclassName="small muted native-spill-note"><Iconname="disk"size={16}/>{tr("May run slower \u00B7 Model data can be read from SSD.")}</p>:null)}
<divclassName="native-choice__col"><spanclassName="native-choice__label">{tr("Computer use")}</span><divclassName="native-resource-choice"><buttonaria-pressed={resourceProfile==='work'}data-selected={resourceProfile==='work'}onClick={()=>setResourceProfile('work')}><Iconname="computer"size={20}/><span><strong>{tr("Leave 20% for my work")}</strong><small>{tr("Headroom for your other apps")}</small></span>{tr(resourceProfile==='work'?<Iconname="check"size={15}/>:null)}</button><buttonaria-pressed={resourceProfile==='performance'}data-selected={resourceProfile==='performance'}onClick={()=>setResourceProfile('performance')}><Iconname="bolt"size={20}/><span><strong>{tr("Full performance")}</strong><small>{tr("Give Strata priority")}</small></span>{tr(resourceProfile==='performance'?<Iconname="check"size={15}/>:null)}</button></div></div>
{tr(expert?<sectionid="native-expert-settings"ref={expertSection}className="st-card native-execution"><h2className="h4">{tr("Expert settings")}</h2>{tr(tunedNow?<pclassName="small muted native-tuned-note">{tr("Showing the values of your tuning '{0}'.",[tunedNow.name])}</p>:null)}<labelclassName="native-toggle"><span><strong>{tr("Run multiple models")}</strong><small>{tr("Loading another model keeps the others loaded.")}</small></span><buttonclassName="st-toggle"role="switch"aria-label={tr("Run multiple models")}aria-checked={multiple}onClick={()=>setMultiple(v=>!v)}/></label>{tr(cardsGpus.length>1?<labelclassName="input-field"><span>{tr("Layer split")}</span><inputclassName="st-input"value={split}placeholder={tr("auto or 16,32")}onChange={e=>setSplit(e.target.value)}/></label>:null)}<divclassName="native-field-grid"><labelclassName="input-field"><span>{tr("Context memory")}</span><SelectariaLabel={tr("Native KV format")}value={kv}options={[{value:'int8',label:'INT8'},{value:'q4_0',label:'Q4'},{value:'k8v4',label:'K8 / V4'}]}onChange={setKv}/></label><labelclassName="input-field"><span>{tr("Low-RAM mode")}</span><SelectariaLabel={tr("Native low RAM mode")}value={lowRam}options={['auto','resident','mmap'].map(v=>({value: v,label: v==='auto'?'Automatic':v==='resident'?'Keep experts in RAM':'Read experts from disk'}))}onChange={setLowRam}/></label></div>{tr(installed?<><NativeSettingsFilemodel={installed.model}/><NativeSettingsmodel={installed.model}/></>:<pclassName="small muted">{tr("More engine parameters become available after preparing this version.")}</p>)}</section>:null)}
</div>
<asideclassName="stack stack--lg"><PcFitCardgpuName={pcGpu?.name}ramGb={pcRam}vramGb={pcVram}inUse={keptModels.length?{ramGb: Math.round(keptRam),vramGb: Math.round(keptVramOn(chosenCards))}:null}fit={selectedSize?fitOn(selectedSize,fitHardware,true,{context,kv}):null}/><sectionclassName="st-card"><h2className="h4">{tr("Model files")}</h2><pclassName="small muted">{tr("Strata prepares all shards, the draft layer and shared files. Completed downloads are reused.")}</p><codeclassName="native-path">{installed?.dataFolder??native?.downloadDirectory??native?.installed?.data_folder??'Choose an installation'}</code><Btnsize="sm"variant="ghost"onClick={()=>{setSettingsSection('Storage');go('settings');}}>{tr("Storage settings")}</Btn></section><Btnsize="sm"variant="ghost"onClick={onOtherModels}>{tr("Other local models\u2026")}</Btn></aside>
<asideclassName="stack stack--lg"><PcFitCardgpuName={pcGpu?.name}ramGb={pcRam}vramGb={pcVram}diskGb={selectedSize?.multi_file?selectedSize.disk_gb??selectedSize.download_gb : null}inUse={keptModels.length?{ramGb: Math.round(keptRam),vramGb: Math.round(keptVramOn(chosenCards))}:null}fit={selectedSize?fitOn(selectedSize,fitHardware,true,{context,kv}):null}/><sectionclassName="st-card"><h2className="h4">{tr("Model files")}</h2><pclassName="small muted">{tr("Strata prepares all shards, the draft layer and shared files. Completed downloads are reused.")}</p><codeclassName="native-path">{installed?.dataFolder??native?.downloadDirectory??native?.installed?.data_folder??'Choose an installation'}</code><Btnsize="sm"variant="ghost"onClick={()=>{setSettingsSection('Storage');go('settings');}}>{tr("Storage settings")}</Btn></section><Btnsize="sm"variant="ghost"onClick={onOtherModels}>{tr("Other local models\u2026")}</Btn></aside>
</div>
<Dialogopen={!!plan}title={tr(installed?.ready?"Apply settings and load":"Download and prepare")}compactonClose={()=>!pending&&setPlan(null)}actions={<><Btndisabled={pending}onClick={()=>setPlan(null)}>{tr("Cancel")}</Btn><Btnvariant="primary"disabled={pending||!!plan?.plan?.disk_short}onClick={()=>voiddownload()}>{tr(installed?.ready?"Apply & load":"Download & load")}</Btn></>}><p><strong>{tr(plan?.plan?.title??selectedFamily?.title)}</strong>{tr(installed?.ready?null:<>{tr(" \u00B7 approximately ")}{tr(plan?.plan?.download_gb??selectedSize?.download_gb)}{tr(" GB")}</>)}</p>{tr(plan?.drafting?.needed?<pclassName="small muted">{tr("Choosing DFlash2 also downloads its drafter: about {0} GB, once. If it does not fit in graphics memory, the model uses the MTP instead.",[plan.drafting.gb])}</p>:null)}{tr(installed?.ready?<pclassName="small muted">{tr("The model files are already on this computer. Nothing will be downloaded; only the settings you chose are applied.")}</p>:null)}{tr(plan?.plan?.disk_short?<ErrorNoticemessage={plan.plan.disk_short}/>:null)}{tr(installed?.ready?<pclassName="small muted">{tr("Any currently loaded native models will be unloaded first.")}</p>:<><pclassName="small muted">{tr("Existing files are checked and reused. Any currently loaded native models will be unloaded before preparing files.")}</p><pclassName="small muted">{tr("Your computer may feel slower while weights load. You can stop the download and resume later.")}</p></>)}{tr(selectedSize?.images?<pclassName="small muted">{tr("Vision encoder files are included. Picture reading loads only when enabled.")}</p>:null)}</Dialog>
/** One of the speed features: what it is called, what it does, whether it is on by default, and the engine environment it writes outside its default (server/native-settings.mjs). */
interfaceSpeedSwitch{
id: string;
label: string;
hint: string;
default:'on'|'off';
env: Record<string,string>;
/** What an earlier test build wrote for the same state: it reads as set, and goes with it. */
@@ -97,8 +115,9 @@ export function NativeSettings({ model }: {
{error?<><ErrorNoticemessage={tr('Couldn’t load model settings.')}next={tr('Check that the engine is connected and this model is installed, then try again.')}onRetry={()=>setRevision(value=>value+1)}/><detailsclassName="native-settings-diagnostic"><summaryclassName="small">{tr('Details')}</summary><pclassName="small muted">{error}</p></details></>:<pclassName="small muted"role="status">{tr('Reading saved settings…')}</p>}
return<detailsclassName="native-settings"><summary>{tr("Native engine settings")}</summary>{tr(tuned?.changed?.length?<divclassName="native-tuned"><pclassName="small"><strong>{tr("Your tuning '{0}' sets",[tuned.preset?.name??''])}</strong></p><ul>{tuned.changed.map(c=>{constd=describeChange(c);return<likey={c.flag}><span>{d.name}</span><b>{d.now||tr('off')}</b>{d.was!==null?<smallclassName="muted">{tr("was {0}",[d.was])}</small>:null}</li>;})}</ul></div>:null)}<pclassName="small muted">{tr(!original?'Reading saved settings… ':'')}{tr("Settings for this model. Blank values use Strata\u2019s defaults.")}</p><divclassName="editor-tabs"role="tablist"aria-label={tr("Native settings editor")}>{tr(['fields','json','tools'].map(t=><buttonkey={t}role="tab"aria-selected={tab===t}onClick={()=>chooseTab(t)}>{tr(t==='fields'?'Parameters':t==='json'?'JSON':'MCP tools')}</button>))}</div>
{tr(tab==='fields'?<><inputclassName="st-input"type="search"aria-label={tr("Search native parameters")}placeholder={tr("Search parameters…")}value={query}onChange={e=>setQuery(e.target.value)}/>{tr(['Responses','GPU & memory','Serving','Image reading','Tools','Recovery'].map(group=>rows.some(f=>f.group===group)?<sectionclassName="native-settings-group"key={group}><h3className="h4">{tr(group)}</h3>{tr(rows.filter(f=>f.group===group).map(f=><labelclassName="native-setting-row"key={f.id}><span><strong>{tr(f.label)}</strong>{tr(f.supported===false?<small>{tr("Requires a newer Strata installation.")}</small>:f.hint?<small>{tr(f.hint)}</small>:null)}</span><NativeFieldInputfield={f}value={values[f.id]}onChange={v=>change(f.id,v)}/></label>))}</section>:null))}<sectionclassName="native-settings-group"><h3className="h4">{tr("Sampling")}</h3>{tr([['temperature','Temperature'],['top_p','Top-p'],['top_k','Top-k'],['min_p','Min-p'],['repetition_penalty','Repetition penalty'],['presence_penalty','Presence penalty'],['frequency_penalty','Frequency penalty'],['penalty_last_n','Penalty window'],['seed','Seed']].map(([id,label])=><labelclassName="native-setting-row"key={id}><span><strong>{tr(label)}</strong></span><DecimalInputariaLabel={tr("Native {0}",[label])}value={values.sampling?.[id]??null}onChange={n=>{constsampling={...values.sampling};if(n===null)deletesampling[id];elsesampling[id]=n;change('sampling',sampling);}}/></label>))}</section></>:tab==='json'?<><pclassName="small muted">{tr("Engine arguments and STRATA environment flags are editable here. Program paths and network access remain managed by the host.")}</p><textareaclassName="st-input native-json"aria-label={tr("Native configuration JSON")}spellCheck={false}value={text}onChange={e=>{setText(e.target.value);setNotice('');}}/></>:<><pclassName="small muted">{tr("Add local programs or HTTP MCP servers. Connected tools can read or change the resources you give them access to. Enable tools in chat when you want the model to use them.")}</p><NativeMcpEditortext={mcp}onChange={value=>{setMcp(value);setNotice('');}}/><details><summaryclassName="small">{tr("Configuration example")}</summary><preclassName="native-example">{'{\n "files": {\n "command": "npx",\n "args": ["-y", "@modelcontextprotocol/server-filesystem", "/your/folder"],\n "disabled": false\n }\n}'}</pre><pclassName="xs muted">{tr("Saved credentials are masked. Keep the mask to preserve an existing value; type a new value to replace it.")}</p></details></>)}
{tr(tab==='fields'?<><inputclassName="st-input"type="search"aria-label={tr("Search native parameters")}placeholder={tr("Search parameters…")}value={query}onChange={e=>setQuery(e.target.value)}/>{tr(['Responses','GPU & memory','Serving','Image reading','Tools','Recovery'].map(group=>rows.some(f=>f.group===group)?<sectionclassName="native-settings-group"key={group}><h3className="h4">{tr(group)}</h3>{tr(rows.filter(f=>f.group===group).map(f=><labelclassName="native-setting-row"key={f.id}><span><strong>{tr(f.label)}</strong>{tr(f.supported===false?<small>{tr("Requires a newer Strata installation.")}</small>:f.hint?<small>{tr(f.hint)}</small>:null)}</span><NativeFieldInputfield={f}value={values[f.id]}onChange={v=>change(f.id,v)}/></label>))}</section>:null))}{tr(switchRows.length?<sectionclassName="native-settings-group"><h3className="h4">{tr("Speed")}</h3>{tr(switchRows.map(sw=><labelclassName="native-setting-row"key={sw.id}><span><strong>{tr(sw.label)}</strong><small>{tr(sw.hint)}</small></span><SelectariaLabel={tr(sw.label)}value={switchIsOn(sw,values.env)?'on':'off'}options={sw.default==='on'?[{value:'on',label:'On (recommended)'},{value:'off',label:'Off'}]:[{value:'off',label:'Off (recommended)'},{value:'on',label:'On'}]}onChange={v=>changeSwitch(sw,v==='on')}/></label>))}</section>:null)}<sectionclassName="native-settings-group"><h3className="h4">{tr("Sampling")}</h3>{tr([['temperature','Temperature'],['top_p','Top-p'],['top_k','Top-k'],['min_p','Min-p'],['repetition_penalty','Repetition penalty'],['presence_penalty','Presence penalty'],['frequency_penalty','Frequency penalty'],['penalty_last_n','Penalty window'],['seed','Seed']].map(([id,label])=><labelclassName="native-setting-row"key={id}><span><strong>{tr(label)}</strong></span><DecimalInputariaLabel={tr("Native {0}",[label])}value={values.sampling?.[id]??null}onChange={n=>{constsampling={...values.sampling};if(n===null)deletesampling[id];elsesampling[id]=n;change('sampling',sampling);}}/></label>))}</section></>:tab==='json'?<><pclassName="small muted">{tr("Engine arguments and STRATA environment flags are editable here. Program paths and network access remain managed by the host.")}</p><textareaclassName="st-input native-json"aria-label={tr("Native configuration JSON")}spellCheck={false}value={text}onChange={e=>{setText(e.target.value);setNotice('');}}/></>:<><pclassName="small muted">{tr("Add local programs or HTTP MCP servers. Connected tools can read or change the resources you give them access to. Enable tools in chat when you want the model to use them.")}</p><NativeMcpEditortext={mcp}onChange={value=>{setMcp(value);setNotice('');}}/><details><summaryclassName="small">{tr("Configuration example")}</summary><preclassName="native-example">{'{\n "files": {\n "command": "npx",\n "args": ["-y", "@modelcontextprotocol/server-filesystem", "/your/folder"],\n "disabled": false\n }\n}'}</pre><pclassName="xs muted">{tr("Saved credentials are masked. Keep the mask to preserve an existing value; type a new value to replace it.")}</p></details></>)}
{tr(inUse?<pclassName="xs muted">{tr("Models that stay loaded use about {0} GB of RAM and {1} GB of this card’s memory.",[inUse.ramGb,inUse.vramGb])}</p>:null)}
{tr(fit?.known?<pclassName="pc-note">{tr(fit.spillGb>=1?'About {0} GB will be read from the SSD, so answers can be a little slower.':'Fits fully in memory.',[Math.round(fit.spillGb)])}</p>:null)}
{tr(diskGb?<pclassName="pc-note">{tr("Needs about {0} GB of free disk space. What does not fit in memory is read from your SSD.",[Math.round(diskGb)])}</p>:null)}
{tr(fit?.known&&(diskGb?fit.spillGb>=1 : true)?<pclassName="pc-note">{tr(fit.spillGb>=1?'About {0} GB will be read from the SSD, so answers can be a little slower.':'Fits fully in memory.',[Math.round(fit.spillGb)])}</p>:null)}
return<buttonclassName="st-toggle"role="switch"aria-checked={shell.minimizeToTray}aria-label={tr("Minimise to tray")}onClick={()=>voidsave({tray:!shell.minimizeToTray})}/>;
}
/** The port the app's window and local address use (#3402). Empty lets Strata choose; the choice applies the next time Strata starts. */
<strong>{selected&&!apart.has(selected.id)?versionName(selected,true):tr(moreLabel??"Choose a version")}</strong>{tr(selected&&!apart.has(selected.id)&&downloaded?.has(selected.id)?<spanclassName="version-row__have">{tr("Downloaded")}</span>:null)}
<spanclassName="version-trigger__need muted">{tr(fit&&selected?.multi_file&&selected.disk_gb?tr("{0} RAM + VRAM, plus about {1} GB of disk",[gb(fit.memoryGb),Math.round(selected.disk_gb)]):fit?`${gb(fit.memoryGb)} RAM + VRAM`:'')}</span>
if(started&&typeof(startedasPromise<void>).catch==='function')(startedasPromise<void>).catch(()=>{/* the browser asked for a click first: no sound this time */});
returntrue;
}catch{
returnfalse;
}
}
/** Called when an answer or a task is finished. */
/** The largest size not flagged as untested; also how far the list goes while the pack does not say more (a pack without ds4.txt behaves as before). */
exportconstTESTED_CONTEXT=262144;
/** The model's own length: the most any pack can say. */
exportconstNATIVE_CONTEXT=1048576;
exportconstUNTESTED_NOTE='Not tested yet. A context this long takes graphics memory from the model and reads long prompts slowly.';
/** The sizes to offer: up to the pack's limit (`contextMax`, from its ds4.txt) when the installed model has one, else up to 256K. */
/* The Tune model dialog's strings live in tune-<lang>.json (apps/host/scripts/make-tune-locales.py) so other interface work does not collide with them. */
/* The tool-server, picture and agent-result strings of the tools and images fixes live in tools-<lang>.json for the same reason. */
/* The strings that had no key in the files above (#3479, #3825) and the 0.2.4.3 fixes live in fix-<lang>.json (scripts/make-fix-locales.py). */
exportinterfaceNativeSize{/** The first engine that reads this size's files (the catalog), and whether the installed engine is older: the screen then says "needs a newer engine" and offers the update. */engine_min?:string|null;needsNewerEngine?:boolean;id:string;model:string;about:string;download_gb:number;ram_needed_gb:number;experts_gb:number;on_this_pc:string|null;installed:boolean;gpu_options?:GpuChoice|null;images?:boolean;imagesCapable?:boolean;/** DeepSeek-V4.1-Flash and the like: several big files and an engine of their own. */multi_file?:boolean}
exportinterfaceNativeSize{/** The first engine that reads this size's files (the catalog), and whether the installed engine is older: the screen then says "needs a newer engine" and offers the update. */engine_min?:string|null;needsNewerEngine?:boolean;id:string;model:string;about:string;download_gb:number;ram_needed_gb:number;experts_gb:number;on_this_pc:string|null;installed:boolean;gpu_options?:GpuChoice|null;images?:boolean;imagesCapable?:boolean;/** DeepSeek-V4.1-Flash and the like: several big files and an engine of their own. */multi_file?:boolean;/** DeepSeek: the free disk the set needs (its download plus the pack and room to work), from the set itself. */disk_gb?:number}
visionReader?: boolean;available:boolean;repo:string;downloadDirectory?:string;/** Free bytes on the drive that holds the models folder. */downloadFreeBytes?:number|null;downloadTotalBytes?:number|null;version?:string|null;summary:string;error:string|null;catalog:{families:NativeFamily[];contexts:number[];recommendation:{family:string|null;model:string|null;context?:number;why?:string};context_note:string}|null;installed:{models:{model:string;config:string;model_name:string;ready:boolean;/** The picture-reading files are on this computer (notes-split-agent8.md, item 8). */visionPrepared?:boolean;context?:number;kv?:string;images?:string;visionEnabled?:boolean;visionDevice?:string;lowRamMode?:string;resourceProfile?:'work'|'performance';/** The Drafting choice saved with the model: auto | mtp | dflash2 | off. */drafting?:string;dataFolder?:string;diskGb?:number;diskBytes?:number;/** Files this model shares with another (the picture reader, the draft layer): counted once, in `usage`. */sharedBytes?:number;/** What Delete frees: files in Strata's own folders that no other model uses. */removableGb?:number;missing_files?:string[]}[];/** What all models take on disk, from their real files, shared files counted once. */usage?:{total_bytes:number;total_gb:number;shared_bytes:number;shared_gb:number;models:Record<string,{bytes:number;gb:number}>}|null;data_folder:string;engine:{version:string|null}}|null;hardware:{ram_gb:number;engine_vendor?:string;gpus:{index:number;name:string;vram_gb:number;usable:boolean;/** SMs x clock: only the order between cards matters (strata_control/gpu_roles.py). */speed?:number;vendor?:string;uma?:boolean;integrated?:boolean;unified_gb?:number;problem?:string|null}[]}|null;fields:NativeField[];instances:NativeInstance[];defaultId:string|null;jobs:NativeJob[];capabilities:{/** The engine serves DFlash2 (Control says): the Drafting choice then offers it. */dflash2?:boolean;responses?:boolean;updates:boolean;rollback:boolean;calibration:boolean;componentUpdates?:boolean}}
visionReader?: boolean;available:boolean;repo:string;downloadDirectory?:string;/** Free bytes on the drive that holds the models folder. */downloadFreeBytes?:number|null;downloadTotalBytes?:number|null;version?:string|null;summary:string;error:string|null;catalog:{families:NativeFamily[];contexts:number[];recommendation:{family:string|null;model:string|null;context?:number;why?:string};context_note:string}|null;installed:{models:{model:string;config:string;model_name:string;ready:boolean;/** The picture-reading files are on this computer (notes-split-agent8.md, item 8). */visionPrepared?:boolean;context?:number;kv?:string;images?:string;visionEnabled?:boolean;visionDevice?:string;lowRamMode?:string;resourceProfile?:'work'|'performance';/** The Drafting choice saved with the model: auto | mtp | dflash2 | off. */drafting?:string;/** DeepSeek: the most context its pack reads (max_position_embeddings in the pack's ds4.txt); absent when the pack does not say. */contextMax?:number;dataFolder?:string;diskGb?:number;diskBytes?:number;/** Files this model shares with another (the picture reader, the draft layer): counted once, in `usage`. */sharedBytes?:number;/** What Delete frees: files in Strata's own folders that no other model uses. */removableGb?:number;missing_files?:string[]}[];/** What all models take on disk, from their real files, shared files counted once. */usage?:{total_bytes:number;total_gb:number;shared_bytes:number;shared_gb:number;models:Record<string,{bytes:number;gb:number}>}|null;data_folder:string;engine:{version:string|null}}|null;hardware:{ram_gb:number;engine_vendor?:string;gpus:{index:number;name:string;vram_gb:number;usable:boolean;/** SMs x clock: only the order between cards matters (strata_control/gpu_roles.py). */speed?:number;vendor?:string;uma?:boolean;integrated?:boolean;unified_gb?:number;problem?:string|null}[]}|null;fields:NativeField[];instances:NativeInstance[];defaultId:string|null;jobs:NativeJob[];capabilities:{/** The engine serves DFlash2 (Control says): the Drafting choice then offers it. */dflash2?:boolean;responses?:boolean;updates:boolean;rollback:boolean;calibration:boolean;componentUpdates?:boolean}}
letstate:NativeState|null=null;
constlisteners=newSet<()=>void>();
/** True once the first read of the native state has ended, whatever its answer: until then the Models page must not pick a model of its own (#3740). */
/** Chrome scale, independent of the conversation's reading size. */
uiScale: number;
/** Price of one kWh for the Monitor's cost line (0 = not set) and the currency the person picked instead of the detected one ('' = automatic) (#3628). */
electricityPrice: number;
electricityCurrency: string;
/** Opt-in: a short sound when a task finishes (#3586). */
doneSound: boolean;
/** Opt-in: a desktop notification when Manual mode waits for an approval (#3477). */
if(shell.portNotice)pushToast('warn',tr('Strata could not use its port'),tr('Port {0} is used by another program, so Strata started on port {1}. Chats and settings saved on port {0} are not shown until Strata can use it again. Close the other program and restart Strata.',[shell.portNotice.wanted,shell.portNotice.used]));
}catch{/* a browser or an older host has no shell */}
/** The electricity price for the Monitor's cost line: a positive price per kWh; 0 turns the line off. The currency is the detected one unless a three-letter code is given (#3628). */
"fx71":"Schnellere Antworten auf Radeon RX 9000 / AI PRO R9700",
"fx72":"DeepSeek-Beschleunigungen",
"fx73":"Liest sehr lange Prompts schneller, indem ein Teil der Arbeit angenähert wird. Standardmäßig aus: Antworten zu langen Dokumenten können Einzelheiten auslassen oder verwechseln.",
"fx74":"Verteilt die Arbeit so zwischen Grafikkarte und Prozessor, dass Antworten auf diesen Karten schneller werden. Ausschalten stellt die vorherige, langsamere Einstellung wieder her.",
"fx75":"Liest lange Prompts schneller und lädt die benötigten Teile des Modells schneller. Schalten Sie die Option aus, wenn DeepSeek nicht mehr funktioniert oder die Antworten falsch aussehen; dann läuft es wie in Version 0.2.6.",
"fx76":"Ein (empfohlen)",
"fx77":"Das ist ein sehr großes Modell. Der Download umfasst {0} GB und braucht etwa so viel freien Speicherplatz. Es läuft auf NVIDIA-RTX-Karten der Serien 30, 40 und 50, auf einer Radeon AI PRO R9700 (Linux) oder auf einem Ryzen-AI-Max-/Strix-Halo-PC (Windows); wie schnell es ist, hängt von Speicher und Grafikkarte ab (die Zeile darunter sagt, was auf diesem Computer zu erwarten ist). Was nicht in den Speicher passt, wird von deiner SSD gelesen, deshalb zählt eine NVMe-SSD.",
"fx78":"Noch nicht getestet. Ein so langer Kontext nimmt dem Modell Grafikspeicher weg, und lange Prompts werden langsam gelesen.",
"fx79":"Aus (empfohlen)",
"fx80":"Braucht etwa {0} GB freien Speicherplatz. Was nicht in den Speicher passt, wird von Ihrer SSD gelesen.",
"fx81":"{0} RAM + VRAM, dazu etwa {1} GB Speicherplatz",
"fx82":"Schnellere Antworten auf NVIDIA-Karten mit 12 GB",
"fx83":"Bevorzugt die Teile des Modells, die schon auf der Grafikkarte liegen. Das macht Antworten auf Karten mit 12 GB deutlich schneller. Schalten Sie die Option aus, wenn die Antworten schlechter aussehen als vorher."
"fx71":"Faster answers on Radeon RX 9000 / AI PRO R9700",
"fx72":"DeepSeek speed-ups",
"fx73":"Reads very long prompts faster by approximating part of the work. Off by default: answers about long documents can miss or mix up details.",
"fx74":"Shares the work between the graphics card and the processor in a way that makes answers faster on these cards. Turn off to go back to the previous setting, which is slower.",
"fx75":"Reads long prompts faster and loads the parts of the model it needs faster. Turn off if DeepSeek stops working or its answers look wrong; it then runs as in version 0.2.6.",
"fx76":"On (recommended)",
"fx77":"This is a very large model. The download is {0} GB and it needs about as much free disk space. It runs on NVIDIA RTX 30, 40 and 50 cards, on a Radeon AI PRO R9700 (Linux) or on a Ryzen AI Max / Strix Halo PC (Windows); how fast it is depends on the memory and card (the line below says what to expect on this computer). What does not fit in memory is read from your SSD, so an NVMe drive matters.",
"fx78":"Not tested yet. A context this long takes graphics memory from the model and reads long prompts slowly.",
"fx79":"Off (recommended)",
"fx80":"Needs about {0} GB of free disk space. What does not fit in memory is read from your SSD.",
"fx81":"{0} RAM + VRAM, plus about {1} GB of disk",
"fx82":"Faster answers on 12 GB NVIDIA cards",
"fx83":"Prefers the parts of the model already on the graphics card, which makes answers much faster on 12 GB cards. Turn off if answers look worse than before."
"fx67":"Predeterminado (Calidad completa en este PC)",
"fx68":"Más rápido (predeterminado en este PC)",
"fx69":"Calidad completa (predeterminado en este PC)"
"fx69":"Calidad completa (predeterminado en este PC)",
"fx70":"Prompts largos más rápidos",
"fx71":"Respuestas más rápidas en Radeon RX 9000 / AI PRO R9700",
"fx72":"Aceleraciones de DeepSeek",
"fx73":"Lee prompts muy largos más rápido aproximando una parte del trabajo. Desactivado de forma predeterminada: las respuestas sobre documentos largos pueden omitir o confundir detalles.",
"fx74":"Reparte el trabajo entre la tarjeta gráfica y el procesador de una forma que acelera las respuestas en estas tarjetas. Si lo desactivas, vuelve el ajuste anterior, que es más lento.",
"fx75":"Lee los prompts largos más rápido y carga más rápido las partes del modelo que necesita. Desactívalo si DeepSeek deja de funcionar o sus respuestas parecen incorrectas; entonces funciona como en la versión 0.2.6.",
"fx76":"Activado (recomendado)",
"fx77":"Es un modelo muy grande. La descarga es de {0} GB y necesita casi el mismo espacio libre en disco. Funciona con tarjetas NVIDIA RTX de las series 30, 40 y 50, con una Radeon AI PRO R9700 (Linux) o en un PC con Ryzen AI Max / Strix Halo (Windows); su velocidad depende de la memoria y de la tarjeta (la línea de abajo indica qué esperar en este equipo). Lo que no cabe en memoria se lee de tu SSD, así que importa que sea NVMe.",
"fx78":"Aún sin probar. Un contexto tan largo le quita memoria gráfica al modelo y lee los prompts largos con lentitud.",
"fx79":"Desactivado (recomendado)",
"fx80":"Necesita unos {0} GB de espacio libre en disco. Lo que no cabe en memoria se lee de tu SSD.",
"fx81":"{0} de RAM + VRAM, más unos {1} GB de disco",
"fx82":"Respuestas más rápidas en tarjetas NVIDIA de 12 GB",
"fx83":"Prefiere las partes del modelo que ya están en la tarjeta gráfica, lo que hace las respuestas mucho más rápidas en tarjetas de 12 GB. Desactívalo si las respuestas parecen peores que antes."
"fx02":"Téléchargé, mais pas chargé. Ouvrez Modèles pour le charger.",
"fx03":"{0} lignes ajoutées, {1} supprimées",
"fx04":"{0} fichiers modifiés",
"fx05":"Déposez des fichiers pour les ajouter",
"fx06":"Images et fichiers texte",
"fx07":"Garder l’objectif",
"fx08":"Supprimer l’objectif",
"fx09":"L’assistant cesse de travailler à l’atteindre. Votre conversation et vos fichiers restent comme ils sont.",
"fx10":"Ce modèle peut lire les images, mais le moteur installé ici n’inclut pas encore le lecteur d’images: Lire les images est donc désactivé. Mettez à jour le moteur dans Paramètres → À propos pour l’activer.",
"fx11":"Affichage des valeurs de votre optimisation « {0} ».",
"fx12":"Votre optimisation « {0} » règle",
"fx13":"était {0}",
"fx14":"· {0} de RAM",
"fx15":"Le message n’a pas été annulé",
"fx16":"Réglages supplémentaires pour le programme",
"fx17":"Rédiger avec l’IA",
"fx18":"Temps de réflexion",
"fx19":"Temps de rédaction",
"fx20":"tokens/s",
"fx21":"% du contexte",
"fx22":"Cette page demande des fichiers à internet, que l’aperçu ne charge pas: elle peut donc paraître incomplète. Voir le code montre exactement ce qui a été écrit.",
"fx23":"Garder A",
"fx24":"Garder B",
"fx25":"A pris",
"fx26":"Variante",
"fx27":"Modifié",
"fx28":"Rien",
"fx29":"Les deux versions sont prêtes. Comparez A et B, puis gardez-en une.",
"fx30":"L’assistant répond encore",
"fx31":"Envoyez votre message quand il a terminé.",
"fx32":"Le message n’a pas été mis en file d’attente",
"fx33":"Aller au dernier message",
"fx34":"Retirer la tâche",
"fx35":"Ouvrir une nouvelle conversation; garder celle-ci",
"fx36":"Faire d’abord rédiger un plan à l’assistant; rien ne s’exécute tant que vous ne l’avez pas approuvé",
"fx37":"Chercher sur le web, lire les pages et répondre avec des sources",
"fx38":"Copié",
"fx39":"Réponse",
"fx40":"Message",
"fx41":"PDF ajouté",
"fx42":"Sélectionnez le texte et copiez-le avec Ctrl+C.",
"fx43":"{0} est dans cette conversation. Posez une question à son sujet dans votre prochain message.",
"fx44":"Le moteur est prêt.",
"fx45":"Étape suivante: choisir un modèle.",
"fx46":"Téléchargement du moteur",
"fx47":"Vous pouvez fermer Strata; le téléchargement reprendra là où il s’est arrêté.",
"fx48":"Bienvenue dans Strata",
"fx49":"Examen de votre carte graphique…",
"fx50":"Strata a trouvé votre carte graphique:",
"fx51":"{0} cartes trouvées. Strata utilise la première pour le moteur; vous pourrez changer cela plus tard dans les Paramètres.",
"fx52":"Strata n’a pas reconnu de carte graphique prise en charge.",
"fx53":"Pour y exécuter des modèles, Strata a besoin d’un téléchargement:",
"fx54":"Le moteur pour cette carte",
"fx55":"Le moteur a besoin d’environ {0} Go d’espace libre sur ce disque et il n’y en a que {1} Go. Libérez de l’espace, puis continuez.",
"fx56":"Il vient du serveur de mise à jour de Strata et il est vérifié avant utilisation. Vos modèles et vos conversations ne sont jamais touchés.",
"fx57":"Strata n’a pas pu joindre github.com. Vérifiez votre connexion et réessayez, ou téléchargez le fichier du moteur sur un autre PC et choisissez-le ici.",
"fx58":"Le moteur installé ici est prévu pour une autre marque de carte graphique.",
"fx59":"Strata ne peut pas choisir de moteur pour ce PC dans sa liste de mises à jour.",
"fx60":"Continuer sans moteur",
"fx61":"Le téléchargement s’est arrêté. Continuer le reprend là où il s’est arrêté.",
"fx62":"C’est un très gros modèle. Le téléchargement fait {0} Go et il faut à peu près autant d’espace disque libre. Il tourne sur une carte graphique de 12 Go avec 46 à 64 Go de RAM, lentement: environ 5 à 7 mots par seconde. Ce qui ne tient pas en mémoire est lu depuis votre SSD, un disque NVMe compte donc beaucoup.",
"fx63":"Vous pouvez arrêter le téléchargement et le continuer plus tard; il reprend là où il s’est arrêté. Strata vous avertit ici mais ne vous bloque jamais.",
"fx64":"Plus rapide",
"fx65":"Qualité maximale",
"fx66":"Par défaut (Plus rapide sur ce PC)",
"fx67":"Par défaut (Qualité maximale sur ce PC)",
"fx68":"Plus rapide (par défaut sur ce PC)",
"fx69":"Qualité maximale (par défaut sur ce PC)",
"fx70":"Longs prompts plus rapides",
"fx71":"Réponses plus rapides sur Radeon RX 9000 / AI PRO R9700",
"fx72":"Accélérations DeepSeek",
"fx73":"Lit les très longs prompts plus vite en approximant une partie du travail. Désactivé par défaut: les réponses sur de longs documents peuvent manquer ou mélanger des détails.",
"fx74":"Répartit le travail entre la carte graphique et le processeur d’une façon qui accélère les réponses sur ces cartes. Désactivez pour revenir au réglage précédent, plus lent.",
"fx75":"Lit les longs prompts plus vite et charge plus vite les parties du modèle dont il a besoin. Désactivez si DeepSeek cesse de fonctionner ou si ses réponses semblent fausses; il fonctionne alors comme dans la version 0.2.6.",
"fx76":"Activé (recommandé)",
"fx77":"C’est un très gros modèle. Le téléchargement fait {0} Go et il faut à peu près autant d’espace disque libre. Il tourne sur les cartes NVIDIA RTX 30, 40 et 50, sur une Radeon AI PRO R9700 (Linux) ou sur un PC Ryzen AI Max / Strix Halo (Windows); sa vitesse dépend de la mémoire et de la carte (la ligne ci-dessous indique à quoi s’attendre sur cet ordinateur). Ce qui ne tient pas en mémoire est lu depuis votre SSD, un disque NVMe compte donc beaucoup.",
"fx78":"Pas encore testé. Un contexte aussi long prend de la mémoire graphique au modèle et lit lentement les longs prompts.",
"fx79":"Désactivé (recommandé)",
"fx80":"Il faut environ {0} Go d’espace disque libre. Ce qui ne tient pas en mémoire est lu depuis votre SSD.",
"fx81":"{0} de RAM + VRAM, plus environ {1} Go de disque",
"fx82":"Réponses plus rapides sur les cartes NVIDIA de 12 Go",
"fx83":"Privilégie les parties du modèle déjà présentes sur la carte graphique, ce qui accélère nettement les réponses sur les cartes de 12 Go. Désactivez-le si les réponses semblent moins bonnes qu’avant."
"fx77":"とても大きなモデルです。ダウンロードは {0} GB で、ほぼ同じ大きさの空き容量が必要です。NVIDIA RTX 30・40・50 シリーズ、Radeon AI PRO R9700（Linux）、または Ryzen AI Max / Strix Halo の PC（Windows）で動きます。速さはメモリとカードによって変わります（この PC での目安は下の行に表示されます）。メモリに入らない分は SSD から読むため、NVMe ドライブが重要です。",
"fx73":"작업의 일부를 근사 처리하여 매우 긴 프롬프트를 더 빠르게 읽습니다. 기본값은 꺼짐입니다. 긴 문서에 대한 답변에서 세부 내용이 빠지거나 뒤섞일 수 있습니다.",
"fx74":"이 그래픽 카드에서 응답이 빨라지도록 그래픽 카드와 프로세서가 작업을 나눕니다. 끄면 이전의 더 느린 설정으로 돌아갑니다.",
"fx75":"긴 프롬프트와 필요한 모델 부분을 더 빠르게 읽고 불러옵니다. DeepSeek가 멈추거나 답변이 이상해 보이면 끄세요. 그러면 0.2.6 버전처럼 동작합니다.",
"fx76":"켬(권장)",
"fx77":"매우 큰 모델입니다. 다운로드는 {0}GB이고, 거의 같은 크기의 여유 디스크 공간이 필요합니다. NVIDIA RTX 30·40·50 시리즈 카드, Radeon AI PRO R9700(Linux), 또는 Ryzen AI Max / Strix Halo PC(Windows)에서 실행됩니다. 속도는 메모리와 카드에 따라 달라집니다(아래 줄에 이 컴퓨터에서 기대할 수 있는 속도가 표시됩니다). 메모리에 들어가지 않는 부분은 SSD에서 읽으므로 NVMe 드라이브가 중요합니다.",
"fx78":"아직 테스트하지 않았습니다. 이렇게 긴 컨텍스트는 모델이 쓸 그래픽 메모리를 차지하고, 긴 프롬프트를 느리게 읽습니다.",
"fx79":"끔(권장)",
"fx80":"약 {0}GB의 여유 디스크 공간이 필요합니다. 메모리에 들어가지 않는 부분은 SSD에서 읽습니다.",
"fx81":"RAM + VRAM {0}, 디스크 약 {1}GB 추가",
"fx82":"12GB NVIDIA 카드에서 더 빠른 응답",
"fx83":"이미 그래픽 카드에 올라와 있는 모델 부분을 우선 사용해 12GB 카드에서 응답이 훨씬 빨라집니다. 답변이 이전보다 나빠 보이면 끄세요."
"fx71":"Respostas mais rápidas em Radeon RX 9000 / AI PRO R9700",
"fx72":"Acelerações do DeepSeek",
"fx73":"Lê prompts muito longos mais rápido aproximando parte do trabalho. Desativado por padrão: as respostas sobre documentos longos podem omitir ou misturar detalhes.",
"fx74":"Divide o trabalho entre a placa de vídeo e o processador de um jeito que deixa as respostas mais rápidas nessas placas. Ao desativar, volta a configuração anterior, que é mais lenta.",
"fx75":"Lê prompts longos mais rápido e carrega mais rápido as partes do modelo de que precisa. Desative se o DeepSeek parar de funcionar ou as respostas parecerem erradas; ele passa a funcionar como na versão 0.2.6.",
"fx76":"Ativado (recomendado)",
"fx77":"Este é um modelo muito grande. O download tem {0} GB e precisa de quase o mesmo espaço livre em disco. Ele roda em placas NVIDIA RTX das séries 30, 40 e 50, em uma Radeon AI PRO R9700 (Linux) ou em um PC com Ryzen AI Max / Strix Halo (Windows); a velocidade depende da memória e da placa (a linha abaixo diz o que esperar neste computador). O que não cabe na memória é lido do seu SSD, então um NVMe faz diferença.",
"fx78":"Ainda não testado. Um contexto tão longo tira memória gráfica do modelo e lê prompts longos devagar.",
"fx79":"Desativado (recomendado)",
"fx80":"Precisa de cerca de {0} GB de espaço livre em disco. O que não cabe na memória é lido do seu SSD.",
"fx81":"{0} de RAM + VRAM, mais cerca de {1} GB de disco",
"fx82":"Respostas mais rápidas em placas NVIDIA de 12 GB",
"fx83":"Prefere as partes do modelo que já estão na placa de vídeo, o que deixa as respostas bem mais rápidas em placas de 12 GB. Desative se as respostas parecerem piores do que antes."
"fx66":"По умолчанию (на этом компьютере — «Быстрее»)",
"fx67":"По умолчанию (на этом компьютере — «Полное качество»)",
"fx68":"Быстрее (по умолчанию на этом компьютере)",
"fx69":"Полное качество (по умолчанию на этом компьютере)"
"fx69":"Полное качество (по умолчанию на этом компьютере)",
"fx70":"Более быстрое чтение длинных запросов",
"fx71":"Более быстрые ответы на Radeon RX 9000 / AI PRO R9700",
"fx72":"Ускорения DeepSeek",
"fx73":"Читает очень длинные запросы быстрее, приближённо выполняя часть работы. По умолчанию выключено: в ответах по длинным документам могут пропадать или путаться детали.",
"fx74":"Распределяет работу между видеокартой и процессором так, что ответы на этих видеокартах получаются быстрее. Если отключить, вернётся прежняя, более медленная настройка.",
"fx75":"Быстрее читает длинные запросы и быстрее загружает нужные части модели. Отключите, если DeepSeek перестаёт работать или его ответы выглядят неверными; тогда он работает как в версии 0.2.6.",
"fx76":"Вкл. (рекомендуется)",
"fx77":"Это очень большая модель. Загрузка — {0} ГБ, и нужно почти столько же свободного места на диске. Она работает на видеокартах NVIDIA RTX серий 30, 40 и 50, на Radeon AI PRO R9700 (Linux) или на ПК с Ryzen AI Max / Strix Halo (Windows); скорость зависит от памяти и видеокарты (строка ниже подскажет, чего ждать на этом компьютере). То, что не помещается в память, читается с вашего SSD, поэтому важен накопитель NVMe.",
"fx78":"Ещё не проверено. Такой длинный контекст отнимает у модели видеопамять, а длинные запросы читаются медленно.",
"fx79":"Выкл. (рекомендуется)",
"fx80":"Нужно около {0} ГБ свободного места на диске. То, что не помещается в память, читается с вашего SSD.",
"fx81":"{0} RAM + VRAM, плюс около {1} ГБ на диске",
"fx82":"Более быстрые ответы на видеокартах NVIDIA с 12 ГБ",
"fx83":"Отдаёт предпочтение частям модели, которые уже находятся на видеокарте, поэтому ответы на картах с 12 ГБ становятся намного быстрее. Отключите, если ответы стали выглядеть хуже, чем раньше."
"gh08":"Preis einer Kilowattstunde. Der Monitor zeigt dann, was Ihre Grafikkarte seit dem Start von Strata gekostet hat. Leer lassen, um die Kosten auszublenden.",
"gh09":"Preis pro kWh",
"gh10":"Ton, wenn Arbeit fertig ist",
"gh11":"Spielt einen kurzen Ton, wenn eine Antwort oder Aufgabe, die länger gedauert hat, fertig ist oder wenn Strata im Hintergrund läuft.",
"gh12":"Ton abspielen",
"gh13":"Benachrichtigen, wenn Strata eine Antwort braucht",
"gh14":"Zeigt im manuellen Modus eine Desktop-Benachrichtigung, wenn der Assistent auf Ihre Freigabe wartet und Strata im Hintergrund läuft.",
"gh15":"Strata braucht Ihre Antwort",
"gh16":"Der Assistent möchte {0} verwenden. Öffnen Sie Strata, um zu erlauben oder abzulehnen.",
"gh17":"ein Werkzeug",
"gh18":"In den Infobereich minimieren",
"gh19":"Beim Schließen des Fensters läuft Strata im Infobereich neben der Uhr weiter. Wählen Sie im Menü des Symbols „Beenden“, um es zu stoppen. Gilt ab dem nächsten Start von Strata.",
"gh20":"App-Port",
"gh21":"Der Port, den das Fenster und die lokale Adresse von Strata verwenden. Leer lassen, damit Strata ihn wählt (47831, wenn frei). Gilt ab dem nächsten Start von Strata.",
"gh22":"Port (leer = automatisch)",
"gh23":"Aktuell wird Port {0} verwendet.",
"gh24":"Strata konnte seinen Port nicht verwenden",
"gh25":"Port {0} wird von einem anderen Programm verwendet, daher wurde Strata auf Port {1} gestartet. Unter Port {0} gespeicherte Chats und Einstellungen werden nicht angezeigt, bis Strata ihn wieder verwenden kann. Schließen Sie das andere Programm und starten Sie Strata neu.",
"gh26":"Wählen Sie einen Port zwischen 1024 und 65535 oder lassen Sie das Feld leer.",
"gh27":"Macht alle Texte, Schaltflächen und Menüs in Strata größer oder kleiner.",
"gh08":"Price of one kilowatt-hour. The Monitor then shows what your graphics card has cost since Strata started. Leave it empty to hide the cost.",
"gh09":"Price per kWh",
"gh10":"Sound when work finishes",
"gh11":"Plays a short sound when an answer or task that took a while is finished, or when Strata is in the background.",
"gh12":"Play sound",
"gh13":"Notify me when Strata needs an answer",
"gh14":"In Manual mode, shows a desktop notification when the assistant waits for your approval and Strata is in the background.",
"gh15":"Strata needs your answer",
"gh16":"The assistant wants to use {0}. Open Strata to approve or deny.",
"gh17":"a tool",
"gh18":"Minimise to tray",
"gh19":"Closing the window keeps Strata running in the tray, next to the clock. Choose Quit in the tray menu to stop it. Applies the next time Strata starts.",
"gh20":"App port",
"gh21":"The port Strata’s window and local address use. Leave it empty to let Strata choose (47831 when it is free). Applies the next time Strata starts.",
"gh22":"Port (empty = automatic)",
"gh23":"Using port {0} now.",
"gh24":"Strata could not use its port",
"gh25":"Port {0} is used by another program, so Strata started on port {1}. Chats and settings saved on port {0} are not shown until Strata can use it again. Close the other program and restart Strata.",
"gh26":"Choose a port between 1024 and 65535, or leave it empty.",
"gh27":"Makes all text, buttons and menus in Strata larger or smaller.",
"gh08":"Precio de un kilovatio hora. El Monitor mostrará entonces lo que ha costado tu tarjeta gráfica desde que se inició Strata. Déjalo vacío para ocultar el coste.",
"gh09":"Precio por kWh",
"gh10":"Sonido al terminar el trabajo",
"gh11":"Reproduce un sonido breve cuando termina una respuesta o tarea que tardó un rato, o cuando Strata está en segundo plano.",
"gh12":"Reproducir sonido",
"gh13":"Avisarme cuando Strata necesite una respuesta",
"gh14":"En el modo manual, muestra una notificación de escritorio cuando el asistente espera tu aprobación y Strata está en segundo plano.",
"gh15":"Strata necesita tu respuesta",
"gh16":"El asistente quiere usar {0}. Abre Strata para aprobar o denegar.",
"gh17":"una herramienta",
"gh18":"Minimizar a la bandeja",
"gh19":"Al cerrar la ventana, Strata sigue funcionando en la bandeja, junto al reloj. Elige Salir en el menú de la bandeja para detenerlo. Se aplica la próxima vez que se inicie Strata.",
"gh20":"Puerto de la aplicación",
"gh21":"El puerto que usan la ventana y la dirección local de Strata. Déjalo vacío para que Strata elija (47831 si está libre). Se aplica la próxima vez que se inicie Strata.",
"gh22":"Puerto (vacío = automático)",
"gh23":"Ahora se usa el puerto {0}.",
"gh24":"Strata no pudo usar su puerto",
"gh25":"El puerto {0} lo usa otro programa, así que Strata se inició en el puerto {1}. Los chats y ajustes guardados en el puerto {0} no se muestran hasta que Strata pueda volver a usarlo. Cierra el otro programa y reinicia Strata.",
"gh26":"Elige un puerto entre 1024 y 65535, o déjalo vacío.",
"gh27":"Hace más grandes o más pequeños todos los textos, botones y menús de Strata.",
"gh08":"Prix d’un kilowattheure. Le Moniteur affiche alors ce que votre carte graphique a coûté depuis le démarrage de Strata. Laissez vide pour masquer le coût.",
"gh09":"Prix par kWh",
"gh10":"Son à la fin du travail",
"gh11":"Joue un court son quand une réponse ou une tâche qui a pris du temps est terminée, ou quand Strata est en arrière-plan.",
"gh12":"Jouer le son",
"gh13":"Me prévenir quand Strata a besoin d’une réponse",
"gh14":"En mode manuel, affiche une notification sur le bureau quand l’assistant attend votre accord et que Strata est en arrière-plan.",
"gh15":"Strata a besoin de votre réponse",
"gh16":"L’assistant veut utiliser {0}. Ouvrez Strata pour accepter ou refuser.",
"gh17":"un outil",
"gh18":"Réduire dans la zone de notification",
"gh19":"Fermer la fenêtre garde Strata actif dans la zone de notification, près de l’horloge. Choisissez Quitter dans le menu de l’icône pour l’arrêter. S’applique au prochain démarrage de Strata.",
"gh20":"Port de l’application",
"gh21":"Le port que la fenêtre et l’adresse locale de Strata utilisent. Laissez vide pour que Strata choisisse (47831 s’il est libre). S’applique au prochain démarrage de Strata.",
"gh22":"Port (vide = automatique)",
"gh23":"Port {0} utilisé en ce moment.",
"gh24":"Strata n’a pas pu utiliser son port",
"gh25":"Le port {0} est utilisé par un autre programme, donc Strata a démarré sur le port {1}. Les conversations et réglages enregistrés sur le port {0} ne s’affichent pas tant que Strata ne peut pas le réutiliser. Fermez l’autre programme et relancez Strata.",
"gh26":"Choisissez un port entre 1024 et 65535, ou laissez vide.",
"gh27":"Agrandit ou réduit tous les textes, boutons et menus de Strata.",
"gh08":"1킬로와트시의 요금입니다. 모니터에 Strata를 시작한 뒤 그래픽 카드에 든 전기 요금이 표시됩니다. 비워 두면 비용을 표시하지 않습니다.",
"gh09":"kWh당 요금",
"gh10":"작업이 끝났을 때 소리",
"gh11":"답변이나 오래 걸린 작업이 끝났을 때, 또는 Strata가 백그라운드에 있을 때 짧은 소리를 재생합니다.",
"gh12":"소리 재생",
"gh13":"Strata가 답을 필요로 할 때 알림",
"gh14":"수동 모드에서 어시스턴트가 승인을 기다리고 Strata가 백그라운드에 있을 때 데스크톱 알림을 표시합니다.",
"gh15":"Strata가 답을 기다리고 있습니다",
"gh16":"어시스턴트가 {0}을(를) 사용하려고 합니다. Strata를 열어 승인하거나 거부하세요.",
"gh17":"도구",
"gh18":"트레이로 최소화",
"gh19":"창을 닫아도 Strata는 시계 옆 트레이에서 계속 실행됩니다. 끝내려면 트레이 메뉴에서 '종료'를 선택하세요. 다음에 Strata를 시작할 때부터 적용됩니다.",
"gh20":"앱 포트",
"gh21":"Strata의 창과 로컬 주소가 사용하는 포트입니다. 비워 두면 Strata가 고릅니다(비어 있으면 47831). 다음에 Strata를 시작할 때부터 적용됩니다.",
"gh22":"포트(비움 = 자동)",
"gh23":"지금은 포트 {0}을(를) 사용 중입니다.",
"gh24":"Strata가 지정한 포트를 사용할 수 없었습니다",
"gh25":"포트 {0}을(를) 다른 프로그램이 사용 중이어서 Strata는 포트 {1}에서 시작했습니다. 포트 {0}에 저장된 채팅과 설정은 Strata가 다시 그 포트를 쓸 수 있을 때까지 표시되지 않습니다. 다른 프로그램을 닫고 Strata를 다시 시작하세요.",
"gh26":"1024에서 65535 사이의 포트를 고르거나 비워 두세요.",
"gh27":"Strata의 모든 글자, 버튼, 메뉴를 더 크게 또는 더 작게 만듭니다.",
"gh08":"Preço de um quilowatt-hora. O Monitor passa a mostrar quanto a sua placa de vídeo custou desde que o Strata iniciou. Deixe vazio para ocultar o custo.",
"gh09":"Preço por kWh",
"gh10":"Som ao terminar o trabalho",
"gh11":"Toca um som curto quando uma resposta ou tarefa que demorou termina, ou quando o Strata está em segundo plano.",
"gh12":"Tocar som",
"gh13":"Avisar quando o Strata precisar de uma resposta",
"gh14":"No modo manual, mostra uma notificação na área de trabalho quando o assistente espera sua aprovação e o Strata está em segundo plano.",
"gh15":"O Strata precisa da sua resposta",
"gh16":"O assistente quer usar {0}. Abra o Strata para aprovar ou negar.",
"gh17":"uma ferramenta",
"gh18":"Minimizar para a bandeja",
"gh19":"Ao fechar a janela, o Strata continua em execução na bandeja, ao lado do relógio. Escolha Sair no menu da bandeja para encerrá-lo. Vale a partir da próxima vez que o Strata iniciar.",
"gh20":"Porta do aplicativo",
"gh21":"A porta usada pela janela e pelo endereço local do Strata. Deixe vazio para o Strata escolher (47831 quando estiver livre). Vale a partir da próxima vez que o Strata iniciar.",
"gh22":"Porta (vazio = automático)",
"gh23":"Usando a porta {0} agora.",
"gh24":"O Strata não conseguiu usar a sua porta",
"gh25":"A porta {0} está em uso por outro programa, então o Strata iniciou na porta {1}. Os chats e as configurações salvos na porta {0} não aparecem até o Strata poder usá-la de novo. Feche o outro programa e reinicie o Strata.",
"gh26":"Escolha uma porta entre 1024 e 65535, ou deixe vazio.",
"gh27":"Deixa todos os textos, botões e menus do Strata maiores ou menores.",
"gh08":"Цена одного киловатт-часа. Тогда монитор покажет, во сколько видеокарта обошлась с запуска Strata. Оставьте поле пустым, чтобы скрыть стоимость.",
"gh09":"Цена за кВт·ч",
"gh10":"Звук по окончании работы",
"gh11":"Проигрывает короткий звук, когда закончился долгий ответ или задача, а также когда Strata работает в фоне.",
"gh12":"Проиграть звук",
"gh13":"Уведомлять, когда Strata нужен ответ",
"gh14":"В ручном режиме показывает уведомление на рабочем столе, когда помощник ждёт вашего подтверждения, а Strata работает в фоне.",
"gh15":"Strata нужен ваш ответ",
"gh16":"Помощник хочет использовать {0}. Откройте Strata, чтобы разрешить или отклонить.",
"gh17":"инструмент",
"gh18":"Сворачивать в трей",
"gh19":"При закрытии окна Strata продолжает работать в трее рядом с часами. Чтобы остановить её, выберите «Выход» в меню значка. Действует со следующего запуска Strata.",
"gh20":"Порт приложения",
"gh21":"Порт, который используют окно и локальный адрес Strata. Оставьте поле пустым, чтобы Strata выбрала сама (47831, если свободен). Действует со следующего запуска Strata.",
"gh22":"Порт (пусто = автоматически)",
"gh23":"Сейчас используется порт {0}.",
"gh24":"Strata не смогла использовать свой порт",
"gh25":"Порт {0} занят другой программой, поэтому Strata запустилась на порту {1}. Чаты и настройки, сохранённые на порту {0}, не показываются, пока Strata не сможет снова его использовать. Закройте другую программу и перезапустите Strata.",
"gh26":"Выберите порт от 1024 до 65535 или оставьте поле пустым.",
"gh27":"Увеличивает или уменьшает весь текст, кнопки и меню в Strata.",
"tl01":"Utilisez un nom de serveur simple de 32 caractères maximum: lettres, chiffres, tirets ou tirets bas.",
"tl02":"Ces serveurs d’outils ont été laissés de côté car leurs noms ne sont pas autorisés: {0}. Un nom peut contenir jusqu’à 32 lettres, chiffres, tirets ou tirets bas.",
"tl03":"Ces serveurs d’outils ont été laissés de côté car ils n’ont ni programme ni adresse web: {0}.",
"tl04":"Ces serveurs d’outils ont été laissés de côté car un autre a presque le même nom: {0}."
"tune21":"Ce PC a changé depuis l’optimisation, ces réglages ne sont donc pas utilisés tant que vous n’optimisez pas à nouveau.",
"tune22":"Ce PC ou le moteur a changé depuis l’optimisation. Une vérification courte prend environ 2 minutes.",
"tune23":"Pas encore d’optimisation enregistrée pour ces choix.",
"tune24":"Strata essaie différents réglages sur ce PC et garde ceux qui accélèrent les réponses. Votre modèle est déchargé pendant ce temps, et l’ordinateur sera occupé.",
"tune25":"Terminez d’abord l’opération en cours.",
"tune26":"Durée à y consacrer",
"tune27":"Rapide (environ 20 minutes)",
"tune28":"Approfondie (jusqu’à 2 heures)",
"tune29":"Ce que vous faites surtout",
"tune30":"Programmation avec de longs prompts",
"tune31":"Autoriser les réglages qui changent très légèrement les réponses (même qualité)",
"tune32":"Optimisations enregistrées",
"tune33":"Utilisée",
"tune34":"Utiliser",
"tune35":"Optimiser à nouveau",
"tune36":"Lancer l’optimisation",
"tune37":"Optimisation de votre modèle",
"tune38":"Les réponses peuvent différer d’une exécution à l’autre sur ce PC même avec la configuration standard; ce réglage n’y ajoute rien.",
"tune39":"Utilisation de votre optimisation « {0} »",
"tune40":"Désactiver",
"tune41":"Activer",
"tune42":"Votre optimisation est désactivée. Les réglages standard sont utilisés.",
"tune43":"Votre PC était occupé, les mesures n’étaient donc pas fiables. Rien n’a été modifié. Réessayez quand il est au calme."
{tr(readingsStrip.length?<sectionclassName="st-card monitor-readings"aria-label={tr("Energy and temperature readings")}><dl>{readingsStrip.map(item=><divkey={item.label}><dt>{tr(item.label)}</dt><ddclassName="num">{item.value}</dd></div>)}</dl></section>:null)}
<divclassName="monitor-context-requests">
<sectionclassName="st-card monitor-context"><h2className="h4">{tr("Context fill")}</h2><divclassName="monitor-context__gauge"><Gaugevalue={contextPct??0}label={tr(hasReading(contextPct)?`${Math.round(contextPct)}%`:'—')}sub={contextKnown?`${tokens(contextUsed)} / ${contextLabel(data.context.maxTokens)}`:selection==='all'&&(runtime?.instances.length??0)>1?'Select a model to see context':'No recorded request usage'}/></div>
<divclassName="monitor-bar-reading"><span>{tr("Experts in VRAM")}</span><spanclassName="muted num">{tr(cacheKnown?`${number(data.expertCache.used)}${hasReading(cacheMemory)?` · ${number(cacheMemory,1)} GB`:''}`:'—')}</span></div><ReadingBarlabel={tr("Experts in VRAM")}value={cachePercent}/>
@@ -26,11 +29,16 @@ export function SettingsScreen() {
control: ReactNode;
}[]=[
{section:'General',label:'Language',hint:'Choose the language used throughout Strata.',control:<SelecttranslateLabels={false}ariaLabel={tr("Language")}value={prefs.language}options={[...LANGUAGES]}onChange={value=>setLanguage(valueastypeofprefs.language)}/>},
{section:'General',label:'Sound when work finishes',hint:'Plays a short sound when an answer or task that took a while is finished, or when Strata is in the background.',aliases:'audio chime done complete notification',control:<DoneSoundSetting/>},
{section:'General',label:'Notify me when Strata needs an answer',hint:'In Manual mode, shows a desktop notification when the assistant waits for your approval and Strata is in the background.',aliases:'approval permission alert notification manual',control:<ApprovalNotifySetting/>},
...(shell?.tray?[{section:'General',label:'Minimise to tray',hint:'Closing the window keeps Strata running in the tray, next to the clock. Choose Quit in the tray menu to stop it. Applies the next time Strata starts.',aliases:'tray background close notification area',control:<TraySettingshell={shell}save={saveShell}/>}]:[]),
...(shell?.desktop?[{section:'General',label:'App port',hint:'The port Strata’s window and local address use. Leave it empty to let Strata choose (47831 when it is free). Applies the next time Strata starts.',aliases:'network address 47831 localhost api',control:<PortSettingshell={shell}save={saveShell}/>}]:[]),
{section:'General',label:'Electricity price',hint:'Price of one kilowatt-hour. The Monitor then shows what your graphics card has cost since Strata started. Leave it empty to hide the cost.',aliases:'cost power energy kwh currency money',control:<ElectricityPrice/>},
{section:'Reading & focus',label:'Font',hint:'Choose an installed font for the interface and conversations.',control:<FontChoice/>},
{section:'General',label:'Keyboard shortcuts',hint:'Set the key combinations that feel natural to you.',aliases:'command keys hotkeys',control:<Btnsize="sm"onClick={()=>setShortcuts(true)}>{tr("Edit shortcuts")}</Btn>},
{section:'Reading & focus',label:'Appearance',hint:'System follows your computer by default.',aliases:'theme light dark system automatic',control:<Selectvalue={prefs.theme}ariaLabel={tr("Appearance")}options={[{value:'light',label:'Light'},{value:'dark',label:'Dark'},{value:'system',label:'System'}]}onChange={setTheme}/>},
{section:'Reading & focus',label:'Reading size',hint:'Conversation text size. Interface controls stay separate.',aliases:'font text zoom',control:<labelclassName="row"><inputaria-label={tr("Reading text size")}type="range"className="st-slider"min={CONTENT_FONT_RANGE.min}max={CONTENT_FONT_RANGE.max}value={prefs.contentFontSize}onChange={e=>setContentFontSize(Number(e.target.value))}/><output>{tr(prefs.contentFontSize)}{tr(" px")}</output></label>},
{section:'Reading & focus',label:'Interface size',hint:'Makes all text, buttons and menus in Strata larger or smaller.',aliases:'zoom scale font text size bigger smaller 4k 90% 100% 110% 125% 150% 175% 200%',control:<Selectvalue={String(prefs.uiScale)}ariaLabel={tr("Interface size")}options={UI_SCALES.map(v=>({value: String(v),label:`${Math.round(v*100)}%`}))}onChange={v=>setUiScale(Number(v))}/>},
{section:'Reading & focus',label:'Spacing',hint:'Comfortable spacing by default.',aliases:'density comfortable compact',control:<Selectvalue={prefs.density}ariaLabel={tr("Interface spacing")}options={[{value:'comfortable',label:'Comfortable'},{value:'compact',label:'Compact'}]}onChange={setDensity}/>},
{section:'Reading & focus',label:'Motion',hint:'Reduced motion also disables smooth scrolling.',aliases:'animation system full reduced',control:<Selectvalue={prefs.motion}ariaLabel={tr("Motion")}options={[{value:'system',label:'Follow system'},{value:'full',label:'Full'},{value:'reduced',label:'Reduced'}]}onChange={setMotion}/>},
{section:'Reading & focus',label:'Transparency',hint:'Translucent controls or solid surfaces. Text stays opaque.',aliases:'blur glass liquid opaque translucent',control:<Selectvalue={prefs.transparency}ariaLabel={tr("Transparency")}options={[{value:'translucent',label:'Translucent'},{value:'opaque',label:'Opaque'}]}onChange={setTransparency}/>},
/* The energy and temperature readings (#3628): one thin row under the cards, a label and a value each, wrapping to 3 and 2 columns on narrow windows. */
for(consttextof[running,done,failed]){constid=idOf(text);assert.ok(id,`${text} is in the catalog`);for(constlof['de','es','ja','ko','pt','ru','zh'])assert.ok(catalog[l][id]?.trim(),`${l}: ${text}`);}
for(consttextof[running,done,failed]){constid=idOf(text);assert.ok(id,`${text} is in the catalog`);for(constlof['de','es','fr','ja','ko','pt','ru','zh'])assert.ok(catalog[l][id]?.trim(),`${l}: ${text}`);}
@@ -154,7 +154,7 @@ test('real Harness: a turn that writes a file says what it is doing, in order, a
test('a sentence with a name or a number also carries its catalog template and values, and every status sentence is in the catalog in every language',async()=>{
@@ -163,7 +163,7 @@ test('a sentence with a name or a number also carries its catalog template and v
constkeys=['Starting the assistant…','Starting…','Reading your message…','Working out the next step…','Thinking… {0} tokens','Writing the answer…','Waiting for your answer…','Helpers are working ({0})…','Working on the goal…',
'Writing {0}…','Writing a file…','Writing {0}… ({1} KB)','Writing a file… ({0} KB)','Changing {0}…','Changing a file…','Reading {0}…','Reading a file…','Running a command…','Searching your files…','Searching the web…',
'Opening {0}…','Opening a web page…','Using the browser…','Setting the goal…','Updating the goal…','Checking the goal…','Using a tool…','Reading your files…','Trying an improved version…','Checking the result…','Comparing the two versions…'];
for(constkeyofkeys){constid=idOf(key);assert.ok(id,`${key} is in the catalog`);for(constlof['de','es','ja','ko','pt','ru','zh']){assert.ok(catalog[l][id]?.trim(),`${l}: ${key}`);assert.deepEqual((catalog[l][id].match(/\{\d\}/g)??[]).sort(),(key.match(/\{\d\}/g)??[]).sort(),`${l} keeps the slots of ${key}`);}}
for(constkeyofkeys){constid=idOf(key);assert.ok(id,`${key} is in the catalog`);for(constlof['de','es','fr','ja','ko','pt','ru','zh']){assert.ok(catalog[l][id]?.trim(),`${l}: ${key}`);assert.deepEqual((catalog[l][id].match(/\{\d\}/g)??[]).sort(),(key.match(/\{\d\}/g)??[]).sort(),`${l} keeps the slots of ${key}`);}}
});
test('a goal step is always sent, even right after another status (it used to be swallowed by the 300 ms gap, which made a test flaky)',()=>{
Loaded 100 of 634 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Niko1221