Compare commits

...
944 Commits
Author SHA1 Message Date
Niko1221 5e8a4cca7d Release notes 0.2.7: update, tray and speed lines confirmed on the release packages; Strix Halo guide names the prompt used for the 4K row 2026-10-27 14:54:33 +01:00
Niko1221 a42cc3d7df Version 0.2.7: Terminal and app 0.2.7, engine and Control 0.1.52 (every Qwen and DeepSeek package, Metal included; both engines changed since 0.2.6)
Same 15 files as the 0.2.6 bump (issue template, AGENTS.md, CMakeLists.txt 0.1.52, desktop and host package files, Cargo.toml and the strata-desktop entry in Cargo.lock, SettingsUpdates comment, qa1-storage test, metal 0.1.52-metal, build_hip_linux.sh PKGVER 0.1.52, strata_cli PRODUCT_VERSION, the two version tests), plus the Strata-0.2.7-<file> names in the md files.
2026-10-27 09:13:22 +01:00
Niko1221 5a843fd841 docs: release notes 0.2.7 - two-card pipelined default (owner choice a), DeepSeek pictures by default, the shard 1 setup fix 2026-10-27 09:13:21 +01:00
Niko1221 e6d5d76010 Merge f-027-dsvis: DeepSeek reads pictures by default (owner yes), and setup accepts ISTA's re-published shard 1
The engine's own tower (319 MiB) through strata launch and setup (--vision off still turns it off); setup fetches
the pinned 972 MB mmproj; the server sends pictures to the tower. Checked on 4x R9700, 1x R9700 and a real
launch on 2x A6000 (download, pack, picture answer over HTTP); 0/10 stalls, KL unchanged. Also: a newer shard 1
from --model-path (ISTA's 2026-10-26 chat template) is accepted after its own size and SHA-256 check.
2026-10-27 09:10:53 +01:00
Niko1221 2526f2ff99 Merge two-card batch default (owner choice a): two NVIDIA cards pipeline by default
Pipelined groups beat the serial default at 2 and 4 streams on 2x A6000 (+17/+22%) and 2x A4000 (+7..12/+17..20%);
one stream equal, -4% on A4000 IQ3_XXS. STRATA_BATCH_GROUPS_NV_SERIAL=1 or STRATA_DEFAULTS=026 gives serial.
2026-10-27 09:10:53 +01:00
Niko1221 94d7aab293 setup: accept the DeepSeek shard 1 with ISTA's 2026-10-26 chat template from files already here
ISTA re-published (and patch_chat_template.py rewrites) the first shard of each set with a larger chat template, so a shard 1 on disk can be 11 KB
larger than the pinned revision's. setup refused it (size mismatch; the GGUF preflight does not read DeepSeek tensors, so it was not taken as a
whole own file). The pinned revision stays the download; the newer shard 1 is accepted by its own size and SHA-256 with --gguf-dir.
2026-10-27 07:18:19 +01:00
Niko1221 1bd17d59d9 launch: the model state's DeepSeek check tolerates a catalog without the strata table (test_model_folder_case) 2026-10-27 06:22:02 +01:00
Niko1221 c451698d6a DeepSeek pictures: the Terminal's pictures step knows the tower is part of the engine (no reader lookup, no GPU/CPU reader choice), and a DeepSeek start without pictures answers a picture request with its own 400 wording 2026-10-27 06:12:25 +01:00
Niko1221 1e617c3f64 DeepSeek pictures by default: setup fetches the vision tower's weights (mmproj-BF16.gguf, pinned) with the shared files, packs them with the model (pack_gguf --mmproj) and rebuilds a pack made without them; --vision no leaves all of it out; the picture-reader parts of the launcher's model state skip DeepSeek (its tower is part of the engine); the installed-config description says pictures on for a --vision on DeepSeek config; tests 2026-10-27 05:49:37 +01:00
Niko1221 9b177f3080 DeepSeek reads pictures by default: strata launch and setup pass --vision on (the engine's own 319 MiB tower, no separate reader), the server sends a picture to that tower as raw RGB in the file of a GENI line (serve/ds_pictures.py: decode with Pillow, both sides a multiple of 42, at most 1024 image tokens, the template's placeholder expanded to the span), --vision off keeps pictures off and the old 400; the plan, the catalog and docs/DEEPSEEK41.md say pictures on; tests for the size rule, the config switch, the prompt expansion and the server route 2026-10-27 05:41:48 +01:00
Niko1221 4c7bce3ad1 generate: the gated-proposal check reads use_df2 only where the DFlash2 drafter is built (the Metal build has none; #3416 broke its compile) 2026-10-27 05:26:53 +01:00
Niko1221 8dce24a0a1 docs: release notes - the two-card batch default marker uses the owner's labels (a) pipelined by default, (b) pipelined only when every expert fits, (c) leave serial; the serial against pipelined numbers are batch scheduling without drafting 2026-10-27 04:33:31 +01:00
Niko1221 12d0f0abe8 docs: release notes 0.2.7 - the engine items from f-027-gh-eng filled in (layer-major prefill on by default for a two-card CUDA split, pipelined drafts opt-in, sm_80 and sm_89 defaults, audit fixes, opt-in switches), markers closed except the owner's two-card batch choice 2026-10-27 04:32:24 +01:00
Niko1221 ac19fecc06 Merge release-notes update: route prior 1.5 wording with the agent-loop caveat 2026-10-27 04:28:46 +01:00
Niko1221 6ca60e9a0d Merge f-027-gh-eng: GitHub engine items for 0.2.7
P0 #3911 #1503; P1 fixes; file-02 items; 22 PR ports (authors kept) incl. #3779 (sm_80 defaults under
STRATA_DEFAULTS=026) and #4121 (STRATA_PIPED_MTP opt-in); audit fixes #3602 #3948 #3638 #3797 #2409; owner
decisions #3935 (DeepSeek sm_89 packages), #3844, #3453 (sm_89 tables on), #3976 (layer-major prefill on by
default for a 2-card CUDA layer split; revert 1e41fa269 to undo). Final gate: sm_60, SYCL, z440 sm_86 CPU 38/38
GPU 23/23, boston CPU 24/24 GPU 23/23, owner-PC MSVC CPU 36/36 GPU 20/20, snapshot 10/10 on z440 and boston.
2026-10-27 04:28:36 +01:00
Niko1221 c41bd44de7 docs: route prior 1.5, final verdict of the 80-task check on the RTX 3060 (no copy or code errors; agent loops may finish slightly less often, within the run-to-run spread), speed and the off switch 2026-10-27 04:28:28 +01:00
Niko1221 447460e8cc native experts: the window-invariant reference rows are used only where they match the linked ggml dot (#3416)
native_arithmetic_parity failed on an Intel i9-13900K (AVX2 + AVX-VNNI, ggml built native)
in five of its eight configurations: "gu=16 down=20 ff=96 nt=2 token=0". Not the VNNI row
path: STRATA_NO_AVXVNNI=1 changes nothing, STRATA_NO_IQ4NL=1 passes. The IQ4_NL down
reference (iq4nl_reference_rows) differs from that ggml's one-token dot on the scalar tail
block of a row with an odd number of blocks (ff=96 is three): that ggml does not contract
`sumf += d * sumi` into an FMA, the reference lets the compiler do it. On the other
machines tested the two agree, so it is a property of the ggml build, not of the CPU.
With an unfused tail in the reference all eight configurations pass on that machine.

The invariant path now checks each reference once against the linked ggml-cpu on synthetic
blocks before using it (iq4nl_reference_ok for IQ4_NL with 2, 3, 5 and 20 blocks;
iq512_reference_ok for types 18, 21 and 22 with SwiGLU, on AVX-512 CPUs). On a mismatch
it prints one line and the invariant path uses ggml's own one-token dot per token: slower
on such a build, never different. The Default path is not touched; only the opt-in
STRATA_SPEC_PROB=1 decode reaches this code.

Tests: native_arithmetic_parity, eight configurations, on the i9-13900K (4,866,048
comparisons, with the fallback line), z440 and the owner PC (see the series notes).
2026-10-27 04:18:06 +01:00
Niko1221 ec8e405909 batch groups: two NVIDIA cards pipeline by default; the serial group is opt-in (STRATA_BATCH_GROUPS_NV_SERIAL=1, STRATA_DEFAULTS=026)
0.2.1 made a layer split over exactly two NVIDIA cards with --batch and no --batch-groups run one serial group (measured
then on 2x RTX A4000, IQ3_XXS: +12..16%). With the 0.2.7 code the pipelined groups are faster at 2 and 4 streams on
every box and quant measured, so the default goes back to the pipeline.

Measured (tools/serve_load.py, --batch 4, 256 tokens per request, interleaved pairs, total tok/s, pipelined / serial):
- 2x RTX A6000, IQ3_S, all experts in VRAM: 1 stream 101.7 / 101.7; 2 streams 125.7 / 107.4 (+17%); 4 streams 171.4 / 141.0
  (+22%). The v0.2.6 binary on the same box: serial default 104.9 and 137.2, --batch-groups auto 124.6 and 169.6.
- 2x RTX A4000, Q2_0 pack (17037 of 24576 pairs cached, 4 clean pairs): 2 streams 80.2-82.5 / 75.0-75.5 (+7..9%, 4 of 4);
  4 streams 110.8-111.5 / 94.8-95.3 (+17%, 4 of 4); 1 stream 63.6-68.7 / 68.2-70.3.
- 2x RTX A4000, IQ3_XXS (13056 of 24576 pairs cached, 3 pairs): 2 streams 73.2-73.7 / 65.8-66.5 (+12%, 3 of 3); 4 streams
  100.5-100.7 / 82.9-83.6 (+20%, 3 of 3); 1 stream 58.4-60.5 / 60.7-62.6 (serial about 4% faster, 3 of 3).
The single stream runs on the solo path in both; only the A4000s show the 4%.

What changed: batch_groups.hpp has a pure rule nv_serial_wanted(env value, STRATA_DEFAULTS=026). STRATA_BATCH_GROUPS_NV_SERIAL
set to anything but 0 asks for the serial group, an explicit 0 wins over STRATA_DEFAULTS=026, and with the variable unset or
empty STRATA_DEFAULTS=026 gives v0.2.6's serial group. An explicit --batch-groups (a number or auto) still wins. Three or more
cards, AMD builds and --batch-mtp/--adapt-async handling are unchanged. The serial message names the opt-in.

Tests: batch_groups_test (the rule over the env values and 026, two cards pipeline, the existing serial cases). Real engine on
2x A6000 (IQ3_S), no --batch-groups: the engine says "2 groups of 2 slots" / "2 groups of 1 slot"; with 4 and 2 slots, greedy,
greedy+seeded sampled, slots finishing at different lengths and a cancel in the middle of a draft, every slot's tokens equal its
solo decode and the pipelined build's. A long-prompt slot differs from the pre-merge base build in the same way with the flag
and without it (it is the parent's prompt path, not this change).

Not covered: no run on AMD or on three cards; the 4% single-stream cost on the A4000s is measured on one box.
2026-10-27 04:17:17 +01:00
Niko1221 5ac66e5005 release notes 0.2.7: layer-major prefill (on by default on a CUDA layer split) and pipelined drafts filled in 2026-10-27 04:05:02 +01:00
Niko1221 e40843dade Merge f-027-int2 (c7c8c0d0a) into f-027-gh-eng
# Conflicts:
#	docs/release-notes-0.2.7.md
2026-10-27 04:04:41 +01:00
Niko1221 5cfa395476 Merge f-027-gh-eng-pm (#4121 MTP drafts in pipelined slots, opt-in STRATA_PIPED_MTP) 2026-10-27 04:04:21 +01:00
Niko1221 f656a721a6 docs: STRATA_PIPED_MTP on 2x RTX A4000 (Q2_0): =1 is the setting to use; =2 is slower than no drafts at 4 streams there (#4121) 2026-10-27 04:02:38 +01:00
Niko1221 42890ca79f metal: stubs for the layer-major prefill kernels (moe_shared_init, moe_scatter_add): the Mac build shares prefill.cpp, the walk is never taken there (#3976) 2026-10-27 03:29:12 +01:00
Niko1221 37e4832ab3 layer-major prefill: on by default for a CUDA layer split, STRATA_PREFILL_LAYER_MAJOR=0 and STRATA_DEFAULTS=026 turn it off (#3976)
THIS COMMIT ALONE MAKES THE WALK A DEFAULT. Drop it and the layer-major prefill stays
an opt-in (STRATA_PREFILL_LAYER_MAJOR=1) with the chunk-major path as the default.

Rule (include/strata/core/sm120_series.hpp, pure, tested in src/core/arch_defaults_test.cpp):
- prefill_layer_major_default(hip, stages, v026): on for two or more stages on CUDA,
  off on one GPU, in a HIP build, and under STRATA_DEFAULTS=026 (qwen_defaults_026());
- prefill_layer_major_resolve(env, ...): STRATA_PREFILL_LAYER_MAJOR=0 / 1 (any text
  atoi reads) wins in both directions, over the rule and over STRATA_DEFAULTS=026.
Every decline of the earlier commits stays and runs in lm_prepare as before (HIP,
image prompts, DFlash2 taps, KVA, stop_at_chunk, a pending force_chunk, a peer GPU,
APR / NVFP4 / WMMA / fused layers, STRATA_EXPERT_NIBBLE, no VRAM, no pinned RAM):
such a prompt reads chunk-major with nothing but a log line. When the walk is on by
default (nothing set) the VRAM margin is 512 MiB instead of 96 and the super-chunk
target 32768 tokens (the size measured) instead of 8192; "a short prompt" is not logged.
STRATA_PREFILL_LM_TOKENS and STRATA_PREFILL_LM_MARGIN_MIB still win.

Measured (z440, 2x RTX A4000 16 GB, Qwen3.8-Flash-Next IQ3_XXS, experts streamed, 5
interleaved pairs, time to first token, medians): 8K 2.8x, 20K 4.8-5.7x, 32K 5.1-5.5x
faster (141 s to 26-28 s); decode 38.6 vs 38.4 tok/s; needles 12 of 12 found in both
arms. The speed-up is larger than the report's 2x because --prefill auto picks 512-token
chunks there. The first-token KL numbers are in the gate notes (3976-kl.sh).

Not covered: one GPU (stays opt-in), HIP, SYCL, Metal, three or more stages, a card
with more VRAM than the pack needs (where the experts are VRAM-resident and the
chunk-major path has little to re-read).
2026-10-27 03:28:36 +01:00
Niko1221 1c2efd789a Merge f-027-gh-eng-d again (9d85432b8: QFUSE one-token self-commit) 2026-10-27 03:12:14 +01:00
Niko1221 0d4b2cdd43 Merge f-027-gh-eng-lm up to 0c7de9868 (#3976 layer-major split prefill, opt-in: STRATA_PREFILL_LAYER_MAJOR=1) 2026-10-27 03:12:05 +01:00
Niko1221 789d83903b STRATA_QFUSE=1: a one-token window did not self-commit, but commit() skipped the commit graph as if it had (#3948)
Problem. #3948: with STRATA_QFUSE=1 the verified text changes and MTP acceptance drops, on CUDA. The QFUSE kernels are bit-exact
(gr_parity, gdn_parity, 10,500 random windows each on sm_86 and sm_120, after 91a4b8a67), yet the engine's greedy text still differed
from QFUSE off on every prompt tried. Model run, l620 (RTX PRO 5000, sm_120), Qwen3.8-Flash-Next IQ3_S, MTP --spec 6, identity-gate
flags, 4 prompts of 4K and 12K tokens, 96 greedy tokens: QFUSE on differed from off on 4 of 4 (the off-vs-off control was identical).

Cause. A window of one token advances the GDN state and the indexer tail itself ("self-commit") so that commit() needs no commit
graph. Verifier::record_window decides this with
`T == 1 && !always_publish_ && !batch_rec_ && !g_qfuse() && one_token_self_commit()`: with STRATA_QFUSE the fused GDN output
kernel takes the q8_1 destination and the window does not self-commit. Verifier::commit() skipped the commit graph with
`last_t_ == 1 && !always_publish_ && one_token_self_commit()`, without the QFUSE term. So with QFUSE on, every accepted one-token
window left the recurrent state (and the indexer tail) one token behind. Windows of one token are the ordinary case when the drafter
offers nothing, which matches the report: fewer drafts offered, lower acceptance, different text, and an unchanged text where
no window of one token occurs. On HIP (where QFUSE is the default on gfx1151) one_token_self_commit() is off unless
STRATA_ONE_TOKEN_COMMIT=1, so the two sides agreed there.

Change. commit() takes the same condition: `... && one_token_self_commit() && !g_qfuse()`. Without STRATA_QFUSE nothing changes.

Result (same l620 run). QFUSE on with this change against QFUSE off, same build: logits md5 and the 96 greedy tokens byte-identical
on all 4 prompts (IDENTITY PASS); the build without the change differs on all 4. The default path (QFUSE unset) of the merged head
against the f-027-int2 head (2a2264c21): first-window logits and greedy tokens byte-identical on the same 4 prompts.

Not covered. Serve answers and draft counters (3948-qfuse.sh step 2, SERVE=1), sm_86, and the reporter's 40-prompt set.
2026-10-27 03:11:15 +01:00
Niko1221 9c624c75a9 Merge f-027-gh-eng-d (#3602 #3948 #3638 #3702 #3797 #2409 #3844: audits, fixes and tests)
# Conflicts:
#	src/core/verify.cpp
#	src/kernels/cuda/fused_gr.cu
2026-10-27 02:48:36 +01:00
Niko1221 8dd297d0f1 PLE table: STRATA_PLE_CHECK=1 names the first row that decodes to an implausible value and says whether the delivered bytes are the file's (#3602)
The report asks for the first data mismatch between the GGUF row bytes and what the direct reader delivered, and for an early
check of the gathered values; neither exists. Its trace stops at "the embedding has 96 non-finite values" without saying whether
the reader gave a wrong row or the table holds one.

With STRATA_PLE_CHECK=1 (a tester switch, off by default; one static flag read per gather call when off) every direct-mode decode (read_row, collect,
gather_batch, also from the prefetched slots) checks the row's 160 values. For a value that is not finite or beyond 1e5 (an IQ4_NL
row cannot decode beyond 8.3e6; real PLE values are orders smaller) it re-reads that row from the file with a buffered read and prints
one line to stderr (at most 8 per table): the row, its file offset and page offset, the value, and either "the delivered bytes ARE the
file's bytes" or "THE DELIVERED BYTES DIFFER FROM THE FILE from byte N" with the first 8 bytes of each. That separates a reader
fault (the bug) from a damaged table. gates/3602-ple-direct.sh sets it in its direct arms.

Checked: ple_reader_test --selftest on z440 with the switch on prints the "ARE the file's bytes" line for the synthetic table's random
rows (which decode to huge values); with it off the output and the bytes are unchanged (ple_reader_selftest passes).
Not checked: the "DIFFER" branch has no natural trigger (that is the point); it was read, not run.
2026-10-27 02:10:24 +01:00
Niko1221 7d1016f3f4 QFUSE: the fused q8_1 producers divide the way the separate quantiser does; they gave a different code at about one element in a million (#3948)
Problem. #3948: with STRATA_QFUSE=1 the verified text changes and MTP acceptance drops (RTX 3080, sm_86). The report suspected the
group counter or the nat_xq_ offsets of fused_gr.cu; the audit and the new two-group test (previous commit of this stream)
rule those out. What the new test found instead, on an RTX 5070 (sm_120, CUDA 13.0, MSVC), over 10,500 random windows of two groups: in
about 1 run of 400 one q8_1 code of the fused image differs by exactly one from native_quantize_q8_1's, in every mode (group-relative,
window-indexed, direct, graph replay). The scale/sum pair and the other codes agree.

Cause. native_mmvq.cu, which holds native_quantize_q8_1 (the quantiser every consumer of a q8_1 image was written for), is
compiled with --use_fast_math (CMakeLists.txt, the native_* list), so its `amax / 127` and `x / d` are the fast full-range division
(div.full.f32, up to 2 ulp off). fused_gr.cu (the hyper-connection read's tail, gr_up_q32_kernel) and verify_kernels.cu (the GDN
output norm, gdn_q8_1_store) are not in that list and divide with IEEE rounding. roundf(x / d) then lands on the other side of a .5
at the rare element whose quotient is next to a boundary. One changed activation code per ~10^6 values is about one per few
hundred tokens at 80 hyper-connection reads of 2560 values per token: enough to move a near-tie in the verify, and the
draft (which quantises with the separate kernel) disagrees with the verifier a little more often, which lowers acceptance. That
matches the report (acceptance 0.755 against 0.79, different text, and an unchanged text where no draft is made).

Change. include/strata/kernels/q8_1_finite.hpp: q81_div(a, b) is `a / b`, except in a translation unit that defines
STRATA_Q81_FASTMATH_DIV (CUDA device code only, never HIP), where it is the same PTX division the fast-math file gets (div.full.f32).
q8_1_quant and the scale use it. fused_gr.cu and verify_kernels.cu define the macro; nothing else does, so every other kernel,
the separate quantiser and every HIP build compile to what they did. STRATA_QFUSE is off by default on CUDA (it is on by default on
gfx1151, which is HIP and unchanged), so the default path is bit-identical.

Result (RTX 5070, sm_120): gr_parity's two-group window test: 10,500 runs with 1,500 random inputs for each T, all modes, zero
differences (before: dozens); the whole of gr_parity, gdn_parity and gdn_parity with STRATA_GDN_SPLIT=1 pass.

Tests. gr_parity: the two-group test prints the first differing bytes (block, field), and STRATA_GRP_REPS=N / STRATA_GRP_KEEP_GOING=1
repeat it longer and count every failure. gdn_parity: the q8_1 destination check uses its own stream (native_quantize_q8_1 refuses the
null stream, which crashed the test on Windows), unbuffered stdout and STRATA_GDN_PARITY_TRACE=1.

Not covered. sm_86 (the report's card) has not run the new test yet (z440 was reserved for other jobs); the cause is not
architecture specific. The model run (gates/3948-qfuse.sh) is still owed: QFUSE on against off must now be byte-identical, and the
draft counters equal.
2026-10-27 02:02:46 +01:00
Niko1221 e0d47034df Presets: a value outside the range the tuner tries is kept with a warning, not dropped (#3844)
Problem. #3844: --spec 5 measured +6.1% decode on an RTX 5060 Ti, but a saved preset that carries it was left out entirely
("outside the safe values [3, 4]"), and an imported one was refused. The tuner's BOUNDS are the values it tries; they are
not a finding that a value is harmful. The never list (--spec 6 and wider: 4% to 16% slower, --expert-cache 0, ...) is.

Change (Terminal / Control; touches strata_control/tune_knobs.py, strata_control/presets.py, strata_cli/tune_commands.py and
serve/server.py).
- tune_knobs.check_setting_parts(args, env, ctx) returns (never, bounds). check_setting is the two lists together, in the
  order it had, so the tuner still skips --spec 5 and its tests are unchanged.
- presets.overlay applies a preset whose only problems are outside BOUNDS and returns info["warnings"], one short line each
  ("--spec 5: not tried by the tuner, kept because it is your setting"). A never-list setting still means "not applied".
  presets.create and import_preset refuse only never-list settings.
- serve/server.py prints each warning as "[strata] warning: ..." when the preset is applied; strata_cli tune_commands
  _effective adds "warnings" to its payload.

Tests. tests/tuning/test_presets.py: a preset with --spec 5 is saved, applied (the flag becomes 5, the input config is not
modified), warns once, and an in-range preset has no warnings; the never list still refuses --spec 6; import of --spec 5 works.
tests/tuning/test_autotune.py: --spec 5 stays in the tuner's refused list. python -m pytest tests/tuning: 274 passed;
tests/terminal -k "tun or preset or overlay": passed.

Not covered. The app's Details text does not show the warning yet (it reads the payload key above).
2026-10-27 02:02:46 +01:00
Niko1221 8c0301484e Conversation cache: skip a restore only when a card would have to create its copy stream and has no room for it (#2409)
Problem. #2408 / PR #2409: restoring a large parked conversation on a layer split ended the server with a CUDA out of memory
on a later card. The PR checks the free VRAM of every card against --vram-reserve-mib before a restore. Its own review
points out that a loaded card is always below that reserve, so the check would skip nearly every restore on a split (and
on one GPU). It is not ported.

What a restore allocates. conversation_snapshot_restore writes into the session's existing buffers (pools(st), the GDN
state, the indexer tails); refresh_residency only launches kv_stream_reset or copies into the existing ring. The elastic K/V
growth (kvg_ensure) allocates, but only on one GPU. The one thing a restore can create on a card is the calling thread's
private copy stream for it (conversation_copy.hpp private_stream), which it takes from the pool of finished threads when
there is one (#3350). A failed restore is fatal by contract (the session may be half written). So the need is one stream
and one event, and only for a card whose stream does not exist yet (a snapshot loaded from disk, or a restore on a thread
that never copied on that card).

Change. conv_cache.hpp: kRestoreFirstUseMib (64) and restore_fits(free, stream_ready, n, need): a restore is refused only
for a card that has no copy stream yet and less than 64 MiB free. conversation_copy_stream_ready(device) says whether a
card has one. generate.cpp: before the restore, each card of the run is checked; a snapshot that would not fit goes back to
the cache and the prompt is read again, with a log line. A card that has its stream is never refused, however full.
No sample (a failed cudaMemGetInfo) means the restore goes ahead as it did.

Tests. conv_cache_test: the predicate (cards with a stream are admitted at any free size; the edge at the need; no
sample / no cards / no need). conversation_copy_streams_test: copy_stream_ready before and after the first copy, in the pool
after a thread ends, and the measured cost of a card's first copy stream and event (printed; the test requires under
16 MiB, so 64 MiB leaves a wide margin). conversation_snapshot_test: after the first restore, six more restores of two
images leave the free VRAM where it was.

Not covered. The pure predicate and the generate.cpp code build on z440 (CUDA 13.0); the GPU tests are listed in the stream
notes. No model run: gates/2409-restore-guard.sh (a held-VRAM start on a layer split) shows the skip line and an
unchanged answer on the second turn.
2026-10-27 02:02:46 +01:00
Niko1221 2a9a67c174 Later-card VRAM: audit fixes for failed allocations, the reserve arithmetic as a tested helper, a per-card out-of-memory report (#3638 #3702 #3797)
Problem. On a layer split the later cards are left with very little VRAM after loading (#3702: 68 MiB on the last card; #3638:
268 and 666 MiB, then every batched prompt read of more than ~400 tokens ends non-finite after hours; #3797: the weight arena
fails with 320 MiB free and the message does not say who holds the rest). The window-graph eviction (12b0bce95) and the copy
stream pool (#3350) are in already; this is the rest of the code audit of what is allocated after load.

Audit. The table of every lazy allocation after load (what, when, size, how a failure is handled) is in
phaseA-tests/notes-027-gh-eng-d.md. Bugs found and fixed here, each small:
- verify.cpp capture / capture_batch and mtp.cpp finish_capture: the cudaGraphUpload and its sync were only printed. The
  upload is a warm-up, but a failure (no VRAM) stayed as the thread's pending error and the next kernel wrapper's
  cudaGetLastError ended the engine. The error is now cleared and the first launch uploads the graph.
- verify.cpp capture_commit and mtp.cpp finish_capture: the commit graph called cudaGraphInstantiate directly (no
  eviction of older layouts, unlike the window graphs) and reported the status of the end-of-capture call, which is success,
  so the message read "no error". capture_commit now goes through instantiate_evicting and reports the instantiate's status.
- instantiate_evicting instantiates into a local and sets the member only on success (what a failed instantiate leaves in
  its output is not specified, and a non-null member means "captured").
- prefill.cpp carve: after a failed take of the shared region the sub-allocators had no base and each cudaMalloc'd its
  buffer (hundreds of MiB in all) on the card that had just run out. carve now returns at once; the caller reports the
  failure as before.
- generate.cpp: a failed set_stage_helper / set_peer is reported and the run goes on, but its CUDA error stayed pending;
  cleared.
Found and not changed: relend() re-plans only the first participant (#3638 comment). Read: it runs only from the VRAM
command, which is off on a layer split, so it cannot touch a later stage's loan. Other candidates (ignored cudaEventCreate for
the pipelined fences, unchecked expert copies, cudaFuncSetAttribute flags set before success) are in the notes.

Changes for the report of #3797 and the reserve (#3702).
- include/strata/program/gpu_report.hpp: on a failed weight-arena allocation (first card and each stage) the engine now
  also prints, for every card of the run, the free and total memory and the largest other programs that hold memory on it
  (NVML, loaded at run time; where the driver does not list them it says so). Text only; the failure is the same.
  tests/program/gpu_report_test.cpp pins the text.
- include/strata/program/stage_reserve.hpp: the arithmetic of stage_room (what the split keeps free on a card) moved out of
  generate.cpp unchanged; tests/program/stage_reserve_test.cpp compares it with the old expression over a grid. The file
  also holds a PROPOSAL for a larger window term on a later card (96 MiB + 6 MiB per further window size + 48 MiB, 174 MiB
  for --spec 4) which nothing calls: the default placement is not changed. gates/3702-later-reserve.sh measures it.
- The low-VRAM warning of a later card now adds how many MiB the plan meant to keep free there and how far off that was.

Tests: gpu_report_test, stage_reserve_test (host only, built on z440).
Not covered: no model was run. What the ~700 MiB the plan does not count on the last card is (window graph sets, kernel code
loads) needs the run in gates/3702-later-reserve.sh.
2026-10-27 02:02:46 +01:00
Niko1221 11b0490228 tests: STRATA_QFUSE parity for a window of two groups and for the GDN q8_1 destination (#3948)
Problem. #3948 reports that STRATA_QFUSE=1 changes the verified text and lowers MTP acceptance on an RTX 3080 (sm_86); the
control (QFUSE off against off) was noisy too. The reporter read the code and suspects the per-group counter in
fused_gr.cu gr_q8_tail (not reset when a graph replays with another T) or the nat_xq_ offsets of a window of two groups.

Audit (no model run). The counter is reset by the block that finishes the group (atomicExch before it reads the 32 values),
so a launch leaves it at zero whatever T is; two blocks own a 32-column group in every kernel that calls gr_q8_tail, the
32-column kernel (gr_up_q32_kernel) quantises on its own without a counter; the fence pattern is the usual
threadFenceReduction one. The two destinations are consistent with their consumers in verify.cpp: the attention half's
xq_ is indexed from the group's first token (the projections read it from row 0), the expert half's nat_xq_ by the window's
token (the experts read it from tb). The GDN side (verify_kernels.cu gdn_q8_1_store, rows from t_out_begin) matches its
consumer too. I found no defect by reading. One thing the tests below will show if it matters: nvcc ignores the
`#pragma clang fp contract(off)` in gdn_q8_1_store, so on CUDA the ds.sum field could differ by one ulp from the separate
quantiser (the q8 bytes would not).

Tests (synthetic data, no model).
- gr_parity: the engine's G = 2 call pattern - the window split at (T + 1) / 2, two fused_gr_read_multi calls in a row on
  one stream with the same counters, group-relative destinations (xq_) and window-indexed ones (nat_xq_), direct and as a
  graph replay; fresh random inputs for every one of 200 repetitions for each T = 2..8; after every run both groups' q8_1
  bytes equal native_quantize_q8_1 of `mixed`, nothing outside the written ranges changed, and all counters are back at zero.
- gdn_parity: gdn_step_norm_multi with a q8_1 destination for every T and t_out_begin (verify and commit calls): the bytes
  equal native_quantize_q8_1 of the output rows, the outputs and the state are the ones of the call without it, and
  nothing is written past the rows. A second ctest, gdn_parity_split, runs all of gdn_parity with STRATA_GDN_SPLIT=1 (the
  4-blocks-a-head step, which has its own output-norm kernel).

Not covered. The reporter's model run (QFUSE on/off, MTP, 40 prompts): gates/3948-qfuse.sh. The tests were built on z440
(CUDA 13.0, sm_86); their GPU run is listed in the stream notes.
2026-10-27 02:02:46 +01:00
Niko1221 ebe3383a3f PLE reader: a failed read no longer leaves the next ticket waiting for ever; stress test for the direct table reader (#3602)
Problem. #3602 reports a deterministic all-logits NaN with --ple-io direct on IQ3_S (Windows, two GGUF shards) where
--ple-io mmap passes; a diagnostic build found non-finite values already in the gathered PLE embedding of one token. The
bytes the model got were not the file's (the largest finite value, 7.9e6, is what a random fp16 scale gives an IQ4_NL
row). This commit audits the reader and builds the test the report asks for; it does not claim to have found that
fault.

Audit (no model run). src/ngram/ple_reader.cpp, src/platform/direct_file.cpp and the consumers (PleTable gather_batch /
prefetch_rows / collect, prefill's double-buffered ple_gather, verify's windows): rows are copied and cached under the
reader's mutex after a complete read, short reads are checked per row, straddling rows get a two-page read, the offset
arithmetic is 64-bit, the set-associative cache and the slot buffers are only touched under the lock, and prefill's two
host buffers are ordered by the ple_copied events. I found no way to hand out a row before its read completed or from a
stale buffer.

One real bug, in the error path. After a failed read the I/O worker parks for good (worker_loop), but issue() went on
queueing jobs, so the next collect() waited for them for ever. issue() now returns a ticket with no jobs once the reader
has failed, and collect() returns the error at once (and drops the ticket). Same change in the SYCL copy
(sycl/src/ngram/ple_reader.cpp).

Tests.
- src/ngram/ple_reader_stress_test.cpp (ctest ple_reader_stress): 7 table shapes (header 0/192/4000/4051, rows of 1, 90,
  110, 160 and 4096 bytes, with and without bytes after the table) x 15 reader configurations (io thread or caller,
  in-flight 1/2/3/64, row cache of 0, 16 and 100000 rows, injected completion delays, keep-alive), four client threads
  with one to three tickets outstanding collected in any order; every delivered row is compared byte for byte with a
  buffered read of the file; collected buffers are compared again later (a late write would show); rows on every page
  phase, the last rows, out-of-range rows; the reader after a failed read (the file is cut short under it); with --big a
  5 GiB sparse table with rows around 2^31 and 2^32 and at its end.
- table_is_consistent_under_concurrent_use in src/ngram/ple_reader_test.cpp: one PleTable used by a prompt thread
  (big gather_batch), a decode window (prefetch_rows, gather_batch) and single gathers at once, against the mapped reader.

Results (z440, Linux, g++ 13, reader + direct_file + test built alone): plain build with --big OK; ASan+UBSan OK. The
error-path test against the unchanged reader hangs (the test exits 3 after 20 s); with the change it passes. ThreadSanitizer
flagged a data race that was a mistake in the test (the delay changed from a second thread in the caller-thread mode,
which has no lock); fixed in the test.

Not covered. The NaN itself: it needs a run on the reporter's configuration (gate script gates/3602-ple-direct.sh). The
Windows run of the new test is listed in the stream notes.
2026-10-27 02:02:46 +01:00
Niko1221 ba8b2bcb9c Merge f-027-gh-eng-a (#3622 ARM64 / DGX Spark port, #3806 Gyro-L TQP5/6/7)
# Conflicts:
#	cmake/ggml_tq_overlay_test.cmake
#	src/core/verify.cpp
2026-10-27 01:59:31 +01:00
Kevin Read 38b06316ef Gyro-L: read the pair-trellis types TQP5/TQP6/TQP7 (ggml 147-149) on the CPU, CUDA and the MMQ prompt path (#3806)
Gyro-L (agentionai's 50 GiB file) stores its experts as a pair trellis: 128
values per block, a fp16 scale and 64 steps of K bits (K = 5, 6, 7), two
weights per step taken from the first pair of the TQ2_T codebook entry the
step's state selects. The engine refused the file because it did not know the
three type ids. This adds them next to the existing trellis types 144-146
without changing those.

What changed:
- gguf_reader.hpp / tools/gguf_reader.py: names and block sizes (42 / 50 / 58 bytes).
- tq_cpu.*, tq_states.hpp, tq_avx2.cpp: tq_supported / tq_kbits / tq_block_bytes
  for the new ids (the bytes per block are 2 + 8K for the pair layout, 2 + 4K
  for the others), a scalar pair kernel (the reference) and an AVX2 pair kernel
  that is bit for bit equal to it; AVX-512 uses the AVX2 pair kernel.
- iq_kernels.cu: TqInfo, the lane decode (tqp_states8, a byte window), the
  dot product, the decode-once Split, the fp16 dequantizer and the format lists
  for 147-149. They are reached only by a pack that holds these types.
- moe_mmq.cu, strata_ggml_tq_overlay.cmake, third_party/ggml-tq/tq.cuh,
  ggml-common-tq.h: the MMQ prompt path for the three types (overlay
  enumerators, traits, tile loader branch, instances).
- native_expert_parity: names tqp5-7 and skips its synthetic check, with a
  message, on a ggml that has no trellis traits.
- docs/GYRO.md: the "not supported" sentence now says how Gyro-L is read.

Changes to the contributor's patch:
- tq_cpu.cpp called the AVX2 pair kernel without the x86 guard the rest of the
  file has, which breaks the Metal build (it compiles tq_cpu.cpp without
  tq_avx2.cpp): the pair rows go through one helper that falls back to the scalar kernel.
- moe_mmq.cu named GGML_TYPE_TQP5-7 under STRATA_MMQ_TQ, but the public
  agentionai llama.cpp has the older trellis types and not these: a new
  STRATA_MMQ_TQP (set for the overlay and for a tree that has the tqp instances)
  guards them, so that tree still builds. The HIP overlay lists get the three
  instances too, and ggml_tq_overlay_test checks them (and sets the CMake
  policy its IN_LIST needs).
- native_expert_parity: the skip test compared a type-name pointer to null, but
  ggml_type_name of an id past the table reads past it (the test crashed on
  an upstream ggml); it compares against GGML_TYPE_COUNT.
- docs/GYRO.md keeps the plain description and leaves out the contributor's
  measured speeds (we have not repeated them).

Tests added: src/kernels/cpu/tqp_cpu_test.cpp (no ggml, no model: the scalar
kernel against a bit-by-bit reference from the format definition, every ISA
equal to scalar for 1-8 tokens and for gate/up rows, the type predicates for ids
0-299 against the old formulas), the pair types in tq_pascal_bench (fp16
dequantizer bit for bit against the definition, decode kernels within 3e-2,
decode-once == per-column), tools/test_gguf_tqp.py, and native_expert_parity
tqp5/6/7 registered when the ggml tree has the tqp instances.

Checked (no model): tqp_cpu_test passes on x86 (AVX2 on z440; AVX2 and
AVX-512 on the owner PC with MSVC); tq_pascal_bench on an RTX 5070 (sm_120,
MSVC build): 0 failures, the TQP dequantizer matches the definition in all
bits, and the checksum lines of TQ2_T/TQK6/TQK7 are byte-identical to the
build without this change. Builds: CUDA sm_86 (gcc), sm_60 (CUDA 12.9),
sm_120 (MSVC), SYCL, and the CPU files for aarch64.

Not covered: the contributor's own parity runs need a ggml tree that carries the
types (not public here), and the 50 GiB model run (load check, coherent
answer, 32K prompt, MTP, Gyro-S/M identity) is still to be done. Metal, SYCL and
HIP have no kernels for the types; HIP was read, not built.
2026-10-27 01:57:57 +01:00
Niko1221 1e57a924d0 Merge f-027-gh-eng-m (#3416 gated MTP: opt-in proposal law, window-invariant CPU dispatch, draft policy)
# Conflicts:
#	docs/DETAILS.md
2026-10-27 01:48:34 +01:00
Keyan Hu ca17d0f5cd spec: a gated request's lookup policy learns from committed rounds only, DraftPolicy choose() unchanged otherwise (#3416)
Problem (KeyanHu-git): the gated proposal needs the choice between the MTP window and a
lookup window to be fixed by the history, not by the draws of the rounds it will verify.
Freezing the policy for the whole request (the first commit) does that, but also freezes
the cost table, so a repeat-heavy request kept the window sizes it inherited. In the
author's diagnostic workload the frozen policy chose mean lookup windows of 5.87 and 3.38
on two repeat requests; with this change 4.31 and 4.05 (from the PR text; not
reproduced here).

Change: the request's copy of the DraftPolicy (source_policy) gets three calls, used only
when gated_proposal is on:
- observe_committed(): the round's accept and probe counts, applied at once. A lookup
  window counts as a probe of its size (tried_), so a size whose cost is only guessed is
  tried kProbes times, not every round.
- defer_cost(end, t, ms): the round's time is held. A time covers the window and the next
  MTP draft, so it depends on every Philox counter up to
  max(p + T - 1, p + a + 1 + S_mtp).
- apply_costs(committed): applies, in order, the held times whose end is behind the
  committed position, with the same update observe() makes (including the STRATA_SPEC_COST
  clamp), in the context bucket the round ran in; delayed_n_ keeps a held time from
  counting as a second probe.
choose() counts the probes as cost_n_ - delayed_n_ + tried_.

Port note: the DraftPolicy on the current tree keeps its costs per context bucket
(STRATA_SPEC_COST), so tried_, delayed_n_ and the held times carry the bucket. A gated
request does not run with STRATA_SPEC_COST (it falls back to the older rule), so its
copy stays in bucket 0.

Default path: unchanged. choose() reads two new terms; on any path that never calls the new
functions tried_ and delayed_n_ are +0.0 and cost_n_ - 0.0 + 0.0 == cost_n_ exactly.
Checked by driving the old DraftPolicy (namespace-renamed copy of f-027-gh-eng) and the
new one with the same random call sequences (observe, observe_chain, observe_probs,
set_context, seed_costs, set_robust, choose, chain, keep_draft; 400 sequences of 3000
steps): 25,983,538 bitwise comparisons of picks, costs and rates, 0 mismatches; one
observe_committed call makes the same harness report 60,528 mismatches. generate.cpp:
nothing calls the new functions when gated_proposal is false.

Tests: draft_policy_test (the PR's added assertions, on the bucketed class) and
spec_admission_test pass on z440 and on the owner PC.
Not covered: the sampling argument for the delay boundary (that a held time cannot steer
a window through a counter that is drawn again) is the author's; a unit test cannot show
it, only the distribution run (gate script 3416-spec-prob-dist.sh, arm NEWLK) can.
2026-10-27 01:45:36 +01:00
Keyan Hu 499e963593 native experts: window-invariant CPU arithmetic for STRATA_SPEC_PROB decode, Default dispatch unchanged (#3416)
Problem (KeyanHu-git): the rejection rule of the previous commit judges each row against
the target's probabilities p. For a native (IQ) pack the CPU experts round a token's rows
differently depending on how many tokens share its expert in the window (the grouped
kernels against ggml's one-token dot, #152). So p of one row moved with the window width,
and the same sampled text could depend on the draft length. On random blocks a grouped
Default call differs from the token alone in 700 to 12000 of about 18000 output floats
for the IQ gate/up and IQ4_NL down formats.

Change: NativeArithmetic { Default, WindowInvariant }, fixed for one pool batch.
native_gu_rows / native_down_rows become thin wrappers of one template with the arithmetic
as a parameter. The Default instance keeps every old branch (the new conditions are
compile-time true there). WindowInvariant gives each token the bits of the one-token call
on this build and CPU: kernels that already match it stay grouped (NVFP4, Trellis and
Hadamard rows, KQ256 when ggml runs its AVX2/FMA dot); the IQ rows that differ use
iq512_reference_gu_rows (types 18, 21, 22: the same decode, ggml AVX2's accumulation
order, one weight decode per group) or iq4nl_reference_rows (IQ4_NL down), and anything
else falls to the one-token dot per token. ExpertPool::run_split_multi_native takes the
arithmetic and expert_pool_dispatch_multi passes it; generate.cpp uses the invariant
pool only when the MTP drafts probabilistically (mtp.prob(), i.e. only with
STRATA_SPEC_PROB=1). Prefill, greedy decode and every other decode run Default.
The pool picks the entry point with one compare and a direct call per chunk.

Default path (STRATA_SPEC_PROB unset): unchanged, shown by running the old and the new
native_gu_rows / native_down_rows (Default) on the same random blocks: 61 format, width
and amplitude lines (IQ2_XXS/XS/S, IQ3_XXS/S, IQ4_XS, NVFP4, Q4_K, Trellis with and
without folded Hadamard, IQ4_NL / Q2_0 / Q5_1 / Q8_0 down; widths 1 to 8; row sub-ranges),
output hashes identical in 10 settings of the environment switches on z440 (AVX2) and
in 13 on the owner PC (Zen4, AVX-512, MSVC), including STRATA_NO_IQ512, STRATA_IQ_GATHER
and STRATA_IQ_MT_MIN=1. g++ codegen of the AVX-512 row kernels is not byte-identical for
5 of 87 functions (inlining and register allocation of the same source); the AVX-512
kernels were run only under MSVC.

Tests: native_arithmetic_parity, eight ctest configurations (default, AVX, KQ256,
IQ_MT_MIN=1, no512, ggml only, NVFP4 AVX2, NVFP4 ggml), registered in CMakeLists.txt and
sycl/CMakeLists.txt. It compares the invariant entry points at every group width 1..8
with the one-token calls, bitwise, for the IQ, NVFP4, Q4_K/Q5_1/Q8_0, Trellis and
Hadamard formats. All eight pass on z440 and on the owner PC; the existing CPU expert
tests (iq_avx2_parity, iq_avx1_parity, iq512_gather_parity, pool_tasks_test,
nvfp4_cpu_parity, router_dot_parity, q2_bitplane_parity) pass.
Not covered: an ARM CPU (not built here), an AVX-512 machine with fast gathers (Sapphire
Rapids), the layer-split call site, and any model run.
2026-10-27 01:45:35 +01:00
Keyan Hu f4612b1add spec: STRATA_SPEC_PROB=1 keeps the confidence gate in the proposal law, opt-in (#3416)
Problem (KeyanHu-git, refs #2028, #2591): with STRATA_SPEC_PROB=1 the default
--spec-min-p gate tests the token drawn from the draft head's distribution q, but the
verifier still judged the draft against the unfiltered q and treated a failed gate as a
plain sample from p. That shifts the output distribution: with p = (0.5, 0.5),
q = (0.875, 0.125) and threshold 0.5 it emits (0.5625, 0.4375).

Change: the gate is part of the proposal. Keep q's mass on the tokens that pass the gate
without renormalising (mu); a draw that fails the gate becomes NULL. An offered token D
is accepted with min(1, p(D)/mu(D)); a rejection and NULL draw from the normalised
positive part of p - mu; a position with no attempted proposal is a plain sample from p.
Serve and the CLI build the same packet (core/gated_proposal.hpp) and give it to the
verifier explicitly (Verifier::set_spec_proposal, sample_tokens_proposal). The explicit
call has no plain-p fallback: an invalid packet row is reported, not sampled around.
STRATA_SPEC_PROB_GATE=top keeps its normalised-q rule.

Default path (STRATA_SPEC_PROB unset): unchanged. mtp.prob() is false, so gated_proposal
is false, the new draft-result argument is null, GatedDraftLimit does nothing,
Verifier::run takes the old branch (set_spec_q clears the measure flag), and the
sampler kernel's old instance is the old code with the new branches compiled out.
Read hunk by hunk; no hunk runs a new branch or a new allocation per token. The
default-identity model run is written as a gate script and not run yet.

Port notes against the PR as submitted:
- Unsupported combinations (parallel or multi-device decode, --lookup-chain,
  --adapt-swaps, PCIe share, --spec-min-p not finite, STRATA_SPEC_COST, DFlash2,
  STRATA_SPEC_T_CYCLE, --spec-corrupt, Metal) used to be an error that ended the
  request or the server. They now warn once and keep the older rule, so setting the
  variable never stops a run. --adapt-swaps 96 and the PCIe share are on by default, so
  the exact rule needs --lookup-chain 0 --adapt-swaps 0 --pcie-frac 0.
- The lookup source of a probabilistic window is chosen by a copy of the policy fixed at
  the request start, without asking whether the MTP's first guess agrees. Learning from
  committed rounds inside a request is the policy commit of this series.
- The CLI loop on the current tree has the out-of-memory window retry, DFlash2 and
  STRATA_SPEC_T_CYCLE; the gated path does not use the retry (a narrower window would
  change the room the packet was built for). Serve keeps merged first windows
  (STRATA_ADMIT_MERGE_FIRST) at their length.
- docs/DETAILS.md: two short bullets instead of the PR's long paragraph (that paragraph
  no longer exists on this tree). bench/results and the PNGs are not ported.

Tests: gated_proposal_test and spec_admission_test (default, pick, top, exact-top-only)
pass on z440 (CPU, g++ 13) and on the owner PC (MSVC). On the owner PC's RTX 5070
(sm_120, MSVC build, synthetic data, no model) spec_proposal_parity, spec_verify_parity and
sampler_parity (default, one-block and old-kernel configurations) pass. Also built:
CUDA sm_86 on z440 and boston, SYCL on P1006 (engine and host tests). The Metal path is
not built here.
Not covered: a model-level distribution test (chi-square of the first tokens against
plain sampling) and the identity run with the variable unset. Both are written as gate
scripts (3416-spec-prob-dist.sh, 3416-default-identity.sh) and not run.
2026-10-27 01:45:35 +01:00
Niko1221 9ac278e124 docs: release notes - split serving (#4000) filled from docs/SPLIT_SERVE.md: opt-in, exact, slower than the layer split on two A6000, credit to MarcoTool 2026-10-27 01:40:17 +01:00
Niko1221 ccf18ea143 Merge f-027-split: split serving (#4000), opt-in, first version
Prefill on one card, decode on another, session handed over through RAM; exact token for token (state hash match at 4K
and 16K on 2x A4000 and 2x A6000). Nothing changes unless split_serve / --split-serve is set. On two alike cards it is
slower than the layer split on every workload measured (docs say so); it can only pay on dissimilar cards. Idea and
manual setup by MarcoTool (#4000).
2026-10-27 01:39:21 +01:00
Niko1221 3480bf2b8c Split serving: docs with the measured results, check tool no longer hashes the state in speed runs
Measured on 2x RTX A6000 (IQ3_S): slower than the layer split on every workload;
exactness checks pass (Q2_0 on 2x A4000, IQ3_S on 2x A6000).

Idea and manual setup by @MarcoTool in #4000.
2026-10-27 01:38:32 +01:00
Niko1221 597df5e601 docs: the KVA opt-in is split 36, final (the split 24 variant is a 0.2.8 study); release notes marker closed 2026-10-27 01:35:28 +01:00
Niko1221 1b89043157 docs: the 1.5 route prior check - all five tasks the grader failed were read: three were grader flags, two were the long-winded answers 2026-10-27 01:29:32 +01:00
Niko1221 129739440a docs: release notes 0.2.7 draft, audit fixes of the engine review and the decision markers brought up to date 2026-10-27 01:28:23 +01:00
Niko1221 8191cb91b4 docs: the route prior stays at 1.5 after the 61-task check on the RTX 3060 (DETAILS.md and the 0.2.7 release notes)
No copy, code-constant, JSON-spelling, retrieval or unanswerable defects (retrieval 12/12, JSON 3/3, unanswerable 9/9, extraction equal to off); two long-winded answers, which the runs with the prior off also show two or three of per run. Story +15%, code +37% against off.
2026-10-27 01:26:22 +01:00
Niko1221 fb8f7d9df6 docs: release notes 0.2.7 draft (every merged item and the owner decisions of 26-27 October, measured numbers per box, markers for what is still pending) 2026-10-27 01:25:46 +01:00
Niko1221 df20d77845 Docker docs, compose defaults, scripts and the file test name the 0.2.7 image tags (0.2.7-cuda13 and the other flavors) 2026-10-27 01:25:46 +01:00
Niko1221 9e20319069 layer-major prefill: declines on STRATA_EXPERT_NIBBLE builds (#3976)
f-027-gh-eng gained STRATA_EXPERT_NIBBLE (4-bit expert slots, the sm_60 experimental
build): only run_impl gathers such a layer for MMQ. The layer-major walk would read
the slot as GGUF blocks, so lm_prepare now declines when any layer is converted and
the prompt takes the chunk-major path. Compiled out of every build without
STRATA_EXPERT_NIBBLE_BUILT.
2026-10-27 01:13:13 +01:00
Niko1221 bd8a254453 Merge f-027-gh-eng into f-027-gh-eng-pm (#4121 on top of the #4010 pipelined adaptive rounds) 2026-10-27 01:12:47 +01:00
Niko1221 0959545e21 Merge f-027-gh-eng into f-027-gh-eng-lm 2026-10-27 01:12:00 +01:00
ferstar a4c41a92ee Build the engine for ARM64 (DGX Spark / GB10) without changing the x86 build (#3622)
The engine did not build on aarch64: the CPU expert path assumed the x86 AVX
kernels, the CPUID probes and the x86 spin/fence intrinsics. After this change
the strata target compiles on aarch64 (CUDA host code, native GGUF packs).
On x86 nothing is selected differently.

What changed:
- CMake: STRATA_CPU_X86 is decided from CMAKE_SYSTEM_PROCESSOR (an unset
  processor stays the x86 build). On aarch64 src/kernels/cpu/portable.cpp
  stands in for the AVX kernel sources; native experts run on ggml-cpu's NEON
  dot products. -ffp-contract=off on aarch64. STRATA_ISA_FLOOR is ignored with
  a warning there.
- include/strata/platform/cpu_relax.hpp: strata_cpu_pause / strata_store_fence
  map to _mm_pause / _mm_sfence on x86 and to yield / dmb oshst on aarch64.
  verify.cpp, pool.cpp, session.cpp, mtp.cpp and graph.cpp use them.
- expert_layout.cpp: the CPUID/XCR0 probes compile on x86 only and report no
  AVX on other CPUs.
- generate.cpp: --cpu-oracle-q8-0 is rejected on a non-x86 CPU.
- setup.py: nvidia-smi's "[N/A]" memory on GB10 (unified memory) takes the
  pool size from /proc/meminfo on ARM64 only; the CPU probe no longer needs
  the x86 "flags" line; setup warns that the ARM64 engine is compiled on the
  PC and which packs work.

Changes to the contributor's patch, found while moving it onto the 0.2.7 line:
- portable.cpp had no iq512x_rows / iq512x_supported / iq512_gather_on (added
  to the x86 kernels after the PR was written): the aarch64 link would fail.
- The Metal build (Apple silicon) shares these files. It keeps its barrier
  (dmb ishst), its plain router loop (its bf16_rows_dot_multi is a stub), its
  Metal start-up block in generate.cpp and its "unknown" CPU name.
- The CPU side of the GPU doorbell: strata_load_fence() after the spins that
  wait for a flag the GPU raised (dmb ishld on ARM, nothing on x86 and on
  Apple silicon), so the data written before the flag is not read early.
- A card whose memory nvidia-smi does not report stays unlisted on x86 (the
  new fallback is for ARM64 only).
- tests/terminal/test_setup_arm64.py.

Checked: with the x86 CUDA build (sm_86, gcc 13 -O3, z440) the objects of
verify, session, mtp, graph, expert_source, pool, expert_layout and generate
are byte-identical before and after (the x86 wrappers are macros for the same
intrinsics). Every C++ file of the strata target compiles for aarch64 with
clang, and the symbols of the CPU kernel library link up (syntax and symbols
only: no ARM machine, no ARM run). The Metal-shared files compile for
aarch64-macos with STRATA_GPU_METAL.

Not covered: no ARM hardware here. The tests that include x86 headers
(iq_avx2_parity and the other AVX parity tools) and the DeepSeek target do
not build on aarch64; the strata target does not need them. Memory ordering
on ARM is reasoned, not run. The Q2_0 and Gyro packs need the x86 kernels.
2026-10-27 01:11:53 +01:00
Niko1221 2651c4a96b Merge f-027-int2 (2a2264c21) into f-027-gh-eng
# Conflicts:
#	tools/calibrate.py
2026-10-27 01:11:52 +01:00
Niko1221 2579635138 sm_80 defaults of #3779 (wide dense dequant, MMVQ-IL sm_80 tables) follow STRATA_DEFAULTS=026: back to the 0.2.6 kernels and the shared table; explicit =1 and =0 still win; arch_defaults_test checks the rule; release-notes line 2026-10-27 00:58:42 +01:00
Niko1221 1e278a280d Merge f-027-gh-eng-n (#3779 sm_80 tuning: wide dequant, MMVQ-IL sm_80 tables and the opt-ins)
# Conflicts:
#	CMakeLists.txt
#	docs/MMVQ_IL_TABLE.md
#	src/kernels/cuda/native_mmvq.cu
#	src/kernels/mmvq_il_parity.cpp
#	src/program/generate.cpp
2026-10-27 00:57:59 +01:00
linxuhao 56d98f164e docs: the sm_80 defaults and the opt-in sm_80 switches of #3779, STRATA_GRAPH_PRIO and STRATA_MMVQ_IL_PF (#3779, #3712)
DETAILS.md gets two sections: the decode switches (GRAPH_PRIO, MMVQ_IL_PF) with the
two sm_80-only defaults named, and the prompt switches (PF_FUSED_NT / BATCH / F16 /
WS, GDN_TC, PLE_COMPACT / PLE_COPY_STREAM, EMBED_VRAM, DEQUANT_SIDE) with what is
bitwise and what is rounding-level. MMVQ_IL_TABLE.md gets the sm_80 row of the table
list and a paragraph on the hoisted-load tables (kIlPf_80 / kIlPf58_80, Q6_K only) and
on measuring a card with mmvq_il_parity --bench --emit-table --pf 0,3,4.

Rewritten from the PR's text: every figure is labelled as the reporter's (one CMP
170HX; we have no sm_80 card and measured none of them), and the prompt-switch
section is new (the PR documented only the decode switches and the conversation
cache).

Original docs by linxuhao (PR #3779, commit 6d6f0f39).
2026-10-27 00:52:53 +01:00
linxuhao 8b95bffb0b Decode: opt-in STRATA_MMVQ_IL_PF=1 hoisted-load, exact-integer interleaved GEMV kernels for Q6_K (#3779, #3712)
Problem: the interleaved small-batch matvec (native_mmvq_il) has one weight block
in flight per warp: its four `if (kbx < bpr)` sections each sit behind their own
branch, so on a GA100 (ncu: 432 GB/s on the qkv Q6_K matrix) the loads of a group
are not issued together.

Change, opt-in (every variant is bitwise native_mmvq): native_mmvq_il_pf_kernel
issues every weight load of a group (4 virtual warps x R rows) before any of the
group's dots, with the addresses clamped to the row's last block for a tail, and
with the ring variants the next group's loads before this group's dots. The
per-lane accumulation order is the plain kernel's. P2's forms replace
conversions by exact ones where the value is a small integer: (float)p becomes
p + 1.5 * 2^23 minus 1.5 * 2^23 for |p| < 2^22, and the saturating byte subtract
of Q6_K's codes (0..63, so nothing saturates) becomes two instructions. Q4_K,
Q5_K, IQ4_XS and Q6_K have the variants; Q3_K (the Q2_0 pack's type) keeps the
plain kernel.
- STRATA_MMVQ_IL_PF=1: the measured table (kIlPf_80 / kIlPf58_80: Q6_K only, read
  only when the card is sm_80; on every other card 1 does nothing);
  =2..6: one variant for every type and shape (A/B only, any CUDA card).
- Unset or 0: the plain kernels through the same launch code (the variant argument
  is 0, and the added code is one cached environment read and a switch).
- CUDA only; HIP has no interleaved path.

Tests: mmvq_il_parity --pf a,b,.. bit-checks each listed variant against
native_mmvq for every type and shape, T = 1..8 and rows 0/1/2/4, and with --bench
times them interleaved (PFBENCH lines) and prints the joint (variant, rows) table
in the source form of kIlPf_<cc>. New ctest entry mmvq_il_parity_pf runs it with
--pf 0,2,3,4,5,6 on whatever card is present.

Reporter's numbers (linxuhao, CMP 170HX): decode +1.4 % (two reference / on / reference /
on sets) over the plain kernels; with STRATA_GRAPH_PRIO decode +1.3 % in their
per-feature pair (4K prompts +3.9 %, 16K +1.2 %, 52K +0.2 % in the same pair, which
the reporter reads as noise). Q4_K, Q5_K and IQ4_XS variants tie the plain kernels
there. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 116992ab).
2026-10-27 00:52:53 +01:00
linxuhao c2d3fe5000 mmvq_il_parity: the L2-residency arms (--hot / --gap / --persist / --ab), the clock warm-up and MMVQ_IL_TMIN (#3779, #3712)
Harness only; no engine code. The bench part of mmvq_il_parity times the
interleaved matvec cold (24 copies of the weights rotated past the L2), hot (one
copy), hot with other weights' reads in between (--gap MB, what a window's GEMVs do
between a prefetch and its consumer), and with the timed weights in a persisting L2
set-aside (--persist); --ab runs the five arms per case in one process, because
the clocks ramp over the first ~100 ms of a run. A 150 ms warm-up of launches runs
before the first timed case, and MMVQ_IL_TMIN bounds T from below.

The parity part is unchanged: without --bench the same checks run in the same
order (every rows choice against native_mmvq, bitwise). The bench code is CUDA only
(the test is built with STRATA_ENABLE_CUDA); the persisting-L2 calls are left out
under STRATA_HIP_GFX906.

Original change by linxuhao (PR #3779, commit 1f4b8139).
2026-10-27 00:52:53 +01:00
linxuhao 0049fa73ff Decode window: opt-in STRATA_GRAPH_PRIO=1 gives the window graph's kernel nodes their stream's priority (#3779, #3712)
Problem: the decode window graph runs every kernel node at the launch stream's
priority, so the one-block indexer kernels of the side branch wait behind the
queued blocks of the big q GEMV, and the shared expert competes with the routed
experts.

Change, opt-in (bitwise by construction: no kernel changes, the same launches on
the same inputs; only the dispatch order moves): with STRATA_GRAPH_PRIO=1 the
side mixer's branch stream (GDN gates, QSA indexer chain) is created at the
device's highest priority, cs_ and side2_ in the middle, the shared expert's
stream at the lowest, and the window graph is instantiated with
cudaGraphInstantiateFlagUseNodePriority. If a driver did not copy the stream
priority into the nodes, the capture-time notes (the branch forks and the
priority streams' tails) set the attribute per node. One log line per captured
window prints the node priority histogram. A card whose priority range is 0..0
changes nothing.

CUDA only: the members and the three methods sit under !STRATA_USE_HIP &&
!STRATA_HIP_GFX906 && !STRATA_GPU_METAL; graph_prio_on() is false there.

Merged by hand with #3702 (the window-graph eviction): instantiate_evicting now
takes both keep_t and the flags, and the retry after freeing other window
sizes keeps the flags (the merged code would otherwise have dropped the node
priorities on that retry).

With the variable unset: every stream is created as before and the capture-time
notes return at once.

Tests: none of its own (a priority is a scheduling hint with no value to compare).
The window graphs are exercised by the engine; see the checklist for the
reporter's greedy-answer identity check.

Reporter's numbers (linxuhao, CMP 170HX, IQ3_S, --spec 3, 3K context, interleaved in
one session, each arm with STRATA_QIL=1): reference 125.3 / 126.6, GRAPH_PRIO
127.8 / 129.0, reference 125.0 / 126.8 tok/s, +1.8 %. Greedy 11 of 11 identical. In
the later per-feature pair together with STRATA_MMVQ_IL_PF: decode +1.3 %. Theirs, not
measured here.

Original change by linxuhao (PR #3779, commit 68c487c3).
2026-10-27 00:52:53 +01:00
linxuhao 0e63d59c01 docs: conversation cache copies on a slow host link, STRATA_CKPT_ASYNC and the park switches (#3779, #3712)
One section in DETAILS.md for the switches the four conversation-cache commits
add: what each copies where, its VRAM and host memory cost, that they stay off
on HIP, gfx906 and Metal, and the measured times.

Rewritten from the PR's section: the times are labelled as the reporter's
(linxuhao, one CMP 170HX on PCIe Gen2 x4; we have no sm_80 card and measured
none of them), the independent RTX 3090 data of bitaccesscomau is added, and the
pool row describes STRATA_CKPT_POOL_READY=N instead of the PR's change to the
meaning of STRATA_CKPT_POOL=N (STRATA_CKPT_POOL=1 is unchanged).

Original section by linxuhao (PR #3779, commit a04f4399).
2026-10-27 00:52:53 +01:00
linxuhao 85e3dde522 Conversation cache: opt-in STRATA_PARK_VRAM_MIB=N keeps parked conversations' K/V in VRAM in front of the host copy (#3779, #3712)
Problem: restoring a parked conversation copies its K/V from the host over the
link: 27-107 ms over a PCIe Gen2 x4 link.

Change, opt-in and only with STRATA_PARK_ASYNC=1 (CUDA only): a park also keeps
the whole K/V of its image in a VRAM slot of its own (64 MiB granules, up to N MiB
in all, only while 1.5 GB of VRAM stays free beside it); the background copy to
the host reads from the slot, and a restore of that image copies device to device
from the slot (about 1 ms) without waiting for the background copy. Each K/V
layer carries its own join-and-release token, so any order of destruction is safe
(the layers outlive their image when they are retained for the next park). Under
VRAM pressure the elastic K/V's growth releases every slot and the stage, then
retries once (both growth sites in generate.cpp).

With the variable unset the slot code is not reached: the budget is 0, a restore
finds dev_tag 0 and takes the host path, and the growth sites see
conversation_park_vram_release_all() return 0 and behave as before.

Tests: none of its own (it needs a card with the VRAM for the slot and a parked
conversation); see the checklist for the reporters' A/B/A conversation check with
STRATA_PARK_VRAM_MIB set.

Reporter's numbers (linxuhao, CMP 170HX): restore of a 3K conversation 28-47 ->
7-12 ms; outputs identical. The first restore of a run is slower (41.6 / 56.9 ms)
than later ones (6.5-12.6 ms) and they have not found why. Theirs, not measured
here.

Original change by linxuhao (PR #3779, commit bafca410).
2026-10-27 00:52:52 +01:00
linxuhao ca378670b2 Conversation cache: opt-in ready checkpoint pool STRATA_CKPT_POOL_READY=N, gathered small checkpoint copies STRATA_CKPT_GATHER=1, park traces (#3779, #3712)
Problem (reporter, CMP 170HX, PCIe Gen2 x4): a conversation switch's park still
paid 38-52 ms for the live state, because the STRATA_CKPT_POOL pool was empty at
every park (a switch takes two checkpoints), so the 118 MB GDN vector was
allocated fresh and page-faulted; and a checkpoint's 37 small synchronous
device-to-host copies queue behind the 4 MiB DMA pieces of STRATA_CKPT_ASYNC.

Change:
- STRATA_CKPT_POOL_READY=N (opt-in, 1..16, with STRATA_CKPT_POOL on): the pool
  keeps N ready checkpoints, pre-sized to the session's state sizes with their
  pages in, topped up on a background thread after any take (not with a layer
  split).
- STRATA_CKPT_GATHER=1 (CUDA, opt-in): the small running-state vectors (ple, and
  each layer's indexer tail, dead row and block position) are gathered device to
  device into one buffer and fetched with one copy into a pinned buffer.
- The park stage is allocated rounded up to 64 MiB (a growing conversation does not
  cudaFree/cudaMalloc it every time) and a VRAM-resident K/V skips the pointer
  attribute query.
- The background helpers (pool top-up, the checkpoint DMA, the park DMA) leave the
  host's pinned core (aux_cpus.hpp); no output depends on where a thread runs.
- A parked image whose live GDN state is still being copied joins before its
  buffers are freed (SavedConversation::live_pending).
- STRATA_PARK_TRACE prints the joins, the live-state vectors' readiness and the
  K/V sub-timers.

Differs from the PR as submitted: the PR made STRATA_CKPT_POOL=1 allocate two
ready checkpoints at startup (+236 MB of host RSS) and read STRATA_CKPT_POOL=N as
a count. That changes what an existing switch does, so here STRATA_CKPT_POOL=1
keeps its meaning (the pool holds dropped checkpoints only, nothing is allocated
ahead of time) and the ready pool has its own variable, STRATA_CKPT_POOL_READY.
A user of the PR's STRATA_CKPT_POOL=2 sets STRATA_CKPT_POOL=1
STRATA_CKPT_POOL_READY=2. The PR's one-checkpoint refill after a park with
STRATA_PARK_ASYNC is replaced by the top-up, as in the PR, and so is only active
with the new variable. Merged by hand into #2526's park_source.

With none of the new variables set: the stage size is rounded up only when the
park stage is on, the extra code is trace output behind STRATA_PARK_TRACE and
the join calls.

Reporter's numbers (linxuhao, CMP 170HX): park at a 3K context 64-75 -> 21-40 ms
(all park switches together, A/B/A conversation check identical, greedy 11 of 11
identical). Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 56088e71).
2026-10-27 00:52:52 +01:00
linxuhao f73b3c86a4 Conversation cache: opt-in STRATA_PARK_ASYNC=1 parks the K/V in the background and moves the dropped checkpoints (#3779, #3712)
Problem: parking the outgoing conversation copies its K/V device to host and
deep-copies its checkpoints before the next prompt can start: 213 ms at a 3K
context, 334 ms at 13K on a PCIe Gen2 x4 card (the reporter's CMP 170HX).

Change, opt-in and CUDA only (off on HIP, gfx906 and Metal):
- STRATA_PARK_ASYNC=1: conversation_kv_save copies the device K/V device to
  device into a VRAM stage (only while 1.5 GB stay free beside it; otherwise the
  save copies as before), and a background thread moves the stage into the host
  buffers in 4 MiB pieces through two pinned buffers. conversation_park_join
  waits for it: every reader of a parked image's K/V, the cache before it frees an
  entry, and the image itself (SavedConversation::pending) before its buffers go.
  ConversationBuffer::resize_for_overwrite leaves the added bytes unfilled for
  that copy; the buffer's segments use an allocator that value-initializes only
  when a fill value is given, and resize(n, value) is unchanged for every other
  caller.
- With it, the checkpoints this request drops right after the park (all of them
  when another state comes in, else the ones off this prompt's path) move into
  the parked image instead of being copied, and a background allocation refills
  STRATA_CKPT_POOL for the next save. Only the main session's checkpoints move,
  not an idle batch slot's (the park_source of #2526 serves both).
- conversation_snapshot_save takes dropped checkpoints' storage (the existing
  STRATA_CKPT_POOL) for its copies; with the pool off, nothing is taken.
- Unset: park_drops is null, no storage is taken, the K/V copies are the
  synchronous ones; the added work is the K/V size sum for the stage request and
  a few join calls.

Differs from the PR as submitted: the PR's join in conversation_file.cpp
(put_payload) is not ported; the SAVE/RESTORE handler of generate.cpp joins the
park first (see the previous commit, which carries the PR's 33a8146a fix).
Hand-merged into the park_source / ParkSource refactor of #2526.

Not covered: if the background copy fails after resize_for_overwrite, the
parked image keeps unfilled bytes; the failure is logged. A restore of such an
image would read them. No test of the park path with a failing copy.

Tests: the PARK_ASYNC path has no unit test in this commit (it needs a card with 1.5 GB free for the
stage); the reporter's A/B/A conversation check covers it, see the checklist.

Reporter's numbers (linxuhao, CMP 170HX): park 213 -> 64-75 ms (3K), 334 -> 68 ms
(13K); greedy answers identical on their 11 prompts and the A/B/A conversation
check. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 3e0c141c).
2026-10-27 00:52:52 +01:00
linxuhao 0f7224d168 Conversation checkpoints: opt-in STRATA_CKPT_ASYNC (background device-to-host copy) and STRATA_CKPT_VRAM_SLOTS (#3779, #3712)
Problem: on a slow host link a turn checkpoint copies the whole GDN state to
the host on the request path (118 MB at 1.64 GB/s on a PCIe Gen2 x4 card, 72 ms;
an independent RTX 3090 on PCIe 3.0 x16 showed 73 ms to 62 ms with the switch).
Restoring a parked conversation copies it back.

Change, both opt-in and CUDA only (off on HIP, gfx906 and Metal):
- STRATA_CKPT_ASYNC=1: the GDN state is copied device to device into a VRAM stage
  (one buffer the size of the state, allocated at the first checkpoint save, plus
  a pinned host buffer of the same size; if either allocation fails the save falls
  back to the synchronous copy), and a background thread moves it in 4 MiB pieces
  into the checkpoint's host vector. Every reader joins first and only for the copy
  it needs (restore, snapshot save and restore, the parked copies, the serve loop at
  each request start, the SAVE/RESTORE session handler).
- STRATA_CKPT_VRAM_SLOTS=N (with ASYNC): the last N saved GDN states stay in VRAM,
  tag-checked, only allocated while 1.5 GB stays free; a restore of a checkpoint
  whose tag a slot holds copies device to device.
- STRATA_TRACE lines carry a steady-clock timestamp; STRATA_PARK_TRACE prints the
  timing of the save steps.
- Unset: the code adds a mutex-guarded no-op join per checkpoint save, restore and
  snapshot, a tag field that stays 0 and a few clock reads; the copies are the
  synchronous ones as before.

Differs from the PR as submitted: the PR put a join into conversation_file.cpp
(put_checkpoint), which broke the conversation_file_test and
session_save_reclaim_test builds without CUDA and matched only the copied vector,
not the one the copy in flight writes, so a SAVE within one checkpoint copy after a
turn could write an incomplete GDN state (found by bitaccesscomau; fixed in the
PR's last commit 33a8146a). This commit carries that fix: conversation_file.cpp
stays as it is and the join runs first in the SAVE/RESTORE handler of
generate.cpp. The commit 33a8146a is therefore not a separate commit here.

One more join, found by running the new ctest entries: conversation_snapshot_sources (the SAVE
path of a session file) saves the running state with the background copy and the caller writes
it to the file at once, after the handler's join has already run. It now joins the copy of
that checkpoint before it returns; without it a session file written with
STRATA_CKPT_ASYNC=1 could hold an incomplete GDN state (conversation_snapshot_test failed
at "both files read" before the join).

Tests: conversation_snapshot_test now joins after each save and, with
STRATA_TEST_BIG_GDN=1 (a GDN state over 1 MiB, so the async path is taken), runs as
the new ctest entries conversation_snapshot_test_ckpt_async and
conversation_snapshot_test_ckpt_vram_slots: every checkpoint, capture and restore
must give the same bytes as the plain run.

Not covered: no test destroys a checkpoint while its copy is in flight (readers
join; the cache's eviction paths do not); the reporter saw 3 of about 35 engine
starts stop answering after a park with evictions, on the paths with the
switches off, and could not reproduce it in 15 reruns. The VRAM stage has no
headroom check of its own (an RTX 3090 with 147 MiB minimum free showed no failure
in 42 of 42 saves; the allocation failure falls back silently).

Reporter's numbers (linxuhao, CMP 170HX): turn checkpoint 71.4 -> 0.9 ms,
restore 114-126 -> 28-47 ms. Independent data (bitaccesscomau, RTX 3090 24 GB,
PCIe 3.0 x16, release defaults, 4 pairs): checkpoint save 73.2 -> 62.4 ms, tiny-prompt
time to first token median ratio 1.038, outputs byte-identical. Theirs, not measured
here.

Original change by linxuhao (PR #3779, commits 0d512925 and 33a8146a).
2026-10-27 00:52:52 +01:00
linxuhao 8c2a846947 Prefill GDN: opt-in STRATA_GDN_TC=1 runs the DeltaNet recurrence as a chunked scan on the tensor cores, sm_80 (#3779, #3712)
Problem: the prompt path's DeltaNet recurrence is a large part of the prompt time
of a GA100 card.

Change, opt-in and rounding-level (the output is not the FP32 kernels' bits):
src/prefill/gdn_tc.cuh holds the chunked WY-form recurrence with every product
on the tensor cores (f16x3 splits with power-of-two scaling, one block per value
head) and its prep kernel; kernels.cu dispatches it before the STRATA_GDN_CHUNKED
branch for prompt chunks of 128+ tokens when STRATA_GDN_TC is set.

Where it runs and where it does not:
- gdn_rec_tc returns "not supported" and the existing kernels run unchanged on a
  card below sm_80, with fewer than 48 SMs, or with less than 122 KB of
  shared memory a block (the kernel needs 124,944 B). That excludes sm_86, sm_89
  and sm_120 (99 KB opt-in): the code is compiled into those builds and never
  launched there. It runs on sm_80 and sm_90. A failed scratch allocation
  (60.3 MB a device) also falls back.
- Device code for cards below sm_80 compiles to trapping stubs behind
  STRATA_GDN_TC_SM80; the whole header sits under !__HIPCC__.
- Unset, the dispatch costs one cached flag test per call.

Tests: gdn_rec_parity gains the tc variant (T = 1..4099, 8192, 32768, six stress
inputs against gdn_rec_kh_kernel, gate 1e-4; "not on this card" where it cannot
run), --selftest-frag (each tensor-core helper on one warp against a double
reference) and --bench-tc. The new ctest entry gdn_rec_parity_selftest_frag runs
the helper self-test, which any sm_80+ card can run (77 below sm_80). The full
tc variant can only run on an sm_80 or sm_90 card.

Reporter's numbers (linxuhao, CMP 170HX): recurrence 105 -> 34 ms at 3.3K tokens
and 1,705 -> 547 ms at 52K; prompts +3.9 % (4K), +6.0 % (16K), +6.7 % (52K) over
the same base; within 2.3e-6 of the FP32 kernel in gdn_rec_parity; teacher-forced
KL against the bitwise base mean 0.0157, median 0.0011, max 0.098, top-1 100 %;
1 of 5 conversation checks identical and decode -2.6 % in their pair (they
think draft acceptance moves with the rounding; not checked). Theirs, not
measured here.

Original change by linxuhao (PR #3779, commit f1cabc99).
2026-10-27 00:52:52 +01:00
linxuhao 10c8cc629f Prefill fused IQ experts: opt-in f16 mma path STRATA_PF_FUSED_F16=1 and warp-specialized kernel STRATA_PF_FUSED_WS=1 (#3779, #3712)
Problem: the NT = 256 kernels still decode every 32 values to int8 and scale
the int32 products in FP32.

Change, both opt-in and both rounding-level (the output is not bit-identical to the
int8 kernels; they are the PR's "rounding-level" switches):
- STRATA_PF_FUSED_F16=1 (implies the NT = 256 set): the decoded weights times
  their block scales and the activations times theirs as F16, m16n8k16 F16
  products into FP32 accumulators. The k order inside a 16-value group is
  permuted the same way for both operands.
- STRATA_PF_FUSED_WS=1: native_ws_kernel, producer warps decode into an F16 ring
  in shared memory and consumer warps run the same F16 products (named barriers),
  so it has the F16 path's rounding. One block per multiprocessor; about 101 KB of
  shared memory at its defaults, so it is set up on sm_80 / sm_90 cards and
  says "not available" and keeps the 512-thread kernels on an sm_86 / sm_89 /
  sm_120 card (99 KB). Compile-time knobs STRATA_WS_PROD / _SLOTS / _NB / _KS / _HTAB.
- Both take a WideBatch like the NT = 256 kernels, so STRATA_PF_BATCH above 128
  reaches them. CUDA only (under !__HIPCC__); the Turing pass compiles the F16
  instances against __trap() stubs and never runs them (dev_info stops before the
  NT = 256 setup on Turing).
- With neither variable set the kernels, their launches and their shared-memory
  sizes are as before (the F16 template parameter defaults to false).

Differs from the PR as submitted: when STRATA_PF_FUSED_Q2N=1 is on as well
(#3809, after the PR was written) the WS setup also sets up the Q2_0 gate/up
instance.

Tests: prefill_fused_iq_test_f16 and prefill_fused_iq_test_ws (ctest) run the
fused-experts test (double-precision reference and MMQ) with the switch set. The
test now skips (77) when a requested NT / F16 / WS kernel set is not set up on the
device, so a card that cannot run WS does not report a pass for the 512-thread
kernels.

Limits of the test for these two arms only: with STRATA_PF_FUSED_F16=1 or _WS=1 the
test uses the limits it already has for STRATA_PF_F16 (F16 products): fused against the
int8-rounded model rel RMS 5e-3 (2e-3 otherwise) and against MMQ 5e-3 (1e-3 otherwise).
Measured with F16 on an RTX A4000 (sm_86, z440), over the 11 pairs: against the int8-rounded
model rel RMS 2.85e-3 to 2.90e-3 and worst row 5.9e-3 to 8.1e-3; against MMQ the same;
against the FP32 reference 1.438e-2 to 1.480e-2 with MMQ at 1.435e-2 to 1.478e-2 (the
fused path stays within 1.003x of MMQ's rel RMS and 1.03x of its worst row). The rounding is the F16 rounding of the
weights times their scales and of the activations times theirs (an 11-bit mantissa, about
2^-11 relative per operand), which the int8-rounded model does not have; it is not a
different summation order. The same arm on an RTX PRO 5000 (sm_120): up to 2.90e-3 (rel RMS), worst row up to 8.1e-3.
The default path and the NT=256 arm keep the old limits and their printed errors are
unchanged: with no switch set the test prints identical output (52 lines) from the base
tree and this tree, on an RTX A4000 (sm_86) and on an RTX PRO 5000 (sm_120).

Reporter's numbers (linxuhao, CMP 170HX): one MoE layer of 3,256 tokens 8.27 ms
(NT = 256) to 7.90 ms (F16) and 7.65 ms (WS); prompts +2..+6 % over the
256-thread kernels, 1 of 11 greedy answers changes. Teacher-forced KL against the
bitwise base on their 20-prompt set: F16 / WS mean 0.0062, median 0.00006, max
0.076, top-1 100 %. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 579cbfbb).
2026-10-27 00:52:52 +01:00
wannagoxuf 17878ac579 batch slots: MTP drafts in the pipelined groups, STRATA_PIPED_MTP=N opt-in with --batch-mtp (#4121, #3794)
Problem: a layer split with --batch pipelines the slots in groups, and the pipelined groups run no MTP drafts.
--batch-mtp with serial windows leaves all but one card waiting. A multi-stream deployment wants both.

What changed: STRATA_PIPED_MTP=N (with --batch-mtp) keeps the pipeline and drafts in it. A group's window carries
each of its slots' rows: the slot's token and up to N MTP proposals (at most --spec - 1; the first always, each
further one while the draft layer was at least --spec-min-p sure of the one before). The last stage judges each
slot's proposals on their own. The number of rows to keep is known only then, and by then the earlier stages run
other groups' windows, which reuse the scratch the commit replays from. So a small save graph copies those rows
right behind each window (Verifier::batch_launch_rows), and every stage queues the commit of the kept rows after
the verdict (Verifier::batch_commit_rows). The slot drafter catches up on its own rows and drafts the next
proposals as an asynchronous chain (MtpDrafter::chain_launch). Unset, nothing changes.

Changes on top of the PR as sent (review):
- The verifier window size is worked out where the verifiers are built, from the slots that fit and the final
  groups, not earlier from --batch as asked. Unset it is the old expression.
- STRATA_PIPED_MTP is decided again once the slots that fit are known. It says it is off, and why, when fewer than
  2 groups are left, when STRATA_BATCH_SELF_COMMIT is on, or when a group's window would hold more than
  kVerifyMaxT rows. The start line names groups and slots correctly.
- batch_groups::resolve keeps its old mtp_dropped rule; only mtp_piped is new, and it also applies with
  --batch-groups given (the PR's fix 1). The "--batch-groups 1" test case now expects "not dropped".
- --remote-expert-opt alone no longer blocks --batch-mtp on a layer split: the object it guards is only made with
  a helper expert cache (the PR's fix 2).
- docs/BATCHING.md rewritten with the numbers measured here.

Tests: batch_groups_test (the PR's cases, four "unset is as before" cases); builds on CUDA sm_86 (Linux), SYCL
(the sycl tree has its own verify.cpp and generate.cpp and does not include this code) and MSVC. Model runs:
see docs/BATCHING.md and the gate scripts 4121-*.sh.

Not covered: AMD (HIP) was not built or run; the draft layer on the first stage; helper expert caches.
2026-10-27 00:21:07 +01:00
Niko1221 1d45da3a8a Merge f-027-rp15: route-prior default 1.5 on discrete NVIDIA cards of 13 GiB or less (owner decision)
2.0 showed copy and code defects on the RTX 3060 (IQ3_XXS); 1.5 showed none in 19 tasks and keeps the speed
(story +15%, code +37% vs off). The 61-task check decides 1.5 vs a 1.0 fallback.
2026-10-26 23:30:18 +01:00
Niko1221 0afaaa399e Route prior: the 0.2.7 default on a discrete NVIDIA card of 13 GiB or less drops from 2.0 to 1.5 (owner decision 2026-10-26). On the RTX 3060 12 GB (IQ3_XXS) the copy and code check found real defects at 2.0 and none in 19 tasks at 1.5 or 1.0; 1.5 keeps story +15% and code +37% against off. The value lives in strata::kRoutePriorSmallCard (sm120_series.hpp); scope (CUDA discrete <= 13 GiB), STRATA_ROUTE_PRIOR=0, an explicit value and STRATA_DEFAULTS=026 are unchanged. arch_defaults_test, verify.cpp comments, DETAILS.md and the 0.2.7 release notes follow. 2026-10-26 23:26:18 +01:00
Niko1221 1e9678cb7c Split serving: check tool uses cold prompts per request and a single-card arm on the shared arena
Idea and manual setup by @MarcoTool in #4000.
2026-10-26 23:15:49 +01:00
Niko1221 b1bea3a17c Merge f-027-shs: STRATA_SH_STREAM off by default on Windows gfx1151 (owner decision)
+6.9% Qwen IQ3_S decode on Strix Halo Windows, prompts unchanged; token ids identical in 4 starts x 5 prompts.
Linux gfx1151 and other cards unchanged; STRATA_DEFAULTS=026 restores it. Docs label the DeepSeek Strix numbers by build.
2026-10-26 23:01:38 +01:00
Niko1221 f932e410e6 docs: Windows Strix Halo leaves STRATA_SH_STREAM off, with the token-id check; DeepSeek numbers say which build they come from
The Windows guide says what 0.2.7 does and how to turn the switch back on, backed by five fixed greedy prompts
(895 ids) run four times with the switch on, off, on, off: identical ids. STRIX_HALO.md and the 0.2.7 release notes
get a line. The DeepSeek 4K prompt figures of 2026-10-25 (78.7, 87.2) stay and are labelled with their build; a table
of the 0.2.7 candidate changes one thing at a time (chunk size, DS4_DEFAULTS=026).
2026-10-26 23:00:50 +01:00
Niko1221 c77292f09d gfx1151 on Windows: the shared-expert fork (STRATA_SH_STREAM) is off by default
Interleaved A/B on a Strix Halo Windows PC (Qwen IQ3_S, engine 0.1.51): chat decode 43.6 tok/s with the fork,
46.6 without (+6.9%), prompts unchanged. Linux gfx1151 and every other architecture keep their table.
STRATA_SH_STREAM=1 turns it back on, STRATA_DEFAULTS=026 keeps v0.2.6's table.
arch_default_env(arch, defaults026, windows) is a pure rule; arch_defaults_test checks both platforms.
2026-10-26 22:28:05 +01:00
Niko1221 e240850324 Docs: Strix Halo Windows guide, plain emphasis and a stray blank line 2026-10-26 22:24:30 +01:00
Niko1221 538acd098c Merge f-027-gh-ctl: GitHub setup, Docker, docs and bench items for 0.2.7
Ported PRs (authors kept) and our fixes, plus the owner's part-2 items: the restored TROUBLESHOOTING entries with a
link check, #3895 (no shared-expert fork calibration on Windows HIP with --kv-resident; a warning on a hand-set
STRATA_SH_STREAM=1), #3930 (docs), #3421 (Docker stops when models are present and MODEL_PATH is unset), #3570
(launch --show-key reads setup's config), #3311 (engine paragraph only) and #3029 (Strix Halo Windows guide,
re-measured on 0.2.7).
2026-10-26 22:24:08 +01:00
Niko1221andHaiYunChuang ee1a6efd5c docs/STRIX_HALO_WINDOWS.md: a Strix Halo Windows guide, measured again on 0.2.7 (#3029)
Structure from the guide in PR #3029 by HaiYunChuang (what the PC reaches, memory and the BIOS, the configuration, what is on by default, traps),
rewritten for the 0.2.7 install and with every number measured again: the Terminal and the gfx1151 engines built from f-027-int2 (3e468f5de), a
local signed test feed, strata launch as a standard test user on the NucBox EVO-X2 (Adrenalin 26.10.1), tools/strata_bench.py over a tunnel.
Qwen IQ3_S: 44 tok/s chat, prompts 755 / 821 / 792 tok/s at 4K / 16K / 32K. DeepSeek 2.00 bpw: 12 to 14 tok/s chat, prompts 214 to 342 tok/s.
An interleaved A/B of STRATA_SH_STREAM on Windows gfx1151 (three pairs): off is 6.9% faster in chat decode, prompts unchanged.
What launch chose and logged is quoted instead of a hand-made config. The first guide's tuning rows (engines 0.1.41 to 0.1.47) are not repeated.

Co-authored-by: HaiYunChuang <[email protected]>
2026-10-26 22:21:04 +01:00
Niko1221 f1dadd1710 fused_gr: the persistent hyper-connection read's barrier only calls __nanosleep on sm_70 and newer (the sm_60 build of f-027-int2 failed on it) 2026-10-26 22:07:55 +01:00
Niko1221 3cabb9e3fe Merge f-027-gh-app: GitHub app and Terminal items for 0.2.7
Part 1 fixes and 12 ported PRs (authors kept), and the owner's part-2 items: the Monitor readings strip with
local currency (#3628), approval notification + opt-in minimise to tray (#3477), opt-in done sound (#3586),
app port setting with a busy warning (#3402) and interface size up to 200% (#3970). Host tests: only the two
known base failures on Linux; connector-setup and the new tests pass on Windows.
2026-10-26 21:56:21 +01:00
Niko1221 3b908cc4f7 Tests: the new catalog is checked in nine languages, French included 2026-10-26 21:55:58 +01:00
Niko1221 0d0ec83b7d Merge f-027-int2 into f-027-gh-app 2026-10-26 21:54:30 +01:00
Niko1221 c17a6f4ce1 App: the Monitor's readings strip (energy since start, cost in the person's own currency, GPU fan and clock, CPU and SSD temperature) with the Electricity price setting (#3628, PR #3628 by hash13 as the starting idea); a sound when work finishes (#3586); desktop notification and tray, pinned port and interface size settings with their host routes (#3477, #3402, #3970); the gh-<lang> catalog in nine languages 2026-10-26 21:31:51 +01:00
Niko1221 902dc814d6 Desktop shell: a pinned app port with a warning when it is busy (#3402); minimise to tray with Tauri's own tray icon and a native desktop notification through the loopback bridge (#3477) 2026-10-26 21:31:47 +01:00
Niko1221 59b4be2a54 App: French for the four new DeepSeek card and 12 GB NVIDIA strings (fx80-fx83) 2026-10-26 21:19:31 +01:00
Niko1221 98e35984c6 Merge f-027-dscard: the DeepSeek card names the free disk the set needs (owner-approved wording)
# Conflicts:
#	apps/host/src/locales/fix-de.json
#	apps/host/src/locales/fix-en.json
#	apps/host/src/locales/fix-es.json
#	apps/host/src/locales/fix-ja.json
#	apps/host/src/locales/fix-ko.json
#	apps/host/src/locales/fix-pt.json
#	apps/host/src/locales/fix-ru.json
#	apps/host/src/locales/fix-zh.json
2026-10-26 21:19:14 +01:00
linxuhao dc390c3864 Prefill: opt-in STRATA_DEQUANT_SIDE=1 dequantizes the dense weights on a side stream into a double-buffered scratch (#3779, #3712)
Problem: Gemm::native dequantizes each dense weight to FP16 in front of its
GEMM on the compute stream, so the dequant never overlaps the kernels queued
ahead of it.

Change, opt-in (CUDA only, always off on HIP): with STRATA_DEQUANT_SIDE=1 the
prompt path takes a second GEMM scratch (64 MiB more of its region, counted in
bytes_needed and carried through relayout) and Gemm::set_dequant_side hands it
to the Gemm. native() then dequantizes the next weight on a non-blocking side
stream into the scratch the compute stream read last but one. Four events order
every write and read of each scratch (free -> dequant -> ready -> GEMM -> free),
so each product sees the same bytes as in the serial form. The row-sliced path
(a weight larger than the scratch) stays serial. Metal answers false and keeps
the serial form.

Differs from the PR as submitted: if the side stream, its events or the
setup fail, the PR failed the prompt-path init. Here one line is printed and the
dense weights are dequantized in front of their GEMMs as before. Unset, nothing
changes (gemm_dequant_side() is one cached flag; the scratch, stream and events
are not created).

Test: new gemm_dequant_side_parity (ctest, GPU, synthetic, 77 without a device):
eight Q4_K / Q5_K / Q6_K weights of the engine's shapes (one larger than the
scratch, so it takes the sliced path) run three rounds through Gemm::native
serially and with the side stream, with and without a spin kernel queued ahead of
each GEMM; every output must be identical bit for bit.

Not covered: dense weights read by the side stream are the resident ones; a weight
that arrived by an upload on another stream would need its own event (the prompt
path has none that do).

Reporter's numbers (linxuhao, CMP 170HX): measured with STRATA_PLE_COPY_STREAM and
STRATA_EMBED_VRAM, the four PLE / embedding / dequant switches together give
+2.2..+2.6 % prompts. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 741fc6ab).
2026-10-26 21:19:13 +01:00
Niko1221 03d709820d Merge f-027-app2: master switches in the app and Terminal settings
DS4_DEFAULTS=026 behind the DeepSeek speed-ups Off toggle, the 12 GB NVIDIA route-prior row, the Advanced-only
Qwen '0.2.6 behaviour' row (STRATA_DEFAULTS=026), and the app-ui Quit test fix. Terminal suites pass; host suite
failures on Linux also fail on the base.
2026-10-26 21:18:49 +01:00
Niko1221 663ec80abf mmvq_il: the sm_89 (Ada) decode tables are on by default on compute capability 8.9 only, STRATA_MMVQ_IL_89=0 and STRATA_DEFAULTS=026 turn them off (#3453)
Owner decision for 0.2.7: the tables of PR #3453 (the cells where an RTX 4070 and an RTX 4080 community run gave the same rows count, and
the one-column Q3_K row) ship on for sm_89 cards, untested on Ada by us. The output is bitwise the same with them off (a table only
changes how many rows a warp takes); no end-to-end gain was seen on a 4090 or 4080 in the report, so the reporter tests 0.2.7 and reports.

The rule is a pure function in sm120_series.hpp: mmvq_il_89_default(hip, cc, v026) is true for cc 89 only, false in HIP builds and under
STRATA_DEFAULTS=026 (defaults_026.hpp, as 38c9080db did for the other defaults); mmvq_il_89_resolve() adds the variable: "1" and "0" win,
unset or other text is the rule. native_mmvq.cu reads it once. arch_defaults_test checks cc 89 on, every other cc off, HIP off, 026 off,
"0" off, "1" wins over 026, the empty text. mmvq_il_parity expects the on-tables by default and the old ones with =0 or 026.

Docs: MMVQ_IL_TABLE.md, DETAILS.md and the 0.2.7 release notes (a line that says it is not measured on an RTX 40 card, with the off switch).
2026-10-26 21:11:03 +01:00
Niko1221 46a9cd87c7 Merge f-027-int2 into f-027-gh-eng (brings STRATA_DEFAULTS=026 and the other 0.2.7 integration work) 2026-10-26 21:09:26 +01:00
Niko1221 52ba0b7816 Merge f-027-i18n: French as the ninth app language and the Polish README
French catalogs for the app (3,528 strings + fix/tune/tools catalogs) and the served web page (247 keys),
README.pl.md from the current README (#3355, Jakub Luwierski), Polski in every README language row, and the
README.md language row's double-encoded names fixed. Tests: 21 host files 180 pass on p1006, Terminal and web
i18n checks on l620.
2026-10-26 21:03:59 +01:00
Niko1221 551f1cdc30 Merge f-027-gh-eng-k1 (#3867 #3751 #3453 #3943 #3820 #3752 #3455: Pascal and sm_89 opt-in kernel ports)
# Conflicts:
#	src/core/expert_cache.cpp
#	src/core/verify.cpp
2026-10-26 21:03:12 +01:00
Niko1221 2b9869eddf Web chat page: French (#3970)
serve/web/locales/fr.json (247 keys, plurals one/other, same terms as the app catalog) and the French line in serve/web/i18n.js. tools/web_i18n.py and serve/test_web_i18n.py (node and python) pass.
2026-10-26 20:59:54 +01:00
Niko1221 710b6a1bfe Merge branch 'f-027-int2' into f-027-i18n 2026-10-26 20:58:00 +01:00
Niko1221andJakub Luwierski 2ce07f8631 Docs: Polish README (#3355)
Ports the Polish translation from #3355 and the Polski entry in the language row of every README. The pull request translated the older long README, so README.pl.md is rewritten from the current README.md (logo, centred language row, DeepSeek speed tables including Strix Halo, Gyro, coffee button). README.md's own language row had garbled names (double-encoded text); it is written out correctly.

Co-authored-by: Jakub Luwierski <[email protected]>
2026-10-26 20:57:57 +01:00
Niko1221 52a558abbe App: French as the ninth interface language (#3970)
fr.json, fix-fr.json, tune-fr.json and tools-fr.json (3,654 strings, vous, plain short words), the Francais entry in the language picker, every locale test and gap check now covers fr, a french-catalog test (spacing before ? ! : ;, apostrophes, short labels, no leftover English), scripts/fr_sync.py to list and merge strings added later, and scripts/language-shots.mjs for screenshots of the app in one language.
2026-10-26 20:57:57 +01:00
Niko1221 5d9f479229 Merge f-027-web: the served web chat page for 0.2.7
Wide/narrow CSS (#3176), page translations with a language choice, opt-in KaTeX math (#2972, #2974),
sandboxed svg/html preview (#1628), Load/Unload in the header (#3550), and one chat-history design
(#1787, #2615, #3823, #4004). Tests: browser suite on the owner PC, node + pytest serve suites on boston
(457 passed, 2 skipped).
2026-10-26 20:57:41 +01:00
ruibeikaa 9067cc64d3 iq_kernels: GP100 4-bit gate/up expert slots, opt-in STRATA_EXPERT_NIBBLE=1 (#3455)
Problem: on a GP100 die the verify window's gate/up kernels for the IQ2_S,
IQ3_XXS and IQ3_S expert formats are not held by memory bandwidth but by each
block's serial chain (unpack the codebook indices, the emulated dp4a, the
16-lane reduction); the same kernel with every dot product removed still takes
about two thirds of its time.

Change: with STRATA_EXPERT_NIBBLE=1 the engine, after the expert cache is
filled and every expert is resident, transcodes the gate/up blocks of every
fully resident IQ2_S / IQ3_XXS / IQ3_S layer (n_embd 2560, n_ff 640) into slots
of 4-bit codes, 16 bytes per 32 weights, with the integer scales and fp16 d kept
as they are; the down rows stay GGUF. The decode gate/up reads those slots with
kernels that run the production integer chain and float tail in the production
order (R4S / R4P / R4SV / R4PX by format and group size); the prompt path's MMQ
gather rebuilds the GGUF blocks from a slot on the fly. At load every item is
round-tripped and every slot's rebuilt GGUF blocks are compared with the staged
blob before the cache is sealed (ExpertCache:🦭 a later fill_slot stops the
engine). New: include/strata/core/expert_nibble.hpp (slot contract, switches,
per-layer table), nib_gu_proto.cuh (STRATA_NIB_GU_PROTO=1 variant), the
transcoder, hash and rebuild kernels and the 4-bit gate/up kernels in
iq_kernels.cu, generate.cpp (the fill, the VRAM accounting, STRATA_EXPERT_NIBBLE_
RECHECK), iq_nibble_parity and native_expert_bench --nibble.

Review changes against the PR:
- An option the switch cannot work with (--peer-device, --kv-grow, --vram-elastic,
  remote caches, STRATA_MMQ_DIRECT, ..., a card that is not cc 6.0, a build
  without the kernels) now prints a warning and the start goes on with the usual
  GGUF slots, where the PR refused the start. What is known only later (an expert
  out of VRAM, a loan, the prompt chunk) still refuses, since slots are
  converted by then.
- The checks this switch adds to expert_source.cpp and prefill.cpp are compiled
  out of every build that has no 4-bit kernels, so the default builds add nothing
  to the hit split or the prompt loop.
- iq_nibble_parity gets --synthetic-only (every gate/up format x both down
  types on random experts, no model file), skips with 77 on a card that is not a
  GP100, and the synthetic and --header runs are ctests.
- The PR's docs section is rewritten for these changes and its measurements are
  labeled as the author's.

Default path (switch unset): the table is empty, nibble::slot_bytes(l) is
expert_layout().blob_bytes(l) and nibble_max_slot_bytes() is max_blob, every
sizing the PR touched returns what it returned before, and the new branches in
generate.cpp (guard_raw before the adaptive tier's copies, the profile fill's
test), expert_cache.cpp (one bool in the fill calls and admit) and verify.cpp
(a capture-time layout choice, compiled out without the kernels) are not taken.
Proof on the kernels: in the sm_60 build (CUDA 12.9) the SASS of all 446
kernels iq_kernels.cu had before is identical with and without this change,
as is that of the 603 in native_mmvq.cu, 29 in qsa_select.cu and 10 in
moe_mmq.cu; 21 kernels are new.

Tests: iq_nibble_parity --header passes on z440 and boston (the host half of the
contract; also a ctest). The GPU parts (--synthetic-only, a model shard) refuse a
card that is not a GP100, so they were only compiled here (sm_60, CUDA 12.9,
P1004: iq_nibble_parity and native_expert_bench build; the whole tree builds).
sm_86 build without the SM60 option (z440): exit 0, the 4-bit paths compiled out.

Not covered: no GP100 run. The model gate is
phaseA-tests/gh027/gates/3455-expert-nibble.sh: iq_nibble_parity --synthetic-only
and on a model's layers on a P100, then identical replies with the switch on and
off (a layer is converted only when every expert of every layer is resident, so
the rig needs the pack to fit whole), then the decode A/B. The contributor's
numbers (four GP100 dies, Swift 1.5 IQ3_S, 47 of 48 layers converted, +12.7 GiB
VRAM, decode 47.3 -> 51.9 tok/s, +9.7%; prompt reading 2.3-3.0% slower; replies
of 15 requests equal) were not re-measured, and the enabled path was written
against an older base than this branch's.
2026-10-26 20:52:54 +01:00
ruibeikaa 5dbd8774d8 prefill: a resident layer's experts weight-stationary on GP100, opt-in STRATA_MMQ_WS=1 (#3752)
Problem: on a GP100 die (compute capability 6.0: no dp4a, no tensor cores) the
prompt pass's routed experts run as MMQ launches per 16-expert group: a
gate/up, a swiglu, a quantize and a down launch per group, each group's experts
gathered and padded to the group's largest row count, every MMQ tile decoding
its weight blocks again. With every expert resident the per-group overhead, not
the weight bytes, is a large share of a prompt read on this card.

Change: with STRATA_MMQ_WS=1, a layer whose routed experts are all resident on
this GPU runs quantize_scatter, then gate/up as ONE weight-stationary launch
over all its experts (read where they lie in the expert cache), ONE
swiglu_quant over all its rows, then the down as ONE launch. New
src/prefill/moe_ws*.cu(h): a CTA holds 128 (or 256) weight rows of one expert
and runs them against up to 64 of that expert's activation rows, a thread per
weight row, so each weight block is decoded once per pass; gate/up for
IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS and IQ3_S, down for Q2_0 and IQ4_NL. Every
product keeps MMQ's int8 values, scales and per-32-value epilogue in its K
order, so GU, H and the down rows are MMQ's bits. ws::faster carries the
author's measured ws / MMQ time ratios per format and picks a layer only where
both products win at its rows per routed expert. prefill.cpp decides per layer
after quantize_scatter (all experts resident, this GPU alone, no peer or CPU
share, not the fused layout, not STRATA_MMQ_DIRECT, no Hadamard, no NVFP4 tail,
no FP4 activations, aligned, not STRATA_DBG_NAN); every other layer keeps its
path. STRATA_WS_FAST=1 (second opt-in, same bits) selects a down kernel for the
IQ4_NL 32-token tier with fewer instructions per pass.

Gating: the kernels and STRATA_PREFILL_WS exist only in the CUDA build with
STRATA_EXPERIMENTAL_SM60 (not HIP, not Metal; prefill.cpp links stubs that
answer mode 0 elsewhere). At run time ws::mode() is 1 only with STRATA_MMQ_WS=1
and every visible GPU at compute capability 6.0. Unset, nothing is allocated
(the offset and chunk buffers, under 1 MiB per die, exist only when the mode is
not 0), the arena layout and the mapped table are the ones before, and the
per-layer test short-circuits on the cached mode.

Not ported: moe_ws_down_bench (needs a model shard) and its CMake target.
Added: ws::set_mode (tests only) and tests/cuda/prefill_ws_parity_test.cpp, a
synthetic layer-level check: random experts of all 5 gate/up x 2 down formats,
random routing, the per-group MMQ (mmq::Context per expert) against ws::gate_up
+ swiglu_quant + ws::down, GU, Hq and Dm compared byte for byte, the down also
alone on the reference H and with STRATA_WS_FAST on. It gives its verdict on a
compute capability 6.0 card only (the kernels follow MMQ's sm_60 SASS); other
cards run it for information with --any.

Tests: prefill_ws_parity_test on boston (2x RTX A6000, sm_86, CUDA 13.1, gpu lock,
SM60 build, --any because it is not a GP100): 40 cases (all 5 gate/up x 2 down
formats, four routings each), 0 declined; the largest GU and down-alone error
against that card's MMQ is 5.2e-07 of the largest value (float rounding: MMQ's
sm_86 epilogue is not the sm_60 one the kernels follow), the full chain's Dm
differs up to 1e-3 where int8 rounding of H flips on those rounding
differences, and STRATA_WS_FAST output equals the default kernel's in all
cases, 0 differing bytes. The byte-for-byte verdict against MMQ is the P100's.
sm_60 compile of the whole tree (CUDA 12.9, P1004): exit 0; sm_86 build without
the SM60 option (z440): exit 0. SASS of the 10 kernels moe_mmq.cu had before
this change is identical in the sm_60 build.

Not covered: no GP100 run here. The contributor's numbers (prompt reads 31-37%
shorter on four GP100 dies with every expert resident: 240 tokens 2,156 -> 1,364
ms, a fresh 26K read 41,231 -> 28,555 ms; one run per arm) were not re-measured.
The model gate is phaseA-tests/gh027/gates/3752-mmq-ws.sh.
2026-10-26 20:52:32 +01:00
Niko1221 60e6a06086 Split serving: docs pointers, check tool puts session files in /dev/shm
Idea and manual setup by @MarcoTool in #4000.
2026-10-26 20:44:45 +01:00
linxuhao 884e37dfdf Native head: opt-in STRATA_EMBED_VRAM=1 keeps the token-embedding table in VRAM (#3779, #3712)
Problem: the token-embedding table of a native pack lives in mapped pinned host
memory, so every prompt's scattered row gather crosses the PCIe link. On a Gen2
x4 link that costs more than the table's size in VRAM is worth.

Change, opt-in: with STRATA_EMBED_VRAM=1 the table is copied into the current
device's VRAM when the table plus 512 MiB stay free (322 MiB for an IQ4_XS
table). The gather kernels read the same bytes through dev_ as before. If the
VRAM is not there or the copy fails, the table stays in mapped host memory as
before and one line says so. The start-up line now says where the table is
("in VRAM" also for the two existing VRAM fallbacks, which it called mapped
host memory).

With the variable unset nothing changes (one getenv at load).

Not covered: a layer split across several devices places the table on the
device that is current when it loads, as the existing VRAM fallback does. No
unit test (it moves the same bytes to a different memory; the gather kernel is
unchanged).

Reporter's numbers (linxuhao, CMP 170HX): measured together with
STRATA_PLE_COPY_STREAM and STRATA_DEQUANT_SIDE on a 51.8K-token prompt, the
embedding step went from 191-201 to 70 ms. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 952dd784).
2026-10-26 20:41:23 +01:00
linxuhao d6428bf291 Prefill: opt-in STRATA_PLE_COPY_STREAM=1 uploads the compact PLE rows on their own stream (#3779, #3712)
Problem: with STRATA_PLE_COMPACT the raw-row upload sat on the compute stream
before layer 1, which idles for the DMA (4.7 MB per 3K-token chunk, 2.9 ms on a
Gen2 x4 link).

Change, opt-in, only with STRATA_PLE_COMPACT=1: the raw rows go up on a
non-blocking stream of their own once the previous chunk's dequant has read the
device buffer (ple_raw_free event); the compute stream waits for the upload
(ple_copied) only at the dequant. Same bytes, same kernel. The stream is drained
when a run ends so no upload is left in flight from a buffer the next run
refills.

Differs from the PR as submitted: if the stream or the event cannot be created,
the PR failed Prefill::init. Here it prints one line and the rows go up on the
compute stream as with STRATA_PLE_COMPACT alone. With the variable unset
nothing changes (the stream and the events stay null).

Reporter's numbers (linxuhao, CMP 170HX): PLE time of a 51.8K-token prompt 95-103
to 47 ms, measured together with STRATA_EMBED_VRAM and STRATA_DEQUANT_SIDE
(+2.2..+2.6 % prompts for the four PLE/embedding/dequant switches together).
Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 0ba47e61).
2026-10-26 20:40:42 +01:00
linxuhao 17b08c9633 Prefill PLE: opt-in STRATA_PLE_COMPACT=1 uploads the raw IQ4_NL rows and dequantizes them on the device (#3779, #3712)
Problem: the prompt path reads 16 PLE rows a token, dequantizes them on the
host and uploads 10,240 floats a token. On a slow host link (PCIe Gen2 x4) that
upload is on the critical path of the first layer.

Change, opt-in with STRATA_PLE_COMPACT=1: for an IQ4_NL table held in memory
(--ple-io ram / mmap) the prompt path copies the raw 90-byte rows (1,440 bytes
a token) into the same pinned buffer, uploads them and runs
ple_iq4nl_dequant on the device (d times the code's value, one multiply, the
split-half nibble order of iq4nl_dequant_row). Any other table (Direct I/O,
pageable fallback, another format) and a failed allocation take the host path
as before. With the variable unset nothing changes: the new code is behind one
cached flag, and the extra members of the Prefill state stay null.

Two things differ from the PR as submitted:
- PleTable::gather_raw_iq4nl writes a row past the table as scale 0 and code 8
  in every nibble. The PR wrote zero bytes, which the device decodes to
  -0.0f (0 times code 0's value -127), while the host path writes +0.0f for such
  a row.
- New test ple_iq4nl_dequant_test (ctest, synthetic, no model, 77 without a
  device): the device decode against iq4nl_dequant_row, bit for bit, over every
  fp16 scale pattern with random codes, random rows, an odd row count and the
  out-of-table row.

Reporter's numbers (linxuhao, CMP 170HX, PCIe Gen2 x4): prompts +1.0..+1.7 %
(4K, 16K) with the switch on. Theirs, not measured here. On a Gen4 x16 card
the gain will be smaller.

Metal gets a stub (the Metal prompt runs through its window and never calls
it).

Original change by linxuhao (PR #3779, commit da818261).
2026-10-26 20:40:20 +01:00
sergiywith dd97421d3f prefill: mmq::swiglu_quant, swiglu and the down product's q8_1 rows in one kernel (#3752)
Problem: the prompt path stores H (swiglu of the gate/up rows) as floats and
then quantizes it to the q8_1 rows the down product reads: two kernels and an
H round trip per MMQ group.

Change: swiglu_quant_kernel, a block per row (128 threads, 4 values each,
blockIdx.y the row's 512-value half), computes swiglu_kernel's expression for
each value and quantize_mmq_q8_1<D4>'s result for each 32-value block, so hq
holds the same bytes and H is never stored. swiglu_quant_ok(type) says where
it applies (a q8_1 D4 down type, no FP4 activations). Built only in the CUDA
build with STRATA_EXPERIMENTAL_SM60; HIP and the other builds get
swiglu_quant_ok() == false and a no-op. Nothing calls it in this commit; the
weight-stationary path of the next commit does. prefill.cpp gets the matching
stubs for builds without the MMQ sources.

Tests: prefill_swiglu_quant_test (new, registered under STRATA_BUILD_TESTS in
the SM60 build, skips with 77 without a device) compares the fused kernel with
swiglu + quantize byte for byte for Q2_0, IQ4_NL and Q8_0 down types, split and
interleaved gate/up rows, a zero row and a row with one huge value.
Run on z440 (RTX A4000, sm_86, CUDA 13.0, SM60 build, gpu lock): all 18 cases print
"0 of N bytes differ" (1, 37 and 4113 rows) and the test ends "prefill_swiglu_quant_test:
ok". sm_60 compile of the tree (CUDA 12.9, P1004): exit 0.

This is a partial port of another contributor's prompt-path fusion change;
only the kernel is taken, as in PR 3752.
2026-10-26 20:38:06 +01:00
linxuhao bdcc34dfa8 Prefill fused IQ experts: opt-in 256-thread kernels STRATA_PF_FUSED_NT=256 and whole-layer launches STRATA_PF_BATCH=N (#3779, #3712)
Problem: on a GA100 card (70 SMs) the 512-thread fused prompt-expert kernels
leave the card idle at every barrier and at every grid tail.

Change, all opt-in:
- STRATA_PF_FUSED_NT=256 sets up a second set of native_kernel instances with
  256 threads and two blocks a multiprocessor (WW 2 only), with two-pass weight
  loads (the loads of the next k step are issued before the current step's
  decode; load_unit_l gives the same words as load_unit). Every output is the
  same thread's sum in the same order.
- STRATA_PF_BATCH=N launches up to N experts at once (WideBatch holds 512 blob
  pointers, 4 KB of kernel parameters: CUDA 12.1+, Volta and newer; HIP, gfx906
  and the Pascal build keep kMaxBatch). Above 128 it needs the NT = 256 kernels.
  Below 128 it only changes the launch grouping.
- With neither variable set the 512-thread kernels, their launches and their
  load sequence are unchanged: tile_rows(ww, 512) equals the old 256 / ww, the
  two-pass code is behind a constant-false TWO_PASS, and the one added test in
  experts_native is occ_nt > 0, which only STRATA_PF_FUSED_NT=256 sets.
- If the 256-thread instances cannot be set up (shared memory, occupancy) the
  engine prints one line and keeps the 512-thread kernels.

Cost: the NT = 256 instances are new device code in every CUDA build (the
reporter measured 365,632 bytes for sm_80).

Tests: new ctest entry prefill_fused_iq_test_nt256 runs prefill_fused_iq_test
(the fused kernels against the double-precision reference and the MMQ path)
with STRATA_PF_FUSED_NT=256; it exits 77 below sm_80.

Reporter's numbers (linxuhao, CMP 170HX): one MoE layer of 3,256 tokens
8.27 ms with NT = 256; prompts +7.6 % (4K), +4.9 % (16K), +4.3 % (52K) for
NT=256 with BATCH=512 on the PR's own build. Greedy answers identical on their
11 prompts. These are theirs.

Original change by linxuhao (PR #3779, commit 056d36a9). Merged onto the Q2_0
fused path of #3809 (STRATA_PF_FUSED_Q2N=1) without textual conflicts; the PR
predates it, so one line is mine: the NT = 256 setup also sets up the Q2_0
gate/up instance when Q2N is on (without it that instance would have launched
without its shared-memory attribute).
2026-10-26 20:38:00 +01:00
Niko1221 945a63bec5 tests: lm_kernels_parity checks the two kernels of the layer-major prompt path (#3976)
moe_shared_init and moe_scatter_add came with the layer-major walk and had no test.
lm_kernels_parity runs them on synthetic data of the model's width (N = 2560, no
model, a few MiB) against a host reference:
- moe_shared_init: acc = shared * sigmoid(sg), within 1e-5 relative (the device
  sigmoid uses __expf);
- moe_scatter_add: the per-token fma chain in row order, bitwise equal to the host
  chain and bitwise equal between two launches (no atomics), over groups of one
  row, one sub-chunk, several sub-chunks, one row per token and eight per token.
Registered in CMakeLists.txt next to gr_upmix_parity (CUDA builds, not gfx906),
SKIP_RETURN_CODE 77 when no device is visible.

Not covered: the walk itself needs a model (see the 3976 gate scripts).
2026-10-26 20:37:49 +01:00
linxuhao 91aebae636 MMVQ-IL: sm_80 row tables, bitwise (#3779, #3712)
Problem: an sm_80 card took the shared interleaved-MMVQ row tables, which
were measured on other cards (#3252 reports A100 slowdowns).

Change: kIlRows_80 (2-4 columns), kIlRows58_80 (5-8 columns) and kIl1Rows_80
(one column), looked up under the key 80 only. They were read off
mmvq_il_parity --bench --emit-table with the same 3 % rule as the other tables.
STRATA_MMVQ_IL_SM80=0 maps an sm_80 card to the key 81, which has no table, so it
gets the shared table again (and no one-column table, as before).

A table only picks the rows a warp (0, 1, 2 or 4); 0 means native_mmvq's kernels.
mmvq_il_parity already checks every rows choice against native_mmvq bitwise for
every type and shape, T = 1..8, so any entry of the new tables is covered.

Reporter's numbers (linxuhao, one CMP 170HX): decode +2.7..+3.2 % over the shared
table, Qwen3.8-Flash-Next IQ3_S. The measured card is not an A100; the rows may
want re-measuring there.

Original change by linxuhao (PR #3779, commit eaedca84); the commit message here
is rewritten. The sm_89 table of #3453 is a separate change in the same list.
2026-10-26 20:34:14 +01:00
linxuhao 269c6d3e4a Dense dequant: the wide kernels by default on sm_80 too, bitwise (#3779, #3712)
Problem: sm_80 (GA100: A100, A30, CMP 170HX) ran the per-group dequant kernels
that sm_61 and sm_86 had already left for the wide ones, because
dequant_wide_on() had no sm_80 entry.

Change: dequant_wide_on() answers yes for compute capability 8.0 as well. The
test is the raw (major, minor) of the device, so it is false on every other
card (sm_60, 70, 75, 86 keep their answer, sm_89, sm_90 and sm_120 stay off as
before). STRATA_DEQUANT_WIDE=0 still turns the wide kernels off.

Reporter's numbers (linxuhao, one CMP 170HX, GA100 sm_80 with 70 SMs): the GPU
time of a 3,259-token prompt went from 857 to 792 ms, prompts +6.3..+7.2 % at
3.3K tokens and +3.9 % at 52K. We have no sm_80 card; these are theirs.

Bit identity: the wide kernels are the ones sm_61 and sm_86 already run by
default. dequant_wide_parity compares them with the per-group kernels byte for
byte for FP16, BF16, FP32, the padded stride and a row slice. The kernels have
no sm_80-only code, so the result does not depend on the card; what only an sm_80
card can show is that ptxas for sm_80 keeps the same arithmetic (see the
checklist in phaseA-tests/gh027/gates/3779-sm80-checklist.md).

Original change by linxuhao (PR #3779, commit 48dc535e); the commit message
here is rewritten.
2026-10-26 20:34:14 +01:00
Niko1221 40c9dae95b Merge f-027-gh-eng-g (#3935 sm_89 in the DeepSeek CUDA package builds, #3796 collapsed tool-name alias and the rejected-calls marker) 2026-10-26 20:31:38 +01:00
Niko1221 805ebe17be serve: a call returned as text for an undeclared name is named in the reply; the collapsed mcp__a__b spelling is the declared tool (#3796)
Problem: a request declares tools; the model writes a complete call for a name that is not among them (bash for shell, a dropped
prefix, an MCP name with every "__" written as "_"). Since #3075 such a call stays text, which is the rule we keep, but the client
saw an ordinary answer with finish_reason stop and nothing else, so the agent turn ended with raw XML and the model never learned
of its mistake. Reporters on OpenCode, Pithagoras, OpenClaw, LibreChat, Open WebUI and Pi hit it.

What changed (declared-only stays: nothing undeclared is ever returned as a call, finish_reason is unchanged):
- serve/frontend.py OutputParser: a declared name that contains "__" is also read when written with every "__" as "_"
  (mcp_playwright_browser_navigate for mcp__playwright__browser_navigate); the call goes to the client under the declared name.
  Same rules as the other aliases: only when unambiguous, and a tool declared under that spelling wins. A reordered or renamed
  tool (server1_label_values for label_values_mcp_server1, bash for shell) is not guessed.
- serve/server.py: when a block went back as text for an undeclared name, the reply carries
  "strata_rejected_tool_calls": [{"name": ..., "reason": "undeclared"}] (the names the parser already collected, up to five): at the
  top level of a Chat Completions reply and in its last streamed chunk, and in the message_delta event and the message of
  /v1/messages. It is absent when nothing was rejected, so every other reply is unchanged. The text itself is untouched.
- The server window line says the response carries the names; the count goes to totals["tool_calls_unread"] and to
  /metrics/prometheus as strata:tool_calls_unread_total.
- docs/DETAILS.md: the alias, the marker, the counter.

Tests: serve/test_tool_name_alias.py (collapsed spelling at several chunk widths, a declared spelling and a plain tool unchanged,
a tool declared in the collapsed spelling wins, two tools with one collapsed spelling are not guessed, reordered/renamed names stay
text and are named), serve/test_server.py ForcedToolChoice (the field in Chat Completions whole and streamed and in /v1/messages,
absent for an ordinary answer and a declared call, the collapsed name is a call; the counter). The whole serve/ suite: 1055 tests,
two failures that also fail on the base tree (test_fatal_recovery on this PC, RestartWindow.test_not_alive_until_ready under load).

Not covered: returning an undeclared call as a real tool call (the owner keeps declared-only); a block for a DECLARED name that came
back as text because it was malformed (counted, no name to list); the /v1/messages field with a real client; a stderr or log-file copy
of the warning.
2026-10-26 20:30:44 +01:00
ruibeikaa b434a8aa6e qsa_select: multi-CTA decode top-k and a shuffle-free decode scorer, opt-in STRATA_SELECT_LONG=1 (#3820)
Problem: a decode window's QSA select runs one CTA per query for the top-k
(block_topk_wide_kernel past ~135K cells) and the warp-shuffle block scorer
(block_scores_multi_kernel), in each QSA layer. Both grow linearly with the
context: on one GP100 die at 300K cells a top-k call takes 0.39 ms with 1-5 SMs
busy and a scores call 0.22 ms (two queries). At short contexts the same work is
small and an extra kernel is pure overhead.

Change: qsa_select.cu (inside !__HIPCC__ && STRATA_EXPERIMENTAL_SM60) gets
block_topk_multi_* (six kernels over several CTAs per query, one per 2,048
blocks of capacity, at most 32, exchanging through caller scratch: no clusters,
no spin waits, capturable) and block_scores_dx_kernel (the multi scorer's
per-head sums built in registers as the same 32-leaf xor-butterfly tree, no
__shfl_xor_sync, the tail block stays block_scores_tail_kernel's). New public
qsa_topk_multi_scratch_ints, qsa_block_topk_long and qsa_block_scores_long
forward to qsa_block_topk / qsa_block_scores outside the build gate or their
conditions. verify.cpp keeps a second set of window graphs (exec_[sel_long_][T])
for windows that reach STRATA_SELECT_LONG_AT cells (default 65536); the choice
is made before a capture and handed to the next stage of an overlapped layer
split, capture_all captures both, pl_launch picks by the window's position.
The ids and the score bits are the replaced kernels'. STRATA_TOPK_MULTI=0 and
STRATA_SCORES_DX=0 turn either half off inside the long graphs.

Review changes against the PR: in a build without the multi-CTA kernels
STRATA_SELECT_LONG=1 now says so and is ignored (the PR would have kept two
sets of graphs for nothing there); Metal gets stubs for the three new calls
(the SYCL tree has its own verify.cpp and qsa_select copy).

Default path: unset, select_long_on() is a cached false, topk_scratch_ is not
allocated, sel_long_ stays 0, and every window uses exec_[0][T], the graph it
used before; the index change is the only difference in verify.cpp.

Merge note for f-027-gh-eng (#3702 evict_window_graph, not in this branch's
base): its exec_ / exec_nr_ / launched_ accesses need the same [sel_long_]
first index.

Tests: src/kernels/qsa_select_long_parity.cpp (new; registered with
STRATA_SCORES_DX=1 STRATA_TOPK_MULTI=1 in the SM60 test block, and a second
case with STRATA_TOPK_MULTI_N=3; skips with 77 without the kernels) compares
qsa_block_scores_long with qsa_block_scores (nq 1-8, n_bid 8 .. 131072, tail
blocks of 0-3 cells, every block 0..n_bid memcmp) and qsa_block_topk_long with
qsa_block_topk (nq 1-16, contexts 1,000 .. 524,288, random scores, all-equal
scores and 7 distinct values for ties across CTAs, selected ids of every query;
the multi scratch is junk-filled and must have been written). z440 (RTX A4000,
sm_86, CUDA 13.0, gpu lock, build with -DSTRATA_EXPERIMENTAL_SM60=ON): "172
cases, 0 differ -> PASS" with one CTA per 2,048 blocks and with N=3. sm_60
compile of the whole tree (CUDA 12.9, P1004): exit 0.

Not covered: no GP100 run here (the kernels' last word is the P100, and the
speed claim). The contributor's numbers (four GP100 dies, every expert resident,
--pipeline-windows 3: 400 tokens at 115K context 51.0 -> 52.5 tok/s, +2.9%;
prompts below 64K unchanged; replies and state hashes equal on 15 requests) were
not re-measured. The model gate is phaseA-tests/gh027/gates/3820-select-long.sh.
2026-10-26 20:25:56 +01:00
Niko1221 40e78366bc Web page: German and dark screenshots in tools/webtest/shots.mjs 2026-10-26 20:23:06 +01:00
Niko1221 4661596ce3 Web page: the chat view in eight languages, and the page's docs
Everything the Chat view shows (header, tabs, message box, settings, toasts,
tool blocks, the sidebar, the history and Load/Unload texts) now goes through the
translation catalog: serve/web/locales/{de,es,ja,ko,pt,ru,zh}.json beside en.json
(written from the English text in the code by tools/web_i18n.py). Plurals follow
the language (Russian has few and many, Japanese and Korean only one form).
serve/test_web_i18n.py fails when a catalog lacks a key, has a stale one or
changes a {placeholder}. A ninth language is one file plus one line in
serve/web/i18n.js. The Monitor, Images and About views are still English (the
language setting says so); they are the next step.

Browser tests: a language pick translates and is kept, /?lang= is for one visit,
an unknown code stays English, all seven catalogs load without an error.

docs/DETAILS.md: a section "The web page" (every switch, what the preview can and
cannot reach, the languages, the tests), the Sampling drawer's new name, and the
credits (KaTeX and the contributors of the PRs ported into the page).
2026-10-26 20:22:05 +01:00
fcd3cdbcd7 Web chat: one chat-history design - saved chats, edit and rewind, branch, send while answering (#1787, #2615, #3823, #4004)
Five overlapping proposals, one set of controls (Chat settings > Chats, all off
by default; off, the page is the page from before: one chat, New chat clears it
with an Undo, Send is off while an answer is written, no new request field).

- Saved chats list (#1787): the chats kept in this browser (IndexedDB, one record
  per chat, names of attachments only), a list with day groups, search over
  titles and text, rename, delete with Undo, collapse; a private window or full
  store keeps the list in memory and says so once. Nothing goes to the server.
  The list has the New chat button, so the composer's one is hidden while it is on.
- Edit and rewind messages (#2615 + #3823): under each message, Regenerate (a new
  answer to the same question), Edit answer (its text only: thinking and tool
  calls go, no request) and Rewind to here. Rewind asks, as #3823 proposes: "Edit
  and send again here" hides this message and the ones after it, puts the
  question and its attachments in the box and keeps one Undo bar ("2 messages
  hidden"); "Branch from here" starts a new saved chat with the messages before
  it and leaves this one alone. A branch keeps a title that says where it came
  from ("Branch of ...", also at the very first message) and links back. One
  control does what #2615 had as Edit on a question and as Remove from here.
- Send while answering (#4004, for the web page): "Wait" (default, as before),
  "Send after" (the message waits and goes when the answer is done) or
  "Interrupt" (the answer stops where it is, what is written stays and is marked
  interrupted, the message goes at once; a running tool finishes first). One
  message waits; Take back returns it to the box; Stop keeps it for you to send.

The pure logic is serve/web/history.js, the store serve/web/chatstore.js, the
page's part serve/web/chats-ui.js. Tests: serve/test_web_history.js (Node, with
the IndexedDB stub and cases of #1787), serve/test_web_history.py, and 17 browser
tests (tools/webtest/browser_test.mjs). Screenshots: tools/webtest/shots.mjs.
Also: the Sampling drawer is now Chat settings (its button says so), a wide
window lets a message use the column (#3176 widened only the column).

Ported by hand and merged: #1787 by Scotty Hudson, #2615 by Joyer Huang (uppet),
issues #3823 and #4004 by xvcezrqpbxtpzblrt (their proposals, for the desktop app,
carried to the web page).

Co-authored-by: Scotty Hudson <[email protected]>
Co-authored-by: Joyer Huang <[email protected]>
Co-authored-by: xvcezrqpbxtpzblrt <[email protected]>
2026-10-26 20:14:14 +01:00
Niko1221 a732a34c65 layer-major prefill: a prompt that needs a run_impl feature the walk lacks takes the chunk-major path (#3976)
The layer-major walk is a copy of run_impl's attention half with the experts moved
out. It does not carry four things run_impl does, and when one of them is asked for
the walk now declines (the prompt is read chunk-major, and the one-time "layer-major
off for this prompt" line names the reason):
- the DFlash2 drafter's prompt taps (tap_layers_): the walk would leave the taps
  stale and the drafter would read old rows;
- STRATA_KVA (approximate late layers): the walk would silently read exactly;
- a stop_at_chunk callback (the pipelined batch reader that yields at chunk
  boundaries): the walk reads every token, and stopped_at() must stay -1;
- a pending force_chunk (a --prefill-pipe plan): it is left set for run_impl.
The walk also resets stopped_at_ like run_impl does at the start of a run.

With STRATA_PREFILL_LAYER_MAJOR unset none of this is reached.

Not covered: no model run in this commit.
2026-10-26 20:13:56 +01:00
Niko1221 d02478b7a1 layer-major prefill: stays off on AMD builds (#3976)
Only NVIDIA CUDA has run the layer-major walk. On a HIP build (wave32 or the
gfx906 compat layer) lm_prepare now declines with a reason, so a prompt takes the
chunk-major path and the one-time "layer-major off for this prompt" line names it.
The code still compiles everywhere the shared prefill.cpp does; nothing else
changes. With STRATA_PREFILL_LAYER_MAJOR unset nothing is reached.

Not covered: HIP and SYCL were not built or run (SYCL has its own prefill.cpp).
2026-10-26 20:07:56 +01:00
Niko1221 0acba452ec Test: app-ui 'Quit works once' waits for the shutdown callback (polling, up to 5 s) instead of a fixed 20 ms sleep, so a busy PC cannot fail it (test only) 2026-10-26 20:06:26 +01:00
Norihiro Nishimura e7318fa23d layer-major prefill on a split: a stale hand_in_ left by a chunk-major prompt no longer turns layer-major off (#3976)
run_impl never clears a later stage's hand_in_, so after one short prompt (read
chunk-major) every later prompt was declined with "a layer split's chunk hand-off
is in use". lm_prepare_chain now declines only when a hand-off is really in flight
(a stage's next_run_ is valid) and otherwise clears the stale hand-in pointers.
2026-10-26 20:03:03 +01:00
Norihiro Nishimura 18c5357666 layer-major prefill: even super-chunks (STRATA_PREFILL_LM_EVEN, default on with layer-major) (#3976)
A prompt longer than the common super-chunk is split into the same number of
super-chunks of near-equal size (32K at a 28K fit: 2 x 16K, not 28K + 4K). That
keeps a stage's residual rows in VRAM and lets a split's stages overlap on every
super-chunk. STRATA_PREFILL_LM_EVEN=0 gives the old largest-first split. It only
acts when STRATA_PREFILL_LAYER_MAJOR is set.
2026-10-26 20:03:03 +01:00
Norihiro Nishimura c6f6db857b layer-major prefill on a split: the PLE layer is fine in a first stage that holds layers 0 and 1 (#3976)
The split was declined whenever the model has PLE layers. The PLE block (layer 1)
reads the token history, which only the first stage has, and a later stage never
reaches it. So the split is declined only when the first stage does not hold both
layer 0 and layer 1.
2026-10-26 20:03:02 +01:00
Norihiro Nishimura 6eff377a79 layer-major prefill on a CUDA layer split: each stage runs its layers over the super-chunk and hands its rows to the next stage (#3976)
Problem: layer-major prefill declined every layer split, so a split model kept
re-reading each layer's streamed experts once per 8192-token chunk.

Changed:
- lm_prepare_chain sizes every stage with the same super-chunk (the smallest any
  stage fits, up to four rounds until they agree) and gives them back together;
- a stage runs only its layers [LB, LE); a later stage's rows arrive in its pinned
  R_host;
- a stage before the last writes its final rows into the next stage's R_host on
  the copy stream, then runs the next stage on a thread (std::async) while it reads
  its own next super-chunk. There is one hand-off buffer: a stage's final pass waits
  until the next stage has finished the previous super-chunk;
- layer_major_mid() asks the last stage, the one that reports the rows;
- run() drains the pipeline after the walk and clears the mid flag on every stage.

Not covered: only two stages were tried by the reporter (RTX 3060 + RTX 5060 Ti);
the peer-GPU tier and the stage helper have no layer-major form and decline.
2026-10-26 20:03:02 +01:00
Norihiro Nishimura cc125e06a0 layer-major prefill: no MMQ row table without an MMQ layer; QSA shapes from core::qsa_shapes (#3976)
run_layer_major called mmq::iota on ids_identity, which is null when no layer is on
MMQ; it now skips that. The QSA shapes come from core::qsa_shapes(g) as run_impl
does, instead of being filled in by hand.
2026-10-26 20:03:00 +01:00
Norihiro Nishimura 5602253e3f layer-major prefill: the FP16 path needs no MMQ identity table or token upload buffer (#3976)
lm_prepare required ids_identity and tok_dev even when no layer is on MMQ. Only a
model with an MMQ layer needs the identity table, so the check now asks for it only
then. Without this a model whose layers all take the FP16 path was declined.
2026-10-26 20:03:00 +01:00
Norihiro Nishimura 42094da9c1 layer-major prefill: the FP16 expert path for layers MMQ does not take, decline reasons name the layers (#3976)
Problem: lm_prepare declined every model whose experts are not all on MMQ. All 48
layers of UD-Q4_K_XL take run_impl's FP16 path, so layer-major never ran on them.

Changed: a layer outside the MMQ plan dequantizes each routed expert to FP16 once
per super-chunk and runs two FP16 GEMMs per entry on that entry's FP16 input rows
(a new per-super-chunk buffer mixed_h, the bits run_impl's gather reads), then
adds the group's rows per token with moe_scatter_add as the MMQ path does. The
buffers each path needs are checked separately, and a missing one gives a decline
reason that names it. The summary line counts the groups on the FP16 path.

Not covered: no model run in this commit.
2026-10-26 20:02:59 +01:00
Norihiro Nishimura df7068b6be layer-major prefill: STRATA_HC_Q8 dequant, guards for APR/NVFP4/WMMA layers, QSA indexer calls of this tree (#3976)
Problem: the layer-major walk (#1670) was written for an older base. It read the
hyper-connection matrices as bf16 only, accepted expert formats its MMQ pass does
not carry, and called the QSA indexer with the older argument lists.

Changed (Norhiro's port of vizdatom's walk, kept as one step):
- the HC projections are dequantized per half when STRATA_HC_Q8 holds them as Q8_0
  with no bf16 copy, and released after the GEMMs that read them are enqueued;
- lm_prepare declines (the prompt then runs chunk-major) for a layer that needs the
  WMMA tiles, an APR (Hadamard-folded) layer, and an NVFP4 layer;
- STRATA_PREFILL_DIRECT is not used by the walk.

Not covered: no model run in this commit.
2026-10-26 20:02:59 +01:00
vizdatom ffe40c1695 prefill: layer-major prefill (STRATA_PREFILL_LAYER_MAJOR=1) (#3976, #1670)
A super-chunk of S tokens is read layer by layer: each 512-token sub-chunk runs the attention half,
the FFN half hyper-connection read, the router and the shared expert in order (GDN / QSA / PLE state
advance as in the default path); then the layer experts run once for the whole super-chunk - each
routed expert staged once, its rows of every sub-chunk in one MMQ group, the results added per token.
Expert traffic drops by about S / 512 (a 12.9K prompt: ~509 GB -> ~79 GB on ryzen).

- STRATA_PREFILL_LM_TOKENS: the super-chunk target (default 8192), shrunk to fit free VRAM minus
  STRATA_PREFILL_LM_MARGIN_MIB (default 96).
- STRATA_PREFILL_LM_R=vram|host|auto: the residual rows of the super-chunk in VRAM, or in pinned RAM
  with the copies overlapped on the copy stream (auto: VRAM only when the whole target fits there).
- STRATA_LM_PREFETCH=0: no prefetch (default on: every streamed expert of a layer goes to the stager
  when its attention halves start; the ones the routing skips are skipped if not started).
- Not for a layer split, a peer GPU, image prompts, the fused layout or a prompt under two chunks.
- on_chunk still reports every sub-chunk in order; layer_major_mid() tells serve not to checkpoint
  before a super-chunk end (the state is already at that end).

Port to 0.1.43.1: the copied run_impl body follows the new base. Every blob the stager or the copy stream reads later
comes from blob_stable() (blob() may hand out a recycled STRATA_IO_PREFETCH buffer); the token ids go up through tok_up,
the QSA step records and the PLE rows' pageable fallback through up_ring (#1712, #1996); the PLE gather threads pin to
--aux-cpus; bf16x2 takes m.f16_io; the K8V4 append is upstream's, which writes the host copy and the staging pool by
halves. The indexer passes the INT8 pooled keys as the original does, in the QSA port's order (pooled_i8 and
pooled_scale after block_pos, the int8 qsa_block_scores(_tc) overloads); STRATA_IDX_INT8 stays opt-in and unsupported on
the prompt path, as in the original. stage_one calls the stager's pass() before wait(): with batched claims a routed job
after about a ring's depth of skipped prefetch jobs waited forever. lm_prepare reads the fused layout from m.fused_bufs
and declines while STRATA_DBG_FEAT, STRATA_DBG_ROUTE_OVERRIDE or STRATA_PREFILL_DUMP_R_ALL is set. serve's near-tail
checkpoint (--prompt-cache-tail) is not taken mid-super-chunk, so under layer-major it is taken only when the last
super-chunk is a single sub-chunk.

From vizdatom's #1670, original commit .
2026-10-26 20:01:55 +01:00
vizdatom a6df0023a4 prefill: kernels for a layer-major expert pass (moe_shared_init, moe_scatter_add) (#3976, #1670)
moe_shared_init writes the shared expert part of moe_combine before the routed experts are known;
moe_scatter_add adds one expert group rows into a per-token accumulator, per token in a fixed order
(deterministic, no atomics).

From vizdatom's #1670, original commit f0173382d0646907917edf99e59617c1412d898e.
2026-10-26 20:01:36 +01:00
vizdatom 162dec3411 prefill: the per-layer walk warms this layer's and the next layer's file-tier experts (STRATA_PREFILL_WARM=1) (#3976, #1670)
Chunks below stream_all_min() stage each layer's unpinned experts on the stager's few threads, and with a mapped
file tier each of those is a chain of 128 KiB page faults, a shallow queue for the drive.  With STRATA_PREFILL_WARM=1
(and a source that warms: the GGUF in place, or experts.bin with STRATA_BIN_WARM=1) a background thread hands the
source the layer's whole list at once before the stager starts, and the next layer's list from the previous chunk
(less what is on the GPU or pinned) while this layer computes.  It only warms pages: the stager copies the same
bytes.  At most four lists queued; a thread that falls behind drops the oldest.  Off by default: the A/B arm.

Port to 0.1.43.1: the warmer thread pins itself with aux_cpus::pin_current_thread() first, like the other helper threads since --aux-cpus (it would otherwise inherit the host thread's CPU).

From vizdatom's #1670, original commit cabe33c4f585d4c9dd0af1fc9f014d00e2920042.
2026-10-26 20:01:36 +01:00
vizdatom fca1936c2c prefill stager: a job waits for the previous job on its buffer; skip() (#3976, #1670)
issued is a watermark, so a job nobody waits for could be passed while its thread was still writing
the pinned buffer the next job on that buffer was filling - a corrupt expert blob. A job now also
waits until the job kRing before it has marked itself ready. skip(j) lets a job the routing turned
out not to need be marked ready without reading it (the layer-major prefetch). The default walk
consumes every job in order, so neither changes it.

Port to 0.1.43.1: the stager now claims batched runs and sleeps in issued.wait, so the ready[i - kRing] wait
sits in the run's per-job wait loop after the issued wait, and a skipped job is left out of the run's gather
(no memcpy, prefill_copy or copy_blobs entry); its ready flag and `active` stay with the run's single store and
decrement. New pass(j): skipped jobs are never issued, so the layer-major walk moves the watermark to the
routed job it is about to wait for (store plus notify_all, which wakes sleeping threads); without it a routed
job after about a ring's depth of skipped ones waited forever. The default walks never call it.

From vizdatom's #1670, original commit 149b899548fbc98977219e6fd4ee7866384e61f6.
2026-10-26 20:01:36 +01:00
Niko1221 8486336f9b Merge f-027-gh-eng-s (#3291 #3955 #3317 #4036 #2893 #3218 #3200: serve, tools and setup ports) 2026-10-26 20:00:53 +01:00
Niko1221andSakurai Momoka bcdb92ac68 Web page: Load and Unload in the header (#3550)
One button in the header of every tab: Load while the model is unloaded, Unload
otherwise. It calls POST /load and POST /unload (the server already had both,
for its own page only). Unload asks first and never interrupts a turn: the
button is off while an answer is being written, an image is made or a request is
queued. The pill and the Monitor say Unloaded / Loading / Unloading, a message or
an image on an unloaded model shows Loading, and a poll that started before the
POST cannot flip the button back (the POST's state is held until a later poll
agrees). On a phone the pill's text is the first thing the header row gives up
(it measures the row, not a fixed width), so Load/Unload and the theme button
stay on the screen from 320 px.

Browser tests (tools/webtest/browser_test.mjs): unload with its question, load
back, declining, not while answering, a message that loads, the header at 320 to
640 px.

Ported by hand from PR #3550 by Sakurai Momoka (sakuumomo); the texts now go
through the page's translation catalog.

Co-authored-by: Sakurai Momoka <[email protected]>
2026-10-26 19:59:54 +01:00
Niko1221andThomas Ivesdal-Tronstad 0aa64b2de3 Web chat: preview a generated svg or html block in a sandboxed frame (#1628)
A complete ```svg or ```html block in an answer gets a Preview button. The
block's own code is drawn in an iframe (srcdoc) that is cut off from the page:
sandbox="" (an opaque origin: no access to the page's DOM, storage, cookies or
server, no scripts, forms, popups, downloads or navigation of the page), a CSP
in the frame's document (default-src 'none': no network, fonts, frames or media;
styles inline, pictures as data: only; no <base>, no form target), no referrer
and no permissions. A well-formed svg is shown as an <img> with a data: URL: an
image runs no script, loads nothing and animates in every browser. The frame
takes the page's light or dark colours and is built again when the theme
changes. A page (html) can also be run WITH scripts, one block at a time, after
a warning that is shown every time: sandbox="allow-scripts" only (still an
opaque origin), the CSP allows the block's inline script and no address; closing
the preview ends it.

What the sandbox flags alone do not stop is a link or <meta refresh> inside the
model's html loading a web page into the frame. The browser test shows it does
(tools/webtest/browser_test.mjs, with and without the header). So the page is
served with Content-Security-Policy: frame-src 'none' (nothing else: a srcdoc
frame inherits the page's policy, and the script preview needs its inline
script), and a <meta http-equiv=refresh> in a previewed page is dropped.

tools/webtest holds the mock server (the page's own files, scripted answers,
no model) and the browser tests; serve/test_web_preview.js checks the documents,
flags and frames, serve/test_web_preview.py the header and that the page never
listens for a frame's postMessage.

Ported by hand from PR #1628 by thoives.

Co-authored-by: Thomas Ivesdal-Tronstad <[email protected]>
2026-10-26 19:57:56 +01:00
Niko1221 675f7fc238 Merge f-027-gh-eng-k2 (#2526 #1864 #1911 #4005 #3060 #4010 #3809: PR ports for the conversation cache, prefill events, expert source and batch groups) 2026-10-26 19:54:59 +01:00
chaog992 c47c699486 prefill: opt-in fused int8 prompt experts for the native Q2_0 pack on sm_80+, STRATA_PF_FUSED_Q2N=1 (#3809)
A PC whose CPU lacks the AVX-512 set that setup checks for gets the native Q2_0 pack. On a CUDA card of sm_80 or newer its
Q2_0 / Q2_0 prompt experts run through MMQ, although the fused int8 kernel for that format (#2145) is in the binary and set
up only on a Turing card (b6443f3ae).

STRATA_PF_FUSED_Q2N=1 sets that kernel up on any sm_80+ card and takes it for the Q2_0 / Q2_0 layers, under its own variable
so that STRATA_PF_FUSED=1 keeps its bits. STRATA_PF_FUSED=0 and STRATA_PF_FUSED_NATIVE=0 still turn it off. A failed setup
leaves the occupancy as it was and the layers on MMQ.
- moe_fused_iq.cu: DevInfo.q2n; dev_info() sets up Q2_0 gate/up when the switch is set; gu_covered() and native_supported()
  take the pair only when q2n is set (a Q2_0 gate with another down type is refused, as on Turing).
- tests/cuda/prefill_fused_iq_test.cpp: --unset-pf-fused leaves STRATA_PF_FUSED unset (the engine's default) and skips below
  sm_80; CMakeLists.txt: the ctest entry prefill_fused_iq_test_q2n runs the Q2_0 / Q2_0 pair with the switch alone (exit 77 =
  skipped without an sm_80+ card; CUDA branch only).
- docs/DETAILS.md: one section in the author's measured numbers (RTX 5080) and the A4000 caveat below.

Unset (the default), nothing changes: dev_info() reads the variable once and sets nothing up, gu_covered() and
native_supported() return what they returned. Read line by line against base: the Turing path (t75) and the HIP path are
untouched. Set, the prompt arithmetic changes (first-token KL against FP16 0.0110, MMQ 0.0112, the author's figure) and the
fused path's smaller buffers let --prefill auto pick a larger chunk, which is most of the author's 16K gain; the generated
text can change.

Not verified: an RTX A4000 (sm_86). The kernel was taken on every CUDA card in #2145 and then kept to Turing after an A4000
no longer matched the old output and read 4K prompts 4.5% slower; the PR does not repeat that run. Merge note: #3277 and #3779
change dev_info() and native_supported() in the same file.

Tests: prefill_fused_iq_test_q2n and prefill_fused_iq_test pass on an RTX A4000 (sm_86, synthetic data); on the Q2_0 / Q2_0 layer the
fused kernel is 1.76x / 1.30x / 1.24x the MMQ path at 2048 / 4864 / 8192 tokens with the same relative RMS against FP32. Builds on CUDA
sm_86 (Linux) and sm_120 (MSVC). Not covered: no model run; the engine-level A4000 check is the gate.
2026-10-26 19:54:00 +01:00
Niko1221andDavid a965c962cf Web chat: draw math with the bundled KaTeX, opt-in and offline (#2972, #2974)
Chat settings > Render math (off by default). When it is on, \( \), \[ \], $x$
and $$ $$ in an answer are drawn as formulas by KaTeX 0.19.0, which is vendored
in serve/web/vendor/katex (script, style, 20 woff2 fonts, MIT licence and a
PROVENANCE.json with the npm integrity and the sha256 of every file, so the page
needs no network). Nothing of KaTeX loads until an answer holds a formula.

The shared core (math.js, from #2972) keeps the text of every message exactly as
the model wrote it: formulas are swapped for placeholders before the Markdown
runs and drawn over them afterwards, so history, copying, the Markdown export
and the requests never see the drawing. Code, link targets, prices ($5 and $10)
and a formula still being streamed stay text. A formula that fails, is over
16 KiB or times out stays as its source text.

Safety: KaTeX runs with trust off (no \href, \url, \includegraphics, \htmlClass:
they are drawn as their own name), maxExpand 1000, maxSize 20 and a fresh macro
table per formula; it builds DOM nodes itself, so no HTML string from a model
reaches the page. A formula already drawn is placed again at once while an
answer streams, so it does not flicker back to source.

serve/test_web_math.js runs the real vendored KaTeX against hostile formulas;
serve/test_web_math.py checks the vendored files against PROVENANCE.json and
that nothing outside this server is addressed. serve/web/vendor is -text in
.gitattributes so the hashes hold on a Windows checkout.

Ported by hand from PRs #2972 (core) and #2974 (KaTeX) by CC-David-CC.

Co-authored-by: David <[email protected]>
2026-10-26 19:52:56 +01:00
Niko1221 d5cca319c9 Web page: a translation mechanism, and a language choice in About > Settings
The served page had no way to translate its text: every string was English in
the HTML and the script. Now the English text stays in the code next to a key
(tr("key", "text", vars), data-i18n="key" in the HTML), a language is a catalog
serve/web/locales/<lang>.json, and a catalog that is missing or has no entry for
a key leaves the English text. The language is the user's choice (About >
Settings), kept in this browser; nothing is chosen from the browser's language,
because a page translated in part would mix two languages. /?lang=de shows one
for a visit.

The server hands out /web/locales/<lang>.json and /web/vendor/katex/** from
serve/webfiles.py: whitelisted names and types, no backslash, colon, percent or
dot-dot, and the resolved path must stay inside the folder (also against a link
out of it). tools/web_i18n.py reads the English text out of the code, writes
en.json and checks every catalog for the same keys and {placeholders}; the
check runs once the catalogs exist.
2026-10-26 19:52:26 +01:00
Niko1221 3813a459c4 Split serving: exactness and speed check tool, docs draft
Idea and manual setup by @MarcoTool in #4000.
2026-10-26 19:48:29 +01:00
gopinath87607 fa128f91e8 docs: --prompt-cache-every does not set the size of the pieces a prompt is read in (#3311, #2834)
Only the part of PR #3311 by gopinath87607 that is about this engine: a periodic mark is saved at a chunk boundary of the batched prefill and the read carries on
(src/program/generate.cpp, periodic_checkpoint), so the interval is not a piece-size dial; what cuts a read into runs is the re-read, the root, a message
boundary and the start of the last turn; --prompt-cache-every 0 skips the periodic and the near-tail marks. The wall-time numbers and the comparison with the Maya
checkout belong to another engine and are left out. New section in docs/PROMPT_CACHE_TAIL.md, linked from the option's row in docs/DETAILS.md.
2026-10-26 19:47:22 +01:00
Niko1221 47c98bb23a strata launch --show-key shows the key of a model setup.py set up (#3570)
--show-key only knew a running launch or a preset. A model set up by setup.py (START-HERE.bat, ./setup.sh) keeps its generated key in strata-<model>.json, so the command
said there was no key and people read the JSON by hand. With no launch and no --preset, it now looks at the run configs of the installed models: one model with a key
prints that key (stdout is the key alone; a hint line with the model and the file goes to stderr), several models ask for --model NAME (any case, the size or the full
id), a name that matches nothing says so. --preset never falls back to a setup config, and no key is printed while the choice is open. Tests:
tests/terminal/test_launch_show_key_setup.py.
2026-10-26 19:45:45 +01:00
Niko1221 726472a229 Merge f-027-int2 into f-027-app2 2026-10-26 19:43:59 +01:00
Niko1221 10a62b8d2f docker: model files left in the older /data/models stop the start instead of being downloaded again (#3421)
A volume or bind mount from the 0.1.x image keeps its models in /data/models. The entry point moves a folder that is part of the volume, but a folder the person
mounted there is left alone, and a failed move leaves the files too; with no MODEL_PATH the start then did not see them and downloaded the model again
(tens of GB) without a word. Now, when .gguf files are still in /data/models and MODEL_PATH is not set, the container stops with code 2 and names the folder,
says what to do (MODEL_PATH to the folder with the shards, or the files into /data/Strata-data/models) and that nothing was downloaded.
ALLOW_DOWNLOAD=1 goes on anyway with the same text as a warning. docs/DOCKER.md lists the setting. The other points of the issue (the key, the removed
variables, the status loop) are the owner's design questions and are not changed here. Tested by running the guard function with sh (tests/terminal/test_docker_files.py);
the whole container was not run (no docker on this PC).
2026-10-26 19:42:58 +01:00
Niko1221 690172c464 Split serving, first version: GEN 0 prefill=1 in the engine, serve/split.py (SplitEngine, config split_serve / --split-serve), tests
Prefill on one card, decode on another, the session handed over in RAM through the engine's
own session file (SAVE/RESTORE). The engine's new request form GEN 0 prefill=1 reads a prompt
as a request does, up to where the first verify window starts, and stops with the session at
n - 1 tokens, which is the place a continuing request resumes from. Opt-in; without
split_serve nothing changes.

Idea and manual setup by @MarcoTool in #4000.
2026-10-26 19:42:41 +01:00
Niko1221 4a759f82d4 DeepSeek engine packages: add sm_89 (RTX 40) to the CUDA builds (#3935)
Problem: the strata-dsv41 packages were built for sm_86 and sm_120 (+ PTX 120). An RTX 4090 user was stopped at setup on 0.2.4.1; since
b750d757f an sm_86 package is accepted on an sm_89 card (cubin same-major rule), but the engine had no sm_89 image of its own, and the docs said
nothing about RTX 40 or about building the engine for a card the packages do not cover. The reporter built strata-dsv41 for 86-real;89-real
themselves and ran the 2.00 set at 40.2 tok/s on an RTX 4090 48 GB (theirs, --dspark on).

What changed:
- packaging/build_dsv41_cuda.sh and build_dsv41_windows.bat: the default architectures are now 86-real;89-real;120, so the DeepSeek CUDA 13
  packages list sm_86, sm_89, sm_120, compute_120 (Windows and Linux alike). The CUDA 12.9 line in the script header says ARCH="86-real;89".
- CMakeLists.txt: a build of strata-dsv41 with no -DCMAKE_CUDA_ARCHITECTURES defaults to 86-real;89-real;120 (the Qwen engine's default
  stays 120, and the warning now names the architectures it picked).
- strata_control/engine_component.py arch_native(): true when a package holds machine code for the card itself. resolve() and
  feed.select_engine() use it as the last tie-breaker, so with otherwise equal packages the one with the sm_89 image is taken on an RTX 40;
  an sm_86-only package still fits (no refusal).
- setup.py: the "engine not available" stop now points at the build recipe (docs/DEEPSEEK41.md); the developer line names the
  architecture to pass. The card list already said RTX 30, 40 and 50.
- docs/DEEPSEEK41.md, docs/MODELS.md: RTX 40 coverage, the reporter's measurement quoted as theirs, no claim of our own (we have no Ada card),
  and a section "Building the engine for your card".

Tests: tests/terminal/test_engine_manifest.py (arch_native; the build scripts and the CMake default name sm_89 and both OSes agree; the
package list they give is sm_86 sm_89 sm_120 compute_120; an sm_89 card is accepted and prefers the package with sm_89). tools/test_setup_dsv41.py
unchanged and passing.

Not covered: no run on an Ada card (gate script in phaseA-tests/gh027/gates/3935-ada-dsv41.sh).
2026-10-26 19:42:11 +01:00
Norwin ab48216a7b docs/MULTI_GPU.md: say when a layer split pays on a slow CPU (#3930)
One bullet from PR #3930 by NorwinYu: on two identical RTX 2080 Ti over a dual-socket Xeon E5-2696 v4 a plain split moved decode from 35.2 to 51.0 tok/s and an
11.6K prompt from 672 to 908 tok/s, because the CPU pool per window fell from 9.52 to 0.79 ms (STRATA_DECODE_TIMING=1). The page's own 5080 + 3090
figures, where the second card is on par, came from a fast CPU; the bullet says so and that it is one measurement. No code changes: setup already splits
across two cards. Shipped without a hardware run of ours; the reply asks owners of two older cards on slow CPUs to compare and report.
2026-10-26 19:40:24 +01:00
Matthias Nott ad1a310f55 Web page: use wide windows and fit narrow screens (#3176)
The chat, About and Monitor columns grow with the window (the chat and the
composer to 1100 px, About to 1100 px, Monitor to 1920 px), the metric cards
wrap with auto-fit, and under 640 / 560 / 480 px the controls get bigger
touch targets, the composer bar wraps, and long words, code and tables stay
inside the viewport.

Ported by hand from PR #3176 by Matthias Nott (mnott); the CSS is his.
2026-10-26 19:40:00 +01:00
Vop3p c2490b49f5 docs/AMD_HIP.md: why STRATA_SH_STREAM goes both ways, and leave it off on Windows with KV streaming (#3895)
Two paragraphs from PR #3895 by Vop3p, put under the table row they explain: the fork pays only when it has a CPU-pool wait to hide (4,859 slots on one
RX 6900 XT, GPU-reach wait 20.7 of 39.1 ms; with every expert resident it is 43.4 against 62.9 tok/s on an R9700), and the Windows RX 6900 XT fault with
the fork plus --kv-resident, its bisection and its limits (one PC, kernel not isolated). The text now says what the code does about it and asks RX 6000 owners on Windows to test.
2026-10-26 19:39:43 +01:00
Niko1221 5f81454038 AMD on Windows: the shared-expert fork with KV streaming is not measured, never picked, and warned about when set by hand (#3895)
On one Windows RX 6900 XT STRATA_SH_STREAM=1 together with --kv-resident failed the engine on its first request (verify: unspecified launch failure).
setup --calibrate (tools/calibrate.py) no longer measures the fork on Windows when KV streaming is on (the fork is off by default on AMD, so
nothing changes unless the person sets it), strata tune's never-pick list holds the combination (strata_control/tune_knobs.py), and a config that
sets STRATA_SH_STREAM=1 with --kv-resident on Windows HIP gets one warning line when the engine starts (serve/server.py sh_stream_warning). Nothing is refused or
removed: the variable still reaches the engine. Shipped without a hardware run, from one report; the reply to #3895 asks RX 6000 owners on Windows to test
and report. Tests: tests/terminal/test_sh_stream_windows.py.
2026-10-26 19:39:42 +01:00
Niko1221 d98d8f977f docs/TROUBLESHOOTING.md: every entry of v0.2.4.2 is back, updated for 0.2.7
The 0.2.5 rewrite dropped about 25 entries: the Mac quarantine steps, strata doctor, sending logs, the first-start freeze, the graphics memory the app keeps free,
Smart App Control, UPDATE.bat and git pull (#1276), the build tools and the gcc/glibc compile problems, the engine checksum, the engine stopped unexpectedly,
slower than the tables, the empty-reply and Turing switches, --expert-cache and --resident-budget-gib, the page-locked OOM, the NVIDIA 0x141 reset, CUDA 13.2 on
RTX 50, the AMD section (KFD, the build list, 0xC0000005, the driver reset, pinned allocations, verify timeouts) and the body of the verify-timeout entry, which
the rewrite had left empty. Each is back with the 0.2.7 facts: the app's reserve defaults (native-resources.mjs), the Drafting setting, --vision on|off|gpu|cpu
in the Terminal and --vision no in a source checkout, Intel Arc's CPU reader, and the Terminal route next to the source-checkout one.

Every pointer resolves: setup.py's driver-reset pointer, the server's Smart App Control message, DETAILS.md's STRATA_KEEP_EMPTY_TURNS / STRATA_TOPK_STREAM line,
OLDER_GPUS.md and MAC.md. tests/terminal/test_troubleshooting_page.py checks the entries, those pointers and the page's own links and anchors.
2026-10-26 19:36:15 +01:00
ruibeikaa a338990186 qsa_pattn: sm_6x prompt attention without partials, opt-in STRATA_PATTN_SM60=1 (#3943)
Problem: below sm_70 the prompt reads each QSA layer's attention through the
decode kernels (qsa_prompt_attn_batch returns false without tensor cores): 32
queries a call, every chunk's partial results (817 KB a query) written to
scratch and merged by a second kernel.

Change: src/kernels/cuda/qsa_pattn.cu keeps the contributor's default variant
only, ps_score_x_kernel (q in registers, the chunk's K staged with 16-byte
loads, two cells at a time through a select-free reduce-scatter) and
ps_value_d4_kernel (four dims per thread, merged into the output as the merge
kernel does). Every float operation of attn_chunk_kernel_pre75 and
attn_merge_kernel is kept in its order, so the output is the decode kernels'
bit for bit. The PR's other variants (per-chunk, fused persistent, bench split
codes), pattn_bench and the variant argument are not ported; pattn_bench needs
a STRATA_QSA_DUMP file from a model run.

Gating: compiled only in the CUDA build with STRATA_EXPERIMENTAL_SM60 (HIP and
Metal link a stub that answers false; prefill.cpp keeps the call out of the
Metal build). At run time only with STRATA_PATTN_SM60=1, on compute capability
6.x, int8 K/V, head_dim 256, 12 query heads per KV head, at most 33 chunks.
Unset, the only cost on the default path is one cached int test per QSA layer,
and only when qsa_prompt_attn_batch has already declined. Any refusal or a
failed launch returns false and the usual 32-query loop runs (one warning, no
exit; the PR called exit(1) on a launch failure).

Tests: src/kernels/qsa_pattn_parity.cpp (new, registered in CMakeLists.txt in
the experimental sm_60 test block, skips with 77 when the kernels are absent)
compares the kernels with the Daily's chunk kernel plus attn_merge_kernel on
random int8 K/V: a shuffled page table with ~3% unresident pages and a run of
64 unresident pages (whole chunks masked, some queries empty), three selection
shapes, caps 64 / 130 / 2051 / 2112, widths 0..cap, n_q up to 40 with a
scratch that holds only 2 decode-layout queries (several batches), junk in the
scratch and output. On a Pascal card it also requires the checked entry to take
the shapes and qsa_decode_attn_batch to match. The kernel entry that skips the
compute-capability test (qsa_pattn_batch_any_arch) exists for this test only.
Results: z440 (RTX A4000, sm_86, CUDA 13.0, build with
-DSTRATA_EXPERIMENTAL_SM60=ON, gpu lock): "qsa_pattn_parity: 468 cases, 0
differ, 29724792 nonzero output values in all -> PASS". sm_60 compile of the
whole tree (CUDA 12.9, P1004): exit 0.

Not covered: no P100 run here (the Pascal-only checks of the test and the
speed claims wait for the gate, phaseA-tests/gh027/gates/3943-pattn.sh). The
contributor's numbers (43.3 -> 30.8 ms a 1,536-query chunk per QSA layer at
26K context, 2-3% less time on 8K-150K token prompts on four dies) were not
re-measured; compute capability 6.1 is covered by the same switch and was not
measured by anyone.
2026-10-26 19:24:50 +01:00
Niko1221 9db497056b docs: the two master switches in the 0.2.7 release-notes draft 2026-10-26 19:14:00 +01:00
Niko1221 a3ab37fc03 Master switches DS4_DEFAULTS=026 (DeepSeek) and STRATA_DEFAULTS=026 (Qwen): every default changed since v0.2.6 gives v0.2.6's behaviour; explicit per-feature variables and flags still win; every rule is a pure function with a v026 argument and a test 2026-10-26 19:13:25 +01:00
Niko1221 6f1bd21e59 App: "Faster answers on 12 GB NVIDIA cards" row; DeepSeek speed-ups Off also writes DS4_DEFAULTS=026
The Speed group gets a row for Qwen on a discrete CUDA card of 13 GiB or less: "Prefers the parts of the model already on the graphics card, which makes answers
much faster on 12 GB cards. Turn off if answers look worse than before." On (recommended) by default; Off writes STRATA_ROUTE_PRIOR=0, On removes it. The
DeepSeek speed-ups row, when Off, writes DS4_DEFAULTS=026 and the six DS4_*=0; On removes all of them, and a file that holds only the six (an earlier test
build) shows as Off. The Qwen "0.2.6 behaviour" switch is Terminal-only and is not in the app. New strings fx82 and fx83 in the eight fix catalogs.
2026-10-26 19:10:55 +01:00
Niko1221 211fca87df Shared speed-switch table and Terminal: DS4_DEFAULTS=026 with the DeepSeek off switch, the 12 GB NVIDIA row, and a Qwen "0.2.6 behaviour" row (Advanced only)
- DeepSeek speed-ups Off now writes DS4_DEFAULTS=026 (the engine's master switch back to the 0.2.6 defaults) plus the six DS4_*=0 for older engines; On removes
  all seven. A file with only the six (an earlier test build) still reads as Off and On removes them (`legacy` entries).
- New switch "Faster answers on 12 GB NVIDIA cards" (the residency-biased routing default): on by default, Off writes STRATA_ROUTE_PRIOR=0. Applies to Qwen
  Flash-Next (no arch) on a CUDA build with a discrete NVIDIA card of 13 GiB or less (not a chip that shares system memory), the engine's own rule.
- New Terminal-only switch "Qwen: 0.2.6 behaviour" (STRATA_DEFAULTS=026, off by default), a row of the Advanced table for every Qwen model, for
  troubleshooting; the app never shows it. DeepSeek has DS4_DEFAULTS instead.
tests/terminal/test_speed_switches.py covers the new defaults, applicability, the seven DeepSeek entries and the legacy six, the Advanced rows and the
comparison with the app's table (the Terminal-only switch is left out of it).
2026-10-26 19:10:55 +01:00
xiezhipeng-gitandLeonardo Murillo 9ea044962a OpenAI logprobs, top_logprobs and allowed_token_ids on /v1/chat/completions (#3200)
/v1/chat/completions on the CUDA and HIP engine accepted logprobs and
returned an empty choices[0].logprobs.content. Now "logprobs": true with
"top_logprobs" 0 to 20 (and the legacy "logprobs": N) returns one entry
per token with its alternatives, streamed or not, and "allowed_token_ids"
returns a distribution renormalised over exactly the ids the caller named.

Engine (src/program/generate.cpp): the GEN line takes logprobs=K and
allowed=a,b,c. After each "T id" the engine writes one "LP <logprob>
[id:logprob ...]" line from the verify window's head logits before
sampling, penalties and temperature. The SYCL engine already wrote LP
lines for logprobs=K; it now reads allowed= as well, with the same line
format. A request without logprobs reaches none of this: both call sites
are behind "req_logprobs >= 0", a local int of the request, so the default
path prints what it printed before and does no extra work per token.
Changes from the PR: the call sites are guarded instead of calling the
function for every token, the function is static, the token index is
checked against the row, and the Chinese comment is in English.

Server (serve/server.py): LP lines are read before the token is yielded
and belong to the request that asked (tagged with its cancel Event), the
entries spell the reply, json_schema replies return no logprobs rather
than a mismatching set, an allowed id past the vocabulary is a 400, and
logprobs on a --parallel server is answered for max_tokens 1 only (a
slot's BT lines carry no LP line). serve/structured.py: _extract_json is
now extract_json (the logprobs alignment uses it); the two uses added by
the #2893 retry follow the rename.

Left out: the PR's bench and result files (none in the diff). #3332 (/v1/completions) and #4056
(logprobs on it) build on this change; this one needs nothing from them.

Tests: serve.test_logprobs and serve.test_logprobs_parallel (30 tests with
the PR's), serve.test_structured, serve.test_responses: 94 tests OK.
Not covered: no model ran here. The PR author reports 28 of 28 on an RTX
5080 with MTP and on a GTX 1080 Ti; the layer-split pipelined decode path
(the first call site) was not run by anyone, and the LP values were not
compared with a CPU softmax of dumped logits.

Co-authored-by: Leonardo Murillo <[email protected]>
2026-10-26 19:08:35 +01:00
Niko1221androdi4eg 2299c8f70a native_mmvq: sm_89 MMVQ-IL tables opt-in with STRATA_MMVQ_IL_89=1 (#3453)
Review of the first port: it registered kIlRows_89 / kIl1Rows_89 for every
compute capability 8.9 card by default. Bit-identity of an interleaved-kernel
choice to native_mmvq is a property of the compiled code, and the release
engine builds its own sm_89 cubin that no card of ours can run; the issue also
saw no end-to-end decode change on a 4090 or a 4080, and the 4070 and 4080
runs agree in 53 of the 75 cells of the 2-4 column table (33 of them non-zero). So the default
change has no measured benefit and an unproven bitwise claim on the card it
changes.

Change: sm_89 answers the shared table, with no one-column row, exactly as
before, unless STRATA_MMVQ_IL_89=1 is set (read once, like the neighbouring
switches). With it set the cells from the two community runs apply. Nothing
else in the lookup changes; sm_86, sm_120 and the others are untouched.

Why it is bitwise: native_mmvq_il_kernel keeps a separate accumulator set per
row (acc[v][i][c]) and sums each row over the same chunk order and the same
warp_sum, so the rows-a-warp count R only changes how many rows a warp walks,
not the order of any row's float additions; the plain-versus-interleaved
equality is what mmvq_il_parity measures (memcmp against native_mmvq for rows
1/2/4, T 1-8, odd row counts). That test passes on sm_86; it has not run on an
Ada card.

Tests: mmvq_il_parity now checks the unset behaviour (sm_89 equals the shared
table, Q3_K and the one-column row answer 0) or, with STRATA_MMVQ_IL_89=1, the
4070/4080 agreement. A second ctest, mmvq_il_parity_89table, runs the table
check with the switch set and no kernel case. On z440 (2x RTX A4000, sm_86,
CUDA 13.0, gpu lock): mmvq_il_parity and mmvq_il_parity_89table both pass, and
the binary prints "mmvq_il_parity: OK (every case, T 1-8, rows 1/2/4 and the
table, bitwise)" with the switch unset and set.

Model gate owed on an sm_89 card: phaseA-tests/gh027/gates/3453-sm89-il.sh
(mmvq_il_parity on the card, then identity of the answers with the switch on,
then a speed A/B that decides whether the default flips).

Co-authored-by: rodi4eg <[email protected]>
2026-10-26 19:07:23 +01:00
SuperDistance a3bb38ec45 batch-groups: opt-in adaptive-tier rounds in the pipelined loop, STRATA_PIPE_ADAPT_EVERY (#4010)
With --batch-groups above 1 on a layer split the pipelined batch loop never starts a round of the adaptive VRAM tier (the
serial batch windows start one every --adapt-every windows). While the slots are busy the tier only moves when they go back
to the solo path, so a fresh engine keeps its start-up profile for minutes.

STRATA_PIPE_ADAPT_EVERY=N (opt-in; unset, 0 or not a number is off): after N group windows pump starts no new group on the
first stage, lets the groups in flight finish, then runs one round as batch_step does between serial windows, adapt() and
apply_pending(true), so nothing in flight reads a cache slot while experts swap and the new experts have landed before the
next launch. Not while a prompt is read beside the slots (in_prompt_read, its loan of cache slots), not with --adapt-async,
not while the last round's copies are in flight, not with the tier off. The rule is two pure functions in
include/strata/program/batch_groups.hpp (pipe_adapt_every, pipe_adapt_due) with 12 new cases in batch_groups_test. With the
switch set the engine says so once at start and the "strata batch (pipelined ...)" line adds the rounds, the experts
swapped in and the time spent. docs/BATCHING.md documents the switch with the author's table (2x RTX 5090, six streams,
309 to 410 rows/s over eight bursts after a fresh start, two engines per setting).

Reviewed: unset costs one counter increment per group window and one false check where a group starts, and prints no new
line. A due round cannot wait forever: no new group starts, the groups in flight finish, then the round runs. A prompt read
sets in_prompt_read before the reader thread starts (including STRATA_PIPED_CONCURRENT) and clears it after the reader
joined; the drain after the read runs the round if it is due.

Tests: batch_groups_test (the new cases). Not covered and still the author's to show (the PR is a draft): a soak with
prompts read beside the slots, a Linux run, another GPU pair, more than two stages, and exactness settings with the switch
set. The model gate is written down.
2026-10-26 19:03:48 +01:00
Niko1221 557aad0265 hip-<package> is accepted wherever the backend is named (#4013)
STRATA_ENGINE_BACKEND=hip-rdna2 worked for strata update (updater.detect_hardware), but strata update --backend hip-rdna2 offered no package and
strata launch stopped with "STRATA_ENGINE_BACKEND='hip-rdna2' is not one of cpu, cuda, hip, metal, sycl". engine_component.split_backend reads
hip-<variant> as the hip family and its package; detect_backend (the start path), detect_hardware and update --backend all use it. Any other name is
checked as before. Tests in tests/terminal/test_gfx_selection.py.
2026-10-26 19:03:30 +01:00
SanghunYun 2b13a44933 expert source: STRATA_RAM_ADAPT copies the incoming expert out of its stage buffer when it is still there, and moves up to 4 pairs a layer (#3060)
STRATA_RAM_ADAPT (opt-in, --serve, one GPU, needs --adapt-every) lets the resident RAM copy follow the conversation:
experts the windows keep routing to take the RAM places of unused experts of their layer. Each move re-read the incoming
expert from the file and a layer moved one pair per window, although the incoming expert was usually read for the window
before and its blob still sits in that read's stage buffer.

What changed:
- FileExpertSource::replace_resident_staged(layer, in, out): the same move as replace_resident, with the bytes copied out
  of the finished stage buffer of `in` (no file read). The copy runs under stage_mu_, so claim_stage cannot give the buffer
  to another blob meanwhile; it needs a non-busy buffer whose key is `in`, and returns false (nothing changed) otherwise,
  and the pair then goes the old way. The Foresight hold/unhold around the copy is the one install_resident uses.
- generate.cpp: a layer pairs up to 4 of its most routed outside experts with up to 4 of its least routed ones in the copy
  (outgoing count below 0.5, incoming 2 or more), largest difference first. A pair whose incoming expert is staged moves by
  copy; `n` now caps only the pairs that need a file read. The log line counts the moves and how many came out of a stage
  buffer (ram_staged()).
- docs/DETAILS.md: the RAM_ADAPT bullet in the low-RAM list in plain words and one row of the measured table (the author's
  numbers: RTX 3060, IQ2_XS and IQ3_XXS, 120 ShareGPT requests, 3 interleaved runs each).

Unset (the default) nothing runs: the block is guarded by ram_adapt_n > 0 as before. The weights moved are byte copies of the
same blob; which experts sit in RAM changes, as it already does with RAM_ADAPT, so greedy token ids may differ from run to
run once it is on. Changed from the PR: the DETAILS.md hunk was rewritten against the current, shorter bullet.

Tests: file_expert_source_test (new staged cases; the positive case needs unbuffered load and is skipped without it).
Not covered: no model run. Model gate written down: RTX 3060 ShareGPT replay =8 against =1.
2026-10-26 19:02:48 +01:00
Niko1221 a3b7d9de3f docs/AMD_HIP.md: a hand-built engine has no hipBLASLt tuning table unless it is pointed at one (#4034)
The Build section said nothing about it. gemm.cu (create_hipblaslt_state, own_tuning_dir) takes STRATA_HIPBLASLT_TUNING, else the file
<engine folder>/tuning/<arch>-hipblaslt-<version>.txt, else plain hipBLAS. The report measured 826 against 1,043 tok/s at 8K prompts on a
gfx1151 (one run per arm, stated as such) and the log line that tells whether a table was taken. The extra log line the issue suggests for the
no-table case is a small engine change and is not part of this commit.
2026-10-26 19:01:50 +01:00
Niko1221 c966d7ad89 setup: Maxwell gets a warning with the build recipe, not an env switch (#3218)
The first #3218 commit made setup.py admit a Maxwell (sm_52) card only
when a tester set STRATA_EXPERIMENTAL_SM52=1, and then routed it to the
CUDA 12 engine. That engine holds no sm_52 code, and users never set
environment variables to get a fix.

sm60_card() is back to Pascal and Volta (6.0 to 7.0). A PC whose only
NVIDIA cards are Maxwell now gets a warning (maxwell_note) that says no
ready-made engine holds code for it, gives the source-build recipe
(-DSTRATA_EXPERIMENTAL_SM60=ON -DCMAKE_CUDA_ARCHITECTURES=52, CUDA 12.x)
and the one reporter's result, and points to docs/OLDER_GPUS.md. The card
table line says Maxwell runs only from a source build. A Maxwell card
beside a newer card prints nothing new. Naming the card (--gpu, --cuda
12) or any env var does not admit it. The CMake floor from the first
commit is unchanged (opt-in build only).

Tests: tools.test_setup_older_gpus 23 tests OK (a Maxwell-only PC through
setup.main() with and without the old switches installs nothing and
prints the recipe; a Maxwell card beside an RTX prints no note).
Not covered: no engine was built for sm_52 by this commit; the P1004
compile is recorded in the notes.
2026-10-26 19:01:27 +01:00
JoShMiQueL c0845b7fc5 bench: tools/strata_bench.py reads the CUDA UMD version, the storage, the engine's build toolkit and GPU power and PCIe detail (#3250, #2111)
The report written by BENCH.bat and bench.sh had fields missing or wrong: nvidia-smi's 'CUDA UMD Version:' (current drivers) was not read, the value
was the newest CUDA the driver supports but was labelled like the engine's toolkit, storage always said 'not measured', and Windows 11 showed as
Windows 10. Now the report says 'CUDA (driver supports up to): X; engine built for CUDA Y' (BUILD.json beside the engine), names the storage device
(Get-PhysicalDisk, system_profiler, lsblk -e 7 to skip snap loop devices), adds a 'GPU detail (after the run, near idle)' line with power limit and draw,
PCIe generation and width, pstate and temperature, and recognises Windows 11. The standard report (strata bench run, strata_cli/bench_facts.py) already
read the UMD version, storage and power limit; this brings the legacy tool to the same. Ported from PR #3250 by JoShMiQueL with its tests.
2026-10-26 19:01:17 +01:00
Norm 7783b1c18e docs: a crashed engine start names itself, and a setup run again keeps the installed size (#3243)
Two 0.1.44.1 changes no page told. TROUBLESHOOTING.md gets the start error 'the engine process ended: ...' (serve/server.py start_exit_note: the signal or
Windows exit code, and for an illegal instruction the AVX, AVX2, FMA and F16C flags the CPU lacks). SOURCE_INSTALL.md says a setup run again
pre-selects the installed size, marked '(installed)', and --yes takes it (setup.py size_tag, installed_size). Written by hand from PR #3243 by
Buddychek, whose hunks no longer fit the rewritten pages; each sentence checked against the two functions.
2026-10-26 19:00:14 +01:00
Niko1221 f217b23bd3 prefill: the copy-wait dedupe skips a wait only when both slots were released in the current walk and the ring is longer than one group (#4005)
Review fix on the #4005 port. STRATA_PREFILL_COPY_WAIT_DEDUPE=1 skipped the copy stream's wait on a slot's release
event when the previous entry had waited on the same event. Two cases break that:
- At idx == ring the previous entry (ring - 1) is still in the first round, so its wait was on the event of an earlier
  walk. The same slot id can be this walk's first flush, which re-records the event after that wait; skipping the wait for
  slot 0 then lets the next copy overwrite a slot that the flush's gather has not finished reading.
- With a ring of one MMQ group (16, the smallest ring) a flush of the whole ring re-records the event right after the
  previous slot's wait.
The rule is now copy_wait_dedupe_ok() in include/strata/prefill/copy_events.hpp: idx > ring, ring > group, same event.

copy_events_test models two to three walks with random group sizes and random issuer/consumer interleaving and checks that
a refill always waits on a record at or after the flush that released its slot. The same model with the PR's original rule
(idx >= ring) finds the stale-event case, so the test can fail. The default path is unchanged (dedupe off).

Tests: copy_events_test passes (z440, g++ -std=c++20). Not covered: a GPU run.
2026-10-26 18:59:50 +01:00
Niko1221 cea3ed40db serve: a slot's shorter checkpoints come along at a give-back only when they are prefixes of the tokens copied back (#2526)
Review fix on the #2526 port. At an admission from a slot the port copies every checkpoint of the slot that is not
longer than the copied-back length into the main session's chain. Such a checkpoint is valid there only when its tokens
are a prefix of what was copied back (its K/V cells are then the ones the main session holds) and it has no pictures
(the main session's pictures are cleared at a give-back). The PR did not check either; a checkpoint left over from a
branch the slot no longer holds would sit in the main chain with cells that are not its own.

Both trees (CUDA/HIP and SYCL) now test the prefix and the empty picture list per checkpoint. The checkpoint the
request continues from is copied as before. Nothing changes when every checkpoint is a prefix, which is the normal case.

Tests: compiles on CUDA and SYCL (see the build lines of the branch). Not covered: no model run; the gate is written down.
2026-10-26 18:59:50 +01:00
Yuu 05c1dabf1a docs: the xe expert tier has no SSD fallback under the wrapper default, and a native pack needs --spec 2 or more (#2153, #1796)
docs/INTEL.md and docs/INTEL_ARC.md described an SSD expert tier that strata-sycl.sh's default (STRATA_VERIFY_NO_HOST=1) does not have: every
expert must be in VRAM or in the pinned RAM mirror, and the engine refuses to start otherwise (sycl/src/program/generate.cpp, 'REFUSED: ... are
neither in VRAM nor mirrored'). A native (IQ) pack refuses --spec 0 and 1 (sycl/src/program/generate.cpp and src/program/generate.cpp). Ported by hand
from PR #2153 by nekomario28 onto the rewritten pages, each sentence checked against those two messages. The hunk for docs/TROUBLESHOOTING.md is left
out: the entry it changed is no longer on that page.
2026-10-26 18:59:27 +01:00
oscarlius e21a1872f7 docs/BATCHING.md: the cache settings that let a long solo request move into a batch slot without reading its prompt again (#3334)
Adds a minimal handoff configuration (--prompt-cache 1, --prompt-cache-every 0, --prompt-cache-root 0, --conversation-cache-mib 0), what each
option does, that it costs RAM, that it does not make cold prompts prefill in parallel, and how to look for the handoff in the engine log, /metrics
and /v1/status. Ported from PR #3334 by oscarlius (continuing #3035). The four options exist in src/program/generate.cpp and the o.prompt_cache > 0
gate matches the text; the log evidence was not reproduced here (no model run).
2026-10-26 18:57:53 +01:00
stuchapin909 6c09a1929c CI: build the Metal engine on macos-26 (#3870, #3824)
metal/src/mtl_rt.mm names MTLGPUFamilyMetal4 (macOS 26 SDK) and MTLCompileOptions.mathMode (macOS 15 SDK) behind @available, a run-time check;
the names must still exist in the SDK the engine is compiled against, so the metal-link job failed on the macOS 14 SDK on every pull request that
touched src/, include/ or metal/. The job now runs on macos-26 (the runner-images repository lists a macos-26 image). Ported from PR #3870
by stuchapin909. A GitHub Actions run cannot be made here: the author ran this workflow on his fork; the first run after the merge proves it.
#3824 changes the same line with macos-15 and an Xcode 26.0.1 path pin, and is not taken.
2026-10-26 18:57:30 +01:00
Heinz N. Gies 5bc1e24696 kq_cpu_bench: i-quant pairs no longer crash, and Q2_0 is timed on the CPU pool's path (#3871)
make_blob quantized its random test rows with ggml-cpu's type traits from_float, which the i-quants do not have: an IQ3_S pair called a null
pointer (an access violation on Windows, SIGSEGV on Linux) before the first timing. It now uses ggml_quantize_chunk, with a flat importance
matrix where the type needs one. A Q2_0 gate/up or down is timed as pool.cpp runs it (activations as an ActQ, Strata's own Q2_0 rows, SwiGLU
in between) instead of with ggml-cpu's Q2_0 dot, and the pair's header line says so. Other formats are timed as before. Ported from PR #3871 by
Licenser.

Built on this PC (MSVC, CPU only) and run: iq3_s/q2_0 0.531, iq2_xs/q2_0 0.336 and q2_0/q2_0 0.214 ms per expert per thread (--mb 512, 1 thread).
2026-10-26 18:57:13 +01:00
Highlander 0fbbf068eb nvfp4 local pack: read the checkpoint with cached sequential reads, not a process-lifetime np.memmap (#3377, #3378)
Checkpoint.raw mapped every shard for the life of the process. On Windows each faulted page of the ~64 GB of expert bytes is charged to the
process working set, so the pack step looked like a 64 GB RAM demand and was killed by free-RAM watchers on a PC with a model loaded (nothing
leaked: it all came back at once). raw() now reads the tensor's byte range into a fresh buffer through one cached file handle per shard; the
pages go to the system file cache, which counts as available. The author measured the same bytes, 1.2x faster, flat working set, and a
byte-identical pack. raw() is no longer thread-safe (this file uses no threads). Ported from PR #3378 by jerry78424 (the fix for his issue #3377)
with its two tests; I ran the tests (13 pass); the full 102 GB pack was not run.
2026-10-26 18:56:54 +01:00
Keyan Hu 409f2466ad tools/gguf_reader.py: bound the header reads and report malformed metadata as ValueError (#2288, #1919)
A GGUF header with an unknown metadata type or an impossible length raised KeyError or OverflowError inside the reader, and setup, which expects
a ValueError, printed a traceback instead of its diagnostic naming the broken shard. Strings, scalars, arrays and tensor dimensions now share
one bounded read; counts are checked against the bytes left in the file before anything is constructed; type ids are checked before their values
are decoded; nested arrays are reported as unsupported; general.alignment must be a positive multiple of 8. The file name is added once to the
message. Ported from PR #2288 by KeyanHu-git with its tests (tools/test_gguf_reader_malformed.py).

Checked against the old reader on the 18 GGUF v3 vocabulary files of the pinned llama.cpp: same metadata, tensors and data start.
2026-10-26 18:55:57 +01:00
Jakub Luwierski 11237f831f serve: Chat Completions tool results keep the order of the calls they answer (#3395)
When one assistant message made several tool calls and the client returned the results in another order, the call ids were dropped before the
prompt was rendered, so the model saw anonymous results in the wrong order. The ids are now kept while the request is normalized; a run of
results is reordered to the call order only when the call ids and result ids are non-empty unique strings with matching sets. Anything else
(missing, duplicate, unknown or incomplete ids) keeps the received order, and the ids never reach the template. Ported from PR #3395 by
j-luwierski with its seven tests (serve/test_chat_tool_results.py); the hunk was merged by hand beside require_messages.
2026-10-26 18:54:34 +01:00
rodi4egandkzhv bdecbffc6b native_mmvq: sm_89 (Ada) MMVQ-IL table from the cells two cards agree on (#3453)
Problem: an Ada card took the shared MMVQ-IL table, which has no Q3_K row, so
Q3_K shapes ran the plain kernel and the other types used a table measured on
sm_86 and sm_120.

Change: kIlRows_89 (2-4 columns) and kIl1Rows_89 (one column) are registered
for compute capability 8.9. The 2-4 column table keeps only the cells where the
RTX 4070 run (rodi4eg) and the RTX 4080 run (kzhv) gave the same rows count and
sets the rest to 0, the plain kernel; the 4070 and 4080 disagree in 13 of 15
cells of the unmerged tables. The one-column row is Q3_K only, which both cards
gave. The issue's 5-8 column table is not taken: the 4080's changed in 10-14 of
20 entries between two runs of the same card, so there is no kIlRows58_89. The
bench output files of the PR are not ported.

Every choice in a table is bitwise native_mmvq's output (kernel row split only),
so answers do not change; this is speed only. No end-to-end decode change was
seen on a 4090 or a 4080 in the issue, so the gain is the microbenchmark's.
This does change which kernel an sm_89 card runs by default, so the model gate
(identity on sm_89) is written down in phaseA-tests/gh027/gates/3453-sm89-il.sh.

Tests: mmvq_il_parity (registered in CMakeLists.txt) now checks sm_89's table
against the two source runs (check_tables) instead of expecting the shared
table, and still checks bitwise parity of every case and rows choice on the GPU
it runs on (sm_86 on z440). docs/MMVQ_IL_TABLE.md updated.

Not covered: no sm_89 card here, so the table speed is the contributors'
numbers, not ours.

Co-authored-by: kzhv <[email protected]>
2026-10-26 18:53:58 +01:00
i-Vladimir-i 092c7d1ef4 serve: return 503 when the elastic image encoder cannot start (#2435)
When the elastic image encoder failed to restart, its exception left request preparation and the client got a closed connection. It now goes
through the EngineDied path: HTTP 503 with the reason, a later picture tries again, text requests and a later good start are unaffected. An idle
encoder stop is logged even when the cache shrink was refused, and a layer split gets a line saying its encoder VRAM has to be kept free with
--vram-reserve-mib. Ported from PR #2435 by i-Vladimir-i with its tests (chat, messages and responses, streamed and not, plus a real encoder
that exits with code 71 on Linux) and the docs/VRAM_ELASTIC.md text.
2026-10-26 18:51:44 +01:00
Niko1221 a8c1d4b7f8 verify: a window graph that does not fit frees the graphs of other window sizes instead of ending the engine (#3702)
On a layer split the later card is left with a few dozen MiB after loading, and each single-request window size is
captured the first time it is used (68, 64, 58, 52 and then 10 MiB free in the report); when one capture's
cudaGraphInstantiate ran out of memory, instantiate_evicting could only free graphs of batch slot layouts, a
single-request run has none, and "verify: instantiate: out of memory" ended the engine.

Now, after the batch layouts, a single-request window's capture frees the graph of the largest other window size (never the
one being captured or a prelaunched one; pick_window_victim) and instantiates again, and says so once: the freed size is
captured again when it is next used. capture_all keeps every size at once and is excluded. Only a failed instantiate does
anything new; with enough VRAM nothing changes.

window_evict_test (CPU, header-only) covers the choice. The real gate needs a model on a 2-GPU split with a thin later card
(listed in the triage file); not changed here: the later-stage reserve of the split search (kWindowMib), which is a default
placement change.
2026-10-26 18:51:29 +01:00
Vop3p a11f8b0aa5 prefill: opt-in sparse copied events and deduplicated slot waits in the streamed prompt walk (#4005)
Under WDDM every cudaEventRecord and cross-stream wait is a submission of its own (the author measured ~80 / ~45 us each
on an RX 6900 XT under Windows 11), and the streamed walk paid one record and one wait per expert although a group
gathers after one wait on its last copy. Two tester switches, threaded issuer only:
- STRATA_PREFILL_COPY_EVENT_EVERY=N (2..64): the copied event is recorded every N-th entry, before the issuer waits and on
  the last; the compute stream waits on the first recorded entry at or after the one it needs (the copy stream is in order).
- STRATA_PREFILL_COPY_WAIT_DEDUPE=1: the copy stream skips its wait on a slot's release event when the previous slot was
  released by the same event.
The same copies into the same slots: the same bytes.

Default path (both unset): read line by line. The issuer records copied[slot] and waits on used[used_of[slot]] exactly as
before (the counters are bookkeeping, cover() is a no-op). Added to the PR so this holds by construction: wait_copied takes
the slot the old code used and, without the sparse switch, waits on copied[slot] at all three consumer sites (the PR derived
the slot from the entry number). The record/wait selection is now three pure functions in
include/strata/prefill/copy_events.hpp, with a host test (copy_events_test, registered in CMakeLists.txt) that models the
issuer and consumer with a full ring and checks that every wait names a live recorded entry at or after the one needed.

Tests: copy_events_test passes (z440, g++ -std=c++20). Not yet: a CUDA build of prefill.cpp in this tree (queued), no GPU
run. Not covered: the Windows HIP / WDDM speed claim and NVIDIA (the gate is written down). Needs the author's numbers
repeated before any default talk.
2026-10-26 18:50:44 +01:00
PurrSec eba3106c0e serve: support IPv6 bind addresses (#3212)
--host :: failed before the model loaded because Server always opened an IPv4 socket. Server now selects AF_INET6 when the bind address contains a
colon; the ready line, the 'from other devices' lines and the browser address bracket an IPv6 address. The bind is still the person's choice and
no default changes. Ported from PR #3212 by purrsec with its tests (serve/test_security.py, including a real :: bind). setup.py's --host help and
docs/DETAILS.md say :: is all IPv6 interfaces; the README hunks of the PR (they name strata launch --host, not checked) are left out.
2026-10-26 18:50:11 +01:00
Niko1221 61ede87e1b Maxwell sm_52 floor for the opt-in old-card build (#3218)
A reporter built Strata for two Quadro M6000 (sm_52) by lowering two
floors and saw about 22 tokens a second with Qwen3.8-Flash-Next IQ1.
CMakeLists.txt: the experimental build (-DSTRATA_EXPERIMENTAL_SM60=ON)
now accepts sm_52 and prints a warning for it (Maxwell, one reporter, no
native half precision, CUDA 12.x). Without the flag the build still
stops at sm_75 as before, and the architectures a build uses are not
changed.

setup.py: sm60_card() admits a 5.2 card only when the tester sets
STRATA_EXPERIMENTAL_SM52=1, because the ready-made CUDA 12 engine holds
no sm_52 code and a Maxwell card must not be routed to it by default.
With the variable set, setup warns about the Maxwell card instead of
refusing. Default: a Maxwell card is "not supported" exactly as before.

docs/OLDER_GPUS.md: a Maxwell row and a short section, with the build
command and a request for an answer-text check.

Tests: tools/test_setup_older_gpus.py (20 tests, new Maxwell cases for
default, opt-in and a newer card present). Not covered: no model run, and
the answer text on a Maxwell card is unchecked.
2026-10-26 18:48:59 +01:00
Keyan Hu a1d2fd7ed1 setup: use the published size when the download length probes fail (#2287)
When HEAD and the one-byte range probe both gave no length, download() took an early EOF as a finished file even though the repository pin (or
ModelScope's published size) names the size. The published size now stands in for the length, so the existing resume, retry and parallel code
publishes the file only after it has all its bytes; a size that disagrees with the pin is still refused before the transfer and the SHA-256
check is unchanged. Ported from PR #2287 by KeyanHu-git with its tests (tools/test_setup_download_length.py).
2026-10-26 18:48:49 +01:00
Günter 560cc9fbbc setup: --gguf-dir files for Qwen3.6 no longer need a .done mark made by setup (#3606)
install_own_path treated a file as present only when <file>.done existed, and that mark is written by setup's own download or by verify_sha256,
which a file from another tool (LM Studio) never reached: setup said it was missing and asked for the full download size in free disk. With
--gguf-dir an existing file and its mmproj go to verify_sha256(own=True), which checks size and SHA-256, leaves a mismatching file alone and
writes the mark. A file that is not in the folder is still reported missing. Ported from PR #3606 by ai-bits with its two tests.
2026-10-26 18:48:14 +01:00
Niko1221 96b6068c32 Fix the follow-up of #4014 (a variable name already used in Chat.tsx) and two of my tests (#3254 iterator, #3656 raw paths) 2026-10-26 18:48:09 +01:00
Xiaomi2008 3100ca05ed expert cache: STRATA_VRAM_FREE=device reads the whole card's free VRAM through NVML, Windows + NVIDIA, opt-in (#1911)
Under WDDM cudaMemGetInfo answers for the calling process: memory another process holds on the card (the image encoder,
a browser, another engine) is not in its free figure. The reporter measured, on an RTX 5090, 30,994 MiB free from CUDA
where NVML read 24,348 with 6 GiB held elsewhere, so --expert-cache auto sized the cache too large and shrank it by
guesses.

device_free_bytes() now takes min(CUDA's figure, NVML's figure for the whole card) when STRATA_VRAM_FREE=device is set.
nvml.dll is loaded from the system directory at first use (LoadLibraryExW with LOAD_LIBRARY_SEARCH_SYSTEM32), the card
is matched by its PCI bus id, and a missing nvml.dll, a missing export or a failed nvmlInit leaves CUDA's figure with
one line on stderr. The serve check of "VRAM free with everything loaded" uses the same function. The value can only get
smaller, so the switch can only shrink the expert cache.

Unset (the default), nothing changes: the NVML code is compiled only for _WIN32 and not for HIP, Metal or the no-VMM
build, and the env variable is read once. The serve check's min() is the same value elsewhere (on a unified-memory
system device_free_bytes() is never below cudaMemGetInfo's figure). Added to the PR: the NVML block is also left out
of a SYCL build (SYCL_LANGUAGE_VERSION), whose CUDA names are a compatibility layer. docs/DETAILS.md: one row in the
measured-and-left-off table instead of the PR's long paragraph.

Tests: builds on Linux CUDA (the Windows-only block is skipped), SYCL (P1006) and on Windows MSVC (owner PC build of
the final tree). Not covered: no run with a model; the Windows gate is written down.
2026-10-26 18:47:52 +01:00
Jakub Luwierski a332cc95db setup: an explicit --rope-scale must be finite and at least 1 (#3393)
resolve_rope accepted nan, inf, zero and values below 1 and handed them to the engine. It now stops with a message that names --rope-scale and the
valid range, before any configuration is written; valid factors and the automatic scaling are unchanged. Ported from PR #3393 by j-luwierski
(follow-up to #2950) with its tests; the --help text gets the range.
2026-10-26 18:47:46 +01:00
Avicennasis 3a3f35c64a tests: cuda_spill_guard fails the CUDA test build when a Strata kernel spills or keeps a per-thread array in local memory (#2683, #2682)
tools/check_cuda_spills.py reads the resource usage the compiler wrote into the kernel libraries (cuobjdump
--dump-resource-usage: STACK and LOCAL per entry function and per sm_) and fails on any Strata kernel outside its ALLOWED
table, which carries the measured allowance of the kernels that hold such an array on purpose; ggml's vendored kernels
(no namespace) are skipped and counted. ctest cuda_spill_guard runs it on eight libraries, skips (77) without cuobjdump or
a demangler, CUDA builds only (HIP has no cuobjdump); check_cuda_spills_unit runs tools/test_check_cuda_spills.py
(stand-in demanglers are sh scripts, so not on Windows).

Changes against the PR: (1) a demangler that exits 0 but prints a blank name used to make the kernel's key empty, which
reads as vendored, so a spilling Strata kernel was skipped; that is now "no verdict" (77) with a test. (2) The unit test is
registered. (3) The ALLOWED table was re-measured on f-027-int: sm_120 with CUDA 13.0 on Windows (MSVC) and sm_86 with the
CUDA 13.0 objects built on Linux (cuobjdump 12.9); allowances are only raised (new DFlash2 gemm8_kernel and
rope_table_kernel, native_gu_swiglu_kernel on sm_86, dequant_kernel / expdec_gu_kernel / prompt_attn_i8v3_kernel /
native_gu_swiglu_kernel on sm_120), none lowered or dropped. Result: 0 violations on both architectures; a test shows the
same object fails once one allowance is lowered.

Ported from PR #2683 (the tracking issue is #2682).
2026-10-26 18:47:35 +01:00
Niko1221 2cc6ea169b serve: opt-in LaTeX-safe JSON recovery for structured output (#2893)
A json_object or json_schema answer that holds LaTeX such as \alpha is
rejected (502) because \a, \g and the like are not JSON escapes. New
opt-in: "repair_latex_escapes": true in response_format (and in
text.format on /v1/responses). Only after the strict parse failed, the
server doubles each backslash that does not start a valid JSON escape and
parses once more with the same duplicate-key and NaN rules, then checks
the schema as before, and prints one log line. Without the option, and
for any answer that parses on the first try, nothing changes.

Escapes that are valid JSON (\b \f \n \r \t \/ \" \ and a well-formed
\uXXXX) are left alone, because they cannot be told from LaTeX: a \beta
next to a \alpha comes back with a backspace. Truncated output, arrays,
NaN and duplicate keys still fail.

Tests: three new tests in serve/test_structured.py (default still 502,
opt-in 200 and stream, failures stay failures, helper cases, Responses
format passthrough). serve/test_structured.py 13 tests OK.
2026-10-26 18:47:25 +01:00
SanghunYun 2251053b42 serve: the serial decode timing line shows the adaptive waits in its sum (#1864)
With STRATA_DECODE_TIMING=1 the serial line listed `verify + commit/emit + draft` after `ms/window =`, while the adaptive
wait before verify and the join after the draft were timed but printed after the sum, so the named parts did not add up
to ms/window. The serial line now reads `adapt wait + verify + commit/emit + draft + adapt join + other`, where `other` is
the decode time minus the named parts (not a timed step). The pipelined decode keeps its old line, byte for byte, and
nothing is printed with the timing off or with no windows. Diagnostic only: no decode path changes.

docs/DETAILS.md: a short paragraph on the new line (the hunk was rebased by hand onto the current file).

Tests: compiles on CUDA (the format string was counted against its 22 arguments); no parser in the repo reads
`ms/window = verify`. Not covered: no run of the line itself (it needs a decode).
2026-10-26 18:46:55 +01:00
l33tm4st3r c471588f3e bench: tools/strata_bench.py (BENCH.bat, bench.sh) names an AMD card on ROCm 7+ too (#3630)
The quick speed check has its own GPU probe, separate from strata_cli/bench_facts.py, and tried nvidia-smi then rocm-smi only. On a Radeon with
amd-smi alone it wrote the server's vendor stub and 'GPU driver: not read'. The same amd-smi probe as #3519 is added in the shape this file
uses, and a bare vendor name from the server no longer replaces a card the probe named. Ported from PR #3630 by l33tm4st3r.

The three new tests are POSIX-only (symlinks, /sys); run on z440 (Linux, Python 3.12): OK. tools/test_strata_bench.py on Windows: OK.
2026-10-26 18:46:33 +01:00
hotschmoe 01e404c140 serve: an idle batch slot's decoded conversation is parked before an admission overwrites it (#2526)
A batch slot holds the reply it decoded. The main session was parked with the conversation's prompt only (the
admission's copy), so when the next admission or a yielding read overwrote the slot, the reply lived nowhere: the
conversation's next turn found its prompt-read state and read the whole reply again (the author measured 255 of a
256-token reply re-read on four 200K chats over two slots).

What changed:
- park_current is split into park_source (one budget / RAM-admission / snapshot path) and two callers: park_current
  (the main session, output line and byte accounting as before) and the new park_slot (an idle slot's ids, checkpoints,
  sessions and draft K/V). The ParkSource passes what a park captures; the main session's retained K/V (#1503
  take_reuse_for) is taken only for the main session, never for a slot.
- park_slot runs before copy_to_slot at a slot admission and before a BYIELD read overwrites a slot. It does nothing
  when the live main session or a parked conversation already holds that state (ConversationCache::holds /
  conversation_covers, new, with unit tests), for a slot with pictures, or without --batch-mtp when the drafter is on.
- When a slot is admitted into the main session its checkpoints come along (the shorter ones too), so the chain keeps its
  branch points and the parked copies a turn back are superseded (#342) instead of taking an entry each.
- Same refactor in sycl/src/program/generate.cpp.

Changed from the PR: parking is an optimization like a checkpoint (#3350). The PR ended the request (ERR) or the batch
(batch_fatal) when park_slot's snapshot failed; here a failed snapshot, a failed device sync and a host allocation
that runs out are logged, the slot is overwritten as before and the request goes on. The slot's extra checkpoints that
come along at an admission are copied under a bad_alloc guard (the one the request continues from is copied as before).

Tests: conversation_cache_test (new holds() cases and the four-conversation replay at policy level) and
conversation_split_failure_test. Not covered: no model run (the batch path itself needs one); the gate is written down.
2026-10-26 18:46:05 +01:00
Niko1221 9c2d35d978 serve: /v1/responses accepts max_output_tokens of 0 and -1 (#4036)
The Responses validator still refused anything below 1 after the chat
path made max_tokens of -1 and 0 mean no cap (documented in
docs/DETAILS.md). It now accepts -1 and 0 as no cap; values below -1,
floats, booleans and strings stay a 400. Nothing else changes: the
server already passes max_output_tokens or 0 to prepare(), which gives
the answer the room left in the context.

Tests: OverHttp.test_max_output_tokens_zero_and_minus_one_mean_no_cap
(stream and non-stream, status completed) and
test_max_output_tokens_bad_values_are_400. The first fails without the
change. serve/test_responses.py 43 tests OK.
2026-10-26 18:45:04 +01:00
Niko1221 407f659763 docs: GR_V3 and PROMPT_ATTN_IMMA defaults in DETAILS.md and the 0.2.7 release-notes draft 2026-10-26 18:45:02 +01:00
ruibeikaa f213f78bae mtp: the drafter's Q8_0 draft head and projections on the Pascal GEMV, opt-in STRATA_MTP_Q8_SM60=1 (#3751)
Problem: with STRATA_Q8_SM60=1 the main model's Q8_0 matrices run on the GP100
Q8_0 kernel, but the MTP drafter's own Q8_0 draft head and projections kept the
generic kernels.

Change: with STRATA_MTP_Q8_SM60=1 (on top of STRATA_Q8_SM60=1) the drafter packs
its Q8_0 projections as packed copies, as the main model does (the prompt path's
draft_kv still reads the GGUF layout, so a copy is kept), and packs the draft
head's subset in place when every column count fits the kernel's shared-memory
planes and the data is 16-byte aligned (otherwise a packed copy). The in-place
pack writes the GGUF blocks back if the device write fails. Release unregisters
the entry without a cudaFree when the data was packed in place. New
native_q8_0_sm60_pack_quiet and native_q8_0_sm60_pack_in_place; HIP returns
false; metal/src/engine/stubs_0145.cpp gets matching stubs that answer
"nothing packed".

Replies do not depend on the drafts (they are verified); only the draft rounding
changes, because the Pascal kernel sums in another order.

Default path: mtp_q8_sm60() is false unless the env is set, so the packing calls
are skipped. The destructor now walks the tensor list once at teardown, not in a
hot loop. The SYCL tree has its own sycl/src/core/mtp.cpp, so it is not touched.

Tests: compile-only on sm_60 (CUDA 12.9) on P1004. Model gate (identical replies
with the flag on and off, decode speed on a P100) is written down in
phaseA-tests/gh027/gates/3751-mtp-q8-sm60.sh.

Not covered: no GPU run here; the contributor measured +4.4% on four GP100 dies
with all experts resident, one run per arm, not on a single streamed P100.
2026-10-26 18:43:53 +01:00
Dzmitry Senchankau 41188ec734 calibrate: an engine that dies mid-measurement drops only its candidate (#3317)
#1337 made a failed engine start in a later calibration step a candidate
that loses. An engine that started and then exited during its own rate
measurement still failed the whole run. measure_candidate() now treats
both the same: the candidate is dropped and recorded in
report["failed_starts"], the earlier results stay. It is used for the CPU
worker step, the adaptive expert tier step and the AMD shared-expert
fork. An operator cancel (CalibrationCancelled) is still raised. The first
start (steps 1 to 3) still fails the run.

src/core/verify.cpp: when adapt_gate_wait times out after 20 s, the
message now says which layer was reached (atomic and copy signal), that
the copy stream is not drained by the release, and dumps the
STRATA_VERIFY_TRACE breadcrumbs when that is on. Diagnostics only, #2188
stays open. The watchdog label is set only once a wait actually starts,
so a gate that is already open costs nothing extra (changed from the
PR, which set it on every call).

Tests: tools/test_calibrate.py 33 tests OK (3 new death tests).
Not covered: engine compile of verify.cpp is checked in the build step.
2026-10-26 18:43:46 +01:00
ruibeikaa 6a6d58ad03 iq_kernels: GP100 decode down projection staged in shared memory, opt-in STRATA_DOWN_WS=1 (#3867)
Problem: on a GP100 die (cc 6.0, no dp4a, no tensor cores) the verify window's
routed down projection for the native packs' IQ4_NL and Q2_0 rows (n_ff 640)
is latency bound. native_down_multi_kernel reads each weight as 2-byte loads,
one item at a time, straight from global memory.

Change: native_down_ws_kernel copies a block's rows (IQ4_NL 32, Q2_0 64, 11,520
bytes) into shared memory with 16-byte loads, then runs the same row dots on the
copy: same sub-warp, lane, items, float expressions and order. Only the integer
sum inside an item changes form (vmad byte selectors, prmt table lookups), the
same int8 x int8 products, so every output is native_down_multi_kernel's (expert
mode 0) bit for bit.

Gating: compiled only in the CUDA build with STRATA_EXPERIMENTAL_SM60 (not HIP,
not Metal). At run time only with STRATA_DOWN_WS=1 on a device of compute
capability exactly 6.0, IQ4_NL / Q2_0 down rows at n_ff 640, and not with
STRATA_OLD_IQ_MMVQ or STRATA_NO_SUB16_GU. Unset, the only change on the default
path is one cached boolean test per native_expert_grouped call.

docs/OLDER_GPUS.md lists the switch. The contributor's microbench numbers (one
PH402 die, Swift 1.5 IQ3_S, one-token window: IQ4_NL 72.6 us mode 8 and 54.5 us
mode 0 become 42.3 us; Q2_0 53.5 / 65.4 us become 38.6 us) were measured on a
four-GP100 rig, not here.

Tests: compile-only on sm_60 (CUDA 12.9) on P1004. The contributor's bitwise
check (native_expert_bench hash against modes 0 and 8) is not run here: it needs
a GP100 GPU run and the model's layers. The gate script is
phaseA-tests/gh027/gates/3867-down-ws.sh.

Not covered: no end-to-end measurement on a single P100 (16 GB cannot hold the
47 GB pack resident).
2026-10-26 18:43:32 +01:00
l33tm4st3r 2ac3482144 bench: the standard report names an AMD card and its ROCm runtime when only amd-smi exists (#3519)
ROCm 7 and newer ship amd-smi and no rocm-smi, and the amdgpu product_name is empty on gfx1201, so a Radeon AI PRO R9700 run was filed
under 'unknown' with 'runtime: not measured'. bench_facts reads the market name from amd-smi static --asic, the VRAM from the amdgpu sysfs
file of the PCI address amd-smi list reports, the ROCm version from the install (or amd-smi -h), and keeps the probe's card name when the
server only says 'AMD Radeon'. Ported from PR #3519 by l33tm4st3r with its tests and the BENCHMARK_PROTOCOL.md text.

Changed while porting: the amd-smi branch also took macOS (os.name is posix there), which would have hidden the Apple chip; it now skips
darwin, and a test keeps the chip.
2026-10-26 18:41:53 +01:00
Niko1221 9e3fd0f73a App: the DeepSeek card names the free disk the set needs, and the version line says "{N} GB RAM + VRAM, plus about {disk} GB of disk"
The "This computer" card said only "Fits fully in memory." for DeepSeek, with no word of the ~265 GB of disk. For a DeepSeek version it now says "Needs about {N} GB
of free disk space. What does not fit in memory is read from your SSD." (the spill line stays beside it when part of the model is read from the SSD), and the
selected version's line reads "{N} GB RAM + VRAM, plus about {disk} GB of disk". {disk} is not hard-coded: the catalog (tools/strata_mcp.py) gives each set
its `disk_gb`, the set's whole download (model files, n-gram table, drafter, metadata) plus the pack and room to work (dsv41_fit.DISK_HEADROOM_GB), so 2.00 and
2.25 differ. Same card and row markup, no design change; every other model keeps its card. Both sentences are fx80 and fx81 in the eight fix catalogs.
The Terminal's DeepSeek description makes no "fits fully in memory" claim (its lines already state the download and the free disk), so it is unchanged.
2026-10-26 18:41:17 +01:00
Niko1221 54d9363c97 CUDA Plan opt-ins become defaults where measured: STRATA_GR_V3 on a discrete 12.0 card above 13 GiB, STRATA_PROMPT_ATTN_IMMA on CUDA sm_80+ (env =0 turns each off); DeepSeek Faster mode (5 experts) by default on 2+ RDNA4 cards that hold half the experts; the route-prior/GR_V3/IMMA rules are pure functions with tests; gr_parity and fused_gr_bench pin STRATA_GR_V3=0 for their bitwise contract 2026-10-26 18:41:09 +01:00
Tsury fb966cff37 pack_gguf: accept older DeepSeek GGUF metadata names (#3955)
Older converters write the Engram scalars as engram.key_length and
engram.pad_id, and omit the sparse-attention geometry. The packer now
reads the old names only when the canonical engram.head_dim and
engram.pad_token_id are absent, and a new --config-json option fills the
omitted geometry from the source model's config.json. A value that
disagrees with metadata already in the GGUF stops the pack with a clear
message. The weights are never rewritten and a set that already has the
canonical names packs exactly as before.

Tests: LegacyModelMetadata (2 new, 5 in the file) in
tests/terminal/test_pack_gguf_engram.py.

Not covered: the author reports the older Q2_K set still needs dense
Q2/Q3 and Engram Q2_K tensor support in the engine; this does not add it.
2026-10-26 18:40:30 +01:00
Patrick Schaub a0fc88abd6 serve: timings on /v1/responses (#3291)
The final response.completed / response.incomplete event and a
non-streaming /v1/responses reply now carry the same top-level timings
object that /v1/chat/completions has. The server already put timings in
the done dict; the Responses assembler dropped them.

Tests: UsageAndStatus.test_responses (stream and non-stream).
Unset timings leave the event unchanged; response.failed has none.
2026-10-26 18:40:04 +01:00
Norm 5b14fe23f7 tests: three suite gaps a live PC showed (#3236)
test_iq_pack imports without llama.cpp's gguf-py and skips the classes that build GGUF fixtures with a named reason; the MCP discovery tests use
a free port, not whatever Strata serves on 8080; the six conversation-cache tests add the repo root to sys.path so they run from the repo root
as the suite does. Ported from PR #3236 by Buddychek (tests only, no shipped code).
2026-10-26 18:38:42 +01:00
Niko1221 f06bfc30e7 docs/TROUBLESHOOTING.md: the Windows AMD driver-reset entry is back
The 0.2.5 rewrite of the page dropped the 'Windows AMD: the driver resets' entry (v0.2.4.2 had it), although setup.py still tells a Windows gfx12
card owner to read it and tools/test_setup_amd.py (TdrPointer) checks for it, so that test failed on main. Restored from v0.2.4.2 with the
log and config locations worded for a source checkout.
2026-10-26 18:37:19 +01:00
Norm e6f598c0a2 docs: STRATA_VCVARS, /metrics/prometheus behind the API key, and the Monitor's prompt speed (#3244)
Three 0.1.42 changes no page told. Ported by hand from PR #3244 by Buddychek (the PR conflicted with the rewritten pages), each sentence
checked against the code: /metrics/prometheus needs the API key like /metrics (serve/server.py _authorized), the prompt-speed row is
(prompt_tokens - reused) / prompt_ms (apps/host/server/monitor-dashboard.mjs), STRATA_VCVARS is read by setup.find_vcvars. The compiler
text now says Visual Studio 2022 (2019 is refused since 0.2.5, #3566) and went to docs/SOURCE_INSTALL.md.
2026-10-26 18:36:43 +01:00
Niko1221 3478382eee conversation copies: the private copy streams and events are handed back to a pool when a thread ends (#3350, #3470)
conversation_detail::private_stream() and order_after_queued() kept one non-blocking stream and one event per device in
thread_local storage and never destroyed them. The layer split's later stages run each prompt chunk on a fresh
std::async thread (STRATA_HANDOFF_PIPE off, the default), and a checkpoint part is saved on that thread, so every
periodic checkpoint of every long cold prompt left a stream and an event behind on each later card: a few MiB of VRAM
per long prompt that never came back (#3470: the later card's free VRAM drops 2-6 MiB per long cold prompt, no plateau),
until cudaStreamCreate failed with "out of memory" (#3350: "conversation snapshot running-state copy: out of memory",
engine exit) on a card that was already nearly full.

A thread now hands its handles to a process-wide pool when it ends and the next thread takes them, so the number of
streams and events is the number of threads copying at once. Same stream, same flags, same contract; only the lifetime
changed. copy_streams_created() / copy_events_created() count what the process made.

conversation_copy_streams_test (CUDA, no model, skips without a device): 200 short-lived threads in a row create at most
one stream and one event; ten rounds of six concurrent threads at most six more; the same on a second device.
2026-10-26 18:36:32 +01:00
Niko1221 f62589fdbe serve: a checkpoint whose copy runs out of memory is skipped, it no longer ends the engine (#3350)
A conversation checkpoint is an optimization, like parking, but the writer treated a failed copy as fatal: "conversation
snapshot running-state copy: out of memory" (cudaErrorMemoryAllocation from the snapshot copy: the private copy stream's
first creation or the driver's staging on a nearly full card, not host RAM) made the prefill fail and the engine exit,
21 times in 4 h for one reporter and 16 in 34 h for another, each exit a reload and the in-flight requests lost.

Now, after a failed save, checkpoint_failure_is_survivable() clears the non-sticky allocation error and synchronizes the
device: when that is clean the checkpoint is skipped with a line on stderr and the request goes on (a layer split's stage
part is left empty, which already means "no checkpoint at this length"; the message-boundary / turn / root checkpoints
after a prompt chunk do the same). A device that is unwell (a sticky error, a GPU that stopped answering) still ends the
request exactly as before. Same edits in sycl/src/program/generate.cpp.

STRATA_TEST_COPY_FAIL_AT=N (tests only, like STRATA_TEST_SLOW_COPY_US): the N-th snapshot copy of the process fails with
cudaErrorMemoryAllocation, so the case can be made without a full card. Unset, nothing changes.
Not changed: "verify: instantiate: out of memory" (graph capture), which the reporter saw once.
2026-10-26 18:36:32 +01:00
Niko1221 164619c5a8 DeepSeek: the 4 x R9700 share-based defaults and the drafter cut apply when every device is RDNA4 (gfx1200/gfx1201), not to every HIP build (Impl::rdna4) 2026-10-26 18:36:12 +01:00
pschaub 3cd33d504c cmake: a CUDA build with no architecture option is for sm_120, and native builds the W4A4 unit for sm_120a (#4029, #4023)
CMakeLists.txt tested "NOT DEFINED CMAKE_CUDA_ARCHITECTURES" after enable_language(CUDA). Since CMake 3.18 that call
fills an unset variable with nvcc's default (sm_75 for CUDA 13), so the sm_120 default never applied: a hand build with
no option was an sm_75 build. The default is now set before enable_language, unless the user gave
-DCMAKE_CUDA_ARCHITECTURES or the CUDAARCHS environment variable, and the configure step prints a warning that says what
to pass for another card (a recommendation, not a refusal).

With -DCMAKE_CUDA_ARCHITECTURES=native, the one NVFP4 W4A4 unit (mmq_nvfp4_w4a4.cu) stayed on plain `native`, where
ptxas refuses the FP4 MMA on an RTX 50. `native` is now replaced by CMAKE_CUDA_ARCHITECTURES_NATIVE (the detected
architectures) before the 12x -> 12xa rewrite, as a list, so a mixed PC works too. docs/SOURCE_INSTALL.md says what
`native` and the other values build.

Release and setup builds pass their architectures explicitly and are unchanged. Configure matrix, CUDA 12.9 on Linux:
none -> 120 (+ the warning), CUDAARCHS=86 -> 86, 89, "75;86;120" (w4a4 120a only for the 12x entry), native -> detected.

Ported from PR #4029.
2026-10-26 18:35:58 +01:00
sergqwer 4a7e373da3 engine: --adapt-decay outside 0..1 falls back to 0.7 with a line on stderr (#1284)
--adapt-decay took any number. Above 1 the adaptive tier's usage counts grow without end (inf after a few hundred
adaptations) and the swap gains turn NaN, which breaks the sort that orders the swaps; a negative value flips the counts.
The value is now checked where it is parsed (include/strata/program/adapt_decay.hpp, also in sycl/): a finite value in 0..1
is used exactly as before (0 and 1 included, the range the calibration profile already allows); anything else, NaN and inf
too, runs with the default 0.7 and says so on stderr. A warning instead of the PR's exit 2: a risky pick is reported,
the engine still starts.

adapt_decay_test (25 checks) covers the range, the fallback and finite counts after 100000 adaptations.

Ported from PR #1284 (the check; the refusal became a fallback).
2026-10-26 18:35:58 +01:00
Niko1221 bb62a048b8 updater: a failed test start keeps the runtime records, not only the engine log tail (#3654)
The sandbox of a test start was deleted at once, so the PID records that show why a start failed were lost; only the last 64 KiB of the engine
log stayed. A failed start now also keeps run.json (without keys), the host, child and ready JSON records of the runtime (tokens cut to their
first 8 characters) and the host and server logs in .updates/logs/test-start-<version>-failed. The last three failed starts are kept.
Best effort, never hides the failure. Tests: tests/terminal/test_updater.py (FailedStartRecords).
2026-10-26 18:35:15 +01:00
Niko1221 484c689337 DeepSeek: the 4 x R9700 share-based defaults (small chunk for multi-card tp runs, whole-chunk slab, small staging ring, pinned copy at share >= 0.5) apply to HIP builds only; the staging ring takes a hip flag; tests for both branches 2026-10-26 18:33:37 +01:00
Niko1221 3b64abf5e0 DeepSeek drafter cumulative cut 0.5 by default on gfx1200/gfx1201 only (Model::gpus_are_gfx12); Strix Halo, other AMD and CUDA keep 0 until measured 2026-10-26 18:29:34 +01:00
Niko1221 c8596ec391 Merge f-027-marlin (DeepSeek on 4x R9700: staging ring from the room, copy pin auto, chunk 1024 where the cards hold most experts, whole-chunk attention slab, F16 gate/up activations, drafter cumulative cut on HIP, diagnostics). Conflicts resolved: attention slab = one rule (attn_slab_rows: Marlin's share >= 0.5 branch plus the automatic growth, unified memory and DS4_ATTN_SLAB=0/N untouched); staging ring = one function (staging_ring_mb: 16 MB where the cards hold the experts, Marlin's small per-device ring at 0.5..0.82, main's streamed 400..1200 MB on unified memory and under half); DS4_COPY_PIN auto = huge-page arena OR cards hold >= half; DS4_FAST_ATTN stays gfx12-only; chunk 1024 for multi-card tp runs too, but not for --threads on or one card on the default tp 2026-10-26 18:28:51 +01:00
Niko1221 bf994ca414 setup: a build folder configured with another generator is configured again instead of stopping cmake (#4038)
The manual build in docs/INSTALL.md uses CMake's default generator (Unix Makefiles on Linux) in build/, and setup --build always asks for
Ninja, so cmake stopped with 'Does not match the generator used previously'. cmake_build now reads CMAKE_GENERATOR from the folder's
CMakeCache.txt; when it is not Ninja it says so in one line and removes the cache and CMakeFiles (no other file). A Ninja folder or a new
folder is untouched. Tests: tools/test_setup_build_dir.py.
2026-10-26 18:28:23 +01:00
Niko1221 590c69a578 cmake/ggml_tq_overlay_test.cmake sets CMP0057 NEW so IN_LIST works under CMake 3.x (#3412)
A cmake -P script has no cmake_minimum_required, so CMake 3.x used the OLD behaviour and stopped at 'if given arguments ... IN_LIST: Unknown
arguments' (ctest ggml_tq_overlay with CMake 3.31). Checked with pip cmake 3.31.6 and 4.4.3 against the pinned llama.cpp: both print
'ggml_tq_overlay_test: ok'. A static test keeps the policy before the first IN_LIST in every cmake/*_test.cmake.
2026-10-26 18:27:27 +01:00
Niko1221 7507b47525 sycl/tools/build.sh exits with the build's own status (#2990)
The script ended with a grep pipeline, so a failed compile exited 0 (and, before the ocloc check was added at the end, a good build exited 1).
A build.sh && next-step chain then ran after a failure. The status of cmake --build is kept and the script exits with it. The test runs the real
script with a stub cmake on PATH (good build: 0, failed compile: 1; the old script exits 0 on the failed compile).
2026-10-26 18:26:40 +01:00
Niko1221 1be7eb77c1 tests: the setup golden compares the platform's vision binary names, and test_nvfp4_local_pack skips without pytest (#3325)
tools/test_setup_golden normalized strata.exe but not strata-vision.exe, so all 25 baseline entries failed on Linux (48 subtests). VEXE
and HVEXE are mapped to <VEXE> and <HVEXE> like EXE, and the baseline records <VEXE>. test_nvfp4_local_pack imports pytest, which is not in
requirements.txt; under python -m unittest discover the module is now skipped with a reason instead of failing to import.
2026-10-26 18:25:25 +01:00
ctr-137 800730103c tools/strata_bench.py: the redactor replaces a folder before the short user name inside it (#3619, #3326)
Substitutions were ordered by the length of their regular expression, and a user name wrapped in word bounds outranked the Strata or home
folder that contains it, so a report kept /home/<user>/Strata instead of <strata>. They are now ordered by the text they replace. Ported from
PR #3619 by ctr-137 with its test (Linux and Windows layouts).
2026-10-26 18:24:35 +01:00
Niko1221 9a0462a011 Chat: sending a message still scrolls to it when the reader had scrolled up (#4014) 2026-10-26 18:24:01 +01:00
Patrick Schaub bc939b6303 docs/SAFETY.md: strata launch updates an installed DeepSeek engine package by itself (#4028, #4022)
The page said that starting a model never downloads and that nothing checks for updates in the background. Since 0.2.4.2 strata launch with a
DeepSeek model checks the signed feed (10 s at most, silent without a connection) and replaces an older installed DeepSeek engine package.
Ported from PR #4028 by pschaub (wording from issue #4022), checked against strata_cli/launch/runner.py upgrade_dsv41_engine.
2026-10-26 18:23:38 +01:00
xxxliqu1dxxx 3cda075b51 setup (AMD, Linux): the engine probes get the engine's lib_dirs on LD_LIBRARY_PATH (#3305)
engine_env put the ROCm library folders first on PATH, which only Windows' loader reads. On AMD Linux strata --families exited 127 on
libhipblas.so.3, so setup falsely said the engine does not run Qwen3.6. On Linux and elsewhere the folders now go first on LD_LIBRARY_PATH
(an existing value is kept); Windows keeps PATH. Ported from PR #3305 by xxxliqu1dxxx (resubmission of #2914), with its tests.
2026-10-26 18:23:22 +01:00
cuijinkuo 7c9b6f3a28 MCP chat: opt-in mcp.image_results caches PNG/JPEG/WebP tool results beside the config, shows them in the web chat with an authenticated download and passes them to the vision reader (PR #2473 by cuijinkuo) 2026-10-26 18:22:31 +01:00
Demetree Kallergis cc2c628cbe setup (Mac): say when an update was asked for a model that is not installed (#3848)
metal/setup_mac.py returned right after the engine step on --update while the model questions had been asked, so choosing a model the Mac did
not have ended with no download and no word. The update now names the missing model (or image encoder) and points at ./setup.sh --setup; an
update for a model that is installed prints what it printed before. Ported from PR #3848 by demetree, with its two tests (mocked; not run on a Mac).
2026-10-26 18:22:30 +01:00
MegaRushX 3e81173d63 chat.py: fenced code blocks in streamed answers are coloured on a terminal; prose is untouched (PR #1895 by MegaRushX) 2026-10-26 18:21:36 +01:00
medking82 df6928400f Agent: the staged Harness keeps output room in small contexts (8K): a guarded, hash-checked correction of pi-ai 1.1.0's safety margin below 64K (PR #3684 by medking82) 2026-10-26 18:21:20 +01:00
medking82 1e4ae3ddbf Models page: a failed preparation is shown only on the model it belongs to, not on the next model you select (PR #3361 by medking82) 2026-10-26 18:21:08 +01:00
kossumandMIDHAT NASHAR e2e99ae54a Windows: the token default DACL grants generic rights, so a child process can still be opened after adopt_user_defaults (#3428, #3399)
adopt_user_defaults() set the token's default DACL with FA (FILE_ALL_ACCESS). That default also applies to new process objects, and the file
mask has no process query or set-information right, so a child could not be reopened by PID: the HighQoS helper timed out and
procs.start_token could not record the child. GA lets Windows map the mask to the object being created; the principals are unchanged.

Ported from PR #3428 by kossum (QoS reopen test). Test from PR #3399 by midhatn (a live child is identified and recorded) added; both fail
without the change and pass with it.

Co-authored-by: MIDHAT NASHAR <[email protected]>
2026-10-26 18:21:07 +01:00
medking82 f499922379 App: the composer's effort (Off, Low, Medium, High, Max) reaches saved API models that support reasoning, with the OpenRouter mapping; unknown support keeps the provider default (PR #3363 by medking82) 2026-10-26 18:20:52 +01:00
jubeater 14bef6bb46 setup: hand component engines to the server for validation (#4027, #4026)
_start() checked cfg['exe'] with Path.exists(), so a run config whose engine is component:strata-engine was refused as a missing file
before the server could resolve and verify it. Ported from PR #4027 by jubeater. A component reference is no longer tested as a file; model
files and literal engine paths are checked as before. ensure_engine_for, the Windows HIP runtime repair and the HIP architecture lookup
skip the source-build engine folder for a component. Eight tests (tools/test_setup_component_start.py).
2026-10-26 18:20:42 +01:00
zapabob c7833932a1 App: a model load validates and parses the same captured text of its settings file, so a file changed between two reads cannot start with unchecked settings (PR #3954 by zapabob) 2026-10-26 18:20:17 +01:00
SuperDistance d2c69e6d4b tools/strata_bench.py: read CMakeLists.txt as UTF-8, not with the locale's code page (#4011, #2792)
Ported from PR #4011 by SuperDistance. project_version() read CMakeLists.txt with Path.read_text() and no encoding, which raised
UnicodeDecodeError on Windows code pages such as cp936. The test makes read_text default to each of 14 Windows code pages. The same crash was
reported in #3235 and #3414.
2026-10-26 18:20:17 +01:00
BColsey 19bfcb33f0 Tests: the Terminal fixtures write settings where the platform keeps them and no longer swap os.name or assume a Windows ACL on Linux (PR #3782 by BColsey) 2026-10-26 18:19:55 +01:00
Niko1221 9e2c0b6eb0 docker: test that both Dockerfiles take the source commit and the source-build commands pass it (#3644) 2026-10-26 18:19:55 +01:00
jackbapa 24c3f24ae1 Tests: the ACL checks compare trustees through the SDDL alias, so an account that Windows writes as LA passes (PR #3406 by jackbapa) 2026-10-26 18:19:43 +01:00
Niko1221 7c4c20799c Residency-biased routing default is CUDA-discrete-only: read from the device properties once in Verifier::init (never first inside a graph capture), off in HIP builds and on integrated GPUs, a line in the log when it is on; the wide HC up read never defaults on in a HIP build (gfx12 reports compute capability 12.0 too) 2026-10-26 18:19:39 +01:00
Niko1221 68ba321acc launch: a model folder is found whatever its letter case, so a downloaded model is not offered again on a case-sensitive disk (#3996); a log path the person set in the config is kept (#3656); --dspark takes on|off|auto|none and the word reaches the engine (#3959) 2026-10-26 18:19:38 +01:00
Niko1221 a8755042bd Delete a model: every part of a multi-part GGUF goes with it (the run config names only the first), and a part another model names stays (#3716) 2026-10-26 18:19:38 +01:00
Roberto Jose Etcheverry Romero 255861c442 docker: the root Dockerfile passes the source commit to the Terminal stage (#3644)
.git is not in the build context, so package_terminal.py had no commit to put in the archive and the documented source build stopped.
Ported from PR #3644 by robertoetcheverryr: ARG STRATA_COMMIT / ENV STRATA_BUILD_COMMIT as in docker/Dockerfile, and the two 'From source'
commands in docs/DOCKER.md pass --build-arg STRATA_COMMIT=$(git rev-parse HEAD).

Added: .dockerignore leaves out the folders docker/entrypoint.sh creates in a mounted data/ (install, state, home, Strata-data). A dangling
data/install/third_party/llama.cpp link in a clone that had been mounted made the Terminal stage fail with a copytree error (seen in the
review thread of #3644). Not built with docker here (no docker on this PC).
2026-10-26 18:19:35 +01:00
i-Vladimir-i e0469a585f MCP chat: at the tool-round limit the model gets one final reply with no tools from the results already found, instead of the turn ending without an answer (PR #3741 by i-Vladimir-i) 2026-10-26 18:19:27 +01:00
天空Blond d398f35325 docker: the build context carries the nested tools/ scripts the Terminal bundle reads (#3974)
docker/build.sh staged only top-level tools/*.py, so the Terminal stage stopped at 'installer tool missing: tools/dsv41/tune.py'.
Ported from PR #3974 by hurui200320: the pathspec is tools/**/*.py, the no-.git fallback finds nested files, and the context test reads
INSTALLER_TOOLS and INSTALLER_SCRIPTS from packaging/package_terminal.py. Fixes the failure that images built from v0.2.4.1 and v0.2.4.2 had.
2026-10-26 18:19:09 +01:00
toel1234 dc1fd367d3 procs: an engine adopted by systemd --user is found as an orphan, and ending it waits on the start token until the process is really gone (PR #3254 by toel1234) 2026-10-26 18:19:05 +01:00
MaKla89 eb1c40038c App sharing: the shared and tunnel address answers a browser's CORS preflight and sends Access-Control headers when an API key is set; without a key nothing changes (PR #3853 by MaKla89, #3208) 2026-10-26 18:18:15 +01:00
thisisspartan cd72accac3 Web page: a request that timed out in the queue shows a Timeout badge in Recent requests (PR #3991 by thisisspartan) 2026-10-26 18:17:49 +01:00
Niko1221 4abccbb360 Merge f-027-cuda (CUDA speed work: wide HC up read on sm_120, residency-biased routing on small cards, STRATA_TRACE, release-notes draft) 2026-10-26 18:16:40 +01:00
Niko1221 f92cfa4374 Merge f-027-win (main 2d07d1c0b with the DeepSeek slab scoping 32 MB unified / 8 MB discrete plus automatic growth, the dq kernel arch choice, and the HIP-only guard for the gfx11 WMMA gemm call) 2026-10-26 18:16:34 +01:00
justindressler b3db5a1c83 serve: the lifeline identity is the child's start time, the sampler survives a bad source (#3590, #3407)
The portable lifeline watcher built a child's identity from `ps -o lstart=,stat=` and kept the whole line, so the token
changed with the process state (R while it starts, S asleep). The watcher compared its own reading with the token taken
at start, decided it was not the child it was told about and exited without ending it (test_lifeline "the child outlived
a killed server (watcher)", #3407). The identity is now the start time alone: field 22 of /proc/<pid>/stat on Linux
(a zombie has none), `ps -o lstart=` elsewhere; strata_control/procs.start_token drops the state column the same way.
A launch that is not a real Popen is not watched.

Telemetry._loop no longer ends on the first failing sample (a machine whose GPU or psutil source is unreadable): it
prints one line and keeps sampling, and NoLeakedSampler checks the thread itself instead of counting threads.

New test: the token is the same while the child runs (R) and sleeps (S); on the old code it fails on Linux
('...R' != '...S').

Ported from PR #3590.
2026-10-26 18:15:01 +01:00
Niko1221 9a1e768388 App: the source-checkout server is started without an empty STRATA_API_KEY (#3851); a card taken out of the PC no longer decides the engine make, and a PC with two makes accepts a build for either (#3892) 2026-10-26 18:14:26 +01:00
Niko1221 085dcded73 Monitor: the host reads the graphics card every 3 s only while a model runs or the Monitor is open, every 30 s otherwise, and the PCIe sample only while a model runs (#4066) 2026-10-26 18:12:22 +01:00
Righteous-AI fd8d37e93d serve: accept Codex spellings of a namespace tool (#3972)
A declared namespace.member tool (Codex MCP: mcp__filesystem.write_file) was accepted only as namespace__member (#3556).
Qwen3.8-Flash-Next also writes namespace=member (the <function= tag's equals sign ends up in the name), namespace_member
(one underscore dropped) and the bare member. Each is now an alias of the declared tool when exactly one declared tool
produces it and no declared name is the same; a name that matches nothing stays text (#3075), two namespaces that share
a member are not guessed, and a declared flat tool always wins. Requests without namespaced tools are unchanged.

serve/test_tool_name_alias.py (10 tests, 5 of the first 9 fail without the change; the tenth pins that a flat tool named
like the underscore alias wins), docs/DETAILS.md names the spellings.

Ported from PR #3972.
2026-10-26 18:12:20 +01:00
Niko1221 89a68d8829 Chat: the transcript follows the answer only while the reader has not left; a wheel notch, finger or key moving up lets go at once, in the app chat and the served web page (#4014) 2026-10-26 18:11:11 +01:00
Stefanos Anastasiou f8ff3420d9 conversation cache: never park one conversation with another's retained K/V (#1503)
With --batch and conversation parking, a restore keeps the restored K/V (ConversationCache::retain) so the next park of
the main session copies only what changed. The reuse tracked a token count (unchanged_tokens), not whose tokens, and
copy_from_slot (a batch slot giving its conversation back to the main session) replaced the main session's K/V without
dropping the retained image. The next park reused the earlier conversation's K/V for the new one's prefix: a restore of
the new conversation then answered from the other one's history.

retain() now records the ids of the conversation the K/V belongs to; a park calls take_reuse_for(live), which reuses only
the prefix live shares with those ids (nothing when they share none or the ids are unknown), and one stderr line is
printed whenever a reuse is cut. copy_from_slot drops the retained image before it replaces the main session's K/V.
Same edits in sycl/src/program/generate.cpp. The guard only cuts reuse: at most one full copy of a parked conversation's
K/V, the saved bytes are otherwise identical; no flag, no default change.

Ported from PR #1503 onto the current tree (the first-stage ids sit after the stages field so the existing aggregate
initialisers keep their meaning). conversation_cache_test gains the cross-conversation cases of the PR and the logged
sequence (restore P, a slot hands Q to the main session, park Q): another conversation never gets the K/V, a diverging
one only the shared prefix on every stage, a slot hand-back drops it, unknown identity is never reused. Removing the
ids limit makes the first case fail.
2026-10-26 18:09:44 +01:00
Niko1221 f681b9ad62 Manual mode: the look-only shell allowlist takes only options known to look (#3975)
A command on the allowlist used to run without a card with any option except -delete/-exec/-execdir/-fprint/-fls/-ok. Write modes (sort -o,
tree -o, uniq IN OUT), program-execution modes (sort --compress-program, rg --pre, git grep -O), date -s and tail -f got through, and so did
git config -e, git branch --set-upstream-to and git remote ... forms.

Each command now has a spec of the options and operands that only look (short and long, spelled in full); every other option, abbreviation
or attached form asks. git subcommands have the same specs, a branch/tag name is taken only with a list option, git config only reads with
--get. date takes only +FORMAT operands, uniq at most one operand, hostname none. less and more leave the list. Quoted texts are checked as
operands too. `du -sh` no longer trips the 'sh' word check.

Tests: apps/host/tests/gate-readonly.test.mjs (deny and allow cases for options, operands, spellings, quoting; every listed command asks for
an unknown option).
2026-10-26 18:08:17 +01:00
Niko1221andyehya 6c01f85add qwen35: a penalty for a history token outside the logit row no longer indexes past the row (#3911)
sample() applied the repeat / frequency / presence penalties to lg[id] for every token id of the last penalty_last_n
tokens of the history. A row shorter than the vocabulary (the MTP draft head picks among the first N token ids,
--draft-vocab N) made a history token with id >= N an out-of-bounds read and write on the host. A token that is not in
the row takes no penalty; the loop skips ids < 0 or >= lg.size(). A full-vocabulary row is bit-identical.

The loop and Sampling move to include/strata/qwen35/sampling.hpp (header-only) so a CPU unit test can run them:
src/qwen35/sampling_test.cpp (qwen35_sampling_test) checks nothing is written past a short row, that in-row tokens
take the penalty, and that a full row matches the unguarded loop bit for bit over 108 penalty/window mixes. Without the
guard the test dies with an access violation.

Co-authored-by: yehya <[email protected]>
2026-10-26 18:07:29 +01:00
Niko1221 3a604e0855 Merge f-027-kva (docs/KVA.md: the shipped opt-in's numbers on the R9700 and the P100, and the decode trade on fast GPUs) 2026-10-26 17:52:48 +01:00
Niko1221 6f45b24864 docs/KVA.md: the R9700 and P100 numbers for the shipped opt-in (split 36), and the trade stated plainly 2026-10-26 17:52:39 +01:00
Niko1221 063b3c5dc8 DeepSeek engine: the RDNA3 WMMA dense GEMM call in gemm() is HIP-only (wmma_gemm.cu is part of the AMD builds; main's 0eea4a3d5 called strata_pf_gemm_f16_ld unconditionally, so a CUDA build failed to link: unresolved external) 2026-10-26 17:48:36 +01:00
Niko1221 20c2f42bf1 ds4_iq.cu: the NUL character literal in dq_use_old_kernel had been written as a raw NUL byte by the merge script (nvcc: quoted string should contain at least one character); it is a backslash-zero escape again 2026-10-26 17:36:16 +01:00
Niko1221 2e1fad783d host test offline-start: point STRATA_COMPONENTS_DIR at the test's own folder so the host cannot find an installed Strata Control. On a PC with the app installed the update check went through the installed Control (no host fetch, 45 s), so the test saw 0 outside calls and failed; it failed the same on v0.2.6. With the fix both tests pass in 2-12 s. 2026-10-26 17:35:48 +01:00
Niko1221 afc45665fc Merge main 2d07d1c0b (Strix Halo DeepSeek work: fd33cb6ee, 0eea4a3d5, 3133160ed, 2d07d1c0b) into f-027-win. The attention slab: the base is STRATA_DS4_A_MB megabytes of q, 32 MB on a unified-memory GPU (where main measured it) and 8 MB on a discrete card (the 0.2.6 slab); the automatic growth (attn_slab_rows) stays on top for discrete cards with spare VRAM, DS4_ATTN_SLAB still overrides, both env names work. The dense dequantizer: dq_rows_kernel2 stays the default; a gfx11 device (gcnArchName) defaults to main's improved row-per-lane kernel, DS4_DQ_OLD=1/0 forces either. 2026-10-26 17:29:43 +01:00
Niko1221 63b6e0e834 Merge branch 'main' into f-027-cuda 2026-10-26 17:19:22 +01:00
Niko1221 259532bf2e prompt: the RAM store's post-prompt preload is off by default - it costs a 4K prompt 4.7 s and the decode that follows gains nothing
cache_settle()'s store preload (DS4_SETTLE_STORE) read 2,243 hot experts into free store slots after every chunked prompt: 4,713 ms of a 4K prompt.  Measured on the box (code fixture, prompt + 256 tokens, interleaved, 3 runs a side): with it on the decode is 13.9/14.4 s, with it off 14.2 s - no benefit - while the 4K prompt goes 31.3 -> 26.6 s (130.89 -> 153.78 tok/s).  The swap loop above it is what the decode uses (it holds the CPU lane at ~5-7 k experts a run against ~14 k without), so that keeps its budget; DS4_SETTLE_STORE=1 restores the preload.
2026-10-26 17:07:53 +01:00
Niko1221 9c059bade1 Merge f-027-kva (KVA ships off in 0.2.7: STRATA_KVA unset is the exact 0.2.6 engine; STRATA_KVA=1 approximates layers 36-47 with a 2048-token exact tail, the configuration that kept agent loops and long extraction at the exact level; geometry gate; draft fixes; opt-in STRATA_KVA_SKIP=moe) 2026-10-26 17:02:35 +01:00
Niko1221 9c8c90391d Merge f-027-hip (gfx12: decode miss split starts at pcie_frac 0.15, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55; DeepSeek matrix-core prompt attention on by default on gfx1200/gfx1201, DS4_FAST_ATTN=0 off; huge-page copy arena and its pinned DMA on only when a reserved pool holds the copy; KVA draft fixes) 2026-10-26 17:02:13 +01:00
Niko1221 bda3005082 KVA, when switched on, approximates layers 36-47 with a 2048-token exact tail (the measured configuration); STRATA_KVA_SPLIT / STRATA_KVA_SUFFIX override
On the RTX 5070 (--agent-long, 22 tasks) split 36 kept agents at 6/10 (exact 4 and 7) and extraction at 585/606 facts (exact
585) for +11% prompt speed at 14K and +18% at 30K; the split-24 default passed 0/10 agent runs and found 556 facts, a tail of
8192 with split 24 found 574. KVA stays off unless STRATA_KVA=1. docs/KVA.md gives the numbers and the reasons; kva_test
checks the opt-in's split 36 / tail 2048 and the overrides. The geometry gate is unchanged.
2026-10-26 17:02:12 +01:00
Niko1221 aeb70de349 Merge f-027-kvaoff (KVA is off by default: the app's and the Terminal's 'Faster long prompts' switch turns it on, shown for Qwen3.8-Flash-Next models only) 2026-10-26 17:01:54 +01:00
Niko1221 76cd7d7970 KVA: the one-time note says how many positions were approximated, not where the tail starts (chunks round the frontier down) 2026-10-26 16:55:44 +01:00
Niko1221 25a62e0fb3 KVA: keep the geometry gate - a model whose shape KVA was not measured on reads exactly; STRATA_KVA_ANY_MODEL=1 is the research override
The app toggle will write STRATA_KVA=1 for Qwen models in general; the split layer, the tail and the measured quality apply to
Qwen3.8-Flash-Next's shape only (48 layers, 512 experts). Any other shape (Qwen3.6-35B-A3B, ...) now logs "KVA configured but
NOT applied: this model's shape (L layers, E experts) is not one KVA was measured on" and reads exactly; the routine reasons
(short prompt, continuation) still come first. STRATA_KVA_ANY_MODEL=1 allows it, with one warning. kva_test and docs/KVA.md
cover both.
2026-10-26 16:40:59 +01:00
Niko1221 72b7696810 Faster long prompts is shown for Qwen3.8-Flash-Next models only (app and Terminal)
KVA was measured on the Flash-Next shape (48 layers, 512 experts) and the engine declines any other shape with a log reason, so the row would do nothing
for Qwen3.6-35B-A3B. Setup writes "arch" in a run config only for the families with an engine path of their own (Qwen3.6-35B-A3B: qwen35moe; DeepSeek:
deepseek41); every Qwen3.8-Flash-Next file (the original, Swift, Coder, Unsloth, Gyro, NVFP4) has none. The row is therefore shown for a model without an
"arch" on CUDA or HIP, in the app's Native engine settings and in the Terminal's Advanced table. The Radeon RX 9000 and DeepSeek rows are unchanged.
Tests: a Qwen3.6 config gets no KVA row (CUDA and HIP), the six Flash-Next family names do, and through the host fixture a model with an arch has no switch row.
2026-10-26 16:38:15 +01:00
Niko1221 e3450e4552 KVA ships OFF in 0.2.7: STRATA_KVA unset is the exact 0.2.6 engine, STRATA_KVA=1 turns it on with an 8192-token exact tail
The quality runs found lost records and invented rows in long extraction tasks and defects in agent loops with a ~10K
system prompt when most of the prompt was approximated. Opt-in now: unset (or 0) nothing runs, the draft layer included.
Switched on, the tail is 8192 tokens (STRATA_KVA_SUFFIX overrides), the safest configuration measured. A model other than
Qwen3.8-Flash-Next's geometry gets a one-line warning instead of a refusal. Kept: the untouched draft layer (cell floor
behind STRATA_KVA_DRAFT_FLOOR=1), the decline reasons, STRATA_KVA_SKIP=moe, the tests. docs/KVA.md says why it is off and
how to turn it on; kva_test checks unset = off and STRATA_KVA=1 = on with the 8192 tail.
2026-10-26 16:36:31 +01:00
Niko1221 9760cbca4b App: "Faster long prompts" (KVA) is Off by default in the model's Expert settings; On writes STRATA_KVA=1, Off removes it
The Speed row now follows the shared table's default: Faster long prompts reads "Off (recommended)" / "On" and its new sentence is "Reads very long prompts
faster by approximating part of the work. Off by default: answers about long documents can miss or mix up details." (replaces fx73; "Off (recommended)"
is fx79; translated in zh, ja, de, es, ru, pt, ko). The other two rows are unchanged ("On (recommended)" / "Off"). A STRATA_KVA=0 stored by an earlier
0.2.7 test build shows as Off. Still shown only where KVA applies (Qwen on CUDA or HIP). tests/speed-switches.test.mjs updated: the default launch has
no KVA entry, On writes STRATA_KVA=1, Off removes it, the old 0 shows as Off, and the other two switches are tested as before.
2026-10-26 16:34:01 +01:00
Niko1221 ae03ef6323 Terminal and the shared speed-switch table: Faster long prompts (Qwen KVA) is OFF by default, STRATA_KVA=1 turns it on
Owner decision (26 October, conditional on the last KVA variant results): KVA ships off. strata_control/speed_switches.py now gives every switch a default
and the environment it writes outside that default: KVA is default off and writes STRATA_KVA=1; the Radeon RX 9000 and DeepSeek switches stay on by
default and write their off entries as before. A launch in the default carries no entry. A file that holds STRATA_KVA=0 (the off switch of an earlier
0.2.7 test build) reads as off, which is what it is now; choosing Off cleans it up.
The Advanced table row says "off by default; set 1 to turn on" with the new sentence (`--set STRATA_KVA=1`; `off` or Del puts the default back).
tests/terminal/test_speed_switches.py covers the defaults, KVA on/off/old-zero, the rows and the comparison with the app's table.
2026-10-26 16:34:01 +01:00
Niko1221 3171291420 KVA: STRATA_KVA_SKIP=moe (opt-in) - the late layers of an approximated chunk run their whole attention half and skip only the MoE half
The default stays the state-only form (byte-identical). With STRATA_KVA_SKIP=moe the GDN recurrence (gate projection, output
normalization, output projection) and the QSA attention (query side, selection, attention, output projection) run in full
for layers >= split, their outputs go into the residual, and the loop leaves before the MoE half; the expert transfers of
those layers are still skipped. The KVA line says "mode moe-skip". Motivation: the extraction losses of the state-only form
come from late-layer K/V and state built from a stale residual; this keeps the attention halves' contribution in it.
kva_test covers the switch.
2026-10-26 16:18:49 +01:00
Niko1221 fd81df6972 DeepSeek engine (AMD): the matrix-core attention on by default for the chunked prompt
fast_attn_enabled() was opt-in (DS4_FAST_ATTN=1); with the prompt chunk at 1024 it is worth far more than when
it was gated: measured on 4x R9700, 2.25 bpw, 8K code prompts, three warm prompts a load, the same ids
(5705e1164a) --

  without it  1159 / 1180 / 1195 ppts   (median 1180)
  with it     1327 / 1354 / 1360 ppts   (median 1354, +14.7%)

and after this commit the un-enved default reads 1314 / 1357 / 1369 (median 1357), i.e. identical to setting the
flag, so a deployment no longer needs any environment at all.  The QK/PV sums are the matrix cores' order, which
the owner already approved for the prefill; the decode path keeps the scalar kernels.  DS4_FAST_ATTN=0 turns it
off (it is a no-op on CUDA, whose dispatch is not compiled).
2026-10-26 16:16:28 +01:00
Niko1221 879b958a3e docs: setup is expected to choose --ple-io ram on a RAM-rich Linux PC - say to check setup --plan rather than asserting it 2026-10-26 16:08:11 +01:00
Niko1221 be4450fdc3 Merge branch 'main' into f-027-cuda 2026-10-26 15:51:50 +01:00
Niko1221 67cfaf2ac6 prompt streaming: the staging ring sizes from the VRAM room (400 -> 1200 MB on Strix Halo) and the attention slab defaults to 32 MB - 4K prompt 27.3 -> 21.0 s
ring_mb_auto() returned a flat 400 MB on any box whose experts do not fit in VRAM, but on such a box the ring is exactly what lets a prompt queue its likely experts early.  400 -> 1200 MB measured on the box (settle off): the queue 3,200 -> 8,087 experts, the disk's effective rate 1.49 -> 2.44 GB/s, the compute's exposed read wait 11.4 -> 3.5 s, the 4K prompt 148.98 -> 190.98 tok/s; 16K 290.97 -> 315.01, 32K 316.92 -> 325.89.  Decode unmoved in an interleaved A/B (17.54/16.39 against 17.61 tok/s).  It is now a sixteenth of the VRAM room clamped to 400-1200 MB, so a smaller pool still loads; DS4_RING_MB still overrides.

c.A: the attention slab decides how many times each dense matrix is re-read a prompt.  8 -> 32 MB (STRATA_DS4_A_MB): 32K 325.89 -> 357.78 tok/s, 16K 315.01 -> 328.04, 8K +1.6%, 4K flat, decode unmoved (16.81 against 17.60 tok/s interleaved, noise-limited) for ~0.4 GB of VRAM.

Best measured prompt set with the two new defaults (DS4_SETTLE_MS=0;DS4_SETTLE_STORE=0, which is a prompt-only trade): 4K 195.18, 8K 258.47, 16K 328.04, 32K 357.78 tok/s.  With the settle work at its default a 4K prompt is 31.3 s = 130.89 tok/s (was 88.30).
2026-10-26 15:43:31 +01:00
Niko1221 bffb41ef47 Merge f-027-win (ctest registers ple_residency_test with the GPU-build tests) 2026-10-26 15:13:40 +01:00
Niko1221 3e2f45e2e1 ple_residency_test is registered with the GPU-build tests (it sat in the conversation-tests block, which is off by default, so ctest never ran it) 2026-10-26 15:12:23 +01:00
Niko1221 de6229fb17 Merge branch 'main' into f-027-cuda 2026-10-26 15:08:46 +01:00
Niko1221 1ee6060afb docs: the 12 GB card's prompt figures are withdrawn - a five-pair repeat read +254% and -20.5% where the first read +35% and +17%, and that box's prompt swings 218-466 tok/s between arms 2026-10-26 15:08:43 +01:00
Niko1221 cca3dd2e33 dense GEMM: the chunked prompt's gemm() prefers the engine's own RDNA3 WMMA kernel (src/prefill/wmma_gemm.cu, now linked) over hipBLAS - 4.97 vs 20-40 TFLOPS at our shapes
The dense GEMM was 11.1 s of a 4K prompt at hipBLAS's measured 4.97 TFLOPS (M 4096 N 2304 K 5120, the engine's call convention). A blocked WMMA kernel reaches 23.96 TFLOPS at that shape (128x128 tile, BK 32, 2-stage LDS, rocwmma; scripts/wmma2.cu) and the engine's own tuned kernel (BM 128 x BN 256, grouped tile order, raw wmma intrinsics) 20.8-20.9 at the wq_b shape and 38-41 at M 16384. gemm() now calls strata_pf_gemm_f16_ld with hipBLAS as the fallback (rows < 64, N < 512, K % 32, unaligned X: unchanged); STRATA_DENSE_PFGEMM=0 restores hipBLAS for an A/B. Same operands, the kernel's own k order (rounding-level).

Measured on the box (eng-028, 4E224E7F -> 467D7320, 4K prompt fixture p4k, chunk 16384): 46.4 -> 38.1 s, 88.30 -> 107.63 tok/s; with DS4_SETTLE_MS=0;DS4_SETTLE_STORE=0 27.5 s = 148.98 tok/s. 8K 134.66 -> 218.31, 16K 158.6 -> 290.97, 32K 178.9 -> 316.92 (settle off). Decode unchanged in an interleaved A/B (old 15.17/16.73, new 17.38/18.48 tok/s) with acceptance 151/164 = 92% (93/98/100/100/90 per position).

c.A (the attention slab, the reason each dense matrix is re-read 10-16x a prompt) takes STRATA_DS4_A_MB: 32 gives 107.63 -> 110.67, small because the expert reads now dominate.
2026-10-26 15:07:37 +01:00
Niko1221 4329c0d380 Merge f-027-ctx (DeepSeek-V4.1-Flash: 512K and 1M contexts in setup, the Terminal and the app up to the pack's own limit, with a 'not tested yet' note past 256K; the app now honours a chosen DeepSeek context; idea from Project Maya v1.1.28 #442 by @needmorevram) 2026-10-26 14:47:11 +01:00
Niko1221 f9fa567e1d Merge branch 'main' into f-027-cuda 2026-10-26 13:58:25 +01:00
Niko1221 6bc27ebef7 docs: the agent scenario across all three boxes 2026-10-26 13:34:58 +01:00
Niko1221 b8946f4c7b Tests: nobody gets a bigger DeepSeek context than before unless they choose it (8K stays the default in setup, the Terminal and the app; an installed model shows and loads the context it was prepared with) 2026-10-26 13:28:18 +01:00
Niko1221 c4c320a755 App: DeepSeek's context goes up to the pack's limit (512K, 1M), with a plain "not tested yet" note past 256K, in eight languages (Project Maya #442, @needmorevram)
The Models screen listed the shared sizes for DeepSeek but installed and loaded it at 8K whatever was chosen. Now the choice counts (8K stays the
default for a version that is not installed) and the list is DeepSeek's own: 8K to the limit the installed pack states in ds4.txt (max_position_embeddings,
read by the host as `contextMax`); a pack without it behaves as before (up to 256K). Past 256K the existing note slot under the model shows: "Not tested yet.
A context this long takes graphics memory from the model and reads long prompts slowly." (fx78 in the eight fix catalogs). Qwen's choices are unchanged.
src/lib/deepseek-context.ts is the app's copy of strata_control/dsv41_context.py; tests/terminal/test_dsv41_context.py compares the two.
2026-10-26 13:25:32 +01:00
Niko1221 33780a4955 DeepSeek-V4.1-Flash: 512K and 1M contexts in setup and the Terminal, as Project Maya v1.1.28 did (#442 by @needmorevram, MIT)
The model reads 1,048,576 positions natively. Setup and `strata launch` offered DeepSeek the shared list that stops at 512K and dressed anything past
256K in Qwen's "rope scaling" words. Now:
- strata_control/dsv41_context.py: the choices are 8K, 32K, 64K, 128K, 256K, 512K and 1M (labelled "512K" and "1M"), limited to the pack's own limit
  (max_position_embeddings in its ds4.txt) when it has one; a size past it (2M, or 1M on a pack that says 256K) is the one refusal; past 256K a plain
  note says it is not tested yet and what it costs.
- setup.py: --context is checked before any download and again against the pack once it is built; the install tool refuses the same sizes.
- Terminal: the context step lists DeepSeek's own sizes (and the note beside them), a typed size is checked, `--context 512k` / `1m` and 524288 / 1048576
  are accepted, plan and launch carry the note as a warning; Qwen's list and wording are unchanged.
- Engine: read, no change needed. strata-dsv41 takes --max-context as given (ds4_main.cpp) and sizes its buffers from it (ds4_model.cu); nothing clamps it
  below the pack's limit, and the rope settings come from the pack. A test keeps it that way.
- docs/DEEPSEEK41.md: the cost (graphics memory per position taken from the expert cache, slow reading of very long prompts) and that 512K / 1M are untested.
2026-10-26 13:25:32 +01:00
Niko1221 11b5a77fe9 docs: the agent scenario - two conversations nearly double throughput 2026-10-26 13:23:56 +01:00
Niko1221 f9907b6d9c docs: the routing default is +67% with two conversations in flight 2026-10-26 13:14:27 +01:00
Niko1221 d56dec89b5 Merge commit '2e6f88b7b' into f-027-hip 2026-10-26 13:04:40 +01:00
Niko1221 6c00851ac1 KVA: the draft layer is untouched by default (window 32768, cells built for approximated chunks, no floor); the cell floor is STRATA_KVA_DRAFT_FLOOR=1
Measured on a 5070 (IQ3_S) the cell floor cut decode after a 14K / 30K approximated prompt by 22 / 34 percent (drafts
accepted 100 -> 83 / 73 percent): a draft layer that reads only the exact tail loses its context. The approximated
chunks' draft cells are usable, so by default nothing about the draft layer changes: not the window, not the cells
built, not the attention. The floor machinery (MtpDrafter::set_cell_floor, window_ids_floor, KvaFloorRegistry) stays
behind STRATA_KVA_DRAFT_FLOOR=1 for research. kva_test now really covers window_start with floor 0, the registry and
wants_cell_floor.
2026-10-26 13:03:41 +01:00
Niko1221 2043b682af Merge f-027-win (DeepSeek: the chunked prompt's attention slab is automatic, up to 2048 rows only where the VRAM would not hold an expert; DS4_ATTN_SLAB=0 = the 0.2.6 rule) 2026-10-26 13:00:52 +01:00
Niko1221 e66d2758ed docs: the release notes carry the two-conversation batching gain 2026-10-26 12:46:27 +01:00
Niko1221 03056168d9 dense dequant: the chunked prompt's dq_rows_kernel packs each sub-block's 32 halves into four 16-byte stores and stages the warp's input through contiguous uint4 loads decoded at a relative index - the same values (bit-identical), 4K prefill 79.5 -> 88.52 tok/s, acceptance 97.4% 2026-10-26 12:21:18 +01:00
Niko1221 2d1db1ec6f Merge commit '9c282fbc5' into f-027-hip 2026-10-26 12:15:19 +01:00
Niko1221 7a34c60d61 KVA: leave the draft window alone; a per-session cell floor keeps the draft layer off approximated cells
The first default-on build set --mtp-window 2048 once per process whenever KVA was on, so short prompts (where KVA never
applies) lost draft context and decode speed. Now the draft layer keeps its window (32768 by default) and
MtpDrafter::set_cell_floor() names the first draft cell built from a real row: a KVA read raises it to the end of its
approximated chunks (they are neither built nor read), the draft attention starts at max(floor, pos - window)
(window_ids_floor, window_start.hpp), and KvaFloorRegistry gives every state a request continues from (live, checkpoint,
parked) the floor of its own tokens. A session KVA did not approximate has floor 0 and attends exactly as with
STRATA_KVA=0; with STRATA_KVA=0 or a process KVA is ruled out for the draft layer's code is 0.2.6's. The tail is 2048
whatever the window. Unit tests: window_start with floor 0 equals the plain window, the registry, wants_cell_floor.
2026-10-26 12:14:30 +01:00
Niko1221 50df0b95a0 Merge f-027-app (off switches for the three 0.2.7 speed features: app Native engine settings > Speed in 8 languages, Terminal Advanced rows, shared table strata_control/speed_switches.py; host tests isolated from the real components folder) 2026-10-26 12:08:29 +01:00
Niko1221 af7f7371c3 DeepSeek attention slab: the two new log lines lost their newline escape in the previous commit (a literal line break inside the string literal); no other change 2026-10-26 11:56:33 +01:00
Niko1221 bdde697225 DeepSeek engine: the chunked prompt's attention slab is automatic (DS4_ATTN_SLAB unset): up to 2048 rows where the bigger buffers take VRAM no expert would have used (the expert tier off, or every expert fits with room to spare), else the 0.2.6 rule; DS4_ATTN_SLAB=0 restores the 0.2.6 rule exactly, N rows overrides. If the automatic slab does not fit the load retries with the 256-row slab before it halves the chunk. Measured on the RTX 5070 (12 GB, 2.25 bpw): 1024 / 2048 rows gave +2% prompt but cost 29 / 83 VRAM experts, -3% decode and different greedy ids (placement), so there the rule keeps 256. attn_slab_rows() in ds4_experts_rule.hpp, 14 new checks in experts_rule_test. 2026-10-26 11:34:15 +01:00
Niko1221 eb448299c0 Merge branch 'f-027-kva' into f-027-hip 2026-10-26 11:04:09 +01:00
Niko1221 621e660604 Host tests green on this PC: the GuidedSetup very-large-model sentence gets its catalog key in all 8 languages; the Control lookup test no longer reads the real components folder
- locale-gaps: the sentence added in 72c37ea2d (RTX 30/40/50, R9700, Strix Halo; "the line below says what to expect") had no key. fx77 in the eight fix catalogs.
- packaged-control: findControl was given an env without STRATA_COMPONENTS_DIR, so it fell back to the real per-user components folder and found the
  updated Control installed on the PC (read only). The test now passes a scratch components folder.
2026-10-26 11:03:37 +01:00
Niko1221 cb2e1dbb4e Merge branch 'main' into f-027-cuda 2026-10-26 11:03:10 +01:00
Niko1221 bc5f84fd62 PLE residency policy: the load log also says the expert arena size the rule counted (the proof that the layout is initialised and the sum is not 0) 2026-10-26 10:55:20 +01:00
Niko1221 8a46ca9061 Merge f-027-int (the PLE residency fix) into f-027-kva 2026-10-26 10:55:03 +01:00
Niko1221 0ee72a5041 Merge branch 'f-027-int' into f-027-hip 2026-10-26 10:54:58 +01:00
Niko1221 ad1660aadf Merge f-027-win (PLE residency: admit the table only when RAM holds table + expert arena + 8 GiB; fixes the IQ3_S load failure, error 1453, on a 64 GB Windows PC) 2026-10-26 10:54:45 +01:00
Niko1221 472de9cdac PLE residency policy: admit the table only when RAM holds table + the expert arena + 8 GiB. The 0.2.7 rule (table + 8 GiB) admitted the 26.8 GiB table on a 64 GB Windows PC (RTX 5070, IQ3_S), where the engine then fills a 51 GiB pinned expert arena: the locked table left too little lockable memory and the expert load died with ERROR_WORKING_SET_QUOTA (1453). ple_resident_admit() (include/strata/core/ple_residency.hpp) sizes the arena from the native expert layout; unknown sizes or telemetry admit nothing. Unit test ple_residency_test. 2026-10-26 10:53:11 +01:00
Niko1221 ddd4d5d42b App: off switches for the three 0.2.7 speed features in the model's Expert settings (Native engine settings > Speed), on by default, in eight languages
Three rows under "Speed" in the native settings, in the existing row markup with the existing Select (On (recommended) / Off): Faster long prompts
(Qwen on CUDA or HIP; off writes STRATA_KVA=0), Faster answers on Radeon RX 9000 / AI PRO R9700 (Qwen on a gfx1200/gfx1201 card; off writes
STRATA_PCIE_FRAC_DEFAULT=old) and DeepSeek speed-ups (DeepSeek; off writes the six DS4_* = 0 of the contract). Each has a plain sentence saying what
turning it off does. A row is shown only where it applies, like the DeepSeek Speed row; the host decides from the model's file and the hardware report.

The switch is the model's file "env": off adds exactly its entries, on removes them, so a default launch is byte for byte what 0.2.6 started and the
Terminal, Control and the app reach the engine the same way (serve/server.py child_env). The host now shows and edits STRATA_* and DS4_* names (the
Strix Halo async entry is visible and kept); other names stay untouched. New strings are fx70..fx76 in the fix catalogs, translated in zh, ja, de, es,
ru, pt, ko. tests/speed-switches.test.mjs covers the default, each switch off and on again (same file), where each applies, and the catalogs.
2026-10-26 10:37:53 +01:00
Niko1221 3b3ea883c3 Terminal: the three 0.2.7 speed features as rows of the Advanced table, with their off values and plain words
strata_control/speed_switches.py holds the engine switch contract (Faster long prompts: STRATA_KVA=0; Radeon RX 9000 / R9700: STRATA_PCIE_FRAC_DEFAULT=old;
DeepSeek speed-ups: six DS4_* = 0), where each applies (Qwen on CUDA/HIP; Qwen on a gfx1200/gfx1201 card; DeepSeek) and the one-sentence effect text.
`strata launch` shows the environment names of the switches that apply as rows of the Advanced table (default "on", the sentence beside them); typing the
off value (`--set STRATA_KVA=0`) writes it into the run config's env, `off` takes it out again. A default launch carries none of them.
tests/terminal/test_speed_switches.py checks the names, the applicability, the config a launch writes, the rows, and that the app's table
(apps/host/server/native-settings.mjs) says the same.
2026-10-26 10:37:47 +01:00
Niko1221 750ca8c3b1 KVA on by default for a cold prompt of 4096+ tokens (Qwen3.8 on one CUDA/HIP device); STRATA_KVA=0 gives the exact 0.2.6 prompt read
The tail follows the draft layer: with KVA on and no --mtp-window given, the draft window becomes 2048 cells and the
fully computed tail 2112 (a given --mtp-window N sets the tail N + 64; a window of every cell, or STRATA_KVA_SUFFIX, is
handled and named). Layer splits, batch slots, --peer-device, pictures, the DFlash2 drafter, other model geometries and
the Metal build read exactly, with the existing "NOT applied: reason" line (quiet for continuation and short reads unless
STRATA_KVA=1). The reasons live in KvaConfig::decline and are unit tested (kva_test). Later chat turns continue from the
live session or a checkpoint and read their new tokens exactly over the cells the first read left; docs/KVA.md says so.
2026-10-26 10:14:06 +01:00
Niko1221 3c8a452900 DeepSeek copy-all: the huge-page arena (DS4_COPY_HUGETLB) is on when the reserved pool already holds the whole copy, and the pin of the copies (DS4_COPY_PIN) is on only when the whole arena is on explicit huge pages; =0 turns either off, an explicit =1 keeps the old opt-in behaviour, no pool means the ordinary arena exactly as before 2026-10-26 10:00:12 +01:00
Niko1221 1f079f9d13 DeepSeek HIP: the matrix-core prompt attention (DS4_FAST_ATTN) is on by default on gfx1200/gfx1201 only; DS4_FAST_ATTN=0 is the off switch, gfx11 and gfx1151 stay scalar unless =1 2026-10-26 10:00:11 +01:00
Niko1221 e5cfa4bbc8 docs: the miss split is the gfx12 start value again (the owner's decision 26 Oct; revert of the opt-in wording) 2026-10-26 09:58:12 +01:00
Niko1221 8d56d41768 gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55 2026-10-26 09:58:11 +01:00
Niko1221 45c9e623e2 STRATA_LOGITS_DUMP: a post-window logits tap for KL quality measurement (off by default); the sampler itself runs inside a CUDA graph capture, so the copy lives in the verify window's host-side epilogue where STRATA_DBG_NAN already copies from 2026-10-26 09:44:39 +01:00
Niko1221 053a41570c KVA: say why a configured run fell back to the full path (the observability criterion: an operator must be able to tell a configured-and-declined run from one that never asked) 2026-10-26 09:44:39 +01:00
Niko1221 5b469ee9a3 KVA: keep gdn_recurrence_variant's public signature - the SYCL tree defines that symbol, so a new parameter would break its link; the state-only form goes through a private impl 2026-10-26 09:44:39 +01:00
Niko1221 4abef6a74e KVA: what the opt-in approximate late-layer prefill does, how to enable and capture, and the measured frontier 2026-10-26 09:44:39 +01:00
Niko1221 c5cf218e54 KVA: state-only GDN recurrence (no gate projection, no output norm) behind STRATA_KVA_GDN_NORM for the A/B 2026-10-26 09:44:39 +01:00
Niko1221 011e5beced KVA: layer-type filter, per-layer transfer skip, stream-entry release on a skipped MoE half 2026-10-26 09:44:38 +01:00
Niko1221 f4012d6369 KVA: optional approximate late-layer prefill (split 24, chunk-aligned suffix, capture/oracle modes) 2026-10-26 09:44:38 +01:00
Niko1221 7dacaf8ea1 Merge f-027-r9700 (Qwen on one R9700: the PLE table kept in RAM when the PC can afford it, four bit-identical gfx12 decode switches back on) into f-027-int 2026-10-26 09:44:24 +01:00
Niko1221 ca7b3d2ca5 Merge f-026-miss (DeepSeek 0.2.7: the RAM store's spare + put, the 12 GB-cell prompt fixes, opt-in matrix-core attention on HIP) into f-027-int 2026-10-26 09:44:16 +01:00
Niko1221 7e1a7cb93c docs: the release notes carry the 12 GB routing default with its held-out evidence 2026-10-26 09:38:45 +01:00
Niko1221 a266bdfe1d markov bias in Q8: the drafter's head is quantised to Q8_0 blocks as it loads (66 MB -> 33 MB, the pack untouched) and read by a Q8 variant of markov_kernel - the markov segment 1.80 -> 1.34 ms, the drafter's round 13.46 -> 12.95, the drafts' acceptance 90-95% (the same band) 2026-10-26 09:35:56 +01:00
Niko1221 e43284572b head: 8 columns a pass for a GGUF head (every window rows<=kWin walks the 129280x5120 head once instead of twice, the same logits per column) - the drafter's fixed T=5 head 4.55 -> 2.83 ms, its round 14.49 -> 13.46; the verify window's head 3.48 -> 2.68 ms 2026-10-26 09:02:28 +01:00
Niko1221 0f66c316ee DeepSeek engine (AMD): the whole chunk's attention slab where the cards hold most of the experts
The slab rule (8 MB of q/o per device, 256 rows here) was set to keep the dense dequant's VRAM small; with a
1024-token chunk it dequantizes q_b/wo once per 256 rows - four times a chunk, 32 times an 8K prompt.
Measured on 4x R9700, 2.25 bpw, share 0.77, chunk 1024, 8K code prompt, ids 5705e1164a:

  256-row rule     1204 / 1236 / 1250 ppts, decode 77.7 tok/s
  chunk's 1024     1324 / 1353 / 1363        decode 78.1        (+9% prefill, decode unchanged)

The 1024 rows cost 268 MB a device (1% of the card).  So high-share runs take min(chunk, 1024) rows and the
old rule stays elsewhere; DS4_ATTN_SLAB still overrides.  The un-enved default verified at 1335 ppts /
77.8 tok/s with the ring at 253 MB.
2026-10-26 08:52:39 +01:00
Niko1221 4aa645c007 DeepSeek engine (AMD): a small staging ring where the cards hold most of the experts
With the 1024-token prompt chunk (the default where the cards hold most of the experts, abfe7d813) a big
staging ring is no longer the trade it was: every MB it takes is a tenth of an expert the decode keeps.
Measured on the same box, 8K code prompt + six 128-token decodes a load, ids 5705e1164a / 81d895366c:

  2000 MB  1243 ppts / 73.2 tok/s
  1000 MB  1287 / 77.5
   500 MB  1314 / 78.5
   256 MB  1324 / 78.4
   128 MB  1338 / 78.3

So high-share runs take clamp(room/device * 0.008, 128, 512) MB (248 MB here) and the old deeper ring stays
for the cases it was measured for (share < 0.5, where the cards stream most of the experts).  The un-enved
default verified at 1339 ppts / 78.1 tok/s, 42 s of summed GPU time in a 36 s pass.
2026-10-26 08:10:37 +01:00
Niko1221 88c2931963 DeepSeek engine (AMD): the small prompt chunk whenever the cards hold most of the experts, 4 cards included
prefill_chunk_default() returned 1024 only for a single-card run (own_default); a multi-card run whose cards
hold most of the experts fell through to 2048 (HIP skips the two 4096 branches) - and the rule's own datapoint
for 1024 was a 4 x R9700 run ('1024 370 tok/s, 8192 214').  Measured again here, 2.25 bpw, 4 cards, 8K code
prompt, --dspark auto, ids 5705e1164a, 4 prompts a load:

  512   1110 / 1137 / 1142 ppts (median 1123)
  1024  1229 / 1257 / 1270        (median 1243)   <- this commit's default
  2048  689 / 695 / 696           (median 692)    <- what the engine picked before
  8192  690 / 690 / 695           (median 690)    <- what the 0.2.7 configs passed

Decode unchanged (73.2 against 73.2 tok/s medians, --dspark auto).  So the owner's default 8K prompt goes from
692 to 1243 ppts (+80%) with no flag; --prefill-chunk still overrides.
2026-10-26 07:24:22 +01:00
Niko1221 85ddf96ffc Merge branch 'main' into f-027-cuda 2026-10-26 06:48:53 +01:00
Niko1221 e83b82afe5 DeepSeek engine (AMD): the drafter's cumulative confidence cut on by default (0.5) on HIP
DeepInfer's rule: verify a draft while the product of the confidences so far stays above the cut, instead of
only each draft's own floor.  The 0.2.7 notes left it off ('not measured faster on code: H100 / owner PC'),
Project Maya runs 0.5 on a Radeon 8065S (+6.5% decode, the same tokens).

Measured on 4x R9700, DeepSeek 2.25 bpw, six 128-token decodes an arm in one load, same ids
(81d895366c): 71.7 -> 76.9 tok/s (medians) with STRATA_DS4_DSPARK_CUM=0.5, and a second pair 74.1/73.6;
the default with the rule on (no env) verified at 74.5 tok/s (6 decodes).  CUDA keeps 0 (the rule was flat
there).  STRATA_DS4_DSPARK_CUM still overrides either way.
2026-10-26 06:09:02 +01:00
Niko1221 2247e5a98d DeepSeek engine (AMD): pin the CPU lane's RAM copy when the cards hold >= half the experts (DS4_COPY_PIN auto)
The prompt path copies every streamed expert from the RAM copy into the pinned staging before its DMA (the
file's own note: 6.8 GB/s through the staging against 24-28 GB/s probes).  Registering the copy lets it DMA
in place.  Three interleaved pairs on 4x R9700, 2.25 bpw (share 0.72): the second 8K prompt of a process
12.0 / 11.9 / 11.9 s against 28.8 / 13.8 / 12.3 s unpinned; decode 35.42 / 35.50 / 34.9-36.0 against
35.43 / 35.45 / 34.9-36.0 tok/s, i.e. no cost; ids unchanged.  Costs 23 s of load (registering a 50 GB
copy) - once a process, which for a server is the right trade.  Auto only when the expert share is >= 0.5
(the CPU lane is then a minority of the decode, so the pinned pages' TLB cost cannot bite); DS4_COPY_PIN=0/1
forces it, and it stays off where the 12 GB cards' decode is CPU-lane-bound (the -15% measured there).
2026-10-26 04:46:46 +01:00
Niko1221 5b969c1f7a DeepSeek engine (AMD, diagnostic): the io threads' busy time in the prompt stream line
The RAM->staging memcpy runs on the IoQueue's threads; on a warm 8K prompt (stream stage 14.3 s, 12.8 GB a
device) they are busy 4.1 s, i.e. ~3 GB/s a device - a real share, and the thing to check in the 50-95 s
first-prompt runs (where the same counters are what to read first).  DS4_PROF only.
2026-10-26 04:30:12 +01:00
Niko1221 3cc132ec51 DeepSeek engine (AMD, diagnostic): where copy_ahead's time goes in the prompt stream line (DS4_PROF only)
The first prompt after a load is 2-9x slower than the second on the 4x R9700 2.25 bpw cell and it is not
the disk (0 MB read during it), the RAM store (removing it is worse), the dense GEMM shapes (q_b+rope
1.46 s) or the compute waiting (0 ms).  The lead is the RAM->pinned-staging->DMA path (the file's own note:
6.8 GB/s through the staging against 24-28 GB/s probes); DS4_COPY_PIN=1 measured 47.9 -> 31.4 s on the
first prompt and 28.8 -> 12.0 s on the second, decode unchanged.  This commit only adds the counters
(staging / DMA issue / slot wait) to the existing prompt stream line under DS4_PROF; no behaviour change.
2026-10-26 04:07:19 +01:00
Niko1221 3b23994568 DeepSeek engine (AMD): the staging ring from each device's room (not the summed room)
No behaviour change on the 4x R9700 cell (126.8 GB of room / 4 = 31.7 GB a device, 8% -> 2000 MB, the value
measured in 76177ec34); on other card counts it is the per-device share that was meant.
2026-10-26 03:36:03 +01:00
Niko1221 1e72cdc4b9 DeepSeek engine (AMD): a staging ring sized from the VRAM room, and expert-tile knobs for the MarLin work
The prompt path streams the experts the cards cannot hold.  ring_mb_auto() gave 16 MB whenever the cards
held >= 1/1.3 of the experts, which is exactly this 2.25 bpw cell (126.8 GB of room, 156.3 GB of experts,
share 0.769): the stream then had no depth and the expert tiles waited 1.7 s inside a 5.2 s stage.  A share
of the VRAM room (5%, 400-2000 MB) when any expert spills, 16 MB when they all fit (the 0.2.6 rule, kept
for the decode it was chosen for).

Measured, 4x R9700, 2.25 plain pack, 6 experts, --dspark off, 8192-token code prompt, ids 5705e1164a:
8K warm 632 -> 675-678 ppts (+6.8%, 5 prompts a run, 3 runs), agent decode 35.9 -> 35.3 tok/s (-1.7%,
4 runs).  The profile of the warm pass: expert stage 5185 -> 4034 ms (kernels 3509 -> 2645 ms, the same
kernels now that the weights arrive in VRAM ahead of them, waits 1676 -> 1389 ms), everything else equal.

Also in this commit (opt-in, defaults byte-identical except the F16 activation path which is bit-identical):
- SetArgs::x16: gate/up take the layer's F16 copy of the activations (rows_f16 rounds exactly as the tile's
  own smem store did) - half the activation bytes, 26% faster gate/up in the standalone bench
  (box/expert_bench.cpp), +0.3% at the pass level (the stage is stream-bound).
- HIPK_CFG 6..11: tile variants for the expert GEMMs (64x128, 32x128, 128x256, 64x128/4 waves,
  128/128+down 256, 64x256).  64x128 (cfg 6) is 21% faster than 128x128 in the bench and +0.9% in the pass.
- HIPK_DBG 1/2/3: switch the MMA / the decode off in the tiles (the breakdown in 09-marlin-kernel-diary.md).
- ds4_iq_hip.inc: a UTF-8 BOM on a source file removed.
2026-10-26 03:01:27 +01:00
Niko1221 81ac4dd83b setup: a 16,384-token prompt chunk on a unified-memory PC whose pool holds ~128 GB (16K prompt 152.2 -> 158.6 tok/s, 32K 174.0 -> 179.7, 8K flat, decode unchanged); smaller unified PCs keep 8,192 2026-10-26 01:52:05 +01:00
Niko1221 44b88bc91e docs: the release notes carry the residency-biased routing default
It now ships on at 2.0 for a card of 13 GiB or less: +37.6% story / +35.2% code
decode over five interleaved pairs on an RTX 3060 12 GB, and it stays off on
bigger cards where it measured neutral to negative (l620 -1.9%/-0.5%, z440
-4.2%/+11.0%).  It changes which experts run - the owner approved that trade
for 12 GB cards - and STRATA_ROUTE_PRIOR=0 turns it off.
2026-10-26 01:01:09 +01:00
Niko1221 2395c8a53e hc: the one-kernel persistent hyper-connection read builds and runs on CUDA
The design was HIP-only: 80 resident blocks, a column slice of D each, two
grid barriers on an atomic counter, a barrier timeout so a card that cannot
hold all 80 blocks resident turns a hang into a refused start-up check.  Ported
by giving grid_sync a CUDA branch (volatile generation counter + clock64() for
the timeout, scaled to SM clocks), guarding the gfx-arch refusal to HIP and
adding a masked-shuffle helper.

On the RTX PRO 5000 Blackwell 72 GB it passes its own check - '80 blocks on 110
multiprocessors, 0 token counts timed out, max relative difference 1.8e-07
against the default read' - and measurably does NOT win: story 201.4 -> 190.6
tok/s (-5.4%), code 245.6 -> 249.6 (+1.6%) over 3 interleaved pairs.  That
matches the AMD note in the same file (an R9700 measured it neutral to 1%
slower), so it stays opt-in (STRATA_HC_PERSIST=1) and is not a default.  The
port is kept as the starting point for a CUDA-tuned block mapping.
2026-10-26 00:47:21 +01:00
Niko1221 c30656aaf9 route: residency-biased routing is on by default at 2.0 on a card of 13 GiB or less
The router's top-10 is picked again with a logit bonus for the experts already
in VRAM (route_prior.cu); the weights stay the router's.  Measured on an RTX
3060 12 GB, IQ3_XXS, 5 interleaved pairs of fresh engines: story 44.2 -> 60.9
tok/s (+37.6%), code 47.3 -> 64.0 (+35.2%); =1.0 gives +22.6% / +24.6%.  Three
more pairs against the switch itself read +23.6% / +36.9%.

It changes which experts run, so it is a default only where the expert cache
cannot hold the model, and the owner approved that trade for 12 GB cards.  It
is NOT a default on a bigger card: on an RTX PRO 5000 72 GB it measures
-1.9% / -0.5% and on 2x RTX A4000 -4.2% / +11.0% (3 pairs each); both were
re-run after this change and are unchanged (201.7 / 246.9 and 87.1 / 107.1
tok/s).  STRATA_ROUTE_PRIOR=0 turns it off and any explicit value wins.
2026-10-26 00:27:14 +01:00
Niko1221 219beb5736 DeepSeek engine: porting guards for the 12 GB work. The matrix-core attention and its dispatch are now inside #if defined(STRATA_USE_HIP) (a CUDA/sm120 build keeps the scalar path and compiles), and a failed DS4_COPY_HUGETLB allocation falls back to the ordinary arena with a warning instead of leaving that layer's experts in place (the flag is safe on a machine with no huge-page pool). HIP build verified: 8K warm 768.7 / 767.4 / 765.6 ppts with DS4_FAST_ATTN=1, ids 5705e1164a, the agent decode's 81d895366c. 2026-10-26 00:21:01 +01:00
Niko1221 32c686fa67 DeepSeek engine: the matrix-core attention's fragments load two halves a word (the scores' Q/K tiles and the values' P tile), and the online softmax runs a warp per two heads with a lane a key (16-lane shuffle max/sum) instead of a thread a head walking 16 keys. R9700 12 GB cell, 2.25 pack, 8K warm with DS4_FAST_ATTN=1: 767.7 / 765.8 ppts (10.67/10.70 s), the sparse attention stage 2516 -> 2311 ms, first prompt 465.6, with the same ids (5705e1164a, the agent decode's 81d895366c) and the same decode speed (16.49 tok/s). Quality gate on this binary (4995-position score, 2.25 plain, ref_base1): KL 0.10776 / ppl 3.389 / top1 87.89% against the scalar kernel's 0.10746 / 3.395 / 87.81% (delta 0.0003; the gate is 0.006). The softmax's sum order differs from the previous matrix-core build (a shuffle tree, not a key walk). 2026-10-25 23:43:12 +01:00
Niko1221 f05ccd3b54 DeepSeek engine: DS4_FAST_ATTN=1 runs the chunked prompt's packed-KV attention on the gfx11/gfx12 matrix cores (new sparse_attn_wmma_kernel, opt-in; the decode and verify paths stay on the scalar kernels). The matrix's rows are 16 heads of ONE query, so a block stages a 16-key tile once for all of them: Q (16 heads x 512) is the A operand of the scores, the key tile is B, and the same tile is B of the values' product; the scores' 64-dim slices are computed by the 8 warps and summed in shared, the values by a warp a 64-dim slice, the online softmax a head a thread. R9700 12 GB cell, 2.25 pack, 8K warm: 752.2 / 748.3 ppts (10.89/10.95 s) against 694.2 / 693.0 / 691.2 for the scalar kernel; the sparse attention stage 3750 -> 2516 ms; first prompt 484.8. Quality gate (4995-position score, 2.25 plain, ref_base1): KL 0.10750 / ppl 3.394 / top1 87.83% against the scalar kernel's 0.10746 / 3.395 / 87.81% (delta 0.00004; the gate is 0.006). The ids are the same (5705e1164a) and the 128-token agent decode's ids too (81d895366c); its decode speed unchanged (16.49 vs 16.87 tok/s). The QK/PV sums are the matrix cores' order, hence opt-in (the owner approved the trade for the prefill). 2026-10-25 23:29:29 +01:00
Niko1221 2dde837d9f DeepSeek engine: DS4_COPY_HUGETLB=1 puts the CPU lane's RAM copy on explicit 2 MB pages (MAP_HUGETLB from the kernel pool; the copy's RAM budget adds the pool, which MemAvailable does not count). With DS4_COPY_PIN=1 the pin then keeps 2 MB pages: on a transparent-huge-page arena the registration pins 4 KB pages and the prompt's DMA from it swings between 11.9 and 30.4 s for the same 140.2 GB stream. R9700 12 GB cell, 2.25 pack, 8K, 3 warm prompts: huge-page arena + pin + DS4_RING_MB=3500 + DS4_ATTN_SLAB=2048 -> 694.2 / 693.0 / 691.2 ppts (11.80-11.82 s), first prompt 435.4 (18.8 s), ids identical (5705e1164a); the THP arena with the pin ran 691/573/270 in one load and 665-691 in another. Copy at load 33.3 s for 156.7 GB; the pool is reserved once per boot (sysctl vm.nr_hugepages=78000). Opt-in: default path unchanged; no-pin staged stays 651. 2026-10-25 22:27:30 +01:00
Niko1221 bc6a39217a docs: 0.2.7 release notes draft from the CUDA speed work
A draft for the owner: the one engine default that changed (the wide
hyper-connection up read on a 12.0 card, +1.9% story / +0.7% code decode on
the RTX PRO 5000, 5 pairs, bitwise), the four settings that are faster where
the PC supports them (PLE in RAM, the English draft vocabulary, --prefill-pipe
on a two-card split, q4_0 KV on 12 GB with its precision cost), and the list
of what was measured and rejected so it is not retried.  No version number is
claimed: the owner decides what else 0.2.7 carries.
2026-10-25 21:26:06 +01:00
Niko1221 d39e2a7539 sm_120: the wide hyper-connection up read is on by default
STRATA_GR_UP_WIDE (one warp a row, N/2 = 1,280 blocks instead of 160)
measured on the RTX PRO 5000 Blackwell 72 GB, IQ3_S, every expert in VRAM,
5 interleaved pairs of fresh engines: story 197.6 -> 201.3 tok/s (+1.9%),
code 243.5 -> 245.1 (+0.7%).  A third run against the switch (default on vs
STRATA_GR_UP_WIDE=0) reads +1.7% / +0.8%.  The produced token ids were
identical in all 6 pairs, and fused_gr_bench's per-T hashes are the plain
read's.  The switch stays: =0 off, =1 on any card, STRATA_SM120_PERF=0 the
whole 12.0 set.  Other cards are untouched.
2026-10-25 21:21:28 +01:00
Niko1221 2465b93708 docs: the miss split is opt-in, not a default: the text can differ (4 of 6 fixed prompts within 64 greedy tokens) even though the first-token KL is 0.0013 2026-10-25 21:03:52 +01:00
Niko1221 1d00c018e1 Revert "gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55"
This reverts commit 550c6ddbf90d57a0d018878115f6d0f9fa86cb74.
2026-10-25 21:03:43 +01:00
Niko1221 1f00c1825e Merge f-027-async: synchronous verify window on Strix Halo (+31% code decode)
setup writes DS4_ASYNC_T=4 into the config env for an integrated Radeon: the engine's mailbox path is sized for cards that hold
most of the experts and costs this box 31% on code (15.9-16.3 -> 20.1-21.3 tok/s), 16% on agent with identical accepted tokens,
9% on chat; the 2.25 set (no drafter) is unchanged.  The same tokens come out.  Discrete cards keep the engine default.
2026-10-25 20:53:56 +01:00
Niko1221 63b8a8d3cc Strix Halo: run the verify window synchronously (DS4_ASYNC_T=4 in the config env), +31% code decode
The engine's windowed decode uses the mailbox ("async") path for windows up to DS4_ASYNC_T (8 by default). That default was sized
for cards that hold most of the experts; on the 128 GB gfx1151 box the 6-row verify window is faster on the synchronous path.
Measured 2026-10-25 with the 2.00 bpw set and the shipped 0.1.51 engine (greedy, 256-token sets, three interleaved reps):

- code 15.87-16.33 -> 20.14-21.28 tok/s (+31%);
- agent, with identical accepted tokens a round (5.07): 15.93 -> 18.48 (+16%);
- chat 10.33 -> 11.30 (+9%);
- 2.25 code unchanged (11.2 -> 10.9): the drafter is unloaded there, so the extra syncs have nothing to hide.

The same tokens come out; only the scheduling changes, and the drafter's own single-token windows (T=1) stay on the async path.
dsv41_config_env(cards) writes DS4_ASYNC_T=4 into the generated config for an integrated Radeon only (a discrete card keeps the
engine default); STRATA_DSV41_ASYNC_T=n overrides the value and =8 restores the default. Tests:
tests/terminal/test_dsv41_async_t.py (5 cases). No engine rebuild; the signed dsv41 package is unchanged.

Docs: the STRIX_HALO and DEEPSEEK41 measured tables and the README row now read 19-21 tok/s code, 18 agent, 11 chat.
2026-10-25 20:53:52 +01:00
Niko1221 3a26753027 STRATA_TRACE: one clock for the whole process, so a layer split's stages compare
The trace origin was thread-local, and the prompt's later stages run on their
own thread, so their timestamps could not be compared with the first stage's.
One process-wide origin makes the hand-off visible.

Found with it on z440 (Q2_0, split 24, 16,383-token prompt, 8K chunks): stage
0 starts at 0 ms and 3,267 ms, stage 1 at 3,268 ms and 5,996 ms, read ends at
8,775 ms - the two stages do overlap chunk-for-chunk, and each takes ~2.7 s
per 8K chunk for its 24 layers.  A split 30 run leaves stage 1 at 3.09 s a
chunk with 18 layers, so the per-chunk cost is not its layer count.
2026-10-25 20:52:26 +01:00
Niko1221 457d8cc791 DeepSeek engine: DS4_ATTN_SLAB=N sets the chunked prompt's attention slab rows (default unchanged: 8 MB / HQ rounded to 64). The slab loop runs the dense projections (q_b, wo_a, wo_b) per slab, and each call dequantizes the whole dense matrix: a 256-row slab dequantized the same weights 32 times for an 8K prompt (the dense dequant 916 ms and rocBLAS 867 ms of that prompt). R9700 12 GB cell, 2.25 pack, 8K warm: slab 256 -> 591.9 ppts (with the earlier fixes), slab 2048 + DS4_RING_MB=1600 -> 668.0/666.2 ppts (12.26/12.30 s), the first prompt 429.9 (from 143-402), ids identical (5705e1164a); q_b+rope 3.97 -> 1.15 s. Opt-in: the default slab is unchanged. 2026-10-25 20:45:17 +01:00
Niko1221 14d076ee2e STRATA_TRACE: the prompt chunk line carries its stage range and elapsed ms
Same diagnostic as the request-step timestamps: with a layer split the chunk
line now says which stage is starting a chunk and when, so a prompt's stage
overlap can be read off the log.  STRATA_TRACE only; no behaviour change.

Measured with it on z440 (Q2_0, split 24, 16,383-token prompt): serial reads
run stage 0's two 8,192-token chunks 3,332 ms apart and the whole read takes
8,788 ms; --prefill-pipe 1500 reads four 4,096-token chunks 1,582 ms apart and
the read takes 8,375 ms, so the stages overlap only partially at best.
2026-10-25 20:15:58 +01:00
Niko1221 13c4aebeca docs: the miss split is the gfx12 start value now (measured A/B on the shipped path: 8K +9.0%, 32K +23.6% decode) 2026-10-25 20:08:41 +01:00
Niko1221 d7ad4a2762 gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55 2026-10-25 19:57:35 +01:00
Niko1221 8abe826f70 DeepSeek engine: the chunked prompt's dense dequantizer writes its 32 values a thread through shared memory, coalesced (a warp covers 1024 contiguous values), instead of each thread writing 2 bytes every 64 - a warp's store touched 32 cache lines. Same arithmetic and the same F16/BF16 values (only the store order changes). R9700 12 GB cell, 2.25 pack, 8K warm: prompt 14474 -> 13840 ms (566 -> 591.9 ppts), the q_b+rope stage 4025 -> 3581 ms, ids identical (5705e1164a); the cold first prompt also uses it. DS4_DQ_OLD=1 keeps the old kernel for A/B. 2026-10-25 19:43:51 +01:00
Niko1221 7901bc5137 docs: RDNA4 decode miss split measured on the R9700 (--pcie-frac 0.15: +9.9% at 8K, +25.6% at 32K, first-token KL 0.0013, top-1 12/12) 2026-10-25 19:15:53 +01:00
Niko1221 91d4d7ff86 STRATA_TRACE: print elapsed milliseconds on each request step
The trace lines named the step a request reached but not how long it took to
get there, so a stall between two marks was invisible.  Each line now carries
the time since the first traced step (thread-local).  A diagnostic only: no
step, no order and no output changes without STRATA_TRACE set.

Found with it on z440 (Q2_0, layer split 24, 16,383-token prompt): the batched
read takes 8,758 ms while stage 0's own GPU timeline is 5,374 ms - stage 1 is
the slow one - and the expert-cache refill after the prompt costs a further
592 ms of the request.
2026-10-25 18:56:26 +01:00
Niko1221 2b5cadd984 Merge f-027-perf: 8,192-token prompt chunk on Strix Halo (11-15% faster prompts)
Measured on the 128 GB box with the 2.00 bpw set: 4K prompt 78.7 -> 87.2 tok/s, 8K 108.9 -> 125.1, 32K 155.2 -> 176.9;
decode unchanged (16.53 -> 16.59).  setup writes --prefill-chunk 8192 only for an integrated Radeon whose whole pool is at
least 64 GB; the escape hatch is STRATA_DSV41_NO_BIG_CHUNK=1.  Also repairs UTF-8 damage in three docs from an earlier
PowerShell edit.
2026-10-25 18:54:28 +01:00
Niko1221 e7e48be2c9 Strix Halo (gfx1151): an 8,192-token prompt chunk on the unified-memory APU (11-15% faster prompts)
The engine's prompt-chunk rule returns 1,024 when a single card holds half or more of the experts; it was calibrated on
cards that hold most of them (4 x R9700).  On the 128 GB Strix Halo (share 0.62, one shared pool) the bigger chunk wins:
measured with the 2.00 bpw set, engine 0.1.51, 2026-10-25: 4K prompt 78.7 -> 87.2 tok/s, 8K 108.9 -> 125.1, 32K 155.2 ->
176.9.  Decode is unchanged (16.53 -> 16.59): the chunk buffers take ~5 GB from the GPU expert tier (81.9 -> 76.6 GB) and the
RAM store's hit rate rises (68.6% -> 76.8%) to compensate.

dsv41_prefill_chunk_args(cards, ram) writes --prefill-chunk 8192 for an integrated Radeon only, and only when the whole pool
(the RAM Windows sees plus the BIOS carve-out) is at least 64 GB; a discrete card or a smaller unified PC keeps the engine's
choice, and STRATA_DSV41_NO_BIG_CHUNK=1 turns it off.  The chunk changes the prompt GEMMs' summation order, not the model: the
repo's chunk-size KL check reads 1024 0.1705 / 2048 0.1692 / 4096 0.1688 / 8192 0.1695.  Test:
tests/terminal/test_dsv41_prefill_chunk.py (5 cases).

Also repairs UTF-8 damage in README.md, docs/STRIX_HALO.md and docs/MODELS.md from an earlier PowerShell edit (the middots,
em dashes and arrows had been double-encoded; no text changed).  No engine rebuild is needed: the existing signed dsv41 package
accepts the flag.
2026-10-25 18:54:22 +01:00
Niko1221 00a1292531 DeepSeek engine: the post-prompt RAM-store preload skips the experts the CPU lane already holds as a RAM copy (copy_mode). In copy-all mode every routed expert is in that copy, so the preload read a second copy of the same bytes: 4152 experts, 40 GB at 6.4 GB/s, 6.2 s of every 8K prompt, with 0.0 store hits in the decode after it; with --mmap-experts off and the experts not all fitting, the in-place ones are still preloaded. R9700 12 GB cell, 2.25 pack, 8K: prompt 27918 -> 21567 ms (293 -> 380 ppts; a repeated prompt 526.6), decode 16.61 -> 16.82 tok/s, ids identical (5705e1164a). With DS4_COPY_PIN=1 the same run is 21347 ms (383.7 ppts, repeat 535.6). 2026-10-25 18:45:40 +01:00
Niko1221 4be0a7ce4d DeepSeek engine: DS4_COPY_PIN=1 DMAs the prompt's experts from the CPU lane's copies in place (cudaHostRegister, read-only with DS4_COPY_PIN_RO=1), so the flow does not copy each one to the pinned staging first; the pinned chunks are unregistered with the arena. R9700 1 card, 2.25 pack, 8K prompt: the stream's compute wait 5517 -> 0 ms, prompt 28035 -> 27392 ms (292 -> 299 tok/s), ids identical; decode unchanged (16.6 vs 16.8 tok/s). The stream itself stays ~7 GB/s (the DMA issue pattern, not the staging), so the gain is small here; on an SSD-bound box the removed wait is the point. Opt-in: default path unchanged. 2026-10-25 18:05:54 +01:00
Niko1221 1d380fad87 arch_defaults_test: the gfx12 table is nine switches now (five prompt, four decode); check the apply path against a clean environment 2026-10-25 17:47:24 +01:00
Niko1221 d18d8ba045 CUDA HC: the wide up read, one warp per row, opt-in (STRATA_GR_UP_WIDE=1)
The up projection of the hyper-connection read runs 160 blocks of 8 warps
(each warp takes 8 rows) on a 12.0 card, so 160 blocks of 192 SMs and ~6.7
warps an SM: the read is latency-bound at about 0.5 TB/s.  This kernel takes
one row per warp and 2 columns x the 4 streams a block, so the grid is
N/2 = 1280 blocks (53 warps an SM).  Every row's dot, xor tree, epilogue and
the cross-stream sum are gr_up_multi_kernel's, in its order, so the outputs
are bitwise the same; the token streams of both arms were identical in all
6 pairs measured.

Measured on l620 (RTX PRO 5000 72 GB, all experts in VRAM, IQ3_S, 3
interleaved pairs, fresh engine each): story 197.4 -> 201.3 tok/s (+1.9%),
code 243.0 -> 244.7 (+0.7%).  Off by default (STRATA_GR_UP_WIDE unset); the
q8_1 tail of S26 needs the 16-column blocks, so the wide kernel is only used
when no q8_1 image is written.
2026-10-25 17:39:28 +01:00
Niko1221 3fd5a748c7 gfx12: the four bit-identical decode switches back on (ATTN_LANECELL, ATTN_MERGE_V2, TSUM, EXPDEC), measured on p620 R9700 IQ3_S 8K: +3.5% decode, 3/3 pairs, answer md5 identical; docs: the PLE residency policy and its numbers; ple_reader_test: table_bytes 2026-10-25 17:34:42 +01:00
Niko1221 3f26ca5c2f Merge f-027-strix: DeepSeek V4.1-Flash on Ryzen AI Max (Strix Halo, gfx1151) on Windows
The Windows AMD DeepSeek engine (feed member dsv41, windows-x86_64-hip-rdna3, gfx1151) ships with these
changes: the 0.2.6 copy-all rule is off on an integrated GPU (it copied 53 GB into RAM beside the GPU's 82 GB
in one 128 GB pool and an 8K prompt stalled; the in-place path is the 0.2.5 soak path), a Windows HIP package's
rocm/bin is on the engine's library path (setup, run config, app), the unified-memory fit plan counts the BIOS
carve-out, and --feed reaches the DeepSeek engine download.  Docs, app text and tests included.

Measured on the Strix box (2.00 bpw, Faster, DSpark auto): 4K prompt 78.7 tok/s, 8K 108.9, code 17.6 tok/s,
chat 11.0.  Clean-user end-to-end from a signed feed: install, download, pack, smoke, serve, chat, tool call,
agent round trip: 5/5.
2026-10-25 17:20:42 +01:00
Niko1221 a6543d6a7f setup: accept --feed (the DeepSeek engine's package comes from it)
strata launch --feed passed the feed to setup, but setup's parser had no --feed (getattr(a, 'feed', None)
was always None), so the flag was refused with 'Unrecognized arguments' and a local or non-default feed could
never serve the DeepSeek engine (found by the clean-user test on strix, 2026-10-25).  It is now the same
option strata update has: a URL or a folder.
2026-10-25 17:06:43 +01:00
Niko1221 932bda54bd Strix Halo: the DeepSeek engine follows --feed, and the unified plan counts the BIOS carve-out
Found by the clean-user test on strix: 'strata launch --feed DIR' read the local feed for its own engine
check and then setup went to GitHub for the DeepSeek package, so a local (or any non-default) feed never
served the DeepSeek engine.  setup's get_dsv41_engine/upgrade_dsv41_engine take the feed, install_dsv41
passes --feed, and runner.setup_args hands --feed to setup (a test in test_launch_dsv41).

The unified-memory plan counted only the RAM Windows sees (64 GB on the owner's box) and ignored the 64 GB
BIOS carve-out the GPU owns: 128 GB in one pool.  uma_total_gb() adds the carve-out (or the GPU figure less
its shared part), so the plan now says about 109 GB of the 2.00 set's 132 GB stay in memory and 23 GB are read
from the SSD; the docs and the test fixtures use the 128 GB box.  The 'measured' text says 128 GB unified too.
2026-10-25 17:00:58 +01:00
Niko1221 3b34b6c7b2 Engine: keep the PLE table resident in RAM when the PC can afford it (opt-out STRATA_PLE_AUTO_RAM=0); measured on p620 R9700 IQ3_S: 8K prompt PLE 845 -> 132 ms, 2125 -> 2611 tok/s, answer md5 identical 2026-10-25 16:42:05 +01:00
Niko1221 6c70d4bc5d Strix Halo (gfx1151) on Windows: DeepSeek V4.1-Flash support and the 0.2.6 shared-memory fix
The Windows AMD DeepSeek engine package (feed member dsv41, windows-x86_64-hip-rdna3, arch gfx1151) is built from this
branch and installed as its own package; these are the code changes it needs.

- ds4_model.cu: on an integrated GPU the CPU lane's experts are no longer copied into RAM by the auto rule.  The GPU's
  expert tier and that copy are the same shared memory: on the 64 GB Strix Halo the GPU held 82 GB of experts, the rule
  saw 62 GB 'free' and copied 53 GB, and an 8K prompt did not finish in 11 minutes.  Found 2026-10-25 on strix; the
  corrected build does the same prompt at 108.9 tok/s (4K 78.7), code answers 17.6 tok/s, chat 11.0.  DS4_COPY_ALL=1
  still forces the copy for a test.
- dsv41_engine.py: lib_dirs()/start_env() put the package's BUILD.json lib_dirs on PATH/LD_LIBRARY_PATH (a Windows HIP
  package keeps hipBLAS/rocBLAS in rocm/bin, or a clean PC fails with 0xC0000135); the upgrade test-start uses it.
- setup.py: the dsv41 engine's library dirs come from the package (so the run config carries rocm/bin), the unsupported-card
  text names Ryzen AI Max on Windows, and the manual package hint names the .tar.gz the release page actually has.
- runner.py: the same library dirs for the app/Terminal path.
- dsv41_fit.py: a unified-memory plan for the official IST-DAS sets (one pool, what stays in RAM, what is read from the SSD,
  measured 14-16 tok/s on code, 10-11 on chat, 4K 78 / 8K 109), never a refusal; tools/strata_mcp.py uses it for the app
  catalog's on-this-PC line.
- Docs: STRIX_HALO.md (DeepSeek section with speeds, driver 26.10.1, memory behaviour, limits), DEEPSEEK41.md, MODELS.md,
  AI_REFERENCE.md, README.md, packaging/README.md (the Windows AMD package recipe).
- Tests: tests/terminal/test_dsv41_fit.py (7) and 4 new tests in test_dsv41_engine.py (lib dirs, feed selection for gfx1151).
2026-10-25 16:41:49 +01:00
Niko1221 4b9e05a2a8 Windows uninstaller: remove the app's own folders (control, host, node, harness) recursively; files the app wrote there while it ran (Python bytecode caches under control) are not in the installer's file list, so those folders and the program folder stayed behind after use (found on strix with a standard user after the process fix); test 2026-10-25 13:56:36 +01:00
Niko1221 af197f96db Windows uninstaller: find Strata's own running processes with the 64-bit PowerShell (the installer is 32-bit, whose Get-Process cannot read a 64-bit process's path) and with Get-Process first, WMI only as a second source: a standard user's WMI query was denied, so the host kept the folder open and it stayed behind (found on strix with a standard user) 2026-10-25 12:32:41 +01:00
Niko1221 dfaae03b00 DeepSeek engine: the RAM store no longer misses the experts the VRAM cache gives back to the CPU lane: the preload fills the slots the lane leaves over (and the 256 reserved ones, evictable first) with the VRAM experts the profile counts least and the lane experts it never saw (DS4_STORE_SPARE=0 off), and an expert leaving VRAM is copied into a store slot from its VRAM blob (put_begin/put_done, DS4_STORE_PUT=0 off) instead of being read from the SSD while the lane waits. r9700 1 card 2.00, store forced (--ssd-cache 140): decode code 23.7 -> 26.2, agent 20.7 -> 24.6 tok/s, misses 1.5/2.6 -> 0.5/0.6 a token; P2003 RTX 3060: no loss, ids identical 2026-10-25 11:54:42 +01:00
Niko1221 53548c8af4 Merge main (41cf05a06: README model order, docs only) into f-026-rel 2026-10-25 11:37:25 +01:00
Niko1221 41cf05a06a README (all 7 languages): models in the order Qwen3.8-Flash-Next, Qwen3.6-35B-A3B, DeepSeek V4.1-Flash 2026-10-25 11:35:36 +01:00
Niko1221 4f0f933941 Merge main (ed4b4db40: README, docs and icons) into f-026-rel; AGENTS.md keeps main's text with Version 0.2.6, engine 0.1.51; the installer names in the READMEs and docs are the 0.2.6 ones 2026-10-25 11:29:12 +01:00
Niko1221 ed4b4db405 docs and packaging: the owner's 2026-10-23 guide set brought in for 0.2.5 (one installer per system, engine downloaded on first start, DeepSeek in the app and the Terminal with three sets and the Speed choice, install.sh form kept, versions 0.2.5 and 0.1.50, source setup labelled as the developer path) 2026-10-25 11:28:02 +01:00
Niko1221 af8c6f8ec7 README translations (zh-CN, ja, de, fr, es, pt-BR) rebuilt in the structure of the new README; the README links source setup to SOURCE_INSTALL.md and names the three 0.2.5 installers 2026-10-25 11:28:01 +01:00
Niko1221 e99c4d2d37 AGENTS.md and AI_SETUP.md: lead an agent to the installer, install.sh/install.ps1 and strata launch --plan then --yes --no-tui; source setup only as the fallback; provider order ISTA-DASLab, Unsloth, Gyro, others 2026-10-25 11:28:01 +01:00
Niko1221 b8b4ca0e32 docs: add the guide index, the source-install page, the AI reference and the media kit (the owner's pages, updated to 0.2.5) 2026-10-25 11:28:01 +01:00
Niko1221 562a0c2ed7 Version 0.2.6: the Qwen engine packages and Metal stay 0.1.50 (no C++ change since 0.2.5: only the DeepSeek engine and a generate.cpp comment changed); Control and the DeepSeek engine are 0.1.51 as the CMake project version, the shape of 0.2.4; build_hip_linux.sh keeps PKGVER 0.1.50 2026-10-25 11:21:36 +01:00
Niko1221 79ec8e2786 Host test: the source check for 'a version that is not downloaded starts with Read pictures on' accepts the braced else that the DeepSeek 8K context fix introduced (test only) 2026-10-25 11:16:25 +01:00
Niko1221 b5d41babfa README: the owner's layout (model dropdowns with Qwen and DeepSeek marks, provider and set tables, short install, hardware and tools tables), updated for 0.2.5: DeepSeek in the app and the Terminal with three sets and the free disk each needs, 0.2.5 speeds, one installer per system; model marks from LobeHub Icons (MIT) 2026-10-25 11:14:21 +01:00
Niko1221 27a8ed96b6 Version 0.2.6: Terminal and app 0.2.6, engine and Control 0.1.51 (every Qwen and DeepSeek package, Metal included) 2026-10-25 11:14:16 +01:00
Niko1221 8aee4538f2 generate.cpp: the --spec-split comment no longer calls its windows exact (they are not bit-exact against no-split: batch-shape divergence); comment only 2026-10-25 11:14:00 +01:00
Niko1221 e9e22db6a7 Setup: the DeepSeek engine step does not stop on 'can't reach GitHub' when the update feed is a local one (STRATA_UPDATE_FEED or --feed naming a folder, file or loopback address); a transient GitHub timeout no longer fails a verify run; tests 2026-10-25 11:13:51 +01:00
Niko1221 7025a99799 WSL on AMD: setup and launch say plainly that the AMD (HIP) engines do not run under WSL yet (no /dev/kfd, the GPU bridge never finishes a command) and name the Windows app or Terminal as the next step; a warning, nothing is refused; tests 2026-10-25 11:12:59 +01:00
Niko1221 36c6dd9ef5 Windows uninstaller: stop the app, host, node and engine that run from the install folder before files are removed, so the program folder no longer stays behind after use; only processes whose image path is inside the install folder; test with a real process 2026-10-25 11:11:37 +01:00
Niko1221 4a6a5fb699 Merge f-026-copyall (DeepSeek copy-all CPU-lane experts) into f-026-rel 2026-10-25 11:10:16 +01:00
Niko1221 d51d71e7c0 Merge f-025-fix (6 owner-QA fixes) into f-026-rel 2026-10-25 11:10:13 +01:00
Niko1221 fa663fabc5 Merge f-0243-rel (721c6f7ff) into f-025-readme 2026-10-25 10:41:20 +01:00
Niko1221 721c6f7ff5 Mac: the engine bundle inside a signed .app is not removed after the first start (deleting it broke the app's signature); outside an .app and on other systems the cleanup is unchanged; test 2026-10-25 10:37:21 +01:00
Niko1221 025cf7d954 DeepSeek engine: the copy-all fit check is one tested rule (copy_all_fits) and also needs the Windows commit room (RAM + page file beside its margin), not only the free RAM, so a 64 GB PC with a 12 GB card, or a big-RAM PC with a small page file, never copies the lane in by itself; the auto store says (verbose) when it holds less than the CPU lane 2026-10-25 10:18:53 +01:00
Niko1221 5d5c26d5bc Monitor/serve: a one-token reply has no decode span, so no tok/s is shown for it (a 1-token probe read 1,250 tok/s); test 2026-10-25 09:29:40 +01:00
Niko1221 8a3ca07c4e serve: a chat request with no message is a 400 (messages must be a non-empty list), as OpenAI answers it; test 2026-10-25 09:28:47 +01:00
Niko1221 dd9b9f746d About/Updates: never show the host's placeholder engine version (strata-engine 0.1.0) while the real one is still being read; a forced status read that meets one already running waits and reads again, so the version after an engine update is not the old one; test 2026-10-25 09:27:38 +01:00
Niko1221 6b5d593639 DeepSeek engine: --mmap-experts auto copies all the CPU lane's experts into RAM at load when they fit beside 8 GiB, with no demand store (all or none; --mmap-experts on/off, --ssd-cache N and DS4_STORE_GB stand as given); the auto store logs when it holds less than the lane. 1 x R9700 + 251 GB, 2.00 pack, warm: decode code 24.5 -> 27.6, agent 21.9 -> 27.1 tok/s, same ids as --ssd-cache 0 --mmap-experts off, load 34 -> 55 s. Rule ported from Project Maya v1.1.27 (MIT), mmap-experts auto copy-all 2026-10-25 09:27:16 +01:00
Niko1221 4d6721822a Storage and Manage models: a model whose files are all gone (old config, 0.0 GB, not ready) is no longer listed as a download 2026-10-25 09:26:19 +01:00
Niko1221 ad66a758a7 Models screen: DeepSeek is installed and shown at its one 8K context (as the guided setup), not the 32K default 2026-10-25 09:25:53 +01:00
Niko1221 59599f69f8 Monitor: All GPUs leaves out a small integrated Radeon/Intel chip beside a real card (totals and count); the hardware report gives each card a stable id since each make numbers from 0; tests with the RTX 5070 + Radeon iGPU pair 2026-10-25 09:25:17 +01:00
Niko1221 a04f5505aa Merge f-0243-rel (ad638b93e) into f-025-readme 2026-10-25 02:12:48 +02:00
Niko1221 36f580a5c3 UPDATES.md: the Mac dmg keeps its Metal engine; only the Windows and Linux installers are engine-less 2026-10-25 02:12:42 +02:00
Niko1221 ad638b93e5 Merge f-0243-dsimport (the app's model-folder preview recognises official DeepSeek sets) into f-0243-rel 2026-10-25 01:07:30 +02:00
Niko1221 32ad109816 Model preview: recognise official DeepSeek-V4.1-Flash IST-DAS sets (1.75/2.00/2.25 bpw, official and first-release names, with or without the Engram) as family deepseek41 2026-10-25 01:07:05 +02:00
Niko1221 6270e2d9d6 collect_libs: a library an earlier binary's pass already put in --out is not copied onto itself (SameFileError stopped the AMD packages' second pass) 2026-10-24 22:51:19 +02:00
Niko1221 ce27d13539 Merge f-0250-tui-uma (Faster is the default Speed on shared-RAM GPUs) into f-0243-rel 2026-10-24 22:31:11 +02:00
Niko1221 2dbdea2a77 Merge f-0243-strixspeed into f-0243-rel 2026-10-24 22:13:52 +02:00
Niko1221 da1eb6bc69 Merge f-0243-copyfix into f-0243-rel 2026-10-24 22:13:51 +02:00
Niko1221 2cea5afbc5 Merge f-0243-locales into f-0243-rel 2026-10-24 22:13:51 +02:00
Niko1221 0b9539877f README and docs for the single installer: three download buttons (Windows, Linux, Mac) with the 0.2.5 file names, one plain line instead of the per-GPU sentence (7 languages); docs/AMD_HIP.md and packaging/UPDATES.md name the single installers 2026-10-24 22:08:00 +02:00
Niko1221 9551b4168d Metal and SYCL builds: the Metal engine gets stubs for the 512-bit i-quant rows native_expert.cpp now references (x86-only file), and SYCL compiles iq_avx512.cpp with -mbmi2 and -mavx512vpopcntdq like the root CMakeLists.txt 2026-10-24 22:01:11 +02:00
Niko1221 1aff6d1411 Version 0.2.5: Terminal and app 0.2.5, engine and Control 0.1.50 (every Qwen and DeepSeek package, Metal included) 2026-10-24 21:52:49 +02:00
Niko1221 3f8f9aeaa6 Translations (de, zh, ja, es, ru, pt, ko) for the 26 new fix-catalog keys: first-run card, DeepSeek note, Speed labels 2026-10-24 21:47:01 +02:00
Niko1221 684881246d DeepSeek Speed label: the default is named by the engine's own rule, integrated / unified-memory GPU (Strix Halo) or cards totalling 16 GiB or less; '(default on this PC)' on the matching choice; test 2026-10-24 21:45:44 +02:00
Niko1221 d44144d7bb English catalog keys for the first-run card, the DeepSeek note and the Speed labels (the locale-gaps test failed on the merged branch); the other seven fix catalogs carry the English text until translated 2026-10-24 21:45:44 +02:00
Niko1221 df7d407742 Merge f-025-fast (Faster is the default on shared-RAM GPUs; ZC warning) into f-0243-rel 2026-10-24 21:40:42 +02:00
Niko1221 0597aaaeb7 launch: an integrated / unified-memory GPU (Strix Halo, gfx1151) is Faster by default, as the engine's --experts auto says: the 'default on this PC' label and the plan mirror 'integrated GPU, or cards totalling 16 GiB or less'; the engine 0.1.50 guard stays; tests 2026-10-24 21:40:29 +02:00
Niko1221 c4ce40bf5e DeepSeek engine: faster mode (5 experts a word) is the default on a GPU that shares the PC's RAM (Strix Halo: decode +11..13%, prompts +5..7% against 6); the 16 GiB rule is unchanged, --experts N wins; rule test rows 2026-10-24 21:40:06 +02:00
Niko1221 add4362139 Merge branch 'f-025-strix' into f-025-fast 2026-10-24 21:39:41 +02:00
Niko1221 ff91bc3729 DeepSeek engine: DS4_STORE_ZC=1 on a GPU that shares the PC's RAM, whose budget the weights already fill, prints one plain line that it will likely fail with a GPU fault and gains nothing (no cap) 2026-10-24 21:36:27 +02:00
Niko1221 67996022cd build_dsv41_windows.bat: configure every time so the build id is the current commit 2026-10-24 21:25:48 +02:00
Niko1221 7988837183 release build scripts: a failed step is a non-zero exit (build_hip_linux.sh), a stale binary or an old configure cannot pass, and strata-dsv41 must carry the release commit (BUILD_ID for a tarball, checked in the binary) 2026-10-24 21:24:53 +02:00
Niko1221 77b1a50541 launch: flow_for reads the settings only when the context has them (a context with only a computer, as test_mixed_vendors passes, raised AttributeError after the Speed step) 2026-10-24 21:18:56 +02:00
Niko1221 902239e7f7 ds4_gg_bench --k512: the two printf strings end in a newline escape (they held literal line breaks) 2026-10-24 20:57:49 +02:00
Niko1221 c1c3eab540 Merge f-026-5070 (in-place chunk 4096 on non-HIP cards under 16 GiB) into f-0243-rel 2026-10-24 20:41:36 +02:00
Niko1221 1fac68d6f2 DeepSeek engine: the experts-read-in-place default prompt chunk is 4096 on non-HIP cards under 16 GiB (was 8192): decode +2% agent / +0.4..9% code (5 pairs on RTX 5070 and RTX 3060, 120 more experts resident), an agent turn of 3K new tokens on 16K takes the same time, a cold 8K prompt 9% slower; DS4_FEEDLOG prints each feed's time; test rows for 12 GB, 15.9 / 16 / 24 GiB, 8 GB, unknown VRAM and HIP 2026-10-24 20:41:08 +02:00
Niko1221 aa4b225237 Merge f-025-strix (gfx11 WMMA expert kernels, iq test) into f-0243-rel 2026-10-24 20:31:03 +02:00
Niko1221 bd71ea81b9 Merge f-0243-pick (prompt-path rule, post-ready warm-up) into f-0243-rel 2026-10-24 20:25:56 +02:00
Niko1221 4d8ff9646f DeepSeek engine tests: the gfx11 / gfx12 matrix-core prompt tiles are forced on and off in ds4_iq_test (launch counted, both against the CPU reference and each other); umaprobe m1/ver probes; test knobs DS4_ZC_COARSE and DS4_STORE_CHUNK_SLOTS for the zero-copy registration 2026-10-24 20:11:27 +02:00
Niko1221 536460c3d8 Copy feedback: one helper (lib/clipboard.ts copyText with a toast); Markdown and Serving use it, lib/copy-text.ts is gone (the toast store is loaded only when a code block is copied) 2026-10-24 20:05:05 +02:00
Niko1221 061a285832 Merge f-0243-gui into f-0243-rel (Chat.tsx and DropZone.tsx: the guifix clipboard helpers kept at the conflicts, copyAndTell stays for Markdown and Serving) 2026-10-24 20:03:44 +02:00
Niko1221 e752c65455 ds4_gg_bench: --k512 calls the AVX-512 rows (iq512x_rows) directly against ggml-cpu's dot, bit for bit, and says whether the CPU and type allow them 2026-10-24 20:02:50 +02:00
Niko1221 dfd1b0f291 DeepSeek engine: the prompt path is chosen from the prompt's length alone (96 tokens and more through the chunked pass, windows of 16 below), one number for every box - the same prompt always takes the same path and gives the same bits (the old per-request timing prediction flipped the logits between runs and took 7 s on a 28-token prompt); the chunked pass is warmed up after the model is ready, on a thread, so the BLAS library's ~3.6 s start is not paid by the first prompt a person types; startup line; DS4_PICK=adaptive, DS4_PICK_MIN=N, DS4_WARM=0 as switches; rule test 2026-10-24 19:56:11 +02:00
Niko1221 8b8f8a10ee Merge f-026-5070 final chunk rule (7bf6e4d3b) into f-0243-rel 2026-10-24 19:15:07 +02:00
Niko1221 7bf6e4d3b7 DeepSeek engine: one default prompt chunk rule, prefill_chunk_default(): 1024 only for a single-card default run where the GPUs hold at least half of the routed experts (4 x R9700; the R9700 worker's gpu_expert_share() from 4cde62a35), 4096 when a GGUF pack's experts are copied into RAM (--mmap-experts off) on a non-HIP card with 10 GiB or more, 8192 for the experts read in place (5070, 3060, 1 x R9700: what a launch on one card already ran), 2048 for a pack that is not a GGUF; real launch logs on the RTX 5070 and the RTX 3060 show no --tp / --threads and 'prompts in chunks of 8192'; one unit test with the 5070, 3060, 5090-class, 1 x and 4 x R9700 cases 2026-10-24 19:14:17 +02:00
Niko1221 56ac21a217 Merge f-026-5070 (prefill chunk default rule) into f-0243-rel 2026-10-24 19:00:37 +02:00
Niko1221 829913ea59 DeepSeek engine: 4096 chunk rule applies to a GGUF pack only when it is copied into RAM (--mmap-experts off) and to an explicit --tp 1 single card; the experts-read-in-place default stays 8192; other packs keep 2048 2026-10-24 19:00:07 +02:00
Niko1221 dba7d0e0ab DeepSeek engine: the prompt chunk of the experts-copied-into-RAM case (2048) and of an explicit --tp 1 single card (1024) becomes 4096 on cards with 10 GiB or more (HIP builds and smaller cards keep theirs; the in-place 8192 is unchanged); startup line; rule test incl. 12/16/24/32 GB, short-VRAM, HIP and max-context cases 2026-10-24 18:49:20 +02:00
Niko1221 535cacb052 Tests never write the owner's real Strata profile: pytest gets scratch LOCALAPPDATA/XDG state and data folders; the host test runner and the native-control test files get a scratch settings file, components folder and app data 2026-10-24 18:43:52 +02:00
Niko1221 67a6ee76f4 Engine bundle cleanup: one function (cleanupAfterInstall) writes the removed and kept lines to the host output, with a test that both reach it 2026-10-24 18:36:50 +02:00
Niko1221 a264168390 One size figure: engine sizes are decimal MB, rounded half up, in the Terminal (strata update) and the app; both tested on the same numbers 2026-10-24 18:14:36 +02:00
Niko1221 e6c5997091 DeepSeek engine: gfx11 WMMA expert kernels for the prompt sets, on by default on gfx1151 (Strix Halo: prompts +22..29% at the same KL), opt-in on other gfx11 parts; STRATA_DS4_HIPK=0 turns them off 2026-10-24 18:13:43 +02:00
Niko1221 7063f3adca Sizes in decimal MB on the first-run card and the update check (one helper); the engine bundle of a per-GPU install is removed after an update only when the same engine is complete in the store (size and SHA-256), else kept with one log line 2026-10-24 18:10:05 +02:00
Niko1221 1c69e23658 Merge f-0243-guifix final head into f-0243-rel 2026-10-24 17:50:44 +02:00
Niko1221 f613637cbd Merge f-026-5070 (experts N|auto, IQ1_M row) into f-0243-rel 2026-10-24 17:50:01 +02:00
Niko1221 4f47009d08 Merge f-0243-fuse into f-0243-rel 2026-10-24 17:48:34 +02:00
Niko1221 94a4a34b05 Merge f-026-qwen into f-0243-rel 2026-10-24 17:46:13 +02:00
Niko1221 8993997e02 Merge f-025-strix into f-0243-rel 2026-10-24 17:46:13 +02:00
Niko1221 9741a61d7b Merge f-0250-tui into f-0243-rel 2026-10-24 17:46:13 +02:00
Niko1221 06f313b019 CPU lane IQ1_M AVX-512 rows: the 64-weight step's scale words by one vpermw and the delta signs by a masked subtract (bit-identical to ggml's dot: 0 of 2.7M rows differ at 1 and 2 tokens on Zen 4 and Sapphire Rapids); +9% on the rows, the 1-token policy is unchanged 2026-10-24 17:41:33 +02:00
Niko1221 236e13b4af Settings update check: a failed check of the app itself says so in one plain line, with the raw error behind Details (it used to show only Control's new-version text) 2026-10-24 17:25:53 +02:00
Niko1221 c893eeebf1 App update: an update job an older host left running is finished when the installer has written uninstall.exe or changed the program file after the job began (the installer keeps the build time as the modified time) 2026-10-24 17:11:55 +02:00
Niko1221 9b4092a94d launch: the Speed choice and --experts need a DeepSeek engine of 0.1.50 or newer (0.1.49.2 does not accept the flag): an older or unknown engine gets no flag, no Speed step and no plan line, and no error; the version comes from the package folder, else --version; tests 2026-10-24 16:47:17 +02:00
Niko1221 3283d2ae49 DeepSeek Speed: offered and passed only when the installed DeepSeek engine is 0.1.50 or newer (an older one ends the start with unknown option --experts) 2026-10-24 16:46:06 +02:00
Niko1221 b79d6930c1 test: the own-GGUF check loads its helper by path (another tests package can already be loaded in a full tools run) 2026-10-24 16:43:46 +02:00
Niko1221 883e21bf2e tests: the app-owned settings list includes the DeepSeek Speed choice 2026-10-24 16:43:31 +02:00
Niko1221 4e4921f63e DeepSeek settings: Speed (Faster / Full quality), default shown by the engine's own 16 GB rule, passed as --experts 5|6 only when chosen 2026-10-24 16:42:29 +02:00
Niko1221 bd7acc0e6d launch: the 16 GiB rule for the Speed label is compared in whole MiB; the catalog's vram_gb is already GiB (nvidia-smi MiB / 1024, AMD bytes / 2**30), so a 16 GB card (16376 MiB) counts as 16 GiB or less; tests with exact MiB values 2026-10-24 16:39:53 +02:00
Niko1221 c4413c0b54 launch: one rule for DeepSeek's speed. Nothing is passed to the engine unless the person chose (its --experts auto decides: 5 at 16 GiB of cards or less); the 'default on this PC' words use the same 16 GiB rule summed over the cards the launch uses, and the summary reads the engine's own 'experts per word: N' line back; tests follow 2026-10-24 16:36:49 +02:00
Niko1221 bc1a3ba2ed launch: DeepSeek Speed choice, Faster (--experts 5, the default on a PC with 16 GB of graphics memory or less) or Full quality (--experts 6): a Speed step in the guided screens, --speed and --full-quality, plain words in --plan and the summary, the quality note; tests, docs 2026-10-24 16:33:51 +02:00
Niko1221 3dd11c6dc1 DeepSeek engine: --experts N (the router's top N, weights renormalised); default 5 where the cards hold 16 GiB or less in all, the model's 6 elsewhere; startup line 'experts per word: 5 (faster mode; full quality: --experts 6)'; DS4_TOPK=N stays as a test override; rule test; DS4_PROF prints the CPU lane's stage split 2026-10-24 16:31:09 +02:00
Niko1221 eb4736c45e DeepSeek engine: on a Windows integrated GPU the experts leave room for the context's later GPU allocations
The Windows commit limit decides how many experts an integrated GPU may hold (place_experts). With AMD driver 26.10.1 the limit
went from 74 GB to 133 GB on a Strix Halo, the engine put 9506 experts (86 GB) on the GPU instead of 8105, every byte of the
GPU's address space was gone, and the 32K prompt's later allocations spilled into RAM: free RAM fell to 0 and the engine died
("rocBLAS: No devices found"). The experts now leave 4 GiB + 0.2 MB a token of context above 16384 (DS4_UMA_GPU_RESERVE_GB sets
it, tests); only an integrated GPU on Windows is affected.

Strix Halo (gfx1151, 2.00 pack, shipped profile), driver 26.10.1:
  32K prompt: killed (min free RAM 0) before, 113.2 tok/s with min free 13.6 GB now; 64K prompt (70000 context): 122.2 tok/s, min free 14.4 GB
  4K / 8K prompt: 62.8 / 83.9 tok/s (64.9 / 86.2 with the 9506 experts: -3%), min free 13.4-13.8 GB
  KL, code segment, 999 positions: 0.1020 (0.1028 before)
A smaller reserve (3 GiB + 0.1 MB a token) gained 0.0% on 4K / 8K / code decode and left 0.9 GB less free at 64K, so it was not taken.
2026-10-24 16:22:28 +02:00
Niko1221 66b43d4a7f Monitor: with All GPUs, a card that cannot report a value is left out and the card that does report is named (35 C . RTX 5070) instead of blanking the figure 2026-10-24 16:09:59 +02:00
Niko1221 91eb05a228 App update: a job an older host left running is finished when the app's program file is newer than the job 2026-10-24 16:07:45 +02:00
Niko1221 4228e6ee11 chat model picker: a downloaded model that is not loaded uses the same two-line row as the API models 2026-10-24 15:59:39 +02:00
Niko1221 9caf39495c Merge f-0243-guifix into f-0243-rel 2026-10-24 15:44:30 +02:00
Niko1221 0c53372a0e Merge f-0250-tui into f-0243-rel 2026-10-24 15:44:30 +02:00
Niko1221 a3e4cb64fd test: serving never reaches the DeepSeek engine check (only launch and setup import it), and 'serve --plan runs no program' means no installer or fetch: a PC's read-only hardware probes (nvidia-smi, Get-CimInstance) no longer fail it 2026-10-24 15:26:07 +02:00
Niko1221 dc2c4229e9 test: launch_engine_first gives the runner a Qwen catalog (a bare Mock answered 'DeepSeek' to the 0.2.4 multi-file question and skipped the engine step); the launch code was right 2026-10-24 15:25:07 +02:00
Niko1221 60d856a18e DeepSeek decode: fewer launches a layer - the int8 roundings of one GEMV call in one launch (identical inputs once) and hc_mix + hc_split in one launch (the token's last mix block runs the split); same arithmetic
R9700 1 card 2.00: 21.30 -> 21.82 tok/s (+2.4%), 4 cards 35.39 -> 37.00 (+4.5%), 3 pairs each, same generated ids. DS4_QUANT_SINGLE=1 / DS4_QUANT_NODEDUP=1 go back to one launch an item / no sharing.
2026-10-24 14:47:48 +02:00
Niko1221 abab458d74 Feed reads: one more try when the first times out or the connection fails (the first wait is 12 s, the second the full 30 s); about one github.com request in 25 hung for 30 s and the first-run card said offline 2026-10-24 14:44:12 +02:00
Niko1221 9cf91f3555 Load of DeepSeek: the start wait is at most the 600 s Control allows 2026-10-24 14:36:14 +02:00
Niko1221 f30d191d54 Load of DeepSeek: the app adds no Flash-Next options (request slots, RAM budget, output clamp, resource profile, memory retry) to the DeepSeek engine and waits as long as its config says 2026-10-24 14:34:47 +02:00
Niko1221 75ee28fd3d Host: the per-model settings routes accept a dotted id; a test walks a dotted model id through list, load and unload 2026-10-24 14:33:40 +02:00
Niko1221 fe2b9f4094 Host: run config file names with a dot (DeepSeek) pass the app's own checks, so the model can be loaded 2026-10-24 14:31:43 +02:00
Niko1221 54768d25b1 test: the own-GGUF check finds tests.terminal when tools/ is run on its own 2026-10-24 14:29:49 +02:00
Niko1221 4499c69681 Installed models: an id with a dot (deepseek-v4.1-flash-ist-das-2.00bpw) is listed; the install finished but the app never showed the model 2026-10-24 14:22:39 +02:00
Niko1221 beeea0abf3 test: the Mac build-tools test follows find_tool's version check 2026-10-24 14:19:31 +02:00
Niko1221 569f7c739d test: the --kv choices check reads the last line of the error (Python 3.12 and 3.13 print the list differently) 2026-10-24 14:18:47 +02:00
Niko1221 ff84ff8164 First-run card: one plain line when the drive for the engine is short on space 2026-10-24 14:14:32 +02:00
Niko1221 61a4b6bd02 First-run card: no dash in the engine size line 2026-10-24 14:14:10 +02:00
Niko1221 192ab03813 DeepSeek-V4.1-Flash in the app: the model picker lists the three official sizes (not the 508 GB multi-card set), an 8K context only, a plain size/speed/disk note that warns and never blocks, no calibration step; the install tool accepts the own-path sizes 2026-10-24 14:11:45 +02:00
Niko1221 16c4da9443 CPU expert rows: 512-bit grid lookups by gather where a gather is cheap; one IQ3 token on the exact 512-bit rows
The 512-bit i-quant rows (IQ2_XXS/XS/S, IQ3_XXS/S) assembled their grid words from scalar loads (~30 uops a 64-value chunk). Where cpu_gather_fast()
(Intel Sapphire Rapids and later) they now load them by one gather, 32-bit for IQ3 and 64-bit for IQ2 (STRATA_IQ_GATHER=1/0 force it; Zen 4 stays scalar:
its gathers are microcoded). The same words are loaded, so every output bit is unchanged. One token of an IQ3_XXS / IQ3_S expert takes the exact 512-bit rows (ggml-cpu's
single-token AVX2 dot bit for bit) with the gather, after a first-use bit check against this ggml; STRATA_NO_IQ512X1=1 turns it off.

Xeon 8468, native_gu_rows (gate+up of an expert), us: IQ3_XXS 1 token 295 -> 219, 2 tokens 462 -> 300, 4 tokens 558 -> 389; IQ3_S 415 -> 305, 603 -> 342, 562 -> 296;
IQ2_S 2 tokens 377 -> 235, 4 tokens 437 -> 327 (one IQ2_S token stays on ggml: 239 against 234). Output checksums identical with and without.
Test iq512_gather_parity (both lookup modes): exact rows == ggml vec_dot at 1-4 tokens, engine rows within float rounding.
2026-10-24 14:09:08 +02:00
Niko1221 9030da09f9 Merge f-026-5070 into f-0243-rel 2026-10-24 14:01:05 +02:00
Niko1221 980c95a6d7 Merge f-0243-store into f-0243-rel 2026-10-24 14:01:04 +02:00
Niko1221 86a751af27 Merge f-025-strix into f-0243-rel 2026-10-24 14:01:04 +02:00
Niko1221 84b712a3d8 Merge f-0243-sm89 into f-0243-rel 2026-10-24 14:01:04 +02:00
Niko1221 db42e59930 First-run card stays until Continue; a processor's own graphics is not counted as a card 2026-10-24 13:58:36 +02:00
Niko1221 c1e80f7952 Markdown: a single newline inside a paragraph is a soft break (a space); two spaces or a backslash at the end of a line keeps a line break (#3818) 2026-10-24 13:55:42 +02:00
Niko1221 ea349eb8b8 Language: 43 interface strings that had no key (/plan /research /new hints, tuning, goal, telemetry, task and picture texts, the new fix texts) now exist in all 8 languages; the four Open hints translate their label; a test fails when a tr() literal or hint has no key (#3479, #3825) 2026-10-24 13:51:18 +02:00
Niko1221 08a5e6b34a packaging smoke: the packaged Terminal carries the installer since 0.2.4, so 'strata install' without a keyboard stops at the first question instead of saying 'no installer'; the check accepts both 2026-10-24 13:48:09 +02:00
Niko1221 353c9f2556 chat: a PDF is added to the chat folder and named in the next message; the file dialog lists PDFs and text on Linux (#3838) 2026-10-24 13:48:06 +02:00
Niko1221 ec03ae0106 First-run engine card: GPU, engine size, one Download button, progress with resume, 'I already have the file' (offline), plain states; host routes /api/engine/first-run, /job and /from-file; install-engine-file job 2026-10-24 13:47:22 +02:00
Niko1221 780a69b41b Expert settings: decimals can be typed in sampling and number fields; Ctrl+V of a screenshot reads clipboard items (#3736, #3748) 2026-10-24 13:47:18 +02:00
Niko1221 c7899f9c89 chat: Copy response and Copy message answer Copied or say they could not (with an older-copy fallback); a typed message keeps its line breaks (#3731, #3720) 2026-10-24 13:45:58 +02:00
Niko1221 e61b7024cf Models page: a neutral placeholder until the engine's first answer, not the bundled Starter (#3740) 2026-10-24 13:45:12 +02:00
Niko1221 25abbdd90c Models page: the chosen conversation context and cache format count in the memory estimate (#3832) 2026-10-24 13:43:25 +02:00
Niko1221 f750086183 Models page: the chosen conversation context and cache format count in the memory estimate (#3832) 2026-10-24 13:43:02 +02:00
Niko1221 31adebc8b7 strata components install-file: install an engine package the person downloaded (signature, every file, fit, test start, switch; no network); the engine switch is shared with the download path 2026-10-24 13:42:41 +02:00
Niko1221 4ccfd7ca30 Windows AMD/Intel cards: real memory size from the DirectX adapter list, load and memory in use from the Windows GPU counters, virtual monitors no longer listed as cards (#3727, #3836) 2026-10-24 13:40:59 +02:00
Niko1221 b750d757f6 CUDA engine selector: sm_MN machine code also covers a card sm_MK of the same major version with K >= N (CUDA cubin compatibility), so an RTX 40 card (sm_89) takes an sm_86 package; tests for the Qwen and DeepSeek packages; DeepSeek card text says RTX 30, 40 and 50 2026-10-24 13:40:27 +02:00
Niko1221 d9d033be86 launch: at 80 columns the model list keeps its verdict column (fits, tight, too big and the recommendation) instead of cutting it off; test 2026-10-24 13:40:22 +02:00
Niko1221 9c375ccb7d launch: setup's DeepSeek engine upgrade lines (the failure one 'A newer DeepSeek engine is available but could not be installed') are shown on the setup path too; test 2026-10-24 13:38:49 +02:00
chicco83 90a7aeea51 Setup wizard: show the VRAM of AMD cards on Windows from the engine's list when the host probe reports 0 (#3727, PR #3798 by chicco83) 2026-10-24 13:38:38 +02:00
Niko1221 d6c1936021 setup: a source build on Windows needs Visual Studio 2022 (2019 cannot compile the C++20 sources) and the stop says so, with the ready-made engine as the way around (#3566); tests follow 2026-10-24 13:37:35 +02:00
Niko1221 fb3bae20c6 setup: the list of installed packages remembers which system wrote it, so a .venv shared by Windows and Linux installs again instead of saying 'already installed' (#3634); test 2026-10-24 13:36:55 +02:00
Niko1221 1d20d3e2b0 chat: reopening a chat restores its workspace and the turn runs in that folder (#3754) 2026-10-24 13:36:35 +02:00
Niko1221 13a4153be4 copy-text: load the toast store only when copying, so the Markdown renderer still renders without a browser 2026-10-24 13:35:56 +02:00
Niko1221 ea897551ee folder picker: a PowerShell that ends before its window opens (security software) falls back to typing the path (#3831) 2026-10-24 13:35:54 +02:00
Niko1221 ed0c8939aa setup: a download that gets no data for two minutes says so with the percentage and the one next step, instead of a line that never moves (#3567); test 2026-10-24 13:35:43 +02:00
Niko1221 47ccec7c6a Delete a model: files Strata downloaded and packs it made go with it even in another models folder; the person's own files stay (#3716) 2026-10-24 13:35:04 +02:00
Niko1221 aef053e353 launch: the Access screen's help names --show-key and --api-key, so a person can see or set a key of their own (#3713 #3951); test 2026-10-24 13:34:18 +02:00
Niko1221 c22e5e7633 missing files: name the engine program in words once, not as component:strata-engine (#3900) 2026-10-24 13:34:11 +02:00
Niko1221 27cbbfe6d6 launch: the start waits longer for a big model by itself (10 minutes, about 8 s more per GB above 40 GB, at most 30), the server's own engine wait follows, and the runtime accepts start waits up to 1800 s (#3949 #3459). Own fix instead of PR #3817: that one needs people to set STRATA_ENGINE_READY_S; tests 2026-10-24 13:34:04 +02:00
Niko1221 97c31168aa chat model picker lists downloaded optimized models that are not loaded yet (#3814) 2026-10-24 13:33:56 +02:00
Niko1221 5faf357b49 Chat: Copy response/message/code say Copied or Couldn't copy and fall back to the older copy route (#3731); a pasted screenshot is also read from clipboard items (#3748); the user bubble keeps its line breaks (#3720) 2026-10-24 13:33:17 +02:00
Niko1221 77284b5e33 update: the test start runs the new engine with a short context and no picture reader (#3842), and an update keeps the CUDA build that runs here (#3835); launch: the summary names the engine log (#3656) and a tuned model says how to measure again (#3874); tests 2026-10-24 13:32:44 +02:00
Niko1221 4b49729a1e monitor: read Windows disk counters from one long-lived PowerShell instead of a cold one every 3 s (#3939) 2026-10-24 13:32:09 +02:00
Niko1221 014fd6b5db App update: log lines for every update step, an alert when the app came back as the old version, and Update engine works before the native snapshot has connected 2026-10-24 13:30:13 +02:00
Niko1221 5bfce4bea0 setup: the build tools come from the virtualenv first and an older system cmake (below the 3.24 CMakeLists.txt needs) counts as missing, so pip installs the right one (#3822); tests 2026-10-24 13:28:02 +02:00
Niko1221 15fdb69563 In-app update: a finished app update is no longer shown as an interrupted operation; a test update feed used once is not remembered as the feed for later checks
After 'Update Strata' installed a new app version, the host that asked for it was ended by the installer, so the job stayed saved as running and the next start turned it into a red 'Interrupted operation' alert in Settings > Update and Diagnostics. The job now records the app version it started from, and a start under a different version marks it done ('Strata is updated to X').

strata update and the app's check reused the feed used last, including a loopback address or a folder from a test of the update feed; every later check on that PC then said 'No internet connection'. Only an https address is remembered now.
2026-10-24 13:27:28 +02:00
Niko1221 ad9bb57ddc own model files: a qwen4exp file set that is no tested build but has the layout of one is used as that model with a plain notice, and setup keeps a complete GGUF of another size or hash with a warning instead of stopping (#3651 #3777); tests 2026-10-24 13:27:08 +02:00
Niko1221 b18700c8f7 Adaptive tier: the engine chooses the asynchronous tier by measurement (arena, one CUDA GPU)
Without --adapt-async on the command line, the expert arena on one CUDA GPU (--serve, adaptive tier on, no --batch, no layer
split, no pipelined windows, no peer or remote tier) starts on the blocking tier, ignores 64 steady decode windows, measures 64,
and when the tier's wait is at least 5% of a window runs 64 windows on the asynchronous tier, going back (for the session) only if
that was more than 2% slower per token. One log line at each switch. --adapt-async 0|1 and STRATA_ADAPT_ASYNC=0 keep their meaning;
the identity gates run with --adapt-every 100000 and never reach it. Decision logic and its unit test are header-only
(adapt_async_default.hpp, adapt_async_default_test).
2026-10-24 13:27:00 +02:00
Niko1221 c02a0b8b0d UPDATE.bat / update.sh: an engine that stayed older than the minimum is said as 'the engine was NOT updated' with one next step, not '[ok] Strata is updated' (#3653); test 2026-10-24 13:08:46 +02:00
demetree 7a03fc9414 strata connect: scale the app configs' compaction point with the server's context 2026-10-24 13:06:11 +02:00
Niko1221 d924beffe5 setup: a Mac with an Intel (Rosetta) Python is still an Apple Silicon Mac; the chip name and hw.optional.arm64 decide, one plain line says the Python is an Intel build (#3894); test 2026-10-24 13:06:00 +02:00
Niko1221 bdaa93edf0 launch: a setup option error ('setup.py: error: argument --kv: invalid choice') is the cause shown, not only 'could not be installed'; setup accepts --kv fp16 as DOCKER.md says (#3785); tests 2026-10-24 13:05:30 +02:00
Niko1221 1736b558bd setup: a NVIDIA driver older than CUDA 13 needs (528 to 579) runs the CUDA 12 engine by itself and says so in one line instead of stopping and naming a flag strata launch does not have (#3843); tests 2026-10-24 13:04:44 +02:00
Niko1221 0cd44323e4 Windows: a card taken out of the PC no longer decides the engine; only the display adapters Windows reports as present count when two makes are in the registry (#3892); test 2026-10-24 13:03:56 +02:00
Niko1221 e3eeb93972 DeepSeek engine: the RAM store's background fill is opt-in (DS4_STORE_BG=1): ready 9 s sooner but the first prompt waits for the same SSD and decode measured 3.7% lower 2026-10-24 12:40:19 +02:00
Niko1221 58f2acc05b DeepSeek engine: the RAM store's fill beyond the first 48 GB is read in the background after the model is ready (hottest first, behind the model's own reads); DS4_STORE_BG=0 reads it all first 2026-10-24 12:33:10 +02:00
Niko1221 6fcde7e305 DeepSeek CPU lane: compute the store hits while the misses are read, each missed expert as its read lands (DS4_MISS_SPLIT=0 restores the wait-for-all order); ExpertStore::ensure split into begin / wait_slots 2026-10-24 12:29:56 +02:00
Niko1221 b3c86c783f DeepSeek engine: the automatic RAM store may take every routed expert, not 48 GB at most, where the free RAM holds them (1 R9700 + 251 GB, 2.00 set: 12.6 -> 21.4 tokens/s); free - 12 GB and the explicit sizes unchanged 2026-10-24 11:58:35 +02:00
Niko1221 f1e41cdee2 DeepSeek engine: the automatic RAM store is sized again after the weights are on the GPU, and keeps room for the context's buffers, so a unified-memory PC (Strix Halo) keeps 12 GB free; explicit sizes unchanged 2026-10-24 11:28:09 +02:00
Niko1221 7dd5cf18e6 release notes 0.2.4.2: the picture-file and already-have-files fixes (#3927, #3917, #3914); no environment variable in the conversations line 2026-10-24 10:52:25 +02:00
Niko1221 8aacdbc7a0 release notes 0.2.4.2: the list of new files 2026-10-24 10:51:27 +02:00
Niko1221 3eabee6049 test: the restyled READMEs state the app version through their download buttons (Strata-0.2.3-...), not the old '0.2.3 · ' header line 2026-10-24 10:51:13 +02:00
Niko1221 c896f73934 release notes 0.2.4.2: the DeepSeek engine packages are 0.1.49.2, the same number the binary prints and the Control has 2026-10-24 10:36:34 +02:00
Niko1221 93a91a2767 README translations (zh-CN, ja, de, fr, es, pt-BR): brought up to the restyled English README (download buttons per system, section menu, coffee badge, plain-word DeepSeek table, shorter Using it) 2026-10-24 10:36:09 +02:00
Niko1221 12939aa64d README: restyled to the owner's GitHub layout (download buttons per system, a section menu and a coffee badge at the top, plain-word DeepSeek table with the measured 0.2.4 speeds, shorter Using it, jargon cut); DEEPSEEK41.md: 4x R9700 speed updated to the 0.2.4 measurements 2026-10-24 10:29:36 +02:00
Niko1221 dfa3f4025c DeepSeek: pack_gguf takes an automatically found mmproj only when its projector is deepseek41v (a Qwen mmproj in the shared models folder is ignored, no error: #3927 #3917); files the person already has are recognised as DeepSeek by the pinned shard names in the guided screen too, not by the generic GGUF check that called the UD n-gram shard unreadable (#3914); tests 2026-10-24 10:27:57 +02:00
Niko1221 ac08a0820d release notes 0.2.4.2: conversations and a shared system prompt are kept on disk 2026-10-24 10:12:46 +02:00
Niko1221 05041e5db0 Merge branch 'f-024-on' into f-0242-rel 2026-10-24 10:12:41 +02:00
Niko1221 3157c38c68 DeepSeek engine: an installed package older than the signed feed's is updated by itself on launch and setup (fetched and checked, put next to the old one, started once, then switched to and the old one removed; one plain line and the old one keeps running on a failure; silent without a network; never for --dsv41-package or STRATA_DSV41_EXE); tests; release notes 2026-10-24 09:35:15 +02:00
Niko1221 d2d198a492 release notes 0.2.4.2: the DeepSeek engine 0.1.49.1 (faster on AMD Zen 1/2 CPUs, answers unchanged) and how to get it 2026-10-24 09:30:42 +02:00
Niko1221 8c9bd206ce Merge branch 'f-0243-iq1m' into f-0242-rel 2026-10-24 09:29:52 +02:00
Niko1221 46bddbbb74 dsv41: BMI2-off ggml-cpu applies only to the DeepSeek engine build (STRATA_BUILD_DSV41); ds4_gg_bench --dump FILE to compare two ggml builds bit for bit (IQ1_S/IQ1_M with and without BMI2: identical at 1,2,4,8,16 tokens) 2026-10-24 09:14:06 +02:00
Niko1221 6953d896d0 dsv41 CPU lane: IQ1_M uses Strata's exact rows from one token (ggml's IQ1_M dot runs at 1.8 GB/s on AMD Zen 2: _pdep_u64 is microcoded there); portable ggml-cpu build has BMI2 off (IQ1_S 2.3 -> 37 GB/s, IQ1_M 1.8 -> 32 GB/s on an EPYC 7452) 2026-10-24 09:09:23 +02:00
Niko1221 baa8f616de DeepSeek-V4.1: conversation slots and the shared-start checkpoint on by default (STRATA_DS4_SLOT_KEEP=0 / STRATA_DS4_AUTO_PIN=0 turn them off) 2026-10-24 08:25:17 +02:00
Niko1221 ac669f199d Terminal 0.2.4.2 / Control 0.1.49.2: version bump for the DeepSeek launch and install fixes (app stays 0.2.3, engine packages unchanged); release notes 2026-10-24 06:29:42 +02:00
Niko1221 0003604859 launch: the second sentence of 'The model could not be installed.' starts with a capital; test 2026-10-24 06:29:42 +02:00
Niko1221 e168b76b8c Merge branch 'f-0242' into f-0242-rel 2026-10-24 06:28:20 +02:00
Niko1221 f53664c159 launch: a setup message that ends with a full stop does not get a second one ('engine..'); test 2026-10-24 05:43:54 +02:00
Niko1221 44f8c334de model list: the DeepSeek disk check counts only what is still to be downloaded (files already in the models folder, also under the first release's names, are done), plus the room for the pack; tests 2026-10-24 05:41:50 +02:00
Niko1221 a22d557eee launch --plan: a DeepSeek plan no longer promises a Strata engine download; test 2026-10-24 05:39:30 +02:00
Niko1221 51d23fdd1e P2003 findings: DeepSeek setup says in one line when GitHub cannot be reached (a 10 s check before any download, not minutes of retries) and no longer fetches or test-starts the Strata engine, llama.cpp source or CUDA libraries it does not use; install.sh is one function called on the last line (a half-downloaded file runs nothing) and every documented one-liner is 'curl ... -o install.sh && sh install.sh' so a failed curl cannot look like success; tests 2026-10-24 05:37:41 +02:00
Niko1221 a62ca064a3 launch: the DeepSeek engine package that runs is the newest of the running Terminal's own install (STRATA_INSTALL_DIR, else its own folder) and the install the model's settings came from (the first remembered install, which can be an older Terminal folder or another checkout); the running Terminal's own on a tie; the choice is logged; versions compare as numbers; tests 2026-10-24 05:09:58 +02:00
Niko1221 8183c6b671 docs: RTX 3060 speed with the release defaults (8 to 10 tokens/s, 149 tokens/s at 8K); DSpark stays off on 12 GB cards and adds 2-13% when turned on; README x7, DEEPSEEK41.md, release notes 2026-10-24 04:59:46 +02:00
Niko1221 c33fa7ba6d Terminal 0.2.4.1 / Control 0.1.49.1: version bump for the DeepSeek-setup hotfix (app stays 0.2.3, engine packages unchanged); release notes 2026-10-24 04:36:57 +02:00
Niko1221 b7958c044f Packaged Terminal: tools/dsv41/pack_gguf.py, smoke.py, tune.py (and strata_inspect, strata_bench, nvfp4_local_pack) are in the bundle; a test reads every tools/*.py that setup.py, strata_cli and strata_control run by path and fails when one is missing from the bundle, and starts the DeepSeek scripts from the bundle alone 2026-10-24 04:36:57 +02:00
Niko1221 06e51dfaef test: the failed-update reason is capitalised where it joins the sentence (follows 00a2cfa07) 2026-10-24 04:13:07 +02:00
Niko1221 96ca4383c7 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-24 03:17:28 +02:00
Niko1221 2ecdc2e511 launch: an installed DeepSeek model whose config names a gone engine, with no package in this install, is not 'already installed': setup runs (feed download, or one plain line with the next step) instead of the start failing on a missing file; tests 2026-10-24 02:58:01 +02:00
Niko1221 5353097c90 DeepSeek quality numbers from the final-tip scores (H100, 4,995 positions): 2.00 KL 0.170 ppl +15.5% (code +8.6%, agent +5.3%), 2.25 KL 0.107 ppl +7.3% (code +3.2%, agent +2.2%); README x7, DEEPSEEK41.md, release notes, setup and fit text 2026-10-24 02:57:14 +02:00
Niko1221 9011bd107a launch: DeepSeek runs on this install's own engine package, not the exe path baked into an installed model's config (which can come from another Strata install on the computer or name a package folder an update replaced); tests 2026-10-24 02:56:16 +02:00
Niko1221 52b104b1ac IQ1_M quality from the H100 (KL 0.217, ppl 3.80, +21%; code +8.9%, agent +10.2%, weaker on plain prose) in setup, the fit text, README x7, DEEPSEEK41.md and the release notes 2026-10-24 02:37:08 +02:00
Niko1221 00a2cfa079 launch: a reason joined after 'The engine could not be installed.' starts with a capital; test 2026-10-24 02:36:17 +02:00
Niko1221 6eaf32075c Merge f-024-maya2 (6c6267f33) into f-024-rel: Windows batch submit, flow pump, Stop during a long prompt, lane-free, opt-in switches (all in src/dsv41)
Five conflicts, resolved keeping every f-024-rel behaviour (slots pin, long-prompt skip, HIPK, ZC fix) and maya2's:
- ds4_main.cpp, request struct: both fields kept (pin=N from f-024-slots and dspark_cum from maya2).
- ds4_main.cpp, request parser: both keys kept (pin, dspark_cum/spec_cum).
- ds4_main.cpp, prompt feed loop: slots' pin_at (the read stops at N for the shared-start checkpoint 3) and maya2's Stop handling (checkpoint 1 before the prompt, set_cancel, the stop check at the top of each feed) combined: the loop checks Stop first, then sizes the feed with the pin cut (n is no longer const).
- ds4_model.cu, forward chunk loop (two places): the skip's 'run.runs(k, il)' (an early chunk skipped past the yoco layer) kept, and maya2's halt_now() / halt_end(di, err) check added after it, so a skipped layer does no halt work.
win_submit (ds4_kernels.cu) is compiled to a no-op outside _WIN32 (#if defined(_WIN32) ... else (void) s), so it is already out of the Linux CUDA and Linux HIP builds.
2026-10-24 02:35:40 +02:00
Niko1221 698a87d999 Merge branch 'f-024-tui' into f-024-rel 2026-10-24 02:34:57 +02:00
Niko1221 6c6267f33c dsv41: Maya-port defaults from the owner-PC A/Bs (ON: win_submit, flow pump, Stop, lane-free; OFF with switch: timer, Engram readers, unbuffered Engram, cold prefetch, DSpark cum)
Owner PC (RTX 5070 12 GB, Ryzen 5 7600, official 2.00 pack, shipped profile-2.00.bin, 3 alternating pairs, medians, off = every switch off):
 4K prompt, off 92.64 tok/s: win_submit 99.12 (+7.0%, 3/3 pairs), flow pump 97.37 (+5.1%, 3/3), 64 Engram readers 96.64 (+4.3%, 3/3),
   unbuffered Engram 85.56 (-7.6%, 0/3), all on 86.84 (-6.3%).
 code decode, off 4.51: timer 4.51 (1/3), 64 readers 4.41 (2/3), unbuffered 4.67 (2/3, one pair 3.83), all 4.57.
 mmap decode, off 1.95: cold-page prefetch 1.71 (-12%, 0/3).
 Cold (no profile) agreed on win_submit and flow pump (+4.5%, +2.7%, 2/2) and showed the timer +12%, readers + unbuffered +22% on decode (2/2, off arm noisy).
 Identity: --vram-experts 0, off vs all IDENTICAL (4K prompt, code decode); text equal to off in every warm arm.
 Stop during a long prompt (H100, 8K and 16K): STOP answered in 0.12 / 0.16 s, the conversation before the prompt kept, the next request identical.
 DSpark cum cut: H100 code 20.13 tok/s off vs 19.60 at 0.5, no gain.
ON: DS4_WIN_SUBMIT, DS4_FLOW_PUMP_WAIT, Stop, STRATA_DS4_LANE_FREE (a no-op on the physical-core lane).
OFF, switch turns it on: DS4_TIMER_MS=1, DS4_ENGRAM_THREADS=64, DS4_EG_DIRECT=1, DS4_IQ_PF_COLD=1, STRATA_DS4_DSPARK_CUM=0.5.
2026-10-24 02:29:57 +02:00
Niko1221 cec75af6ae launch: the Save screen shows DeepSeek's own tuning length (5-10 / 10-15 min) like the Tuning screen; walk test: save text, run paths, error paths 2026-10-24 02:29:04 +02:00
Niko1221 8ac2a8385e tests: the 1.75 bpw set also runs with --tp 1 on 4 AMD cards; release notes: set up again on 4+ AMD cards 2026-10-24 02:24:31 +02:00
Niko1221 894dc24c6c Merge branch 'f-024-mgpu' into f-024-rel 2026-10-24 02:23:57 +02:00
Niko1221 c4dcde7145 dsv41 iq test: IQ1_S set gate_up (f16) tolerance 1.5e-2 (data-order dependent, 7.1e-3..1.08e-2, same on the older build); fix a stale comment in ds4_model.cu (IQ1_S/IQ1_M/IQ4_NL have device kernels) 2026-10-24 02:18:45 +02:00
Niko1221 67d5f44bf8 dsv41 plan: four or more AMD cards run the IST-DAS sets with --tp 1 (8K prompt 370 against 196 tokens a second on 4 x R9700) 2026-10-24 02:17:25 +02:00
Niko1221 6a35eb6c5f release notes: three sets everywhere, the DeepSeek engine is separate, Mac Control 0.1.49 2026-10-24 02:15:32 +02:00
Niko1221 12d0a4a051 Merge f-dsv41-tune (c5e0e1482) into f-024-rel: the third set IST-DAS 1.75 bpw (IQ1_M) in setup, fit and the launch list
One conflict, docs/DEEPSEEK41.md (both sides edited the three-set paragraph and the file table). Resolution: f-024-rel's text kept (IQ1_M first, the model-card sentence, 533 GB for all three, the 'not a model to pick' Engram row), with the setup worker's sentence that setup recommends 1.75 where 2.00 is tight in memory or disk added to the 1.75 entry.
2026-10-24 02:09:11 +02:00
Niko1221 c5e0e14823 DeepSeek: IST-DAS 1.75 bpw (IQ1_M), the smallest official set, as model ist-das-1.75bpw: pins at ISTA-DASLab cb02422c (the 9 earlier files are unchanged there), fit numbers, setup recommends it where 2.00 is tight in memory or disk (warn, never refuse), catalog/TUI text, docs, tests 2026-10-24 02:08:43 +02:00
Niko1221 9bb00346a2 release notes: R9700 prompt numbers from the r97 command (4 cards 8K 505, 32K 818; kernels +36% / +49%) 2026-10-24 02:04:13 +02:00
Niko1221 672612ac1c DeepSeek tables: the IQ1_M (1.75 bpw, ist-das-1.75bpw) row first, 239 GB; quality 'being measured' plus ISTA-DASLab's model-card figure; README + 6 translations, DEEPSEEK41.md, release notes (placeholder until the KL is in) 2026-10-24 02:03:56 +02:00
Niko1221 f4e7685ccf Merge f-024-rel into f-024-tui 2026-10-24 01:59:28 +02:00
Niko1221 267a2f4462 launch: a Runner that cannot be built is shown on the run screen as a plain failure instead of crashing the screens; tests: launch TUI walk (every screen, every key, Qwen and DeepSeek, 80x24 and wide, run paths, error paths) 2026-10-24 01:59:28 +02:00
Niko1221 eec705cc98 release notes: the prompt-skip agreement at the final tip, 99.5% of 799 positions 2026-10-24 01:58:45 +02:00
Niko1221 b0d5f6e57c Merge branch 'f-024-slots' into f-024-rel 2026-10-24 01:52:42 +02:00
Niko1221 f2413d77e9 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-24 01:52:27 +02:00
Niko1221 4f8770d13e release notes: R9700 prompt numbers with the new kernels (4 cards 8K about 320, 32K about 750) 2026-10-24 01:52:08 +02:00
Niko1221 984f52c827 Merge branch 'f-024-rel' into f-dsv41-tune 2026-10-24 01:51:58 +02:00
Niko1221 aebdc298f9 Merge f-024-hipk (cdf8b98f5) into f-024-rel: gfx12 WMMA i-quant prompt tiles for DeepSeek V4.1 (HIP only, default on for gfx12, STRATA_DS4_HIPK=0 off)
One conflict, src/dsv41/ds4_model.cu (forward_chunks): f-024-rel has the long-prompt skip (kv_only layers), f-024-hipk added DS4_PROF stage marks around the same lines.
Resolution: the f-024-rel code kept as it is, and hipk's three prof_mark(d, 5/6/7) calls added at the same places (before the q/kv gemms, before the snapshot, after the kv_only return). They are no-ops with DS4_PROF unset.
2026-10-24 01:51:58 +02:00
Niko1221 fd5823ab83 Merge branch 'f-dsv41-cuk-zcfix' into f-024-rel 2026-10-24 01:48:40 +02:00
Niko1221 d50da4e3a1 release notes: the run-to-run difference is near-tied words, not only expert placement 2026-10-24 01:48:18 +02:00
Niko1221 624a226d13 dsv41 store zero-copy: clear the device's zc_tab entry when VRAM takes an expert over (a stale pointer into a reused slot made zc_stage copy another expert's blob into the plan: garbage, NaN routing, illegal access) 2026-10-24 01:44:40 +02:00
Niko1221 fa3252ad5d dsv41 serve: with slots off the loop is as before (no anchor copy, same continuation test on an empty engine) 2026-10-24 01:43:35 +02:00
Niko1221 9797056e4d Merge branch 'f-024-rel' into f-024-slots
# Conflicts:
#	serve/test_dsv41.py
2026-10-24 01:43:15 +02:00
Niko1221 e2361d80d2 DeepSeek-V4.1: slot keep and auto-pin default off (switches STRATA_DS4_SLOT_KEEP=1, STRATA_DS4_AUTO_PIN=1) 2026-10-24 01:39:15 +02:00
Niko1221 afd24ea6ae launch: the last lines suggest 'strata launch --stop' (and --save-preset NAME to keep the settings) when no preset was saved; --preset "<name>" only when one exists; tests 2026-10-24 01:27:07 +02:00
Niko1221 4ebba26767 pack_gguf: an Engram file ('engram' or 'deepseek41-engram') is never taken for a model shard or for the model's metadata; real header fixture and tests; setup test: the first release's files are packed as they are with no shard download 2026-10-24 01:25:23 +02:00
Niko1221 598372c3fd setup: repin the DeepSeek 2.00/2.25 sets to ISTA-DASLab's final files (flat repository 0d8f49fb, IQ2_XXS = 2.00 bpw, IQ2_XS = 2.25 bpw, DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf); the first release's local files (model-2.00bpw-..., engram-IQ4_NL.gguf, in 2.00bpw/ folders) stay accepted by their old size and SHA-256 and are never downloaded again; tests 2026-10-24 01:24:14 +02:00
Niko1221 2e477ecd40 release notes: the 99.75% is agreement between skip on and off over 799 positions 2026-10-24 01:21:59 +02:00
Niko1221 0e121c7a13 DeepSeek table: the IQ4_NL Engram table row (README and 6 translations, DEEPSEEK41.md, release notes); notes: long-prompt skip and the 32,768 thinking budget 2026-10-24 01:21:32 +02:00
Niko1221 415e618085 Merge branch 'f-dsv41-maya' into f-024-rel 2026-10-24 01:20:31 +02:00
Niko1221 dfe29bce61 serve: DeepSeek-V4.1 thinks at most 32768 tokens when neither the request nor the config names a budget
Same default as Maya #363. For the deepseek41 family only: Service.reasoning_budget() answers 32768 when the request has
no reasoning_budget_tokens / thinking_budget_tokens and the config has no reasoning_budget_tokens at all (a config value,
0 included, wins; a request's 0 turns it off). The existing wrap-up (#123) closes the thinking at the budget, and the
answer-room clamp still cuts it to leave the answer its share of max_tokens. Thinking off and other families unchanged.
Tests in serve/test_dsv41.py: the default, a request's 0 and own value, the config's value and 0, another family, and a
5000-token cap that is wrapped up and answered. serve suite: 1027 tests OK (15 skipped).
2026-10-24 01:20:04 +02:00
Niko1221 f51b80c222 README: R9700 numbers in the DeepSeek block (11 tokens/s on one card, 4x about 31, 55 to 69 with DSpark); the same block in the six translated READMEs; release notes: one-card R9700 row 2026-10-24 01:19:04 +02:00
Niko1221 17c585c402 Merge branch 'f-024-hsamsg' into f-024-rel 2026-10-24 01:18:51 +02:00
Niko1221 41e804d93c launch: the hsa_exception sentence ignores hsaKmt lines, and a real cause in the same log (out of memory, no permission on the card) still wins; tests for both 2026-10-24 01:18:16 +02:00
Niko1221 524ff90e1a docs: the final DeepSeek quants (ISTA-DASLab repository) in the release notes and DEEPSEEK41.md 2026-10-24 01:17:00 +02:00
Niko1221 86834afedc Merge branch 'f-024-hsamsg' into f-024-rel 2026-10-24 01:16:46 +02:00
Niko1221 fa77717a57 README: DeepSeek V4.1-Flash (ISTA-DASLab IQ2_XXS / IQ2_XS) in the model list, Terminal only 2026-10-24 01:16:46 +02:00
Niko1221 60d847ba33 launch: a GPU runtime start failure (hsa_exception) says the card was not ready, with a next step; the runtime text stays in the log 2026-10-24 01:16:19 +02:00
Niko1221 a3f76dfd63 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-24 01:13:51 +02:00
Niko1221 b4418e30df serve: DeepSeek-V4.1 with a short max_tokens (48-1023) caps the thinking at half of it, so a text block always arrives
/v1/messages and /v1/chat/completions with max_tokens 200-600 and the thinking on spent all of max_tokens on the
thinking: the 0.1.45 answer room starts at max_tokens 1024 and a client that gave no budget of its own got no text block.
For the deepseek41 family only, with no budget from the client or the config, the thinking is now capped at max_tokens / 2
for max_tokens from 48 up to 1023 (the existing wrap-up, </think> and the answer follow in the other half). A client's
own reasoning_budget_tokens / budget_tokens wins, "answer_room": false turns it off, thinking off is unchanged, other
families (Qwen) are untouched. Maya #363 closes the thinking in the engine instead; the server's #123 budget already does
that for every engine, so only the short-cap gap needed closing.

Tests (serve/test_dsv41.py SmallCap, fake engine that thinks without end): max_tokens 200, 400 and 600 on both APIs return a
non-empty text block with the thinking wrapped up at half; thinking off answers directly; a client budget wins; a tool
call under a short cap still parses; another family keeps the old behaviour. serve suite: 1025 tests OK (15 skipped).
2026-10-24 01:11:34 +02:00
Niko1221 8a7102ec4d dsv41: a long prompt skips layers 20-39 on all but its last 2,668 rows (STRATA_DS4_PROMPT_SKIP, on by default)
Layers 20-39 of DeepSeek-V4.1-Flash read a prompt's earlier tokens only through layer 20's compressed KV and index
keys, and each looks back one 128-token window more than the layer before. So for the answer and for the rings the
decode reads they need their own work only on the last (40 - 20) x 127 + 128 = 2,668 rows. A prompt of ~3,700 or more
new tokens on one GPU now runs its earlier chunks through layers 0-19 plus layer 20's kv_only part (the hyper-connection
pre-norm, the compressor latent, the index keys, the compressed KV) and nothing of the later layers' attention or
experts; the chunk the cut falls in runs layers 20-39 from the cut row on; later chunks run every layer. The expert flow
plans nothing for the skipped (chunk, layer)s. STRATA_DS4_PROMPT_SKIP: 1 (default) the skip, 2 only the last 128 rows
(DeepSeek's own serving; answers shift a little), 0 off. Same approach as Maya #318.

H100 emulation (64 GB cgroup, 12 GiB VRAM, store 41 GB, 3.6 GB/s SSD cap): 16K prompt 69.3 s -> 51.5 s (236 -> 318
tok/s, two alternating pairs), 4K 32.5 s -> 30.7 s; 800 teacher-forced positions after a 4,200-token skipped prompt: KL
0.00029 against the full work, top-1 against the text 88.11% against 87.98%, ppl 1.5110 against 1.5111.

--score gets DS4_SCORE_PREFIX=N (the segments joined, the first N tokens read as one prompt with the last row's logits,
the rest scored) so the skip can be scored; tools/dsv41/test_prompt_skip.py runs it against a pack when one is named.
The Maya notice file now says later Maya changes are covered by the same notice.
2026-10-24 01:11:34 +02:00
Niko1221 be26cb7995 DeepSeek-V4.1: slots on by default (2 GB cap, one week), docs 2026-10-24 01:10:21 +02:00
Niko1221 736a4136e1 serve: DeepSeek-V4.1 engine gets the slot switch and a slot folder in Strata's data folder (other families untouched) 2026-10-24 01:09:06 +02:00
Niko1221 1d31e2a1e4 tools/dsv41/slot_keep_test.py: two agents taking turns, fresh reads vs shared start vs slots vs restart vs folder cap 2026-10-24 00:59:52 +02:00
Niko1221 b8f9c63ae7 serve: DeepSeek-V4.1 marks the system prompt and tools as the shared start (pin=N) so a sub-agent goes back to it
Only the deepseek41 family; STRATA_DS4_AUTO_PIN=0 switches it off, a client's own strata_prefix wins, Qwen is untouched.
2026-10-24 00:59:52 +02:00
Niko1221 cdc3a26577 dsv41: keep a conversation another request displaces and take it back at its checkpoints (STRATA_DS4_SLOT_KEEP), shared-start checkpoint (pin=N)
A prompt that does not continue the conversation in the model first sets that one aside (position, window rings,
compressor partial groups, each KV source's compressed KV and index keys, its checkpoints), then takes back the kept
conversation it does continue, in RAM or from the slot folder, also across restarts. The latest checkpoint the prompt
still agrees with is used (end of the last prompt, before it, the anchor of the first prompt of a run, and the shared
start the server marks with pin=N). The .meta format and the prefix matching are in ds4_slotmeta.hpp with a GPU-free
test (ds4_slot_test). STRATA_DS4_REUSE=0 reads every request from token 0 (reference for tests). Off unless
STRATA_DS4_SLOT_KEEP=1.
2026-10-24 00:59:52 +02:00
Niko1221 cdf8b98f54 Merge f-024-rel d914211e6 into f-024-hipk 2026-10-24 00:58:00 +02:00
Niko1221 5be4964c2b launch for DeepSeek: the tune text says what it really does and its result lists what was timed (or that nothing could be compared); the plan names an installed DeepSeek engine ('installed, 0.1.49') instead of saying setup will download it; preset test; tests 2026-10-24 00:56:08 +02:00
Niko1221 d914211e64 release notes 0.2.4: the manual DeepSeek engine command 2026-10-24 00:41:47 +02:00
Niko1221 71c674dcdb DeepSeek engine by hand: --dsv41-package FILE (setup and strata launch) or a file put in engine-dsv41/; the package is checked (DeepSeek engine for this system, signed manifest, every file's hash) before it is installed; plain steps in the missing-engine message and docs; tests 2026-10-24 00:41:28 +02:00
Niko1221 0d38517888 dsv41 prompts: expert copies keep going while the host waits for the routing (flow_pump on the busy stream); DS4_FLOW_PUMP_WAIT=0 as before 2026-10-24 00:40:48 +02:00
Niko1221 e51e84e22a dsv41 CPU lane: two cores left free beside the host thread when the lane would take every logical CPU; STRATA_DS4_LANE_FREE=0 as before 2026-10-24 00:40:42 +02:00
Niko1221 cffd240834 dsv41 serve: Stop ends a long prompt's reading at a layer/chunk and keeps the conversation before it (set_cancel / cancelled) 2026-10-24 00:40:09 +02:00
Niko1221 062fd4a95e release notes 0.2.4: the DeepSeek engine is a separate download (automatic and by hand), Qwen engines unchanged 2026-10-24 00:38:34 +02:00
Niko1221 f73ae5a62e dsv41 DSpark: cumulative cut on the drafts (product of confidences >= 0.5); STRATA_DS4_DSPARK_CUM, request key dspark_cum, 0 off 2026-10-24 00:38:21 +02:00
Niko1221 e6e14f2714 dsv41 CPU lane: prefetch only the IQ experts' pages not in RAM (os::prefetch_cold); DS4_IQ_PF_ALL=1 as before 2026-10-24 00:38:01 +02:00
Niko1221 a6cbc3621e dsv41 Windows: Engram rows read unbuffered from their unmapped file; DS4_EG_DIRECT=0 as before, DS4_EG_CHECK compares against a buffered read 2026-10-24 00:37:44 +02:00
Niko1221 aee5db5eb1 dsv41 Windows: 64 Engram reader threads (each ReadFile is synchronous, so the threads are the queue depth); DS4_ENGRAM_THREADS overrides 2026-10-24 00:37:14 +02:00
Niko1221 611c1701d3 dsv41 Windows: 1 ms timer resolution while the engine runs; DS4_TIMER_MS=0 leaves it 2026-10-24 00:37:04 +02:00
Niko1221 5ec845f6bc dsv41 Windows: queued GPU work submitted at once on NVIDIA too (win_submit), after every flag hand-off; DS4_WIN_SUBMIT=0 turns it off 2026-10-24 00:36:40 +02:00
Niko1221 a23b5f209c dsv41: gfx12 WMMA expert tiles on by default (STRATA_DS4_HIPK=0 turns them off) 2026-10-24 00:35:47 +02:00
Niko1221 94413c7057 dsv41: gfx12 WMMA prompt tiles for the i-quant expert sets (STRATA_DS4_HIPK=1, default off) 2026-10-24 00:35:38 +02:00
Niko1221 8fda835383 dsv41: DS4_PROF expert-kernel timers on both set paths 2026-10-24 00:35:37 +02:00
Niko1221 4278304914 dsv41: finer DS4_PROF for the chunked prompt (stages, expert kernels by type) 2026-10-24 00:35:37 +02:00
Niko1221 75c57be831 release notes 0.2.4: Thanks names IST-DAS and Unsloth only 2026-10-24 00:34:09 +02:00
Niko1221 2dbb263b02 launch/setup for DeepSeek: setup installs the screens packages (textual) and launch goes on in plain lines without them; the Tuning screen shows the DeepSeek tuner's real length and what it tries; the Advanced list knows strata-dsv41's own flags; the model menu counts the disk (download + preparation) in 'fits'; docs: tuning and the separate engine package; tests 2026-10-24 00:18:48 +02:00
Niko1221 955c709d96 Merge branch 'f-024-dsml' into f-024-rel 2026-10-24 00:12:55 +02:00
Niko1221 70efffb5b6 serve: a DeepSeek-V4.1 pack that ships the V4-Flash chat template (tool_calls, no space) is rendered with the V4.1 template, so tool calls come back as real calls; STRATA_DEBUG prints the raw ids; test 2026-10-24 00:07:13 +02:00
Niko1221 027691b33d release notes 0.2.4: RTX 3060 row 6-9 tokens/s 2026-10-24 00:06:34 +02:00
Niko1221 00ce92068a setup: banner names the chosen family; the user's own home (root's included) is allowed as a data folder; a missing DeepSeek engine package is said plainly (supported cards), the developer hint only with --verbose; tests 2026-10-24 00:05:11 +02:00
Niko1221 18022ceaf3 launch: the plan says 'drafting DSpark' for DeepSeek, not MTP; test 2026-10-23 23:48:59 +02:00
Niko1221 1904c0261b dsv41 tune: with the drafter on, also try a draft block of 3 against the pack's own 2026-10-23 23:47:48 +02:00
Niko1221 642e76948e setup: shared DeepSeek files (drafter, Engram) are also looked for two folders above the set's own folder (strata launch passes .../off/2.00bpw); test 2026-10-23 23:47:30 +02:00
Niko1221 1d0a8051d5 setup: DeepSeek's engine comes from the signed feed's dsv41 member (strata_control/dsv41_engine.py: same download, resume and verification as an engine update) into engine-dsv41/, beside the engine; the flat-zip route is dropped; tests 2026-10-23 23:46:46 +02:00
Niko1221 9f04da3afc Merge branch 'f-024-rel' into f-dsv41-tune 2026-10-23 23:37:40 +02:00
Niko1221 134f24f199 setup: DeepSeek's engine is its own package (engine-dsv41/ beside the engine, never inside it): downloaded with resume and a SHA-256 check from the --prebuilt base, found before the old all-in-one place; launch's note says so; tests 2026-10-23 23:37:19 +02:00
Niko1221 6655733d5e feed: a separate dsv41 member for the DeepSeek engine packages (Feed.dsv41, pack-engine --prefix, build-feed --dsv41); Qwen engine list untouched; tests 2026-10-23 23:35:00 +02:00
Niko1221 f9f297bf90 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-23 23:33:46 +02:00
Niko1221 ec5b2bdc4e dsv41 win: --dspark-block N (at most N drafts a round) 2026-10-23 23:33:40 +02:00
Niko1221 a2aeb4ecb1 launch: a real short tune for DeepSeek-V4.1-Flash (tools/dsv41/tune.py: the drafter on or off, CPU threads; gains over 5% only, written into the launch config; any failure leaves the standard settings with a plain line); the launch no longer forces --tune skip for it; tests 2026-10-23 23:31:51 +02:00
Niko1221 3feaa2aa3f Merge branch 'f-dsv41-setup' into f-dsv41-tune 2026-10-23 23:30:35 +02:00
Niko1221 8ea7fef827 Terminal 0.2.4 / Control and engine 0.1.49: version bump for the DeepSeek V4.1-Flash release (app stays 0.2.3, Mac stays 0.1.48); release notes 2026-10-23 23:29:00 +02:00
Niko1221 36a94857a1 dsv41_fit: remove a duplicated constants block; one unit (tokens a second) 2026-10-23 23:28:05 +02:00
Niko1221 28ce98bf16 Merge f-dsv41-rel (0be62ccad) into v0.2.3.1: DeepSeek V4.1-Flash on top of the released 0.2.3 line
f-dsv41-rel was based on the 0.2.2-era tree (merge-base 1f727f4b5), so the merge brings it onto 7ddc17a38 without dropping 0.2.3 / 0.2.3.1.
Conflict resolutions (0.2.3.1's behaviour kept everywhere that is not DeepSeek):
- packaging/package_engine.py: build() keeps 0.2.3.1's check_reader / allow_stale_vision / allow_no_reader and adds require_dsv41; both the
  picture-reader check and the strata-dsv41 presence check run; the CLI call passes both sets of arguments.
- packaging/ENGINE-COMPONENT.md: 0.2.3.1's cuda13 / cuda12 rows (vision built per variant), with strata-dsv41.exe added to the cuda13 contents.
- serve/server.py: the parser is self.parser_cls (OutputParser, or the DeepSeek DSML parser) and keeps 0.2.3.1's declared_only argument.
- strata_cli/launch/cmd.py: the --plan output prints 0.2.3.1's drafting line and then the DeepSeek placement lines.
- strata_cli/launch/runner.py: plan() returns 0.2.3.1's dict with "drafting", plus "placement" for multi-file (DeepSeek) models;
  drafting_plan_line() kept; dsv41_engine() and placement() added.
2026-10-23 23:27:29 +02:00
Niko1221 0be62ccad7 Merge f-dsv41-setup 86ba33845 into f-dsv41-rel (serve: DeepSeek turn without </think> returns its text as content; pack recipe noise token/markov rank; exec bits in packages; console-safe smoke) 2026-10-23 23:00:32 +02:00
Niko1221 86ba33845f serve: a DeepSeek turn that stops without </think> and wrote no answer returns its text as the answer (streaming ends with it as content; the non-streaming OpenAI and Anthropic replies drop the identical reasoning); tests 2026-10-23 23:00:05 +02:00
Niko1221 6f507fb210 packaging: strata-dsv41 and strata-vision get execute bits in the package and in the tar (ELF header decides when packed on Windows); setup restores them on an engine copied without modes 2026-10-23 22:49:29 +02:00
Niko1221 1a7b4ae236 setup: pack recipe states --dspark-noise-token 128799 and --dspark-markov-rank 256; plain wording (nothing to download, one unit: tokens a second, the one-time SHA-256 check and its time, the remembered-folder line) 2026-10-23 22:48:21 +02:00
Niko1221 a9dd554e13 setup: the smoke reply is made printable for the console code page before it is shown 2026-10-23 21:54:52 +02:00
Niko1221 3724818db5 Merge f-dsv41-setup into f-dsv41-rel (setup installs the official IST-DAS 2.00/2.25 sets; strata-dsv41 in the engine packages) 2026-10-23 21:54:42 +02:00
Niko1221 e9546cb564 Merge f-dsv41-cpul e5f536a6c into f-dsv41-rel (AVX-512 one-token rows for Zen 4/5, bit-identical to ggml, per-type start-up self-check, chosen by the detected CPU) 2026-10-23 21:10:53 +02:00
Niko1221 8de8d90287 dsv41 smoke: utf-8 output (the reply has DeepSeek's special tokens; a Windows console code page failed on them) 2026-10-23 21:09:46 +02:00
Niko1221 e5f536a6cb dsv41 cpu lane: AVX-512 rows from 1 token on Zen 4 as well (IQ2_XXS, IQ2_XS; IQ3_XXS and IQ1_M from 2), measured 2026-10-23 21:02:09 +02:00
Niko1221 b6cc7e681b setup: DeepSeek family line names IST-DAS for the official 2-bit GGUFs and Unsloth for UD-Q8_K_XL; says who made the files of the chosen version 2026-10-23 20:41:48 +02:00
Niko1221 f2323d876d setup test + docs: no --prefill-chunk, one plain line about the engine's own chunked prompt 2026-10-23 20:37:24 +02:00
Niko1221 10af5bb1e4 setup: drop --prefill-chunk (the chunked prompt is the engine's default now); doc line 2026-10-23 20:37:14 +02:00
Niko1221 237eb667d3 Merge branch 'f-dsv41-rel' into f-dsv41-setup 2026-10-23 20:37:12 +02:00
Niko1221 d6f18babe9 Merge f-dsv41-r97 fb321d62c into f-dsv41-rel (layer-major chunked prompt as the engine default) 2026-10-23 20:12:55 +02:00
Niko1221 1747c8e538 dsv41 cpu lane: one-token AVX-512 rows - sign masks by popcount (IQ2_XXS, IQ2_XS, IQ3_XXS), IQ1_M AVX-512 rows with pdep indices; chosen per CPU (Zen 5 from 1 token) 2026-10-23 20:09:59 +02:00
Niko1221 f1cb66d09e dsv41: strata-dsv41 --version and a 'ds4: build <commit>' line with --verbose (commit from git at configure, a BUILD_ID file in an archive, or -DSTRATA_BUILD_ID) 2026-10-23 20:03:24 +02:00
Niko1221 4eedb86b4f Merge f-dsv41-h100 (store pre-grow: register the whole arena before the first kernel on Linux, fixes the --ssd-cache zero-copy hang) 2026-10-23 20:01:41 +02:00
Niko1221 e917cf5b37 Merge branch 'f-dsv41-rel' into f-dsv41-setup 2026-10-23 20:01:11 +02:00
Niko1221 c506238ce2 setup: DeepSeek official sets start with --dspark-conf 0.65 (H100 measured default) 2026-10-23 20:00:50 +02:00
Niko1221 c35710e388 dsv41: store arena pre-grown by default on Linux when the GPU maps it 2026-10-23 19:59:29 +02:00
Niko1221 fb321d62cb dsv41: the layer-major (chunked) prompt is the default on every run: chunks of 1024 for the single-card default run, staging ring 16 MB where the cards hold most of the experts 2026-10-23 19:59:13 +02:00
Niko1221 f7d7072841 dsv41: DS4_STORE_PREGROW=1 (map the whole store arena before the first kernel; cuMemHostRegister mid-decode hangs on Linux) 2026-10-23 19:58:22 +02:00
Niko1221 1c1005952a Merge f-dsv41-cuk 86bdfbd0d (HIP keeps the release grids and kernels). One conflict: the identical IQ1_M code-table HIP guard (9c46ab821 vs cuk); took cuk's wording. 2026-10-23 19:52:52 +02:00
Niko1221 41856f6594 dsv41: staging ring 64 MB when the experts are within 30% of fitting the cards (R9700 x4 plain 2.00: ratio 1.16) 2026-10-23 19:45:21 +02:00
Niko1221 0beae0bca6 dsv41: the store fill after a chunked prompt reaches the store's whole capacity, not only its allocated slots 2026-10-23 19:44:54 +02:00
Niko1221 9469412db2 setup: DeepSeek official sets start with --prefill-chunk 1024 (layer-major prompt path, measured 231 tok/s at 8K on a 12 GB PC) 2026-10-23 19:44:40 +02:00
Niko1221 a345d9f8cf Merge branch 'f-dsv41-rel' into f-dsv41-setup 2026-10-23 19:44:39 +02:00
Niko1221 a82477069b Merge branch 'f-dsv41-pkg' into f-dsv41-setup 2026-10-23 19:41:53 +02:00
Niko1221 81dd215de9 setup: the two official IST-DAS DeepSeek-V4.1-Flash sets (2.00 / 2.25 bpw) as sizes: pinned files, shared Engram + DSpark, plain pack recipe, shipped profiles, pack-info and a 16-token text check, --dspark auto; fit words from the measured numbers 2026-10-23 19:41:49 +02:00
Niko1221 4b5f8d89f2 tests + ENGINE-COMPONENT.md: strata-dsv41 in the engine package and the --require-dsv41 refusal 2026-10-23 19:39:24 +02:00
Niko1221 dc352b839b dsv41: say how the chunk staging ring was sized 2026-10-23 19:36:53 +02:00
Niko1221 e838e2beca Merge f-dsv41-cpul 02b07b686 into f-dsv41-rel (exact multi-row CPU-lane rows for IQ types, bit-identical to ggml-cpu, self-check per type, DS4_GG_IQ256=0 switch) 2026-10-23 19:35:58 +02:00
Niko1221 6053a43841 packaging: build_dsv41_windows.bat and build_dsv41_cuda.sh build strata-dsv41 for the NVIDIA release packages (sm_86 + sm_120 fat) 2026-10-23 19:35:51 +02:00
Niko1221 faaf82acd7 build_hip_linux.sh: build strata-dsv41 (own build tree) and ship it in the rx9070 package; DSV41=1|0 overrides, --require-dsv41 on the package 2026-10-23 19:35:51 +02:00
Niko1221 8fee63f4f0 package_engine: ship strata-dsv41 beside the engine (picked up like the picture readers); --require-dsv41 refuses a package or a --verify without it 2026-10-23 19:34:59 +02:00
Niko1221 7519b1c75c pack_gguf: warn loudly when --skip-tensors / --dense-override are used on an official IST-DAS set (engram_k/q/wkv present): the rebuilt pack loads but gives garbage text; the plain recipe is right 2026-10-23 19:27:17 +02:00
Niko1221 dbf3bb2e64 dsv41: the layer-major run's RAM is given back before the cache catch-up and the store fill 2026-10-23 19:24:01 +02:00
Niko1221 86bdfbd0da dsv41 iq: HIP keeps the release grids and kernels (no persistent work-item grid there) 2026-10-23 19:23:13 +02:00
Niko1221 796dbb77b4 dsv41 iq: the IQ1_M code-table copy is CUDA only (kIq1mCodes does not exist on HIP; found by building the HIP-guarded paths under nvcc with -DSTRATA_USE_HIP) 2026-10-23 19:17:14 +02:00
Niko1221 96a47b79d5 dsv41: after a chunked prompt the RAM store is filled with the prompt's hottest CPU-lane experts (a window prompt fills it through the CPU lane; the chunk path streams past it) 2026-10-23 19:14:44 +02:00
Niko1221 9c46ab821a dsv41 iq: kIq1mCodes is CUDA-only, so its use in load_tab_fast is under #if !STRATA_USE_HIP (HIP failed 'undeclared identifier kIq1mCodes': non-dependent names are looked up in the dead constexpr branch) 2026-10-23 19:13:07 +02:00
Niko1221 12595032e7 Merge f-dsv41-cuk a3d955201 into f-dsv41-rel (persistent grids only for codebooks >= 8 KiB: fixes 2.5x slowdown with 1 active expert) 2026-10-23 19:06:12 +02:00
Niko1221 a3d9552015 dsv41 iq test: golden compare also covers dynamic-count window groups (5/1/3 entries, ne_max 5; 9 groups with 4 live) 2026-10-23 19:05:14 +02:00
Niko1221 01220edb67 dsv41 iq test: DS4_BENCH_WIN (window path / DSpark verify bench: n entries a group) 2026-10-23 19:01:30 +02:00
Niko1221 02b07b686a dsv41 cpu lane: AVX-512 rows chosen by the detected CPU (Zen 5 family 0x1A: IQ2_XXS and IQ3_XXS; Zen 4 0x19: IQ2_XXS; others none), prompt chunks of 7+ tokens only 2026-10-23 18:51:19 +02:00
Niko1221 a70b006929 dsv41 iq (CUDA): persistent work-item grids only for codebooks of 8 KiB+ (IQ1_M/IQ1_S/IQ2_S), lazy codebook load, small-codebook types keep the release grids and kernels (no regression with few active experts) 2026-10-23 18:49:14 +02:00
Niko1221 4cca3fac20 dsv41: --spec-eval honours --dspark off 2026-10-23 18:44:21 +02:00
Niko1221 995ff61f3c dsv41 cpu lane: AVX-512 rows by default for IQ2_XXS prompt chunks (7+ tokens), measured on Zen 4 2026-10-23 18:43:28 +02:00
Niko1221 f5fc88c599 dsv41 iq (CUDA): token and light kernels take work items (group, row tile) round a persistent grid (any number of active groups shares all blocks), IQ1_M codes from a generated table (ds4_iq1m_codes.inc), persistence only for codebooks of 8 KiB+; many-row light sets keep the release kernel; test: --bench-tiles, DS4_BENCH_ACTIVE 2026-10-23 18:41:55 +02:00
Niko1221 428950b96c Merge catch-up: f-dsv41-win fd19f9d6b, f-dsv41-h100 c0f944e12, f-dsv41-cuk 2026-10-23 18:35:43 +02:00
Niko1221 0553ba6c3f dsv41 iq (CUDA): light sets (DSpark verify, short prompts) take the persistent grid and the IQ1_M code table for experts of <= 4 rows, 4-block occupancy kept for 8+ rows; test: --bench-light 2026-10-23 18:16:11 +02:00
Niko1221 fd19f9d6bf dsv41 win: store off the lookahead by default 2026-10-23 18:09:27 +02:00
Niko1221 b43b0bb518 dsv41: kEntMax declared before Ent 2026-10-23 18:06:26 +02:00
Niko1221 d2411af9fd dsv41: i-quant CPU-lane entries take a whole window's tokens of an expert (cap 16, was 8) for the exact multi-token rows; the chunk staging ring is 64 MB where the cards hold 90% of the experts (decode after a chunked prompt back to the windowed path's speed) 2026-10-23 18:04:51 +02:00
Niko1221 49e35e8bb7 dsv41 cpu lane: exact AVX-512 rows (iq512x_rows, 1-6 tokens and prompt tiles), opt-in DS4_GG_IQ512=1 until measured on Zen 4 2026-10-23 18:04:12 +02:00
Niko1221 c25724a222 Merge branch 'f-dsv41-cpul' into f-dsv41-r97 2026-10-23 18:01:40 +02:00
Niko1221 d689e6080d Merge f-dsv41-cuk into f-dsv41-rel (per-device light-set scratch and tensor-core check, faster IQ1_M/IQ2_S/IQ3_XXS token kernels) 2026-10-23 17:59:37 +02:00
Niko1221 cf4f8befe7 dsv41 cpu lane: take the AVX2 rows by ggml-cpu's own AVX2/FMA/F16C answer (MSVC builds have no __builtin_cpu_supports) 2026-10-23 17:59:03 +02:00
Niko1221 1cace5c864 dsv41 iq (CUDA): per-device light-set scratch and tensor-core check (stream 0 is one handle on every device: the scratch of the first GPU was reused on the second), block/sub-block loop form in the token kernels; test: --devices (in-process multi-GPU, bit-compared) 2026-10-23 17:58:59 +02:00
Niko1221 7e9b0dcc25 dsv41 cpu lane: exact AVX2 rows for 7..64+ tokens an expert (the prompt path): decode once, u8 activations, no per-token sign; IQ1_M too
T tokens in tiles of 16 and rows in groups of 8 (activation slice and accumulators in L1): 7.7-9 ns per block and token on IQ2/IQ3 rows against
28-60 for ggml's dot (x3.6-6.6), IQ1_M 16 ns (x1.8); bit-identical to ggml (ds4_gg_bench --exact at 7, 8, 16, 33, 64 tokens, all six types).
gg_cpu_gate_up takes up to 64 tokens a call (and any number in pieces); ds4_gg_bench --expert times the whole expert pipeline.
2026-10-23 17:57:53 +02:00
Niko1221 0b48e23fa5 dsv41 win: --dspark auto (not loaded when the experts do not fit the GPU), --ids @file, h100sim 2026-10-23 17:57:34 +02:00
Niko1221 f6e4cd6899 dsv41: chunked prompt for a single card (--prefill-chunk N), the expert cache learns from chunks and catches up after a prompt, chunk buffers fall back by halving, CLI feeds a whole prompt
The layer-major chunked prompt path only ran with --threads on / --tp > 1. --prefill-chunk N (N > 0) now enables it on one card too (kChunkedPromptDefault stays false until the decode gate is signed off). A chunk's routing bumps the layers' heat and CPU-lane hits, and cache_settle() runs the swap rule in rounds after the prompt so the decode that follows finds the hot experts in VRAM (8K prompt: 1699 swaps, 5.6 s). If the chunk buffers do not fit, the chunk is halved down to 256 rows, then windows of 16 with one line saying so. ds4_main: --window 0 (default) feeds a whole prompt when the chunk path is on.
2026-10-23 17:47:43 +02:00
Niko1221 c0f944e127 dsv41: DS4_PRED_DUMP / DS4_XIN_DUMP (router logits + projected router input; router inputs for the expert-similarity scan) 2026-10-23 17:44:34 +02:00
Niko1221 6aba354c16 dsv41 cpu lane: the exact AVX2 rows on by default from 2 tokens (1 on IQ3_S, 3 on IQ1_M), 5-6 tokens in one pass, IQ1_M rows rewritten
The rows are ggml-cpu's vec_dot bit for bit at any token count, so the default cannot change an answer; a one-time per-type check against the
ggml-cpu build in use turns them off if they ever differ (DS4_GG_IQ256=0 never, =1 always).  EX rows: 5 and 6 tokens in one decode pass
(were 4+rest), independent integer chains for 1-2 tokens, IQ1_M: delta dot from per-call pair sums of the activation, group signs from one
shifted load.  The main engine's iq256_rows hash is unchlanged (phaseA-tests/cpul/hash_iq256.cpp).
2026-10-23 17:42:52 +02:00
Niko1221 71abb6c163 dsv41 iq (CUDA): persistent token kernels (codebook loaded once a block) and IQ1_M decode through a 4096-entry code table (delta sign folded in): bit-identical, IQ1_M layer -15% sm_120 / -30% sm_86 2026-10-23 17:39:02 +02:00
Niko1221 cf45533cf6 dsv41 cpu lane: AVX2 multi-token i-quant rows in ggml's float order (bit-identical to ggml-cpu at 1..8 tokens), IQ1_M rows
The main engine's iq256_* rows keep their float order (EX=false, the default; hash of their outputs on all types and 1..8 tokens is
identical before/after, phaseA-tests/cpul/hash_iq256.cpp).  The new iq256x_rows (EX=true) sums as ggml-cpu does - block scale f16(d)*y.d, the
format constant after the sum, ggml's hsum - so a token's dot no longer depends on how many tokens share the pass (the greedy ids of a
DSpark run changed with DS4_GG_IQ256=1 because the old order differed in the last bits).  IQ1_M (29), single-token in ggml, gets a
multi-token AVX2 row.  ds4_gg_bench --exact: 0 differing bits for IQ1_M, IQ2_XXS/XS/S, IQ3_XXS/S at 1..8 tokens.
2026-10-23 17:28:19 +02:00
Niko1221 95994c45c5 dsv41 iq test: --bench-big (>=640 MiB working set, rotated experts: DRAM rates), --dump/--cmp (bit-exact golden compare incl. real shapes and grouped path) 2026-10-23 17:18:21 +02:00
Niko1221 ea081e3312 dsv41 rel: one IQ1_M implementation in ds4_iq.cu (the textual merge kept both; port's kernels stay), win's dequant_rows (chunked-prompt dense dequant) kept and takes IQ1_S too; strix add4 dropped with win's IQ1_M decode (port's decode has no __vadd4) 2026-10-23 17:06:08 +02:00
Niko1221 8cfe988383 Merge f-dsv41-strix into f-dsv41-rel (HIP build fixes: portable add4 for the IQ1_M decode, strata_core link for the DXGI budget, HIP build scripts, umaprobe)
No conflicts. The strix add4 edit applies to the merged IQ1_M decode; no __vadd4 is left in ds4_iq.cu.
2026-10-23 17:02:44 +02:00
Niko1221 e5c70bbaa0 Merge f-dsv41-h100 into f-dsv41-rel (--profile-run, DS4_TRACE on the chunked path, --spec-eval) 2026-10-23 17:02:30 +02:00
Niko1221 c70ae83d4c Merge f-dsv41-port into f-dsv41-rel (release integration)
Conflict decisions (win side = f-dsv41-win, port side = f-dsv41-port):
- ds4_iq.cu: kept port. Its IQ1_S/IQ1_M/IQ4_NL kernels are verified bit-exact on HIP gfx1201, sm_86 and sm_120; win's
  IQ1_M table load and supported() list are a subset (IQ1_S, IQ4_NL added).
- ds4_kernels.cu/.cuh: kept port's engram_dequant fmt int (0 MXFP8, 1 Q6_K, 2 IQ4_NL) and its iq4nl_value, replacing
  win's q6k flag and iq4nl_value; same arithmetic.
- ds4_model.cu: kept port's eg_fmt vector and its Engram type check (unknown type exits 2 instead of segfaulting);
  win's eg_q6k is gone, the duplicate IQ4_NL row branch from the textual merge is removed.
- ds4_packinfo.cpp: kept port (device types derived from iq::supported()).
- iq_expert_test.cpp: kept port (hip_or_cuda.h); win's tests/dsv41/hip_as_cuda.h removed (nothing else used it).
  The IQ1_M test-d placement is the same bytes in both versions.
- CMakeLists.txt: kept port's test-language block (HIP, CUDA or CXX for the CPU-only build); win's
  _strata_dsv41_test_lang variable dropped.
- pack_gguf.py: kept port's type table (IQ1_S/IQ1_M comment) and combined both missing-Engram checks
  (has_part from win's split Engram layout AND port's dense-override names).
2026-10-23 17:02:22 +02:00
Niko1221 f232cbfed6 dsv41 win: store test covers cancel and shrink; shrink finalizes read slots 2026-10-23 16:59:01 +02:00
Niko1221 08224f228f dsv41 win: priority SSD queue with cancellation (demand before guesses), partial GPU visibility, store shrink on RAM pressure, predsim 2026-10-23 16:53:02 +02:00
Niko1221 c8ead78419 dsv41 win: store init last, zc off by default, flow reads the store, release notes 2026-10-23 16:37:01 +02:00
Niko1221 55e1a53cfd dsv41: --spec-eval and DS4_ROUND_LOG (DSpark acceptance per round, every draft verified) 2026-10-23 16:32:59 +02:00
Niko1221 8da42765dd dsv41: DS4_TRACE also records the chunked prompt path's picks 2026-10-23 16:29:22 +02:00
Niko1221 75c9f2d81e dsv41: profile-run accepted without --ids 2026-10-23 16:25:51 +02:00
Niko1221 2e61fef4d1 dsv41 win: store auto size, profile preload, decode staging zc_tab (T=1), release plumbing 2026-10-23 16:21:59 +02:00
Niko1221 fb141be3e0 dsv41: profile-run string fix 2026-10-23 16:21:19 +02:00
Niko1221 9051f31306 dsv41: --profile-run mode (feed id segments without logits, expert profile saved as it goes) 2026-10-23 16:20:57 +02:00
Niko1221 17bc654b6d dsv41 strix: umaprobe (UMA budget, hipMalloc reach, host registration, zero-copy read bandwidth), build script 2026-10-23 16:10:12 +02:00
Niko1221 084e654bcb dsv41 on Windows HIP (gfx1151): build script for ROCm 10.2 + MSVC, Linux HIP build script, portable per-byte add for the IQ1_M decode (no __vadd4 on HIP), strata_core link for the DXGI-budgeted cudaMemGetInfo 2026-10-23 15:38:17 +02:00
Niko1221 7d2307deb0 dsv41 win: the store's slots in blob layout, GPU-visible (zero-copy mapped) with a PCIe staging copy for decode picks, node-free reserved arena 2026-10-23 15:33:54 +02:00
Niko1221 226d73b45b tools/dsv41/engram_fold.py --gate-kv-from: make the ple table the only variable, and the answer it gave
Deriving their engram.gate from their k*q and transposing their wkv scored 0.409 then 2.996 KL - both my arithmetic, not
their data (their k/q/wkv are stored transposed and the gate is not k*q in the engine's layout).  --gate-kv-from <pack>
copies those two tensors from a pack that is known good, so a pack can carry their ple table and nothing else of theirs
differ.  That is how the comparison the plan asks for was finally made (numbers in EACH-PC-PLAN/R9700-BOX.md, step 2):
their 110.6 GB IQ4_NL table scores KL 0.16787 / ppl 3.627 against 0.16502 / 3.588 for our 203 GB MXFP8 one - +0.003 KL,
inside this engine's run-to-run placement noise - while saving 92 GB of disk.

Also fixes the fold's reads: Gguf's tensor offsets are relative to the shard's data section (pack_gguf adds data_start),
so the fold must add it too; without that every tensor came out as NaN.
2026-10-23 15:10:00 +02:00
Niko1221 52543768c7 dsv41: one runtime header for the tests on both backends (HIP or CUDA), so a CUDA box needs no shim on -I
tests/dsv41/{iq,tq}_expert_test.cpp included <hip/hip_runtime.h> unconditionally, which no CUDA toolkit ships - every CUDA
box had to pass a hand-written shim on -I (found on l620 and z440, 2026-10-23, and reported as the one thing worth
upstreaming).  include/strata/dsv41/hip_or_cuda.h now provides the handful of HIP names those tests use on a HIP build and
maps them onto the CUDA runtime otherwise (hipMalloc/Free/Memcpy, device sync, the error type and the events).

Verified on r9700 that the HIP build is untouched: incremental build rc=0, ds4_iq_test --type 29 ALL OK, ds4_tq_test ALL OK.
2026-10-23 15:00:19 +02:00
Niko1221 3826885f4b tools/dsv41/engram_fold.py: transpose their Engram layout, but the fold's reader is still wrong
Their engram_k/q/wkv are transposed against the engine's layout (wkv [6144, 25600] against [25600, 6144], k/q [5120, 4]
against a [4, 5120] gate), so the fold now transposes and writes the engine's shape.  The pack builds (their IQ4_NL table,
all files present) and runs without crashing, but it scores KL 0.40915 - and that is this tool's bug, not their data: a
direct read of their BF16 engram_k through Gguf's offsets already comes out as NaN, while the engine reads the same tensors
through a pack fine, so the fault is in how this code turns an offset into bytes (the Q5_K dequant inherits it).

Do not quote 0.409 as the IQ4_NL-vs-MXFP8 comparison; that measurement is still open.  Next step is to read the tensors the
way pack_gguf writes them into tensors.txt and check one row against the engine's own dump.
2026-10-23 14:53:02 +02:00
Niko1221 e7b3f15240 dsv41 win: RAM expert store (unbuffered reads, policy, lookahead fill), Windows direct reads, dense dequant for the chunked prompt, SMT option, store test 2026-10-23 14:44:51 +02:00
Niko1221 7ddc17a38f tests: the old-launcher test expects the 0.2.3.1 version line 2026-10-23 14:37:21 +02:00
Niko1221 d62ff4a608 tools/dsv41: engram_fold.py (their k*q gate and wkv from their own shards) and pack_gguf accepts an override for the Engram tensors
Their sets keep the Engram as three per-layer tensors (blk.N.engram_k/q/wkv) while the engine wants engram.gate (k*q
folded) and engram.wkv; their engram-IQ4_NL.gguf carries only the ple table, so a pack built around their Engram stopped at
'blk.1.engram_kv.weight: missing'.  engram_fold.py folds the gate and renames the kv into a dense_requant-style folder, and
pack_gguf now treats a tensor a --dense-override supplies as present (it checked the quant's shards and the Engram set only).

Result: packs/ista_2.00_native - 48784 tensors, their IQ4_NL table, Engram layers 1,14, 2896 Engram reads for a 32-token
prompt, no crash.

NOT yet a measure of their table: it scored KL 0.40915 / ppl 4.769 against 0.16502 / 3.588 for the same set with our MXFP8
table, because their engram_k/q/wkv are stored TRANSPOSED relative to the engine's layout (their engram_wkv is [6144, 25600]
against [25600, 6144]; their k/q [5120, 4] against the [4, 5120] gate).  Copying the bytes as they are yields a valid pack
with the wrong data.  engram_fold.py still needs the transpose (gate in float; the Q5_K kv dequantised, transposed, written
BF16) - Phase 1.1's 'load their layout as published'.
2026-10-23 14:36:12 +02:00
Niko1221 488dd8459c dsv41: --pack-info's device-type list is derived from iq::supported() instead of copied
It was a hand-kept copy of the supported() switch and fell behind when IQ1_S, IQ1_M and IQ4_NL got kernels: the 2.00 set's
60 IQ1_M expert tensors (59.5 GB) were reported as 'no GPU kernel ... the CPU lane takes them' while the kernels were in the
same binary and ds4_iq_test passed them.  That is exactly the line an operator trusts before a download, so it now walks
ids 0..63, asks iq::name()/iq::supported(), and adds MXFP4 and the trellis types by hand (they are not in that switch).
Verified: packs/ista_2.00bpw_9e2528f and packs/h225 both report 'no GPU kernel: none'.  Also fixes the intake's eval call
(session16.sh takes the pack directory name, not a path - the 2.00 score had failed with an empty runs/s16_<path>.tsv).
2026-10-23 14:21:38 +02:00
Niko1221 b2d2cf22eb dsv41: the Engram reader takes IST-DAS's IQ4_NL table, and a pack with an unknown Engram type stops cleanly instead of segfaulting
Found and fixed on p620, the only box with their engram-IQ4_NL.gguf (110.6 GB). Before this, a pack built around their
file loaded for 38 s and then died in Model::Impl::eg_issue at the first token (3 of 3 runs, addr2line 0x6d3450).

- Model::Impl::load: one guard line and exit(2) for an Engram tensor whose type the reader does not handle, naming the
  layer and the type (verified with a pack retyped to IQ4_XS, rc 2; a missing embed tensor also exits 2). The guard stays
  for the next unknown type.
- eg_q6k becomes eg_fmt per layer: 0 MXFP8 ([scale, 32 codes] blocks, 264 B a row), 1 ggml Q6_K (210 B a 256), 2 ggml
  IQ4_NL (18 B a 32: an fp16 d and 16 nibble bytes), 3 an old pack's split FP8. The load loop sniffs the type from the
  pack's tensor table, eg_issue builds the row descriptor from it (144 B a row, off + id*144, no scale tensor), and
  EgReader::read() needed no new branch: its contiguous, scale-free path already served Q6_K.
- ds4_kernels.cu: engram_dequant_kernel and engram_dequant_tok_kernel take the format and add the IQ4_NL branch, a literal
  port of ggml's dequantize_row_iq4_nl (fp16 d per 18-byte block, low nibbles 0-15, high 16-31, kvalues_iq4nl).
- The Engram row slot stays cols*(hd+hd/32) for every format, so existing packs are byte-identical: --pack-info on the
  MXFP8 h225 pack and on a Q6_K zero pack is unchanged, and a Q6_K pack's 32-token ids are identical old against new.

Verified with their real table: the pack that segfaulted runs 32 tokens in 1692-1711 ms (18.1-18.3 tok/s) and produces
coherent Python (GGML_TYPE_Q2_0 = 42, GGML_TYPE_NAMES: dict[int, str]); 64 tokens on the code prompt, DSpark off 17.16
and on 23.36 tok/s (44 of 53 drafts accepted). IQ4_NL decode is bit-exact against ggml on 8 of 8 sampled rows, and the row
addresses match the pack's tensor table (data_start 320, 144 B a row).

Not covered: no non-zero MXFP8 table existed on the box to re-check that path in-engine (the Q6_K and synthetic-zero MXFP8
packs were), there is no in-engine float dump against the Python reference (the proof is formula- and address-level), and
the legacy split-FP8 format (3) is kept but untested.
2026-10-23 14:13:26 +02:00
Niko1221 7b3e2eaa0b Control 0.1.48.1 / Terminal 0.2.3.1: version bump for the Control-only hotfix of #3810 (app stays 0.2.3, engine packages stay 0.1.48); release notes 2026-10-23 14:10:33 +02:00
Niko1221 f5f205bee0 setup: the HIP ready-made-engine checks read gfx names with arch_names, not arch_numbers (an installed AMD engine was never recognised); tests for archs as a list and as a string 2026-10-23 14:07:48 +02:00
Niko1221 0edd96ccf6 setup: BUILD.json archs / vision_archs are read as a list or as the ';'-separated string the packaging writes (setup stopped on the released Windows CUDA package: int(';'), #3810) 2026-10-23 14:05:32 +02:00
Niko1221 eb6c0a23f0 dsv41 win: pack_gguf reads the IST-DAS Engram parameters from tensors, IQ1_M type, packinfo 64-bit sizes, DS4_TRACE_AHEAD, cachesim 2026-10-23 13:17:02 +02:00
Niko1221 c523890228 dsv41: a CPU-only build path, and the intake's pack recipe fixed
1. CMakeLists: -DSTRATA_BUILD_DSV41=ON was silently dropped when no GPU backend was on, because the whole DSV41 block
   sits inside if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP).  Found on p1004 (2026-10-23), whose P100 is not a DeepSeek
   target but which is exactly the box the CPU-lane A/B belongs on.  A new branch builds ds4_gg_bench with no GPU
   runtime at all.  Verified on r9700 with both backends off: configure rc=0, build rc=0, the benchmark runs IQ3_XXS at
   60.5 GB/s (ggml-cpu) against 64.1 (AVX2 rows, x1.06) and 8.02e-09 against a double reference.

2. tools/dsv41/ista_intake.py: the pack step now uses the recipe that actually builds a loadable pack for these sets -
   --engram-meta (the UD set's first shard carries the Engram hash parameters, without it pack_gguf sees no Engram layers
   and refuses per_layer_token_embd), --mmproj, the combined --skip-tensors regex, and the DSpark noise token and markov
   rank (pack_gguf fills the rest of the geometry from the drafter but not those two, so a pack came out with
   dspark_noise_token_id 0).  Verified end to end on the 2.25 set: the intake rebuilds packs/h225's content exactly -
   same 48794 tensor lines, --pack-info 'files: 10, all present', 'no GPU kernel: none', dspark geometry 5/128799/256/128/3.
2026-10-23 12:56:24 +02:00
Niko1221 5c523d55f7 dsv41: device kernels for IQ1_S, IQ4_NL and IQ1_M (the three IST-DAS types that had none)
Found and implemented on z440's CUDA bring-up, applied here and re-verified on HIP.

ds4_iq.cu: decode<T>/deq8<T> for kIQ1_S, kIQ4_NL, kIQ1_M, plus iq1_codes() (iq1s_grid_gpu's 4-bit nibbles -> the int8 codes
8*(n-1)+-1 through two __vsub4) and iq1m_d() (IQ1_M carries its F16 d in the top nibbles of its four u16 scales - there is no d
field).  kQK<T> is 32 for IQ4_NL as well as MXFP4; load_tab uses iq1s_grid_gpu / kvalues_iq4nl; DS4_IQ_SWITCH and
supported() cover the three types, so they leave the CPU lane and join the device path.

tests/dsv41/iq_expert_test.cpp: make_mat now writes IQ1_M's scale where the format keeps it (bytes 49/51/53/55, high nibbles).
Without that the derived d is random and roughly 1/32 of the blocks come out NaN, which std::max then hides.

Parity (z440, one A4000, ds4_iq_test --type 19,20,29): token gate_up 6.70e-08 / 2.04e-08 / 5.97e-08, token down 3.62e-08 /
4.71e-08 / 3.71e-08 against ggml's to_float and a double reference; CPU lane 3.4e-8..8.2e-8; F16 prompt tiles inside their
own tolerances.  Bandwidth: IQ4_NL 376 GB/s (84% of the A4000's 448, bandwidth-bound), IQ1_S 214, IQ1_M 202 (both
compute-bound at 0.92-1.10e12 weight-values/s against IQ2_XXS's 1.22e12).

Not covered: no end-to-end run with a pack that uses these expert types (none on z440); a combined-seed run put IQ1_M's
light-path gate_up at 1.10e-2 against its 1e-2 tolerance, which is the int8 activation rounding of that path (it does not
move with tensor cores off), not the dequant.
2026-10-23 12:34:16 +02:00
Niko1221 45dc5e3400 dsv41: the CUDA build of the tree needs two portable edits (found on z440's sm_86 bring-up)
1. ds4_tq.cu:262 used the two-argument __shfl_xor, which is HIP-only; the rest of the tree already uses
   __shfl_xor_sync(0xffffffffu, v, lane), which both backends have.
2. tests/dsv41/{tq,iq}_expert_test.cpp were forced to LANGUAGE HIP, so a CUDA configure failed at generate time
   ('The language HIP was requested for compilation but was not enabled'). The language now follows the enabled
   backend: HIP, CUDA, else CXX.

Verified on r9700 that the HIP build is untouched: incremental rebuild rc=0, ds4_iq_test --type 17 ALL OK, the engine
--pack-info clean. Verified on z440 (CUDA 12.9, sm_86) by the bring-up agent: HEAD builds, ds4_iq_test ALL OK for
IQ2_XXS/XXS/IQ2_XS/IQ2_S/IQ3_XXS/IQ3_S/IQ4_XS at 5e-8 against ggml, 272-369 GB/s on one A4000. IQ1_M, IQ1_S and
IQ4_NL still have no CUDA kernel (CPU lane only) - that is the Phase 5 kernel list.
2026-10-23 12:08:49 +02:00
Niko1221 c42f4f2969 dsv41 on Windows+CUDA: IQ1_M device kernels (bit-exact vs ggml), IQ4_NL Engram rows, CUDA build fixes (strtok_r, shfl_xor, test language), IST-DAS pack names, hardware bench 2026-10-23 11:57:51 +02:00
Niko1221 5c8dd32252 serve: max_tokens -1 and 0 mean no cap again, as in 0.2.2 (clients send -1 for that); documented 2026-10-23 07:57:09 +02:00
Niko1221 5964d113f8 release notes 0.2.3: hotfix heading, status/logs names, launch not setup, drop the unreleased-pack limit 2026-10-23 06:37:32 +02:00
Niko1221 fea2d02aaa docs: Strata 0.2.3 release notes 2026-10-23 06:36:54 +02:00
Niko1221 81a7260194 launch config never keeps --mtp beside "mtp": false; one line says where the pack and draft layer are read from (B10, C08 note) 2026-10-23 03:28:36 +02:00
Niko1221 3b1ab2848d Save and launch saves the preset only after the restart question is answered Restart; the question names the running model and port 2026-10-23 03:27:48 +02:00
Niko1221 80a70b86de Folder step: models Strata already keeps stay where they are (the text said they were moved) 2026-10-23 03:24:46 +02:00
Niko1221 9222b6a1d3 a failed start ends its own runtime host, checks the port first and quotes only this attempt's logs (X01) 2026-10-23 03:23:58 +02:00
Niko1221 67a48427b8 Merge f-023-int into f-023-tui2 2026-10-23 00:27:00 +02:00
Niko1221 d64ae1b0b1 tests: the smoke test reads the engine's own words from .reason, the message is plain (X01) 2026-10-23 00:26:59 +02:00
Niko1221 27c0cabd81 Merge f-023-tui1: TUI stress fixes (second launch reuse-or-ask with --restart, context change on the same port, status/logs find the launch, key kept, retune stops its own model, folder checks, Ctrl+C, --model-path wins) 2026-10-23 00:13:45 +02:00
Niko1221 2f5f7c1135 Failed engine test start gives a plain cause and next step; rollback never says the same version twice; max_tokens below 1 is a 400 (TUI stress X01, #18, #15)
X01: TestStartFailed has one text builder (TestStartFailed.plain): what did not work in plain words, that nothing was installed and the current engine is unchanged, and one next step (`strata update --force`, or a plain retry when the card was busy). The engine's own words stay in .reason, the state file and the JSON. The app's message matcher also accepts the new wording. Hints that a caller passes, such as the one for a busy launch, are used as they are, so the launch-holds-the-card line of f-023-tui1 composes with it.
#18: `components rollback` when an earlier rollback is still in force says the engine is already the previous one and what to run, instead of "0.1.45.3 -> 0.1.45.3".
#15 (K05): max_tokens, max_completion_tokens and max_output_tokens below 1 are a 400 naming the field (they used to mean "no cap"); an unknown model id is still answered by the loaded model, now documented in DETAILS.md.
2026-10-23 00:06:13 +02:00
Niko1221 3bf804c9c8 tests follow the folder check and the compact save summary 2026-10-23 00:05:00 +02:00
Niko1221 0af0f9f125 Installer: an older feed shows its real reason; a folder with accented letters unpacks (TUI stress #11 A01b/A05, #12 A04)
#11: the refusal of a feed older than the one already accepted now says both sequence numbers, the folder that holds the newer one (.updates) and one next step (use the newest release folder, or delete state.json to take the older one). install.ps1 and install.sh print the whole check output, not its first 6 lines, and no longer blame a missing signature when the check gave a reason.
#12: tar.exe got the path through the ANSI/OEM code page and failed on accented letters. The installer now runs Windows' own bsdtar (System32 first, so a GNU tar from Git for Windows cannot be picked up for a zip) with ASCII relative arguments from inside the data folder and renames the result, so no non-ASCII path ever crosses that boundary. Tested on C:\...\<U-umlaut>n<i-umlaut> test with a stub release folder; the old script fails there with "could not chdir".
2026-10-22 23:54:57 +02:00
Niko1221 ea67f6999a A missing shard is named with its file name in the CLI, the TUI and the app's plain issues (TUI stress #9, C05, #3478)
plan_import now returns missing_shards (the file names of the split set that are not in the folder). problem_words and plain_issues say "<name>-00002-of-00002.gguf is missing" instead of "some parts of the model are missing"; more than two are counted. The three call sites (models import, strata launch --model-path, the Model screen's own-files field) pass the list.
2026-10-22 23:47:37 +02:00
Niko1221 eb818c50ed Download size counts the draft layer and the picture reader; --plan no longer calls an installed engine missing; --drafting off writes no --mtp (TUI stress #6, B03, B10/E04)
#6 (A07/B03): a model that is not installed now counts the draft layer (5.2 GB) and the picture reader (0.9 GB) in the one place (modelstate.assess), so the Folder step, the confirm and --plan agree (Gyro-S 58.5 + 5.2 + 0.9 = 64.6 GB). Gyro joins the families that get the helpers. --mtp off and --drafting off drop the draft layer from the count and from what setup is told. setup.py's disk check counts the draft layer it is about to fetch.
B03: --plan keeps "engine_missing" for no engine at all; an installed engine that fails the check says so in plain words and adds engine_usable.
B10/E04: with --drafting off (or an installed config that says off) the written run config carries "mtp": false and no --mtp / --spec-min-p arguments.
2026-10-22 23:44:27 +02:00
Niko1221 092a62daaa --model-path and --data-dir win over an installed copy with the same model and context (C08) 2026-10-22 23:37:20 +02:00
Niko1221 1ed28f8863 the Folder step checks the path before going on, the Save summary shows the storage path, a denied folder is told in plain words (B05, B12) 2026-10-22 23:36:16 +02:00
Niko1221 38c7f7017d ctrl+c quits the launch screens like ctrl+q; only a running download or tune asks first (B13) 2026-10-22 23:36:15 +02:00
Niko1221 c924d3c9d7 An old Terminal that cannot hand over to a new Control is told to run the install command once more (TUI stress #4, H03)
The Terminals before 0.2.1.3 (0.2.0-beta, 0.2.1, 0.2.1.1, 0.2.1.2) install a newer Control and report "updated", but their launcher has no hand-over, so the old Control keeps running. Their updater test-starts the staged Control with --version (STRATA_NO_ACQUIRE=1, no STRATA_HANDOVER, from the .s staging folder) and shows the first 120 characters of the answer when that fails. Control now answers that start with one plain sentence and exit 7, so the old updater keeps the old Control and prints the sentence instead of "updated". Every updater that can hand over sets STRATA_HANDOVER and is not touched. Release note added.
2026-10-22 23:33:24 +02:00
Niko1221 6f8800aefe Version bump for 0.2.3: app 0.2.3, engine and Control 0.1.48
Cargo.toml and Cargo.lock, desktop and host package.json and lock, PRODUCT_VERSION, CMake project and Metal STRATA_VERSION, build_hip_linux PKGVER default, About comments, tests that pin the number, Docker image tags and docs (0.2.3-<flavor>), download names in the READMEs, AMD_HIP.md and UPDATES.md, AGENTS.md, the bug report placeholder. The engine moves too: it shares the CMake number and its source changed since 0.2.2 (DFlash2, verify and prefill paths). A new test checks every place agrees and that a 0.2.2 install (Control 0.1.47) is offered Control 0.1.48.
2026-10-22 23:33:15 +02:00
Niko1221 814716336a a second launch reuses the same choices, asks or needs --restart for other choices, and frees the port first (G04/E01) 2026-10-22 23:31:30 +02:00
Niko1221 0530a1f7c2 retune stops its own server first; engine update gives one next step when the launch holds the card (E07/H04/H06) 2026-10-22 23:22:57 +02:00
Niko1221 f6db09bae2 launch keeps the API key it made until --new-key is given (E05/G01) 2026-10-22 23:21:37 +02:00
Niko1221 217ddda0eb status and logs find the running launch when no address is given (F02) 2026-10-22 23:20:36 +02:00
Niko1221 734ada7e1b requant_gguf: check the output's free space against the plan before writing (or in --dry-run, return 3)
The release runbook promises the pack/requant builder checks free space before it writes (DEEPSEEK-RELEASE.MD 9.6 item 5);
it did not.  --dry-run now prints 'space: N GB planned, M GB free on <fs>' and returns 3 when 98% of the free space does
not cover the plan, so 'requant_dry && requant' cannot start a run that will fill the disk; a real run raises instead.
With --resume only the bytes still missing are counted against the free space.

Verified on the box with two recipes that do not fit /data today: rq3 (224.3 GB planned against 128.6 GB free) and rq_gu3
(251.2 GB against 128.6 GB) both refuse with the numbers printed, rc 3 in dry-run.  pack_gguf's own output is a few MB
(tensors.txt, engram.bin, tokenizer) and needs no guard.
2026-10-22 22:38:16 +02:00
Niko1221 a14e85d8e6 tests: the pinned DFlash2 drafter address is allowed in strata_control (only strata drafting fetch reaches it, behind acquisition.require) 2026-10-22 21:19:41 +02:00
Niko1221 996b8a4044 dsv41: a type with no device kernel is known, laid out, and kept on the CPU lane (stream H1, second half)
The rule from DEEPSEEK-RELEASE.MD 6.5.2 - every ggml type works, a tensor whose type has no GPU kernel stays on the CPU
lane with one warning instead of stopping the load - is now implemented for IQ1_S (19), IQ1_M (29) and IQ4_NL (20):

- iq::known(t) (a layout exists: block_values/block_bytes/name) is separated from iq::supported(t) (a device kernel
  exists).  type_row_bytes/type_block use known(), so a pack lays out such a tensor instead of refusing it;
  gg::type_on_device(t) is what the VRAM tier tests.
- the CPU lane takes them: gg_cpu_type() covers 10-14, 16-23 and 29 (ggml-cpu's row dots).
- a layer whose experts are such a type is flagged cpu_only at load, printed once ('no GPU kernel for it, its experts
  stay on the CPU lane') and excluded from the VRAM budget in place_experts().
- iq::block_values(IQ4_NL) = 32 (QK4_NL): the only block here that is not 256 values.  Without it the CPU lane read
  IQ4_NL at the wrong row stride (2.4e-02 and 1.4e+05 error against the reference).

Verified: ds4_iq_test --type 29,19,20 runs the CPU lane against ggml's own decode and a double reference - IQ1_M
1.6e-08 / 4.6e-08, IQ1_S 5.4e-08 / 4.0e-08, IQ4_NL 3.1e-08 / 3.2e-08, ALL OK; the device types still pass; --pack-info
reports the layer and role for each such tensor.  A full model with real IQ1_M weights has not been run (none on the
box): the arithmetic of the lane is what is verified.
2026-10-22 20:27:19 +02:00
Niko1221 164dd3bbef Merge f-023-fix3: drafting network code loads only when used, rollback tests wait for the restored start, Arc B-series runs with pictures on 2026-10-22 19:58:52 +02:00
Niko1221 6577b69d5d Intel Arc B-series (Battlemage, e.g. Arc Pro B70) runs like any Arc: pictures on by default with the CPU reader, one plain line that the card is new and how to report a problem
Owner decision 2026-10-22. setup_intel no longer switches pictures off for a B-series card; it prints the line instead (known risk: the first-picture xe reset on the B70, 2026-10-19). strata launch gets the same line as a warn-level note (Computer.new_card_note) and its picture default for an Intel engine now follows the CPU-reader rule at any VRAM size (it used the GPU reader for a 32 GB B70). Gyro adoption on Arc was already allowed in this tree (adopt.FAMILIES). Tests that pinned the opt-in are updated; docs/INTEL.md, packaging/README.md and docs/GYRO.md say B-series is supported and untested by us.
2026-10-22 19:39:27 +02:00
Niko1221 dca4217501 test_runtime: the rollback tests wait 30 s for the restored launch (5 s was shorter than a Windows engine start); the two failures and the teardown error were a timeout, the rollback itself works
Same failure on v0.2.2 (011c9e600): the failed new start is refused for its port as intended, the old config is relaunched, but it was still loading when the 5 s startup wait ended, so rolled_back was False and the left-over process held the temp folder at teardown.
2026-10-22 19:35:29 +02:00
Niko1221 cf4bef2f78 dsv41: --pack-info prints the DSpark and Engram geometry a pack needs at run time
Today a pack built by another tool had dspark_block_size 0: DSpark ran one token a round and was worth +0.8% instead of
+28-77%.  The report now shows block_size / noise_token_id / markov_rank / the drafter's expert counts, the mtp tensor
count, and the Engram layer ids, with a <- marker when the block size is 0 or there is no drafter.  Verified on h225,
c0q3hQ6_K, c0q2d5 and mxfp4 (all: block 5, noise 128799, rank 256, 1242 mtp tensors, engram layers 1,14).
2026-10-22 19:33:47 +02:00
Niko1221 84b1e9815e strata drafting: load the network code only when the command runs, so a started engine's command line does not import it (437a60676 regressed the start-path rule) 2026-10-22 19:30:23 +02:00
Niko1221 c266cfb3cb Merge f-023-int into f-023-gui 2026-10-22 18:59:33 +02:00
Niko1221 a8301a2709 Merge f-023-metalfix2: no expert-cache loan in the Metal CLI prompt path either 2026-10-22 18:42:02 +02:00
Niko1221 669023fc08 Merge f-023-strixpp: Strix Halo gets its 8192 prompt chunk back on Windows, and the HIP engine finds its own tuning table 2026-10-22 18:36:58 +02:00
Niko1221 2f3f363c33 dsv41: --pack-info checks the files the pack points at (a cleaned-up model_dir is a dead pack, not a description)
The pack is pointers: ds4.txt + tensors.txt + the weights where model_dir says.  Today /home/niko/dsv41-c0/t3 was empty
and the c0q3 pack (all IQ3_XXS) described itself perfectly while being unable to load.  --pack-info now stats every file,
compares the length with the largest tensor end, prints MISSING/SHORT lines and exits 2 when any is bad; a healthy pack
prints 'files: N, all present and long enough'.  Verified: h225 10/10, c0q3hQ6_K 19/19, c0q2d5 12/12, c0q3 fails with
six MISSING lines.
2026-10-22 18:31:53 +02:00
Niko1221 b440640929 metal: no expert-cache loan in the CLI prompt path either (the 4K+ prompt of a plain strata run aborted in ExpertCache::fill_slot_queued) 2026-10-22 18:31:04 +02:00
Niko1221 155995b65c Merge f-023-drafting: the app hides DFlash2 until the engine serves it 2026-10-22 18:07:21 +02:00
Niko1221 e890e30766 The app offers DFlash2 only once the engine serves it: the Drafting setting is Auto / MTP / Off until then (Control reports it) 2026-10-22 18:07:03 +02:00
Niko1221 7b04360f93 hip prefill: the package tuning folder lookup split the exe path on '/' only (a backslash escape lost in the string): split on both 2026-10-22 18:05:47 +02:00
Niko1221 31785cce47 pack_gguf: build a pack from a quant that was made by another tool (the IST-DAS sets)
The IST-DAS 2.25bpw GGUF differs from ours in five ways that stopped pack_gguf, each found by building its pack:

- geometry(): ALIASES for the keys they spell differently (kv_source_layer_ids, index_source_layer_ids, the indexer's
  candidate_*, dspark_target_layer_ids, engram_layer_ids).  Without the source ids the engine refused the pack
  ('layer 2: compress ratio 2 without a KV source of that ratio before it').
- --engram-meta FILE: their GGUF carries no Engram hash parameters (no ple.*/engram.layers, no token map, no
  multipliers/primes/offsets - those are tensors of integer type in their file); the parameters come from a GGUF of the
  same model that still has them, the tables from --engram.
- integer tensors (I8/I16/I32/I64/F64) are left out with a line instead of stopping the build (engram_token_map.weight,
  engram_primes.weight, ...).
- --skip-tensors REGEX: their attn_output_a is one interleaved 3-D tensor ([8,1024,4096] ne) that this engine's
  o_groups slices cannot take; our Q5_K requant of the same BF16 weights supplies it through --dense-override, which
  now also *adds* a tensor the quant leaves out (it could only replace before).
- the DSpark geometry (dspark_block_size / n_routed_experts / num_experts_per_tok / target_layer_ids / markov_rank) is
  filled from the drafter's dflash.* keys and the markov tensor's shape, and --dspark-noise-token sets the model's noise
  token (128799).  Without it d_block stayed 0 and DSpark ran one token a round: h225 measured 16.77 tok/s off vs
  25.93 on (3.16 tokens a round) after the fix, 1.01 before.
2026-10-22 18:04:57 +02:00
Niko1221 e22a3c9ba9 Merge f-023-drafting: strata launch --drafting auto|mtp|dflash2|off and the app's Drafting setting; suggestion table (Auto = MTP until a card wins everywhere); pinned drafter download and numpy packer; gated until the engine serves DFlash2 2026-10-22 17:43:34 +02:00
Niko1221 b7196c23de Drafting in the app: "Drafting: Auto (recommended) / MTP / DFlash2 / Off" beside context in the model's settings
- The Models screen gets one more column in the context row, the existing Select, with the four choices; DFlash2 is offered for Qwen3.8-Flash-Next
  (the qwen and gyro families) only. The choice is saved with the model (a setting the app owns, like Draft layer) and shown again next time.
- Before a load the host asks Control (strata drafting plan / fetch --if-needed): when the choice would use the DFlash2 drafter and its file is not
  here, it is downloaded pinned and checked and packed, with plain progress in the load. A drafter that cannot be had never stops the load: the MTP
  starts and the job says so. The download plan dialog lists the drafter's size when it is chosen.
- What ran is told in the engine's words under Details and below the row: "Drafting: MTP (DFlash2 didn't fit)" and the other three reasons, all in
  the eight languages. Off is the draft layer's own switch.
- The app never reads the suggestion table: Control decides (strata_control/drafting.py), so the Terminal and the app cannot disagree.
2026-10-22 17:41:10 +02:00
Niko1221 fcb0c2d0ec Merge f-023-int into f-023-gui 2026-10-22 17:34:22 +02:00
Niko1221andmedking82 e17efa53ce Updates: an interruption that a later finished update settled says so in the same red alert (#3578)
Logic and tests ported from #3578 by medking82. The alert stays red and in place; only its words change for the restart interruption of an update that a later update of the same component finished. Sentence-case text in all eight languages; the PR's screenshots and validation files are left out.

Co-authored-by: medking82 <[email protected]>
2026-10-22 17:33:54 +02:00
Niko1221andmedking82 3f81245864 Reuse a retained legacy model library through a guarded native data root, without moving weights (#3362)
Ported from #3362 by medking82.

Co-authored-by: medking82 <[email protected]>
2026-10-22 17:33:01 +02:00
Niko1221andG0-0000 9471faa5f0 Web page: model settings rows no longer overflow their card on narrow windows (#3601)
The overflow was found and reported in #3601 by G0-0000. Its 75% column made the inputs narrower at every width, so this keeps the current layout when it fits: the label column may shrink, long names wrap, the input keeps 360 px.

Co-authored-by: G0-0000 <[email protected]>
2026-10-22 17:32:50 +02:00
Niko1221andjkir4n 3b51719abc Web page: keep long words and wide tables inside the viewport on narrow screens (#3594)
Ported from #3594 by jkir4n.

Co-authored-by: jkir4n <[email protected]>
2026-10-22 17:32:35 +02:00
Niko1221andengharat 2ba95cf61f Web page: use an edited API key at once, even when browser storage is blocked (#3435)
Ported from #3435 by engharat.

Co-authored-by: engharat <[email protected]>
2026-10-22 17:32:35 +02:00
Niko1221 437a60676f Drafting: strata launch --drafting auto|mtp|dflash2|off, one suggestion table, and the DFlash2 drafter fetched pinned and packed on the person's PC
- strata_control/drafting.py is the one place: the choice (saved in the model's run config as "drafting"), the table card class -> suggested
  drafter with the measured evidence (Auto picks DFlash2 only where it was faster on every tested prompt; today no class qualifies, so Auto is
  the MTP everywhere), the pinned drafter (incoai/Qwen3.8-Flash-Next-DFlash2 rev a22a1744, sizes and SHA-256 of the repository), what each choice
  puts on the engine arguments, and the plain "Drafting: MTP (DFlash2 didn't fit)" words read from the engine's own log.
- serve/server.py applies the choice at every start and reports "drafting" / "drafting_note" in /metrics.
- strata launch --drafting writes it to the launch config and presets, shows it in --plan, adds the drafter to the download plan when it would
  be used, fetches it before the start (a failure starts the MTP and says so), and prints the engine's fallback line.
- strata drafting plan|fetch (JSON/JSONL for the app). tools/dflash2_pack.py now reads safetensors with numpy alone (no torch on the person's
  PC), byte for byte the same file as before (checked against the earlier version), and ships in the Terminal bundle.
- DFlash2 is for Qwen3.8-Flash-Next (qwen, gyro) only. The engine in this release runs it only in its CLI run, not with --serve, so while
  drafting.ENGINE_SERVES_DFLASH2 is False the choice starts the MTP, says "DFlash2 isn't available in this mode yet" and downloads nothing.
2026-10-22 17:27:39 +02:00
Niko1221 f1b6a2144c Merge f-023-df2: DFlash2 drafter (opt-in --draft dflash2) merged onto the combined branch, builds on CUDA, HIP and Metal 2026-10-22 17:05:26 +02:00
Niko1221 c62d980112 hip prefill: with no STRATA_HIPBLASLT_TUNING in the environment the engine takes the table beside it (tuning/<arch>-hipblaslt-<version>.txt) itself, so a start that did not come through the launcher (hand-made config, CLI) does not read prompts at half speed on a Strix Halo; STRATA_HIPBLASLT_TABLE=off keeps plain hipBLAS 2026-10-22 17:04:58 +02:00
Niko1221 9ee8b0cf79 dsv41: --pack-info (the pack as a description: type map, bytes an expert/a token, tiers, tensors with no GPU kernel)
strata-dsv41 --pack-info --pack <dir> reads only ds4.txt and tensors.txt: no device, no weights, no lock.  It prints the
layer x role type map with an expert's bytes and a token's MB (top-k picks), the type totals, all-experts GB at the mean
bpw, the dense side split into embed / head / mtp / vision / engram, and every tensor whose type has no device kernel
(IQ1_S, IQ1_M, IQ4_NL today) with the layer and role - the H1 rule's report, and the first line of every benchmark log.
Verified on c0q3 (208.1 GB at 3.06 bpw, 3.251 GB a token - the document's figures) and c0q3d5.
2026-10-22 16:57:11 +02:00
Niko1221 051db74cdf gfx1151 on Windows: a bare --prefill auto may reach 8192 again (the 6144 cap of #1630 stays for other cards); STRATA_GFX1151_CHUNK=0 keeps 6144 2026-10-22 16:55:16 +02:00
Niko1221 49c157fae3 dsv41: IQ2_XS device kernels (bit-exact vs ggml), Q5_K/Q4_K/Q6_K embedding rows on the host, i-quant expert test, CPU-lane bench reference
- ds4_iq.cu: IQ2_XS (type 17) through the same kernels as IQ2_XXS: Ty<17> (74 B, 1024-word grid),
  decode<17> and deq8<17>, block_bytes/name/dispatch.  ds4_iq_test compares the token path, the window
  path, the prompt sets and the CPU lane against ggml's own to_float + a double reference: worst
  4.6e-08 (token gate_up), 4.6e-08 (down), 8.9e-03 (F16 tiles), CPU lane 3.5e-08.  IQ2_XXS / IQ2_S /
  IQ3_XXS / IQ3_S / IQ4_XS pass the same test unchanged.
- ds4_model.cu: gg_type_of takes 17; embed.weight may be any ggml block type (Q5_K / Q4_K / Q6_K ...)
  through gg_row_to_bf16 (ggml's to_float, the same BF16 rounding as the Q8_0 path) - the IST-DAS sets'
  Q5_K token_embd needs no dense override now.
- gg_dot_bench: the A/B's pass criterion was |x-y|/|x| on single outputs, which fails on cancellation
  while the two paths agree to a few ulps; it now also compares 64 sampled rows against a double
  reference scaled by the L1 norm (worst 1.1e-08) and requires |x-y| <= 1e-6 of the output scale.
  A/B at 32 threads: single token x1.03 (IQ2_XXS) / x1.13 (IQ2_S); 4 rows x2.4 / x2.3.
- CMake: ds4_iq_test target; ds4_tq_test and ds4_tq_test_portable link ds4_gg_cpu.cpp (the ggml CPU lane
  ds4_iq_cpu.cpp refers to since 49e82263a, missing from those targets).
2026-10-22 16:53:39 +02:00
Niko1221 0c467e9d45 Merge f-023-agent: agent context pressure from measured tokens, summaries in pieces that fit, recovery from the context-window 400, preflight of oversized messages, quote-aware read-only shell detection, visible compaction, plain context errors, stale child marker after a force-kill
# Conflicts:
#	apps/host/src/screens/Chat.tsx
2026-10-22 16:52:15 +02:00
Niko1221 76ac5842ff Merge f-023-tune: Work profile CPU workers ceil(0.8 x physical cores), VRAM reserve capped at 1536 MiB on 16 GB cards (picture floor only for a GPU reader); #3493 conversation cache sized automatically after load; #3438 bench reports an engine that did not stop 2026-10-22 16:26:51 +02:00
Niko1221 5de8d26949 tests: setup no longer tips about the conversation cache (the engine sizes it, #3493) 2026-10-22 16:22:37 +02:00
Niko1221 20fed551ba dsv41: Strata AVX2 i-quant rows opt-in (DS4_GG_IQ256=1) until the engine A/B 2026-10-22 16:21:25 +02:00
Niko1221 e4b31d6f69 strata-context: drop the development log hook 2026-10-22 16:14:56 +02:00
Niko1221 b96f174775 Merge f-dflash2 (the DFlash2 drafter, opt-in) into f-023-int 2026-10-22 16:10:08 +02:00
Niko1221 f0010a607f serve: the conversation cache is sized from the free RAM when --conversation-cache-mib is not given (#3493): 40% of what is left above the parking floor, at most 8192, off under 1024, one log line; an explicit value (0 too) is kept; harness_reuse_probe.py; docs 2026-10-22 16:06:17 +02:00
Niko1221 ed0ed38e2d Merge f-023-tls: #3722 HTTPS uses the OS certificate store (truststore, certifi fallback) in setup, Control, serve and the fetch tools; the app host adds the system roots; a certificate failure says plainly what is intercepting and what to try 2026-10-22 15:43:33 +02:00
Niko1221 c4aec611b2 tls: downloads check the server against the operating system's certificates; a failed check gives a plain message (#3722)
Python's OpenSSL checks against certifi's roots and does not fetch a missing intermediate; Chrome and curl.exe use the Windows store
(schannel), which also holds the roots antivirus HTTPS scanning, a company proxy or a preview build adds. strata_control/tls.py
install() puts truststore (pinned in requirements.txt and the three runtime locks with hashes, CPython 3.12 in every archive) under
every HTTPS connection of setup, Control's transfer and component download, serve and the fetch tools, with certifi as the fallback.
A certificate failure is no longer the raw urlopen text and is not retried: it names the likely causes (antivirus web scanning, a
company or school proxy), says what to try, and never offers to skip the check.

The host: server/system-ca.mjs adds the system roots to Node's bundled list at start (Node 24.5+), and the agent process gets
NODE_USE_SYSTEM_CA=1.
2026-10-22 15:43:12 +02:00
Niko1221 3ae6b3041b A big message on a chat near the compaction threshold summarises first; the real-engine script can also run against a server that is already up 2026-10-22 15:40:55 +02:00
Niko1221 ebaeda85f1 Summarize and continue asks for the answer instead of sending the refused message a second time; agent README describes the context pieces 2026-10-22 15:37:35 +02:00
Niko1221 815256d89b Tests for the two ways to send a message that is too large (file in the chat folder, shortened preview) and the host route 2026-10-22 15:26:53 +02:00
Niko1221 b14862351b The chat checks a message against the model's window before sending it, handles /compact itself, and explains a full context in plain words
- /compact was typed into the chat and sent to the model as text (it answered "nothing to compact, this conversation is still short" at 90%
  full). The host now runs the Harness's /compact and says how full the chat was and is, measured.
- Before a message goes out the model server counts it (its own tokenizer). Fits: sent. Fits once the older messages are a summary: summarised
  first, with a line in the chat. Can never fit (a 103 KB paste is 70K tokens in a 64K window): not sent, and the card offers "Attach as a file"
  (saved in the chat folder, the assistant reads it in parts) or "Send a shortened preview".
- A refusal that remains after the Harness's own compact-and-retry is a plain sentence with the engine's text under Details, and the card
  offers "Summarize and continue" or "Start a new chat" instead of "Try sending the message again".
- Compaction shows: a status line while it runs and one plain line afterwards. The context meter follows the running turn (the host reports
  what the context holds after every model request) so it no longer reads 1% between two tool steps.
- A Harness error "lost host child is still present" reads as "the last Strata session is still closing down".
- All new sentences are in the catalog in the eight languages. The real-engine run is scripts/context-e2e.mjs.
2026-10-22 15:26:30 +02:00
Niko1221 f8f4e8a7df bench: a failed engine stop is loud and fails the run (#3438): stop_engine returns why the process may still be alive (taskkill result and exit, the 60 s and 30 s timeouts), the run prints an ERROR, writes ENGINE-NOT-STOPPED.txt, exits 6 and reports ok false 2026-10-22 15:20:59 +02:00
Niko1221 acf63bef73 app: the Work profile takes 80% of the physical cores rounded up (the engine's own count at most), keeps no more than 1536 MiB on a card of 16 GB or less, and a CPU picture reader no longer raises the reserve to 2048 (RTX 5070, Gyro-S 64K: 9 workers + 2048 33 tok/s, 5 + 1536 46) 2026-10-22 15:17:24 +02:00
Niko1221 098fd545b3 A marker left by a force-killed app no longer blocks the next load when its process number belongs to a later program
The child marker without a creation time could not tell the recorded engine from another Python or Strata program that had reused the
number, so every load said "lost host child is still present" until the marker was deleted by hand. The child is always started before
its marker is written, so a live process created after the marker's file time (with a two-second slack) is not the child. Unreadable
times still keep refusing; a recorded creation time still decides by itself. New procs.started_epoch reads a creation time on Windows,
Linux and macOS.
2026-10-22 15:02:49 +02:00
Niko1221 a7cd2de97a A look-only shell command with an & or a bracket inside quotes no longer asks for approval
The read-only check refused any command containing & ; ( ) { } before it understood quoting, so listing "C:\...\GUI Q&A" showed an
approval card. Quoted text is now set aside first (maskQuotes): an expansion inside double quotes ($ or a backtick), an unterminated
quote, a quote glued to other characters and a quoted text that starts with a dash are still not understood, so those and everything
outside quotes ask as before.
2026-10-22 14:46:52 +02:00
Niko1221 be58604387 Compaction reads the prompt size the model server measured, and a summary request always fits the window
The Harness prices a conversation at 4 characters a token; code, logs and JSON are about twice as dense. A new Harness plugin
(plugins/strata-context.mjs, arithmetic in context-budget.mjs) wraps the token meter so pressure is the last measured prompt plus what
came after, priced at the density that measurement showed. It also wraps the compaction summary request: when it would not fit the
window it is asked for in pieces (a message bigger than a piece is cut for the request, each piece builds on the summary before it, a piece
the server still refuses is halved). A chat the engine refused once (a 70K-token message in a 64K window) is no longer stuck: the next
message compacts and goes through. Tests run the real Harness against a scripted server that counts and refuses like the engine.
2026-10-22 14:40:20 +02:00
Niko1221 5701fc65ce Merge f-023-tools: tool-returned pictures with elastic vision, picture-reader start errors and per-variant packaging, Markdown images (safe sources), PR #3556 tool aliases, MCP wiring, command-tool cwd/shell/timeout, image-token limit, gate fails closed 2026-10-22 14:31:55 +02:00
Niko1221 0351ecb11d Two GTX 1080s are a working PC: an 8 GB card starts Off only beside a bigger card, and a PC whose only NVIDIA cards are Pascal uses them on the CUDA 12 engine (#3655)
recommend() (Terminal, app and shared vectors) now turns a card with 8 GB or less Off only when a bigger usable card is there or it has under 4 GB. The hardware report (tools/strata_mcp.py) applies setup's own opt-in rule for a PC with only Pascal/Volta NVIDIA cards and no usable AMD card, so those cards are no longer flagged 'older than the RTX 20 series' (engine_variant cuda12); the feed already picks the cuda12 package for sm_61 and refuses cuda13. Tests: shared gpu-roles vectors, strata_mcp hardware, feed selection for sm_61, and Tune keeping a two-card role map.
2026-10-22 13:47:41 +02:00
Niko1221 d9042a16bd Missing model files are named, with one next step (#3478)
The load used to stop with 'Some model files are missing' and nothing else, so Retry looked the same every time. Control's model list now carries the required parts it cannot find (part, file, status); the host joins them with the installer's missing GGUF files and shards, puts the names in the error (four shown, all in Details and in the load log), and the app shows 'These model files are missing: ...' with 'Press Download & load again to fetch them. Files that are already there are kept.' in all eight languages.
2026-10-22 13:29:59 +02:00
Niko1221 642f5120a7 Route change: the view transition's update callback no longer waits for requestAnimationFrame (#3673)
While the callback is pending the browser holds rendering, so the frame callback never ran and every click waited for the browser's own guard: about 4 s in Chromium (and so the WebView2 app window, reproduced in Edge 155: callback 4023 ms) and about 5 s in Firefox. The callback now resolves on the first DOM change after the hash flipped, or after 150 ms. Measured in Edge 155: callback 9-19 ms, full transition about 350 ms, with and without a 3.6 MB chat history.
2026-10-22 13:24:33 +02:00
Niko1221 98a139c3c7 Updates never cross GPU makes: Windows display adapters decide the backend, an installed engine of another make is not replaced (#3688)
On an AMD R9700 PC a leftover nvidia-smi made detection say cuda, so the update staged cuda13 and its test start failed. detect_backend (Control) and gpuBackendHere (app) now let the Windows display adapters decide first and ignore a tool for a make with no adapter. The update decision keeps the installed engine's make when the PC has a card of it, and otherwise stages nothing and says so in plain words (status wrong-make, shown by the app). update_engine refuses a package of another make before any download. --backend and STRATA_ENGINE_BACKEND are the explicit way to change make.
2026-10-22 13:21:36 +02:00
Niko1221 828a2559a9 vision: the image-token limit has a default and is enforced (I7, #3690)
The reader was clamped only when the config carried vision.max_tokens; an old, hand-edited or migrated config left it unclamped and
one picture became 93,700 image tokens. Vision.token_limit gives every reader a limit (GPU 1024, CPU 300, the numbers setup writes)
unless the config names one (a value the person set is kept, 0 still means none). strata-vision refuses a picture that comes back
over the limit plus an eighth with a plain ERR instead of passing 90 times the budget on, and prints a WARNING when its own warm-up
picture is over it. The server logs a picture that is more than 4 times the limit with the numbers to send with a report.
2026-10-22 13:10:12 +02:00
Niko1221 bf8285b8c9 Merge f-dflash2-check: the per-context cost table for the cost-aware MTP window (opt-in, STRATA_SPEC_COST), phase 0 tools and launch-script fixes 2026-10-22 13:09:05 +02:00
Niko1221 a71fd34758 agent: custom command tools run in the right shell and folder, with a time the tool can raise (T7)
shell: true meant cmd.exe on Windows (Get-Content failed), the host's own folder (python tools/foo.py found nothing) and a fixed two
minutes with silently cut output. custom-command-run.mjs: a Windows command that reads as PowerShell runs in pwsh or powershell.exe,
any other in cmd.exe as before, sh elsewhere (or the tool's named shell); the folder is the call's optional cwd, then the tool's cwd,
then the home folder, and a missing folder is a plain refusal; the time is the tool's timeoutSec (1 to 3600, default 120) and ends the
programs the command started; a cut output says so. Settings stores cwd, shell and timeoutSec for a command tool and an edit that
does not send them keeps them. No new field in the Settings screen: that needs a design decision.
2026-10-22 13:09:00 +02:00
Niko1221 7d55581527 agent: stop claiming a window opened; point the model at the present tool (T9, O6, #3576)
A launch command that printed nothing was reported as 'The program, folder or file was opened.', and the system prompt told the model
to say it is open. A browser given a file:// address also returns without an error, and the model then reported that it had previewed
the page. The card now says what is known (the command finished without an error, so the program was started; Strata cannot confirm
that a window or page opened). The prompt (and the plan view step) send local images, SVG, HTML and PDF to the present tool and
forbid claiming a window opened or a page rendered unless a tool confirmed it.
2026-10-22 13:07:18 +02:00
Niko1221 034f2fdebe agent: tool servers follow the default model, one naming rule, a dropped server is reported (T5, T6)
T5: the agent's tool servers came from local[0], the first model loaded, not the model the chat runs on. buildProfile takes
defaultLocalId (the native default instance, agent-bridge option defaultLocalId); the agent's default route follows it too.
T6: the editor and the save check accepted 64 characters while the profile dropped everything over 32 or with a dot, silently.
server/mcp-names.mjs holds the one rule (32, letters, digits, hyphen, underscore; src/lib/mcp-names.ts is its copy, a test
keeps them equal). A name saved before the rule stays saveable. mcpRows reports what it leaves out (name, no program or address,
a name that collapses to another's id); the agent status carries skippedServers and the chat shows one notice per reason before
the turn. The sentences are in all eight languages (src/locales/tools-<lang>.json).
2026-10-22 13:06:11 +02:00
Niko1221 c5b13accfd metal: serve_probe.py, the owner round 9 short/long prompt probe on the default config 2026-10-22 13:05:58 +02:00
Niko1221 4791d6b676 agent: the approval gate fails closed when it cannot read the permission mode (T4)
A session whose preset lookup threw ran the call unasked (return next()). It is treated as Manual now: a call that is not provably
read-only asks, a read goes through, and the reason is logged.
2026-10-22 13:03:11 +02:00
Niko1221andAmit Tsur 8bf178febf serve: tool-call aliases and wrapper boundaries (port of PR #3556), and an unparsed call is logged (T1)
Ported from #3556 (Amit Tsur): a declared namespaced tool written with its flat alias (mcp__server__tool for mcp__server.tool) keeps
its schema while streaming and is reported under the declared name; an ambiguous alias is no alias; a quoted incomplete wrapper
(`Format: <tool_call>... </function> and then I explain it.`) stays text; a following call opener separates concatenated calls;
streamed JSON is closed only once the wrapper or the next call confirms the boundary; a call cut by the output limit stays
incomplete. The Responses-only alias map is gone (the parser resolves aliases now).

Reconciled with the 0.2.2 envelope rule: with tools declared, a call for an undeclared name stays text on every route (the PR's
tests that expected a native call for an undeclared name now expect text, or a native call only where no tools were declared and
declared_only is off: the Responses/Codex compaction route keeps that). The aliases attribute replaces the envelope work's _alias
set, so the lenient opener (#3441) and the declared-only gate use the PR's unambiguous alias table.

New rail: a call that named a tool the request did not declare is counted (parser.unread) with the names the model wrote
(parser.unread_names), and the server's existing log line for unread calls now prints them.

Co-authored-by: Amit Tsur <[email protected]>
2026-10-22 13:01:04 +02:00
Niko1221 1fa22cd29a app: the chat renders Markdown images from data: URLs and the chat's own folder; remote pictures stay links (O3)
![alt](src) used to come out as a link with a stray '!', and a data: URI as literal text. Allowlist (src/lib/markdown-images.ts):
data: PNG/JPEG/WebP/GIF (never SVG), the host's /api/files/raw address, and a scheme-less path read through that address for the
active chat. http(s) pictures are not fetched: they show as a link with the alt text. file:, javascript:, blob:, share paths and
every other scheme show as text. One .md-image rule keeps the picture inside the bubble.
2026-10-22 12:58:22 +02:00
Niko1221 e2858b9855 packaging: the picture reader is built per CUDA variant and checked against the package's archs (I1, I2)
build_cuda_linux.sh builds strata-vision for the same architectures as the engine (cuda12 includes sm_60/61/70) and records
vision_archs in BUILD.json; the Dockerfile records it too. packaging/build_vision_windows.bat does the same for the Windows
cuda12 and cuda13 zips (they shared one reader built before Pascal support, #3669). package_engine.py refuses a CUDA package whose
reader's vision_archs do not cover its archs and a macOS package without strata-vision (#3697); --allow-stale-vision and
--no-picture-reader are the explicit exits; --stamp-vision records a hand-built reader. ENGINE-COMPONENT.md describes the release steps.
2026-10-22 12:56:30 +02:00
Niko1221 e6dbbc312a metal: no expert-cache loan for the prompt path (Prefill::relayout and the refill into given-back allocations), auto expert cache capped at 1500 slots; CI: Metal build job on a macOS runner 2026-10-22 12:55:09 +02:00
Niko1221 425bca7199 Agent watchdog: the engine's prompt-read progress counts as activity; local routes get a 60 min stream-idle timeout (#3623)
A local engine reading a long prompt sends the Harness nothing for minutes, so the 4 minute stall limit ended the turn mid-read. The host now hands the bridge the engine's /metrics progress (live.prompt_read, generated, slots); a moving mark keeps the turn alive for at most 30 minutes after the Harness last spoke. An engine that stops reporting still ends at the usual limit. API routes are unchanged.
2026-10-22 12:54:48 +02:00
Niko1221 1298a3971c Chat store: save the chat list at most once a second, serialise only changed chats, cap the saved copy (#3604, #3603)
Every streamed token stringified every chat and wrote the lot to localStorage, which grew the WebView2 renderer to gigabytes. Writes from the streaming path are now coalesced (1 s), flushed when a turn ends, is cancelled or fails, and on pagehide/hidden. Unchanged chats reuse their JSON by object identity; a field over 200k characters, a chat over 1.5M and a list over 3.5M are shortened in the saved copy only. The stored format is unchanged.
2026-10-22 12:52:53 +02:00
Niko1221 c40a03e6dc serve: a tool's picture with an elastic image encoder starts the encoder (I6); a failed encoder start carries its stderr tail, exit status and card hint (I3)
A picture a tool returned wrote to an elastic encoder that was not running (AttributeError, then a 400 that every later
turn repeated). _note_unreadable_tool_images now calls _vision_up and turns a start failure into the usual
[image omitted: why] note; Vision.encode raises a clear ValueError when no reader runs.

vision_start_error no longer returns a blank sentence: last stderr lines, the exit status in words, the seconds of a
timeout, and a kernel-image error (sm_60 on the Windows cuda12 reader, #3669) names the card/architecture and the CPU way out.
2026-10-22 12:52:37 +02:00
Niko1221 eebb870922 dflash2: reserve the drafter's head before the expert cache is sized, loud fallback lines naming the VRAM needed, Blackwell window cap 6 (draft_cap_test) 2026-10-22 12:44:45 +02:00
Niko1221 dd8201d20d metal: link stubs for hc_mean_streams and route_marginal_stats (STRATA_TAP_FILE / STRATA_ROUTE_STATS hooks of the shared Verifier; owner round 9 G6 link failure) 2026-10-22 12:44:40 +02:00
Niko1221 a4c503d7ce nv_defaults: the sm_86 speculative start-up adaptation is opt-in (STRATA_SPEC_ADAPT_SM86=1); owner round 9 measured 0.995 on the A4000s and a length-confounded 3060, so the earlier gain did not reproduce 2026-10-22 12:43:35 +02:00
Niko1221 23b3dfc2ca Merge f-0221-sycl: a Linux SYCL (Intel Arc) engine package recipe and feed entry, --version without GPU access, RUNPATH $ORIGIN; Arc PCs get an engine from the feed 2026-10-22 12:13:33 +02:00
Niko1221 8be3f916f2 Intel Arc engine package: packaging/build_sycl_linux.sh, tests, docs
No release feed ever carried a linux-x86_64-sycl engine package, because no recipe built one (the 0.1.45 package of stream split-s1 was made by
hand). build_sycl_linux.sh builds the JIT engine with oneAPI, bundles the oneAPI runtime and the Unified Runtime adapters with collect_libs.py,
sets RUNPATH $ORIGIN on every ELF file (Intel's libraries have none or a wrong one), packages it with archs intel-arc and checks the package
with --verify, binary_deps.py and an env -i strata --version. _drm_vendor_backend takes the sysfs root so the Arc detection can be tested with a
fake /sys/class/drm; tests/terminal/test_sycl_package.py covers the choice (an Arc PC with no oneAPI tools gets the package, nobody else does,
Windows gets a reason) and the recipe; the release-matrix test has the sycl package.
2026-10-22 12:12:30 +02:00
Niko1221 424b364a88 SYCL engine: strata --version answers before any GPU is touched; no SYCL device is built before main
The migrated generate.cpp had no --version (the CUDA and HIP engines answer it first thing), and three things in dpct's helper headers
built a SYCL device at program start: blas::descriptor's inline _saved_queue_ptr, the dpct::device_memory constructor (pointer table,
mem_mgr, dev_mgr) and its default-built sycl::queue member. On a PC whose user cannot open the GPU the engine therefore aborted with
'No device of requested type available' before it could print anything, which also failed the engine smoke test of strata update.
The headers are edited and sycl/tools/fixups.py reproduces each edit after a re-migration. A frozen first-token-logits run on an Arc A750
(4095-token prompt, IQ3_XXS) is byte-identical before and after.
2026-10-22 12:12:30 +02:00
Niko1221 0788a91c64 Merge f-0221-fix: #3683 test start behind a Windows venv launcher, #3670 certifi 2026.10.19, #3613/#3467 calibrate in the bundle, and the owner's round 8 Terminal fixes (real errors, cards and roles kept, --mtp off, every download planned, engine-version preflight) 2026-10-22 11:33:43 +02:00
Niko1221 ad98bf1d17 N6: Gyro on macOS needs engine 0.1.47 (the Metal TQK6/TQK7/TQ2_T kernels first shipped there) 2026-10-22 11:33:30 +02:00
Niko1221 c67180edf1 Owner round 8: show the real engine error, keep cards and roles, honour --mtp off, plan every download, check the engine version
F5: a start that did not reach ready names the stage, the cause line from the engine log, a next step (render group on Linux, smaller
context, strata update) and the log path; --no-tui and runs without a keyboard never ask 'Keep or move?' and print what was kept.
F3: a context or picture change forwards the installed cards and Main/Helper roles to setup.
N3: --mtp on|off reaches setup, and setup no longer fetches the draft layer for --mtp off.
N4/N2: --plan counts a picture reader unless one sits beside the files, names the --vision off and --mtp off switches, and counts
the ready-made pack (NVFP4) setup would fetch; setup's disk hint is --data-dir under launch.
N5/N6: STRATA_DATA_DIR/install is looked at first, --plan says when an earlier install is the one in use, and an installed engine
older than the model needs is updated or named.
N8: --plan warns when the file headers say another family or model than the one named.
N9: update --check names the package it selected.
N11: the Arc adopt path takes Gyro and the sentence has no double full stop.
2026-10-22 11:32:32 +02:00
Niko1221 49e82263ad dsv41: CPU lane takes Strata's AVX2 i-quant rows (switchable), ds4_gg_bench, pack_gguf repeatable --dense-override 2026-10-22 10:31:03 +02:00
Niko1221 736d7a840c dsv41: pack_gguf --override-experts takes several folders with roles 2026-10-22 10:16:05 +02:00
Niko1221 a539e12c97 Merge f-strix-hcq8: Strix Halo 8-bit hyper-connection weights by default (gfx1151), sm_86 start-up adaptation default, gfx1151 expert layout mode 8, env-gated DFlash2/spec instrumentation and the opt-in cost-aware MTP window 2026-10-22 10:10:32 +02:00
Niko1221 2a22daa4f0 Fix #3613 #3467: the Terminal/Control bundle ships tools/calibrate.py; a setup crash now says what it was
setup.py imports tools/calibrate.py after a tune, but INSTALLER_TOOLS did not list it, so every launch after a tune died with
ModuleNotFoundError and only 'The model could not be installed'. The list now names it, and add_installer also reads the
tools/ modules setup.py imports so the list cannot go stale again (stage-control.py uses the same list).

A new test stages the bundle the way package_terminal.py does and imports setup.py and its tools with tools/ limited to the bundle.
strata launch keeps the last 400 lines of setup's output in launch/<name>.setup.log, shows the Python error line in plain words
when setup crashed without an [X] line, and names the file.
2026-10-22 10:08:50 +02:00
Niko1221 27a5707de5 Fix #3670: certifi 2026.10.19 (drops the emSign ECC Root CA - G3 root) in requirements.txt and the three runtime locks 2026-10-22 09:38:34 +02:00
Niko1221 72d0bf667e Fix #3683: the test start no longer waits out its 10 minutes when a Windows venv launcher sits above the server
A source checkout's .venv\Scripts\python.exe starts the real interpreter as its child, so the pid in the server's ready file
was one below the pid the runtime host started, the ready file was never matched, no probe was sent and the update ended
with startup_wait_expired. The host now accepts a ready pid within three parents of its child (Windows only).

The test start also shows progress (loading, loaded and waiting for one token) and a wait that runs out says what the engine was
doing, with the kept log, instead of blaming a busy graphics card because the log mentions VRAM.
2026-10-22 09:38:34 +02:00
Niko1221 e7ded666be docs: the Strix Halo 8-bit hyper-connection line goes into a new release-notes-0.2.3.md (0.2.2 is released) 2026-10-22 09:13:17 +02:00
Niko1221 3ae2123bce dflash2: --draft-conf (window cut by the selector's cumulative probability), drafter streams at high priority on CUDA 2026-10-22 08:51:54 +02:00
Niko1221 6d4874a491 hip: 8-bit hyper-connection weights (STRATA_HC_Q8) on by default on gfx1151, STRATA_HC_Q8=0 keeps the bf16 read; one start-up line; docs and 0.2.2 release note with the KL table and the CPU-vs-GPU rounding baseline 2026-10-22 08:16:47 +02:00
Niko1221 b1ebc3d7ca spec: the cost-aware window keeps one row-cost table per context length (< 6K, < 24K, longer), seedable per bucket 2026-10-22 07:24:46 +02:00
Niko1221 5e52cf569f dsv41: pack_gguf --override-experts (mixed-bit packs from two requantized sets); stageB tooling 2026-10-22 07:03:10 +02:00
Niko1221 ff9ca03f3c dflash2: a verify window whose graph runs out of memory narrows the window instead of failing; --draft-cap, 5 by default on Strix Halo, a plain line on 12 GB NVIDIA 2026-10-22 06:24:00 +02:00
Niko1221 5278ba0043 dsv41: dense K-quant GEMV (Q5_K/Q6_K) + dense_requant, ggml-type CPU lane, imatrix builder, requant fixes (C0 experiments) 2026-10-22 04:03:32 +02:00
Niko1221 1ef68f83e7 docs: note the STRATA_HC_Q8 measurement on the Strix Halo
+8.4% on json_extract at fixed windows (53.55 -> 58.03 tok/s), 94 of 250
output tokens differ from the BF16 read, cold HC read 93.8 -> 20.3 us
(142 -> 352 GB/s).  Experimental, opt-in only, never a default.
2026-10-21 23:43:02 +02:00
Niko1221 aec327ee51 gr: fold the hyper-connection inject and gate/mean into their neighbours
gr_read launched six kernels per hyper-connection read: norm, down,
gate, mean, inject, write.  The inject projection is now the down grid's
last block (one warp per stream, the same lane stride and warp_sumf as
gr_inject_kernel), and gate + mean are one kernel (one block owns WARPS
columns for every stream, the per-stream gated values meet in shared and
are added in the old c order).  Both keep the old per-row arithmetic and
reduction order, so lo/lq, mixed and inject are bitwise unchanged; gated
is no longer written to global because nothing outside gr_read reads it.
Six launches per read become four.

This is a no-op for the default IQ3_S path on the Strix Halo: that engine
already takes the fused (staged) hyper-connection read, not gr_read.  The
change is kept for the unfused gr_read path (CUDA, other HIP
architectures and --no-fused-gr).

Verified on gfx1151: json_extract at fixed windows (--spec 4
--spec-min-p 0 --suffix-draft 0) gives an identical output line before
and after.
2026-10-21 23:42:55 +02:00
Niko1221 9fe2ad3f55 wip: gfx1151 takes mode 8 gate/up (LDS, fused SwiGLU + q8_1) and the CUDA-layout down for IQ4_NL 2026-10-21 19:55:22 +02:00
Niko1221 ba6966d83f verify: the DFlash2 tap kernel only where the DFlash2 sources are built (Metal and SYCL keep the per-row dump path) 2026-10-21 19:52:11 +02:00
Niko1221 4c953b424d wip (experiment): expert layouts compiled for every HIP arch 2026-10-21 19:46:43 +02:00
Niko1221 6f83fe2c42 dflash2: drop the cache-touch kernel (no gain), wait-commit measurement switch 2026-10-21 19:46:26 +02:00
Niko1221 2ccf57ec46 spec: faster start-up adaptation of the expert tier in speculative runs (swap every 2 verify rounds for the first 200, then the normal interval); default on sm_86 only, STRATA_SPEC_ADAPT_EVERY / STRATA_SPEC_ADAPT_WARM set it elsewhere, an explicit --adapt-every is kept 2026-10-21 19:45:52 +02:00
Niko1221 89799596a6 dflash2: fused conv+residual+norm+quant, q/k norm+rope, K/V ring write and attention-output quantization; stage skip switches and small benches 2026-10-21 16:34:27 +02:00
Niko1221 48ec4a59ac spec: cost-aware window: the first round of a width is replaced by the guess from the other widths, a running mean for the first samples 2026-10-21 16:10:56 +02:00
Niko1221 e0d79b02f9 spec: with STRATA_SPEC_COST a round far above the known cost of its width (the first use captures a graph) counts as a tenth of a sample 2026-10-21 16:04:10 +02:00
Niko1221 d45cd9d3cf spec: STRATA_SPEC_COST_TABLE seeds the row costs (a per-card table), the CLI prints the learned table at the end 2026-10-21 15:35:07 +02:00
Niko1221 5bc7f8cab3 dflash2: tiled Q8 GEMM, fused quantizers, head-major K/V ring with a prefetching attention, forked context update, PLE read-ahead, benches 2026-10-21 15:31:29 +02:00
Niko1221 b7d5ea2f82 native_expert_bench: BENCH_CAP sizes the launch for more possible groups than real ones (what the empty block rows of a verify window cost); bench only 2026-10-21 15:08:13 +02:00
Niko1221 d232721db6 spec: cost-aware chain rule keeps the best width seen and looks one or two drafts ahead (the row cost is not convex); unmeasured widths are probed a few times 2026-10-21 15:03:51 +02:00
Niko1221 5f49ec4599 spec: STRATA_SPEC_COST keeps an MTP draft only while the next row pays at the measured row cost (step-by-step chain rule in MtpDrafter, DraftPolicy::keep_draft) 2026-10-21 14:46:56 +02:00
Niko1221 54f81315bf dsv41: DS4_TRACE routing trace (env), routestats / budget model scripts live in phaseA-tests 2026-10-21 14:12:19 +02:00
Niko1221 6878105560 spec: STRATA_SPEC_COST, a cost-aware MTP window (DraftPolicy::best_rows: the window with the best expected tokens per ms from the chain's probabilities, a measured probability-to-acceptance table and the measured row costs); the trace now logs the probabilities the window was chosen from (A/B arm, off by default) 2026-10-21 14:07:25 +02:00
Niko1221 7f97c35aca DFlash2 drafter for Qwen3.8-Flash-Next: Q8_0 pack tool, GPU forward (conv, K/V ring, windowed attention, top-16 selector), permanent verify/prefill taps, --draft dflash2 --draft-model (opt-in, MTP stays the fallback) 2026-10-21 14:04:15 +02:00
Niko1221 651a56806a generate: STRATA_SPEC_TRACE, a per-round trace of the window rows, accepted drafts, round ms and MTP draft probabilities for offline window-length studies (bench only, off by default) 2026-10-21 13:46:06 +02:00
Niko1221 d276adf277 verify: STRATA_ROUTE_STATS (distinct and new-per-row routed experts per layer per window width) and a per-width GPU stage profile line for STRATA_SPEC_T_CYCLE runs (bench only, off by default) 2026-10-21 13:28:02 +02:00
Niko1221 d8f58af1b5 generate: STRATA_SPEC_T_CYCLE, a bench-only cycle of the verify window size with a per-width cost line (off by default) 2026-10-21 13:25:20 +02:00
Niko1221 dcba249d01 verify: STRATA_TAP_FILE, an env-gated dump of the hidden states at chosen layers per verified row (DFlash2 acceptance experiment; off by default, no new graph nodes without it) 2026-10-21 13:25:19 +02:00
Niko1221 5f426c38cb tools/dflash2: q8test, the drafter with Q8_0 matrices keeps its acceptance 2026-10-21 13:24:21 +02:00
Niko1221 128223891e tools/dflash2: the torch port of the DFlash2 drafter and the scripts of the acceptance experiment (research tooling) 2026-10-21 13:24:21 +02:00
Niko1221 17c9b92bc9 generate: STRATA_SPEC_T_CYCLE, a bench-only cycle of the verify window size with a per-width cost line (off by default) 2026-10-21 13:24:21 +02:00
Niko1221 d1dec64362 verify: STRATA_TAP_FILE, an env-gated dump of the hidden states at chosen layers per verified row (DFlash2 acceptance experiment; off by default, no new graph nodes without it) 2026-10-21 13:24:21 +02:00
Niko1221 2355265fdc dsv41: pack_gguf --tq-layers (a subset of the converted layers) 2026-10-21 08:24:45 +02:00
Niko1221 1c2388876c dsv41: trellis gate/up fused in one pass sharing the activation fragments (token path 454 -> ~355 us a layer, outputs unchanged: ds4_tq_test ALL OK); optional 32-bit stream loads (no gain) 2026-10-21 01:30:01 +02:00
Niko1221 e8cb1cba7f dsv41: tq_convert resumes (skips finished layers), pack_gguf --tq-experts takes several folders 2026-10-21 00:26:31 +02:00
Niko1221 da70196350 dsv41: tq_convert --act (input-column weighted encode from the gate-3 calibration sums) 2026-10-20 23:24:48 +02:00
Niko1221 88fcf2b443 dsv41: engine-kernel bench in ds4_tq_test (--bench) 2026-10-20 23:21:41 +02:00
Niko1221 12a3c0739e dsv41: portable (FMA) tile test target, docs for the trellis experts 2026-10-20 22:31:25 +02:00
Niko1221 7df82763e3 dsv41: tq_convert.py (re-encode a pack's MXFP4 experts with the trellis encoder into the engine's file layout) and pack_gguf --tq-experts 2026-10-20 21:52:59 +02:00
Niko1221 e5b4021c6f dsv41: trellis routed experts end to end on the device and the CPU lane (TQK6/7, TQK5X/4X computed, TQK5T/4T table): ds4_tq.cu (gfx12 WMMA + portable FMA tiles over the file layout), ds4_tq_cpu.cpp (Strata tq_cpu kernels), iq entry points route the types; test ds4_tq_test (all six types pass: device and CPU lane within 1e-6 of a double reference) 2026-10-20 21:49:29 +02:00
Niko1221 d6a6c1c797 Merge f-dsv41-enc (trellis encoder, K4/K5 computed-code types 224/225, table versions 226/227) into f-dsv41-port; conflicting trellis files taken from f-dsv41-enc 2026-10-20 21:23:42 +02:00
Niko1221 a5f54c9796 dsv41: test includes point at src/dsv41 2026-10-20 21:02:33 +02:00
Niko1221 e38025769d dsv41: picture test (mkimg.py) and serve_prompts GENI / text output 2026-10-20 20:40:41 +02:00
Niko1221 3d8d1079bc dsv41: gfx12 WMMA trellis GEMV (ds4_tqw.cuh, from the CUDA lab's recipe) with a parity + bench test: all K4-K7, widths 1/3/5, table and computed codes within 8e-8 of the double reference on the R9700; ~1.07 Tw/s 2026-10-20 20:09:32 +02:00
Niko1221 6c8247af60 Merge f-dsv41-serve into f-dsv41-port 2026-10-20 20:03:45 +02:00
Niko1221 2713121725 trellis: computed codebook on the GPU encoder, README for the K5/K4 versions 2026-10-20 19:59:48 +02:00
Niko1221 c11e66aa8b tqe_table takes a type 2026-10-20 19:37:54 +02:00
Niko1221 9fcc101844 docs: DEEPSEEK41.md (where the model goes, install, the API) 2026-10-20 19:26:21 +02:00
Niko1221 a7c50df002 K5/K4 version 2: computed (LUT-free) codebook for TQK5X/TQK4X, point-table versions kept as TQK5T/TQK4T; CPU decoder tables per type; tests 2026-10-20 19:25:16 +02:00
Niko1221 a4c9336416 serve: engine_ready_s in the run config (DeepSeek-V4.1-Flash reads 300 GB at start; setup writes 3600) 2026-10-20 19:23:44 +02:00
Niko1221 2189884a59 dsv41: DS4_PROF events per device (invalid resource handle on several GPUs), exl3_legacy.cu in CUDA builds (reported by the CUDA lab) 2026-10-20 19:22:51 +02:00
Niko1221 f7c645c065 DeepSeek-V4.1-Flash in the catalog, setup and strata launch
- setup.py: the nine pinned shards of unsloth/DeepSeek-V4.1-Flash-1017-GGUF UD-Q8_K_XL (revision f924b94f, sizes and SHA-256 from the reference run; the DSpark drafter and picture reader pinned but not fetched), MULTI_FILE_FAMILIES, install_dsv41: plain warnings for the 508 GB download, RAM and disk (a refusal only where the disk cannot hold the files), the strata-dsv41 check before any download, --gguf-dir for files you already have, pack_gguf.py, config with --pack --max-context 8192 --eos 1 --tp 2 and the cards; menu entry and a --check line.
- strata_control/dsv41_fit.py: where the model goes on the cards and in RAM, from the measured run (6,240 experts in graphics memory predicted on 4 x 32 GB, 6,197 measured), the n-gram table's SSD note.
- tools/strata_mcp.py and strata_cli/launch: the model is in the list; strata launch --plan prints the placement (layer blocks per card, experts on the cards and on the CPU, RAM, disk, the SSD note); --model-path takes the nine shards where they are; parallel, pictures and tuning are said, not passed to an engine that does not take them.
- tests: tools/test_setup_dsv41.py (22), tests/terminal/test_launch_dsv41.py (13).
2026-10-20 19:22:11 +02:00
Niko1221 4455c8bd1d tqenc.py: GPU lock wait with CPU fallback 2026-10-20 19:18:57 +02:00
Niko1221 d9220bd1a5 Merge f-dsv41-cudalab (trellis GEMV lab, tqenc) into f-dsv41-port 2026-10-20 18:54:44 +02:00
Niko1221 d251b2c0db dsv41 cudalab: gfx12 WMMA port recipe of the trellis GEMV: host emulation of the data flow (matches the reference), HIP kernel source, layout probe 2026-10-20 18:45:22 +02:00
Niko1221 5a52007005 serve: DeepSeek-V4.1 through the server (deepseek-v3 pre-tokenizer, DSML tool calls, effort words, end token, card sampling)
- tools/strata_tokenizer.py: the deepseek-v3 pre-tokenizer (llama.cpp's DEEPSEEK3_LLM list as a sequence of splits) chosen by the pack's tokenizer.json 'pre'; special tokens decode to their literal text (the DSML token is not in the byte alphabet). 20 of 20 reference prompts tokenize to the ids llama.cpp made.
- serve/dsml.py: DSML calls (the GGUF template's spelling) translated as they stream into the form OutputParser reads, so tool streaming, tool_choice and calls in the thinking work as for the other families; string=true values stay strings.
- serve/server.py: Service.set_family('deepseek41'): end token from the pack, DsmlOutputParser, forced-call opening in DSML, effort words low/medium/high/max/1..100, model name and card sampling.
- serve/test_dsv41.py (39 tests, fake engine) and fixtures from the GGUF's own template and the llama.cpp reference prompts.
2026-10-20 18:45:21 +02:00
Niko1221 6f703a6dbe dsv41 cudalab: computed (LUT-free) codebooks for the 15-bit trellis (1mad, 3inst, 3sum, 3single), encoder hook TQE_CODE, LUT-free GEMV kernel, code SQNR tool, pilot scripts from f-dsv41-enc 2026-10-20 18:41:32 +02:00
Niko1221 befdd9d3e1 tqenc.py: optional per-call GPU locks (TQENC_GPU_LOCK) 2026-10-20 18:30:12 +02:00
Niko1221andmw00 2d718defc0 dsv41: pack_gguf takes unsloth's DSpark drafter GGUF (dflash) and the mmproj vision GGUF into the pack; the vision loader reads F32 rows and gate/up halves
Co-authored-by: mw00 <[email protected]>
2026-10-20 18:28:30 +02:00
Niko1221 9f78a77e01 Merge f-022-int into f-dsv41-port 2026-10-20 18:22:49 +02:00
Niko1221 01389205e9 dsv41: split-KV attention opt-in (DS4_ATTN_SPLIT=1): A/B on 4x R9700 shows no decode gain, KL unchanged 2026-10-20 18:22:02 +02:00
Niko1221 a63b2d4dae dsv41: --score segment file really carries end_line 2026-10-20 18:16:03 +02:00
Niko1221 8e13982710 build_swap: ldl damping 0.3 2026-10-20 17:58:45 +02:00
Niko1221andmw00 ff5577ad70 dsv41: split-KV sparse attention from the CUDA lab for decode / short verify windows (packed rows, T<=5), DS4_ATTN_SPLIT=0 off
Co-authored-by: mw00 <[email protected]>
2026-10-20 17:54:04 +02:00
Niko1221 d17c32d6a3 dsv41: --score writes end_line in the segment file (quant_quality.py compare reads it) 2026-10-20 17:51:12 +02:00
Niko1221 1c20a59675 dsv41 cudalab: split-KV sparse attention (2.9x at T=1), CPU-kernel cross-check of the trellis GEMV, build for the repo CPU kernels 2026-10-20 17:47:29 +02:00
Niko1221 cb725bb209 dsv41 cudalab: DeepSeek kernel parity + speed harness (mHC, router, compressor, indexer, top-k, sparse attention, Engram) over src/dsv41/ds4_kernels.cu 2026-10-20 17:39:16 +02:00
Niko1221 770292e41c dsv41 cudalab: tensor-core trellis GEMV (fp16 point table, TQL coalesced layout, prepared B fragments), K4-K7, widths 1-5 2026-10-20 17:34:48 +02:00
Niko1221 090121c31b dsv41 cudalab: trellis K4-K7 dequant + GEMV v1 on sm_86 with parity and speed harness (tqenc and the K5/K4 CPU decoder taken from f-dsv41-enc) 2026-10-20 17:20:22 +02:00
Niko1221 6b5ad4f27d trellis: proxy_ldl 2026-10-20 17:11:51 +02:00
Niko1221 c3e9a998ad dsv41: tools to turn the llama.cpp reference's 10 greedy prompts into ids and compare the engine's greedy output (serve protocol) 2026-10-20 16:50:46 +02:00
Niko1221andmw00 a0dbad8078 dsv41: pack_gguf uses the GGUF's own chat template (fixes Maya #299), drop Maya's template and the EXL3 packer
Maya's tools/ds4/chat_template.jinja lost tool associations and later system instructions; the GGUF's embedded
template matched DeepSeek's encoding.py byte for byte on 9 cases (notes-dsv41-ref.md).

Co-authored-by: mw00 <[email protected]>
2026-10-20 16:36:35 +02:00
453cc0d56f dsv41: build the imported DeepSeek-V4.1 engine path in Strata (HIP gfx1201), add --score
CMake option STRATA_BUILD_DSV41 (default OFF, adds the target strata-dsv41 and a private static kernel lib; no
existing target changes). The vendored exl3 / sampler / window-grouping kernels live in namespace strata::dsv41k so
no symbol is shared with strata_kernels. Missing CUDA names for HIP come from a force-included private header
(include/strata/dsv41/hip_extra.h), the IQ CPU lane stays the stub branch, MAYA_* env aliases are gone.
New --score IDS.json --score-out RUN.tsv: teacher-forced top-64 log-probabilities in tools/quant_quality.py's
row format (the gate against the llama.cpp reference). MIT notice added to every ported file.

Co-authored-by: mw00 <[email protected]>
Co-authored-by: needmorevram <[email protected]>
2026-10-20 16:34:12 +02:00
Niko1221 b73bd26a5c trellis: run_parts (engine runs that survive a failed segment) 2026-10-20 16:26:00 +02:00
mw00andneedmorevram bbd0a2c673 dsv41: import Project Maya's DeepSeek-V4.1 engine unmodified (v1.1.17, 8d9a9b5d)
Verbatim copy of mw00/project-maya src/ds4, tests/ds4_*, tools/ds4 and the exl3 / sampler
kernel sources it needs, MIT, with their LICENSE and a NOTICE. Not built yet; the next
commits adapt it to Strata's tree.

Co-authored-by: needmorevram <[email protected]>
2026-10-20 16:19:45 +02:00
Niko1221 a7eb99bba9 trellis: block-feedback (BlockLDLQ) encoder and the ldl mode of build_swap 2026-10-20 15:59:36 +02:00
Niko1221 431803abb9 STRATA_ACT_DUMP: rarer flush with the covariance on; merge_dumps keeps it 2026-10-20 15:57:45 +02:00
Niko1221 adf7ca9729 trellis: swap comparison and greedy sanity scripts 2026-10-20 15:34:41 +02:00
Niko1221 0b4e603cfd STRATA_ACT_DUMP: keep the accumulator alive for the exit flush 2026-10-20 14:36:38 +02:00
Niko1221 2ec3d2e381 trellis: corpus builder, original-weight fetch, swap builder for Gyro-S layer variants 2026-10-20 14:36:38 +02:00
Niko1221 a775b04695 native experts: STRATA_ACT_DUMP, per-layer second moments of the CPU expert inputs (research, env-gated, off by default) 2026-10-20 14:36:38 +02:00
Niko1221 6f99190f82 trellis: README, column-profile check of the Gyro error 2026-10-20 14:36:38 +02:00
Niko1221 4292678b8f trellis: GPU path in tqenc.py and the pilot, diagonal activation second moments (importance-weighted encode), error profile script 2026-10-20 14:36:38 +02:00
Niko1221 68a8cad3ed trellis HIP: register pressure tweaks, CU mode 2026-10-20 14:36:38 +02:00
Niko1221 c23936be78 trellis: HIP encoder (gfx1201), bit-identical to the CPU encoder; host helpers; pilot driver reads remote GGUF shards 2026-10-20 14:36:38 +02:00
Niko1221 7e9531b3da trellis: python wrapper, Hadamard helpers, .tqx container, Gyro re-encode and original-weight comparison scripts, pilot driver 2026-10-20 14:36:38 +02:00
Niko1221 512e298e6e trellis encoder: tail-biting Viterbi for TQ2_T/TQK6/TQK7 plus experimental TQK5X/TQK4X (CPU decoder reads them)
tools/trellis/tqenc.{hpp,cpp}: shift-trellis Viterbi (group-min ACS), tail-biting by a rotated free pass or the exact
search over all start groups, fp16 scale refinement, importance weights.  tqenc_test.cpp checks the streams, the
Viterbi against a naive DP, planted solutions, bit-exact decode against tq_states.hpp and the CPU kernels.
The CPU decoder gets K = 5 and 4 (ids 224, 225, experimental v1); existing types and layouts are unchanged.
2026-10-20 14:36:38 +02:00
970 changed files with 132152 additions and 12471 deletions

No files matched your search

+5
View File
@@ -17,6 +17,11 @@ __pycache__/
/mtp/
/models/
/Strata-data/
# the folders docker/entrypoint.sh creates in a mounted data/ (a link to llama.cpp, the models, the keys): never part of a build (#3644)
/data/install/
/data/state/
/data/home/
/data/Strata-data/
third_party/llama.cpp/
third_party/*.zip
*.done
+3
View File
@@ -25,3 +25,6 @@ docker/compose/* text eol=lf
# Apple tools (codesign, plutil) read these: LF only, wherever the repository is cloned
*.plist text eol=lf
# vendored KaTeX (serve/web/vendor): the files are hashed byte for byte (PROVENANCE.json), never convert their line endings.
serve/web/vendor/** -text
+1 -1
View File
@@ -66,7 +66,7 @@ body:
attributes:
label: Strata version
description: "In the app: Settings > About. In the Terminal: `strata --version`. From source: the About tab of the web page, or `engine` in `GET /v1/status`, or the git commit."
placeholder: "Strata 0.2.2 · engine 0.1.47"
placeholder: "Strata 0.2.7 · engine 0.1.52"
validations:
required: true
- type: input
+1 -1
View File
@@ -13,7 +13,7 @@ on:
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag whose engine packages go into the images (for example v0.2.2)'
description: 'Release tag whose engine packages go into the images (for example v0.2.7)'
required: true
type: string
flavors:
+53
View File
@@ -0,0 +1,53 @@
name: Metal build
# Compile and link the Metal engine on a macOS runner (no GPU needed: the kernels are compiled at run time). The shared host files
# (src/core/verify.cpp, src/prefill, src/program/generate.cpp) call kernel entry points that exist only for CUDA / HIP until Metal gets a
# stub or an implementation; a missing one is a link error here, not on a user's Mac (owner round 9, G6).
on:
pull_request:
paths:
- 'src/**'
- 'include/**'
- 'metal/**'
- 'third_party/**'
- 'CMakeLists.txt'
- '.github/workflows/metal-build.yml'
workflow_dispatch:
permissions:
contents: read
jobs:
metal-link:
# the macOS 26 SDK: metal/src/mtl_rt.mm names MTLGPUFamilyMetal4 (and MTLCompileOptions.mathMode, macOS 15) behind @available,
# which is a run-time check; the names must still be in the SDK the engine is compiled against
runs-on: macos-26
timeout-minutes: 45
env:
# setup.py's LLAMA_CPP_COMMIT and metal/setup_mac.py's metal-cpp pin
LLAMA_CPP_COMMIT: 3cf03257f219afbe7334045ff7c6a06ac68c627d
METAL_CPP_URL: https://developer.apple.com/metal/cpp/files/metal-cpp_26.zip
METAL_CPP_SHA256: 4df3c078b9aadcb516212e9cb03004cbc5ce9a3e9c068fa3144d021db585a3a4
steps:
- uses: actions/checkout@v4
- name: Tools
run: brew install ninja
- name: metal-cpp
run: |
curl -fsSL "$METAL_CPP_URL" -o metal-cpp.zip
echo "$METAL_CPP_SHA256 metal-cpp.zip" | shasum -a 256 -c -
unzip -q metal-cpp.zip -d _mcpp
- name: llama.cpp (ggml) at the pinned commit
run: |
mkdir llama.cpp && cd llama.cpp
git init -q
git remote add origin https://github.com/ggml-org/llama.cpp.git
git fetch -q --depth 1 origin "$LLAMA_CPP_COMMIT"
git checkout -q FETCH_HEAD
- name: Configure
run: |
cmake -S metal -B build-metal -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DMETAL_CPP_DIR="$PWD/_mcpp/metal-cpp" -DSTRATA_GGML_DIR="$PWD/llama.cpp"
- name: Build and link the engine
run: cmake --build build-metal --target strata
+4
View File
@@ -63,6 +63,10 @@ build-vision-cuda12.bat
/.ccache/
/.rocm-win/
# Optional MCP raster cache
*.tool-media/
/.tool-media/
# local Windows HIP packaging output
dist-hip/
+29 -13
View File
@@ -1,25 +1,40 @@
# AGENTS.md
Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5, Unsloth, Gyro and NVFP4
variants, and Qwen3.6-35B-A3B) on a normal PC: one NVIDIA or AMD graphics card plus system RAM on Windows or Linux,
or an Apple Silicon Mac. Version 0.2.2, engine 0.1.47. It has three parts:
Version 0.2.7, engine 0.1.52. The public model families are DeepSeekV4.1-Flash-1017,
Qwen3.8-Flash-Next and Qwen3.6-35B-A3B. DeepSeek V4.1-Flash runs in the app and in the Terminal
(`strata launch --family deepseek41 --model ist-das-2.00bpw`; sets in
[docs/MODELS.md](docs/MODELS.md#deepseek-v41-flash)). Read **[docs/AI_REFERENCE.md](docs/AI_REFERENCE.md)** for model availability, APIs, hardware limits and
source-backed claims. Articles and videos should use **[docs/MEDIA_KIT.md](docs/MEDIA_KIT.md)**.
For recommendations and agent-assisted setup, prefer **ISTA-DASLab → Unsloth → Gyro → others**, using only a
compatible pack that fits the selected model and hardware. ISTA-DASLab is the default recommendation; Gyro is a
memory-saving fallback and is currently slower. Honor an explicit user choice. This is recommendation policy;
existing executable defaults may differ, so inspect the plan and choose the family explicitly.
Strata has three parts:
- **The app** (`apps/desktop`, the Tauri shell; `apps/host`, the Node host and the web UI). It is for people who do
not want a terminal. The installer has the engine inside.
not want a terminal. One installer per system ([latest release](https://github.com/Niko1221/Strata/releases/latest)):
`Strata-0.2.7-windows-x64-setup.exe`, `Strata-0.2.7-linux-x86_64.AppImage`, `Strata-0.2.7-macos-arm64.dmg`.
The Windows and Linux installers carry no engine: the app downloads the engine for the graphics card on first
start, or installs a file the user already has. The Mac dmg keeps its small Metal engine inside.
- **The Terminal** (`strata.py`, `strata_cli/`, `strata_control/`): the `strata` command. It sets up a model, serves
it and watches it. Agents use this part.
- **The engine and the server** (`src/`, `include/`, `metal/`, `sycl/` for the C++/CUDA/HIP/SYCL/Metal engine;
`serve/` for the Python server with an OpenAI- and Anthropic-compatible API and a web page). `setup.py`, started by
`START-HERE.bat` or `setup.sh`, is the older one-click installer from source.
`START-HERE.bat` or `setup.sh`, is setup from a source checkout, for developers. The release paths are the
installer and `strata launch`.
## Installing Strata for a user
Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**. For an agent that has a shell, the short version is:
Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**. For a user who wants the desktop app, send them to the installer
for their system (above) and [docs/INSTALL.md](docs/INSTALL.md). For an agent that has a shell, use the Terminal; the
short version is:
1. Install the Terminal. It needs no administrator rights.
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh # Linux, Mac
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh # Linux, Mac
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex # Windows PowerShell
```
@@ -27,27 +42,28 @@ Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**. For an agent that has a shell,
2. Check what would happen. Nothing is changed:
```sh
strata launch --plan --model q2_0
strata launch --plan --family qwen --model q2_0
```
3. Set up and serve without questions. It downloads the engine for the card and the model (tens of GB), starts the
server, prints the addresses and returns. The server keeps running.
```sh
strata launch --model q2_0 --yes --no-tui
strata launch --family qwen --model q2_0 --yes --no-tui
```
Useful flags (all of them: `strata launch --help`): `--family`, `--model-path` (files the user already has),
`--data-dir`, `--context 32k|64k|128k`, `--vision off|on|gpu|cpu`, `--parallel N`, `--gpu 0,1`,
`--access local|lan|cloudflare|tailscale`, `--port`, `--api-key-env NAME`, `--new-key`, `--save-preset NAME`,
`--output json`. `strata launch --status` shows what is serving, `strata launch --stop` stops it, and
`--speed faster|full` (DeepSeek), `--output json`. `strata launch --status` shows what is serving, `strata launch --stop` stops it, and
`strata launch --show-key` prints the API key.
4. Connect the user's apps: `strata connect` lists them, `strata connect <app> --check` sends one prompt and one tool
call the way that app does.
Keep the server on `127.0.0.1` unless the user asks for more. Beyond it an API key is recommended: `--access lan` makes one for you, and `--no-api-key` serves without one (a warning, never a refusal; on a public tunnel the warning is stronger). The old route (clone
the repository and run `START-HERE.bat --yes` or `./setup.sh --yes`) still works and is the fallback when the
Terminal cannot run. Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md)) offers the same steps as tools.
Keep the server on `127.0.0.1` unless the user asks for more. Beyond it an API key is recommended: `--access lan` makes one for you, and `--no-api-key` serves without one (a warning, never a refusal; on a public tunnel the warning is stronger). Source setup (`START-HERE.bat`, `setup.sh`) is the fallback for a developer or for a system with no release package; it is
documented in [docs/SOURCE_INSTALL.md](docs/SOURCE_INSTALL.md). Those wrappers exist in a source checkout, not in the
released app. Do not tell an app user to find a `.bat` file.
Source setup does not make an unsupported model or backend compatible. Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md)) offers the same steps as tools.
## Working on the code
+491 -21
View File
@@ -1,4 +1,4 @@
# Strata Engine - top-level build (P1.S1).
# Strata Engine - top-level build (P1.S1).
#
# Targets (phase-1 §P1.S1): `strata_artifact` (no CUDA dependency), `tests`. `strata_plan` and the C++
# `strata-pack` are added when their sources exist - a target with no sources would make the build graph
@@ -8,7 +8,7 @@
# below refuses anything older: silently building for a pre-Ampere arch would produce a binary that runs
# and is wrong about its own performance, which is the worst of both outcomes.
cmake_minimum_required(VERSION 3.24)
project(strata VERSION 0.1.47 LANGUAGES CXX)
project(strata VERSION 0.1.52 LANGUAGES CXX)
# EXPERIMENTAL: the Intel Arc engine, the SYCL port in sycl/ (PR #423; icpx + oneMKL, docs/INTEL_ARC.md). It is its
# own project that takes the unported sources from src/; with this on, nothing below is configured.
option(STRATA_ENABLE_SYCL "EXPERIMENTAL: build the Intel Arc (SYCL) engine from sycl/ - docs/INTEL_ARC.md" OFF)
@@ -59,6 +59,17 @@ function(strata_utf8_manifest target)
endif()
endfunction()
# The AVX kernels exist for x86 only. On aarch64 (DGX Spark / GB10) portable.cpp stands in for them and
# native experts run on ggml-cpu's NEON dot products.
if(NOT CMAKE_SYSTEM_PROCESSOR OR CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64|amd64|x64|X64|i[3-6]86|x86|X86)$") # an unset processor stays the x86 build, as before
set(STRATA_CPU_X86 ON)
else()
set(STRATA_CPU_X86 OFF)
endif()
if(STRATA_ISA_FLOOR AND NOT STRATA_CPU_X86)
message(WARNING "STRATA_ISA_FLOOR is an x86 older-CPU build and does not apply to ${CMAKE_SYSTEM_PROCESSOR}; ignored")
set(STRATA_ISA_FLOOR "" CACHE STRING "" FORCE)
endif()
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
@@ -174,13 +185,30 @@ elseif(STRATA_ENABLE_CUDA)
if(MSVC AND STRATA_EXPERIMENTAL_SM60 AND NOT DEFINED CMAKE_CUDA_RUNTIME_LIBRARY)
set(CMAKE_CUDA_RUNTIME_LIBRARY Shared)
endif()
# #4023: the default below has to be set BEFORE enable_language(CUDA): since CMake 3.18 that call fills an unset
# CMAKE_CUDA_ARCHITECTURES with nvcc's own default (sm_75 for CUDA 13), so a test after it never saw "undefined" and
# a build with no option was an sm_75 build. -DCMAKE_CUDA_ARCHITECTURES and the CUDAARCHS environment variable win.
set(_strata_cuda_arch_defaulted OFF)
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES AND NOT DEFINED ENV{CUDAARCHS})
if(STRATA_BUILD_DSV41)
# #3935: the DeepSeek engine's own release shape (sm_86, sm_89, sm_120 + PTX 120, as packaging/build_dsv41_cuda.sh), so a hand
# build of strata-dsv41 with no architecture runs on an RTX 30, 40 or 50 card; the Qwen engine's default below is unchanged
set(CMAKE_CUDA_ARCHITECTURES "86-real;89-real;120" CACHE STRING "CUDA architectures")
else()
set(CMAKE_CUDA_ARCHITECTURES 120 CACHE STRING "CUDA architectures")
endif()
set(_strata_cuda_arch_defaulted ON)
endif()
enable_language(CUDA)
# `enable_language(CUDA)` gives the .cu compile rules but NOT the imported targets, so `CUDA::cudart` has to
# be brought in explicitly for the host-side tests that call cudaMalloc themselves.
find_package(CUDAToolkit REQUIRED)
option(STRATA_EXPERIMENTAL_SM60 "community build for GPUs below compute capability 7.5: Pascal sm_60, Volta sm_70 (#236)" OFF)
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES 120 CACHE STRING "CUDA architectures" FORCE)
if(_strata_cuda_arch_defaulted)
# a recommendation, not a refusal: the release engine is built for 75;86;89;120, a hand build names its card
message(WARNING "Strata: CMAKE_CUDA_ARCHITECTURES was not set, so this build is for ${CMAKE_CUDA_ARCHITECTURES} only (sm_120 = RTX 50). For another "
"card pass it: -DCMAKE_CUDA_ARCHITECTURES=native (the GPUs in this PC), 75 (RTX 20), 86 (RTX 30), "
"89 (RTX 40), or a list like \"86;120\".")
endif()
foreach(_a IN LISTS CMAKE_CUDA_ARCHITECTURES)
string(REGEX REPLACE "-.*$" "" _base "${_a}") # 120-real / 120-virtual are still 120
@@ -202,9 +230,16 @@ elseif(STRATA_ENABLE_CUDA)
"CMAKE_CUDA_ARCHITECTURES is '${CMAKE_CUDA_ARCHITECTURES}'. (A Pascal sm_60 or Volta sm_70 build is "
"-DSTRATA_EXPERIMENTAL_SM60=ON; upstream does not support it.)")
endif()
if(_base LESS 60)
if(_base LESS 52)
message(FATAL_ERROR "CMAKE_CUDA_ARCHITECTURES '${CMAKE_CUDA_ARCHITECTURES}': even the experimental build "
"needs compute capability 6.0 (sm_60) or newer")
"needs compute capability 5.2 (sm_52, Maxwell) or newer")
endif()
# Maxwell (sm_52, #3218): one reporter built and ran this on two Quadro M6000 (CUDA 12.x, Qwen3.8-Flash-Next
# IQ1, about 22 t/s). It has no native half-precision arithmetic, so it is slower per byte than Pascal; only
# this opt-in build gets the floor. Warn, do not refuse.
if(_base LESS 60)
message(WARNING "Building for sm_${_base} (Maxwell): community-tested on one reporter's Quadro M6000 only, "
"no native half-precision arithmetic, CUDA 12.x toolkit required (docs/OLDER_GPUS.md).")
endif()
# #220 / #224: RTX 50 engines built with CUDA 12.8 crashed in the prompt path on Linux (13.0 and newer did not)
if(_base GREATER_EQUAL 120 AND CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 13.0)
@@ -304,6 +339,18 @@ if(STRATA_BUILD_TESTS)
target_include_directories(split_rules_test PRIVATE include)
add_test(NAME split_rules_test COMMAND split_rules_test)
# #3797: the out-of-memory report at start (per card free/total, the largest other programs): the text is a pure
# function; the driver query (NVML, loaded at run time) is only checked to be safe to call. No GPU needed.
add_executable(gpu_report_test tests/program/gpu_report_test.cpp)
target_include_directories(gpu_report_test PRIVATE include)
target_link_libraries(gpu_report_test PRIVATE ${CMAKE_DL_LIBS})
add_test(NAME gpu_report_test COMMAND gpu_report_test)
# #3702: the later-stage reserve of the layer split search (header-only arithmetic, pinned against the old expression)
add_executable(stage_reserve_test tests/program/stage_reserve_test.cpp)
target_include_directories(stage_reserve_test PRIVATE include)
add_test(NAME stage_reserve_test COMMAND stage_reserve_test)
# the prompt planner's host-memory model (#2824): header-only, no GPU
add_executable(prefill_plan_test tests/program/prefill_plan_test.cpp)
target_include_directories(prefill_plan_test PRIVATE include)
@@ -336,6 +383,8 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
add_library(strata_core STATIC src/core/device.cu src/core/pinned.cu src/platform/memory.cpp src/core/graph.cpp src/core/weights.cpp
src/core/layout.cpp)
target_include_directories(strata_core PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
# the DFlash2 drafter (src/core/dflash2.cpp, src/kernels/cuda/dflash2.cu) exists in the CUDA and HIP builds; the Metal and SYCL builds compile without it
target_compile_definitions(strata_core PUBLIC STRATA_HAVE_DFLASH2=1)
if(STRATA_ENABLE_CUDA AND STRATA_EXPERIMENTAL_SM60)
# the runtime device check (src/core/device.cu) admits the cards this build was made for (#236)
target_compile_definitions(strata_core PRIVATE STRATA_EXPERIMENTAL_SM60=1)
@@ -417,7 +466,9 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
src/kernels/cuda/fused_gdn.cu
src/kernels/cuda/dequant_bf16.cu
src/kernels/cuda/qsa_select.cu
src/kernels/cuda/qsa_pattn.cu
src/kernels/cuda/verify_kernels.cu
src/kernels/cuda/dflash2.cu
src/kernels/cuda/iq_kernels.cu
src/kernels/ngram.cpp
src/ngram/ple_reader.cpp
@@ -445,6 +496,29 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_include_directories(qsa_prompt_attn_parity PRIVATE ${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
target_link_libraries(qsa_prompt_attn_parity PRIVATE strata_kernels CUDA::cudart)
endif()
# PR #3943: STRATA_PATTN_SM60's score and value kernels against the decode kernels' output (the Daily's chunk kernel and
# the merge), bitwise, on random int8 K/V (GPU, synthetic, no model). Only in the experimental sm_60 build; it skips
# (77) on a build without the kernels.
if(STRATA_BUILD_TESTS AND STRATA_ENABLE_CUDA AND STRATA_EXPERIMENTAL_SM60)
add_executable(qsa_pattn_parity src/kernels/qsa_pattn_parity.cpp)
target_include_directories(qsa_pattn_parity PRIVATE ${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
target_link_libraries(qsa_pattn_parity PRIVATE strata_kernels CUDA::cudart)
add_test(NAME qsa_pattn_parity COMMAND qsa_pattn_parity)
set_tests_properties(qsa_pattn_parity PROPERTIES SKIP_RETURN_CODE 77)
endif()
# PR #3820: STRATA_SELECT_LONG's multi-CTA top-k and shuffle-free decode scorer against qsa_block_topk and
# qsa_block_scores, bitwise, on random scores (GPU, synthetic, no model). The switches force the new kernels on any
# card; the "n3" case fixes three CTAs a query. Only in the experimental sm_60 build (skips with 77 otherwise).
if(STRATA_BUILD_TESTS AND STRATA_ENABLE_CUDA AND STRATA_EXPERIMENTAL_SM60)
add_executable(qsa_select_long_parity src/kernels/qsa_select_long_parity.cpp)
target_include_directories(qsa_select_long_parity PRIVATE ${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
target_link_libraries(qsa_select_long_parity PRIVATE strata_kernels CUDA::cudart)
add_test(NAME qsa_select_long_parity COMMAND qsa_select_long_parity)
add_test(NAME qsa_select_long_parity_n3 COMMAND qsa_select_long_parity)
set_tests_properties(qsa_select_long_parity qsa_select_long_parity_n3 PROPERTIES SKIP_RETURN_CODE 77)
set_tests_properties(qsa_select_long_parity PROPERTIES ENVIRONMENT "STRATA_SCORES_DX=1;STRATA_TOPK_MULTI=1")
set_tests_properties(qsa_select_long_parity_n3 PROPERTIES ENVIRONMENT "STRATA_SCORES_DX=1;STRATA_TOPK_MULTI=1;STRATA_TOPK_MULTI_N=3")
endif()
# PR #600: the Volta (sm_70) prompt attention's latency pipeline against the FP32 kernel it replaces. Only in the
# experimental sm_60/sm_70 build, where prompt_attn_v70_kernel is compiled in.
if(STRATA_BUILD_TESTS AND STRATA_ENABLE_CUDA AND STRATA_EXPERIMENTAL_SM60)
@@ -579,7 +653,7 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
# about arithmetic, and this one knows the ORDER the arithmetic happens in.
add_library(strata_engine STATIC src/core/layer.cpp src/core/session.cpp src/core/expert_source.cpp src/core/remote_experts.cpp
src/core/remote_expert_opt.cu
src/core/expert_cache.cpp src/core/native_head.cpp src/core/native_dense.cpp src/core/verify.cpp src/core/mtp.cpp
src/core/expert_cache.cpp src/core/native_head.cpp src/core/native_dense.cpp src/core/verify.cpp src/core/mtp.cpp src/core/dflash2.cpp
src/core/conversation_snapshot.cpp src/core/conversation_state.cpp src/core/conversation_memory.cpp src/core/peer_experts.cpp
src/core/arch_defaults.cpp src/core/nv_defaults.cpp src/core/vmm.cpp src/core/conversation_file.cpp)
target_include_directories(strata_engine PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
@@ -683,6 +757,14 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(conversation_snapshot_test PRIVATE strata_engine)
add_test(NAME conversation_snapshot_test COMMAND conversation_snapshot_test)
set_tests_properties(conversation_snapshot_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 120)
# the same test with a GDN state over 1 MiB and the opt-in background checkpoint copy (STRATA_CKPT_ASYNC=1), alone and with
# recent states kept in VRAM (STRATA_CKPT_VRAM_SLOTS): every checkpoint, capture and restore must give the same bytes
add_test(NAME conversation_snapshot_test_ckpt_async COMMAND conversation_snapshot_test)
set_tests_properties(conversation_snapshot_test_ckpt_async PROPERTIES
ENVIRONMENT "STRATA_TEST_BIG_GDN=1;STRATA_CKPT_ASYNC=1" SKIP_RETURN_CODE 77 TIMEOUT 300)
add_test(NAME conversation_snapshot_test_ckpt_vram_slots COMMAND conversation_snapshot_test)
set_tests_properties(conversation_snapshot_test_ckpt_vram_slots PROPERTIES
ENVIRONMENT "STRATA_TEST_BIG_GDN=1;STRATA_CKPT_ASYNC=1;STRATA_CKPT_VRAM_SLOTS=3" SKIP_RETURN_CODE 77 TIMEOUT 300)
add_executable(conversation_validation_test src/core/conversation_validation_test.cpp)
target_link_libraries(conversation_validation_test PRIVATE strata_engine)
add_test(NAME conversation_validation_test COMMAND conversation_validation_test)
@@ -732,6 +814,27 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(tune_rocblas PRIVATE hip::host roc::rocblas)
endif()
# DFlash2 drafter tools (not built by default: cmake --build <dir> --target dflash2_parity dflash2_gemm_bench dflash2_attn_bench): the drafter against
# the torch port on a synthetic case (needs the pack and a case file, tools/dflash2/mk_parity_case.py), and the timing of its Q8 GEMM and attention
add_executable(dflash2_parity EXCLUDE_FROM_ALL src/core/dflash2_parity.cpp)
target_include_directories(dflash2_parity PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(dflash2_parity PRIVATE strata_engine strata_kernels ${_strata_gpu_runtime_target})
add_executable(dflash2_gemm_bench EXCLUDE_FROM_ALL src/core/dflash2_gemm_bench.cpp)
target_include_directories(dflash2_gemm_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(dflash2_gemm_bench PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_executable(dflash2_launch_bench EXCLUDE_FROM_ALL src/core/dflash2_launch_bench.cu)
target_include_directories(dflash2_launch_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(dflash2_launch_bench PRIVATE ${_strata_gpu_runtime_target})
add_executable(dflash2_bw_bench EXCLUDE_FROM_ALL src/core/dflash2_bw_bench.cu)
target_include_directories(dflash2_bw_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(dflash2_bw_bench PRIVATE ${_strata_gpu_runtime_target})
add_executable(dflash2_small_bench EXCLUDE_FROM_ALL src/core/dflash2_small_bench.cpp)
target_include_directories(dflash2_small_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(dflash2_small_bench PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_executable(dflash2_attn_bench EXCLUDE_FROM_ALL src/core/dflash2_attn_bench.cpp)
target_include_directories(dflash2_attn_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(dflash2_attn_bench PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_executable(strata src/program/generate.cpp)
strata_utf8_manifest(strata)
target_include_directories(strata PRIVATE ${_strata_gpu_include_directories})
@@ -770,6 +873,116 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_compile_definitions(strata PRIVATE STRATA_HAVE_QWEN35=1)
endif()
# DeepSeek-V4.1-Flash: the third engine path (src/dsv41, from Project Maya's maya-ds4, MIT, third_party/project-maya).
# Its own executable for now (strata-dsv41); a family switch in `strata` comes after first light. OFF by default: it
# changes no existing target (the vendored kernels live in strata::dsv41k, so no symbol is shared with strata_kernels).
option(STRATA_BUILD_DSV41 "Build the DeepSeek-V4.1 engine path (strata-dsv41)" OFF)
if(STRATA_BUILD_DSV41 AND NOT STRATA_HIP_GFX906)
add_library(strata_dsv41_kernels STATIC src/dsv41/vendor/exl3.cu src/dsv41/vendor/sampler.cu src/dsv41/vendor/moe_win.cu
src/dsv41/vendor/exl3_cpu.cpp)
if(NOT STRATA_ENABLE_HIP) # (the pre-sm_80 NVIDIA decode kernels exl3.cu dispatches to)
target_sources(strata_dsv41_kernels PRIVATE src/dsv41/vendor/exl3_legacy.cu)
endif()
target_include_directories(strata_dsv41_kernels PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include ${_strata_gpu_include_directories})
target_link_libraries(strata_dsv41_kernels PUBLIC ${_strata_gpu_runtime_target})
add_executable(strata-dsv41 src/dsv41/ds4_main.cpp src/dsv41/ds4_model.cu src/dsv41/ds4_kernels.cu src/dsv41/ds4_vision.cu
src/dsv41/ds4_gguf.cu src/dsv41/ds4_gguf_cpu.cpp src/dsv41/ds4_iq.cu src/dsv41/ds4_iq_cpu.cpp
src/dsv41/ds4_tq.cu src/dsv41/ds4_tq_cpu.cpp src/dsv41/ds4_gg_cpu.cpp src/dsv41/ds4_packinfo.cpp)
# the commit the binary was built from, readable with --version (set -DSTRATA_BUILD_ID=... for a tree without .git)
if(NOT STRATA_BUILD_ID AND EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/BUILD_ID) # (a git archive has no .git: the packer writes the commit here)
file(STRINGS ${CMAKE_CURRENT_SOURCE_DIR}/BUILD_ID STRATA_BUILD_ID LIMIT_COUNT 1)
endif()
if(NOT STRATA_BUILD_ID)
execute_process(COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} rev-parse --short=9 HEAD OUTPUT_VARIABLE STRATA_BUILD_ID
OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET)
endif()
if(NOT STRATA_BUILD_ID)
set(STRATA_BUILD_ID unknown)
endif()
set_source_files_properties(src/dsv41/ds4_main.cpp PROPERTIES COMPILE_DEFINITIONS "DS4_BUILD_ID=\"${STRATA_BUILD_ID}\"")
target_include_directories(strata-dsv41 PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include ${_strata_gpu_include_directories}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml ${CMAKE_CURRENT_SOURCE_DIR}/src/prefill)
# strata_prefill: the RDNA3 WMMA dense GEMM (src/prefill/wmma_gemm.cu), which gemm() prefers over hipBLAS on gfx11 (measured
# 4.97 -> 20-40 TFLOPS on this part: the chunked prompt's dense GEMM is 11.1 s of a 4K prompt at the library's rate)
target_link_libraries(strata-dsv41 PRIVATE strata_prefill strata_dsv41_kernels ${_strata_gpu_runtime_target} ${_strata_gpu_blas_target})
find_package(Threads REQUIRED)
target_link_libraries(strata-dsv41 PRIVATE Threads::Threads)
if(WIN32 AND STRATA_ENABLE_HIP) # cudaMemGetInfo is the DXGI-budgeted hip_compat::mem_get_info (src/core/device.cu): on a Strix Halo the
# carve-out is not all the GPU may use, and the engine sizes its expert tiers by what the budget says is free
target_link_libraries(strata-dsv41 PRIVATE strata_core)
endif()
if(STRATA_NATIVE_EXPERTS) # Strata's CPU trellis kernels (tq_cpu.hpp) for the trellis experts on the CPU lane
target_link_libraries(strata-dsv41 PRIVATE strata_kernels_cpu)
target_compile_definitions(strata-dsv41 PRIVATE STRATA_DSV41_TQ_CPU=1)
endif()
if(STRATA_NATIVE_EXPERTS)
# the CPU lane's ggml-type rows: ggml-cpu's dots against Strata's AVX2 i-quant rows (bits and GB/s); no GPU, no model
add_executable(ds4_gg_bench tests/dsv41/gg_dot_bench.cpp src/dsv41/ds4_gg_cpu.cpp)
target_include_directories(ds4_gg_bench PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include ${CMAKE_CURRENT_SOURCE_DIR}/src/dsv41
${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml)
target_link_libraries(ds4_gg_bench PRIVATE strata_kernels_cpu Threads::Threads)
target_compile_definitions(ds4_gg_bench PRIVATE STRATA_DSV41_TQ_CPU=1)
endif()
if(STRATA_NATIVE_EXPERTS)
# the trellis experts (device kernels ds4_tq.cu, CPU lane ds4_tq_cpu.cpp) against a double reference; needs a GPU, no model
# both test .cpp files launch kernels, so they compile as HIP on the AMD backend and as CUDA on the NVIDIA one
if(STRATA_ENABLE_HIP)
set(_ds4_test_lang HIP)
elseif(STRATA_ENABLE_CUDA)
set(_ds4_test_lang CUDA)
else()
set(_ds4_test_lang CXX)
endif()
set_source_files_properties(tests/dsv41/tq_expert_test.cpp PROPERTIES LANGUAGE ${_ds4_test_lang})
add_executable(ds4_tq_test tests/dsv41/tq_expert_test.cpp src/dsv41/ds4_tq.cu src/dsv41/ds4_iq.cu src/dsv41/ds4_tq_cpu.cpp
src/dsv41/ds4_iq_cpu.cpp src/dsv41/ds4_gg_cpu.cpp tools/trellis/tqenc.cpp)
target_include_directories(ds4_tq_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include ${_strata_gpu_include_directories}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml ${CMAKE_CURRENT_SOURCE_DIR}/src/kernels/cpu)
target_link_libraries(ds4_tq_test PRIVATE strata_kernels_cpu ${_strata_gpu_runtime_target} Threads::Threads)
target_compile_definitions(ds4_tq_test PRIVATE STRATA_DSV41_TQ_CPU=1)
# the same with the plain-FMA tile instead of the gfx12 WMMA one (what CUDA and gfx11 builds run)
add_executable(ds4_tq_test_portable tests/dsv41/tq_expert_test.cpp src/dsv41/ds4_tq.cu src/dsv41/ds4_iq.cu src/dsv41/ds4_tq_cpu.cpp
src/dsv41/ds4_iq_cpu.cpp src/dsv41/ds4_gg_cpu.cpp tools/trellis/tqenc.cpp)
target_include_directories(ds4_tq_test_portable PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include ${_strata_gpu_include_directories}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml ${CMAKE_CURRENT_SOURCE_DIR}/src/kernels/cpu)
target_link_libraries(ds4_tq_test_portable PRIVATE strata_kernels_cpu ${_strata_gpu_runtime_target} Threads::Threads)
target_compile_definitions(ds4_tq_test_portable PRIVATE STRATA_DSV41_TQ_CPU=1 DS4_TQ_PORTABLE=1)
if(STRATA_ENABLE_HIP)
target_compile_options(ds4_tq_test_portable PRIVATE "$<$<COMPILE_LANGUAGE:HIP,CXX>:SHELL:-include ${CMAKE_CURRENT_SOURCE_DIR}/include/strata/dsv41/hip_extra.h>")
endif()
if(STRATA_ENABLE_HIP)
target_compile_options(ds4_tq_test PRIVATE "$<$<COMPILE_LANGUAGE:HIP,CXX>:SHELL:-include ${CMAKE_CURRENT_SOURCE_DIR}/include/strata/dsv41/hip_extra.h>")
endif()
# the i-quant experts (ds4_iq.cu) against ggml's own decode of the same blocks + a double reference, device and CPU
# lane, for every type the device takes: the gate a new type (IQ2_XS ..) has to pass before the engine uses it
set_source_files_properties(tests/dsv41/iq_expert_test.cpp PROPERTIES LANGUAGE ${_ds4_test_lang})
add_executable(ds4_iq_test tests/dsv41/iq_expert_test.cpp src/dsv41/ds4_iq.cu src/dsv41/ds4_iq_cpu.cpp
src/dsv41/ds4_gg_cpu.cpp src/dsv41/ds4_tq.cu src/dsv41/ds4_tq_cpu.cpp)
target_include_directories(ds4_iq_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include ${_strata_gpu_include_directories}
${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml ${CMAKE_CURRENT_SOURCE_DIR}/src/kernels/cpu)
target_link_libraries(ds4_iq_test PRIVATE strata_kernels_cpu ${_strata_gpu_runtime_target} Threads::Threads)
target_compile_definitions(ds4_iq_test PRIVATE STRATA_DSV41_TQ_CPU=1)
if(STRATA_ENABLE_HIP)
target_compile_options(ds4_iq_test PRIVATE "$<$<COMPILE_LANGUAGE:HIP,CXX>:SHELL:-include ${CMAKE_CURRENT_SOURCE_DIR}/include/strata/dsv41/hip_extra.h>")
endif()
# the kept conversations' .meta file and prefix matching (ds4_slotmeta.hpp): no GPU, no model
add_executable(ds4_slot_test tests/dsv41/slot_test.cpp)
target_include_directories(ds4_slot_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/dsv41)
target_compile_features(ds4_slot_test PRIVATE cxx_std_17)
add_executable(ds4_experts_rule_test tests/dsv41/experts_rule_test.cpp)
target_include_directories(ds4_experts_rule_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/dsv41)
target_compile_features(ds4_experts_rule_test PRIVATE cxx_std_17)
add_executable(ds4_pick_rule_test tests/dsv41/pick_rule_test.cpp)
target_include_directories(ds4_pick_rule_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/dsv41)
target_compile_features(ds4_pick_rule_test PRIVATE cxx_std_17)
endif()
if(STRATA_ENABLE_HIP)
foreach(_t strata_dsv41_kernels strata-dsv41)
target_compile_options(${_t} PRIVATE "$<$<COMPILE_LANGUAGE:HIP,CXX>:SHELL:-include ${CMAKE_CURRENT_SOURCE_DIR}/include/strata/dsv41/hip_extra.h>")
endforeach()
endif()
endif()
if(STRATA_ENABLE_HIP)
enable_testing()
set(_strata_hip_tests)
@@ -968,6 +1181,13 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_include_directories(mmvq_il_parity PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(mmvq_il_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME mmvq_il_parity COMMAND mmvq_il_parity)
# sm_89's opt-in tables (#3453, STRATA_MMVQ_IL_89=1): the table check only (no case matches MMVQ_IL_ONLY)
add_test(NAME mmvq_il_parity_89table COMMAND mmvq_il_parity)
set_tests_properties(mmvq_il_parity_89table PROPERTIES ENVIRONMENT "STRATA_MMVQ_IL_89=1;MMVQ_IL_ONLY=no-such-case")
# the opt-in STRATA_MMVQ_IL_PF kernels (hoisted group loads, exact integer forms): every variant bit-checked against
# native_mmvq for every type, shape, T and rows choice
add_test(NAME mmvq_il_parity_pf COMMAND mmvq_il_parity --pf 0,2,3,4,5,6)
set_tests_properties(mmvq_il_parity_pf PROPERTIES TIMEOUT 900)
# the wide dequant kernels (a thread per 8 values; sm_61's default), bitwise vs the per-group ones (GPU, synthetic,
# no model; --bench adds old/new timings)
add_executable(dequant_wide_parity src/kernels/dequant_wide_parity.cpp)
@@ -995,7 +1215,7 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
# the Gyro trellis decode kernels (TQ2_T / TQK6 / TQK7) on synthetic blocks: decode-once vs per-column bitwise, output
# checksums to compare two builds byte for byte, --bench for timings (GPU, no model)
add_executable(tq_pascal_bench src/kernels/tq_pascal_bench.cpp)
target_include_directories(tq_pascal_bench PRIVATE ${_strata_gpu_include_directories})
target_include_directories(tq_pascal_bench PRIVATE ${_strata_gpu_include_directories} ${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml)
target_link_libraries(tq_pascal_bench PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME tq_pascal_bench COMMAND tq_pascal_bench)
set_tests_properties(iq_multi_parity PROPERTIES ENVIRONMENT "STRATA_KQ_SPLIT=1") # the Q4_K / Q5_K / Q8_0 Split path against the old kernels
@@ -1023,11 +1243,22 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
add_executable(gdn_rec_parity src/prefill/gdn_rec_parity.cu)
target_link_libraries(gdn_rec_parity PRIVATE ${_strata_gpu_runtime_target})
add_test(NAME gdn_rec_parity COMMAND gdn_rec_parity)
# STRATA_GDN_TC=1: the tensor-core helpers of the scan, each on one warp against a double-precision reference. Any sm_80+
# card runs them; the scan itself needs 122 KB of shared memory a block (sm_80, sm_90), where gdn_rec_parity prints
# "not on this card" for the tc variant
add_test(NAME gdn_rec_parity_selftest_frag COMMAND gdn_rec_parity --selftest-frag)
set_tests_properties(gdn_rec_parity_selftest_frag PROPERTIES SKIP_RETURN_CODE 77)
# the prompt path's BF16 products against cuBLAS's own BF16 GEMM, within fp32 rounding (sm_7x runs them on the
# FP16 tensor cores; GPU, synthetic, no model; --bench adds timings)
add_executable(gemm_bf16_parity src/prefill/gemm_bf16_parity.cu)
target_link_libraries(gemm_bf16_parity PRIVATE strata_prefill )
add_test(NAME gemm_bf16_parity COMMAND gemm_bf16_parity)
# STRATA_DEQUANT_SIDE: Gemm::native with the dense dequant on a side stream (two scratches) against the serial form,
# bitwise (GPU, synthetic, no model; 77 without a device)
add_executable(gemm_dequant_side_parity src/prefill/gemm_dequant_side_parity.cu)
target_link_libraries(gemm_dequant_side_parity PRIVATE strata_prefill )
add_test(NAME gemm_dequant_side_parity COMMAND gemm_dequant_side_parity)
set_tests_properties(gemm_dequant_side_parity PROPERTIES SKIP_RETURN_CODE 77)
# Pascal, opt-in STRATA_DENSE_F32=1: the dense projections' FP16 GEMMs as cublasSgemm on FP32 copies, bitwise vs
# the cuBLAS FP16 call over a sweep of chunk sizes (GPU, synthetic, no model; --full sweeps every size, --bench
# adds timings). Not a ctest: whether the two agree is a property of the cuBLAS build, which this checks.
@@ -1040,6 +1271,12 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(gr_upmix_parity PRIVATE strata_prefill )
add_test(NAME gr_upmix_parity COMMAND gr_upmix_parity)
set_tests_properties(gr_upmix_parity PROPERTIES SKIP_RETURN_CODE 77)
# the two kernels of the layer-major prompt path (STRATA_PREFILL_LAYER_MAJOR=1, #3976): moe_shared_init and
# moe_scatter_add against a host reference, bitwise for the scatter (GPU, synthetic, no model)
add_executable(lm_kernels_parity src/prefill/lm_kernels_parity.cu)
target_link_libraries(lm_kernels_parity PRIVATE strata_prefill )
add_test(NAME lm_kernels_parity COMMAND lm_kernels_parity)
set_tests_properties(lm_kernels_parity PROPERTIES SKIP_RETURN_CODE 77)
endif()
# native_expert_grouped's group stride and fused SwiGLU + q8_1, bitwise vs the launches before them (GPU, synthetic,
# no model; --bench adds v1/new timings of a window's 48 calls in a graph)
@@ -1081,7 +1318,6 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(verify_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME verify_parity COMMAND verify_parity --selftest)
endif()
add_executable(fused_gr_bench src/kernels/fused_gr_bench.cpp)
target_include_directories(fused_gr_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(fused_gr_bench PRIVATE strata_kernels ${_strata_gpu_runtime_target})
@@ -1103,6 +1339,10 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_include_directories(gdn_parity PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(gdn_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME gdn_parity COMMAND gdn_parity --selftest)
# #3948: the same checks through the 4-blocks-a-head GDN step (the default on gfx1151), whose output-norm kernel writes the
# STRATA_QFUSE q8_1 image on its own
add_test(NAME gdn_parity_split COMMAND gdn_parity --selftest)
set_tests_properties(gdn_parity_split PROPERTIES ENVIRONMENT "STRATA_GDN_SPLIT=1")
# v100/gdn-chunk: the chunked GDN prompt recurrence vs the token recurrence and a host reference (GPU,
# synthetic, no model). The chunked path changes the summation order - FP32-level, not bit-exact - so the
@@ -1136,6 +1376,12 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(spec_verify_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME spec_verify_parity COMMAND spec_verify_parity)
set_tests_properties(spec_verify_parity PROPERTIES ENVIRONMENT "STRATA_OLD_SAMPLER=0;STRATA_SAMPLER_ONE_BLOCK=0")
# the explicit mu / NULL verifier of the gated proposal (STRATA_SPEC_PROB=1, pick gate) against a CPU oracle
add_executable(spec_proposal_parity src/kernels/spec_proposal_parity.cpp)
target_include_directories(spec_proposal_parity PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(spec_proposal_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME spec_proposal_parity COMMAND spec_proposal_parity)
set_tests_properties(spec_proposal_parity PROPERTIES ENVIRONMENT "STRATA_OLD_SAMPLER=0;STRATA_SAMPLER_ONE_BLOCK=0" TIMEOUT 120)
# the same fixtures on the other two sampled paths: the one-block kernel and the 0.1.20 kernel
add_test(NAME sampler_parity_one_block COMMAND sampler_parity --selftest)
set_tests_properties(sampler_parity_one_block PROPERTIES ENVIRONMENT "STRATA_OLD_SAMPLER=0;STRATA_SAMPLER_ONE_BLOCK=1")
@@ -1273,6 +1519,12 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
add_test(NAME ple_parity COMMAND ple_parity --selftest
--in "${STRATA_PLE_FIXTURE_DIR}/ple_in.bin" --out "${STRATA_PLE_FIXTURE_DIR}/ple_out.bin"
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
# STRATA_PLE_COMPACT: the raw IQ4_NL rows dequantized on the device against the host decode, bit for bit (synthetic, no model)
add_executable(ple_iq4nl_dequant_test src/kernels/ple_iq4nl_dequant_test.cpp)
target_include_directories(ple_iq4nl_dequant_test PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(ple_iq4nl_dequant_test PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME ple_iq4nl_dequant_test COMMAND ple_iq4nl_dequant_test)
set_tests_properties(ple_iq4nl_dequant_test PROPERTIES SKIP_RETURN_CODE 77)
# Plan v0.3 P7: INT8 KV storage vs a host reference and vs the FP16 path (GPU, synthetic, no model).
add_executable(kv_q8_parity src/kernels/kv_q8_parity.cpp)
@@ -1344,6 +1596,13 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(pin_budget_test PRIVATE strata_core ${_strata_gpu_runtime_target})
add_test(NAME pin_budget_test COMMAND pin_budget_test)
# #3350 / #3470: the conversation copy streams and events are pooled across short-lived threads (needs a device, else skips)
add_executable(conversation_copy_streams_test src/core/conversation_copy_streams_test.cpp)
target_include_directories(conversation_copy_streams_test PRIVATE include)
target_link_libraries(conversation_copy_streams_test PRIVATE strata_core ${_strata_gpu_runtime_target})
add_test(NAME conversation_copy_streams_test COMMAND conversation_copy_streams_test)
set_tests_properties(conversation_copy_streams_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 120)
if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
add_executable(pinned_shared_test src/core/pinned_shared_test.cpp)
target_link_libraries(pinned_shared_test PRIVATE strata_core ${_strata_gpu_runtime_target})
@@ -1356,6 +1615,24 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
target_link_libraries(ple_reader_test PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME ple_reader_selftest COMMAND ple_reader_test --selftest)
# #3602: the same reader under load - several clients, a tiny row cache, one in-flight slot, injected delays, rows on
# every page phase, late-writer checks, the reader after a failed read. CPU and a temp file only; 77 = no direct reads here.
add_executable(ple_reader_stress_test src/ngram/ple_reader_stress_test.cpp)
target_include_directories(ple_reader_stress_test PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(ple_reader_stress_test PRIVATE strata_kernels ${_strata_gpu_runtime_target})
add_test(NAME ple_reader_stress COMMAND ple_reader_stress_test)
set_tests_properties(ple_reader_stress PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 300)
# 0.2.7: the PLE residency rule (table + expert arena + headroom); header-only, needs nothing
add_executable(ple_residency_test src/core/ple_residency_test.cpp)
target_include_directories(ple_residency_test PRIVATE include)
add_test(NAME ple_residency_test COMMAND ple_residency_test)
# 0.2.7 (#3911): the qwen35 host sampler's penalties on a short (draft-vocab) logit row; header-only, CPU
add_executable(qwen35_sampling_test src/qwen35/sampling_test.cpp)
target_include_directories(qwen35_sampling_test PRIVATE include)
add_test(NAME qwen35_sampling_test COMMAND qwen35_sampling_test)
# The GPU test runs ON THE TARGET PC. `--selftest` allocates
# and writes real VRAM, so a machine without a device fails this test rather than skipping it silently -
# which is correct: an engine that targets one GPU should not report a pass for having no GPU.
@@ -1413,6 +1690,12 @@ if(STRATA_BUILD_TESTS)
add_executable(arch_defaults_test src/core/arch_defaults_test.cpp src/core/arch_defaults.cpp)
target_include_directories(arch_defaults_test PRIVATE include)
add_test(NAME arch_defaults_test COMMAND arch_defaults_test)
add_executable(kva_test src/prefill/kva_test.cpp) # KVA: the switch, the automatic tail, the reasons a read is turned away
target_include_directories(kva_test PRIVATE include)
add_test(NAME kva_test COMMAND kva_test)
add_executable(adapt_async_default_test src/core/adapt_async_default_test.cpp)
target_include_directories(adapt_async_default_test PRIVATE include)
add_test(NAME adapt_async_default_test COMMAND adapt_async_default_test)
add_executable(uma_cache_test src/core/uma_cache_test.cpp)
target_include_directories(uma_cache_test PRIVATE include)
add_test(NAME uma_cache_test COMMAND uma_cache_test)
@@ -1441,6 +1724,14 @@ if(STRATA_BUILD_TESTS OR STRATA_BUILD_CONVERSATION_TESTS)
add_executable(conv_cache_test src/program/conv_cache_test.cpp)
target_include_directories(conv_cache_test PRIVATE include)
add_test(NAME conv_cache_test COMMAND conv_cache_test)
# --adapt-decay outside 0..1 falls back to 0.7 with a line (#1284; header-only, CPU)
add_executable(adapt_decay_test src/program/adapt_decay_test.cpp)
target_include_directories(adapt_decay_test PRIVATE include)
add_test(NAME adapt_decay_test COMMAND adapt_decay_test)
# which window-size graph is freed when the next one does not fit (#3702; header-only, CPU)
add_executable(window_evict_test src/core/window_evict_test.cpp)
target_include_directories(window_evict_test PRIVATE include)
add_test(NAME window_evict_test COMMAND window_evict_test)
# which path computes the draft layer's prompt K/V, and STRATA_SPLIT_MTP_BATCH's parsing (header-only, CPU)
add_executable(draft_kv_plan_test src/program/draft_kv_plan_test.cpp)
target_include_directories(draft_kv_plan_test PRIVATE include)
@@ -1453,6 +1744,21 @@ if(STRATA_BUILD_TESTS OR STRATA_BUILD_CONVERSATION_TESTS)
add_executable(spec_prob_test src/core/spec_prob_test.cpp)
target_include_directories(spec_prob_test PRIVATE include)
add_test(NAME spec_prob_test COMMAND spec_prob_test)
# gated proposal packets (STRATA_SPEC_PROB=1, pick gate): masked measure, NULL tail, root rescue (header-only, CPU)
add_executable(gated_proposal_test src/core/gated_proposal_test.cpp)
target_include_directories(gated_proposal_test PRIVATE include)
add_test(NAME gated_proposal_test COMMAND gated_proposal_test)
# the admission law of that packet, enumerated exactly (no Monte Carlo), in the three gate modes (CPU)
add_executable(spec_admission_test src/core/spec_admission_test.cpp src/spec/draft_policy.cpp)
target_include_directories(spec_admission_test PRIVATE include)
add_test(NAME spec_admission_default COMMAND ${CMAKE_COMMAND} -E env --unset=STRATA_SPEC_PROB_GATE $<TARGET_FILE:spec_admission_test>)
add_test(NAME spec_admission_pick COMMAND ${CMAKE_COMMAND} -E env STRATA_SPEC_PROB_GATE=pick $<TARGET_FILE:spec_admission_test> pick)
add_test(NAME spec_admission_top COMMAND ${CMAKE_COMMAND} -E env STRATA_SPEC_PROB_GATE=top $<TARGET_FILE:spec_admission_test> top)
add_test(NAME spec_admission_exact_top_only COMMAND ${CMAKE_COMMAND} -E env STRATA_SPEC_PROB_GATE=TOP $<TARGET_FILE:spec_admission_test> pick)
# the per-card window cap of --draft dflash2 (header-only, CPU)
add_executable(draft_cap_test src/core/draft_cap_test.cpp)
target_include_directories(draft_cap_test PRIVATE include)
add_test(NAME draft_cap_test COMMAND draft_cap_test)
# the f32 -> bf16 rounding against ggml's, NaN included, over every f32 bit pattern (header-only, CPU)
add_executable(bf16_bits_test src/kernels/bf16_bits_test.cpp)
target_include_directories(bf16_bits_test PRIVATE include)
@@ -1467,8 +1773,13 @@ if(STRATA_BUILD_TESTS OR STRATA_BUILD_CONVERSATION_TESTS)
add_test(NAME batch_groups_test COMMAND batch_groups_test)
endif()
add_library(strata_kernels_cpu STATIC src/kernels/cpu/expert.cpp src/kernels/cpu/pool.cpp
src/kernels/cpu/expert_layout.cpp src/kernels/cpu/q2_avx2.cpp src/kernels/cpu/hadamard.cpp)
if(STRATA_CPU_X86)
add_library(strata_kernels_cpu STATIC src/kernels/cpu/expert.cpp src/kernels/cpu/pool.cpp
src/kernels/cpu/expert_layout.cpp src/kernels/cpu/q2_avx2.cpp src/kernels/cpu/hadamard.cpp)
else()
add_library(strata_kernels_cpu STATIC src/kernels/cpu/portable.cpp src/kernels/cpu/pool.cpp
src/kernels/cpu/expert_layout.cpp src/kernels/cpu/hadamard.cpp)
endif()
target_include_directories(strata_kernels_cpu PUBLIC include)
# Plan v0.3 P6: only the kernel files get the AVX-512 / AVX2 flags; the pool and the probe run on any x86-64,
# and the engine picks the kernels at run time (cpu_avx512_ok).
@@ -1538,6 +1849,13 @@ if(STRATA_NATIVE_EXPERTS)
foreach(_f GGML_AVX GGML_AVX2 GGML_FMA GGML_F16C GGML_BMI2)
set(${_f} ON CACHE BOOL "" FORCE)
endforeach()
# The DeepSeek engine (STRATA_BUILD_DSV41) builds ggml-cpu with BMI2 OFF: ggml's IQ1_S / IQ1_M dots use _pdep_u64 with it, and AMD
# Zen 1/2 run PDEP in microcode (EPYC 7452, IQ1_M one-token dot 1.8 GB/s against 32 GB/s without it; IQ1_S 2.3 against 37). The
# results are bit-identical either way (ds4_gg_bench --dump, types 19 and 29, 1-16 tokens). Other engines keep BMI2 until their
# next rebuild is looked at: no Qwen package changes with this commit.
if(STRATA_BUILD_DSV41)
set(GGML_BMI2 OFF CACHE BOOL "" FORCE)
endif()
elseif(STRATA_ISA_FLOOR)
# the experimental older-CPU build (STRATA_ISA_FLOOR above): one ggml-cpu for the floor, ggml's "sandybridge"
# feature set (SSE4.2 + AVX) or SSE4.2 alone - not the build host's, which may be newer than the PC it runs on.
@@ -1573,11 +1891,13 @@ if(STRATA_NATIVE_EXPERTS)
endif()
# part of strata_kernels_cpu (the pool calls these and they call its CPU probe: one library, so GNU ld's
# one-pass static linking has no cycle to resolve)
target_sources(strata_kernels_cpu PRIVATE src/kernels/cpu/native_expert.cpp src/kernels/cpu/iq_avx512.cpp
src/kernels/cpu/iq_avx2.cpp src/kernels/cpu/kq_avx2.cpp
src/kernels/cpu/kq_avx1.cpp src/kernels/cpu/q2_avx1.cpp src/kernels/cpu/iq_avx1.cpp
src/kernels/cpu/nvfp4_avx512.cpp src/kernels/cpu/nvfp4_avx2.cpp
src/kernels/cpu/tq_cpu.cpp src/kernels/cpu/tq_avx2.cpp src/kernels/cpu/tq_avx512.cpp)
target_sources(strata_kernels_cpu PRIVATE src/kernels/cpu/native_expert.cpp)
if(STRATA_CPU_X86)
target_sources(strata_kernels_cpu PRIVATE src/kernels/cpu/iq_avx512.cpp src/kernels/cpu/iq_avx2.cpp
src/kernels/cpu/kq_avx2.cpp src/kernels/cpu/kq_avx1.cpp src/kernels/cpu/q2_avx1.cpp
src/kernels/cpu/iq_avx1.cpp src/kernels/cpu/nvfp4_avx512.cpp src/kernels/cpu/nvfp4_avx2.cpp
src/kernels/cpu/tq_cpu.cpp src/kernels/cpu/tq_avx2.cpp src/kernels/cpu/tq_avx512.cpp)
endif()
target_include_directories(strata_kernels_cpu PRIVATE third_party/ggml)
if(MSVC)
set_source_files_properties(src/kernels/cpu/iq_avx512.cpp src/kernels/cpu/nvfp4_avx512.cpp
@@ -1596,7 +1916,7 @@ if(STRATA_NATIVE_EXPERTS)
set_source_files_properties(src/kernels/cpu/iq_avx1.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX")
else()
set_source_files_properties(src/kernels/cpu/iq_avx512.cpp src/kernels/cpu/nvfp4_avx512.cpp PROPERTIES COMPILE_OPTIONS
"-mavx512f;-mavx512bw;-mavx512vl;-mavx512dq;-mavx512vnni;-mavx512vbmi;-mf16c;-mfma")
"-mavx512f;-mavx512bw;-mavx512vl;-mavx512dq;-mavx512vnni;-mavx512vbmi;-mavx512vpopcntdq;-mbmi2;-mf16c;-mfma")
set_source_files_properties(src/kernels/cpu/iq_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mf16c")
set_source_files_properties(src/kernels/cpu/kq_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mf16c")
set_source_files_properties(src/kernels/cpu/nvfp4_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mf16c")
@@ -1633,6 +1953,21 @@ if(STRATA_NATIVE_EXPERTS)
add_executable(router_dot_parity src/kernels/cpu/router_dot_parity.cpp)
target_link_libraries(router_dot_parity PRIVATE strata_kernels_cpu)
if(STRATA_BUILD_TESTS)
# a token's native expert rows are the same bits at every number of tokens that share its expert (window-invariant arithmetic)
add_executable(native_arithmetic_parity src/kernels/cpu/native_arithmetic_parity.cpp)
target_link_libraries(native_arithmetic_parity PRIVATE strata_kernels_cpu)
add_test(NAME native_arithmetic_parity COMMAND native_arithmetic_parity)
add_test(NAME native_arithmetic_parity_avx COMMAND ${CMAKE_COMMAND} -E env
STRATA_FORCE_ISA=avx $<TARGET_FILE:native_arithmetic_parity>)
add_test(NAME native_arithmetic_parity_kq256 COMMAND ${CMAKE_COMMAND} -E env
STRATA_KQ256=1 $<TARGET_FILE:native_arithmetic_parity>)
add_test(NAME native_arithmetic_parity_mt1 COMMAND ${CMAKE_COMMAND} -E env STRATA_IQ_MT_MIN=1 $<TARGET_FILE:native_arithmetic_parity>)
add_test(NAME native_arithmetic_parity_no512 COMMAND ${CMAKE_COMMAND} -E env STRATA_NO_IQ512=1 $<TARGET_FILE:native_arithmetic_parity>)
add_test(NAME native_arithmetic_parity_ggml COMMAND ${CMAKE_COMMAND} -E env STRATA_NO_IQ512=1 STRATA_NO_IQ256=1 STRATA_NO_IQ4NL=1 $<TARGET_FILE:native_arithmetic_parity>)
add_test(NAME native_arithmetic_parity_nvfp4_avx2 COMMAND ${CMAKE_COMMAND} -E env
STRATA_NO_NVFP4_512=1 $<TARGET_FILE:native_arithmetic_parity>)
add_test(NAME native_arithmetic_parity_nvfp4_ggml COMMAND ${CMAKE_COMMAND} -E env
STRATA_NO_NVFP4_512=1 STRATA_NO_NVFP4_256=1 $<TARGET_FILE:native_arithmetic_parity>)
add_executable(pool_tasks_test src/kernels/cpu/pool_tasks_test.cpp)
target_link_libraries(pool_tasks_test PRIVATE strata_kernels_cpu)
add_test(NAME pool_tasks_test COMMAND pool_tasks_test)
@@ -1644,6 +1979,14 @@ if(STRATA_NATIVE_EXPERTS)
# against ggml-cpu's dot, on random blocks; --bench: ms per expert on one thread, the weights streamed from DRAM
add_executable(iq_avx2_parity src/kernels/cpu/iq_avx2_parity.cpp)
target_link_libraries(iq_avx2_parity PRIVATE strata_kernels_cpu)
# the 512-bit i-quant rows against ggml-cpu's dot: the exact rows bit for bit at 1-4 tokens, the engine's multi-token rows to float
# rounding, with the grid lookups as scalar loads and as gathers (SKIP without AVX-512)
add_executable(iq512_gather_parity src/kernels/cpu/iq512_gather_parity.cpp)
target_link_libraries(iq512_gather_parity PRIVATE strata_kernels_cpu)
if(STRATA_BUILD_TESTS)
add_test(NAME iq512_gather_parity_scalar COMMAND ${CMAKE_COMMAND} -E env STRATA_IQ_GATHER=0 $<TARGET_FILE:iq512_gather_parity>)
add_test(NAME iq512_gather_parity_gather COMMAND ${CMAKE_COMMAND} -E env STRATA_IQ_GATHER=1 $<TARGET_FILE:iq512_gather_parity>)
endif()
# MODEL-UPDATE: the pool's expert rows per format on T threads against the RAM bandwidth (a benchmark, no test)
add_executable(kq_cpu_bench src/kernels/cpu/kq_cpu_bench.cpp)
target_link_libraries(kq_cpu_bench PRIVATE strata_kernels_cpu)
@@ -1671,6 +2014,11 @@ if(STRATA_NATIVE_EXPERTS)
# tree leaves them to the FP16 prompt path
if(STRATA_GGML_DIR AND EXISTS "${STRATA_GGML_DIR}/ggml/src/ggml-cuda/template-instances/mmq-instance-tqk6.cu")
list(APPEND _strata_mmq_types tq2_t tqk6 tqk7)
# Gyro-L's pair trellis (types 147-149): the same fork's ggml has their MMQ instances
if(EXISTS "${STRATA_GGML_DIR}/ggml/src/ggml-cuda/template-instances/mmq-instance-tqp5.cu")
list(APPEND _strata_mmq_types tqp5 tqp6 tqp7)
set(_strata_mmq_tqp ON) # moe_mmq.cu names GGML_TYPE_TQP5-7 only where the ggml has them (STRATA_MMQ_TQP)
endif()
set(_strata_mmq_tq ON)
elseif(STRATA_MMQ_TQ_OVERLAY AND (STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP))
# an upstream llama.cpp: the trellis MMQ instances come from an overlay of the pinned ggml-cuda headers written at
@@ -1687,6 +2035,7 @@ if(STRATA_NATIVE_EXPERTS)
configure_file(third_party/ggml/ggml-common-tq.h "${_strata_tq_overlay_dir}/ggml-common-tq.h" COPYONLY)
configure_file(src/kernels/cuda/tq_lut_i8.cuh "${_strata_tq_overlay_dir}/tq_lut_i8.cuh" COPYONLY)
set(_strata_mmq_tq ON)
set(_strata_mmq_tqp ON) # the overlay adds the pair-trellis enumerators and instances
set(_strata_mmq_tq_overlay ON)
message(STATUS "trellis MMQ (Gyro prompts): overlay on the pinned ggml-cuda in ${_strata_tq_overlay_dir}")
if(STRATA_BUILD_TESTS)
@@ -1718,6 +2067,12 @@ if(STRATA_NATIVE_EXPERTS)
add_library(strata_mmq STATIC src/prefill/moe_mmq.cu src/prefill/moe_fused.cu src/prefill/moe_fused_iq.cu
src/prefill/mmq_nvfp4_w4a8.cu
src/prefill/ggml_cuda_host.cu ${_strata_quantize_cu})
if(STRATA_EXPERIMENTAL_SM60 AND NOT STRATA_HIP_GFX906)
# GP100 (sm_60): a resident layer's experts weight-stationary, one launch per product (moe_ws.hpp; opt-in at
# run time, STRATA_MMQ_WS=1, and only where every visible GPU is cc 6.0)
target_sources(strata_mmq PRIVATE src/prefill/moe_ws.cu src/prefill/moe_ws_gu_iq2.cu src/prefill/moe_ws_gu_iq3.cu
src/prefill/moe_ws_down.cu)
endif()
# q8_0: the drafter's matrices (E-9); STRATA_ORCA_Q4KS_MMQ adds q5_0 (#296, in _strata_mmq_types), and
# STRATA_MMQ_KQUANTS q4_k q5_k q5_1 q6_k, the experts of Unsloth's UD-Q4/Q5/Q6_K_XL (docs/QUANTS.md; the dispatch
# follows eddoursul/Strata 7706c1b). MODEL-UPDATE Phase 1: on CUDA they are a library of their own, strata_kq
@@ -1736,7 +2091,7 @@ if(STRATA_NATIVE_EXPERTS)
target_sources(strata_mmq PRIVATE ${_strata_mmq}/template-instances/mmq-instance-${_t}.cu)
endforeach()
if(_strata_mmq_tq_overlay)
foreach(_t tq2_t tqk6 tqk7)
foreach(_t tq2_t tqk6 tqk7 tqp5 tqp6 tqp7)
target_sources(strata_mmq PRIVATE ${_strata_tq_overlay_dir}/template-instances/mmq-instance-${_t}.cu)
endforeach()
# first on the include path: the patched mmq.cuh / ggml.h shadow the pinned ones for this target only
@@ -1746,6 +2101,12 @@ if(STRATA_NATIVE_EXPERTS)
# which exists only in the arch-specific target, so its one unit alone is built with 12x -> 12xa (the default
# binary is unchanged and runs on sm_121; a card without that image falls back to W4A8 at run time)
set(_strata_w4a4_archs ${CMAKE_CUDA_ARCHITECTURES})
# `native` is first replaced by the detected architectures (e.g. 120-real), so the 12x -> 12xa rewrite below applies
# to it too: with plain sm_120 ptxas refuses the FP4 MMA of this one unit (#4023)
if("native" IN_LIST _strata_w4a4_archs AND CMAKE_CUDA_ARCHITECTURES_NATIVE)
list(REMOVE_ITEM _strata_w4a4_archs native)
list(APPEND _strata_w4a4_archs ${CMAKE_CUDA_ARCHITECTURES_NATIVE})
endif()
# (two steps: CMake before 3.29 rejects a \2 whose optional group did not match, "120" alone)
list(TRANSFORM _strata_w4a4_archs REPLACE "^(12[0-9])$" "\\1a")
list(TRANSFORM _strata_w4a4_archs REPLACE "^(12[0-9])-(real|virtual)$" "\\1a-\\2")
@@ -1848,8 +2209,15 @@ if(STRATA_NATIVE_EXPERTS)
if(_strata_mmq_tq)
target_compile_definitions(strata_mmq PRIVATE STRATA_MMQ_TQ=1)
endif()
if(_strata_mmq_tqp)
target_compile_definitions(strata_mmq PRIVATE STRATA_MMQ_TQP=1)
endif()
target_link_libraries(strata_prefill PUBLIC strata_mmq)
target_compile_definitions(strata_prefill PUBLIC STRATA_PREFILL_MMQ=1 STRATA_PREFILL_FUSED=1)
if(STRATA_EXPERIMENTAL_SM60 AND NOT STRATA_HIP_GFX906)
# the weight-stationary experts (moe_ws.hpp) are in strata_mmq: the prompt path may take them (STRATA_MMQ_WS=1)
target_compile_definitions(strata_prefill PUBLIC STRATA_PREFILL_WS=1)
endif()
if(STRATA_BUILD_TESTS)
# #136: the fused int8 experts against a double-precision reference and the MMQ path (exit 77 below sm_80)
add_executable(prefill_fused_moe_test tests/cuda/prefill_fused_moe_test.cpp)
@@ -1861,12 +2229,40 @@ if(STRATA_NATIVE_EXPERTS)
target_link_libraries(prefill_fused_iq_test PRIVATE strata_mmq ggml-base ${_strata_gpu_runtime_target})
add_test(NAME prefill_fused_iq_test COMMAND prefill_fused_iq_test)
set_tests_properties(prefill_fused_iq_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 600)
# the opt-in STRATA_PF_FUSED_Q2N=1 on its own (the test leaves STRATA_PF_FUSED unset): the native Q2_0 pack's Q2_0 / Q2_0
# layers on sm_80+ (77 below sm_80 and without a card)
add_test(NAME prefill_fused_iq_test_q2n COMMAND prefill_fused_iq_test "--only=Q2_0 / Q2_0" --unset-pf-fused)
set_tests_properties(prefill_fused_iq_test_q2n PROPERTIES ENVIRONMENT "STRATA_PF_FUSED_Q2N=1" SKIP_RETURN_CODE 77 TIMEOUT 600)
# the opt-in STRATA_PF_FUSED_NT=256 kernels (256 threads, two blocks a multiprocessor, two-pass weight loads) against the
# same references; the batches of the test reach them through the Batch overload of experts_native (77 below sm_80)
add_test(NAME prefill_fused_iq_test_nt256 COMMAND prefill_fused_iq_test)
set_tests_properties(prefill_fused_iq_test_nt256 PROPERTIES ENVIRONMENT "STRATA_PF_FUSED_NT=256" SKIP_RETURN_CODE 77 TIMEOUT 600)
# the opt-in STRATA_PF_FUSED_F16=1 (f16 mma, rounding level) and STRATA_PF_FUSED_WS=1 (warp-specialized, the same F16 products;
# about 101 KB of shared memory at its defaults, more than an sm_86 / sm_89 / sm_120 card's 99 KB, so it is set up on
# sm_80 and sm_90 cards and the test skips (77) elsewhere)
add_test(NAME prefill_fused_iq_test_f16 COMMAND prefill_fused_iq_test)
set_tests_properties(prefill_fused_iq_test_f16 PROPERTIES ENVIRONMENT "STRATA_PF_FUSED_F16=1" SKIP_RETURN_CODE 77 TIMEOUT 600)
add_test(NAME prefill_fused_iq_test_ws COMMAND prefill_fused_iq_test)
set_tests_properties(prefill_fused_iq_test_ws PROPERTIES ENVIRONMENT "STRATA_PF_FUSED_WS=1" SKIP_RETURN_CODE 77 TIMEOUT 600)
# the MoE input quantized per token and scattered to its rows (mmq::quantize_scatter) against the per-row
# gather: the same q8_1 bytes for every MMQ layout (--bench: both at 2K, 8K and 32K tokens)
add_executable(prefill_mmq_scatter_test tests/cuda/prefill_mmq_scatter_test.cpp)
target_link_libraries(prefill_mmq_scatter_test PRIVATE strata_mmq ggml-base ${_strata_gpu_runtime_target})
add_test(NAME prefill_mmq_scatter_test COMMAND prefill_mmq_scatter_test)
set_tests_properties(prefill_mmq_scatter_test PROPERTIES TIMEOUT 120)
if(STRATA_EXPERIMENTAL_SM60)
# mmq::swiglu_quant: swiglu + the down product's q8_1 rows in one kernel against the two, byte for byte
add_executable(prefill_swiglu_quant_test tests/cuda/prefill_swiglu_quant_test.cpp)
target_link_libraries(prefill_swiglu_quant_test PRIVATE strata_mmq ggml-base ${_strata_gpu_runtime_target})
add_test(NAME prefill_swiglu_quant_test COMMAND prefill_swiglu_quant_test)
set_tests_properties(prefill_swiglu_quant_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 120)
# the weight-stationary experts (STRATA_MMQ_WS) against the per-group MMQ, byte for byte, on random experts; the
# verdict is given on a compute capability 6.0 card only (77 elsewhere)
add_executable(prefill_ws_parity_test tests/cuda/prefill_ws_parity_test.cpp)
target_link_libraries(prefill_ws_parity_test PRIVATE strata_mmq ggml-base ${_strata_gpu_runtime_target})
add_test(NAME prefill_ws_parity_test COMMAND prefill_ws_parity_test)
set_tests_properties(prefill_ws_parity_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 600)
endif()
endif()
if(STRATA_BUILD_TESTS AND _strata_mmq_tab_on)
# STRATA_MMQ_DIRECT: the pointer-table MMQ bit for bit against the gathered MMQ (exit 77 below sm_80)
@@ -1925,7 +2321,7 @@ if(STRATA_NATIVE_EXPERTS)
list(APPEND _strata_mmq_sources "${_strata_mmq}/template-instances/mmq-instance-${_t}.cu")
endforeach()
if(_strata_mmq_tq_overlay)
foreach(_t tq2_t tqk6 tqk7)
foreach(_t tq2_t tqk6 tqk7 tqp5 tqp6 tqp7)
list(APPEND _strata_mmq_sources "${_strata_tq_overlay_dir}/template-instances/mmq-instance-${_t}.cu")
endforeach()
endif()
@@ -1950,6 +2346,9 @@ if(STRATA_NATIVE_EXPERTS)
if(_strata_mmq_tq)
target_compile_definitions(strata_mmq PRIVATE STRATA_MMQ_TQ=1)
endif()
if(_strata_mmq_tqp)
target_compile_definitions(strata_mmq PRIVATE STRATA_MMQ_TQP=1)
endif()
target_link_libraries(strata_mmq PUBLIC ggml-base hip::host roc::hipblas)
target_link_libraries(strata_prefill PUBLIC strata_mmq)
target_compile_definitions(strata_prefill PUBLIC STRATA_PREFILL_MMQ=1 STRATA_PREFILL_FUSED=1)
@@ -2003,6 +2402,20 @@ if(STRATA_NATIVE_EXPERTS)
add_executable(native_expert_bench src/kernels/native_expert_bench.cpp)
target_include_directories(native_expert_bench PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(native_expert_bench PRIVATE strata_kernels strata_kernels_cpu strata_artifact ${_strata_gpu_runtime_target})
if(STRATA_ENABLE_CUDA AND STRATA_EXPERIMENTAL_SM60)
# STRATA_EXPERT_NIBBLE (include/strata/core/expert_nibble.hpp): the 4-bit gate/up expert slots against the
# production kernels and the GGUF blobs, bit for bit, on a real model's layers (needs a GP100); --header: the
# contract's host half, no GPU
add_executable(iq_nibble_parity src/kernels/iq_nibble_parity.cpp)
target_include_directories(iq_nibble_parity PRIVATE ${_strata_gpu_include_directories})
target_link_libraries(iq_nibble_parity PRIVATE strata_kernels strata_kernels_cpu strata_artifact ggml-base ${_strata_gpu_runtime_target})
if(STRATA_BUILD_TESTS)
add_test(NAME iq_nibble_parity_header COMMAND iq_nibble_parity --header)
# the synthetic experts of every gate/up x down format pair, no model (skips with 77 on a card that is not a GP100)
add_test(NAME iq_nibble_parity_synthetic COMMAND iq_nibble_parity --synthetic-only)
set_tests_properties(iq_nibble_parity_synthetic PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 900)
endif()
endif()
# MODEL-UPDATE: the dense Q8_0 GEMVs at the engine's shapes against the memory bandwidth (a benchmark, no test)
add_executable(q8_gemv_bench src/kernels/q8_gemv_bench.cpp)
target_include_directories(q8_gemv_bench PRIVATE ${_strata_gpu_include_directories})
@@ -2041,7 +2454,11 @@ if(STRATA_NATIVE_EXPERTS)
# APR: the trellis expert types and the Hadamard activation rotation, when ggml comes from the agention fork
# (the only tree with these types; marked by its ggml-cuda/tq.cuh)
if(STRATA_GGML_DIR AND EXISTS "${STRATA_GGML_DIR}/ggml/src/ggml-cuda/tq.cuh")
foreach(_pair tqk6/tqk7 tq2_t/tq2_t)
set(_strata_tq_pairs tqk6/tqk7 tq2_t/tq2_t)
if(EXISTS "${STRATA_GGML_DIR}/ggml/src/ggml-cuda/template-instances/mmq-instance-tqp5.cu") # Gyro-L's pair trellis
list(APPEND _strata_tq_pairs tqp5/tqp5 tqp6/tqp6 tqp7/tqp7)
endif()
foreach(_pair ${_strata_tq_pairs})
string(REPLACE "/" "_" _name "${_pair}")
add_test(NAME native_expert_parity_${_name} COMMAND native_expert_parity --synthetic ${_pair})
add_test(NAME native_expert_parity_hadamard_${_name} COMMAND native_expert_parity --synthetic-hadamard ${_pair})
@@ -2115,6 +2532,13 @@ if(STRATA_BUILD_TESTS)
add_test(NAME tq_cpu_parity COMMAND tq_cpu_parity)
add_test(NAME tq_cpu_parity_avx2 COMMAND ${CMAKE_COMMAND} -E env STRATA_FORCE_AVX2=1 $<TARGET_FILE:tq_cpu_parity>)
set_tests_properties(tq_cpu_parity tq_cpu_parity_avx2 PROPERTIES TIMEOUT 300 SKIP_RETURN_CODE 77)
# Gyro-L's pair trellis (types 147-149) and the older trellis rows on synthetic blocks, no ggml and no model: the scalar
# kernel against a bit-by-bit reference, every ISA bit for bit equal to scalar, the type predicates unchanged
add_executable(tqp_cpu_test src/kernels/cpu/tqp_cpu_test.cpp)
target_link_libraries(tqp_cpu_test PRIVATE strata_kernels_cpu)
add_test(NAME tqp_cpu_test COMMAND tqp_cpu_test)
add_test(NAME tqp_cpu_test_avx2 COMMAND ${CMAKE_COMMAND} -E env STRATA_FORCE_AVX2=1 $<TARGET_FILE:tqp_cpu_test>)
set_tests_properties(tqp_cpu_test tqp_cpu_test_avx2 PROPERTIES TIMEOUT 120)
endif()
add_executable(expert_parity src/kernels/cpu/expert_parity.cpp)
@@ -2144,6 +2568,10 @@ if(STRATA_BUILD_TESTS)
add_executable(warm_policy_test src/prefill/warm_policy_test.cpp)
target_include_directories(warm_policy_test PRIVATE include)
add_test(NAME warm_policy_test COMMAND warm_policy_test)
# #4005: which streamed expert copies get a copied event (header-only, CPU)
add_executable(copy_events_test src/prefill/copy_events_test.cpp)
target_include_directories(copy_events_test PRIVATE include)
add_test(NAME copy_events_test COMMAND copy_events_test)
# issue #29: tiny batches with sleeping workers, as on a big-VRAM card; a hang fails it
add_executable(pool_stress src/kernels/cpu/pool_stress.cpp)
target_link_libraries(pool_stress PRIVATE strata_kernels_cpu)
@@ -2197,6 +2625,32 @@ if(STRATA_BUILD_TESTS)
add_test(NAME aux_cpus_test COMMAND aux_cpus_test)
endif()
# CUDA release kernels must not spill or keep a per-thread array in local memory: every access there is a round trip
# to device memory in the hottest loops the engine has, and nothing on the build machine says so without -Xptxas -v
# or a GPU. tools/check_cuda_spills.py reads the resource usage the compiler wrote into the kernel libraries
# (cuobjdump: STACK and LOCAL per entry function and per sm_) and fails on any kernel outside its ALLOWED table, which
# carries the measured allowance of the kernels that hold such an array on purpose. CUDA only: HIP has no cuobjdump.
if(STRATA_ENABLE_CUDA AND STRATA_BUILD_TESTS AND NOT STRATA_ENABLE_HIP AND NOT STRATA_HIP_GFX906)
find_package(Python3 QUIET COMPONENTS Interpreter)
if(Python3_Interpreter_FOUND)
set(_strata_spill_libs)
foreach(_t strata_kernels strata_core strata_engine strata_prefill strata_qwen35 strata_mmq strata_mmq_w4a4 strata_kq)
if(TARGET ${_t})
list(APPEND _strata_spill_libs $<TARGET_FILE:${_t}>)
endif()
endforeach()
add_test(NAME cuda_spill_guard COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/tools/check_cuda_spills.py
--cuobjdump ${CUDAToolkit_BIN_DIR}/cuobjdump${CMAKE_EXECUTABLE_SUFFIX} ${_strata_spill_libs})
set_tests_properties(cuda_spill_guard PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 600)
if(NOT WIN32) # its stand-in demanglers are sh scripts
add_test(NAME check_cuda_spills_unit COMMAND ${Python3_EXECUTABLE} -m unittest tools.test_check_cuda_spills
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
endif()
else()
message(STATUS "strata: no python3 - the cuda_spill_guard test is NOT registered")
endif()
endif()
if(STRATA_BUILD_TESTS)
enable_testing()
# Model-level checks may be absent from source distributions.
@@ -2224,7 +2678,7 @@ if(STRATA_HIP_GFX906)
# f16kq::covers() answers false and the int8 kernels run, as on every other card)
if(_strata_mmq_tq_overlay)
set(_strata_tq_hip ${_strata_tq_overlay_dir}/quantize.cu)
foreach(_t tq2_t tqk6 tqk7)
foreach(_t tq2_t tqk6 tqk7 tqp5 tqp6 tqp7)
list(APPEND _strata_tq_hip ${_strata_tq_overlay_dir}/template-instances/mmq-instance-${_t}.cu)
endforeach()
list(APPEND _strata_cu ${_strata_tq_hip})
@@ -2238,3 +2692,19 @@ if(STRATA_HIP_GFX906)
add_executable(gfx906_qsa_query_swizzle_probe bench/gfx906_qsa_query_swizzle_probe.cpp)
target_link_libraries(gfx906_qsa_query_swizzle_probe PRIVATE strata_kernels ${_strata_gpu_runtime_target})
endif()
# ---- DeepSeek-V4.1 on a box with no GPU toolkit ------------------------------------------------
# The whole DSV41 block above lives inside if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP), so with neither backend on
# -DSTRATA_BUILD_DSV41=ON was silently unused and the host-side benchmark could not be built at all (found on p1004,
# 2026-10-23: the CPU lane is exactly what a box without a usable GPU needs to measure). This branch builds the one target
# that has no device code, so the CPU-lane A/B runs anywhere.
if(STRATA_BUILD_DSV41 AND NOT STRATA_ENABLE_CUDA AND NOT STRATA_ENABLE_HIP AND NOT STRATA_HIP_GFX906)
message(STATUS "STRATA_BUILD_DSV41: no GPU backend, building the host-side benchmark only (ds4_gg_bench)")
if(STRATA_NATIVE_EXPERTS)
add_executable(ds4_gg_bench tests/dsv41/gg_dot_bench.cpp src/dsv41/ds4_gg_cpu.cpp)
target_include_directories(ds4_gg_bench PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include ${CMAKE_CURRENT_SOURCE_DIR}/src/dsv41
${CMAKE_CURRENT_SOURCE_DIR}/third_party/ggml)
target_link_libraries(ds4_gg_bench PRIVATE strata_kernels_cpu Threads::Threads)
target_compile_definitions(ds4_gg_bench PRIVATE STRATA_DSV41_TQ_CPU=1)
endif()
endif()
+4 -1
View File
@@ -56,7 +56,8 @@ RUN set -e; \
ver="${ENGINE_VERSION:-$(sed -n 's/^[[:space:]]*project([^)]*VERSION[[:space:]]*\([0-9][0-9.]*\).*/\1/p' CMakeLists.txt | head -1)}"; \
cudafull=$(nvcc --version | sed -n 's/.*release \([0-9][0-9]*\.[0-9][0-9]*\).*/\1/p' | head -1); cudamajor=${cudafull%%.*}; \
if [ "$cudamajor" = 13 ]; then mindriver=580.65; else mindriver=575.51; fi; \
echo "{\"version\": \"$ver\", \"source\": \"docker build\", \"archs\": \"$CUDA_ARCHITECTURES\", \"cuda\": \"$cudafull\", \"portable\": true}" > /stage/BUILD.json; \
vision_archs=""; [ -f /stage/strata-vision ] && vision_archs=", \"vision_archs\": \"$CUDA_ARCHITECTURES\""; \
echo "{\"version\": \"$ver\", \"source\": \"docker build\", \"archs\": \"$CUDA_ARCHITECTURES\", \"cuda\": \"$cudafull\", \"portable\": true${vision_archs}}" > /stage/BUILD.json; \
python3 packaging/collect_libs.py --binary build/strata --search /usr/local/cuda/lib64 --out /stage --also 'libcublasLt.so*'; \
archs=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do archs="$archs --arch sm_$a"; done; \
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
@@ -67,6 +68,8 @@ RUN set -e; \
# ---- 2. the Terminal (the release archive, built here)
FROM ${RUNTIME_BASE} AS terminal
ARG STRATA_COMMIT=
ENV STRATA_BUILD_COMMIT=${STRATA_COMMIT}
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends python3 ca-certificates git && rm -rf /var/lib/apt/lists/*
COPY . /src/
+121 -337
View File
@@ -1,404 +1,188 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <b>Deutsch</b> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<h3 align="center">Ein Spitzenmodell auf deinem eigenen Gaming-PC</h3>
<p align="center">Unterstützt NVIDIA, AMD, Intel · Windows, Linux, Mac · kostenlos und Open Source</p>
<h3 align="center">Leistungsstarke KI auf deinem eigenen Computer.</h3>
<p align="center">Chatten, programmieren und mit Bildern arbeiten, lokal.</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#modelle">Modelle</a> · <a href="#installation"><b>Installation</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#tools">Tools</a> ·
<a href="#hilfe">Hilfe</a> · <a href="#strata-unterstützen">Unterstützen</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Ein Voxel-Pagodengarten aus einem einzigen Prompt, auf einer RTX 5070 mit Strata (IQ3_S, 128K Kontext) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">ganzes Video (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Strata herunterladen"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT-Lizenz"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Strata auf Buy Me a Coffee unterstützen"></a>
</p>
Strata lässt **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** laufen, ein Modell mit 125
Milliarden Parametern, für das man sonst einen Server braucht, auf einem normalen PC. Es chattet, schreibt Code, liest
Bilder und arbeitet mit deinen Apps und Coding-Agents zusammen. Deine Chats und Dateien bleiben auf deinem PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <b>Deutsch</b> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
**Neu hier?** Sieh dir an, [was du brauchst](#was-du-brauchst), und [installiere dann die App](#option-a-die-strata-app).
**Kostenlos, Open Source und ganz in deiner Hand.** Nutze Strata auf dem Desktop oder mit deinen liebsten Coding-Tools. Kein Konto nötig.
- **Läuft auf einem Gaming-PC.** 32 GB Arbeitsspeicher (RAM) und Grafikspeicher zusammen reichen für den Anfang.
- **Passt zu deinen Werkzeugen.** Strata spricht die OpenAI- und die Anthropic-API, deshalb verbinden sich Claude Code, Codex CLI, OpenCode und andere Agents damit.
- **Liest Bilder.** Das ist standardmäßig eingeschaltet.
- **Kostenlos.** MIT-Lizenz, kein Konto, keine Cloud.
- **Privat.** Keine Telemetrie. Was Strata herunterlädt, wann es das Internet nutzt und wie es Downloads prüft, steht in [docs/SAFETY.md](docs/SAFETY.md).
## Modelle
Version 0.2.2 · Engine 0.1.47
Wir empfehlen **zuerst ISTA-DASLab**, danach **Unsloth** und andere Anbieter, soweit ein passendes Modell in deinen Speicher passt.
## Was du brauchst
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**Du brauchst mindestens 32 GB Arbeitsspeicher (RAM) und Grafikspeicher (VRAM) zusammen.** Bei einem Mac oder einem Ryzen AI Max
ist das der gemeinsame Speicher. Mit mehr Speicher läuft ein größeres oder ein schnelleres Modell.
- Ein PC mit 32 GB und einer Grafikkarte mit 12-16 GB führt den Coder aus.
- Mit 64 GB RAM laufen alle Größen.
| Dein Computer | Details |
| Anbieter | Auswahl |
| --- | --- |
| **NVIDIA** GeForce RTX 20, 30, 40, 50 Serie (12 GB oder mehr) | Installiere die App oder das Terminal. |
| **AMD** Radeon RX 6800 und neuer, Radeon AI PRO R9700 (12 GB oder mehr) | [AMD-Karten](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [Intel-Engine](docs/INTEL.md) |
| **Apple-Silicon-Mac** (M1 oder neuer, macOS 15 oder neuer) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** und andere Systeme mit gemeinsamem Speicher | [Strix Halo](docs/STRIX_HALO.md) |
| Ältere Karten und Prozessoren | [Ältere GPUs](docs/OLDER_GPUS.md), [Ältere CPUs](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · empfohlen** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) zum Programmieren. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Geringerer Speicherbedarf, aber langsamer. |
| **Weitere Optionen** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
Außerdem nötig:
[Größen, Speicherbedarf und Kompatibilität vergleichen](docs/MODELS.md).
- etwa 80 GB freier Speicherplatz (mit einer SSD startet es beim ersten Mal viel schneller)
- Windows 10 / 11, Linux oder macOS
- ein aktueller Grafiktreiber
</details>
Zwei oder drei Karten desselben Herstellers können sich ein Modell teilen ([Multi-GPU](docs/MULTI_GPU.md)). Strata teilt ein
Modell nie auf NVIDIA- und AMD-Karten auf. Die vollständige Liste: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Das sind die derzeit dokumentierten Pakete für dieses Modell.
[Modellanleitung und Kompatibilität](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
In der **Desktop-App** und im **Terminal**. Was nicht in den Arbeitsspeicher passt, wird von deiner SSD gelesen. Darum brauchst du viel freien Speicherplatz.
| Set (IST-DAS) | Download | Freier Speicherplatz nötig |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | etwa 245 GB |
| 2.00 bpw | 257 GB | etwa 265 GB |
| 2.25 bpw | 274 GB | etwa 280 GB |
Läuft auf NVIDIA RTX 30, 40 und 50 (Windows und Linux) und auf AMD Radeon AI PRO R9700 (Linux). Das Setup lädt die Dateien und die DeepSeek-Engine herunter und startet dann das Modell.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Gemessen mit dem 2.00-bpw-Set auf Strata 0.2.5. „Ausgabe“ ist das Tempo der Antwort, „Prompt“ zeigt, wie schnell Strata deine Eingabe liest.
| Hardware | Ausgabe | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4 bis 6 Token/s | etwa 110 Token/s (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9 bis 11 Token/s | etwa 155 Token/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21 bis 25 Token/s | 320 bis 360 Token/s (8K) |
| 4x Radeon AI PRO R9700 | 74 bis 77 Token/s | 400 bis 450 Token/s (8K) |
[Speicher, Festplatte und Details](docs/DEEPSEEK41.md) · [Versionshinweise](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Unsicher, was passt? [Wähle ein Modell für deinen Computer](docs/MODELS.md#pick-by-memory).
## Installation
Such dir eine Option aus. Alle starten dieselbe Engine und nutzen dieselben Modelle.
<a id="option-a-the-strata-app"></a>
### Option A: die Strata-App
### Desktop-App
Der einfachste Weg. Die App bringt die Engine mit: installieren, ein Modell wählen, chatten. Lade die Datei für dein
System und deine Grafikkarte von der [Releases-Seite](https://github.com/Niko1221/Strata/releases/latest) herunter. Du bist
unsicher, welche du brauchst? Lies den Absatz unter der Tabelle.
**[Strata für Windows, Linux oder Mac herunterladen →](https://github.com/Niko1221/Strata/releases/latest)**
| System | Deine Grafikkarte | Datei zum Herunterladen |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, Treiber 580 oder neuer | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA, ältere Karte oder Treiber älter als 580 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon RX 9000 Serie oder Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon RX 7000 Serie oder Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, Treiber 580 oder neuer | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, ältere Karte oder Treiber älter als 580 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 Serie oder Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 Serie | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 oder neuer) | eingebaut | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
Installiere die App, wähle ein Modell und leg mit dem Chat los. Es gibt einen Installer pro System. Beim ersten Start lädt die App die Engine für deine Grafikkarte herunter. [Hilfe zum Installer](docs/INSTALL.md#the-app).
**Wenn du unsicher bist:** Bei NVIDIA nimmst du `cuda13`, wenn dein Treiber Version 580 oder neuer ist, sonst `cuda12`.
`cuda12` ist für ältere Karten und ältere Treiber. Die Dateien `hip` und `amd` sind für AMD-Karten. Für andere AMD-Karten
(RX 6000) gibt es noch keine fertige App: siehe [AMD-Karten](docs/AMD_HIP.md).
<sub>Dateien: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
Danach:
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Strata-Desktop-App mit Chat und Live-Codevorschau nebeneinander">
<br><sub>Chat, Dateien und Live-Vorschau in einem Fenster.</sub>
</p>
- **Windows:** Starte die `.exe`. Sie wird für deinen Benutzer installiert und braucht keine Administratorrechte. Wenn
Windows "Der Computer wurde durch Windows geschützt" meldet, wähle **Weitere Informationen** und dann **Trotzdem
ausführen** (der Installer ist noch nicht signiert).
- **Linux:** Mach die Datei ausführbar (`chmod +x`) und starte sie. Sie braucht FUSE 2: `sudo apt install libfuse2`
(Ubuntu 22.04) oder `libfuse2t64` (Ubuntu 24.04 und neuer).
- **Mac:** Öffne die `.dmg` und zieh Strata in den Ordner "Programme".
<a id="option-b-the-terminal-one-line"></a>
Auf dem ersten Bildschirm liest die App deinen PC aus und empfiehlt ein passendes Modell. Drück auf Installieren. Das Modell
ist ein großer Download (etwa 40-70 GB) und wird nach einer Unterbrechung dort fortgesetzt, wo er aufgehört hat. Wenn du
das Fenster schließt, beenden sich Strata und die Engine.
### Terminal-App
### Option B: das Terminal (eine Zeile)
Lieber im Terminal? Installiere Strata und starte dann **`strata launch`**.
Das Terminal ist der Befehl `strata`, gedacht für Skripte, Server und KI-Agents.
<details open>
<summary><b>Installationsbefehle</b></summary>
Linux und Mac:
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows (PowerShell):
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Das Skript prüft deinen PC, lädt das Terminal herunter, kontrolliert die SHA-256-Prüfsumme und entpackt es in deinen
Home-Ordner. Es braucht keine Administratorrechte. Öffne unter Windows danach ein neues Terminalfenster. Dann startest du
Strata:
Öffne unter Windows danach ein neues Terminal. Dann:
```sh
strata launch
```
Es stellt dir eine Frage nach der anderen: Modell, Kontext, Bilder, wer sich verbinden darf. Danach lädt es die Engine für
deine Grafikkarte herunter (signiert und vor der Nutzung probeweise gestartet), dazu das Modell, und beginnt zu laufen.
Für Skripte und KI-Agents gibt es Flags:
```sh
strata launch --model q2_0 --yes --no-tui
```
Wenn du das Installationsskript vor dem Ausführen lesen willst, lade es zuerst mit `curl -fsSL <address> -o install.sh`
herunter, lies es und führe dann `sh install.sh` aus. Alle Bildschirme und Flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Option C: deine KI richtet es ein
Füge das in Claude Code, Cursor, Codex, GitHub Copilot oder einen anderen Coding-Assistenten ein:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
Der Assistent prüft deine Grafikkarte, deinen Arbeitsspeicher und den Speicherplatz, wählt das passende Modell, installiert
und startet es und sagt dir, wie du deine Apps verbindest. KI-Werkzeuge können Strata auch über den
[MCP-Server](docs/MCP_SERVER.md) installieren, starten und stoppen.
Du willst es lieber selbst bauen? Siehe [Aus dem Quellcode bauen](#aus-dem-quellcode-bauen).
## Modelle
Die App und `strata launch` empfehlen ein Modell passend zu deinem Speicher. Es ist ein einziges Modell in mehreren
Größen, mehr oder weniger stark komprimiert. Kleinere Größen sind schneller. Größere Größen sind etwas klüger.
### Wenig Speicher? Probier Gyro
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (von agentionai) ist die kleinste Möglichkeit, dieses Modell auszuführen.
| Größe | Benötigter Speicher (RAM + VRAM) | Download | Was es ist |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 GB | 58,5 GB | Für PCs, auf denen nichts anderes läuft. Die Qualität ist geringer als bei IQ2_XS. |
| **Gyro-M** | 35 GB | 92,0 GB | Ungefähr die Qualität von IQ3_XXS bei etwa 20 % weniger GPU-Speicher. Es schreibt Antworten langsamer als Q2_0: 16 % auf einer RTX 3060 mit 12 GB, 21 % auf einer R9700. |
Gyro läuft auf NVIDIA- und AMD-Karten (bei AMD experimentell) auf Intel-Arc-Karten unter Linux (getestet auf einer Arc A750) und ab 0.2.2 auf Macs mit Apple Silicon (Gyro-S auf einem Mac mini mit 32 GB: 9 Tokens/s, die Experten werden von der SSD gelesen). Details und Messwerte:
[docs/GYRO.md](docs/GYRO.md). Starte es mit `strata launch --model gyro-s`.
### Die anderen Größen
| Dein Speicher (RAM + VRAM) | Nimm | Hinweise |
| --- | --- | --- |
| **32 GB** | **Gyro-S**, oder den **[Coder](docs/MODELS.md#coder)** für Code | Der Coder braucht 32 GB RAM und behält die Hälfte der Experten, deshalb ist er außerhalb von Code schwächer |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 GB), oder **Gyro-M** (35 GB) | Swift denkt kürzer nach, bevor es antwortet. Gyro-M kommt der Qualität von IQ3_XXS nahe, schreibt Antworten aber 16-21 % langsamer als Q2_0 |
| **48 GB** | **IQ3_XXS** (47,0 GB) | Langsamer, besser |
| **64 GB** | **IQ3_S** (54,8 GB), das Original oder Swift 1.5 | Am langsamsten, am besten. Es erreicht in den veröffentlichten Tests das volle Modell |
| **96 GB oder mehr** | **UD-IQ4_XS** von Unsloth (etwa 72 GB RAM) bis **Q8_0** (etwa 136 GB), 4 bis 8 Bit | Näher am vollen Modell. Bei weniger RAM liest Strata einen Teil von der SSD, das ist viel langsamer |
Auch mit weniger RAM, als eine Zeile nennt, kann es funktionieren: Strata wechselt von selbst in seinen
[Modus für wenig RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Rechne mit weniger Tempo, mit einer kleinen
Grafikkarte mit deutlich weniger.
Weitere Auswahl:
- **[Swift 1.5](docs/MODELS.md#swift-15)** ist eine Feinabstimmung, die kürzer nachdenkt, bevor sie antwortet, bei etwa gleicher Qualität und gleichem RAM-Bedarf.
- **[NVFP4](docs/NVFP4.md)** ist NVIDIAs 4-Bit-Version für NVIDIA-Karten. Sie braucht etwa 80 GB RAM, um alle Experten im RAM zu halten.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** ist ein kleineres, anderes Modell, das Bilder liest und mehrere Anfragen gleichzeitig beantworten kann.
- **[OrcaRouters Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** richtest du von Hand ein.
Alle Größen, Downloads und was wo läuft: [docs/MODELS.md](docs/MODELS.md).
## Wie schnell ist es?
Gemessen mit einer Grafikkarte mit 12 GB (RTX 5070, Ryzen 5 7600, 64 GB RAM). Ergebnisse anderer Karten findest du hier:
[Tempo jeder Größe](docs/MODELS.md#how-fast-is-each-size), [Ergebnisse der Community](docs/COMMUNITY_BENCHMARKS.md).
| Modell | Antworten erscheinen mit | Liest deinen Prompt mit |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 Tokens/s | `████████████` 2.650 Tokens/s |
| **IQ2_XS** | `██████████░░` 79 Tokens/s | `█████████░░░` 2.090 Tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 Tokens/s | `████████░░░░` 1.750 Tokens/s |
| **IQ3_S** | `███████░░░░░` 53 Tokens/s | `███████░░░░░` 1.620 Tokens/s |
| **Coder** | `███████░░░░░` 55 Tokens/s | `██████████░░` 2.180 Tokens/s |
Ein Token ist etwa ¾ eines Wortes, 60 Tokens pro Sekunde sind also schneller, als du lesen kannst. "Liest deinen Prompt"
beschreibt, wie schnell Strata ein langes Dokument, Code oder einen Chatverlauf aufnimmt.
Gemessen mit einer AMD-Karte mit 16 GB (RX 9070 XT, Ryzen 9 3900X, 47 GB RAM):
| Modell | Antworten erscheinen mit | Liest deinen Prompt mit |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 Tokens/s | `█████░░░░░░░` 1.160 Tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 Tokens/s | `█████░░░░░░░` 1.110 Tokens/s |
| **Coder** | `██████░░░░░░` 44 Tokens/s | `██████░░░░░░` 1.420 Tokens/s |
Die Balken haben in beiden Tabellen denselben Maßstab. Eine Karte mit mehr VRAM ist schneller. Eine RTX 3090 (24 GB) sollte
etwa 100-140 Tokens pro Sekunde schreiben. Die Engine-Version hinter jeder Zahl steht in [DETAILS.md](docs/DETAILS.md#speed-measured).
## Benutzung
### In der App
Die App hat einen **Chat**, einen **Monitor** für das Modell sowie für GPU, CPU und RAM, eine Seite **Modelle** und die
**Einstellungen**. Der Chat kann in einem Ordner auf deinem PC arbeiten. Im Modus **Manuell** fragt er nach, bevor er eine
Datei ändert oder einen Befehl ausführt. Mit `/loop 10m <what to do>` wiederholst du eine Nachricht alle 10 Minuten, solange
der Chat geöffnet ist.
Der Chat kann das Web mit DuckDuckGo (oder Bing) durchsuchen, ohne Konto. Unter **Einstellungen, Tools** kannst du einen
Suchschlüssel hinzufügen. Die Suche ist die einzige Funktion, die etwas von deinem PC sendet (die Suchbegriffe).
Um zwei Modelle auszuführen, schalte **Mehrere Modelle ausführen** ein und gib jedem Modell eine eigene Karte desselben
Herstellers.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Ein Chat in der Strata-App: Der Agent hat einen kleinen Voxel-Pagodengarten als eine HTML-Datei geschrieben, und die Seite öffnet sich in der Vorschau neben dem Gespräch</sub></p>
### Im Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code>, während das Modell antwortet: Geschwindigkeit, GPU, Speicher und die letzten Anfragen</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` listet alles auf. Starten und Bereitstellen laden nichts herunter. Nur die Installation, `strata launch`
(wenn du ein Modell wählst) und `strata update` tun das. Um Strata von deinem Handy oder einem anderen PC aus zu erreichen,
nutze `strata launch --access lan`. Der Befehl erzeugt einen API-Schlüssel und startet in einem Netzwerk nicht ohne einen.
Ein Browser öffnet sich nur, wenn du `--open` angibst. Anleitung: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Der ganze Client: [docs/TERMINAL.md](docs/TERMINAL.md).
### Mit KI-Agents
Ein Agent installiert Strata mit dem Einzeiler aus Option B und führt
`strata launch --model NAME --yes --no-tui` aus. Die Schritte, Flags und Exit-Codes stehen in [docs/AI_SETUP.md](docs/AI_SETUP.md),
die Regeln für das Repository in [AGENTS.md](AGENTS.md).
### Andere Apps verbinden
| App-Typ | Basis-URL |
| --- | --- |
| OpenAI-kompatibel (die meisten Apps und Agents) | `http://127.0.0.1:8080/v1` (beliebiger API-Schlüssel, beliebiger Modellname) |
| Anthropic-API (Claude Code und andere) | `http://127.0.0.1:8080`, ohne `/v1`. Diese Apps hängen `/v1/messages` selbst an. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([Einrichtung](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` gibt die genauen Einstellungen für OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent und OpenClaw aus.
`strata connect opencode --write` schreibt die Datei nach einer Rückfrage und legt eine Sicherung an.
### Bilder
Bilder sind für Modelle, die sie lesen können, standardmäßig eingeschaltet. Hänge ein Bild im Chat an oder füge es ein, oder
sende es aus deiner App. Der Bildleser läuft auf dem Prozessor, wenn deine Grafikkarte weniger als 24 GB hat (einige Sekunden
pro Bild, ohne Speicher der Karte zu belegen), und ab 24 GB auf der Grafikkarte. Die Regel gilt für NVIDIA und AMD gleich.
Du kannst Bilder pro Modell ausschalten oder den Leser mit `--vision off`, `--vision gpu` oder `--vision cpu` wählen.
Die Zahlen hinter der Regel: Auf einer Radeon AI PRO R9700 brauchte der Bildleser der Karte 57 ms für ein Bild mit 260 Tokens,
der des Prozessors 3,1 s. Auf Karten mit 16 GB (Tesla P100 und RTX A4000, Linux) kostete der Bildleser der Karte 3-5 % des
Texttempos, weil er etwa 1,4 GB belegt, die sonst das Modell nutzen würde. Auf dem Mac liest die GPU, egal wie viel Speicher er hat (0,45 s für ein Bild mit 260 Tokens auf einem M4 mit 32 GB, etwa 1,2 GB des gemeinsamen
Speichers); der Prozessor braucht dort etwa 50 s pro Bild und liest deshalb nur mit `--vision cpu`.
### Gut zu wissen
- Das Denkniveau (**aus, niedrig, mittel oder hoch**) stellst du im Chat-Menü oder unter "reasoning effort" in deiner App ein. Aus ist am schnellsten.
- Strata beantwortet standardmäßig eine Anfrage nach der anderen. Für mehrere gleichzeitig nutze `--parallel N` in `strata launch` oder
`"parallel": 2` in der Konfiguration ([BATCHING.md](docs/BATCHING.md)). Auf einer Karte mit 12 GB wird dadurch jede Antwort langsamer.
- Strata liest die erste Nachricht eines Chats komplett, etwa 1 Minute pro 30.000 Tokens. Folgenachrichten starten in Sekunden.
- Wo deine Chats gespeichert werden und die ganze API: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Aus dem Quellcode bauen
<details>
<summary>Windows, Linux und Mac aus dem Repository</summary>
[Lade Strata herunter](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) und entpacke es, oder nutze `git clone`.
**Windows:** Doppelklick auf `START-HERE.bat`. **Linux:** Führe `./setup.sh` im Strata-Ordner aus.
**Mac (Apple Silicon, macOS 15 oder neuer):** Installiere einmal die Command Line Tools von Apple mit `xcode-select --install`
(Xcode selbst brauchst du nicht). Öffne das Terminal im Strata-Ordner und führe `./setup.sh` aus. Das Skript baut die
Mac-Engine auf deinem Mac (etwa 7 Minuten), stellt dieselben Fragen, lädt das Modell herunter und startet es. Beim nächsten
Mal startet `./setup.sh` sofort. Welche Größe zu deinem Mac passt: [docs/MAC.md](docs/MAC.md).
Der Installer erkennt deine Karte und richtet die passende Engine ein. Er fragt, welches Modell und welche Größe du willst,
wie viel Kontext (wie viel Text sich das Modell merkt) und ob es Bilder lesen soll. Drück Enter für die empfohlene Antwort.
Dann lädt er das Modell herunter (etwa 70 GB) und startet es. Wenn der Download abbricht, führe das Skript erneut aus, dann
geht es weiter. Dein Browser öffnet die Strata-Seite unter `http://127.0.0.1:8080`.
> Während das Modell startet, kann dein PC 1-3 Minuten langsam sein oder nicht reagieren (beim ersten Mal am längsten). Strata
> lädt 35-55 GB in deinen RAM und sperrt einen Teil davon für die Grafikkarte. Warte ab und schließe das Fenster nicht.
Starte beim nächsten Mal wieder `START-HERE.bat` (oder `./setup.sh`). Es startet sofort und lädt nichts doppelt herunter.
Schließ sein Fenster, um das Modell zu beenden. `UPDATE.bat` (`./update.sh`) aktualisiert Strata, ohne es zu starten.
Aktualisieren, Docker, mehrere Karten, wohin die Dateien kommen und alle Optionen: [docs/INSTALL.md](docs/INSTALL.md).
Um deinen PC zu messen, führe `BENCH.bat` (`./bench.sh`) aus. Das dauert 5-10 Minuten und schreibt einen Bericht, den du teilen
kannst ([so geht's](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Optional: mehr Tempo auf einer Karte, wenn das ganze Modell in den RAM passt.** Zwei Einstellungen zum Einschalten von
RubenTorney (#1904, #1905). Sie helfen, wenn du eine Grafikkarte nutzt und dein RAM alle Experten aufnimmt, zum Beispiel bei
Q2_0 oder IQ3_XXS auf einem PC mit 64 GB. Öffne `strata-<model>.json` im Strata-Datenordner ([wo](docs/INSTALL.md#where-things-are-stored)) und ergänze:
- `"--adapt-async", "1"` in der Liste `"args"`. Der Expertencache wird dann in einem Hilfsthread aktualisiert, statt die
Antwort anzuhalten. Gemessen auf einer RTX 5070 (Windows, Q2_0): +4-5 % beim Decodieren, +7-10 % bei Code und langen
Dokumenten. Die Antworten können sich leicht von der Standardeinstellung unterscheiden und bleiben von Lauf zu Lauf gleich.
- `"STRATA_PIN_BY_NEED": "1"` im Block `"env"` (Windows). Windows kann nur etwa die Hälfte des RAM für die Grafikkarte sperren,
und diese Einstellung sperrt zuerst die Experten, die die Karte am dringendsten braucht. Gemessen auf der RTX 5070: +4,4 % beim Lesen des Prompts.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Starte Strata nach der Änderung neu. Zum Rückgängigmachen entfernst du die beiden Einträge.
Folge den Anweisungen, um ein Modell herunterzuladen und zu starten. [Terminal-Anleitung](docs/TERMINAL_LAUNCH.md).
</details>
## Wie funktioniert das?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Strata-Terminal-App mit Modellaktivität, Tempo, Speicher und letzten Anfragen">
<br><sub>Behalte Modell und Hardware im Blick, während es läuft.</sub>
</p>
Modelle wie dieses laufen normalerweise auf Servern mit Hunderten Gigabyte Grafikspeicher. Deine Grafikkarte hat 12-24 GB.
Strata bringt das Modell auf deinen PC, indem es die Arbeit auf den ganzen Rechner verteilt. Stell dir eine Bibliothek vor.
<a id="build-from-source"></a>
- Das Modell ist ein Team aus 24.576 kleinen Spezialisten ("Experten"), wie 24.576 Nachschlagewerke. Für jedes Wort braucht es nur 10 davon.
- **Deine Grafikkarte ist der Schreibtisch.** Er ist klein und schnell. Die etwa 4.000 am häufigsten genutzten Bücher liegen
dort aufgeschlagen.
- **Dein RAM sind die Regale im Lesesaal.** Alle 24.576 Bücher stehen dort. Braucht ein Wort ein Buch, das nicht auf dem
Schreibtisch liegt, holt dein Prozessor es sofort aus dem Regal, während die Karte weiterarbeitet.
- **Deine SSD ist das Archiv.** Sie enthält eine Nachschlagetabelle von 29 GB. Das Modell liest pro Wort nur wenige Zeilen
daraus, deshalb muss sie nicht im RAM liegen.
[Aus dem Quellcode bauen](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Vollständige Installationsanleitung](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
Ein kleiner Helfer rät außerdem die nächsten Wörter. Das große Modell prüft sie alle auf einmal und behält die richtigen. Du
bekommst dieselbe Antwort, 1,6-1,8-mal früher. Lange Texte werden in großen Stücken gelesen, bis zu 8.192 Tokens auf einmal,
mit über 1.000 Tokens pro Sekunde.
## Hardware
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| Unterstützte Hardware | Anleitung |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [NVIDIA-Setup](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [AMD-Setup](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimentell** | [RX-6000-Setup](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel-Setup](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 oder neuer** | [Mac-Setup](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix-Halo-Setup](docs/STRIX_HALO.md) |
Die ausführliche Erklärung: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Jeder Teil mit seinen Zahlen:
[die Details](docs/DETAILS.md#how-it-works) und das [Paper](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Etwas ist schiefgegangen?
## Tools
- **Mein PC ist beim ersten Start von Strata eingefroren.** Strata lädt das Modell in den RAM. Warte ab und schließe das
Fenster nicht. Nach 10 Minuten immer noch eingefroren? Starte den PC neu, schließe andere Programme und versuch es noch
einmal, oder wähle eine kleinere Größe.
- **Es ist beim Herunterladen oder Installieren stehen geblieben.** Starte die App oder `strata launch` noch einmal, oder
führe `START-HERE.bat` (oder `./setup.sh`) erneut aus. Es macht dort weiter, wo es aufgehört hat.
- **Es ist sehr langsam, die Festplattenanzeige blinkt ständig, oder es meldet "the engine stopped unexpectedly".** Dein PC
hat nicht genug freien RAM. Schließe andere Programme (Browser brauchen viel) oder wähle eine kleinere Größe wie Q2_0 oder IQ2_XS.
- **Es meldet, dass es keine Engine für diesen PC gibt.** Öffne in der App **Einstellungen, Engine** und drücke **Engine
aktualisieren**. Führe im Terminal `strata update` aus. `strata doctor` zeigt, welche Engine-Dateien installiert sind.
- **Es meldet, dass Port 8080 schon belegt ist.** Strata läuft bereits. Such das Fenster.
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, und weitere über die OpenAI- und Anthropic-kompatiblen APIs.
Weitere Probleme und Lösungen: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Du kommst nicht weiter? Öffne ein
[Issue](https://github.com/Niko1221/Strata/issues). Hänge in der App die Datei aus **Einstellungen, Engine, Diagnose, Protokolle anzeigen, Protokolle speichern** an. Füge im Terminal die Ausgabe von `strata doctor` ein. Hänge beim Start aus dem Quellcode die
Datei `strata-<model>.log` aus dem Strata-Ordner an. Ein Sicherheitsproblem meldest du bitte privat: [SECURITY.md](SECURITY.md).
Führe **`strata connect`** aus, um die Einrichtung für dein Tool zu sehen. [Verbindungsanleitung](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
## Danksagung und Lizenz
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
Das Modell ist [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) vom Qwen-Team. Komprimiert wurde es von
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
und Unsloth. Strata nutzt Teile von [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Alle Danksagungen:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata ist Open Source unter der [MIT-Lizenz](LICENSE). Einige Teile und
alle Modelle haben eigene Lizenzen ([welche](docs/HOW_IT_WORKS.md#license)).
## Hilfe
## Mitmachen
Du hast einen Fehler gefunden oder willst eine Änderung oder einen Benchmark einsenden? Lies zuerst [CONTRIBUTING.md](CONTRIBUTING.md).
Dort steht, was dazugehört, damit andere deine Arbeit nachprüfen können.
[Alle Anleitungen](docs) · [Fehlerbehebung](docs/TROUBLESHOOTING.md) · [Fehler melden](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [So funktioniert es](docs/HOW_IT_WORKS.md) · [Mitmachen](CONTRIBUTING.md)
## Strata unterstützen
Strata ist kostenlos und steht unter der MIT-Lizenz. Ich baue es und teste es auf jeder Hardware, die ich bekommen kann. Wenn es auf deinem PC gut läuft, hilft mir ein Kaffee,
weiter neue Karten zu testen und das zu beheben, was ihr meldet. Es besteht kein Druck. Strata zu nutzen und zu melden, was dir auffällt, hilft genauso viel.
**Hilf mit, lokale KI für alle offen zu halten.** Ein Kaffee hilft, Entwicklung und Tests auf mehr Hardware zu finanzieren.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: Strata unterstützen" height="48"></a>
Du möchtest Strata sponsern? Schreib mir über [GitHub](https://github.com/Niko1221).
[MIT-Lizenz](LICENSE) · [Danksagung und Modelllizenzen](docs/HOW_IT_WORKS.md#credits) · [Datenschutz](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Für KI-Tools und automatische Leser: <a href="AGENTS.md">technische Referenz</a> · <a href="docs/AI_SETUP.md">Setup-Anleitung</a>.</sub>
**Sponsoring, Hardware-Partnerschaften oder Zusammenarbeit:** [[email protected]](mailto:[email protected]).
+121 -336
View File
@@ -1,403 +1,188 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <b>Español</b> · <a href="README.pt-BR.md">Português</a></p>
<h3 align="center">Ejecuta un modelo de última generación en tu propio PC gaming</h3>
<p align="center">Compatible con NVIDIA, AMD, Intel · Windows, Linux, Mac · libre y de código abierto</p>
<h3 align="center">IA potente en tu propio ordenador.</h3>
<p align="center">Chatea, programa y trabaja con imágenes, en local.</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#modelos">Modelos</a> · <a href="#instalación"><b>Instalación</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#herramientas">Herramientas</a> ·
<a href="#ayuda">Ayuda</a> · <a href="#apoya-strata">Apoyar</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Un jardín con una pagoda de vóxeles, hecho con un solo prompt en una RTX 5070 con Strata (IQ3_S, contexto de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vídeo completo (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Descargar Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licencia MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Apoya Strata en Buy Me a Coffee"></a>
</p>
Strata ejecuta **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, un modelo de 125 mil millones
de parámetros que normalmente necesita un servidor, en un PC normal. Conversa, escribe código, lee imágenes y funciona
con tus aplicaciones y tus agentes de programación. Tus chats y tus archivos se quedan en tu PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <b>Español</b> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
**¿Eres nuevo?** Mira [lo que necesitas](#lo-que-necesitas) y luego [instala la aplicación](#opción-a-la-aplicación-strata).
**Gratis, de código abierto y tuyo para ejecutar.** Usa Strata en tu escritorio o con tus herramientas de programación favoritas. No necesitas cuenta.
- **Funciona en un PC gaming.** Con 32 GB entre RAM y memoria gráfica ya puedes empezar.
- **Funciona con tus herramientas.** Habla las API de OpenAI y de Anthropic, así que Claude Code, Codex CLI, OpenCode y otros agentes se conectan a él.
- **Lee imágenes.** Viene activado por defecto.
- **Gratis.** Licencia MIT, sin cuenta, sin nube.
- **Privado.** Sin telemetría. Lo que Strata descarga, cuándo usa internet y cómo comprueba las descargas está en [docs/SAFETY.md](docs/SAFETY.md).
## Modelos
Versión 0.2.2 · motor 0.1.47
Recomendamos **primero ISTA-DASLab**, después **Unsloth** y otros proveedores, siempre que haya un modelo compatible que quepa en tu memoria.
## Lo que necesitas
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**Necesitas al menos 32 GB entre RAM y memoria gráfica (VRAM).** En un Mac o un Ryzen AI Max, es la memoria unificada.
Con más memoria puedes usar un modelo más grande o más rápido.
- Un PC de 32 GB con una tarjeta de 12 a 16 GB ejecuta el Coder.
- Con 64 GB de RAM funcionan todos los tamaños.
| Tu ordenador | Detalles |
| Proveedor | Opciones |
| --- | --- |
| **NVIDIA** GeForce RTX serie 20, 30, 40, 50 (12 GB o más) | Instala la aplicación o la Terminal. |
| **AMD** Radeon RX 6800 y posteriores, Radeon AI PRO R9700 (12 GB o más) | [Tarjetas AMD](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [motor Intel](docs/INTEL.md) |
| **Mac con Apple Silicon** (M1 o posterior, macOS 15 o posterior) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** y otros sistemas con memoria unificada | [Strix Halo](docs/STRIX_HALO.md) |
| Tarjetas y procesadores antiguos | [GPU antiguas](docs/OLDER_GPUS.md), [CPU antiguas](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · recomendado** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) para programar. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Usa menos memoria, pero es más lento. |
| **Otras opciones** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
También hace falta:
[Compara tamaños, necesidades de memoria y compatibilidad](docs/MODELS.md).
- unos 80 GB de disco libre (con un SSD el primer arranque es mucho más rápido)
- Windows 10 / 11, Linux o macOS
- un controlador gráfico actual
</details>
Dos o tres tarjetas de la misma marca pueden compartir un modelo ([multi-GPU](docs/MULTI_GPU.md)). Strata nunca reparte un
modelo entre tarjetas NVIDIA y AMD. Lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Son los paquetes documentados actualmente para este modelo.
[Guía del modelo y compatibilidad](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
En la **aplicación de escritorio** y en el **Terminal**. Lo que no cabe en la memoria se lee desde tu SSD, así que hace falta mucho espacio libre en disco.
| Conjunto (IST-DAS) | Descarga | Espacio libre necesario |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | unos 245 GB |
| 2.00 bpw | 257 GB | unos 265 GB |
| 2.25 bpw | 274 GB | unos 280 GB |
Funciona en NVIDIA RTX 30, 40 y 50 (Windows y Linux) y en AMD Radeon AI PRO R9700 (Linux). La instalación descarga los archivos y el motor de DeepSeek, y después inicia el modelo.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Medido con el conjunto de 2.00 bpw en Strata 0.2.5. «Salida» es la velocidad de la respuesta; «Prompt» es la rapidez con la que Strata lee tu texto.
| Hardware | Salida | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB de RAM | 4 a 6 tokens/s | unos 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB de RAM | 9 a 11 tokens/s | unos 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB de RAM | 21 a 25 tokens/s | 320 a 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 a 77 tokens/s | 400 a 450 tokens/s (8K) |
[Memoria, disco y detalles](docs/DEEPSEEK41.md) · [Notas de la versión](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
¿No sabes qué cabe? [Elige un modelo para tu ordenador](docs/MODELS.md#pick-by-memory).
## Instalación
Elige una opción. Todas arrancan el mismo motor y usan los mismos modelos.
<a id="option-a-the-strata-app"></a>
### Opción A: la aplicación Strata
### Aplicación de escritorio
La forma más fácil. La aplicación lleva el motor dentro: la instalas, eliges un modelo y chateas. Descarga el archivo para
tu sistema y tu tarjeta gráfica desde la [página de versiones](https://github.com/Niko1221/Strata/releases/latest). ¿No
sabes cuál elegir? Mira el párrafo que hay debajo de la tabla.
**[Descargar Strata para Windows, Linux o Mac →](https://github.com/Niko1221/Strata/releases/latest)**
| Sistema | Tu tarjeta gráfica | Archivo que descargar |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, controlador 580 o posterior | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA, tarjeta más antigua o controlador anterior a 580 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon serie RX 9000 o Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon serie RX 7000 o Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, controlador 580 o posterior | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, tarjeta más antigua o controlador anterior a 580 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon serie RX 9000 o Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon serie RX 7000 | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 o posterior) | integrada | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
Instala la aplicación, elige un modelo y empieza a chatear. Hay un instalador por sistema. En el primer arranque, la aplicación descarga el motor para tu tarjeta gráfica. [Ayuda con el instalador](docs/INSTALL.md#the-app).
**Si no estás seguro:** con NVIDIA, elige `cuda13` si la versión de tu controlador es la 580 o posterior, y `cuda12` en caso
contrario. `cuda12` es para tarjetas y controladores más antiguos. Los archivos `hip` y `amd` son para tarjetas AMD. Las
demás tarjetas AMD (RX 6000) todavía no tienen una aplicación ya preparada: mira
[Tarjetas AMD](docs/AMD_HIP.md).
<sub>Archivos: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
Después:
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Aplicación de escritorio de Strata con un chat y una vista previa del código en vivo, una junto a otra">
<br><sub>Chat, archivos y vista previa en vivo en una sola ventana.</sub>
</p>
- **Windows:** ejecuta el `.exe`. Se instala para tu usuario y no necesita permisos de administrador. Si Windows muestra
"Windows protegió tu PC", elige **Más información** y luego **Ejecutar de todas formas** (el instalador todavía no está
firmado).
- **Linux:** haz el archivo ejecutable (`chmod +x`) y lánzalo. Necesita FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04)
o `libfuse2t64` (Ubuntu 24.04 y posteriores).
- **Mac:** abre el `.dmg` y arrastra Strata a Aplicaciones.
<a id="option-b-the-terminal-one-line"></a>
En la primera pantalla, la aplicación analiza tu PC y recomienda un modelo adecuado. Pulsa Instalar. El modelo es una
descarga grande (unos 40 a 70 GB) y, si se interrumpe, continúa donde se quedó. Al cerrar la ventana, Strata y el motor se
detienen.
### Aplicación de Terminal
### Opción B: la Terminal (una línea)
¿Prefieres la terminal? Instala Strata y ejecuta **`strata launch`**.
La Terminal es el comando `strata`, pensado para scripts, servidores y agentes de IA.
<details open>
<summary><b>Comandos de instalación</b></summary>
Linux y Mac:
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows (PowerShell):
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
El script analiza tu PC, descarga la Terminal, verifica su SHA-256 y la descomprime en tu carpeta personal. No necesita
permisos de administrador. En Windows, abre una ventana de terminal nueva después. Luego arranca Strata:
En Windows, abre una terminal nueva después. Luego:
```sh
strata launch
```
Te hace una pregunta cada vez: modelo, contexto, imágenes, quién puede conectarse. Después descarga el motor para tu tarjeta
gráfica (firmado y arrancado como prueba antes de usarlo) y el modelo, y empieza a servir. Para scripts y agentes de IA, usa
opciones:
```sh
strata launch --model q2_0 --yes --no-tui
```
Si quieres leer el script de instalación antes de ejecutarlo, descárgalo primero con `curl -fsSL <address> -o install.sh`,
léelo y luego ejecuta `sh install.sh`. Todas las pantallas y opciones: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Opción C: que lo instale tu IA
Pega esto en Claude Code, Cursor, Codex, GitHub Copilot u otro asistente de programación:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
El asistente revisa tu tarjeta gráfica, tu RAM y tu disco, elige el modelo adecuado, lo instala y lo arranca, y te explica
cómo conectar tus aplicaciones. Las herramientas de IA también pueden instalar, arrancar y detener Strata mediante su
[servidor MCP](docs/MCP_SERVER.md).
¿Prefieres compilarlo tú? Mira [Compilar desde el código fuente](#compilar-desde-el-código-fuente).
## Modelos
La aplicación y `strata launch` recomiendan un modelo según tu memoria. Es un solo modelo en varios tamaños, más o menos
comprimido. Los tamaños pequeños son más rápidos. Los grandes son algo más inteligentes.
### ¿Poca memoria? Prueba Gyro
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (de agentionai) es la forma más pequeña de ejecutar este modelo.
| Tamaño | Memoria que necesita (RAM + VRAM) | Descarga | Qué es |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 GB | 58,5 GB | Para PC donde no cabe nada más. La calidad es menor que la de IQ2_XS. |
| **Gyro-M** | 35 GB | 92,0 GB | Calidad parecida a IQ3_XXS con un 20 % menos de memoria de GPU. Escribe las respuestas un 16 % más despacio que Q2_0 en una RTX 3060 de 12 GB y un 21 % más despacio en una R9700. |
Gyro funciona en tarjetas NVIDIA y AMD (en AMD es experimental) en tarjetas Intel Arc con Linux (probado en una Arc A750) y, desde 0.2.2, en Mac con Apple Silicon (Gyro-S en un Mac mini de 32 GB: 9 tokens/s, con los expertos leídos del SSD). Detalles y mediciones:
[docs/GYRO.md](docs/GYRO.md). Arráncalo con `strata launch --model gyro-s`.
### Los demás tamaños
| Tu memoria (RAM + VRAM) | Elige | Notas |
| --- | --- | --- |
| **32 GB** | **Gyro-S**, o el **[Coder](docs/MODELS.md#coder)** para código | El Coder necesita 32 GB de RAM y conserva la mitad de los expertos, así que es más flojo fuera del código |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 GB), o **Gyro-M** (35 GB) | Swift piensa menos tiempo antes de responder. Gyro-M se acerca a la calidad de IQ3_XXS, pero escribe las respuestas un 16-21 % más despacio que Q2_0 |
| **48 GB** | **IQ3_XXS** (47,0 GB) | Más lento, mejor |
| **64 GB** | **IQ3_S** (54,8 GB), el original o Swift 1.5 | El más lento, el mejor. Iguala al modelo completo en las pruebas publicadas |
| **96 GB o más** | **UD-IQ4_XS** de Unsloth (unos 72 GB de RAM) hasta **Q8_0** (unos 136 GB), de 4 a 8 bits | Más cerca del modelo completo. Con menos RAM, Strata lee una parte desde el SSD, que es mucho más lento |
Con menos RAM de la que indica una fila también puede funcionar: Strata pasa solo a su
[modo de poca RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Espera que sea más lento, y mucho más lento con una
tarjeta gráfica pequeña.
Más opciones:
- **[Swift 1.5](docs/MODELS.md#swift-15)** es un ajuste fino que piensa menos tiempo antes de responder, con una calidad y un uso de RAM casi iguales.
- **[NVFP4](docs/NVFP4.md)** es la versión de 4 bits de NVIDIA para tarjetas NVIDIA. Necesita unos 80 GB de RAM para tener todos los expertos en RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** es un modelo más pequeño y distinto que lee imágenes y puede atender varias peticiones a la vez.
- **[El Uncensored IQ3_XXS de OrcaRouter](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** se configura a mano.
Todos los tamaños, descargas y lo que cabe en cada equipo: [docs/MODELS.md](docs/MODELS.md).
## ¿Qué velocidad tiene?
Medido con una tarjeta gráfica de 12 GB (RTX 5070, Ryzen 5 7600, 64 GB de RAM). Resultados con otras tarjetas:
[velocidad de cada tamaño](docs/MODELS.md#how-fast-is-each-size), [resultados de la comunidad](docs/COMMUNITY_BENCHMARKS.md).
| Modelo | Las respuestas aparecen a | Lee tu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2.650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2.090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1.750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1.620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2.180 tokens/s |
Un token equivale a unos ¾ de palabra, así que 60 tokens por segundo es más rápido de lo que puedes leer. "Lee tu prompt"
es lo rápido que Strata procesa un documento largo, código o un historial de chat.
Medido con una tarjeta AMD de 16 GB (RX 9070 XT, Ryzen 9 3900X, 47 GB de RAM):
| Modelo | Las respuestas aparecen a | Lee tu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1.160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1.110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1.420 tokens/s |
Las barras usan la misma escala en las dos tablas. Una tarjeta con más VRAM es más rápida. Una RTX 3090 (24 GB) debería
escribir unos 100-140 tokens por segundo. La versión del motor detrás de cada cifra está en [DETAILS.md](docs/DETAILS.md#speed-measured).
## Uso
### En la aplicación
La aplicación tiene un **Chat**, un **Monitor** del modelo y de tu GPU, CPU y RAM, una página de **Modelos** y los
**Ajustes**. El chat puede trabajar en una carpeta de tu PC. En modo **Manual** pregunta antes de cambiar un archivo o
ejecutar un comando. Escribe `/loop 10m <what to do>` para repetir un mensaje cada 10 minutos mientras el chat esté abierto.
El chat puede buscar en la web con DuckDuckGo (o Bing), sin cuenta. Puedes añadir una clave de búsqueda en **Ajustes,
Herramientas**. La búsqueda es la única función que envía algo fuera de tu PC (las palabras que buscas).
Para ejecutar dos modelos, activa **Ejecutar varios modelos** y dale a cada modelo su propia tarjeta de la misma marca.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Un chat en la app de Strata: el agente escribió un pequeño jardín de pagoda de vóxeles en un solo archivo HTML, y la página se abre en la vista previa junto a la conversación</sub></p>
### En la Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> mientras el modelo responde: velocidad, GPU, memoria y las últimas solicitudes</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` lo lista todo. Arrancar y servir no descargan nada. Solo lo hacen la instalación, `strata launch` (cuando
eliges un modelo) y `strata update`. Para llegar a Strata desde tu móvil u otro PC, usa `strata launch --access lan`. Crea
una clave de API y no arranca en una red sin ella. No se abre ningún navegador salvo que añadas `--open`. Guía:
[docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). El cliente completo: [docs/TERMINAL.md](docs/TERMINAL.md).
### Con agentes de IA
Un agente instala Strata con el comando de una línea de la opción B y ejecuta
`strata launch --model NAME --yes --no-tui`. Los pasos, las opciones y los códigos de salida están en [docs/AI_SETUP.md](docs/AI_SETUP.md),
y las reglas del repositorio en [AGENTS.md](AGENTS.md).
### Conecta tus otras aplicaciones
| Tipo de aplicación | URL base |
| --- | --- |
| Compatible con OpenAI (la mayoría de aplicaciones y agentes) | `http://127.0.0.1:8080/v1` (cualquier clave de API, cualquier nombre de modelo) |
| API de Anthropic (Claude Code y otros) | `http://127.0.0.1:8080`, sin `/v1`. Estas aplicaciones añaden `/v1/messages` por su cuenta. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| API OpenAI Responses (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([configuración](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` muestra los ajustes exactos para OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent y OpenClaw.
`strata connect opencode --write` escribe el archivo después de preguntar y guarda una copia de seguridad.
### Imágenes
Las imágenes vienen activadas por defecto en los modelos que pueden leerlas. Adjunta o pega una imagen en el chat, o envíala
desde tu aplicación. El lector de imágenes funciona en el procesador cuando tu tarjeta gráfica tiene menos de 24 GB (unos
segundos por imagen, sin quitar memoria a la tarjeta) y en la tarjeta gráfica a partir de 24 GB. La regla es la misma para
NVIDIA y AMD. Puedes desactivar las imágenes por modelo, o elegir el lector con `--vision off`, `--vision gpu` o
`--vision cpu`.
Las cifras detrás de la regla: en una Radeon AI PRO R9700, el lector de la tarjeta tardó 57 ms en una imagen de 260 tokens y
el del procesador 3,1 s. En tarjetas de 16 GB (Tesla P100 y RTX A4000, Linux), el lector de la tarjeta costó un 3-5 % de
velocidad en texto, porque ocupa unos 1,4 GB que de otro modo usaría el modelo. En un Mac lee la GPU, tenga la memoria que tenga (0,45 s para una imagen de 260 tokens en un M4 de 32 GB, unos 1,2 GB de la memoria compartida);
el procesador tarda unos 50 s por imagen, así que solo lee con `--vision cpu`.
### Conviene saber
- Elige el nivel de razonamiento (**desactivado, bajo, medio o alto**) en el menú del chat o en "reasoning effort" de tu aplicación. Desactivado es lo más rápido.
- Strata responde a una petición a la vez por defecto. Para varias a la vez, usa `--parallel N` en `strata launch` o
`"parallel": 2` en la configuración ([BATCHING.md](docs/BATCHING.md)). En una tarjeta de 12 GB, esto hace más lenta cada respuesta.
- Strata lee entero el primer mensaje de un chat, unos 1 minuto por cada 30.000 tokens. Los mensajes siguientes empiezan en segundos.
- Dónde se guardan tus chats y la API completa: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Compilar desde el código fuente
<details>
<summary>Windows, Linux y Mac desde el repositorio</summary>
[Descarga Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) y descomprímelo, o haz `git clone`.
**Windows:** haz doble clic en `START-HERE.bat`. **Linux:** ejecuta `./setup.sh` en la carpeta de Strata.
**Mac (Apple Silicon, macOS 15 o posterior):** instala una vez las Command Line Tools de Apple con `xcode-select --install`
(no hace falta Xcode). Abre la Terminal en la carpeta de Strata y ejecuta `./setup.sh`. El script compila el motor para Mac
en tu Mac (unos 7 minutos), hace las mismas preguntas, descarga el modelo y lo arranca. La próxima vez, `./setup.sh` lo
arranca enseguida. Qué tamaño cabe en tu Mac: [docs/MAC.md](docs/MAC.md).
El instalador detecta tu tarjeta y prepara el motor adecuado. Pregunta qué modelo y qué tamaño quieres, cuánto contexto
(cuánto texto mantiene el modelo en mente) y si debe leer imágenes. Pulsa Enter para aceptar la respuesta recomendada.
Luego descarga el modelo (unos 70 GB) y lo arranca. Si la descarga se detiene, vuelve a ejecutar el script y continúa. Tu
navegador abre la página de Strata en `http://127.0.0.1:8080`.
> Mientras el modelo arranca, tu PC puede ir lento o dejar de responder durante 1-3 minutos (más tiempo la primera vez).
> Strata carga 35-55 GB en tu RAM y bloquea una parte para la tarjeta gráfica. Espera y no cierres la ventana.
La próxima vez, ejecuta de nuevo `START-HERE.bat` (o `./setup.sh`). Arranca enseguida y no descarga nada dos veces. Cierra
su ventana para detener el modelo. `UPDATE.bat` (`./update.sh`) actualiza Strata sin arrancarlo. Actualización, Docker,
varias tarjetas, dónde van los archivos y todas las opciones: [docs/INSTALL.md](docs/INSTALL.md).
Para medir tu PC, ejecuta `BENCH.bat` (`./bench.sh`). Tarda 5-10 minutos y escribe un informe que puedes compartir
([cómo](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Opcional: más velocidad en una tarjeta cuando todo el modelo cabe en la RAM.** Dos ajustes que se activan a mano, de
RubenTorney (#1904, #1905). Ayudan cuando usas una sola tarjeta gráfica y tu RAM guarda todos los expertos, por ejemplo con
Q2_0 o IQ3_XXS en un PC de 64 GB. Abre `strata-<model>.json` en la carpeta de datos de Strata ([dónde](docs/INSTALL.md#where-things-are-stored)) y añade:
- `"--adapt-async", "1"` a la lista `"args"`. Así la caché de expertos se actualiza en un hilo auxiliar en vez de detener la
respuesta. Medido en una RTX 5070 (Windows, Q2_0): +4-5 % en decodificación, +7-10 % en código y documentos largos. Las
respuestas pueden diferir un poco de las del ajuste por defecto y son iguales de una ejecución a otra.
- `"STRATA_PIN_BY_NEED": "1"` en el bloque `"env"` (Windows). Windows solo puede bloquear aproximadamente la mitad de la RAM
para la tarjeta gráfica, y este ajuste bloquea primero los expertos que la tarjeta necesita antes. Medido en la RTX 5070: +4,4 % en lectura del prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Reinicia Strata después del cambio. Para deshacerlo, quita las dos entradas.
Sigue las indicaciones para descargar e iniciar un modelo. [Guía del Terminal](docs/TERMINAL_LAUNCH.md).
</details>
## ¿Cómo funciona?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplicación de Terminal de Strata con la actividad del modelo, la velocidad, la memoria y las peticiones recientes">
<br><sub>Observa tu modelo y tu hardware mientras se ejecuta.</sub>
</p>
Los modelos como este suelen ejecutarse en servidores con cientos de gigabytes de memoria gráfica. Tu tarjeta gráfica tiene
12-24 GB. Strata hace que el modelo quepa repartiendo el trabajo por todo tu PC. Imagina una biblioteca.
<a id="build-from-source"></a>
- El modelo es un equipo de 24.576 pequeños especialistas ("expertos"), como 24.576 libros de consulta. Cada palabra necesita solo 10 de ellos.
- **Tu tarjeta gráfica es el escritorio.** Es pequeño y rápido. Los unos 4.000 libros que más se usan están abiertos encima.
- **Tu RAM son las estanterías de la sala de lectura.** Los 24.576 libros están ahí. Cuando una palabra necesita un libro que
no está en el escritorio, tu procesador lo coge enseguida de la estantería mientras la tarjeta sigue trabajando.
- **Tu SSD es el archivo.** Guarda una tabla de consulta de 29 GB. El modelo lee unas pocas filas por palabra, así que no
hace falta que esté en la RAM.
[Compilar desde el código fuente](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Guía de instalación completa](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
Un pequeño ayudante también adivina las siguientes palabras. El modelo grande las comprueba todas a la vez y se queda con
las correctas. Obtienes la misma respuesta, entre 1,6 y 1,8 veces antes. Los textos largos se leen en trozos grandes, de
hasta 8.192 tokens a la vez, a más de 1.000 tokens por segundo.
## Hardware
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| Hardware compatible | Guía |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [Instalación en NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Instalación en AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimental** | [Instalación en RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Instalación en Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 o posterior** | [Instalación en Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Instalación en Strix Halo](docs/STRIX_HALO.md) |
La explicación más larga: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte con sus cifras:
[los detalles](docs/DETAILS.md#how-it-works) y el [artículo](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## ¿Algo ha ido mal?
## Herramientas
- **Mi PC se congeló la primera vez que arrancó Strata.** Carga el modelo en la RAM. Espera y no cierres la ventana. ¿Sigue
congelado después de 10 minutos? Reinicia el PC, cierra otros programas e inténtalo de nuevo, o elige un tamaño más pequeño.
- **Se detuvo durante la descarga o la instalación.** Abre de nuevo la aplicación o `strata launch`, o vuelve a ejecutar
`START-HERE.bat` (o `./setup.sh`). Continúa donde se quedó.
- **Va muy lento, la luz del disco parpadea sin parar, o dice "the engine stopped unexpectedly".** Tu PC no tiene bastante
RAM libre. Cierra otros programas (los navegadores gastan mucha) o elige un tamaño más pequeño, como Q2_0 o IQ2_XS.
- **Dice que no hay motor para este PC.** En la aplicación, abre **Ajustes, Motor** y pulsa **Actualizar motor**. En la
Terminal, ejecuta `strata update`. `strata doctor` muestra qué archivos del motor están instalados.
- **Dice que el puerto 8080 ya está en uso.** Strata ya se está ejecutando. Busca su ventana.
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, y más, mediante las API compatibles con OpenAI y Anthropic.
Más problemas y soluciones: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). ¿Sigues atascado? Abre una
[incidencia](https://github.com/Niko1221/Strata/issues). En la aplicación, adjunta el archivo de **Ajustes, Motor,
Diagnóstico, Ver registros, Guardar registros**. En la Terminal, pega la salida de `strata doctor`. Si lo ejecutas desde el
código fuente, adjunta `strata-<model>.log` de la carpeta de Strata. Para un problema de seguridad, avísanos en privado:
[SECURITY.md](SECURITY.md).
Ejecuta **`strata connect`** para ver las instrucciones de tu herramienta. [Guía de conexión](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
## Créditos y licencia
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
El modelo es [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), del equipo de Qwen. Lo comprimieron
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
y Unsloth. Strata usa partes de [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Todos los créditos:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata es de código abierto con la [licencia MIT](LICENSE). Algunas
partes y todos los modelos tienen sus propias licencias ([cuáles](docs/HOW_IT_WORKS.md#license)).
## Ayuda
## Contribuir
¿Has encontrado un error, o quieres enviar un cambio o un benchmark? Lee primero [CONTRIBUTING.md](CONTRIBUTING.md). Explica
qué incluir para que otros puedan comprobar tu trabajo.
[Todas las guías](docs) · [Solución de problemas](docs/TROUBLESHOOTING.md) · [Informar de un error](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [Cómo funciona](docs/HOW_IT_WORKS.md) · [Contribuir](CONTRIBUTING.md)
## Apoya Strata
Strata es gratis y tiene licencia MIT. Lo construyo y lo pruebo en el hardware que consigo. Si te funciona bien en tu PC, un café me ayuda
a seguir probando tarjetas nuevas y a arreglar lo que nos cuentas. Sin presión. Usarlo y contar lo que encuentras ayuda igual de mucho.
**Ayuda a mantener la IA local abierta a todos.** Un café ayuda a financiar el desarrollo y las pruebas en más hardware.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: apoya Strata" height="48"></a>
¿Te interesa patrocinar Strata? Escríbeme a través de [GitHub](https://github.com/Niko1221).
[Licencia MIT](LICENSE) · [Créditos y licencias de los modelos](docs/HOW_IT_WORKS.md#credits) · [Privacidad](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Para herramientas de IA y lectores automáticos: <a href="AGENTS.md">referencia técnica</a> · <a href="docs/AI_SETUP.md">instrucciones de instalación</a>.</sub>
**Patrocinios, alianzas de hardware o colaboraciones:** [[email protected]](mailto:[email protected]).
+121 -338
View File
@@ -1,405 +1,188 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <b>Français</b> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<h3 align="center">Faites tourner un modèle de pointe sur votre propre PC de jeu</h3>
<p align="center">Compatible avec NVIDIA, AMD, Intel · Windows, Linux, Mac · gratuit et open source</p>
<h3 align="center">Une IA puissante sur votre propre ordinateur.</h3>
<p align="center">Discutez, programmez et travaillez avec des images, en local.</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#modèles">Modèles</a> · <a href="#installation"><b>Installation</b></a> ·
<a href="#matériel">Matériel</a> · <a href="#outils">Outils</a> ·
<a href="#aide">Aide</a> · <a href="#soutenir-strata">Soutenir</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Un jardin de pagode en voxels, généré en une seule requête sur une RTX 5070 avec Strata (IQ3_S, contexte de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vidéo complète (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Télécharger Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licence MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Soutenir Strata sur Buy Me a Coffee"></a>
</p>
Strata fait tourner **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, un modèle de 125 milliards
de paramètres qui demande normalement un serveur, sur un PC ordinaire. Il discute, écrit du code, lit des images et
fonctionne avec vos applications et vos agents de code. Vos conversations et vos fichiers restent sur votre PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <b>Français</b> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
**Nouveau ici ?** Regardez [ce qu'il vous faut](#ce-quil-vous-faut), puis [installez l'application](#option-a--lapplication-strata).
**Gratuit, open source, et à vous de l'exécuter.** Utilisez Strata sur votre bureau ou avec vos outils de programmation préférés. Aucun compte nécessaire.
- **Tourne sur un PC de jeu.** 32 Go de RAM et de mémoire graphique réunis suffisent pour commencer.
- **Fonctionne avec vos outils.** Il parle les API d'OpenAI et d'Anthropic, donc Claude Code, Codex CLI, OpenCode et d'autres agents peuvent s'y connecter.
- **Lit les images.** C'est activé par défaut.
- **Gratuit.** Licence MIT, pas de compte, pas de cloud.
- **Privé.** Aucune télémétrie. Ce que Strata télécharge, quand il utilise internet et comment il vérifie les téléchargements est expliqué dans [docs/SAFETY.md](docs/SAFETY.md).
## Modèles
Version 0.2.2 · moteur 0.1.47
Nous recommandons **d'abord ISTA-DASLab**, puis **Unsloth** et d'autres fournisseurs, lorsqu'un modèle compatible tient en mémoire.
## Ce qu'il vous faut
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**Il vous faut au moins 32 Go de RAM et de mémoire graphique (VRAM) réunis.** Sur un Mac ou un Ryzen AI Max, c'est la
mémoire unifiée. Plus de mémoire permet un modèle plus gros ou plus rapide.
- Un PC de 32 Go avec une carte de 12 à 16 Go fait tourner le Coder.
- Avec 64 Go de RAM, toutes les tailles fonctionnent.
| Votre ordinateur | Détails |
| Fournisseur | Choix |
| --- | --- |
| **NVIDIA** GeForce RTX série 20, 30, 40, 50 (12 Go ou plus) | Installez l'application ou le Terminal. |
| **AMD** Radeon RX 6800 et plus récent, Radeon AI PRO R9700 (12 Go ou plus) | [Cartes AMD](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [moteur Intel](docs/INTEL.md) |
| **Mac Apple Silicon** (M1 ou plus récent, macOS 15 ou plus récent) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** et autres systèmes à mémoire unifiée | [Strix Halo](docs/STRIX_HALO.md) |
| Cartes et processeurs plus anciens | [GPU anciens](docs/OLDER_GPUS.md), [CPU anciens](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · recommandé** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder : IQ1\_M](docs/MODELS.md#coder) pour programmer. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Utilise moins de mémoire, mais plus lent. |
| **Autres options** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
Il faut aussi :
[Comparer les tailles, la mémoire nécessaire et la compatibilité](docs/MODELS.md).
- environ 80 Go d'espace disque libre (un SSD accélère beaucoup le premier démarrage)
- Windows 10 / 11, Linux ou macOS
- un pilote graphique récent
</details>
Deux ou trois cartes de la même marque peuvent se partager un modèle ([multi-GPU](docs/MULTI_GPU.md)). Strata ne répartit
jamais un modèle entre des cartes NVIDIA et AMD. Liste complète : [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Ce sont les paquets actuellement documentés pour ce modèle.
[Guide du modèle et compatibilité](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
Dans l'**application de bureau** et dans le **Terminal**. Ce qui ne tient pas en mémoire est lu depuis votre SSD : il faut donc beaucoup d'espace disque libre.
| Jeu (IST-DAS) | Téléchargement | Espace disque libre nécessaire |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | environ 245 GB |
| 2.00 bpw | 257 GB | environ 265 GB |
| 2.25 bpw | 274 GB | environ 280 GB |
Fonctionne sur les NVIDIA RTX 30, 40 et 50 (Windows et Linux) et sur l'AMD Radeon AI PRO R9700 (Linux). L'installation télécharge les fichiers et le moteur DeepSeek, puis démarre le modèle.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Mesuré avec le jeu 2.00 bpw sur Strata 0.2.5. « Sortie » est la vitesse de la réponse, « Prompt » la vitesse à laquelle Strata lit votre texte.
| Matériel | Sortie | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB de RAM | 4 à 6 tokens/s | environ 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB de RAM | 9 à 11 tokens/s | environ 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB de RAM | 21 à 25 tokens/s | 320 à 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 à 77 tokens/s | 400 à 450 tokens/s (8K) |
[Mémoire, disque et détails](docs/DEEPSEEK41.md) · [Notes de version](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Vous ne savez pas ce qui tient ? [Choisir un modèle pour votre ordinateur](docs/MODELS.md#pick-by-memory).
## Installation
Choisissez une option. Elles lancent toutes le même moteur et utilisent les mêmes modèles.
<a id="option-a-the-strata-app"></a>
### Option A : l'application Strata
### Application de bureau
Le plus simple. L'application contient le moteur : installez-la, choisissez un modèle, discutez. Téléchargez le fichier
correspondant à votre système et à votre carte graphique depuis la [page des versions](https://github.com/Niko1221/Strata/releases/latest).
Vous ne savez pas lequel prendre ? Lisez le paragraphe sous le tableau.
**[Télécharger Strata pour Windows, Linux ou Mac →](https://github.com/Niko1221/Strata/releases/latest)**
| Système | Votre carte graphique | Fichier à télécharger |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, pilote 580 ou plus récent | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA, carte plus ancienne ou pilote antérieur à 580 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon série RX 7000 ou Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, pilote 580 ou plus récent | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, carte plus ancienne ou pilote antérieur à 580 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon série RX 7000 | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 ou plus récent) | intégrée | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
Installez l'application, choisissez un modèle et commencez à discuter. Il y a un seul installateur par système. Au premier lancement, l'application télécharge le moteur de votre carte graphique. [Aide pour l'installateur](docs/INSTALL.md#the-app).
**Si vous hésitez :** pour NVIDIA, prenez `cuda13` si la version de votre pilote est 580 ou plus récente, sinon `cuda12`.
`cuda12` convient aux cartes plus anciennes et aux pilotes plus anciens. Les fichiers `hip` et `amd` sont pour les cartes
AMD. Les autres cartes AMD (RX 6000) n'ont pas encore d'application prête à l'emploi : voir
[Cartes AMD](docs/AMD_HIP.md).
<sub>Fichiers: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
Ensuite :
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Application de bureau Strata avec une discussion et un aperçu du code en direct côte à côte">
<br><sub>Discussion, fichiers et aperçu en direct dans une seule fenêtre.</sub>
</p>
- **Windows :** lancez le `.exe`. Il s'installe pour votre utilisateur et ne demande pas de droits d'administrateur. Si
Windows affiche « Windows a protégé votre ordinateur », choisissez **Informations complémentaires**, puis **Exécuter
quand même** (l'installateur n'est pas encore signé).
- **Linux :** rendez le fichier exécutable (`chmod +x`) et lancez-le. Il a besoin de FUSE 2 : `sudo apt install libfuse2`
(Ubuntu 22.04) ou `libfuse2t64` (Ubuntu 24.04 et plus récent).
- **Mac :** ouvrez le `.dmg` et faites glisser Strata dans Applications.
<a id="option-b-the-terminal-one-line"></a>
Sur le premier écran, l'application analyse votre PC et recommande un modèle adapté. Appuyez sur Installer. Le modèle est un
gros téléchargement (environ 40 à 70 Go) qui reprend là où il s'est arrêté en cas d'interruption. Quand vous fermez la
fenêtre, Strata et le moteur s'arrêtent.
### Application Terminal
### Option B : le Terminal (une ligne)
Vous préférez le terminal ? Installez Strata, puis lancez **`strata launch`**.
Le Terminal est la commande `strata`, pensée pour les scripts, les serveurs et les agents d'IA.
<details open>
<summary><b>Commandes d'installation</b></summary>
Linux et Mac :
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows (PowerShell) :
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Le script analyse votre PC, télécharge le Terminal, vérifie son empreinte SHA-256 et le décompresse dans votre dossier
personnel. Il ne demande pas de droits d'administrateur. Sous Windows, ouvrez ensuite une nouvelle fenêtre de terminal.
Puis lancez Strata :
Sous Windows, ouvrez ensuite un nouveau terminal. Puis :
```sh
strata launch
```
Il pose une question à la fois : modèle, contexte, images, qui peut se connecter. Ensuite il télécharge le moteur pour votre
carte graphique (signé, et lancé à titre d'essai avant usage) ainsi que le modèle, puis commence à servir. Pour les scripts
et les agents d'IA, utilisez des options :
```sh
strata launch --model q2_0 --yes --no-tui
```
Pour lire le script d'installation avant de l'exécuter, téléchargez-le d'abord avec `curl -fsSL <address> -o install.sh`,
lisez-le, puis lancez `sh install.sh`. Tous les écrans et toutes les options : [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Option C : laissez votre IA l'installer
Collez ceci dans Claude Code, Cursor, Codex, GitHub Copilot ou un autre assistant de programmation :
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
L'assistant vérifie votre carte graphique, votre RAM et votre disque, choisit le modèle adapté, l'installe, le démarre et
vous explique comment connecter vos applications. Les outils d'IA peuvent aussi installer, démarrer et arrêter Strata via son
[serveur MCP](docs/MCP_SERVER.md).
Vous préférez le compiler vous-même ? Voir [Compiler depuis les sources](#compiler-depuis-les-sources).
## Modèles
L'application et `strata launch` recommandent un modèle selon votre mémoire. C'est un seul modèle en plusieurs tailles,
plus ou moins compressé. Les petites tailles sont plus rapides. Les grandes sont un peu plus intelligentes.
### Peu de mémoire ? Essayez Gyro
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (par agentionai) est la façon la plus légère de faire tourner ce modèle.
| Taille | Mémoire nécessaire (RAM + VRAM) | Téléchargement | Ce que c'est |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 Go | 58,5 Go | Pour les PC où rien d'autre ne tient. La qualité est inférieure à celle d'IQ2_XS. |
| **Gyro-M** | 35 Go | 92,0 Go | À peu près la qualité d'IQ3_XXS avec environ 20 % de mémoire GPU en moins. Il écrit les réponses 16 % plus lentement que Q2_0 sur une RTX 3060 de 12 Go et 21 % plus lentement sur une R9700. |
Gyro fonctionne sur les cartes NVIDIA et AMD (expérimental sur AMD) sur les cartes Intel Arc sous Linux (testé sur une Arc A750) et, depuis la 0.2.2, sur les Mac Apple Silicon (Gyro-S sur un Mac mini de 32 Go : 9 tokens/s, experts lus sur le SSD). Détails et mesures :
[docs/GYRO.md](docs/GYRO.md). Lancez-le avec `strata launch --model gyro-s`.
### Les autres tailles
| Votre mémoire (RAM + VRAM) | Prenez | Remarques |
| --- | --- | --- |
| **32 Go** | **Gyro-S**, ou le **[Coder](docs/MODELS.md#coder)** pour le code | Le Coder demande 32 Go de RAM et garde la moitié des experts, il est donc moins bon hors du code |
| **40 Go** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 Go), ou **Gyro-M** (35 Go) | Swift réfléchit moins longtemps avant de répondre. Gyro-M approche la qualité d'IQ3_XXS, mais il écrit les réponses 16 à 21 % plus lentement que Q2_0 |
| **48 Go** | **IQ3_XXS** (47,0 Go) | Plus lent, meilleur |
| **64 Go** | **IQ3_S** (54,8 Go), l'original ou Swift 1.5 | Le plus lent, le meilleur. Il égale le modèle complet dans les tests publiés |
| **96 Go ou plus** | **UD-IQ4_XS** d'Unsloth (environ 72 Go de RAM) jusqu'à **Q8_0** (environ 136 Go), 4 à 8 bits | Plus proche du modèle complet. Avec moins de RAM, Strata lit une partie depuis le SSD, ce qui est beaucoup plus lent |
Avoir moins de RAM que ce qu'indique une ligne peut encore fonctionner : Strata passe tout seul en
[mode peu de RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Attendez-vous à plus de lenteur, et à bien plus avec
une petite carte graphique.
Autres choix :
- **[Swift 1.5](docs/MODELS.md#swift-15)** est une version affinée qui réfléchit moins longtemps avant de répondre, pour une qualité et un besoin en RAM à peu près identiques.
- **[NVFP4](docs/NVFP4.md)** est la version 4 bits de NVIDIA pour les cartes NVIDIA. Elle demande environ 80 Go de RAM pour garder tous les experts en RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** est un modèle plus petit et différent, qui lit les images et peut répondre à plusieurs requêtes à la fois.
- **[L'Uncensored IQ3_XXS d'OrcaRouter](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** se configure à la main.
Toutes les tailles, les téléchargements et ce qui tient où : [docs/MODELS.md](docs/MODELS.md).
## Quelle est sa vitesse ?
Mesuré avec une carte graphique de 12 Go (RTX 5070, Ryzen 5 7600, 64 Go de RAM). Voir les résultats d'autres cartes :
[vitesse de chaque taille](docs/MODELS.md#how-fast-is-each-size), [résultats de la communauté](docs/COMMUNITY_BENCHMARKS.md).
| Modèle | Les réponses s'affichent à | Lit votre prompt à |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2 650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2 090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1 750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1 620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2 180 tokens/s |
Un token vaut environ ¾ de mot, donc 60 tokens par seconde, c'est plus rapide que votre lecture. « Lit votre prompt »
indique la vitesse à laquelle Strata absorbe un long document, du code ou un historique de discussion.
Mesuré avec une carte AMD de 16 Go (RX 9070 XT, Ryzen 9 3900X, 47 Go de RAM) :
| Modèle | Les réponses s'affichent à | Lit votre prompt à |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1 160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1 110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1 420 tokens/s |
Les barres utilisent la même échelle dans les deux tableaux. Une carte avec plus de VRAM est plus rapide. Une RTX 3090
(24 Go) devrait écrire environ 100 à 140 tokens par seconde. La version du moteur derrière chaque chiffre est indiquée dans
[DETAILS.md](docs/DETAILS.md#speed-measured).
## Utilisation
### Dans l'application
L'application comprend un **Chat**, un **Monitor** du modèle, de votre GPU, de votre CPU et de votre RAM, une page **Models** et les **Settings**. Le chat peut travailler dans un dossier de votre PC. En mode **Manual**, il demande
avant de modifier un fichier ou d'exécuter une commande. Tapez `/loop 10m <what to do>` pour répéter un message toutes les
10 minutes tant que le chat est ouvert.
Le chat peut chercher sur le web avec DuckDuckGo (ou Bing), sans compte. Vous pouvez ajouter une clé de recherche sous
**Settings, Tools**. La recherche est la seule fonction qui envoie quelque chose hors de votre PC (les mots recherchés).
Pour faire tourner deux modèles, activez **Run multiple models** (exécuter plusieurs modèles) et donnez à chacun sa propre carte de la même marque.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Une discussion dans l'app Strata : l'agent a écrit un petit jardin de pagode en voxels dans un seul fichier HTML, et la page s'ouvre dans l'aperçu à côté de la conversation</sub></p>
### Dans le Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> pendant que le modèle répond : vitesse, GPU, mémoire et les dernières requêtes</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` liste tout. Démarrer et servir ne téléchargent rien. Seuls l'installation, `strata launch` (quand vous
choisissez un modèle) et `strata update` le font. Pour joindre Strata depuis votre téléphone ou un autre PC, utilisez
`strata launch --access lan`. La commande crée une clé d'API et refuse de démarrer sur un réseau sans elle. Aucun
navigateur ne s'ouvre, sauf si vous ajoutez `--open`. Guide : [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Le client complet : [docs/TERMINAL.md](docs/TERMINAL.md).
### Avec des agents d'IA
Un agent installe Strata avec la commande en une ligne de l'option B et lance
`strata launch --model NAME --yes --no-tui`. Les étapes, les options et les codes de sortie sont dans [docs/AI_SETUP.md](docs/AI_SETUP.md),
et les règles du dépôt dans [AGENTS.md](AGENTS.md).
### Connecter vos autres applications
| Type d'application | URL de base |
| --- | --- |
| Compatible OpenAI (la plupart des applications et des agents) | `http://127.0.0.1:8080/v1` (n'importe quelle clé d'API, n'importe quel nom de modèle) |
| API Anthropic (Claude Code et d'autres) | `http://127.0.0.1:8080`, sans `/v1`. Ces applications ajoutent `/v1/messages` elles-mêmes. Claude Code : `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| API OpenAI Responses (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([configuration](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` affiche les réglages exacts pour OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent et OpenClaw.
`strata connect opencode --write` écrit le fichier après confirmation et garde une sauvegarde.
### Images
Les images sont activées par défaut pour les modèles qui savent les lire. Joignez ou collez une image dans le chat, ou
envoyez-la depuis votre application. Le lecteur d'images tourne sur le processeur quand votre carte graphique a moins de
24 Go (quelques secondes par image, sans prendre de mémoire à la carte) et sur la carte graphique à partir de 24 Go. La
règle est la même pour NVIDIA et AMD. Vous pouvez désactiver les images par modèle, ou choisir le lecteur avec
`--vision off`, `--vision gpu` ou `--vision cpu`.
Les chiffres derrière la règle : sur une Radeon AI PRO R9700, le lecteur de la carte a pris 57 ms pour une image de
260 tokens et celui du processeur 3,1 s. Sur des cartes de 16 Go (Tesla P100 et RTX A4000, Linux), le lecteur de la carte
a coûté 3 à 5 % de vitesse sur le texte, parce qu'il occupe environ 1,4 Go que le modèle utiliserait sinon. Sur un Mac, le lecteur tourne sur le GPU quelle que soit la mémoire (0,45 s pour une image de 260 tokens sur un M4 de 32 Go, environ 1,2 Go de la
mémoire partagée) ; le processeur met environ 50 s par image, il ne lit donc qu'avec `--vision cpu`.
### Bon à savoir
- Choisissez le niveau de réflexion (**désactivé, faible, moyen ou élevé**) dans le menu du chat ou dans le réglage « reasoning effort » de votre application. Désactivé est le plus rapide.
- Strata répond à une requête à la fois par défaut. Pour en traiter plusieurs en même temps, utilisez `--parallel N` dans `strata launch` ou
`"parallel": 2` dans la configuration ([BATCHING.md](docs/BATCHING.md)). Sur une carte de 12 Go, chaque réponse devient alors plus lente.
- Strata lit en entier le premier message d'une discussion, soit environ 1 minute pour 30 000 tokens. Les messages suivants démarrent en quelques secondes.
- Où sont stockées vos discussions et l'API complète : [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Compiler depuis les sources
<details>
<summary>Windows, Linux et Mac depuis le dépôt</summary>
[Téléchargez Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) et décompressez-le, ou faites un `git clone`.
**Windows :** double-cliquez sur `START-HERE.bat`. **Linux :** lancez `./setup.sh` dans le dossier Strata.
**Mac (Apple Silicon, macOS 15 ou plus récent) :** installez une fois les Command Line Tools d'Apple avec `xcode-select --install`
(Xcode lui-même n'est pas nécessaire). Ouvrez le Terminal dans le dossier Strata et lancez `./setup.sh`. Le script compile
le moteur Mac sur votre Mac (environ 7 minutes), pose les mêmes questions, télécharge le modèle et le démarre. La fois
suivante, `./setup.sh` le démarre tout de suite. Quelle taille convient à votre Mac : [docs/MAC.md](docs/MAC.md).
L'installateur détecte votre carte et prépare le bon moteur. Il demande quel modèle et quelle taille, combien de contexte
(la quantité de texte que le modèle garde en tête) et s'il doit lire les images. Appuyez sur Entrée pour la réponse
recommandée. Puis il télécharge le modèle (environ 70 Go) et le démarre. Si le téléchargement s'arrête, relancez le script
et il reprend. Votre navigateur ouvre la page de Strata à l'adresse `http://127.0.0.1:8080`.
> Pendant le démarrage du modèle, votre PC peut être lent ou ne plus répondre pendant 1 à 3 minutes (le plus longtemps la
> première fois). Strata charge 35 à 55 Go dans votre RAM et en verrouille une partie pour la carte graphique. Patientez et
> ne fermez pas la fenêtre.
La fois suivante, relancez `START-HERE.bat` (ou `./setup.sh`). Il démarre tout de suite et ne télécharge rien deux fois.
Fermez sa fenêtre pour arrêter le modèle. `UPDATE.bat` (`./update.sh`) met Strata à jour sans le démarrer. Mise à jour,
Docker, plusieurs cartes, où vont les fichiers et toutes les options : [docs/INSTALL.md](docs/INSTALL.md).
Pour mesurer votre PC, lancez `BENCH.bat` (`./bench.sh`). Cela prend 5 à 10 minutes et produit un rapport que vous pouvez
partager ([comment](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Facultatif : plus de vitesse sur une carte quand tout le modèle tient en RAM.** Deux réglages à activer soi-même, par
RubenTorney (#1904, #1905). Ils aident quand vous utilisez une seule carte graphique et que votre RAM contient tous les
experts, par exemple Q2_0 ou IQ3_XXS sur un PC de 64 Go. Ouvrez `strata-<model>.json` dans le dossier de données de Strata ([où](docs/INSTALL.md#where-things-are-stored)) et ajoutez :
- `"--adapt-async", "1"` à la liste `"args"`. Le cache des experts se met alors à jour sur un thread auxiliaire au lieu de
mettre la réponse en pause. Mesuré sur une RTX 5070 (Windows, Q2_0) : +4 à 5 % en décodage, +7 à 10 % sur le code et les
longs documents. Les réponses peuvent différer légèrement de celles du réglage par défaut, et restent identiques d'une exécution à l'autre.
- `"STRATA_PIN_BY_NEED": "1"` dans le bloc `"env"` (Windows). Windows ne peut verrouiller qu'environ la moitié de la RAM
pour la carte graphique, et ce réglage verrouille d'abord les experts dont la carte a le plus besoin. Mesuré sur la RTX 5070 : +4,4 % en lecture du prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Redémarrez Strata après la modification. Pour annuler, supprimez les deux entrées.
Suivez les invites pour télécharger et démarrer un modèle. [Guide du Terminal](docs/TERMINAL_LAUNCH.md).
</details>
## Comment ça marche ?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Application Terminal de Strata affichant l'activité du modèle, la vitesse, la mémoire et les requêtes récentes">
<br><sub>Suivez votre modèle et votre matériel pendant qu'il tourne.</sub>
</p>
Les modèles de ce genre tournent d'habitude sur des serveurs qui disposent de centaines de gigaoctets de mémoire graphique.
Votre carte graphique en a 12 à 24. Strata fait tenir le modèle en répartissant le travail sur tout votre PC. Imaginez une
bibliothèque.
<a id="build-from-source"></a>
- Le modèle est une équipe de 24 576 petits spécialistes (les « experts »), comme 24 576 ouvrages de référence. Chaque mot n'en demande que 10.
- **Votre carte graphique est le bureau.** Il est petit et rapide. Les quelque 4 000 livres les plus utilisés y sont ouverts.
- **Votre RAM, ce sont les étagères de la salle de lecture.** Les 24 576 livres s'y trouvent. Quand un mot demande un livre
qui n'est pas sur le bureau, votre processeur va le chercher aussitôt sur l'étagère, pendant que la carte continue de travailler.
- **Votre SSD, ce sont les archives.** Il contient une table de consultation de 29 Go. Le modèle n'en lit que quelques lignes par
mot, elle n'a donc pas besoin de tenir en RAM.
[Compiler depuis les sources](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Guide d'installation complet](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
Un petit assistant devine aussi les mots suivants. Le grand modèle les vérifie tous d'un coup et garde les bons. Vous
obtenez la même réponse, 1,6 à 1,8 fois plus tôt. Les textes longs sont lus par gros morceaux, jusqu'à 8 192 tokens à la
fois, à plus de 1 000 tokens par seconde.
## Matériel
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| Matériel pris en charge | Guide |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [Installation NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Installation AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · expérimental** | [Installation RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Installation Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 ou plus récent** | [Installation sur Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Installation Strix Halo](docs/STRIX_HALO.md) |
L'explication plus longue : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Chaque élément avec ses chiffres :
[les détails](docs/DETAILS.md#how-it-works) et l'[article](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Un problème ?
## Outils
- **Mon PC s'est figé au premier démarrage de Strata.** Il charge le modèle en RAM. Patientez et ne fermez pas la fenêtre.
Toujours figé après 10 minutes ? Redémarrez le PC, fermez les autres programmes et réessayez, ou choisissez une taille plus petite.
- **Ça s'est arrêté pendant le téléchargement ou l'installation.** Relancez l'application ou `strata launch`, ou relancez
`START-HERE.bat` (ou `./setup.sh`). Cela reprend là où ça s'était arrêté.
- **C'est très lent, le voyant du disque clignote sans arrêt, ou il affiche « the engine stopped unexpectedly ».** Votre PC
n'a pas assez de RAM libre. Fermez d'autres programmes (les navigateurs en consomment beaucoup) ou choisissez une taille plus petite, comme Q2_0 ou IQ2_XS.
- **Il indique qu'il n'y a pas de moteur pour ce PC.** Dans l'application, ouvrez **Settings, Engine** et appuyez sur **Update engine** (mettre à jour le moteur). Dans le Terminal, lancez `strata update`. `strata doctor` montre quels fichiers du moteur sont installés.
- **Il indique que le port 8080 est déjà utilisé.** Strata tourne déjà. Cherchez sa fenêtre.
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, et d'autres via les API compatibles OpenAI et Anthropic.
Autres problèmes et solutions : [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Toujours bloqué ? Ouvrez une
[issue](https://github.com/Niko1221/Strata/issues). Dans l'application, joignez le fichier obtenu par **Settings, Engine, Diagnostics, View logs, Save logs**. Dans le Terminal, collez la sortie de `strata doctor`. Si
vous avez lancé depuis les sources, joignez `strata-<model>.log` du dossier Strata. Pour un problème de sécurité,
signalez-le en privé : [SECURITY.md](SECURITY.md).
Lancez **`strata connect`** pour voir la configuration de votre outil. [Guide de connexion](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
## Remerciements et licence
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
Le modèle est [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), de l'équipe Qwen. Il a été compressé par
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
et Unsloth. Strata utilise des parties de [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Tous les
remerciements : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata est open source sous [licence MIT](LICENSE).
Quelques parties et tous les modèles ont leurs propres licences ([lesquelles](docs/HOW_IT_WORKS.md#license)).
## Aide
## Contribuer
Vous avez trouvé un bug, ou vous voulez proposer une modification ou un benchmark ? Lisez d'abord [CONTRIBUTING.md](CONTRIBUTING.md).
Il indique ce qu'il faut inclure pour que d'autres puissent vérifier votre travail.
[Tous les guides](docs) · [Dépannage](docs/TROUBLESHOOTING.md) · [Signaler un bogue](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [Comment ça marche](docs/HOW_IT_WORKS.md) · [Contribuer](CONTRIBUTING.md)
## Soutenir Strata
Strata est gratuit et sous licence MIT. Je le construis et je le teste sur le matériel que je peux obtenir. S'il tourne bien sur votre PC, un café m'aide
à continuer de tester de nouvelles cartes et à corriger ce que vous signalez. Aucune obligation. L'utiliser et signaler ce que vous constatez aide tout autant.
**Aidez à garder l'IA locale ouverte à tous.** Un café aide à financer le développement et les tests sur davantage de matériel.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee : soutenir Strata" height="48"></a>
Vous souhaitez sponsoriser Strata ? Contactez-moi via [GitHub](https://github.com/Niko1221).
[Licence MIT](LICENSE) · [Crédits et licences des modèles](docs/HOW_IT_WORKS.md#credits) · [Confidentialité](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Pour les outils d'IA et les lecteurs automatiques : <a href="AGENTS.md">référence technique</a> · <a href="docs/AI_SETUP.md">instructions d'installation</a>.</sub>
**Parrainages, partenariats matériel ou collaborations :** [[email protected]](mailto:[email protected]).
+121 -318
View File
@@ -1,385 +1,188 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <b>日本語</b> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<h3 align="center">最先端のモデルを、手元のゲーミングPCで動かす</h3>
<p align="center">NVIDIA、AMD、Intel に対応 · Windows、Linux、Mac · 無料のオープンソース</p>
<h3 align="center">あなた自身のパソコンで、高性能な AI を。</h3>
<p align="center">チャット、コーディング、画像の処理を、すべてローカルで。</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#モデル">モデル</a> · <a href="#インストール"><b>インストール</b></a> ·
<a href="#ハードウェア">ハードウェア</a> · <a href="#ツール">ツール</a> ·
<a href="#ヘルプ">ヘルプ</a> · <a href="#strata-を支援する">支援</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>ボクセルの塔の庭園。1回のプロンプトで生成し、RTX 5070 上の Strata で動かしています(IQ3_S、128K コンテキスト) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">動画全体(49秒)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Strata をダウンロード"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT ライセンス"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Buy Me a Coffee で Strata を支援する"></a>
</p>
Strata は **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** を動かします。1250億パラメータのモデルで、
ふつうはサーバーが必要ですが、Strata なら一般的なPCで使えます。チャット、コード作成、画像の読み取りができ、
お使いのアプリやコーディングエージェントとも連携します。チャットとファイルはPCの外に出ません。
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <b>日本語</b> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
**はじめての方へ。** まず[必要な環境](#必要な環境)を確認して、[アプリをインストール](#方法a-strata-アプリ)してください。
**無料でオープンソース。自分の環境で動かせます。** デスクトップで、またはお好みのコーディングツールと一緒に Strata を使えます。アカウントは不要です。
- **ゲーミングPCで動きます。** RAMとグラフィックスメモリの合計が32 GBあれば始められます。
- **お使いのツールと連携します。** OpenAI と Anthropic の API に対応しているので、Claude Code、Codex CLI、OpenCode などのエージェントから接続できます。
- **画像を読めます。** 初期設定でオンです。
- **無料です。** MIT ライセンス。アカウントもクラウドも不要です。
- **プライバシーを守ります。** テレメトリはありません。Strata が何をダウンロードするか、いつインターネットを使うか、ダウンロードをどう検証するかは [docs/SAFETY.md](docs/SAFETY.md) に書いてあります。
## モデル
バージョン 0.2.2 · エンジン 0.1.47
まず **ISTA-DASLab** をおすすめします。次に **Unsloth** などの提供元です。メモリに収まる互換モデルがある場合に選べます。
## 必要な環境
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**RAMとグラフィックスメモリ(VRAM)の合計が、少なくとも32 GB必要です。** MacやRyzen AI Maxでは、ユニファイドメモリがその量にあたります。
メモリが多いほど、大きなモデルを使えたり、速く動いたりします。
- RAM 32 GBのPCに12〜16 GBのグラフィックスカードがあれば、Coder が動きます。
- RAM 64 GBなら、どのサイズも動きます。
| お使いのPC | 詳細 |
| 提供元 | 選択肢 |
| --- | --- |
| **NVIDIA** GeForce RTX 20、30、40、50 シリーズ(12 GB以上) | アプリまたはターミナル版をインストールしてください。 |
| **AMD** Radeon RX 6800 以降、Radeon AI PRO R9700(12 GB以上) | [AMD のカード](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md)、[Intel エンジン](docs/INTEL.md) |
| **Apple Silicon の Mac**(M1 以降、macOS 15 以降) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max(Strix Halo)** などのユニファイドメモリ搭載機 | [Strix Halo](docs/STRIX_HALO.md) |
| 古いカードやプロセッサ | [古いGPU](docs/OLDER_GPUS.md)、[古いCPU](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · おすすめ** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S。[Coder: IQ1\_M](docs/MODELS.md#coder)(コーディング向け)。 |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)。メモリ使用量は少ないですが、動作は遅くなります。 |
| **その他の選択肢** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
そのほかに必要なもの:
[サイズ、必要なメモリ、互換性を比べる](docs/MODELS.md)。
- 約80 GBの空きディスク容量(SSDなら初回の起動がずっと速くなります)
- Windows 10 / 11、Linux、または macOS
- 最新のグラフィックスドライバー
</details>
同じメーカーのカードを2〜3枚使って、1つのモデルを動かせます([マルチGPU](docs/MULTI_GPU.md))。NVIDIA と AMD のカードにまたがってモデルを分けることはありません。
一覧は [docs/INSTALL.md](docs/INSTALL.md#what-you-need) にあります。
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0。現在、このモデルで文書化されているパックです。
[モデルガイドと互換性](docs/QWEN36.md)。
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
**デスクトップアプリ**と**ターミナル**で使えます。メモリに収まらない分は SSD から読み込むため、空きディスク容量がたくさん必要です。
| セット(IST-DAS) | ダウンロード | 必要な空き容量 |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | 約245 GB |
| 2.00 bpw | 257 GB | 約265 GB |
| 2.25 bpw | 274 GB | 約280 GB |
NVIDIA RTX 30・40・50 シリーズ(Windows と Linux)と、AMD Radeon AI PRO R9700(Linux)で動きます。セットアップがファイルと DeepSeek エンジンをダウンロードし、モデルを起動します。
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Strata 0.2.5 で 2.00 bpw のセットを使って測定しました。「出力」は回答の速さ、「プロンプト」は Strata が入力を読み込む速さです。
| ハードウェア | 出力 | プロンプト |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4〜6 トークン/秒 | 約 110 トークン/秒 (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9〜11 トークン/秒 | 約 155 トークン/秒 (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21〜25 トークン/秒 | 320〜360 トークン/秒 (8K) |
| 4x Radeon AI PRO R9700 | 74〜77 トークン/秒 | 400〜450 トークン/秒 (8K) |
[メモリ、ディスク、詳細](docs/DEEPSEEK41.md) · [リリースノート](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
どれが動くかわからない場合は、[お使いのパソコンに合うモデルを選ぶ](docs/MODELS.md#pick-by-memory)。
## インストール
3つの方法から1つ選んでください。どれも同じエンジンを起動し、同じモデルを使います。
<a id="option-a-the-strata-app"></a>
### 方法A: Strata アプリ
### デスクトップアプリ
いちばん簡単な方法です。エンジンはアプリに入っているので、インストールしてモデルを選べばチャットを始められます。お使いのOSとグラフィックスカードに合うファイルを
[リリースページ](https://github.com/Niko1221/Strata/releases/latest)からダウンロードしてください。どれか分からない場合は、表の下の説明を見てください。
**[Windows、Linux、Mac 用の Strata をダウンロード →](https://github.com/Niko1221/Strata/releases/latest)**
| OS | グラフィックスカード | ダウンロードするファイル |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX、ドライバー 580 以降 | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA、古いカード、またはドライバーが 580 より古い場合 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon RX 9000 シリーズまたは Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon RX 7000 シリーズまたは Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX、ドライバー 580 以降 | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA、古いカード、またはドライバーが 580 より古い場合 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 シリーズまたは Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 シリーズ | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac**(Apple Silicon、macOS 15 以降) | 内蔵 | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
アプリをインストールし、モデルを選んで、チャットを始めましょう。インストーラーはシステムごとに1つです。初回起動時に、アプリがお使いのグラフィックスカード用のエンジンをダウンロードします。[インストーラーのヘルプ](docs/INSTALL.md#the-app)。
**迷ったときは:** NVIDIA の場合、ドライバーがバージョン 580 以降なら `cuda13`、それより古ければ `cuda12` を選びます。`cuda12` は古いカードと古いドライバー向けです。
`hip` と `amd` のファイルは AMD のカード用です。それ以外の AMD カード(RX 6000)には、まだ用意されたアプリがありません。
[AMD のカード](docs/AMD_HIP.md)を見てください。
<sub>ファイル: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
そのあとの手順:
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="チャットとライブのコードプレビューを並べて表示した Strata デスクトップアプリ">
<br><sub>チャット、ファイル、ライブプレビューを1つのウィンドウで。</sub>
</p>
- **Windows:** `.exe` を実行します。ユーザー単位でインストールされ、管理者権限は不要です。「Windows によってPCが保護されました」と表示されたら、
**詳細情報**を選び、**実行**を選んでください(インストーラーにはまだコード署名がありません)。
- **Linux:** ファイルを実行可能にして(`chmod +x`)起動します。FUSE 2 が必要です。`sudo apt install libfuse2`(Ubuntu 22.04)
または `libfuse2t64`(Ubuntu 24.04 以降)を実行してください。
- **Mac:** `.dmg` を開き、Strata を「アプリケーション」にドラッグします。
<a id="option-b-the-terminal-one-line"></a>
最初の画面で、アプリがPCを調べて合うモデルをすすめます。インストールを押してください。モデルは大きく(約40〜70 GB)、中断しても続きからダウンロードできます。
ウィンドウを閉じると、Strata とエンジンが止まります。
### ターミナルアプリ
### 方法B: ターミナル(1行)
ターミナルのほうが好みですか? Strata をインストールして、**`strata launch`** を実行します。
ターミナル版は `strata` コマンドです。スクリプトやサーバー、AI エージェント向けです。
<details open>
<summary><b>インストールコマンド</b></summary>
Linux と Mac:
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows(PowerShell):
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
スクリプトはPCを調べ、ターミナル版をダウンロードし、SHA-256 を検証して、ホームフォルダに展開します。管理者権限は不要です。Windows では、終わったら
新しいターミナルを開いてください。そのあと Strata を起動します。
Windows では、インストール後に新しいターミナルを開きます。そのあと:
```sh
strata launch
```
質問は1つずつ表示されます。モデル、コンテキスト、画像、接続を許可する相手の順です。そのあと、お使いのグラフィックスカード用のエンジン
(署名付きで、使う前にテスト起動します)とモデルをダウンロードして、サーバーを開始します。スクリプトや AI エージェントからは、フラグを使います。
```sh
strata launch --model q2_0 --yes --no-tui
```
実行前にインストールスクリプトを読みたい場合は、先に `curl -fsSL <address> -o install.sh` でダウンロードし、内容を確認してから
`sh install.sh` を実行してください。すべての画面とフラグは [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md) にあります。
### 方法C: AI にセットアップを任せる
Claude Code、Cursor、Codex、GitHub Copilot などのコーディングアシスタントに、次の文を貼り付けます。
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
AI はグラフィックスカード、RAM、ディスクを調べ、合うモデルを選んで、インストールと起動をし、アプリの接続方法を教えてくれます。
AI ツールは [MCP サーバー](docs/MCP_SERVER.md)を通じて、Strata のインストール、起動、停止もできます。
自分でビルドしたい方は[ソースからビルド](#ソースからビルド)を見てください。
## モデル
アプリと `strata launch` は、メモリ量に合うモデルをすすめます。モデルは1つで、圧縮の度合いが違う複数のサイズがあります。
小さいサイズは速く、大きいサイズは少し賢くなります。
### メモリが少ないなら Gyro
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)(agentionai 製)は、このモデルをいちばん小さく動かせる方法です。
| サイズ | 必要なメモリ(RAM + VRAM) | ダウンロード | 内容 |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27.6 GB | 58.5 GB | ほかのサイズが入らないPC向け。品質は IQ2_XS より下がります。 |
| **Gyro-M** | 35 GB | 92.0 GB | 品質は IQ3_XXS 程度で、GPUメモリの使用量は約20%少なくなります。回答の書き出しは Q2_0 より、12 GBの RTX 3060 で16%、R9700 で21%遅くなります。 |
Gyro は NVIDIA と AMD のカード(AMD は実験的)、および Linux 上の Intel Arc カード(Arc A750 で確認)で動きます。0.2.2 からは Apple Silicon の Mac でも動きます(32 GB の Mac mini で Gyro-S は 9 トークン/秒、エキスパートは SSD から読み込み)。詳細と測定結果は
[docs/GYRO.md](docs/GYRO.md) にあります。`strata launch --model gyro-s` で起動します。
### ほかのサイズ
| メモリ(RAM + VRAM) | 選ぶモデル | 備考 |
| --- | --- | --- |
| **32 GB** | **Gyro-S**、またはコード向けの **[Coder](docs/MODELS.md#coder)** | Coder はRAM 32 GBが必要で、エキスパートを半分だけ保持するため、コード以外は弱くなります |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS(39.2 GB)、または **Gyro-M**(35 GB) | Swift は答える前に考える時間が短くなります。Gyro-M は品質が IQ3_XXS に近いものの、回答の書き出しは Q2_0 より16〜21%遅くなります |
| **48 GB** | **IQ3_XXS**(47.0 GB) | 遅くなりますが、品質は上がります |
| **64 GB** | **IQ3_S**(54.8 GB)、オリジナルまたは Swift 1.5 | いちばん遅く、いちばん高品質です。公開されているテストでは完全版と同等です |
| **96 GB 以上** | Unsloth の **UD-IQ4_XS**(RAM 約72 GB)から **Q8_0**(約136 GB)まで、4〜8ビット | 完全版に近づきます。RAM が足りないと、Strata は一部を SSD から読むため、ずっと遅くなります |
表の行より RAM が少なくても動くことがあります。Strata は自動で[低RAMモード](docs/MODELS.md#a-big-graphics-card-and-little-ram)に切り替わります。
速度は落ち、グラフィックスカードが小さいとさらに大きく落ちます。
そのほかの選択肢:
- **[Swift 1.5](docs/MODELS.md#swift-15)** はファインチューン版で、答える前に考える時間が短くなります。品質と RAM の必要量はほぼ同じです。
- **[NVFP4](docs/NVFP4.md)** は NVIDIA カード向けの NVIDIA 製4ビット版です。すべてのエキスパートを RAM に置くには、約80 GBの RAM が必要です。
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** は、より小さい別のモデルです。画像を読め、複数のリクエストに同時に答えられます。
- **[OrcaRouter の Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** は手動で設定します。
すべてのサイズ、ダウンロード、どの構成で何が動くかは [docs/MODELS.md](docs/MODELS.md) にあります。
## どのくらい速い?
12 GBのグラフィックスカード(RTX 5070、Ryzen 5 7600、RAM 64 GB)での数値です。ほかのカードの結果は
[各モデルの速度](docs/MODELS.md#how-fast-is-each-size)、[コミュニティの結果](docs/COMMUNITY_BENCHMARKS.md)を見てください。
| モデル | 回答が出る速さ | プロンプトを読む速さ |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |
1トークンは英単語の約¾にあたるので、毎秒60トークンは読む速さより速くなります。「プロンプトを読む速さ」は、長い文書やコード、チャット履歴を Strata が取り込む速さです。
16 GBの AMD カード(RX 9070 XT、Ryzen 9 3900X、RAM 47 GB)での測定値:
| モデル | 回答が出る速さ | プロンプトを読む速さ |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |
バーは2つの表で同じ目盛りです。VRAM が多いカードほど速くなり、RTX 3090(24 GB)なら毎秒100〜140トークンほどになるはずです。各数値のエンジンのバージョンは
[DETAILS.md](docs/DETAILS.md#speed-measured) にあります。
## 使い方
### アプリで
アプリには、**チャット**、モデルとGPU・CPU・RAMの状態を見る**モニター**、**モデル**ページ、**設定**があります。チャットはPC上のフォルダで作業できます。
**手動**モードでは、ファイルを変更したりコマンドを実行したりする前に確認します。`/loop 10m <やること>` と入力すると、チャットを開いている間、メッセージを10分ごとに繰り返し送ります。
チャットは DuckDuckGo(または Bing)でウェブ検索ができ、アカウントは不要です。**設定、ツール**で検索キーを追加することもできます。
検索は、PCの外に何かを送る唯一の機能で、送られるのは検索語です。
2つのモデルを動かすには、**複数のモデルを実行**をオンにして、それぞれのモデルに同じメーカーのカードを1枚ずつ割り当てます。
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Strata アプリでのチャット。エージェントが小さなボクセルの塔の庭を 1 つの HTML ファイルとして書き、そのページが会話の横のプレビューで開きます</sub></p>
### ターミナルで
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub>モデルが返信を書いている間の <code>strata monitor</code>。速度、GPU、メモリ、最近のリクエストが表示されます</sub></p>
```sh
strata launch # ガイド付きの画面: モデル、コンテキスト、画像、アクセス、そして開始
strata launch --model q2_0 --yes --no-tui # 質問なし: 開始し、アドレスを表示して戻る
strata launch --plan --model q2_0 # 実行内容を表示するだけで、何も変更しない
strata launch --status # 今、何が動いているか
strata launch --stop # 停止する
strata monitor --url http://127.0.0.1:8080 # 速度、GPU、メモリ、リクエストを見る
strata update --check # 新しいエンジンがあるか確認(strata update でインストール)
strata doctor # インストール済みのエンジンファイルを調べる
```
`strata --help` ですべてのコマンドを確認できます。起動とサーバーの稼働では、何もダウンロードしません。ダウンロードするのは、インストール、
`strata launch`(モデルを選んだとき)、`strata update` だけです。スマートフォンや別のPCから Strata に接続するには、
`strata launch --access lan` を使います。API キーが作られ、キーがない状態ではネットワーク上で起動しません。`--open` を付けない限り、ブラウザは開きません。
ガイド: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md)。クライアント全体: [docs/TERMINAL.md](docs/TERMINAL.md)。
### AI エージェントと使う
エージェントは、方法Bの1行コマンドで Strata をインストールし、
`strata launch --model NAME --yes --no-tui` を実行します。手順、フラグ、終了コードは [docs/AI_SETUP.md](docs/AI_SETUP.md) に、
リポジトリのルールは [AGENTS.md](AGENTS.md) にあります。
### ほかのアプリをつなぐ
| アプリの種類 | ベース URL |
| --- | --- |
| OpenAI 互換(ほとんどのアプリとエージェント) | `http://127.0.0.1:8080/v1`(API キーもモデル名も何でも可) |
| Anthropic API(Claude Code など) | `http://127.0.0.1:8080`、`/v1` は付けません。これらのアプリは `/v1/messages` を自分で付けます。Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API(Codex CLI) | `http://127.0.0.1:8080/v1/responses`([設定方法](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` は、OpenCode、Codex CLI、Claude Code、oh-my-pi、pi、Hermes Agent、OpenClaw の正確な設定を表示します。
`strata connect opencode --write` は、確認したうえでファイルを書き込み、バックアップを残します。
### 画像
画像を読めるモデルでは、画像は初期設定でオンです。チャットに画像を添付または貼り付けるか、アプリから送ってください。
グラフィックスカードが24 GB未満のときは、画像リーダーがプロセッサで動きます(画像1枚につき数秒、カードのメモリは使いません)。24 GB以上のときはグラフィックスカードで動きます。
このルールは NVIDIA と AMD で同じです。画像はモデルごとにオフにでき、`--vision off`、`--vision gpu`、`--vision cpu` でリーダーを選べます。
このルールの根拠となる数値です。Radeon AI PRO R9700 では、カードのリーダーは260トークンの画像に57 ms、プロセッサのリーダーは3.1秒かかりました。
16 GBのカード(Tesla P100 と RTX A4000、Linux)では、カードのリーダーを使うとテキストの速度が3〜5%下がりました。
本来モデルが使う約1.4 GBをリーダーが占めるためです。Mac ではメモリ量にかかわらず GPU が読み取ります(32 GB の M4 で260トークンの画像に0.45秒、共有メモリを約1.2 GB使います)。プロセッサでは1枚あたり約50秒かかるため、`--vision cpu` を指定したときだけ使います。
### 知っておくと便利なこと
- 思考のレベル(**off、low、medium、high**)は、チャットのメニューか、お使いのアプリの「reasoning effort」で選びます。off がいちばん速くなります。
- Strata は初期設定では、一度に1つのリクエストに答えます。同時に複数処理するには、`strata launch` で `--parallel N` を使うか、設定に
`"parallel": 2` を書きます([BATCHING.md](docs/BATCHING.md))。12 GBのカードでは、1つずつの回答が遅くなります。
- Strata はチャットの最初のメッセージを全文読みます。30,000トークンあたり約1分かかります。続きのメッセージは数秒で始まります。
- チャットの保存場所と API 全体: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored)、[docs/DETAILS.md](docs/DETAILS.md#using-it)。
## ソースからビルド
<details>
<summary>リポジトリから Windows、Linux、Mac 用にビルドする</summary>
[Strata をダウンロード](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)して展開するか、`git clone` します。
**Windows:** `START-HERE.bat` をダブルクリックします。**Linux:** Strata フォルダで `./setup.sh` を実行します。
**Mac(Apple Silicon、macOS 15 以降):** 最初に一度、`xcode-select --install` で Apple のコマンドラインツールをインストールします
(Xcode 本体は不要です)。Strata フォルダでターミナルを開き、`./setup.sh` を実行します。Mac 用エンジンをお使いの Mac 上でビルドし
(約7分)、同じ質問をして、モデルをダウンロードし、起動します。2回目以降は `./setup.sh` ですぐに起動します。
お使いの Mac に合うサイズは [docs/MAC.md](docs/MAC.md) を見てください。
インストーラーはカードを見つけて、合うエンジンを設定します。どのモデルとサイズにするか、コンテキストをどれだけにするか(モデルが一度に覚えておけるテキストの量)、
画像を読ませるかを聞かれます。おすすめの答えでよければ Enter を押します。そのあとモデル(約70 GB)をダウンロードして起動します。
ダウンロードが止まったら、もう一度実行すると続きから再開します。ブラウザで `http://127.0.0.1:8080` の Strata ページが開きます。
> モデルの起動中は、PCが1〜3分ほど(初回がいちばん長い)遅くなったり、応答しなくなったりすることがあります。Strata は35〜55 GBをRAMに読み込み、
> その一部をグラフィックスカード用に固定するためです。ウィンドウを閉じずに待ってください。
次回からは `START-HERE.bat`(または `./setup.sh`)をもう一度実行するだけです。すぐに起動し、同じものを2回ダウンロードすることはありません。ウィンドウを閉じるとモデルが止まります。
`UPDATE.bat`(`./update.sh`)は、起動せずに Strata を更新します。更新、Docker、複数のカード、ファイルの保存先、すべてのオプションは
[docs/INSTALL.md](docs/INSTALL.md) にあります。
PCの性能を測るには、`BENCH.bat`(`./bench.sh`)を実行します。5〜10分かかり、共有できるレポートが作られます
([方法](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark))。
**オプション: モデル全体がRAMに入るとき、1枚のカードをさらに速くする。** RubenTorney による、自分でオンにする2つの設定です(#1904、#1905)。
グラフィックスカードが1枚で、RAM がすべてのエキスパートを保持できるときに効果があります。たとえば64 GBのPCでの Q2_0 や IQ3_XXS です。
Strata のデータフォルダにある `strata-<model>.json`([場所](docs/INSTALL.md#where-things-are-stored))を開いて、次を追加します。
- `"args"` のリストに `"--adapt-async", "1"` を追加します。エキスパートキャッシュが、回答を止めずに補助スレッドで更新されます。
RTX 5070(Windows、Q2_0)での測定: デコードが4〜5%、コードや長い文書では7〜10%速くなりました。回答は初期設定と少し違うことがありますが、実行ごとの結果は変わりません。
- `"env"` ブロックに `"STRATA_PIN_BY_NEED": "1"` を追加します(Windows)。Windows ではグラフィックスカード用に固定できる RAM が半分ほどしかありません。
この設定は、カードが最初に必要とするエキスパートを優先して固定します。RTX 5070 での測定: プロンプトの読み取りが4.4%速くなりました。
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
変更したら Strata を再起動してください。元に戻すには、2つの項目を削除します。
画面の案内に従って、モデルをダウンロードして起動します。[ターミナルガイド](docs/TERMINAL_LAUNCH.md)。
</details>
## 仕組み
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="モデルの動作、速度、メモリ、最近のリクエストを表示する Strata ターミナルアプリ">
<br><sub>動作中のモデルとハードウェアの状態を確認できます。</sub>
</p>
このようなモデルは、ふつう数百GBのグラフィックスメモリを持つサーバーで動きます。お使いのグラフィックスカードは12〜24 GBです。Strata は、PC全体で作業を分担して、モデルを収めます。
図書館にたとえてみます。
<a id="build-from-source"></a>
- モデルは、24,576人の小さな専門家(「エキスパート」)からなるチームです。24,576冊の参考書のようなものです。1つの単語に必要なのは、そのうち10冊だけです。
- **グラフィックスカードは机です。** 小さくて速く、最もよく使う約4,000冊が開いて置いてあります。
- **RAM は閲覧室の書棚です。** 24,576冊すべてがここにあります。単語に必要な本が机になければ、プロセッサがすぐに書棚から読み、その間もカードは作業を続けます。
- **SSD は書庫です。** 29 GBの参照テーブルがしまってあります。モデルは単語ごとにその数行だけを読むので、RAM に置いておく必要はありません。
[ソースからビルド](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [インストールガイド(全文)](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
小さな助手が、次の数単語も予想します。大きなモデルがそれをまとめて確認し、正しいものだけを残します。答えは同じで、1.6〜1.8倍早く返ります。
長い文章は、一度に最大8,192トークンの大きな塊で、毎秒1,000トークンを超える速さで読みます。
## ハードウェア
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| 対応ハードウェア | ガイド |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 シリーズ** | [NVIDIA のセットアップ](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900、RX 9060 / 9070、AI PRO R9700** | [AMD のセットアップ](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · 試験的** | [RX 6000 のセットアップ](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel のセットアップ](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 以降** | [Mac のセットアップ](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo のセットアップ](docs/STRIX_HALO.md) |
詳しい説明は [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md) にあります。各部分と数値は
[詳細](docs/DETAILS.md#how-it-works)と[論文](docs/paper/Strata-Paper.pdf)を見てください。
<a id="using-it"></a>
## うまくいかないとき
## ツール
- **Strata を初めて起動したときに、PCが固まりました。** モデルを RAM に読み込んでいます。ウィンドウを閉じずに待ってください。10分たっても固まったままなら、
PCを再起動し、ほかのプログラムを閉じてからもう一度試すか、小さいサイズを選んでください。
- **ダウンロードやインストールの途中で止まりました。** アプリか `strata launch` をもう一度起動するか、`START-HERE.bat`(または
`./setup.sh`)をもう一度実行してください。止まったところから再開します。
- **とても遅い、ディスクのランプが点滅し続ける、または「エンジンが予期せず停止しました」と表示される。** PCの空き RAM が足りません。
ほかのプログラムを閉じるか(ブラウザはメモリを多く使います)、Q2_0 や IQ2_XS のような小さいサイズを選んでください。
- **このPCに合うエンジンがないと表示される。** アプリでは、**設定、エンジン**を開いて**エンジンを更新**を押します。ターミナルでは
`strata update` を実行します。`strata doctor` で、どのエンジンファイルがインストールされているか分かります。
- **ポート 8080 はすでに使われていると表示される。** Strata がすでに動いています。ウィンドウを探してください。
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**。OpenAI 互換と Anthropic 互換の API を通じて、ほかのツールも使えます。
そのほかの問題と対処法は [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) にあります。それでも解決しない場合は、
[issue](https://github.com/Niko1221/Strata/issues) を作成してください。アプリでは、**設定、エンジン、診断、ログを見る、ログを保存**で作られるファイルを添付します。
ターミナルでは、`strata doctor` の出力を貼り付けます。ソースから動かしている場合は、Strata フォルダの `strata-<model>.log` を添付してください。
セキュリティ上の問題は、非公開で報告してください: [SECURITY.md](SECURITY.md)。
**`strata connect`** を実行すると、お使いのツールの設定方法が表示されます。[接続ガイド](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect)。
## クレジットとライセンス
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
モデルは Qwen チームの [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) です。圧縮は
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)、agentionai(Gyro)、
Unsloth が行いました。Strata は [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) の一部を使っています。すべてのクレジットは
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits) にあります。Strata は [MIT ライセンス](LICENSE)のオープンソースです。一部のコンポーネントと
すべてのモデルには、それぞれのライセンスがあります([該当するもの](docs/HOW_IT_WORKS.md#license))。
## ヘルプ
## コントリビューション
[すべてのガイド](docs) · [トラブルシューティング](docs/TROUBLESHOOTING.md) · [不具合を報告](https://github.com/Niko1221/Strata/issues/new/choose) ·
[ベンチマーク](docs/COMMUNITY_BENCHMARKS.md) · [仕組み](docs/HOW_IT_WORKS.md) · [貢献する](CONTRIBUTING.md)
バグを見つけた、変更やベンチマークを送りたい、という方は、先に [CONTRIBUTING.md](CONTRIBUTING.md) を読んでください。
ほかの人があなたの成果を確認できるように、何を書くべきかが説明してあります。
## Strata を支援する
## Strata を応援する
**ローカル AI をみんなに開かれたものに。** コーヒー1杯が、開発と、より多くのハードウェアでのテストの支えになります。
Strata は無料で、MIT ライセンスです。私が入手できるハードウェアで開発とテストをしています。お使いのPCで快適に動いたなら、コーヒー1杯分の支援は、
新しいカードのテストと、報告いただいた不具合の修正を続ける助けになります。無理のない範囲で大丈夫です。使って、気づいたことを報告してもらえるだけでも同じように助かります。
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: Strata を支援する" height="48"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
[MIT ライセンス](LICENSE) · [クレジットとモデルのライセンス](docs/HOW_IT_WORKS.md#credits) · [プライバシー](docs/SAFETY.md)
Strata のスポンサーに関心がある方は、[GitHub](https://github.com/Niko1221) から連絡してください。
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>AI ツールと自動リーダー向け: <a href="AGENTS.md">技術リファレンス</a> · <a href="docs/AI_SETUP.md">セットアップ手順</a></sub>
**スポンサー、ハードウェア提携、共同開発のご相談:** [[email protected]](mailto:[email protected]).
+126 -332
View File
@@ -1,399 +1,193 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><b>English</b> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<h3 align="center">Run a state-of-the-art model on your own gaming PC</h3>
<p align="center">Support for NVIDIA, AMD, Intel · Windows, Linux, Mac · free and open source</p>
<h3 align="center">Run powerful AI on your own computer.</h3>
<p align="center">Chat, code and work with images, locally.</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#models">Models</a> · <a href="#install"><b>Install</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#tools">Tools</a> ·
<a href="#help">Help</a> · <a href="#support-strata">Support</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">full video (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Download Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Support Strata on Buy Me a Coffee"></a>
</p>
Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, a 125-billion-parameter model that
normally needs a server, on a normal PC. It chats, writes code, reads pictures and works with your apps and coding
agents. Your chats and files stay on your PC.
<p align="center"><sub><b>English</b> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
**New here?** Check [what you need](#what-you-need), then [install the app](#option-a-the-strata-app).
**Free, open source, and yours to run.** Use Strata on your desktop or with your favorite coding tools. No account needed.
- **Runs on a gaming PC.** 32 GB of RAM and graphics memory together is enough to start.
- **Works with your tools.** It speaks the OpenAI and Anthropic APIs, so Claude Code, Codex CLI, OpenCode and other agents connect to it.
- **Reads pictures.** They are on by default.
- **Free.** MIT license, no account, no cloud.
- **Private.** No telemetry. What Strata downloads, when it uses the internet and how it checks downloads is in [docs/SAFETY.md](docs/SAFETY.md).
## Models
Version 0.2.2 · engine 0.1.47
We recommend **ISTA-DASLab first**, then **Unsloth** and other providers, where a compatible model fits.
## What you need
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**You need at least 32 GB of RAM and graphics memory (VRAM) together.** On a Mac or a Ryzen AI Max, that is the unified
memory. More memory gives a bigger model or a faster one.
- A 32 GB PC with a 12-16 GB card runs the Coder.
- 64 GB of RAM runs every size.
| Your computer | Details |
| Provider | Choices |
| --- | --- |
| **NVIDIA** GeForce RTX 20, 30, 40, 50 series (12 GB or more) | Install the app or the Terminal. |
| **AMD** Radeon RX 6800 and newer, Radeon AI PRO R9700 (12 GB or more) | [AMD cards](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [Intel engine](docs/INTEL.md) |
| **Apple Silicon Mac** (M1 or newer, macOS 15 or newer) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** and other unified-memory systems | [Strix Halo](docs/STRIX_HALO.md) |
| Older cards and processors | [Older GPUs](docs/OLDER_GPUS.md), [Older CPUs](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · recommended** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) for coding. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Lower memory use but slower. |
| **Other options** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
Also needed:
[Compare sizes, memory needs and compatibility](docs/MODELS.md).
- about 80 GB of free disk (an SSD makes the first start much faster)
- Windows 10 / 11, Linux or macOS
- a current graphics driver
</details>
Two or three cards of the same make can share a model ([multi-GPU](docs/MULTI_GPU.md)). Strata never splits a model across
NVIDIA and AMD cards. Full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. These are the currently documented packs for this model.
[Model guide and compatibility](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
In the **desktop app** and the **Terminal**. What does not fit in memory is read from your SSD, so it needs a lot of free
disk space.
| Set (IST-DAS) | Download | Free disk needed |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | about 245 GB |
| 2.00 bpw | 257 GB | about 265 GB |
| 2.25 bpw | 274 GB | about 280 GB |
Runs on NVIDIA RTX 30, 40 and 50 series (Windows and Linux), AMD Radeon AI PRO R9700 (Linux), and AMD Ryzen AI Max — Strix Halo
(Windows). Setup downloads the files and the DeepSeek engine, then starts the model.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Measured with the 2.00 bpw set on Strata 0.2.5 (Strix Halo: 0.2.7). Output is the speed of the answer, prompt is how fast Strata reads your input.
| Hardware | Output | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4 to 6 tokens/s | about 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9 to 11 tokens/s | about 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21 to 25 tokens/s | 320 to 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 to 77 tokens/s | 400 to 450 tokens/s (8K) |
| Ryzen AI Max+ 395 (Strix Halo), 128 GB unified | 19 to 21 tokens/s (code), 11 (chat) | 87 tokens/s (4K), 125 tokens/s (8K) |
[Memory, disk and details](docs/DEEPSEEK41.md) · [Release notes](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Not sure what fits? [Choose a model for your computer](docs/MODELS.md#pick-by-memory).
## Install
Pick one option. They all start the same engine and use the same models.
<a id="option-a-the-strata-app"></a>
### Option A: the Strata app
### Desktop app
The easiest way. The app has the engine inside: install it, pick a model, chat. Download the file for your system and
graphics card from the [releases page](https://github.com/Niko1221/Strata/releases/latest). Not sure which one? See the
line under the table.
**[Download Strata for Windows, Linux or Mac →](https://github.com/Niko1221/Strata/releases/latest)**
| System | Your graphics card | File to download |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, driver 580 or newer | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA, older card or driver older than 580 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon RX 9000 series or Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon RX 7000 series or Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, driver 580 or newer | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, older card or driver older than 580 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 series or Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 series | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 or newer) | built in | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
Install the app, choose a model, and start chatting. There is one installer for each system. On first start the app
downloads the engine for your graphics card. [Help with the installer](docs/INSTALL.md#the-app).
**If you are unsure:** for NVIDIA, take `cuda13` if your driver is version 580 or newer, otherwise `cuda12`. `cuda12` is
for older cards and older drivers. The `hip` and `amd` files are for AMD cards. RX 6000 cards have no installer of
their own: install `hip-rdna3` (Windows) or `amd-rx7900` (Linux), and on a PC with an RX 6000 card it fetches the RX 6000 engine by itself
(also `strata update`). That engine is built but not yet tested on a real RX 6000 card: see [AMD cards](docs/AMD_HIP.md).
<sub>Files: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
Then:
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Strata desktop app with a chat and live code preview side by side">
<br><sub>Chat, files and a live preview in one window.</sub>
</p>
- **Windows:** run the `.exe`. It installs for your user and needs no administrator rights. If Windows says "Windows
protected your PC", choose **More info**, then **Run anyway** (the installer is not code-signed yet).
- **Linux:** make the file runnable (`chmod +x`) and start it. It needs FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04)
or `libfuse2t64` (Ubuntu 24.04 and newer).
- **Mac:** open the `.dmg` and drag Strata onto Applications.
<a id="option-b-the-terminal-one-line"></a>
On the first screen the app reads your PC and recommends a model that fits. Press install. The model is a big download
(about 40-70 GB) and continues where it stopped if it is interrupted. Closing the window stops Strata and the engine.
### Terminal app
### Option B: the Terminal (one line)
Prefer the terminal? Install Strata, then run **`strata launch`**.
The Terminal is the `strata` command, for scripts, servers and AI agents.
<details open>
<summary><b>Install commands</b></summary>
Linux and Mac:
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows (PowerShell):
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
The script checks your PC, downloads the Terminal, verifies its SHA-256 and unpacks it into your home folder. It needs no
administrator rights. On Windows, open a new terminal window afterwards. Then start Strata:
On Windows, open a new terminal afterwards. Then:
```sh
strata launch
```
It asks one question at a time: model, context, pictures, who may connect. Then it downloads the engine for your graphics
card (signed, and test-started before use) and the model, and starts serving. For scripts and AI agents, use flags:
```sh
strata launch --model q2_0 --yes --no-tui
```
To read the install script before it runs, download it first with `curl -fsSL <address> -o install.sh`, read it, then run
`sh install.sh`. All screens and flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Option C: let your AI set it up
Paste this into Claude Code, Cursor, Codex, GitHub Copilot or another coding assistant:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
It checks your graphics card, RAM and disk, picks the model that fits, installs and starts it, and tells you how to
connect your apps. AI tools can also install, start and stop Strata through its [MCP server](docs/MCP_SERVER.md).
Prefer to build it yourself? See [Build from source](#build-from-source).
## Models
The app and `strata launch` recommend a model for your memory. It is one model in several sizes, compressed more or less.
Smaller sizes are faster. Larger sizes are a bit smarter.
### Low on memory? Try Gyro
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (by agentionai) is the smallest way to run this model.
| Size | Memory it needs (RAM + VRAM) | Download | What it is |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27.6 GB | 58.5 GB | For PCs that fit nothing else. Quality is lower than IQ2_XS. |
| **Gyro-M** | 35 GB | 92.0 GB | About IQ3_XXS quality with about 20% less GPU memory. It writes answers 16% (12 GB RTX 3060) to 21% (R9700) slower than Q2_0. |
Gyro runs on NVIDIA and AMD cards (AMD is experimental) on Intel Arc cards on Linux (tested on an Arc A750) and, from 0.2.2, on Apple Silicon Macs (Gyro-S on a 32 GB Mac mini: 9 tok/s, streaming from the SSD; [docs/MAC.md](docs/MAC.md#gyro-s-and-gyro-m-agentionais-trellis-experts)). Details and measurements:
[docs/GYRO.md](docs/GYRO.md). Start it with `strata launch --model gyro-s`.
### The other sizes
The tiers count your RAM plus your VRAM, less 5 GB that Windows and the system keep for themselves. Every other size stays a choice.
| Your memory (RAM + VRAM) | Take | Notes |
| --- | --- | --- |
| **32 GB** | **Gyro-S**, or the **[Coder](docs/MODELS.md#coder)** for code | The Coder needs 32 GB of RAM and keeps half of the experts, so it is weaker outside code |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39.2 GB), or **Gyro-M** (35 GB) | Swift thinks for a shorter time before it answers. Gyro-M is close to IQ3_XXS in quality, but it writes answers 16-21% slower than Q2_0 |
| **48 GB** | **IQ3_XXS** (47.0 GB) | Slower, better |
| **64 GB** | **IQ3_S** (54.8 GB), the original or Swift 1.5 | The slowest, the best: it matches the full model on the published tests |
| **96 GB or more** | Unsloth's **UD-IQ4_XS** (about 72 GB of RAM) to **Q8_0** (about 136 GB), 4 to 8 bit | Closer to the full model. With less RAM, Strata reads part of them from the SSD, which is much slower |
Less RAM than a row says can still work: Strata switches to its
[low-RAM mode](docs/MODELS.md#a-big-graphics-card-and-little-ram) by itself. Expect it to be slower, and much slower
with a small graphics card.
More choices:
- **[Swift 1.5](docs/MODELS.md#swift-15)** is a fine-tune that thinks for a shorter time before it answers, at about the same quality and RAM.
- **[NVFP4](docs/NVFP4.md)** is NVIDIA's 4-bit version for NVIDIA cards. It needs about 80 GB of RAM to keep every expert in RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** is a smaller, different model that reads pictures and can answer several requests at once.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** is set up by hand.
Every size, download and what fits where: [docs/MODELS.md](docs/MODELS.md).
## How fast is it?
Based on a 12 GB graphics card (RTX 5070, Ryzen 5 7600, 64 GB RAM). See results from other cards:
[speed of each model](docs/MODELS.md#how-fast-is-each-size), [community results](docs/COMMUNITY_BENCHMARKS.md).
| Model | Answers appear at | Reads your prompt at |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |
A token is about ¾ of a word, so 60 tokens per second is faster than you can read. "Reads your prompt" is how fast Strata
takes in a long document, code or chat history.
Measured on a 16 GB AMD card (RX 9070 XT, Ryzen 9 3900X, 47 GB RAM):
| Model | Answers appear at | Reads your prompt at |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |
The bars use one scale for both tables. A card with more VRAM is faster: an RTX 3090 (24 GB) should write about 100-140
tokens per second. The engine version behind each number is in [DETAILS.md](docs/DETAILS.md#speed-measured).
## Using it
### In the app
The app has a **Chat**, a **Monitor** of the model and your GPU, CPU and RAM, a **Models** page and **Settings**. The chat
can work in a folder on your PC. In **Manual** mode it asks before it changes a file or runs a command. Type
`/loop 10m <what to do>` to repeat a message every 10 minutes while the chat is open.
The chat can search the web with DuckDuckGo (or Bing), with no account. You can add a search key under **Settings, Tools**.
Search is the one feature that sends something off your PC, the search words.
To run two models, turn on **Run multiple models** and give each model its own card of the same make.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>A chat in the Strata app: the agent wrote a small voxel pagoda garden as one HTML file, and the page opens in the preview next to the conversation</sub></p>
### In the Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> while the model writes a reply: speed, GPU, memory and the recent requests</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` lists everything. Starting and serving never download anything. Only the install, `strata launch` (when
you choose a model) and `strata update` do. To reach Strata from your phone or another PC, use
`strata launch --access lan`. It makes an API key (recommended; add `--no-api-key` to go without, with a warning). No browser opens unless
you add `--open`. Guide: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Whole client: [docs/TERMINAL.md](docs/TERMINAL.md).
### With AI agents
An agent installs Strata with the one-line command in Option B and runs
`strata launch --model NAME --yes --no-tui`. The steps, flags and exit codes are in [docs/AI_SETUP.md](docs/AI_SETUP.md),
and the repository rules are in [AGENTS.md](AGENTS.md).
### Connect your other apps
| App type | Base URL |
| --- | --- |
| OpenAI-compatible (most apps and agents) | `http://127.0.0.1:8080/v1` (any API key, any model name) |
| Anthropic API (Claude Code and others) | `http://127.0.0.1:8080`, without `/v1`. These apps add `/v1/messages` themselves. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` prints the exact settings for OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent and OpenClaw.
`strata connect opencode --write` writes the file after asking and keeps a backup.
### Pictures
Pictures are on by default for models that can read them. Attach or paste a picture in the chat, or send it from your app.
The picture reader runs on the processor when your graphics card has less than 24 GB (a few seconds per picture, nothing
taken from the card's memory) and on the graphics card from 24 GB. The rule is the same for NVIDIA and AMD. Turn pictures
off per model, or choose the reader with `--vision off`, `--vision gpu` or `--vision cpu`.
The numbers behind the rule: on a Radeon AI PRO R9700 the card's reader took 57 ms for a 260-token picture and the
processor's took 3.1 s. On 16 GB cards (Tesla P100 and RTX A4000, Linux) the card's reader cost 3-5% of text speed,
because it holds about 1.4 GB that the model would use otherwise. On a Mac the reader runs on the Mac's GPU whatever its memory (0.45 s for a 260-token picture on an M4 with 32 GB, about 1.2 GB of the shared
memory); the processor reader needs about 50 s per picture there, so it runs only when you ask for it with `--vision cpu`.
### Good to know
- Choose the thinking level (**off, low, medium or high**) in the chat menu or in your app's "reasoning effort". Off is fastest.
- Strata answers one request at a time by default. For several at once, use `--parallel N` in `strata launch` or
`"parallel": 2` in the config ([BATCHING.md](docs/BATCHING.md)). On a 12 GB card this makes each answer slower.
- Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up messages start in seconds.
- Where your chats are stored and the whole API: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Build from source
<details>
<summary>Windows, Linux and Mac from the repository</summary>
[Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it, or `git clone` it.
**Windows:** double-click `START-HERE.bat`. **Linux:** run `./setup.sh` in the Strata folder.
**Mac (Apple Silicon, macOS 15 or newer):** install Apple's Command Line Tools once with `xcode-select --install`
(Xcode itself is not needed). Open Terminal in the Strata folder and run `./setup.sh`. It builds the Mac engine on your
Mac (about 7 minutes), asks the same questions, downloads the model and starts it. Next time, `./setup.sh` starts it
right away. Which size fits your Mac: [docs/MAC.md](docs/MAC.md).
The installer finds your card and sets up the right engine. It asks which model and size, how much context (how much text
the model keeps in mind) and whether it should read pictures. Press Enter for the recommended answer. Then it downloads the
model (about 70 GB) and starts it. If the download stops, run it again and it continues. Your browser opens the Strata
page at `http://127.0.0.1:8080`.
> While the model starts, your PC can be slow or stop responding for 1-3 minutes (longest the first time). Strata loads
> 35-55 GB into your RAM and locks part of it for the graphics card. Wait, and don't close the window.
Next time, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close its
window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker, several cards,
where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md).
To measure your PC, run `BENCH.bat` (`./bench.sh`). It takes 5-10 minutes and writes a report you can share
([how](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Optional: more speed on one card when the whole model fits in RAM.** Two opt-in settings by RubenTorney (#1904, #1905).
They help when you run one graphics card and your RAM holds every expert, for example Q2_0 or IQ3_XXS on a 64 GB PC. Open
`strata-<model>.json` in the Strata data folder ([where](docs/INSTALL.md#where-things-are-stored)) and add:
- `"--adapt-async", "1"` to the `"args"` list. The expert cache then updates on a helper thread instead of pausing the
answer. Measured on an RTX 5070 (Windows, Q2_0): +4-5% decode, +7-10% on code and long documents. Answers can differ
slightly from the default and stay the same from run to run.
- `"STRATA_PIN_BY_NEED": "1"` in the `"env"` block (Windows). Windows can lock only about half the RAM for the graphics
card, and this locks the experts the card needs first. Measured on the RTX 5070: +4.4% prompt reading.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Restart Strata after the change. To undo, remove the two entries.
Follow the prompts to download and start a model. [Terminal guide](docs/TERMINAL_LAUNCH.md).
</details>
## How does it work?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Strata terminal app showing model activity, speed, memory and recent requests">
<br><sub>Watch your model and hardware while it runs.</sub>
</p>
Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has
12-24 GB. Strata makes the model fit by sharing the work across your whole PC. Picture a library.
<a id="build-from-source"></a>
- The model is a team of 24,576 small specialists ("experts"), like 24,576 reference books. Each word needs only 10 of them.
- **Your graphics card is the desk.** It is small and fast. The roughly 4,000 books used most often lie open on it.
- **Your RAM is the shelves in the reading room.** All 24,576 books are there. When a word needs a book that is not on the
desk, your processor reads it from the shelf at once, while the card keeps working.
- **Your SSD is the archive.** It holds a 29 GB lookup table. The model reads a few rows of it per word, so it does not
need to sit in RAM.
[Build from source](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Full install guide](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
A small helper also guesses the next few words. The big model checks all of them at once and keeps the right ones. You get
the same answer, 1.6-1.8x sooner. Long texts are read in big pieces, up to 8,192 tokens at a time, at over 1,000 tokens
per second.
## Hardware
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| Supported hardware | Guide |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 series** | [NVIDIA setup](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [AMD setup](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimental** | [RX 6000 setup](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel setup](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 or newer** | [Mac setup](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo setup](docs/STRIX_HALO.md) |
The longer explanation: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Every part and its numbers:
[the details](docs/DETAILS.md#how-it-works) and the [paper](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Something went wrong?
## Tools
- **My PC froze the first time Strata started.** It loads the model into RAM. Wait, and don't close the window. Still
frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.
- **It stopped while downloading or installing.** Start the app or `strata launch` again, or run `START-HERE.bat` (or
`./setup.sh`) again. It continues where it stopped.
- **It is very slow, the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does not have
enough free RAM. Close other programs (browsers use a lot), or pick a smaller size such as Q2_0 or IQ2_XS.
- **It says there is no engine for this PC.** In the app, open **Settings, Engine** and press **Update engine**. In the
Terminal, run `strata update`. `strata doctor` shows which engine files are installed.
- **It says port 8080 is already in use.** Strata is already running. Look for its window.
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, and more through the OpenAI- and Anthropic-compatible APIs.
More problems and fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an
[issue](https://github.com/Niko1221/Strata/issues). In the app, attach the file from **Settings, Engine, Diagnostics, View
logs, Save logs**. In the Terminal, paste the output of `strata doctor`. From source, attach `strata-<model>.log` from the
Strata folder. For a security problem, report it privately: [SECURITY.md](SECURITY.md).
Run **`strata connect`** to see setup instructions for your tool. [Connection guide](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
## Credits and license
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was compressed by
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
and Unsloth. Strata uses parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). All credits:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few parts and
every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).
## Help
## Contributing
Found a bug, or want to send a change or a benchmark? Read [CONTRIBUTING.md](CONTRIBUTING.md) first. It says what to
include so that others can check your work.
[All guides](docs) · [Troubleshooting](docs/TROUBLESHOOTING.md) · [Report a bug](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [How it works](docs/HOW_IT_WORKS.md) · [Contribute](CONTRIBUTING.md)
## Support Strata
Strata is free and MIT-licensed. I build it and test it on whatever hardware I can get. If it runs well on your PC, a coffee helps
me keep testing new cards and fixing what you report. No pressure. Using it and reporting what you find helps just as much.
**Help keep local AI open to everyone.** A coffee helps fund development and testing on more hardware.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: support Strata" height="48"></a>
[MIT license](LICENSE) · [Credits and model licenses](docs/HOW_IT_WORKS.md#credits) · [Privacy](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>For AI tools and automated readers: <a href="AGENTS.md">technical reference</a> · <a href="docs/AI_SETUP.md">setup instructions</a>.</sub>
**Sponsorships, hardware partnerships or collaborations:** [[email protected]](mailto:[email protected]).
Interested in sponsoring Strata? Contact me via [GitHub](https://github.com/Niko1221).
+192
View File
@@ -0,0 +1,192 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<h3 align="center">Uruchom potężną sztuczną inteligencję na własnym komputerze.</h3>
<p align="center">Rozmawiaj, programuj i pracuj z obrazami — lokalnie.</p>
<p align="center">
<a href="#modele">Modele</a> · <a href="#instalacja"><b>Instalacja</b></a> ·
<a href="#sprzęt">Sprzęt</a> · <a href="#narzędzia">Narzędzia</a> ·
<a href="#pomoc">Pomoc</a> · <a href="#wesprzyj-stratę">Wsparcie</a>
</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Pobierz Stratę"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licencja MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Wesprzyj Stratę na Buy Me a Coffee"></a>
</p>
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <b>Polski</b></sub></p>
**Bezpłatna, z otwartym kodem źródłowym i do uruchomienia u siebie.** Używaj Straty na swoim komputerze albo razem z ulubionymi narzędziami do programowania. Konto nie jest potrzebne.
## Modele
Najpierw polecamy **ISTA-DASLab**, potem **Unsloth** i innych dostawców — o ile zgodny model mieści się w pamięci.
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Dostawca | Do wyboru |
| --- | --- |
| **ISTA-DASLab · polecany** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) do programowania. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Zużywa mniej pamięci, ale działa wolniej. |
| **Inne opcje** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
[Porównaj rozmiary, wymagania pamięciowe i zgodność](docs/MODELS.md).
</details>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. To obecnie udokumentowane pakiety dla tego modelu.
[Przewodnik po modelu i zgodność](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
W **aplikacji na komputer** i w **Terminalu**. To, co nie mieści się w pamięci, jest czytane z dysku SSD, więc potrzebujesz
dużo wolnego miejsca na dysku.
| Zestaw (IST-DAS) | Do pobrania | Potrzebne wolne miejsce |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | około 245 GB |
| 2.00 bpw | 257 GB | około 265 GB |
| 2.25 bpw | 274 GB | około 280 GB |
Działa na kartach NVIDIA RTX serii 30, 40 i 50 (Windows i Linux), AMD Radeon AI PRO R9700 (Linux) oraz AMD Ryzen AI Max — Strix Halo
(Windows). Instalacja pobiera pliki i silnik DeepSeek, a potem uruchamia model.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Zmierzone na zestawie 2.00 bpw w Strata 0.2.5 (Strix Halo: 0.2.7). „Wyjście” to szybkość odpowiedzi, „prompt” — jak szybko Strata czyta twój tekst.
| Sprzęt | Wyjście | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4–6 tokenów/s | około 110 tokenów/s (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9–11 tokenów/s | około 155 tokenów/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21–25 tokenów/s | 320–360 tokenów/s (8K) |
| 4x Radeon AI PRO R9700 | 74–77 tokenów/s | 400–450 tokenów/s (8K) |
| Ryzen AI Max+ 395 (Strix Halo), 128 GB pamięci współdzielonej | 19–21 tokenów/s (kod), 11 (czat) | 87 tokenów/s (4K), 125 tokenów/s (8K) |
[Pamięć, dysk i szczegóły](docs/DEEPSEEK41.md) · [Informacje o wydaniach](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Nie wiesz, co się zmieści? [Wybierz model do swojego komputera](docs/MODELS.md#pick-by-memory).
## Instalacja
<a id="option-a-the-strata-app"></a>
### Aplikacja na komputer
**[Pobierz Stratę dla Windows, Linuxa lub Maca →](https://github.com/Niko1221/Strata/releases/latest)**
Zainstaluj aplikację, wybierz model i zacznij rozmawiać. Dla każdego systemu jest jeden instalator. Przy pierwszym
uruchomieniu aplikacja pobiera silnik dla twojej karty graficznej. [Pomoc z instalatorem](docs/INSTALL.md#the-app).
<sub>Pliki: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Aplikacja Strata na komputer: czat i podgląd kodu na żywo obok siebie">
<br><sub>Czat, pliki i podgląd na żywo w jednym oknie.</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### Aplikacja Terminal
Wolisz terminal? Zainstaluj Stratę, a potem uruchom **`strata launch`**.
<details open>
<summary><b>Polecenia instalacji</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
W Windows otwórz potem nowe okno terminala. Następnie:
```sh
strata launch
```
Postępuj zgodnie z podpowiedziami, aby pobrać i uruchomić model. [Przewodnik po Terminalu](docs/TERMINAL_LAUNCH.md).
</details>
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplikacja Terminal Straty pokazująca aktywność modelu, szybkość, pamięć i ostatnie zapytania">
<br><sub>Obserwuj model i sprzęt w trakcie pracy.</sub>
</p>
<a id="build-from-source"></a>
[Budowanie ze źródeł](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Pełny przewodnik instalacji](docs/INSTALL.md)
<a id="what-you-need"></a>
## Sprzęt
| Obsługiwany sprzęt | Przewodnik |
| --- | --- |
| **NVIDIA GeForce RTX serii 20 / 30 / 40 / 50** | [Konfiguracja NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Konfiguracja AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · eksperymentalnie** | [Konfiguracja RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Konfiguracja Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 lub nowszy** | [Konfiguracja Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Konfiguracja Strix Halo](docs/STRIX_HALO.md) |
<a id="using-it"></a>
## Narzędzia
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw** i inne — przez API zgodne z OpenAI i Anthropic.
Uruchom **`strata connect`**, aby zobaczyć instrukcję konfiguracji dla swojego narzędzia. [Przewodnik po połączeniach](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## Pomoc
[Wszystkie przewodniki](docs) · [Rozwiązywanie problemów](docs/TROUBLESHOOTING.md) · [Zgłoś błąd](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Testy wydajności](docs/COMMUNITY_BENCHMARKS.md) · [Jak to działa](docs/HOW_IT_WORKS.md) · [Współtworzenie](CONTRIBUTING.md)
## Wesprzyj Stratę
**Pomóż zachować lokalną sztuczną inteligencję otwartą dla wszystkich.** Kawa pomaga finansować rozwój i testy na większej liczbie sprzętu.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: wesprzyj Stratę" height="48"></a>
[Licencja MIT](LICENSE) · [Podziękowania i licencje modeli](docs/HOW_IT_WORKS.md#credits) · [Prywatność](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Dla narzędzi AI i automatycznych czytników: <a href="AGENTS.md">dokumentacja techniczna</a> · <a href="docs/AI_SETUP.md">instrukcja instalacji</a>.</sub>
**Sponsoring, partnerstwa sprzętowe i współpraca:** [[email protected]](mailto:[email protected]).
+121 -327
View File
@@ -1,394 +1,188 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <b>Português</b></p>
<h3 align="center">Rode um modelo de ponta no seu próprio PC gamer</h3>
<p align="center">Suporte a NVIDIA, AMD, Intel · Windows, Linux, Mac · gratuito e de código aberto</p>
<h3 align="center">IA poderosa no seu próprio computador.</h3>
<p align="center">Converse, programe e trabalhe com imagens, localmente.</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#modelos">Modelos</a> · <a href="#instalação"><b>Instalação</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#ferramentas">Ferramentas</a> ·
<a href="#ajuda">Ajuda</a> · <a href="#apoie-o-strata">Apoiar</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Um jardim de pagodes em voxels, gerado com um único prompt, rodando numa RTX 5070 com o Strata (IQ3_S, contexto de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vídeo completo (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Baixar o Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licença MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Apoie o Strata no Buy Me a Coffee"></a>
</p>
O Strata roda o **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, um modelo de 125 bilhões de parâmetros que
normalmente precisa de um servidor, num PC comum. Ele conversa, escreve código, lê imagens e funciona com os seus aplicativos e
agentes de programação. Suas conversas e arquivos ficam no seu PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <b>Português</b> · <a href="README.pl.md">Polski</a></sub></p>
**Primeira vez aqui?** Veja [o que você precisa](#o-que-você-precisa) e depois [instale o app](#opção-a-o-app-strata).
**Gratuito, de código aberto e seu para executar.** Use o Strata no desktop ou com suas ferramentas de programação favoritas. Não precisa de conta.
- **Roda num PC gamer.** 32 GB somando memória RAM e memória da placa de vídeo já bastam para começar.
- **Funciona com as suas ferramentas.** Ele fala as APIs da OpenAI e da Anthropic, então Claude Code, Codex CLI, OpenCode e outros agentes se conectam a ele.
- **Lê imagens.** Já vêm ativadas.
- **Gratuito.** Licença MIT, sem conta, sem nuvem.
- **Privado.** Sem telemetria. O que o Strata baixa, quando usa a internet e como confere os downloads está em [docs/SAFETY.md](docs/SAFETY.md).
## Modelos
Versão 0.2.2 · mecanismo 0.1.47
Recomendamos **primeiro a ISTA-DASLab**, depois a **Unsloth** e outros provedores, quando um modelo compatível cabe na memória.
## O que você precisa
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**Você precisa de pelo menos 32 GB somando a memória RAM e a memória da placa de vídeo (VRAM).** Num Mac ou num Ryzen AI Max, isso é a
memória unificada. Mais memória permite um modelo maior ou mais rápido.
- Um PC de 32 GB com uma placa de 12 a 16 GB roda o Coder.
- 64 GB de RAM rodam todos os tamanhos.
| Seu computador | Detalhes |
| Provedor | Opções |
| --- | --- |
| **NVIDIA** GeForce RTX série 20, 30, 40, 50 (12 GB ou mais) | Instale o app ou o Terminal. |
| **AMD** Radeon RX 6800 e mais novas, Radeon AI PRO R9700 (12 GB ou mais) | [Placas AMD](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [mecanismo Intel](docs/INTEL.md) |
| **Mac com Apple Silicon** (M1 ou mais novo, macOS 15 ou mais novo) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** e outros sistemas de memória unificada | [Strix Halo](docs/STRIX_HALO.md) |
| Placas e processadores mais antigos | [GPUs antigas](docs/OLDER_GPUS.md), [CPUs antigas](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · recomendado** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) para programar. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Usa menos memória, mas é mais lento. |
| **Outras opções** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
Você também precisa de:
[Compare tamanhos, necessidades de memória e compatibilidade](docs/MODELS.md).
- cerca de 80 GB de espaço livre no disco (um SSD deixa a primeira inicialização bem mais rápida)
- Windows 10 / 11, Linux ou macOS
- um driver de vídeo atualizado
</details>
Duas ou três placas da mesma marca podem dividir um modelo ([multi-GPU](docs/MULTI_GPU.md)). O Strata nunca divide um modelo entre
placas NVIDIA e AMD. Lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Esses são os pacotes documentados atualmente para este modelo.
[Guia do modelo e compatibilidade](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
No **aplicativo de desktop** e no **Terminal**. O que não cabe na memória é lido do seu SSD, por isso é preciso muito espaço livre em disco.
| Conjunto (IST-DAS) | Download | Espaço livre necessário |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | cerca de 245 GB |
| 2.00 bpw | 257 GB | cerca de 265 GB |
| 2.25 bpw | 274 GB | cerca de 280 GB |
Roda em NVIDIA RTX 30, 40 e 50 (Windows e Linux) e em AMD Radeon AI PRO R9700 (Linux). A instalação baixa os arquivos e o motor do DeepSeek e depois inicia o modelo.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Medido com o conjunto de 2.00 bpw no Strata 0.2.5. “Saída” é a velocidade da resposta; “Prompt” é a rapidez com que o Strata lê o seu texto.
| Hardware | Saída | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB de RAM | 4 a 6 tokens/s | cerca de 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB de RAM | 9 a 11 tokens/s | cerca de 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB de RAM | 21 a 25 tokens/s | 320 a 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 a 77 tokens/s | 400 a 450 tokens/s (8K) |
[Memória, disco e detalhes](docs/DEEPSEEK41.md) · [Notas da versão](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Não sabe o que cabe? [Escolha um modelo para o seu computador](docs/MODELS.md#pick-by-memory).
## Instalação
Escolha uma opção. Todas iniciam o mesmo mecanismo e usam os mesmos modelos.
<a id="option-a-the-strata-app"></a>
### Opção A: o app Strata
### Aplicativo de desktop
O jeito mais fácil. O app já traz o mecanismo: instale, escolha um modelo e converse. Baixe o arquivo para o seu sistema e a sua
placa de vídeo na [página de releases](https://github.com/Niko1221/Strata/releases/latest). Não sabe qual? Veja a explicação
logo abaixo da tabela.
**[Baixar o Strata para Windows, Linux ou Mac →](https://github.com/Niko1221/Strata/releases/latest)**
| Sistema | Sua placa de vídeo | Arquivo para baixar |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, driver 580 ou mais novo | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA, placa mais antiga ou driver anterior ao 580 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon série RX 7000 ou Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, driver 580 ou mais novo | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, placa mais antiga ou driver anterior ao 580 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon série RX 7000 | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 ou mais novo) | integrada | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
Instale o aplicativo, escolha um modelo e comece a conversar. Há um instalador por sistema. Na primeira vez que abre, o aplicativo baixa o motor da sua placa de vídeo. [Ajuda com o instalador](docs/INSTALL.md#the-app).
**Se você estiver em dúvida:** para NVIDIA, use `cuda13` se o seu driver for a versão 580 ou mais nova, e `cuda12` nos outros casos. O `cuda12`
serve para placas e drivers mais antigos. Os arquivos `hip` e `amd` são para placas AMD. Outras placas AMD (RX 6000) ainda não têm app pronto: veja [Placas AMD](docs/AMD_HIP.md).
<sub>Arquivos: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
Depois:
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Aplicativo de desktop do Strata com um chat e uma prévia do código ao vivo lado a lado">
<br><sub>Chat, arquivos e prévia ao vivo em uma só janela.</sub>
</p>
- **Windows:** execute o `.exe`. Ele instala só para o seu usuário e não pede direitos de administrador. Se o Windows disser "O Windows
protegeu o computador", escolha **Mais informações** e depois **Executar assim mesmo** (o instalador ainda não tem assinatura de código).
- **Linux:** torne o arquivo executável (`chmod +x`) e inicie. É preciso ter o FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04)
ou `libfuse2t64` (Ubuntu 24.04 e mais novos).
- **Mac:** abra o `.dmg` e arraste o Strata para Aplicativos.
<a id="option-b-the-terminal-one-line"></a>
Na primeira tela, o app lê o seu PC e recomenda um modelo que caiba nele. Clique em instalar. O modelo é um download grande
(cerca de 40 a 70 GB) e continua de onde parou se for interrompido. Fechar a janela encerra o Strata e o mecanismo.
### Aplicativo de Terminal
### Opção B: o Terminal (uma linha)
Prefere o terminal? Instale o Strata e execute **`strata launch`**.
O Terminal é o comando `strata`, para scripts, servidores e agentes de IA.
<details open>
<summary><b>Comandos de instalação</b></summary>
Linux e Mac:
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows (PowerShell):
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
O script confere o seu PC, baixa o Terminal, verifica o SHA-256 e descompacta na sua pasta pessoal. Não precisa de direitos de
administrador. No Windows, abra uma nova janela do terminal depois. Então inicie o Strata:
No Windows, abra um novo terminal depois. Em seguida:
```sh
strata launch
```
Ele faz uma pergunta por vez: modelo, contexto, imagens, quem pode se conectar. Depois baixa o mecanismo para a sua placa de vídeo
(assinada e testada antes do uso) e o modelo, e começa a servir. Em scripts e agentes de IA, use flags:
```sh
strata launch --model q2_0 --yes --no-tui
```
Para ler o script de instalação antes de executá-lo, baixe-o primeiro com `curl -fsSL <address> -o install.sh`, leia e então rode
`sh install.sh`. Todas as telas e flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Opção C: deixe a sua IA configurar
Cole isto no Claude Code, Cursor, Codex, GitHub Copilot ou outro assistente de programação:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
Ele confere a sua placa de vídeo, a RAM e o disco, escolhe o modelo que cabe, instala e inicia, e diz como conectar os seus
aplicativos. Ferramentas de IA também podem instalar, iniciar e parar o Strata pelo [servidor MCP](docs/MCP_SERVER.md) dele.
Prefere compilar por conta própria? Veja [Compilar a partir do código-fonte](#compilar-a-partir-do-código-fonte).
## Modelos
O app e o `strata launch` recomendam um modelo para a sua memória. É um único modelo em vários tamanhos, mais ou menos comprimidos.
Os tamanhos menores são mais rápidos. Os maiores são um pouco mais inteligentes.
### Pouca memória? Experimente o Gyro
O [Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (da agentionai) é a menor forma de rodar este modelo.
| Tamanho | Memória necessária (RAM + VRAM) | Download | O que é |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 GB | 58,5 GB | Para PCs onde nenhum outro tamanho cabe. A qualidade é menor que a do IQ2_XS. |
| **Gyro-M** | 35 GB | 92,0 GB | Qualidade próxima à do IQ3_XXS, com cerca de 20% menos memória da GPU. Ele escreve as respostas 16% mais devagar que o Q2_0 numa RTX 3060 de 12 GB e 21% mais devagar numa R9700. |
O Gyro roda em placas NVIDIA e AMD (AMD é experimental) em placas Intel Arc no Linux (testado numa Arc A750) e, desde a 0.2.2, em Macs com Apple Silicon (Gyro-S num Mac mini de 32 GB: 9 tokens/s, com os especialistas lidos do SSD). Detalhes e medições:
[docs/GYRO.md](docs/GYRO.md). Inicie com `strata launch --model gyro-s`.
### Os outros tamanhos
| Sua memória (RAM + VRAM) | Escolha | Observações |
| --- | --- | --- |
| **32 GB** | **Gyro-S**, ou o **[Coder](docs/MODELS.md#coder)** para código | O Coder precisa de 32 GB de RAM e mantém metade dos experts, então é mais fraco fora de código |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 GB), ou **Gyro-M** (35 GB) | O Swift pensa por menos tempo antes de responder. O Gyro-M chega perto do IQ3_XXS em qualidade, mas escreve as respostas 16 a 21% mais devagar que o Q2_0 |
| **48 GB** | **IQ3_XXS** (47,0 GB) | Mais lento, melhor |
| **64 GB** | **IQ3_S** (54,8 GB), o original ou o Swift 1.5 | O mais lento, o melhor: empata com o modelo completo nos testes publicados |
| **96 GB ou mais** | **UD-IQ4_XS** da Unsloth (cerca de 72 GB de RAM) até **Q8_0** (cerca de 136 GB), de 4 a 8 bits | Mais perto do modelo completo. Com menos RAM, o Strata lê parte deles do SSD, o que é bem mais lento |
Com menos RAM do que a linha indica, ainda pode funcionar: o Strata muda sozinho para o
[modo de pouca RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Espere mais lentidão, e muito mais com uma placa de vídeo pequena.
Mais opções:
- **[Swift 1.5](docs/MODELS.md#swift-15)** é um ajuste fino que pensa por menos tempo antes de responder, com qualidade e RAM parecidas.
- **[NVFP4](docs/NVFP4.md)** é a versão de 4 bits da NVIDIA para placas NVIDIA. Precisa de cerca de 80 GB de RAM para manter todos os experts na RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** é outro modelo, menor, que lê imagens e pode atender vários pedidos ao mesmo tempo.
- **[Uncensored IQ3_XXS da OrcaRouter](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** é configurado manualmente.
Todos os tamanhos, downloads e o que cabe em cada configuração: [docs/MODELS.md](docs/MODELS.md).
## Qual é a velocidade?
Com base numa placa de vídeo de 12 GB (RTX 5070, Ryzen 5 7600, 64 GB de RAM). Veja resultados de outras placas:
[velocidade de cada modelo](docs/MODELS.md#how-fast-is-each-size), [resultados da comunidade](docs/COMMUNITY_BENCHMARKS.md).
| Modelo | As respostas aparecem a | Lê o seu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2.650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2.090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1.750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1.620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2.180 tokens/s |
Um token equivale a cerca de ¾ de palavra, então 60 tokens por segundo é mais rápido do que você lê. "Lê o seu prompt" é a velocidade com que o Strata
absorve um documento longo, um código ou um histórico de conversa.
Medido numa placa AMD de 16 GB (RX 9070 XT, Ryzen 9 3900X, 47 GB de RAM):
| Modelo | As respostas aparecem a | Lê o seu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1.160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1.110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1.420 tokens/s |
As barras usam a mesma escala nas duas tabelas. Uma placa com mais VRAM é mais rápida: uma RTX 3090 (24 GB) deve escrever cerca de 100 a 140
tokens por segundo. A versão do mecanismo por trás de cada número está em [DETAILS.md](docs/DETAILS.md#speed-measured).
## Como usar
### No app
O app tem um **Chat**, um **Monitor** do modelo e da sua GPU, CPU e RAM, uma página de **Modelos** e as **Configurações**. O chat
pode trabalhar numa pasta do seu PC. No modo **Manual**, ele pergunta antes de alterar um arquivo ou executar um comando. Digite
`/loop 10m <o que fazer>` para repetir uma mensagem a cada 10 minutos enquanto o chat estiver aberto.
O chat pode pesquisar na web com o DuckDuckGo (ou o Bing), sem conta. Você pode adicionar uma chave de busca em **Configurações, Ferramentas**.
A busca é o único recurso que envia algo para fora do seu PC: os termos da pesquisa.
Para rodar dois modelos, ative **Executar vários modelos** e dê a cada modelo a sua própria placa da mesma marca.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Um chat no app Strata: o agente escreveu um pequeno jardim de pagode em voxels em um único arquivo HTML, e a página abre na pré-visualização ao lado da conversa</sub></p>
### No Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> enquanto o modelo responde: velocidade, GPU, memória e as solicitações recentes</sub></p>
```sh
strata launch # telas guiadas: modelo, contexto, imagens, acesso e depois servir
strata launch --model q2_0 --yes --no-tui # sem perguntas: serve, mostra o endereço e volta
strata launch --plan --model q2_0 # mostra o que aconteceria; nada muda
strata launch --status # o que está sendo servido agora
strata launch --stop # para o serviço
strata monitor --url http://127.0.0.1:8080 # acompanha velocidade, GPU, memória e requisições
strata update --check # há um mecanismo novo? (strata update instala)
strata doctor # confere os arquivos do mecanismo instalada
```
`strata --help` lista tudo. Iniciar e servir nunca baixam nada. Só a instalação, o `strata launch` (quando você
escolhe um modelo) e o `strata update` baixam. Para acessar o Strata do celular ou de outro PC, use
`strata launch --access lan`. Ele cria uma chave de API e não inicia numa rede sem uma. Nenhum navegador abre, a menos que
você adicione `--open`. Guia: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Cliente completo: [docs/TERMINAL.md](docs/TERMINAL.md).
### Com agentes de IA
Um agente instala o Strata com o comando de uma linha da Opção B e executa
`strata launch --model NAME --yes --no-tui`. Os passos, as flags e os códigos de saída estão em [docs/AI_SETUP.md](docs/AI_SETUP.md),
e as regras do repositório estão em [AGENTS.md](AGENTS.md).
### Conecte os seus outros aplicativos
| Tipo de aplicativo | URL base |
| --- | --- |
| Compatível com OpenAI (a maioria dos apps e agentes) | `http://127.0.0.1:8080/v1` (qualquer chave de API, qualquer nome de modelo) |
| API da Anthropic (Claude Code e outros) | `http://127.0.0.1:8080`, sem `/v1`. Esses aplicativos adicionam `/v1/messages` sozinhos. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([configuração](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` mostra as configurações exatas para OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent e OpenClaw.
`strata connect opencode --write` grava o arquivo depois de perguntar e mantém um backup.
### Imagens
As imagens vêm ativadas nos modelos que sabem lê-las. Anexe ou cole uma imagem no chat, ou envie pelo seu aplicativo.
O leitor de imagens roda no processador quando a sua placa de vídeo tem menos de 24 GB (alguns segundos por imagem, sem tirar nada
da memória da placa) e roda na placa de vídeo a partir de 24 GB. A regra é a mesma para NVIDIA e AMD. Desative as imagens
por modelo, ou escolha o leitor com `--vision off`, `--vision gpu` ou `--vision cpu`.
Os números por trás da regra: numa Radeon AI PRO R9700, o leitor da placa levou 57 ms para uma imagem de 260 tokens e o do
processador levou 3,1 s. Em placas de 16 GB (Tesla P100 e RTX A4000, Linux), o leitor da placa custou de 3 a 5% da velocidade do texto,
porque ocupa cerca de 1,4 GB que o modelo usaria de outra forma. No Mac, a GPU faz a leitura, seja qual for a memória (0,45 s para uma imagem de 260 tokens em um M4 de 32 GB, cerca de 1,2 GB da memória compartilhada);
o processador leva cerca de 50 s por imagem, então só lê com `--vision cpu`.
### Bom saber
- Escolha o nível de raciocínio (**off, low, medium ou high**) no menu do chat ou no "reasoning effort" do seu aplicativo. Off é o mais rápido.
- Por padrão, o Strata responde a um pedido por vez. Para vários ao mesmo tempo, use `--parallel N` no `strata launch` ou
`"parallel": 2` na configuração ([BATCHING.md](docs/BATCHING.md)). Numa placa de 12 GB, isso deixa cada resposta mais lenta.
- O Strata lê a primeira mensagem de uma conversa por inteiro, cerca de 1 minuto a cada 30.000 tokens. As mensagens seguintes começam em segundos.
- Onde ficam as suas conversas e a API completa: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Compilar a partir do código-fonte
<details>
<summary>Windows, Linux e Mac a partir do repositório</summary>
[Baixe o Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) e descompacte, ou use `git clone`.
**Windows:** dê dois cliques em `START-HERE.bat`. **Linux:** execute `./setup.sh` na pasta do Strata.
**Mac (Apple Silicon, macOS 15 ou mais novo):** instale uma vez as Command Line Tools da Apple com `xcode-select --install`
(o Xcode em si não é necessário). Abra o Terminal na pasta do Strata e execute `./setup.sh`. Ele compila o mecanismo do Mac no seu
Mac (cerca de 7 minutos), faz as mesmas perguntas, baixa o modelo e inicia. Da próxima vez, `./setup.sh` inicia
na hora. Qual tamanho cabe no seu Mac: [docs/MAC.md](docs/MAC.md).
O instalador encontra a sua placa e configura o mecanismo certo. Ele pergunta qual modelo e tamanho, quanto contexto (quanto texto
o modelo mantém em mente) e se deve ler imagens. Pressione Enter para aceitar a resposta recomendada. Depois ele baixa o
modelo (cerca de 70 GB) e inicia. Se o download parar, execute de novo e ele continua. O navegador abre a página do Strata em
`http://127.0.0.1:8080`.
> Enquanto o modelo inicia, o seu PC pode ficar lento ou parar de responder por 1 a 3 minutos (mais tempo na primeira vez). O Strata carrega
> de 35 a 55 GB na sua RAM e trava parte dela para a placa de vídeo. Espere e não feche a janela.
Da próxima vez, execute `START-HERE.bat` (ou `./setup.sh`) de novo. Ele inicia na hora e não baixa nada duas vezes. Feche a janela
dele para parar o modelo. `UPDATE.bat` (`./update.sh`) atualiza o Strata sem iniciá-lo. Atualização, Docker, várias placas,
onde ficam os arquivos e todas as opções: [docs/INSTALL.md](docs/INSTALL.md).
Para medir o seu PC, execute `BENCH.bat` (`./bench.sh`). Leva de 5 a 10 minutos e gera um relatório que você pode compartilhar
([como](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Opcional: mais velocidade numa placa quando o modelo inteiro cabe na RAM.** Duas configurações opcionais do RubenTorney (#1904, #1905).
Elas ajudam quando você usa uma placa de vídeo e a sua RAM comporta todos os experts, por exemplo Q2_0 ou IQ3_XXS num PC de 64 GB. Abra
`strata-<model>.json` na pasta de dados do Strata ([onde fica](docs/INSTALL.md#where-things-are-stored)) e adicione:
- `"--adapt-async", "1"` à lista `"args"`. O cache de experts passa a ser atualizado numa thread auxiliar, em vez de pausar a
resposta. Medido numa RTX 5070 (Windows, Q2_0): +4 a 5% na geração, +7 a 10% em código e documentos longos. As respostas podem diferir
um pouco do padrão e continuam iguais de uma execução para outra.
- `"STRATA_PIN_BY_NEED": "1"` no bloco `"env"` (Windows). O Windows só consegue travar cerca de metade da RAM para a placa
de vídeo, e isso trava primeiro os experts de que a placa precisa. Medido na RTX 5070: +4,4% na leitura do prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Reinicie o Strata depois da alteração. Para desfazer, remova as duas entradas.
Siga as instruções para baixar e iniciar um modelo. [Guia do Terminal](docs/TERMINAL_LAUNCH.md).
</details>
## Como funciona?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplicativo de Terminal do Strata mostrando a atividade do modelo, a velocidade, a memória e as solicitações recentes">
<br><sub>Acompanhe seu modelo e seu hardware enquanto ele roda.</sub>
</p>
Modelos como este costumam rodar em servidores com centenas de gigabytes de memória de vídeo. A sua placa de vídeo tem
12 a 24 GB. O Strata faz o modelo caber dividindo o trabalho entre o PC inteiro. Imagine uma biblioteca.
<a id="build-from-source"></a>
- O modelo é uma equipe de 24.576 pequenos especialistas ("experts"), como 24.576 livros de consulta. Cada palavra precisa de apenas 10 deles.
- **A sua placa de vídeo é a mesa.** Ela é pequena e rápida. Os cerca de 4.000 livros mais usados ficam abertos nela.
- **A sua RAM são as estantes da sala de leitura.** Os 24.576 livros estão lá. Quando uma palavra precisa de um livro que não está na
mesa, o seu processador lê da estante na hora, enquanto a placa continua trabalhando.
- **O seu SSD é o arquivo.** Ele guarda uma tabela de consulta de 29 GB. O modelo lê algumas linhas dela por palavra, então ela não
precisa ficar na RAM.
[Compilar a partir do código-fonte](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Guia de instalação completo](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
Um pequeno ajudante também adivinha as próximas palavras. O modelo grande confere todas de uma vez e fica com as certas. Você recebe
a mesma resposta, de 1,6 a 1,8 vez mais rápido. Textos longos são lidos em blocos grandes, de até 8.192 tokens por vez, a mais de 1.000 tokens
por segundo.
## Hardware
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| Hardware compatível | Guia |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [Instalação para NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Instalação para AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimental** | [Instalação para RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Instalação para Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 ou mais novo** | [Instalação para Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Instalação para Strix Halo](docs/STRIX_HALO.md) |
A explicação mais longa: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte e seus números:
[os detalhes](docs/DETAILS.md#how-it-works) e o [artigo](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Algo deu errado?
## Ferramentas
- **Meu PC travou na primeira vez que o Strata iniciou.** Ele carrega o modelo na RAM. Espere e não feche a janela. Ainda
travado depois de 10 minutos? Reinicie o PC, feche outros programas e tente de novo, ou escolha um tamanho menor.
- **Parou durante o download ou a instalação.** Abra o app ou o `strata launch` de novo, ou execute `START-HERE.bat` (ou
`./setup.sh`) outra vez. Ele continua de onde parou.
- **Está muito lento, a luz do disco não para de piscar, ou aparece "o mecanismo parou inesperadamente".** O seu PC não tem
RAM livre suficiente. Feche outros programas (os navegadores usam muita), ou escolha um tamanho menor, como Q2_0 ou IQ2_XS.
- **Diz que não há mecanismo para este PC.** No app, abra **Configurações, Mecanismo** e clique em **Atualizar mecanismo**. No
Terminal, execute `strata update`. O `strata doctor` mostra quais arquivos do mecanismo estão instalados.
- **Diz que a porta 8080 já está em uso.** O Strata já está rodando. Procure a janela dele.
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, e outras, pelas APIs compatíveis com OpenAI e Anthropic.
Mais problemas e soluções: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Ainda sem resolver? Abra uma
[issue](https://github.com/Niko1221/Strata/issues). No app, anexe o arquivo de **Configurações, Mecanismo, Diagnóstico, Ver
registros, Salvar logs**. No Terminal, cole a saída de `strata doctor`. Se estiver rodando pelo código-fonte, anexe o `strata-<model>.log` da pasta do
Strata. Para um problema de segurança, informe em particular: [SECURITY.md](SECURITY.md).
Execute **`strata connect`** para ver as instruções da sua ferramenta. [Guia de conexão](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
## Créditos e licença
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
O modelo é o [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), da equipe Qwen. Ele foi comprimido por
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
e Unsloth. O Strata usa partes do [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Todos os créditos:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). O Strata é de código aberto sob a [Licença MIT](LICENSE). Algumas partes e
todos os modelos têm licenças próprias ([quais](docs/HOW_IT_WORKS.md#license)).
## Ajuda
## Contribuindo
Encontrou um bug, ou quer enviar uma alteração ou um benchmark? Leia primeiro o [CONTRIBUTING.md](CONTRIBUTING.md). Ele diz o que
incluir para que outras pessoas possam conferir o seu trabalho.
[Todos os guias](docs) · [Solução de problemas](docs/TROUBLESHOOTING.md) · [Relatar um bug](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [Como funciona](docs/HOW_IT_WORKS.md) · [Contribuir](CONTRIBUTING.md)
## Apoie o Strata
O Strata é gratuito e tem licença MIT. Eu o desenvolvo e testo em todo hardware que consigo arranjar. Se ele roda bem no seu PC, um café me ajuda
a continuar testando placas novas e corrigindo o que você relatar. Sem pressão. Usar e relatar o que encontrar ajuda tanto quanto.
**Ajude a manter a IA local aberta para todos.** Um café ajuda a financiar o desenvolvimento e os testes em mais hardware.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: apoie o Strata" height="48"></a>
Quer patrocinar o Strata? Fale comigo pelo [GitHub](https://github.com/Niko1221).
[Licença MIT](LICENSE) · [Créditos e licenças dos modelos](docs/HOW_IT_WORKS.md#credits) · [Privacidade](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Para ferramentas de IA e leitores automáticos: <a href="AGENTS.md">referência técnica</a> · <a href="docs/AI_SETUP.md">instruções de instalação</a>.</sub>
**Patrocínios, parcerias de hardware ou colaborações:** [[email protected]](mailto:[email protected]).
+121 -314
View File
@@ -1,381 +1,188 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<p align="center"><a href="README.md">English</a> · <b>简体中文</b> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<h3 align="center">在你自己的游戏电脑上运行最先进的模型</h3>
<p align="center">支持 NVIDIA、AMD、Intel · Windows、Linux、Mac · 免费开源</p>
<h3 align="center">在你自己的电脑上运行强大的 AI。</h3>
<p align="center">本地聊天、写代码、处理图片。</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
<a href="#模型">模型</a> · <a href="#安装"><b>安装</b></a> ·
<a href="#硬件">硬件</a> · <a href="#工具">工具</a> ·
<a href="#帮助">帮助</a> · <a href="#支持-strata">支持</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>一座体素宝塔花园,由一条提示词一次生成,在装有 RTX 5070 的电脑上用 Strata 运行(IQ3_S,128K 上下文) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">完整视频(49 秒)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="下载 Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT 许可证"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="在 Buy Me a Coffee 上支持 Strata"></a>
</p>
Strata 运行 **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**。这是一个 1250 亿参数的模型,
通常需要服务器才能跑,现在用普通电脑就行。它能聊天、写代码、看图片,也能配合你的应用和编程智能体使用。
你的聊天记录和文件都留在你自己的电脑上。
<p align="center"><sub><a href="README.md">English</a> · <b>简体中文</b> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
**第一次用?** 先看[硬件要求](#硬件要求),再[安装应用](#方式-astrata-应用)。
**免费、开源,由你自己运行。** 在桌面上使用 Strata,或搭配你喜欢的编程工具。无需账号。
- **游戏电脑就能跑。** 内存和显存加起来有 32 GB 就可以开始。
- **能接入你的工具。** 支持 OpenAI 和 Anthropic 的 API,Claude Code、Codex CLI、OpenCode 等智能体都能连上。
- **能看图片。** 默认开启。
- **免费。** MIT 许可证,不用注册账号,不走云端。
- **隐私。** 没有遥测。Strata 下载什么、什么时候联网、如何校验下载内容,都写在 [docs/SAFETY.md](docs/SAFETY.md)。
## 模型
版本 0.2.2 · 引擎 0.1.47
我们**首先推荐 ISTA-DASLab**,其次是 **Unsloth** 和其他提供方,前提是有内存放得下的兼容模型。
## 硬件要求
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
**内存和显存(VRAM)加起来至少要有 32 GB。** Mac 和 Ryzen AI Max 上指的是统一内存。内存越大,能跑的模型越大,
或者速度越快。
- 32 GB 内存加一张 12-16 GB 显存的显卡,可以跑 Coder。
- 64 GB 内存可以跑所有尺寸。
| 你的电脑 | 说明 |
| 提供方 | 可选项 |
| --- | --- |
| **NVIDIA** GeForce RTX 20、30、40、50 系列(12 GB 及以上) | 安装应用或终端版即可。 |
| **AMD** Radeon RX 6800 及更新型号、Radeon AI PRO R9700(12 GB 及以上) | [AMD 显卡](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md)、[Intel 引擎](docs/INTEL.md) |
| **Apple 芯片 Mac**(M1 及更新,macOS 15 及以上) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max(Strix Halo)** 及其他统一内存系统 | [Strix Halo](docs/STRIX_HALO.md) |
| 较旧的显卡和处理器 | [旧显卡](docs/OLDER_GPUS.md)、[旧处理器](docs/INSTALL.md#older-cpus-experimental) |
| **ISTA-DASLab · 推荐** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S。[Coder:IQ1\_M](docs/MODELS.md#coder),适合写代码。 |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)。占用内存更少,但速度更慢。 |
| **其他选项** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
另外还需要:
[比较大小、内存需求和兼容性](docs/MODELS.md)。
- 约 80 GB 的可用磁盘空间(用固态硬盘,首次启动会快很多)
- Windows 10 / 11、Linux 或 macOS
- 较新的显卡驱动
</details>
同一品牌的两三张显卡可以共同运行一个模型([多显卡](docs/MULTI_GPU.md))。Strata 不会把一个模型拆到 NVIDIA 和 AMD
显卡上。完整列表见 [docs/INSTALL.md](docs/INSTALL.md#what-you-need)。
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0。这些是目前文档中收录的该模型的版本。
[模型指南和兼容性](docs/QWEN36.md)。
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
可在**桌面应用**和**终端**中使用。内存放不下的部分会从 SSD 读取,所以需要大量空闲磁盘空间。
| 版本(IST-DAS) | 下载大小 | 所需空闲磁盘 |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | 约245 GB |
| 2.00 bpw | 257 GB | 约265 GB |
| 2.25 bpw | 274 GB | 约280 GB |
支持 NVIDIA RTX 30、40 和 50 系列(Windows 和 Linux),以及 AMD Radeon AI PRO R9700(Linux)。安装程序会下载模型文件和 DeepSeek 引擎,然后启动模型。
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
以下为 Strata 0.2.5 上 2.00 bpw 版本的实测结果。“输出”是回答的速度,“提示词”是 Strata 读取你输入内容的速度。
| 硬件 | 输出 | 提示词 |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB 内存 | 4 到 6 tokens/s | 约 110 tokens/s(4K) |
| RTX 3060 12 GB, 46 GB 内存 | 9 到 11 tokens/s | 约 155 tokens/s(4K) |
| 1x Radeon AI PRO R9700, 251 GB 内存 | 21 到 25 tokens/s | 320 到 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 到 77 tokens/s | 400 到 450 tokens/s (8K) |
[内存、磁盘和详细信息](docs/DEEPSEEK41.md) · [发行说明](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
不确定哪个放得下?[为你的电脑选择模型](docs/MODELS.md#pick-by-memory)。
## 安装
三种方式任选一种。它们启动的是同一个引擎,用的是同样的模型。
<a id="option-a-the-strata-app"></a>
### 方式 A:Strata 应用
### 桌面应用
最简单的方式。应用内置引擎:安装、选模型、开始聊天。请根据你的系统和显卡,从
[发布页](https://github.com/Niko1221/Strata/releases/latest)下载对应的文件。不确定选哪个?看表格下面那一段。
**[下载适用于 Windows、Linux 或 Mac 的 Strata →](https://github.com/Niko1221/Strata/releases/latest)**
| 系统 | 你的显卡 | 要下载的文件 |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX,驱动 580 或更新 | [`Strata-0.2.2-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda13.exe) |
| | NVIDIA,较旧的显卡,或驱动低于 580 | [`Strata-0.2.2-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-cuda12.exe) |
| | AMD Radeon RX 9000 系列或 Radeon AI PRO R9700 | [`Strata-0.2.2-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip.exe) |
| | AMD Radeon RX 7000 系列或 Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX,驱动 580 或更新 | [`Strata-0.2.2-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda13.AppImage) |
| | NVIDIA,较旧的显卡,或驱动低于 580 | [`Strata-0.2.2-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 系列或 Radeon AI PRO R9700 | [`Strata-0.2.2-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 系列 | [`Strata-0.2.2-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.2-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-linux-amd-strixhalo.AppImage) |
| **Mac**(Apple 芯片,macOS 15 及以上) | 内置 | [`Strata-0.2.2-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.2-mac.dmg) |
安装应用,选择模型,就可以开始聊天。每个系统只有一个安装程序。首次启动时,应用会下载适合你显卡的引擎。[安装程序帮助](docs/INSTALL.md#the-app)。
**不确定时:** NVIDIA 显卡的驱动版本在 580 及以上就选 `cuda13`,否则选 `cuda12`。`cuda12` 适用于较旧的显卡和驱动。
`hip` 和 `amd` 的文件是给 AMD 显卡用的。其他 AMD 显卡(RX 6000)目前还没有现成的应用,
请看 [AMD 显卡](docs/AMD_HIP.md)。
<sub>文件: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
然后:
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Strata 桌面应用,聊天和实时代码预览并排显示">
<br><sub>聊天、文件和实时预览都在一个窗口里。</sub>
</p>
- **Windows:** 运行 `.exe`。它只为当前用户安装,不需要管理员权限。如果 Windows 提示“Windows 已保护你的电脑”,
点击**更多信息**,再点**仍要运行**(安装程序还没有代码签名)。
- **Linux:** 给文件加上可执行权限(`chmod +x`),然后启动。需要 FUSE 2:`sudo apt install libfuse2`(Ubuntu 22.04),
或 `libfuse2t64`(Ubuntu 24.04 及更新)。
- **Mac:** 打开 `.dmg`,把 Strata 拖到“应用程序”。
<a id="option-b-the-terminal-one-line"></a>
第一个界面里,应用会检测你的电脑,并推荐合适的模型。点击安装即可。模型很大,要下载约 40-70 GB,中断后会从断点继续。
关闭窗口会同时停止 Strata 和引擎。
### 终端应用
### 方式 B:终端(一行命令)
更喜欢终端?安装 Strata,然后运行 **`strata launch`**。
终端版就是 `strata` 命令,适合脚本、服务器和 AI 智能体。
<details open>
<summary><b>安装命令</b></summary>
Linux 和 Mac:
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows(PowerShell):
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
脚本会检查你的电脑,下载终端版,校验 SHA-256,然后解压到你的用户目录。不需要管理员权限。在 Windows 上,装完后请
打开一个新的终端窗口。然后启动 Strata:
在 Windows 上,安装后请打开一个新的终端。然后运行:
```sh
strata launch
```
它会一次问一个问题:模型、上下文、图片、允许谁连接。接着为你的显卡下载引擎(已签名,使用前会先试启动),再下载
模型,并开始提供服务。在脚本和 AI 智能体里,请使用参数:
```sh
strata launch --model q2_0 --yes --no-tui
```
如果想在运行前先看看安装脚本,先用 `curl -fsSL <address> -o install.sh` 下载,读完后再用 `sh install.sh` 运行。
所有界面和参数见 [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md)。
### 方式 C:让你的 AI 来安装
把下面这段话粘贴到 Claude Code、Cursor、Codex、GitHub Copilot 或其他编程助手里:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
它会检查你的显卡、内存和磁盘,选出合适的模型,完成安装和启动,并告诉你怎么连接你的应用。AI 工具也可以通过 Strata 的
[MCP 服务器](docs/MCP_SERVER.md)安装、启动和停止 Strata。
想自己动手编译?请看[从源码构建](#从源码构建)。
## 模型
应用和 `strata launch` 会按你的内存推荐模型。它是同一个模型的几种尺寸,压缩程度不同。尺寸小的更快,尺寸大的稍微更聪明一点。
### 内存不够?试试 Gyro
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)(作者 agentionai)是运行这个模型占用最小的方式。
| 尺寸 | 所需内存(RAM + VRAM) | 下载大小 | 说明 |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27.6 GB | 58.5 GB | 给其他尺寸都放不下的电脑。质量低于 IQ2_XS。 |
| **Gyro-M** | 35 GB | 92.0 GB | 质量约相当于 IQ3_XXS,显存占用少约 20%。输出回答比 Q2_0 慢,12 GB 的 RTX 3060 上慢 16%,R9700 上慢 21%。 |
Gyro 可在 NVIDIA 和 AMD 显卡(AMD 为实验性支持)以及 Linux 上的 Intel Arc 显卡(已在 Arc A750 上测试)上运行,自 0.2.2 起也可在 Apple Silicon 的 Mac 上运行(32 GB 的 Mac mini 上 Gyro-S 为 9 token/秒,专家权重从 SSD 读取)。详情和测试数据:
[docs/GYRO.md](docs/GYRO.md)。用 `strata launch --model gyro-s` 启动。
### 其他尺寸
| 你的内存(RAM + VRAM) | 选择 | 备注 |
| --- | --- | --- |
| **32 GB** | **Gyro-S**,写代码可选 **[Coder](docs/MODELS.md#coder)** | Coder 需要 32 GB 内存,只保留一半的专家,所以代码以外的能力较弱 |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS(39.2 GB),或 **Gyro-M**(35 GB) | Swift 回答前思考的时间更短。Gyro-M 的质量接近 IQ3_XXS,但输出回答比 Q2_0 慢 16-21% |
| **48 GB** | **IQ3_XXS**(47.0 GB) | 更慢,更好 |
| **64 GB** | **IQ3_S**(54.8 GB),原版或 Swift 1.5 | 最慢,最好:在公开测试中与完整模型持平 |
| **96 GB 及以上** | Unsloth 的 **UD-IQ4_XS**(约 72 GB 内存)到 **Q8_0**(约 136 GB),4 到 8 位 | 更接近完整模型。内存不足时,Strata 会从固态硬盘读取一部分,速度会慢很多 |
内存比这一行写的少,也可能能用:Strata 会自动切换到[低内存模式](docs/MODELS.md#a-big-graphics-card-and-little-ram)。
速度会变慢,显卡较小时会慢得多。
更多选择:
- **[Swift 1.5](docs/MODELS.md#swift-15)** 是微调版本,回答前思考的时间更短,质量和内存占用大致相同。
- **[NVFP4](docs/NVFP4.md)** 是 NVIDIA 为 NVIDIA 显卡提供的 4 位版本。要把所有专家放进内存,需要约 80 GB 内存。
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** 是另一个更小的模型,能看图片,还能同时回答多个请求。
- **[OrcaRouter 的 Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** 需要手动设置。
所有尺寸、下载大小,以及各种配置能跑什么:[docs/MODELS.md](docs/MODELS.md)。
## 速度如何?
以下数据基于 12 GB 显卡(RTX 5070、Ryzen 5 7600、64 GB 内存)。其他显卡的结果见:
[各模型的速度](docs/MODELS.md#how-fast-is-each-size)、[社区测试结果](docs/COMMUNITY_BENCHMARKS.md)。
| 模型 | 回答输出速度 | 读取提示词速度 |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |
一个 token 约等于 ¾ 个英文单词,所以每秒 60 个 token 比你阅读的速度还快。“读取提示词”指 Strata 吸收长文档、代码或聊天历史的速度。
在 16 GB 的 AMD 显卡上测得的结果(RX 9070 XT、Ryzen 9 3900X、47 GB 内存):
| 模型 | 回答输出速度 | 读取提示词速度 |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |
两张表的进度条使用同一个比例。显存更大的显卡更快:RTX 3090(24 GB)每秒大约能输出 100-140 个 token。每个数字对应的引擎版本见
[DETAILS.md](docs/DETAILS.md#speed-measured)。
## 使用方法
### 在应用里
应用包含**聊天**、显示模型以及 GPU、CPU 和内存状态的**监控**、**模型**页面和**设置**。聊天可以在你电脑上的某个文件夹里工作。
在**手动**模式下,修改文件或运行命令之前它会先询问你。输入 `/loop 10m <要做的事>`,在聊天保持打开期间每 10 分钟重复发送一次消息。
聊天可以用 DuckDuckGo(或 Bing)搜索网页,不需要账号。你也可以在**设置,工具**里添加搜索密钥。搜索是唯一会把内容发出电脑的功能,
发出的就是搜索词。
要同时运行两个模型,请打开**运行多个模型**,并为每个模型分配同一品牌的单独一张显卡。
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Strata 应用中的一次对话:智能体用一个 HTML 文件写出了一个小型体素宝塔花园,页面在对话旁的预览中打开</sub></p>
### 在终端里
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub>模型回复时的 <code>strata monitor</code>:速度、GPU、内存和最近的请求</sub></p>
```sh
strata launch # 引导界面:模型、上下文、图片、访问方式,然后开始服务
strata launch --model q2_0 --yes --no-tui # 不提问:开始服务,打印地址,然后返回
strata launch --plan --model q2_0 # 显示将要执行的操作;不做任何更改
strata launch --status # 当前正在提供什么服务
strata launch --stop # 停止服务
strata monitor --url http://127.0.0.1:8080 # 查看速度、GPU、内存和请求
strata update --check # 有新引擎吗?(strata update 会安装它)
strata doctor # 检查已安装的引擎文件
```
`strata --help` 会列出所有命令。启动和提供服务时不会下载任何东西。只有安装、`strata launch`(在你选择模型时)和
`strata update` 会下载。要从手机或另一台电脑访问 Strata,请使用 `strata launch --access lan`。它会生成一个 API 密钥,
没有密钥就不会在网络上启动。除非你加上 `--open`,否则不会打开浏览器。指南:[docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md)。
完整的客户端说明:[docs/TERMINAL.md](docs/TERMINAL.md)。
### 配合 AI 智能体
智能体用方式 B 中的一行命令安装 Strata,然后运行 `strata launch --model NAME --yes --no-tui`。步骤、参数和退出码见
[docs/AI_SETUP.md](docs/AI_SETUP.md),仓库规则见 [AGENTS.md](AGENTS.md)。
### 连接你的其他应用
| 应用类型 | 基础 URL |
| --- | --- |
| 兼容 OpenAI(大多数应用和智能体) | `http://127.0.0.1:8080/v1`(API 密钥和模型名称随便填) |
| Anthropic API(Claude Code 等) | `http://127.0.0.1:8080`,不带 `/v1`。这些应用会自己加上 `/v1/messages`。Claude Code:`ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API(Codex CLI) | `http://127.0.0.1:8080/v1/responses`([设置方法](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` 会打印 OpenCode、Codex CLI、Claude Code、oh-my-pi、pi、Hermes Agent 和 OpenClaw 的具体设置。
`strata connect opencode --write` 会在询问后写入配置文件,并保留备份。
### 图片
能看图片的模型默认开启图片功能。在聊天里附加或粘贴图片,或者从你的应用发送图片。显卡的显存小于 24 GB 时,读图由处理器完成
(每张图片几秒钟,不占用显卡的显存);显存达到 24 GB 时,由显卡完成。NVIDIA 和 AMD 的规则相同。你可以按模型关闭图片功能,
也可以用 `--vision off`、`--vision gpu` 或 `--vision cpu` 选择读图方式。
这条规则背后的数据:在 Radeon AI PRO R9700 上,显卡读一张 260 token 的图片用了 57 ms,处理器用了 3.1 秒。在 16 GB 的显卡
(Tesla P100 和 RTX A4000,Linux)上,用显卡读图会让文字速度下降 3-5%,因为它占用约 1.4 GB 显存,否则这部分会被模型使用。
在 Mac 上,无论内存多大,都由 GPU 读图(32 GB 的 M4 上读一张 260 token 的图片用 0.45 秒,占用约 1.2 GB 共享内存);处理器读一张约需 50 秒,所以只有指定 `--vision cpu` 时才用。
### 使用须知
- 在聊天菜单或你的应用的“reasoning effort”里选择思考级别(**off、low、medium 或 high**)。off 最快。
- Strata 默认一次回答一个请求。要同时处理多个,请在 `strata launch` 中使用 `--parallel N`,或在配置里写
`"parallel": 2`([BATCHING.md](docs/BATCHING.md))。在 12 GB 显卡上,这会让每个回答变慢。
- Strata 会完整读取聊天的第一条消息,每 30,000 个 token 约需 1 分钟。之后的消息几秒钟就开始。
- 聊天记录存放的位置和完整的 API:[docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored)、[docs/DETAILS.md](docs/DETAILS.md#using-it)。
## 从源码构建
<details>
<summary>从仓库构建 Windows、Linux 和 Mac 版本</summary>
[下载 Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)并解压,或者用 `git clone` 获取。
**Windows:** 双击 `START-HERE.bat`。**Linux:** 在 Strata 文件夹里运行 `./setup.sh`。
**Mac(Apple 芯片,macOS 15 及以上):** 先用 `xcode-select --install` 安装一次 Apple 的命令行工具(不需要完整的 Xcode)。
在 Strata 文件夹里打开终端,运行 `./setup.sh`。它会在你的 Mac 上构建 Mac 引擎(约 7 分钟),问同样的问题,下载模型并启动。
下次运行 `./setup.sh` 会直接启动。你的 Mac 适合哪个尺寸:[docs/MAC.md](docs/MAC.md)。
安装程序会找到你的显卡,并设置合适的引擎。它会问你用哪个模型和尺寸、上下文多长(模型能记住多少文字),以及是否要读取图片。
直接按回车就是推荐的选择。然后它会下载模型(约 70 GB)并启动。如果下载中断,再运行一次就会继续。浏览器会打开
`http://127.0.0.1:8080` 上的 Strata 页面。
> 模型启动期间,你的电脑可能会变慢,或者在 1-3 分钟内没有响应(第一次最久)。Strata 会把 35-55 GB 加载到内存里,并锁定其中一部分
> 给显卡使用。请等待,不要关闭窗口。
下次再运行 `START-HERE.bat`(或 `./setup.sh`)会立即启动,不会重复下载。关闭它的窗口就能停止模型。`UPDATE.bat`
(`./update.sh`)只更新 Strata,不启动。更新、Docker、多显卡、文件存放位置和所有选项:[docs/INSTALL.md](docs/INSTALL.md)。
要测试你的电脑,运行 `BENCH.bat`(`./bench.sh`)。需要 5-10 分钟,并生成一份可以分享的报告
([方法](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark))。
**可选:整个模型都放得进内存时,让单卡更快。** 这是 RubenTorney 提供的两个需要手动开启的设置(#1904、#1905)。
当你只用一张显卡,并且内存装得下所有专家时,它们会有帮助,例如在 64 GB 的电脑上跑 Q2_0 或 IQ3_XXS。打开 Strata 数据文件夹里的
`strata-<model>.json`([位置](docs/INSTALL.md#where-things-are-stored)),然后添加:
- 在 `"args"` 列表中加入 `"--adapt-async", "1"`。这样专家缓存会在辅助线程上更新,不会让回答暂停。在 RTX 5070
(Windows,Q2_0)上测得:解码快 4-5%,代码和长文档快 7-10%。回答可能与默认设置略有不同,但每次运行的结果保持一致。
- 在 `"env"` 块中加入 `"STRATA_PIN_BY_NEED": "1"`(Windows)。Windows 只能为显卡锁定大约一半的内存,
这个设置会优先锁定显卡最先需要的专家。在 RTX 5070 上测得:读取提示词快 4.4%。
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
修改后请重启 Strata。要撤销,删除这两项即可。
按提示下载并启动模型。[终端指南](docs/TERMINAL_LAUNCH.md)。
</details>
## 工作原理
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Strata 终端应用,显示模型活动、速度、内存和最近的请求">
<br><sub>模型运行时,随时查看模型和硬件状态。</sub>
</p>
这类模型通常运行在有几百 GB 显存的服务器上。你的显卡只有 12-24 GB。Strata 把工作分给整台电脑,让模型装得下。
可以把它想成一座图书馆。
<a id="build-from-source"></a>
- 模型是一个由 24,576 位小专家(“experts”)组成的团队,好比 24,576 本参考书。每个词只需要其中 10 本。
- **你的显卡是书桌。** 它小而快。最常用的大约 4,000 本书摊开放在上面。
- **你的内存是阅览室的书架。** 24,576 本书都在这里。某个词需要一本不在书桌上的书时,处理器会立刻从书架上取来读,显卡则继续工作。
- **你的固态硬盘是档案室。** 里面存着一张 29 GB 的查找表。模型每个词只读其中几行,所以不用常驻内存。
[从源码构建](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [完整安装指南](docs/INSTALL.md)
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<a id="what-you-need"></a>
另外有一个小助手会猜接下来的几个词。大模型一次检查所有猜测,保留正确的部分。得到的答案不变,但快了 1.6-1.8 倍。
长文本按大块读取,一次最多 8,192 个 token,速度超过每秒 1,000 个 token。
## 硬件
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
| 支持的硬件 | 指南 |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 系列** | [NVIDIA 安装](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900、RX 9060 / 9070、AI PRO R9700** | [AMD 安装](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · 实验性** | [RX 6000 安装](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel 安装](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 或更新** | [Mac 安装](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo 安装](docs/STRIX_HALO.md) |
更详细的解释:[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md)。每个部分和它的数据:
[详细说明](docs/DETAILS.md#how-it-works)和[论文](docs/paper/Strata-Paper.pdf)。
<a id="using-it"></a>
## 出问题了?
## 工具
- **第一次启动 Strata 时我的电脑卡住了。** 它在把模型加载到内存。请等待,不要关闭窗口。10 分钟后还是卡住?重启电脑,
关闭其他程序后再试,或者选一个更小的尺寸。
- **下载或安装时中断了。** 重新启动应用或 `strata launch`,或者再运行一次 `START-HERE.bat`(或 `./setup.sh`)。
它会从中断的地方继续。
- **非常慢、硬盘灯一直闪,或者提示“引擎意外停止”。** 你的电脑可用内存不够。关闭其他程序(浏览器很占内存),
或者选一个更小的尺寸,比如 Q2_0 或 IQ2_XS。
- **提示这台电脑没有可用的引擎。** 在应用里打开**设置,引擎**,点击**更新引擎**。在终端里运行 `strata update`。
`strata doctor` 会显示已安装哪些引擎文件。
- **提示 8080 端口已被占用。** Strata 已经在运行了。找一下它的窗口。
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**,也可以通过兼容 OpenAI 和 Anthropic 的 API 使用更多工具。
更多问题和解决办法:[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)。还是解决不了?请提交
[issue](https://github.com/Niko1221/Strata/issues)。在应用里,附上**设置,引擎,诊断,查看日志,保存日志**得到的文件。
在终端里,粘贴 `strata doctor` 的输出。从源码运行时,附上 Strata 文件夹里的 `strata-<model>.log`。
如果是安全问题,请私下报告:[SECURITY.md](SECURITY.md)。
运行 **`strata connect`**,查看你所用工具的设置方法。[连接指南](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect)。
## 致谢与许可证
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
模型是 Qwen 团队的 [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)。压缩工作由
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)、agentionai(Gyro)
和 Unsloth 完成。Strata 使用了 [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) 的部分代码。全部致谢:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits)。Strata 以 [MIT 许可证](LICENSE)开源。少数部分和所有模型有各自的许可证
([具体是哪些](docs/HOW_IT_WORKS.md#license))。
## 帮助
## 参与贡献
发现了 bug,或者想提交修改或基准测试结果?请先阅读 [CONTRIBUTING.md](CONTRIBUTING.md)。其中说明了需要附上哪些内容,
方便其他人核实你的工作。
[全部指南](docs) · [故障排查](docs/TROUBLESHOOTING.md) · [报告问题](https://github.com/Niko1221/Strata/issues/new/choose) ·
[基准测试](docs/COMMUNITY_BENCHMARKS.md) · [工作原理](docs/HOW_IT_WORKS.md) · [参与贡献](CONTRIBUTING.md)
## 支持 Strata
Strata 免费,采用 MIT 许可证。我在能拿到的各种硬件上开发和测试它。如果它在你的电脑上跑得不错,请我喝杯咖啡,能帮我继续测试新显卡、
修复你们反馈的问题。不用有压力。使用它并反馈你的发现,同样很有帮助。
**帮助本地 AI 保持开放。** 一杯咖啡能帮助我们在更多硬件上开发和测试。
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee:支持 Strata" height="48"></a>
有兴趣赞助 Strata?请通过 [GitHub](https://github.com/Niko1221) 联系我。
[MIT 许可证](LICENSE) · [致谢和模型许可证](docs/HOW_IT_WORKS.md#credits) · [隐私](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>面向 AI 工具和自动阅读器: <a href="AGENTS.md">技术参考</a> · <a href="docs/AI_SETUP.md">安装说明</a></sub>
**赞助、硬件合作或其他合作:** [[email protected]](mailto:[email protected]).
+4 -3
View File
@@ -23,8 +23,9 @@ this PC only. The details and every setting are in [docs/DETAILS.md](docs/DETAIL
- **Where it listens.** `127.0.0.1` by default. `--host 0.0.0.0` (or `"host"` in `strata-<model>.json`) opens it to
your network, and the server then warns when no API key is set.
- **API key.** `"api_key"` in the run config (or `STRATA_API_KEY`) is required on `/v1/*` and on every endpoint
that shows the model's state, requests or answers (`/status`, `/metrics`, `/settings`, `/mcp`, `/props`, `/slots`,
`/api/requests`, `/config`) and on every `POST`. It is compared in constant time. Set one before you open the
that shows the model's state, requests or answers (`/status`, `/metrics`, `/metrics/prometheus`, `/settings`,
`/mcp`, `/props`, `/slots`, `/api/requests`, `/config`) and on every `POST`. It is compared in constant time.
Set one before you open the
server to your network or put a tunnel in front of it.
- **Host check (DNS rebinding, 0.1.38).** Without an API key the server answers only requests whose `Host` is a name
it knows (`localhost`, an IP address, the address it listens on and, when it listens beyond this PC, this PC's
@@ -74,7 +75,7 @@ the model. The section above is about the engine's HTTP server. This is what the
The Terminal installs with one line:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "strata-desktop",
"productName": "Strata",
"version": "0.2.2",
"version": "0.2.7",
"private": true,
"description": "Strata Desktop - local AI, tuned to one model.",
"scripts": {
+52
View File
@@ -0,0 +1,52 @@
/** Strata's output-budget correction for the staged pi-ai copy. */
import { createHash } from 'node:crypto';
import { lstatSync, readFileSync, realpathSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
export const HARNESS_BUDGET_PATCH = Object.freeze({
id: 'strata-pi-ai-context-safety-v2',
package: '@earendil-works/pi-ai',
version: '1.1.0',
file: 'node_modules/@earendil-works/pi-ai/dist/api/simple-options.js',
beforeSha256: '9b3cc0ae6daa86fdc050bc0cb0f3f986401e64ccddbde6757183c9f8ad411be2',
});
const sha256 = bytes => createHash('sha256').update(bytes).digest('hex');
const oldClamp = 'const available = model.contextWindow - estimateContextTokens(context).tokens - CONTEXT_SAFETY_TOKENS;';
// Small windows also need room after tool results; a fixed margin must not
// consume all remaining estimated space. Keep the old 64K+ calculation.
const newClamp = `const remaining = model.contextWindow - estimateContextTokens(context).tokens;
const safety = model.contextWindow >= 65536 ? CONTEXT_SAFETY_TOKENS
: Math.min(CONTEXT_SAFETY_TOKENS, Math.floor(model.contextWindow / 16), Math.max(0, Math.floor(remaining / 2)));
const available = remaining - safety;`;
/**
* Patch only a flattened, disposable staged Harness. Returns the BUNDLE.json
* provenance entry; the caller records it when writing the bundle manifest.
* A repeat application or any unknown package/source fails before writing.
*/
export function patchStagedHarnessBudget(stagedRoot) {
const fail = message => { throw new Error(`Harness output-budget patch: ${message}`); };
const root = realpathSync(resolve(stagedRoot));
const packageDir = join(root, 'node_modules', '@earendil-works', 'pi-ai');
const target = join(root, HARNESS_BUDGET_PATCH.file);
let manifest, before;
try {
manifest = JSON.parse(readFileSync(join(packageDir, 'package.json'), 'utf8'));
// Flattening must have removed links; never write through a shared source file.
if (realpathSync(target) !== target || !lstatSync(target).isFile() || lstatSync(target).nlink !== 1)
fail('expected an unlinked staged module');
before = readFileSync(target);
} catch (error) {
fail(`cannot read a standalone staged pi-ai dependency (${error.message})`);
}
if (manifest?.name !== HARNESS_BUDGET_PATCH.package || manifest?.version !== HARNESS_BUDGET_PATCH.version)
fail('expected @earendil-works/pi-ai 1.1.0; review the patch before staging a different dependency');
if (sha256(before) !== HARNESS_BUDGET_PATCH.beforeSha256)
fail('unexpected simple-options.js SHA-256; review the dependency source before staging');
const source = before.toString('utf8');
if (source.split(oldClamp).length !== 2) fail('expected exactly one known context clamp');
const after = Buffer.from(source.replace(oldClamp, newClamp));
writeFileSync(target, after);
return { ...HARNESS_BUDGET_PATCH, afterSha256: sha256(after) };
}
+5
View File
@@ -19,6 +19,7 @@ import { join, relative, resolve, sep } from 'node:path';
import { createHash } from 'node:crypto';
import { flattenPnpm } from './flatten-pnpm.mjs';
import { isOtherOsBinaryFolder, missingHarnessImports } from './harness-prune.mjs';
import { patchStagedHarnessBudget } from './harness-budget.mjs';
const args = process.argv.slice(2);
const opt = (name, fallback) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : fallback; };
@@ -144,6 +145,9 @@ function prune(dir) {
}
prune(out);
// Strata's provenance-recorded output-budget correction applies only to this copy.
const patches = [patchStagedHarnessBudget(out)];
// --- licence: the MIT text and the third-party notices travel with the files ---------------------------------------------
for (const name of ['LICENSE', 'THIRD_PARTY_NOTICES.md']) {
if (existsSync(join(src, name))) cpSync(join(src, name), join(out, name));
@@ -156,6 +160,7 @@ const total = size(out);
writeFileSync(join(out, 'BUNDLE.json'), JSON.stringify({
component: 'deepseek-harness', version: cli.version, commit, license: 'MIT', platform, node: process.version,
bytes: total.bytes, files: total.files, entry: 'apps/cli/lib/bin.js', builtAt: new Date().toISOString(),
patches,
note: 'Pruned production install (pnpm deploy --prod). No package manager runs on the user\'s PC.',
}, null, 2));
+155 -1
View File
@@ -349,6 +349,12 @@ version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
[[package]]
name = "byteorder-lite"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f1fe948ff07f4bd06c30984e69f5b4899c516a3ef74f34df92a2df2ab535495"
[[package]]
name = "bytes"
version = "1.12.1"
@@ -1755,6 +1761,19 @@ dependencies = [
"icu_properties",
]
[[package]]
name = "image"
version = "0.25.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85ab80394333c02fe689eaf900ab500fbd0c2213da414687ebf995a65d5a6104"
dependencies = [
"bytemuck",
"byteorder-lite",
"moxcms",
"num-traits",
"png 0.18.1",
]
[[package]]
name = "indexmap"
version = "1.9.3"
@@ -2097,6 +2116,20 @@ version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "mac-notification-sys"
version = "0.6.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd604973958ddcc11b561193c0fb96ba146506ef2f231ef2e7c35fd2cbc9beca"
dependencies = [
"cc",
"log",
"objc2",
"objc2-foundation",
"time",
"uuid",
]
[[package]]
name = "markup5ever"
version = "0.39.0"
@@ -2166,6 +2199,16 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "moxcms"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb85c154ba489f01b25c0d36ae69a87e4a1c73a72631fc6c0eb6dde34a73e44b"
dependencies = [
"num-traits",
"pxfm",
]
[[package]]
name = "muda"
version = "0.20.0"
@@ -2229,6 +2272,20 @@ version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]]
name = "notify-rust"
version = "4.18.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4587364a9a0074333429b3df75a30a205340c56a536ca3eb6ca0e59b87bbf8af"
dependencies = [
"futures-lite",
"log",
"mac-notification-sys",
"serde",
"tauri-winrt-notification",
"zbus",
]
[[package]]
name = "num-conv"
version = "0.2.2"
@@ -2757,6 +2814,15 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a6394b9e965e73d0a289ee54f589087e2c676aedf60885baf52c76b771e4958"
[[package]]
name = "ppv-lite86"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9"
dependencies = [
"zerocopy",
]
[[package]]
name = "precomputed-hash"
version = "0.1.1"
@@ -2825,6 +2891,12 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "pxfm"
version = "0.1.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "quick-xml"
version = "0.42.0"
@@ -2855,6 +2927,35 @@ version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "rand"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9ef1d0d795eb7d84685bca4f72f3649f064e6641543d3a8c415898726a57b41"
dependencies = [
"rand_chacha",
"rand_core",
]
[[package]]
name = "rand_chacha"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
dependencies = [
"ppv-lite86",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c"
dependencies = [
"getrandom 0.3.4",
]
[[package]]
name = "raw-window-handle"
version = "0.6.2"
@@ -3524,13 +3625,14 @@ checksum = "ff7589a1859b09232522bb9edf076ca7fcf14ded96685d3fa629db73023ffbb7"
[[package]]
name = "strata-desktop"
version = "0.2.2"
version = "0.2.7"
dependencies = [
"gtk",
"serde",
"serde_json",
"tauri",
"tauri-build",
"tauri-plugin-notification",
"tauri-plugin-shell",
"tauri-plugin-single-instance",
"tauri-plugin-updater",
@@ -3755,6 +3857,7 @@ dependencies = [
"gtk",
"heck 0.5.0",
"http",
"image",
"jni 0.21.1",
"libc",
"log",
@@ -3869,6 +3972,26 @@ dependencies = [
"walkdir",
]
[[package]]
name = "tauri-plugin-notification"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b939d7b19c8af1381a72ef8474b3aba2b56c1ddac2c23f0c8e96abbcf86b198b"
dependencies = [
"log",
"notify-rust",
"rand",
"serde",
"serde_json",
"serde_repr",
"tauri",
"tauri-plugin",
"thiserror 2.0.21",
"time",
"url",
"zbus",
]
[[package]]
name = "tauri-plugin-shell"
version = "2.5.0"
@@ -4042,6 +4165,17 @@ dependencies = [
"toml 1.1.8+spec-1.1.0",
]
[[package]]
name = "tauri-winrt-notification"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f37a6c354fd28fc9e322ed9bd47e3959576dad28c9d58ea1cf888cce1c7ccb36"
dependencies = [
"thiserror 2.0.21",
"windows",
"windows-version",
]
[[package]]
name = "tempfile"
version = "3.27.0"
@@ -5332,6 +5466,26 @@ dependencies = [
"serde",
]
[[package]]
name = "zerocopy"
version = "0.8.63"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5fe1f8f1b06191a00962174c61aa5005e0bb391a6d80d07e24d115c01a92ed8"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.63"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "863ad3ac83293fb4d740aedbfdc9240dd8d1a50c1099acd76ce80ce7c7230c7f"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "zerofrom"
version = "0.1.8"
+4 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "strata-desktop"
version = "0.2.2"
version = "0.2.7"
description = "Strata Desktop — local AI, tuned to one model."
edition = "2021"
rust-version = "1.77"
@@ -15,8 +15,10 @@ webkit2gtk = "2.0"
tauri-build = { version = "2", features = [] }
[dependencies]
tauri = { version = "2", features = [] }
tauri = { version = "2", features = ["tray-icon", "image-png"] }
tauri-plugin-shell = "2"
# Desktop notifications for an approval that waits (#3477); the tray icon above is Tauri's own.
tauri-plugin-notification = "2"
tauri-plugin-updater = "2"
tauri-plugin-single-instance = "2"
serde = { version = "1", features = ["derive"] }
@@ -1 +1 @@
{"core":{"default_permission":{"identifier":"default","description":"Default core plugins set.","permissions":["core:path:default","core:event:default","core:window:default","core:webview:default","core:app:default","core:image:default","core:resources:default","core:menu:default","core:tray:default"]},"permissions":{},"permission_sets":{},"global_scope_schema":null},"core:app":{"default_permission":{"identifier":"default","description":"Default permissions for the plugin.","permissions":["allow-version","allow-name","allow-tauri-version","allow-identifier","allow-bundle-type","allow-register-listener","allow-remove-listener","allow-supports-multiple-windows"]},"permissions":{"allow-app-hide":{"identifier":"allow-app-hide","description":"Enables the app_hide command without any pre-configured scope.","commands":{"allow":["app_hide"],"deny":[]}},"allow-app-show":{"identifier":"allow-app-show","description":"Enables the app_show command without any pre-configured scope.","commands":{"allow":["app_show"],"deny":[]}},"allow-bundle-type":{"identifier":"allow-bundle-type","description":"Enables the bundle_type command without any pre-configured scope.","commands":{"allow":["bundle_type"],"deny":[]}},"allow-default-window-icon":{"identifier":"allow-default-window-icon","description":"Enables the default_window_icon command without any pre-configured scope.","commands":{"allow":["default_window_icon"],"deny":[]}},"allow-exit":{"identifier":"allow-exit","description":"Enables the exit command without any pre-configured scope.","commands":{"allow":["exit"],"deny":[]}},"allow-fetch-data-store-identifiers":{"identifier":"allow-fetch-data-store-identifiers","description":"Enables the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":["fetch_data_store_identifiers"],"deny":[]}},"allow-identifier":{"identifier":"allow-identifier","description":"Enables the identifier command without any pre-configured scope.","commands":{"allow":["identifier"],"deny":[]}},"allow-name":{"identifier":"allow-name","description":"Enables the name command without any pre-configured scope.","commands":{"allow":["name"],"deny":[]}},"allow-register-listener":{"identifier":"allow-register-listener","description":"Enables the register_listener command without any pre-configured scope.","commands":{"allow":["register_listener"],"deny":[]}},"allow-remove-data-store":{"identifier":"allow-remove-data-store","description":"Enables the remove_data_store command without any pre-configured scope.","commands":{"allow":["remove_data_store"],"deny":[]}},"allow-remove-listener":{"identifier":"allow-remove-listener","description":"Enables the remove_listener command without any pre-configured scope.","commands":{"allow":["remove_listener"],"deny":[]}},"allow-set-app-theme":{"identifier":"allow-set-app-theme","description":"Enables the set_app_theme command without any pre-configured scope.","commands":{"allow":["set_app_theme"],"deny":[]}},"allow-set-dock-visibility":{"identifier":"allow-set-dock-visibility","description":"Enables the set_dock_visibility command without any pre-configured scope.","commands":{"allow":["set_dock_visibility"],"deny":[]}},"allow-supports-multiple-windows":{"identifier":"allow-supports-multiple-windows","description":"Enables the supports_multiple_windows command without any pre-configured scope.","commands":{"allow":["supports_multiple_windows"],"deny":[]}},"allow-tauri-version":{"identifier":"allow-tauri-version","description":"Enables the tauri_version command without any pre-configured scope.","commands":{"allow":["tauri_version"],"deny":[]}},"allow-version":{"identifier":"allow-version","description":"Enables the version command without any pre-configured scope.","commands":{"allow":["version"],"deny":[]}},"deny-app-hide":{"identifier":"deny-app-hide","description":"Denies the app_hide command without any pre-configured scope.","commands":{"allow":[],"deny":["app_hide"]}},"deny-app-show":{"identifier":"deny-app-show","description":"Denies the app_show command without any pre-configured scope.","commands":{"allow":[],"deny":["app_show"]}},"deny-bundle-type":{"identifier":"deny-bundle-type","description":"Denies the bundle_type command without any pre-configured scope.","commands":{"allow":[],"deny":["bundle_type"]}},"deny-default-window-icon":{"identifier":"deny-default-window-icon","description":"Denies the default_window_icon command without any pre-configured scope.","commands":{"allow":[],"deny":["default_window_icon"]}},"deny-exit":{"identifier":"deny-exit","description":"Denies the exit command without any pre-configured scope.","commands":{"allow":[],"deny":["exit"]}},"deny-fetch-data-store-identifiers":{"identifier":"deny-fetch-data-store-identifiers","description":"Denies the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":[],"deny":["fetch_data_store_identifiers"]}},"deny-identifier":{"identifier":"deny-identifier","description":"DenLine truncated
{"core":{"default_permission":{"identifier":"default","description":"Default core plugins set.","permissions":["core:path:default","core:event:default","core:window:default","core:webview:default","core:app:default","core:image:default","core:resources:default","core:menu:default","core:tray:default"]},"permissions":{},"permission_sets":{},"global_scope_schema":null},"core:app":{"default_permission":{"identifier":"default","description":"Default permissions for the plugin.","permissions":["allow-version","allow-name","allow-tauri-version","allow-identifier","allow-bundle-type","allow-register-listener","allow-remove-listener","allow-supports-multiple-windows"]},"permissions":{"allow-app-hide":{"identifier":"allow-app-hide","description":"Enables the app_hide command without any pre-configured scope.","commands":{"allow":["app_hide"],"deny":[]}},"allow-app-show":{"identifier":"allow-app-show","description":"Enables the app_show command without any pre-configured scope.","commands":{"allow":["app_show"],"deny":[]}},"allow-bundle-type":{"identifier":"allow-bundle-type","description":"Enables the bundle_type command without any pre-configured scope.","commands":{"allow":["bundle_type"],"deny":[]}},"allow-default-window-icon":{"identifier":"allow-default-window-icon","description":"Enables the default_window_icon command without any pre-configured scope.","commands":{"allow":["default_window_icon"],"deny":[]}},"allow-exit":{"identifier":"allow-exit","description":"Enables the exit command without any pre-configured scope.","commands":{"allow":["exit"],"deny":[]}},"allow-fetch-data-store-identifiers":{"identifier":"allow-fetch-data-store-identifiers","description":"Enables the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":["fetch_data_store_identifiers"],"deny":[]}},"allow-identifier":{"identifier":"allow-identifier","description":"Enables the identifier command without any pre-configured scope.","commands":{"allow":["identifier"],"deny":[]}},"allow-name":{"identifier":"allow-name","description":"Enables the name command without any pre-configured scope.","commands":{"allow":["name"],"deny":[]}},"allow-register-listener":{"identifier":"allow-register-listener","description":"Enables the register_listener command without any pre-configured scope.","commands":{"allow":["register_listener"],"deny":[]}},"allow-remove-data-store":{"identifier":"allow-remove-data-store","description":"Enables the remove_data_store command without any pre-configured scope.","commands":{"allow":["remove_data_store"],"deny":[]}},"allow-remove-listener":{"identifier":"allow-remove-listener","description":"Enables the remove_listener command without any pre-configured scope.","commands":{"allow":["remove_listener"],"deny":[]}},"allow-set-app-theme":{"identifier":"allow-set-app-theme","description":"Enables the set_app_theme command without any pre-configured scope.","commands":{"allow":["set_app_theme"],"deny":[]}},"allow-set-dock-visibility":{"identifier":"allow-set-dock-visibility","description":"Enables the set_dock_visibility command without any pre-configured scope.","commands":{"allow":["set_dock_visibility"],"deny":[]}},"allow-supports-multiple-windows":{"identifier":"allow-supports-multiple-windows","description":"Enables the supports_multiple_windows command without any pre-configured scope.","commands":{"allow":["supports_multiple_windows"],"deny":[]}},"allow-tauri-version":{"identifier":"allow-tauri-version","description":"Enables the tauri_version command without any pre-configured scope.","commands":{"allow":["tauri_version"],"deny":[]}},"allow-version":{"identifier":"allow-version","description":"Enables the version command without any pre-configured scope.","commands":{"allow":["version"],"deny":[]}},"deny-app-hide":{"identifier":"deny-app-hide","description":"Denies the app_hide command without any pre-configured scope.","commands":{"allow":[],"deny":["app_hide"]}},"deny-app-show":{"identifier":"deny-app-show","description":"Denies the app_show command without any pre-configured scope.","commands":{"allow":[],"deny":["app_show"]}},"deny-bundle-type":{"identifier":"deny-bundle-type","description":"Denies the bundle_type command without any pre-configured scope.","commands":{"allow":[],"deny":["bundle_type"]}},"deny-default-window-icon":{"identifier":"deny-default-window-icon","description":"Denies the default_window_icon command without any pre-configured scope.","commands":{"allow":[],"deny":["default_window_icon"]}},"deny-exit":{"identifier":"deny-exit","description":"Denies the exit command without any pre-configured scope.","commands":{"allow":[],"deny":["exit"]}},"deny-fetch-data-store-identifiers":{"identifier":"deny-fetch-data-store-identifiers","description":"Denies the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":[],"deny":["fetch_data_store_identifiers"]}},"deny-identifier":{"identifier":"deny-identifier","description":"DenLine truncated
@@ -2426,6 +2426,216 @@
"const": "core:window:deny-unminimize",
"markdownDescription": "Denies the unminimize command without any pre-configured scope."
},
{
"description": "This permission set configures which\nnotification features are by default exposed.\n\n#### Granted Permissions\n\nIt allows all notification related features.\n\n\n#### This default permission set includes:\n\n- `allow-is-permission-granted`\n- `allow-request-permission`\n- `allow-notify`\n- `allow-register-action-types`\n- `allow-register-listener`\n- `allow-remove-listener`\n- `allow-cancel`\n- `allow-get-pending`\n- `allow-remove-active`\n- `allow-get-active`\n- `allow-check-permissions`\n- `allow-show`\n- `allow-batch`\n- `allow-list-channels`\n- `allow-delete-channel`\n- `allow-create-channel`\n- `allow-permission-state`",
"type": "string",
"const": "notification:default",
"markdownDescription": "This permission set configures which\nnotification features are by default exposed.\n\n#### Granted Permissions\n\nIt allows all notification related features.\n\n\n#### This default permission set includes:\n\n- `allow-is-permission-granted`\n- `allow-request-permission`\n- `allow-notify`\n- `allow-register-action-types`\n- `allow-register-listener`\n- `allow-remove-listener`\n- `allow-cancel`\n- `allow-get-pending`\n- `allow-remove-active`\n- `allow-get-active`\n- `allow-check-permissions`\n- `allow-show`\n- `allow-batch`\n- `allow-list-channels`\n- `allow-delete-channel`\n- `allow-create-channel`\n- `allow-permission-state`"
},
{
"description": "Enables the batch command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-batch",
"markdownDescription": "Enables the batch command without any pre-configured scope."
},
{
"description": "Enables the cancel command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-cancel",
"markdownDescription": "Enables the cancel command without any pre-configured scope."
},
{
"description": "Enables the check_permissions command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-check-permissions",
"markdownDescription": "Enables the check_permissions command without any pre-configured scope."
},
{
"description": "Enables the create_channel command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-create-channel",
"markdownDescription": "Enables the create_channel command without any pre-configured scope."
},
{
"description": "Enables the delete_channel command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-delete-channel",
"markdownDescription": "Enables the delete_channel command without any pre-configured scope."
},
{
"description": "Enables the get_active command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-get-active",
"markdownDescription": "Enables the get_active command without any pre-configured scope."
},
{
"description": "Enables the get_pending command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-get-pending",
"markdownDescription": "Enables the get_pending command without any pre-configured scope."
},
{
"description": "Enables the is_permission_granted command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-is-permission-granted",
"markdownDescription": "Enables the is_permission_granted command without any pre-configured scope."
},
{
"description": "Enables the list_channels command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-list-channels",
"markdownDescription": "Enables the list_channels command without any pre-configured scope."
},
{
"description": "Enables the notify command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-notify",
"markdownDescription": "Enables the notify command without any pre-configured scope."
},
{
"description": "Enables the permission_state command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-permission-state",
"markdownDescription": "Enables the permission_state command without any pre-configured scope."
},
{
"description": "Enables the register_action_types command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-register-action-types",
"markdownDescription": "Enables the register_action_types command without any pre-configured scope."
},
{
"description": "Enables the register_listener command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-register-listener",
"markdownDescription": "Enables the register_listener command without any pre-configured scope."
},
{
"description": "Enables the remove_active command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-remove-active",
"markdownDescription": "Enables the remove_active command without any pre-configured scope."
},
{
"description": "Enables the remove_listener command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-remove-listener",
"markdownDescription": "Enables the remove_listener command without any pre-configured scope."
},
{
"description": "Enables the request_permission command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-request-permission",
"markdownDescription": "Enables the request_permission command without any pre-configured scope."
},
{
"description": "Enables the show command without any pre-configured scope.",
"type": "string",
"const": "notification:allow-show",
"markdownDescription": "Enables the show command without any pre-configured scope."
},
{
"description": "Denies the batch command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-batch",
"markdownDescription": "Denies the batch command without any pre-configured scope."
},
{
"description": "Denies the cancel command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-cancel",
"markdownDescription": "Denies the cancel command without any pre-configured scope."
},
{
"description": "Denies the check_permissions command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-check-permissions",
"markdownDescription": "Denies the check_permissions command without any pre-configured scope."
},
{
"description": "Denies the create_channel command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-create-channel",
"markdownDescription": "Denies the create_channel command without any pre-configured scope."
},
{
"description": "Denies the delete_channel command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-delete-channel",
"markdownDescription": "Denies the delete_channel command without any pre-configured scope."
},
{
"description": "Denies the get_active command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-get-active",
"markdownDescription": "Denies the get_active command without any pre-configured scope."
},
{
"description": "Denies the get_pending command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-get-pending",
"markdownDescription": "Denies the get_pending command without any pre-configured scope."
},
{
"description": "Denies the is_permission_granted command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-is-permission-granted",
"markdownDescription": "Denies the is_permission_granted command without any pre-configured scope."
},
{
"description": "Denies the list_channels command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-list-channels",
"markdownDescription": "Denies the list_channels command without any pre-configured scope."
},
{
"description": "Denies the notify command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-notify",
"markdownDescription": "Denies the notify command without any pre-configured scope."
},
{
"description": "Denies the permission_state command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-permission-state",
"markdownDescription": "Denies the permission_state command without any pre-configured scope."
},
{
"description": "Denies the register_action_types command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-register-action-types",
"markdownDescription": "Denies the register_action_types command without any pre-configured scope."
},
{
"description": "Denies the register_listener command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-register-listener",
"markdownDescription": "Denies the register_listener command without any pre-configured scope."
},
{
"description": "Denies the remove_active command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-remove-active",
"markdownDescription": "Denies the remove_active command without any pre-configured scope."
},
{
"description": "Denies the remove_listener command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-remove-listener",
"markdownDescription": "Denies the remove_listener command without any pre-configured scope."
},
{
"description": "Denies the request_permission command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-request-permission",
"markdownDescription": "Denies the request_permission command without any pre-configured scope."
},
{
"description": "Denies the show command without any pre-configured scope.",
"type": "string",
"const": "notification:deny-show",
"markdownDescription": "Denies the show command without any pre-configured scope."
},
{
"description": "This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type": "string",
@@ -2491,6 +2701,60 @@
"type": "string",
"const": "shell:deny-stdin-write",
"markdownDescription": "Denies the stdin_write command without any pre-configured scope."
},
{
"description": "This permission set configures which kind of\nupdater functions are exposed to the frontend.\n\n#### Granted Permissions\n\nThe full workflow from checking for updates to installing them\nis enabled.\n\n\n#### This default permission set includes:\n\n- `allow-check`\n- `allow-download`\n- `allow-install`\n- `allow-download-and-install`",
"type": "string",
"const": "updater:default",
"markdownDescription": "This permission set configures which kind of\nupdater functions are exposed to the frontend.\n\n#### Granted Permissions\n\nThe full workflow from checking for updates to installing them\nis enabled.\n\n\n#### This default permission set includes:\n\n- `allow-check`\n- `allow-download`\n- `allow-install`\n- `allow-download-and-install`"
},
{
"description": "Enables the check command without any pre-configured scope.",
"type": "string",
"const": "updater:allow-check",
"markdownDescription": "Enables the check command without any pre-configured scope."
},
{
"description": "Enables the download command without any pre-configured scope.",
"type": "string",
"const": "updater:allow-download",
"markdownDescription": "Enables the download command without any pre-configured scope."
},
{
"description": "Enables the download_and_install command without any pre-configured scope.",
"type": "string",
"const": "updater:allow-download-and-install",
"markdownDescription": "Enables the download_and_install command without any pre-configured scope."
},
{
"description": "Enables the install command without any pre-configured scope.",
"type": "string",
"const": "updater:allow-install",
"markdownDescription": "Enables the install command without any pre-configured scope."
},
{
"description": "Denies the check command without any pre-configured scope.",
"type": "string",
"const": "updater:deny-check",
"markdownDescription": "Denies the check command without any pre-configured scope."
},
{
"description": "Denies the download command without any pre-configured scope.",
"type": "string",
"const": "updater:deny-download",
"markdownDescription": "Denies the download command without any pre-configured scope."
},
{
"description": "Denies the download_and_install command without any pre-configured scope.",
"type": "string",
"const": "updater:deny-download-and-install",
"markdownDescription": "Denies the download_and_install command without any pre-configured scope."
},
{
"description": "Denies the install command without any pre-configured scope.",
"type": "string",
"const": "updater:deny-install",
"markdownDescription": "Denies the install command without any pre-configured scope."
}
]
},
+32
View File
@@ -701,6 +701,35 @@ SectionEnd
!macroend
; STRATA-MIGRATE-END
; STRATA-STOP-BEGIN
; Stop what Strata itself started from the install folder (the app, its host, the node runtime, the engine) before files are removed:
; a running copy keeps its files open, so the folder stayed behind after use. Only processes whose image path is inside INSTDIR are
; stopped; this uninstaller's own process and any program elsewhere are left alone.
!macro StrataStopOwnProcesses INSTALLDIR
System::Call 'kernel32::SetEnvironmentVariableW(w "STRATA_INSTDIR", w "${INSTALLDIR}")'
System::Call 'kernel32::GetCurrentProcessId()i.r9'
System::Call 'kernel32::SetEnvironmentVariableW(w "STRATA_SELFPID", w "$9")'
; the installer is a 32-bit program, so plain powershell.exe is the 32-bit one, which cannot read the image path of a 64-bit process: use the 64-bit one
StrCpy $8 "$WINDIR\Sysnative\WindowsPowerShell\v1.0\powershell.exe"
IfFileExists $8 +2 0
StrCpy $8 "powershell.exe"
nsExec::Exec '"$8" -NoProfile -NonInteractive -ExecutionPolicy Bypass -Command "$$d = [IO.Path]::GetFullPath($$env:STRATA_INSTDIR).TrimEnd([char]92) + [char]92; $$ids = @(); $$ids += Get-Process -ErrorAction SilentlyContinue | Where-Object { $$_.Path -and [IO.Path]::GetFullPath($$_.Path).StartsWith($$d, [StringComparison]::OrdinalIgnoreCase) } | ForEach-Object { $$_.Id }; try { $$ids += Get-CimInstance Win32_Process -ErrorAction Stop | Where-Object { $$_.ExecutablePath -and [IO.Path]::GetFullPath($$_.ExecutablePath).StartsWith($$d, [StringComparison]::OrdinalIgnoreCase) } | ForEach-Object { $$_.ProcessId } } catch {}; $$ids | Sort-Object -Unique | Where-Object { $$_ -ne [int]$$env:STRATA_SELFPID } | ForEach-Object { Stop-Process -Id $$_ -Force -ErrorAction SilentlyContinue }; Start-Sleep -Milliseconds 1500"'
Pop $9
!macroend
; STRATA-STOP-END
; STRATA-CLEAN-BEGIN
; The app writes into its own folders while it runs (Python bytecode caches under control\, for one), and the file-by-file delete below does not know them,
; so those folders stayed behind with a few dozen files. These four folders hold only what the installer put there and what the app made there;
; user data is never in them (it lives in the data folder), and the install folder itself is still removed with a plain RMDir.
!macro StrataRemoveAppFolders INSTALLDIR
RMDir /r "${INSTALLDIR}\control"
RMDir /r "${INSTALLDIR}\host"
RMDir /r "${INSTALLDIR}\node"
RMDir /r "${INSTALLDIR}\harness"
!macroend
; STRATA-CLEAN-END
Section Install
SetOutPath $INSTDIR
@@ -847,6 +876,7 @@ Section Uninstall
!insertmacro NSIS_HOOK_PREUNINSTALL
!endif
!insertmacro StrataStopOwnProcesses "$INSTDIR"
!insertmacro CheckIfAppIsRunning "$INSTDIR\${MAINBINARYNAME}.exe" "${PRODUCTNAME}"
; Delete the app directory and its content from disk
@@ -879,6 +909,8 @@ Section Uninstall
{{/each}}
!insertmacro StrataRemoveAppFolders "$INSTDIR"
; Delete uninstaller
Delete "$INSTDIR\uninstall.exe"
+33 -3
View File
@@ -39,6 +39,7 @@ use tauri::{Manager, WebviewUrl, WebviewWindowBuilder};
mod updater;
mod desktoplog;
mod shell;
#[cfg(target_os = "linux")]
mod supervisor;
@@ -252,13 +253,24 @@ fn pick_port() -> Result<u16, String> {
/// launch starts with empty storage, as before).
const PREFERRED_PORT: u16 = 47831;
/// Set when the port the person (or the app) wanted was busy: "busy:<wanted>:<used>". The host gets it as STRATA_PORT_NOTICE and tells the
/// person (the chats and settings saved on the page belong to the port, so a different port starts empty) (#3402).
static PORT_NOTICE: std::sync::Mutex<Option<String>> = std::sync::Mutex::new(None);
fn pick_stable_port() -> Result<u16, String> {
match TcpListener::bind(("127.0.0.1", PREFERRED_PORT)) {
// The person may have chosen a port in Settings (shell-prefs.json); otherwise the app's usual one.
let wanted = shell::wanted_port(&shell::read_prefs(), PREFERRED_PORT);
*PORT_NOTICE.lock().unwrap() = None;
match TcpListener::bind(("127.0.0.1", wanted)) {
Ok(listener) => {
drop(listener);
Ok(PREFERRED_PORT)
Ok(wanted)
}
Err(_) => {
let used = pick_port()?;
*PORT_NOTICE.lock().unwrap() = Some(shell::busy_notice(wanted, used));
Ok(used)
}
Err(_) => pick_port(),
}
}
@@ -469,6 +481,9 @@ fn host_command(res: &Path, port: u16, shutdown_token: &str) -> Result<Command,
.stderr(Stdio::inherit());
apply_runtime_policy(&mut cmd, res);
hide_console(&mut cmd);
if let Some(notice) = PORT_NOTICE.lock().unwrap().clone() {
cmd.env("STRATA_PORT_NOTICE", notice);
}
// A user with the Strata API worker on a different port sets this; unset means the worker's
// documented loopback default.
if let Ok(url) = std::env::var("STRATA_SERVER_URL") {
@@ -875,6 +890,7 @@ fn run_embedded() {
}))
.plugin(tauri_plugin_shell::init())
.plugin(tauri_plugin_updater::Builder::new().build())
.plugin(tauri_plugin_notification::init())
.manage(HostProcess(Mutex::new(None)))
.manage(HostPort(Mutex::new(None)))
.manage(ShutdownToken(Mutex::new(None)))
@@ -901,6 +917,20 @@ fn run_embedded() {
Ok(())
})
.on_window_event(|window, event| {
// "Minimise to tray" (Settings, off by default): closing the main window hides it and keeps Strata running in the tray; Quit in the
// tray menu ends it (#3477). If no tray could be made the window closes as before.
if let tauri::WindowEvent::CloseRequested { api, .. } = event {
if window.label() == "main" {
let prefs = shell::read_prefs();
if prefs.tray {
api.prevent_close();
shell::hide_to_tray(&window.app_handle().clone(), &prefs.language);
if !shell::tray_exists(&window.app_handle().clone()) {
let _ = window.destroy();
}
}
}
}
if let tauri::WindowEvent::Destroyed = event {
stop_host(&window.app_handle().clone());
}
+238
View File
@@ -0,0 +1,238 @@
//! The shell's own choices and the two things the host asks of it besides updates (#3402, #3477).
//!
//! `<Strata data folder>/shell-prefs.json` is written by the App Host when the person changes a setting
//! (`apps/host/server/shell-prefs.mjs`) and read here when Strata starts, before any page exists:
//!
//! { "port": 0 | 1024..65535, "tray": false, "language": "en" }
//!
//! * `port`: the port the window's origin uses. 0 is the default, 47831. When the port is busy the shell takes a free one and tells the
//! host (`STRATA_PORT_NOTICE=busy:<wanted>:<used>`), which shows the person a warning: the chats and settings of the page are kept per
//! port, so a different port starts empty.
//! * `tray`: closing the window hides it and keeps Strata running in the tray (Windows, macOS). The tray menu follows `language`.
//! * `/shell/notify` on the loopback bridge shows a desktop notification (an approval is waiting) whether the window is open or hidden.
use std::path::PathBuf;
use serde_json::Value;
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Prefs {
pub port: u16,
pub tray: bool,
pub language: String,
}
impl Default for Prefs {
fn default() -> Self {
Prefs { port: 0, tray: false, language: "en".into() }
}
}
pub const LANGUAGES: [&str; 8] = ["en", "zh", "ja", "de", "es", "ru", "pt", "ko"];
pub fn prefs_path() -> PathBuf {
crate::desktoplog::data_dir().join("shell-prefs.json")
}
/// The file's text as choices; anything missing or out of range is the default.
pub fn parse_prefs(text: &str) -> Prefs {
let value: Value = serde_json::from_str(text.trim_start_matches('\u{feff}')).unwrap_or(Value::Null);
let port = value.get("port").and_then(Value::as_u64).filter(|p| (1024..=65535).contains(p)).unwrap_or(0) as u16;
let tray = value.get("tray").and_then(Value::as_bool).unwrap_or(false);
let language = value
.get("language")
.and_then(Value::as_str)
.filter(|l| LANGUAGES.contains(l))
.unwrap_or("en")
.to_string();
Prefs { port, tray, language }
}
pub fn read_prefs() -> Prefs {
std::fs::read_to_string(prefs_path()).map(|t| parse_prefs(&t)).unwrap_or_default()
}
/// The port to ask for: the person's choice, else the app's usual one.
pub fn wanted_port(prefs: &Prefs, usual: u16) -> u16 {
if prefs.port != 0 { prefs.port } else { usual }
}
/// The notice the host shows when the wanted port was taken.
pub fn busy_notice(wanted: u16, used: u16) -> String {
format!("busy:{wanted}:{used}")
}
/// `%41%20b` -> "A b" (a query value from the host). Bad escapes are kept as typed.
pub fn percent_decode(text: &str) -> String {
let bytes = text.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'%' && i + 2 < bytes.len() {
if let Some(v) = std::str::from_utf8(&bytes[i + 1..i + 3]).ok().and_then(|h| u8::from_str_radix(h, 16).ok()) {
out.push(v);
i += 3;
continue;
}
}
out.push(if bytes[i] == b'+' { b' ' } else { bytes[i] });
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
/// The value of `key` in the query of a request path (`/shell/notify?title=..&body=..`).
pub fn query_value(path: &str, key: &str) -> Option<String> {
let query = path.split_once('?')?.1;
query.split('&').find_map(|pair| {
let (k, v) = pair.split_once('=')?;
(k == key).then(|| percent_decode(v))
})
}
/// A short single-line text for a notification: no control characters, bounded.
pub fn clean_text(text: &str, max: usize) -> String {
text.chars().filter(|c| !c.is_control()).take(max).collect::<String>().trim().to_string()
}
/// The tray menu: (open, quit, tooltip), in the app's language.
pub fn tray_labels(language: &str) -> (&'static str, &'static str, &'static str) {
match language {
"zh" => ("打开 Strata", "退出 Strata", "Strata"),
"ja" => ("Strata を開く", "Strata を終了", "Strata"),
"de" => ("Strata öffnen", "Strata beenden", "Strata"),
"es" => ("Abrir Strata", "Salir de Strata", "Strata"),
"ru" => ("Открыть Strata", "Выйти из Strata", "Strata"),
"pt" => ("Abrir o Strata", "Sair do Strata", "Strata"),
"ko" => ("Strata 열기", "Strata 종료", "Strata"),
_ => ("Open Strata", "Quit Strata", "Strata"),
}
}
#[cfg(not(target_os = "linux"))]
mod native {
use tauri::menu::{Menu, MenuItem};
use tauri::tray::{MouseButton, MouseButtonState, TrayIconBuilder, TrayIconEvent};
use tauri::{AppHandle, Manager};
use tauri_plugin_notification::NotificationExt;
const TRAY_ID: &str = "strata-tray";
fn show_main(app: &AppHandle) {
if let Some(window) = app.get_webview_window("main") {
let _ = window.unminimize();
let _ = window.show();
let _ = window.set_focus();
}
}
/// Hide the window and keep Strata running in the tray (made on the first need, with the menu in the app's language).
pub fn hide_to_tray(app: &AppHandle, language: &str) {
if app.tray_by_id(TRAY_ID).is_none() {
let (open, quit, tip) = super::tray_labels(language);
let built = (|| -> tauri::Result<()> {
let open_item = MenuItem::with_id(app, "open", open, true, None::<&str>)?;
let quit_item = MenuItem::with_id(app, "quit", quit, true, None::<&str>)?;
let menu = Menu::with_items(app, &[&open_item, &quit_item])?;
let mut builder = TrayIconBuilder::with_id(TRAY_ID)
.tooltip(tip)
.menu(&menu)
.show_menu_on_left_click(false)
.on_menu_event(|app, event| match event.id().as_ref() {
"open" => show_main(app),
"quit" => app.exit(0),
_ => {}
})
.on_tray_icon_event(|tray, event| {
if let TrayIconEvent::Click { button: MouseButton::Left, button_state: MouseButtonState::Up, .. } = event {
show_main(tray.app_handle());
}
});
if let Some(icon) = app.default_window_icon() {
builder = builder.icon(icon.clone());
}
builder.build(app)?;
Ok(())
})();
if built.is_err() {
// No tray could be made (a desktop without one): do not hide a window nobody could bring back.
return;
}
}
if let Some(window) = app.get_webview_window("main") {
let _ = window.hide();
}
}
pub fn tray_exists(app: &AppHandle) -> bool {
app.tray_by_id(TRAY_ID).is_some()
}
/// Show a desktop notification; a hidden or minimised window also asks for attention (the taskbar button flashes).
pub fn notify(app: &AppHandle, title: &str, body: &str) -> bool {
if title.is_empty() {
return false;
}
if let Some(window) = app.get_webview_window("main") {
let hidden = !window.is_visible().unwrap_or(true) || window.is_minimized().unwrap_or(false);
if hidden {
let _ = window.request_user_attention(Some(tauri::UserAttentionType::Informational));
}
}
app.notification().builder().title(title).body(body).show().is_ok()
}
}
#[cfg(not(target_os = "linux"))]
pub use native::{hide_to_tray, notify, tray_exists};
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_file_gives_choices_and_anything_wrong_is_the_default() {
assert_eq!(parse_prefs(""), Prefs::default());
assert_eq!(parse_prefs("{ not json"), Prefs::default());
let p = parse_prefs(r#"{"port":47900,"tray":true,"language":"de"}"#);
assert_eq!((p.port, p.tray, p.language.as_str()), (47900, true, "de"));
assert_eq!(parse_prefs("\u{feff}{\"port\":47900}").port, 47900);
assert_eq!(parse_prefs(r#"{"port":80}"#).port, 0, "a port below 1024 is not used");
assert_eq!(parse_prefs(r#"{"port":70000}"#).port, 0);
assert_eq!(parse_prefs(r#"{"port":"47831"}"#).port, 0, "a text is not a number");
assert_eq!(parse_prefs(r#"{"tray":"yes","language":"xx"}"#), Prefs::default());
}
#[test]
fn the_wanted_port_is_the_choice_or_the_usual_one() {
assert_eq!(wanted_port(&Prefs::default(), 47831), 47831);
assert_eq!(wanted_port(&Prefs { port: 47900, ..Prefs::default() }, 47831), 47900);
assert_eq!(busy_notice(47831, 51234), "busy:47831:51234");
}
#[test]
fn query_values_are_decoded_and_bounded() {
let path = "/shell/notify?title=Strata%20needs%20your%20answer&body=a%20%26%20b%3F+ok";
assert_eq!(query_value(path, "title").as_deref(), Some("Strata needs your answer"));
assert_eq!(query_value(path, "body").as_deref(), Some("a & b? ok"));
assert_eq!(query_value(path, "missing"), None);
assert_eq!(query_value("/shell/notify", "title"), None);
assert_eq!(percent_decode("100%"), "100%");
assert_eq!(percent_decode("%zz%41"), "%zzA");
assert_eq!(percent_decode("%C3%A4"), "ä");
assert_eq!(clean_text("a\nb\u{7}c", 10), "abc");
assert_eq!(clean_text("abcdef", 3), "abc");
}
#[test]
fn every_language_has_tray_labels() {
for language in LANGUAGES {
let (open, quit, tip) = tray_labels(language);
assert!(!open.is_empty() && !quit.is_empty() && !tip.is_empty(), "{language}");
if language != "en" {
assert_ne!(open, tray_labels("en").0, "{language}");
}
}
assert_eq!(tray_labels("xx"), tray_labels("en"));
}
}
+10
View File
@@ -210,6 +210,16 @@ fn handle(mut stream: TcpStream, app: &AppHandle, token: &str, progress: &Arc<Mu
respond(&mut stream, "202 Accepted", &progress.lock().unwrap().json());
}
("GET", "/app-update/status") => respond(&mut stream, "200 OK", &progress.lock().unwrap().json()),
// The host asks for a desktop notification (an approval is waiting): title and body are in the query, bounded and cleaned here.
("POST", p) if p.starts_with("/shell/notify") => {
let title = crate::shell::clean_text(&crate::shell::query_value(p, "title").unwrap_or_default(), 120);
let body = crate::shell::clean_text(&crate::shell::query_value(p, "body").unwrap_or_default(), 300);
#[cfg(not(target_os = "linux"))]
let shown = crate::shell::notify(app, &title, &body);
#[cfg(target_os = "linux")]
let shown = false;
respond(&mut stream, "200 OK", &serde_json::json!({ "shown": shown }))
}
_ => respond(&mut stream, "404 Not Found", &serde_json::json!({ "error": "unknown route" })),
}
}
+12
View File
@@ -10,3 +10,15 @@ npm run preview:ui
The built UI is included. To rebuild after source edits, run `npm run build`. `preview:ui` uses isolated simulated data; `npm run dev` connects the normal host.
The previous project README is retained in [docs/README-historical.md](docs/README-historical.md). It contains older implementation status and is historical reference.
## Languages
The app speaks English, Chinese, Japanese, German, French, Spanish, Russian, Portuguese and Korean. The English text is the key: `src/locales/en.json`
(with `fix-en.json`, `tune-en.json` and `tools-en.json`) holds every string, and each language has the same four files (`fr.json`, `fix-fr.json`, ...).
- **A new string.** Write it with `tr("...")`, then add it to the English catalog (`node scripts/append-localization.mjs`, or the next `fx` number in
`scripts/make-fix-locales.py`). The language tests fail until every language has it.
- **French.** `python scripts/fr_sync.py missing todo.tsv` lists the English strings that have no French yet, one per line (`id`, a tab, the text).
Translate the second column ("vous", short plain words, one next step; a lone `=` keeps the English), then `python scripts/fr_sync.py apply todo.tsv` merges it
(it also sets the French spacing before `: ; ? !` and the typographic apostrophe) and `python scripts/fr_sync.py check` confirms nothing is missing.
- **Screenshots in a language.** `npm run build`, then `node scripts/language-shots.mjs fr <folder> models settings setup` (hermetic preview host, scratch profile, headless Chrome).
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "strata",
"version": "0.2.2",
"version": "0.2.7",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "strata",
"version": "0.2.2",
"version": "0.2.7",
"dependencies": {
"html-to-image": "^1.11.13",
"katex": "^0.19.0",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "strata",
"private": true,
"version": "0.2.2",
"version": "0.2.7",
"type": "module",
"description": "Strata \u2014 local AI, tuned to one model. Web shell + engine host.",
"scripts": {
Binary file not shown.
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env node
/**
* The context fixes against a REAL engine (not the scripted worker): Gyro-S at 64K on this PC, the agent bridge and the real Harness.
*
* gpu + heavy locks held first (python phaseA-tests\lock.py acquire gpu "023-agent: context e2e", then heavy); one engine; nothing of the
* installed app is used or touched: its model files are read, its Harness is copied (STRATA_HARNESS_DIR), data goes to a scratch folder.
*
* node scripts/context-e2e.mjs (needs STRATA_HARNESS_DIR; see the E2E_* names below)
*
* It starts serve/server.py as ITS OWN child (foreground: this script waits for it and stops it by PID at the end or on a memory alarm),
* runs four scenarios and prints one table. Exit code 0 when every expectation held.
*
* A a message that can never fit (about 60K tokens in a 64K window) is refused in seconds, before the engine reads it
* B a chat of dense log messages grows past the window several times over: automatic compaction runs, nothing is refused
* C /compact at a high fill: summarises, says the figures; and a second /compact is answered in plain words
* D the needle: a fact from the first message is asked about at the end (reported, not asserted: it is the summary's quality)
*/
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { createAgentBridge } from '../server/agent/agent-bridge.mjs';
import { countTokens } from '../server/agent/context-fit.mjs';
const REPO = join(import.meta.dirname, '..', '..', '..');
const env = (name, fallback) => process.env[name] ?? fallback;
const DOCS = env('E2E_DOCS', 'C:\\Users\\AI-Server\\Documents\\Strata');
const ENGINE = env('E2E_ENGINE', 'C:\\Users\\AI-Server\\Desktop\\Strata\\GitHub\\wt-023-int\\build-cuda\\strata.exe');
const PROFILE = env('E2E_EXPERT_PROFILE', 'C:\\Users\\AI-Server\\AppData\\Local\\Programs\\Strata\\control\\data\\expert-profile.bin');
const PYTHON = env('E2E_PYTHON', 'python');
const PORT = Number(env('E2E_PORT', '18590'));
const MIN_FREE_GB = Number(env('E2E_MIN_FREE_GB', '7'));
const GGUF = join(DOCS, 'models', 'gyro-Gyro-S', 'Qwen3.8-Flash-Next-Gyro-S-TQ1_0.gguf');
const CUDA_BIN = env('E2E_CUDA_BIN', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v13.0\\bin');
const WINDOW = 65536;
/** A model server that is already running (the scripted worker, to try this script without a GPU): nothing is started or stopped. */
const EXISTING = process.env.E2E_BASE_URL ?? '';
const sleep = ms => new Promise(r => setTimeout(r, ms));
const freeGb = () => {
const out = spawnSync('powershell', ['-NoProfile', '-Command', '[math]::Round((Get-CimInstance Win32_OperatingSystem).FreePhysicalMemory/1MB,1)'], { encoding: 'utf8' }).stdout.trim();
return Number(out);
};
const rows = [];
const note = (id, ok, detail) => { rows.push({ id, ok, detail }); console.log(`${ok ? 'PASS' : 'FAIL'} ${id} ${detail}`); };
if (!process.env.STRATA_HARNESS_DIR) { console.error('STRATA_HARNESS_DIR is not set'); process.exit(2); }
if (!EXISTING) for (const f of [ENGINE, GGUF]) if (!existsSync(f)) { console.error(`missing: ${f}`); process.exit(2); }
if (!EXISTING && freeGb() < 20) { console.error(`only ${freeGb()} GB of RAM are free; wait for the other streams (lock.py status) and run again`); process.exit(2); }
const scratch = mkdtempSync(join(tmpdir(), 'strata-ctx-e2e-'));
const configFile = join(scratch, 'server.json');
const logFile = join(scratch, 'server.log');
const readyFile = join(scratch, 'ready.json');
writeFileSync(configFile, JSON.stringify({
exe: ENGINE,
args: ['--pack', join(DOCS, 'packs', 'gyro-gyro-s'), '--native', GGUF, '--ple-gguf', GGUF, ...(existsSync(PROFILE) ? ['--expert-profile', PROFILE] : []),
'--prefill', 'auto', '--spec-min-p', '0.5', '--mtp', join(DOCS, 'mtp', 'rt'), '--max-context', String(WINDOW), '--kv', 'int8', '--pool-workers', '5',
'--expert-cache', 'auto', '--kv-resident', '32768', '--spec', '4', '--vram-reserve-mib', '1536'],
cwd: REPO, tokenizer: join(DOCS, 'packs', 'gyro-gyro-s', 'tokenizer'), model_name: 'qwen3.8-flash-next-gyro-gyro-s', log: join(scratch, 'engine.log'), lib_dirs: [], port: PORT, gpu: 0,
}, null, 2));
let server = null, bridge = null, alarm = null;
const stopServer = () => {
if (server?.pid) spawnSync('taskkill', ['/T', '/F', '/PID', String(server.pid)], { stdio: 'ignore' });
server = null;
};
const finish = async (code) => {
clearInterval(alarm);
try { await bridge?.close(); } catch { /* going away */ }
stopServer();
await sleep(1500);
console.log(`\nscratch folder: ${scratch}`);
process.exit(code);
};
process.on('SIGINT', () => finish(130));
if (!EXISTING) server = spawn(PYTHON, ['serve/server.py', '--engine', 'strata', '--config', configFile, '--port', String(PORT), '--no-api-key', '--ready-file', readyFile], {
cwd: REPO, env: { ...process.env, PATH: `${CUDA_BIN};${process.env.PATH}`, STRATA_DATA_DIR: join(scratch, 'data') }, stdio: ['ignore', 'pipe', 'pipe'], windowsHide: true,
});
const sink = [];
if (server) { server.stdout.on('data', d => sink.push(String(d))); server.stderr.on('data', d => sink.push(String(d))); server.on('exit', code => { sink.push(`[server exited ${code}]`); server = null; }); }
// Memory alarm: this run must never be what freezes the PC. Under the floor the engine is stopped (by PID) and the run ends.
if (!EXISTING) alarm = setInterval(() => { const free = freeGb(); if (free < MIN_FREE_GB) { console.error(`ALARM: ${free} GB of RAM free, stopping the engine`); finish(3); } }, 3000);
alarm?.unref?.();
const base = EXISTING || `http://127.0.0.1:${PORT}`;
let ready = false;
for (let i = 0; i < 120 && !ready; i++) {
if (!EXISTING && !server) break;
try { ready = (await fetch(`${base}/health`, { signal: AbortSignal.timeout(2000) })).ok; } catch { /* still loading */ }
if (!ready) await sleep(2000);
}
writeFileSync(logFile, sink.join(''));
if (!ready) { console.error('the engine did not become ready:\n' + sink.join('').slice(-1500)); await finish(4); }
const model = (await (await fetch(`${base}/v1/models`)).json()).data[0].id;
console.log(`engine ready: ${model}, free RAM ${freeGb()} GB`);
// ---------------------------------------------------------------------------------------------------------------- the pieces
const line = i => `2026-10-22 13:${String(Math.floor(i / 60) % 60).padStart(2, '0')}:${String(i % 60).padStart(2, '0')} INFO svc-${i % 17} req=${(i * 2654435761 >>> 0).toString(16)} status=${200 + (i % 7) * 3} ms=${(i * 37) % 900} path=/api/v1/items/${(i * 7919) % 100000}`;
const logText = (chars, from = 0, needle = '') => {
const out = [];
let n = 0, i = from;
while (n < chars) { const l = line(i++); out.push(l); n += l.length + 1; }
if (needle) out.splice(Math.floor(out.length / 2), 0, needle);
return out.join('\n');
};
const count = text => countTokens({ baseUrl: `${base}/v1`, model, text });
const turnsOf = async (message) => {
const frames = [];
const t0 = Date.now();
await bridge.runTurn({ chatId: 'e2e', message, workspacePath: scratch, mode: 'bypass', thinking: 'Off', onFrame: f => frames.push(f) });
return { frames, seconds: (Date.now() - t0) / 1000, last: frames.at(-1) };
};
const textOf = frames => frames.filter(f => f.type === 'delta' && !f.noteReason).map(f => f.text).join('').trim();
const noteOf = frames => frames.find(f => f.type === 'delta' && f.noteReason === 'compacted')?.text?.trim() ?? null;
bridge = createAgentBridge({
dataDir: join(scratch, 'agent'), env: process.env, startTimeoutMs: 120_000, log: l => console.log(' bridge:', l),
localInstances: () => [{ id: 'e2e', label: 'Gyro-S', baseUrl: `${base}/v1`, modelId: model, context: WINDOW, slots: 1, key: '' }], connections: () => [],
});
mkdirSync(join(scratch, 'agent'), { recursive: true });
// ---------------------------------------------------------------------------------------------------------------- A
{
// Grow a log until the engine itself counts about 60K tokens.
let chars = 150_000, tokens = await count(logText(chars));
while (tokens && tokens < 58_000 && chars < 400_000) { chars = Math.round(chars * (60_000 / tokens)); tokens = await count(logText(chars)); }
const t = await turnsOf(logText(chars) + '\n\nWhat is the most common status?');
const refused = t.last.type === 'error' && t.last.code === 'CONTEXT_MESSAGE_TOO_LARGE';
note('A too large is refused before it is read', refused && t.seconds < 15, `${chars} characters = ${tokens} tokens by the engine; answered in ${t.seconds.toFixed(1)} s; frame ${t.last.type}/${t.last.code ?? '-'}: ${String(t.last.message ?? '').slice(0, 110)}`);
}
// ---------------------------------------------------------------------------------------------------------------- B
const needle = 'NOTE FROM OPS: the vault code is COBALT-27. Do not lose it.';
const timeline = [];
let notes = 0, refusedAny = false, peak = 0;
{
for (let k = 0; k < 5; k++) {
const body = logText(34_000, k * 1000, k === 0 ? needle : '');
const t = await turnsOf(`${body}\n\nReply with the single word OK.`);
const note = noteOf(t.frames);
if (note) notes++;
if (t.last.type === 'error') refusedAny = true;
const used = t.last.contextTokens ?? 0;
peak = Math.max(peak, ...t.frames.filter(f => f.type === 'context').map(f => f.contextTokens), used);
timeline.push(`msg${k + 1}: ${t.last.type}${t.last.code ? '/' + t.last.code : ''} ${t.seconds.toFixed(0)} s context ${used} (${Math.round(100 * used / WINDOW)}%)${note ? ' [' + note.slice(0, 80) + ']' : ''}`);
console.log(' ' + timeline.at(-1));
if (t.last.type === 'error') break;
}
note('B five dense messages (about 14K tokens each) in a 64K window: no error', !refusedAny, `peak context ${peak} of ${WINDOW}; ${notes} compaction note(s) in the chat`);
note('B compaction ran and was announced', notes >= 1, timeline.join(' | '));
}
// ---------------------------------------------------------------------------------------------------------------- C
{
const run = async (line) => { const frames = []; await bridge.compactCommand({ chatId: 'e2e', message: line, workspacePath: scratch, mode: 'bypass', choice: null, onFrame: f => frames.push(f) }); return frames; };
await turnsOf(`${logText(30_000, 90_000)}\n\nReply with the single word OK.`);
const t0 = Date.now();
const first = await run('/compact');
const said = first.find(f => f.type === 'delta');
const ok = said?.key === 'Older messages were summarized to make room. The chat was {0}% full and is now {1}% full.' && said.values[0] > said.values[1];
note('C /compact summarises and states the figures', ok, `${((Date.now() - t0) / 1000).toFixed(0)} s: ${said?.text}`);
const second = await run('/compact');
const again = second.find(f => f.type === 'delta')?.text ?? '';
note('C a second /compact is answered in plain words', /summarized|nothing|could not/i.test(again) && !/\{|400|invalid_request/.test(again), again);
}
// ---------------------------------------------------------------------------------------------------------------- D
{
const t = await turnsOf('What was the vault code in the very first message? Answer with the code only.');
const answer = textOf(t.frames);
console.log(`D needle after compaction: "${answer.slice(0, 80)}" (${t.last.type})`);
rows.push({ id: 'D the needle', ok: /COBALT-27/.test(answer), detail: `reported only: answer "${answer.slice(0, 80)}"` });
}
console.log('\n' + rows.map(r => `${r.ok ? 'PASS' : r.id.startsWith('D') ? 'INFO' : 'FAIL'} ${r.id}${r.id.startsWith('D') ? ` ${r.detail}` : ''}`).join('\n'));
const failed = rows.filter(r => !r.ok && !r.id.startsWith('D')).length;
writeFileSync(join(scratch, 'result.json'), JSON.stringify(rows, null, 2));
await finish(failed ? 1 : 0);
+156
View File
@@ -0,0 +1,156 @@
"""French (fr) catalog tooling for the desktop app (#3970).
The French texts live in src/locales/fr.json, fix-fr.json, tune-fr.json and tools-fr.json, next to the other languages.
They are edited by hand (they are the translations, not generated), so a string added to an English catalog later has no
French yet. This script lists and merges those. From apps/host:
python scripts/fr_sync.py missing [out.tsv] write every English string that has no French yet
python scripts/fr_sync.py apply done.tsv merge translations, rewrite the four fr files in English key order
python scripts/fr_sync.py check exit 1 if a French string is missing, empty or breaks a placeholder / tag
The TSV has one string per line: <id><TAB><text>. In the text, a newline is written \\n and a backslash \\\\.
In the "done" file a lone = means "keep the English text as it is" (class names, product names, file names).
The id prefix picks the file: s0001 -> fr.json, fx01 -> fix-fr.json, tune01 -> tune-fr.json, tl01 -> tools-fr.json.
Use "vous", plain short words, a next step, and a no-break space (U+00A0) before : ; ? ! and inside guillemets.
"""
import json
import re
import sys
from pathlib import Path
LOCALES = Path(__file__).resolve().parent.parent / "src" / "locales"
PAIRS = [("en.json", "fr.json"), ("fix-en.json", "fix-fr.json"), ("tune-en.json", "tune-fr.json"), ("tools-en.json", "tools-fr.json")]
SLOT = re.compile(r"\{\d+\}")
TAG = re.compile(r"</?[A-Za-z][^>]*>")
def load(name):
path = LOCALES / name
return json.loads(path.read_text(encoding="utf-8")) if path.exists() else {}
def file_of(identifier):
for en_name, fr_name in PAIRS:
if identifier in load(en_name):
return fr_name
raise SystemExit(f"unknown id {identifier}: it is in no English catalog")
def escape(text):
return text.replace("\\", "\\\\").replace("\n", "\\n")
def unescape(text):
out, i = [], 0
while i < len(text):
if text[i] == "\\" and i + 1 < len(text):
out.append({"n": "\n", "\\": "\\"}.get(text[i + 1], text[i + 1]))
i += 2
else:
out.append(text[i])
i += 1
return "".join(out)
NBSP = " "
def typography(text):
"""French spacing and apostrophes, applied to every translated (not kept) string.
l'image -> l'image with a typographic apostrophe (also before a {0} slot); a no-break space before : ; ? ! % and inside the guillemets."""
text = re.sub(r"(?<=[A-Za-zÀ-ſ])'(?=[A-Za-zÀ-ſ{])", "’", text)
text = re.sub(r"(?<=\S) ([:;?!])", NBSP + r"\1", text)
text = re.sub(r"(?<=[\d}]) %", NBSP + "%", text)
text = re.sub("« ?", "«" + NBSP, text)
text = re.sub(" ?»", NBSP + "»", text)
return text
def problems(identifier, source, text):
found = []
if not text.strip():
found.append("empty")
expected = [slot for slot in SLOT.findall(source) if not (identifier == "s2463" and slot == "{1}")] # the tests allow this one to drop {1} (the plural suffix)
if sorted(expected) != sorted(SLOT.findall(text)):
found.append("placeholders differ")
if sorted(TAG.findall(source)) != sorted(TAG.findall(text)):
found.append("markup differs")
return found
def missing():
todo = {}
for en_name, fr_name in PAIRS:
en, fr = load(en_name), load(fr_name)
for identifier, text in en.items():
if identifier not in fr:
todo[identifier] = text
return todo
def main(argv):
command = argv[1] if len(argv) > 1 else ""
if command == "missing":
todo = missing()
lines = [f"{identifier}\t{escape(text)}" for identifier, text in todo.items()]
if len(argv) > 2:
Path(argv[2]).write_text("\n".join(lines) + ("\n" if lines else ""), encoding="utf-8")
print(f"{len(lines)} strings without French written to {argv[2]}")
else:
print("\n".join(lines))
print(f"{len(lines)} strings without French", file=sys.stderr)
return 0
if command == "apply" and len(argv) > 2:
incoming = {}
for number, line in enumerate(Path(argv[2]).read_text(encoding="utf-8").splitlines(), 1):
if not line.strip():
continue
if "\t" not in line:
raise SystemExit(f"{argv[2]}:{number}: no tab")
identifier, text = line.split("\t", 1)
incoming[identifier.strip()] = text
bad = 0
for en_name, fr_name in PAIRS:
en, fr = load(en_name), load(fr_name)
changed = False
for identifier, source in en.items():
if identifier in incoming:
text = source if incoming[identifier] == "=" else typography(unescape(incoming[identifier]))
for why in problems(identifier, source, text):
print(f"{identifier}: {why}: {source!r} -> {text!r}")
bad += 1
fr[identifier] = text
changed = True
if changed:
ordered = {identifier: fr[identifier] for identifier in en if identifier in fr}
(LOCALES / fr_name).write_text(json.dumps(ordered, ensure_ascii=False, indent=1 if en_name != "en.json" else 2) + "\n", encoding="utf-8")
unknown = [identifier for identifier in incoming if not any(identifier in load(en_name) for en_name, _ in PAIRS)]
for identifier in unknown:
print(f"{identifier}: not in any English catalog")
bad += 1
print(f"applied {len(incoming)} strings, {bad} problems")
return 1 if bad else 0
if command == "check":
bad = 0
for en_name, fr_name in PAIRS:
en, fr = load(en_name), load(fr_name)
for identifier, source in en.items():
if identifier not in fr:
print(f"{fr_name}: {identifier} has no French yet: {source!r}")
bad += 1
continue
for why in problems(identifier, source, fr[identifier]):
print(f"{fr_name}: {identifier}: {why}")
bad += 1
for identifier in fr:
if identifier not in en:
print(f"{fr_name}: {identifier} is not in {en_name}")
bad += 1
print("French catalogs complete" if not bad else f"{bad} problems")
return 1 if bad else 0
print(__doc__)
return 2
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env node
/**
* Screenshots of the app screens in one interface language, from a hermetic preview host (simulated hardware and models,
* scratch profile, no engine, no weights). The browser is a throw-away headless Chrome profile.
*
* node scripts/language-shots.mjs <lang> <outDir> [route ...] (needs `npm run build` first)
* example: node scripts/language-shots.mjs fr C:/Users/me/Desktop/fr-screens models settings setup
*
* Env: CHROME (path to Chrome/Edge), WIDTH (default 1280), HEIGHT (default 800).
*/
import { spawn } from 'node:child_process';
import { chromium } from 'playwright-core';
import { existsSync, mkdirSync, mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
const root = resolve(import.meta.dirname, '..');
const [, , language = 'fr', outDir = 'language-shots', ...routes] = process.argv;
const screens = routes.length ? routes : ['models', 'settings', 'setup'];
if (!existsSync(join(root, 'dist/index.html'))) { console.error('Run npm run build first.'); process.exit(1); }
const chrome = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', 'C:/Program Files (x86)/Microsoft/Edge/Application/msedge.exe', '/usr/bin/google-chrome', '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find(existsSync);
if (!chrome) { console.error('No Chrome or Edge found; set CHROME.'); process.exit(1); }
const home = mkdtempSync(join(tmpdir(), 'strata-language-shots-'));
const hostPort = 8830 + Math.floor(Math.random() * 40), webPort = hostPort + 100;
const scratch = { STRATA_HOME: home, STRATA_MODELS_DIR: join(home, 'models'), STRATA_WORKSPACE: home, STRATA_REAL_ENGINE: '0', STRATA_NATIVE_REPO: join(home, 'no-native-installation'),
STRATA_SETTINGS_FILE: join(home, 'app-settings.json'), STRATA_COMPONENTS_DIR: join(home, 'components'), LOCALAPPDATA: join(home, 'Local'), APPDATA: join(home, 'Roaming'),
XDG_DATA_HOME: join(home, 'xdg-data'), XDG_STATE_HOME: join(home, 'xdg-state'), XDG_CONFIG_HOME: join(home, 'xdg-config'), STRATA_PORT: String(hostPort) };
const children = [];
const run = (args, extra = {}) => { const child = spawn(process.execPath, args, { cwd: root, env: { ...process.env, ...scratch, ...extra }, stdio: 'ignore' }); children.push(child); return child; };
const stop = () => { for (const child of children) { try { child.kill(); } catch {} } try { rmSync(home, { recursive: true, force: true }); } catch {} };
process.on('exit', stop);
run(['server/index.mjs']);
for (let attempt = 0; ; attempt++) {
try { if ((await fetch(`http://127.0.0.1:${hostPort}/api/health`)).ok) break; } catch {}
if (attempt > 200) { console.error('The preview host did not start.'); process.exit(1); }
await new Promise(done => setTimeout(done, 150));
}
run(['tests/fixtures/onboarding-ui-host.mjs'], { STRATA_UI_PREVIEW: '1', STRATA_ONBOARDING_UPSTREAM: `http://127.0.0.1:${hostPort}`, STRATA_ONBOARDING_PORT: String(webPort), STRATA_ONBOARDING_START: 'ready', STRATA_ONBOARDING_SCENARIO: 'telemetry', STRATA_ONBOARDING_PREVIEW: '1' });
for (let attempt = 0; ; attempt++) {
try { if ((await fetch(`http://127.0.0.1:${webPort}/`)).ok) break; } catch {}
if (attempt > 200) { console.error('The preview page did not start.'); process.exit(1); }
await new Promise(done => setTimeout(done, 150));
}
mkdirSync(resolve(outDir), { recursive: true });
const browser = await chromium.launch({ executablePath: chrome, headless: true, args: ['--no-sandbox', '--disable-gpu', '--force-device-scale-factor=1'] });
const page = await browser.newPage({ viewport: { width: Number(process.env.WIDTH ?? 1280), height: Number(process.env.HEIGHT ?? 800) }, deviceScaleFactor: 1 });
await page.addInitScript(code => { try { localStorage.setItem('strata.prefs.v2', JSON.stringify({ language: code, onboardingSeen: true })); } catch {} }, language);
const problems = [];
page.on('pageerror', error => problems.push(`pageerror: ${error.message}`));
for (const screen of screens) {
await page.goto(`http://127.0.0.1:${webPort}/#/${screen}`, { waitUntil: 'load' });
await page.reload({ waitUntil: 'load' });
await page.waitForTimeout(2500);
const file = join(resolve(outDir), `${language}-${screen}.png`);
await page.screenshot({ path: file });
console.log(`${screen}: ${file} (lang=${await page.evaluate(() => document.documentElement.lang)})`);
}
await browser.close();
if (problems.length) console.log(problems.join('\n'));
stop();
process.exit(0);
+385
View File
@@ -0,0 +1,385 @@
"""Writes src/locales/fix-<lang>.json: the interface strings that had no key in the locale files (#3479, #3825) and the strings of the 0.2.4.3 bug fixes.
Run from apps/host: python scripts/make-fix-locales.py
The keys are fx01.. in this order; fix-en.json is the English source. Placeholders ({0}, {1}) must be kept in every language."""
import json
from pathlib import Path
EN = [
"will stop running and be removed.",
"Downloaded, not loaded. Open Models to load it.",
"{0} lines added, {1} removed",
"Edited {0} files",
"Drop files to add them",
"Pictures and text files",
"Keep goal",
"Remove goal",
"The assistant stops working towards it. Your chat and files stay as they are.",
"This model can read pictures, but the engine installed here does not include the picture reader yet, so Read pictures is off. Update the engine in Settings → About to turn it on.",
"Showing the values of your tuning '{0}'.",
"Your tuning '{0}' sets",
"was {0}",
"· {0} RAM",
"The message was not cancelled",
"Extra settings for the program",
"Draft it with AI",
"Thinking time",
"Writing time",
"tokens/s",
"% of context",
"This page asks for files from the internet, which the preview does not load, so it may look incomplete. View code shows exactly what was written.",
"Keep A",
"Keep B",
"Took",
"Variation",
"Changed",
"Nothing",
"Both versions are ready. Compare A and B, then keep one.",
"The assistant is still answering",
"Send your message when it has finished.",
"The message was not queued",
"Scroll to the latest message",
"Remove task",
"Start a new conversation; keep this one",
"Have the assistant write a plan first; nothing runs until you approve it",
"Search the web, read the pages and answer with sources",
"Copied",
"Response",
"Message",
"PDF added",
"Select the text and copy it with Ctrl+C.",
"{0} is in this chat. Ask about it in your next message.",
]
T = {
"zh": [
"将停止运行并被移除。",
"已下载,尚未加载。打开“模型”页面来加载它。",
"新增 {0} 行,删除 {1} 行",
"已编辑 {0} 个文件",
"拖放文件即可添加",
"图片和文本文件",
"保留目标",
"移除目标",
"助手将不再朝这个目标工作。你的对话和文件保持不变。",
"此模型可以读取图片,但这里安装的引擎还不包含图片读取器,所以“读取图片”已关闭。请在“设置 → 关于”中更新引擎后再开启。",
"显示你的调优“{0}”的取值。",
"你的调优“{0}”设置了",
"原为 {0}",
"· {0} 内存",
"消息没有被取消",
"程序的额外设置",
"用 AI 起草",
"思考时间",
"写作时间",
"词元/秒",
"% 的上下文",
"此页面需要从互联网加载文件,预览不会加载这些文件,所以看起来可能不完整。“查看代码”会显示实际写入的内容。",
"保留 A",
"保留 B",
"用时",
"波动",
"已更改",
"无",
"两个版本都已就绪。比较 A 和 B,然后保留其中一个。",
"助手仍在回答",
"等它回答完再发送你的消息。",
"消息没有进入队列",
"滚动到最新消息",
"移除任务",
"开始新对话;保留当前这个",
"让助手先写一份计划;在你批准之前不会运行任何内容",
"搜索网络、阅读网页,并附上来源作答",
"已复制",
"回复",
"消息",
"已添加 PDF",
"请选中文字,然后按 Ctrl+C 复制。",
"{0} 已在此对话中。请在下一条消息里询问它。",
],
"ja": [
"は停止され、削除されます。",
"ダウンロード済みですが、読み込まれていません。「モデル」を開いて読み込んでください。",
"{0} 行追加、{1} 行削除",
"{0} 個のファイルを編集しました",
"ファイルをドロップして追加",
"画像とテキストファイル",
"目標を残す",
"目標を削除",
"アシスタントはこの目標に向けた作業をやめます。チャットとファイルはそのまま残ります。",
"このモデルは画像を読めますが、ここにインストールされているエンジンにはまだ画像リーダーが含まれていないため、「画像を読む」はオフです。設定 → 情報 でエンジンを更新するとオンにできます。",
"チューニング「{0}」の値を表示しています。",
"チューニング「{0}」が設定する項目",
"変更前: {0}",
"· {0} RAM",
"メッセージはキャンセルされませんでした",
"プログラムの追加設定",
"AI で下書きを作成",
"思考時間",
"執筆時間",
"トークン/秒",
"% のコンテキスト",
"このページはインターネット上のファイルを必要としますが、プレビューでは読み込まれないため、表示が不完全になることがあります。「コードを表示」で書かれた内容をそのまま確認できます。",
"A を残す",
"B を残す",
"所要時間",
"ばらつき",
"変更点",
"なし",
"両方のバージョンの準備ができました。A と B を比べて、どちらかを残してください。",
"アシスタントはまだ回答中です",
"回答が終わってからメッセージを送ってください。",
"メッセージは待機列に入りませんでした",
"最新のメッセージへスクロール",
"タスクを削除",
"新しい会話を始めます。今の会話は残ります",
"アシスタントにまず計画を書かせます。承認するまで何も実行されません",
"ウェブを検索し、ページを読み、出典付きで回答します",
"コピーしました",
"返信",
"メッセージ",
"PDF を追加しました",
"テキストを選択して Ctrl+C でコピーしてください。",
"{0} をこのチャットに追加しました。次のメッセージで質問してください。",
],
"de": [
"wird nicht mehr ausgeführt und entfernt.",
"Heruntergeladen, aber nicht geladen. Öffne „Modelle“, um es zu laden.",
"{0} Zeilen hinzugefügt, {1} entfernt",
"{0} Dateien bearbeitet",
"Dateien hier ablegen, um sie hinzuzufügen",
"Bilder und Textdateien",
"Ziel behalten",
"Ziel entfernen",
"Der Assistent arbeitet nicht mehr darauf hin. Dein Chat und deine Dateien bleiben unverändert.",
"Dieses Modell kann Bilder lesen, aber die hier installierte Engine enthält den Bildleser noch nicht, deshalb ist „Bilder lesen“ aus. Aktualisiere die Engine unter Einstellungen → Info, um es einzuschalten.",
"Zeigt die Werte deiner Abstimmung „{0}“.",
"Deine Abstimmung „{0}“ setzt",
"war {0}",
"· {0} RAM",
"Die Nachricht wurde nicht abgebrochen",
"Zusätzliche Einstellungen für das Programm",
"Mit KI entwerfen",
"Denkzeit",
"Schreibzeit",
"Token/s",
"% des Kontexts",
"Diese Seite lädt Dateien aus dem Internet, was die Vorschau nicht tut. Sie kann deshalb unvollständig aussehen. „Code anzeigen“ zeigt genau, was geschrieben wurde.",
"A behalten",
"B behalten",
"Dauer",
"Schwankung",
"Geändert",
"Nichts",
"Beide Versionen sind fertig. Vergleiche A und B und behalte eine.",
"Der Assistent antwortet noch",
"Sende deine Nachricht, wenn er fertig ist.",
"Die Nachricht wurde nicht eingereiht",
"Zur neuesten Nachricht scrollen",
"Aufgabe entfernen",
"Neue Unterhaltung beginnen; diese bleibt erhalten",
"Der Assistent schreibt zuerst einen Plan; nichts läuft, bis du ihn freigibst",
"Im Web suchen, die Seiten lesen und mit Quellen antworten",
"Kopiert",
"Antwort",
"Nachricht",
"PDF hinzugefügt",
"Markiere den Text und kopiere ihn mit Strg+C.",
"{0} ist in diesem Chat. Frag in deiner nächsten Nachricht danach.",
],
"es": [
"dejará de ejecutarse y se eliminará.",
"Descargado, pero no cargado. Abre Modelos para cargarlo.",
"{0} líneas añadidas, {1} eliminadas",
"{0} archivos editados",
"Suelta los archivos para añadirlos",
"Imágenes y archivos de texto",
"Mantener el objetivo",
"Quitar el objetivo",
"El asistente deja de trabajar en ello. Tu chat y tus archivos se quedan como están.",
"Este modelo puede leer imágenes, pero el motor instalado aquí aún no incluye el lector de imágenes, así que Leer imágenes está desactivado. Actualiza el motor en Ajustes → Acerca de para activarlo.",
"Se muestran los valores de tu ajuste «{0}».",
"Tu ajuste «{0}» establece",
"antes {0}",
"· {0} de RAM",
"El mensaje no se canceló",
"Ajustes adicionales del programa",
"Redactar con IA",
"Tiempo de pensar",
"Tiempo de escribir",
"tokens/s",
"% del contexto",
"Esta página pide archivos de internet, que la vista previa no carga, así que puede verse incompleta. Ver código muestra exactamente lo que se escribió.",
"Mantener A",
"Mantener B",
"Tardó",
"Variación",
"Cambiado",
"Nada",
"Las dos versiones están listas. Compara A y B y quédate con una.",
"El asistente sigue respondiendo",
"Envía tu mensaje cuando termine.",
"El mensaje no se puso en cola",
"Ir al último mensaje",
"Quitar tarea",
"Empieza una conversación nueva; esta se conserva",
"El asistente escribe primero un plan; no se ejecuta nada hasta que lo apruebes",
"Busca en la web, lee las páginas y responde con fuentes",
"Copiado",
"Respuesta",
"Mensaje",
"PDF añadido",
"Selecciona el texto y cópialo con Ctrl+C.",
"{0} está en este chat. Pregunta por él en tu próximo mensaje.",
],
"ru": [
"перестанет работать и будет удалена.",
"Скачана, но не загружена. Откройте «Модели», чтобы загрузить её.",
"Добавлено строк: {0}, удалено: {1}",
"Изменено файлов: {0}",
"Перетащите файлы сюда, чтобы добавить",
"Картинки и текстовые файлы",
"Оставить цель",
"Удалить цель",
"Ассистент перестанет работать над ней. Ваш чат и файлы останутся как есть.",
"Эта модель умеет читать картинки, но установленный здесь движок пока не содержит модуль чтения картинок, поэтому «Читать картинки» выключено. Обновите движок в разделе «Настройки → О программе», чтобы включить.",
"Показаны значения вашей настройки «{0}».",
"Ваша настройка «{0}» задаёт",
"было {0}",
"· {0} ОЗУ",
"Сообщение не было отменено",
"Дополнительные настройки программы",
"Набросать с помощью ИИ",
"Время размышления",
"Время написания",
"токенов/с",
"% контекста",
"Эта страница запрашивает файлы из интернета, а предпросмотр их не загружает, поэтому она может выглядеть неполной. «Показать код» показывает, что именно было написано.",
"Оставить A",
"Оставить B",
"Заняло",
"Разброс",
"Изменено",
"Ничего",
"Обе версии готовы. Сравните A и B и оставьте одну.",
"Ассистент ещё отвечает",
"Отправьте сообщение, когда он закончит.",
"Сообщение не поставлено в очередь",
"К последнему сообщению",
"Удалить задачу",
"Начать новый разговор; этот сохраняется",
"Ассистент сначала напишет план; ничего не запустится, пока вы его не одобрите",
"Искать в интернете, читать страницы и отвечать со ссылками на источники",
"Скопировано",
"Ответ",
"Сообщение",
"PDF добавлен",
"Выделите текст и скопируйте его клавишами Ctrl+C.",
"{0} добавлен в этот чат. Спросите о нём в следующем сообщении.",
],
"pt": [
"vai parar de funcionar e será removido.",
"Baixado, mas não carregado. Abra Modelos para carregá-lo.",
"{0} linhas adicionadas, {1} removidas",
"{0} arquivos editados",
"Solte os arquivos para adicioná-los",
"Imagens e arquivos de texto",
"Manter a meta",
"Remover a meta",
"O assistente deixa de trabalhar nela. Seu chat e seus arquivos continuam como estão.",
"Este modelo consegue ler imagens, mas o motor instalado aqui ainda não inclui o leitor de imagens, então Ler imagens está desligado. Atualize o motor em Configurações → Sobre para ligar.",
"Mostrando os valores do seu ajuste \"{0}\".",
"Seu ajuste \"{0}\" define",
"era {0}",
"· {0} de RAM",
"A mensagem não foi cancelada",
"Configurações extras do programa",
"Rascunhar com IA",
"Tempo de raciocínio",
"Tempo de escrita",
"tokens/s",
"% do contexto",
"Esta página pede arquivos da internet, que a visualização não carrega, então pode parecer incompleta. Ver código mostra exatamente o que foi escrito.",
"Manter A",
"Manter B",
"Levou",
"Variação",
"Alterado",
"Nada",
"As duas versões estão prontas. Compare A e B e mantenha uma.",
"O assistente ainda está respondendo",
"Envie sua mensagem quando ele terminar.",
"A mensagem não entrou na fila",
"Ir para a última mensagem",
"Remover tarefa",
"Começar uma nova conversa; esta continua salva",
"O assistente escreve um plano primeiro; nada roda até você aprovar",
"Pesquisar na web, ler as páginas e responder com fontes",
"Copiado",
"Resposta",
"Mensagem",
"PDF adicionado",
"Selecione o texto e copie com Ctrl+C.",
"{0} está neste chat. Pergunte sobre ele na próxima mensagem.",
],
"ko": [
"실행이 중지되고 제거됩니다.",
"다운로드했지만 아직 불러오지 않았습니다. 모델 화면을 열어 불러오세요.",
"{0}줄 추가, {1}줄 삭제",
"파일 {0}개 편집함",
"파일을 놓아서 추가하세요",
"그림과 텍스트 파일",
"목표 유지",
"목표 제거",
"어시스턴트가 이 목표를 향한 작업을 멈춥니다. 대화와 파일은 그대로 남습니다.",
"이 모델은 그림을 읽을 수 있지만, 여기에 설치된 엔진에는 아직 그림 읽기 기능이 없어서 '그림 읽기'가 꺼져 있습니다. 설정 → 정보에서 엔진을 업데이트하면 켤 수 있습니다.",
"튜닝 '{0}'의 값을 보여 줍니다.",
"튜닝 '{0}'이(가) 설정하는 항목",
"이전 값 {0}",
"· RAM {0}",
"메시지가 취소되지 않았습니다",
"프로그램 추가 설정",
"AI로 초안 작성",
"생각한 시간",
"작성한 시간",
"토큰/초",
"컨텍스트 %",
"이 페이지는 인터넷의 파일을 불러와야 하는데 미리보기는 이를 불러오지 않으므로 불완전하게 보일 수 있습니다. '코드 보기'에서 작성된 내용을 그대로 볼 수 있습니다.",
"A 유지",
"B 유지",
"걸린 시간",
"편차",
"변경됨",
"없음",
"두 버전이 모두 준비되었습니다. A와 B를 비교한 뒤 하나를 유지하세요.",
"어시스턴트가 아직 답하는 중입니다",
"답이 끝나면 메시지를 보내세요.",
"메시지가 대기열에 들어가지 않았습니다",
"최신 메시지로 이동",
"작업 제거",
"새 대화를 시작합니다. 이 대화는 그대로 둡니다",
"어시스턴트가 먼저 계획을 작성합니다. 승인하기 전에는 아무것도 실행되지 않습니다",
"웹을 검색하고 페이지를 읽어 출처와 함께 답합니다",
"복사했습니다",
"답변",
"메시지",
"PDF를 추가했습니다",
"텍스트를 선택한 뒤 Ctrl+C로 복사하세요.",
"{0}이(가) 이 채팅에 있습니다. 다음 메시지에서 물어보세요.",
],
}
here = Path(__file__).resolve().parent.parent / "locales" if False else Path(__file__).resolve().parent.parent / "src" / "locales"
def write(lang, values):
assert len(values) == len(EN), (lang, len(values), len(EN))
for source, text in zip(EN, values):
assert sorted(__import__("re").findall(r"\{\d+\}", source)) == sorted(__import__("re").findall(r"\{\d+\}", text)), (lang, source, text)
data = {f"fx{i + 1:02d}": text for i, text in enumerate(values)}
(here / f"fix-{lang}.json").write_text(json.dumps(data, ensure_ascii=False, indent=1) + "\n", encoding="utf-8")
write("en", EN)
for lang, values in T.items():
write(lang, values)
print("written", len(EN), "strings in", len(T) + 1, "languages")
+365
View File
@@ -0,0 +1,365 @@
"""Writes src/locales/gh-<lang>.json: the interface strings of the 0.2.7 GitHub items done in f-027-gh-app (the Monitor's Power card #3628,
approval notifications and the tray #3477, the finished-task sound #3586, the pinned port #3402, the interface size #3970).
Run from apps/host: python scripts/make-gh-locales.py
The keys are gh01.. in this order; gh-en.json is the English source. Placeholders ({0}, {1}) must be kept in every language.
A separate file keeps this work from colliding with the other catalogs (the same reason tune-, tools- and fix- exist)."""
import json
import re
from pathlib import Path
EN = [
"Energy since start",
"Cost since start",
"GPU fan",
"GPU clock",
"CPU temperature",
"SSD temperature",
"Electricity price",
"Price of one kilowatt-hour. The Monitor then shows what your graphics card has cost since Strata started. Leave it empty to hide the cost.",
"Price per kWh",
"Sound when work finishes",
"Plays a short sound when an answer or task that took a while is finished, or when Strata is in the background.",
"Play sound",
"Notify me when Strata needs an answer",
"In Manual mode, shows a desktop notification when the assistant waits for your approval and Strata is in the background.",
"Strata needs your answer",
"The assistant wants to use {0}. Open Strata to approve or deny.",
"a tool",
"Minimise to tray",
"Closing the window keeps Strata running in the tray, next to the clock. Choose Quit in the tray menu to stop it. Applies the next time Strata starts.",
"App port",
"The port Strata’s window and local address use. Leave it empty to let Strata choose (47831 when it is free). Applies the next time Strata starts.",
"Port (empty = automatic)",
"Using port {0} now.",
"Strata could not use its port",
"Port {0} is used by another program, so Strata started on port {1}. Chats and settings saved on port {0} are not shown until Strata can use it again. Close the other program and restart Strata.",
"Choose a port between 1024 and 65535, or leave it empty.",
"Makes all text, buttons and menus in Strata larger or smaller.",
"Energy and temperature readings",
"per kWh, in {0}",
"per kWh",
"Change currency",
"Automatic ({0})",
"Currency",
"less than {0}",
]
T = {
"zh": [
"启动以来的能耗",
"启动以来的费用",
"GPU 风扇",
"GPU 频率",
"CPU 温度",
"SSD 温度",
"电价",
"每千瓦时的价格。监控页会据此显示 Strata 启动以来显卡用电的费用。留空则不显示费用。",
"每千瓦时价格",
"工作完成时的提示音",
"回答或耗时较长的任务完成时,或 Strata 在后台时,播放一段简短的提示音。",
"播放提示音",
"Strata 需要您答复时通知我",
"在手动模式下,当助手等待您批准而 Strata 在后台时,显示桌面通知。",
"Strata 需要您的答复",
"助手想使用 {0}。打开 Strata 以批准或拒绝。",
"一个工具",
"最小化到托盘",
"关闭窗口后,Strata 会继续在托盘(时钟旁边)运行。要停止它,请在托盘菜单中选择“退出”。下次启动 Strata 时生效。",
"应用端口",
"Strata 窗口和本地地址使用的端口。留空则由 Strata 自行选择(空闲时为 47831)。下次启动 Strata 时生效。",
"端口(留空 = 自动)",
"当前使用端口 {0}。",
"Strata 无法使用它的端口",
"端口 {0} 已被其他程序占用,因此 Strata 改用端口 {1} 启动。在端口 {0} 上保存的聊天和设置在 Strata 能再次使用该端口之前不会显示。请关闭其他程序并重新启动 Strata。",
"请选择 1024 到 65535 之间的端口,或留空。",
"让 Strata 中的所有文字、按钮和菜单变大或变小。",
"能耗和温度读数",
"每千瓦时,币种 {0}",
"每千瓦时",
"更改货币",
"自动({0})",
"货币",
"不足 {0}",
],
"ja": [
"起動後の消費電力量",
"起動後の電気代",
"GPU ファン",
"GPU クロック",
"CPU 温度",
"SSD 温度",
"電気料金",
"1キロワット時あたりの料金です。モニターに、Strata の起動後にグラフィックスカードが使った電気代が表示されます。空欄のままなら電気代は表示されません。",
"1 kWh あたりの料金",
"作業完了時の音",
"回答や時間のかかった作業が終わったとき、または Strata がバックグラウンドにあるときに、短い音を鳴らします。",
"音を鳴らす",
"Strata の返答が必要なときに通知する",
"手動モードで、アシスタントが承認を待っていて Strata がバックグラウンドにあるときに、デスクトップ通知を表示します。",
"Strata があなたの返答を待っています",
"アシスタントが {0} を使おうとしています。Strata を開いて承認または拒否してください。",
"ツール",
"トレイに最小化",
"ウィンドウを閉じても、Strata は時計の横のトレイで動き続けます。止めるには、トレイのメニューで「終了」を選びます。次に Strata を起動したときから有効になります。",
"アプリのポート",
"Strata のウィンドウとローカルアドレスが使うポートです。空欄なら Strata が選びます(空いていれば 47831)。次に Strata を起動したときから有効になります。",
"ポート(空欄 = 自動)",
"現在のポートは {0} です。",
"Strata は指定のポートを使えませんでした",
"ポート {0} は別のプログラムが使っているため、Strata はポート {1} で起動しました。ポート {0} に保存されたチャットと設定は、Strata が再びそのポートを使えるようになるまで表示されません。他のプログラムを閉じて Strata を再起動してください。",
"1024 から 65535 の間のポートを選ぶか、空欄にしてください。",
"Strata のすべての文字、ボタン、メニューを大きくしたり小さくしたりします。",
"消費電力量と温度の値",
"1 kWh あたり、通貨は {0}",
"1 kWh あたり",
"通貨を変更",
"自動({0})",
"通貨",
"{0} 未満",
],
"de": [
"Energie seit dem Start",
"Kosten seit dem Start",
"GPU-Lüfter",
"GPU-Takt",
"CPU-Temperatur",
"SSD-Temperatur",
"Strompreis",
"Preis einer Kilowattstunde. Der Monitor zeigt dann, was Ihre Grafikkarte seit dem Start von Strata gekostet hat. Leer lassen, um die Kosten auszublenden.",
"Preis pro kWh",
"Ton, wenn Arbeit fertig ist",
"Spielt einen kurzen Ton, wenn eine Antwort oder Aufgabe, die länger gedauert hat, fertig ist oder wenn Strata im Hintergrund läuft.",
"Ton abspielen",
"Benachrichtigen, wenn Strata eine Antwort braucht",
"Zeigt im manuellen Modus eine Desktop-Benachrichtigung, wenn der Assistent auf Ihre Freigabe wartet und Strata im Hintergrund läuft.",
"Strata braucht Ihre Antwort",
"Der Assistent möchte {0} verwenden. Öffnen Sie Strata, um zu erlauben oder abzulehnen.",
"ein Werkzeug",
"In den Infobereich minimieren",
"Beim Schließen des Fensters läuft Strata im Infobereich neben der Uhr weiter. Wählen Sie im Menü des Symbols „Beenden“, um es zu stoppen. Gilt ab dem nächsten Start von Strata.",
"App-Port",
"Der Port, den das Fenster und die lokale Adresse von Strata verwenden. Leer lassen, damit Strata ihn wählt (47831, wenn frei). Gilt ab dem nächsten Start von Strata.",
"Port (leer = automatisch)",
"Aktuell wird Port {0} verwendet.",
"Strata konnte seinen Port nicht verwenden",
"Port {0} wird von einem anderen Programm verwendet, daher wurde Strata auf Port {1} gestartet. Unter Port {0} gespeicherte Chats und Einstellungen werden nicht angezeigt, bis Strata ihn wieder verwenden kann. Schließen Sie das andere Programm und starten Sie Strata neu.",
"Wählen Sie einen Port zwischen 1024 und 65535 oder lassen Sie das Feld leer.",
"Macht alle Texte, Schaltflächen und Menüs in Strata größer oder kleiner.",
"Energie- und Temperaturwerte",
"pro kWh, in {0}",
"pro kWh",
"Währung ändern",
"Automatisch ({0})",
"Währung",
"weniger als {0}",
],
"es": [
"Energía desde el inicio",
"Coste desde el inicio",
"Ventilador de la GPU",
"Reloj de la GPU",
"Temperatura de la CPU",
"Temperatura del SSD",
"Precio de la electricidad",
"Precio de un kilovatio hora. El Monitor mostrará entonces lo que ha costado tu tarjeta gráfica desde que se inició Strata. Déjalo vacío para ocultar el coste.",
"Precio por kWh",
"Sonido al terminar el trabajo",
"Reproduce un sonido breve cuando termina una respuesta o tarea que tardó un rato, o cuando Strata está en segundo plano.",
"Reproducir sonido",
"Avisarme cuando Strata necesite una respuesta",
"En el modo manual, muestra una notificación de escritorio cuando el asistente espera tu aprobación y Strata está en segundo plano.",
"Strata necesita tu respuesta",
"El asistente quiere usar {0}. Abre Strata para aprobar o denegar.",
"una herramienta",
"Minimizar a la bandeja",
"Al cerrar la ventana, Strata sigue funcionando en la bandeja, junto al reloj. Elige Salir en el menú de la bandeja para detenerlo. Se aplica la próxima vez que se inicie Strata.",
"Puerto de la aplicación",
"El puerto que usan la ventana y la dirección local de Strata. Déjalo vacío para que Strata elija (47831 si está libre). Se aplica la próxima vez que se inicie Strata.",
"Puerto (vacío = automático)",
"Ahora se usa el puerto {0}.",
"Strata no pudo usar su puerto",
"El puerto {0} lo usa otro programa, así que Strata se inició en el puerto {1}. Los chats y ajustes guardados en el puerto {0} no se muestran hasta que Strata pueda volver a usarlo. Cierra el otro programa y reinicia Strata.",
"Elige un puerto entre 1024 y 65535, o déjalo vacío.",
"Hace más grandes o más pequeños todos los textos, botones y menús de Strata.",
"Lecturas de energía y temperatura",
"por kWh, en {0}",
"por kWh",
"Cambiar moneda",
"Automática ({0})",
"Moneda",
"menos de {0}",
],
"ru": [
"Энергия с запуска",
"Стоимость с запуска",
"Вентилятор GPU",
"Частота GPU",
"Температура CPU",
"Температура SSD",
"Цена электричества",
"Цена одного киловатт-часа. Тогда монитор покажет, во сколько видеокарта обошлась с запуска Strata. Оставьте поле пустым, чтобы скрыть стоимость.",
"Цена за кВт·ч",
"Звук по окончании работы",
"Проигрывает короткий звук, когда закончился долгий ответ или задача, а также когда Strata работает в фоне.",
"Проиграть звук",
"Уведомлять, когда Strata нужен ответ",
"В ручном режиме показывает уведомление на рабочем столе, когда помощник ждёт вашего подтверждения, а Strata работает в фоне.",
"Strata нужен ваш ответ",
"Помощник хочет использовать {0}. Откройте Strata, чтобы разрешить или отклонить.",
"инструмент",
"Сворачивать в трей",
"При закрытии окна Strata продолжает работать в трее рядом с часами. Чтобы остановить её, выберите «Выход» в меню значка. Действует со следующего запуска Strata.",
"Порт приложения",
"Порт, который используют окно и локальный адрес Strata. Оставьте поле пустым, чтобы Strata выбрала сама (47831, если свободен). Действует со следующего запуска Strata.",
"Порт (пусто = автоматически)",
"Сейчас используется порт {0}.",
"Strata не смогла использовать свой порт",
"Порт {0} занят другой программой, поэтому Strata запустилась на порту {1}. Чаты и настройки, сохранённые на порту {0}, не показываются, пока Strata не сможет снова его использовать. Закройте другую программу и перезапустите Strata.",
"Выберите порт от 1024 до 65535 или оставьте поле пустым.",
"Увеличивает или уменьшает весь текст, кнопки и меню в Strata.",
"Показания энергии и температуры",
"за кВт·ч, в валюте {0}",
"за кВт·ч",
"Сменить валюту",
"Автоматически ({0})",
"Валюта",
"меньше {0}",
],
"pt": [
"Energia desde o início",
"Custo desde o início",
"Ventoinha da GPU",
"Clock da GPU",
"Temperatura da CPU",
"Temperatura do SSD",
"Preço da eletricidade",
"Preço de um quilowatt-hora. O Monitor passa a mostrar quanto a sua placa de vídeo custou desde que o Strata iniciou. Deixe vazio para ocultar o custo.",
"Preço por kWh",
"Som ao terminar o trabalho",
"Toca um som curto quando uma resposta ou tarefa que demorou termina, ou quando o Strata está em segundo plano.",
"Tocar som",
"Avisar quando o Strata precisar de uma resposta",
"No modo manual, mostra uma notificação na área de trabalho quando o assistente espera sua aprovação e o Strata está em segundo plano.",
"O Strata precisa da sua resposta",
"O assistente quer usar {0}. Abra o Strata para aprovar ou negar.",
"uma ferramenta",
"Minimizar para a bandeja",
"Ao fechar a janela, o Strata continua em execução na bandeja, ao lado do relógio. Escolha Sair no menu da bandeja para encerrá-lo. Vale a partir da próxima vez que o Strata iniciar.",
"Porta do aplicativo",
"A porta usada pela janela e pelo endereço local do Strata. Deixe vazio para o Strata escolher (47831 quando estiver livre). Vale a partir da próxima vez que o Strata iniciar.",
"Porta (vazio = automático)",
"Usando a porta {0} agora.",
"O Strata não conseguiu usar a sua porta",
"A porta {0} está em uso por outro programa, então o Strata iniciou na porta {1}. Os chats e as configurações salvos na porta {0} não aparecem até o Strata poder usá-la de novo. Feche o outro programa e reinicie o Strata.",
"Escolha uma porta entre 1024 e 65535, ou deixe vazio.",
"Deixa todos os textos, botões e menus do Strata maiores ou menores.",
"Leituras de energia e temperatura",
"por kWh, em {0}",
"por kWh",
"Alterar moeda",
"Automática ({0})",
"Moeda",
"menos de {0}",
],
"ko": [
"시작 이후 에너지",
"시작 이후 비용",
"GPU 팬",
"GPU 클럭",
"CPU 온도",
"SSD 온도",
"전기 요금",
"1킬로와트시의 요금입니다. 모니터에 Strata를 시작한 뒤 그래픽 카드에 든 전기 요금이 표시됩니다. 비워 두면 비용을 표시하지 않습니다.",
"kWh당 요금",
"작업이 끝났을 때 소리",
"답변이나 오래 걸린 작업이 끝났을 때, 또는 Strata가 백그라운드에 있을 때 짧은 소리를 재생합니다.",
"소리 재생",
"Strata가 답을 필요로 할 때 알림",
"수동 모드에서 어시스턴트가 승인을 기다리고 Strata가 백그라운드에 있을 때 데스크톱 알림을 표시합니다.",
"Strata가 답을 기다리고 있습니다",
"어시스턴트가 {0}을(를) 사용하려고 합니다. Strata를 열어 승인하거나 거부하세요.",
"도구",
"트레이로 최소화",
"창을 닫아도 Strata는 시계 옆 트레이에서 계속 실행됩니다. 끝내려면 트레이 메뉴에서 '종료'를 선택하세요. 다음에 Strata를 시작할 때부터 적용됩니다.",
"앱 포트",
"Strata의 창과 로컬 주소가 사용하는 포트입니다. 비워 두면 Strata가 고릅니다(비어 있으면 47831). 다음에 Strata를 시작할 때부터 적용됩니다.",
"포트(비움 = 자동)",
"지금은 포트 {0}을(를) 사용 중입니다.",
"Strata가 지정한 포트를 사용할 수 없었습니다",
"포트 {0}을(를) 다른 프로그램이 사용 중이어서 Strata는 포트 {1}에서 시작했습니다. 포트 {0}에 저장된 채팅과 설정은 Strata가 다시 그 포트를 쓸 수 있을 때까지 표시되지 않습니다. 다른 프로그램을 닫고 Strata를 다시 시작하세요.",
"1024에서 65535 사이의 포트를 고르거나 비워 두세요.",
"Strata의 모든 글자, 버튼, 메뉴를 더 크게 또는 더 작게 만듭니다.",
"에너지 및 온도 측정값",
"kWh당, 통화 {0}",
"kWh당",
"통화 변경",
"자동({0})",
"통화",
"{0} 미만",
],
"fr": [
"Énergie depuis le démarrage",
"Coût depuis le démarrage",
"Ventilateur du GPU",
"Fréquence du GPU",
"Température du CPU",
"Température du SSD",
"Prix de l'électricité",
"Prix d'un kilowattheure. Le Moniteur affiche alors ce que votre carte graphique a coûté depuis le démarrage de Strata. Laissez vide pour masquer le coût.",
"Prix par kWh",
"Son à la fin du travail",
"Joue un court son quand une réponse ou une tâche qui a pris du temps est terminée, ou quand Strata est en arrière-plan.",
"Jouer le son",
"Me prévenir quand Strata a besoin d'une réponse",
"En mode manuel, affiche une notification sur le bureau quand l'assistant attend votre accord et que Strata est en arrière-plan.",
"Strata a besoin de votre réponse",
"L'assistant veut utiliser {0}. Ouvrez Strata pour accepter ou refuser.",
"un outil",
"Réduire dans la zone de notification",
"Fermer la fenêtre garde Strata actif dans la zone de notification, près de l'horloge. Choisissez Quitter dans le menu de l'icône pour l'arrêter. S'applique au prochain démarrage de Strata.",
"Port de l'application",
"Le port que la fenêtre et l'adresse locale de Strata utilisent. Laissez vide pour que Strata choisisse (47831 s'il est libre). S'applique au prochain démarrage de Strata.",
"Port (vide = automatique)",
"Port {0} utilisé en ce moment.",
"Strata n'a pas pu utiliser son port",
"Le port {0} est utilisé par un autre programme, donc Strata a démarré sur le port {1}. Les conversations et réglages enregistrés sur le port {0} ne s'affichent pas tant que Strata ne peut pas le réutiliser. Fermez l'autre programme et relancez Strata.",
"Choisissez un port entre 1024 et 65535, ou laissez vide.",
"Agrandit ou réduit tous les textes, boutons et menus de Strata.",
"Relevés d'énergie et de température",
"par kWh, en {0}",
"par kWh",
"Changer de devise",
"Automatique ({0})",
"Devise",
"moins de {0}",
],
}
here = Path(__file__).resolve().parent.parent / "src" / "locales"
NBSP = "\u00a0"
def french(text):
"""The same French spacing as scripts/fr_sync.py: a typographic apostrophe, a no-break space before : ; ? ! % and inside the guillemets."""
text = re.sub(r"(?<=[A-Za-z\u00c0-\u017f])'(?=[A-Za-z\u00c0-\u017f{])", "\u2019", text)
text = re.sub(r"(?<=\S) ([:;?!])", NBSP + r"\1", text)
text = re.sub(r"(?<=[\d}]) %", NBSP + "%", text)
return text
def write(lang, values):
assert len(values) == len(EN), (lang, len(values), len(EN))
for source, text in zip(EN, values):
assert sorted(re.findall(r"\{\d+\}", source)) == sorted(re.findall(r"\{\d+\}", text)), (lang, source, text)
if lang == "fr":
values = [french(v) for v in values]
data = {f"gh{i + 1:02d}": text for i, text in enumerate(values)}
(here / f"gh-{lang}.json").write_text(json.dumps(data, ensure_ascii=False, indent=1) + "\n", encoding="utf-8")
write("en", EN)
for lang, values in T.items():
write(lang, values)
print("written", len(EN), "strings in", len(T) + 1, "languages")
+7 -1
View File
@@ -5,6 +5,12 @@ import {spawn} from 'node:child_process';
import {resolve} from 'node:path';
const root=resolve(import.meta.dirname,'..');
const files=readdirSync(resolve(root,'tests')).filter(name=>name.endsWith('.test.mjs')).sort().map(name=>resolve(root,'tests',name));
const child=spawn(process.execPath,['--test',...files],{cwd:root,stdio:'inherit'});
/* Every test file runs against scratch copies of the per-user places the host writes to (the owner's real profile is never touched). */
import {mkdtempSync} from 'node:fs';
import {tmpdir} from 'node:os';
import {join} from 'node:path';
const scratch=mkdtempSync(join(tmpdir(),'strata-host-suite-profile-'));
const env={...process.env,STRATA_SETTINGS_FILE:join(scratch,'app-settings.json'),STRATA_COMPONENTS_DIR:join(scratch,'components'),LOCALAPPDATA:join(scratch,'Local'),APPDATA:join(scratch,'Roaming'),XDG_DATA_HOME:join(scratch,'xdg-data'),XDG_STATE_HOME:join(scratch,'xdg-state'),XDG_CONFIG_HOME:join(scratch,'xdg-config')};
const child=spawn(process.execPath,['--test',...files],{cwd:root,stdio:'inherit',env});
child.on('error',error=>{console.error(error.message);process.exitCode=1;});
child.on('exit',(code,signal)=>{process.exitCode=code??(signal?1:0);});
+40 -7
View File
@@ -1,5 +1,8 @@
# Agent bridge (Strata chats as DeepSeek Harness sessions)
This page is for developers who work on the app host's agent code. It lists the files, the frames and routes, and the rules the
bridge enforces. Tests are listed at the end.
The Harness is a separate, pinned component (`DSH 0.2.1-alpha.3`, commit `57190202`, MIT). The host finds it through
`STRATA_HARNESS_DIR` (or `<STRATA_COMPONENTS_DIR>/harness`), starts one `dsh --profile web` process on a loopback port,
and talks to its Gateway. Nothing here downloads or builds anything.
@@ -14,11 +17,13 @@ and talks to its Gateway. Nothing here downloads or builds anything.
| `children.mjs` | pure projection of one helper agent's events into its row and its approval card |
| `preset.mjs` | the agent preset (which tools the chat has) |
| `plugins/strata-gate.mjs` | the Manual-mode approval gate (a Harness plugin, loaded by path) |
| `plugins/strata-search.mjs`, `plugins/strata-models.mjs` | the app's web search registered with the Harness; helper agents on a named model |
| `plugins/strata-context.mjs`, `context-budget.mjs`, `context-fit.mjs` | measured context pressure, summaries that fit the window, the check before a message goes out |
| `agent-bridge.mjs` | the chat-to-session map, turns, approvals, Stop, status |
## Frames sent on `/api/chat/stream` for an agent turn
`queued`, `start` (`agent:true`), `reasoning`, `delta`, `agent-tool` (new: upserts a tool card by call id with
`queued`, `start` (`agent:true`), `reasoning`, `delta`, `agent-tool` (upserts a tool card by call id with
`running | proposed | done | denied | error`), `done` (usage, `finishReason`, `cancelled`), `error`, and `notice`
(why a chat is not using the agent). The chat UI renders them with its existing components.
@@ -56,7 +61,7 @@ The chat's Mode menu is real Harness behaviour. Each turn the page sends the cha
| Bypass | `strata-bypass` | `danger-full-access` | `never` | Full access without prompts, as before. |
- Manual relies on the gate, so a Manual turn does not start unless the bridge has confirmed the gate is loaded (`/strata-gate` command): without it Manual would be an open sandbox that nobody asks about.
- The gate listens to `tools/pre-execute` (the seam the Harness's own Auto-review layer uses) and returns an `ask` decision; the Harness's approval service then asks, runs the tool only after `allowed-once`, and records the answer. Read-only tools and a short allowlist of look-only shell commands (Get-ChildItem, git status, ...) run unasked; MCP and unknown tools ask.
- The gate listens to `tools/pre-execute` (the seam the Harness's own Auto-review layer uses) and returns an `ask` decision; the Harness's approval service then asks, runs the tool only after `allowed-once`, and records the answer. Read-only tools and a short allowlist of look-only shell commands (Get-ChildItem, git status, ...) run unasked, each only with the options listed for it in `strata-gate.mjs` (an unknown option, or one that writes, runs a program, sets something or never ends, asks); MCP and unknown tools ask.
- Web search and web pages (`web_search`, `web_fetch`) ask once per chat in Manual, not once per call. The gate asks through the approval service itself (`webDecision`: "The assistant wants to search the web for: ... Search words go to <service>." / "... open the web page ..."), remembers a yes for the chat (a helper agent shares its chat's answer, and two searches started together share one card), and on a no gives the model `WEB_REFUSAL` ("The person did not allow web access ... Go on without it and say so."). Plan and Bypass are unchanged. Settings -> Tools has a switch, "Let the assistant search the web without asking" (off by default, `webWithoutAsking` in `tools-settings.mjs`, passed to the gate as `config.web.skipAsk`), that skips the question in Manual. A scheduled task has nobody at the card, so its web call is answered no unless that switch is on. Tests: `tests/web-ask.test.mjs`.
- The mode is read once per turn; an unknown or missing value is Manual.
@@ -69,7 +74,7 @@ The chat's Mode menu is real Harness behaviour. Each turn the page sends the cha
What the person chose is saved by `tools-settings.mjs` (inside the host's settings file) and shown by `src/components/SettingsTools.tsx` (`GET/PUT /api/agent/tools`, `POST/PATCH/DELETE /api/agent/tools/custom[/id]`). A change reaches the agent from the next message: the profile carries a `toolsSignature`, and the bridge restarts the Harness (after the running turns end) when it differs.
- Tool groups (`tools-catalog.mjs`): each is one switch in plain words and is one or more preset plugins (`withoutDisabled`); `fetch` and `search` are the two halves of the Harness's `tool-web` (both off drops it). Goals off: `/goal` answers that goals are turned off.
- Web search needs no account. The default is DuckDuckGo's HTML result page (`../web-search-ddg.mjs`), with Bing's plain result page (`../web-search-bing.mjs`) as the second source. Manners, because DuckDuckGo refuses a program that asks fast (a real box: two searches in 1.2 s answered, the third got HTTP 202 and every DuckDuckGo page refused for many minutes): a browser User-Agent and the page's own request shape (Referer, Origin, Sec-Fetch-*), one request at a time per source, 2.5 to 4 s apart (random), at most 8 a minute, a 30 minute session cache per question, a cooldown after a refusal that doubles (1, 2, 4 ... 10 minutes, `Retry-After` honoured) during which that source gets no request, no lite-page follow-up after a refusal, one retry for a timeout or a 5xx, a 15 s limit. When every source refuses, the tool answers one plain sentence (`LIMITED_SENTENCE`: "Web search is limited right now. Try again in a minute, or add a search key (Exa, Perplexity or DeepSeek) in Settings -> Tools -> Web search.") plus an instruction to the model not to retry; the turn guard in `agent-bridge.mjs` (see Turn guard) stops a model that retries anyway. **The results come from DuckDuckGo or Bing**; they are not ranked or filtered here. The person may switch to Exa, Perplexity or DeepSeek with their own key (`../web-search-keyed.mjs`; DeepSeek uses the Harness's own provider): the key is saved in the settings file, reaches the agent only as the environment variable `STRATA_SEARCH_KEY`, never the profile file and never the screen (it only learns `keySet`). A keyed service without its key falls back to DuckDuckGo. The profile pins `web.searchProvider` to the chosen id so two usable backends cannot make the choice ambiguous.
- Web search needs no account. The default is DuckDuckGo's HTML result page (`../web-search-ddg.mjs`), with Bing's plain result page (`../web-search-bing.mjs`) as the second source. Manners, because DuckDuckGo refuses a program that asks fast: a browser User-Agent and the page's own request shape (Referer, Origin, Sec-Fetch-*), one request at a time per source, 2.5 to 4 s apart (random), at most 8 a minute, a 30 minute session cache per question, a cooldown after a refusal that doubles (1, 2, 4 ... 10 minutes, `Retry-After` honoured) during which that source gets no request, no lite-page follow-up after a refusal, one retry for a timeout or a 5xx, a 15 s limit. When every source refuses, the tool answers one plain sentence (`LIMITED_SENTENCE`: "Web search is limited right now. Try again in a minute, or add a search key (Exa, Perplexity or DeepSeek) in Settings -> Tools -> Web search.") plus an instruction to the model not to retry; the turn guard in `agent-bridge.mjs` (see Turn guard) stops a model that retries anyway. **The results come from DuckDuckGo or Bing**; they are not ranked or filtered here. The person may switch to Exa, Perplexity or DeepSeek with their own key (`../web-search-keyed.mjs`; DeepSeek uses the Harness's own provider): the key is saved in the settings file, reaches the agent only as the environment variable `STRATA_SEARCH_KEY`, never the profile file and never the screen (it only learns `keySet`). A keyed service without its key falls back to DuckDuckGo. The profile pins `web.searchProvider` to the chosen id so two usable backends cannot make the choice ambiguous.
- The browser: Playwright's MCP server (`@playwright/mcp`, staged next to the Harness by `apps/desktop/scripts/stage-harness.mjs`) on the person's own Chrome, Edge or Chromium (`browser-probe.mjs`), `--isolated` so it never sees their profile. Tools appear as `mcp__browser__browser_*`. In Manual, opening a page, clicking, typing, uploading, running a script in the page and closing tabs ask (plain reasons: "Open the web page ...", "Click on the page: ..."); looking (snapshot, screenshot, console, network, waiting, listing tabs) does not. Plan refuses every connected tool. The screen says plainly when no browser, or no tool server, is found.
- The person's own tools: `instructions` (the tool returns their text; look-only, never asks), `command` (runs their command with the assistant's text in the `STRATA_INPUT` environment variable, never spliced into the command line; asks in Manual, refused in Plan) and `server` (a web address or a program, added as `mcp__custom-<name>__*`; asks in Manual). Instruction and command tools are served by `custom-tools-mcp.mjs` (stdio MCP) from a JSON file the settings module rewrites on every change, so a switch in Settings is seen without a restart.
- `strata-gate.mjs` implements the Manual/Plan rules above (`planDecision`, `browserDecision`, the look-only instruction server).
@@ -80,7 +85,7 @@ What the person chose is saved by `tools-settings.mjs` (inside the host's settin
- A helper's request for approval is the same card (state `proposed`, id `<helper session>:<call>`), answered with the same `/api/chat/approval`; a chat can only answer its own helpers.
- The run (the chat's one NDJSON response) stays open until the parent is idle, every helper has finished and reported back, and no armed goal is waiting; `done` is sent then.
- Stop: pauses an active goal, interrupts every helper (`subagents/interruptByParent`; `session/cancel` on a helper session is refused by the Harness), cancels the parent, waits; a report that would wake the parent afterwards is cancelled too.
- The number of helpers at once equals the engine's request slots (`maxActiveSubagents` in the profile); the Harness refuses more with a message the model reads.
- The number of helpers at once is the engine's request slots, with a floor of 4 for a local model and 8 for an API model (`maxActiveSubagents` in the profile). The Harness refuses more with a message the model reads.
## /goal
@@ -88,8 +93,15 @@ What the person chose is saved by `tools-settings.mjs` (inside the host's settin
## Thinking and the output budget
The Harness staging script (`apps/desktop/scripts/stage-harness.mjs`) applies Strata's guarded pi-ai 1.1.0 patch to its staged copy
(`harness-budget.mjs`, #3684): below 64K the context safety room is at most `min(4096, floor(contextWindow / 16))` and never more
than half the estimated remaining room, so an 8K window keeps output room after a tool result instead of ending with a one-token
cap. The caller's output cap and the one-token floor remain; windows of 64K or more keep the previous calculation. `BUNDLE.json.patches`
records the patch identity, the dependency version and the before/after SHA-256. Unknown dependency bytes stop staging for review.
Source and already-installed Harness copies are not changed.
- A local route is reasoning-capable (`reasoning: true`) with `compat.thinkingFormat: 'chat-template'`: the request carries `chat_template_kwargs {enable_thinking, reasoning_effort}`, which the Strata server reads (a top-level `enable_thinking` is ignored by it, which is why pi-ai's `qwen` format left the model thinking). The chat's Reasoning menu travels as `thinking` in the stream body; the bridge sends it with `session/selectModel` (`reasoningEffort`) whenever the model or the level changed; a level a route does not have is dropped (never an error). API routes keep the provider default (the menu is disabled for them in the UI).
- `localOutputBudget(context)` = context minus 4096 (it was context/4 capped at 8192, which cut big answers off at exactly 8192 tokens); pi-ai sends the window minus the prompt. App-loaded models get `fit_max_tokens: true` so the engine clamps instead of answering 400. Sampling is not set here: the Harness sends none, so the server's own defaults apply.
- `localOutputBudget(context)` is 30 percent of the window, at least 4096 and at most 32768 (9,830 tokens at 32K, 19,660 at 64K). The Harness's automatic compaction reads it as the room a reply reserves. pi-ai sends the window minus the prompt. App-loaded models get `fit_max_tokens: true` so the engine clamps instead of answering 400. Sampling is not set here: the Harness sends none, so the server's own defaults apply.
## Tasks
@@ -107,7 +119,7 @@ Failures the person sees are plain sentences with a next step (`AgentUnavailable
## Helper agents on a named model (`plugins/strata-models.mjs`, `model-routes.mjs`)
The Harness's `subagent` tool takes an exact `provider` route key and model id, and the app had its model choice switched off, so every helper ran on the chat's model. The profile now enables the Harness's `subagent-model-selection-settings` with every route the app can reach (local instances and saved API models), and the `strata-models` plugin wraps the two tool definitions the Harness registers for each chat (`subagent`, `list_subagent_models`) in place:
The Harness's `subagent` tool takes an exact `provider` route key and model id. The profile enables the Harness's `subagent-model-selection-settings` with every route the app can reach (local instances and saved API models), and the `strata-models` plugin wraps the two tool definitions the Harness registers for each chat (`subagent`, `list_subagent_models`) in place:
- the model is found from its id, `provider/id`, the name in the model picker, `connection name`, or "local" (any letters). A close guess is not started: the tool answers "Did you mean ...?" and nothing runs until the call is made again with that model. Several fits are listed; nothing fitting lists the models that can be used and says they can be added in Models.
- the tool's own text says when to use it ("ask a subagent", "ask model X"), not to look for the model with shell commands, and not to wait with sleep. The system prompt (`preset.mjs`) says the same, so it reaches every model, local or hosted.
@@ -125,10 +137,31 @@ Pictures are on by default for every model that can read them: an API model read
## Turn guard (`turn-guard.mjs`)
A turn that keeps failing the same way is stopped by the app, never left to the model. Seen on a search-heavy `/plan`: every web search was refused, the model searched again and rewrote its plan after each one until the context reached 100%, then the turn ended with no answer and no message.
A turn that keeps failing the same way is stopped by the app, never left to the model.
- **Counted, from what the turn shows:** web searches that failed in a row (3), searches in one turn (10 in a plan turn, 25 otherwise), a plan turn that starts its numbered list from `1.` again (4 times), and a context of 93% or more (the Harness's own compaction did not keep up).
- **Stopped the way Stop stops** (`session/cancel`, helpers first) but ended as a normal turn (`done`, not cancelled, no error card), so the plan the model had written is still proposed. The plan card carries the reason (`plan.caution`); any other turn ends with one `delta` frame that has `own: true` and `key` (the English sentence, translated by the page; `GUARD_NOTES` are in the interface catalog in all eight languages). Each sentence says what happened and gives one next step.
- **Compaction:** after a guard stop (and after a turn the model server refused because the context was full) the bridge asks the Harness for `/compact` on that session, so the next message has room. Best effort and logged.
- **Never silent:** a turn that ends (`completed` or `max-tokens`) with no answer text gets "I ran out of room in this chat..." when the last request used 85% of the window or more, otherwise "I stopped without writing an answer..." (not when tool cards were shown, not after Stop, not after an error, which says why by itself). Goal runs have their own ending lines and are not guarded.
- Tests: `tests/agent-turn-guard.test.mjs` (counting, always; end to end with the real Harness, the scripted model's `searchloop` and `bloat` scripts and `searchMock: 'limited'` on the bridge, which runs the real search code on a pretend web that refuses, no network).
## Context: measured pressure, summaries that fit, the check before a message goes out
- **`plugins/strata-context.mjs`** (inserted by the profile, like the gate) wraps `tokenMeter.measure` and `llm.stream` in place, once each. Pressure = the last prompt size the model server reported for an answer, plus what changed since priced at the density that measurement showed (`context-budget.mjs`: `pressureNow`, `densityOf`); never lower than the Harness's own price. Node prices are scaled the same way, so the retained tail is in real tokens. A summary request (`purpose: 'compaction'`) that would not fit the window is asked for in pieces: a message bigger than a piece is cut for the request only, each piece builds on the summary before it, a piece the server refuses is halved and tried again. `/strata-context` (a Harness command, like `/strata-gate`) answers `{tokens, window}` for a session so the host can tell the person true figures.
- **`context-fit.mjs`**: before a message is sent the model server counts it (`POST /v1/messages/count_tokens`, its own tokenizer). `decideFit`: send; summarise first and then send (the message fits an otherwise empty chat); or never fits. The last one is not sent: the error frame has `code: 'CONTEXT_MESSAGE_TOO_LARGE'` and the page offers "Attach as a file" (`oversize.mjs`, `/api/chat/oversize`: the text is saved in the chat's folder, the assistant reads it in parts) or "Send a shortened preview". An engine that cannot count lets the message through; the Harness's own compact-and-retry is the net.
- **`/compact`** is handled by the host (`compactCommand`): the Harness's `/compact`, then a plain line with the measured figures ("The chat was 90% full and is now 38% full"), or "nothing older to summarize" when the newest message is what fills the chat.
- **Visible:** `compaction/start` shows "Summarizing earlier messages..." in the status line; `compaction/end` adds one plain line to the reply. The projector also sends a `context` frame after every model request, so the meter follows the running turn.
- **A refusal that is left** (`CONTEXT_WINDOW_EXCEEDED`) is a plain sentence with the engine's text under Details and the buttons "Summarize and continue" and "Start a new chat". "Try sending the message again" is for other errors only.
- Tests: `tests/agent-context.test.mjs` (arithmetic; the real Harness against the scripted worker with `window`, `charsPerToken` and `noCount`), `tests/oversize.test.mjs`, `tests/context-ui.test.mjs`. The run against a real engine is `scripts/context-e2e.mjs`.
## History / older measurements
- **Search refusals (2026-10):** on a real box two DuckDuckGo searches in 1.2 s answered, the third got HTTP 202, and every DuckDuckGo page refused for many minutes.
The pacing, cooldown and cache rules above come from that test.
- **Search-heavy `/plan` (2026-10):** every web search was refused, the model searched again and rewrote its plan after each one until the context reached 100%.
The turn ended with no answer and no message. The turn guard comes from that run.
- **Large log paste (2026-10-22, installed 0.2.1, Gyro-S, 64K):** a 118 KB log was 66,500 tokens to the engine and about 29,500 to the Harness
(4 characters a token; logs and code are about 2). The engine refused it with a 400. The refusal stayed in the history, so neither the next message nor `/compact` could
go through, because the summary request carried the same history. `/compact` was also not a command of the app yet: it went to the model as a question. This
produced `strata-context.mjs`, `context-fit.mjs` and the host's own `/compact`.
- **Output budget:** an earlier local budget of a quarter of the window, capped at 8192, cut big answers off at exactly 8192 tokens.
+136 -7
View File
@@ -22,7 +22,7 @@ import { randomUUID } from 'node:crypto';
import { GatewayClient } from './gateway-client.mjs';
import { HarnessProcess, resolveHarness, redact, stopStaleAgent } from './harness-runtime.mjs';
import { StatusReporter } from './status.mjs';
import { buildProfile, routeFor, MODE_PRESETS, modeOf, effortOf } from './profile.mjs';
import { buildProfile, routeFor, MODE_PRESETS, modeOf, effortOf, localOutputBudget, compactionHeadroom } from './profile.mjs';
import { TurnProjector, parseArguments as parseArgs } from './events.mjs';
import { ChildProjector } from './children.mjs';
import { nearestBudget } from './llm-shape.mjs';
@@ -30,6 +30,7 @@ import { DeliverableTracker, ActivityGroups } from './deliverables.mjs';
import { SkillsUsed, skillInfoFrom } from './skills-used.mjs';
import { turnEngineSpeed } from './engine-speed.mjs';
import { TurnGuard, guardNote, GUARD_NOTES } from './turn-guard.mjs';
import { BASE_TOKENS, COUNT_ABOVE_CHARS, FIT_TEXT, countTokens, decideFit, fill, grouped } from './context-fit.mjs';
import { PlanStore, planPrompt, parsePlan, parsePoints, stepObjective, stepLine, firstSentence, stepSummary, planAsPrompt, editedSteps, planView, researchPrompt, researchWritePrompt, RESEARCH_NUDGE, parseSources } from './plan.mjs';
/**
@@ -111,11 +112,12 @@ const deferred = () => { let resolve; const promise = new Promise(r => { resolve
* @param {string|null} [options.componentsDir] where installed components live
* @param {() => Array} options.localInstances ready local instances (see profile.mjs)
* @param {() => Array} options.connections saved API connections WITH their keys (host-internal)
* @param {() => (string|null)} [options.defaultLocalId] the local instance the chat runs on: its tool servers are the agent's (T5)
* @param {NodeJS.ProcessEnv} [options.env]
* @param {string} [options.nodePath] the Node runtime that runs the Harness
* @param {(line:string)=>void} [options.log]
*/
export function createAgentBridge({ dataDir, componentsDir = null, bundledDir = null, gate = true, localInstances, connections, tools = () => undefined, skillsDir = () => null, onGoal = () => {}, onPlan = () => {}, onQueue = () => {}, proxyBase = null, searchMock = null, expertSampling = () => null, stallMs = 4 * 60_000, stallToolMs = 20 * 60_000, env = process.env, nodePath = process.execPath, log = () => {}, startTimeoutMs = 120_000, sessionReadyMs = 60_000, sessionRetryMs = 1_500 }) {
export function createAgentBridge({ dataDir, componentsDir = null, bundledDir = null, gate = true, localInstances, connections, tools = () => undefined, defaultLocalId = () => null, skillsDir = () => null, onGoal = () => {}, onPlan = () => {}, onQueue = () => {}, proxyBase = null, searchMock = null, expertSampling = () => null, engineProgress = () => null, engineGraceMs = 30 * 60_000, stallMs = 4 * 60_000, stallToolMs = 20 * 60_000, env = process.env, nodePath = process.execPath, log = () => {}, startTimeoutMs = 120_000, sessionReadyMs = 60_000, sessionRetryMs = 1_500 }) {
const root = join(dataDir, 'agent');
const home = join(root, 'home');
const sessionsFile = join(root, 'sessions.json');
@@ -171,7 +173,7 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
/** The agent's profile now, plus where its shaped routes really lead (read by the forwarding endpoint). */
let shapeTargets = {};
function buildProfileNow() {
const profile = buildProfile({ local: localInstances(), connections: connections(), gate, tools: tools(), skillsDir: skillsDir(), proxyBase, searchMock });
const profile = buildProfile({ local: localInstances(), connections: connections(), gate, tools: tools(), skillsDir: skillsDir(), proxyBase, searchMock, defaultLocalId: defaultLocalId() });
shapeTargets = profile.shapeTargets ?? {};
return profile;
}
@@ -193,6 +195,8 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
harness: { version: h.version },
runtime: rt?.hp.running ? { pid: rt.hp.pid, generation: rt.generation } : null,
routes: profile.routes.map(({ route, kind, modelId, name, contextWindow }) => ({ route, kind, modelId, name, contextWindow })),
// tool servers that could not be wired (a name the rule does not allow, no program or address): the chat says so
skippedServers: profile.mcpSkipped ?? [],
activeTurns: activeTurns(),
pendingApprovals: pending.size,
};
@@ -406,7 +410,27 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
const appliedMode = planTask ? 'plan' : modeOf(mode);
await applyMode(runtime, sessionId, appliedMode);
const projector = new TurnProjector({ policy: MODE_PRESETS[appliedMode].approval });
// Does the message fit the model's window? Asked before anything is sent, with the model server's own count (context-fit.mjs). A message
// that fits once the older messages are a summary is sent after a summary; one that can never fit is not sent, and the person is told
// plainly and offered the two ways out. Whatever cannot be counted goes through: the Harness recovers from a refusal by itself.
if (!goal && route.kind === 'local') {
const fit = await fitCheck({ runtime, route, sessionId, fresh, history, message }).catch(() => null);
if (fit?.action === 'too-large') {
reporter.clear();
const values = [grouped(fit.tokens), grouped(fit.window)];
const text = fill(FIT_TEXT.tooLarge, values);
emit({ type: 'error', message: text, key: FIT_TEXT.tooLarge, values, code: 'CONTEXT_MESSAGE_TOO_LARGE', tokens: fit.tokens, window: fit.window });
return { error: text };
}
if (fit?.action === 'summarize') {
reporter.set({ phase: 'compacting', code: 'compacting', text: FIT_TEXT.summarizing, key: FIT_TEXT.summarizing, values: [] });
const done = await compactChat(runtime, sessionId);
reporter.clear();
if (done.ok) emit(summarizedNote(done.before, done.after, fit.window));
}
}
const projector = new TurnProjector({ policy: MODE_PRESETS[appliedMode].approval, window: route.contextWindow });
if (!goal) guard = new TurnGuard({ plan: !!planTask, contextMax: route.kind === 'local' ? route.contextWindow : 0 }); // a saved connection's window is often only assumed: never judged by a guess
projector.onStatus = update => { if (!turn?.parentOpen && turn?.children.size && [...turn.children.values()].some(h => !h.done) && update.phase === 'waiting') return; reporter.set(update); };
const turn = {
@@ -415,7 +439,7 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
abortFromRuntime: reason => finishTurn(reason), finish: failure => finishTurn(failure),
// Helper agents (subagents) the Harness started for this chat, and the state of the run around them.
children: new Map(), subagentCalls: new Map(), parentOpen: false, stopping: false, settleTimer: null, mode: appliedMode,
plan: null, advancing: false, goalRun: false, lastActivity: Date.now(), watchdog: null,
plan: null, advancing: false, goalRun: false, lastActivity: Date.now(), heard: Date.now(), engineMark: null, watchdog: null,
research: researchTopic ? { phase: 'gather', nudged: false, waitingStart: false } : null,
};
c.turn = turn;
@@ -428,6 +452,13 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
if (turn.finished || turn.stopping) return;
const toolRunning = [...projector.tools.values()].some(x => x.card?.state === 'running' || x.card?.state === 'proposed') || pending.size > 0 && pendingFor(chatId).length > 0;
const limit = toolRunning ? stallToolMs : stallMs;
// A local engine reading a long prompt sends the Harness nothing for minutes (#3623). Its /metrics progress counts as activity,
// for at most engineGraceMs after the Harness last spoke: progress for another chat cannot keep a lost turn alive forever.
if (turn.route?.kind === 'local' && Date.now() - turn.heard < engineGraceMs) {
let mark = null;
try { mark = engineProgress(turn.route.instanceId); } catch { /* no progress known: the Harness alone decides */ }
if (mark != null && mark !== turn.engineMark) { turn.engineMark = mark; turn.lastActivity = Date.now(); return; }
}
if (Date.now() - turn.lastActivity < limit) return;
turn.stalledFor = Math.round((Date.now() - turn.lastActivity) / 1000);
turn.stopping = true;
@@ -501,6 +532,7 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
function handle(item) {
if (turn.finished) return;
turn.lastActivity = Date.now();
turn.heard = turn.lastActivity;
if (item.type === 'event') {
const e = item.event;
if (e.seq <= turn.lastSeq) return;
@@ -538,7 +570,7 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
emitRow(child, true);
const fold = item => {
if (turn.finished) return;
turn.lastActivity = Date.now();
turn.lastActivity = turn.heard = Date.now();
// A helper that finishes before its throttled "working" row went out would show queued -> completed and never "working": send the held row first.
if (item.type === 'event' && item.event.type === 'turn/end' && child.rowPending) emitRow(child, true);
const { frames, row } = cp.consume(item);
@@ -951,6 +983,16 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
// -- /plan -----------------------------------------------------------------------------------------------
/** A short answer in the chat that the page translates (the sentence is in the interface catalog, "values" fill its slots). */
function sayKey(options, template, values = []) {
const { onFrame } = options;
onFrame({ type: 'queued', requestId: 'compact-' + randomUUID() });
onFrame({ type: 'start', model: '', contextMax: 0, demo: false, remoteModel: false, agent: true, mode: modeOf(options.mode), skills: [] });
onFrame({ type: 'delta', text: fill(template, values), key: template, values });
onFrame({ type: 'done', finishReason: 'stop', tokensIn: null, tokensOut: null, usageSource: 'unavailable', durationMs: 0 });
return { compact: true };
}
/** A short answer in the chat (no model): the same frames a turn sends. */
function sayOnce(options, reply) {
const { onFrame } = options;
@@ -1150,6 +1192,93 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
* Stop one chat's turn. Resolves only once the turn has ended (or the wait times out), because the
* Harness acknowledges a cancel as admitted, not as done.
*/
/** What the chat holds now and the model's window, as the context plugin measures them (null when it cannot be read). */
async function contextOf(runtime, sessionId) {
try {
const r = await runtime.gw.rpc('commands/execute', { agentId: sessionId, line: '/strata-context', submittedAttachments: [] });
if (r?.result?.kind !== 'success') return null;
const v = JSON.parse(String(r.result.text));
return Number.isFinite(v.tokens) ? { tokens: Math.round(v.tokens), window: Number(v.window) > 0 ? Math.round(v.window) : 0 } : null;
} catch { return null; }
}
/** The decision of context-fit.mjs for the message about to be sent, or null when the model server cannot count it. */
async function fitCheck({ runtime, route, sessionId, fresh, history, message }) {
const inst = localInstances().find(i => i.id === route.instanceId);
if (!inst || !(route.contextWindow > 0)) return null;
const carried = fresh && history.length ? withEarlierConversation(history, message) : message;
if (carried.length < COUNT_ABOVE_CHARS) return null;
const count = text => countTokens({ baseUrl: inst.baseUrl, key: inst.key, model: inst.modelId, text });
let tokens = await count(carried);
if (tokens == null) return null;
const now = fresh ? null : await contextOf(runtime, sessionId);
const window = now?.window || route.contextWindow;
const pressure = now?.tokens ?? 0;
// Where the Harness starts summarising by itself on this window (profile.mjs: the reply keeps its room, then the headroom).
const reply = localOutputBudget(window);
const threshold = Math.floor(Math.min(window * 0.8, window - reply - compactionHeadroom(window, reply)));
let verdict = decideFit({ window, pressure, messageTokens: tokens, threshold });
// The earlier conversation carried into a fresh session is what pushed it over: send the message alone instead.
if (verdict.action === 'too-large' && carried !== message) {
const alone = await count(message);
if (alone != null) { tokens = alone; verdict = decideFit({ window, pressure, messageTokens: alone, threshold }); }
}
return { ...verdict, tokens, window, pressure };
}
/**
* Summarise the older part of a chat with the Harness's /compact and say what happened. Never throws.
* ok:true carries the figures before and after; ok:false says why (nothing to summarise, busy, failed).
*/
async function compactChat(runtime, sessionId) {
const before = await contextOf(runtime, sessionId);
let result;
try { result = (await runtime.gw.rpc('commands/execute', { agentId: sessionId, line: '/compact', submittedAttachments: [] }))?.result; }
catch (e) { return { ok: false, reason: /not.found|no such/i.test(String(e.message)) ? 'nothing' : 'failed', before, text: String(e.message ?? e) }; }
const text = String(result?.text ?? '');
log('agent · compact: ' + (result?.kind ?? 'none') + ' ' + text.slice(0, 140));
if (result?.kind === 'success' && /^Compacted/i.test(text)) return { ok: true, before, after: await contextOf(runtime, sessionId) };
if (result?.kind === 'success') return { ok: false, reason: 'nothing', before, text };
if (/active compaction|not idle/i.test(text)) return { ok: false, reason: 'busy', before, text };
// The Harness cannot make a summary smaller than what is left when the chat holds little beyond its first prompt: that is nothing to summarise, not a failure.
const little = before && before.window > 0 && before.tokens - BASE_TOKENS < before.window * 0.25;
return { ok: false, reason: little && /useful summary|not smaller/i.test(text) ? 'nothing' : 'failed', before, text };
}
/** The line in the chat that explains why the context meter fell: how full it was and how full it is. */
function summarizedNote(before, after, window) {
const w = after?.window || before?.window || window || 0;
const pct = v => Math.max(0, Math.min(100, Math.round(100 * v.tokens / w)));
const known = w > 0 && before && after;
const template = known ? FIT_TEXT.summarized : FIT_TEXT.summarizedShort;
const values = known ? [pct(before), pct(after)] : [];
return { type: 'delta', own: true, text: '\n\n' + fill(template, values), key: template, values, noteReason: 'compacted' };
}
/** "/compact": summarise the older messages of this chat now, and say in plain words what happened. */
async function compactCommand(options) {
const { chatId, choice, signal } = options;
const c = chat(chatId);
const say = (template, values = []) => sayKey(options, template, values);
if (c.turn || c.starting) return say(FIT_TEXT.compactBusy);
const known = sessionMap[chatId]?.sessionId;
if (!known) return say(FIT_TEXT.noChatYet);
let runtime;
try { ({ rt: runtime } = await ensureRuntime(choice, signal)); } catch { return say(FIT_TEXT.compactFailed); }
const done = await compactChat(runtime, known);
if (done.ok) {
const note = summarizedNote(done.before, done.after, 0);
return say(note.key, note.values);
}
if (done.reason === 'busy') return say(FIT_TEXT.compactBusy);
if (done.reason === 'nothing') {
const w = done.before?.window || 0;
const pct = w > 0 && done.before ? Math.round(100 * done.before.tokens / w) : 0;
return say(pct >= 60 ? FIT_TEXT.nothingFull : FIT_TEXT.nothingToSummarize, pct >= 60 ? [pct] : []);
}
return say(FIT_TEXT.compactFailed);
}
/** Summarise the older part of a session with the Harness's own /compact. Best effort: the next message compacts again by itself when it must. */
async function compactSession(runtime, sessionId) {
try {
@@ -1204,7 +1333,7 @@ export function createAgentBridge({ dataDir, componentsDir = null, bundledDir =
/** Let go of a chat's Harness session mapping (a task starts every run in a session of its own). */
function forget(chatId) { if (chats.get(chatId)?.turn) return; delete sessionMap[chatId]; chats.delete(chatId); persist(); }
return { goalState, goalControl, harnessDir, status, runTurn, goalCommand, planCommand, researchCommand, llmTarget, setPlanBriefing, resetSession, reopenPlan, skipStep, planGoal, planControl, planState, queueMessage, cancelQueued, takeQueue, forget, approve, setMode, stop, pendingFor, pendingAll, close, sessionOf: chatId => sessionMap[chatId]?.sessionId ?? null, workingFolder: chatId => sessionMap[chatId]?.cwd ?? null, isRunning: chatId => !!chats.get(chatId)?.turn, _debug: () => ({ rt, sessionMap }) };
return { compactCommand, goalState, goalControl, harnessDir, status, runTurn, goalCommand, planCommand, researchCommand, llmTarget, setPlanBriefing, resetSession, reopenPlan, skipStep, planGoal, planControl, planState, queueMessage, cancelQueued, takeQueue, forget, approve, setMode, stop, pendingFor, pendingAll, close, sessionOf: chatId => sessionMap[chatId]?.sessionId ?? null, workingFolder: chatId => sessionMap[chatId]?.cwd ?? null, isRunning: chatId => !!chats.get(chatId)?.turn, _debug: () => ({ rt, sessionMap }) };
}
/** A goal that stopped by itself, in words. */
+162
View File
@@ -0,0 +1,162 @@
/**
* Token arithmetic for the agent's context, with no I/O (the Harness plugin `plugins/strata-context.mjs` and the host both use it).
*
* Why it exists. The Harness prices a conversation with a fixed 4 characters per token. Code, logs and JSON are about twice as dense
* (2.09 characters per token in the 2026-10-22 hands-on test), so its pressure read 31,794 tokens while the engine had been sent 59,048,
* and compaction waited until the window was already past full. The engine reports the real prompt size with every answer; this module
* turns that report into the number compaction decides on, and splits a summarising request that would not fit the window.
*/
/** The Harness's own price of a text: characters over this, plus a fixed cost per block and per message. */
export const CHARS_PER_TOKEN = 4;
const BLOCK_OVERHEAD = 4;
const MESSAGE_OVERHEAD = 4;
/** The most a real token can be worth in the Harness's units (one character per token is the worst a tokenizer does). */
export const MAX_DENSITY = 6;
/** Room kept free in a window for the engine's own framing and the chat template (the engine keeps 64 tokens of slack itself). */
export const WINDOW_SLACK = 1024;
/** The Harness's price (4 characters a token) of one message, the same arithmetic as its token meter. */
export function estimateMessage(message) {
const content = Array.isArray(message?.content) ? message.content : [];
if (message?.role === 'system') {
if (!content.length) return 0;
let chars = 0;
for (const block of content) chars += block?.type === 'text' ? String(block.text ?? '').length : JSON.stringify(block ?? {}).length;
return Math.ceil(chars / CHARS_PER_TOKEN) + MESSAGE_OVERHEAD;
}
let tokens = MESSAGE_OVERHEAD;
for (const block of content) {
if (block?.type === 'text' || block?.type === 'reasoning') tokens += Math.ceil(String(block.text ?? '').length / CHARS_PER_TOKEN) + BLOCK_OVERHEAD;
else if (block?.type === 'tool-call') tokens += Math.ceil(String(block.name ?? '').length / CHARS_PER_TOKEN) + Math.ceil(String(block.arguments ?? '').length / CHARS_PER_TOKEN) + BLOCK_OVERHEAD;
else tokens += BLOCK_OVERHEAD + Math.ceil(JSON.stringify(block ?? {}).length / CHARS_PER_TOKEN);
}
return tokens;
}
/** The Harness's price of the tool list of a request header. */
export function estimateTools(header) {
const tools = header?.tools;
if (!Array.isArray(tools) || !tools.length) return 0;
return Math.ceil(JSON.stringify(tools).length / CHARS_PER_TOKEN) + BLOCK_OVERHEAD;
}
/**
* How many real tokens one of the Harness's tokens is worth, from one measured request: the engine said the prompt was `real` tokens,
* the Harness priced the same prompt at `estimated`. Never below 1 (a conversation of plain prose is priced about right, and a lower
* number would only delay compaction) and never above MAX_DENSITY (a wild sample must not make every step compact).
*/
export function densityOf(real, estimated) {
if (!(real > 0) || !(estimated > 0)) return 1;
return Math.min(MAX_DENSITY, Math.max(1, real / estimated));
}
/**
* The prompt size compaction decides on. `sample` is the last measured request: `real` tokens for a prompt the Harness priced at
* `estimated`. `now` is the Harness's price of the conversation now. What was added or removed since the sample is scaled by the sample's
* density; the measured part is exact. Without a sample, or when the measurement is lower than the Harness's own price, the Harness's
* price stands (it is never an underestimate that makes this smaller).
*/
export function pressureNow({ sample = null, now }) {
if (!sample || !(sample.real > 0) || !(sample.estimated > 0)) return Math.max(0, now);
const density = densityOf(sample.real, sample.estimated);
return Math.max(Math.round(now), Math.round(sample.real + density * (now - sample.estimated)));
}
/** The usage a finished model request reported, as the prompt side only: input plus the cache's reads and writes (output is not in the prompt). */
export function promptTokensOf(usage) {
if (!usage || typeof usage !== 'object') return 0;
const n = value => (Number.isFinite(Number(value)) ? Math.max(0, Number(value)) : 0);
return n(usage.inputTokens) + n(usage.cacheReadTokens) + n(usage.cacheWriteTokens);
}
/** The usage of a settled assistant event (a `usage` on the message, or the last usage chunk of its stream). */
export function usageOfEvent(event) {
const data = event?.data;
if (!data) return null;
if (data.usage && typeof data.usage === 'object') return data.usage;
const stream = Array.isArray(data.stream) ? data.stream : [];
for (let i = stream.length - 1; i >= 0; i--) if (stream[i]?.type === 'chunk' && stream[i].chunk?.type === 'usage') return stream[i].chunk.usage ?? null;
return null;
}
/**
* A summarising request is: the conversation's system prompt and tools, the messages to condense, then the instruction. This is how many
* real tokens the messages may take in a window so that all of it, the summary and a safety margin fit.
*/
export function summaryBudget({ window, summaryTokens, toolsTokens = 0, systemTokens = 0, instructionTokens = 900, slack = WINDOW_SLACK }) {
return Math.floor(window - summaryTokens - toolsTokens - systemTokens - instructionTokens - slack);
}
/** Text blocks of a message, longest first: the ones worth shortening when a single message is bigger than the budget. */
function textBlocks(message) {
return (Array.isArray(message?.content) ? message.content : []).filter(b => (b?.type === 'text' || b?.type === 'reasoning') && typeof b.text === 'string');
}
/**
* One message cut down to `maxTokens` (in the Harness's price times `density`): the start and the end of its longest text are kept and
* the middle is replaced by a line saying how much was left out. Only the copy sent to the summariser is cut; the conversation keeps
* the whole message until the summary replaces it.
*/
export function clipMessage(message, maxTokens, density = 1) {
if (estimateMessage(message) * density <= maxTokens) return message;
const blocks = textBlocks(message);
if (!blocks.length) return message;
const others = estimateMessage({ ...message, content: (message.content ?? []).filter(b => !blocks.includes(b)) }) * density;
const room = Math.max(256, maxTokens - others);
const share = room / blocks.reduce((n, b) => n + b.text.length, 0) / density * CHARS_PER_TOKEN; // characters kept per character
const content = (message.content ?? []).map((block) => {
if (!blocks.includes(block)) return block;
const keep = Math.max(200, Math.floor(block.text.length * Math.min(1, share) - 120));
if (block.text.length <= keep) return block;
const head = Math.ceil(keep * 0.6), tail = keep - head;
const left = block.text.length - head - tail;
return { ...block, text: `${block.text.slice(0, head)}\n\n[… ${left.toLocaleString('en-US')} characters from the middle are left out of this summary request …]\n\n${block.text.slice(block.text.length - tail)}` };
});
return { ...message, content };
}
/** Whether cutting the list in front of index `i` would separate a tool call from its result. */
function balancedBefore(messages, i) {
let open = 0;
for (let k = 0; k < i; k++) {
const m = messages[k];
if (m?.role === 'assistant') open += (Array.isArray(m.content) ? m.content : []).filter(b => b?.type === 'tool-call').length;
else if (m?.role === 'tool') open -= 1;
}
return open <= 0;
}
/**
* Split messages into runs that each fit `budget` real tokens (price `density`), cutting between a tool call's result and the next
* message when it can. A message bigger than the budget by itself is clipped first. Order is kept.
*/
export function splitMessages(messages, budget, density = 1) {
const room = Math.max(512, budget);
const sized = messages.map(m => clipMessage(m, Math.floor(room * 0.8), density));
const cost = sized.map(m => estimateMessage(m) * density);
const runs = [];
let start = 0, used = 0;
for (let i = 0; i < sized.length; i++) {
if (i > start && used + cost[i] > room) {
// Cut as late as possible, but never between a tool call and its result when a clean place exists inside the run.
let cut = i;
while (cut > start + 1 && !balancedBefore(sized, cut)) cut--;
if (!balancedBefore(sized, cut)) cut = i;
runs.push(sized.slice(start, cut));
start = cut;
used = cost.slice(start, i).reduce((n, x) => n + x, 0);
}
used += cost[i];
}
if (start < sized.length) runs.push(sized.slice(start));
return runs;
}
/** Whether a failure is the model server saying the prompt does not fit its window. */
export function isContextOverflow(error) {
if (!error) return false;
if (error.code === 'CONTEXT_WINDOW_EXCEEDED') return true;
const text = [error.message, error.cause?.message, error.failure?.message, error.failure?.code].filter(Boolean).join(' ');
return /request exceeds the context window|leaves no room to answer|context[_ ]length|context window|too many tokens|maximum context/i.test(text);
}
+102
View File
@@ -0,0 +1,102 @@
/**
* Does a message fit the model's window? Asked before it is sent, with the model server's own count, so a message the server would refuse
* is never read for half a minute first (2026-10-22: a 103 KB paste was 70,141 tokens in a 65,536-token window, and the 400 came back after
* the whole prompt had been read). No I/O except the one count request; everything else is arithmetic the tests call directly.
*
* Three answers: send it; summarise the chat first and then send it (the message fits once the older messages are a summary); or it can
* never fit, and the person is offered the two ways out that do (the text as a file the assistant reads in parts, or a shortened preview).
*/
/** What a new chat already holds before the person's first word: the system prompt and the tool list. Measured 10,510 by the Harness's price. */
export const BASE_TOKENS = 12000;
/** The least room a reply needs after the prompt: an eighth of the window, at most this. */
export const REPLY_FLOOR = 4096;
/** A message shorter than this many characters cannot matter to a window of 8K or more, so it is not even counted. */
export const COUNT_ABOVE_CHARS = 3000;
export const replyRoom = window => Math.max(512, Math.min(REPLY_FLOOR, Math.floor(window / 8)));
/**
* @param {{window:number, pressure?:number, base?:number, messageTokens:number, threshold?:number}} input
* `pressure` is what the chat holds now (measured), `base` what an empty chat holds, `messageTokens` the new message, `threshold` where the
* Harness starts summarising by itself. The Harness looks at that only before its NEXT step, so a big message that arrives now would otherwise
* be sent on top of a chat already near the threshold (2026-10-22: 54K of 64K held by the time it was looked at): summarise first.
* @returns {{action:'send'|'summarize'|'too-large', room:number, needs:number}}
*/
export function decideFit({ window, pressure = 0, base: given = BASE_TOKENS, messageTokens, threshold = 0 }) {
const reply = replyRoom(window);
const base = Math.min(given, Math.floor(window * 0.6)); // on a small window the first prompt cannot be what the window is made of
// A chat holds at least what an empty one does (the system prompt and the tools), so a new chat is not counted as empty.
const needs = Math.max(pressure, base) + messageTokens + reply;
if (!(window > 0) || !(messageTokens >= 0)) return { action: 'send', room: Infinity, needs: 0 };
if (needs <= window) {
const worthIt = pressure > base + window * 0.1; // older messages worth summarising: more than a tenth of the window beyond the first prompt
if (threshold > 0 && worthIt && pressure + messageTokens > threshold) return { action: 'summarize', room: window - needs, needs, why: 'threshold' };
return { action: 'send', room: window - needs, needs };
}
// Not as it stands. Does the message fit in an otherwise empty chat (only the system prompt and the tools before it)?
if (base + messageTokens + reply <= window) return { action: 'summarize', room: window - (base + messageTokens + reply), needs };
return { action: 'too-large', room: window - (base + messageTokens + reply), needs };
}
/**
* The model server's count of a text (its tokenizer and chat template), through its Anthropic-style count endpoint.
* Returns the number, or null when it cannot be had (an old engine, a server that is busy, a timeout): then nothing is blocked.
*/
export async function countTokens({ baseUrl, key = '', model = '', text, fetchImpl = fetch, timeoutMs = 8000 }) {
try {
const root = String(baseUrl ?? '').replace(/\/v1\/?$/, '').replace(/\/$/, '');
const res = await fetchImpl(`${root}/v1/messages/count_tokens`, {
method: 'POST',
headers: { 'content-type': 'application/json', ...(key ? { authorization: `Bearer ${key}`, 'x-api-key': key } : {}) },
body: JSON.stringify({ model, messages: [{ role: 'user', content: String(text) }] }),
signal: AbortSignal.timeout(timeoutMs),
});
if (!res.ok) return null;
const n = Number((await res.json())?.input_tokens);
return Number.isFinite(n) && n >= 0 ? Math.round(n) : null;
} catch { return null; }
}
/** The plain sentences (also in the interface catalog in all nine languages). `{0}` slots are filled by the page. */
export const FIT_TEXT = Object.freeze({
tooLarge: 'This message is about {0} tokens long, and this model can read about {1} at a time. It is too large to send as it is.',
tooLargeNext: 'Attach it as a file for the assistant to read in parts, or send a shortened preview.',
full: 'This conversation no longer fits the model’s window, and summarizing the older messages did not make enough room.',
fullNext: 'Summarize it now and continue, or start a new chat.',
summarizing: 'Summarizing earlier messages…',
summarized: 'Older messages were summarized to make room. The chat was {0}% full and is now {1}% full.',
summarizedWas: 'Older messages were summarized to make room. The chat was {0}% full.',
summarizedShort: 'Older messages were summarized to make room.',
nothingFull: 'This chat is {0}% full, but there is nothing older to summarize: the newest message is what fills it. Start a new chat, or send a shorter message.',
nothingToSummarize: 'There is nothing older to summarize yet. Everything in this chat is still needed by the assistant.',
noChatYet: 'There is nothing to summarize yet. This chat has not started a conversation with the assistant.',
compactFailed: 'The older messages could not be summarized right now. Try again in a moment, or start a new chat.',
compactBusy: 'This chat is still working. Stop it or wait for it to finish, then try again.',
});
export const fill = (template, values = []) => String(template).replace(/\{(\d+)\}/g, (_, i) => String(values[Number(i)] ?? ''));
/** A number with thousands separators, the way the page writes it. */
export const grouped = n => Math.round(n).toLocaleString('en-US');
/**
* The two ways to send a message that is too large.
* file the whole text goes in a file in the chat's folder; the message names it and shows how it starts
* preview the start and the end of the text, cut to what the window can hold, with the number of characters left out
* @param {{text:string, mode:'file'|'preview', tokens:number, window:number, pressure?:number, base?:number, path?:string}} input
* @returns {{message:string, kept:number}}
*/
export function oversizedReplacement({ text, mode, tokens, window, pressure = 0, base = BASE_TOKENS, path = '' }) {
const t = String(text);
const charsPerToken = tokens > 0 ? Math.max(0.8, t.length / tokens) : 3;
if (mode === 'file') {
const start = t.slice(0, 1200).trimEnd();
return { kept: start.length, message: `My message was too long to send in the chat, so I saved it as a file: ${path}\n\nIt starts like this:\n\n${start}\n\n[the rest is in the file]\n\nPlease read the file in parts (it is long) and then do what my message asks.` };
}
const room = Math.max(1000, Math.floor((window - Math.max(pressure, base) - replyRoom(window)) * 0.7));
const keep = Math.max(500, Math.floor(room * charsPerToken * 0.9));
const head = Math.ceil(keep * 0.7), tail = keep - head;
const left = Math.max(0, t.length - head - tail);
if (!left) return { kept: t.length, message: t };
return { kept: head + tail, message: `${t.slice(0, head)}\n\n[… ${grouped(left)} characters from the middle are left out because the message was too long to send …]\n\n${t.slice(t.length - tail)}` };
}
@@ -0,0 +1,103 @@
/**
* Runs the command of a command tool the person wrote (Settings -> Tools). Split from custom-tools-mcp.mjs so it can be tested.
*
* T7 (tools and images review, 2026-10-22). It used to run every command with `shell: true` (cmd.exe on Windows, so `Get-Content x` failed with
* "not recognized"), in whatever folder the host happened to run in (so `python tools/foo.py` could not find its file), and ended it after two
* minutes with 20,000 characters of output, silently cut.
*
* shell 'auto' (default): PowerShell for a Windows command that reads as PowerShell (Get-Content, Start-Process, $env:X, $(...)), cmd.exe for
* any other Windows command (so every command that worked keeps working), /bin/sh elsewhere. 'cmd', 'powershell', 'bash' and 'sh' name one.
* cwd the folder to run in: the tool's own, or the one the assistant names in the call (it knows the chat's folder); else the person's home.
* timeoutSec the time allowed, 1 to 3600 (default 120). A command that runs over is ended together with the programs it started.
*/
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, statSync } from 'node:fs';
import { homedir } from 'node:os';
export const DEFAULT_TIMEOUT_SEC = 120;
export const MAX_TIMEOUT_SEC = 3600;
export const OUT_MAX = 20000;
export const SHELLS = ['auto', 'cmd', 'powershell', 'bash', 'sh'];
/** A Windows command that is PowerShell and would fail in cmd.exe. */
const POWERSHELL_LOOKS = /(^|[\s;|(&])[A-Z][A-Za-z]+-[A-Z][A-Za-z]+\b|\$\w|\$\(|\$env:|@\(|\|\s*(?:%|\?)\s*\{/;
export const looksLikePowerShell = command => POWERSHELL_LOOKS.test(String(command ?? ''));
let powershellFound;
/** pwsh when it is installed (PowerShell 7), else the Windows PowerShell every Windows has. */
function powershellExe(platform) {
if (powershellFound) return powershellFound;
if (platform !== 'win32') return (powershellFound = 'pwsh');
try { if (spawnSync('where', ['pwsh'], { windowsHide: true, timeout: 5000 }).status === 0) return (powershellFound = 'pwsh'); } catch { /* none */ }
return (powershellFound = 'powershell.exe');
}
/**
* -> { file, args, shell } for spawn: which program runs `command`.
* @param {string} command
* @param {{shell?: string, platform?: string}} [options]
*/
export function planShell(command, { shell = 'auto', platform = process.platform } = {}) {
const windows = platform === 'win32';
let choice = SHELLS.includes(shell) ? shell : 'auto';
if (choice === 'auto') choice = windows ? (looksLikePowerShell(command) ? 'powershell' : 'cmd') : 'sh';
if (choice === 'powershell') return { file: powershellExe(platform), args: ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-Command', command], shell: false, name: 'PowerShell' };
if (choice === 'bash') return { file: 'bash', args: ['-c', command], shell: false, name: 'bash' };
if (choice === 'sh') return windows ? { file: 'sh', args: ['-c', command], shell: false, name: 'sh' } : { file: command, args: [], shell: true, name: 'sh' };
return { file: command, args: [], shell: true, name: windows ? 'cmd.exe' : 'sh' }; // cmd: spawn's shell:true is cmd.exe on Windows
}
/** The folder a command runs in: -> { cwd } or { error } with a plain sentence. */
export function resolveCwd(...candidates) {
for (const candidate of candidates) {
const text = typeof candidate === 'string' ? candidate.trim() : '';
if (!text) continue;
const expanded = text.replace(/^~(?=$|[\\/])/, homedir());
try {
if (!existsSync(expanded) || !statSync(expanded).isDirectory()) return { error: `The folder “${text}” does not exist, so the command was not run.` };
} catch { return { error: `The folder “${text}” could not be opened, so the command was not run.` }; }
return { cwd: expanded };
}
return { cwd: homedir() };
}
export const clampTimeoutSec = value => {
const n = Number(value);
return Number.isFinite(n) && n >= 1 ? Math.min(Math.floor(n), MAX_TIMEOUT_SEC) : DEFAULT_TIMEOUT_SEC;
};
function endTree(child) {
try {
if (process.platform === 'win32' && child.pid) spawnSync('taskkill', ['/PID', String(child.pid), '/T', '/F'], { windowsHide: true, timeout: 10000 });
else child.kill('SIGKILL');
} catch { /* gone */ }
}
/**
* @param {{command: string, input?: string, cwd?: string, callCwd?: string, shell?: string, timeoutSec?: number, platform?: string}} job
* @returns {Promise<{ok: boolean, text: string}>}
*/
export function runCommand({ command, input = '', cwd, callCwd, shell = 'auto', timeoutSec, platform = process.platform }) {
return new Promise(resolve => {
const where = resolveCwd(callCwd, cwd);
if (where.error) return resolve({ ok: false, text: where.error });
const plan = planShell(command, { shell, platform });
const limit = clampTimeoutSec(timeoutSec);
let child;
try {
child = spawn(plan.file, plan.args, { shell: plan.shell, windowsHide: true, cwd: where.cwd, env: { ...process.env, STRATA_INPUT: String(input ?? '') }, stdio: ['ignore', 'pipe', 'pipe'] });
} catch (e) { return resolve({ ok: false, text: `The command could not start in ${plan.name}: ${e.message}` }); }
let text = '', cut = false, timedOut = false;
const take = chunk => { if (text.length < OUT_MAX + 1) text += chunk.toString(); else cut = true; };
child.stdout.on('data', take); child.stderr.on('data', take);
const timer = setTimeout(() => { timedOut = true; endTree(child); }, limit * 1000);
child.on('error', e => { clearTimeout(timer); resolve({ ok: false, text: `The command could not start in ${plan.name}: ${e.message}` }); });
child.on('close', code => {
clearTimeout(timer);
if (text.length > OUT_MAX) { cut = true; text = text.slice(0, OUT_MAX); }
const out = text.trim() + (cut ? `\n\n(The output was cut after ${OUT_MAX.toLocaleString('en-US')} characters.)` : '');
if (timedOut) return resolve({ ok: false, text: `The command was stopped after ${limit} seconds.${out ? '\n' + out : ''}` });
resolve({ ok: code === 0, text: out || (code === 0 ? 'The command finished without any error. It printed nothing.' : `The command failed (exit code ${code}) and printed nothing.`) });
});
});
}
+9 -23
View File
@@ -8,49 +8,35 @@
* It speaks the Model Context Protocol over stdio (newline-delimited JSON-RPC), the transport the Harness's MCP client uses for a program
* it starts. The file is re-read on every request, so a change in Settings is seen without restarting anything. The assistant's text for
* a command tool reaches the command in the STRATA_INPUT environment variable (never spliced into the command line, so it cannot add
* commands of its own). A command gets 2 minutes and 20,000 characters of output.
* commands of its own). A command runs in the shell that suits it (PowerShell for a Windows command that reads as PowerShell, cmd.exe for
* the rest, sh elsewhere), in its own folder or the one the assistant names, gets 2 minutes unless its tool allows more (up to an hour) and
* 20,000 characters of output (custom-command-run.mjs).
*/
import { spawn } from 'node:child_process';
import { readFileSync } from 'node:fs';
import { createInterface } from 'node:readline';
import { runCommand } from './custom-command-run.mjs';
const [mode, file] = process.argv.slice(2);
if (!['instructions', 'commands'].includes(mode) || !file) { console.error('usage: custom-tools-mcp.mjs instructions|commands <tools.json>'); process.exit(2); }
const load = () => { try { return JSON.parse(readFileSync(file, 'utf8'))[mode] ?? {}; } catch { return {}; } };
const OUT_MAX = 20000;
const TIME_MS = 120000;
const schema = mode === 'commands'
? { type: 'object', properties: { input: { type: 'string', description: 'Text to give the command (optional).' } }, additionalProperties: false }
? { type: 'object', properties: {
input: { type: 'string', description: 'Text to give the command (optional).' },
cwd: { type: 'string', description: 'The folder to run the command in (optional). Give the working folder of this chat when the command uses relative paths.' },
}, additionalProperties: false }
: { type: 'object', properties: {}, additionalProperties: false };
function listTools() {
return Object.entries(load()).map(([name, t]) => ({ name, description: String(t.description ?? ''), inputSchema: schema, annotations: mode === 'instructions' ? { readOnlyHint: true } : {} }));
}
function runCommand(command, input) {
return new Promise(resolve => {
const child = spawn(command, { shell: true, windowsHide: true, env: { ...process.env, STRATA_INPUT: String(input ?? '') }, stdio: ['ignore', 'pipe', 'pipe'] });
let text = '';
const take = chunk => { if (text.length < OUT_MAX) text += chunk.toString(); };
child.stdout.on('data', take); child.stderr.on('data', take);
const timer = setTimeout(() => { try { child.kill(); } catch { /* gone */ } }, TIME_MS);
child.on('error', e => { clearTimeout(timer); resolve({ ok: false, text: `The command could not start: ${e.message}` }); });
child.on('close', (code, signal) => {
clearTimeout(timer);
const out = text.slice(0, OUT_MAX).trim();
if (signal) resolve({ ok: false, text: `The command was stopped after ${TIME_MS / 1000} seconds.${out ? '\n' + out : ''}` });
else resolve({ ok: code === 0, text: out || (code === 0 ? 'The command finished without any error. It printed nothing.' : `The command failed (exit code ${code}) and printed nothing.`) });
});
});
}
async function call(name, args) {
const t = load()[name];
if (!t) return { isError: true, content: [{ type: 'text', text: 'That tool is no longer available.' }] };
if (mode === 'instructions') return { content: [{ type: 'text', text: String(t.text ?? '') }] };
const r = await runCommand(String(t.command ?? ''), args?.input);
const r = await runCommand({ command: String(t.command ?? ''), input: args?.input, callCwd: args?.cwd, cwd: t.cwd, shell: t.shell, timeoutSec: t.timeoutSec });
return { isError: !r.ok, content: [{ type: 'text', text: r.text }] };
}
@@ -0,0 +1,11 @@
/**
* How far a local engine has got, as a short text that changes whenever it makes progress. The agent's stall watchdog
* compares it with the last one it saw: while a long prompt is read the Harness hears nothing for minutes, but the
* engine's /metrics (polled every 2 s) shows `live.prompt_read` moving (#3623). Null when the engine is idle or says nothing.
*/
export function engineProgressMark(instance) {
const live = instance?.metrics?.live;
if (!live || live.state === 'idle' || live.state === 'unloaded') return null;
const slots = Array.isArray(live.slots) ? live.slots.map(s => `${s?.state}:${s?.prompt_read ?? ''}:${s?.generated ?? ''}`).join(',') : '';
return `${live.state}|${live.prompt_read ?? ''}|${live.generated ?? ''}|${slots}`;
}
+36 -6
View File
@@ -9,6 +9,8 @@
import { toolUpdate, TokenCount, fmtTokens } from './status.mjs';
import { askFor } from './ask.mjs';
import { FIT_TEXT, fill } from './context-fit.mjs';
import { isContextOverflow } from './context-budget.mjs';
const SHELL = new Set(['pwsh', 'bash', 'job_kill', 'job_output', 'job_list']);
const WRITE = new Set(['write', 'edit']);
@@ -157,6 +159,9 @@ export function withPlain(raw) {
return plain ? `${plain}\n\nDetails: ${raw.replace(/^Error:\s*/, '')}` : raw;
}
/** What a launch that printed nothing proves, and no more. */
export const LAUNCHED_TEXT = 'The command finished without an error, so the program was started. Strata cannot confirm that a window or page actually opened.';
/** A command whose whole point is to open something: it has no output to show when it worked. */
const LAUNCH = /(^|[\s;|&(])(start-process|start|invoke-item|ii|explorer(?:\.exe)?|xdg-open|open|notepad(?:\.exe)?|code)(?=[\s;|&)]|$)/i;
@@ -172,9 +177,10 @@ export function outcomeText(tool, text, args = {}) {
text = text.replace(/\r?\n?\[stderr\]\r?\n(?:Cannot create type\. Only core types are supported in this language mode\.[\s\S]*?CannotCreateTypeConstrainedLanguage\s*)+$/, '');
if (!text.trim()) text = '(no output)';
}
// Starting a program or opening a folder prints nothing when it worked. The model used to read "finished, nothing to show" as "cannot tell
// whether it opened" and told the person so; a launch that returned without an error did start, and the result says that plainly.
if ((tool === 'pwsh' || tool === 'bash') && /^\(no output\)$/.test(text.trim()) && LAUNCH.test(String(args?.command ?? ''))) return 'Started. The program, folder or file was opened.';
// Starting a program or opening a folder prints nothing when it worked, and also when it did not (a browser given a file:// address, a
// program that closed at once). T9 / #3576: the card used to say "was opened", and the model repeated it ("I previewed it in a browser to
// confirm it renders"). It says what is known: the command returned without an error.
if ((tool === 'pwsh' || tool === 'bash') && /^\(no output\)$/.test(text.trim()) && LAUNCH.test(String(args?.command ?? ''))) return LAUNCHED_TEXT;
if ((tool === 'pwsh' || tool === 'bash') && /^\(no output\)$/.test(text.trim())) return 'The command finished without any error. It printed nothing, so there is nothing more to show.';
return text;
}
@@ -196,8 +202,10 @@ export class TurnProjector {
* no question ever reaches the person: the card goes straight from running to denied, without a moment where it
* offers Approve and Deny buttons that would do nothing.
*/
constructor({ startedAt = Date.now(), policy = 'ask' } = {}) {
constructor({ startedAt = Date.now(), policy = 'ask', window = 0 } = {}) {
this.startedAt = startedAt;
this.window = Number(window) || 0; // the model's window, for the figures in the line that says the chat was summarised
this.summarizing = false;
this.policy = policy;
this.byPolicy = new Set(); // callIds the approval policy rejected without asking
this.tools = new Map(); // callId -> { card, calledAt }
@@ -303,6 +311,7 @@ export class TurnProjector {
this.tokensIn += prompt;
this.tokensOut += Number(c.usage.outputTokens) || 0;
this.contextTokens = prompt + (Number(c.usage.outputTokens) || 0);
out.push({ type: 'context', contextTokens: this.contextTokens, contextMax: this.window || undefined });
}
}
return out;
@@ -371,6 +380,22 @@ export class TurnProjector {
if (frame) out.push(frame);
break;
}
case 'compaction/start':
// The Harness is summarising the older messages to make room: say so while it does, and again (once) when it is done.
this.summarizing = true;
this.#status({ phase: 'compacting', code: 'compacting', text: FIT_TEXT.summarizing, key: FIT_TEXT.summarizing, values: [] });
break;
case 'compaction/end':
if (!this.summarizing) break;
this.summarizing = false;
this.#afterTool();
if (!d.error) {
const before = this.window > 0 && this.contextTokens ? Math.min(100, Math.round(100 * this.contextTokens / this.window)) : null;
const text = before === null ? FIT_TEXT.summarizedShort : FIT_TEXT.summarizedWas;
const values = before === null ? [] : [before];
out.push({ type: 'delta', own: true, text: '\n\n' + fill(text, values), key: text, values, noteReason: 'compacted' });
}
break;
case 'turn/start':
// The parent woke up again (a helper finished and reported back): the message goes on, as a new paragraph.
this.ended = null;
@@ -420,7 +445,7 @@ export class TurnProjector {
case 'aborted':
return [{ type: 'done', ...base, ...usage, finishReason: 'aborted', cancelled: true }];
case 'error':
return [{ type: 'error', message: friendlyFailure(reason.error), code: reason.error?.code ?? 'UNKNOWN' }];
return [{ type: 'error', message: friendlyFailure(reason.error), code: isContextFailure(reason.error) ? 'CONTEXT_WINDOW_EXCEEDED' : reason.error?.code ?? 'UNKNOWN', ...(isContextFailure(reason.error) ? { key: FIT_TEXT.full, details: String(reason.error?.message ?? '') } : {}) }];
case 'blocked':
return [{ type: 'error', message: 'The agent stopped because this turn was blocked by policy.', code: 'BLOCKED' }];
default:
@@ -429,10 +454,15 @@ export class TurnProjector {
}
}
/** Whether a failure is the model server saying the conversation does not fit its window (its code, or the engine's own words). */
export function isContextFailure(failure) {
return isContextOverflow(failure) || /context window|too long for|exceeds the context/i.test(String(failure?.message ?? ''));
}
/** A model-route failure in words a user can act on. */
export function friendlyFailure(failure) {
const message = String(failure?.message ?? 'The model request failed.');
if (/context window|too long|exceeds/i.test(message)) return `The conversation no longer fits the model's context window. ${clip(message, 200)}`;
if (isContextFailure(failure)) return FIT_TEXT.full;
if (/ECONNREFUSED|fetch failed|connect/i.test(message)) return `The model is not reachable. ${clip(message, 200)}`;
if (/401|403|credential|api key/i.test(message)) return 'The model rejected the credentials. Check the connection in Settings.';
return clip(message, 400);
+1 -1
View File
@@ -33,7 +33,7 @@ export function agentEnvironment(base = process.env, extra = {}) {
}
// DSH_PERMISSION_MODE is the composition default for a session no chat mode has touched (a child agent, a
// seeded session): the strictest sandbox, which matches the Manual preset, never the Harness's wider default.
return { ...out, DSH_TELEMETRY_DISABLED: '1', DSH_PERMISSION_MODE: 'read-only', ...extra };
return { ...out, DSH_TELEMETRY_DISABLED: '1', DSH_PERMISSION_MODE: 'read-only', NODE_USE_SYSTEM_CA: '1', ...extra };
}
/**
+1 -1
View File
@@ -117,7 +117,7 @@ export function stepObjective(plan, index) {
`You are on step ${index + 1} of ${n}: ${s.text}${s.detail ? ` (${s.detail})` : ''}.`,
done ? `Already done: ${done}.` : '',
plan.briefing ? String(plan.briefing) : '',
isViewStep(s.text) ? 'This step is about seeing the result in a browser. Never wait for a program to close. If a person is at this computer, start the browser so the command returns at once (Windows: Start-Process; macOS: open; Linux: xdg-open) and say it is open. If there is no screen, check the page without a visible window: use the headless browser tool if you have one, otherwise read the file and its script for mistakes, and say in one plain sentence what you could and could not check. Then finish the step.' : '',
isViewStep(s.text) ? 'This step is about seeing the result. To show a local image, SVG, HTML page or PDF, use the present tool (a browser cannot open file:// addresses). Never wait for a program to close. If a person is at this computer and you start a browser, start it so the command returns at once (Windows: Start-Process; macOS: open; Linux: xdg-open) and say only that you started it, not that the page rendered. If there is no screen, check the page without a visible window: use the headless browser tool if you have one, otherwise read the file and its script for mistakes, and say in one plain sentence what you could and could not check. Then finish the step.' : '',
'Do only this step, check that it worked, and then finish the goal.',
index + 1 < n ? 'The next steps are handled separately: do not start them.' : '',
].filter(Boolean).join(' ');
@@ -0,0 +1,159 @@
/**
* Strata's context plugin for the Harness (plain ESM, loaded by absolute path from the profile, like strata-gate and strata-models).
*
* What it fixes (2026-10-22 hands-on test, Gyro-S at 64K):
* 1. Compaction decided from the Harness's own price of the conversation, 4 characters a token. Code, logs and JSON are about twice as
* dense, so a chat the engine had read as 59,048 tokens was priced at 31,794 and compaction started far too late. Here the token meter
* is calibrated by what the model server reports for each answer (its real prompt size): the measured size is exact, and what was
* added since is priced with the density that measurement showed. Automatic compaction and the manual /compact both measure through
* the token meter, so both read the same number as the context meter in the app.
* 2. The summarising request itself could be bigger than the window ("/compact" sent a 66,507-token prompt to a 65,536 window). The
* summary is now asked for in pieces that fit: a message bigger than a piece is cut for the request (the conversation keeps it until
* the summary replaces it), the pieces are summarised one after the other, each building on the summary before it, and a piece the
* server still refuses is halved and tried again.
*
* How. It wraps `tokenMeter.measure` and `llm.stream` in place, once each (the same way strata-models wraps the subagent tools). Only a
* request the Harness marks `purpose: 'compaction'` is touched in `llm.stream`. Nothing in the Harness is rewritten: a Harness update
* that renames either leaves compaction as it was, and the test that runs the real Harness fails.
*/
import { densityOf, estimateMessage, estimateTools, isContextOverflow, MAX_DENSITY, pressureNow, promptTokensOf, splitMessages, summaryBudget, usageOfEvent, WINDOW_SLACK } from '../context-budget.mjs';
export const name = 'strata-context';
export const inject = ['tokenMeter', 'llm'];
/** Real prompt measurements per session: { real, estimated } for the last answer the model server reported. */
const samples = new WeakMap();
const bySessionId = new Map();
const wrapped = new WeakSet();
/** Densities that made a summary request fail on this server, per session id (the next request starts from a more careful one). */
const careful = new Map();
export function apply(ctx) {
const meter = ctx.tokenMeter;
const llm = ctx.llm;
const log = (level, text) => { try { ctx.logger?.[level]?.(`strata-context: ${text}`); } catch { /* logging never blocks */ } };
// 1. The measured size of the prompt, from the model server's own report for each answer.
if (meter && typeof meter.measure === 'function' && !wrapped.has(meter)) {
wrapped.add(meter);
const original = meter.measure;
meter.measure = function measure(session, requestHeader) {
const m = original.call(this, session, requestHeader);
const sample = samples.get(session);
if (!sample) return m;
const now = m.surfaceTokens + estimateTools(requestHeader ?? session.requestHeader?.());
const total = Math.max(m.totalTokens, pressureNow({ sample, now }));
if (total <= m.totalTokens) return m;
const density = densityOf(sample.real, sample.estimated);
return { ...m, totalTokens: total, nodes: m.nodes.map(n => ({ ...n, tokens: Math.ceil(n.tokens * density) })) };
};
ctx.on('session/event', (session, event) => {
if (event.type !== 'assistant/message') return;
const real = promptTokensOf(usageOfEvent(event));
if (!(real > 0)) return;
try {
// The request that was just answered saw the conversation without this answer: its price is the surface before the answer's node.
const m = original.call(meter, session);
const heuristic = m.nodes.reduce((n, x) => n + (x.seq === event.seq ? 0 : x.heuristicTokens), 0);
const estimated = heuristic + estimateTools(session.requestHeader?.());
if (!(estimated > 0)) return;
const sample = { real, estimated };
samples.set(session, sample);
if (session.id) bySessionId.set(String(session.id), sample);
} catch (e) { log('warn', `could not record a measurement: ${e.message}`); }
});
}
// 2. A summary request that fits the window.
if (llm && typeof llm.stream === 'function' && !wrapped.has(llm)) {
wrapped.add(llm);
const original = llm.stream;
llm.stream = function stream(options) {
if (options?.purpose !== 'compaction') return original.call(this, options);
return summaryStream.call(this, options, original);
};
}
// The host asks, with /strata-context, how much of the window the chat holds right now: the same measured number compaction decides on
// (JSON in the command's text: { tokens, window }). Before a message goes out, and after /compact, so the person is told true figures.
ctx.inject(['commands'], (commandCtx) => {
commandCtx.commands.register({
definitionId: 'strata-context',
name: 'strata-context',
description: 'Reports how many tokens the conversation holds now and the size of the model window',
handler: async (invocation) => {
try {
const session = invocation.agent.session;
const tokens = meter.measure(session).totalTokens;
const config = session.requestHeader?.()?.config;
let window = 0;
if (config) window = (await llm.resolveModelInfo(config.provider, config.model))?.context?.contextWindow ?? 0;
return { kind: 'success', text: JSON.stringify({ tokens, window }) };
} catch (e) { return { kind: 'error', text: `strata-context: ${e.message}` }; }
},
});
});
/** A finished stream: its chunks, the text it wrote and why it ended. */
async function drain(iterable) {
let text = '', finish = null;
const chunks = [];
for await (const chunk of iterable) {
chunks.push(chunk);
if (chunk.type === 'text-delta') text += chunk.text ?? '';
else if (chunk.type === 'finish') finish = chunk.reason;
}
return { text, finish, chunks };
}
const refusedForSize = finish => finish?.kind === 'error' && isContextOverflow(finish.failure ?? finish);
async function* summaryStream(options, original) {
let window = 0;
try { window = (await llm.resolveModelInfo(options.provider, options.model, options.signal))?.context?.contextWindow ?? 0; } catch { window = 0; }
const messages = options.messages ?? [];
const key = String(options.sessionId ?? '');
const summaryTokens = options.maxTokens ?? 4096;
if (!(window > 0) || messages.length < 2) { yield* original.call(this, options); return; }
const own = bySessionId.get(key);
let density = Math.max(careful.get(key) ?? 1, own ? densityOf(own.real, own.estimated) : 1);
const price = list => list.reduce((n, m) => n + estimateMessage(m), 0) * density;
const limit = window - summaryTokens - WINDOW_SLACK;
// The whole request fits by this price: ask as the Harness would (the model server's cache of the conversation is reused).
if (price(messages) + estimateTools({ tools: options.tools }) * density <= limit) {
const first = await drain(original.call(this, options));
if (!refusedForSize(first.finish)) { for (const chunk of first.chunks) yield chunk; return; }
density = Math.min(MAX_DENSITY, density * 1.6);
careful.set(key, density);
log('warn', 'the summary request did not fit the window; asking in pieces');
}
// In pieces. The instruction is the last message; the system prompt, when there is one, the first.
const instruction = messages[messages.length - 1];
const system = messages[0]?.role === 'system' ? messages[0] : null;
const body = messages.slice(system ? 1 : 0, -1);
let tools = options.tools;
if (tools && estimateTools({ tools }) * density > window / 3) tools = undefined; // a long tool list only describes tools: the summary does not need it
const fixed = (system ? estimateMessage(system) : 0) * density + estimateTools({ tools }) * density + estimateMessage(instruction) * density;
let budget = summaryBudget({ window, summaryTokens: summaryTokens * 2, systemTokens: fixed, instructionTokens: 0 });
for (let attempt = 0; ; attempt++) {
const runs = splitMessages(body, Math.max(1024, budget), density);
log('info', `summarising ${body.length} messages in ${runs.length} piece(s) for a ${window}-token window (density ${density.toFixed(2)}, about ${Math.round(budget)} tokens a piece)`);
let failed = null, summary = '';
for (let i = 0; i < runs.length && !failed; i++) {
options.signal?.throwIfAborted?.();
const carried = summary ? [{ role: 'user', content: [{ type: 'text', text: `<compacted-summary>\n${summary}\n</compacted-summary>` }] }] : [];
const { tools: _drop, ...rest } = options;
const piece = { ...rest, ...(tools ? { tools } : {}), messages: [...(system ? [system] : []), ...carried, ...runs[i], instruction] };
const { text, finish, chunks } = await drain(original.call(this, piece));
if (refusedForSize(finish)) { failed = finish; break; }
// The last piece's answer is the summary the Harness reads; a piece that ended in another error ends the request the same way.
if (i === runs.length - 1 || finish?.kind === 'error' || finish?.kind === 'aborted') { for (const chunk of chunks) yield chunk; return; }
summary = text;
}
if (!failed || attempt >= 3) throw Object.assign(new Error(failed?.failure?.message ?? 'the summary request does not fit the window'), { code: 'CONTEXT_WINDOW_EXCEEDED' });
density = Math.min(MAX_DENSITY, density * 1.6);
careful.set(key, density);
budget = Math.max(1024, budget / 2);
log('warn', `a piece did not fit; asking again with pieces of about ${Math.round(budget)} tokens`);
}
}
}
+239 -34
View File
@@ -48,28 +48,230 @@ export const READ_ONLY_TOOLS = new Set([
const SHELL_TOOLS = new Set(['pwsh', 'bash']);
/** Commands that only look. Matched against the first word of every pipeline segment. */
const SAFE_COMMANDS = new Set([
// PowerShell
'get-childitem', 'gci', 'ls', 'dir', 'get-content', 'gc', 'cat', 'type', 'get-item', 'gi', 'get-location', 'pwd', 'gl',
'select-string', 'sls', 'measure-object', 'measure', 'select-object', 'select', 'where-object', 'where', 'sort-object', 'sort',
'format-table', 'ft', 'format-list', 'fl', 'out-string', 'test-path', 'resolve-path', 'get-filehash', 'get-date', 'get-command',
'get-process', 'get-itemproperty', 'write-output', 'echo', 'hostname', 'whoami', 'get-host', 'get-psdrive', 'get-help',
'group-object', 'group', 'convertto-json', 'convertfrom-json', 'select-xml', 'compare-object', 'get-unique', 'tee-object-none',
// POSIX
'head', 'tail', 'wc', 'grep', 'rg', 'egrep', 'fgrep', 'stat', 'file', 'du', 'df', 'uname', 'which', 'basename', 'dirname',
'realpath', 'readlink', 'sort', 'uniq', 'cut', 'tr', 'less', 'more', 'date', 'id', 'env-none', 'tree',
]);
/**
* Which shell commands run without a card in Manual mode, and with which options. Safe by construction (#3975): a command is on the list WITH the
* options that are known to only look. Every other option, long or short, abbreviated or attached, makes the command ask. So a write mode
* (`sort -o`, `tree -o`, `uniq in out`), a program-execution mode (`sort --compress-program`, `rg --pre`, `git grep -O`), a setting mode
* (`date -s`) or one that never ends (`tail -f`) cannot slip in, and a word added to the list later starts with no options at all.
*
* A spec says what one command accepts:
* f short options without a value (letters, so `-la` is `-l -a`) v short options that need a value (`-k2`, `-k 2`)
* vn the same, and the value must be a number o short options with an optional value that must be attached (`-Iseconds`)
* n a bare `-5` is allowed l long options (`--name`, `--name=value`), spelled in full
* lv long options that need a value (`--name=value`, `--name value`) ps PowerShell parameter names, lower case, spelled in full
* max how many operands (files, patterns) are allowed op a check for each operand
* post a last check from the options seen and the operand count
*/
const words = (text) => (text ? text.trim().split(/\s+/) : []);
const spec = ({ f = '', v = '', vn = '', o = '', n = false, l = '', lv = '', ps = '', max = Infinity, op = null, post = null }) =>
({ f, v, vn, o, n, l: new Set(words(l)), lv: new Set(words(lv)), ps: ps ? new Set([...words(ps), ...PS_COMMON]) : null, max, op, post });
/** Parameters every cmdlet has that only change how errors and warnings show. (-Debug asks a question and -OutVariable sets one: not listed.) */
const PS_COMMON = ['erroraction', 'ea', 'warningaction', 'wa', 'informationaction', 'ia', 'verbose', 'vb'];
const NUMBER = /^[+-]?\d+[A-Za-z]{0,3}$/;
/** git subcommands that only read. */
const SAFE_GIT = new Set(['status', 'log', 'diff', 'show', 'branch', 'rev-parse', 'ls-files', 'blame', 'describe', 'remote', 'tag', 'shortlog', 'rev-list', 'cat-file', 'grep', 'ls-tree', 'config']);
const SAFE_VERSION = /^(node|python|python3|py|npm|pnpm|git|dotnet|cargo|rustc|go|java|pip|pip3)\s+(--version|-v|-version|version)$/i;
/** What `ls`, `dir`, `gci` and `Get-ChildItem` take. */
const LS = { f: 'aAbBcdDFgGhHiklLmnNopqQrRsStuUvxX1CZ', v: 'ITw', ps: 'path literalpath filter include exclude recurse depth name force file directory hidden readonly system attributes',
l: 'all almost-all human-readable recursive directory classify reverse size inode dereference color group-directories-first sort time time-style block-size indicator-style si numeric-uid-gid no-group literal hide ignore quote-name escape author file-type full-time width tabsize' };
/** What `cat`, `type`, `gc` and `Get-Content` take. (-Wait never ends, so it is not here.) */
const CAT = { f: 'AbeEnstTuv', ps: 'path literalpath totalcount tail head raw encoding readcount force filter include exclude delimiter stream asbytestream',
l: 'show-all number-nonblank show-ends number squeeze-blank show-tabs show-nonprinting' };
const MEASURE = { ps: 'property sum average maximum minimum line word character ignorewhitespace inputobject standarddeviation allstats' };
const SELECT = { ps: 'first last skip skiplast property expandproperty unique index excludeproperty inputobject' };
const WHERE = { ps: 'property value inputobject eq ne gt lt ge le like notlike match notmatch contains notcontains in notin is isnot ceq cne cgt clt cge cle clike cnotlike cmatch cnotmatch ccontains cnotcontains cin cnotin' };
const GROUP = { ps: 'property noelement ashashtable asstring casesensitive inputobject culture' };
const FT = { ps: 'property autosize wrap hidetableheaders groupby view force expand repeatheader displayerror showerror inputobject' };
const FL = { ps: 'property view groupby expand force displayerror showerror inputobject' };
const SLS = { ps: 'pattern path literalpath simplematch casesensitive context list notmatch quiet allmatches include exclude encoding raw inputobject culture noemphasis' };
const GETITEM = { ps: 'path literalpath name filter include exclude force stream' };
const GETLOC = { ps: 'psprovider psdrive stack stackname' };
const ECHO = { f: 'neE', ps: 'inputobject noenumerate' };
/** The GNU/BSD grep family. */
const GREP = { f: 'aiIlLcnvwHhEFGPqrRsTUyzZbxo', v: 'efdD', vn: 'ABCm', n: true,
l: 'extended-regexp fixed-strings basic-regexp perl-regexp ignore-case no-ignore-case word-regexp line-regexp invert-match count color colour files-with-matches files-without-match max-count only-matching quiet silent no-messages byte-offset with-filename no-filename label line-buffered initial-tab null after-context before-context context text recursive dereference-recursive include exclude exclude-dir binary-files line-number null-data no-group-separator group-separator regexp file directories devices binary' };
/** Anything that can chain, redirect, expand or start something else is not read-only. */
const UNSAFE_SYNTAX = /[;&<>`${}()]|\r|\n|--%|\bout-file\b|\bset-content\b|\badd-content\b|\binvoke-|\biex\b|\bstart-|\bnew-|\bremove-|\bcopy-|\bmove-|\brename-|\bclear-|\bset-|\bpush-|\bexpand-|\bsave-|\binstall-|\bcurl\b|\bwget\b|\bsudo\b|\bsh\b|\bbash\b|\bpwsh\b|\bpowershell\b|\bcmd\b|-exec\b|-delete\b|-execdir\b/i;
/** Commands that only look, each with the options that are known to be harmless. Matched against the first word of every pipeline segment. */
const SPECS = new Map();
const add = (names, s) => { for (const name of words(names)) SPECS.set(name, spec(s)); };
// PowerShell
add('get-childitem gci', { ps: LS.ps });
add('ls dir', LS);
add('get-content gc', { ps: CAT.ps });
add('cat type', CAT);
add('get-item gi get-itemproperty', GETITEM);
add('get-location gl', GETLOC);
add('pwd', { ...GETLOC, f: 'LP' });
add('select-string sls', SLS);
add('measure-object measure', MEASURE);
add('select-object select', SELECT);
add('where-object where', WHERE);
add('sort-object', { ps: 'property descending unique top bottom casesensitive culture stable inputobject' });
add('format-table ft', FT);
add('format-list fl', FL);
add('out-string', { ps: 'stream width inputobject' });
add('test-path', { ps: 'path literalpath pathtype isvalid filter include exclude olderthan newerthan' });
add('resolve-path', { ps: 'path literalpath relative relativebasepath force' });
add('get-filehash', { ps: 'path literalpath algorithm' });
add('get-date', { ps: 'format uformat date asutc displayhint year month day hour minute second millisecond' });
add('get-command', { ps: 'name commandtype module all syntax listimported totalcount noun verb fullyqualifiedmodule parametername parametertype' });
add('get-process', { ps: 'name id includeusername module fileversioninfo inputobject' });
add('write-output', { ps: ECHO.ps });
add('echo', ECHO);
add('get-host', { ps: '' });
add('get-psdrive', { ps: 'name psprovider scope literalname' });
add('get-help', { ps: 'name full detailed examples parameter category component functionality role' });
add('group-object group', GROUP);
add('convertto-json', { ps: 'depth compress enumsasstrings inputobject asarray escapehandling' });
add('convertfrom-json', { ps: 'ashashtable depth inputobject noenumerate' });
add('select-xml', { ps: 'xml path literalpath xpath content namespace' });
add('compare-object', { ps: 'referenceobject differenceobject property includeequal excludedifferent passthru syncwindow casesensitive culture' });
add('get-unique', { ps: 'inputobject asstring caseinsensitive onetype' });
// POSIX. `sort` and `date` take no PowerShell names here: `sort -o FILE` writes, `date -s` sets the clock, and a PowerShell-looking `-Second` or `-Top`
// must not be read by a real sort or date as one of those. (Sort-Object and Get-Date above are the PowerShell forms.)
add('head', { f: 'qvz', vn: 'nc', n: true, l: 'quiet silent verbose zero-terminated lines bytes' });
add('tail', { f: 'qvz', vn: 'nc', n: true, l: 'quiet silent verbose zero-terminated lines bytes' }); // no -f, -F, --follow, --retry, --pid, -s: those never end
add('wc', { f: 'lwcmL', l: 'lines words bytes chars max-line-length' });
add('grep egrep fgrep', GREP);
add('rg', { // no --pre, --pre-glob, --hostname-bin (they run a program) and no -z/--search-zip (it runs decompressors)
f: 'iIsSnNHlcwFxvUoLaquhVpbP', v: 'efgtTr', vn: 'ABCjdMm',
l: 'ignore-case no-ignore-case smart-case case-sensitive fixed-strings word-regexp line-regexp invert-match count count-matches files-with-matches files-without-match files type-list hidden no-ignore no-ignore-vcs no-ignore-parent no-ignore-dot no-ignore-global no-ignore-messages follow line-number no-line-number with-filename no-filename column heading no-heading null only-matching quiet text multiline multiline-dotall pcre2 no-messages json stats trim unrestricted binary no-config vimgrep pretty byte-offset passthru crlf one-file-system no-unicode regexp file glob iglob type type-not max-count max-depth maxdepth context after-context before-context color replace threads sort sortr max-filesize max-columns path-separator encoding engine ignore-file glob-case-insensitive no-glob-case-insensitive' });
add('stat', { f: 'LtxlnqrsFf', v: 'c', l: 'dereference file-system terse format printf' });
add('file', { f: 'bhikLnNrvI', l: 'brief dereference no-dereference mime mime-type mime-encoding keep-going no-pad no-buffer raw' }); // no -C, -m, -f, -P (they write or read other files), no -z/-s/-p (they run decompressors, read devices, reset times)
add('du', { f: 'abchHklmsxLPSD0', v: 'tB', vn: 'd', l: 'all apparent-size bytes total dereference-args human-readable si summarize one-file-system dereference no-dereference separate-dirs count-links inodes null max-depth threshold block-size time time-style exclude' });
add('df', { f: 'ahHiklmPTv', v: 'txB', l: 'all human-readable si inodes local portability print-type total sync no-sync output block-size type exclude-type' });
add('uname', { f: 'asnrvmpio', max: 0, l: 'all kernel-name nodename kernel-release kernel-version machine processor hardware-platform operating-system' });
add('which', { f: 'as', l: 'all' });
add('basename', { f: 'az', v: 's', l: 'multiple zero suffix' });
add('dirname', { f: 'z', l: 'zero' });
add('realpath', { f: 'emsqzLP', l: 'canonicalize-existing canonicalize-missing no-symlinks quiet zero strip logical physical relative-to relative-base' });
add('readlink', { f: 'femnqsvz', l: 'canonicalize canonicalize-existing canonicalize-missing no-newline quiet silent verbose zero' });
add('sort', { f: 'bdfghiMnRrsuVzmcC', v: 'kt', l: 'ignore-leading-blanks dictionary-order ignore-case general-numeric-sort human-numeric-sort ignore-nonprinting month-sort numeric-sort random-sort reverse stable unique version-sort zero-terminated check merge sort field-separator key' });
// uniq's second operand is its OUTPUT file, so it takes at most one.
add('uniq', { f: 'cdDuiz', vn: 'fsw', max: 1, l: 'count repeated all-repeated unique ignore-case skip-fields skip-chars check-chars zero-terminated group' });
add('cut', { f: 'snz', v: 'bcfd', l: 'complement only-delimited zero-terminated bytes characters fields delimiter output-delimiter' });
add('tr', { f: 'cCdst', max: 2 });
add('date', { f: 'uR', o: 'I', v: 'dr', lv: 'date reference', l: 'utc universal rfc-email rfc-2822 iso-8601 rfc-3339',
op: (word) => word.startsWith('+') }); // a bare operand like 1026120026 SETS the clock; only +FORMAT is a format
add('id', { f: 'ugGnrazZ', l: 'user group groups name real context zero' });
add('hostname', { f: 'sfiIdaAyv', max: 0 }); // `hostname NAME` renames the machine
add('whoami', { l: 'help version', op: (word) => /^\/(all|user|groups|priv|upn|fqdn|logonid|nh)$/i.test(word) });
add('tree', { f: 'adfilpsugDFqNQrtvxCnASUchJX', v: 'LPIHT', vn: '', l: 'help version filelimit charset sort timefmt noreport dirsfirst filesfirst prune du si gitignore matchdirs ignore-case inodes device nolinks' }); // no -o, no -R
// `less` and `more` are not here: they wait for keys, and `less` can run commands.
/** The names that run without a card (for the tests). */
export const readOnlyCommandNames = () => [...SPECS.keys()];
/** git subcommands that only read, with the options they may take. */
const GIT_VIEW = 'oneline graph decorate no-decorate all branches tags remotes stat shortstat numstat dirstat name-only name-status patch no-patch raw summary compact-summary abbrev-commit no-abbrev-commit abbrev reverse follow no-merges merges first-parent full-history simplify-by-decoration simplify-merges ancestry-path left-right left-only right-only cherry-pick cherry-mark cherry topo-order date-order author-date-order relative-date all-match invert-grep regexp-ignore-case extended-regexp fixed-strings perl-regexp no-color color color-words word-diff word-diff-regex minimal patience histogram diff-algorithm check exit-code quiet cached staged merge-base no-index no-ext-diff no-textconv find-renames find-copies find-copies-harder no-renames ignore-space-change ignore-all-space ignore-space-at-eol ignore-blank-lines ignore-cr-at-eol function-context show-function unified inter-hunk-context src-prefix dst-prefix no-prefix full-index binary text source use-mailmap mailmap no-mailmap log-size notes no-notes show-signature diff-merges boundary children parents no-walk sparse date format pretty author committer grep since until after before max-count skip min-parents max-parents diff-filter encoding stat-width stat-name-width stat-graph-width stat-count count';
const LISTING = ['l', 'list', 'contains', 'no-contains', 'merged', 'no-merged', 'points-at'];
const GIT = new Map(Object.entries({
status: spec({ f: 'sbvz', o: 'u', l: 'short branch porcelain long ignored untracked-files ignore-submodules renames no-renames find-renames show-stash ahead-behind no-ahead-behind column no-column verbose null' }),
log: spec({ f: 'pswbMCizuacmrtqEFPR', vn: 'nU', v: 'SGL', n: true, l: GIT_VIEW }),
diff: spec({ f: 'pswbMCizuacmrtqEFPR', vn: 'nU', v: 'SGL', n: true, l: GIT_VIEW }),
show: spec({ f: 'pswbMCizuacmrtqEFPR', vn: 'nU', v: 'SGL', n: true, l: GIT_VIEW }),
'rev-list': spec({ f: 'pswbMCizuacmrtqEFPR', vn: 'nU', v: 'SGL', n: true, l: GIT_VIEW }),
shortlog: spec({ f: 'sneciw', n: false, l: `${GIT_VIEW} summary numbered email committer group` }),
// Listing forms only. A name with no list option would CREATE a branch or a tag, and the options that change things are not here.
branch: spec({ f: 'avri', l: 'list all remotes verbose show-current ignore-case contains no-contains merged no-merged points-at sort format abbrev no-abbrev color no-color column no-column omit-empty',
post: (seen, count) => count === 0 || LISTING.some(name => seen.has(name)) }),
tag: spec({ f: 'li', o: 'n', l: 'list sort contains no-contains merged no-merged points-at format column no-column ignore-case omit-empty color no-color',
post: (seen, count) => count === 0 || LISTING.some(name => seen.has(name)) }),
remote: spec({ f: 'v', l: 'verbose', max: 0 }),
// `git config KEY` reads and `git config KEY VALUE` writes, so a key is only taken with --get; no --edit, --add, --unset, --file.
config: spec({ f: 'lz', l: 'list get get-all get-regexp show-origin show-scope local global system worktree bool int bool-or-int path expiry-date type null name-only includes no-includes', lv: 'default',
post: (seen, count) => (count > 0 ? ['get', 'get-all', 'get-regexp'].some(name => seen.has(name)) : seen.has('list') || seen.has('l')) }),
'rev-parse': spec({ f: 'q', l: 'show-toplevel git-dir git-common-dir absolute-git-dir abbrev-ref symbolic symbolic-full-name short verify is-inside-work-tree is-inside-git-dir is-bare-repository is-shallow-repository show-prefix show-cdup show-superproject-working-tree all branches tags remotes quiet revs-only no-revs flags no-flags local-env-vars show-object-format path-format', lv: 'default git-path' }),
'ls-files': spec({ f: 'cdmoiskuzvtfD', l: 'cached deleted modified others ignored stage directory unmerged killed no-empty-directory error-unmatch exclude-standard exclude full-name eol deduplicate recurse-submodules sparse abbrev resolve-undo format' }),
'ls-tree': spec({ f: 'rtdlz', l: 'name-only name-status long full-name full-tree object-only abbrev format' }),
blame: spec({ f: 'wMCespltbfnc', v: 'L', l: 'porcelain line-porcelain incremental show-email show-name show-number date root abbrev' }),
describe: spec({ l: 'all tags always long dirty broken contains abbrev match exclude first-parent exact-match candidates' }),
'cat-file': spec({ f: 'tsep', l: 'batch batch-check batch-all-objects allow-unknown-type buffer unordered follow-symlinks mailmap use-mailmap' }), // no --textconv, --filters
// No -O / --open-files-in-pager: they run a program on the matches.
grep: spec({ f: 'aiIlLcnvwHhEFGPqpWrzUo', v: 'e', vn: 'ABCm', n: true, l: 'no-index untracked cached no-exclude-standard exclude-standard recurse-submodules text ignore-case word-regexp invert-match full-name extended-regexp basic-regexp perl-regexp fixed-strings line-number column files-with-matches name-only files-without-match count null only-matching quiet no-color color break heading show-function function-context all-match and or not max-depth max-count threads after-context before-context context recursive no-recursive' }),
}));
/**
* Whether a shell command only reads. Conservative by design: one unrecognised word and the answer is no.
* One command's words against its spec: every option has to be one the spec knows (anything else is not understood, so it asks).
* @param {ReturnType<typeof spec>} sp
* @param {{t:string,q:boolean}[]} args the words after the command; q marks a quoted text, which is data and never an option
*/
function argumentsAreSafe(sp, args) {
const seen = new Set();
let operands = 0, ended = false;
const operand = (arg) => { operands++; return operands <= sp.max && (!sp.op || sp.op(arg.t, arg.q)); };
const valueOf = (i, numeric) => { // the next word as an option's value; it can be a quoted text, a number, or a word that is not an option
const arg = args[i + 1];
if (!arg) return false;
return numeric ? NUMBER.test(arg.t) : arg.q || !arg.t.startsWith('-') || NUMBER.test(arg.t);
};
for (let i = 0; i < args.length; i++) {
const { t, q } = args[i];
if (q || ended || t === '-' || t[0] !== '-') { if (!operand(args[i])) return false; continue; }
if (t === '--') { ended = true; continue; }
if (t.startsWith('--')) {
const eq = t.indexOf('=');
const name = eq < 0 ? t.slice(2) : t.slice(2, eq);
if (sp.l.has(name)) { seen.add(name); continue; }
if (sp.lv.has(name)) { seen.add(name); if (eq < 0) { if (!valueOf(i, false)) return false; i++; } continue; }
return false;
}
if (sp.ps) {
const named = /^-([A-Za-z][A-Za-z0-9]*)(?::|$)/.exec(t);
if (named && sp.ps.has(named[1].toLowerCase())) { seen.add(named[1].toLowerCase()); continue; }
}
if (sp.n && /^-\d+$/.test(t)) continue;
for (let j = 1; j < t.length; j++) {
const c = t[j];
if (sp.f.includes(c)) { seen.add(c); continue; }
if (sp.o.includes(c)) { seen.add(c); break; }
const numeric = sp.vn.includes(c);
if (!numeric && !sp.v.includes(c)) return false;
seen.add(c);
const rest = t.slice(j + 1);
if (rest) { if (numeric && !NUMBER.test(rest)) return false; } else { if (!valueOf(i, numeric)) return false; i++; }
break;
}
}
return sp.post ? sp.post(seen, operands) : true;
}
const SAFE_VERSION = /^(node|python|python3|py|npm|pnpm|git|dotnet|cargo|rustc|go|java|pip|pip3)\s+(--version|-v|-version|version)$/i;
/** Anything that can chain, redirect, expand or start something else is not read-only. (`sh` as a word, not inside an option like `du -sh`.) */
const UNSAFE_SYNTAX = /[;&<>`${}()]|\r|\n|--%|\bout-file\b|\bset-content\b|\badd-content\b|\binvoke-|\biex\b|\bstart-|\bnew-|\bremove-|\bcopy-|\bmove-|\brename-|\bclear-|\bset-|\bpush-|\bexpand-|\bsave-|\binstall-|\bcurl\b|\bwget\b|\bsudo\b|(?<![-\w])sh(?![-\w])|\bbash\b|\bpwsh\b|\bpowershell\b|\bcmd\b|-exec\b|-delete\b|-execdir\b/i;
/**
* The command with every quoted text replaced by a plain stand-in, or null when it cannot be read safely. What is inside quotes is data
* (a path like "C:\Users\x\Q&A", a pattern like 'a|b'), so an & ; | ( ) < > or { in it must not decide anything. Two things still can:
* inside double quotes a $ or a backtick expands (a variable, a sub-command), and an unterminated quote means the text was not understood.
* Either one gives null, and the command asks as before.
* @param {string} text
* @param {string[]} [quoted] when given, receives the text inside each quote, in order (the checks on operands read it)
* @returns {string|null}
*/
export function maskQuotes(text, quoted) {
let out = '';
for (let i = 0; i < text.length; i++) {
const c = text[i];
if (c !== '"' && c !== "'") { out += c; continue; }
let end = text.indexOf(c, i + 1);
// PowerShell writes a quote inside a quoted text by doubling it ('it''s'): the text goes on past the pair.
while (end >= 0 && text[end + 1] === c) end = text.indexOf(c, end + 2);
if (end < 0) return null;
if (c === '"' && /[$`]/.test(text.slice(i + 1, end))) return null;
// A quoted text is a whole word. One glued to other characters (--out"put"=f, "a"b) or one that starts with a dash could hide an option from the
// word checks below (git diff "--output=f" writes a file), so those are not understood either.
const before = text[i - 1], after = text[end + 1];
if ((before !== undefined && !/[\s|]/.test(before)) || (after !== undefined && !/[\s|]/.test(after)) || text[i + 1] === '-') return null;
out += '"_"';
quoted?.push(text.slice(i + 1, end));
i = end;
}
return out;
}
/**
* Whether a shell command only reads. Conservative by design: one unrecognised word or option and the answer is no.
* Quoted text is data and is looked at after it has been set aside (maskQuotes): Get-ChildItem "C:\Users\x\Q&A" only lists.
* @param {string} command
* @returns {boolean}
*/
@@ -77,23 +279,21 @@ export function isReadOnlyShell(command) {
const text = String(command ?? '').trim();
if (!text || text.length > 600) return false;
if (SAFE_VERSION.test(text)) return true;
if (UNSAFE_SYNTAX.test(text)) return false;
const segments = text.split('|').map(s => s.trim());
return segments.every((segment) => {
if (!segment) return false;
const words = segment.split(/\s+/);
const first = words[0].toLowerCase().replace(/^\.\\/, '');
const quoted = [];
const masked = maskQuotes(text, quoted);
if (masked === null || UNSAFE_SYNTAX.test(masked)) return false;
let next = 0;
const tokens = masked.split('|').map(segment => segment.trim()).map(segment => words(segment).map(w => (w === '"_"' ? { t: quoted[next++] ?? '', q: true } : { t: w, q: false })));
return tokens.every((segment) => {
if (!segment.length || segment[0].q) return false;
const first = segment[0].t.toLowerCase().replace(/^\.\\/, '');
if (first === 'git') {
// `git config` and `git tag`/`branch`/`remote` can write when given a value; only the bare listing forms are safe.
const sub = (words[1] ?? '').toLowerCase();
if (!SAFE_GIT.has(sub)) return false;
if (['config', 'tag', 'branch', 'remote'].includes(sub)) {
const rest = words.slice(2).filter(w => !w.startsWith('-') || /^(-l|-a|-v|-vv|--list|--all|--show-current|--get|--get-all)$/i.test(w));
return rest.length === 0 || (sub === 'config' && words[2] === '--get') || (sub === 'remote' && words[2] === '-v');
}
return !words.some(w => /^--(output|ext-diff|textconv)\b/i.test(w));
const sub = segment[1];
const sp = sub && !sub.q ? GIT.get(sub.t) : null;
return !!sp && argumentsAreSafe(sp, segment.slice(2));
}
return SAFE_COMMANDS.has(first) && !words.some(w => /^-(delete|exec|execdir|fprint|fls|ok)$/i.test(w));
const sp = SPECS.get(first);
return !!sp && argumentsAreSafe(sp, segment.slice(1));
});
}
@@ -219,8 +419,13 @@ export function apply(ctx, config = {}) {
ctx.on('tools/pre-execute', async (exec, next) => {
const agent = exec.agent;
if (!agent || exec.name === 'run_code') return next();
// Fail closed (T4): a session whose preset cannot be read (a reset in progress, a helper whose chat is gone) is treated as Manual, so
// anything this file cannot prove is read-only still asks. It used to run unasked.
let preset;
try { preset = effectivePreset(agent.session); } catch { return next(); }
try { preset = effectivePreset(agent.session); } catch (e) {
preset = MANUAL_PRESET;
try { console.warn(`[strata-gate] could not read the permission mode for this call (${e?.message ?? e}); treating it as Manual`); } catch { /* logging must never decide a permission */ }
}
// A helper that was set to ask while the chat was in Manual must stop asking the moment the chat leaves Manual,
// so a mode change applies to the very next tool call of a running goal and its helpers.
if (preset !== MANUAL_PRESET && agent.session.header?.origin === 'subagent' && ctx.approval.overrideOf(agent.session) === 'ask') {
+2 -2
View File
@@ -42,8 +42,8 @@ export const AGENT_PROMPT = [
'You are a helpful assistant that can also work on this computer through your tools. You are running as {{model}}.',
'Answer questions directly and briefly; use a tool only when the task needs one. The person reads your replies in a chat: use plain words, not tool names or technical jargon.',
'Look before you change: read a file before you edit or overwrite it, and prefer a small edit to a rewrite. Search with the glob and grep tools and read files with the read tool; use the shell for running programs.',
'When you start a program or open a folder or file for the person, the command returning without an error means it worked: say plainly that it is open. Do not say you could not confirm it. If a command fails, say what went wrong in one sentence and what you will try next.',
'To open a page or a file for the person (a browser, an app, a folder), start it so the command returns at once and never wait for the program to close: on Windows use Start-Process, on macOS open, on Linux xdg-open. Then say plainly that it is open.',
'A command that starts a program, folder or page only tells you that it started. Never say a window opened, a page rendered or that you checked how it looks unless a tool confirmed it. If a command fails, say what went wrong in one sentence and what you will try next.',
'To show the person an image, SVG, HTML page or PDF from their folders, use the present tool: the app shows it next to the chat. A browser cannot open file:// addresses. If you do start a program, start it so the command returns at once and never wait for it to close: Start-Process on Windows, open on macOS, xdg-open on Linux.',
'When the person asks you to ask a subagent, a helper or another model ("ask a subagent", "ask model X"), use the subagent tool and name the model in its model field. Never look for a model with shell commands, and never wait for a helper with sleep: end your turn; its answer arrives as a new message.',
'Tokens prefixed with @ are files or folders the person pointed at. A failed or refused action changed nothing; say so, and do not claim work you did not do.',
].join('\n\n');
+42 -6
View File
@@ -20,6 +20,7 @@
import { createHash } from 'node:crypto';
import { fileURLToPath } from 'node:url';
import { standardPreset } from './preset.mjs';
import { validMcpName } from '../mcp-names.mjs';
import { BUDGETS, isPrivateHost } from './llm-shape.mjs';
import { DDG_PROVIDER_ID } from '../web-search-ddg.mjs';
@@ -55,6 +56,8 @@ export const modeOf = value => (Object.hasOwn(MODE_PRESETS, value) ? value : DEF
/** The approval gate plugin (plain ESM, loaded by absolute path). */
export const GATE_PLUGIN = fileURLToPath(new URL('./plugins/strata-gate.mjs', import.meta.url));
/** The app's web search (DuckDuckGo, or the person's own Exa / Perplexity key), registered with the Harness's web seam. */
/** Compaction that measures the real prompt and summarises in pieces that fit the window (plugins/strata-context.mjs). */
export const CONTEXT_PLUGIN = fileURLToPath(new URL('./plugins/strata-context.mjs', import.meta.url));
export const MODELS_PLUGIN = fileURLToPath(new URL('./plugins/strata-models.mjs', import.meta.url));
export const SEARCH_PLUGIN = fileURLToPath(new URL('./plugins/strata-search.mjs', import.meta.url));
/** The app's own tool server for the tools a person writes (instructions, commands). */
@@ -71,11 +74,17 @@ const NO_TOOLS_SETTINGS = Object.freeze({ off: new Set(), search: { provider: 'b
* `mcp__<server>__<tool>`. A command or URL goes into the patch as text; a secret (an env value or a header) never does: the row
* carries `!!js process.env.<NAME>` and the value travels in the agent process's environment (returned in `env`).
*/
export function mcpRows(servers, env) {
export function mcpRows(servers, env, skipped = []) {
const rows = [];
const ids = new Set();
let index = 0;
for (const [name, server] of Object.entries(servers ?? {})) {
if (!server || server.disabled === true || !/^[A-Za-z0-9_-]{1,32}$/.test(name)) continue;
if (!server || server.disabled === true) continue; // switched off on purpose: not an error
// A server that cannot be wired is REPORTED (T6): the model never sees its tools, and a silent drop looked like "the tool does not work".
if (!validMcpName(name)) { skipped.push({ name, why: 'name' }); continue; }
if (!server.command && !server.url) { skipped.push({ name, why: 'address' }); continue; }
if (ids.has(`mcp-${slug(name)}`)) { skipped.push({ name, why: 'duplicate' }); continue; }
ids.add(`mcp-${slug(name)}`);
const token = (kind, key, value) => {
const variable = `STRATA_MCP_${index}_${kind}_${slug(key).replace(/-/g, '_').toUpperCase()}`;
env[variable] = String(value);
@@ -91,6 +100,24 @@ export function mcpRows(servers, env) {
return rows;
}
/**
* What the chat says when a tool server was left out (one notice per reason). The sentences are interface copy: they are in every language's
* catalog (src/locales/tools-<lang>.json), and the {0} is the list of names exactly as the person wrote them.
*/
const SKIPPED_TEXT = {
name: 'These tool servers were left out because their names are not allowed: {0}. A name can have up to 32 letters, numbers, hyphens or underscores.',
address: 'These tool servers were left out because they have no program or web address: {0}.',
duplicate: 'These tool servers were left out because another one has almost the same name: {0}.',
};
export function mcpSkippedNotices(skipped) {
const out = [];
for (const why of Object.keys(SKIPPED_TEXT)) {
const names = (skipped ?? []).filter(s => s.why === why).map(s => String(s.name).slice(0, 40));
if (names.length) out.push({ code: `mcp-skipped-${why}-${names.join(',')}`.slice(0, 160), message: SKIPPED_TEXT[why].replace('{0}', names.join(', ')) });
}
return out;
}
const slug = value => String(value).toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '').slice(0, 24) || 'x';
/** Identity of one route's wire configuration, secret included by hash only. */
@@ -166,7 +193,7 @@ export function outputBudget(context, maxOutput = null) {
/** What a local model, or an OpenAI-style server on this network, takes for the thinking level (see the comment where a local route is built). */
const THINKING_COMPAT = { supportsStore: false, thinkingFormat: 'chat-template', chatTemplateKwargs: { enable_thinking: { $var: 'thinking.enabled' }, reasoning_effort: { $var: 'thinking.effort', omitWhenOff: true } } };
export function buildProfile({ local = [], connections = [], gate = true, tools = NO_TOOLS_SETTINGS, skillsDir = null, proxyBase = null, searchMock = null }) {
export function buildProfile({ local = [], connections = [], gate = true, tools = NO_TOOLS_SETTINGS, skillsDir = null, proxyBase = null, searchMock = null, defaultLocalId = null }) {
const providers = {};
const env = {};
const routes = [];
@@ -202,6 +229,9 @@ export function buildProfile({ local = [], connections = [], gate = true, tools
api: 'openai-completions',
baseURL: inst.baseUrl,
apiKeyEnv: keyEnv,
// A long prompt read sends only SSE comments, which pi-ai's OpenAI client drops, so its 300 s stream-idle timer would cut the
// request five minutes in (#3623). 60 minutes is a backstop: serve ends a hung engine and the bridge's watchdog ends a lost turn.
streamIdleTimeoutMs: 60 * 60_000,
// The Strata server reads the thinking level from `chat_template_kwargs` (enable_thinking, reasoning_effort) or from a top-level
// reasoning_effort; a top-level `enable_thinking` (pi-ai's `qwen` format) is ignored, so the model went on thinking with the menu on Off.
compat: THINKING_COMPAT,
@@ -257,9 +287,13 @@ export function buildProfile({ local = [], connections = [], gate = true, tools
for (const m of conn.models) routes.push({ route, fp, kind: 'api', connectionId: conn.id, connectionName: conn.name, modelId: m.id, name: m.name || m.id, contextWindow: m.contextMax ?? ASSUMED_API_CONTEXT, ...(shaped ? { shaped } : {}) });
});
const defaultRoute = routes.find(r => r.kind === 'local') ?? null;
// The model the chat really runs on is the default instance, not whichever one was loaded first (T5: with two models loaded, the tool servers of the
// first one were wired to a chat on the second).
const defaultInstance = local.find(i => i.id === defaultLocalId) ?? local[0] ?? null;
const defaultRoute = routes.find(r => r.kind === 'local' && r.instanceId === defaultInstance?.id) ?? routes.find(r => r.kind === 'local') ?? null;
// One Harness, one set of tool servers: the default local model's.
const mcp = mcpRows({ ...(local[0]?.mcpServers ?? {}), ...(tools.servers ?? {}) }, mcpEnv);
const mcpSkipped = [];
const mcp = mcpRows({ ...(defaultInstance?.mcpServers ?? {}), ...(tools.servers ?? {}) }, mcpEnv, mcpSkipped);
Object.assign(env, mcpEnv);
// The person's own instruction and command tools come from the app's tool server; a command tool asks in Manual like any connected tool.
const own = (kind, on) => on && tools.toolFile ? [{ id: `mcp-strata-${kind}`, name: '@deepseek-ai/dsh-mcp-client', config: { serverName: `strata-${kind}`, transport: 'stdio', command: process.execPath, args: [CUSTOM_TOOLS_SCRIPT, kind, tools.toolFile] } }] : [];
@@ -286,6 +320,8 @@ export function buildProfile({ local = [], connections = [], gate = true, tools
// Manual asks before a change runs, so the person sees one Approve/Deny card and nothing has happened yet.
// Its web question names the search service the person chose (a keyed one only when its key is saved), and the person's switch skips the question.
...(gate ? [{ insert: [{ id: 'strata-gate', name: GATE_PLUGIN, config: { web: { provider: SEARCH_NAMES[keyed ?? 'builtin'], skipAsk: tools.webWithoutAsking === true } } }] }] : []),
// Compaction decides from what the model server measured, and its summary requests always fit the window (plugins/strata-context.mjs).
{ insert: [{ id: 'strata-context', name: CONTEXT_PLUGIN, config: {} }] },
// Helper agents on a model the person names: every model the app can reach is allowed, and the plugin makes the choice forgiving (plugins/strata-models.mjs).
...(subagentRoutes.length ? [{ id: 'subagent-model-selection-settings', config: { enabled: true, allowedModels: subagentRoutes.map(r => ({ provider: r.provider, model: r.model })) } }] : []),
...(subagentRoutes.length ? [{ insert: [{ id: 'strata-models', name: MODELS_PLUGIN, config: { routes: subagentRoutes } }] }] : []),
@@ -309,7 +345,7 @@ export function buildProfile({ local = [], connections = [], gate = true, tools
// What the agent is offered (tool groups, web search, tool servers, the person's own tools) decides whether a running agent must be
// restarted for the next message; the routes are compared separately (a changed context size alone does not restart it).
const toolsSignature = createHash('sha256').update(JSON.stringify([mcp, ownRows, webRows, entries.find(e => e.id === 'preset-standard'), env.STRATA_SEARCH_KEY ?? '', mcpEnv])).digest('hex').slice(0, 16);
return { patch, env, routes, defaultRoute, signature, toolsSignature, shapeTargets };
return { patch, env, routes, defaultRoute, signature, toolsSignature, shapeTargets, mcpSkipped };
}
/** Find the route for a chat's model choice. */
+14 -2
View File
@@ -13,6 +13,7 @@
import { mkdirSync, renameSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { GROUP_IDS, TOOL_GROUPS } from './tools-catalog.mjs';
import { SHELLS, clampTimeoutSec } from './custom-command-run.mjs';
export const SEARCH_PROVIDERS = ['builtin', 'exa', 'perplexity', 'deepseek'];
const KINDS = ['instructions', 'command', 'server'];
@@ -38,6 +39,8 @@ function normalize(raw) {
id: String(t.id), name: String(t.name), about: clean(t.about, 300), kind: t.kind, enabled: t.enabled !== false,
instructions: clean(t.instructions, 20000), command: clean(t.command, 2000), address: clean(t.address, 2000),
args: (Array.isArray(t.args) ? t.args : []).slice(0, 50).map(a => clean(a, 1000)),
// a command tool's folder, shell and time allowed (custom-command-run.mjs); '' / 'auto' / 120 when not set
cwd: clean(t.cwd, 1000).trim(), shell: SHELLS.includes(t.shell) ? t.shell : 'auto', timeoutSec: clampTimeoutSec(t.timeoutSec),
})),
};
}
@@ -49,7 +52,7 @@ function validateCustom(body, existing) {
if (!about) throw new Error('Say what the tool does, in a sentence. The assistant reads this to decide when to use it.');
const kind = body?.kind;
if (!KINDS.includes(kind)) throw new Error('Choose how the tool works.');
const out = { name, about, kind, instructions: '', command: '', address: '', args: [] };
const out = { name, about, kind, instructions: '', command: '', address: '', args: [], cwd: '', shell: 'auto', timeoutSec: clampTimeoutSec(undefined) };
if (kind === 'instructions') {
out.instructions = clean(body.instructions, 20000).trim();
if (!out.instructions) throw new Error('Write the instructions the assistant should follow when it uses this tool.');
@@ -57,6 +60,14 @@ function validateCustom(body, existing) {
out.command = clean(body.command, 2000).trim();
if (!out.command) throw new Error('Enter the command to run.');
if (/[\r\n]/.test(out.command)) throw new Error('Put the command on one line.');
// The edit form of Settings does not send these three: an edit keeps what the tool had.
const before = existing.find(t => t.id === body.id && t.kind === 'command');
out.cwd = body.cwd !== undefined ? clean(body.cwd, 1000).trim() : before?.cwd ?? '';
if (/[\r\n]/.test(out.cwd)) throw new Error('Put the folder on one line.');
if (body.shell !== undefined && !SHELLS.includes(body.shell)) throw new Error('Choose how the command runs: automatic, cmd, PowerShell, bash or sh.');
out.shell = body.shell !== undefined ? body.shell : before?.shell ?? 'auto';
if (body.timeoutSec !== undefined && !(Number(body.timeoutSec) >= 1 && Number(body.timeoutSec) <= 3600)) throw new Error('Give the time allowed in seconds, from 1 to 3600.');
out.timeoutSec = body.timeoutSec !== undefined ? clampTimeoutSec(body.timeoutSec) : before?.timeoutSec ?? clampTimeoutSec(undefined);
} else {
out.address = clean(body.address, 2000).trim();
if (!out.address) throw new Error('Enter the tool server’s web address, or the program that starts it.');
@@ -91,7 +102,7 @@ export function createAgentTools({ read, write, dataDir, browser = () => ({ foun
for (const t of s.custom.filter(t => t.enabled)) {
const key = t.id;
if (t.kind === 'instructions') instructions[key] = { description: t.about, text: t.instructions };
else if (t.kind === 'command') commands[key] = { description: t.about, command: t.command };
else if (t.kind === 'command') commands[key] = { description: t.about, command: t.command, cwd: t.cwd, shell: t.shell, timeoutSec: t.timeoutSec };
}
mkdirSync(join(dataDir, 'agent'), { recursive: true });
const tmp = `${filePath}.tmp`;
@@ -115,6 +126,7 @@ export function createAgentTools({ read, write, dataDir, browser = () => ({ foun
id: t.id, name: t.name, about: t.about, kind: t.kind, enabled: t.enabled,
summary: t.kind === 'instructions' ? t.instructions.slice(0, 80) : t.kind === 'command' ? t.command : t.address,
instructions: t.instructions, command: t.command, address: t.address, args: t.args,
...(t.kind === 'command' ? { cwd: t.cwd, shell: t.shell, timeoutSec: t.timeoutSec } : {}),
})),
};
}
+1 -1
View File
@@ -10,7 +10,7 @@
* rewrites a plan turn wrote its numbered plan from "1." again this many times
* context the context is nearly full: the Harness compaction did not keep up
*
* What the person is told is `guardNote`: fixed English sentences that are also in the interface catalog in all eight languages (the page
* What the person is told is `guardNote`: fixed English sentences that are also in the interface catalog in all nine languages (the page
* translates a `delta` frame that carries `key`). Each says what happened and gives one next step.
*/
+116 -16
View File
@@ -67,24 +67,70 @@ export function nvidiaDriverVersion() {
/** Where the builds are published; the wrong-make sentence points here. */
export const RELEASES_URL = 'https://github.com/Niko1221/Strata/releases';
/** The PCI vendor ids of the display adapters Windows reports as present right now (PowerShell, Win32_VideoController); empty when that cannot be read. */
export function windowsPresentVendors({ platform = process.platform, run = execFileSync } = {}) {
const found = new Set();
if (platform !== 'win32') return found;
try {
const out = run('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', 'Get-CimInstance Win32_VideoController | ForEach-Object { $_.PNPDeviceID }'], { encoding: 'utf8', timeout: 15000, windowsHide: true, stdio: ['ignore', 'pipe', 'ignore'] });
for (const match of String(out).matchAll(/VEN_([0-9A-F]{4})/gi)) found.add(match[1].toLowerCase());
} catch { /* no PowerShell or a limited account: the registry answer stands */ }
return found;
}
/**
* The PCI vendor ids (`10de`, `1002`, `8086`) of the Windows display adapters; empty off Windows or when the registry cannot be read.
* The registry keeps the entry of a card that was taken out (an NVIDIA card swapped for a Radeon, #3892): when NVIDIA and AMD are both
* listed, only the makes Windows reports as present now count, the same rule Strata Control uses (engine_component.windows_display_adapters).
*/
export function windowsDisplayVendors({ platform = process.platform, run = execFileSync, present = windowsPresentVendors } = {}) {
const found = new Set();
if (platform !== 'win32') return found;
try {
const out = run('reg', ['query', 'HKLM\\SYSTEM\\CurrentControlSet\\Control\\Class\\{4d36e968-e325-11ce-bfc1-08002be10318}', '/s', '/v', 'MatchingDeviceId'], { encoding: 'utf8', timeout: 8000, windowsHide: true, stdio: ['ignore', 'pipe', 'ignore'] });
for (const match of String(out).matchAll(/VEN_([0-9A-F]{4})/gi)) found.add(match[1].toLowerCase());
} catch { /* no registry access: the tools decide */ }
if (found.has('10de') && found.has('1002')) {
const now = present({ platform, run });
const kept = new Set([...found].filter(vendor => now.has(vendor)));
if (kept.size) return kept;
}
return found;
}
const BACKEND_OF_VENDOR = [['10de', 'cuda'], ['1002', 'hip'], ['8086', 'sycl']];
/**
* Every engine family this PC's graphics cards can run, in the order NVIDIA, AMD, Intel; STRATA_ENGINE_BACKEND wins. A PC with two
* makes lists both, so a build for either make is accepted (recommend, never force).
*/
export function gpuBackendsHere(env = process.env, { vendors = windowsDisplayVendors() } = {}) {
const forced = String(env.STRATA_ENGINE_BACKEND ?? '').trim().toLowerCase();
if (forced) return ['cuda', 'hip', 'sycl'].includes(forced) ? [forced] : [];
if (vendors.size) {
const list = BACKEND_OF_VENDOR.filter(([vendor]) => vendors.has(vendor)).map(([, backend]) => backend);
if (list.length) return list;
}
if (nvidiaDriverVersion()) return ['cuda'];
try {
const base = '/sys/class/drm';
const seen = new Set();
for (const card of readdirSync(base)) {
if (!/^card\d+$/.test(card)) continue;
try { seen.add(readFileSync(`${base}/${card}/device/vendor`, 'utf8').trim().toLowerCase()); } catch { /* no vendor file */ }
}
const list = [['0x10de', 'cuda'], ['0x1002', 'hip'], ['0x8086', 'sycl']].filter(([vendor]) => seen.has(vendor)).map(([, backend]) => backend);
if (list.length) return list;
} catch { /* not Linux */ }
return [];
}
/**
* The engine family this PC's graphics card needs ('cuda', 'hip', 'sycl'), or null when it cannot be told. The same order as Control
* (NVIDIA first, then AMD, then Intel); STRATA_ENGINE_BACKEND wins. Reads only the driver and (Linux) the PCI vendor of the display adapters.
*/
export function gpuBackendHere(env = process.env) {
const forced = String(env.STRATA_ENGINE_BACKEND ?? '').trim().toLowerCase();
if (forced) return ['cuda', 'hip', 'sycl'].includes(forced) ? forced : null;
if (nvidiaDriverVersion()) return 'cuda';
try {
const base = '/sys/class/drm';
const vendors = new Set();
for (const card of readdirSync(base)) {
if (!/^card\d+$/.test(card)) continue;
try { vendors.add(readFileSync(`${base}/${card}/device/vendor`, 'utf8').trim().toLowerCase()); } catch { /* no vendor file */ }
}
for (const [vendor, backend] of [['0x10de', 'cuda'], ['0x1002', 'hip'], ['0x8086', 'sycl']]) if (vendors.has(vendor)) return backend;
} catch { /* not Linux */ }
return null;
export function gpuBackendHere(env = process.env, options = {}) {
return gpuBackendsHere(env, options)[0] ?? null;
}
const HIP_FAMILIES = [['RX 9000', ['gfx1200', 'gfx1201']], ['RX 7000', ['gfx1100', 'gfx1101', 'gfx1102']], ['RX 6000', ['gfx1030', 'gfx1031']], ['Ryzen AI Max', ['gfx1151']]];
@@ -131,10 +177,12 @@ export async function installBundledEngine({ bundleDir, components, platform = n
if (platform) packages = packages.filter(p => p.manifest.platform === platform);
if (!packages.length) return { action: 'none' };
// A build for another make of graphics card than this PC has can never run here: it is not unpacked, and the caller says so in plain words.
const wanted = backend === undefined ? gpuBackendHere() : backend;
// A PC with two makes (an AMD card for the model, an NVIDIA card for the desktop) accepts a build for either (#3892).
const wantedList = backend === undefined ? gpuBackendsHere() : (Array.isArray(backend) ? backend : backend ? [backend] : []);
const wanted = wantedList[0] ?? null;
if (wanted && ['cuda', 'hip', 'sycl'].includes(wanted)) {
const gpu = packages.filter(p => ['cuda', 'hip', 'sycl'].includes(p.manifest.backend));
const fits = packages.filter(p => !gpu.includes(p) || p.manifest.backend === wanted);
const fits = packages.filter(p => !gpu.includes(p) || wantedList.includes(p.manifest.backend));
if (gpu.length && !fits.length) {
const first = gpu.sort((a, b) => compareVersions(b.manifest.version, a.manifest.version))[0].manifest;
return { action: 'wrong-make', built: first.backend, here: wanted, notice: wrongMakeNotice(first, wanted) };
@@ -218,3 +266,55 @@ export async function installBundledEngine({ bundleDir, components, platform = n
return { action: 'refused', reason: error?.message ?? String(error) };
}
}
/**
* After an update from an installer that carried an engine (a per-GPU build), its `engine-bundle` folder is still in the install folder
* (about 570 MB) although the engine was copied to the engine store long ago. It is removed only when EVERY package in it exists complete in
* the store (every file there with the size and SHA-256 its manifest lists); otherwise it stays and one line says why.
* Returns { removed: bool, reason?: string, packages?: number }; never throws.
*/
export async function removeVerifiedBundle({ bundleDir, components, log = () => {} } = {}) {
try {
if (!bundleDir || !components || !existsSync(bundleDir)) return { removed: false, reason: 'no bundle' };
// Files inside a signed macOS app must stay intact after installation.
if (process.platform === 'darwin' && resolve(bundleDir).split(sep).some(part => part.toLowerCase().endsWith('.app'))) return keepBundle(log, 'macOS app bundles must remain unchanged after signing');
const packages = [];
for (const entry of await readdir(bundleDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const folder = join(bundleDir, entry.name);
const manifest = await readManifest(folder);
packages.push({ folder, manifest, name: entry.name });
}
if (!packages.length) return { removed: false, reason: 'nothing in the bundle' };
const store = join(components, 'engine');
const stored = existsSync(store) ? (await readdir(store, { withFileTypes: true })).filter(e => e.isDirectory() && !e.name.startsWith('.')).map(e => join(store, e.name)) : [];
for (const pkg of packages) {
if (!pkg.manifest) return keepBundle(log, `${pkg.name} has no readable manifest`);
let found = false;
for (const candidate of stored) {
const theirs = await readManifest(candidate);
if (!theirs || theirs.version !== pkg.manifest.version || theirs.platform !== pkg.manifest.platform || theirs.backend !== pkg.manifest.backend
|| (theirs.variant ?? '') !== (pkg.manifest.variant ?? '') || (theirs.build_id ?? '') !== (pkg.manifest.build_id ?? '')) continue;
if (await packageIsWhole(candidate, pkg.manifest, { deep: true })) { found = true; break; }
}
if (!found) return keepBundle(log, `${pkg.name} is not (or not completely) in the engine store`);
}
for (const pkg of packages) await rm(pkg.folder, { recursive: true, force: true });
log(`engine bundle: the copy that came with the installer (${packages.map(p => p.name).join(', ')}) was removed; the same engine is complete in the engine store`);
return { removed: true, packages: packages.length };
} catch (error) {
return keepBundle(log, `could not be checked (${error?.message ?? error})`);
}
}
function keepBundle(log, why) {
log(`engine bundle: kept in the install folder (${why})`);
return { removed: false, reason: why };
}
/**
* What the host does once its engine step has settled: for an engine that is (or was) there, look at the install folder's leftover bundle.
* The lines go to the host's output (console.log), which the desktop shell copies to its log file (`[host] engine bundle: ...`).
*/
export function cleanupAfterInstall(settled, { bundleDir, components, log = line => console.log(line) } = {}) {
return Promise.resolve(settled).then(result => (['installed', 'kept', 'repaired'].includes(result?.action) ? removeVerifiedBundle({ bundleDir, components, log }) : null), () => null);
}
+33
View File
@@ -74,6 +74,17 @@ export function findControl({ env = process.env, platform = process.platform, ex
return control;
}
/**
* The environment of the server the host starts from a source checkout (#3851): the key travels in the run configuration, so a
* STRATA_API_KEY the host itself was started with is removed. Setting it to an empty string made serve/server.py stop with
* "an API key was given but it is empty".
*/
export function sourceServerEnvironment(env = process.env) {
const next = { ...env, PYTHONUNBUFFERED: '1', PYTHONUTF8: '1', PYTHONIOENCODING: 'utf-8' };
delete next.STRATA_API_KEY;
return next;
}
/** The environment Control runs in: nothing from the user's Python, and acquiring forbidden. */
export function controlEnvironment(control, { env = process.env, components = componentsDirectory({ env }), acquire = false } = {}) {
const next = { ...env };
@@ -183,6 +194,12 @@ export function createControl({ control = findControl(), env = process.env, run
if (failed) throw Object.assign(new ControlError(failed.error, { exitCode: failed.exit_code ?? result.code, output: result.json }), { code: result.code === 7 ? 'engine_serving' : 'update_failed' });
throw failure(result, 'The update failed.');
},
/** Install an engine package the person downloaded and chose (the offline path): no network; Control checks the package's own signature. */
async installEngineFile(path, { onEvent = null, signal = null } = {}) {
const result = await call(['components', 'install-file', path, '--output', 'jsonl'], { timeoutMs: 3600_000, onLine: onEvent ?? (() => {}), signal });
if (result.json?.ok) return { ok: true, results: [result.json] };
throw failure(result, 'This file could not be installed.');
},
async rollbackEngine() {
const result = await call(['components', 'rollback', '--output', 'json'], { timeoutMs: 120_000 });
if (result.json?.ok) return result.json;
@@ -244,6 +261,22 @@ export function createControl({ control = findControl(), env = process.env, run
if (result.json?.ok) return result.json;
throw Object.assign(failure(result, 'The partial copies could not be removed.'), { code: result.json?.error?.code ?? 'leave_failed', details: result.json?.error?.details ?? '' });
},
/**
* The DFlash2 drafter for a model's run config and the drafting choice (`strata drafting plan`): { needed, gb, line, note, drafter }. Control decides
* (the table of strata_control/drafting.py); the app never reads it. Nothing is downloaded.
*/
async draftingPlan({ configPath = null, mode = null, family = null, dataDir = null } = {}) {
const result = await call(['drafting', 'plan', ...(configPath ? ['--config', configPath] : []), ...(mode ? ['--mode', mode] : []), ...(family ? ['--family', family] : []), ...(dataDir ? ['--data-dir', dataDir] : []), '--output', 'json'], { timeoutMs: 30_000 });
if (result.json?.ok) return result.json;
throw failure(result, 'The drafter could not be checked.');
},
/** Fetch and pack the drafter when the choice would use it and it is not here (`strata drafting fetch --if-needed`); `onEvent` gets {step, done, total}. */
async draftingFetch({ configPath, mode = null, onEvent = null, signal = null } = {}) {
const args = ['drafting', 'fetch', '--config', configPath, ...(mode ? ['--mode', mode] : []), '--if-needed', '--output', 'jsonl'];
const result = await call(args, { timeoutMs: 3 * 3600_000, acquire: true, onLine: onEvent ?? (() => {}), signal });
if (result.json?.ok) return result.json;
throw Object.assign(new ControlError(result.json?.error?.message ?? (result.stderr.split(/\r?\n/).filter(Boolean).pop() || 'The DFlash2 drafter could not be downloaded.'), { exitCode: result.code, output: result.json }), { code: 'drafting_failed' });
},
async saveProfile(name, configPath) {
const listed = await call(['profile', 'list', '--output', 'json'], { timeoutMs: 20_000 });
const known = (listed.json?.profiles ?? []).some(p => p.id === name);
+18 -8
View File
@@ -1,6 +1,7 @@
/** Physical topology and GPU sensors; unavailable values stay null. */
import { readFile, readdir, realpath } from 'node:fs/promises';
import { probe, round } from './util.mjs';
import { windowsGpuReading } from './windows-gpu.mjs';
/** Unified GPUs share one system memory pool; dedicated GPUs each have their own. */
export function gpuMemoryTotals(devices, readings) {
@@ -39,13 +40,15 @@ export function parseNvidiaDevices(raw) {
return [{id:uuid,index:Number(index),sensorIndex:Number(index),busId:bus?.toLowerCase(),vendor:'nvidia',name,vramGb:round((number(memory)??0)/1024,1),unified:false,backend:'cuda',driver}];
});
}
/** A percentage reading; "[N/A]" and out-of-range values are no reading. */
const percentOf = v => { const n=number(v); return n!==null&&n>=0&&n<=100?n:null; };
export function parseNvidiaReadings(raw) {
return raw.split('\n').flatMap(line=>{
const [id,temp,load,memory,power,limit,gen,maxGen,width]=line.split(',').map(s=>s.trim());
const [id,temp,load,memory,power,limit,gen,maxGen,width,fan,clock]=line.split(',').map(s=>s.trim());
if (!id?.startsWith('GPU-')) return [];
const t=number(temp),u=number(load),m=number(memory);
const positive = v => { const n=number(v); return n!==null&&n>=0?n:null; };
return [{id,temperatureC:t!==null&&t>=0&&t<=150?t:null,utilizationPct:u!==null&&u>=0&&u<=100?u:null,memoryUsedGb:m!==null&&m>=0?round(m/1024,2):null,powerW:positive(power),powerLimitW:positive(limit),pcieGen:positive(gen),pcieGenMax:positive(maxGen),pcieWidth:positive(width),pcieRxMbs:null}];
return [{id,temperatureC:t!==null&&t>=0&&t<=150?t:null,utilizationPct:u!==null&&u>=0&&u<=100?u:null,memoryUsedGb:m!==null&&m>=0?round(m/1024,2):null,powerW:positive(power),powerLimitW:positive(limit),pcieGen:positive(gen),pcieGenMax:positive(maxGen),pcieWidth:positive(width),pcieRxMbs:null,fanPct:percentOf(fan),clockMhz:positive(clock)}];
});
}
export function parsePcieThroughput(raw) {
@@ -72,23 +75,30 @@ export async function detectDrmGpus() {
}
return result;
}
export async function readGpuTelemetry(gpus) {
/** `transfer: false` skips the PCIe throughput sample (a second nvidia-smi); it only means something while a model runs. */
export async function readGpuTelemetry(gpus, { transfer: wantTransfer = true, run = probe } = {}) {
const now=new Date().toISOString();
const hasNvidia=gpus.some(g=>g.vendor==='nvidia');
let raw=hasNvidia?await probe('nvidia-smi',['--query-gpu=uuid,temperature.gpu,utilization.gpu,memory.used,power.draw,power.limit,pcie.link.gen.current,pcie.link.gen.max,pcie.link.width.current','--format=csv,noheader,nounits'],2000):'';
let raw=hasNvidia?await run('nvidia-smi',['--query-gpu=uuid,temperature.gpu,utilization.gpu,memory.used,power.draw,power.limit,pcie.link.gen.current,pcie.link.gen.max,pcie.link.width.current,fan.speed,clocks.current.graphics','--format=csv,noheader,nounits'],2000):'';
// Older drivers can reject a field; retain their basic sensors in that case.
if(hasNvidia&&!raw)raw=await probe('nvidia-smi',['--query-gpu=uuid,temperature.gpu,utilization.gpu,memory.used','--format=csv,noheader,nounits'],2000);
if(hasNvidia&&!raw)raw=await run('nvidia-smi',['--query-gpu=uuid,temperature.gpu,utilization.gpu,memory.used','--format=csv,noheader,nounits'],2000);
const nvidia=new Map(parseNvidiaReadings(raw).map(r=>[r.id,r]));
const transfer=hasNvidia?parsePcieThroughput(await probe('nvidia-smi',['dmon','-s','t','-c','1'],1500)):new Map();
const transfer=hasNvidia&&wantTransfer?parsePcieThroughput(await run('nvidia-smi',['dmon','-s','t','-c','1'],1500)):new Map();
return Promise.all(gpus.map(async gpu=>{
let reading=nvidia.get(gpu.id)??{id:gpu.id??`${gpu.vendor}:${gpu.index}`,temperatureC:null,utilizationPct:null,memoryUsedGb:null,powerW:null,powerLimitW:null,pcieGen:null,pcieGenMax:null,pcieWidth:null,pcieRxMbs:null};
if(gpu.vendor==='nvidia')reading={...reading,pcieRxMbs:transfer.get(gpu.sensorIndex??gpu.index)??null};
if(gpu.luid&&!nvidia.has(gpu.id)) {
const counters=windowsGpuReading(gpu.luid);
if(counters)reading={...reading,utilizationPct:counters.utilizationPct,memoryUsedGb:round(counters.memoryUsedGb,2)};
}
if(gpu.sensorPath&&!nvidia.has(gpu.id)) {
let temperature=null,power=null,powerLimit=null;
let temperature=null,power=null,powerLimit=null,fanPwm=null,clockHz=null;
for(const hwmon of await list(`${gpu.sensorPath}/hwmon`)) {
const folder=`${gpu.sensorPath}/hwmon/${hwmon}`,inputs=(await list(folder)).filter(n=>/^temp\d+_input$/.test(n));
power ??= number(await read(`${folder}/power1_average`)) ?? number(await read(`${folder}/power1_input`));
powerLimit ??= number(await read(`${folder}/power1_cap`));
fanPwm ??= number(await read(`${folder}/pwm1`)); // 0-255, the fan's duty as the amdgpu driver reports it
clockHz ??= number(await read(`${folder}/freq1_input`)); // the graphics clock in Hz
for(const input of inputs) {
const label=await read(`${folder}/${input.replace('_input','_label')}`);
if(label&&!/edge|gpu/i.test(label))continue;
@@ -100,7 +110,7 @@ export async function readGpuTelemetry(gpus) {
const utilization=number(await read(`${gpu.sensorPath}/gpu_busy_percent`)),bytes=number(await read(`${gpu.sensorPath}/mem_info_vram_used`));
const gen=path=>read(path).then(raw=>({2.5:1,5:2,8:3,16:4,32:5,64:6}[Number.parseFloat(raw)]??null));
const [pcieGen,pcieGenMax,width]=await Promise.all([gen(`${gpu.sensorPath}/current_link_speed`),gen(`${gpu.sensorPath}/max_link_speed`),read(`${gpu.sensorPath}/current_link_width`)]);
reading={...reading,temperatureC:temperature,utilizationPct:utilization!==null&&utilization>=0&&utilization<=100?utilization:null,memoryUsedGb:bytes!==null&&bytes>=0?round(bytes/1024**3,2):null,powerW:power!==null&&power>=0?power/1e6:null,powerLimitW:powerLimit!==null&&powerLimit>=0?powerLimit/1e6:null,pcieGen,pcieGenMax,pcieWidth:number(width)};
reading={...reading,fanPct:fanPwm!==null&&fanPwm>=0&&fanPwm<=255?Math.round(fanPwm/255*100):null,clockMhz:clockHz!==null&&clockHz>0?Math.round(clockHz/1e6):null,temperatureC:temperature,utilizationPct:utilization!==null&&utilization>=0&&utilization<=100?utilization:null,memoryUsedGb:bytes!==null&&bytes>=0?round(bytes/1024**3,2):null,powerW:power!==null&&power>=0?power/1e6:null,powerLimitW:powerLimit!==null&&powerLimit>=0?powerLimit/1e6:null,pcieGen,pcieGenMax,pcieWidth:number(width)};
}
return {...reading,sampledAt:now,source:Object.entries(reading).some(([key,value])=>key!=='id'&&typeof value==='number')?'sensor':'none'};
}));
+20
View File
@@ -0,0 +1,20 @@
// DeepSeek-V4.1-Flash's own context limit, read from its pack (the app offers sizes up to it: src/lib/deepseek-context.ts).
// Project Maya v1.1.28 (#442, @needmorevram, MIT) did the same in its dashboard. strata_control/dsv41_context.py reads the same file for setup and the Terminal.
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
/** The pack's own limit: `max_position_embeddings` in <pack>/ds4.txt. Null when there is no pack, no file or no such line (the choices then stay as before). */
export function packContextLimit(pack) {
if (!pack) return null;
try {
const match = /^max_position_embeddings\s+(\d+)\s*$/m.exec(readFileSync(join(pack, 'ds4.txt'), 'utf8'));
const value = match ? Number(match[1]) : 0;
return Number.isSafeInteger(value) && value >= 1024 ? value : null;
} catch { return null; }
}
/** The limit of the pack a DeepSeek run config names (`--pack DIR` in its arguments). */
export function configContextLimit(config) {
const args = (config?.args ?? []).map(String), at = args.indexOf('--pack');
return at >= 0 && at + 1 < args.length ? packContextLimit(args[at + 1]) : null;
}
+14
View File
@@ -0,0 +1,14 @@
/**
* GPU energy since the host started, in Wh (#3628): the power reading held for the time since the last tick, added up.
* A long gap (the PC slept, the host was paused) counts for at most 5 s, so a sleep never adds a made-up hour of use.
*/
const meter = { wh: 0, at: 0 };
export function addEnergy(powerW, at) {
const dt = meter.at ? Math.min((at - meter.at) / 1000, 5) : 0;
meter.at = at;
if (typeof powerW === 'number' && Number.isFinite(powerW) && powerW >= 0 && dt > 0) meter.wh += powerW * dt / 3600;
return meter.wh;
}
export const energyWh = () => meter.wh;
+4 -1
View File
@@ -62,7 +62,10 @@ export function watchedWindowsDialog(shell, args, { readyMs = 30_000, openMs = 2
const later = () => { clearTimeout(timer); ready = true; setTimeout(() => end(resolve, { stdout: '' }), openMs).unref?.(); };
child.stdout?.on('data', chunk => { stdout += chunk; if (!ready && /^READY\s*$/m.test(stdout)) { clearTimeout(timer); later(); } });
child.on('error', error => end(reject, error));
child.on('close', code => { if (settled) return; settled = true; clearTimeout(timer); code === 0 ? resolve({ stdout }) : reject(Object.assign(new Error('exit ' + code), { code })); });
child.on('close', code => { if (settled) return; settled = true; clearTimeout(timer);
// PowerShell ended before its window opened (security software ended it, #3831): the person types the path instead.
if (!ready) return resolve({ noDialog: true });
code === 0 ? resolve({ stdout }) : reject(Object.assign(new Error('exit ' + code), { code })); });
});
}
+7 -5
View File
@@ -12,6 +12,7 @@ import { CONTEXT_TOKENS, VARIANTS, footprint } from './catalog.mjs';
import { MODELS_DIR, diskFreeGb, diskTotalGb } from './store.mjs';
import { probe, round } from './util.mjs';
import { utf8Script } from './powershell-utf8.mjs';
import { DX_ADAPTERS_SCRIPT, parseDxAdapters, windowsOtherGpus } from './windows-gpu.mjs';
let cached = null;
let cachedAt = 0;
@@ -187,11 +188,12 @@ async function detectGpus(platform, totalGb) {
}
if(platform==='win32') {
const raw=await probe('powershell.exe',['-NoProfile','-NonInteractive','-Command',utf8Script('Get-CimInstance Win32_VideoController | Select-Object Name,PNPDeviceID | ConvertTo-Json -Compress')]);
try {const parsed=JSON.parse(raw);for(const row of Array.isArray(parsed)?parsed:[parsed]) {
if(!row?.Name||(/nvidia/i.test(row.Name)&&gpus.some(g=>g.vendor==='nvidia')))continue;
const vendor=/amd|radeon/i.test(row.Name)?'amd':/intel/i.test(row.Name)?'intel':'unknown';
gpus.push({id:String(row.PNPDeviceID),index:gpus.length,vendor,name:row.Name,vramGb:0,unified:false,backend:'vulkan'});
}}catch{}
try {
const parsed=JSON.parse(raw);
const rows=(Array.isArray(parsed)?parsed:[parsed]).filter(row=>row?.Name&&!(/nvidia/i.test(row.Name)&&gpus.some(g=>g.vendor==='nvidia')));
const dx=rows.length?parseDxAdapters(await probe('powershell.exe',['-NoProfile','-NonInteractive','-Command',DX_ADAPTERS_SCRIPT],8000)):[];
for(const card of windowsOtherGpus(rows,dx))gpus.push({...card,index:gpus.length,unified:false,backend:'vulkan'});
}catch{}
}
gpus.forEach((g,i)=>{g.index=i;});
+87 -10
View File
@@ -1,4 +1,6 @@
#!/usr/bin/env node
import { engineProgressMark } from './agent/engine-progress.mjs';
import './system-ca.mjs'; // #3722: the operating system's certificates, before any HTTPS request
import { createCloudflare } from './cloudflare.mjs';
import { createConnectorInstaller } from './cloudflared-install.mjs';
import { findRunningConnector } from './cloudflared-find.mjs';
@@ -15,6 +17,7 @@ import { materializeSkills, parseSkillFile, skillSlug, writeCreatedSkill } from
import { systemFonts } from './fonts.mjs';
import { tuningDir } from './platform-paths.mjs';
import { createAgentBridge, plainTurnFailure } from './agent/agent-bridge.mjs';
import { mcpSkippedNotices } from './agent/profile.mjs';
import { createAgentTools } from './agent/tools-settings.mjs';
import { withPlainStatus } from './agent/status.mjs';
import { findBrowser, findPlaywrightMcp } from './agent/browser-probe.mjs';
@@ -29,7 +32,8 @@ import { findBrowser, findPlaywrightMcp } from './agent/browser-probe.mjs';
*/
import http from 'node:http';
import { homedir } from 'node:os';
import { cpSync, createReadStream, existsSync, mkdirSync, readFileSync, readdirSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { statfsSync, cpSync, createReadStream, createWriteStream, existsSync, mkdirSync, readFileSync, readdirSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { pipeline } from 'node:stream/promises';
import { spawn } from 'node:child_process';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
@@ -55,9 +59,10 @@ import { RequestQueue } from './request-queue.mjs';
import { selectSkills, skillInstructions } from './skill-selection.mjs';
import { createModelProviders } from './model-providers.mjs';
import { createForkTimeskip } from './timeskip-fork.mjs';
import { prepareOversized } from './oversize.mjs';
import { createChatFolders, chatRootOf, openFile, resolveChatFile, previewFile, rawFile, createAttachments, withAttachments } from './chat-files.mjs';
import { createLlmProxy, samplingFromExpert } from './agent/llm-shape.mjs';
import { checkUpdates, checkComponentUpdates } from './updates.mjs';
import { checkUpdates, checkComponentUpdates, engineFirstRun } from './updates.mjs';
import { plainNoModel } from './no-model-text.mjs';
import { createAppUi } from './app-ui.mjs';
import { appVersionFrom, appVersionToShow } from './app-version.mjs';
@@ -65,7 +70,9 @@ import { createImageModels } from './image-models.mjs';
import { createRuntimeModels } from './runtime-models.mjs';
import { createNativeControl } from './native-control.mjs';
import { nativeUnloadPlan } from './storage-check.mjs';
import { installBundledEngine } from './bundled-engine.mjs';
import { installBundledEngine, cleanupAfterInstall } from './bundled-engine.mjs';
import { systemLocale } from './region.mjs';
import { createShellNotifier, DEFAULT_PORT, parsePortNotice, readShellPrefs, writeShellPrefs } from './shell-prefs.mjs';
import { findControl, createControl, componentsDirectory } from './control-runtime.mjs';
import { forbidden as acquisitionForbidden } from './acquisition.mjs';
import { randomBytes, timingSafeEqual } from 'node:crypto';
@@ -110,7 +117,7 @@ const WORKSPACE_NAME = process.env.STRATA_WORKSPACE ? basename(WORKSPACE) : 'Str
const insideApp = path => { const rel = relative(ROOT, resolve(path)); return rel === '' || (!rel.startsWith('..') && !isAbsolute(rel)); };
const hub = new EventHub();
const modelProviders = createModelProviders({ directory: store.DATA_DIR, changed: value => hub.send('providers', withPictureAnswers(value)) });
const modelProviders = createModelProviders({ directory: store.DATA_DIR, knownThinks: (connection, modelId) => visionProbe.knownThinks(connection, modelId), changed: value => hub.send('providers', withPictureAnswers(value)) });
const imageModels = createImageModels({ dataDir: store.DATA_DIR, modelsDir: () => store.MODELS_DIR, real: models.REAL, changed: value => hub.send('image-models', value) });
/* The installed app ships Strata Control (python + strata_control) next to the host; a source checkout has none and keeps its older paths. */
const CONTROL = findControl();
@@ -122,10 +129,26 @@ const appUi = createAppUi({ shutdown: () => shutdown() });
/* The app's version has one source: the desktop shell passes it (STRATA_APP_VERSION, from apps/desktop/src-tauri/Cargo.toml). Outside the app it is read from the same file in a checkout, else 'dev'. */
function appVersion() { return appVersionFrom({ env: process.env, here: HERE }); }
const appUpdater = appUi.updater;
const shellNotifier = createShellNotifier();
/** What the page needs for the desktop settings: whether this is the desktop app, the port in use, the saved choices and a busy-port notice. */
function shellState() {
const saved = readShellPrefs();
const desktop = !!shellNotifier;
return {
desktop,
tray: desktop && process.platform !== 'linux',
port: PORT, defaultPort: DEFAULT_PORT,
pinnedPort: saved.port || null, minimizeToTray: saved.tray,
portNotice: parsePortNotice(process.env.STRATA_PORT_NOTICE),
};
}
/* The engine that ships inside the installer is put in the components folder when this PC has none (or an older one); no network, hashes checked. The model list waits for it. */
const bundledEngine = control && !process.env.STRATA_NO_BUNDLED_ENGINE
? installBundledEngine({ bundleDir: process.env.STRATA_ENGINE_BUNDLE_DIR || resolve(HERE, '..', '..', 'engine-bundle'), components: componentsDirectory(), log: line => console.log(line) })
: Promise.resolve({ action: 'none' });
const engineBundleDir = process.env.STRATA_ENGINE_BUNDLE_DIR || resolve(HERE, '..', '..', 'engine-bundle');
/* An update from a per-GPU installer leaves its engine-bundle behind; it goes only when the same engine is complete in the store (sizes and hashes). */
cleanupAfterInstall(bundledEngine, { bundleDir: engineBundleDir, components: componentsDirectory() });
let bundledEngineSettled = !(control && !process.env.STRATA_NO_BUNDLED_ENGINE);
bundledEngine.then(() => { bundledEngineSettled = true; }, () => { bundledEngineSettled = true; });
/* A build made for another make of graphics card is not unpacked; Control's answer then carries the plain sentence (with the right download) instead of its own. */
@@ -174,12 +197,15 @@ const agent = createAgentBridge({
// The person's Response settings (thinking budget, Expert sampling) reach the agent's model through this host's own forwarding endpoint.
proxyBase: `http://127.0.0.1:${PORT}/api/internal/agent-llm`,
expertSampling: () => samplingFromExpert(store.get().expert),
// How far a local engine has got (the 2 s /metrics poll), for the agent's stall watchdog: it moves while a long prompt is read.
engineProgress: instanceId => engineProgressMark(native.readyInstances().find(i => i.id === instanceId)),
onPlan: (chatId, event, plan) => hub.send('plan', { chatId, event, plan }),
onQueue: (chatId, queue) => hub.send('queue', { chatId, queue }),
dataDir: store.DATA_DIR,
componentsDir: componentsDirectory(),
// The installer ships the Harness next to the host (<install>/host/server -> <install>/harness).
bundledDir: resolve(HERE, '..', '..', 'harness'),
defaultLocalId: () => native.defaultInstanceId(),
localInstances: () => native.readyInstances().map(i => ({
id: i.id, label: i.label, baseUrl: `http://127.0.0.1:${i.port}/v1`, modelId: i.apiModelId,
context: i.metrics?.engine?.max_context ?? i.context ?? undefined, key: i.key, vision: i.vision,
@@ -213,6 +239,7 @@ models.init(hub);
engine.init(hub);
engineInstall.init(hub);
monitor.init(hub, boot);
monitor.setLiveProbe(() => native.hasLive());
serving.init(hub, { port: PORT, host: HOST });
runtimeModels=createRuntimeModels({hardware:()=>hardware.cachedHardware()??boot,library:()=>models.library(),requests:()=>serving.state().requests,real:models.REAL,changed:value=>{
if(!models.REAL){const primary=value.instances.find(i=>i.id===value.defaultId);if(store.get().loaded!==(primary?.variantId??null))store.update({loaded:primary?.variantId??null});}
@@ -546,17 +573,18 @@ function clientName(req) {
return 'Local client';
}
async function runRemoteTurn({ selection, message, history, images = [], onFrame, signal, workspaceId }) {
async function runRemoteTurn({ selection, message, history, images = [], thinking, onFrame, signal, workspaceId }) {
const controller = new AbortController();
const requestSignal = signal ? AbortSignal.any([signal, controller.signal]) : controller.signal;
let entry;
const started = Date.now();
try {
const { connection, model } = modelProviders.resolve(selection);
if (connection.protocol === 'openai-chat' && model.reasoning == null) await visionProbe.check(connection, model.id);
const appliedSkills = selectSkills(skillState, message, workspaceId);
entry = serving.open({ client: 'Strata chat', endpoint: connection.protocol === 'anthropic' ? '/messages' : connection.protocol === 'openai-responses' ? '/responses' : '/chat/completions', model: `${connection.name} · ${model.name}`, cancel: () => controller.abort(new DOMException('Cancelled by the user', 'AbortError')) });
serving.mark(entry.id, { state: 'generating' });
const result = await modelProviders.stream({ selection, message, history, images, signal: requestSignal, systemPrompt: ['You are Strata, a helpful assistant. Answer clearly. Do not claim to have read files or performed actions unless their results are included in the conversation.', skillInstructions(appliedSkills)].filter(Boolean).join('\n\n'), onFrame: frame => {
const result = await modelProviders.stream({ selection, message, history, images, thinking, signal: requestSignal, systemPrompt: ['You are Strata, a helpful assistant. Answer clearly. Do not claim to have read files or performed actions unless their results are included in the conversation.', skillInstructions(appliedSkills)].filter(Boolean).join('\n\n'), onFrame: frame => {
if (frame.type === 'start') noteSkillUse(appliedSkills);
if (frame.type === 'done') serving.mark(entry.id, { tokensIn: frame.tokensIn, tokensOut: frame.tokensOut, usageSource: frame.usageSource });
onFrame({ ...frame, requestId: entry.id, model: model.name, contextMax: model.contextMax, remoteModel: true, ...(frame.type === 'start' ? { skills: appliedSkills.map(s => s.name) } : {}) });
@@ -601,8 +629,11 @@ async function runAgentTurn({ res, controller, chatId, body, message, workspaceI
const goalLine = /^\/goal(?:\s+([\s\S]*))?$/i.exec(message.trim());
const planLine = /^\/plan(?:\s+([\s\S]*))?$/i.exec(message.trim());
const researchLine = /^\/research(?:\s+([\s\S]*))?$/i.exec(message.trim());
// "/compact" summarises the older messages of this chat now (it used to go to the model as text, which answered it as a question).
const compactLine = /^\/compact\s*$/i.test(message.trim());
const turnOptions = { chatId, message, workspacePath: cwd, choice, images, mode: typeof body.mode === 'string' ? body.mode : null, thinking: typeof body.thinking === 'string' ? body.thinking : null, thinkingBudget: Number.isFinite(body.thinkingBudget) ? body.thinkingBudget : 0, history: Array.isArray(body.history) ? body.history : [], onFrame: write, signal: controller.signal };
const result = typeof body.planId === 'string' && body.message === '/plan start' ? await agent.planGoal({ ...turnOptions, planId: body.planId })
: compactLine ? await agent.compactCommand(turnOptions)
: researchLine ? await agent.researchCommand({ ...turnOptions, line: researchLine[1] ?? '' })
: planLine ? await agent.planCommand({ ...turnOptions, line: planLine[1] ?? '' })
: goalLine ? await agent.goalCommand({ ...turnOptions, line: goalLine[1] ?? '' }) : await agent.runTurn(turnOptions);
@@ -616,7 +647,7 @@ async function runAgentTurn({ res, controller, chatId, body, message, workspaceI
}
async function runTurn({ message, history = [], onFrame, signal, client, workspaceId = null, sessionId = null, remoteModel = null, runtimeModelId = undefined, nativeModelId, images=[], useTools=false, thinking='medium', thinkingBudget=0 }) {
if (remoteModel) return runRemoteTurn({ selection: remoteModel, message, history, images, onFrame, signal, workspaceId });
if (remoteModel) return runRemoteTurn({ selection: remoteModel, message, history, images, thinking, onFrame, signal, workspaceId });
if(native.hasLive()||nativeModelId){
let instance;try{instance=native.resolveInstance(nativeModelId??native.state().defaultId);}catch(e){const message=plainNoModel(e.message);onFrame({type:'error',message});return {error:message};}const started=Date.now(),cancel=new AbortController();
const entry=serving.open({client:client??'Strata chat',endpoint:'/chat/completions',model:instance.apiModelId,instanceId:instance.id,gpuIds:instance.gpus.map(String),cancel:()=>cancel.abort()});
@@ -1012,11 +1043,11 @@ async function api(req, res, path, method, url) {
if(nativeOperation&&method==='POST')return json(res,200,await native.cancel(nativeOperation[1]));
const nativeInstance=/^\/api\/native\/instances\/([a-z0-9-]+)\/(unload|default)$/.exec(path);
if(nativeInstance&&method==='POST')return json(res,200,nativeInstance[2]==='unload'?await native.unload(nativeInstance[1]):native.setDefault(nativeInstance[1]));
const nativeSettings=/^\/api\/native\/models\/([a-z0-9_-]+)\/settings$/.exec(path);
const nativeSettings=/^\/api\/native\/models\/([a-z0-9_.-]+)\/settings$/.exec(path);
/* Expert mode shows what will actually run: the saved tuning's values come with the settings (null when none applies or unreadable). */
if(nativeSettings&&method==='GET'){const settings=native.settings(nativeSettings[1]);let tuned=null;try{const t=await native.tuneEffective({model:nativeSettings[1],resourceProfile:url.searchParams.get('resourceProfile')||undefined,vision:url.searchParams.get('vision')||undefined});tuned=t.applied?t:null;}catch{tuned=null;}return json(res,200,{...settings,tuned});}
if(nativeSettings&&method==='PATCH')return json(res,200,native.save(nativeSettings[1],await readBody(req)));
const nativeSettingsFile=/^\/api\/native\/models\/([a-z0-9_-]+)\/settings-file$/.exec(path);
const nativeSettingsFile=/^\/api\/native\/models\/([a-z0-9_.-]+)\/settings-file$/.exec(path);
if(nativeSettingsFile&&method==='GET')return json(res,200,native.settingsFile(nativeSettingsFile[1]));
if(nativeSettingsFile&&method==='POST')return json(res,200,await native.openSettingsFile(nativeSettingsFile[1]));
@@ -1317,6 +1348,27 @@ async function api(req, res, path, method, url) {
}
if (path === '/api/setup' && method === 'GET') return json(res, 200, setupState());
if (path === '/api/engine/first-run' && method === 'GET') {
if (!control?.updateCheck) return json(res, 200, { status: 'unavailable', gpus: [] });
const info = await engineFirstRun({ control, gpus: hardware.cachedHardware()?.gpus ?? [] });
try { let at = componentsDirectory(); while (at && !existsSync(at) && dirname(at) !== at) at = dirname(at); const f = statfsSync(at); info.freeBytes = Number(f.bavail) * Number(f.bsize); } catch { /* unknown: no warning */ }
return json(res, 200, info);
}
if (path === '/api/engine/job' && method === 'GET') {
// The newest engine download / file install, read without waiting for the native snapshot to connect (the first-run card needs it before that).
const kinds = ['update-engine', 'install-engine-file'];
return json(res, 200, { job: native.state().jobs.filter(j => kinds.includes(j.kind)).slice(-1)[0] ?? null });
}
if (path === '/api/engine/from-file' && method === 'POST') {
// The person chose an engine package in the page (the offline path). The bytes come here; Control checks and installs them.
const name = String(req.headers['x-file-name'] ?? 'engine.tar.gz').replace(/[^A-Za-z0-9._-]/g, '_').slice(-120);
if (!/\.(tar\.gz|tgz|tar\.xz|zip)$/i.test(name)) return error(res, 400, 'This is not a Strata engine package. Choose the .tar.gz or .zip file from the release page.');
const folder = join(store.DATA_DIR, 'incoming-engine');
mkdirSync(folder, { recursive: true, mode: 0o700 });
const target = join(folder, name);
await pipeline(req, createWriteStream(target, { mode: 0o600 }));
return json(res, 202, native.operation('install-engine-file', { path: target }));
}
if (path === '/api/updates/check' && method === 'GET' && control?.updateCheck) return json(res, 200, await checkComponentUpdates({ control, appUpdater, appVersion: process.env.STRATA_APP_VERSION || null }));
if (path === '/api/updates/check' && method === 'GET') return json(res, 200, await checkUpdates({ appVersion:appVersionToShow(setupState().app.version,native.state().version), engineVersion:native.state().available?(native.state().installed?.engine?.version??null):(await engine.config()).engine.version }));
@@ -1551,7 +1603,7 @@ async function api(req, res, path, method, url) {
if (cancelMatch && method === 'POST') return json(res, 200, serving.cancel(cancelMatch[1]));
// --- monitor ------------------------------------------------------------
if (path === '/api/monitor') return json(res, 200, monitor.snapshot());
if (path === '/api/monitor') { monitor.watch(); return json(res, 200, monitor.snapshot()); }
// --- skills / workspace -------------------------------------------------
if (path === '/api/skills' && method === 'GET') {
@@ -1717,6 +1769,16 @@ async function api(req, res, path, method, url) {
try { const f = rawFile(chatRoot(url.searchParams.get('sessionId') ?? ''), url.searchParams.get('path')); res.writeHead(200, { 'content-type': f.mime, 'content-length': f.bytes.length, 'cache-control': 'no-store', 'x-content-type-options': 'nosniff' }); res.end(f.bytes); return undefined; }
catch (e) { return error(res, e.code === 'outside' ? 403 : 400, e.message); }
}
/* A message too large for the model's window, made small enough to send: its text as a file in the chat's folder, or a shortened preview (oversize.mjs). */
if (path === '/api/chat/oversize' && method === 'POST') {
const body = await readBody(req, 12 * 1024 * 1024);
const text = String(body.text ?? '');
if (!text.trim()) return error(res, 400, 'text is required');
let instance = null;
try { instance = native.resolveInstance(body.nativeModelId ?? native.state().defaultId); } catch { /* no local model: the length alone decides */ }
try { return json(res, 200, await prepareOversized({ mode: body.mode === 'file' ? 'file' : 'preview', text, root: chatRoot(String(body.sessionId ?? '')), instance })); }
catch (e) { return error(res, 400, 'The message could not be prepared. Try a shorter message.'); }
}
/* "Attach as context": the file is named in the next message of the chat, so the assistant reads it again. */
if (path === '/api/chat/attachments' && method === 'GET') return json(res, 200, { files: chatAttachments.list(url.searchParams.get('sessionId') ?? '') });
if (path === '/api/chat/attachments' && method === 'POST') {
@@ -1885,6 +1947,19 @@ async function api(req, res, path, method, url) {
try { return json(res, 200, await agent.approve({ chatId: b.sessionId, callId: b.toolId, decision: b.decision })); }
catch (e) { return error(res, e.code === 'approval-gone' ? 409 : 400, e.message); }
}
/* The desktop shell's own choices (#3402, #3477): the port it asks for, the tray, and a desktop notification through the shell. */
if (path === '/api/locale' && method === 'GET') return json(res, 200, systemLocale());
if (path === '/api/shell' && method === 'GET') return json(res, 200, shellState());
if (path === '/api/shell/prefs' && method === 'POST') {
const b = await readBody(req);
try { writeShellPrefs(b); } catch (e) { return error(res, 400, e.message); }
return json(res, 200, shellState());
}
if (path === '/api/shell/notify' && method === 'POST') {
const b = await readBody(req);
if (!shellNotifier || typeof b.title !== 'string') return json(res, 200, { shown: false });
try { return json(res, 200, await shellNotifier.notify({ title: b.title, body: String(b.body ?? '') })); } catch { return json(res, 200, { shown: false }); }
}
/* The person changed the chat's mode while a turn runs: it applies to the next tool call, goal and helpers included. */
if (path === '/api/chat/mode' && method === 'POST') {
const b = await readBody(req);
@@ -1934,6 +2009,8 @@ async function api(req, res, path, method, url) {
if (chatId && !agentSkip) {
const capability = agent.status();
if (capability.state !== 'unavailable') {
const skipped = mcpSkippedNotices(capability.skippedServers);
for (const notice of skipped) res.write(`${JSON.stringify({ type: 'notice', ...notice })}\n`);
await runAgentTurn({ res, controller, chatId, body, message, workspaceId, images });
res.end();
return undefined;
+15 -1
View File
@@ -14,8 +14,22 @@ export function createLocalShare({ handle, hasKey, changed = () => {}, host = '0
if (listener) throw new Error('Sharing is already starting.');
listener = http.createServer(async (req, res) => {
const path = apiPath(new URL(req.url ?? '/', 'http://localhost').pathname);
const method = req.method ?? 'GET';
// #3208: with an API key set, a browser page of any site can call the shared API - the key decides, not the
// origin. The browser's preflight (OPTIONS) never carries the key (the spec says so), so it is answered here,
// before the path check and the key check; the real request that follows still goes through both. Without a
// key no CORS headers go out, as before: the listener stays "a device on this network" (SECURITY.md), and a
// web page of another site on the internet cannot read the model through it.
if (hasKey()) {
res.setHeader('access-control-allow-origin', '*');
res.setHeader('access-control-allow-methods', 'GET, POST, OPTIONS');
// the headers the preflight asks for (SDKs add their own; Anthropic browser clients send anthropic-version)
res.setHeader('access-control-allow-headers', String(req.headers['access-control-request-headers'] ?? '') || 'authorization, content-type, x-api-key, x-strata-client, anthropic-version, anthropic-beta');
res.setHeader('access-control-max-age', '600');
if (method === 'OPTIONS') { res.writeHead(204); res.end(); return; }
}
if (!['/v1/models','/v1/chat/completions','/v1/messages'].includes(path)) { res.writeHead(404); res.end('Not found'); return; }
try { await handle(req, res, path, req.method ?? 'GET'); }
try { await handle(req, res, path, method); }
catch { if (!res.headersSent) res.writeHead(500); res.end('Request failed'); }
});
try { await new Promise((resolve,reject) => { listener.once('error',reject); listener.listen(port,host,resolve); }); }
+10
View File
@@ -0,0 +1,10 @@
// The one rule for the name of a tool server (MCP), shared by the Settings editor (src/lib/mcp-names.ts keeps a copy; tests/mcp-rows.test.mjs
// checks they agree), the save check (native-settings.mjs) and the agent profile (agent/profile.mjs).
//
// The model sees every tool as `mcp__<server>__<tool>`, and an OpenAI-style API takes at most 64 characters for that name, so the server's
// part is limited to 32 (T6, tools and images review 2026-10-22). Before, the editor and the save check accepted 64 and the profile quietly
// dropped everything over 32 (or with a dot): the model never saw the tool and nothing said why.
export const MCP_NAME_MAX = 32;
export const MCP_NAME_RE = /^[A-Za-z0-9_-]{1,32}$/;
export const validMcpName = name => typeof name === 'string' && MCP_NAME_RE.test(name);
export const MCP_NAME_HELP = 'Use a simple server name of up to 32 characters: letters, numbers, hyphens or underscores.';
+30
View File
@@ -0,0 +1,30 @@
/**
* Which files of a model are missing, in words a person can act on (#3478). Control's model list says which required parts of the run
* config are absent (`missing`: part + file name), and the installer's status says which GGUF files and shards it could not find
* (`missing_files`, full paths). Together they give the names; the app shows them instead of only "some files are missing".
*/
import { basename } from 'node:path';
const PART_WORDS = { engine: 'the engine program', 'component:strata-engine': 'the engine program', 'tokenizer/vocab.json': 'the tokenizer', 'tokenizer/merges.txt': 'the tokenizer', 'tokenizer/token_type.json': 'the tokenizer' };
/** Short readable names, no duplicates, most useful first. */
export function missingNames(controlParts, installerFiles) {
const names = [];
const add = text => { const t = String(text ?? '').trim(); if (t && !names.includes(t)) names.push(t); };
for (const file of Array.isArray(installerFiles) ? installerFiles : []) { const name = basename(String(file).split(String.fromCharCode(92)).join('/')); add(PART_WORDS[name] ?? name); }
for (const item of Array.isArray(controlParts) ? controlParts : []) {
add(PART_WORDS[item?.part] ?? PART_WORDS[item?.file] ?? item?.file ?? item?.part);
}
return names;
}
/** The error a load or a tune raises for a model that is not whole. Names are in the sentence (at most four) and all of them in `details`. */
export function missingFilesError(model, names) {
const list = names ?? [];
const shown = list.slice(0, 4).join(', ') + (list.length > 4 ? ` and ${list.length - 4} more` : '');
const error = new Error(list.length
? `Some model files are missing: ${shown}. Use Download / repair to restore them.`
: 'Some model files are missing. Use Download / repair to restore them.');
error.details = [list.length ? `Missing for ${model}: ${list.join(', ')}` : `Missing for ${model}: Strata could not tell which files`];
return error;
}
+9 -2
View File
@@ -1,5 +1,6 @@
/** Saved API connections. Secrets never leave the host's private credentials file. */
import { modelReadsPictures, modelSupportsReasoning } from './vision-probe.mjs';
import { isOpenRouter, OPENROUTER_EFFORTS } from './agent/profile.mjs';
import { existsSync, readFileSync, writeFileSync, renameSync, mkdirSync } from 'node:fs';
import { join } from 'node:path';
import { randomUUID } from 'node:crypto';
@@ -49,7 +50,7 @@ export function normalizeBaseUrl(value) {
const publicConnection = ({ apiKey, ...connection }) => ({ ...connection, apiKeySet: !!apiKey });
const numeric = value => typeof value === 'number' && Number.isFinite(value) && value >= 0 ? value : null;
export function createModelProviders({ directory, changed = () => {}, fetchImpl = fetch }) {
export function createModelProviders({ directory, changed = () => {}, fetchImpl = fetch, knownThinks = () => null }) {
mkdirSync(directory, { recursive: true });
const file = join(directory, 'model-providers.json');
let connections = [];
@@ -142,7 +143,7 @@ export function createModelProviders({ directory, changed = () => {}, fetchImpl
const models = [...new Map(rows.filter(m => text(m.id, 256)).map(m => [m.id, { id: text(m.id, 256), name: text(m.display_name ?? m.name) || text(m.id, 256), contextMax: numeric(m.context_length ?? m.max_context_length ?? m.context_window), maxOutput: numeric(m.max_output_tokens ?? m.max_completion_tokens ?? m.top_provider?.max_completion_tokens), vision: modelReadsPictures(m), reasoning: modelSupportsReasoning(m) }])).values()];
return { models: models.sort((a, b) => a.name.localeCompare(b.name)) };
}
async function stream({ selection, message, history = [], systemPrompt = '', images = [], signal, onFrame }) {
async function stream({ selection, message, history = [], systemPrompt = '', images = [], thinking, signal, onFrame }) {
const { connection, model } = resolve(selection);
const started = Date.now();
const messages = history.filter(m => ['user', 'assistant'].includes(m.role) && typeof m.content === 'string').slice(-40).map(m => ({ role: m.role, content: m.content }));
@@ -155,6 +156,12 @@ export function createModelProviders({ directory, changed = () => {}, fetchImpl
: protocol === 'openai-responses'
? { model: model.id, input: messages, instructions: systemPrompt, stream: true, store: false }
: { model: model.id, messages: [...(systemPrompt ? [{ role: 'system', content: systemPrompt }] : []), ...messages], stream: true, ...(['openai', 'deepseek', 'openrouter', 'google', 'moonshotai'].includes(connection.provider) ? { stream_options: { include_usage: true } } : {}) };
// Match the native composer's effort names. Unknown support and other protocols keep provider defaults.
const choice = typeof thinking === 'string' ? thinking.toLowerCase() : '';
if (protocol === 'openai-chat' && (model.reasoning ?? knownThinks(connection, model.id)) === true && ['off', 'low', 'medium', 'high', 'max'].includes(choice)) {
if (isOpenRouter(connection)) payload.reasoning = { effort: OPENROUTER_EFFORTS[choice] };
else payload.reasoning_effort = ({ off: 'none', max: 'high' })[choice] ?? choice;
}
const requestSignal = signal ? AbortSignal.any([signal, AbortSignal.timeout(10 * 60 * 1000)]) : AbortSignal.timeout(10 * 60 * 1000);
const res = await request(connection, protocol === 'anthropic' ? 'messages' : protocol === 'openai-responses' ? 'responses' : 'chat/completions', { method: 'POST', body: JSON.stringify(payload), signal: requestSignal });
if (!res.body) fail('The model API returned an empty response.');
+22 -5
View File
@@ -3,6 +3,9 @@ import { variant as catalogVariant, footprint } from './catalog.mjs';
import { detect, cachedHardware } from './hardware.mjs';
import { readGpuTelemetry, gpuMemoryTotals } from './device-probes.mjs';
import { createSystemSampler } from './system-telemetry.mjs';
import { sensorPace, WATCH_MS } from './sensor-pace.mjs';
import { readTemperatures } from './temperature-telemetry.mjs';
import { addEnergy, energyWh } from './energy-meter.mjs';
import { state as servingState } from './serving.mjs';
import { originalDashboard } from './monitor-dashboard.mjs';
import { get as getStore } from './store.mjs';
@@ -58,7 +61,17 @@ let hwCache = null;
let ticker = null;
let memoryWarningShown = false;
let sensorsPending = false, sensorsAt = 0;
/* The Monitor screen asks (GET /api/monitor) every few seconds while it is open; a model that is loaded counts as live (#4066). */
let watchedUntil = 0, liveProbe = () => false;
export function watch() {
const wasIdle = Date.now() >= watchedUntil;
watchedUntil = Date.now() + WATCH_MS;
if (wasIdle) sensorsAt = 0; // opening the Monitor after a quiet spell reads the card at once
}
export function setLiveProbe(probe) { liveProbe = typeof probe === 'function' ? probe : () => false; }
let systemReading=null;
/* CPU and SSD temperatures (null where this PC cannot give one) and the GPU energy used since the host started (#3628). */
let temperatures={cpuTempC:null,ssdTempC:null};
const sampleSystem=createSystemSampler();
const finite = value => typeof value==='number'&&Number.isFinite(value);
function aggregate(readings,key,mode='sum') {
@@ -71,7 +84,7 @@ function hardwareSnapshot() {
const demo=process.env.STRATA_REAL_ENGINE!=='1',hw=state.hardware;
const devices=cachedHardware()?.gpus??hwCache?.gpus??[];
const memoryPools=Math.max(1,devices.filter(g=>!g.unified).length+Number(devices.some(g=>g.unified)));
const readings=demo?devices.map((gpu,i)=>({id:gpu.id??`${gpu.vendor}:${gpu.index}`,source:'demo',sampledAt:new Date().toISOString(),utilizationPct:Math.round(clamp(hw.gpuLoad+i*2,0,100)),temperatureC:Math.round(hw.gpuTempC+i),memoryUsedGb:round(Math.min(gpu.vramGb,hw.vramUsedGb/memoryPools),2),powerW:gpu.unified?null:round(35+hw.gpuLoad*2,0),powerLimitW:gpu.unified?null:300,pcieGen:gpu.unified?null:4,pcieGenMax:gpu.unified?null:4,pcieWidth:gpu.unified?null:16,pcieRxMbs:gpu.unified?null:round(hw.gpuLoad*1.2,1)})):hw.gpus.map(g=>({...g}));
const readings=demo?devices.map((gpu,i)=>({id:gpu.id??`${gpu.vendor}:${gpu.index}`,source:'demo',sampledAt:new Date().toISOString(),utilizationPct:Math.round(clamp(hw.gpuLoad+i*2,0,100)),temperatureC:Math.round(hw.gpuTempC+i),memoryUsedGb:round(Math.min(gpu.vramGb,hw.vramUsedGb/memoryPools),2),powerW:gpu.unified?null:round(35+hw.gpuLoad*2,0),powerLimitW:gpu.unified?null:300,pcieGen:gpu.unified?null:4,pcieGenMax:gpu.unified?null:4,pcieWidth:gpu.unified?null:16,pcieRxMbs:gpu.unified?null:round(hw.gpuLoad*1.2,1),fanPct:gpu.unified?null:Math.round(30+hw.gpuLoad*0.5),clockMhz:gpu.unified?null:Math.round(1400+hw.gpuLoad*14)})):hw.gpus.map(g=>({...g}));
const memory = gpuMemoryTotals(devices,readings);
return {
gpus:readings,cpus:systemReading?.cpus??[],cpuLoad:systemReading?.cpuLoad??null,
@@ -81,7 +94,9 @@ function hardwareSnapshot() {
ramUsedGb:systemReading?.ramUsedGb??null,ramTotalGb:systemReading?.ramTotalGb??hw.ramTotalGb,
gpuTempC:aggregate(readings,'temperatureC','max'),tempLimitC:hw.tempLimitC,
powerW:aggregate(readings,'powerW'),powerLimitW:aggregate(readings,'powerLimitW'),
pcieRxMbs:aggregate(readings,'pcieRxMbs'),diskReadGbs:systemReading?.diskReadGbs??null,diskWriteMbs:systemReading?.diskWriteMbs??null,
pcieRxMbs:aggregate(readings,'pcieRxMbs'),
cpuTempC:demo?54:temperatures.cpuTempC,ssdTempC:demo?41:temperatures.ssdTempC,energyWh:round(energyWh(),3),
diskReadGbs:systemReading?.diskReadGbs??null,diskWriteMbs:systemReading?.diskWriteMbs??null,
};
}
@@ -108,14 +123,16 @@ function approach(current, target, factor = 0.22) {
}
function tick() {
if(!sensorsPending&&Date.now()-sensorsAt>=3000){
const pace=sensorPace({live:supervisor.status().running||!!liveProbe(),watchedUntil});
if(!sensorsPending&&Date.now()-sensorsAt>=pace.intervalMs){
sensorsPending=true;sensorsAt=Date.now();
const hardware=cachedHardware()??hwCache;
void Promise.all([readGpuTelemetry(hardware?.gpus??[]),sampleSystem(hardware?.processors??[])])
.then(([readings,system])=>{state.hardware.gpus=readings;systemReading=system;})
void Promise.all([readGpuTelemetry(hardware?.gpus??[],{transfer:pace.transfer}),sampleSystem(hardware?.processors??[]),(process.env.STRATA_REAL_ENGINE==='1'?readTemperatures():Promise.resolve(temperatures)).catch(()=>({cpuTempC:null,ssdTempC:null}))])
.then(([readings,system,temps])=>{state.hardware.gpus=readings;systemReading=system;temperatures=temps;})
.catch(()=>{state.hardware.gpus=[];systemReading=null;})
.finally(()=>{sensorsPending=false;});
}
addEnergy(hardwareSnapshot().powerW,Date.now());
const hw = state.hardware;
const busy = state.phase === 'generating' || state.phase === 'reading' || state.phase === 'thinking';
+86 -35
View File
@@ -1,4 +1,5 @@
/** Native Strata control plane. All user actions are explicit; opening the UI only reads state. */
import {missingFilesError,missingNames} from './missing-files.mjs';
import { spawn } from 'node:child_process';
import {chooseParallel,applyParallel,withoutParallel,FALLBACK_NOTE} from './native-parallel.mjs';
import { randomBytes, randomUUID, createHash } from 'node:crypto';
@@ -6,25 +7,26 @@ import { existsSync, readdirSync, readFileSync, writeFileSync, mkdirSync, realpa
import { join, resolve, dirname, basename, sep, relative, isAbsolute } from 'node:path';
import { fileURLToPath } from 'node:url';
import { createServer } from 'node:net';
import { NATIVE_FIELDS, APP_OWNED_FIELDS, withoutDraftLayer, validateNativeSettings, validateMcpServers, publicMcpServers, publicNativeEnv, restoreNativeEnv } from './native-settings.mjs';
import { NATIVE_FIELDS, APP_OWNED_FIELDS, withoutDraftLayer, validateNativeSettings, validateMcpServers, publicMcpServers, publicNativeEnv, restoreNativeEnv, speedSwitchRows } from './native-settings.mjs';
import { resourceSettings, saferStart, reserveChoice } from './native-resources.mjs';
import { USER_KEY, capture as captureUserArgs, merge as mergeUserArgs, setFlag as setUserFlag, stamped as stampedUserArgs } from './native-user-args.mjs';
import { resolvePython, venvPython } from './python-resolver.mjs';
import os from 'node:os';
import { runtimeVision } from './native-vision.mjs';
import { componentsDirectory, installerEnvironment, controlEnvironment } from './control-runtime.mjs';
import { componentsDirectory, installerEnvironment, controlEnvironment, sourceServerEnvironment } from './control-runtime.mjs';
import { createEngineStep } from './engine-step.mjs';
import { dataFolderFromPick, isDriveRoot, makeFolder, rememberDataFolder, syncDataFolder, terminalSettingsFile } from './model-folder.mjs';
import { createAutoTune, writeTuneConfig } from './native-autotune.mjs';
import { plainError, PHASE_TEXT } from './update-messages.mjs';
import { checkRunConfig, describeIssues } from './run-config-check.mjs';
import { checkAndParseRunConfig, checkRunConfig, describeIssues } from './run-config-check.mjs';
import { openTextFile } from './open-text-file.mjs';
import { markEngine, engineRefusal } from './engine-version.mjs';
import { friendlyModelName } from './model-names.mjs';
import { parseSaved as parseRoles, mirror as mirrorRoles } from './gpu-roles.mjs';
import { configContextLimit } from './dsv41-context.mjs';
const ROOT=resolve(dirname(fileURLToPath(import.meta.url)),'..');
const safeId=value=>typeof value==='string'&&/^[a-z0-9_-]{1,80}$/i.test(value);
const safeId=value=>typeof value==='string'&&/^[a-z0-9](?:[a-z0-9_-]|\.(?!\.)){0,79}$/i.test(value); /* a dot inside the name: deepseek-v4.1-flash-ist-das-2.00bpw */
const busy=requests=>requests.some(r=>['queued','generating'].includes(r.state));
function readJson(path,fallback={}){try{return JSON.parse(readFileSync(path,'utf8').replace(/^\uFEFF/,''));}catch{return fallback;}}
@@ -37,7 +39,13 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
let child=null,seq=0,buffer='',pending=new Map(),definitions=[],refreshing=null,lastRead=0,connecting=null;
let snapshot={available:false,repo:preferences.repo,summary:'Connect a local Strata installation.',catalog:null,installed:null,hardware:null,error:null};
const instances=new Map(),processes=new Map(),jobs=new Map(),logs=[];
const savedJobs=readJson(join(directory,'native-jobs.json'),[]);for(const j of (Array.isArray(savedJobs)?savedJobs:[]).slice(-12))jobs.set(j.id,{...j,...(j.state==='running'?{state:'error',error:'The host restarted before completion. Check diagnostics, then retry; completed files are kept.',summary:'Interrupted operation'}:{})});
const savedJobs=readJson(join(directory,'native-jobs.json'),[]);/* An app update ends the app on purpose: the installer replaces it and starts it again. The job that asked for it was saved as running together with the app version it started from (appUpdateFrom); a start that finds a different version knows the update finished, instead of calling it an interrupted operation. */
const thisAppVersion=()=>process.env.STRATA_APP_VERSION||null;
/* A record an OLDER host wrote (no appUpdateFrom): the app's own program file was replaced after the job began, so the installer finished. <install>/host/server is ROOT's parent chain: ROOT = <install>/host. */
const appProgramNewerThan=started=>{try{const t=Date.parse(started);if(!Number.isFinite(t))return false;const dir=dirname(ROOT),exe=join(dir,process.platform==='win32'?'strata-desktop.exe':'strata-desktop'),times=[];for(const f of [exe,join(dir,'uninstall.exe')])if(existsSync(f)){const st=statSync(f);times.push(st.mtimeMs,st.ctimeMs);}/* the installer keeps the build time as the program's modified time but writes uninstall.exe and changes the program file's metadata when it installs */return times.some(x=>x>t);}catch{return false;}};
const settledByAppUpdate=j=>j.state==='running'&&j.kind==='update-strata'&&((typeof j.appUpdateFrom==='string'&&!!thisAppVersion()&&thisAppVersion()!==j.appUpdateFrom)||(typeof j.appUpdateFrom!=='string'&&appProgramNewerThan(j.startedAt)));
const neverFinishedAppUpdate=j=>j.state==='running'&&j.kind==='update-strata'&&typeof j.appUpdateFrom==='string'&&!!thisAppVersion()&&thisAppVersion()===j.appUpdateFrom;
for(const j of (Array.isArray(savedJobs)?savedJobs:[]).slice(-12))jobs.set(j.id,{...j,...(settledByAppUpdate(j)?{state:'done',error:null,summary:`Strata is updated to ${thisAppVersion()}.`,finishedAt:new Date().toISOString()}:neverFinishedAppUpdate(j)?{state:'error',error:`The update to ${j.appUpdateTo??'the new version'} did not finish: the app closed for the installer, but the old version ${j.appUpdateFrom} started again. Try Update again, or download the installer from the Strata release page (https://github.com/Niko1221/Strata/releases/latest).`,summary:'The app update did not finish',finishedAt:new Date().toISOString()}:j.state==='running'?{state:'error',error:'The host restarted before completion. Check diagnostics, then retry; completed files are kept.',summary:'Interrupted operation'}:{})});
/* Packaged mode: the app ships Strata Control (control-runtime.mjs). Models are run configs in an install folder, the engine
comes from the component folder, and a start is `strata runtime start` under the owner runtime. Nothing is installed,
updated or fetched from here. Without `control` (a source checkout) every path below is the original one. */
@@ -49,12 +57,18 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
The install only runs when the user clicks Download; starting and serving (control.start) keep STRATA_NO_ACQUIRE. */
const defaultInstall=()=>process.env.STRATA_INSTALL_DIR?resolve(process.env.STRATA_INSTALL_DIR):join(dirname(componentsDirectory()),'install');
let packagedInstall=null;
/* The component updates and the rollback go through Control, not through the native snapshot, so they work in the first seconds after the app starts and on a PC that has no engine yet. */
const COMPONENT_KINDS=['update-engine','update-strata','rollback','install-engine-file'];
const updateLog=line=>{try{console.log(`update · ${line}`);}catch{/* no console */}};
const NO_INSTALL='Installing, updating, tuning with calibration or rolling back model files and engines happens from the Terminal (strata install), never while the app runs.';
let defaultId=null,timer=null,closing=false,polling=false;
const emit=()=>{const value=state();writeFileSync(join(directory,'native-jobs.json'),JSON.stringify(value.jobs),{mode:0o600});changed(value);};
const append=(instanceId,line)=>{logs.push({at:new Date().toISOString(),instanceId,line:String(line).slice(0,2000)});if(logs.length>400)logs.shift();};
function diskSpace(path){try{const s=statfsSync(path);return {free:Number(s.bavail)*Number(s.bsize),total:Number(s.blocks)*Number(s.bsize)};}catch{return null;}}
function state(){const folder=preferences.downloadDirectory??(modelsDirectory?modelsDirectory():snapshot.installed?.data_folder),space=folder?diskSpace(folder):null;return {...snapshot,downloadDirectory:folder,downloadFreeBytes:space?.free??null,downloadTotalBytes:space?.total??null,fields:nativeFields(),instances:[...instances.values()].map(({key,configPath,fingerprint,mcpServers,...i})=>structuredClone(i)),defaultId,jobs:[...jobs.values()].slice(-12).map(({process,abort,engineAbort,rawResult,...j})=>j),capabilities:{responses:sourceText().includes('/v1/responses'),updates:!!snapshot.available,rollback:previousEngine(),componentUpdates:packaged&&typeof control.update==='function',calibration:packaged?false:snapshot.hardware?.gpus?.some(g=>/nvidia/i.test(g.name??''))??false}};}
/* Does the engine in this release serve DFlash2? Control knows (strata_control/drafting.py); asked once, false until it answers. The Models screen offers the choice only when true. */
let dflash2Served=null;
function probeDrafting(){if(!packaged||dflash2Served!==null||typeof control.draftingPlan!=='function')return;dflash2Served=false;control.draftingPlan({mode:'auto'}).then(p=>{dflash2Served=p.serves===true;emit();}).catch(()=>{});}
function state(){const folder=preferences.downloadDirectory??(modelsDirectory?modelsDirectory():snapshot.installed?.data_folder),space=folder?diskSpace(folder):null;return {...snapshot,downloadDirectory:folder,downloadFreeBytes:space?.free??null,downloadTotalBytes:space?.total??null,fields:nativeFields(),instances:[...instances.values()].map(({key,configPath,fingerprint,mcpServers,...i})=>structuredClone(i)),defaultId,jobs:[...jobs.values()].slice(-12).map(({process,abort,engineAbort,rawResult,...j})=>j),capabilities:{responses:sourceText().includes('/v1/responses'),updates:!!snapshot.available,rollback:previousEngine(),componentUpdates:packaged&&typeof control.update==='function',calibration:packaged?false:snapshot.hardware?.gpus?.some(g=>/nvidia/i.test(g.name??''))??false,dflash2:dflash2Served===true}};}
let sourceCache={path:'',time:0,text:''};
function sourceText(){try{const path=join(packaged?control.control.dir:preferences.repo,'serve/server.py'),time=statSync(path).mtimeMs;if(sourceCache.path===path&&sourceCache.time===time)return sourceCache.text;sourceCache={path,time,text:readFileSync(path,'utf8')};return sourceCache.text;}catch{return '';}}
function nativeFields(){const source=sourceText();return NATIVE_FIELDS.map(field=>({...field,supported:APP_OWNED_FIELDS.includes(field.id)||field.id.startsWith('vision_')||field.id.startsWith('mcp_')||field.id==='vram_reserve_mib'||source.includes(field.id)}));}
@@ -67,13 +81,13 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
app is reinstalled to another folder, or an update moves Control, those absolute paths point at nothing and every installed
model breaks. At every read of the installed models, a path that ends up inside a folder called `control` that no longer
exists is pointed at the Control this app has now, when the file is there. The config is saved (with a .bak) only then. */
function repairControlPaths(dir){if(!packaged||!dir||!existsSync(dir))return;let names=[];try{names=readdirSync(dir).filter(n=>/^strata-[a-z0-9_-]+\.json$/i.test(n));}catch{return;}
function repairControlPaths(dir){if(!packaged||!dir||!existsSync(dir))return;let names=[];try{names=readdirSync(dir).filter(n=>/^strata-[a-z0-9_.-]+\.json$/i.test(n));}catch{return;}
const here=control.control.dir,inside=/^(.+?[\/]control)(?=[\/]|$)(.*)$/i;
const fix=value=>{if(typeof value!=='string')return value;const m=value.match(inside);if(!m||existsSync(value)||resolve(m[1]).toLowerCase()===resolve(here).toLowerCase())return value;const candidate=join(here,m[2]);return existsSync(candidate)?candidate:value;};
for(const name of names){const path=join(dir,name),config=readJson(path,null);if(!config||typeof config!=='object')continue;
const next={...config,...(config.cwd!==undefined?{cwd:fix(config.cwd)}:{}),...(Array.isArray(config.args)?{args:config.args.map(fix)}:{})};
if(JSON.stringify(next)!==JSON.stringify(config)){try{writeModelConfig(path,next);}catch{/* read-only folder: the load reports it */}}}}
function installedInfo(info){if(!info)return info;return {...info,models:(info.models??[]).map(entry=>{const cfg=/^strata-[a-z0-9_-]+\.json$/i.test(entry.config??'')?readJson(entry.configPath??join(repo(),entry.config)):{};const at=cfg.args?.indexOf('--native')??-1,nativeFile=at>=0?cfg.args[at+1]:null;const dataFolder=typeof nativeFile==='string'&&basename(dirname(dirname(nativeFile)))==='models'?dirname(dirname(dirname(nativeFile))):info.data_folder;/* A model the person pointed at their own GGUF folder (not <data>/models/<tag>): remembered, so a later load runs setup with that folder again instead of downloading the files (#3199). */const ownGgufDir=typeof nativeFile==='string'&&basename(dirname(dirname(nativeFile)))!=='models'&&existsSync(nativeFile)?dirname(nativeFile):undefined;return {...entry,ownGgufDir,diskGb:entry.diskGb??entry.disk_gb,diskBytes:entry.diskBytes??entry.disk_bytes,sharedBytes:entry.sharedBytes??entry.shared_bytes,removableGb:entry.removableGb??entry.removable_gb,dataFolder,visionPrepared:!!cfg.vision,/* The picture files are prepared when the run config has a vision section: the Models screen then loads straight away instead of running setup again (it used to read 'off' for every model, which sent every load of a picture-capable model through "Apply settings and load", and setup writes the arguments, the draft layer and the reader back each time). */images:cfg.vision?'yes':'off',draftLayer:cfg.mtp!==false,/* Picture reading is ON unless the person turned it off: no saved choice and not loaded is undefined, not false (owner rule: pictures on by default). */visionEnabled:cfg.ui_vision_enabled??(()=>{const running=[...instances.values()].find(i=>i.model===entry.model&&i.status==='ready');return running?!!running.vision:undefined;})(),visionDevice:cfg.ui_vision_device,resourceProfile:cfg.ui_resource_profile,lowRamMode:cfg.args?.includes('--resident-experts')?'resident':cfg.args?.includes('--mmap-experts')?'mmap':'auto'};})};}
function installedInfo(info){if(!info)return info;return {...info,models:(info.models??[]).map(entry=>{const cfg=/^strata-[a-z0-9_.-]+\.json$/i.test(entry.config??'')?readJson(entry.configPath??join(repo(),entry.config)):{};const at=cfg.args?.indexOf('--native')??-1,nativeFile=at>=0?cfg.args[at+1]:null;const dataFolder=typeof nativeFile==='string'&&basename(dirname(dirname(nativeFile)))==='models'?dirname(dirname(dirname(nativeFile))):info.data_folder;/* A model the person pointed at their own GGUF folder (not <data>/models/<tag>): remembered, so a later load runs setup with that folder again instead of downloading the files (#3199). */const ownGgufDir=typeof nativeFile==='string'&&basename(dirname(dirname(nativeFile)))!=='models'&&existsSync(nativeFile)?dirname(nativeFile):undefined;return {...entry,ownGgufDir,contextMax:configContextLimit(cfg)??undefined,diskGb:entry.diskGb??entry.disk_gb,diskBytes:entry.diskBytes??entry.disk_bytes,sharedBytes:entry.sharedBytes??entry.shared_bytes,removableGb:entry.removableGb??entry.removable_gb,dataFolder,visionPrepared:!!cfg.vision,/* The picture files are prepared when the run config has a vision section: the Models screen then loads straight away instead of running setup again (it used to read 'off' for every model, which sent every load of a picture-capable model through "Apply settings and load", and setup writes the arguments, the draft layer and the reader back each time). */images:cfg.vision?'yes':'off',draftLayer:cfg.mtp!==false,drafting:cfg.drafting??'auto',/* Picture reading is ON unless the person turned it off: no saved choice and not loaded is undefined, not false (owner rule: pictures on by default). */visionEnabled:cfg.ui_vision_enabled??(()=>{const running=[...instances.values()].find(i=>i.model===entry.model&&i.status==='ready');return running?!!running.vision:undefined;})(),visionDevice:cfg.ui_vision_device,resourceProfile:cfg.ui_resource_profile,lowRamMode:cfg.args?.includes('--resident-experts')?'resident':cfg.args?.includes('--mmap-experts')?'mmap':'auto'};})};}
function installRoot(){const found=[preferences.installDir].find(d=>d&&existsSync(d));if(found)return realpathSync(found);mkdirSync(defaultInstall(),{recursive:true});return realpathSync(defaultInstall());}
/* The settings file Strata Control and the installer read (control-runtime.mjs sets STRATA_SETTINGS_FILE to it): the app's data folder is written there. */
function controlSettingsFile(){return process.env.STRATA_SETTINGS_FILE||join(dirname(componentsDirectory()),'app-settings.json');}
@@ -106,8 +120,8 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
packagedInstall=list.install_dir;
const engine=await control.engine();
const installer=await mcpCall('strata_status',{hardware:true}).catch(error=>({error:error.message}));
const disk=Object.fromEntries((installer.installed?.models??[]).map(m=>[m.model,{diskGb:m.disk_gb,diskBytes:m.disk_bytes,sharedBytes:m.shared_bytes,removableGb:m.removable_gb,partsMissing:m.parts_missing??0}]));
const models=(list.models??[]).filter(m=>m.component_status!=='invalid'&&m.source_config).map(m=>({...(disk[m.id]??{}),model:m.id,config:basename(m.source_config),configPath:m.source_config,model_name:m.name,context:m.context_tokens??0,backend:m.backend,ready:m.component_status==='present'&&!(disk[m.id]?.partsMissing),images:'off',port:undefined,last_used:null}));
const disk=Object.fromEntries((installer.installed?.models??[]).map(m=>[m.model,{diskGb:m.disk_gb,diskBytes:m.disk_bytes,sharedBytes:m.shared_bytes,removableGb:m.removable_gb,partsMissing:m.parts_missing??0,missingFiles:m.missing_files}]));
const models=(list.models??[]).filter(m=>m.component_status!=='invalid'&&m.source_config).map(m=>({...(disk[m.id]??{}),model:m.id,config:basename(m.source_config),configPath:m.source_config,model_name:m.name,context:m.context_tokens??0,backend:m.backend,ready:m.component_status==='present'&&!(disk[m.id]?.partsMissing),missing:missingNames(m.missing,disk[m.id]?.missingFiles),images:'off',port:undefined,last_used:null}));
const cores=os.cpus().length;
return {summary:engine.usable?`Strata Control is ready; engine ${engine.engine.version} (${engine.engine.platform}-${engine.engine.backend}).`:(engine.notice??'No engine for this PC was found.'),
installDir:list.install_dir,engine:{usable:engine.usable,notice:engine.notice,wrongMake:engine.wrongMake===true,version:engine.engine?.version??null,backend:engine.backend,componentsDir:engine.components_dir,previous:engine.previous??null,pinned:engine.pinned??null,updateNotice:engine.update_notice??null},
@@ -124,22 +138,26 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
if(closing)return state();
/* A read that is not forced never makes the window wait for the (slow) status of Control: it gets the last state at once, and a stale one
is refreshed in the background. Only the first read, and a forced one after an operation, waits. */
if(refreshing)return !force&&snapshot.available?state():refreshing;if(!force&&Date.now()-lastRead<45000)return state();
if(refreshing)return !force&&snapshot.available?state():force?refreshing.then(()=>refresh(true)):refreshing; /* a forced read that meets one already running (started before an engine update finished) waits for it and reads again, so it never settles on the old answer */if(!force&&Date.now()-lastRead<45000)return state();
probeDrafting();
refreshing=(async()=>{try{const [catalog,status]=packaged?await (async()=>{const s=await tool('strata_status',{hardware:true});const c=await tool('strata_models').catch(error=>({families:[],contexts:[],recommendation:{family:null,model:null},context_note:'',error:error.message}));return [c,s];})():await Promise.all([tool('strata_models'),tool('strata_status',{hardware:true})]);let visionReader=false;if(packaged){try{const e=await control.engine();const root=e?.engine?.root??(e?.engine?.exe?dirname(e.engine.exe):'');visionReader=!!root&&['strata-vision','strata-vision.exe'].some(n=>existsSync(join(root,n)));}catch{visionReader=false;}}const markImages=c=>c&&Array.isArray(c.families)?{...c,families:c.families.map(f=>({...f,imagesCapable:!!f.images,images:!!f.images&&visionReader,sizes:(f.sizes??[]).map(s=>({...s,imagesCapable:!!s.images,images:!!s.images&&visionReader}))}))}:c;snapshot={available:true,repo:packaged?status.installDir:repo(),version:packaged?(control.version??null):await execute('git',['describe','--tags','--always'],repo()).then(s=>s.trim()).catch(()=>null),summary:status.summary,catalog:markEngine(packaged?markImages(catalog):catalog,status.engine),engineInfo:{version:status.engine?.version??null,source:status.engine?.source??null},visionReader:packaged?visionReader:true,installed:installedInfo(status.installed),hardware:status.hardware,error:null,install:status.install,external:status.running??null,...(packaged?{packaged:true,installDir:status.installDir,engine:status.engine,installerError:status.installerError}:{})};lastRead=Date.now();if(status.install?.running&&![...jobs.values()].some(j=>j.state==='running'))begin('install',j=>watchInstall(j,status.install));}catch(error){snapshot={...snapshot,available:false,error:error.message};lastRead=Date.now();}emit();return state();})().finally(()=>{refreshing=null;});const run=refreshing;if(!force&&snapshot.available){run.catch(()=>{});return state();}return run;
}
function configure({repository}){if(packaged){if(instances.size||[...jobs.values()].some(j=>j.state==='running'))throw new Error('Unload models and finish operations before changing the installation.');const next=realpathSync(String(repository));if(!statSync(next).isDirectory())throw new Error('Choose the folder that holds your Strata model configurations.');preferences={...preferences,installDir:next};packagedInstall=null;disconnect();writeFileSync(settingsPath,JSON.stringify(preferences,null,2),{mode:0o600});lastRead=0;return refresh(true);}if(instances.size||[...jobs.values()].some(j=>j.state==='running'))throw new Error('Unload models and finish operations before changing the installation.');const next=realpathSync(String(repository));const old=preferences;preferences={...preferences,repo:next};try{repo();}catch(e){preferences=old;throw e;}disconnect();writeFileSync(settingsPath,JSON.stringify(preferences,null,2),{mode:0o600});lastRead=0;return refresh(true);}
function setDownloadDirectory(input,{insideJob=false}={}){if(!insideJob&&[...jobs.values()].some(j=>j.state==='running'))throw new Error('Finish the current operation before changing the folder.');if(typeof input!=='string'||!input.trim()||/[\r\n\0]/.test(input))throw new Error('Choose a model folder.');let path;try{path=realpathSync(input.trim());}catch{throw new Error('That folder does not exist. Check the path, or that the drive is connected.');}if(!statSync(path).isDirectory())throw new Error('Choose a folder, not a file.');/* A whole drive (D:\) is not filled with models: D:\Strata is made and used. */if(isDriveRoot(path)){const own=dataFolderFromPick(path).path;makeFolder(own);path=realpathSync(own);}try{accessSync(path,constants.R_OK|constants.W_OK);}catch{throw new Error('Strata cannot write in that folder. Choose a folder you can save files in.');}preferences={...preferences,downloadDirectory:path};/* the same choice for the installer and the Terminal: the Control settings file */if(packaged)rememberDataFolder(controlSettingsFile(),path,Date.now()/1000,true);writeFileSync(settingsPath,JSON.stringify(preferences,null,2),{mode:0o600});shareDataFolder();emit();return state();}
function recordFile(model){if(!safeId(model))throw new Error('Choose an installed model.');const entry=snapshot.installed?.models?.find(i=>i.model===model);if(!entry)throw new Error('This model is not installed. Download and prepare it first.');const path=realpathSync(entry.configPath??join(repo(),entry.config));if(dirname(path)!==repo()||!/^strata-[a-z0-9_-]+\.json$/i.test(basename(path)))throw new Error('The model configuration is outside the Strata installation.');return {entry,path};}
function recordFile(model){if(!safeId(model))throw new Error('Choose an installed model.');const entry=snapshot.installed?.models?.find(i=>i.model===model);if(!entry)throw new Error('This model is not installed. Download and prepare it first.');const path=realpathSync(entry.configPath??join(repo(),entry.config));if(dirname(path)!==repo()||!/^strata-[a-z0-9_.-]+\.json$/i.test(basename(path)))throw new Error('The model configuration is outside the Strata installation.');return {entry,path};}
function record(model){const {entry,path}=recordFile(model);return {entry,path,config:readJson(path)};}
/* The text file a power user edits by hand: where it is and what is wrong in it (run-config-check.mjs). */
function settingsFile(model){const {path}=recordFile(model);return {model,path,name:basename(path),...checkRunConfig(readFileSync(path,'utf8')),note:'Changes you make in the file apply the next time the model starts.'};}
async function openSettingsFile(model,options={}){const info=settingsFile(model);await openTextFile(info.path,options);return info;}
/* A mistake in the file holds the start back with the line, instead of an engine that dies on it. Warnings (an unknown key) never do. */
function assertSettingsFile(path){const check=checkRunConfig(readFileSync(path,'utf8'));if(!check.ok)throw new Error(`The settings file for this model has a mistake, so the model was not started. ${describeIssues(check.issues,'error').join(' ')}`);}
function settings(model){const {config}=record(model);const values=Object.fromEntries(NATIVE_FIELDS.map(f=>[f.id,config[f.id]??null]));if(typeof values.aliases==='string')values.aliases=values.aliases.split(',').map(v=>v.trim()).filter(Boolean);values.sampling=config.sampling??{};values.args=config.args??[];values.env=publicNativeEnv(config.env??{});return {model,values,mcp:publicMcpServers(config.mcp_servers??config.mcpServers??{}),restartNeeded:true};}
function save(model,{values,mcp}){if([...jobs.values()].some(j=>j.state==='running'))throw new Error('Finish the current operation before saving native settings.');const {path,config}=record(model);if(busy(requests().filter(r=>instances.get(r.instanceId)?.model===model)))throw new Error('Finish this model’s requests before saving its settings.');const patch=validateNativeSettings(values??{}),next={...config};for(const field of nativeFields())if(field.supported===false&&patch[field.id]!=null&&JSON.stringify(patch[field.id])!==JSON.stringify(config[field.id]))throw new Error(`${field.label} needs a newer native Strata installation. Update it in Settings → About.`);for(const [key,value] of Object.entries(patch)){if(key==='env')next.env=restoreNativeEnv(value,config.env??{});else if(value===null)delete next[key];else next[key]=value;}if('vram_reserve_mib' in patch){const base=captureUserArgs(next);const mine=setUserFlag(base,'--vram-reserve-mib',patch.vram_reserve_mib);if(mine.length)next[USER_KEY]=mine;else delete next[USER_KEY];next.args=mine.length&&patch.vram_reserve_mib!==null?mergeUserArgs(next.args??[],mine):setUserFlag(next.args??[],'--vram-reserve-mib',null);delete next.ui_reserve_applied;if(Array.isArray(next.args_written))next.args_written=[...next.args];}if(mcp!==undefined){next.mcp_servers=validateMcpServers(mcp,config.mcp_servers??config.mcpServers??{});delete next.mcpServers;}copyFileSync(path,path+'.bak');chmodSync(path+'.bak',0o600);const temp=path+'.ui-tmp';writeFileSync(temp,JSON.stringify(next,null,2)+'\n',{mode:0o600});renameSync(temp,path);return settings(model);}
function checkedSettingsConfig(text){const check=checkAndParseRunConfig(text);if(!check.ok)throw new Error(`The settings file for this model has a mistake, so the model was not started. ${describeIssues(check.issues,'error').join(' ')}`);return check.config;}
function settings(model){const {entry,config}=record(model);const values=Object.fromEntries(NATIVE_FIELDS.map(f=>[f.id,config[f.id]??null]));if(typeof values.aliases==='string')values.aliases=values.aliases.split(',').map(v=>v.trim()).filter(Boolean);values.sampling=config.sampling??{};values.args=config.args??[];values.env=publicNativeEnv(config.env??{});return {model,values,mcp:publicMcpServers(config.mcp_servers??config.mcpServers??{}),restartNeeded:true,capabilities:{dsExperts:ownEngine(config)&&dsExpertsSupported(config)},speedSwitches:speedSwitchRows(config.env??{},{deepseek:ownEngine(config),arch:config.arch??null,backend:config.backend??entry?.backend??snapshot.installed?.engine?.backend??null,gpus:snapshot.hardware?.gpus??[]})};}
function save(model,{values,mcp}){if([...jobs.values()].some(j=>j.state==='running'))throw new Error('Finish the current operation before saving native settings.');const {path,config}=record(model);if(busy(requests().filter(r=>instances.get(r.instanceId)?.model===model)))throw new Error('Finish this model’s requests before saving its settings.');const patch=validateNativeSettings(values??{}),next={...config};if(patch.ds_experts!=null&&!dsExpertsSupported(config))throw new Error('The Speed choice needs the newer DeepSeek engine. Update the engine in Settings, then choose it.');for(const field of nativeFields())if(field.supported===false&&patch[field.id]!=null&&JSON.stringify(patch[field.id])!==JSON.stringify(config[field.id]))throw new Error(`${field.label} needs a newer native Strata installation. Update it in Settings → About.`);for(const [key,value] of Object.entries(patch)){if(key==='env')next.env=restoreNativeEnv(value,config.env??{});else if(value===null)delete next[key];else next[key]=value;}if('vram_reserve_mib' in patch){const base=captureUserArgs(next);const mine=setUserFlag(base,'--vram-reserve-mib',patch.vram_reserve_mib);if(mine.length)next[USER_KEY]=mine;else delete next[USER_KEY];next.args=mine.length&&patch.vram_reserve_mib!==null?mergeUserArgs(next.args??[],mine):setUserFlag(next.args??[],'--vram-reserve-mib',null);delete next.ui_reserve_applied;if(Array.isArray(next.args_written))next.args_written=[...next.args];}if(mcp!==undefined){next.mcp_servers=validateMcpServers(mcp,config.mcp_servers??config.mcpServers??{});delete next.mcpServers;}copyFileSync(path,path+'.bak');chmodSync(path+'.bak',0o600);const temp=path+'.ui-tmp';writeFileSync(temp,JSON.stringify(next,null,2)+'\n',{mode:0o600});renameSync(temp,path);return settings(model);}
function previousEngine(){if(packaged)return !!snapshot.engine?.previous;try{const data=snapshot.installed?.data_dir??snapshot.installed?.data_folder;return [join(preferences.repo,'engine/.previous'),data&&join(data,'engine/.previous')].filter(Boolean).some(existsSync)&&readFileSync(join(preferences.repo,'setup.py'),'utf8').includes('--rollback-engine');}catch{return false;}}
async function plan(args){await refresh();if(!snapshot.available)throw new Error(snapshot.error);return tool('strata_install',installArgs(args,false));}
async function plan(args){await refresh();if(!snapshot.available)throw new Error(snapshot.error);const result=await tool('strata_install',installArgs(args,false));
/* A chosen drafter that is not on this computer yet is part of the download: its size is listed beside the model's (Control decides; nothing is fetched here). */
const choice=args.drafting;if(packaged&&typeof choice==='string'&&choice!=='auto'&&typeof control.draftingPlan==='function'){try{const d=await control.draftingPlan({mode:choice,family:args.family??null,dataDir:snapshot.installed?.data_folder??null});if(d.needed)return {...result,drafting:{needed:true,gb:d.gb,line:d.line}};}catch{/* the model plan still stands */}}
return result;}
/* The folder the person picked in the first-run chooser can be the one that holds their GGUF files (or a folder inside it). It is theirs, not a Strata data folder: packs are never made in it, and a Strata folder around it (<Strata>/models/<tag>) is used instead (#3507, #3424). */
function ownFilesDataFolder(candidate,args){if(!candidate||args.data_dir!==undefined||!args.gguf_dir)return candidate;const same=(a,b)=>{const x=resolve(a),y=resolve(b);return process.platform==='win32'?x.toLowerCase()===y.toLowerCase():x===y;};const inside=(a,b)=>{const rel=relative(resolve(b),resolve(a));return rel===''||(!!rel&&!rel.startsWith('..')&&!isAbsolute(rel));};if(!same(candidate,args.gguf_dir)&&!inside(candidate,args.gguf_dir))return candidate;const models=dirname(resolve(args.gguf_dir)),top=dirname(models);return basename(models).toLowerCase()==='models'&&/strata/i.test(basename(top))?top:undefined;}
function installArgs(args,confirm){const allowed=['family','model','context','vision','kv','gpu','gpus','backend','low_ram','port','gguf_dir'];const size=snapshot.catalog?.families?.find(f=>f.family===args.family)?.sizes.find(s=>s.model===args.model);const existing=snapshot.installed?.models?.find(m=>m.model===size?.id);/* A model adopted from the person's own GGUF folder reports that folder's parent as its data folder: it is theirs, not Strata's, so a second preparation never uses it for the packs. */if(args.gguf_dir===undefined&&existing?.ownGgufDir)args={...args,gguf_dir:existing.ownGgufDir};const data=nativeDataDirectory(ownFilesDataFolder(args.data_dir??(args.gguf_dir?undefined:existing?.dataFolder)??preferences.downloadDirectory??modelsDirectory?.(),args));if(data&&packaged&&!args.data_dir)rememberDataFolder(controlSettingsFile(),data);return {...(data?{data_dir:data}:{}),...Object.fromEntries(allowed.filter(k=>args[k]!==undefined).map(k=>[k,args[k]])),/* One card is "gpu", never "gpus" (the installer wants two or more there: "Gpus='0' is not valid", #3203). */...(/^\d{1,2}$/.test(String(args.gpus??''))?{gpu:Number(args.gpus),gpus:undefined}:{}),/* Picture reading is only downloaded when it was asked for (Read pictures); the default stays the smaller download. */...(packaged&&args.vision===undefined?{vision:'no'}:{}),confirm};}
@@ -176,7 +194,7 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
const note=engineStep?.summaryNote();
const finalResult=engineStep?job.result:result;
if(note)return {...finalResult,summary:note};
if(args.loadAfter){const model=(result.install.family??args.family??'qwen')==='qwen'?String(args.model).toLowerCase():`${args.family}-${String(args.model).toLowerCase()}`;return loadEngine(job,{model,multiple:!!args.multiple,resourceProfile:args.resourceProfile,vision:args.loadVision});}
if(args.loadAfter){const model=(result.install.family??args.family??'qwen')==='qwen'?String(args.model).toLowerCase():`${args.family}-${String(args.model).toLowerCase()}`;return loadEngine(job,{model,multiple:!!args.multiple,resourceProfile:args.resourceProfile,vision:args.loadVision,...(typeof args.drafting==='string'?{values:{drafting:args.drafting}}:{})});}
return finalResult;
}
}
@@ -199,6 +217,12 @@ export function createNativeControl({directory,requests=()=>[],changed=()=>{},sp
if(still.length)throw new Error(`Some files could not be removed: ${still.slice(0,3).map(p=>basename(p)).join('; ')}`);
return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
/* The configuration a load starts the engine with (before its port and key): the saved model config, the user's choices and the resource profile. The tuner measures exactly this, so a saved tuning belongs to the same choices a load makes. */
/* DeepSeek-V4.1-Flash runs on its own engine (strata-dsv41) that takes none of the Flash-Next options the app adds on its own (request slots, the RAM budget, the output clamp, the resource profile): a flag it does not know ends the start with "unknown option". */
const ownEngine=config=>config?.arch==='deepseek41'||/strata-dsv41/i.test(String(config?.exe??''));
/* The DeepSeek engine's version, read from the folder the run config names (engine-dsv41/<version>-<platform>-...). --experts exists from engine 0.1.50: an older engine ends the start with "unknown option", so the choice is offered and passed only for 0.1.50 or newer. */
const DS_EXPERTS_MIN=[0,1,50];
function dsEngineVersion(config){const m=/engine-dsv41[\\/](\d+(?:\.\d+)+)-/.exec(String(config?.exe??''))||/engine-dsv41[\\/](\d+(?:\.\d+)+)-/.exec(String((config?.lib_dirs??[])[0]??''));return m?m[1].split('.').map(Number):null;}
function dsExpertsSupported(config){const v=dsEngineVersion(config);if(!v)return false;for(let i=0;i<DS_EXPERTS_MIN.length;i++){const a=v[i]??0,b=DS_EXPERTS_MIN[i];if(a!==b)return a>b;}return true;}
function buildConfig(args,config){
const vision=runtimeVision(config,args.vision);
const visionPreferences=args.vision!==undefined?{ui_vision_enabled:args.vision!=='no',ui_vision_device:args.vision==='no'?config.ui_vision_device??'yes':args.vision}:{};
@@ -207,7 +231,7 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
const values0=args.values??{},cleared='vram_reserve_mib' in values0&&(values0.vram_reserve_mib===null||values0.vram_reserve_mib===''),base0=cleared?{...config,vram_reserve_mib:undefined,[USER_KEY]:setUserFlag(config[USER_KEY],'--vram-reserve-mib',null)}:config;
let mine=captureUserArgs(base0);const asked=cleared?null:reserveChoice({...base0,[USER_KEY]:mine},snapshot.hardware,values0).asked;
if(asked!==null)mine=setUserFlag(mine,'--vram-reserve-mib',asked);
const resources=args.resourceProfile?resourceSettings({...base0,[USER_KEY]:mine,vision},snapshot.hardware,args.resourceProfile,modelArena(args.model),values0):null;
const resources=args.resourceProfile&&!ownEngine(config)?resourceSettings({...base0,[USER_KEY]:mine,vision},snapshot.hardware,args.resourceProfile,modelArena(args.model),values0):null;
const cfg={...config,...visionPreferences,...(resources?{...resources,ui_resource_profile:args.resourceProfile}:{}),host:'127.0.0.1',open_browser:false,...validateNativeSettings(args.values??{}),vision};
/* The app's suggested cards only fill a gap: a model whose settings file already names its GPUs ("gpu": [1, 0], with "gpu_order": "as_given") keeps them (#3256). A card the person picked in the screen always wins. */
const namedInFile=(Array.isArray(config.gpu)?config.gpu.length>0:config.gpu!==undefined&&config.gpu!==null&&config.gpu!=='')||!!config.gpu_roles;
@@ -218,23 +242,46 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
if(mine.length){cfg.args=mergeUserArgs(cfg.args??[],mine);cfg[USER_KEY]=mine;}else delete cfg[USER_KEY];
if(cfg.vision){if(cfg.vision_tokens!=null)cfg.vision={...cfg.vision,max_tokens:cfg.vision_tokens};if(cfg.vision_min_tokens!=null)cfg.vision={...cfg.vision,min_tokens:cfg.vision_min_tokens};}cfg.mcp={...(cfg.mcp??{}),...(cfg.mcp_timeout_s!=null?{timeout_s:cfg.mcp_timeout_s}:{}),...(cfg.mcp_max_rounds!=null?{max_rounds:cfg.mcp_max_rounds}:{}),...(cfg.mcp_max_result_chars!=null?{max_result_chars:cfg.mcp_max_result_chars}:{})};
/* The model's own choices win over what setup wrote into the arguments: "Draft layer (MTP)" off leaves --mtp out of every start, and a picture reading that is off leaves --vision out (setup writes both again whenever it runs; the saved settings are applied here, on every load). */
if(cfg.mtp===false&&Array.isArray(cfg.args))cfg.args=withoutDraftLayer(cfg.args);
if((cfg.mtp===false||cfg.drafting==='off')&&Array.isArray(cfg.args))cfg.args=withoutDraftLayer(cfg.args);
if(!cfg.vision&&Array.isArray(cfg.args))cfg.args=cfg.args.filter(a=>a!=='--vision');
for(const arg of cfg.args??[])if(arg.endsWith('.gguf')&&!existsSync(resolve(repo(),arg)))throw new Error('An engine argument names a missing GGUF. Restore the saved arguments or repair the files.');
/* The app's own choice of request slots (4, fewer where the card has no room) is part of the configuration, so a tune measures the engine a load will start: slots take VRAM from the expert cache. */
const parallelChoice=packaged?chooseParallel({config:cfg,values:args.values??{},hardware:snapshot.hardware?.gpus?.length?snapshot.hardware:{gpus:hardwareProbe()??[]}}):null;
const parallelChoice=packaged&&!ownEngine(config)?chooseParallel({config:cfg,values:args.values??{},hardware:snapshot.hardware?.gpus?.length?snapshot.hardware:{gpus:hardwareProbe()??[]}}):null;
let parallelNote=null;
/* The agent asks for long answers (a thinking model can use most of the window): let the engine clamp the output to what is left of the context instead of refusing the request. */
if(packaged&&cfg.fit_max_tokens===undefined)cfg.fit_max_tokens=true;
if(packaged&&!ownEngine(config)&&cfg.fit_max_tokens===undefined)cfg.fit_max_tokens=true;
if(parallelChoice){const applied=applyParallel(cfg,parallelChoice);for(const k of Object.keys(cfg))if(!(k in applied.cfg))delete cfg[k];Object.assign(cfg,applied.cfg);parallelNote=applied.note;}
/* DeepSeek's Speed choice: the engine decides by itself (--experts auto: 5 when the cards total 16 GB or less, else 6), so nothing is passed unless the person chose 5 or 6. */
if(ownEngine(config)){const list=[];const old=Array.isArray(cfg.args)?cfg.args:[];for(let i=0;i<old.length;i++){if(old[i]==='--experts'){i++;continue;}list.push(old[i]);}if(dsExpertsSupported(config)&&(cfg.ds_experts==='5'||cfg.ds_experts==='6'))list.push('--experts',cfg.ds_experts);else if(!dsExpertsSupported(config)&&old.includes('--experts'))list.splice(0,list.length,...old);cfg.args=list;delete cfg.ds_experts;}
else delete cfg.ds_experts;
return {cfg,resources,visionPreferences,parallelChoice,parallelNote};
}
/* The DFlash2 drafter, before a load that uses it: Control says whether the choice needs it (and the file is not here), downloads it pinned and checked, and packs it, with plain progress in the job.
Never stops the load: a drafter that cannot be had leaves the MTP running, and the job says so. The choice itself is remembered in the model's settings file. */
async function prepareDrafter(job,args,cfg,configFile){
const asked=args.values?.drafting;
if(asked!==undefined)rememberSetting(configFile,'drafting',asked===null||asked==='auto'?null:asked);
if(!packaged||typeof control.draftingFetch!=='function'||!cfg.drafting||cfg.drafting==='auto'||cfg.drafting==='mtp'||cfg.drafting==='off')return;
const work=join(directory,'native-instances');mkdirSync(work,{recursive:true});const probe=join(work,'drafting-'+randomUUID()+'.json');
try{
writeFileSync(probe,JSON.stringify(cfg),{mode:0o600});
job.engineAbort??=new AbortController();
await control.draftingFetch({configPath:probe,mode:cfg.drafting,signal:job.engineAbort.signal,onEvent:event=>{if(event.event==='progress'&&job.state==='running'){job.summary=event.step;job.details=event.total>1&&event.done<event.total?`${(event.done/1e9).toFixed(2)} of ${(event.total/1e9).toFixed(2)} GB`:undefined;emit();}}});
if(job.state==='running'){job.summary='Loading the model…';job.details=undefined;emit();}
}catch(error){if(job.state==='running'){job.drafterNote=error.message;job.summary='The DFlash2 drafter could not be downloaded, so the model starts with the MTP. Loading the model…';emit();}}
finally{try{rmSync(probe,{force:true});}catch{/* best effort */}}
}
/* One key of a model's settings file, set or removed (the Drafting choice): written like the Settings screen writes it, to be there for the next load. */
function rememberSetting(configFile,key,value){try{const current=readJson(configFile);if(value===null?!(key in current):current[key]===value)return;const next={...current};if(value===null)delete next[key];else next[key]=value;const temp=configFile+'.ui-tmp';writeFileSync(temp,JSON.stringify(next,null,2)+'\n',{mode:0o600});renameSync(temp,configFile);}catch{/* the load goes on with the choice for this start */}}
/* Which drafter runs, when it is not the one that was asked for (the engine's own words, "Drafting: MTP (DFlash2 didn't fit)"): under Details of the load. */
async function noteDrafting(job,instance,key){let note='';try{const r=await fetcher(`http://127.0.0.1:${instance.port}/metrics`,{headers:{authorization:`Bearer ${key}`},signal:AbortSignal.timeout(2000)});if(r.ok)note=String((await r.json())?.engine?.drafting_note??'');}catch{/* a convenience */}const lines=[job.drafterNote,note].filter(Boolean);if(lines.length)job.details=[job.details,...lines].filter(Boolean).join('\n');}
async function loadEngine(job,args){
await refresh(true);if(job.state!=='running')return;const {config,entry,path:configFile}=record(args.model);assertSettingsFile(configFile);if(packaged){const engine=await control.engine();if(!engine.usable)throw new Error(ENGINE_NOT_RECOGNISED+(engine.notice?` ${engine.notice}`:''));}if(!entry.ready)throw new Error('Some model files are missing. Use Download / repair to restore them.');
await refresh(true);if(job.state!=='running')return;const {entry,path:configFile}=recordFile(args.model);const config=checkedSettingsConfig(readFileSync(configFile,'utf8'));if(packaged){const engine=await control.engine();if(!engine.usable)throw new Error(ENGINE_NOT_RECOGNISED+(engine.notice?` ${engine.notice}`:''));}if(!entry.ready){const error=missingFilesError(args.model,entry.missing);append(args.model,error.details[0]);throw error;}
if(externalRunning().length)throw new Error('A Strata server is already running outside this UI. Stop it before loading here, or connect its API in Settings → Models.');
if(args.multiple&&instances.size>=8)throw new Error('Unload an instance before adding another. Up to eight native instances can run here.');
const {cfg,resources,visionPreferences,parallelChoice,parallelNote:chosenNote}=buildConfig(args,config);
let parallelNote=chosenNote;
await prepareDrafter(job,args,cfg,configFile);if(job.state!=='running')return;
const fingerprint=createHash('sha256').update(JSON.stringify({...cfg,api_key:undefined,port:undefined})).digest('hex');
const placement=Array.isArray(cfg.gpu)?cfg.gpu:cfg.gpu===undefined?[]:[cfg.gpu];const matching=[...instances.values()].find(i=>i.model===args.model&&JSON.stringify(i.gpus)===JSON.stringify(placement));
if(matching?.status==='ready'&&matching.fingerprint===fingerprint&&(args.multiple||instances.size===1))return {summary:`${matching.label} is already ready.`,instanceId:matching.id};
@@ -258,7 +305,7 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
/* The engine starts under Control's owner runtime, not as our child: a saved profile per slot, then `runtime start`. */
const used=new Set([...instances.values()].map(i=>i.profile));let slot=1;while(used.has('app-'+slot))slot++;instance.profile='app-'+slot;
instances.set(id,instance);if(!defaultId||!args.multiple||wasDefault)defaultId=id;emit();
const startWith=value=>{writeFileSync(configPath,JSON.stringify(value,null,2),{mode:0o600});return control.stop(instance.profile).catch(()=>{}).then(()=>control.saveProfile(instance.profile,configPath)).then(()=>control.start(instance.profile,{waitSeconds:300}));};
const startWith=value=>{writeFileSync(configPath,JSON.stringify(value,null,2),{mode:0o600});return control.stop(instance.profile).catch(()=>{}).then(()=>control.saveProfile(instance.profile,configPath)).then(()=>control.start(instance.profile,{waitSeconds:ownEngine(config)?Math.min(600,Math.max(300,Number(config.engine_ready_s)||300)):300}));};
started=startWith(cfg);
started.then(()=>append(id,'Strata Control reports the engine is ready.'),error=>{append(id,error.message);});
try{await started;}catch(firstError){let error=firstError;
@@ -271,7 +318,7 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
/* ONE automatic second try in all, never saved, never repeated (args.retrySafer false turns it off): with a safer memory setting when the
failure looks like a memory stop, and without several-requests-at-once when that was the app's own default (not a number the user chose).
Both changes ride on the same single try, with the plain sentence of the memory change when there is one. */
const allowed=args.retrySafer!==false&&job.state==='running';
const allowed=args.retrySafer!==false&&job.state==='running'&&!ownEngine(config);
const safer=allowed?saferStart(cfg,snapshot.hardware,modelArena(args.model),said):null;
const dropParallel=allowed&&cfg.parallel>1&&!parallelChoice?.explicit;
if(!safer&&!dropParallel){instance.status='error';instance.error=said;emit();throw new Error(said);}
@@ -282,14 +329,14 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
try{await startWith(retryConfig);append(id,'Strata Control reports the engine is ready (second try).');}
catch(second){const said2=humanStartError(second);instance.status='error';instance.error=said2;/* the first stop and the second try stay in job.details; the pop-up shows the last stop once */emit();throw new Error(instance.error);}}}
} else {
proc=spawnProcess(py,[join(repo(),'serve/server.py'),'--engine','strata','--config',configPath,'--port',String(cfg.port),'--host','127.0.0.1'],{cwd:repo(),detached:process.platform!=='win32',stdio:['ignore','pipe','pipe'],env:{...process.env,PYTHONUNBUFFERED:'1',PYTHONUTF8:'1',PYTHONIOENCODING:'utf-8',STRATA_API_KEY:''}});
proc=spawnProcess(py,[join(repo(),'serve/server.py'),'--engine','strata','--config',configPath,'--port',String(cfg.port),'--host','127.0.0.1'],{cwd:repo(),detached:process.platform!=='win32',stdio:['ignore','pipe','pipe'],env:sourceServerEnvironment()});
instances.set(id,instance);processes.set(id,proc);if(!defaultId||!args.multiple||wasDefault)defaultId=id;
for(const stream of [proc.stdout,proc.stderr])stream.on('data',chunk=>{const lines=chunk.toString().split('\n').filter(Boolean);lines.forEach(line=>append(id,line.replaceAll(key,'[redacted]')));job.summary=lines.at(-1)??job.summary;emit();});
proc.on('error',error=>{instance.status='error';instance.error=error.message;emit();});proc.on('exit',code=>{if(instances.has(id)&&instance.status!=='unloading'){instance.status='error';instance.error=`Engine stopped${code===null?'':` (exit ${code})`}. Open diagnostics or reload.`;emit();}});emit();
}
const deadline=Date.now()+300000;while(Date.now()<deadline&&job.state==='running'){
if(instance.status==='error')throw new Error(instance.error);
try{const response=await fetcher(`http://127.0.0.1:${instance.port}/health`,{headers:{authorization:`Bearer ${key}`},signal:AbortSignal.timeout(2000)});if(job.state!=='running'){await unload(id);return;}if(response.ok){instance.status='ready';instance.phase='idle';if(resources||args.vision!==undefined||JSON.stringify(cfg[USER_KEY]??null)!==JSON.stringify(config[USER_KEY]??null))writeModelConfig(record(args.model).path,{...config,...(resources?{...resources,ui_resource_profile:args.resourceProfile}:{}),...visionPreferences,...(cfg[USER_KEY]?{[USER_KEY]:cfg[USER_KEY]}:{})});emit();ensurePoll();instance.parallelNote=parallelNote;return {summary:retried?`${instance.label} is ready, after a second try: ${retried.words}`:`${instance.label} is ready.${parallelNote?' '+parallelNote:''}`,instanceId:id};}}catch{}
try{const response=await fetcher(`http://127.0.0.1:${instance.port}/health`,{headers:{authorization:`Bearer ${key}`},signal:AbortSignal.timeout(2000)});if(job.state!=='running'){await unload(id);return;}if(response.ok){instance.status='ready';instance.phase='idle';await noteDrafting(job,instance,key);if(resources||args.vision!==undefined||JSON.stringify(cfg[USER_KEY]??null)!==JSON.stringify(config[USER_KEY]??null))writeModelConfig(record(args.model).path,{...config,...(resources?{...resources,ui_resource_profile:args.resourceProfile}:{}),...visionPreferences,...(cfg[USER_KEY]?{[USER_KEY]:cfg[USER_KEY]}:{})});emit();ensurePoll();instance.parallelNote=parallelNote;return {summary:retried?`${instance.label} is ready, after a second try: ${retried.words}`:`${instance.label} is ready.${parallelNote?' '+parallelNote:''}`,instanceId:id};}}catch{}
await new Promise(resolve=>setTimeout(resolve,1200));
}
if(job.state!=='running'){await unload(id);return;}
@@ -303,7 +350,7 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
function ensurePoll(){if(!timer){timer=setInterval(()=>void poll(),2000);timer.unref?.();}void poll();}
async function benchmark(job,args){const instance=resolveInstance(args.instanceId);job.instanceId=instance.id;if(busy(requests().filter(r=>r.instanceId===instance.id)))throw new Error('Wait for this model to finish before benchmarking.');const started=Date.now();job.abort=new AbortController();const response=await fetcher(`http://127.0.0.1:${instance.port}/v1/chat/completions`,{method:'POST',headers:{'content-type':'application/json',authorization:`Bearer ${instance.key}`},signal:AbortSignal.any([job.abort.signal,AbortSignal.timeout(180000)]),body:JSON.stringify({model:instance.apiModelId,messages:[{role:'user',content:'Explain how a rainbow forms in clear, simple language.'}],temperature:0,reasoning_effort:'none',max_tokens:200,stream:false})});const body=await response.json();if(!response.ok)throw new Error(body.error?.message??'Benchmark failed.');if(job.state!=='running')return;const result={summary:'Measured benchmark complete.',at:new Date().toISOString(),model:instance.apiModelId,durationMs:Date.now()-started,tokensIn:body.usage?.prompt_tokens??null,tokensOut:body.usage?.completion_tokens??null,outputTps:body.timings?.predicted_per_second??null,promptTps:body.timings?.prompt_per_second??null,source:'engine',settings:'Greedy · thinking off · up to 200 output tokens'};writeFileSync(join(directory,'native-benchmark.json'),JSON.stringify(result,null,2),{mode:0o600});return result;}
function resolveInstance(id){const aliases=[...instances.values()].filter(i=>i.aliases?.includes(id));if(aliases.length>1&&!instances.has(id)&&![...instances.values()].some(i=>i.apiModelId===id))throw new Error('This alias matches more than one instance. Choose its exact model ID.');const i=instances.get(id)??[...instances.values()].find(i=>i.apiModelId===id)??aliases[0]??(!id&&instances.size===1?[...instances.values()][0]:undefined);if(!i)throw new Error('Choose a loaded model ID.');if(i.status!=='ready')throw new Error('This model is not ready.');return i;}
async function program(job,args){if(packaged)return componentProgram(job);if(externalRunning().length)throw new Error('Stop the native server running outside this UI before changing engine files.');if(instances.size)throw new Error('Unload native models before changing engine files.');const folder=repo();let {command,args:prefix}=python(),argv;
async function program(job,args){if(packaged)return componentProgram(job,args);if(externalRunning().length)throw new Error('Stop the native server running outside this UI before changing engine files.');if(instances.size)throw new Error('Unload native models before changing engine files.');const folder=repo();let {command,args:prefix}=python(),argv;
if(job.kind==='calibrate'||job.kind==='tune'){if(!state().capabilities.calibration)throw new Error('This installation calibrates NVIDIA GPUs only. Use the short benchmark on other devices.');const model=args.model??snapshot.installed?.models?.[0]?.model;const chosen=record(model);if(!chosen.entry.ready)throw new Error('Prepare this model before calibrating.');command=venvPython(folder);prefix=[];if(!existsSync(command))throw new Error('Prepare the Python environment first.');argv=['-c','import sys; from pathlib import Path; import setup; sys.exit(0 if setup.calibrate_config(Path(sys.argv[1])) else 1)',chosen.path];}
else if(job.kind==='rollback'){if(!previousEngine())throw new Error('No saved previous engine is available.');argv=[join(folder,'setup.py'),'--rollback-engine'];}
else if(job.kind==='update-engine'){if(!snapshot.installed?.models?.length)throw new Error('Prepare a model first. Its installer downloads the matching engine.');argv=[join(folder,'setup.py'),'--update'];}
@@ -314,29 +361,33 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
/* Packaged app: engine and Control updates and the engine rollback go through Control's explicit update step (`strata update`,
`strata components rollback`), never through git, setup.py or a download in this process. The app itself is updated by the
shell's signed Tauri updater. Models are not part of any of it. */
async function componentProgram(job){
if(!['update-engine','update-strata','rollback'].includes(job.kind)||typeof control.update!=='function')throw new Error(NO_INSTALL);
async function componentProgram(job,args={}){
if(!COMPONENT_KINDS.includes(job.kind)||typeof control.update!=='function')throw new Error(NO_INSTALL);
if(instances.size)throw new Error('Unload native models before changing engine files.');
job.abort=new AbortController();
const subject=job.kind==='update-strata'?'Strata':'engine';
try{return await componentSteps(job,subject);}catch(error){throw error?.details!==undefined&&error.message?error:plainError(error,subject);}
try{return await componentSteps(job,subject,args);}catch(error){updateLog(`${job.kind} failed: ${String(error?.message??error).slice(0,400)}`);throw error?.details!==undefined&&error.message?error:plainError(error,subject);}
}
async function componentSteps(job,subject){
async function componentSteps(job,subject,args={}){
const onEvent=event=>{
if(event.event==='progress'){job.summary=`Downloading: ${event.percent}%`;job.result={install:{step:job.summary,download:{file:event.label,percent:event.percent}}};}
else if(event.event==='phase'&&PHASE_TEXT[event.phase])job.summary=PHASE_TEXT[event.phase];
else if(event.event==='waiting')job.summary='Waiting for the running model to stop…';
emit();
};
updateLog(`${job.kind} started (app ${thisAppVersion()??'unknown'})`);
if(job.kind==='install-engine-file'){if(typeof args.path!=='string'||!args.path)throw new Error('Choose the engine file first.');let done;try{done=await control.installEngineFile(args.path,{onEvent,signal:job.abort.signal});}finally{try{const own=resolve(directory,'incoming-engine')+sep;if(resolve(args.path).startsWith(own))rmSync(args.path,{force:true});}catch{/* the copy in the host's own folder; a leftover is harmless */}}return {summary:updateSummary(done)==='Everything is already up to date.'?(done.results?.[0]?.note??'Nothing was changed.'):updateSummary(done),result:done};}
if(job.kind==='rollback'){const done=await control.rollbackEngine();return {summary:'The previous engine is back. Your models and settings are unchanged.',result:done};}
if(job.kind==='update-engine'){const done=await control.update({engine:true,control:false,onEvent,signal:job.abort.signal});return {summary:updateSummary(done),result:done};}
const done=await control.update({engine:false,control:true,onEvent,signal:job.abort.signal});
updateLog(`Control step finished: ${updateSummary(done)}`);
const notes=[updateSummary(done)];
if(appUpdater){
job.summary='Looking for a new version of the app…';emit();
let verdict;
try{verdict=await appUpdater.check();}catch(error){if(error?.code!=='app-update-unavailable')throw error;verdict={available:false};notes.push(appUpdater.unavailable||error.message);}
if(verdict.available){await appUpdater.install({signal:job.abort.signal,onProgress:p=>{if(p.total){job.summary=`Downloading the app update: ${Math.round(100*p.downloaded/p.total)}%`;job.result={install:{step:job.summary,download:{file:'Strata app',percent:Math.round(100*p.downloaded/p.total)}}};emit();}}});notes.push('The app update is installed. Strata will restart.');}
updateLog(`app check: ${verdict.available?`version ${verdict.version} is available`:'no newer app version'}`);
if(verdict.available){job.appUpdateFrom=thisAppVersion();job.appUpdateTo=verdict.version??null;emit();updateLog(`app update to ${verdict.version??'?'} starting; the installer will end this app`);await appUpdater.install({signal:job.abort.signal,onProgress:p=>{if(p.total){job.summary=`Downloading the app update: ${Math.round(100*p.downloaded/p.total)}%`;job.result={install:{step:job.summary,download:{file:'Strata app',percent:Math.round(100*p.downloaded/p.total)}}};emit();}}});notes.push('The app update is installed. Strata will restart.');}
}
return {summary:notes.filter(Boolean).join(' '),result:done};
}
@@ -382,7 +433,7 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
const room=args.resourceProfile==='work';
return autotuner.start(job,job.kind==='recheck'?'recheck':'tune',{configPath:path,presetId:args.presetId,mode:args.mode,use:args.use,goal:room?'room':'speed',leaveRam:room?20:0,allowSmallDifferences:args.allowSmallDifferences===true,name:args.name},{onProgress:p=>{job.summary=p.text??job.summary;emit();}});
}
function operation(kind,args={}){if(!['install','load','benchmark','tune','autotune','recheck','calibrate','update-engine','update-strata','rollback','remove'].includes(kind))throw new Error('Choose a supported native operation.');if(!snapshot.available)throw new Error(snapshot.error??'Connect the native Strata installation first.');if(kind==='install'||kind==='load'){const refusal=engineRefusal(snapshot.catalog,args.model,snapshot.engineInfo);if(refusal)throw new Error(refusal);}return begin(kind,job=>kind==='install'?install(job,args):kind==='load'?loadEngine(job,args):kind==='benchmark'?benchmark(job,args):kind==='tune'?tune(job,args):kind==='autotune'||kind==='recheck'?autotune(job,args):kind==='remove'?removeModel(job,args):program(job,args),{guided:args.guided===true,model:args.model,...(args.family?{family:args.family}:{})});}
function operation(kind,args={}){if(!['install','load','benchmark','tune','autotune','recheck','calibrate','update-engine','update-strata','rollback','install-engine-file','remove'].includes(kind))throw new Error('Choose a supported native operation.');if(!snapshot.available&&!(packaged&&COMPONENT_KINDS.includes(kind)))throw new Error(snapshot.error??'Connect the native Strata installation first.');if(kind==='install'||kind==='load'){const refusal=engineRefusal(snapshot.catalog,args.model,snapshot.engineInfo);if(refusal)throw new Error(refusal);}return begin(kind,job=>kind==='install'?install(job,args):kind==='load'?loadEngine(job,args):kind==='benchmark'?benchmark(job,args):kind==='tune'?tune(job,args):kind==='autotune'||kind==='recheck'?autotune(job,args):kind==='remove'?removeModel(job,args):program(job,args),{guided:args.guided===true,model:args.model,...(args.family?{family:args.family}:{})});}
/* ---- The models folder card (0.2.2): what a picked folder means, and the three choices. The copying is Control's (strata models move). ---- */
const currentModelsFolder=()=>preferences.downloadDirectory??(modelsDirectory?modelsDirectory():snapshot.installed?.data_folder)??null;
const moveFile=()=>join(directory,'models-move.cancel');
@@ -405,7 +456,7 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
const under=value=>typeof value==='string'&&[ 'models','packs','mtp' ].some(item=>{const base=join(old,item).toLowerCase();const v=value.toLowerCase();return v===base||v.startsWith(base+sep)||v.startsWith(base+'/');});
const scan=value=>Array.isArray(value)?value.some(scan):value&&typeof value==='object'?Object.values(value).some(scan):under(value);
for(const name of readdirSync(folder)){
if(!/^strata-[a-z0-9_-]+\.json$/i.test(name)||name.endsWith('.shared-settings.json'))continue;
if(!/^strata-[a-z0-9_.-]+\.json$/i.test(name)||name.endsWith('.shared-settings.json'))continue;
if(!scan(readJson(join(folder,name))))continue;
mkdirSync(kept,{recursive:true});renameSync(join(folder,name),join(kept,name));aside.push(name);
}
@@ -482,5 +533,5 @@ return {summary:result.summary??'Deleted',removed:result.plan?.gb??null};}
// Remove nonserializable process references from public jobs.
const publicState=()=>{const value=state();value.jobs=value.jobs.map(({process,engineAbort,rawResult,...job})=>job);return value;};
async function localPreview(path){if(!packaged)throw new Error('Checking local model files is part of the Strata app.');if(typeof path!=='string'||!path.trim()||[...path].some(c=>c.charCodeAt(0)<32))throw new Error('Choose a folder.');return control.importPlan(path.trim());}
return {state:publicState,tuneStatus,tuneEffective,tuneOff,tunePresets:{list:()=>autotuner.list(),action:(kind,id,extra)=>autotuner.action(kind,id,extra),undoFor:async args=>autotuner.undoFor(tuneConfig(args).path)},refresh,localPreview,configure,setDownloadDirectory,planModelsFolder,startModelsFolder,interruptedMove,finishInterruptedMove,leaveInterruptedMove,plan,settings,save,settingsFile,openSettingsFile,operation,cancel,unload,resolveInstance,poll,logs:()=>logs.slice(),benchmarkReport:()=>readJson(join(directory,'native-benchmark.json'),null),setDefault(id){defaultId=resolveInstance(id).id;emit();return publicState();},hasLive:()=>[...instances.values()].some(i=>i.status==='ready'),readyInstances:()=>[...instances.values()].filter(i=>i.status==='ready'),close};
return {state:publicState,tuneStatus,tuneEffective,tuneOff,tunePresets:{list:()=>autotuner.list(),action:(kind,id,extra)=>autotuner.action(kind,id,extra),undoFor:async args=>autotuner.undoFor(tuneConfig(args).path)},refresh,localPreview,configure,setDownloadDirectory,planModelsFolder,startModelsFolder,interruptedMove,finishInterruptedMove,leaveInterruptedMove,plan,settings,save,settingsFile,openSettingsFile,operation,cancel,unload,resolveInstance,poll,logs:()=>logs.slice(),benchmarkReport:()=>readJson(join(directory,'native-benchmark.json'),null),setDefault(id){defaultId=resolveInstance(id).id;emit();return publicState();},hasLive:()=>[...instances.values()].some(i=>i.status==='ready'),readyInstances:()=>[...instances.values()].filter(i=>i.status==='ready'),defaultInstanceId:()=>defaultId,close};
}
+9 -4
View File
@@ -28,12 +28,16 @@ export function defaultReserves(config,hardware,baselineReserve=0){
const placement=Array.isArray(config.gpu)?config.gpu:[config.gpu??0];
const placed=(hardware?.gpus??[]).filter(g=>placement.includes(g.index)),gpuMemory=placed.map(g=>g.vram_gb).filter(n=>Number.isFinite(n)&&n>0);
const headless=placed.length>0&&placed.every(computeOnly); // no display on the card (the desktop is on an integrated GPU): no desktop to keep room for
const floor=headless?(config.vision?ENGINE_DEFAULT_RESERVE_MIB:COMPUTE_ONLY_RESERVE_MIB):gpuMemory.length&&Math.min(...gpuMemory)>16?1024:1536;
const safety=headless?floor:Math.max(config.vision?2048:floor,baselineReserve);
/* Only a picture reader that runs on the GPU needs VRAM of its own: the CPU reader (vision.gpu false) took the same 2048 floor and cost a 12 GB card a quarter of its expert cache (0.2.3 tuning, notes-023-tune.md). */
const pictures=config.vision?(typeof config.vision==='object'?Boolean(config.vision.gpu):true):false;
const floor=headless?(pictures?ENGINE_DEFAULT_RESERVE_MIB:COMPUTE_ONLY_RESERVE_MIB):gpuMemory.length&&Math.min(...gpuMemory)>16?1024:1536;
const safety=headless?floor:Math.max(pictures?2048:floor,baselineReserve);
/* A card that shares the PC's memory (Strix Halo, other APUs) has no memory of its own to keep a fifth of: that fifth is RAM, and "Leave 20% for my work" already keeps it
* through the RAM budget. A fifth of the shared pool here held back 16 GB of expert cache for nothing (#3510); the display floor stays. */
const shared=placed.length>0&&placed.some(g=>g.uma===true||g.unified===true);
return {work:gpuMemory.length&&!headless&&!shared?Math.max(safety,Math.floor(Math.min(...gpuMemory)*1024*.2)):safety,performance:safety};
/* "work" keeps a fifth of the card, but on a card of 16 GB or less no more than 1536 MiB: a fifth of 12 GB (2457) cost 20% of the speed against 1536, which already leaves 1.1 GiB of VRAM free after the load (RTX 5070, Gyro-S 64K, 5 pairs; notes-023-tune.md). */
const fifth=gpuMemory.length?Math.floor(Math.min(...gpuMemory)*1024*.2):0,cap=gpuMemory.length&&Math.min(...gpuMemory)<=16?1536:Infinity;
return {work:gpuMemory.length&&!headless&&!shared?Math.max(safety,Math.min(fifth,cap)):safety,performance:safety};
}
/** The reserve the person asked for (null: nothing) and the default for each profile. A 0.2.0-beta file has the key and the same number in "args" with no note that the
@@ -63,7 +67,8 @@ export function resourceSettings(config,hardware,profile,model={},values={}){
const cores=Number(hardware?.cpu?.cores)||0;
if(profile==='work'&&cores>1){
const at=args.indexOf('--pool-workers'),previous=at>=0?Number(args[at+1]):0;
const workers=Math.max(1,Math.floor(cores*.8));remove('--pool-workers');
/* 80% of the physical cores rounded UP, never more than the cores less the host thread (the engine's own count): floor(0.8 x 6) = 4 ran 8-10% slower than the engine's 5 on a 6-core Ryzen (notes-023-tune.md); 12 logical threads gave 9 in 0.2.1. */
const workers=Math.max(1,Math.min(cores-1,Math.ceil(cores*.8)));remove('--pool-workers');
args.push('--pool-workers',String(previous>0?Math.min(previous,workers):workers));
}
// Preserve calibrated worker counts and the installation's safety/vision reserve.
+68 -5
View File
@@ -1,3 +1,4 @@
import { validMcpName, MCP_NAME_HELP } from './mcp-names.mjs';
/** Editable native keys. Unknown config keys and program paths stay intact. */
export const NATIVE_FIELDS=[
{id:'reasoning_budget_tokens',label:'Thinking token budget',group:'Responses',type:'number',min:0,max:1048576,hint:'Leave blank for the model default.'},
@@ -8,7 +9,9 @@ export const NATIVE_FIELDS=[
{id:'idle_unload_s',label:'Unload after idle seconds',group:'Serving',type:'number',min:0,max:86400,hint:'Zero keeps the model loaded.'},
{id:'lazy_load',label:'Load on first request',group:'Serving',type:'boolean'},
{id:'mtp',label:'Draft layer (MTP)',group:'Serving',type:'boolean',hint:'The draft layer guesses several words ahead, which makes answers faster, and costs about 1 GB of graphics memory and some RAM. Turn it off if the model stops while it starts, or the card is small: the expert cache gets that memory back and answers are slower. Default is on.'},
{id:'drafting',label:'Drafting',group:'Serving',type:'enum',choices:['auto','mtp','dflash2','off'],hint:'Which drafter guesses words ahead to make answers faster. Auto (recommended) lets Strata pick from what it measured on cards like yours; today that is the MTP. DFlash2 is for Qwen3.8-Flash-Next only and downloads a 1.2 GB drafter once. If DFlash2 cannot run, the MTP is used and Details says why. Off turns the draft layer off.'},
{id:'batchMtp',label:'MTP per request',group:'Serving',type:'boolean',hint:'Each concurrent request uses the draft layer. Left alone, Strata turns it on for one graphics card with 24 GB or more and several requests at once, and off otherwise; choose on or off here to decide yourself.'},
{id:'ds_experts',label:'Speed (DeepSeek)',group:'Serving',type:'enum',choices:['5','6'],forModel:'deepseek',hint:'Faster uses 5 of 6 experts per word: a small quality change, usually not noticeable. Full quality uses all 6. Left alone, the engine picks Faster on a PC whose graphics cards total 16 GB or less.'},
{id:'parallel',label:'Concurrent requests',group:'Serving',type:'number',min:1,max:8,hint:'More slots need more memory and can slow each response.'},
{id:'engine_silence_s',label:'Engine stall timeout (seconds)',group:'Recovery',type:'number',min:10,max:3600},
{id:'api_monitor',label:'Keep API request metadata',group:'Recovery',type:'boolean'},
@@ -23,8 +26,66 @@ export const NATIVE_FIELDS=[
{id:'mcp_max_rounds',label:'Maximum tool rounds',group:'Tools',type:'number',min:1,max:100},
{id:'mcp_max_result_chars',label:'Maximum tool result characters',group:'Tools',type:'number',min:100,max:1000000},
];
/** Engine environment names the app shows and edits: STRATA_* and the DeepSeek engine's DS4_*. Any other name in a model's file is kept untouched and not shown. */
export const ENV_FLAG=/^(?:STRATA|DS4)_[A-Z0-9_]+$/;
/**
* The speed features of 0.2.7 that have a switch in Expert settings. A switch has a default ('on' or 'off'); outside it, it writes exactly the
* environment entries `env` into the model's settings file ("env"); back in its default it removes them, so a default launch carries none of them.
* "Faster long prompts" (Qwen KVA) is OFF by default (owner decision, 26 October): STRATA_KVA=1 turns it on, and an earlier test build's STRATA_KVA=0 reads as off.
* The other two are on by default. The names are the engine switch contract (phaseA-tests/notes-027-plan.md); strata_control/speed_switches.py says the
* same for the Terminal (tests/terminal/test_speed_switches.py compares the two).
*/
export const SPEED_SWITCHES=[
{id:'faster_long_prompts',label:'Faster long prompts',group:'Speed',
hint:'Reads very long prompts faster by approximating part of the work. Off by default: answers about long documents can miss or mix up details.',
default:'off',env:{STRATA_KVA:'1'}},
{id:'faster_12gb_nvidia',label:'Faster answers on 12 GB NVIDIA cards',group:'Speed',
hint:'Prefers the parts of the model already on the graphics card, which makes answers much faster on 12 GB cards. Turn off if answers look worse than before.',
default:'on',env:{STRATA_ROUTE_PRIOR:'0'}},
{id:'faster_radeon_9000',label:'Faster answers on Radeon RX 9000 / AI PRO R9700',group:'Speed',
hint:'Shares the work between the graphics card and the processor in a way that makes answers faster on these cards. Turn off to go back to the previous setting, which is slower.',
default:'on',env:{STRATA_PCIE_FRAC_DEFAULT:'old'}},
{id:'deepseek_speedups',label:'DeepSeek speed-ups',group:'Speed',
hint:'Reads long prompts faster and loads the parts of the model it needs faster. Turn off if DeepSeek stops working or its answers look wrong; it then runs as in version 0.2.6.',
default:'on',env:{DS4_DEFAULTS:'026',DS4_FAST_ATTN:'0',DS4_ATTN_SLAB:'0',DS4_COPY_PIN:'0',DS4_COPY_HUGETLB:'0',DS4_STORE_SPARE:'0',DS4_STORE_PUT:'0'},
/* an earlier test build wrote only these six for the same state: they read as the switch being set, and go with it */
legacy:{DS4_FAST_ATTN:'0',DS4_ATTN_SLAB:'0',DS4_COPY_PIN:'0',DS4_COPY_HUGETLB:'0',DS4_STORE_SPARE:'0',DS4_STORE_PUT:'0'}},
];
const GFX12=/^gfx120[01]$/i, RADEON_9000=/\b(?:RX\s*9\d{3}|AI\s*PRO\s*R97\d{2}|R97\d{2})\b/i;
/**
* Which switches make sense for this model on this PC: only those are shown. `backend` is the engine's (cuda, hip, sycl, metal, cpu). `arch` is the "arch"
* of the model's run config: setup writes it only for the families with an engine path of their own (Qwen3.6-35B-A3B "qwen35moe", DeepSeek "deepseek41");
* the Qwen3.8-Flash-Next files (the original, Swift, Coder, Unsloth, Gyro, NVFP4) have none. KVA was measured on the Flash-Next shape (48 layers, 512
* experts) and the engine declines it on any other, so "Faster long prompts" is shown for a model without an "arch" only.
*/
/** A discrete NVIDIA card of 13 GiB or less: the engine's rule for residency-biased routing (not an integrated chip, not a card that shares system memory). */
const smallNvidia=gpus=>(gpus??[]).some(g=>{
if(g.usable===false||g.uma||g.integrated||(g.unified_gb??0)>0)return false;
const vendor=String(g.vendor??'').toLowerCase();
if(vendor!=='nvidia'&&!(vendor===''&&/nvidia|geforce|\brtx\b|\bgtx\b|quadro|tesla/i.test(String(g.name??''))))return false;
return g.vram_gb>0&&g.vram_gb<=13;
});
export function speedSwitchApplies(id,{deepseek=false,backend=null,gpus=[],arch=null}={}){
if(id==='deepseek_speedups')return !!deepseek;
if(deepseek)return false; // the Qwen engine's switches
if(id==='faster_long_prompts')return !arch&&(backend==='cuda'||backend==='hip');
if(id==='faster_12gb_nvidia')return !arch&&backend==='cuda'&&smallNvidia(gpus); // the engine's routing needs the Flash-Next shape (512 experts, top-10) and a CUDA build
if(id==='faster_radeon_9000')return backend==='hip'&&(gpus??[]).some(g=>(!g.vendor||g.vendor==='amd')&&(GFX12.test(String(g.arch??''))||(!g.arch&&RADEON_9000.test(String(g.name??'')))));
return false;
}
/** The switch is outside its default when every entry it writes is in the environment with its value; anything less or other (a hand-edited file, STRATA_KVA=0) is the default state and is left alone. */
export function speedSwitchIsSet(env,id){const sw=SPEED_SWITCHES.find(s=>s.id===id),has=entries=>!!entries&&Object.keys(entries).length>0&&Object.entries(entries).every(([name,value])=>env?.[name]===value);return !!sw&&(has(sw.env)||has(sw.legacy));}
/** Is the feature on for a launch with this environment? */
export function speedSwitchIsOn(env,id){const sw=SPEED_SWITCHES.find(s=>s.id===id);return !!sw&&speedSwitchIsSet(env,id)!==(sw.default==='on');}
/** The environment with the feature on or off: its default removes the switch's entries, the other state writes them. Nothing else in it changes. */
export function setSpeedSwitch(env,id,on){
const sw=SPEED_SWITCHES.find(s=>s.id===id);if(!sw)throw new Error(`Unknown speed switch: ${id}`);
const write=on!==(sw.default==='on'),next={...(env??{})};for(const [name,value] of Object.entries(sw.env)){if(write)next[name]=value;else delete next[name];}if(!write)for(const name of Object.keys(sw.legacy??{}))delete next[name];return next;
}
/** What the settings answer carries: each switch that applies, with its words, its default, the entries it writes outside the default, and whether the feature is on. */
export function speedSwitchRows(env,context){return SPEED_SWITCHES.filter(s=>speedSwitchApplies(s.id,context)).map(s=>({...s,on:speedSwitchIsOn(env,s.id)}));}
/** Settings the app itself applies when it starts a model (they are not read from the engine's source, so the engine's text never says it supports them). */
export const APP_OWNED_FIELDS=['mtp','batchMtp'];
export const APP_OWNED_FIELDS=['mtp','batchMtp','drafting','ds_experts'];
/** The engine arguments without the draft layer: --mtp <folder or count>, --spec-min-p <p> (its confidence threshold) and --batch-mtp. --spec stays: prompt lookup still uses its windows. Same rule as strata_control/draft_layer.py. */
export function withoutDraftLayer(args){
const out=[];const list=(args??[]).map(String);
@@ -37,7 +98,7 @@ export function validateNativeSettings(values){
for(const [key,value] of Object.entries(values)){
if(key==='args'){if(!Array.isArray(value)||value.length>200||value.some(v=>typeof v!=='string'||v.length>4000||/[\n\r\0]/.test(v))||value.some(v=>['--host','--port','--api-key'].includes(v)))throw new Error('Engine arguments must be text values. Network access stays managed by Strata.');result.args=[...value];continue;}
if(key==='env'){if(!value||typeof value!=='object'||Array.isArray(value))throw new Error('Engine environment must be an object.');const env={};for(const [name,text] of Object.entries(value)){if(!/^STRATA_[A-Z0-9_]+$/.test(name)||typeof text!=='string'||text.length>4000||text.includes('\0'))throw new Error('Use STRATA_ environment flags with text values.');env[name]=text;}result.env=env;continue;}
if(key==='env'){if(!value||typeof value!=='object'||Array.isArray(value))throw new Error('Engine environment must be an object.');const env={};for(const [name,text] of Object.entries(value)){if(!ENV_FLAG.test(name)||typeof text!=='string'||text.length>4000||text.includes('\0'))throw new Error('Use STRATA_ environment flags with text values.');env[name]=text;}result.env=env;continue;}
if(key==='sampling'){if(!value||typeof value!=='object'||Array.isArray(value))throw new Error('Sampling must be an object.');const limits={temperature:[0,10],top_p:[Number.MIN_VALUE,1],top_k:[1,64],min_p:[0,1],repetition_penalty:[Number.MIN_VALUE,10],presence_penalty:[0,10],frequency_penalty:[0,10],penalty_last_n:[0,1048576],seed:[1,2147483647]};for(const [name,n] of Object.entries(value)){if(name==='experimental_speed_projection'){if(typeof n!=='boolean')throw new Error('Speed projection must be on or off.');continue;}if(!limits[name]||!Number.isFinite(n)||(['top_k','seed','penalty_last_n'].includes(name)&&!Number.isInteger(n))||n<limits[name][0]||n>limits[name][1])throw new Error(`Choose a valid sampling ${name}.`);}result.sampling={...value};continue;}
const field=NATIVE_FIELDS.find(f=>f.id===key);if(!field)throw new Error(`Unknown native setting: ${key}`);
if(value===null||value===''){result[key]=null;continue;}
@@ -56,7 +117,9 @@ export function publicMcpServers(servers){return Object.fromEntries(Object.entri
export function validateMcpServers(servers,previous={}){
if(!servers||typeof servers!=='object'||Array.isArray(servers)||Object.keys(servers).length>30)throw new Error('MCP servers must be an object with at most 30 entries.');
const out={};for(const [name,s] of Object.entries(servers)){
if(!/^[a-z0-9_-]{1,64}$/i.test(name)||!s||typeof s!=='object'||Array.isArray(s))throw new Error('Give each tool server a simple, unique name.');
if(!s||typeof s!=='object'||Array.isArray(s))throw new Error('Give each tool server a simple, unique name.');
// a name that was saved before the 32-character rule stays saveable (unrelated settings must not be blocked); the agent reports it as left out
if(!validMcpName(name)&&!Object.hasOwn(previous??{},name))throw new Error(MCP_NAME_HELP);
if(!!s.command===!!s.url)throw new Error(`${name}: choose a command or HTTP URL.`);
for(const key of Object.keys(s))if(!['command','args','env','cwd','url','headers','disabled'].includes(key))throw new Error(`${name}: unsupported property ${key}.`);
if(s.command&&(typeof s.command!=='string'||s.command.length>1000||/[\n\r\0]/.test(s.command)))throw new Error(`${name}: choose a valid program.`);
@@ -70,5 +133,5 @@ export function validateMcpServers(servers,previous={}){
}
const sensitive=name=>/KEY|TOKEN|SECRET|PASSWORD|CREDENTIAL/i.test(name);
export function publicNativeEnv(env){return Object.fromEntries(Object.entries(env).filter(([name])=>/^STRATA_[A-Z0-9_]+$/.test(name)).map(([name,value])=>[name,sensitive(name)?MASK:value]));}
export function restoreNativeEnv(env,previous={}){const result=Object.fromEntries(Object.entries(previous).filter(([name])=>!/^STRATA_[A-Z0-9_]+$/.test(name)));for(const [name,value] of Object.entries(env)){if(value===MASK){if(previous[name]===undefined)throw new Error(`Enter a new value for ${name}.`);result[name]=previous[name];}else result[name]=value;}return result;}
export function publicNativeEnv(env){return Object.fromEntries(Object.entries(env).filter(([name])=>ENV_FLAG.test(name)).map(([name,value])=>[name,sensitive(name)?MASK:value]));}
export function restoreNativeEnv(env,previous={}){const result=Object.fromEntries(Object.entries(previous).filter(([name])=>!ENV_FLAG.test(name)));for(const [name,value] of Object.entries(env)){if(value===MASK){if(previous[name]===undefined)throw new Error(`Enter a new value for ${name}.`);result[name]=previous[name];}else result[name]=value;}return result;}
+30
View File
@@ -0,0 +1,30 @@
/**
* A message too large for the model's window, made small enough to send (agent/context-fit.mjs decides that it is too large).
* The person picks how, on the error card:
* file the whole text is saved in the chat's folder and the message names the file; the assistant reads it in parts
* preview the start and the end of the text, cut to what the window can hold
* Nothing is sent from here: the page sends the returned message as the person's next message.
*/
import { mkdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { countTokens, oversizedReplacement } from './agent/context-fit.mjs';
const stamp = d => `${d.getFullYear()}${String(d.getMonth() + 1).padStart(2, '0')}${String(d.getDate()).padStart(2, '0')}-${String(d.getHours()).padStart(2, '0')}${String(d.getMinutes()).padStart(2, '0')}${String(d.getSeconds()).padStart(2, '0')}`;
/**
* @param {{mode:'file'|'preview', text:string, root:string|null, instance?:{port:number, key?:string, apiModelId?:string, context?:number, metrics?:any}|null, now?:Date, fetchImpl?:typeof fetch}} input
* @returns {Promise<{message:string, mode:string, tokens:number|null, kept:number, path?:string}>}
*/
export async function prepareOversized({ mode, text, root, instance = null, now = new Date(), fetchImpl = fetch }) {
const window = instance?.metrics?.engine?.max_context ?? instance?.context ?? 32768;
const tokens = instance ? await countTokens({ baseUrl: `http://127.0.0.1:${instance.port}/v1`, key: instance.key ?? '', model: instance.apiModelId ?? '', text, fetchImpl }) : null;
const counted = tokens ?? Math.ceil(text.length / 3); // not countable: three characters a token is on the safe side for code and logs
if (mode === 'file') {
if (!root) throw new Error('This chat has no folder yet. Choose one first.');
mkdirSync(root, { recursive: true });
const path = join(root, `pasted-message-${stamp(now)}.txt`);
writeFileSync(path, text, { encoding: 'utf8', flag: 'wx' });
return { mode, tokens, path, ...oversizedReplacement({ text, mode, tokens: counted, window, path }) };
}
return { mode: 'preview', tokens, ...oversizedReplacement({ text, mode: 'preview', tokens: counted, window }) };
}
+66
View File
@@ -0,0 +1,66 @@
/**
* The computer's regional format, for the Monitor's cost line (#3628): which region and, where the system names it, which currency the
* person uses. Read once, without PowerShell:
* Windows the regional format (Settings > Time & language > Region > Regional format) from the registry: LocaleName ("en-US"); when it
* names no region, the country or region setting (Geo\Name).
* macOS `defaults read -g AppleLocale` ("en_US", or "de_DE@currency=CHF" when the person set another currency).
* Linux LC_ALL, LC_MONETARY or LANG ("de_DE.UTF-8"), and the locale's own currency when `locale -k int_curr_symbol` answers.
* Nothing here is a guess: no region means no answer, and the page then falls back to its own language list.
*/
import { execFileSync } from 'node:child_process';
const NONE = Object.freeze({ locale: null, region: null, currency: null, source: null });
/** "en_US.UTF-8" / "de_DE@euro" / "zh-Hans-CN" -> { locale: "en-US", region: "US" } (null parts when the tag has none). */
export function parseLocaleTag(text) {
const tag = String(text ?? '').trim().replace(/[.@].*$/, '').replace(/_/g, '-');
if (!tag || /^(c|posix)$/i.test(tag)) return { locale: null, region: null };
const parts = tag.split('-');
const region = parts.slice(1).find(p => /^[A-Za-z]{2}$/.test(p)) ?? null;
const language = /^[A-Za-z]{2,3}$/.test(parts[0]) ? parts[0].toLowerCase() : null;
if (!language) return { locale: null, region: null };
return { locale: region ? `${language}-${region.toUpperCase()}` : language, region: region ? region.toUpperCase() : null };
}
export function parseRegValue(out, name) {
const m = new RegExp(`^\\s*${name}\\s+REG_SZ\\s+(.*?)\\s*$`, 'mi').exec(String(out ?? ''));
return m ? m[1] : null;
}
const currencyCode = value => (/^[A-Z]{3}$/.test(String(value ?? '').trim().toUpperCase()) ? String(value).trim().toUpperCase() : null);
export function detectRegion({ platform = process.platform, env = process.env, run = (cmd, args) => execFileSync(cmd, args, { encoding: 'utf8', timeout: 4000, windowsHide: true, stdio: ['ignore', 'pipe', 'ignore'] }) } = {}) {
try {
if (platform === 'win32') {
const intl = 'HKCU\\Control Panel\\International';
const named = parseLocaleTag(parseRegValue(run('reg', ['query', intl, '/v', 'LocaleName']), 'LocaleName'));
if (named.region) return { ...named, currency: null, source: 'windows' };
let geo = null;
try { geo = parseRegValue(run('reg', ['query', `${intl}\\Geo`, '/v', 'Name']), 'Name'); } catch { /* no country setting */ }
const region = /^[A-Za-z]{2}$/.test(geo ?? '') ? geo.toUpperCase() : null;
return region ? { locale: named.locale ? `${named.locale}-${region}` : null, region, currency: null, source: 'windows' } : { ...NONE };
}
if (platform === 'darwin') {
const raw = String(run('defaults', ['read', '-g', 'AppleLocale'])).trim();
const named = parseLocaleTag(raw);
const currency = currencyCode(/@currency=([A-Za-z]{3})/.exec(raw)?.[1]);
return named.region || currency ? { ...named, currency, source: 'macos' } : { ...NONE };
}
const named = parseLocaleTag(env.LC_ALL || env.LC_MONETARY || env.LANG);
let currency = null;
try { currency = currencyCode(/int_curr_symbol="?([A-Za-z]{3})/.exec(run('locale', ['-k', 'int_curr_symbol']))?.[1]); } catch { /* no locale command */ }
return named.region || currency ? { ...named, currency, source: 'linux' } : { ...NONE };
} catch {
return { ...NONE };
}
}
let cached = null;
/** The answer is cached: the regional format does not change while the host runs. */
export function systemLocale(options) {
if (options) return detectRegion(options);
// A test or a demo can name the regional format (the screenshots of the Monitor); a packaged app never reads it.
if (process.env.STRATA_TEST_LOCALE && process.env.STRATA_PACKAGED !== '1') { const named = parseLocaleTag(process.env.STRATA_TEST_LOCALE); return { ...named, currency: null, source: 'test' }; }
cached ??= detectRegion();
return cached;
}
+8
View File
@@ -134,6 +134,14 @@ export function checkRunConfig(text, { known = KNOWN_KEYS } = {}) {
return { ok: !issues.some(i => i.level === 'error'), issues };
}
/** Validate and parse one captured text value, so callers use the checked snapshot. */
export function checkAndParseRunConfig(text, options) {
text = String(text ?? '');
const checked = checkRunConfig(text, options);
if (!checked.ok) return { ...checked, config: null };
return { ...checked, config: JSON.parse(text.replace(/^\uFEFF/, '')) };
}
/** The English lines of the issues, `Line 12: …`, for the Details of a refused start. */
export function describeIssues(issues, level) {
return issues.filter(i => !level || i.level === level).map(i => `${i.line ? `Line ${i.line}: ` : ''}${i.message}`);
+16
View File
@@ -0,0 +1,16 @@
/**
* How often the host asks the graphics card driver for its readings (#4066).
* It used to start two nvidia-smi processes every 3 s for as long as the app was open, even with nothing loaded and
* the Monitor out of sight; on some PCs that was felt as a stutter while typing. Now the 3 s pace is kept while a model
* runs or the Monitor is being looked at, and an idle app asks once every 30 s.
*/
export const ACTIVE_MS = 3000;
export const IDLE_MS = 30000;
/** The Monitor screen asks again every 10 s while it is open; this is how long one ask keeps the fast pace. */
export const WATCH_MS = 25000;
/** `live`: a model is loaded or an engine runs. `watchedUntil`: ms timestamp until which the Monitor was last seen open. */
export function sensorPace({ live = false, watchedUntil = 0, now = Date.now() } = {}) {
const watched = now < watchedUntil;
return { intervalMs: live || watched ? ACTIVE_MS : IDLE_MS, transfer: !!live };
}
+77
View File
@@ -0,0 +1,77 @@
/**
* The desktop shell's own choices, kept in <Strata data folder>/shell-prefs.json (#3402, #3477): the shell reads the file when it starts
* (the port it asks for, whether closing the window keeps Strata in the tray, the language of the tray menu), before any page exists.
* The page changes them through the host (POST /api/shell/prefs); a change of the port or the tray applies the next time Strata starts.
*
* { "port": 0 | 1024..65535, "tray": false, "language": "en" }
*
* Port 0 is the default: the shell asks for 47831 and takes a free port when that one is busy.
*/
import { mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { componentsDirectory } from './control-runtime.mjs';
export const DEFAULT_PORT = 47831;
export const LANGUAGES = ['en', 'zh', 'ja', 'de', 'es', 'ru', 'pt', 'ko'];
export const DEFAULTS = Object.freeze({ port: 0, tray: false, language: 'en' });
export const validPort = value => Number.isInteger(value) && value >= 1024 && value <= 65535;
export function shellPrefsFile({ env = process.env } = {}) {
return join(dirname(componentsDirectory({ env })), 'shell-prefs.json');
}
/** The file's content with every field checked; anything missing or wrong is the default. */
export function cleanPrefs(raw) {
const value = raw && typeof raw === 'object' ? raw : {};
return {
port: validPort(value.port) ? value.port : 0,
tray: value.tray === true,
language: LANGUAGES.includes(value.language) ? value.language : 'en',
};
}
export function readShellPrefs(file = shellPrefsFile()) {
try { return cleanPrefs(JSON.parse(readFileSync(file, 'utf8').replace(/^/, ''))); } catch { return { ...DEFAULTS }; }
}
/** Merge a change into the file. Throws a plain sentence for a port that cannot be used. */
export function writeShellPrefs(patch, file = shellPrefsFile()) {
const next = readShellPrefs(file);
if (patch && 'port' in patch) {
const wanted = patch.port === null || patch.port === '' || patch.port === 0 ? 0 : Number(patch.port);
if (wanted !== 0 && !validPort(wanted)) throw new Error('Choose a port between 1024 and 65535, or leave it empty to let Strata choose.');
next.port = wanted;
}
if (patch && 'tray' in patch) next.tray = patch.tray === true;
if (patch && 'language' in patch && LANGUAGES.includes(patch.language)) next.language = patch.language;
mkdirSync(dirname(file), { recursive: true });
const temporary = `${file}.${process.pid}.tmp`;
writeFileSync(temporary, JSON.stringify(next, null, 2) + '\n');
renameSync(temporary, file);
return next;
}
/**
* What the shell tells the host about this start: STRATA_PORT_NOTICE is "busy:<wanted>:<used>" when the port it wanted was taken
* (set by the shell, never by the person). The page turns it into a sentence in the person's language.
*/
export function parsePortNotice(value) {
const m = /^busy:(\d{1,5}):(\d{1,5})$/.exec(String(value ?? ''));
return m ? { wanted: Number(m[1]), used: Number(m[2]) } : null;
}
/** The notification route of the shell's bridge (loopback, token). Null outside the desktop app. */
export function createShellNotifier({ env = process.env, fetcher = fetch } = {}) {
const base = env.STRATA_SHELL_BRIDGE, token = env.STRATA_SHELL_TOKEN;
if (!base || !token || !/^http:\/\/127\.0\.0\.1:\d+$/.test(base)) return null;
return {
async notify({ title, body }) {
const query = `title=${encodeURIComponent(String(title ?? '').slice(0, 120))}&body=${encodeURIComponent(String(body ?? '').slice(0, 300))}`;
const response = await fetcher(`${base}/shell/notify?${query}`, { method: 'POST', headers: { authorization: `Bearer ${token}` }, signal: AbortSignal.timeout(5000) });
let reply = null;
try { reply = await response.json(); } catch { /* no body */ }
return { shown: response.ok && reply?.shown === true };
},
};
}
+14 -3
View File
@@ -88,10 +88,21 @@ export let MODELS_DIR = MODELS_PICK.directory;
/** Native setup writes weights to data_dir/models. Keep its data root one level up. */
export function nativeDataDirectory(input) {
if(!input||process.env.STRATA_MODELS_DIR||!existsSync(DEFAULT_MODELS_DIRECTORY))return input;
if(!input||!existsSync(input))return input;
if(MODELS_DIR_NOTICE)return input;
const root=realpathSync(DEFAULT_MODELS_DIRECTORY);
if(resolve(input)!==resolve(DEFAULT_MODELS_DIRECTORY)&&realpathSync(input)!==root)return input;
const root=realpathSync(input),legacy=realpathSync(join(HOME,'models'));
// Retained legacy libraries are model folders too, not native data roots.
// A named data root satisfies the installer's guard; its alias leaves weights in place.
if(root===legacy){
const data=join(HOME,'Strata-data'),nativeModels=join(data,'models');
mkdirSync(data,{recursive:true});
// Unlike a fresh default, a retained library must not fall back to an empty folder if linking fails.
if(!existsSync(nativeModels))symlinkSync(process.platform==='win32'?root:relative(data,root),nativeModels,process.platform==='win32'?'junction':'dir');
const a=statSync(nativeModels),b=statSync(root);
if(realpathSync(nativeModels)!==root&&!(a.ino&&a.ino===b.ino&&a.dev===b.dev))throw new Error('The model folder conflicts with another library. Choose a different folder.');
return data;
}
if(process.env.STRATA_MODELS_DIR||!existsSync(DEFAULT_MODELS_DIRECTORY)||root!==realpathSync(DEFAULT_MODELS_DIRECTORY))return input;
const data=dirname(root);
// `models` and `Models` are one folder on Windows and macOS; where they are not (Linux) a link makes them one, and where no link can be made (exFAT, a share) models go in `models`.
alignModels(data,root);
+24
View File
@@ -0,0 +1,24 @@
// The host's own HTTPS downloads (engine releases from GitHub, web search, a cloud model's API) check the server against the operating
// system's certificates as well as Node's bundled list (#3722). Node carries its own list of roots; Chrome and curl.exe use the Windows
// store, which also holds the roots an antivirus's HTTPS scanning, a company proxy or an OS preview build adds. Nothing is turned off:
// the system roots are ADDED to the bundled ones, and a connection that still fails keeps failing.
//
// Node 24.5 and newer: tls.getCACertificates('system') and tls.setDefaultCACertificates. An older Node (the app bundles 24.x) does nothing here;
// the agent process gets NODE_USE_SYSTEM_CA=1 (harness-runtime.mjs) for the same reason.
import tlsModule from 'node:tls';
/** @returns {'system'|'bundled'} what the process trusts after the call */
export function useSystemCertificates(tls = tlsModule, env = process.env) {
if (env.STRATA_NO_SYSTEM_CA === '1') return 'bundled'; // a way back for a PC where reading the store misbehaves; not a way to skip checks
try {
if (typeof tls.getCACertificates !== 'function' || typeof tls.setDefaultCACertificates !== 'function') return 'bundled';
const system = tls.getCACertificates('system');
if (!Array.isArray(system) || !system.length) return 'bundled';
tls.setDefaultCACertificates([...new Set([...tls.getCACertificates('bundled'), ...system])]);
return 'system';
} catch {
return 'bundled';
}
}
export const trust = useSystemCertificates();
+38 -7
View File
@@ -1,5 +1,7 @@
/** Host counters. CPU sockets are measured only when their logical CPU IDs are known. */
import os from 'node:os';
import { spawn } from 'node:child_process';
import { createPowerShellLoop, endWithHost } from './windows-poll.mjs';
import { readFile, readdir } from 'node:fs/promises';
import { availableMemoryGb } from './hardware.mjs';
import { probe } from './util.mjs';
@@ -18,7 +20,7 @@ export function cpuUtilization(previous, current, ids) {
}
export function diskRate(previous, current, time) {
const elapsed=previous?(time-previous.time)/1000:0;
const elapsed=previous?((current?.at??time)-(previous.counters?.at??previous.time))/1000:0;
if(elapsed<=0||!current||!previous?.counters)return {readMbs:null,writeMbs:null};
const read=current.read-previous.counters.read,write=current.write-previous.counters.write;
return {readMbs:read>=0?read/1024**2/elapsed:null,writeMbs:write>=0?write/1024**2/elapsed:null};
@@ -41,6 +43,33 @@ export function parseMacDiskCounters(raw) {
return reads.length&&writes.length?{read:reads.reduce((n,m)=>n+Number(m[1]),0),write:writes.reduce((n,m)=>n+Number(m[1]),0)}:null;
}
export function parseWindowsDiskLine(line) {
const m=/^\s*(\d+)\s+(\d+)\s*$/.exec(String(line));
return m?{read:Number(m[1]),write:Number(m[2])}:null;
}
/** The disk counters, read by one long-lived PowerShell (windows-poll.mjs; #3939). */
export function createWindowsDiskReader({ spawnFn=spawn, intervalSec=4, maxStarts=3, now=Date.now }={}) {
let latest=null;
const script=`while($true){$d=Get-CimInstance Win32_PerfRawData_PerfDisk_PhysicalDisk -Filter "Name='_Total'";if($d){[Console]::Out.WriteLine(([string]$d.DiskReadBytesPersec)+' '+([string]$d.DiskWriteBytesPersec));[Console]::Out.Flush()};Start-Sleep -Seconds ${intervalSec}}`;
const loop=createPowerShellLoop({script,spawnFn,maxStarts,onLine:line=>{
const parsed=parseWindowsDiskLine(line);
if(parsed)latest={...parsed,at:now()};
return !!parsed;
}});
return {
read() { loop.start(); return latest&&now()-latest.at<5*intervalSec*1000?latest:null; },
stop() { loop.stop(); },
get starts() { return loop.starts; },
};
}
let windowsReader=null;
function windowsDisks() {
windowsReader??=endWithHost(createWindowsDiskReader());
return windowsReader.read();
}
async function disks() {
try {
if(process.platform==='linux') {
@@ -49,24 +78,26 @@ async function disks() {
}
if(process.platform==='darwin')return parseMacDiskCounters(await probe('ioreg',['-r','-c','IOBlockStorageDriver','-l'],1500));
if(process.platform==='win32') {
const raw=await probe('powershell.exe',['-NoProfile','-NonInteractive','-Command',"Get-CimInstance Win32_PerfRawData_PerfDisk_PhysicalDisk -Filter \"Name='_Total'\" | Select-Object DiskReadBytesPersec,DiskWriteBytesPersec | ConvertTo-Json -Compress"],2000);
const value=JSON.parse(raw);
if(value.DiskReadBytesPersec!=null&&value.DiskWriteBytesPersec!=null)return {read:Number(value.DiskReadBytesPersec),write:Number(value.DiskWriteBytesPersec)};
return windowsDisks();
}
} catch { /* An unsupported counter stays unavailable. */ }
return null;
}
export function createSystemSampler() {
let previousCpu=null,previousDisk=null;
let previousCpu=null,previousDisk=null,lastRate={readMbs:null,writeMbs:null};
return async processors => {
const current=os.cpus(),time=Date.now();
const cpuLoad=cpuUtilization(previousCpu,current,current.map((_,i)=>i));
const cpus=processors.map(p=>({id:p.id,utilizationPct:cpuUtilization(previousCpu,current,p.logicalIds??(processors.length===1?current.map((_,i)=>i):[]))}));
previousCpu=current;
const [available,counters]=await Promise.all([availableMemoryGb(process.platform),disks()]);
const rate=diskRate(previousDisk,counters,time);
previousDisk={time,counters};
let rate=lastRate;
if(!(counters&&counters.at!==undefined&&previousDisk?.counters?.at===counters.at)) {
rate=diskRate(previousDisk,counters,time);
previousDisk={time,counters};
lastRate=rate;
}
const total=os.totalmem()/1024**3,free=available??os.freemem()/1024**3;
return {cpuLoad,cpus,ramUsedGb:Math.max(0,Math.min(total,total-free)),ramTotalGb:total,diskReadGbs:rate.readMbs===null?null:rate.readMbs/1024,diskWriteMbs:rate.writeMbs};
};
+124
View File
@@ -0,0 +1,124 @@
/**
* CPU and SSD temperatures for the Monitor's Power card (#3628). A value that this PC cannot give stays null and its line is left out,
* so the card never shows a made-up number.
*
* Linux: the kernel's hwmon sensors (coretemp, k10temp, zenpower for the processor; nvme and drivetemp for the drives).
* Windows: the drive reports its own temperature to a query that needs no administrator (IOCTL_STORAGE_QUERY_PROPERTY with the device
* temperature property). Get-StorageReliabilityCounter is tried first because it is the documented route; on a normal account it is
* refused, then the query is used. One long-lived PowerShell asks every 30 s (windows-poll.mjs).
* There is no unprivileged, trustworthy processor temperature on Windows, so none is shown there.
* macOS: none.
*/
import { readFile, readdir } from 'node:fs/promises';
import { spawn } from 'node:child_process';
import { createPowerShellLoop, endWithHost } from './windows-poll.mjs';
const CPU_SENSORS = new Set(['coretemp', 'k10temp', 'zenpower', 'cpu_thermal', 'cpu-thermal']);
const DISK_SENSORS = new Set(['nvme', 'drivetemp']);
const plausible = c => Number.isFinite(c) && c > 0 && c < 150;
/** hwmon entries [{name, temps: [milli-degrees…]}] -> the hottest processor and drive sensor. */
export function pickTemperatures(entries) {
let cpu = null, ssd = null;
for (const { name, temps } of entries) {
const hottest = Math.max(...temps.map(t => t / 1000).filter(plausible), -Infinity);
if (!Number.isFinite(hottest)) continue;
if (CPU_SENSORS.has(name)) cpu = Math.max(cpu ?? -Infinity, hottest);
else if (DISK_SENSORS.has(name)) ssd = Math.max(ssd ?? -Infinity, hottest);
}
return { cpuTempC: cpu === null ? null : Math.round(cpu), ssdTempC: ssd === null ? null : Math.round(ssd) };
}
export async function readLinuxTemperatures({ base = '/sys/class/hwmon', read = readFile, list = readdir } = {}) {
const entries = [];
let folders = [];
try { folders = await list(base); } catch { return { cpuTempC: null, ssdTempC: null }; }
for (const folder of folders) {
try {
const name = String(await read(`${base}/${folder}/name`, 'utf8')).trim();
if (!CPU_SENSORS.has(name) && !DISK_SENSORS.has(name)) continue;
const files = (await list(`${base}/${folder}`)).filter(f => /^temp\d+_input$/.test(f));
const temps = [];
for (const file of files) temps.push(Number(String(await read(`${base}/${folder}/${file}`, 'utf8')).trim()));
entries.push({ name, temps });
} catch { /* a sensor that disappears is skipped */ }
}
return pickTemperatures(entries);
}
/** One line of the Windows loop: "ssd 53" or "ssd -". */
export function parseSsdLine(line) {
const m = /^\s*ssd\s+(-?\d+|-)\s*$/.exec(String(line));
if (!m) return undefined;
const value = m[1] === '-' ? NaN : Number(m[1]);
return plausible(value) ? value : null;
}
const WINDOWS_SCRIPT = String.raw`
$ErrorActionPreference='SilentlyContinue'
$cs=@'
using System;
using System.Runtime.InteropServices;
public static class StrataDiskTemp {
[DllImport("kernel32.dll", SetLastError=true, CharSet=CharSet.Unicode)] static extern IntPtr CreateFile(string n, uint a, uint s, IntPtr p, uint d, uint f, IntPtr t);
[DllImport("kernel32.dll", SetLastError=true)] static extern bool DeviceIoControl(IntPtr h, uint c, byte[] i, uint il, byte[] o, uint ol, out uint r, IntPtr ov);
[DllImport("kernel32.dll")] static extern bool CloseHandle(IntPtr h);
public static int Read(int n) {
IntPtr h = CreateFile("\\\\.\\PhysicalDrive" + n, 0, 3, IntPtr.Zero, 3, 0, IntPtr.Zero);
if (h == (IntPtr)(-1)) return -1;
try {
byte[] q = new byte[12]; q[0] = 0x34; byte[] o = new byte[512]; uint r;
if (!DeviceIoControl(h, 0x002D1400, q, 12, o, 512, out r, IntPtr.Zero)) return -1;
if (BitConverter.ToUInt16(o, 12) < 1) return -1;
return BitConverter.ToInt16(o, 26);
} finally { CloseHandle(h); }
}
}
'@
$compiled=$false
while($true){
$max=-1
$c=@(Get-PhysicalDisk | Get-StorageReliabilityCounter | Where-Object { $_.Temperature -gt 0 })
if($c.Count -gt 0){ $max=($c | Measure-Object Temperature -Maximum).Maximum }
else {
if(-not $compiled){ Add-Type -TypeDefinition $cs; $compiled=$true }
foreach($n in 0..15){ $v=[StrataDiskTemp]::Read($n); if($v -gt $max){ $max=$v } }
}
if($max -gt 0){ [Console]::Out.WriteLine('ssd '+$max) } else { [Console]::Out.WriteLine('ssd -') }
[Console]::Out.Flush()
Start-Sleep -Seconds 30
}`;
export function createWindowsSsdReader({ spawnFn = spawn, maxStarts = 2, now = Date.now, intervalSec = 30 } = {}) {
let latest = null;
const loop = createPowerShellLoop({
script: WINDOWS_SCRIPT, spawnFn, maxStarts,
onLine: line => {
const parsed = parseSsdLine(line);
if (parsed === undefined) return false;
latest = { value: parsed, at: now() };
return true;
},
});
return {
read() { loop.start(); return latest && now() - latest.at < 4 * intervalSec * 1000 ? latest.value : null; },
stop() { loop.stop(); },
get starts() { return loop.starts; },
};
}
let windowsReader = null;
let linuxCache = { at: 0, value: { cpuTempC: null, ssdTempC: null } };
/** The temperatures this PC can give now: { cpuTempC, ssdTempC } (null where unreadable). Cheap to call every few seconds. */
export async function readTemperatures({ platform = process.platform, now = Date.now } = {}) {
if (platform === 'linux') {
if (now() - linuxCache.at > 10_000) linuxCache = { at: now(), value: await readLinuxTemperatures() };
return linuxCache.value;
}
if (platform === 'win32') {
windowsReader ??= endWithHost(createWindowsSsdReader());
return { cpuTempC: null, ssdTempC: windowsReader.read() };
}
return { cpuTempC: null, ssdTempC: null };
}
+1 -1
View File
@@ -35,7 +35,7 @@ export function plainUpdateError(error, subject = 'engine') {
} else if (/another update is already running/i.test(raw)) {
message = 'An update is already running.';
next = `Wait a moment, then press ${button} in Settings.`;
} else if (/test start|did not answer a one-token|did not become ready/i.test(raw)) {
} else if (/test start|did not start correctly|did not answer a one-token|did not become ready/i.test(raw)) {
message = `The new ${subject} didn't start correctly, so nothing was changed. ${keep}`;
} else if (looksOffline(error) || looksOffline(raw)) {
message = 'No internet connection.';
+34 -3
View File
@@ -59,7 +59,7 @@ export async function checkComponentUpdates({ control, appUpdater = null, appVer
const message = /update server|aren't available|No internet/.test(plain.headline) ? plain.message : `Couldn't check for updates. Try again in a moment.`;
return { checkedAt, strata: { installedVersion: appVersion ?? control.version ?? null, status: 'error', message, details: plain.details, canInstall: false }, engine: { installedVersion: null, status: 'error', message, details: plain.details, canInstall: false } };
}
const mib = bytes => `${Math.round((bytes ?? 0) / (1 << 20))} MB`;
const mib = bytes => `${Math.round((bytes ?? 0) / 1_000_000)} MB`; // decimal megabytes, like the Terminal and the release page
const engine = report.engine ?? {};
const installedEngine = engine.installed?.version ?? null;
const offered = engine.offered?.version ?? null;
@@ -67,12 +67,14 @@ export async function checkComponentUpdates({ control, appUpdater = null, appVer
if (engine.status === 'available') Object.assign(engineView, { status: 'available', message: `A new engine is available (${mib(engine.download_bytes)}).` });
else if (engine.status === 'install') Object.assign(engineView, { status: 'available', message: `This PC needs an engine to run models. Press Update engine to download it (${mib(engine.download_bytes)}).` });
else if (engine.status === 'current' || engine.status === 'failed-before' || engine.status === 'skipped') Object.assign(engineView, { status: 'current', message: 'The engine is up to date.', details: engine.note });
else if (engine.status === 'wrong-make') Object.assign(engineView, { status: 'unavailable', message: engine.note ?? 'Nothing was changed: the update was for another make of graphics card.', details: (engine.reasons ?? []).join('\n') || undefined });
else if (engine.status === 'not-in-feed') Object.assign(engineView, { status: 'current', message: 'No engine update is listed right now.' });
else Object.assign(engineView, { status: 'unavailable', message: "There isn't an engine for this PC's graphics card yet.", details: (engine.reasons ?? []).join('\n') || undefined });
const control_ = report.control ?? {};
const app = report.app ?? {};
let appNew = app.status === 'available' ? app.offered : null;
let appNotes = app.notes ?? '';
let appCheckFailed = null; // the shell's own (Tauri) check threw: before this, the Settings check hid it and showed only Control's text
let appCannot = null; // browser mode: the app window's updater is not running, so the app itself cannot be updated from here
if (appUpdater) {
try {
@@ -80,12 +82,41 @@ export async function checkComponentUpdates({ control, appUpdater = null, appVer
appNew = verdict.available ? verdict.version : null;
appNotes = verdict.notes ?? appNotes;
} catch (error) {
if (error?.code === 'app-update-unavailable' && appUpdater.unavailable) appCannot = appUpdater.unavailable; // otherwise the feed's own view stands
if (error?.code === 'app-update-unavailable' && appUpdater.unavailable) appCannot = appUpdater.unavailable;
else appCheckFailed = String(error?.message ?? error ?? 'The app update check failed.'); // the app's own check did not work: say so, raw text behind Details
}
}
const controlNew = control_.status === 'available' ? control_.offered : null;
const strata = { installedVersion: appVersion ?? control_.installed ?? control.version ?? null, canInstall: !!((appNew && !appCannot) || controlNew) };
if (appNew || controlNew) Object.assign(strata, { status: 'available', latestVersion: appNew ?? controlNew, message: `A new version of Strata is available.${appNotes ? ` ${appNotes}` : ''}${appNew && appCannot ? ` ${appCannot}` : ''}` });
const appCheckLine = "Strata couldn't check for a new version of the app itself. Try again later.";
if (appNew || controlNew) Object.assign(strata, { status: 'available', latestVersion: appNew ?? controlNew, message: `A new version of Strata is available.${appNotes ? ` ${appNotes}` : ''}${appNew && appCannot ? ` ${appCannot}` : ''}${appCheckFailed ? ` ${appCheckLine}` : ''}` });
else if (appCheckFailed) Object.assign(strata, { status: 'error', message: appCheckLine });
else Object.assign(strata, { status: 'current', message: 'Strata is up to date.' });
if (appCheckFailed) strata.details = appCheckFailed;
return { checkedAt, feed: report.feed, strata, engine: engineView, serving: report.serving ?? [] };
}
/**
* What the first-run card shows: this PC's graphics card(s), the engine package the signed feed offers for it and how big it is.
* One feed read, nothing downloaded. `status`: offer (a package fits), none (nothing fits; `reasons` say why), offline (the feed could not be
* read), installed (an engine is already here), wrong-make (an engine for another make of card is installed).
*/
export async function engineFirstRun({ control, gpus = [] }) {
const cards = gpus.map(g => ({ name: g.name ?? 'Graphics card', vramGb: g.vramGb ?? null }));
let report;
try {
report = await control.updateCheck({});
} catch (error) {
const plain = plainUpdateError(error, 'Strata');
return { status: looksOffline(error) || /update server|No internet/.test(plain.headline ?? '') ? 'offline' : 'error', gpus: cards, message: plain.message, details: plain.details };
}
const engine = report.engine ?? {};
const hardware = report.hardware ?? {};
const base = { gpus: cards, driver: hardware.nvidia_driver ?? null, backend: hardware.backend ?? null };
if (engine.installed) return { ...base, status: 'installed', installedVersion: engine.installed.version ?? null };
if (engine.status === 'wrong-make') return { ...base, status: 'wrong-make', message: engine.note ?? null, reasons: engine.reasons ?? [] };
if (engine.offered && (engine.status === 'install' || engine.status === 'available')) {
return { ...base, status: 'offer', offer: { version: engine.offered.version, bytes: engine.download_bytes || engine.offered.size || 0, variant: engine.offered.variant ?? '', backend: engine.offered.backend ?? null } };
}
return { ...base, status: 'none', message: engine.note ?? null, reasons: engine.reasons ?? [] };
}
+90
View File
@@ -0,0 +1,90 @@
/**
* Windows graphics cards that nvidia-smi does not cover (AMD, Intel): their size and their load.
* - size: the DirectX adapter list in the registry (HKLM\SOFTWARE\Microsoft\DirectX) has a 64-bit DedicatedVideoMemory and the adapter
* LUID. Win32_VideoController.AdapterRAM is 32 bits and reads 0 or 4 GB for a 16 GB card (#3727, #3836).
* - load and memory in use: the Windows GPU performance counters (the ones Task Manager shows), read by one long-lived PowerShell and
* matched to the card by LUID. Temperature, power and the PCIe link need the vendor's own library, so they stay "not available".
*/
import { createPowerShellLoop, endWithHost } from './windows-poll.mjs';
import { utf8Script } from './powershell-utf8.mjs';
export const DX_ADAPTERS_SCRIPT = utf8Script(
"Get-ChildItem HKLM:\\SOFTWARE\\Microsoft\\DirectX | ForEach-Object { $p = Get-ItemProperty $_.PSPath; " +
"if ($p.Description -and $null -ne $p.AdapterLuid) { [pscustomobject]@{ Description = [string]$p.Description; VendorId = [string]$p.VendorId; " +
"Luid = [string]$p.AdapterLuid; Dedicated = [string]$p.DedicatedVideoMemory; Software = [string]$p.SoftwareAdapter } } } | ConvertTo-Json -Compress");
/** luid_0x<high>_0x<low> in the spelling of the performance counters, from the registry's 64-bit number. */
export function luidName(value) {
try {
const n = BigInt.asUintN(64, BigInt(String(value)));
const hex = v => '0x' + v.toString(16).padStart(8, '0');
return `luid_${hex(n >> 32n)}_${hex(n & 0xffffffffn)}`;
} catch { return null; }
}
/** The adapters the DirectX registry lists: [{ description, vendorId, luid, bytes, software }]. Never throws. */
export function parseDxAdapters(raw) {
try {
const parsed = JSON.parse(raw), rows = Array.isArray(parsed) ? parsed : [parsed];
return rows.filter(r => r?.Description).map(r => ({
description: String(r.Description), vendorId: Number(r.VendorId) || 0, luid: luidName(r.Luid),
bytes: Number(r.Dedicated) || 0, software: String(r.Software) === '1' || String(r.Software).toLowerCase() === 'true',
}));
} catch { return []; }
}
const norm = text => String(text ?? '').toLowerCase().replace(/[^a-z0-9]/g, '');
const VIRTUAL = /virtual|remote|indirect|mirror|basic render|meta |parsec|displaylink|idd/i;
/**
* Win32_VideoController rows + the DirectX adapter list -> the cards this app can show. A card keeps the size from the registry;
* a display driver for a virtual monitor (no size, no known maker) is not a graphics card and is dropped.
*/
export function windowsOtherGpus(videoRows, dx) {
const used = new Set();
const out = [];
for (const row of videoRows) {
if (!row?.Name) continue;
const match = dx.find((a, i) => !used.has(i) && !a.software && norm(a.description) === norm(row.Name));
if (match) used.add(dx.indexOf(match));
const vendor = /amd|radeon/i.test(row.Name) ? 'amd' : /intel/i.test(row.Name) ? 'intel' : 'unknown';
const vramGb = match?.bytes ? Math.round(match.bytes / 1024 ** 3 * 10) / 10 : 0;
if (vendor === 'unknown' && (!vramGb || VIRTUAL.test(row.Name))) continue;
out.push({ id: String(row.PNPDeviceID), vendor, name: row.Name, vramGb, luid: match?.luid ?? null });
}
return out;
}
/** One counter line: "G <luid> <load percent> <dedicated bytes in use>". */
export function parseGpuCounterLine(line) {
const m = /^G\s+(luid_0x[0-9a-f]+_0x[0-9a-f]+)\s+(\d+(?:\.\d+)?)\s+(\d+)\s*$/i.exec(String(line));
return m ? { luid: m[1].toLowerCase(), utilizationPct: Math.max(0, Math.min(100, Number(m[2]))), memoryUsedGb: Number(m[3]) / 1024 ** 3 } : null;
}
export function createWindowsGpuReader({ spawnFn, intervalSec = 4, maxStarts = 3, now = Date.now } = {}) {
const readings = new Map();
const script = utf8Script(
`while($true){` +
`$e=@(Get-CimInstance Win32_PerfFormattedData_GPUPerformanceCounters_GPUEngine);$m=@(Get-CimInstance Win32_PerfFormattedData_GPUPerformanceCounters_GPUAdapterMemory);` +
`$u=@{};foreach($r in $e){if($r.Name -match '(luid_0x[0-9A-Fa-f]+_0x[0-9A-Fa-f]+)_phys_\\d+_eng_\\d+_engtype_(\\w+)'){$k=$matches[1].ToLower()+'|'+$matches[2];$u[$k]=[double]$u[$k]+[double]$r.UtilizationPercentage}};` +
`$b=@{};foreach($k in $u.Keys){$l=$k.Split('|')[0];if(-not $b.ContainsKey($l) -or $u[$k] -gt $b[$l]){$b[$l]=$u[$k]}};` +
`foreach($r in $m){if($r.Name -match '(luid_0x[0-9A-Fa-f]+_0x[0-9A-Fa-f]+)'){$l=$matches[1].ToLower();[Console]::Out.WriteLine('G '+$l+' '+[math]::Round([double]$b[$l],0)+' '+$r.DedicatedUsage)}};` +
`[Console]::Out.Flush();Start-Sleep -Seconds ${intervalSec}}`);
const loop = createPowerShellLoop({ script, spawnFn, maxStarts, onLine: line => {
const parsed = parseGpuCounterLine(line);
if (parsed) readings.set(parsed.luid, { ...parsed, at: now() });
return !!parsed;
} });
return {
read(luid) { loop.start(); const r = readings.get(String(luid ?? '').toLowerCase()); return r && now() - r.at < 6 * intervalSec * 1000 ? r : null; },
stop() { loop.stop(); },
};
}
let shared = null;
/** The card's load and memory in use from the Windows counters; null when there is no reading. */
export function windowsGpuReading(luid) {
if (process.platform !== 'win32' || !luid) return null;
shared ??= endWithHost(createWindowsGpuReader());
return shared.read(luid);
}
+45
View File
@@ -0,0 +1,45 @@
/**
* One long-lived PowerShell that prints a line per reading, in a loop. The monitor used to start a cold PowerShell for every sample;
* its WMI start cost more than the probe timeout, so every query failed and WmiPrvSE burned a core (#3939).
* The loop gives up after `maxStarts` early exits in a row, so a counter that cannot be read stays quiet.
*/
import { spawn } from 'node:child_process';
export function createPowerShellLoop({ script, onLine, spawnFn = spawn, maxStarts = 3 }) {
let child = null, starts = 0, buffer = '';
function start() {
if (child || starts >= maxStarts) return;
starts++;
let proc;
try { proc = spawnFn('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', script], { windowsHide: true, stdio: ['ignore', 'pipe', 'ignore'] }); }
catch { return; }
child = proc;
buffer = '';
proc.on?.('error', () => { if (child === proc) child = null; });
proc.on?.('exit', () => { if (child === proc) child = null; });
proc.stdout?.on('data', chunk => {
buffer += chunk;
let i;
while ((i = buffer.indexOf('\n')) >= 0) {
const line = buffer.slice(0, i);
buffer = buffer.slice(i + 1);
if (onLine(line)) starts = 0;
}
});
proc.unref?.();
proc.stdout?.unref?.();
}
return {
start,
stop() { try { child?.kill(); } catch { /* gone */ } child = null; },
get starts() { return starts; },
};
}
let stoppers = null;
/** Every loop is ended with the host. */
export function endWithHost(loop) {
if (!stoppers) { stoppers = new Set(); process.once('exit', () => stoppers.forEach(l => l.stop())); }
stoppers.add(loop);
return loop;
}
+4 -2
View File
@@ -4,7 +4,7 @@ import { useId, useRef, useState } from 'react';
import { FloatingMenu } from './FloatingMenu';
import { Bar, Btn, Dialog, ErrorNotice, Icon } from './ui';
import { go, useLifecycle, loadVariant, selectRemoteModel, setSettingsSection, useStore } from '../lib/store';
import { pickerGroups, pickerVersionState } from '../lib/model-picker';
import { pickerGroups, pickerNativeModels, pickerVersionState } from '../lib/model-picker';
import { versionFit } from '../lib/model-versions';
import type { ModelVariant } from '../lib/types';
import { sizeGb } from '../lib/format';
@@ -37,6 +37,7 @@ function LegacyChatModelPicker() {
const running = library?.variants.find(v => v.id === library.loaded);
const version = library?.variants.find(v => v.id === selected);
const groups = library ? pickerGroups(library) : [];
const installedNative = pickerNativeModels(native?.installed?.models, native?.instances);
const status = version ? pickerVersionState(version, library?.loaded ?? null) : null;
const tooBig = !!version && library?.mode === 'real' && versionFit(version, hardware, contextTokens(engine?.beginner.contextLength ?? '32K')).zone === 'too-big';
async function apply() {
@@ -78,7 +79,8 @@ function LegacyChatModelPicker() {
<button ref={trigger} className="chat-model-badge chat-model-picker" aria-label={tr("Choose model: {0}", [apiModel?.name ?? (running?.local ? running.name : running?.family) ?? 'No model loaded'])} title={tr(apiModel ? `${connection?.name} · ${apiModel.id}` : 'View and switch models')} disabled={!library && !providers} aria-haspopup="menu" aria-expanded={open} onClick={() => setOpen(v => !v)}><Icon name={apiModel ? 'link' : modelFamilyIcon(running?.family ?? groups[0]?.family)} size={16}/><span>{tr(apiModel?.name ?? (running?.local ? running.name : running?.family) ?? groups[0]?.family ?? 'No model loaded')}</span><Icon name="chevron" size={12}/></button>
{tr(open ? <FloatingMenu anchor={trigger.current} title={tr("Models")} width={380} onClose={() => setOpen(false)}>
{tr(native?.instances.filter(i => i.status !== 'error').map(i => <button key={i.id} className="menu__item" role="menuitem" disabled={streaming || i.status !== 'ready'} onClick={() => void nativeAction(`instances/${i.id}/default`).then(() => { selectRemoteModel(null); setOpen(false); }).catch(e => setError(e.message))}><Icon name="models"/><span>{tr(i.label)}<small>{tr(i.gpus.map(g => `GPU ${g + 1}`).join(" + ") || "Automatic GPU")}</small></span></button>))}<p className="model-picker-heading">{tr("Local models")}</p>
{tr(!groups.length ? <p className="model-picker-empty">{tr("No local models downloaded yet.")}</p> : null)}
{tr(installedNative.map(m => <button key={m.config || m.model} className="menu__item model-picker-option model-picker-api" role="menuitem" disabled={streaming} onClick={() => { setOpen(false); go('models'); }}><span><strong>{tr(m.model_name || m.model)}</strong><small>{tr("Downloaded, not loaded. Open Models to load it.")}</small></span></button>))}
{tr(!groups.length && !installedNative.length ? <p className="model-picker-empty">{tr("No local models downloaded yet.")}</p> : null)}
{tr(groups.map(({ family, versions }, index) => {
const main = versions[0], state = pickerVersionState(main, library!.loaded);
const others = versions.slice(1), isExpanded = expanded.includes(family), id = `${groupsId}-${index}`;
@@ -5,10 +5,11 @@ import { FloatingMenu } from './FloatingMenu';
import { DictationControl } from './DictationControl';
import { Btn, Dialog, Icon, ErrorNotice } from './ui';
import { MOD } from '../lib/commands';
import { setChatPreferences, useStore } from '../lib/store';
import { pushToast, setChatPreferences, useStore } from '../lib/store';
import { artifactFromUrl, type PreviewArtifact } from '../lib/artifacts';
import type { ChatMode, Session } from '../lib/types';
import { attachChatPhoto } from '../lib/chat-photos';
import { uploadChatFile, FILE_ACCEPT } from '../lib/chat-attach-file';
import { useNative } from '../lib/native';
import { pictureStatus } from '../lib/pictures';
const modes: {
@@ -91,6 +92,12 @@ export function ComposerControls({ session, onPreview, onAttach, onError, onSlas
await attachChatPhoto(file, session?.id);
continue;
}
if (!directory && session?.id && /\.pdf$/i.test(file.name) && file.size <= 12 * 1024 * 1024) {
// A PDF goes to the chat's folder and is named in the next message; "Open a file" still previews it.
const saved = await uploadChatFile(file, session.id);
pushToast('success', 'PDF added', `${saved.name} is in this chat. Ask about it in your next message.`);
continue;
}
if (!directory && (photoTypes.test(file.name) || /\.pdf$/i.test(file.name))) {
await openFile(file);
if (photoTypes.test(file.name))
@@ -157,7 +164,7 @@ export function ComposerControls({ session, onPreview, onAttach, onError, onSlas
return <>
{tr(!remoteModel && nativeModel?.metrics?.mcp?.servers?.length ? <button className="composer-choice" aria-label={tr("Use connected MCP tools")} aria-pressed={session?.preferences?.useTools === true} onClick={() => session?.preferences?.useTools ? setChatPreferences(session.id, { useTools: false }) : setToolsOpen(true)}><Icon name="skills" size={16}/>{tr("Tools")}{tr(session?.preferences?.useTools ? ' on' : '')}</button> : null)}
<Dialog open={toolsOpen} compact title={tr("Enable connected tools?")} onClose={() => setToolsOpen(false)} actions={<><Btn onClick={() => setToolsOpen(false)}>{tr("Keep tools off")}</Btn><Btn variant="primary" onClick={() => { setChatPreferences(session?.id, { useTools: true, mode: 'bypass' }); setToolsOpen(false); }}>{tr("Enable tools")}</Btn></>}><p>{tr("Connected servers can act on the resources you configured. Calls run automatically when the model chooses them, and their results appear in chat.")}</p><p className="small muted">{tr("This switches this conversation to Bypass permissions. Servers: ")}{tr((nativeModel?.metrics?.mcp?.servers ?? []).map((s: any) => s.name).join(', '))}.</p></Dialog>
<input ref={files} type="file" multiple hidden accept="image/*,.pdf,.txt,.md,.markdown,.csv,.ts,.tsx,.js,.jsx,.py,.json,.css,.rs,.go,.sh,.html,.htm,.yaml,.yml,.toml,.xml,.log" onChange={e => { void addFiles(Array.from(e.target.files ?? [])); e.target.value = ''; }}/>
<input ref={files} type="file" multiple hidden accept={FILE_ACCEPT} onChange={e => { void addFiles(Array.from(e.target.files ?? [])); e.target.value = ''; }}/>
<input ref={folder} type="file" multiple hidden onChange={e => { void addFiles(Array.from(e.target.files ?? []), true); e.target.value = ''; }}/>
<input ref={previewFile} type="file" hidden onChange={e => { void openFile(e.target.files?.[0]); e.target.value = ''; }}/>
<button ref={trigger} className="composer-add" aria-label={tr("Add or open")} title={tr("Add or open")} disabled={reading || streaming} aria-haspopup="menu" aria-expanded={menu} onClick={() => { setPreferences(null); setMenu(v => !v); }}><Icon name="plus" size={22}/></button>
+28
View File
@@ -0,0 +1,28 @@
import { useEffect, useState } from 'react';
import { parseDecimal, partialDecimal } from '../lib/decimal';
/** A number field that lets the person type a decimal point: it keeps the text while it is typed and tells the page the number once it is one (#3736). */
export function DecimalInput({ value, onChange, ariaLabel, className = 'st-input', placeholder, disabled }: {
value: number | null | undefined;
onChange: (value: number | null) => void;
ariaLabel: string;
className?: string;
placeholder?: string;
disabled?: boolean;
}) {
const [text, setText] = useState(value === null || value === undefined ? '' : String(value));
// A change that came from elsewhere (Restore saved, another tab) replaces the text, but never while the typed text already means this number.
useEffect(() => {
const shown = text === '' ? null : parseDecimal(text);
if ((value ?? null) !== shown) setText(value === null || value === undefined ? '' : String(value));
}, [value]);
return <input className={className} aria-label={ariaLabel} inputMode="decimal" disabled={disabled} placeholder={placeholder} value={text}
onChange={e => {
const next = e.target.value;
if (!partialDecimal(next)) return;
setText(next);
if (next.trim() === '') onChange(null);
else { const n = parseDecimal(next); if (n !== null) onChange(n); }
}}
onBlur={() => { const n = text.trim() === '' ? null : parseDecimal(text); setText(n === null ? '' : String(n)); }}/>;
}
+1 -10
View File
@@ -6,16 +6,7 @@ import { Icon } from './ui';
/** Files dropped on the chat go where the attach button would put them: the composer reads the 'strata:drop-files' event. */
export const DROP_EVENT = 'strata:drop-files';
/** The pictures on the clipboard, named so each one is told apart in the composer (a pasted screenshot arrives as "image.png"). */
export function pastedFiles(data: DataTransfer | null | undefined): File[] {
const out: File[] = [];
for (const file of Array.from(data?.files ?? [])) {
if (!file.type.startsWith('image/')) continue;
const extension = file.type.split('/')[1]?.replace('jpeg', 'jpg').replace(/[+].*/, '') || 'png';
out.push(new File([file], `Pasted picture ${new Date().toLocaleTimeString().replace(/[^0-9]+/g, '-').replace(/-$/, '')}${out.length ? `-${out.length + 1}` : ''}.${extension}`, { type: file.type }));
}
return out;
}
export { pastedFiles } from '../lib/pasted-files';
const hasFiles = (e: React.DragEvent) => Array.from(e.dataTransfer?.types ?? []).includes('Files');
/** A file dropped outside the chat must not take the window to that file: the page ignores it (called once at start). */
@@ -0,0 +1,29 @@
import { useEffect, useMemo, useState } from 'react';
import { currencyChoices, currencyLabel } from '../lib/currency';
import { tr } from '../lib/i18n';
import { useMoney } from '../lib/money';
import { setElectricity, useStore } from '../lib/store';
import { Btn, Select } from './ui';
/**
* The price of one kWh for the Monitor's cost line (#3628). The person types only the number: the currency is the one of the computer's
* region, shown beside it. "Change currency" is a way to pick another one, never a step that has to be taken.
*/
export function ElectricityPrice() {
const price = useStore(s => s.electricityPrice), override = useStore(s => s.electricityCurrency);
const money = useMoney();
const [text, setText] = useState(price > 0 ? String(price) : ''), [picking, setPicking] = useState(false);
useEffect(() => { setText(price > 0 ? String(price) : ''); }, [price]);
const commit = () => setElectricity(Number(text.replace(',', '.')), override);
const choices = useMemo(() => currencyChoices(money.locale), [money.locale]);
return <span className="stack">
<span className="row">
<input className="st-input" inputMode="decimal" size={7} aria-label={tr("Price per kWh")} placeholder="0.30" value={text} onChange={e => setText(e.target.value)} onBlur={commit} onKeyDown={e => { if (e.key === 'Enter') commit(); }}/>
<span className="small muted">{money.currency ? tr("per kWh, in {0}", [currencyLabel(money.currency, money.locale)]) : tr("per kWh")}</span>
<Btn size="sm" onClick={() => setPicking(v => !v)}>{tr("Change currency")}</Btn>
</span>
{picking ? <Select ariaLabel={tr("Currency")} translateLabels={false} value={override}
options={[{ value: '', label: tr("Automatic ({0})", [money.detected ?? '—']) }, ...choices]}
onChange={value => { setElectricity(price, String(value)); setPicking(false); }}/> : null}
</span>;
}
+173
View File
@@ -0,0 +1,173 @@
import { tr } from '../lib/i18n';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Brand } from './Brand';
import { Bar, Btn, Icon } from './ui';
import { api } from '../lib/client';
import { useDownloadEta } from '../lib/download-eta';
import type { NativeJob } from '../lib/native';
import { megabytes } from '../lib/format';
/** What the host says about this PC's engine (GET /api/engine/first-run). */
export interface FirstRunInfo {
status: 'offer' | 'none' | 'offline' | 'error' | 'installed' | 'wrong-make' | 'unavailable';
gpus: { name: string; vramGb: number | null }[];
driver?: string | null;
offer?: { version: string; bytes: number; variant: string; backend: string | null };
message?: string | null;
reasons?: string[];
details?: string;
freeBytes?: number;
}
export const mb = megabytes;
/**
* Whether this PC still needs its engine. Reads only Control's local answer (no network): the engine is missing and the engine that came
* with the app is not being unpacked. `null` while that is not known yet.
*/
export function useEngineNeeded(): boolean | null {
const [needed, setNeeded] = useState<boolean | null>(null);
useEffect(() => {
let live = true;
const ask = () => api.get<{ engine?: { usable?: boolean } | null; engineSetup?: boolean }>('/api/control/status').then(s => {
if (!live) return;
if (s.engineSetup) return setNeeded(null);
setNeeded(!!s.engine && s.engine.usable === false);
}, () => { if (live) setNeeded(false); });
void ask();
const timer = setInterval(() => void ask(), 3000);
return () => { live = false; clearInterval(timer); };
}, []);
return needed;
}
function gpuLine(info: FirstRunInfo) {
// a processor's own tiny graphics (no memory of its own) is not "a card"
const first = info.gpus.find(g => (g.vramGb ?? 0) >= 1) ?? info.gpus[0];
if (!first) return '';
const vram = first.vramGb ? `, ${Math.round(first.vramGb)} GB` : '';
const driver = info.driver ? `, driver ${info.driver}` : '';
return `${first.name}${vram}${driver}`;
}
/**
* The first-run card: the engine this PC needs, one button to download it, progress, and the offline way (choose a downloaded file).
* Nothing is downloaded before a button is pressed.
*/
export function EngineFirstRun({ onDone }: { onDone: () => void }) {
const [info, setInfo] = useState<FirstRunInfo | null>(null);
const [job, setJob] = useState<NativeJob | null>(null);
const [busy, setBusy] = useState(false), [problem, setProblem] = useState('');
const [finished, setFinished] = useState(false);
const file = useRef<HTMLInputElement>(null);
const started = useRef(false);
const look = useCallback(() => {
setInfo(null);
setProblem('');
api.get<FirstRunInfo>('/api/engine/first-run').then(setInfo, e => setInfo({ status: 'error', gpus: [], message: e instanceof Error ? e.message : String(e) }));
}, []);
useEffect(() => { look(); }, [look]);
// follow the download / install job
useEffect(() => {
const timer = setInterval(() => {
if (!started.current) return;
api.get<{ job: NativeJob | null }>('/api/engine/job').then(r => {
setJob(r.job);
if (r.job?.state === 'done') setFinished(true);
if (r.job && r.job.state !== 'running') started.current = false;
}, () => { });
}, 700);
return () => clearInterval(timer);
}, []);
async function start() {
setBusy(true);
setProblem('');
try {
started.current = true;
setJob(await api.post<NativeJob>('/api/native/operations', { kind: 'update-engine', args: {} }));
}
catch (e) {
started.current = false;
setProblem(e instanceof Error ? e.message : String(e));
}
finally {
setBusy(false);
}
}
async function chosen(files: FileList | null) {
const picked = files?.[0];
if (!picked) return;
setBusy(true);
setProblem('');
try {
const response = await fetch('/api/engine/from-file', { method: 'POST', headers: { 'x-file-name': picked.name }, body: picked });
const body = await response.json().catch(() => ({}));
if (!response.ok) throw new Error(body?.error ?? `The file could not be used (${response.status}).`);
started.current = true;
setJob(body as NativeJob);
}
catch (e) {
setProblem(e instanceof Error ? e.message : String(e));
}
finally {
setBusy(false);
if (file.current) file.current.value = '';
}
}
const running = job?.state === 'running';
const failed = !!job && (job.state === 'error' || job.state === 'cancelled');
const percent = Number(job?.result?.install?.download?.percent ?? 0);
const bytes = info?.offer?.bytes ?? 0;
const doneMb = Math.round(mb(bytes) * percent / 100);
const eta = useDownloadEta(running && bytes ? { done_gb: bytes * percent / 100 / 1e9, remaining_gb: bytes * (100 - percent) / 100 / 1e9 } : undefined, job?.id);
const cards = (info?.gpus ?? []).filter(g => (g.vramGb ?? 0) >= 1).length;
const hidden = <input ref={file} type="file" accept=".gz,.tgz,.xz,.zip" hidden onChange={e => void chosen(e.target.files)}/>;
const pick = (label: string) => <button type="button" className="onboard-continue-anyway" disabled={busy || running} onClick={() => file.current?.click()}>{tr(label)}</button>;
let body;
if (finished) {
body = <><div className="onboard-welcome-copy"><h1 tabIndex={-1} data-initial-focus>{tr("The engine is ready.")}</h1><p>{tr("Next: pick a model.")}</p></div>
<footer className="onboard-footer onboard-footer--welcome"><Btn variant="primary" onClick={onDone}>{tr("Continue")}<Icon name="chevron" size={16}/></Btn></footer></>;
}
else if (running) {
body = <><div className="onboard-welcome-copy"><h1 tabIndex={-1} data-initial-focus>{tr("Downloading the engine")}</h1>
<p>{bytes && job?.result?.install?.download ? `${doneMb} of ${mb(bytes)} MB${eta ? ' · ' + eta : ''}` : tr(job?.summary ?? 'Starting…')}</p></div>
<div className="onboard-error" style={{ maxWidth: 520, margin: '0 auto', width: '100%' }}><Bar value={percent} label="Downloading the engine"/><p className="small muted" style={{ textAlign: 'center' }}>{tr("You can close Strata; it continues where it stopped.")}</p></div></>;
}
else if (!info) {
body = <div className="onboard-welcome-copy"><h1 tabIndex={-1} data-initial-focus>{tr("Welcome to Strata")}</h1><p role="status">{tr("Looking at your graphics card…")}</p></div>;
}
else {
const card = gpuLine(info);
const found = card ? <p>{tr("Strata found your graphics card:")}<br/><strong>{card}</strong>{cards > 1 ? <><br/><span className="small muted">{tr("{0} cards found. Strata uses the first one for the engine; change it later in Settings.", [cards])}</span></> : null}</p>
: <p>{tr("Strata did not recognise a supported graphics card.")}</p>;
let main, actions;
if (info.status === 'offer' && info.offer) {
main = <>{found}<p>{tr("To run models on it, Strata needs one download:")}<br/><strong>{tr("The engine for this card")}: {mb(info.offer.bytes)} MB</strong></p>
{info.freeBytes !== undefined && info.freeBytes < info.offer.bytes * 2.5 ? <p className="small inline-error">{tr("The engine needs about {0} GB of free space on this drive and only {1} GB is free. Free some space, then continue.", [Math.ceil(info.offer.bytes * 2.5 / 1e9), (info.freeBytes / 1e9).toFixed(1)])}</p> : null}
<p className="small muted">{tr("It comes from Strata’s own update server and is checked before it is used. Your models and chats are never touched by it.")}</p></>;
actions = <><Btn variant="primary" disabled={busy} onClick={() => void start()}>{tr(failed ? 'Continue' : 'Download the engine')}</Btn>{pick("I already have the file…")}</>;
}
else if (info.status === 'offline') {
main = <>{found}<p>{tr("Strata could not reach github.com. Check your connection and try again, or download the engine file on another PC and choose it here.")}</p></>;
actions = <><Btn variant="primary" disabled={busy} onClick={look}>{tr("Try again")}</Btn>{pick("Choose a downloaded file…")}</>;
}
else if (info.status === 'wrong-make') {
main = <>{found}<p>{info.message ?? tr("The engine installed here is for another make of graphics card.")}</p></>;
actions = <><Btn variant="primary" disabled={busy} onClick={look}>{tr("Try again")}</Btn>{pick("Choose a downloaded file…")}</>;
}
else {
main = <>{found}<p>{tr("Strata can’t pick an engine for this PC from its update list.")}</p>{info.message ? <p className="small muted">{info.message}</p> : null}{(info.reasons ?? []).length ? <ul className="small muted" style={{ textAlign: 'left', display: 'inline-block' }}>{(info.reasons ?? []).map(r => <li key={r}>{r}</li>)}</ul> : null}</>;
actions = <><Btn variant="primary" disabled={busy} onClick={look}>{tr("Try again")}</Btn>{pick("Choose a downloaded file…")}<button type="button" className="onboard-continue-anyway" onClick={onDone}>{tr("Continue without an engine")}</button></>;
}
body = <><div className="onboard-welcome-copy"><h1 tabIndex={-1} data-initial-focus>{tr("Welcome to Strata")}</h1>{main}</div>
{failed ? <div className="onboard-error" style={{ maxWidth: 560, margin: '0 auto' }}><p role="alert" className="small inline-error">{job?.error ?? job?.summary}</p><p className="small muted">{tr("The download stopped. Continue picks it up where it stopped.")}</p></div> : null}
{problem ? <div className="onboard-error" style={{ maxWidth: 560, margin: '0 auto' }}><p role="alert" className="small inline-error">{problem}</p></div> : null}
<footer className="onboard-footer onboard-footer--welcome">{actions}</footer></>;
}
return <div className="setup guided-onboarding" data-step="engine">{hidden}<header className="onboard-header"><Brand variant="lockup" height={22}/></header>{body}</div>;
}
+14 -3
View File
@@ -22,6 +22,15 @@ type Choices = NonNullable<NonNullable<SetupState['choices']>['onboarding']>;
function radioKeys(event: KeyboardEvent<HTMLDivElement>) { if (!['ArrowLeft', 'ArrowRight', 'ArrowUp', 'ArrowDown', 'Home', 'End'].includes(event.key))
return; event.preventDefault(); const items = Array.from(event.currentTarget.querySelectorAll<HTMLButtonElement>('[role=radio]')); const current = items.indexOf(event.target as HTMLButtonElement); const next = event.key === 'Home' ? 0 : event.key === 'End' ? items.length - 1 : (current + (['ArrowLeft', 'ArrowUp'].includes(event.key) ? -1 : 1) + items.length) % items.length; items[next]?.focus(); items[next]?.click(); }
const steps = ['Welcome', 'Your model', 'Prepare', 'Make it yours'];
/** What a person is told before choosing DeepSeek: the size, where it runs, how fast, what it needs. Every line is a warning or a fact; nothing here stops the download. */
function DeepSeekNote({ version, plan }: { version: NonNullable<ReturnType<typeof nativeCatalogModels>[number]['sizes'][number]>; plan: Record<string, any> | null }) {
void plan; // the plan's own disk line repeats what the version says in on_this_pc
return <div className="engine-warning" role="status"><Icon name="info" size={18}/><div>
<p>{tr("This is a very large model. The download is {0} GB and it needs about as much free disk space. It runs on NVIDIA RTX 30, 40 and 50 cards, on a Radeon AI PRO R9700 (Linux) or on a Ryzen AI Max / Strix Halo PC (Windows); how fast it is depends on the memory and card (the line below says what to expect on this computer). What does not fit in memory is read from your SSD, so an NVMe drive matters.", [Math.round(version.download_gb)])}</p>
{version.on_this_pc ? <p className="small muted">{version.on_this_pc}</p> : null}
<p className="small muted">{tr("You can stop the download and continue later; it picks up where it stopped. Strata warns you here but never blocks you.")}</p>
</div></div>;
}
export function GuidedSetup() {
const native = useNative(), setup = useStore(s => s.setup), hardware = useStore(s => s.hardware);
const families = nativeCatalogModels(native?.catalog?.families ?? []);
@@ -132,7 +141,8 @@ export function GuidedSetup() {
setTuneJobId(tuneJob.id);
void persist(4, { tuneJobId: tuneJob.id }).catch(e => setError(e.message));
} }, [step, installJob?.id, tuneJob?.id]);
const values = { guided: true, family: version?.sourceFamily, model: version?.model, context, vision: version?.images ? 'yes' : 'no', kv: 'int8', low_ram: 'auto', loadAfter: false, multiple: false, ...(directory ? { data_dir: directory } : {}) };
const fixedContext = version?.multi_file ? 8192 : null; // the one context DeepSeek's engine was checked at
const values = { guided: true, family: version?.sourceFamily, model: version?.model, context: fixedContext ?? context, vision: version?.images ? 'yes' : 'no', kv: 'int8', low_ram: 'auto', loadAfter: false, multiple: false, ...(directory ? { data_dir: directory } : {}) };
async function review() {
await perform(async () => {
setInstallJobId('');
@@ -154,7 +164,7 @@ export function GuidedSetup() {
}
async function tune() {
await perform(async () => {
const result = await nativeAction<NativeJob>('operations', { kind: 'tune', args: { guided: true, model: version?.id, profile, calibrate, vision: version?.images && vision ? 'yes' : 'no', multiple: false } });
const result = await nativeAction<NativeJob>('operations', { kind: 'tune', args: { guided: true, model: version?.id, profile, calibrate: calibrate && !version?.multi_file, vision: version?.images && vision ? 'yes' : 'no', multiple: false } });
setTuneJobId(result.id);
await persist(4, { tuneJobId: result.id });
});
@@ -201,9 +211,10 @@ export function GuidedSetup() {
<div className="onboard-stage-heading"><h1 ref={titleRef} tabIndex={-1} data-initial-focus>{tr(title)}</h1>{tr(step === 2 ? <p>{tr("You can always change your models later.")}</p> : step === 4 && tuneJob?.state === 'running' ? <p>{tr("We\u2019re measuring, applying your preference, and getting ready to chat.")}</p> : null)}</div>
{tr(step === 2 ? <div className="onboard-model-stage">
{tr(selectedFamily ? <>
<div className="onboard-model-top"><div className="onboard-model-picker"><span className="onboard-control-label">{tr("Model")}</span><div><span className="onboard-model-symbol"><Icon name="models" size={23}/></span><Select ariaLabel={tr("Setup model")} value={selectedFamily.family} options={families.map(f => ({ value: f.family, label: f.title }))} onChange={value => chooseNative(value, '')}/></div></div><div className="onboard-context"><span className="onboard-control-label">{tr("Conversation context")}</span><Select ariaLabel={tr("Setup conversation context")} value={String(context)} options={(native?.catalog?.contexts ?? [32768]).map(n => ({ value: String(n), label: `${n / 1024}K` }))} onChange={value => setContext(Number(value))}/></div></div>
<div className="onboard-model-top"><div className="onboard-model-picker"><span className="onboard-control-label">{tr("Model")}</span><div><span className="onboard-model-symbol"><Icon name="models" size={23}/></span><Select ariaLabel={tr("Setup model")} value={selectedFamily.family} options={families.map(f => ({ value: f.family, label: f.title }))} onChange={value => chooseNative(value, '')}/></div></div><div className="onboard-context"><span className="onboard-control-label">{tr("Conversation context")}</span><Select ariaLabel={tr("Setup conversation context")} value={String(version?.multi_file ? 8192 : context)} options={(version?.multi_file ? [8192] : native?.catalog?.contexts ?? [32768]).map(n => ({ value: String(n), label: `${n / 1024}K` }))} onChange={value => setContext(Number(value))}/></div></div>
<OnboardingMemory hardware={hardware} native={native?.hardware}/>{tr(!version || version.needsNewerEngine ? <p className="small muted">{tr("This model needs a newer engine. Update your engine or import a local GGUF for this model.")}</p> : null)}
<div className="onboard-version-heading"><strong>{tr("Choose a version")}</strong><span>{tr(version ? `${catalogVersionLabel(version)} selected` : '')}</span></div>
{tr(version?.multi_file ? <DeepSeekNote version={version} plan={plan}/> : null)}
<div className="onboard-versions" role="radiogroup" aria-label={tr("Model version")} onKeyDown={radioKeys}>{tr(cards.map(v => versionCard(v)))}</div>
{tr(others.length ? <div className="onboard-more-versions"><VersionPicker versions={others} selectedId={version?.id ?? ''} hardware={hardware ? { ram_gb: native?.hardware?.ram_gb ?? hardware.memory.totalGb, vram_gb: Math.max(0, ...(native?.hardware?.gpus ?? []).filter(g => g.usable !== false).map(g => g.vram_gb)) } : null} downloaded={readyIds} showRecommended={false} moreLabel="More versions" onSelect={id => { setSize(id); setPlan(null); }} onOwnGguf={() => void ownModel()}/></div> : null)}
<div className="onboard-folder"><Icon name="folder" size={20}/><div><strong>{tr(directory ? 'Your model folder' : 'Already downloaded the model?')}</strong><small>{tr(directory || "Choose the folder and we'll start your models with Strata.")}</small></div><Btn size="sm" onClick={() => setFolderOpen(v => !v)}>{tr(folderOpen ? 'Close' : directory ? 'Change' : 'Choose folder')}</Btn></div>
+52 -11
View File
@@ -1,12 +1,26 @@
import { tr } from '../lib/i18n';
import { copyText } from '../lib/clipboard';
/** Safe, dependency-free rendering of the Markdown structures used in answers. */
import { Fragment, type ReactNode } from 'react';
import { Btn, IconBtn } from './ui';
import { SyntaxCode } from './SyntaxCode';
import { MathView } from './MathView';
import { displayBlock, hasMath, splitMath } from '../lib/math';
function inlineMarks(text: string, onOpenLink?: (url: string) => void): ReactNode {
return text.split(/(`[^`]+`|\*\*[^*]+\*\*|\*[^*\n]+\*|\[[^\]]+\]\([^\s)]+\))/g).map((part, i) => {
import { imagePlan } from '../lib/markdown-images';
function inlineMarks(text: string, onOpenLink?: (url: string) => void, imageSession?: string | null): ReactNode {
return text.split(/(!\[[^\]]*\]\([^\s)]+\)|`[^`]+`|\*\*[^*]+\*\*|\*[^*\n]+\*|\[[^\]]+\]\([^\s)]+\))/g).map((part, i) => {
const image = part.match(/^!\[([^\]]*)\]\(([^\s)]+)\)$/);
if (image) {
const plan = imagePlan(image[2], imageSession);
if (plan.kind === 'img')
return <img key={i} src={plan.src} alt={image[1]} className="md-image" loading="lazy" referrerPolicy="no-referrer"/>;
if (plan.kind === 'link')
return <a key={i} href={plan.href} target="_blank" rel="noreferrer" onClick={e => { if (onOpenLink) {
e.preventDefault();
onOpenLink(plan.href);
} }}>{image[1] || plan.href}</a>;
return <Fragment key={i}>{image[1] || part}</Fragment>;
}
if (part.startsWith('`') && part.endsWith('`'))
return <code key={i} className="md-code">{part.slice(1, -1)}</code>;
if (part.startsWith('**') && part.endsWith('**'))
@@ -23,21 +37,37 @@ function inlineMarks(text: string, onOpenLink?: (url: string) => void): ReactNod
});
}
/** Text with its inline marks and, outside code spans, its math (#3268). Math is cut out first: a * or _ inside a formula is not emphasis. */
function inline(text: string, onOpenLink?: (url: string) => void): ReactNode {
function inline(text: string, onOpenLink?: (url: string) => void, imageSession?: string | null): ReactNode {
if (!hasMath(text))
return inlineMarks(text, onOpenLink);
return inlineMarks(text, onOpenLink, imageSession);
return text.split(/(`[^`]+`)/g).map((chunk, c) => {
if (chunk.length > 1 && chunk.startsWith('`') && chunk.endsWith('`'))
return <Fragment key={c}>{inlineMarks(chunk, onOpenLink)}</Fragment>;
return <Fragment key={c}>{inlineMarks(chunk, onOpenLink, imageSession)}</Fragment>;
return <Fragment key={c}>{splitMath(chunk).map((part, p) => part.kind === 'math'
? <MathView key={p} tex={part.tex} display={part.display}/>
: <Fragment key={p}>{inlineMarks(part.text, onOpenLink)}</Fragment>)}</Fragment>;
: <Fragment key={p}>{inlineMarks(part.text, onOpenLink, imageSession)}</Fragment>)}</Fragment>;
});
}
export function Markdown({ text, onOpenLink, onPreviewHtml }: {
/** Does this line continue the paragraph above it, or does it start a block of its own? */
export function continuesParagraph(lines: string[], at: number): boolean {
const line = lines[at];
if (!line.trim() || /^\s*```/.test(line) || /^(#{1,6})\s+(.+)$/.test(line) || /^\s*[-*+]\s+\S/.test(line) || /^\s*\d+[.)]\s+\S/.test(line) || /^>\s?/.test(line))
return false;
if (line.includes('|') && at + 1 < lines.length && /^\s*\|?\s*:?-{3,}/.test(lines[at + 1]))
return false;
return !displayBlock(lines, at);
}
/** Copy a code block. The toast store needs a browser, so it is loaded here, when someone copies: the renderer itself must render without one. */
async function copyCode(text: string) {
const { pushToast } = await import('../lib/store');
await copyText(text, 'Code', { toast: pushToast });
}
export function Markdown({ text, onOpenLink, onPreviewHtml, imageSession }: {
text: string;
onOpenLink?: (url: string) => void;
onPreviewHtml?: (html: string) => void;
/** The chat whose folder a picture path in the answer is read from (the host serves only files inside it). */
imageSession?: string | null;
}) {
const nodes: ReactNode[] = [];
const lines = text.split('\n');
@@ -54,7 +84,7 @@ export function Markdown({ text, onOpenLink, onPreviewHtml }: {
while (i < lines.length && !/^\s*```/.test(lines[i]))
body.push(lines[i++]);
i++;
nodes.push(<div className="st-code" key={key}><div className="st-code__head"><span>{language || tr('code')}</span>{tr(/^html$/i.test(language) && onPreviewHtml ? <Btn size="sm" onClick={() => onPreviewHtml(body.join('\n'))}>{tr("Open preview")}</Btn> : null)}<IconBtn name="copy" title={tr("Copy code")} onClick={() => void navigator.clipboard.writeText(body.join('\n'))}/></div><SyntaxCode text={body.join('\n')} language={language}/></div>);
nodes.push(<div className="st-code" key={key}><div className="st-code__head"><span>{language || tr('code')}</span>{tr(/^html$/i.test(language) && onPreviewHtml ? <Btn size="sm" onClick={() => onPreviewHtml(body.join('\n'))}>{tr("Open preview")}</Btn> : null)}<IconBtn name="copy" title={tr("Copy code")} onClick={() => void copyCode(body.join('\n'))}/></div><SyntaxCode text={body.join('\n')} language={language}/></div>);
continue;
}
const block = displayBlock(lines, i);
@@ -66,7 +96,7 @@ export function Markdown({ text, onOpenLink, onPreviewHtml }: {
const heading = line.match(/^(#{1,6})\s+(.+)$/);
if (heading) {
const Tag = heading[1].length <= 2 ? 'h2' : heading[1].length === 3 ? 'h3' : 'h4';
nodes.push(<Tag key={i}>{tr(inline(heading[2], onOpenLink))}</Tag>);
nodes.push(<Tag key={i}>{tr(inline(heading[2], onOpenLink, imageSession))}</Tag>);
i++;
continue;
}
@@ -90,7 +120,7 @@ export function Markdown({ text, onOpenLink, onPreviewHtml }: {
i += 2;
while (i < lines.length && lines[i].includes('|'))
rows.push(cells(lines[i++]));
nodes.push(<div className="md-table-wrap" key={key}><table className="md-table"><thead><tr>{tr(header.map((c, j) => <th key={j}>{tr(inline(c, onOpenLink))}</th>))}</tr></thead><tbody>{tr(rows.map((r, j) => <tr key={j}>{tr(r.map((c, k) => <td key={k}>{tr(inline(c, onOpenLink))}</td>))}</tr>))}</tbody></table></div>);
nodes.push(<div className="md-table-wrap" key={key}><table className="md-table"><thead><tr>{tr(header.map((c, j) => <th key={j}>{tr(inline(c, onOpenLink, imageSession))}</th>))}</tr></thead><tbody>{tr(rows.map((r, j) => <tr key={j}>{tr(r.map((c, k) => <td key={k}>{tr(inline(c, onOpenLink, imageSession))}</td>))}</tr>))}</tbody></table></div>);
continue;
}
if (/^>\s?/.test(line)) {
@@ -98,8 +128,19 @@ export function Markdown({ text, onOpenLink, onPreviewHtml }: {
i++;
continue;
}
nodes.push(<p key={i}>{tr(inline(line, onOpenLink))}</p>);
// One paragraph: a single newline inside it is a soft break (a space); two spaces or a backslash at the end of a line is a hard break (#3818).
const key = i, parts: ReactNode[] = [];
let current = line;
i++;
while (true) {
const hard = /( {2,}|\\)$/.test(current);
parts.push(<Fragment key={parts.length}>{tr(inline(current.replace(/( {2,}|\\)$/, ''), onOpenLink, imageSession))}</Fragment>);
if (i >= lines.length || !continuesParagraph(lines, i))
break;
parts.push(hard ? <br key={`b${parts.length}`}/> : ' ');
current = lines[i++].trimStart();
}
nodes.push(<p key={key}>{tr(parts)}</p>);
}
return <>{tr(nodes)}</>;
}
+3 -2
View File
@@ -1,6 +1,7 @@
import { tr } from '../lib/i18n';
import { useState } from 'react';
import { Btn, Dialog, ErrorNotice, IconBtn, Select } from './ui';
import { MCP_NAME_HELP, validMcpName } from '../lib/mcp-names';
export function NativeMcpEditor({ text, onChange }: {
text: string;
onChange: (text: string) => void;
@@ -19,8 +20,8 @@ export function NativeMcpEditor({ text, onChange }: {
function open(id: string) { const server = servers[id] ?? {}; setEditing(id); setName(id); setTransport(server.url ? 'url' : 'command'); setAddress(server.url ?? server.command ?? ''); setArgs((server.args ?? []).join('\n')); setAdvanced(JSON.stringify(Object.fromEntries(Object.entries(server).filter(([k]) => !['url', 'command', 'args', 'disabled'].includes(k))), null, 2)); setError(''); }
function save() {
try {
if (!/^[a-z0-9_-]{1,64}$/i.test(name))
throw new Error('Use a simple server name: letters, numbers, hyphens or underscores.');
if (!validMcpName(name))
throw new Error(MCP_NAME_HELP);
if (!address.trim())
throw new Error('Enter the server URL or program.');
if (name !== editing && servers[name])
+27 -9
View File
@@ -20,6 +20,8 @@ import { go, setSettingsSection, selectRemoteModel, pushToast } from '../lib/sto
import { nativeAction, useNative, type NativeJob } from '../lib/native';
import { tuneEffective } from '../lib/native-tune';
import { tunedFields } from '../lib/tune-values';
import { draftingChoices as draftingOptions } from '../lib/drafting';
import { deepseekContexts, deepseekContextNote } from '../lib/deepseek-context';
import { NativeSettings } from './NativeSettings';
import { NativeSettingsFile } from './NativeSettingsFile';
import { RunImageModel } from './RunImageModel';
@@ -41,6 +43,7 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
const manual = useStore(s => s.setup?.choices?.onboarding?.manual ?? false);
const [expert, setExpert] = useKept('models.expert', manual), [multiple, setMultiple] = useKept('models.multiple', false), [roles, setRoles] = useKept<Roles | null>('models.roles', null), [kv, setKv] = useKept('models.kv', 'int8'), [lowRam, setLowRam] = useKept('models.lowRam', 'auto'), [split, setSplit] = useKept('models.split', 'auto');
const [resourceProfile, setResourceProfile] = useKept<'work' | 'performance'>('models.profile', 'work');
const [drafting, setDrafting] = useState('auto');
const [pending, setPending] = useState(false), [error, setError] = useState(''), [plan, setPlan] = useState<Record<string, any> | null>(null), [loadedError, setLoadedError] = useState(''), [tuneOpen, setTuneOpen] = useState(false), [jobId, setJobId] = useKept('models.job', ''), [manage, setManage] = useState(false);
const [toastedJob, setToastedJob] = useKept('models.toasted', ''), [appliedSig, setAppliedSig] = useKept('models.applied', '');
const presetPending = useRef(false);
@@ -81,7 +84,12 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
}, [selectedSize?.id, installed?.ready, resourceProfile, vision, native?.jobs.filter(j => j.state === 'done').length]);
useEffect(() => { if (tunedNow?.fields.kv) setKv(tunedNow.fields.kv as typeof kv); if (tunedNow?.fields.lowRam) setLowRam(tunedNow.fields.lowRam as typeof lowRam); }, [tunedNow]);
const active = native?.jobs.find(j => j.state === 'running');
const job = native?.jobs.find(j => j.id === jobId) ?? (!jobId ? native?.jobs.filter(j => ['load', 'install'].includes(j.kind)).at(-1) : undefined);
// Catalog groups share quantization names. A remembered job must still belong
// to the selected version; another family's failure stays in diagnostics.
const selectedJob = (j: NativeJob) => !!selectedSize && (j.model === selectedSize.id ||
(j.model === selectedSize.model && j.family === selectedSize.sourceFamily));
const job = native?.jobs.find(j => j.id === jobId && selectedJob(j)) ??
native?.jobs.filter(j => ['load', 'install'].includes(j.kind) && selectedJob(j)).at(-1);
const noFit = selectedSize?.on_this_pc === 'does not fit';
// Graphics cards (strata_control/gpu_fit.py: the one rule of the app, the Terminal and setup): the recommended option is pre-selected, the person may pick another
// A card a loaded model already uses (a second model on it usually does not fit): said on its row, and not the pre-selected one while another is free.
@@ -99,8 +107,11 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
const fitHardware = { ram_gb: pcRam ? Math.max(0, pcRam - keptRam) : pcRam, vram_gb: Math.max(0, (roleCards.length > 1 ? vramOn(effectiveRoles, roleCards) : pcVram) - keptVramOn(chosenCards)) };
// A layer split written for another number of cards no longer fits: back to automatic.
useEffect(() => { if (cardsGpus.length <= 1 && split !== 'auto') setSplit('auto'); }, [cardsGpus.length]);
// Drafting (which drafter guesses ahead): DFlash2 is for Qwen3.8-Flash-Next only (the qwen and gyro families), so no other model is offered it.
const draftingChoices = selectedSize ? draftingOptions(selectedSize.sourceFamily ?? selectedFamily?.family, native?.capabilities?.dflash2) : null;
const draftingWords: Record<string, string> = { auto: 'Auto (recommended)', mtp: 'MTP', dflash2: 'DFlash2', off: 'Off' };
const values = useMemo(() => ({ family: selectedSize?.sourceFamily, model: selectedSize?.model, context, vision: selectedSize?.images ? vision : 'no', kv, low_ram: lowRam, resourceProfile, ...(cardsGpus.length > 1 ? { gpus: cardsGpus.join(',') } : cardsGpus.length === 1 ? { gpu: cardsGpus[0] } : {}), loadAfter: true, multiple: expert && multiple }), [selectedSize?.sourceFamily, selectedSize?.model, context, vision, kv, lowRam, resourceProfile, cardsGpus.join(','), expert, multiple]);
const downloadValues = { ...values, vision: selectedSize?.images && vision !== 'no' ? 'yes' : 'no', loadVision: selectedSize?.images ? vision : 'no' };
const downloadValues = { ...values, ...(draftingChoices ? { drafting } : {}), vision: selectedSize?.images && vision !== 'no' ? 'yes' : 'no', loadVision: selectedSize?.images ? vision : 'no' };
useEffect(() => { if (!native?.catalog || family)
return; const current = native.instances.find(i => i.id === native.defaultId); const rec = native.catalog.recommendation ?? { family: null, model: null }; const loadedFamily = families.find(f => f.sizes.some(s => s.id === current?.model)); const have = new Set((native.installed?.models ?? []).filter(m => m.ready).map(m => m.model)); const installedFamily = families.find(f => f.sizes.some(s => have.has(s.id))); const f = loadedFamily ?? installedFamily ?? families.find(f => f.family === rec.family || f.sizes.some(s => s.sourceFamily === rec.family)) ?? families[0]; if (f) {
setFamily(f.family);
@@ -117,7 +128,7 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
}
if (!selectedSize)
return;
const sig = installed ? ['have', installed.model, installed.ready, installed.context, installed.kv, installed.images, installed.visionEnabled, installed.visionDevice, installed.lowRamMode, installed.resourceProfile].join('|') : `new|${selectedSize.id}`;
const sig = installed ? ['have', installed.model, installed.ready, installed.context, installed.kv, installed.images, installed.visionEnabled, installed.visionDevice, installed.lowRamMode, installed.drafting, installed.resourceProfile].join('|') : `new|${selectedSize.id}`;
if (sig === appliedSig)
return;
setAppliedSig(sig);
@@ -128,12 +139,16 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
setVision(installed.visionEnabled === false ? 'no' : installed.visionDevice ?? (installed.images && installed.images !== 'off' ? installed.images : 'yes'));
setKv(installed.kv ?? 'int8');
setLowRam(installed.lowRamMode ?? 'auto');
setDrafting(installed.drafting ?? 'auto');
// A model never tuned has no saved choice: keep the page's own (Leave 20%) rather than flipping to Full performance.
if (installed.resourceProfile && !loadedSelected?.resourceProfile)
setResourceProfile(installed.resourceProfile);
}
else
else {
setVision('yes');
if (selectedSize.multi_file)
setContext(8192); // DeepSeek runs at the one context its engine was checked at (as in the guided setup)
}
}, [selectedSize?.id, installed?.model, installed?.ready, installed?.context, installed?.kv, installed?.images, installed?.visionEnabled, installed?.visionDevice, installed?.lowRamMode, installed?.resourceProfile]);
useEffect(() => { if (jobId && job?.state === 'done' && job.id !== toastedJob && (job.kind === 'load' || job.kind === 'install')) {
setToastedJob(job.id);
@@ -151,7 +166,7 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
} }
async function load() { if (!selectedSize)
return; await perform(async () => { if (installed?.ready && context === installed.context && (!selectedSize?.images || vision === 'no' || installed.images && installed.images !== 'off') && (!installed.kv || kv === installed.kv || kv === tunedNow?.fields.kv) && (lowRam === (installed.lowRamMode ?? 'auto') || lowRam === tunedNow?.fields.lowRam)) {
const result = await nativeAction<NativeJob>('operations', { kind: 'load', args: { model: selectedSize.id, resourceProfile, vision: selectedSize?.images ? vision : 'no', multiple: expert && multiple, ...(roleCards.length > 1 ? (roles && customRoles ? { gpuRoles: effectiveRoles, layerSplit: expert && cardsGpus.length > 1 ? split : 'auto' } : cardsGpus.length ? { gpus: cardsGpus, gpusFromRecommendation: true, layerSplit: expert && cardsGpus.length > 1 ? split : 'auto' } : {}) : {}), values: {} } });
const result = await nativeAction<NativeJob>('operations', { kind: 'load', args: { model: selectedSize.id, resourceProfile, vision: selectedSize?.images ? vision : 'no', multiple: expert && multiple, ...(draftingChoices ? { values: { drafting } } : {}), ...(roleCards.length > 1 ? (roles && customRoles ? { gpuRoles: effectiveRoles, layerSplit: expert && cardsGpus.length > 1 ? split : 'auto' } : cardsGpus.length ? { gpus: cardsGpus, gpusFromRecommendation: true, layerSplit: expert && cardsGpus.length > 1 ? split : 'auto' } : {}) : {}) } });
setJobId(result.id);
}
else
@@ -177,10 +192,13 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
return; presetPending.current = true; setFamily(f.family); setSize(f.sizes.find(s => s.model === v.model && (s.sourceFamily === v.family || f.family === v.family))?.id ?? ''); setContext(v.context ?? 32768); setVision(v.vision ?? 'no'); setKv(v.kv ?? 'int8'); setLowRam(v.low_ram ?? 'auto'); }}/><Btn icon="models" onClick={() => setManage(true)}>{tr("Manage models")}</Btn></span></div>
<div className="native-version"><VersionPicker pick={native?.catalog?.recommendation} versions={allSizes} selectedId={selectedSize.id} hardware={fitHardware} downloaded={readyIds} onSelect={id => { const f = families.find(f => f.sizes.some(s => s.id === id)); if (f) { setFamily(f.family); setSize(id); setError(''); } }} onOwnGguf={() => (onImport ?? onOtherModels)()}/></div>
{tr(noFit ? <p className="small muted native-spill-note"><Icon name="disk" size={16}/>{tr("May run slower \u00B7 Model data can be read from SSD.")}</p> : null)}
{tr(selectedSize?.multi_file && deepseekContextNote(context) ? <p className="small muted native-spill-note"><Icon name="info" size={16}/>{tr(deepseekContextNote(context))}</p> : null)}
<div className="native-choice__row">
<div className="native-choice__col"><span className="native-choice__label">{tr("Conversation context")}</span><Select ariaLabel={tr("Conversation context")} value={String(context)} options={contextChoices(native?.catalog?.contexts ?? [8192, 32768, 65536, 131072, 262144], context).map(n => ({ value: String(n), label: n % 1024 === 0 ? `${n / 1024}K` : `${n.toLocaleString()} tokens` }))} onChange={n => setContext(Number(n))}/></div>
<div className="native-choice__col"><span className="native-choice__label">{tr("Conversation context")}</span><Select ariaLabel={tr("Conversation context")} value={String(context)} options={contextChoices(selectedSize?.multi_file ? deepseekContexts(installed?.contextMax) : native?.catalog?.contexts ?? [8192, 32768, 65536, 131072, 262144], context).map(n => ({ value: String(n), label: n % 1024 === 0 ? `${n / 1024}K` : `${n.toLocaleString()} tokens` }))} onChange={n => setContext(Number(n))}/></div>
{tr(draftingChoices ? <div className="native-choice__col"><span className="native-choice__label">{tr("Drafting")}</span><Select ariaLabel={tr("Drafting")} value={drafting} disabled={!!active} options={draftingChoices.map(v => ({ value: v, label: tr(draftingWords[v]) }))} onChange={v => setDrafting(String(v))}/></div> : null)}
<div className="native-choice__col"><span className="native-choice__label">{tr("Computer use")}</span><div className="native-resource-choice"><button aria-pressed={resourceProfile === 'work'} data-selected={resourceProfile === 'work'} onClick={() => setResourceProfile('work')}><Icon name="computer" size={20}/><span><strong>{tr("Leave 20% for my work")}</strong><small>{tr("Headroom for your other apps")}</small></span>{tr(resourceProfile === 'work' ? <Icon name="check" size={15}/> : null)}</button><button aria-pressed={resourceProfile === 'performance'} data-selected={resourceProfile === 'performance'} onClick={() => setResourceProfile('performance')}><Icon name="bolt" size={20}/><span><strong>{tr("Full performance")}</strong><small>{tr("Give Strata priority")}</small></span>{tr(resourceProfile === 'performance' ? <Icon name="check" size={15}/> : null)}</button></div></div>
</div>
{tr(loadedSelected?.metrics?.engine?.drafting_note ? <p className="small muted native-spill-note">{tr(String(loadedSelected.metrics.engine.drafting_note))}</p> : null)}
{roleCards.length > 1 ? <GpuRoles cards={roleCards} roles={effectiveRoles} custom={customRoles} recommendedRoles={recommended.roles} vendor={engineVendor} disabled={!!active} onChange={changeRoles} onBack={() => setRoles(null)} memoryGb={selectedSize ? memoryNeeded(selectedSize) || null : null} ramGb={pcRam} spillGb={selectedSize ? fitOn(selectedSize, fitHardware).spillGb : 0} usedBy={i => usedBy([i])}/> : null}
{tr(roleCards.length > 1 && (native?.hardware?.gpus ?? []).length > roleCards.length ? <p className="small muted native-cards__note">{tr('Not used: {0} (another make of graphics card than this engine drives).', [(native?.hardware?.gpus ?? []).filter(g => !roleCards.some(r => r.index === g.index && r.vendor === g.vendor)).map(g => g.name).join(', ')])}</p> : null)}
{tr(selectedSize?.imagesCapable && !selectedSize.images ? <p className="small muted native-spill-note">{tr("This model can read pictures, but the engine installed here does not include the picture reader yet, so Read pictures is off. Update the engine in Settings → About to turn it on.")}</p> : null)}
@@ -196,13 +214,13 @@ export function NativeModels({ onOtherModels, onReady, onImport, onConnectEngine
{tr(expert && IMAGE_GENERATION_VISIBLE ? <><details className="native-generation"><summary>{tr("Image generation ")}<span className="muted">{tr("Optional")}</span></summary><RunImageModel library={imageLibrary} disabled={pending || !!active} onChange={() => { }}/></details></> : null)}
{tr(expert ? <section id="native-expert-settings" ref={expertSection} className="st-card native-execution"><h2 className="h4">{tr("Expert settings")}</h2>{tr(tunedNow ? <p className="small muted native-tuned-note">{tr("Showing the values of your tuning '{0}'.", [tunedNow.name])}</p> : null)}<label className="native-toggle"><span><strong>{tr("Run multiple models")}</strong><small>{tr("Loading another model keeps the others loaded.")}</small></span><button className="st-toggle" role="switch" aria-label={tr("Run multiple models")} aria-checked={multiple} onClick={() => setMultiple(v => !v)}/></label>{tr(cardsGpus.length > 1 ? <label className="input-field"><span>{tr("Layer split")}</span><input className="st-input" value={split} placeholder={tr("auto or 16,32")} onChange={e => setSplit(e.target.value)}/></label> : null)}<div className="native-field-grid"><label className="input-field"><span>{tr("Context memory")}</span><Select ariaLabel={tr("Native KV format")} value={kv} options={[{ value: 'int8', label: 'INT8' }, { value: 'q4_0', label: 'Q4' }, { value: 'k8v4', label: 'K8 / V4' }]} onChange={setKv}/></label><label className="input-field"><span>{tr("Low-RAM mode")}</span><Select ariaLabel={tr("Native low RAM mode")} value={lowRam} options={['auto', 'resident', 'mmap'].map(v => ({ value: v, label: v === 'auto' ? 'Automatic' : v === 'resident' ? 'Keep experts in RAM' : 'Read experts from disk' }))} onChange={setLowRam}/></label></div>{tr(installed ? <><NativeSettingsFile model={installed.model}/><NativeSettings model={installed.model}/></> : <p className="small muted">{tr("More engine parameters become available after preparing this version.")}</p>)}</section> : null)}
</div>
<aside className="stack stack--lg"><PcFitCard gpuName={pcGpu?.name} ramGb={pcRam} vramGb={pcVram} inUse={keptModels.length ? { ramGb: Math.round(keptRam), vramGb: Math.round(keptVramOn(chosenCards)) } : null} fit={selectedSize ? fitOn(selectedSize, fitHardware) : null}/><section className="st-card"><h2 className="h4">{tr("Model files")}</h2><p className="small muted">{tr("Strata prepares all shards, the draft layer and shared files. Completed downloads are reused.")}</p><code className="native-path">{installed?.dataFolder ?? native?.downloadDirectory ?? native?.installed?.data_folder ?? 'Choose an installation'}</code><Btn size="sm" variant="ghost" onClick={() => { setSettingsSection('Storage'); go('settings'); }}>{tr("Storage settings")}</Btn></section><Btn size="sm" variant="ghost" onClick={onOtherModels}>{tr("Other local models\u2026")}</Btn></aside>
<aside className="stack stack--lg"><PcFitCard gpuName={pcGpu?.name} ramGb={pcRam} vramGb={pcVram} diskGb={selectedSize?.multi_file ? selectedSize.disk_gb ?? selectedSize.download_gb : null} inUse={keptModels.length ? { ramGb: Math.round(keptRam), vramGb: Math.round(keptVramOn(chosenCards)) } : null} fit={selectedSize ? fitOn(selectedSize, fitHardware, true, { context, kv }) : null}/><section className="st-card"><h2 className="h4">{tr("Model files")}</h2><p className="small muted">{tr("Strata prepares all shards, the draft layer and shared files. Completed downloads are reused.")}</p><code className="native-path">{installed?.dataFolder ?? native?.downloadDirectory ?? native?.installed?.data_folder ?? 'Choose an installation'}</code><Btn size="sm" variant="ghost" onClick={() => { setSettingsSection('Storage'); go('settings'); }}>{tr("Storage settings")}</Btn></section><Btn size="sm" variant="ghost" onClick={onOtherModels}>{tr("Other local models\u2026")}</Btn></aside>
</div>
<Dialog open={!!plan} title={tr(installed?.ready ? "Apply settings and load" : "Download and prepare")} compact onClose={() => !pending && setPlan(null)} actions={<><Btn disabled={pending} onClick={() => setPlan(null)}>{tr("Cancel")}</Btn><Btn variant="primary" disabled={pending || !!plan?.plan?.disk_short} onClick={() => void download()}>{tr(installed?.ready ? "Apply & load" : "Download & load")}</Btn></>}><p><strong>{tr(plan?.plan?.title ?? selectedFamily?.title)}</strong>{tr(installed?.ready ? null : <>{tr(" \u00B7 approximately ")}{tr(plan?.plan?.download_gb ?? selectedSize?.download_gb)}{tr(" GB")}</>)}</p>{tr(installed?.ready ? <p className="small muted">{tr("The model files are already on this computer. Nothing will be downloaded; only the settings you chose are applied.")}</p> : null)}{tr(plan?.plan?.disk_short ? <ErrorNotice message={plan.plan.disk_short}/> : null)}{tr(installed?.ready ? <p className="small muted">{tr("Any currently loaded native models will be unloaded first.")}</p> : <><p className="small muted">{tr("Existing files are checked and reused. Any currently loaded native models will be unloaded before preparing files.")}</p><p className="small muted">{tr("Your computer may feel slower while weights load. You can stop the download and resume later.")}</p></>)}{tr(selectedSize?.images ? <p className="small muted">{tr("Vision encoder files are included. Picture reading loads only when enabled.")}</p> : null)}</Dialog>
<Dialog open={!!plan} title={tr(installed?.ready ? "Apply settings and load" : "Download and prepare")} compact onClose={() => !pending && setPlan(null)} actions={<><Btn disabled={pending} onClick={() => setPlan(null)}>{tr("Cancel")}</Btn><Btn variant="primary" disabled={pending || !!plan?.plan?.disk_short} onClick={() => void download()}>{tr(installed?.ready ? "Apply & load" : "Download & load")}</Btn></>}><p><strong>{tr(plan?.plan?.title ?? selectedFamily?.title)}</strong>{tr(installed?.ready ? null : <>{tr(" \u00B7 approximately ")}{tr(plan?.plan?.download_gb ?? selectedSize?.download_gb)}{tr(" GB")}</>)}</p>{tr(plan?.drafting?.needed ? <p className="small muted">{tr("Choosing DFlash2 also downloads its drafter: about {0} GB, once. If it does not fit in graphics memory, the model uses the MTP instead.", [plan.drafting.gb])}</p> : null)}{tr(installed?.ready ? <p className="small muted">{tr("The model files are already on this computer. Nothing will be downloaded; only the settings you chose are applied.")}</p> : null)}{tr(plan?.plan?.disk_short ? <ErrorNotice message={plan.plan.disk_short}/> : null)}{tr(installed?.ready ? <p className="small muted">{tr("Any currently loaded native models will be unloaded first.")}</p> : <><p className="small muted">{tr("Existing files are checked and reused. Any currently loaded native models will be unloaded before preparing files.")}</p><p className="small muted">{tr("Your computer may feel slower while weights load. You can stop the download and resume later.")}</p></>)}{tr(selectedSize?.images ? <p className="small muted">{tr("Vision encoder files are included. Picture reading loads only when enabled.")}</p> : null)}</Dialog>
<NativeDeleteDialog id={deleting} onClose={() => setDeleting(null)}/>
<Dialog open={manage} title={tr("Manage models")} onClose={() => setManage(false)} actions={<Btn onClick={() => setManage(false)}>{tr("Done")}</Btn>}><div className="native-manage">
{tr(downloaded.length ? <div className="native-manage-list" role="list">{tr(downloaded.map(m => { const v = allSizes.find(x => x.id === m.model), loaded = native?.instances.some(i => i.model === m.model && i.status === 'ready'); return <div className="native-manage-row" role="listitem" key={m.model} data-selected={selectedSize?.id === m.model || undefined}>
{tr(downloaded.some(m => m.ready || m.diskGb) ? <div className="native-manage-list" role="list">{tr(downloaded.filter(m => m.ready || m.diskGb).map(m => { const v = allSizes.find(x => x.id === m.model), loaded = native?.instances.some(i => i.model === m.model && i.status === 'ready'); return <div className="native-manage-row" role="listitem" key={m.model} data-selected={selectedSize?.id === m.model || undefined}>
<span className="native-manage-name"><strong>{tr(v ? `${catalogVersionLabel(v)}` : m.model_name ?? m.model)}</strong><small>{tr(v ? v.model : m.model)}{tr(m.diskGb ? ` · ${sizeGb(m.diskGb)}` : '')}</small></span>
<span className={loaded ? 'st-pill st-pill--accent' : 'st-pill'}>{tr(loaded ? 'Loaded' : 'Downloaded')}</span>
<span className="native-manage-actions"><Btn size="sm" disabled={!!active || !v} onClick={() => { if (v) setSize(v.id); setManage(false); }}>{tr(selectedSize?.id === m.model ? 'Selected' : 'Use version')}</Btn><Btn size="sm" variant="danger" disabled={!!active} onClick={() => setDeleting(m.model)}>{tr("Delete")}</Btn></span>
@@ -6,6 +6,7 @@ import { fetchUpdateCheck, publishUpdateCheck, useInstalledVersions, useLastUpda
import { nativeAction, useNative, type NativeJob } from '../lib/native';
import { Btn, Dialog, ErrorNotice, Select } from './ui';
import { NativeProgress } from './NativeModels';
import { jobErrorText } from '../lib/native-job-diagnostics';
export function NativeBenchmark({ open, onClose }: {
open: boolean;
onClose: () => void;
@@ -114,7 +115,7 @@ export function NativeOperations() {
}
return <div className="stack stack--lg native-engine-settings">
<section className="st-card"><div className="section-toolbar"><h2 className="h3">{tr("Update component")}</h2>{tr(update === 'available' ? <Btn size="sm" variant="primary" disabled={!!active || (native?.instances.length ?? 0) > 0} onClick={() => setConfirm(true)}>{tr("Update available")}</Btn> : null)}{tr(update !== 'available' ? <Btn size="sm" disabled={update === 'checking'} onClick={() => void checkForUpdates()}>{tr(update === 'checking' ? 'Checking…' : 'Check for updates')}</Btn> : null)}</div><p className="small muted">{tr("Update the engine for this computer. Your models and settings are kept.")}</p>{tr(update === 'current' ? <p className="small muted">{tr("Up to date")}</p> : null)}{tr(checkError ? <p className="small inline-error" role="alert">{tr(checkError)}</p> : null)}{tr(update === 'available' && (native?.instances.length ?? 0) > 0 ? <p className="small muted">{tr("Open Models and unload the running native models first.")}</p> : null)}</section>
<section className="st-card"><div className="section-toolbar"><h2 className="h3">{tr("Diagnostics")}</h2><Btn size="sm" onClick={() => setShowLogs(true)}>{tr("View logs")}</Btn></div><p className="small muted">{tr("Downloads, preparation and engine failures include their original messages here.")}</p>{tr(native?.jobs.filter(j => j.state === 'error').slice(-3).map(j => <ErrorNotice key={j.id} message={j.error ?? j.summary} detail={j.details}/>))}</section>
<section className="st-card"><div className="section-toolbar"><h2 className="h3">{tr("Diagnostics")}</h2><Btn size="sm" onClick={() => setShowLogs(true)}>{tr("View logs")}</Btn></div><p className="small muted">{tr("Downloads, preparation and engine failures include their original messages here.")}</p>{tr(native?.jobs.filter(j => j.state === 'error').slice(-3).map(j => <ErrorNotice key={j.id} message={jobErrorText(j, native.jobs)} detail={j.details}/>))}</section>
<Dialog open={showLogs} wide title={tr("Engine logs")} onClose={() => setShowLogs(false)} actions={<><Btn icon="download" disabled={saving} onClick={() => void saveLogs()}>{tr(saving ? 'Working…' : 'Save logs')}</Btn><Btn variant="primary" onClick={() => setShowLogs(false)}>{tr("Close")}</Btn></>}>
<pre className="native-log" aria-label={tr("Engine log")}>{logs.length ? logs.map(l => `${l.at} · ${l.line}`).join(String.fromCharCode(10)) : tr('No engine messages yet.')}</pre>
{tr(saved ? <p className="small" role="status">{tr("Saved to {0}", [saved])}</p> : null)}
+39 -10
View File
@@ -1,3 +1,4 @@
import { DecimalInput } from './DecimalInput';
import { tr } from '../lib/i18n';
import {nativeSettingsResponse,nativeSettingsValues} from '../lib/native-settings-response';
import { describeChange, type TunedChange } from '../lib/tune-values';
@@ -5,12 +6,29 @@ import { useEffect, useState } from 'react';
import { api } from '../lib/client';
import { refreshNative, useNative, type NativeField } from '../lib/native';
import { Btn, ErrorNotice, Select } from './ui';
import { speedOptions } from '../lib/deepseek-speed';
import type { RunCard } from '../lib/native-fit';
import { NativeMcpEditor } from './NativeMcpEditor';
/** One of the speed features: what it is called, what it does, whether it is on by default, and the engine environment it writes outside its default (server/native-settings.mjs). */
interface SpeedSwitch {
id: string;
label: string;
hint: string;
default: 'on' | 'off';
env: Record<string, string>;
/** What an earlier test build wrote for the same state: it reads as set, and goes with it. */
legacy?: Record<string, string>;
}
const switchHas = (entries: Record<string, string> | undefined, env: Record<string, string> | undefined) => !!entries && Object.keys(entries).length > 0 && Object.entries(entries).every(([name, value]) => env?.[name] === value);
const switchIsSet = (sw: SpeedSwitch, env: Record<string, string> | undefined) => switchHas(sw.env, env) || switchHas(sw.legacy, env);
const switchIsOn = (sw: SpeedSwitch, env: Record<string, string> | undefined) => switchIsSet(sw, env) !== (sw.default === 'on');
export function NativeSettings({ model }: {
model: string;
}) {
const native = useNative();
const [revision,setRevision]=useState(0);
const [switches, setSwitches] = useState<SpeedSwitch[]>([]); // the speed switches that apply to this model on this PC
const [dsExperts, setDsExperts] = useState(false); // the installed DeepSeek engine knows --experts (0.1.50 or newer)
const [tuned, setTuned] = useState<{ preset: { name: string } | null; changed: TunedChange[] } | null>(null);
const [values, setValues] = useState<Record<string, any>>({}), [mcp, setMcp] = useState('{}'), [original, setOriginal] = useState(''), [text, setText] = useState(''), [tab, setTab] = useState('fields'), [pending, setPending] = useState(false), [error, setError] = useState(''), [notice, setNotice] = useState(''), [query, setQuery] = useState('');
useEffect(() => {
@@ -25,9 +43,13 @@ export function NativeSettings({ model }: {
values: Record<string, any>;
mcp: Record<string, any>;
tuned?: { preset: { name: string } | null; changed: TunedChange[] } | null;
capabilities?: { dsExperts?: boolean };
speedSwitches?: SpeedSwitch[];
}>(`/api/native/models/${model}/settings`).then(input => {
if (!active)
return;
setDsExperts(!!input.capabilities?.dsExperts);
setSwitches(input.speedSwitches ?? []);
setTuned(input.tuned ?? null);
const value = nativeSettingsResponse(input);
setValues(value.values);
@@ -41,6 +63,8 @@ export function NativeSettings({ model }: {
return () => { active = false; };
}, [model,revision]);
const change = (id: string, value: any) => { const next = { ...values, [id]: value }; setValues(next); setText(JSON.stringify(next, null, 2)); setNotice(''); };
/** The default removes the switch's environment entries, the other state writes them: the model's file then carries nothing for a default launch. */
const changeSwitch = (sw: SpeedSwitch, on: boolean) => { const env = { ...(values.env ?? {}) }; const write = on !== (sw.default === 'on'); for (const [name, value] of Object.entries(sw.env)) { if (write) env[name] = value; else delete env[name]; } if (!write) for (const name of Object.keys(sw.legacy ?? {})) delete env[name]; change('env', env); };
async function save() {
setPending(true);
setError('');
@@ -90,16 +114,10 @@ export function NativeSettings({ model }: {
<summary>{tr('Native engine settings')}</summary>
{error ? <><ErrorNotice message={tr('Couldn’t load model settings.')} next={tr('Check that the engine is connected and this model is installed, then try again.')} onRetry={() => setRevision(value => value + 1)}/><details className="native-settings-diagnostic"><summary className="small">{tr('Details')}</summary><p className="small muted">{error}</p></details></> : <p className="small muted" role="status">{tr('Reading saved settings…')}</p>}
</details>;
const rows = (native?.fields ?? []).filter(f => `${tr(f.label)} ${f.id} ${tr(f.group)}`.toLowerCase().includes(query.toLowerCase()));
const rows = (native?.fields ?? []).filter(f => (!f.forModel || (model.toLowerCase().includes(f.forModel) && (f.id !== 'ds_experts' || dsExperts))) && `${tr(f.label)} ${f.id} ${tr(f.group)}`.toLowerCase().includes(query.toLowerCase()));
const switchRows = switches.filter(sw => `${tr(sw.label)} ${sw.id} ${tr('Speed')}`.toLowerCase().includes(query.toLowerCase()));
return <details className="native-settings"><summary>{tr("Native engine settings")}</summary>{tr(tuned?.changed?.length ? <div className="native-tuned"><p className="small"><strong>{tr("Your tuning '{0}' sets", [tuned.preset?.name ?? ''])}</strong></p><ul>{tuned.changed.map(c => { const d = describeChange(c); return <li key={c.flag}><span>{d.name}</span><b>{d.now || tr('off')}</b>{d.was !== null ? <small className="muted">{tr("was {0}", [d.was])}</small> : null}</li>; })}</ul></div> : null)}<p className="small muted">{tr(!original ? 'Reading saved settings… ' : '')}{tr("Settings for this model. Blank values use Strata\u2019s defaults.")}</p><div className="editor-tabs" role="tablist" aria-label={tr("Native settings editor")}>{tr(['fields', 'json', 'tools'].map(t => <button key={t} role="tab" aria-selected={tab === t} onClick={() => chooseTab(t)}>{tr(t === 'fields' ? 'Parameters' : t === 'json' ? 'JSON' : 'MCP tools')}</button>))}</div>
{tr(tab === 'fields' ? <><input className="st-input" type="search" aria-label={tr("Search native parameters")} placeholder={tr("Search parameters…")} value={query} onChange={e => setQuery(e.target.value)}/>{tr(['Responses', 'GPU & memory', 'Serving', 'Image reading', 'Tools', 'Recovery'].map(group => rows.some(f => f.group === group) ? <section className="native-settings-group" key={group}><h3 className="h4">{tr(group)}</h3>{tr(rows.filter(f => f.group === group).map(f => <label className="native-setting-row" key={f.id}><span><strong>{tr(f.label)}</strong>{tr(f.supported === false ? <small>{tr("Requires a newer Strata installation.")}</small> : f.hint ? <small>{tr(f.hint)}</small> : null)}</span><NativeFieldInput field={f} value={values[f.id]} onChange={v => change(f.id, v)}/></label>))}</section> : null))}<section className="native-settings-group"><h3 className="h4">{tr("Sampling")}</h3>{tr([['temperature', 'Temperature'], ['top_p', 'Top-p'], ['top_k', 'Top-k'], ['min_p', 'Min-p'], ['repetition_penalty', 'Repetition penalty'], ['presence_penalty', 'Presence penalty'], ['frequency_penalty', 'Frequency penalty'], ['penalty_last_n', 'Penalty window'], ['seed', 'Seed']].map(([id, label]) => <label className="native-setting-row" key={id}><span><strong>{tr(label)}</strong></span><input className="st-input" aria-label={tr("Native {0}", [label])} inputMode="decimal" value={values.sampling?.[id] ?? ''} onChange={e => {
const sampling = { ...values.sampling };
if (e.target.value === '')
delete sampling[id];
else
sampling[id] = Number(e.target.value);
change('sampling', sampling);
}}/></label>))}</section></> : tab === 'json' ? <><p className="small muted">{tr("Engine arguments and STRATA environment flags are editable here. Program paths and network access remain managed by the host.")}</p><textarea className="st-input native-json" aria-label={tr("Native configuration JSON")} spellCheck={false} value={text} onChange={e => { setText(e.target.value); setNotice(''); }}/></> : <><p className="small muted">{tr("Add local programs or HTTP MCP servers. Connected tools can read or change the resources you give them access to. Enable tools in chat when you want the model to use them.")}</p><NativeMcpEditor text={mcp} onChange={value => { setMcp(value); setNotice(''); }}/><details><summary className="small">{tr("Configuration example")}</summary><pre className="native-example">{'{\n "files": {\n "command": "npx",\n "args": ["-y", "@modelcontextprotocol/server-filesystem", "/your/folder"],\n "disabled": false\n }\n}'}</pre><p className="xs muted">{tr("Saved credentials are masked. Keep the mask to preserve an existing value; type a new value to replace it.")}</p></details></>)}
{tr(tab === 'fields' ? <><input className="st-input" type="search" aria-label={tr("Search native parameters")} placeholder={tr("Search parameters…")} value={query} onChange={e => setQuery(e.target.value)}/>{tr(['Responses', 'GPU & memory', 'Serving', 'Image reading', 'Tools', 'Recovery'].map(group => rows.some(f => f.group === group) ? <section className="native-settings-group" key={group}><h3 className="h4">{tr(group)}</h3>{tr(rows.filter(f => f.group === group).map(f => <label className="native-setting-row" key={f.id}><span><strong>{tr(f.label)}</strong>{tr(f.supported === false ? <small>{tr("Requires a newer Strata installation.")}</small> : f.hint ? <small>{tr(f.hint)}</small> : null)}</span><NativeFieldInput field={f} value={values[f.id]} onChange={v => change(f.id, v)}/></label>))}</section> : null))}{tr(switchRows.length ? <section className="native-settings-group"><h3 className="h4">{tr("Speed")}</h3>{tr(switchRows.map(sw => <label className="native-setting-row" key={sw.id}><span><strong>{tr(sw.label)}</strong><small>{tr(sw.hint)}</small></span><Select ariaLabel={tr(sw.label)} value={switchIsOn(sw, values.env) ? 'on' : 'off'} options={sw.default === 'on' ? [{ value: 'on', label: 'On (recommended)' }, { value: 'off', label: 'Off' }] : [{ value: 'off', label: 'Off (recommended)' }, { value: 'on', label: 'On' }]} onChange={v => changeSwitch(sw, v === 'on')}/></label>))}</section> : null)}<section className="native-settings-group"><h3 className="h4">{tr("Sampling")}</h3>{tr([['temperature', 'Temperature'], ['top_p', 'Top-p'], ['top_k', 'Top-k'], ['min_p', 'Min-p'], ['repetition_penalty', 'Repetition penalty'], ['presence_penalty', 'Presence penalty'], ['frequency_penalty', 'Frequency penalty'], ['penalty_last_n', 'Penalty window'], ['seed', 'Seed']].map(([id, label]) => <label className="native-setting-row" key={id}><span><strong>{tr(label)}</strong></span><DecimalInput ariaLabel={tr("Native {0}", [label])} value={values.sampling?.[id] ?? null} onChange={n => { const sampling = { ...values.sampling }; if (n === null) delete sampling[id]; else sampling[id] = n; change('sampling', sampling); }}/></label>))}</section></> : tab === 'json' ? <><p className="small muted">{tr("Engine arguments and STRATA environment flags are editable here. Program paths and network access remain managed by the host.")}</p><textarea className="st-input native-json" aria-label={tr("Native configuration JSON")} spellCheck={false} value={text} onChange={e => { setText(e.target.value); setNotice(''); }}/></> : <><p className="small muted">{tr("Add local programs or HTTP MCP servers. Connected tools can read or change the resources you give them access to. Enable tools in chat when you want the model to use them.")}</p><NativeMcpEditor text={mcp} onChange={value => { setMcp(value); setNotice(''); }}/><details><summary className="small">{tr("Configuration example")}</summary><pre className="native-example">{'{\n "files": {\n "command": "npx",\n "args": ["-y", "@modelcontextprotocol/server-filesystem", "/your/folder"],\n "disabled": false\n }\n}'}</pre><p className="xs muted">{tr("Saved credentials are masked. Keep the mask to preserve an existing value; type a new value to replace it.")}</p></details></>)}
<div className="native-settings-actions"><Btn disabled={pending || !original || !dirty} variant="primary" onClick={() => void save()}>{tr(pending ? 'Saving…' : 'Save settings')}</Btn><Btn disabled={pending} onClick={() => { const saved = JSON.parse(original || '{"values":{},"mcp":{}}'); setValues(saved.values); setText(JSON.stringify(saved.values, null, 2)); setMcp(JSON.stringify(saved.mcp, null, 2)); setError(''); }}>{tr("Restore saved")}</Btn></div>{tr(notice ? <p role="status" className="small">{tr(notice)}</p> : null)}{tr(error ? <ErrorNotice message={error} onRetry={() => setRevision(value => value + 1)}/> : null)}
</details>;
}
@@ -112,9 +130,20 @@ function NativeFieldInput({ field, value, onChange }: {
return <NativeAliases field={field} value={value} onChange={onChange}/>;
if (field.type === 'boolean')
return <Select disabled={field.supported === false} ariaLabel={tr(field.label)} value={value === null || value === undefined ? '' : String(value)} options={[{ value: '', label: 'Default' }, { value: 'true', label: 'On' }, { value: 'false', label: 'Off' }]} onChange={v => onChange(v === '' ? null : v === 'true')}/>;
if (field.id === 'ds_experts')
return <DeepSeekSpeed field={field} value={value} onChange={onChange}/>;
if (field.type === 'enum')
return <Select disabled={field.supported === false} ariaLabel={tr(field.label)} value={value ?? ''} options={[{ value: '', label: 'Default' }, ...(field.choices ?? []).map(v => ({ value: v, label: v }))]} onChange={v => onChange(v || null)}/>;
return <input disabled={field.supported === false} className="st-input" aria-label={tr(field.label)} inputMode={field.type === 'number' ? 'numeric' : undefined} placeholder={tr("Default")} value={Array.isArray(value) ? value.join(', ') : value ?? ''} onChange={e => onChange(e.target.value === '' ? null : field.type === 'array' ? e.target.value.split(',').map(s => s.trim()).filter(Boolean) : field.type === 'number' ? Number(e.target.value) : e.target.value)}/>;
if (field.type === 'number')
return <DecimalInput disabled={field.supported === false} ariaLabel={tr(field.label)} placeholder={tr("Default")} value={value === null || value === undefined || value === '' || !Number.isFinite(Number(value)) ? null : Number(value)} onChange={onChange}/>;
return <input disabled={field.supported === false} className="st-input" aria-label={tr(field.label)} placeholder={tr("Default")} value={Array.isArray(value) ? value.join(', ') : value ?? ''} onChange={e => onChange(e.target.value === '' ? null : field.type === 'array' ? e.target.value.split(',').map(s => s.trim()).filter(Boolean) : field.type === 'number' ? Number(e.target.value) : e.target.value)}/>;
}
/** DeepSeek's Speed choice. The engine decides the default itself (--experts auto); the labels name the same default (lib/deepseek-speed.ts). */
function DeepSeekSpeed({ field, value, onChange }: { field: NativeField; value: any; onChange: (v: any) => void }) {
const native = useNative();
const options = speedOptions(native?.hardware?.gpus as RunCard[] | undefined).map(o => ({ ...o, label: tr(o.label) }));
return <Select disabled={field.supported === false} ariaLabel={tr(field.label)} value={value === null || value === undefined ? '' : String(value)}
options={options} onChange={v => onChange(v === '' ? null : v)}/>;
}
function NativeAliases({ field, value, onChange }: {
field: NativeField;
+13 -2
View File
@@ -8,9 +8,20 @@ export function OnboardingMemory({ hardware, native }: {
hardware: Hardware | null;
native: NativeState['hardware'] | undefined;
}) {
const gpus = hardware ? hardware.gpus.filter(gpu => gpu.backend !== 'cpu').map(gpu => ({ id: gpu.id ?? String(gpu.index), name: gpu.name, memory: gpu.vramGb, shared: gpu.unified })) : native?.gpus.map(gpu => ({ id: String(gpu.index), name: gpu.name, memory: gpu.vram_gb, shared: false })) ?? [];
// The Windows host probe (Win32_VideoController) cannot read the VRAM of AMD cards and reports 0 for them; the engine's own list has
// the real size. For a dedicated card the host reports as 0, take the engine's figure for the card of the same name, or for the
// card in the same position when both lists have the same length. A card the engine does not list stays at 0.
const nativeGpus = native?.gpus ?? [];
const hostGpus = hardware ? hardware.gpus.filter(gpu => gpu.backend !== 'cpu') : [];
const engineVram = (name: string, position: number) => {
const norm = (value: string) => value.toLowerCase().replace(/[^a-z0-9]/g, '');
const byName = nativeGpus.find(g => norm(g.name).includes(norm(name)) || norm(name).includes(norm(g.name)));
return (byName ?? (nativeGpus.length === hostGpus.length ? nativeGpus[position] : undefined))?.vram_gb || 0;
};
const gpus = hardware ? hostGpus.map((gpu, position) => ({ id: gpu.id ?? String(gpu.index), name: gpu.name, memory: gpu.unified || gpu.vramGb > 0 ? gpu.vramGb : engineVram(gpu.name, position), shared: gpu.unified })) : nativeGpus.map(gpu => ({ id: String(gpu.index), name: gpu.name, memory: gpu.vram_gb, shared: false }));
const dedicatedCount = gpus.filter(gpu => !gpu.shared).length;
const memory = hardware ? memoryInventory(hardware) : native ? { ram: native.ram_gb, vram: native.gpus.reduce((sum, gpu) => sum + Math.max(0, gpu.vram_gb || 0), 0), total: 0, shared: false } : null;
const base = hardware ? memoryInventory(hardware) : null;
const memory = base ? (base.shared ? base : { ...base, vram: gpus.reduce((sum, gpu) => sum + (gpu.shared ? 0 : Math.max(0, gpu.memory || 0)), 0) }) : native ? { ram: native.ram_gb, vram: native.gpus.reduce((sum, gpu) => sum + Math.max(0, gpu.vram_gb || 0), 0), total: 0, shared: false } : null;
if (!memory)
return <div className="onboard-hardware" role="group" aria-label={tr("Computer memory")}><div className="onboard-memory-intro"><Icon name="memory" size={18}/><span>{tr("Reading memory\u2026")}</span></div></div>;
const total = memory.ram + memory.vram;
+3 -2
View File
@@ -15,7 +15,7 @@ function Meter({ label, total, use }: { label: string; total: number; use: numbe
* "This computer": the graphics memory and RAM, and what the chosen version would take of each. One plain line says whether it fits
* or how much would be read from the SSD (the n-gram table, which always sits on the SSD, is not counted).
*/
export function PcFitCard({ gpuName, ramGb, vramGb, fit, inUse = null }: { gpuName?: string | null; ramGb: number | null; vramGb: number; fit: Fit | null; inUse?: { ramGb: number; vramGb: number } | null }) {
export function PcFitCard({ gpuName, ramGb, vramGb, fit, inUse = null, diskGb = null }: { gpuName?: string | null; ramGb: number | null; vramGb: number; fit: Fit | null; inUse?: { ramGb: number; vramGb: number } | null; diskGb?: number | null }) {
const freeVram = Math.max(0, vramGb - (inUse?.vramGb ?? 0)), freeRam = ramGb ? Math.max(0, ramGb - (inUse?.ramGb ?? 0)) : 0;
return <section className="st-card pc-card">
<div><h2 className="h4">{tr("This computer")}</h2>
@@ -23,6 +23,7 @@ export function PcFitCard({ gpuName, ramGb, vramGb, fit, inUse = null }: { gpuNa
{tr(fit?.known && vramGb > 0 ? <Meter label="Graphics memory" total={freeVram} use={fit.vramUseGb}/> : null)}
{tr(fit?.known && ramGb ? <Meter label="RAM" total={freeRam} use={fit.ramUseGb}/> : null)}
{tr(inUse ? <p className="xs muted">{tr("Models that stay loaded use about {0} GB of RAM and {1} GB of this card’s memory.", [inUse.ramGb, inUse.vramGb])}</p> : null)}
{tr(fit?.known ? <p className="pc-note">{tr(fit.spillGb >= 1 ? 'About {0} GB will be read from the SSD, so answers can be a little slower.' : 'Fits fully in memory.', [Math.round(fit.spillGb)])}</p> : null)}
{tr(diskGb ? <p className="pc-note">{tr("Needs about {0} GB of free disk space. What does not fit in memory is read from your SSD.", [Math.round(diskGb)])}</p> : null)}
{tr(fit?.known && (diskGb ? fit.spillGb >= 1 : true) ? <p className="pc-note">{tr(fit.spillGb >= 1 ? 'About {0} GB will be read from the SSD, so answers can be a little slower.' : 'Fits fully in memory.', [Math.round(fit.spillGb)])}</p> : null)}
</section>;
}
+1 -1
View File
@@ -19,7 +19,7 @@ export function SettingsStorage() {
const library = state.library;
const versions = library ? pickerGroups(library).flatMap(group => group.versions) : [];
// ONE list and ONE total: the optimized models (sizes from their real files, shared files counted once) and the other local models together.
const nativeModels = native?.installed?.models ?? [];
const nativeModels = (native?.installed?.models ?? []).filter(m => m.ready || (m.diskGb ?? 0) > 0); // a model whose files are all gone (an old config) is not a 0.0 GB download
const sharedGb = (native?.installed?.usage?.shared_bytes ?? 0) / 1e9;
const nativeGb = native?.installed?.usage ? native.installed.usage.total_gb : nativeModels.reduce((n, m) => n + (m.diskGb ?? 0), 0);
const totalGb = nativeGb + versions.reduce((n, v) => n + (v.diskGb ?? 0), 0);
+8 -6
View File
@@ -6,8 +6,10 @@ import { useStore } from '../lib/store';
import type { SoftwareUpdate, UpdateCheckResult } from '../lib/types';
import { nativeAction, useNative } from '../lib/native';
import { NativeProgress } from './NativeModels';
import { jobErrorText } from '../lib/native-job-diagnostics';
import { appVersionLabel } from '../lib/app-version';
import { fetchUpdateCheck, publishUpdateCheck, useInstalledVersions, useLastUpdateCheck } from '../lib/update-check';
import { shownEngineVersion } from '../lib/engine-version';
/** A check from an earlier day shows its date too, because the result is kept between visits. */
function checkedLabel(at: string) {
const when = new Date(at), sameDay = when.toDateString() === new Date().toDateString();
@@ -29,7 +31,7 @@ export function SettingsUpdates() {
const [error, setError] = useState('');
const request = useRef<AbortController | null>(null);
useEffect(() => () => request.current?.abort(), []);
const installed = component === 'strata' ? appVersion : native?.available ? (native.installed?.engine?.version ?? 'Not installed') : engineVersion;
const installed = component === 'strata' ? appVersion : native?.available ? (native.installed?.engine?.version ?? 'Not installed') : shownEngineVersion(native, engineVersion);
const active = native?.jobs.find(j => j.state === 'running');
const result = checking || error ? null : (fresh ?? last);
async function operate(kind: string) {
@@ -70,14 +72,14 @@ export function SettingsUpdates() {
const labels: Record<SoftwareUpdate['status'], string> = { available: 'Update available', current: 'Up to date', unavailable: 'Check unavailable', error: 'Check failed', 'no-releases': 'No release yet' };
return <div className="settings-version" data-update={update?.status}>
<Icon name={name === 'Strata' ? 'layers' : 'engine'} size={20}/>
<span><strong>{tr(name)}</strong><small>{tr(version ?? 'Version unavailable')}</small>{tr(!checking && update ? <small className="settings-version__message">{tr(update.message)}</small> : null)}</span>
<span><strong>{tr(name)}</strong><small>{tr(version ?? 'Version unavailable')}</small>{tr(!checking && update ? <small className="settings-version__message">{tr(update.message)}</small> : null)}{tr(!checking && name === 'Strata' && update?.details ? <details className="native-settings-diagnostic"><summary className="small">{tr("Details")}</summary><p className="small muted">{update.details}</p></details> : null)}</span>
{tr(!checking && update?.status === 'available' ? (native?.available ? <Btn size="sm" variant="primary" disabled={!!active || (name === 'Engine' && !native.installed?.models.length && !native.capabilities.componentUpdates)} onClick={() => setConfirm(name === 'Strata' ? 'update-strata' : 'update-engine')}>{tr("Update available")}</Btn> : update.releaseUrl ? <a className="st-btn st-btn--secondary st-btn--sm" href={update.releaseUrl} target="_blank" rel="noreferrer" title={tr("Open the release page to update this installation")}>{tr("Update available")}<Icon name="download" size={15}/></a> : <span className="settings-version__label">{tr(labels[update.status])}</span>) : <span className="settings-version__label">{tr(checking ? 'Checking…' : update ? labels[update.status] : version === 'Not installed' ? 'Not installed' : 'Installed')}</span>)}
</div>;
}
return <>
<div className="settings-versions" aria-busy={checking}>
{tr(versionRow('Strata', appVersion, result?.strata))}
{tr(versionRow('Engine', native?.available ? (native.installed?.engine?.version ?? 'Not installed') : engineVersion, result?.engine))}
{tr(versionRow('Engine', native?.available ? (native.installed?.engine?.version ?? 'Not installed') : shownEngineVersion(native, engineVersion), result?.engine))}
<div className="settings-update-actions"><Btn size="sm" icon="refresh" disabled={checking} className={checking ? 'settings-update-checking' : undefined} onClick={() => void check()}>{tr(checking ? 'Checking…' : error || result?.strata.status === 'error' ? 'Try again' : result ? 'Check again' : 'Check for updates')}</Btn><Btn size="sm" variant="ghost" onClick={() => setRollback(true)}>{tr("Roll back\u2026")}</Btn></div>
<div className="settings-update-inline" role="status" aria-live="polite">{tr(checking ? 'Checking for new versions…' : error ? <span className="inline-error">{tr(error)}</span> : result ? <><span className="sr-only">{tr(result.strata.message)} {tr(result.engine.message)} </span>{tr("Checked at ")}{tr(checkedLabel(result.checkedAt))}</> : null)}</div>
</div>
@@ -89,16 +91,16 @@ export function SettingsUpdates() {
<Btn variant="primary" disabled={component !== 'engine' || !native?.capabilities.rollback || !!active || (native?.instances.length ?? 0) > 0} onClick={() => void operate('rollback')}>{tr("Restore previous engine")}</Btn>
</div>
</Dialog>
{tr(active ? <NativeProgress job={active}/> : null)}{tr(native?.jobs.filter(j => j.state === 'error' && ['update-strata', 'update-engine', 'rollback'].includes(j.kind)).slice(-1).map(j => <p role="alert" key={j.id} className="small inline-error">{tr(j.error ?? j.summary)}</p>))}
{tr(active ? <NativeProgress job={active}/> : null)}{tr(native?.jobs.filter(j => j.state === 'error' && ['update-strata', 'update-engine', 'rollback'].includes(j.kind)).slice(-1).map(j => <p role="alert" key={j.id} className="small inline-error">{tr(jobErrorText(j, native.jobs))}</p>))}
<Dialog open={!!confirm} compact title={tr(confirm === 'update-strata' ? 'Update Strata?' : 'Update engine?')} onClose={() => setConfirm('')} actions={<><Btn onClick={() => setConfirm('')}>{tr("Cancel")}</Btn><Btn variant="primary" disabled={!!active || (native?.instances.length ?? 0) > 0} onClick={() => void operate(confirm)}>{tr("Update")}</Btn></>}><p>{tr("The native installer updates the installation. Model files and settings are kept. Unload native models before updating.")}</p>{tr((native?.instances.length ?? 0) > 0 ? <p className="small muted">{tr("Open Models and unload the running native models first.")}</p> : null)}</Dialog>
</>;
}
/** "Strata 0.2.2 · Engine 0.1.47": the app's own version first, the engine's beside it (About). */
/** "Strata 0.2.7 · Engine 0.1.52": the app's own version first, the engine's beside it (About). */
export function AboutVersions() {
const native = useNative();
const app = appVersionLabel(useStore(s => s.setup?.app.version), native?.version);
const engineVersion = useStore(s => s.engine?.engine.version);
const engine = native?.available ? native.installed?.engine?.version : engineVersion;
const engine = shownEngineVersion(native, engineVersion);
return <p className="settings-about-version"><strong>{tr("Strata")} {app ?? ''}</strong>{engine ? <span className="muted"> · {tr("Engine")} {engine}</span> : null}</p>;
}
Loaded 100 of 970 files, more files were not shown because too many files have changed in this diff. Show more