Compare commits

...
2400 Commits
Author SHA1 Message Date
Niko1221 5e8a4cca7d Release notes 0.2.7: update, tray and speed lines confirmed on the release packages; Strix Halo guide names the prompt used for the 4K row 2026-10-27 14:54:33 +01:00
Niko1221 a42cc3d7df Version 0.2.7: Terminal and app 0.2.7, engine and Control 0.1.52 (every Qwen and DeepSeek package, Metal included; both engines changed since 0.2.6)
Same 15 files as the 0.2.6 bump (issue template, AGENTS.md, CMakeLists.txt 0.1.52, desktop and host package files, Cargo.toml and the strata-desktop entry in Cargo.lock, SettingsUpdates comment, qa1-storage test, metal 0.1.52-metal, build_hip_linux.sh PKGVER 0.1.52, strata_cli PRODUCT_VERSION, the two version tests), plus the Strata-0.2.7-<file> names in the md files.
2026-10-27 09:13:22 +01:00
Niko1221 5a843fd841 docs: release notes 0.2.7 - two-card pipelined default (owner choice a), DeepSeek pictures by default, the shard 1 setup fix 2026-10-27 09:13:21 +01:00
Niko1221 e6d5d76010 Merge f-027-dsvis: DeepSeek reads pictures by default (owner yes), and setup accepts ISTA's re-published shard 1
The engine's own tower (319 MiB) through strata launch and setup (--vision off still turns it off); setup fetches
the pinned 972 MB mmproj; the server sends pictures to the tower. Checked on 4x R9700, 1x R9700 and a real
launch on 2x A6000 (download, pack, picture answer over HTTP); 0/10 stalls, KL unchanged. Also: a newer shard 1
from --model-path (ISTA's 2026-10-26 chat template) is accepted after its own size and SHA-256 check.
2026-10-27 09:10:53 +01:00
Niko1221 2526f2ff99 Merge two-card batch default (owner choice a): two NVIDIA cards pipeline by default
Pipelined groups beat the serial default at 2 and 4 streams on 2x A6000 (+17/+22%) and 2x A4000 (+7..12/+17..20%);
one stream equal, -4% on A4000 IQ3_XXS. STRATA_BATCH_GROUPS_NV_SERIAL=1 or STRATA_DEFAULTS=026 gives serial.
2026-10-27 09:10:53 +01:00
Niko1221 94d7aab293 setup: accept the DeepSeek shard 1 with ISTA's 2026-10-26 chat template from files already here
ISTA re-published (and patch_chat_template.py rewrites) the first shard of each set with a larger chat template, so a shard 1 on disk can be 11 KB
larger than the pinned revision's. setup refused it (size mismatch; the GGUF preflight does not read DeepSeek tensors, so it was not taken as a
whole own file). The pinned revision stays the download; the newer shard 1 is accepted by its own size and SHA-256 with --gguf-dir.
2026-10-27 07:18:19 +01:00
Niko1221 1bd17d59d9 launch: the model state's DeepSeek check tolerates a catalog without the strata table (test_model_folder_case) 2026-10-27 06:22:02 +01:00
Niko1221 c451698d6a DeepSeek pictures: the Terminal's pictures step knows the tower is part of the engine (no reader lookup, no GPU/CPU reader choice), and a DeepSeek start without pictures answers a picture request with its own 400 wording 2026-10-27 06:12:25 +01:00
Niko1221 1e617c3f64 DeepSeek pictures by default: setup fetches the vision tower's weights (mmproj-BF16.gguf, pinned) with the shared files, packs them with the model (pack_gguf --mmproj) and rebuilds a pack made without them; --vision no leaves all of it out; the picture-reader parts of the launcher's model state skip DeepSeek (its tower is part of the engine); the installed-config description says pictures on for a --vision on DeepSeek config; tests 2026-10-27 05:49:37 +01:00
Niko1221 9b177f3080 DeepSeek reads pictures by default: strata launch and setup pass --vision on (the engine's own 319 MiB tower, no separate reader), the server sends a picture to that tower as raw RGB in the file of a GENI line (serve/ds_pictures.py: decode with Pillow, both sides a multiple of 42, at most 1024 image tokens, the template's placeholder expanded to the span), --vision off keeps pictures off and the old 400; the plan, the catalog and docs/DEEPSEEK41.md say pictures on; tests for the size rule, the config switch, the prompt expansion and the server route 2026-10-27 05:41:48 +01:00
Niko1221 4c7bce3ad1 generate: the gated-proposal check reads use_df2 only where the DFlash2 drafter is built (the Metal build has none; #3416 broke its compile) 2026-10-27 05:26:53 +01:00
Niko1221 8dce24a0a1 docs: release notes - the two-card batch default marker uses the owner's labels (a) pipelined by default, (b) pipelined only when every expert fits, (c) leave serial; the serial against pipelined numbers are batch scheduling without drafting 2026-10-27 04:33:31 +01:00
Niko1221 12d0f0abe8 docs: release notes 0.2.7 - the engine items from f-027-gh-eng filled in (layer-major prefill on by default for a two-card CUDA split, pipelined drafts opt-in, sm_80 and sm_89 defaults, audit fixes, opt-in switches), markers closed except the owner's two-card batch choice 2026-10-27 04:32:24 +01:00
Niko1221 ac19fecc06 Merge release-notes update: route prior 1.5 wording with the agent-loop caveat 2026-10-27 04:28:46 +01:00
Niko1221 6ca60e9a0d Merge f-027-gh-eng: GitHub engine items for 0.2.7
P0 #3911 #1503; P1 fixes; file-02 items; 22 PR ports (authors kept) incl. #3779 (sm_80 defaults under
STRATA_DEFAULTS=026) and #4121 (STRATA_PIPED_MTP opt-in); audit fixes #3602 #3948 #3638 #3797 #2409; owner
decisions #3935 (DeepSeek sm_89 packages), #3844, #3453 (sm_89 tables on), #3976 (layer-major prefill on by
default for a 2-card CUDA layer split; revert 1e41fa269 to undo). Final gate: sm_60, SYCL, z440 sm_86 CPU 38/38
GPU 23/23, boston CPU 24/24 GPU 23/23, owner-PC MSVC CPU 36/36 GPU 20/20, snapshot 10/10 on z440 and boston.
2026-10-27 04:28:36 +01:00
Niko1221 c41bd44de7 docs: route prior 1.5, final verdict of the 80-task check on the RTX 3060 (no copy or code errors; agent loops may finish slightly less often, within the run-to-run spread), speed and the off switch 2026-10-27 04:28:28 +01:00
Niko1221 447460e8cc native experts: the window-invariant reference rows are used only where they match the linked ggml dot (#3416)
native_arithmetic_parity failed on an Intel i9-13900K (AVX2 + AVX-VNNI, ggml built native)
in five of its eight configurations: "gu=16 down=20 ff=96 nt=2 token=0". Not the VNNI row
path: STRATA_NO_AVXVNNI=1 changes nothing, STRATA_NO_IQ4NL=1 passes. The IQ4_NL down
reference (iq4nl_reference_rows) differs from that ggml's one-token dot on the scalar tail
block of a row with an odd number of blocks (ff=96 is three): that ggml does not contract
`sumf += d * sumi` into an FMA, the reference lets the compiler do it. On the other
machines tested the two agree, so it is a property of the ggml build, not of the CPU.
With an unfused tail in the reference all eight configurations pass on that machine.

The invariant path now checks each reference once against the linked ggml-cpu on synthetic
blocks before using it (iq4nl_reference_ok for IQ4_NL with 2, 3, 5 and 20 blocks;
iq512_reference_ok for types 18, 21 and 22 with SwiGLU, on AVX-512 CPUs). On a mismatch
it prints one line and the invariant path uses ggml's own one-token dot per token: slower
on such a build, never different. The Default path is not touched; only the opt-in
STRATA_SPEC_PROB=1 decode reaches this code.

Tests: native_arithmetic_parity, eight configurations, on the i9-13900K (4,866,048
comparisons, with the fallback line), z440 and the owner PC (see the series notes).
2026-10-27 04:18:06 +01:00
Niko1221 ec8e405909 batch groups: two NVIDIA cards pipeline by default; the serial group is opt-in (STRATA_BATCH_GROUPS_NV_SERIAL=1, STRATA_DEFAULTS=026)
0.2.1 made a layer split over exactly two NVIDIA cards with --batch and no --batch-groups run one serial group (measured
then on 2x RTX A4000, IQ3_XXS: +12..16%). With the 0.2.7 code the pipelined groups are faster at 2 and 4 streams on
every box and quant measured, so the default goes back to the pipeline.

Measured (tools/serve_load.py, --batch 4, 256 tokens per request, interleaved pairs, total tok/s, pipelined / serial):
- 2x RTX A6000, IQ3_S, all experts in VRAM: 1 stream 101.7 / 101.7; 2 streams 125.7 / 107.4 (+17%); 4 streams 171.4 / 141.0
  (+22%). The v0.2.6 binary on the same box: serial default 104.9 and 137.2, --batch-groups auto 124.6 and 169.6.
- 2x RTX A4000, Q2_0 pack (17037 of 24576 pairs cached, 4 clean pairs): 2 streams 80.2-82.5 / 75.0-75.5 (+7..9%, 4 of 4);
  4 streams 110.8-111.5 / 94.8-95.3 (+17%, 4 of 4); 1 stream 63.6-68.7 / 68.2-70.3.
- 2x RTX A4000, IQ3_XXS (13056 of 24576 pairs cached, 3 pairs): 2 streams 73.2-73.7 / 65.8-66.5 (+12%, 3 of 3); 4 streams
  100.5-100.7 / 82.9-83.6 (+20%, 3 of 3); 1 stream 58.4-60.5 / 60.7-62.6 (serial about 4% faster, 3 of 3).
The single stream runs on the solo path in both; only the A4000s show the 4%.

What changed: batch_groups.hpp has a pure rule nv_serial_wanted(env value, STRATA_DEFAULTS=026). STRATA_BATCH_GROUPS_NV_SERIAL
set to anything but 0 asks for the serial group, an explicit 0 wins over STRATA_DEFAULTS=026, and with the variable unset or
empty STRATA_DEFAULTS=026 gives v0.2.6's serial group. An explicit --batch-groups (a number or auto) still wins. Three or more
cards, AMD builds and --batch-mtp/--adapt-async handling are unchanged. The serial message names the opt-in.

Tests: batch_groups_test (the rule over the env values and 026, two cards pipeline, the existing serial cases). Real engine on
2x A6000 (IQ3_S), no --batch-groups: the engine says "2 groups of 2 slots" / "2 groups of 1 slot"; with 4 and 2 slots, greedy,
greedy+seeded sampled, slots finishing at different lengths and a cancel in the middle of a draft, every slot's tokens equal its
solo decode and the pipelined build's. A long-prompt slot differs from the pre-merge base build in the same way with the flag
and without it (it is the parent's prompt path, not this change).

Not covered: no run on AMD or on three cards; the 4% single-stream cost on the A4000s is measured on one box.
2026-10-27 04:17:17 +01:00
Niko1221 5ac66e5005 release notes 0.2.7: layer-major prefill (on by default on a CUDA layer split) and pipelined drafts filled in 2026-10-27 04:05:02 +01:00
Niko1221 e40843dade Merge f-027-int2 (c7c8c0d0a) into f-027-gh-eng
# Conflicts:
#	docs/release-notes-0.2.7.md
2026-10-27 04:04:41 +01:00
Niko1221 5cfa395476 Merge f-027-gh-eng-pm (#4121 MTP drafts in pipelined slots, opt-in STRATA_PIPED_MTP) 2026-10-27 04:04:21 +01:00
Niko1221 f656a721a6 docs: STRATA_PIPED_MTP on 2x RTX A4000 (Q2_0): =1 is the setting to use; =2 is slower than no drafts at 4 streams there (#4121) 2026-10-27 04:02:38 +01:00
Niko1221 42890ca79f metal: stubs for the layer-major prefill kernels (moe_shared_init, moe_scatter_add): the Mac build shares prefill.cpp, the walk is never taken there (#3976) 2026-10-27 03:29:12 +01:00
Niko1221 37e4832ab3 layer-major prefill: on by default for a CUDA layer split, STRATA_PREFILL_LAYER_MAJOR=0 and STRATA_DEFAULTS=026 turn it off (#3976)
THIS COMMIT ALONE MAKES THE WALK A DEFAULT. Drop it and the layer-major prefill stays
an opt-in (STRATA_PREFILL_LAYER_MAJOR=1) with the chunk-major path as the default.

Rule (include/strata/core/sm120_series.hpp, pure, tested in src/core/arch_defaults_test.cpp):
- prefill_layer_major_default(hip, stages, v026): on for two or more stages on CUDA,
  off on one GPU, in a HIP build, and under STRATA_DEFAULTS=026 (qwen_defaults_026());
- prefill_layer_major_resolve(env, ...): STRATA_PREFILL_LAYER_MAJOR=0 / 1 (any text
  atoi reads) wins in both directions, over the rule and over STRATA_DEFAULTS=026.
Every decline of the earlier commits stays and runs in lm_prepare as before (HIP,
image prompts, DFlash2 taps, KVA, stop_at_chunk, a pending force_chunk, a peer GPU,
APR / NVFP4 / WMMA / fused layers, STRATA_EXPERT_NIBBLE, no VRAM, no pinned RAM):
such a prompt reads chunk-major with nothing but a log line. When the walk is on by
default (nothing set) the VRAM margin is 512 MiB instead of 96 and the super-chunk
target 32768 tokens (the size measured) instead of 8192; "a short prompt" is not logged.
STRATA_PREFILL_LM_TOKENS and STRATA_PREFILL_LM_MARGIN_MIB still win.

Measured (z440, 2x RTX A4000 16 GB, Qwen3.8-Flash-Next IQ3_XXS, experts streamed, 5
interleaved pairs, time to first token, medians): 8K 2.8x, 20K 4.8-5.7x, 32K 5.1-5.5x
faster (141 s to 26-28 s); decode 38.6 vs 38.4 tok/s; needles 12 of 12 found in both
arms. The speed-up is larger than the report's 2x because --prefill auto picks 512-token
chunks there. The first-token KL numbers are in the gate notes (3976-kl.sh).

Not covered: one GPU (stays opt-in), HIP, SYCL, Metal, three or more stages, a card
with more VRAM than the pack needs (where the experts are VRAM-resident and the
chunk-major path has little to re-read).
2026-10-27 03:28:36 +01:00
Niko1221 1c2efd789a Merge f-027-gh-eng-d again (9d85432b8: QFUSE one-token self-commit) 2026-10-27 03:12:14 +01:00
Niko1221 0d4b2cdd43 Merge f-027-gh-eng-lm up to 0c7de9868 (#3976 layer-major split prefill, opt-in: STRATA_PREFILL_LAYER_MAJOR=1) 2026-10-27 03:12:05 +01:00
Niko1221 789d83903b STRATA_QFUSE=1: a one-token window did not self-commit, but commit() skipped the commit graph as if it had (#3948)
Problem. #3948: with STRATA_QFUSE=1 the verified text changes and MTP acceptance drops, on CUDA. The QFUSE kernels are bit-exact
(gr_parity, gdn_parity, 10,500 random windows each on sm_86 and sm_120, after 91a4b8a67), yet the engine's greedy text still differed
from QFUSE off on every prompt tried. Model run, l620 (RTX PRO 5000, sm_120), Qwen3.8-Flash-Next IQ3_S, MTP --spec 6, identity-gate
flags, 4 prompts of 4K and 12K tokens, 96 greedy tokens: QFUSE on differed from off on 4 of 4 (the off-vs-off control was identical).

Cause. A window of one token advances the GDN state and the indexer tail itself ("self-commit") so that commit() needs no commit
graph. Verifier::record_window decides this with
`T == 1 && !always_publish_ && !batch_rec_ && !g_qfuse() && one_token_self_commit()`: with STRATA_QFUSE the fused GDN output
kernel takes the q8_1 destination and the window does not self-commit. Verifier::commit() skipped the commit graph with
`last_t_ == 1 && !always_publish_ && one_token_self_commit()`, without the QFUSE term. So with QFUSE on, every accepted one-token
window left the recurrent state (and the indexer tail) one token behind. Windows of one token are the ordinary case when the drafter
offers nothing, which matches the report: fewer drafts offered, lower acceptance, different text, and an unchanged text where
no window of one token occurs. On HIP (where QFUSE is the default on gfx1151) one_token_self_commit() is off unless
STRATA_ONE_TOKEN_COMMIT=1, so the two sides agreed there.

Change. commit() takes the same condition: `... && one_token_self_commit() && !g_qfuse()`. Without STRATA_QFUSE nothing changes.

Result (same l620 run). QFUSE on with this change against QFUSE off, same build: logits md5 and the 96 greedy tokens byte-identical
on all 4 prompts (IDENTITY PASS); the build without the change differs on all 4. The default path (QFUSE unset) of the merged head
against the f-027-int2 head (2a2264c21): first-window logits and greedy tokens byte-identical on the same 4 prompts.

Not covered. Serve answers and draft counters (3948-qfuse.sh step 2, SERVE=1), sm_86, and the reporter's 40-prompt set.
2026-10-27 03:11:15 +01:00
Niko1221 9c624c75a9 Merge f-027-gh-eng-d (#3602 #3948 #3638 #3702 #3797 #2409 #3844: audits, fixes and tests)
# Conflicts:
#	src/core/verify.cpp
#	src/kernels/cuda/fused_gr.cu
2026-10-27 02:48:36 +01:00
Niko1221 8dd297d0f1 PLE table: STRATA_PLE_CHECK=1 names the first row that decodes to an implausible value and says whether the delivered bytes are the file's (#3602)
The report asks for the first data mismatch between the GGUF row bytes and what the direct reader delivered, and for an early
check of the gathered values; neither exists. Its trace stops at "the embedding has 96 non-finite values" without saying whether
the reader gave a wrong row or the table holds one.

With STRATA_PLE_CHECK=1 (a tester switch, off by default; one static flag read per gather call when off) every direct-mode decode (read_row, collect,
gather_batch, also from the prefetched slots) checks the row's 160 values. For a value that is not finite or beyond 1e5 (an IQ4_NL
row cannot decode beyond 8.3e6; real PLE values are orders smaller) it re-reads that row from the file with a buffered read and prints
one line to stderr (at most 8 per table): the row, its file offset and page offset, the value, and either "the delivered bytes ARE the
file's bytes" or "THE DELIVERED BYTES DIFFER FROM THE FILE from byte N" with the first 8 bytes of each. That separates a reader
fault (the bug) from a damaged table. gates/3602-ple-direct.sh sets it in its direct arms.

Checked: ple_reader_test --selftest on z440 with the switch on prints the "ARE the file's bytes" line for the synthetic table's random
rows (which decode to huge values); with it off the output and the bytes are unchanged (ple_reader_selftest passes).
Not checked: the "DIFFER" branch has no natural trigger (that is the point); it was read, not run.
2026-10-27 02:10:24 +01:00
Niko1221 7d1016f3f4 QFUSE: the fused q8_1 producers divide the way the separate quantiser does; they gave a different code at about one element in a million (#3948)
Problem. #3948: with STRATA_QFUSE=1 the verified text changes and MTP acceptance drops (RTX 3080, sm_86). The report suspected the
group counter or the nat_xq_ offsets of fused_gr.cu; the audit and the new two-group test (previous commit of this stream)
rule those out. What the new test found instead, on an RTX 5070 (sm_120, CUDA 13.0, MSVC), over 10,500 random windows of two groups: in
about 1 run of 400 one q8_1 code of the fused image differs by exactly one from native_quantize_q8_1's, in every mode (group-relative,
window-indexed, direct, graph replay). The scale/sum pair and the other codes agree.

Cause. native_mmvq.cu, which holds native_quantize_q8_1 (the quantiser every consumer of a q8_1 image was written for), is
compiled with --use_fast_math (CMakeLists.txt, the native_* list), so its `amax / 127` and `x / d` are the fast full-range division
(div.full.f32, up to 2 ulp off). fused_gr.cu (the hyper-connection read's tail, gr_up_q32_kernel) and verify_kernels.cu (the GDN
output norm, gdn_q8_1_store) are not in that list and divide with IEEE rounding. roundf(x / d) then lands on the other side of a .5
at the rare element whose quotient is next to a boundary. One changed activation code per ~10^6 values is about one per few
hundred tokens at 80 hyper-connection reads of 2560 values per token: enough to move a near-tie in the verify, and the
draft (which quantises with the separate kernel) disagrees with the verifier a little more often, which lowers acceptance. That
matches the report (acceptance 0.755 against 0.79, different text, and an unchanged text where no draft is made).

Change. include/strata/kernels/q8_1_finite.hpp: q81_div(a, b) is `a / b`, except in a translation unit that defines
STRATA_Q81_FASTMATH_DIV (CUDA device code only, never HIP), where it is the same PTX division the fast-math file gets (div.full.f32).
q8_1_quant and the scale use it. fused_gr.cu and verify_kernels.cu define the macro; nothing else does, so every other kernel,
the separate quantiser and every HIP build compile to what they did. STRATA_QFUSE is off by default on CUDA (it is on by default on
gfx1151, which is HIP and unchanged), so the default path is bit-identical.

Result (RTX 5070, sm_120): gr_parity's two-group window test: 10,500 runs with 1,500 random inputs for each T, all modes, zero
differences (before: dozens); the whole of gr_parity, gdn_parity and gdn_parity with STRATA_GDN_SPLIT=1 pass.

Tests. gr_parity: the two-group test prints the first differing bytes (block, field), and STRATA_GRP_REPS=N / STRATA_GRP_KEEP_GOING=1
repeat it longer and count every failure. gdn_parity: the q8_1 destination check uses its own stream (native_quantize_q8_1 refuses the
null stream, which crashed the test on Windows), unbuffered stdout and STRATA_GDN_PARITY_TRACE=1.

Not covered. sm_86 (the report's card) has not run the new test yet (z440 was reserved for other jobs); the cause is not
architecture specific. The model run (gates/3948-qfuse.sh) is still owed: QFUSE on against off must now be byte-identical, and the
draft counters equal.
2026-10-27 02:02:46 +01:00
Niko1221 e0d47034df Presets: a value outside the range the tuner tries is kept with a warning, not dropped (#3844)
Problem. #3844: --spec 5 measured +6.1% decode on an RTX 5060 Ti, but a saved preset that carries it was left out entirely
("outside the safe values [3, 4]"), and an imported one was refused. The tuner's BOUNDS are the values it tries; they are
not a finding that a value is harmful. The never list (--spec 6 and wider: 4% to 16% slower, --expert-cache 0, ...) is.

Change (Terminal / Control; touches strata_control/tune_knobs.py, strata_control/presets.py, strata_cli/tune_commands.py and
serve/server.py).
- tune_knobs.check_setting_parts(args, env, ctx) returns (never, bounds). check_setting is the two lists together, in the
  order it had, so the tuner still skips --spec 5 and its tests are unchanged.
- presets.overlay applies a preset whose only problems are outside BOUNDS and returns info["warnings"], one short line each
  ("--spec 5: not tried by the tuner, kept because it is your setting"). A never-list setting still means "not applied".
  presets.create and import_preset refuse only never-list settings.
- serve/server.py prints each warning as "[strata] warning: ..." when the preset is applied; strata_cli tune_commands
  _effective adds "warnings" to its payload.

Tests. tests/tuning/test_presets.py: a preset with --spec 5 is saved, applied (the flag becomes 5, the input config is not
modified), warns once, and an in-range preset has no warnings; the never list still refuses --spec 6; import of --spec 5 works.
tests/tuning/test_autotune.py: --spec 5 stays in the tuner's refused list. python -m pytest tests/tuning: 274 passed;
tests/terminal -k "tun or preset or overlay": passed.

Not covered. The app's Details text does not show the warning yet (it reads the payload key above).
2026-10-27 02:02:46 +01:00
Niko1221 8c0301484e Conversation cache: skip a restore only when a card would have to create its copy stream and has no room for it (#2409)
Problem. #2408 / PR #2409: restoring a large parked conversation on a layer split ended the server with a CUDA out of memory
on a later card. The PR checks the free VRAM of every card against --vram-reserve-mib before a restore. Its own review
points out that a loaded card is always below that reserve, so the check would skip nearly every restore on a split (and
on one GPU). It is not ported.

What a restore allocates. conversation_snapshot_restore writes into the session's existing buffers (pools(st), the GDN
state, the indexer tails); refresh_residency only launches kv_stream_reset or copies into the existing ring. The elastic K/V
growth (kvg_ensure) allocates, but only on one GPU. The one thing a restore can create on a card is the calling thread's
private copy stream for it (conversation_copy.hpp private_stream), which it takes from the pool of finished threads when
there is one (#3350). A failed restore is fatal by contract (the session may be half written). So the need is one stream
and one event, and only for a card whose stream does not exist yet (a snapshot loaded from disk, or a restore on a thread
that never copied on that card).

Change. conv_cache.hpp: kRestoreFirstUseMib (64) and restore_fits(free, stream_ready, n, need): a restore is refused only
for a card that has no copy stream yet and less than 64 MiB free. conversation_copy_stream_ready(device) says whether a
card has one. generate.cpp: before the restore, each card of the run is checked; a snapshot that would not fit goes back to
the cache and the prompt is read again, with a log line. A card that has its stream is never refused, however full.
No sample (a failed cudaMemGetInfo) means the restore goes ahead as it did.

Tests. conv_cache_test: the predicate (cards with a stream are admitted at any free size; the edge at the need; no
sample / no cards / no need). conversation_copy_streams_test: copy_stream_ready before and after the first copy, in the pool
after a thread ends, and the measured cost of a card's first copy stream and event (printed; the test requires under
16 MiB, so 64 MiB leaves a wide margin). conversation_snapshot_test: after the first restore, six more restores of two
images leave the free VRAM where it was.

Not covered. The pure predicate and the generate.cpp code build on z440 (CUDA 13.0); the GPU tests are listed in the stream
notes. No model run: gates/2409-restore-guard.sh (a held-VRAM start on a layer split) shows the skip line and an
unchanged answer on the second turn.
2026-10-27 02:02:46 +01:00
Niko1221 2a9a67c174 Later-card VRAM: audit fixes for failed allocations, the reserve arithmetic as a tested helper, a per-card out-of-memory report (#3638 #3702 #3797)
Problem. On a layer split the later cards are left with very little VRAM after loading (#3702: 68 MiB on the last card; #3638:
268 and 666 MiB, then every batched prompt read of more than ~400 tokens ends non-finite after hours; #3797: the weight arena
fails with 320 MiB free and the message does not say who holds the rest). The window-graph eviction (12b0bce95) and the copy
stream pool (#3350) are in already; this is the rest of the code audit of what is allocated after load.

Audit. The table of every lazy allocation after load (what, when, size, how a failure is handled) is in
phaseA-tests/notes-027-gh-eng-d.md. Bugs found and fixed here, each small:
- verify.cpp capture / capture_batch and mtp.cpp finish_capture: the cudaGraphUpload and its sync were only printed. The
  upload is a warm-up, but a failure (no VRAM) stayed as the thread's pending error and the next kernel wrapper's
  cudaGetLastError ended the engine. The error is now cleared and the first launch uploads the graph.
- verify.cpp capture_commit and mtp.cpp finish_capture: the commit graph called cudaGraphInstantiate directly (no
  eviction of older layouts, unlike the window graphs) and reported the status of the end-of-capture call, which is success,
  so the message read "no error". capture_commit now goes through instantiate_evicting and reports the instantiate's status.
- instantiate_evicting instantiates into a local and sets the member only on success (what a failed instantiate leaves in
  its output is not specified, and a non-null member means "captured").
- prefill.cpp carve: after a failed take of the shared region the sub-allocators had no base and each cudaMalloc'd its
  buffer (hundreds of MiB in all) on the card that had just run out. carve now returns at once; the caller reports the
  failure as before.
- generate.cpp: a failed set_stage_helper / set_peer is reported and the run goes on, but its CUDA error stayed pending;
  cleared.
Found and not changed: relend() re-plans only the first participant (#3638 comment). Read: it runs only from the VRAM
command, which is off on a layer split, so it cannot touch a later stage's loan. Other candidates (ignored cudaEventCreate for
the pipelined fences, unchecked expert copies, cudaFuncSetAttribute flags set before success) are in the notes.

Changes for the report of #3797 and the reserve (#3702).
- include/strata/program/gpu_report.hpp: on a failed weight-arena allocation (first card and each stage) the engine now
  also prints, for every card of the run, the free and total memory and the largest other programs that hold memory on it
  (NVML, loaded at run time; where the driver does not list them it says so). Text only; the failure is the same.
  tests/program/gpu_report_test.cpp pins the text.
- include/strata/program/stage_reserve.hpp: the arithmetic of stage_room (what the split keeps free on a card) moved out of
  generate.cpp unchanged; tests/program/stage_reserve_test.cpp compares it with the old expression over a grid. The file
  also holds a PROPOSAL for a larger window term on a later card (96 MiB + 6 MiB per further window size + 48 MiB, 174 MiB
  for --spec 4) which nothing calls: the default placement is not changed. gates/3702-later-reserve.sh measures it.
- The low-VRAM warning of a later card now adds how many MiB the plan meant to keep free there and how far off that was.

Tests: gpu_report_test, stage_reserve_test (host only, built on z440).
Not covered: no model was run. What the ~700 MiB the plan does not count on the last card is (window graph sets, kernel code
loads) needs the run in gates/3702-later-reserve.sh.
2026-10-27 02:02:46 +01:00
Niko1221 11b0490228 tests: STRATA_QFUSE parity for a window of two groups and for the GDN q8_1 destination (#3948)
Problem. #3948 reports that STRATA_QFUSE=1 changes the verified text and lowers MTP acceptance on an RTX 3080 (sm_86); the
control (QFUSE off against off) was noisy too. The reporter read the code and suspects the per-group counter in
fused_gr.cu gr_q8_tail (not reset when a graph replays with another T) or the nat_xq_ offsets of a window of two groups.

Audit (no model run). The counter is reset by the block that finishes the group (atomicExch before it reads the 32 values),
so a launch leaves it at zero whatever T is; two blocks own a 32-column group in every kernel that calls gr_q8_tail, the
32-column kernel (gr_up_q32_kernel) quantises on its own without a counter; the fence pattern is the usual
threadFenceReduction one. The two destinations are consistent with their consumers in verify.cpp: the attention half's
xq_ is indexed from the group's first token (the projections read it from row 0), the expert half's nat_xq_ by the window's
token (the experts read it from tb). The GDN side (verify_kernels.cu gdn_q8_1_store, rows from t_out_begin) matches its
consumer too. I found no defect by reading. One thing the tests below will show if it matters: nvcc ignores the
`#pragma clang fp contract(off)` in gdn_q8_1_store, so on CUDA the ds.sum field could differ by one ulp from the separate
quantiser (the q8 bytes would not).

Tests (synthetic data, no model).
- gr_parity: the engine's G = 2 call pattern - the window split at (T + 1) / 2, two fused_gr_read_multi calls in a row on
  one stream with the same counters, group-relative destinations (xq_) and window-indexed ones (nat_xq_), direct and as a
  graph replay; fresh random inputs for every one of 200 repetitions for each T = 2..8; after every run both groups' q8_1
  bytes equal native_quantize_q8_1 of `mixed`, nothing outside the written ranges changed, and all counters are back at zero.
- gdn_parity: gdn_step_norm_multi with a q8_1 destination for every T and t_out_begin (verify and commit calls): the bytes
  equal native_quantize_q8_1 of the output rows, the outputs and the state are the ones of the call without it, and
  nothing is written past the rows. A second ctest, gdn_parity_split, runs all of gdn_parity with STRATA_GDN_SPLIT=1 (the
  4-blocks-a-head step, which has its own output-norm kernel).

Not covered. The reporter's model run (QFUSE on/off, MTP, 40 prompts): gates/3948-qfuse.sh. The tests were built on z440
(CUDA 13.0, sm_86); their GPU run is listed in the stream notes.
2026-10-27 02:02:46 +01:00
Niko1221 ebe3383a3f PLE reader: a failed read no longer leaves the next ticket waiting for ever; stress test for the direct table reader (#3602)
Problem. #3602 reports a deterministic all-logits NaN with --ple-io direct on IQ3_S (Windows, two GGUF shards) where
--ple-io mmap passes; a diagnostic build found non-finite values already in the gathered PLE embedding of one token. The
bytes the model got were not the file's (the largest finite value, 7.9e6, is what a random fp16 scale gives an IQ4_NL
row). This commit audits the reader and builds the test the report asks for; it does not claim to have found that
fault.

Audit (no model run). src/ngram/ple_reader.cpp, src/platform/direct_file.cpp and the consumers (PleTable gather_batch /
prefetch_rows / collect, prefill's double-buffered ple_gather, verify's windows): rows are copied and cached under the
reader's mutex after a complete read, short reads are checked per row, straddling rows get a two-page read, the offset
arithmetic is 64-bit, the set-associative cache and the slot buffers are only touched under the lock, and prefill's two
host buffers are ordered by the ple_copied events. I found no way to hand out a row before its read completed or from a
stale buffer.

One real bug, in the error path. After a failed read the I/O worker parks for good (worker_loop), but issue() went on
queueing jobs, so the next collect() waited for them for ever. issue() now returns a ticket with no jobs once the reader
has failed, and collect() returns the error at once (and drops the ticket). Same change in the SYCL copy
(sycl/src/ngram/ple_reader.cpp).

Tests.
- src/ngram/ple_reader_stress_test.cpp (ctest ple_reader_stress): 7 table shapes (header 0/192/4000/4051, rows of 1, 90,
  110, 160 and 4096 bytes, with and without bytes after the table) x 15 reader configurations (io thread or caller,
  in-flight 1/2/3/64, row cache of 0, 16 and 100000 rows, injected completion delays, keep-alive), four client threads
  with one to three tickets outstanding collected in any order; every delivered row is compared byte for byte with a
  buffered read of the file; collected buffers are compared again later (a late write would show); rows on every page
  phase, the last rows, out-of-range rows; the reader after a failed read (the file is cut short under it); with --big a
  5 GiB sparse table with rows around 2^31 and 2^32 and at its end.
- table_is_consistent_under_concurrent_use in src/ngram/ple_reader_test.cpp: one PleTable used by a prompt thread
  (big gather_batch), a decode window (prefetch_rows, gather_batch) and single gathers at once, against the mapped reader.

Results (z440, Linux, g++ 13, reader + direct_file + test built alone): plain build with --big OK; ASan+UBSan OK. The
error-path test against the unchanged reader hangs (the test exits 3 after 20 s); with the change it passes. ThreadSanitizer
flagged a data race that was a mistake in the test (the delay changed from a second thread in the caller-thread mode,
which has no lock); fixed in the test.

Not covered. The NaN itself: it needs a run on the reporter's configuration (gate script gates/3602-ple-direct.sh). The
Windows run of the new test is listed in the stream notes.
2026-10-27 02:02:46 +01:00
Niko1221 ba8b2bcb9c Merge f-027-gh-eng-a (#3622 ARM64 / DGX Spark port, #3806 Gyro-L TQP5/6/7)
# Conflicts:
#	cmake/ggml_tq_overlay_test.cmake
#	src/core/verify.cpp
2026-10-27 01:59:31 +01:00
Kevin Read 38b06316ef Gyro-L: read the pair-trellis types TQP5/TQP6/TQP7 (ggml 147-149) on the CPU, CUDA and the MMQ prompt path (#3806)
Gyro-L (agentionai's 50 GiB file) stores its experts as a pair trellis: 128
values per block, a fp16 scale and 64 steps of K bits (K = 5, 6, 7), two
weights per step taken from the first pair of the TQ2_T codebook entry the
step's state selects. The engine refused the file because it did not know the
three type ids. This adds them next to the existing trellis types 144-146
without changing those.

What changed:
- gguf_reader.hpp / tools/gguf_reader.py: names and block sizes (42 / 50 / 58 bytes).
- tq_cpu.*, tq_states.hpp, tq_avx2.cpp: tq_supported / tq_kbits / tq_block_bytes
  for the new ids (the bytes per block are 2 + 8K for the pair layout, 2 + 4K
  for the others), a scalar pair kernel (the reference) and an AVX2 pair kernel
  that is bit for bit equal to it; AVX-512 uses the AVX2 pair kernel.
- iq_kernels.cu: TqInfo, the lane decode (tqp_states8, a byte window), the
  dot product, the decode-once Split, the fp16 dequantizer and the format lists
  for 147-149. They are reached only by a pack that holds these types.
- moe_mmq.cu, strata_ggml_tq_overlay.cmake, third_party/ggml-tq/tq.cuh,
  ggml-common-tq.h: the MMQ prompt path for the three types (overlay
  enumerators, traits, tile loader branch, instances).
- native_expert_parity: names tqp5-7 and skips its synthetic check, with a
  message, on a ggml that has no trellis traits.
- docs/GYRO.md: the "not supported" sentence now says how Gyro-L is read.

Changes to the contributor's patch:
- tq_cpu.cpp called the AVX2 pair kernel without the x86 guard the rest of the
  file has, which breaks the Metal build (it compiles tq_cpu.cpp without
  tq_avx2.cpp): the pair rows go through one helper that falls back to the scalar kernel.
- moe_mmq.cu named GGML_TYPE_TQP5-7 under STRATA_MMQ_TQ, but the public
  agentionai llama.cpp has the older trellis types and not these: a new
  STRATA_MMQ_TQP (set for the overlay and for a tree that has the tqp instances)
  guards them, so that tree still builds. The HIP overlay lists get the three
  instances too, and ggml_tq_overlay_test checks them (and sets the CMake
  policy its IN_LIST needs).
- native_expert_parity: the skip test compared a type-name pointer to null, but
  ggml_type_name of an id past the table reads past it (the test crashed on
  an upstream ggml); it compares against GGML_TYPE_COUNT.
- docs/GYRO.md keeps the plain description and leaves out the contributor's
  measured speeds (we have not repeated them).

Tests added: src/kernels/cpu/tqp_cpu_test.cpp (no ggml, no model: the scalar
kernel against a bit-by-bit reference from the format definition, every ISA
equal to scalar for 1-8 tokens and for gate/up rows, the type predicates for ids
0-299 against the old formulas), the pair types in tq_pascal_bench (fp16
dequantizer bit for bit against the definition, decode kernels within 3e-2,
decode-once == per-column), tools/test_gguf_tqp.py, and native_expert_parity
tqp5/6/7 registered when the ggml tree has the tqp instances.

Checked (no model): tqp_cpu_test passes on x86 (AVX2 on z440; AVX2 and
AVX-512 on the owner PC with MSVC); tq_pascal_bench on an RTX 5070 (sm_120,
MSVC build): 0 failures, the TQP dequantizer matches the definition in all
bits, and the checksum lines of TQ2_T/TQK6/TQK7 are byte-identical to the
build without this change. Builds: CUDA sm_86 (gcc), sm_60 (CUDA 12.9),
sm_120 (MSVC), SYCL, and the CPU files for aarch64.

Not covered: the contributor's own parity runs need a ggml tree that carries the
types (not public here), and the 50 GiB model run (load check, coherent
answer, 32K prompt, MTP, Gyro-S/M identity) is still to be done. Metal, SYCL and
HIP have no kernels for the types; HIP was read, not built.
2026-10-27 01:57:57 +01:00
Niko1221 1e57a924d0 Merge f-027-gh-eng-m (#3416 gated MTP: opt-in proposal law, window-invariant CPU dispatch, draft policy)
# Conflicts:
#	docs/DETAILS.md
2026-10-27 01:48:34 +01:00
Keyan Hu ca17d0f5cd spec: a gated request's lookup policy learns from committed rounds only, DraftPolicy choose() unchanged otherwise (#3416)
Problem (KeyanHu-git): the gated proposal needs the choice between the MTP window and a
lookup window to be fixed by the history, not by the draws of the rounds it will verify.
Freezing the policy for the whole request (the first commit) does that, but also freezes
the cost table, so a repeat-heavy request kept the window sizes it inherited. In the
author's diagnostic workload the frozen policy chose mean lookup windows of 5.87 and 3.38
on two repeat requests; with this change 4.31 and 4.05 (from the PR text; not
reproduced here).

Change: the request's copy of the DraftPolicy (source_policy) gets three calls, used only
when gated_proposal is on:
- observe_committed(): the round's accept and probe counts, applied at once. A lookup
  window counts as a probe of its size (tried_), so a size whose cost is only guessed is
  tried kProbes times, not every round.
- defer_cost(end, t, ms): the round's time is held. A time covers the window and the next
  MTP draft, so it depends on every Philox counter up to
  max(p + T - 1, p + a + 1 + S_mtp).
- apply_costs(committed): applies, in order, the held times whose end is behind the
  committed position, with the same update observe() makes (including the STRATA_SPEC_COST
  clamp), in the context bucket the round ran in; delayed_n_ keeps a held time from
  counting as a second probe.
choose() counts the probes as cost_n_ - delayed_n_ + tried_.

Port note: the DraftPolicy on the current tree keeps its costs per context bucket
(STRATA_SPEC_COST), so tried_, delayed_n_ and the held times carry the bucket. A gated
request does not run with STRATA_SPEC_COST (it falls back to the older rule), so its
copy stays in bucket 0.

Default path: unchanged. choose() reads two new terms; on any path that never calls the new
functions tried_ and delayed_n_ are +0.0 and cost_n_ - 0.0 + 0.0 == cost_n_ exactly.
Checked by driving the old DraftPolicy (namespace-renamed copy of f-027-gh-eng) and the
new one with the same random call sequences (observe, observe_chain, observe_probs,
set_context, seed_costs, set_robust, choose, chain, keep_draft; 400 sequences of 3000
steps): 25,983,538 bitwise comparisons of picks, costs and rates, 0 mismatches; one
observe_committed call makes the same harness report 60,528 mismatches. generate.cpp:
nothing calls the new functions when gated_proposal is false.

Tests: draft_policy_test (the PR's added assertions, on the bucketed class) and
spec_admission_test pass on z440 and on the owner PC.
Not covered: the sampling argument for the delay boundary (that a held time cannot steer
a window through a counter that is drawn again) is the author's; a unit test cannot show
it, only the distribution run (gate script 3416-spec-prob-dist.sh, arm NEWLK) can.
2026-10-27 01:45:36 +01:00
Keyan Hu 499e963593 native experts: window-invariant CPU arithmetic for STRATA_SPEC_PROB decode, Default dispatch unchanged (#3416)
Problem (KeyanHu-git): the rejection rule of the previous commit judges each row against
the target's probabilities p. For a native (IQ) pack the CPU experts round a token's rows
differently depending on how many tokens share its expert in the window (the grouped
kernels against ggml's one-token dot, #152). So p of one row moved with the window width,
and the same sampled text could depend on the draft length. On random blocks a grouped
Default call differs from the token alone in 700 to 12000 of about 18000 output floats
for the IQ gate/up and IQ4_NL down formats.

Change: NativeArithmetic { Default, WindowInvariant }, fixed for one pool batch.
native_gu_rows / native_down_rows become thin wrappers of one template with the arithmetic
as a parameter. The Default instance keeps every old branch (the new conditions are
compile-time true there). WindowInvariant gives each token the bits of the one-token call
on this build and CPU: kernels that already match it stay grouped (NVFP4, Trellis and
Hadamard rows, KQ256 when ggml runs its AVX2/FMA dot); the IQ rows that differ use
iq512_reference_gu_rows (types 18, 21, 22: the same decode, ggml AVX2's accumulation
order, one weight decode per group) or iq4nl_reference_rows (IQ4_NL down), and anything
else falls to the one-token dot per token. ExpertPool::run_split_multi_native takes the
arithmetic and expert_pool_dispatch_multi passes it; generate.cpp uses the invariant
pool only when the MTP drafts probabilistically (mtp.prob(), i.e. only with
STRATA_SPEC_PROB=1). Prefill, greedy decode and every other decode run Default.
The pool picks the entry point with one compare and a direct call per chunk.

Default path (STRATA_SPEC_PROB unset): unchanged, shown by running the old and the new
native_gu_rows / native_down_rows (Default) on the same random blocks: 61 format, width
and amplitude lines (IQ2_XXS/XS/S, IQ3_XXS/S, IQ4_XS, NVFP4, Q4_K, Trellis with and
without folded Hadamard, IQ4_NL / Q2_0 / Q5_1 / Q8_0 down; widths 1 to 8; row sub-ranges),
output hashes identical in 10 settings of the environment switches on z440 (AVX2) and
in 13 on the owner PC (Zen4, AVX-512, MSVC), including STRATA_NO_IQ512, STRATA_IQ_GATHER
and STRATA_IQ_MT_MIN=1. g++ codegen of the AVX-512 row kernels is not byte-identical for
5 of 87 functions (inlining and register allocation of the same source); the AVX-512
kernels were run only under MSVC.

Tests: native_arithmetic_parity, eight ctest configurations (default, AVX, KQ256,
IQ_MT_MIN=1, no512, ggml only, NVFP4 AVX2, NVFP4 ggml), registered in CMakeLists.txt and
sycl/CMakeLists.txt. It compares the invariant entry points at every group width 1..8
with the one-token calls, bitwise, for the IQ, NVFP4, Q4_K/Q5_1/Q8_0, Trellis and
Hadamard formats. All eight pass on z440 and on the owner PC; the existing CPU expert
tests (iq_avx2_parity, iq_avx1_parity, iq512_gather_parity, pool_tasks_test,
nvfp4_cpu_parity, router_dot_parity, q2_bitplane_parity) pass.
Not covered: an ARM CPU (not built here), an AVX-512 machine with fast gathers (Sapphire
Rapids), the layer-split call site, and any model run.
2026-10-27 01:45:35 +01:00
Keyan Hu f4612b1add spec: STRATA_SPEC_PROB=1 keeps the confidence gate in the proposal law, opt-in (#3416)
Problem (KeyanHu-git, refs #2028, #2591): with STRATA_SPEC_PROB=1 the default
--spec-min-p gate tests the token drawn from the draft head's distribution q, but the
verifier still judged the draft against the unfiltered q and treated a failed gate as a
plain sample from p. That shifts the output distribution: with p = (0.5, 0.5),
q = (0.875, 0.125) and threshold 0.5 it emits (0.5625, 0.4375).

Change: the gate is part of the proposal. Keep q's mass on the tokens that pass the gate
without renormalising (mu); a draw that fails the gate becomes NULL. An offered token D
is accepted with min(1, p(D)/mu(D)); a rejection and NULL draw from the normalised
positive part of p - mu; a position with no attempted proposal is a plain sample from p.
Serve and the CLI build the same packet (core/gated_proposal.hpp) and give it to the
verifier explicitly (Verifier::set_spec_proposal, sample_tokens_proposal). The explicit
call has no plain-p fallback: an invalid packet row is reported, not sampled around.
STRATA_SPEC_PROB_GATE=top keeps its normalised-q rule.

Default path (STRATA_SPEC_PROB unset): unchanged. mtp.prob() is false, so gated_proposal
is false, the new draft-result argument is null, GatedDraftLimit does nothing,
Verifier::run takes the old branch (set_spec_q clears the measure flag), and the
sampler kernel's old instance is the old code with the new branches compiled out.
Read hunk by hunk; no hunk runs a new branch or a new allocation per token. The
default-identity model run is written as a gate script and not run yet.

Port notes against the PR as submitted:
- Unsupported combinations (parallel or multi-device decode, --lookup-chain,
  --adapt-swaps, PCIe share, --spec-min-p not finite, STRATA_SPEC_COST, DFlash2,
  STRATA_SPEC_T_CYCLE, --spec-corrupt, Metal) used to be an error that ended the
  request or the server. They now warn once and keep the older rule, so setting the
  variable never stops a run. --adapt-swaps 96 and the PCIe share are on by default, so
  the exact rule needs --lookup-chain 0 --adapt-swaps 0 --pcie-frac 0.
- The lookup source of a probabilistic window is chosen by a copy of the policy fixed at
  the request start, without asking whether the MTP's first guess agrees. Learning from
  committed rounds inside a request is the policy commit of this series.
- The CLI loop on the current tree has the out-of-memory window retry, DFlash2 and
  STRATA_SPEC_T_CYCLE; the gated path does not use the retry (a narrower window would
  change the room the packet was built for). Serve keeps merged first windows
  (STRATA_ADMIT_MERGE_FIRST) at their length.
- docs/DETAILS.md: two short bullets instead of the PR's long paragraph (that paragraph
  no longer exists on this tree). bench/results and the PNGs are not ported.

Tests: gated_proposal_test and spec_admission_test (default, pick, top, exact-top-only)
pass on z440 (CPU, g++ 13) and on the owner PC (MSVC). On the owner PC's RTX 5070
(sm_120, MSVC build, synthetic data, no model) spec_proposal_parity, spec_verify_parity and
sampler_parity (default, one-block and old-kernel configurations) pass. Also built:
CUDA sm_86 on z440 and boston, SYCL on P1006 (engine and host tests). The Metal path is
not built here.
Not covered: a model-level distribution test (chi-square of the first tokens against
plain sampling) and the identity run with the variable unset. Both are written as gate
scripts (3416-spec-prob-dist.sh, 3416-default-identity.sh) and not run.
2026-10-27 01:45:35 +01:00
Niko1221 9ac278e124 docs: release notes - split serving (#4000) filled from docs/SPLIT_SERVE.md: opt-in, exact, slower than the layer split on two A6000, credit to MarcoTool 2026-10-27 01:40:17 +01:00
Niko1221 ccf18ea143 Merge f-027-split: split serving (#4000), opt-in, first version
Prefill on one card, decode on another, session handed over through RAM; exact token for token (state hash match at 4K
and 16K on 2x A4000 and 2x A6000). Nothing changes unless split_serve / --split-serve is set. On two alike cards it is
slower than the layer split on every workload measured (docs say so); it can only pay on dissimilar cards. Idea and
manual setup by MarcoTool (#4000).
2026-10-27 01:39:21 +01:00
Niko1221 3480bf2b8c Split serving: docs with the measured results, check tool no longer hashes the state in speed runs
Measured on 2x RTX A6000 (IQ3_S): slower than the layer split on every workload;
exactness checks pass (Q2_0 on 2x A4000, IQ3_S on 2x A6000).

Idea and manual setup by @MarcoTool in #4000.
2026-10-27 01:38:32 +01:00
Niko1221 597df5e601 docs: the KVA opt-in is split 36, final (the split 24 variant is a 0.2.8 study); release notes marker closed 2026-10-27 01:35:28 +01:00
Niko1221 1b89043157 docs: the 1.5 route prior check - all five tasks the grader failed were read: three were grader flags, two were the long-winded answers 2026-10-27 01:29:32 +01:00
Niko1221 129739440a docs: release notes 0.2.7 draft, audit fixes of the engine review and the decision markers brought up to date 2026-10-27 01:28:23 +01:00
Niko1221 8191cb91b4 docs: the route prior stays at 1.5 after the 61-task check on the RTX 3060 (DETAILS.md and the 0.2.7 release notes)
No copy, code-constant, JSON-spelling, retrieval or unanswerable defects (retrieval 12/12, JSON 3/3, unanswerable 9/9, extraction equal to off); two long-winded answers, which the runs with the prior off also show two or three of per run. Story +15%, code +37% against off.
2026-10-27 01:26:22 +01:00
Niko1221 fb8f7d9df6 docs: release notes 0.2.7 draft (every merged item and the owner decisions of 26-27 October, measured numbers per box, markers for what is still pending) 2026-10-27 01:25:46 +01:00
Niko1221 df20d77845 Docker docs, compose defaults, scripts and the file test name the 0.2.7 image tags (0.2.7-cuda13 and the other flavors) 2026-10-27 01:25:46 +01:00
Niko1221 9e20319069 layer-major prefill: declines on STRATA_EXPERT_NIBBLE builds (#3976)
f-027-gh-eng gained STRATA_EXPERT_NIBBLE (4-bit expert slots, the sm_60 experimental
build): only run_impl gathers such a layer for MMQ. The layer-major walk would read
the slot as GGUF blocks, so lm_prepare now declines when any layer is converted and
the prompt takes the chunk-major path. Compiled out of every build without
STRATA_EXPERT_NIBBLE_BUILT.
2026-10-27 01:13:13 +01:00
Niko1221 bd8a254453 Merge f-027-gh-eng into f-027-gh-eng-pm (#4121 on top of the #4010 pipelined adaptive rounds) 2026-10-27 01:12:47 +01:00
Niko1221 0959545e21 Merge f-027-gh-eng into f-027-gh-eng-lm 2026-10-27 01:12:00 +01:00
ferstar a4c41a92ee Build the engine for ARM64 (DGX Spark / GB10) without changing the x86 build (#3622)
The engine did not build on aarch64: the CPU expert path assumed the x86 AVX
kernels, the CPUID probes and the x86 spin/fence intrinsics. After this change
the strata target compiles on aarch64 (CUDA host code, native GGUF packs).
On x86 nothing is selected differently.

What changed:
- CMake: STRATA_CPU_X86 is decided from CMAKE_SYSTEM_PROCESSOR (an unset
  processor stays the x86 build). On aarch64 src/kernels/cpu/portable.cpp
  stands in for the AVX kernel sources; native experts run on ggml-cpu's NEON
  dot products. -ffp-contract=off on aarch64. STRATA_ISA_FLOOR is ignored with
  a warning there.
- include/strata/platform/cpu_relax.hpp: strata_cpu_pause / strata_store_fence
  map to _mm_pause / _mm_sfence on x86 and to yield / dmb oshst on aarch64.
  verify.cpp, pool.cpp, session.cpp, mtp.cpp and graph.cpp use them.
- expert_layout.cpp: the CPUID/XCR0 probes compile on x86 only and report no
  AVX on other CPUs.
- generate.cpp: --cpu-oracle-q8-0 is rejected on a non-x86 CPU.
- setup.py: nvidia-smi's "[N/A]" memory on GB10 (unified memory) takes the
  pool size from /proc/meminfo on ARM64 only; the CPU probe no longer needs
  the x86 "flags" line; setup warns that the ARM64 engine is compiled on the
  PC and which packs work.

Changes to the contributor's patch, found while moving it onto the 0.2.7 line:
- portable.cpp had no iq512x_rows / iq512x_supported / iq512_gather_on (added
  to the x86 kernels after the PR was written): the aarch64 link would fail.
- The Metal build (Apple silicon) shares these files. It keeps its barrier
  (dmb ishst), its plain router loop (its bf16_rows_dot_multi is a stub), its
  Metal start-up block in generate.cpp and its "unknown" CPU name.
- The CPU side of the GPU doorbell: strata_load_fence() after the spins that
  wait for a flag the GPU raised (dmb ishld on ARM, nothing on x86 and on
  Apple silicon), so the data written before the flag is not read early.
- A card whose memory nvidia-smi does not report stays unlisted on x86 (the
  new fallback is for ARM64 only).
- tests/terminal/test_setup_arm64.py.

Checked: with the x86 CUDA build (sm_86, gcc 13 -O3, z440) the objects of
verify, session, mtp, graph, expert_source, pool, expert_layout and generate
are byte-identical before and after (the x86 wrappers are macros for the same
intrinsics). Every C++ file of the strata target compiles for aarch64 with
clang, and the symbols of the CPU kernel library link up (syntax and symbols
only: no ARM machine, no ARM run). The Metal-shared files compile for
aarch64-macos with STRATA_GPU_METAL.

Not covered: no ARM hardware here. The tests that include x86 headers
(iq_avx2_parity and the other AVX parity tools) and the DeepSeek target do
not build on aarch64; the strata target does not need them. Memory ordering
on ARM is reasoned, not run. The Q2_0 and Gyro packs need the x86 kernels.
2026-10-27 01:11:53 +01:00
Niko1221 2651c4a96b Merge f-027-int2 (2a2264c21) into f-027-gh-eng
# Conflicts:
#	tools/calibrate.py
2026-10-27 01:11:52 +01:00
Niko1221 2579635138 sm_80 defaults of #3779 (wide dense dequant, MMVQ-IL sm_80 tables) follow STRATA_DEFAULTS=026: back to the 0.2.6 kernels and the shared table; explicit =1 and =0 still win; arch_defaults_test checks the rule; release-notes line 2026-10-27 00:58:42 +01:00
Niko1221 1e278a280d Merge f-027-gh-eng-n (#3779 sm_80 tuning: wide dequant, MMVQ-IL sm_80 tables and the opt-ins)
# Conflicts:
#	CMakeLists.txt
#	docs/MMVQ_IL_TABLE.md
#	src/kernels/cuda/native_mmvq.cu
#	src/kernels/mmvq_il_parity.cpp
#	src/program/generate.cpp
2026-10-27 00:57:59 +01:00
linxuhao 56d98f164e docs: the sm_80 defaults and the opt-in sm_80 switches of #3779, STRATA_GRAPH_PRIO and STRATA_MMVQ_IL_PF (#3779, #3712)
DETAILS.md gets two sections: the decode switches (GRAPH_PRIO, MMVQ_IL_PF) with the
two sm_80-only defaults named, and the prompt switches (PF_FUSED_NT / BATCH / F16 /
WS, GDN_TC, PLE_COMPACT / PLE_COPY_STREAM, EMBED_VRAM, DEQUANT_SIDE) with what is
bitwise and what is rounding-level. MMVQ_IL_TABLE.md gets the sm_80 row of the table
list and a paragraph on the hoisted-load tables (kIlPf_80 / kIlPf58_80, Q6_K only) and
on measuring a card with mmvq_il_parity --bench --emit-table --pf 0,3,4.

Rewritten from the PR's text: every figure is labelled as the reporter's (one CMP
170HX; we have no sm_80 card and measured none of them), and the prompt-switch
section is new (the PR documented only the decode switches and the conversation
cache).

Original docs by linxuhao (PR #3779, commit 6d6f0f39).
2026-10-27 00:52:53 +01:00
linxuhao 8b95bffb0b Decode: opt-in STRATA_MMVQ_IL_PF=1 hoisted-load, exact-integer interleaved GEMV kernels for Q6_K (#3779, #3712)
Problem: the interleaved small-batch matvec (native_mmvq_il) has one weight block
in flight per warp: its four `if (kbx < bpr)` sections each sit behind their own
branch, so on a GA100 (ncu: 432 GB/s on the qkv Q6_K matrix) the loads of a group
are not issued together.

Change, opt-in (every variant is bitwise native_mmvq): native_mmvq_il_pf_kernel
issues every weight load of a group (4 virtual warps x R rows) before any of the
group's dots, with the addresses clamped to the row's last block for a tail, and
with the ring variants the next group's loads before this group's dots. The
per-lane accumulation order is the plain kernel's. P2's forms replace
conversions by exact ones where the value is a small integer: (float)p becomes
p + 1.5 * 2^23 minus 1.5 * 2^23 for |p| < 2^22, and the saturating byte subtract
of Q6_K's codes (0..63, so nothing saturates) becomes two instructions. Q4_K,
Q5_K, IQ4_XS and Q6_K have the variants; Q3_K (the Q2_0 pack's type) keeps the
plain kernel.
- STRATA_MMVQ_IL_PF=1: the measured table (kIlPf_80 / kIlPf58_80: Q6_K only, read
  only when the card is sm_80; on every other card 1 does nothing);
  =2..6: one variant for every type and shape (A/B only, any CUDA card).
- Unset or 0: the plain kernels through the same launch code (the variant argument
  is 0, and the added code is one cached environment read and a switch).
- CUDA only; HIP has no interleaved path.

Tests: mmvq_il_parity --pf a,b,.. bit-checks each listed variant against
native_mmvq for every type and shape, T = 1..8 and rows 0/1/2/4, and with --bench
times them interleaved (PFBENCH lines) and prints the joint (variant, rows) table
in the source form of kIlPf_<cc>. New ctest entry mmvq_il_parity_pf runs it with
--pf 0,2,3,4,5,6 on whatever card is present.

Reporter's numbers (linxuhao, CMP 170HX): decode +1.4 % (two reference / on / reference /
on sets) over the plain kernels; with STRATA_GRAPH_PRIO decode +1.3 % in their
per-feature pair (4K prompts +3.9 %, 16K +1.2 %, 52K +0.2 % in the same pair, which
the reporter reads as noise). Q4_K, Q5_K and IQ4_XS variants tie the plain kernels
there. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 116992ab).
2026-10-27 00:52:53 +01:00
linxuhao c2d3fe5000 mmvq_il_parity: the L2-residency arms (--hot / --gap / --persist / --ab), the clock warm-up and MMVQ_IL_TMIN (#3779, #3712)
Harness only; no engine code. The bench part of mmvq_il_parity times the
interleaved matvec cold (24 copies of the weights rotated past the L2), hot (one
copy), hot with other weights' reads in between (--gap MB, what a window's GEMVs do
between a prefetch and its consumer), and with the timed weights in a persisting L2
set-aside (--persist); --ab runs the five arms per case in one process, because
the clocks ramp over the first ~100 ms of a run. A 150 ms warm-up of launches runs
before the first timed case, and MMVQ_IL_TMIN bounds T from below.

The parity part is unchanged: without --bench the same checks run in the same
order (every rows choice against native_mmvq, bitwise). The bench code is CUDA only
(the test is built with STRATA_ENABLE_CUDA); the persisting-L2 calls are left out
under STRATA_HIP_GFX906.

Original change by linxuhao (PR #3779, commit 1f4b8139).
2026-10-27 00:52:53 +01:00
linxuhao 0049fa73ff Decode window: opt-in STRATA_GRAPH_PRIO=1 gives the window graph's kernel nodes their stream's priority (#3779, #3712)
Problem: the decode window graph runs every kernel node at the launch stream's
priority, so the one-block indexer kernels of the side branch wait behind the
queued blocks of the big q GEMV, and the shared expert competes with the routed
experts.

Change, opt-in (bitwise by construction: no kernel changes, the same launches on
the same inputs; only the dispatch order moves): with STRATA_GRAPH_PRIO=1 the
side mixer's branch stream (GDN gates, QSA indexer chain) is created at the
device's highest priority, cs_ and side2_ in the middle, the shared expert's
stream at the lowest, and the window graph is instantiated with
cudaGraphInstantiateFlagUseNodePriority. If a driver did not copy the stream
priority into the nodes, the capture-time notes (the branch forks and the
priority streams' tails) set the attribute per node. One log line per captured
window prints the node priority histogram. A card whose priority range is 0..0
changes nothing.

CUDA only: the members and the three methods sit under !STRATA_USE_HIP &&
!STRATA_HIP_GFX906 && !STRATA_GPU_METAL; graph_prio_on() is false there.

Merged by hand with #3702 (the window-graph eviction): instantiate_evicting now
takes both keep_t and the flags, and the retry after freeing other window
sizes keeps the flags (the merged code would otherwise have dropped the node
priorities on that retry).

With the variable unset: every stream is created as before and the capture-time
notes return at once.

Tests: none of its own (a priority is a scheduling hint with no value to compare).
The window graphs are exercised by the engine; see the checklist for the
reporter's greedy-answer identity check.

Reporter's numbers (linxuhao, CMP 170HX, IQ3_S, --spec 3, 3K context, interleaved in
one session, each arm with STRATA_QIL=1): reference 125.3 / 126.6, GRAPH_PRIO
127.8 / 129.0, reference 125.0 / 126.8 tok/s, +1.8 %. Greedy 11 of 11 identical. In
the later per-feature pair together with STRATA_MMVQ_IL_PF: decode +1.3 %. Theirs, not
measured here.

Original change by linxuhao (PR #3779, commit 68c487c3).
2026-10-27 00:52:53 +01:00
linxuhao 0e63d59c01 docs: conversation cache copies on a slow host link, STRATA_CKPT_ASYNC and the park switches (#3779, #3712)
One section in DETAILS.md for the switches the four conversation-cache commits
add: what each copies where, its VRAM and host memory cost, that they stay off
on HIP, gfx906 and Metal, and the measured times.

Rewritten from the PR's section: the times are labelled as the reporter's
(linxuhao, one CMP 170HX on PCIe Gen2 x4; we have no sm_80 card and measured
none of them), the independent RTX 3090 data of bitaccesscomau is added, and the
pool row describes STRATA_CKPT_POOL_READY=N instead of the PR's change to the
meaning of STRATA_CKPT_POOL=N (STRATA_CKPT_POOL=1 is unchanged).

Original section by linxuhao (PR #3779, commit a04f4399).
2026-10-27 00:52:53 +01:00
linxuhao 85e3dde522 Conversation cache: opt-in STRATA_PARK_VRAM_MIB=N keeps parked conversations' K/V in VRAM in front of the host copy (#3779, #3712)
Problem: restoring a parked conversation copies its K/V from the host over the
link: 27-107 ms over a PCIe Gen2 x4 link.

Change, opt-in and only with STRATA_PARK_ASYNC=1 (CUDA only): a park also keeps
the whole K/V of its image in a VRAM slot of its own (64 MiB granules, up to N MiB
in all, only while 1.5 GB of VRAM stays free beside it); the background copy to
the host reads from the slot, and a restore of that image copies device to device
from the slot (about 1 ms) without waiting for the background copy. Each K/V
layer carries its own join-and-release token, so any order of destruction is safe
(the layers outlive their image when they are retained for the next park). Under
VRAM pressure the elastic K/V's growth releases every slot and the stage, then
retries once (both growth sites in generate.cpp).

With the variable unset the slot code is not reached: the budget is 0, a restore
finds dev_tag 0 and takes the host path, and the growth sites see
conversation_park_vram_release_all() return 0 and behave as before.

Tests: none of its own (it needs a card with the VRAM for the slot and a parked
conversation); see the checklist for the reporters' A/B/A conversation check with
STRATA_PARK_VRAM_MIB set.

Reporter's numbers (linxuhao, CMP 170HX): restore of a 3K conversation 28-47 ->
7-12 ms; outputs identical. The first restore of a run is slower (41.6 / 56.9 ms)
than later ones (6.5-12.6 ms) and they have not found why. Theirs, not measured
here.

Original change by linxuhao (PR #3779, commit bafca410).
2026-10-27 00:52:52 +01:00
linxuhao ca378670b2 Conversation cache: opt-in ready checkpoint pool STRATA_CKPT_POOL_READY=N, gathered small checkpoint copies STRATA_CKPT_GATHER=1, park traces (#3779, #3712)
Problem (reporter, CMP 170HX, PCIe Gen2 x4): a conversation switch's park still
paid 38-52 ms for the live state, because the STRATA_CKPT_POOL pool was empty at
every park (a switch takes two checkpoints), so the 118 MB GDN vector was
allocated fresh and page-faulted; and a checkpoint's 37 small synchronous
device-to-host copies queue behind the 4 MiB DMA pieces of STRATA_CKPT_ASYNC.

Change:
- STRATA_CKPT_POOL_READY=N (opt-in, 1..16, with STRATA_CKPT_POOL on): the pool
  keeps N ready checkpoints, pre-sized to the session's state sizes with their
  pages in, topped up on a background thread after any take (not with a layer
  split).
- STRATA_CKPT_GATHER=1 (CUDA, opt-in): the small running-state vectors (ple, and
  each layer's indexer tail, dead row and block position) are gathered device to
  device into one buffer and fetched with one copy into a pinned buffer.
- The park stage is allocated rounded up to 64 MiB (a growing conversation does not
  cudaFree/cudaMalloc it every time) and a VRAM-resident K/V skips the pointer
  attribute query.
- The background helpers (pool top-up, the checkpoint DMA, the park DMA) leave the
  host's pinned core (aux_cpus.hpp); no output depends on where a thread runs.
- A parked image whose live GDN state is still being copied joins before its
  buffers are freed (SavedConversation::live_pending).
- STRATA_PARK_TRACE prints the joins, the live-state vectors' readiness and the
  K/V sub-timers.

Differs from the PR as submitted: the PR made STRATA_CKPT_POOL=1 allocate two
ready checkpoints at startup (+236 MB of host RSS) and read STRATA_CKPT_POOL=N as
a count. That changes what an existing switch does, so here STRATA_CKPT_POOL=1
keeps its meaning (the pool holds dropped checkpoints only, nothing is allocated
ahead of time) and the ready pool has its own variable, STRATA_CKPT_POOL_READY.
A user of the PR's STRATA_CKPT_POOL=2 sets STRATA_CKPT_POOL=1
STRATA_CKPT_POOL_READY=2. The PR's one-checkpoint refill after a park with
STRATA_PARK_ASYNC is replaced by the top-up, as in the PR, and so is only active
with the new variable. Merged by hand into #2526's park_source.

With none of the new variables set: the stage size is rounded up only when the
park stage is on, the extra code is trace output behind STRATA_PARK_TRACE and
the join calls.

Reporter's numbers (linxuhao, CMP 170HX): park at a 3K context 64-75 -> 21-40 ms
(all park switches together, A/B/A conversation check identical, greedy 11 of 11
identical). Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 56088e71).
2026-10-27 00:52:52 +01:00
linxuhao f73b3c86a4 Conversation cache: opt-in STRATA_PARK_ASYNC=1 parks the K/V in the background and moves the dropped checkpoints (#3779, #3712)
Problem: parking the outgoing conversation copies its K/V device to host and
deep-copies its checkpoints before the next prompt can start: 213 ms at a 3K
context, 334 ms at 13K on a PCIe Gen2 x4 card (the reporter's CMP 170HX).

Change, opt-in and CUDA only (off on HIP, gfx906 and Metal):
- STRATA_PARK_ASYNC=1: conversation_kv_save copies the device K/V device to
  device into a VRAM stage (only while 1.5 GB stay free beside it; otherwise the
  save copies as before), and a background thread moves the stage into the host
  buffers in 4 MiB pieces through two pinned buffers. conversation_park_join
  waits for it: every reader of a parked image's K/V, the cache before it frees an
  entry, and the image itself (SavedConversation::pending) before its buffers go.
  ConversationBuffer::resize_for_overwrite leaves the added bytes unfilled for
  that copy; the buffer's segments use an allocator that value-initializes only
  when a fill value is given, and resize(n, value) is unchanged for every other
  caller.
- With it, the checkpoints this request drops right after the park (all of them
  when another state comes in, else the ones off this prompt's path) move into
  the parked image instead of being copied, and a background allocation refills
  STRATA_CKPT_POOL for the next save. Only the main session's checkpoints move,
  not an idle batch slot's (the park_source of #2526 serves both).
- conversation_snapshot_save takes dropped checkpoints' storage (the existing
  STRATA_CKPT_POOL) for its copies; with the pool off, nothing is taken.
- Unset: park_drops is null, no storage is taken, the K/V copies are the
  synchronous ones; the added work is the K/V size sum for the stage request and
  a few join calls.

Differs from the PR as submitted: the PR's join in conversation_file.cpp
(put_payload) is not ported; the SAVE/RESTORE handler of generate.cpp joins the
park first (see the previous commit, which carries the PR's 33a8146a fix).
Hand-merged into the park_source / ParkSource refactor of #2526.

Not covered: if the background copy fails after resize_for_overwrite, the
parked image keeps unfilled bytes; the failure is logged. A restore of such an
image would read them. No test of the park path with a failing copy.

Tests: the PARK_ASYNC path has no unit test in this commit (it needs a card with 1.5 GB free for the
stage); the reporter's A/B/A conversation check covers it, see the checklist.

Reporter's numbers (linxuhao, CMP 170HX): park 213 -> 64-75 ms (3K), 334 -> 68 ms
(13K); greedy answers identical on their 11 prompts and the A/B/A conversation
check. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 3e0c141c).
2026-10-27 00:52:52 +01:00
linxuhao 0f7224d168 Conversation checkpoints: opt-in STRATA_CKPT_ASYNC (background device-to-host copy) and STRATA_CKPT_VRAM_SLOTS (#3779, #3712)
Problem: on a slow host link a turn checkpoint copies the whole GDN state to
the host on the request path (118 MB at 1.64 GB/s on a PCIe Gen2 x4 card, 72 ms;
an independent RTX 3090 on PCIe 3.0 x16 showed 73 ms to 62 ms with the switch).
Restoring a parked conversation copies it back.

Change, both opt-in and CUDA only (off on HIP, gfx906 and Metal):
- STRATA_CKPT_ASYNC=1: the GDN state is copied device to device into a VRAM stage
  (one buffer the size of the state, allocated at the first checkpoint save, plus
  a pinned host buffer of the same size; if either allocation fails the save falls
  back to the synchronous copy), and a background thread moves it in 4 MiB pieces
  into the checkpoint's host vector. Every reader joins first and only for the copy
  it needs (restore, snapshot save and restore, the parked copies, the serve loop at
  each request start, the SAVE/RESTORE session handler).
- STRATA_CKPT_VRAM_SLOTS=N (with ASYNC): the last N saved GDN states stay in VRAM,
  tag-checked, only allocated while 1.5 GB stays free; a restore of a checkpoint
  whose tag a slot holds copies device to device.
- STRATA_TRACE lines carry a steady-clock timestamp; STRATA_PARK_TRACE prints the
  timing of the save steps.
- Unset: the code adds a mutex-guarded no-op join per checkpoint save, restore and
  snapshot, a tag field that stays 0 and a few clock reads; the copies are the
  synchronous ones as before.

Differs from the PR as submitted: the PR put a join into conversation_file.cpp
(put_checkpoint), which broke the conversation_file_test and
session_save_reclaim_test builds without CUDA and matched only the copied vector,
not the one the copy in flight writes, so a SAVE within one checkpoint copy after a
turn could write an incomplete GDN state (found by bitaccesscomau; fixed in the
PR's last commit 33a8146a). This commit carries that fix: conversation_file.cpp
stays as it is and the join runs first in the SAVE/RESTORE handler of
generate.cpp. The commit 33a8146a is therefore not a separate commit here.

One more join, found by running the new ctest entries: conversation_snapshot_sources (the SAVE
path of a session file) saves the running state with the background copy and the caller writes
it to the file at once, after the handler's join has already run. It now joins the copy of
that checkpoint before it returns; without it a session file written with
STRATA_CKPT_ASYNC=1 could hold an incomplete GDN state (conversation_snapshot_test failed
at "both files read" before the join).

Tests: conversation_snapshot_test now joins after each save and, with
STRATA_TEST_BIG_GDN=1 (a GDN state over 1 MiB, so the async path is taken), runs as
the new ctest entries conversation_snapshot_test_ckpt_async and
conversation_snapshot_test_ckpt_vram_slots: every checkpoint, capture and restore
must give the same bytes as the plain run.

Not covered: no test destroys a checkpoint while its copy is in flight (readers
join; the cache's eviction paths do not); the reporter saw 3 of about 35 engine
starts stop answering after a park with evictions, on the paths with the
switches off, and could not reproduce it in 15 reruns. The VRAM stage has no
headroom check of its own (an RTX 3090 with 147 MiB minimum free showed no failure
in 42 of 42 saves; the allocation failure falls back silently).

Reporter's numbers (linxuhao, CMP 170HX): turn checkpoint 71.4 -> 0.9 ms,
restore 114-126 -> 28-47 ms. Independent data (bitaccesscomau, RTX 3090 24 GB,
PCIe 3.0 x16, release defaults, 4 pairs): checkpoint save 73.2 -> 62.4 ms, tiny-prompt
time to first token median ratio 1.038, outputs byte-identical. Theirs, not measured
here.

Original change by linxuhao (PR #3779, commits 0d512925 and 33a8146a).
2026-10-27 00:52:52 +01:00
linxuhao 8c2a846947 Prefill GDN: opt-in STRATA_GDN_TC=1 runs the DeltaNet recurrence as a chunked scan on the tensor cores, sm_80 (#3779, #3712)
Problem: the prompt path's DeltaNet recurrence is a large part of the prompt time
of a GA100 card.

Change, opt-in and rounding-level (the output is not the FP32 kernels' bits):
src/prefill/gdn_tc.cuh holds the chunked WY-form recurrence with every product
on the tensor cores (f16x3 splits with power-of-two scaling, one block per value
head) and its prep kernel; kernels.cu dispatches it before the STRATA_GDN_CHUNKED
branch for prompt chunks of 128+ tokens when STRATA_GDN_TC is set.

Where it runs and where it does not:
- gdn_rec_tc returns "not supported" and the existing kernels run unchanged on a
  card below sm_80, with fewer than 48 SMs, or with less than 122 KB of
  shared memory a block (the kernel needs 124,944 B). That excludes sm_86, sm_89
  and sm_120 (99 KB opt-in): the code is compiled into those builds and never
  launched there. It runs on sm_80 and sm_90. A failed scratch allocation
  (60.3 MB a device) also falls back.
- Device code for cards below sm_80 compiles to trapping stubs behind
  STRATA_GDN_TC_SM80; the whole header sits under !__HIPCC__.
- Unset, the dispatch costs one cached flag test per call.

Tests: gdn_rec_parity gains the tc variant (T = 1..4099, 8192, 32768, six stress
inputs against gdn_rec_kh_kernel, gate 1e-4; "not on this card" where it cannot
run), --selftest-frag (each tensor-core helper on one warp against a double
reference) and --bench-tc. The new ctest entry gdn_rec_parity_selftest_frag runs
the helper self-test, which any sm_80+ card can run (77 below sm_80). The full
tc variant can only run on an sm_80 or sm_90 card.

Reporter's numbers (linxuhao, CMP 170HX): recurrence 105 -> 34 ms at 3.3K tokens
and 1,705 -> 547 ms at 52K; prompts +3.9 % (4K), +6.0 % (16K), +6.7 % (52K) over
the same base; within 2.3e-6 of the FP32 kernel in gdn_rec_parity; teacher-forced
KL against the bitwise base mean 0.0157, median 0.0011, max 0.098, top-1 100 %;
1 of 5 conversation checks identical and decode -2.6 % in their pair (they
think draft acceptance moves with the rounding; not checked). Theirs, not
measured here.

Original change by linxuhao (PR #3779, commit f1cabc99).
2026-10-27 00:52:52 +01:00
linxuhao 10c8cc629f Prefill fused IQ experts: opt-in f16 mma path STRATA_PF_FUSED_F16=1 and warp-specialized kernel STRATA_PF_FUSED_WS=1 (#3779, #3712)
Problem: the NT = 256 kernels still decode every 32 values to int8 and scale
the int32 products in FP32.

Change, both opt-in and both rounding-level (the output is not bit-identical to the
int8 kernels; they are the PR's "rounding-level" switches):
- STRATA_PF_FUSED_F16=1 (implies the NT = 256 set): the decoded weights times
  their block scales and the activations times theirs as F16, m16n8k16 F16
  products into FP32 accumulators. The k order inside a 16-value group is
  permuted the same way for both operands.
- STRATA_PF_FUSED_WS=1: native_ws_kernel, producer warps decode into an F16 ring
  in shared memory and consumer warps run the same F16 products (named barriers),
  so it has the F16 path's rounding. One block per multiprocessor; about 101 KB of
  shared memory at its defaults, so it is set up on sm_80 / sm_90 cards and
  says "not available" and keeps the 512-thread kernels on an sm_86 / sm_89 /
  sm_120 card (99 KB). Compile-time knobs STRATA_WS_PROD / _SLOTS / _NB / _KS / _HTAB.
- Both take a WideBatch like the NT = 256 kernels, so STRATA_PF_BATCH above 128
  reaches them. CUDA only (under !__HIPCC__); the Turing pass compiles the F16
  instances against __trap() stubs and never runs them (dev_info stops before the
  NT = 256 setup on Turing).
- With neither variable set the kernels, their launches and their shared-memory
  sizes are as before (the F16 template parameter defaults to false).

Differs from the PR as submitted: when STRATA_PF_FUSED_Q2N=1 is on as well
(#3809, after the PR was written) the WS setup also sets up the Q2_0 gate/up
instance.

Tests: prefill_fused_iq_test_f16 and prefill_fused_iq_test_ws (ctest) run the
fused-experts test (double-precision reference and MMQ) with the switch set. The
test now skips (77) when a requested NT / F16 / WS kernel set is not set up on the
device, so a card that cannot run WS does not report a pass for the 512-thread
kernels.

Limits of the test for these two arms only: with STRATA_PF_FUSED_F16=1 or _WS=1 the
test uses the limits it already has for STRATA_PF_F16 (F16 products): fused against the
int8-rounded model rel RMS 5e-3 (2e-3 otherwise) and against MMQ 5e-3 (1e-3 otherwise).
Measured with F16 on an RTX A4000 (sm_86, z440), over the 11 pairs: against the int8-rounded
model rel RMS 2.85e-3 to 2.90e-3 and worst row 5.9e-3 to 8.1e-3; against MMQ the same;
against the FP32 reference 1.438e-2 to 1.480e-2 with MMQ at 1.435e-2 to 1.478e-2 (the
fused path stays within 1.003x of MMQ's rel RMS and 1.03x of its worst row). The rounding is the F16 rounding of the
weights times their scales and of the activations times theirs (an 11-bit mantissa, about
2^-11 relative per operand), which the int8-rounded model does not have; it is not a
different summation order. The same arm on an RTX PRO 5000 (sm_120): up to 2.90e-3 (rel RMS), worst row up to 8.1e-3.
The default path and the NT=256 arm keep the old limits and their printed errors are
unchanged: with no switch set the test prints identical output (52 lines) from the base
tree and this tree, on an RTX A4000 (sm_86) and on an RTX PRO 5000 (sm_120).

Reporter's numbers (linxuhao, CMP 170HX): one MoE layer of 3,256 tokens 8.27 ms
(NT = 256) to 7.90 ms (F16) and 7.65 ms (WS); prompts +2..+6 % over the
256-thread kernels, 1 of 11 greedy answers changes. Teacher-forced KL against the
bitwise base on their 20-prompt set: F16 / WS mean 0.0062, median 0.00006, max
0.076, top-1 100 %. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 579cbfbb).
2026-10-27 00:52:52 +01:00
wannagoxuf 17878ac579 batch slots: MTP drafts in the pipelined groups, STRATA_PIPED_MTP=N opt-in with --batch-mtp (#4121, #3794)
Problem: a layer split with --batch pipelines the slots in groups, and the pipelined groups run no MTP drafts.
--batch-mtp with serial windows leaves all but one card waiting. A multi-stream deployment wants both.

What changed: STRATA_PIPED_MTP=N (with --batch-mtp) keeps the pipeline and drafts in it. A group's window carries
each of its slots' rows: the slot's token and up to N MTP proposals (at most --spec - 1; the first always, each
further one while the draft layer was at least --spec-min-p sure of the one before). The last stage judges each
slot's proposals on their own. The number of rows to keep is known only then, and by then the earlier stages run
other groups' windows, which reuse the scratch the commit replays from. So a small save graph copies those rows
right behind each window (Verifier::batch_launch_rows), and every stage queues the commit of the kept rows after
the verdict (Verifier::batch_commit_rows). The slot drafter catches up on its own rows and drafts the next
proposals as an asynchronous chain (MtpDrafter::chain_launch). Unset, nothing changes.

Changes on top of the PR as sent (review):
- The verifier window size is worked out where the verifiers are built, from the slots that fit and the final
  groups, not earlier from --batch as asked. Unset it is the old expression.
- STRATA_PIPED_MTP is decided again once the slots that fit are known. It says it is off, and why, when fewer than
  2 groups are left, when STRATA_BATCH_SELF_COMMIT is on, or when a group's window would hold more than
  kVerifyMaxT rows. The start line names groups and slots correctly.
- batch_groups::resolve keeps its old mtp_dropped rule; only mtp_piped is new, and it also applies with
  --batch-groups given (the PR's fix 1). The "--batch-groups 1" test case now expects "not dropped".
- --remote-expert-opt alone no longer blocks --batch-mtp on a layer split: the object it guards is only made with
  a helper expert cache (the PR's fix 2).
- docs/BATCHING.md rewritten with the numbers measured here.

Tests: batch_groups_test (the PR's cases, four "unset is as before" cases); builds on CUDA sm_86 (Linux), SYCL
(the sycl tree has its own verify.cpp and generate.cpp and does not include this code) and MSVC. Model runs:
see docs/BATCHING.md and the gate scripts 4121-*.sh.

Not covered: AMD (HIP) was not built or run; the draft layer on the first stage; helper expert caches.
2026-10-27 00:21:07 +01:00
Niko1221 1d45da3a8a Merge f-027-rp15: route-prior default 1.5 on discrete NVIDIA cards of 13 GiB or less (owner decision)
2.0 showed copy and code defects on the RTX 3060 (IQ3_XXS); 1.5 showed none in 19 tasks and keeps the speed
(story +15%, code +37% vs off). The 61-task check decides 1.5 vs a 1.0 fallback.
2026-10-26 23:30:18 +01:00
Niko1221 0afaaa399e Route prior: the 0.2.7 default on a discrete NVIDIA card of 13 GiB or less drops from 2.0 to 1.5 (owner decision 2026-10-26). On the RTX 3060 12 GB (IQ3_XXS) the copy and code check found real defects at 2.0 and none in 19 tasks at 1.5 or 1.0; 1.5 keeps story +15% and code +37% against off. The value lives in strata::kRoutePriorSmallCard (sm120_series.hpp); scope (CUDA discrete <= 13 GiB), STRATA_ROUTE_PRIOR=0, an explicit value and STRATA_DEFAULTS=026 are unchanged. arch_defaults_test, verify.cpp comments, DETAILS.md and the 0.2.7 release notes follow. 2026-10-26 23:26:18 +01:00
Niko1221 1e9678cb7c Split serving: check tool uses cold prompts per request and a single-card arm on the shared arena
Idea and manual setup by @MarcoTool in #4000.
2026-10-26 23:15:49 +01:00
Niko1221 b1bea3a17c Merge f-027-shs: STRATA_SH_STREAM off by default on Windows gfx1151 (owner decision)
+6.9% Qwen IQ3_S decode on Strix Halo Windows, prompts unchanged; token ids identical in 4 starts x 5 prompts.
Linux gfx1151 and other cards unchanged; STRATA_DEFAULTS=026 restores it. Docs label the DeepSeek Strix numbers by build.
2026-10-26 23:01:38 +01:00
Niko1221 f932e410e6 docs: Windows Strix Halo leaves STRATA_SH_STREAM off, with the token-id check; DeepSeek numbers say which build they come from
The Windows guide says what 0.2.7 does and how to turn the switch back on, backed by five fixed greedy prompts
(895 ids) run four times with the switch on, off, on, off: identical ids. STRIX_HALO.md and the 0.2.7 release notes
get a line. The DeepSeek 4K prompt figures of 2026-10-25 (78.7, 87.2) stay and are labelled with their build; a table
of the 0.2.7 candidate changes one thing at a time (chunk size, DS4_DEFAULTS=026).
2026-10-26 23:00:50 +01:00
Niko1221 c77292f09d gfx1151 on Windows: the shared-expert fork (STRATA_SH_STREAM) is off by default
Interleaved A/B on a Strix Halo Windows PC (Qwen IQ3_S, engine 0.1.51): chat decode 43.6 tok/s with the fork,
46.6 without (+6.9%), prompts unchanged. Linux gfx1151 and every other architecture keep their table.
STRATA_SH_STREAM=1 turns it back on, STRATA_DEFAULTS=026 keeps v0.2.6's table.
arch_default_env(arch, defaults026, windows) is a pure rule; arch_defaults_test checks both platforms.
2026-10-26 22:28:05 +01:00
Niko1221 e240850324 Docs: Strix Halo Windows guide, plain emphasis and a stray blank line 2026-10-26 22:24:30 +01:00
Niko1221 538acd098c Merge f-027-gh-ctl: GitHub setup, Docker, docs and bench items for 0.2.7
Ported PRs (authors kept) and our fixes, plus the owner's part-2 items: the restored TROUBLESHOOTING entries with a
link check, #3895 (no shared-expert fork calibration on Windows HIP with --kv-resident; a warning on a hand-set
STRATA_SH_STREAM=1), #3930 (docs), #3421 (Docker stops when models are present and MODEL_PATH is unset), #3570
(launch --show-key reads setup's config), #3311 (engine paragraph only) and #3029 (Strix Halo Windows guide,
re-measured on 0.2.7).
2026-10-26 22:24:08 +01:00
Niko1221andHaiYunChuang ee1a6efd5c docs/STRIX_HALO_WINDOWS.md: a Strix Halo Windows guide, measured again on 0.2.7 (#3029)
Structure from the guide in PR #3029 by HaiYunChuang (what the PC reaches, memory and the BIOS, the configuration, what is on by default, traps),
rewritten for the 0.2.7 install and with every number measured again: the Terminal and the gfx1151 engines built from f-027-int2 (3e468f5de), a
local signed test feed, strata launch as a standard test user on the NucBox EVO-X2 (Adrenalin 26.10.1), tools/strata_bench.py over a tunnel.
Qwen IQ3_S: 44 tok/s chat, prompts 755 / 821 / 792 tok/s at 4K / 16K / 32K. DeepSeek 2.00 bpw: 12 to 14 tok/s chat, prompts 214 to 342 tok/s.
An interleaved A/B of STRATA_SH_STREAM on Windows gfx1151 (three pairs): off is 6.9% faster in chat decode, prompts unchanged.
What launch chose and logged is quoted instead of a hand-made config. The first guide's tuning rows (engines 0.1.41 to 0.1.47) are not repeated.

Co-authored-by: HaiYunChuang <[email protected]>
2026-10-26 22:21:04 +01:00
Niko1221 f1dadd1710 fused_gr: the persistent hyper-connection read's barrier only calls __nanosleep on sm_70 and newer (the sm_60 build of f-027-int2 failed on it) 2026-10-26 22:07:55 +01:00
Niko1221 3cabb9e3fe Merge f-027-gh-app: GitHub app and Terminal items for 0.2.7
Part 1 fixes and 12 ported PRs (authors kept), and the owner's part-2 items: the Monitor readings strip with
local currency (#3628), approval notification + opt-in minimise to tray (#3477), opt-in done sound (#3586),
app port setting with a busy warning (#3402) and interface size up to 200% (#3970). Host tests: only the two
known base failures on Linux; connector-setup and the new tests pass on Windows.
2026-10-26 21:56:21 +01:00
Niko1221 3b908cc4f7 Tests: the new catalog is checked in nine languages, French included 2026-10-26 21:55:58 +01:00
Niko1221 0d0ec83b7d Merge f-027-int2 into f-027-gh-app 2026-10-26 21:54:30 +01:00
Niko1221 c17a6f4ce1 App: the Monitor's readings strip (energy since start, cost in the person's own currency, GPU fan and clock, CPU and SSD temperature) with the Electricity price setting (#3628, PR #3628 by hash13 as the starting idea); a sound when work finishes (#3586); desktop notification and tray, pinned port and interface size settings with their host routes (#3477, #3402, #3970); the gh-<lang> catalog in nine languages 2026-10-26 21:31:51 +01:00
Niko1221 902dc814d6 Desktop shell: a pinned app port with a warning when it is busy (#3402); minimise to tray with Tauri's own tray icon and a native desktop notification through the loopback bridge (#3477) 2026-10-26 21:31:47 +01:00
Niko1221 59b4be2a54 App: French for the four new DeepSeek card and 12 GB NVIDIA strings (fx80-fx83) 2026-10-26 21:19:31 +01:00
Niko1221 98e35984c6 Merge f-027-dscard: the DeepSeek card names the free disk the set needs (owner-approved wording)
# Conflicts:
#	apps/host/src/locales/fix-de.json
#	apps/host/src/locales/fix-en.json
#	apps/host/src/locales/fix-es.json
#	apps/host/src/locales/fix-ja.json
#	apps/host/src/locales/fix-ko.json
#	apps/host/src/locales/fix-pt.json
#	apps/host/src/locales/fix-ru.json
#	apps/host/src/locales/fix-zh.json
2026-10-26 21:19:14 +01:00
linxuhao dc390c3864 Prefill: opt-in STRATA_DEQUANT_SIDE=1 dequantizes the dense weights on a side stream into a double-buffered scratch (#3779, #3712)
Problem: Gemm::native dequantizes each dense weight to FP16 in front of its
GEMM on the compute stream, so the dequant never overlaps the kernels queued
ahead of it.

Change, opt-in (CUDA only, always off on HIP): with STRATA_DEQUANT_SIDE=1 the
prompt path takes a second GEMM scratch (64 MiB more of its region, counted in
bytes_needed and carried through relayout) and Gemm::set_dequant_side hands it
to the Gemm. native() then dequantizes the next weight on a non-blocking side
stream into the scratch the compute stream read last but one. Four events order
every write and read of each scratch (free -> dequant -> ready -> GEMM -> free),
so each product sees the same bytes as in the serial form. The row-sliced path
(a weight larger than the scratch) stays serial. Metal answers false and keeps
the serial form.

Differs from the PR as submitted: if the side stream, its events or the
setup fail, the PR failed the prompt-path init. Here one line is printed and the
dense weights are dequantized in front of their GEMMs as before. Unset, nothing
changes (gemm_dequant_side() is one cached flag; the scratch, stream and events
are not created).

Test: new gemm_dequant_side_parity (ctest, GPU, synthetic, 77 without a device):
eight Q4_K / Q5_K / Q6_K weights of the engine's shapes (one larger than the
scratch, so it takes the sliced path) run three rounds through Gemm::native
serially and with the side stream, with and without a spin kernel queued ahead of
each GEMM; every output must be identical bit for bit.

Not covered: dense weights read by the side stream are the resident ones; a weight
that arrived by an upload on another stream would need its own event (the prompt
path has none that do).

Reporter's numbers (linxuhao, CMP 170HX): measured with STRATA_PLE_COPY_STREAM and
STRATA_EMBED_VRAM, the four PLE / embedding / dequant switches together give
+2.2..+2.6 % prompts. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 741fc6ab).
2026-10-26 21:19:13 +01:00
Niko1221 03d709820d Merge f-027-app2: master switches in the app and Terminal settings
DS4_DEFAULTS=026 behind the DeepSeek speed-ups Off toggle, the 12 GB NVIDIA route-prior row, the Advanced-only
Qwen '0.2.6 behaviour' row (STRATA_DEFAULTS=026), and the app-ui Quit test fix. Terminal suites pass; host suite
failures on Linux also fail on the base.
2026-10-26 21:18:49 +01:00
Niko1221 663ec80abf mmvq_il: the sm_89 (Ada) decode tables are on by default on compute capability 8.9 only, STRATA_MMVQ_IL_89=0 and STRATA_DEFAULTS=026 turn them off (#3453)
Owner decision for 0.2.7: the tables of PR #3453 (the cells where an RTX 4070 and an RTX 4080 community run gave the same rows count, and
the one-column Q3_K row) ship on for sm_89 cards, untested on Ada by us. The output is bitwise the same with them off (a table only
changes how many rows a warp takes); no end-to-end gain was seen on a 4090 or 4080 in the report, so the reporter tests 0.2.7 and reports.

The rule is a pure function in sm120_series.hpp: mmvq_il_89_default(hip, cc, v026) is true for cc 89 only, false in HIP builds and under
STRATA_DEFAULTS=026 (defaults_026.hpp, as 38c9080db did for the other defaults); mmvq_il_89_resolve() adds the variable: "1" and "0" win,
unset or other text is the rule. native_mmvq.cu reads it once. arch_defaults_test checks cc 89 on, every other cc off, HIP off, 026 off,
"0" off, "1" wins over 026, the empty text. mmvq_il_parity expects the on-tables by default and the old ones with =0 or 026.

Docs: MMVQ_IL_TABLE.md, DETAILS.md and the 0.2.7 release notes (a line that says it is not measured on an RTX 40 card, with the off switch).
2026-10-26 21:11:03 +01:00
Niko1221 46a9cd87c7 Merge f-027-int2 into f-027-gh-eng (brings STRATA_DEFAULTS=026 and the other 0.2.7 integration work) 2026-10-26 21:09:26 +01:00
Niko1221 52ba0b7816 Merge f-027-i18n: French as the ninth app language and the Polish README
French catalogs for the app (3,528 strings + fix/tune/tools catalogs) and the served web page (247 keys),
README.pl.md from the current README (#3355, Jakub Luwierski), Polski in every README language row, and the
README.md language row's double-encoded names fixed. Tests: 21 host files 180 pass on p1006, Terminal and web
i18n checks on l620.
2026-10-26 21:03:59 +01:00
Niko1221 551f1cdc30 Merge f-027-gh-eng-k1 (#3867 #3751 #3453 #3943 #3820 #3752 #3455: Pascal and sm_89 opt-in kernel ports)
# Conflicts:
#	src/core/expert_cache.cpp
#	src/core/verify.cpp
2026-10-26 21:03:12 +01:00
Niko1221 2b9869eddf Web chat page: French (#3970)
serve/web/locales/fr.json (247 keys, plurals one/other, same terms as the app catalog) and the French line in serve/web/i18n.js. tools/web_i18n.py and serve/test_web_i18n.py (node and python) pass.
2026-10-26 20:59:54 +01:00
Niko1221 710b6a1bfe Merge branch 'f-027-int2' into f-027-i18n 2026-10-26 20:58:00 +01:00
Niko1221andJakub Luwierski 2ce07f8631 Docs: Polish README (#3355)
Ports the Polish translation from #3355 and the Polski entry in the language row of every README. The pull request translated the older long README, so README.pl.md is rewritten from the current README.md (logo, centred language row, DeepSeek speed tables including Strix Halo, Gyro, coffee button). README.md's own language row had garbled names (double-encoded text); it is written out correctly.

Co-authored-by: Jakub Luwierski <[email protected]>
2026-10-26 20:57:57 +01:00
Niko1221 52a558abbe App: French as the ninth interface language (#3970)
fr.json, fix-fr.json, tune-fr.json and tools-fr.json (3,654 strings, vous, plain short words), the Francais entry in the language picker, every locale test and gap check now covers fr, a french-catalog test (spacing before ? ! : ;, apostrophes, short labels, no leftover English), scripts/fr_sync.py to list and merge strings added later, and scripts/language-shots.mjs for screenshots of the app in one language.
2026-10-26 20:57:57 +01:00
Niko1221 5d9f479229 Merge f-027-web: the served web chat page for 0.2.7
Wide/narrow CSS (#3176), page translations with a language choice, opt-in KaTeX math (#2972, #2974),
sandboxed svg/html preview (#1628), Load/Unload in the header (#3550), and one chat-history design
(#1787, #2615, #3823, #4004). Tests: browser suite on the owner PC, node + pytest serve suites on boston
(457 passed, 2 skipped).
2026-10-26 20:57:41 +01:00
ruibeikaa 9067cc64d3 iq_kernels: GP100 4-bit gate/up expert slots, opt-in STRATA_EXPERT_NIBBLE=1 (#3455)
Problem: on a GP100 die the verify window's gate/up kernels for the IQ2_S,
IQ3_XXS and IQ3_S expert formats are not held by memory bandwidth but by each
block's serial chain (unpack the codebook indices, the emulated dp4a, the
16-lane reduction); the same kernel with every dot product removed still takes
about two thirds of its time.

Change: with STRATA_EXPERT_NIBBLE=1 the engine, after the expert cache is
filled and every expert is resident, transcodes the gate/up blocks of every
fully resident IQ2_S / IQ3_XXS / IQ3_S layer (n_embd 2560, n_ff 640) into slots
of 4-bit codes, 16 bytes per 32 weights, with the integer scales and fp16 d kept
as they are; the down rows stay GGUF. The decode gate/up reads those slots with
kernels that run the production integer chain and float tail in the production
order (R4S / R4P / R4SV / R4PX by format and group size); the prompt path's MMQ
gather rebuilds the GGUF blocks from a slot on the fly. At load every item is
round-tripped and every slot's rebuilt GGUF blocks are compared with the staged
blob before the cache is sealed (ExpertCache:🦭 a later fill_slot stops the
engine). New: include/strata/core/expert_nibble.hpp (slot contract, switches,
per-layer table), nib_gu_proto.cuh (STRATA_NIB_GU_PROTO=1 variant), the
transcoder, hash and rebuild kernels and the 4-bit gate/up kernels in
iq_kernels.cu, generate.cpp (the fill, the VRAM accounting, STRATA_EXPERT_NIBBLE_
RECHECK), iq_nibble_parity and native_expert_bench --nibble.

Review changes against the PR:
- An option the switch cannot work with (--peer-device, --kv-grow, --vram-elastic,
  remote caches, STRATA_MMQ_DIRECT, ..., a card that is not cc 6.0, a build
  without the kernels) now prints a warning and the start goes on with the usual
  GGUF slots, where the PR refused the start. What is known only later (an expert
  out of VRAM, a loan, the prompt chunk) still refuses, since slots are
  converted by then.
- The checks this switch adds to expert_source.cpp and prefill.cpp are compiled
  out of every build that has no 4-bit kernels, so the default builds add nothing
  to the hit split or the prompt loop.
- iq_nibble_parity gets --synthetic-only (every gate/up format x both down
  types on random experts, no model file), skips with 77 on a card that is not a
  GP100, and the synthetic and --header runs are ctests.
- The PR's docs section is rewritten for these changes and its measurements are
  labeled as the author's.

Default path (switch unset): the table is empty, nibble::slot_bytes(l) is
expert_layout().blob_bytes(l) and nibble_max_slot_bytes() is max_blob, every
sizing the PR touched returns what it returned before, and the new branches in
generate.cpp (guard_raw before the adaptive tier's copies, the profile fill's
test), expert_cache.cpp (one bool in the fill calls and admit) and verify.cpp
(a capture-time layout choice, compiled out without the kernels) are not taken.
Proof on the kernels: in the sm_60 build (CUDA 12.9) the SASS of all 446
kernels iq_kernels.cu had before is identical with and without this change,
as is that of the 603 in native_mmvq.cu, 29 in qsa_select.cu and 10 in
moe_mmq.cu; 21 kernels are new.

Tests: iq_nibble_parity --header passes on z440 and boston (the host half of the
contract; also a ctest). The GPU parts (--synthetic-only, a model shard) refuse a
card that is not a GP100, so they were only compiled here (sm_60, CUDA 12.9,
P1004: iq_nibble_parity and native_expert_bench build; the whole tree builds).
sm_86 build without the SM60 option (z440): exit 0, the 4-bit paths compiled out.

Not covered: no GP100 run. The model gate is
phaseA-tests/gh027/gates/3455-expert-nibble.sh: iq_nibble_parity --synthetic-only
and on a model's layers on a P100, then identical replies with the switch on and
off (a layer is converted only when every expert of every layer is resident, so
the rig needs the pack to fit whole), then the decode A/B. The contributor's
numbers (four GP100 dies, Swift 1.5 IQ3_S, 47 of 48 layers converted, +12.7 GiB
VRAM, decode 47.3 -> 51.9 tok/s, +9.7%; prompt reading 2.3-3.0% slower; replies
of 15 requests equal) were not re-measured, and the enabled path was written
against an older base than this branch's.
2026-10-26 20:52:54 +01:00
ruibeikaa 5dbd8774d8 prefill: a resident layer's experts weight-stationary on GP100, opt-in STRATA_MMQ_WS=1 (#3752)
Problem: on a GP100 die (compute capability 6.0: no dp4a, no tensor cores) the
prompt pass's routed experts run as MMQ launches per 16-expert group: a
gate/up, a swiglu, a quantize and a down launch per group, each group's experts
gathered and padded to the group's largest row count, every MMQ tile decoding
its weight blocks again. With every expert resident the per-group overhead, not
the weight bytes, is a large share of a prompt read on this card.

Change: with STRATA_MMQ_WS=1, a layer whose routed experts are all resident on
this GPU runs quantize_scatter, then gate/up as ONE weight-stationary launch
over all its experts (read where they lie in the expert cache), ONE
swiglu_quant over all its rows, then the down as ONE launch. New
src/prefill/moe_ws*.cu(h): a CTA holds 128 (or 256) weight rows of one expert
and runs them against up to 64 of that expert's activation rows, a thread per
weight row, so each weight block is decoded once per pass; gate/up for
IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS and IQ3_S, down for Q2_0 and IQ4_NL. Every
product keeps MMQ's int8 values, scales and per-32-value epilogue in its K
order, so GU, H and the down rows are MMQ's bits. ws::faster carries the
author's measured ws / MMQ time ratios per format and picks a layer only where
both products win at its rows per routed expert. prefill.cpp decides per layer
after quantize_scatter (all experts resident, this GPU alone, no peer or CPU
share, not the fused layout, not STRATA_MMQ_DIRECT, no Hadamard, no NVFP4 tail,
no FP4 activations, aligned, not STRATA_DBG_NAN); every other layer keeps its
path. STRATA_WS_FAST=1 (second opt-in, same bits) selects a down kernel for the
IQ4_NL 32-token tier with fewer instructions per pass.

Gating: the kernels and STRATA_PREFILL_WS exist only in the CUDA build with
STRATA_EXPERIMENTAL_SM60 (not HIP, not Metal; prefill.cpp links stubs that
answer mode 0 elsewhere). At run time ws::mode() is 1 only with STRATA_MMQ_WS=1
and every visible GPU at compute capability 6.0. Unset, nothing is allocated
(the offset and chunk buffers, under 1 MiB per die, exist only when the mode is
not 0), the arena layout and the mapped table are the ones before, and the
per-layer test short-circuits on the cached mode.

Not ported: moe_ws_down_bench (needs a model shard) and its CMake target.
Added: ws::set_mode (tests only) and tests/cuda/prefill_ws_parity_test.cpp, a
synthetic layer-level check: random experts of all 5 gate/up x 2 down formats,
random routing, the per-group MMQ (mmq::Context per expert) against ws::gate_up
+ swiglu_quant + ws::down, GU, Hq and Dm compared byte for byte, the down also
alone on the reference H and with STRATA_WS_FAST on. It gives its verdict on a
compute capability 6.0 card only (the kernels follow MMQ's sm_60 SASS); other
cards run it for information with --any.

Tests: prefill_ws_parity_test on boston (2x RTX A6000, sm_86, CUDA 13.1, gpu lock,
SM60 build, --any because it is not a GP100): 40 cases (all 5 gate/up x 2 down
formats, four routings each), 0 declined; the largest GU and down-alone error
against that card's MMQ is 5.2e-07 of the largest value (float rounding: MMQ's
sm_86 epilogue is not the sm_60 one the kernels follow), the full chain's Dm
differs up to 1e-3 where int8 rounding of H flips on those rounding
differences, and STRATA_WS_FAST output equals the default kernel's in all
cases, 0 differing bytes. The byte-for-byte verdict against MMQ is the P100's.
sm_60 compile of the whole tree (CUDA 12.9, P1004): exit 0; sm_86 build without
the SM60 option (z440): exit 0. SASS of the 10 kernels moe_mmq.cu had before
this change is identical in the sm_60 build.

Not covered: no GP100 run here. The contributor's numbers (prompt reads 31-37%
shorter on four GP100 dies with every expert resident: 240 tokens 2,156 -> 1,364
ms, a fresh 26K read 41,231 -> 28,555 ms; one run per arm) were not re-measured.
The model gate is phaseA-tests/gh027/gates/3752-mmq-ws.sh.
2026-10-26 20:52:32 +01:00
Niko1221 60e6a06086 Split serving: docs pointers, check tool puts session files in /dev/shm
Idea and manual setup by @MarcoTool in #4000.
2026-10-26 20:44:45 +01:00
linxuhao 884e37dfdf Native head: opt-in STRATA_EMBED_VRAM=1 keeps the token-embedding table in VRAM (#3779, #3712)
Problem: the token-embedding table of a native pack lives in mapped pinned host
memory, so every prompt's scattered row gather crosses the PCIe link. On a Gen2
x4 link that costs more than the table's size in VRAM is worth.

Change, opt-in: with STRATA_EMBED_VRAM=1 the table is copied into the current
device's VRAM when the table plus 512 MiB stay free (322 MiB for an IQ4_XS
table). The gather kernels read the same bytes through dev_ as before. If the
VRAM is not there or the copy fails, the table stays in mapped host memory as
before and one line says so. The start-up line now says where the table is
("in VRAM" also for the two existing VRAM fallbacks, which it called mapped
host memory).

With the variable unset nothing changes (one getenv at load).

Not covered: a layer split across several devices places the table on the
device that is current when it loads, as the existing VRAM fallback does. No
unit test (it moves the same bytes to a different memory; the gather kernel is
unchanged).

Reporter's numbers (linxuhao, CMP 170HX): measured together with
STRATA_PLE_COPY_STREAM and STRATA_DEQUANT_SIDE on a 51.8K-token prompt, the
embedding step went from 191-201 to 70 ms. Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 952dd784).
2026-10-26 20:41:23 +01:00
linxuhao d6428bf291 Prefill: opt-in STRATA_PLE_COPY_STREAM=1 uploads the compact PLE rows on their own stream (#3779, #3712)
Problem: with STRATA_PLE_COMPACT the raw-row upload sat on the compute stream
before layer 1, which idles for the DMA (4.7 MB per 3K-token chunk, 2.9 ms on a
Gen2 x4 link).

Change, opt-in, only with STRATA_PLE_COMPACT=1: the raw rows go up on a
non-blocking stream of their own once the previous chunk's dequant has read the
device buffer (ple_raw_free event); the compute stream waits for the upload
(ple_copied) only at the dequant. Same bytes, same kernel. The stream is drained
when a run ends so no upload is left in flight from a buffer the next run
refills.

Differs from the PR as submitted: if the stream or the event cannot be created,
the PR failed Prefill::init. Here it prints one line and the rows go up on the
compute stream as with STRATA_PLE_COMPACT alone. With the variable unset
nothing changes (the stream and the events stay null).

Reporter's numbers (linxuhao, CMP 170HX): PLE time of a 51.8K-token prompt 95-103
to 47 ms, measured together with STRATA_EMBED_VRAM and STRATA_DEQUANT_SIDE
(+2.2..+2.6 % prompts for the four PLE/embedding/dequant switches together).
Theirs, not measured here.

Original change by linxuhao (PR #3779, commit 0ba47e61).
2026-10-26 20:40:42 +01:00
linxuhao 17b08c9633 Prefill PLE: opt-in STRATA_PLE_COMPACT=1 uploads the raw IQ4_NL rows and dequantizes them on the device (#3779, #3712)
Problem: the prompt path reads 16 PLE rows a token, dequantizes them on the
host and uploads 10,240 floats a token. On a slow host link (PCIe Gen2 x4) that
upload is on the critical path of the first layer.

Change, opt-in with STRATA_PLE_COMPACT=1: for an IQ4_NL table held in memory
(--ple-io ram / mmap) the prompt path copies the raw 90-byte rows (1,440 bytes
a token) into the same pinned buffer, uploads them and runs
ple_iq4nl_dequant on the device (d times the code's value, one multiply, the
split-half nibble order of iq4nl_dequant_row). Any other table (Direct I/O,
pageable fallback, another format) and a failed allocation take the host path
as before. With the variable unset nothing changes: the new code is behind one
cached flag, and the extra members of the Prefill state stay null.

Two things differ from the PR as submitted:
- PleTable::gather_raw_iq4nl writes a row past the table as scale 0 and code 8
  in every nibble. The PR wrote zero bytes, which the device decodes to
  -0.0f (0 times code 0's value -127), while the host path writes +0.0f for such
  a row.
- New test ple_iq4nl_dequant_test (ctest, synthetic, no model, 77 without a
  device): the device decode against iq4nl_dequant_row, bit for bit, over every
  fp16 scale pattern with random codes, random rows, an odd row count and the
  out-of-table row.

Reporter's numbers (linxuhao, CMP 170HX, PCIe Gen2 x4): prompts +1.0..+1.7 %
(4K, 16K) with the switch on. Theirs, not measured here. On a Gen4 x16 card
the gain will be smaller.

Metal gets a stub (the Metal prompt runs through its window and never calls
it).

Original change by linxuhao (PR #3779, commit da818261).
2026-10-26 20:40:20 +01:00
sergiywith dd97421d3f prefill: mmq::swiglu_quant, swiglu and the down product's q8_1 rows in one kernel (#3752)
Problem: the prompt path stores H (swiglu of the gate/up rows) as floats and
then quantizes it to the q8_1 rows the down product reads: two kernels and an
H round trip per MMQ group.

Change: swiglu_quant_kernel, a block per row (128 threads, 4 values each,
blockIdx.y the row's 512-value half), computes swiglu_kernel's expression for
each value and quantize_mmq_q8_1<D4>'s result for each 32-value block, so hq
holds the same bytes and H is never stored. swiglu_quant_ok(type) says where
it applies (a q8_1 D4 down type, no FP4 activations). Built only in the CUDA
build with STRATA_EXPERIMENTAL_SM60; HIP and the other builds get
swiglu_quant_ok() == false and a no-op. Nothing calls it in this commit; the
weight-stationary path of the next commit does. prefill.cpp gets the matching
stubs for builds without the MMQ sources.

Tests: prefill_swiglu_quant_test (new, registered under STRATA_BUILD_TESTS in
the SM60 build, skips with 77 without a device) compares the fused kernel with
swiglu + quantize byte for byte for Q2_0, IQ4_NL and Q8_0 down types, split and
interleaved gate/up rows, a zero row and a row with one huge value.
Run on z440 (RTX A4000, sm_86, CUDA 13.0, SM60 build, gpu lock): all 18 cases print
"0 of N bytes differ" (1, 37 and 4113 rows) and the test ends "prefill_swiglu_quant_test:
ok". sm_60 compile of the tree (CUDA 12.9, P1004): exit 0.

This is a partial port of another contributor's prompt-path fusion change;
only the kernel is taken, as in PR 3752.
2026-10-26 20:38:06 +01:00
linxuhao bdcc34dfa8 Prefill fused IQ experts: opt-in 256-thread kernels STRATA_PF_FUSED_NT=256 and whole-layer launches STRATA_PF_BATCH=N (#3779, #3712)
Problem: on a GA100 card (70 SMs) the 512-thread fused prompt-expert kernels
leave the card idle at every barrier and at every grid tail.

Change, all opt-in:
- STRATA_PF_FUSED_NT=256 sets up a second set of native_kernel instances with
  256 threads and two blocks a multiprocessor (WW 2 only), with two-pass weight
  loads (the loads of the next k step are issued before the current step's
  decode; load_unit_l gives the same words as load_unit). Every output is the
  same thread's sum in the same order.
- STRATA_PF_BATCH=N launches up to N experts at once (WideBatch holds 512 blob
  pointers, 4 KB of kernel parameters: CUDA 12.1+, Volta and newer; HIP, gfx906
  and the Pascal build keep kMaxBatch). Above 128 it needs the NT = 256 kernels.
  Below 128 it only changes the launch grouping.
- With neither variable set the 512-thread kernels, their launches and their
  load sequence are unchanged: tile_rows(ww, 512) equals the old 256 / ww, the
  two-pass code is behind a constant-false TWO_PASS, and the one added test in
  experts_native is occ_nt > 0, which only STRATA_PF_FUSED_NT=256 sets.
- If the 256-thread instances cannot be set up (shared memory, occupancy) the
  engine prints one line and keeps the 512-thread kernels.

Cost: the NT = 256 instances are new device code in every CUDA build (the
reporter measured 365,632 bytes for sm_80).

Tests: new ctest entry prefill_fused_iq_test_nt256 runs prefill_fused_iq_test
(the fused kernels against the double-precision reference and the MMQ path)
with STRATA_PF_FUSED_NT=256; it exits 77 below sm_80.

Reporter's numbers (linxuhao, CMP 170HX): one MoE layer of 3,256 tokens
8.27 ms with NT = 256; prompts +7.6 % (4K), +4.9 % (16K), +4.3 % (52K) for
NT=256 with BATCH=512 on the PR's own build. Greedy answers identical on their
11 prompts. These are theirs.

Original change by linxuhao (PR #3779, commit 056d36a9). Merged onto the Q2_0
fused path of #3809 (STRATA_PF_FUSED_Q2N=1) without textual conflicts; the PR
predates it, so one line is mine: the NT = 256 setup also sets up the Q2_0
gate/up instance when Q2N is on (without it that instance would have launched
without its shared-memory attribute).
2026-10-26 20:38:00 +01:00
Niko1221 945a63bec5 tests: lm_kernels_parity checks the two kernels of the layer-major prompt path (#3976)
moe_shared_init and moe_scatter_add came with the layer-major walk and had no test.
lm_kernels_parity runs them on synthetic data of the model's width (N = 2560, no
model, a few MiB) against a host reference:
- moe_shared_init: acc = shared * sigmoid(sg), within 1e-5 relative (the device
  sigmoid uses __expf);
- moe_scatter_add: the per-token fma chain in row order, bitwise equal to the host
  chain and bitwise equal between two launches (no atomics), over groups of one
  row, one sub-chunk, several sub-chunks, one row per token and eight per token.
Registered in CMakeLists.txt next to gr_upmix_parity (CUDA builds, not gfx906),
SKIP_RETURN_CODE 77 when no device is visible.

Not covered: the walk itself needs a model (see the 3976 gate scripts).
2026-10-26 20:37:49 +01:00
linxuhao 91aebae636 MMVQ-IL: sm_80 row tables, bitwise (#3779, #3712)
Problem: an sm_80 card took the shared interleaved-MMVQ row tables, which
were measured on other cards (#3252 reports A100 slowdowns).

Change: kIlRows_80 (2-4 columns), kIlRows58_80 (5-8 columns) and kIl1Rows_80
(one column), looked up under the key 80 only. They were read off
mmvq_il_parity --bench --emit-table with the same 3 % rule as the other tables.
STRATA_MMVQ_IL_SM80=0 maps an sm_80 card to the key 81, which has no table, so it
gets the shared table again (and no one-column table, as before).

A table only picks the rows a warp (0, 1, 2 or 4); 0 means native_mmvq's kernels.
mmvq_il_parity already checks every rows choice against native_mmvq bitwise for
every type and shape, T = 1..8, so any entry of the new tables is covered.

Reporter's numbers (linxuhao, one CMP 170HX): decode +2.7..+3.2 % over the shared
table, Qwen3.8-Flash-Next IQ3_S. The measured card is not an A100; the rows may
want re-measuring there.

Original change by linxuhao (PR #3779, commit eaedca84); the commit message here
is rewritten. The sm_89 table of #3453 is a separate change in the same list.
2026-10-26 20:34:14 +01:00
linxuhao 269c6d3e4a Dense dequant: the wide kernels by default on sm_80 too, bitwise (#3779, #3712)
Problem: sm_80 (GA100: A100, A30, CMP 170HX) ran the per-group dequant kernels
that sm_61 and sm_86 had already left for the wide ones, because
dequant_wide_on() had no sm_80 entry.

Change: dequant_wide_on() answers yes for compute capability 8.0 as well. The
test is the raw (major, minor) of the device, so it is false on every other
card (sm_60, 70, 75, 86 keep their answer, sm_89, sm_90 and sm_120 stay off as
before). STRATA_DEQUANT_WIDE=0 still turns the wide kernels off.

Reporter's numbers (linxuhao, one CMP 170HX, GA100 sm_80 with 70 SMs): the GPU
time of a 3,259-token prompt went from 857 to 792 ms, prompts +6.3..+7.2 % at
3.3K tokens and +3.9 % at 52K. We have no sm_80 card; these are theirs.

Bit identity: the wide kernels are the ones sm_61 and sm_86 already run by
default. dequant_wide_parity compares them with the per-group kernels byte for
byte for FP16, BF16, FP32, the padded stride and a row slice. The kernels have
no sm_80-only code, so the result does not depend on the card; what only an sm_80
card can show is that ptxas for sm_80 keeps the same arithmetic (see the
checklist in phaseA-tests/gh027/gates/3779-sm80-checklist.md).

Original change by linxuhao (PR #3779, commit 48dc535e); the commit message
here is rewritten.
2026-10-26 20:34:14 +01:00
Niko1221 40c9dae95b Merge f-027-gh-eng-g (#3935 sm_89 in the DeepSeek CUDA package builds, #3796 collapsed tool-name alias and the rejected-calls marker) 2026-10-26 20:31:38 +01:00
Niko1221 805ebe17be serve: a call returned as text for an undeclared name is named in the reply; the collapsed mcp__a__b spelling is the declared tool (#3796)
Problem: a request declares tools; the model writes a complete call for a name that is not among them (bash for shell, a dropped
prefix, an MCP name with every "__" written as "_"). Since #3075 such a call stays text, which is the rule we keep, but the client
saw an ordinary answer with finish_reason stop and nothing else, so the agent turn ended with raw XML and the model never learned
of its mistake. Reporters on OpenCode, Pithagoras, OpenClaw, LibreChat, Open WebUI and Pi hit it.

What changed (declared-only stays: nothing undeclared is ever returned as a call, finish_reason is unchanged):
- serve/frontend.py OutputParser: a declared name that contains "__" is also read when written with every "__" as "_"
  (mcp_playwright_browser_navigate for mcp__playwright__browser_navigate); the call goes to the client under the declared name.
  Same rules as the other aliases: only when unambiguous, and a tool declared under that spelling wins. A reordered or renamed
  tool (server1_label_values for label_values_mcp_server1, bash for shell) is not guessed.
- serve/server.py: when a block went back as text for an undeclared name, the reply carries
  "strata_rejected_tool_calls": [{"name": ..., "reason": "undeclared"}] (the names the parser already collected, up to five): at the
  top level of a Chat Completions reply and in its last streamed chunk, and in the message_delta event and the message of
  /v1/messages. It is absent when nothing was rejected, so every other reply is unchanged. The text itself is untouched.
- The server window line says the response carries the names; the count goes to totals["tool_calls_unread"] and to
  /metrics/prometheus as strata:tool_calls_unread_total.
- docs/DETAILS.md: the alias, the marker, the counter.

Tests: serve/test_tool_name_alias.py (collapsed spelling at several chunk widths, a declared spelling and a plain tool unchanged,
a tool declared in the collapsed spelling wins, two tools with one collapsed spelling are not guessed, reordered/renamed names stay
text and are named), serve/test_server.py ForcedToolChoice (the field in Chat Completions whole and streamed and in /v1/messages,
absent for an ordinary answer and a declared call, the collapsed name is a call; the counter). The whole serve/ suite: 1055 tests,
two failures that also fail on the base tree (test_fatal_recovery on this PC, RestartWindow.test_not_alive_until_ready under load).

Not covered: returning an undeclared call as a real tool call (the owner keeps declared-only); a block for a DECLARED name that came
back as text because it was malformed (counted, no name to list); the /v1/messages field with a real client; a stderr or log-file copy
of the warning.
2026-10-26 20:30:44 +01:00
ruibeikaa b434a8aa6e qsa_select: multi-CTA decode top-k and a shuffle-free decode scorer, opt-in STRATA_SELECT_LONG=1 (#3820)
Problem: a decode window's QSA select runs one CTA per query for the top-k
(block_topk_wide_kernel past ~135K cells) and the warp-shuffle block scorer
(block_scores_multi_kernel), in each QSA layer. Both grow linearly with the
context: on one GP100 die at 300K cells a top-k call takes 0.39 ms with 1-5 SMs
busy and a scores call 0.22 ms (two queries). At short contexts the same work is
small and an extra kernel is pure overhead.

Change: qsa_select.cu (inside !__HIPCC__ && STRATA_EXPERIMENTAL_SM60) gets
block_topk_multi_* (six kernels over several CTAs per query, one per 2,048
blocks of capacity, at most 32, exchanging through caller scratch: no clusters,
no spin waits, capturable) and block_scores_dx_kernel (the multi scorer's
per-head sums built in registers as the same 32-leaf xor-butterfly tree, no
__shfl_xor_sync, the tail block stays block_scores_tail_kernel's). New public
qsa_topk_multi_scratch_ints, qsa_block_topk_long and qsa_block_scores_long
forward to qsa_block_topk / qsa_block_scores outside the build gate or their
conditions. verify.cpp keeps a second set of window graphs (exec_[sel_long_][T])
for windows that reach STRATA_SELECT_LONG_AT cells (default 65536); the choice
is made before a capture and handed to the next stage of an overlapped layer
split, capture_all captures both, pl_launch picks by the window's position.
The ids and the score bits are the replaced kernels'. STRATA_TOPK_MULTI=0 and
STRATA_SCORES_DX=0 turn either half off inside the long graphs.

Review changes against the PR: in a build without the multi-CTA kernels
STRATA_SELECT_LONG=1 now says so and is ignored (the PR would have kept two
sets of graphs for nothing there); Metal gets stubs for the three new calls
(the SYCL tree has its own verify.cpp and qsa_select copy).

Default path: unset, select_long_on() is a cached false, topk_scratch_ is not
allocated, sel_long_ stays 0, and every window uses exec_[0][T], the graph it
used before; the index change is the only difference in verify.cpp.

Merge note for f-027-gh-eng (#3702 evict_window_graph, not in this branch's
base): its exec_ / exec_nr_ / launched_ accesses need the same [sel_long_]
first index.

Tests: src/kernels/qsa_select_long_parity.cpp (new; registered with
STRATA_SCORES_DX=1 STRATA_TOPK_MULTI=1 in the SM60 test block, and a second
case with STRATA_TOPK_MULTI_N=3; skips with 77 without the kernels) compares
qsa_block_scores_long with qsa_block_scores (nq 1-8, n_bid 8 .. 131072, tail
blocks of 0-3 cells, every block 0..n_bid memcmp) and qsa_block_topk_long with
qsa_block_topk (nq 1-16, contexts 1,000 .. 524,288, random scores, all-equal
scores and 7 distinct values for ties across CTAs, selected ids of every query;
the multi scratch is junk-filled and must have been written). z440 (RTX A4000,
sm_86, CUDA 13.0, gpu lock, build with -DSTRATA_EXPERIMENTAL_SM60=ON): "172
cases, 0 differ -> PASS" with one CTA per 2,048 blocks and with N=3. sm_60
compile of the whole tree (CUDA 12.9, P1004): exit 0.

Not covered: no GP100 run here (the kernels' last word is the P100, and the
speed claim). The contributor's numbers (four GP100 dies, every expert resident,
--pipeline-windows 3: 400 tokens at 115K context 51.0 -> 52.5 tok/s, +2.9%;
prompts below 64K unchanged; replies and state hashes equal on 15 requests) were
not re-measured. The model gate is phaseA-tests/gh027/gates/3820-select-long.sh.
2026-10-26 20:25:56 +01:00
Niko1221 40e78366bc Web page: German and dark screenshots in tools/webtest/shots.mjs 2026-10-26 20:23:06 +01:00
Niko1221 4661596ce3 Web page: the chat view in eight languages, and the page's docs
Everything the Chat view shows (header, tabs, message box, settings, toasts,
tool blocks, the sidebar, the history and Load/Unload texts) now goes through the
translation catalog: serve/web/locales/{de,es,ja,ko,pt,ru,zh}.json beside en.json
(written from the English text in the code by tools/web_i18n.py). Plurals follow
the language (Russian has few and many, Japanese and Korean only one form).
serve/test_web_i18n.py fails when a catalog lacks a key, has a stale one or
changes a {placeholder}. A ninth language is one file plus one line in
serve/web/i18n.js. The Monitor, Images and About views are still English (the
language setting says so); they are the next step.

Browser tests: a language pick translates and is kept, /?lang= is for one visit,
an unknown code stays English, all seven catalogs load without an error.

docs/DETAILS.md: a section "The web page" (every switch, what the preview can and
cannot reach, the languages, the tests), the Sampling drawer's new name, and the
credits (KaTeX and the contributors of the PRs ported into the page).
2026-10-26 20:22:05 +01:00
fcd3cdbcd7 Web chat: one chat-history design - saved chats, edit and rewind, branch, send while answering (#1787, #2615, #3823, #4004)
Five overlapping proposals, one set of controls (Chat settings > Chats, all off
by default; off, the page is the page from before: one chat, New chat clears it
with an Undo, Send is off while an answer is written, no new request field).

- Saved chats list (#1787): the chats kept in this browser (IndexedDB, one record
  per chat, names of attachments only), a list with day groups, search over
  titles and text, rename, delete with Undo, collapse; a private window or full
  store keeps the list in memory and says so once. Nothing goes to the server.
  The list has the New chat button, so the composer's one is hidden while it is on.
- Edit and rewind messages (#2615 + #3823): under each message, Regenerate (a new
  answer to the same question), Edit answer (its text only: thinking and tool
  calls go, no request) and Rewind to here. Rewind asks, as #3823 proposes: "Edit
  and send again here" hides this message and the ones after it, puts the
  question and its attachments in the box and keeps one Undo bar ("2 messages
  hidden"); "Branch from here" starts a new saved chat with the messages before
  it and leaves this one alone. A branch keeps a title that says where it came
  from ("Branch of ...", also at the very first message) and links back. One
  control does what #2615 had as Edit on a question and as Remove from here.
- Send while answering (#4004, for the web page): "Wait" (default, as before),
  "Send after" (the message waits and goes when the answer is done) or
  "Interrupt" (the answer stops where it is, what is written stays and is marked
  interrupted, the message goes at once; a running tool finishes first). One
  message waits; Take back returns it to the box; Stop keeps it for you to send.

The pure logic is serve/web/history.js, the store serve/web/chatstore.js, the
page's part serve/web/chats-ui.js. Tests: serve/test_web_history.js (Node, with
the IndexedDB stub and cases of #1787), serve/test_web_history.py, and 17 browser
tests (tools/webtest/browser_test.mjs). Screenshots: tools/webtest/shots.mjs.
Also: the Sampling drawer is now Chat settings (its button says so), a wide
window lets a message use the column (#3176 widened only the column).

Ported by hand and merged: #1787 by Scotty Hudson, #2615 by Joyer Huang (uppet),
issues #3823 and #4004 by xvcezrqpbxtpzblrt (their proposals, for the desktop app,
carried to the web page).

Co-authored-by: Scotty Hudson <[email protected]>
Co-authored-by: Joyer Huang <[email protected]>
Co-authored-by: xvcezrqpbxtpzblrt <[email protected]>
2026-10-26 20:14:14 +01:00
Niko1221 a732a34c65 layer-major prefill: a prompt that needs a run_impl feature the walk lacks takes the chunk-major path (#3976)
The layer-major walk is a copy of run_impl's attention half with the experts moved
out. It does not carry four things run_impl does, and when one of them is asked for
the walk now declines (the prompt is read chunk-major, and the one-time "layer-major
off for this prompt" line names the reason):
- the DFlash2 drafter's prompt taps (tap_layers_): the walk would leave the taps
  stale and the drafter would read old rows;
- STRATA_KVA (approximate late layers): the walk would silently read exactly;
- a stop_at_chunk callback (the pipelined batch reader that yields at chunk
  boundaries): the walk reads every token, and stopped_at() must stay -1;
- a pending force_chunk (a --prefill-pipe plan): it is left set for run_impl.
The walk also resets stopped_at_ like run_impl does at the start of a run.

With STRATA_PREFILL_LAYER_MAJOR unset none of this is reached.

Not covered: no model run in this commit.
2026-10-26 20:13:56 +01:00
Niko1221 d02478b7a1 layer-major prefill: stays off on AMD builds (#3976)
Only NVIDIA CUDA has run the layer-major walk. On a HIP build (wave32 or the
gfx906 compat layer) lm_prepare now declines with a reason, so a prompt takes the
chunk-major path and the one-time "layer-major off for this prompt" line names it.
The code still compiles everywhere the shared prefill.cpp does; nothing else
changes. With STRATA_PREFILL_LAYER_MAJOR unset nothing is reached.

Not covered: HIP and SYCL were not built or run (SYCL has its own prefill.cpp).
2026-10-26 20:07:56 +01:00
Niko1221 0acba452ec Test: app-ui 'Quit works once' waits for the shutdown callback (polling, up to 5 s) instead of a fixed 20 ms sleep, so a busy PC cannot fail it (test only) 2026-10-26 20:06:26 +01:00
Norihiro Nishimura e7318fa23d layer-major prefill on a split: a stale hand_in_ left by a chunk-major prompt no longer turns layer-major off (#3976)
run_impl never clears a later stage's hand_in_, so after one short prompt (read
chunk-major) every later prompt was declined with "a layer split's chunk hand-off
is in use". lm_prepare_chain now declines only when a hand-off is really in flight
(a stage's next_run_ is valid) and otherwise clears the stale hand-in pointers.
2026-10-26 20:03:03 +01:00
Norihiro Nishimura 18c5357666 layer-major prefill: even super-chunks (STRATA_PREFILL_LM_EVEN, default on with layer-major) (#3976)
A prompt longer than the common super-chunk is split into the same number of
super-chunks of near-equal size (32K at a 28K fit: 2 x 16K, not 28K + 4K). That
keeps a stage's residual rows in VRAM and lets a split's stages overlap on every
super-chunk. STRATA_PREFILL_LM_EVEN=0 gives the old largest-first split. It only
acts when STRATA_PREFILL_LAYER_MAJOR is set.
2026-10-26 20:03:03 +01:00
Norihiro Nishimura c6f6db857b layer-major prefill on a split: the PLE layer is fine in a first stage that holds layers 0 and 1 (#3976)
The split was declined whenever the model has PLE layers. The PLE block (layer 1)
reads the token history, which only the first stage has, and a later stage never
reaches it. So the split is declined only when the first stage does not hold both
layer 0 and layer 1.
2026-10-26 20:03:02 +01:00
Norihiro Nishimura 6eff377a79 layer-major prefill on a CUDA layer split: each stage runs its layers over the super-chunk and hands its rows to the next stage (#3976)
Problem: layer-major prefill declined every layer split, so a split model kept
re-reading each layer's streamed experts once per 8192-token chunk.

Changed:
- lm_prepare_chain sizes every stage with the same super-chunk (the smallest any
  stage fits, up to four rounds until they agree) and gives them back together;
- a stage runs only its layers [LB, LE); a later stage's rows arrive in its pinned
  R_host;
- a stage before the last writes its final rows into the next stage's R_host on
  the copy stream, then runs the next stage on a thread (std::async) while it reads
  its own next super-chunk. There is one hand-off buffer: a stage's final pass waits
  until the next stage has finished the previous super-chunk;
- layer_major_mid() asks the last stage, the one that reports the rows;
- run() drains the pipeline after the walk and clears the mid flag on every stage.

Not covered: only two stages were tried by the reporter (RTX 3060 + RTX 5060 Ti);
the peer-GPU tier and the stage helper have no layer-major form and decline.
2026-10-26 20:03:02 +01:00
Norihiro Nishimura cc125e06a0 layer-major prefill: no MMQ row table without an MMQ layer; QSA shapes from core::qsa_shapes (#3976)
run_layer_major called mmq::iota on ids_identity, which is null when no layer is on
MMQ; it now skips that. The QSA shapes come from core::qsa_shapes(g) as run_impl
does, instead of being filled in by hand.
2026-10-26 20:03:00 +01:00
Norihiro Nishimura 5602253e3f layer-major prefill: the FP16 path needs no MMQ identity table or token upload buffer (#3976)
lm_prepare required ids_identity and tok_dev even when no layer is on MMQ. Only a
model with an MMQ layer needs the identity table, so the check now asks for it only
then. Without this a model whose layers all take the FP16 path was declined.
2026-10-26 20:03:00 +01:00
Norihiro Nishimura 42094da9c1 layer-major prefill: the FP16 expert path for layers MMQ does not take, decline reasons name the layers (#3976)
Problem: lm_prepare declined every model whose experts are not all on MMQ. All 48
layers of UD-Q4_K_XL take run_impl's FP16 path, so layer-major never ran on them.

Changed: a layer outside the MMQ plan dequantizes each routed expert to FP16 once
per super-chunk and runs two FP16 GEMMs per entry on that entry's FP16 input rows
(a new per-super-chunk buffer mixed_h, the bits run_impl's gather reads), then
adds the group's rows per token with moe_scatter_add as the MMQ path does. The
buffers each path needs are checked separately, and a missing one gives a decline
reason that names it. The summary line counts the groups on the FP16 path.

Not covered: no model run in this commit.
2026-10-26 20:02:59 +01:00
Norihiro Nishimura df7068b6be layer-major prefill: STRATA_HC_Q8 dequant, guards for APR/NVFP4/WMMA layers, QSA indexer calls of this tree (#3976)
Problem: the layer-major walk (#1670) was written for an older base. It read the
hyper-connection matrices as bf16 only, accepted expert formats its MMQ pass does
not carry, and called the QSA indexer with the older argument lists.

Changed (Norhiro's port of vizdatom's walk, kept as one step):
- the HC projections are dequantized per half when STRATA_HC_Q8 holds them as Q8_0
  with no bf16 copy, and released after the GEMMs that read them are enqueued;
- lm_prepare declines (the prompt then runs chunk-major) for a layer that needs the
  WMMA tiles, an APR (Hadamard-folded) layer, and an NVFP4 layer;
- STRATA_PREFILL_DIRECT is not used by the walk.

Not covered: no model run in this commit.
2026-10-26 20:02:59 +01:00
vizdatom ffe40c1695 prefill: layer-major prefill (STRATA_PREFILL_LAYER_MAJOR=1) (#3976, #1670)
A super-chunk of S tokens is read layer by layer: each 512-token sub-chunk runs the attention half,
the FFN half hyper-connection read, the router and the shared expert in order (GDN / QSA / PLE state
advance as in the default path); then the layer experts run once for the whole super-chunk - each
routed expert staged once, its rows of every sub-chunk in one MMQ group, the results added per token.
Expert traffic drops by about S / 512 (a 12.9K prompt: ~509 GB -> ~79 GB on ryzen).

- STRATA_PREFILL_LM_TOKENS: the super-chunk target (default 8192), shrunk to fit free VRAM minus
  STRATA_PREFILL_LM_MARGIN_MIB (default 96).
- STRATA_PREFILL_LM_R=vram|host|auto: the residual rows of the super-chunk in VRAM, or in pinned RAM
  with the copies overlapped on the copy stream (auto: VRAM only when the whole target fits there).
- STRATA_LM_PREFETCH=0: no prefetch (default on: every streamed expert of a layer goes to the stager
  when its attention halves start; the ones the routing skips are skipped if not started).
- Not for a layer split, a peer GPU, image prompts, the fused layout or a prompt under two chunks.
- on_chunk still reports every sub-chunk in order; layer_major_mid() tells serve not to checkpoint
  before a super-chunk end (the state is already at that end).

Port to 0.1.43.1: the copied run_impl body follows the new base. Every blob the stager or the copy stream reads later
comes from blob_stable() (blob() may hand out a recycled STRATA_IO_PREFETCH buffer); the token ids go up through tok_up,
the QSA step records and the PLE rows' pageable fallback through up_ring (#1712, #1996); the PLE gather threads pin to
--aux-cpus; bf16x2 takes m.f16_io; the K8V4 append is upstream's, which writes the host copy and the staging pool by
halves. The indexer passes the INT8 pooled keys as the original does, in the QSA port's order (pooled_i8 and
pooled_scale after block_pos, the int8 qsa_block_scores(_tc) overloads); STRATA_IDX_INT8 stays opt-in and unsupported on
the prompt path, as in the original. stage_one calls the stager's pass() before wait(): with batched claims a routed job
after about a ring's depth of skipped prefetch jobs waited forever. lm_prepare reads the fused layout from m.fused_bufs
and declines while STRATA_DBG_FEAT, STRATA_DBG_ROUTE_OVERRIDE or STRATA_PREFILL_DUMP_R_ALL is set. serve's near-tail
checkpoint (--prompt-cache-tail) is not taken mid-super-chunk, so under layer-major it is taken only when the last
super-chunk is a single sub-chunk.

From vizdatom's #1670, original commit .
2026-10-26 20:01:55 +01:00
vizdatom a6df0023a4 prefill: kernels for a layer-major expert pass (moe_shared_init, moe_scatter_add) (#3976, #1670)
moe_shared_init writes the shared expert part of moe_combine before the routed experts are known;
moe_scatter_add adds one expert group rows into a per-token accumulator, per token in a fixed order
(deterministic, no atomics).

From vizdatom's #1670, original commit f0173382d0646907917edf99e59617c1412d898e.
2026-10-26 20:01:36 +01:00
vizdatom 162dec3411 prefill: the per-layer walk warms this layer's and the next layer's file-tier experts (STRATA_PREFILL_WARM=1) (#3976, #1670)
Chunks below stream_all_min() stage each layer's unpinned experts on the stager's few threads, and with a mapped
file tier each of those is a chain of 128 KiB page faults, a shallow queue for the drive.  With STRATA_PREFILL_WARM=1
(and a source that warms: the GGUF in place, or experts.bin with STRATA_BIN_WARM=1) a background thread hands the
source the layer's whole list at once before the stager starts, and the next layer's list from the previous chunk
(less what is on the GPU or pinned) while this layer computes.  It only warms pages: the stager copies the same
bytes.  At most four lists queued; a thread that falls behind drops the oldest.  Off by default: the A/B arm.

Port to 0.1.43.1: the warmer thread pins itself with aux_cpus::pin_current_thread() first, like the other helper threads since --aux-cpus (it would otherwise inherit the host thread's CPU).

From vizdatom's #1670, original commit cabe33c4f585d4c9dd0af1fc9f014d00e2920042.
2026-10-26 20:01:36 +01:00
vizdatom fca1936c2c prefill stager: a job waits for the previous job on its buffer; skip() (#3976, #1670)
issued is a watermark, so a job nobody waits for could be passed while its thread was still writing
the pinned buffer the next job on that buffer was filling - a corrupt expert blob. A job now also
waits until the job kRing before it has marked itself ready. skip(j) lets a job the routing turned
out not to need be marked ready without reading it (the layer-major prefetch). The default walk
consumes every job in order, so neither changes it.

Port to 0.1.43.1: the stager now claims batched runs and sleeps in issued.wait, so the ready[i - kRing] wait
sits in the run's per-job wait loop after the issued wait, and a skipped job is left out of the run's gather
(no memcpy, prefill_copy or copy_blobs entry); its ready flag and `active` stay with the run's single store and
decrement. New pass(j): skipped jobs are never issued, so the layer-major walk moves the watermark to the
routed job it is about to wait for (store plus notify_all, which wakes sleeping threads); without it a routed
job after about a ring's depth of skipped ones waited forever. The default walks never call it.

From vizdatom's #1670, original commit 149b899548fbc98977219e6fd4ee7866384e61f6.
2026-10-26 20:01:36 +01:00
Niko1221 8486336f9b Merge f-027-gh-eng-s (#3291 #3955 #3317 #4036 #2893 #3218 #3200: serve, tools and setup ports) 2026-10-26 20:00:53 +01:00
Niko1221andSakurai Momoka bcdb92ac68 Web page: Load and Unload in the header (#3550)
One button in the header of every tab: Load while the model is unloaded, Unload
otherwise. It calls POST /load and POST /unload (the server already had both,
for its own page only). Unload asks first and never interrupts a turn: the
button is off while an answer is being written, an image is made or a request is
queued. The pill and the Monitor say Unloaded / Loading / Unloading, a message or
an image on an unloaded model shows Loading, and a poll that started before the
POST cannot flip the button back (the POST's state is held until a later poll
agrees). On a phone the pill's text is the first thing the header row gives up
(it measures the row, not a fixed width), so Load/Unload and the theme button
stay on the screen from 320 px.

Browser tests (tools/webtest/browser_test.mjs): unload with its question, load
back, declining, not while answering, a message that loads, the header at 320 to
640 px.

Ported by hand from PR #3550 by Sakurai Momoka (sakuumomo); the texts now go
through the page's translation catalog.

Co-authored-by: Sakurai Momoka <[email protected]>
2026-10-26 19:59:54 +01:00
Niko1221andThomas Ivesdal-Tronstad 0aa64b2de3 Web chat: preview a generated svg or html block in a sandboxed frame (#1628)
A complete ```svg or ```html block in an answer gets a Preview button. The
block's own code is drawn in an iframe (srcdoc) that is cut off from the page:
sandbox="" (an opaque origin: no access to the page's DOM, storage, cookies or
server, no scripts, forms, popups, downloads or navigation of the page), a CSP
in the frame's document (default-src 'none': no network, fonts, frames or media;
styles inline, pictures as data: only; no <base>, no form target), no referrer
and no permissions. A well-formed svg is shown as an <img> with a data: URL: an
image runs no script, loads nothing and animates in every browser. The frame
takes the page's light or dark colours and is built again when the theme
changes. A page (html) can also be run WITH scripts, one block at a time, after
a warning that is shown every time: sandbox="allow-scripts" only (still an
opaque origin), the CSP allows the block's inline script and no address; closing
the preview ends it.

What the sandbox flags alone do not stop is a link or <meta refresh> inside the
model's html loading a web page into the frame. The browser test shows it does
(tools/webtest/browser_test.mjs, with and without the header). So the page is
served with Content-Security-Policy: frame-src 'none' (nothing else: a srcdoc
frame inherits the page's policy, and the script preview needs its inline
script), and a <meta http-equiv=refresh> in a previewed page is dropped.

tools/webtest holds the mock server (the page's own files, scripted answers,
no model) and the browser tests; serve/test_web_preview.js checks the documents,
flags and frames, serve/test_web_preview.py the header and that the page never
listens for a frame's postMessage.

Ported by hand from PR #1628 by thoives.

Co-authored-by: Thomas Ivesdal-Tronstad <[email protected]>
2026-10-26 19:57:56 +01:00
Niko1221 675f7fc238 Merge f-027-gh-eng-k2 (#2526 #1864 #1911 #4005 #3060 #4010 #3809: PR ports for the conversation cache, prefill events, expert source and batch groups) 2026-10-26 19:54:59 +01:00
chaog992 c47c699486 prefill: opt-in fused int8 prompt experts for the native Q2_0 pack on sm_80+, STRATA_PF_FUSED_Q2N=1 (#3809)
A PC whose CPU lacks the AVX-512 set that setup checks for gets the native Q2_0 pack. On a CUDA card of sm_80 or newer its
Q2_0 / Q2_0 prompt experts run through MMQ, although the fused int8 kernel for that format (#2145) is in the binary and set
up only on a Turing card (b6443f3ae).

STRATA_PF_FUSED_Q2N=1 sets that kernel up on any sm_80+ card and takes it for the Q2_0 / Q2_0 layers, under its own variable
so that STRATA_PF_FUSED=1 keeps its bits. STRATA_PF_FUSED=0 and STRATA_PF_FUSED_NATIVE=0 still turn it off. A failed setup
leaves the occupancy as it was and the layers on MMQ.
- moe_fused_iq.cu: DevInfo.q2n; dev_info() sets up Q2_0 gate/up when the switch is set; gu_covered() and native_supported()
  take the pair only when q2n is set (a Q2_0 gate with another down type is refused, as on Turing).
- tests/cuda/prefill_fused_iq_test.cpp: --unset-pf-fused leaves STRATA_PF_FUSED unset (the engine's default) and skips below
  sm_80; CMakeLists.txt: the ctest entry prefill_fused_iq_test_q2n runs the Q2_0 / Q2_0 pair with the switch alone (exit 77 =
  skipped without an sm_80+ card; CUDA branch only).
- docs/DETAILS.md: one section in the author's measured numbers (RTX 5080) and the A4000 caveat below.

Unset (the default), nothing changes: dev_info() reads the variable once and sets nothing up, gu_covered() and
native_supported() return what they returned. Read line by line against base: the Turing path (t75) and the HIP path are
untouched. Set, the prompt arithmetic changes (first-token KL against FP16 0.0110, MMQ 0.0112, the author's figure) and the
fused path's smaller buffers let --prefill auto pick a larger chunk, which is most of the author's 16K gain; the generated
text can change.

Not verified: an RTX A4000 (sm_86). The kernel was taken on every CUDA card in #2145 and then kept to Turing after an A4000
no longer matched the old output and read 4K prompts 4.5% slower; the PR does not repeat that run. Merge note: #3277 and #3779
change dev_info() and native_supported() in the same file.

Tests: prefill_fused_iq_test_q2n and prefill_fused_iq_test pass on an RTX A4000 (sm_86, synthetic data); on the Q2_0 / Q2_0 layer the
fused kernel is 1.76x / 1.30x / 1.24x the MMQ path at 2048 / 4864 / 8192 tokens with the same relative RMS against FP32. Builds on CUDA
sm_86 (Linux) and sm_120 (MSVC). Not covered: no model run; the engine-level A4000 check is the gate.
2026-10-26 19:54:00 +01:00
Niko1221andDavid a965c962cf Web chat: draw math with the bundled KaTeX, opt-in and offline (#2972, #2974)
Chat settings > Render math (off by default). When it is on, \( \), \[ \], $x$
and $$ $$ in an answer are drawn as formulas by KaTeX 0.19.0, which is vendored
in serve/web/vendor/katex (script, style, 20 woff2 fonts, MIT licence and a
PROVENANCE.json with the npm integrity and the sha256 of every file, so the page
needs no network). Nothing of KaTeX loads until an answer holds a formula.

The shared core (math.js, from #2972) keeps the text of every message exactly as
the model wrote it: formulas are swapped for placeholders before the Markdown
runs and drawn over them afterwards, so history, copying, the Markdown export
and the requests never see the drawing. Code, link targets, prices ($5 and $10)
and a formula still being streamed stay text. A formula that fails, is over
16 KiB or times out stays as its source text.

Safety: KaTeX runs with trust off (no \href, \url, \includegraphics, \htmlClass:
they are drawn as their own name), maxExpand 1000, maxSize 20 and a fresh macro
table per formula; it builds DOM nodes itself, so no HTML string from a model
reaches the page. A formula already drawn is placed again at once while an
answer streams, so it does not flicker back to source.

serve/test_web_math.js runs the real vendored KaTeX against hostile formulas;
serve/test_web_math.py checks the vendored files against PROVENANCE.json and
that nothing outside this server is addressed. serve/web/vendor is -text in
.gitattributes so the hashes hold on a Windows checkout.

Ported by hand from PRs #2972 (core) and #2974 (KaTeX) by CC-David-CC.

Co-authored-by: David <[email protected]>
2026-10-26 19:52:56 +01:00
Niko1221 d5cca319c9 Web page: a translation mechanism, and a language choice in About > Settings
The served page had no way to translate its text: every string was English in
the HTML and the script. Now the English text stays in the code next to a key
(tr("key", "text", vars), data-i18n="key" in the HTML), a language is a catalog
serve/web/locales/<lang>.json, and a catalog that is missing or has no entry for
a key leaves the English text. The language is the user's choice (About >
Settings), kept in this browser; nothing is chosen from the browser's language,
because a page translated in part would mix two languages. /?lang=de shows one
for a visit.

The server hands out /web/locales/<lang>.json and /web/vendor/katex/** from
serve/webfiles.py: whitelisted names and types, no backslash, colon, percent or
dot-dot, and the resolved path must stay inside the folder (also against a link
out of it). tools/web_i18n.py reads the English text out of the code, writes
en.json and checks every catalog for the same keys and {placeholders}; the
check runs once the catalogs exist.
2026-10-26 19:52:26 +01:00
Niko1221 3813a459c4 Split serving: exactness and speed check tool, docs draft
Idea and manual setup by @MarcoTool in #4000.
2026-10-26 19:48:29 +01:00
gopinath87607 fa128f91e8 docs: --prompt-cache-every does not set the size of the pieces a prompt is read in (#3311, #2834)
Only the part of PR #3311 by gopinath87607 that is about this engine: a periodic mark is saved at a chunk boundary of the batched prefill and the read carries on
(src/program/generate.cpp, periodic_checkpoint), so the interval is not a piece-size dial; what cuts a read into runs is the re-read, the root, a message
boundary and the start of the last turn; --prompt-cache-every 0 skips the periodic and the near-tail marks. The wall-time numbers and the comparison with the Maya
checkout belong to another engine and are left out. New section in docs/PROMPT_CACHE_TAIL.md, linked from the option's row in docs/DETAILS.md.
2026-10-26 19:47:22 +01:00
Niko1221 47c98bb23a strata launch --show-key shows the key of a model setup.py set up (#3570)
--show-key only knew a running launch or a preset. A model set up by setup.py (START-HERE.bat, ./setup.sh) keeps its generated key in strata-<model>.json, so the command
said there was no key and people read the JSON by hand. With no launch and no --preset, it now looks at the run configs of the installed models: one model with a key
prints that key (stdout is the key alone; a hint line with the model and the file goes to stderr), several models ask for --model NAME (any case, the size or the full
id), a name that matches nothing says so. --preset never falls back to a setup config, and no key is printed while the choice is open. Tests:
tests/terminal/test_launch_show_key_setup.py.
2026-10-26 19:45:45 +01:00
Niko1221 726472a229 Merge f-027-int2 into f-027-app2 2026-10-26 19:43:59 +01:00
Niko1221 10a62b8d2f docker: model files left in the older /data/models stop the start instead of being downloaded again (#3421)
A volume or bind mount from the 0.1.x image keeps its models in /data/models. The entry point moves a folder that is part of the volume, but a folder the person
mounted there is left alone, and a failed move leaves the files too; with no MODEL_PATH the start then did not see them and downloaded the model again
(tens of GB) without a word. Now, when .gguf files are still in /data/models and MODEL_PATH is not set, the container stops with code 2 and names the folder,
says what to do (MODEL_PATH to the folder with the shards, or the files into /data/Strata-data/models) and that nothing was downloaded.
ALLOW_DOWNLOAD=1 goes on anyway with the same text as a warning. docs/DOCKER.md lists the setting. The other points of the issue (the key, the removed
variables, the status loop) are the owner's design questions and are not changed here. Tested by running the guard function with sh (tests/terminal/test_docker_files.py);
the whole container was not run (no docker on this PC).
2026-10-26 19:42:58 +01:00
Niko1221 690172c464 Split serving, first version: GEN 0 prefill=1 in the engine, serve/split.py (SplitEngine, config split_serve / --split-serve), tests
Prefill on one card, decode on another, the session handed over in RAM through the engine's
own session file (SAVE/RESTORE). The engine's new request form GEN 0 prefill=1 reads a prompt
as a request does, up to where the first verify window starts, and stops with the session at
n - 1 tokens, which is the place a continuing request resumes from. Opt-in; without
split_serve nothing changes.

Idea and manual setup by @MarcoTool in #4000.
2026-10-26 19:42:41 +01:00
Niko1221 4a759f82d4 DeepSeek engine packages: add sm_89 (RTX 40) to the CUDA builds (#3935)
Problem: the strata-dsv41 packages were built for sm_86 and sm_120 (+ PTX 120). An RTX 4090 user was stopped at setup on 0.2.4.1; since
b750d757f an sm_86 package is accepted on an sm_89 card (cubin same-major rule), but the engine had no sm_89 image of its own, and the docs said
nothing about RTX 40 or about building the engine for a card the packages do not cover. The reporter built strata-dsv41 for 86-real;89-real
themselves and ran the 2.00 set at 40.2 tok/s on an RTX 4090 48 GB (theirs, --dspark on).

What changed:
- packaging/build_dsv41_cuda.sh and build_dsv41_windows.bat: the default architectures are now 86-real;89-real;120, so the DeepSeek CUDA 13
  packages list sm_86, sm_89, sm_120, compute_120 (Windows and Linux alike). The CUDA 12.9 line in the script header says ARCH="86-real;89".
- CMakeLists.txt: a build of strata-dsv41 with no -DCMAKE_CUDA_ARCHITECTURES defaults to 86-real;89-real;120 (the Qwen engine's default
  stays 120, and the warning now names the architectures it picked).
- strata_control/engine_component.py arch_native(): true when a package holds machine code for the card itself. resolve() and
  feed.select_engine() use it as the last tie-breaker, so with otherwise equal packages the one with the sm_89 image is taken on an RTX 40;
  an sm_86-only package still fits (no refusal).
- setup.py: the "engine not available" stop now points at the build recipe (docs/DEEPSEEK41.md); the developer line names the
  architecture to pass. The card list already said RTX 30, 40 and 50.
- docs/DEEPSEEK41.md, docs/MODELS.md: RTX 40 coverage, the reporter's measurement quoted as theirs, no claim of our own (we have no Ada card),
  and a section "Building the engine for your card".

Tests: tests/terminal/test_engine_manifest.py (arch_native; the build scripts and the CMake default name sm_89 and both OSes agree; the
package list they give is sm_86 sm_89 sm_120 compute_120; an sm_89 card is accepted and prefers the package with sm_89). tools/test_setup_dsv41.py
unchanged and passing.

Not covered: no run on an Ada card (gate script in phaseA-tests/gh027/gates/3935-ada-dsv41.sh).
2026-10-26 19:42:11 +01:00
Norwin ab48216a7b docs/MULTI_GPU.md: say when a layer split pays on a slow CPU (#3930)
One bullet from PR #3930 by NorwinYu: on two identical RTX 2080 Ti over a dual-socket Xeon E5-2696 v4 a plain split moved decode from 35.2 to 51.0 tok/s and an
11.6K prompt from 672 to 908 tok/s, because the CPU pool per window fell from 9.52 to 0.79 ms (STRATA_DECODE_TIMING=1). The page's own 5080 + 3090
figures, where the second card is on par, came from a fast CPU; the bullet says so and that it is one measurement. No code changes: setup already splits
across two cards. Shipped without a hardware run of ours; the reply asks owners of two older cards on slow CPUs to compare and report.
2026-10-26 19:40:24 +01:00
Matthias Nott ad1a310f55 Web page: use wide windows and fit narrow screens (#3176)
The chat, About and Monitor columns grow with the window (the chat and the
composer to 1100 px, About to 1100 px, Monitor to 1920 px), the metric cards
wrap with auto-fit, and under 640 / 560 / 480 px the controls get bigger
touch targets, the composer bar wraps, and long words, code and tables stay
inside the viewport.

Ported by hand from PR #3176 by Matthias Nott (mnott); the CSS is his.
2026-10-26 19:40:00 +01:00
Vop3p c2490b49f5 docs/AMD_HIP.md: why STRATA_SH_STREAM goes both ways, and leave it off on Windows with KV streaming (#3895)
Two paragraphs from PR #3895 by Vop3p, put under the table row they explain: the fork pays only when it has a CPU-pool wait to hide (4,859 slots on one
RX 6900 XT, GPU-reach wait 20.7 of 39.1 ms; with every expert resident it is 43.4 against 62.9 tok/s on an R9700), and the Windows RX 6900 XT fault with
the fork plus --kv-resident, its bisection and its limits (one PC, kernel not isolated). The text now says what the code does about it and asks RX 6000 owners on Windows to test.
2026-10-26 19:39:43 +01:00
Niko1221 5f81454038 AMD on Windows: the shared-expert fork with KV streaming is not measured, never picked, and warned about when set by hand (#3895)
On one Windows RX 6900 XT STRATA_SH_STREAM=1 together with --kv-resident failed the engine on its first request (verify: unspecified launch failure).
setup --calibrate (tools/calibrate.py) no longer measures the fork on Windows when KV streaming is on (the fork is off by default on AMD, so
nothing changes unless the person sets it), strata tune's never-pick list holds the combination (strata_control/tune_knobs.py), and a config that
sets STRATA_SH_STREAM=1 with --kv-resident on Windows HIP gets one warning line when the engine starts (serve/server.py sh_stream_warning). Nothing is refused or
removed: the variable still reaches the engine. Shipped without a hardware run, from one report; the reply to #3895 asks RX 6000 owners on Windows to test
and report. Tests: tests/terminal/test_sh_stream_windows.py.
2026-10-26 19:39:42 +01:00
Niko1221 d98d8f977f docs/TROUBLESHOOTING.md: every entry of v0.2.4.2 is back, updated for 0.2.7
The 0.2.5 rewrite dropped about 25 entries: the Mac quarantine steps, strata doctor, sending logs, the first-start freeze, the graphics memory the app keeps free,
Smart App Control, UPDATE.bat and git pull (#1276), the build tools and the gcc/glibc compile problems, the engine checksum, the engine stopped unexpectedly,
slower than the tables, the empty-reply and Turing switches, --expert-cache and --resident-budget-gib, the page-locked OOM, the NVIDIA 0x141 reset, CUDA 13.2 on
RTX 50, the AMD section (KFD, the build list, 0xC0000005, the driver reset, pinned allocations, verify timeouts) and the body of the verify-timeout entry, which
the rewrite had left empty. Each is back with the 0.2.7 facts: the app's reserve defaults (native-resources.mjs), the Drafting setting, --vision on|off|gpu|cpu
in the Terminal and --vision no in a source checkout, Intel Arc's CPU reader, and the Terminal route next to the source-checkout one.

Every pointer resolves: setup.py's driver-reset pointer, the server's Smart App Control message, DETAILS.md's STRATA_KEEP_EMPTY_TURNS / STRATA_TOPK_STREAM line,
OLDER_GPUS.md and MAC.md. tests/terminal/test_troubleshooting_page.py checks the entries, those pointers and the page's own links and anchors.
2026-10-26 19:36:15 +01:00
ruibeikaa a338990186 qsa_pattn: sm_6x prompt attention without partials, opt-in STRATA_PATTN_SM60=1 (#3943)
Problem: below sm_70 the prompt reads each QSA layer's attention through the
decode kernels (qsa_prompt_attn_batch returns false without tensor cores): 32
queries a call, every chunk's partial results (817 KB a query) written to
scratch and merged by a second kernel.

Change: src/kernels/cuda/qsa_pattn.cu keeps the contributor's default variant
only, ps_score_x_kernel (q in registers, the chunk's K staged with 16-byte
loads, two cells at a time through a select-free reduce-scatter) and
ps_value_d4_kernel (four dims per thread, merged into the output as the merge
kernel does). Every float operation of attn_chunk_kernel_pre75 and
attn_merge_kernel is kept in its order, so the output is the decode kernels'
bit for bit. The PR's other variants (per-chunk, fused persistent, bench split
codes), pattn_bench and the variant argument are not ported; pattn_bench needs
a STRATA_QSA_DUMP file from a model run.

Gating: compiled only in the CUDA build with STRATA_EXPERIMENTAL_SM60 (HIP and
Metal link a stub that answers false; prefill.cpp keeps the call out of the
Metal build). At run time only with STRATA_PATTN_SM60=1, on compute capability
6.x, int8 K/V, head_dim 256, 12 query heads per KV head, at most 33 chunks.
Unset, the only cost on the default path is one cached int test per QSA layer,
and only when qsa_prompt_attn_batch has already declined. Any refusal or a
failed launch returns false and the usual 32-query loop runs (one warning, no
exit; the PR called exit(1) on a launch failure).

Tests: src/kernels/qsa_pattn_parity.cpp (new, registered in CMakeLists.txt in
the experimental sm_60 test block, skips with 77 when the kernels are absent)
compares the kernels with the Daily's chunk kernel plus attn_merge_kernel on
random int8 K/V: a shuffled page table with ~3% unresident pages and a run of
64 unresident pages (whole chunks masked, some queries empty), three selection
shapes, caps 64 / 130 / 2051 / 2112, widths 0..cap, n_q up to 40 with a
scratch that holds only 2 decode-layout queries (several batches), junk in the
scratch and output. On a Pascal card it also requires the checked entry to take
the shapes and qsa_decode_attn_batch to match. The kernel entry that skips the
compute-capability test (qsa_pattn_batch_any_arch) exists for this test only.
Results: z440 (RTX A4000, sm_86, CUDA 13.0, build with
-DSTRATA_EXPERIMENTAL_SM60=ON, gpu lock): "qsa_pattn_parity: 468 cases, 0
differ, 29724792 nonzero output values in all -> PASS". sm_60 compile of the
whole tree (CUDA 12.9, P1004): exit 0.

Not covered: no P100 run here (the Pascal-only checks of the test and the
speed claims wait for the gate, phaseA-tests/gh027/gates/3943-pattn.sh). The
contributor's numbers (43.3 -> 30.8 ms a 1,536-query chunk per QSA layer at
26K context, 2-3% less time on 8K-150K token prompts on four dies) were not
re-measured; compute capability 6.1 is covered by the same switch and was not
measured by anyone.
2026-10-26 19:24:50 +01:00
Niko1221 9db497056b docs: the two master switches in the 0.2.7 release-notes draft 2026-10-26 19:14:00 +01:00
Niko1221 a3ab37fc03 Master switches DS4_DEFAULTS=026 (DeepSeek) and STRATA_DEFAULTS=026 (Qwen): every default changed since v0.2.6 gives v0.2.6's behaviour; explicit per-feature variables and flags still win; every rule is a pure function with a v026 argument and a test 2026-10-26 19:13:25 +01:00
Niko1221 6f1bd21e59 App: "Faster answers on 12 GB NVIDIA cards" row; DeepSeek speed-ups Off also writes DS4_DEFAULTS=026
The Speed group gets a row for Qwen on a discrete CUDA card of 13 GiB or less: "Prefers the parts of the model already on the graphics card, which makes answers
much faster on 12 GB cards. Turn off if answers look worse than before." On (recommended) by default; Off writes STRATA_ROUTE_PRIOR=0, On removes it. The
DeepSeek speed-ups row, when Off, writes DS4_DEFAULTS=026 and the six DS4_*=0; On removes all of them, and a file that holds only the six (an earlier test
build) shows as Off. The Qwen "0.2.6 behaviour" switch is Terminal-only and is not in the app. New strings fx82 and fx83 in the eight fix catalogs.
2026-10-26 19:10:55 +01:00
Niko1221 211fca87df Shared speed-switch table and Terminal: DS4_DEFAULTS=026 with the DeepSeek off switch, the 12 GB NVIDIA row, and a Qwen "0.2.6 behaviour" row (Advanced only)
- DeepSeek speed-ups Off now writes DS4_DEFAULTS=026 (the engine's master switch back to the 0.2.6 defaults) plus the six DS4_*=0 for older engines; On removes
  all seven. A file with only the six (an earlier test build) still reads as Off and On removes them (`legacy` entries).
- New switch "Faster answers on 12 GB NVIDIA cards" (the residency-biased routing default): on by default, Off writes STRATA_ROUTE_PRIOR=0. Applies to Qwen
  Flash-Next (no arch) on a CUDA build with a discrete NVIDIA card of 13 GiB or less (not a chip that shares system memory), the engine's own rule.
- New Terminal-only switch "Qwen: 0.2.6 behaviour" (STRATA_DEFAULTS=026, off by default), a row of the Advanced table for every Qwen model, for
  troubleshooting; the app never shows it. DeepSeek has DS4_DEFAULTS instead.
tests/terminal/test_speed_switches.py covers the new defaults, applicability, the seven DeepSeek entries and the legacy six, the Advanced rows and the
comparison with the app's table (the Terminal-only switch is left out of it).
2026-10-26 19:10:55 +01:00
xiezhipeng-gitandLeonardo Murillo 9ea044962a OpenAI logprobs, top_logprobs and allowed_token_ids on /v1/chat/completions (#3200)
/v1/chat/completions on the CUDA and HIP engine accepted logprobs and
returned an empty choices[0].logprobs.content. Now "logprobs": true with
"top_logprobs" 0 to 20 (and the legacy "logprobs": N) returns one entry
per token with its alternatives, streamed or not, and "allowed_token_ids"
returns a distribution renormalised over exactly the ids the caller named.

Engine (src/program/generate.cpp): the GEN line takes logprobs=K and
allowed=a,b,c. After each "T id" the engine writes one "LP <logprob>
[id:logprob ...]" line from the verify window's head logits before
sampling, penalties and temperature. The SYCL engine already wrote LP
lines for logprobs=K; it now reads allowed= as well, with the same line
format. A request without logprobs reaches none of this: both call sites
are behind "req_logprobs >= 0", a local int of the request, so the default
path prints what it printed before and does no extra work per token.
Changes from the PR: the call sites are guarded instead of calling the
function for every token, the function is static, the token index is
checked against the row, and the Chinese comment is in English.

Server (serve/server.py): LP lines are read before the token is yielded
and belong to the request that asked (tagged with its cancel Event), the
entries spell the reply, json_schema replies return no logprobs rather
than a mismatching set, an allowed id past the vocabulary is a 400, and
logprobs on a --parallel server is answered for max_tokens 1 only (a
slot's BT lines carry no LP line). serve/structured.py: _extract_json is
now extract_json (the logprobs alignment uses it); the two uses added by
the #2893 retry follow the rename.

Left out: the PR's bench and result files (none in the diff). #3332 (/v1/completions) and #4056
(logprobs on it) build on this change; this one needs nothing from them.

Tests: serve.test_logprobs and serve.test_logprobs_parallel (30 tests with
the PR's), serve.test_structured, serve.test_responses: 94 tests OK.
Not covered: no model ran here. The PR author reports 28 of 28 on an RTX
5080 with MTP and on a GTX 1080 Ti; the layer-split pipelined decode path
(the first call site) was not run by anyone, and the LP values were not
compared with a CPU softmax of dumped logits.

Co-authored-by: Leonardo Murillo <[email protected]>
2026-10-26 19:08:35 +01:00
Niko1221androdi4eg 2299c8f70a native_mmvq: sm_89 MMVQ-IL tables opt-in with STRATA_MMVQ_IL_89=1 (#3453)
Review of the first port: it registered kIlRows_89 / kIl1Rows_89 for every
compute capability 8.9 card by default. Bit-identity of an interleaved-kernel
choice to native_mmvq is a property of the compiled code, and the release
engine builds its own sm_89 cubin that no card of ours can run; the issue also
saw no end-to-end decode change on a 4090 or a 4080, and the 4070 and 4080
runs agree in 53 of the 75 cells of the 2-4 column table (33 of them non-zero). So the default
change has no measured benefit and an unproven bitwise claim on the card it
changes.

Change: sm_89 answers the shared table, with no one-column row, exactly as
before, unless STRATA_MMVQ_IL_89=1 is set (read once, like the neighbouring
switches). With it set the cells from the two community runs apply. Nothing
else in the lookup changes; sm_86, sm_120 and the others are untouched.

Why it is bitwise: native_mmvq_il_kernel keeps a separate accumulator set per
row (acc[v][i][c]) and sums each row over the same chunk order and the same
warp_sum, so the rows-a-warp count R only changes how many rows a warp walks,
not the order of any row's float additions; the plain-versus-interleaved
equality is what mmvq_il_parity measures (memcmp against native_mmvq for rows
1/2/4, T 1-8, odd row counts). That test passes on sm_86; it has not run on an
Ada card.

Tests: mmvq_il_parity now checks the unset behaviour (sm_89 equals the shared
table, Q3_K and the one-column row answer 0) or, with STRATA_MMVQ_IL_89=1, the
4070/4080 agreement. A second ctest, mmvq_il_parity_89table, runs the table
check with the switch set and no kernel case. On z440 (2x RTX A4000, sm_86,
CUDA 13.0, gpu lock): mmvq_il_parity and mmvq_il_parity_89table both pass, and
the binary prints "mmvq_il_parity: OK (every case, T 1-8, rows 1/2/4 and the
table, bitwise)" with the switch unset and set.

Model gate owed on an sm_89 card: phaseA-tests/gh027/gates/3453-sm89-il.sh
(mmvq_il_parity on the card, then identity of the answers with the switch on,
then a speed A/B that decides whether the default flips).

Co-authored-by: rodi4eg <[email protected]>
2026-10-26 19:07:23 +01:00
SuperDistance a3bb38ec45 batch-groups: opt-in adaptive-tier rounds in the pipelined loop, STRATA_PIPE_ADAPT_EVERY (#4010)
With --batch-groups above 1 on a layer split the pipelined batch loop never starts a round of the adaptive VRAM tier (the
serial batch windows start one every --adapt-every windows). While the slots are busy the tier only moves when they go back
to the solo path, so a fresh engine keeps its start-up profile for minutes.

STRATA_PIPE_ADAPT_EVERY=N (opt-in; unset, 0 or not a number is off): after N group windows pump starts no new group on the
first stage, lets the groups in flight finish, then runs one round as batch_step does between serial windows, adapt() and
apply_pending(true), so nothing in flight reads a cache slot while experts swap and the new experts have landed before the
next launch. Not while a prompt is read beside the slots (in_prompt_read, its loan of cache slots), not with --adapt-async,
not while the last round's copies are in flight, not with the tier off. The rule is two pure functions in
include/strata/program/batch_groups.hpp (pipe_adapt_every, pipe_adapt_due) with 12 new cases in batch_groups_test. With the
switch set the engine says so once at start and the "strata batch (pipelined ...)" line adds the rounds, the experts
swapped in and the time spent. docs/BATCHING.md documents the switch with the author's table (2x RTX 5090, six streams,
309 to 410 rows/s over eight bursts after a fresh start, two engines per setting).

Reviewed: unset costs one counter increment per group window and one false check where a group starts, and prints no new
line. A due round cannot wait forever: no new group starts, the groups in flight finish, then the round runs. A prompt read
sets in_prompt_read before the reader thread starts (including STRATA_PIPED_CONCURRENT) and clears it after the reader
joined; the drain after the read runs the round if it is due.

Tests: batch_groups_test (the new cases). Not covered and still the author's to show (the PR is a draft): a soak with
prompts read beside the slots, a Linux run, another GPU pair, more than two stages, and exactness settings with the switch
set. The model gate is written down.
2026-10-26 19:03:48 +01:00
Niko1221 557aad0265 hip-<package> is accepted wherever the backend is named (#4013)
STRATA_ENGINE_BACKEND=hip-rdna2 worked for strata update (updater.detect_hardware), but strata update --backend hip-rdna2 offered no package and
strata launch stopped with "STRATA_ENGINE_BACKEND='hip-rdna2' is not one of cpu, cuda, hip, metal, sycl". engine_component.split_backend reads
hip-<variant> as the hip family and its package; detect_backend (the start path), detect_hardware and update --backend all use it. Any other name is
checked as before. Tests in tests/terminal/test_gfx_selection.py.
2026-10-26 19:03:30 +01:00
SanghunYun 2b13a44933 expert source: STRATA_RAM_ADAPT copies the incoming expert out of its stage buffer when it is still there, and moves up to 4 pairs a layer (#3060)
STRATA_RAM_ADAPT (opt-in, --serve, one GPU, needs --adapt-every) lets the resident RAM copy follow the conversation:
experts the windows keep routing to take the RAM places of unused experts of their layer. Each move re-read the incoming
expert from the file and a layer moved one pair per window, although the incoming expert was usually read for the window
before and its blob still sits in that read's stage buffer.

What changed:
- FileExpertSource::replace_resident_staged(layer, in, out): the same move as replace_resident, with the bytes copied out
  of the finished stage buffer of `in` (no file read). The copy runs under stage_mu_, so claim_stage cannot give the buffer
  to another blob meanwhile; it needs a non-busy buffer whose key is `in`, and returns false (nothing changed) otherwise,
  and the pair then goes the old way. The Foresight hold/unhold around the copy is the one install_resident uses.
- generate.cpp: a layer pairs up to 4 of its most routed outside experts with up to 4 of its least routed ones in the copy
  (outgoing count below 0.5, incoming 2 or more), largest difference first. A pair whose incoming expert is staged moves by
  copy; `n` now caps only the pairs that need a file read. The log line counts the moves and how many came out of a stage
  buffer (ram_staged()).
- docs/DETAILS.md: the RAM_ADAPT bullet in the low-RAM list in plain words and one row of the measured table (the author's
  numbers: RTX 3060, IQ2_XS and IQ3_XXS, 120 ShareGPT requests, 3 interleaved runs each).

Unset (the default) nothing runs: the block is guarded by ram_adapt_n > 0 as before. The weights moved are byte copies of the
same blob; which experts sit in RAM changes, as it already does with RAM_ADAPT, so greedy token ids may differ from run to
run once it is on. Changed from the PR: the DETAILS.md hunk was rewritten against the current, shorter bullet.

Tests: file_expert_source_test (new staged cases; the positive case needs unbuffered load and is skipped without it).
Not covered: no model run. Model gate written down: RTX 3060 ShareGPT replay =8 against =1.
2026-10-26 19:02:48 +01:00
Niko1221 a3b7d9de3f docs/AMD_HIP.md: a hand-built engine has no hipBLASLt tuning table unless it is pointed at one (#4034)
The Build section said nothing about it. gemm.cu (create_hipblaslt_state, own_tuning_dir) takes STRATA_HIPBLASLT_TUNING, else the file
<engine folder>/tuning/<arch>-hipblaslt-<version>.txt, else plain hipBLAS. The report measured 826 against 1,043 tok/s at 8K prompts on a
gfx1151 (one run per arm, stated as such) and the log line that tells whether a table was taken. The extra log line the issue suggests for the
no-table case is a small engine change and is not part of this commit.
2026-10-26 19:01:50 +01:00
Niko1221 c966d7ad89 setup: Maxwell gets a warning with the build recipe, not an env switch (#3218)
The first #3218 commit made setup.py admit a Maxwell (sm_52) card only
when a tester set STRATA_EXPERIMENTAL_SM52=1, and then routed it to the
CUDA 12 engine. That engine holds no sm_52 code, and users never set
environment variables to get a fix.

sm60_card() is back to Pascal and Volta (6.0 to 7.0). A PC whose only
NVIDIA cards are Maxwell now gets a warning (maxwell_note) that says no
ready-made engine holds code for it, gives the source-build recipe
(-DSTRATA_EXPERIMENTAL_SM60=ON -DCMAKE_CUDA_ARCHITECTURES=52, CUDA 12.x)
and the one reporter's result, and points to docs/OLDER_GPUS.md. The card
table line says Maxwell runs only from a source build. A Maxwell card
beside a newer card prints nothing new. Naming the card (--gpu, --cuda
12) or any env var does not admit it. The CMake floor from the first
commit is unchanged (opt-in build only).

Tests: tools.test_setup_older_gpus 23 tests OK (a Maxwell-only PC through
setup.main() with and without the old switches installs nothing and
prints the recipe; a Maxwell card beside an RTX prints no note).
Not covered: no engine was built for sm_52 by this commit; the P1004
compile is recorded in the notes.
2026-10-26 19:01:27 +01:00
JoShMiQueL c0845b7fc5 bench: tools/strata_bench.py reads the CUDA UMD version, the storage, the engine's build toolkit and GPU power and PCIe detail (#3250, #2111)
The report written by BENCH.bat and bench.sh had fields missing or wrong: nvidia-smi's 'CUDA UMD Version:' (current drivers) was not read, the value
was the newest CUDA the driver supports but was labelled like the engine's toolkit, storage always said 'not measured', and Windows 11 showed as
Windows 10. Now the report says 'CUDA (driver supports up to): X; engine built for CUDA Y' (BUILD.json beside the engine), names the storage device
(Get-PhysicalDisk, system_profiler, lsblk -e 7 to skip snap loop devices), adds a 'GPU detail (after the run, near idle)' line with power limit and draw,
PCIe generation and width, pstate and temperature, and recognises Windows 11. The standard report (strata bench run, strata_cli/bench_facts.py) already
read the UMD version, storage and power limit; this brings the legacy tool to the same. Ported from PR #3250 by JoShMiQueL with its tests.
2026-10-26 19:01:17 +01:00
Norm 7783b1c18e docs: a crashed engine start names itself, and a setup run again keeps the installed size (#3243)
Two 0.1.44.1 changes no page told. TROUBLESHOOTING.md gets the start error 'the engine process ended: ...' (serve/server.py start_exit_note: the signal or
Windows exit code, and for an illegal instruction the AVX, AVX2, FMA and F16C flags the CPU lacks). SOURCE_INSTALL.md says a setup run again
pre-selects the installed size, marked '(installed)', and --yes takes it (setup.py size_tag, installed_size). Written by hand from PR #3243 by
Buddychek, whose hunks no longer fit the rewritten pages; each sentence checked against the two functions.
2026-10-26 19:00:14 +01:00
Niko1221 f217b23bd3 prefill: the copy-wait dedupe skips a wait only when both slots were released in the current walk and the ring is longer than one group (#4005)
Review fix on the #4005 port. STRATA_PREFILL_COPY_WAIT_DEDUPE=1 skipped the copy stream's wait on a slot's release
event when the previous entry had waited on the same event. Two cases break that:
- At idx == ring the previous entry (ring - 1) is still in the first round, so its wait was on the event of an earlier
  walk. The same slot id can be this walk's first flush, which re-records the event after that wait; skipping the wait for
  slot 0 then lets the next copy overwrite a slot that the flush's gather has not finished reading.
- With a ring of one MMQ group (16, the smallest ring) a flush of the whole ring re-records the event right after the
  previous slot's wait.
The rule is now copy_wait_dedupe_ok() in include/strata/prefill/copy_events.hpp: idx > ring, ring > group, same event.

copy_events_test models two to three walks with random group sizes and random issuer/consumer interleaving and checks that
a refill always waits on a record at or after the flush that released its slot. The same model with the PR's original rule
(idx >= ring) finds the stale-event case, so the test can fail. The default path is unchanged (dedupe off).

Tests: copy_events_test passes (z440, g++ -std=c++20). Not covered: a GPU run.
2026-10-26 18:59:50 +01:00
Niko1221 cea3ed40db serve: a slot's shorter checkpoints come along at a give-back only when they are prefixes of the tokens copied back (#2526)
Review fix on the #2526 port. At an admission from a slot the port copies every checkpoint of the slot that is not
longer than the copied-back length into the main session's chain. Such a checkpoint is valid there only when its tokens
are a prefix of what was copied back (its K/V cells are then the ones the main session holds) and it has no pictures
(the main session's pictures are cleared at a give-back). The PR did not check either; a checkpoint left over from a
branch the slot no longer holds would sit in the main chain with cells that are not its own.

Both trees (CUDA/HIP and SYCL) now test the prefix and the empty picture list per checkpoint. The checkpoint the
request continues from is copied as before. Nothing changes when every checkpoint is a prefix, which is the normal case.

Tests: compiles on CUDA and SYCL (see the build lines of the branch). Not covered: no model run; the gate is written down.
2026-10-26 18:59:50 +01:00
Yuu 05c1dabf1a docs: the xe expert tier has no SSD fallback under the wrapper default, and a native pack needs --spec 2 or more (#2153, #1796)
docs/INTEL.md and docs/INTEL_ARC.md described an SSD expert tier that strata-sycl.sh's default (STRATA_VERIFY_NO_HOST=1) does not have: every
expert must be in VRAM or in the pinned RAM mirror, and the engine refuses to start otherwise (sycl/src/program/generate.cpp, 'REFUSED: ... are
neither in VRAM nor mirrored'). A native (IQ) pack refuses --spec 0 and 1 (sycl/src/program/generate.cpp and src/program/generate.cpp). Ported by hand
from PR #2153 by nekomario28 onto the rewritten pages, each sentence checked against those two messages. The hunk for docs/TROUBLESHOOTING.md is left
out: the entry it changed is no longer on that page.
2026-10-26 18:59:27 +01:00
oscarlius e21a1872f7 docs/BATCHING.md: the cache settings that let a long solo request move into a batch slot without reading its prompt again (#3334)
Adds a minimal handoff configuration (--prompt-cache 1, --prompt-cache-every 0, --prompt-cache-root 0, --conversation-cache-mib 0), what each
option does, that it costs RAM, that it does not make cold prompts prefill in parallel, and how to look for the handoff in the engine log, /metrics
and /v1/status. Ported from PR #3334 by oscarlius (continuing #3035). The four options exist in src/program/generate.cpp and the o.prompt_cache > 0
gate matches the text; the log evidence was not reproduced here (no model run).
2026-10-26 18:57:53 +01:00
stuchapin909 6c09a1929c CI: build the Metal engine on macos-26 (#3870, #3824)
metal/src/mtl_rt.mm names MTLGPUFamilyMetal4 (macOS 26 SDK) and MTLCompileOptions.mathMode (macOS 15 SDK) behind @available, a run-time check;
the names must still exist in the SDK the engine is compiled against, so the metal-link job failed on the macOS 14 SDK on every pull request that
touched src/, include/ or metal/. The job now runs on macos-26 (the runner-images repository lists a macos-26 image). Ported from PR #3870
by stuchapin909. A GitHub Actions run cannot be made here: the author ran this workflow on his fork; the first run after the merge proves it.
#3824 changes the same line with macos-15 and an Xcode 26.0.1 path pin, and is not taken.
2026-10-26 18:57:30 +01:00
Heinz N. Gies 5bc1e24696 kq_cpu_bench: i-quant pairs no longer crash, and Q2_0 is timed on the CPU pool's path (#3871)
make_blob quantized its random test rows with ggml-cpu's type traits from_float, which the i-quants do not have: an IQ3_S pair called a null
pointer (an access violation on Windows, SIGSEGV on Linux) before the first timing. It now uses ggml_quantize_chunk, with a flat importance
matrix where the type needs one. A Q2_0 gate/up or down is timed as pool.cpp runs it (activations as an ActQ, Strata's own Q2_0 rows, SwiGLU
in between) instead of with ggml-cpu's Q2_0 dot, and the pair's header line says so. Other formats are timed as before. Ported from PR #3871 by
Licenser.

Built on this PC (MSVC, CPU only) and run: iq3_s/q2_0 0.531, iq2_xs/q2_0 0.336 and q2_0/q2_0 0.214 ms per expert per thread (--mb 512, 1 thread).
2026-10-26 18:57:13 +01:00
Highlander 0fbbf068eb nvfp4 local pack: read the checkpoint with cached sequential reads, not a process-lifetime np.memmap (#3377, #3378)
Checkpoint.raw mapped every shard for the life of the process. On Windows each faulted page of the ~64 GB of expert bytes is charged to the
process working set, so the pack step looked like a 64 GB RAM demand and was killed by free-RAM watchers on a PC with a model loaded (nothing
leaked: it all came back at once). raw() now reads the tensor's byte range into a fresh buffer through one cached file handle per shard; the
pages go to the system file cache, which counts as available. The author measured the same bytes, 1.2x faster, flat working set, and a
byte-identical pack. raw() is no longer thread-safe (this file uses no threads). Ported from PR #3378 by jerry78424 (the fix for his issue #3377)
with its two tests; I ran the tests (13 pass); the full 102 GB pack was not run.
2026-10-26 18:56:54 +01:00
Keyan Hu 409f2466ad tools/gguf_reader.py: bound the header reads and report malformed metadata as ValueError (#2288, #1919)
A GGUF header with an unknown metadata type or an impossible length raised KeyError or OverflowError inside the reader, and setup, which expects
a ValueError, printed a traceback instead of its diagnostic naming the broken shard. Strings, scalars, arrays and tensor dimensions now share
one bounded read; counts are checked against the bytes left in the file before anything is constructed; type ids are checked before their values
are decoded; nested arrays are reported as unsupported; general.alignment must be a positive multiple of 8. The file name is added once to the
message. Ported from PR #2288 by KeyanHu-git with its tests (tools/test_gguf_reader_malformed.py).

Checked against the old reader on the 18 GGUF v3 vocabulary files of the pinned llama.cpp: same metadata, tensors and data start.
2026-10-26 18:55:57 +01:00
Jakub Luwierski 11237f831f serve: Chat Completions tool results keep the order of the calls they answer (#3395)
When one assistant message made several tool calls and the client returned the results in another order, the call ids were dropped before the
prompt was rendered, so the model saw anonymous results in the wrong order. The ids are now kept while the request is normalized; a run of
results is reordered to the call order only when the call ids and result ids are non-empty unique strings with matching sets. Anything else
(missing, duplicate, unknown or incomplete ids) keeps the received order, and the ids never reach the template. Ported from PR #3395 by
j-luwierski with its seven tests (serve/test_chat_tool_results.py); the hunk was merged by hand beside require_messages.
2026-10-26 18:54:34 +01:00
rodi4egandkzhv bdecbffc6b native_mmvq: sm_89 (Ada) MMVQ-IL table from the cells two cards agree on (#3453)
Problem: an Ada card took the shared MMVQ-IL table, which has no Q3_K row, so
Q3_K shapes ran the plain kernel and the other types used a table measured on
sm_86 and sm_120.

Change: kIlRows_89 (2-4 columns) and kIl1Rows_89 (one column) are registered
for compute capability 8.9. The 2-4 column table keeps only the cells where the
RTX 4070 run (rodi4eg) and the RTX 4080 run (kzhv) gave the same rows count and
sets the rest to 0, the plain kernel; the 4070 and 4080 disagree in 13 of 15
cells of the unmerged tables. The one-column row is Q3_K only, which both cards
gave. The issue's 5-8 column table is not taken: the 4080's changed in 10-14 of
20 entries between two runs of the same card, so there is no kIlRows58_89. The
bench output files of the PR are not ported.

Every choice in a table is bitwise native_mmvq's output (kernel row split only),
so answers do not change; this is speed only. No end-to-end decode change was
seen on a 4090 or a 4080 in the issue, so the gain is the microbenchmark's.
This does change which kernel an sm_89 card runs by default, so the model gate
(identity on sm_89) is written down in phaseA-tests/gh027/gates/3453-sm89-il.sh.

Tests: mmvq_il_parity (registered in CMakeLists.txt) now checks sm_89's table
against the two source runs (check_tables) instead of expecting the shared
table, and still checks bitwise parity of every case and rows choice on the GPU
it runs on (sm_86 on z440). docs/MMVQ_IL_TABLE.md updated.

Not covered: no sm_89 card here, so the table speed is the contributors'
numbers, not ours.

Co-authored-by: kzhv <[email protected]>
2026-10-26 18:53:58 +01:00
i-Vladimir-i 092c7d1ef4 serve: return 503 when the elastic image encoder cannot start (#2435)
When the elastic image encoder failed to restart, its exception left request preparation and the client got a closed connection. It now goes
through the EngineDied path: HTTP 503 with the reason, a later picture tries again, text requests and a later good start are unaffected. An idle
encoder stop is logged even when the cache shrink was refused, and a layer split gets a line saying its encoder VRAM has to be kept free with
--vram-reserve-mib. Ported from PR #2435 by i-Vladimir-i with its tests (chat, messages and responses, streamed and not, plus a real encoder
that exits with code 71 on Linux) and the docs/VRAM_ELASTIC.md text.
2026-10-26 18:51:44 +01:00
Niko1221 a8c1d4b7f8 verify: a window graph that does not fit frees the graphs of other window sizes instead of ending the engine (#3702)
On a layer split the later card is left with a few dozen MiB after loading, and each single-request window size is
captured the first time it is used (68, 64, 58, 52 and then 10 MiB free in the report); when one capture's
cudaGraphInstantiate ran out of memory, instantiate_evicting could only free graphs of batch slot layouts, a
single-request run has none, and "verify: instantiate: out of memory" ended the engine.

Now, after the batch layouts, a single-request window's capture frees the graph of the largest other window size (never the
one being captured or a prelaunched one; pick_window_victim) and instantiates again, and says so once: the freed size is
captured again when it is next used. capture_all keeps every size at once and is excluded. Only a failed instantiate does
anything new; with enough VRAM nothing changes.

window_evict_test (CPU, header-only) covers the choice. The real gate needs a model on a 2-GPU split with a thin later card
(listed in the triage file); not changed here: the later-stage reserve of the split search (kWindowMib), which is a default
placement change.
2026-10-26 18:51:29 +01:00
Vop3p a11f8b0aa5 prefill: opt-in sparse copied events and deduplicated slot waits in the streamed prompt walk (#4005)
Under WDDM every cudaEventRecord and cross-stream wait is a submission of its own (the author measured ~80 / ~45 us each
on an RX 6900 XT under Windows 11), and the streamed walk paid one record and one wait per expert although a group
gathers after one wait on its last copy. Two tester switches, threaded issuer only:
- STRATA_PREFILL_COPY_EVENT_EVERY=N (2..64): the copied event is recorded every N-th entry, before the issuer waits and on
  the last; the compute stream waits on the first recorded entry at or after the one it needs (the copy stream is in order).
- STRATA_PREFILL_COPY_WAIT_DEDUPE=1: the copy stream skips its wait on a slot's release event when the previous slot was
  released by the same event.
The same copies into the same slots: the same bytes.

Default path (both unset): read line by line. The issuer records copied[slot] and waits on used[used_of[slot]] exactly as
before (the counters are bookkeeping, cover() is a no-op). Added to the PR so this holds by construction: wait_copied takes
the slot the old code used and, without the sparse switch, waits on copied[slot] at all three consumer sites (the PR derived
the slot from the entry number). The record/wait selection is now three pure functions in
include/strata/prefill/copy_events.hpp, with a host test (copy_events_test, registered in CMakeLists.txt) that models the
issuer and consumer with a full ring and checks that every wait names a live recorded entry at or after the one needed.

Tests: copy_events_test passes (z440, g++ -std=c++20). Not yet: a CUDA build of prefill.cpp in this tree (queued), no GPU
run. Not covered: the Windows HIP / WDDM speed claim and NVIDIA (the gate is written down). Needs the author's numbers
repeated before any default talk.
2026-10-26 18:50:44 +01:00
PurrSec eba3106c0e serve: support IPv6 bind addresses (#3212)
--host :: failed before the model loaded because Server always opened an IPv4 socket. Server now selects AF_INET6 when the bind address contains a
colon; the ready line, the 'from other devices' lines and the browser address bracket an IPv6 address. The bind is still the person's choice and
no default changes. Ported from PR #3212 by purrsec with its tests (serve/test_security.py, including a real :: bind). setup.py's --host help and
docs/DETAILS.md say :: is all IPv6 interfaces; the README hunks of the PR (they name strata launch --host, not checked) are left out.
2026-10-26 18:50:11 +01:00
Niko1221 61ede87e1b Maxwell sm_52 floor for the opt-in old-card build (#3218)
A reporter built Strata for two Quadro M6000 (sm_52) by lowering two
floors and saw about 22 tokens a second with Qwen3.8-Flash-Next IQ1.
CMakeLists.txt: the experimental build (-DSTRATA_EXPERIMENTAL_SM60=ON)
now accepts sm_52 and prints a warning for it (Maxwell, one reporter, no
native half precision, CUDA 12.x). Without the flag the build still
stops at sm_75 as before, and the architectures a build uses are not
changed.

setup.py: sm60_card() admits a 5.2 card only when the tester sets
STRATA_EXPERIMENTAL_SM52=1, because the ready-made CUDA 12 engine holds
no sm_52 code and a Maxwell card must not be routed to it by default.
With the variable set, setup warns about the Maxwell card instead of
refusing. Default: a Maxwell card is "not supported" exactly as before.

docs/OLDER_GPUS.md: a Maxwell row and a short section, with the build
command and a request for an answer-text check.

Tests: tools/test_setup_older_gpus.py (20 tests, new Maxwell cases for
default, opt-in and a newer card present). Not covered: no model run, and
the answer text on a Maxwell card is unchecked.
2026-10-26 18:48:59 +01:00
Keyan Hu a1d2fd7ed1 setup: use the published size when the download length probes fail (#2287)
When HEAD and the one-byte range probe both gave no length, download() took an early EOF as a finished file even though the repository pin (or
ModelScope's published size) names the size. The published size now stands in for the length, so the existing resume, retry and parallel code
publishes the file only after it has all its bytes; a size that disagrees with the pin is still refused before the transfer and the SHA-256
check is unchanged. Ported from PR #2287 by KeyanHu-git with its tests (tools/test_setup_download_length.py).
2026-10-26 18:48:49 +01:00
Günter 560cc9fbbc setup: --gguf-dir files for Qwen3.6 no longer need a .done mark made by setup (#3606)
install_own_path treated a file as present only when <file>.done existed, and that mark is written by setup's own download or by verify_sha256,
which a file from another tool (LM Studio) never reached: setup said it was missing and asked for the full download size in free disk. With
--gguf-dir an existing file and its mmproj go to verify_sha256(own=True), which checks size and SHA-256, leaves a mismatching file alone and
writes the mark. A file that is not in the folder is still reported missing. Ported from PR #3606 by ai-bits with its two tests.
2026-10-26 18:48:14 +01:00
Niko1221 96b6068c32 Fix the follow-up of #4014 (a variable name already used in Chat.tsx) and two of my tests (#3254 iterator, #3656 raw paths) 2026-10-26 18:48:09 +01:00
Xiaomi2008 3100ca05ed expert cache: STRATA_VRAM_FREE=device reads the whole card's free VRAM through NVML, Windows + NVIDIA, opt-in (#1911)
Under WDDM cudaMemGetInfo answers for the calling process: memory another process holds on the card (the image encoder,
a browser, another engine) is not in its free figure. The reporter measured, on an RTX 5090, 30,994 MiB free from CUDA
where NVML read 24,348 with 6 GiB held elsewhere, so --expert-cache auto sized the cache too large and shrank it by
guesses.

device_free_bytes() now takes min(CUDA's figure, NVML's figure for the whole card) when STRATA_VRAM_FREE=device is set.
nvml.dll is loaded from the system directory at first use (LoadLibraryExW with LOAD_LIBRARY_SEARCH_SYSTEM32), the card
is matched by its PCI bus id, and a missing nvml.dll, a missing export or a failed nvmlInit leaves CUDA's figure with
one line on stderr. The serve check of "VRAM free with everything loaded" uses the same function. The value can only get
smaller, so the switch can only shrink the expert cache.

Unset (the default), nothing changes: the NVML code is compiled only for _WIN32 and not for HIP, Metal or the no-VMM
build, and the env variable is read once. The serve check's min() is the same value elsewhere (on a unified-memory
system device_free_bytes() is never below cudaMemGetInfo's figure). Added to the PR: the NVML block is also left out
of a SYCL build (SYCL_LANGUAGE_VERSION), whose CUDA names are a compatibility layer. docs/DETAILS.md: one row in the
measured-and-left-off table instead of the PR's long paragraph.

Tests: builds on Linux CUDA (the Windows-only block is skipped), SYCL (P1006) and on Windows MSVC (owner PC build of
the final tree). Not covered: no run with a model; the Windows gate is written down.
2026-10-26 18:47:52 +01:00
Jakub Luwierski a332cc95db setup: an explicit --rope-scale must be finite and at least 1 (#3393)
resolve_rope accepted nan, inf, zero and values below 1 and handed them to the engine. It now stops with a message that names --rope-scale and the
valid range, before any configuration is written; valid factors and the automatic scaling are unchanged. Ported from PR #3393 by j-luwierski
(follow-up to #2950) with its tests; the --help text gets the range.
2026-10-26 18:47:46 +01:00
Avicennasis 3a3f35c64a tests: cuda_spill_guard fails the CUDA test build when a Strata kernel spills or keeps a per-thread array in local memory (#2683, #2682)
tools/check_cuda_spills.py reads the resource usage the compiler wrote into the kernel libraries (cuobjdump
--dump-resource-usage: STACK and LOCAL per entry function and per sm_) and fails on any Strata kernel outside its ALLOWED
table, which carries the measured allowance of the kernels that hold such an array on purpose; ggml's vendored kernels
(no namespace) are skipped and counted. ctest cuda_spill_guard runs it on eight libraries, skips (77) without cuobjdump or
a demangler, CUDA builds only (HIP has no cuobjdump); check_cuda_spills_unit runs tools/test_check_cuda_spills.py
(stand-in demanglers are sh scripts, so not on Windows).

Changes against the PR: (1) a demangler that exits 0 but prints a blank name used to make the kernel's key empty, which
reads as vendored, so a spilling Strata kernel was skipped; that is now "no verdict" (77) with a test. (2) The unit test is
registered. (3) The ALLOWED table was re-measured on f-027-int: sm_120 with CUDA 13.0 on Windows (MSVC) and sm_86 with the
CUDA 13.0 objects built on Linux (cuobjdump 12.9); allowances are only raised (new DFlash2 gemm8_kernel and
rope_table_kernel, native_gu_swiglu_kernel on sm_86, dequant_kernel / expdec_gu_kernel / prompt_attn_i8v3_kernel /
native_gu_swiglu_kernel on sm_120), none lowered or dropped. Result: 0 violations on both architectures; a test shows the
same object fails once one allowance is lowered.

Ported from PR #2683 (the tracking issue is #2682).
2026-10-26 18:47:35 +01:00
Niko1221 2cc6ea169b serve: opt-in LaTeX-safe JSON recovery for structured output (#2893)
A json_object or json_schema answer that holds LaTeX such as \alpha is
rejected (502) because \a, \g and the like are not JSON escapes. New
opt-in: "repair_latex_escapes": true in response_format (and in
text.format on /v1/responses). Only after the strict parse failed, the
server doubles each backslash that does not start a valid JSON escape and
parses once more with the same duplicate-key and NaN rules, then checks
the schema as before, and prints one log line. Without the option, and
for any answer that parses on the first try, nothing changes.

Escapes that are valid JSON (\b \f \n \r \t \/ \" \ and a well-formed
\uXXXX) are left alone, because they cannot be told from LaTeX: a \beta
next to a \alpha comes back with a backspace. Truncated output, arrays,
NaN and duplicate keys still fail.

Tests: three new tests in serve/test_structured.py (default still 502,
opt-in 200 and stream, failures stay failures, helper cases, Responses
format passthrough). serve/test_structured.py 13 tests OK.
2026-10-26 18:47:25 +01:00
SanghunYun 2251053b42 serve: the serial decode timing line shows the adaptive waits in its sum (#1864)
With STRATA_DECODE_TIMING=1 the serial line listed `verify + commit/emit + draft` after `ms/window =`, while the adaptive
wait before verify and the join after the draft were timed but printed after the sum, so the named parts did not add up
to ms/window. The serial line now reads `adapt wait + verify + commit/emit + draft + adapt join + other`, where `other` is
the decode time minus the named parts (not a timed step). The pipelined decode keeps its old line, byte for byte, and
nothing is printed with the timing off or with no windows. Diagnostic only: no decode path changes.

docs/DETAILS.md: a short paragraph on the new line (the hunk was rebased by hand onto the current file).

Tests: compiles on CUDA (the format string was counted against its 22 arguments); no parser in the repo reads
`ms/window = verify`. Not covered: no run of the line itself (it needs a decode).
2026-10-26 18:46:55 +01:00
l33tm4st3r c471588f3e bench: tools/strata_bench.py (BENCH.bat, bench.sh) names an AMD card on ROCm 7+ too (#3630)
The quick speed check has its own GPU probe, separate from strata_cli/bench_facts.py, and tried nvidia-smi then rocm-smi only. On a Radeon with
amd-smi alone it wrote the server's vendor stub and 'GPU driver: not read'. The same amd-smi probe as #3519 is added in the shape this file
uses, and a bare vendor name from the server no longer replaces a card the probe named. Ported from PR #3630 by l33tm4st3r.

The three new tests are POSIX-only (symlinks, /sys); run on z440 (Linux, Python 3.12): OK. tools/test_strata_bench.py on Windows: OK.
2026-10-26 18:46:33 +01:00
hotschmoe 01e404c140 serve: an idle batch slot's decoded conversation is parked before an admission overwrites it (#2526)
A batch slot holds the reply it decoded. The main session was parked with the conversation's prompt only (the
admission's copy), so when the next admission or a yielding read overwrote the slot, the reply lived nowhere: the
conversation's next turn found its prompt-read state and read the whole reply again (the author measured 255 of a
256-token reply re-read on four 200K chats over two slots).

What changed:
- park_current is split into park_source (one budget / RAM-admission / snapshot path) and two callers: park_current
  (the main session, output line and byte accounting as before) and the new park_slot (an idle slot's ids, checkpoints,
  sessions and draft K/V). The ParkSource passes what a park captures; the main session's retained K/V (#1503
  take_reuse_for) is taken only for the main session, never for a slot.
- park_slot runs before copy_to_slot at a slot admission and before a BYIELD read overwrites a slot. It does nothing
  when the live main session or a parked conversation already holds that state (ConversationCache::holds /
  conversation_covers, new, with unit tests), for a slot with pictures, or without --batch-mtp when the drafter is on.
- When a slot is admitted into the main session its checkpoints come along (the shorter ones too), so the chain keeps its
  branch points and the parked copies a turn back are superseded (#342) instead of taking an entry each.
- Same refactor in sycl/src/program/generate.cpp.

Changed from the PR: parking is an optimization like a checkpoint (#3350). The PR ended the request (ERR) or the batch
(batch_fatal) when park_slot's snapshot failed; here a failed snapshot, a failed device sync and a host allocation
that runs out are logged, the slot is overwritten as before and the request goes on. The slot's extra checkpoints that
come along at an admission are copied under a bad_alloc guard (the one the request continues from is copied as before).

Tests: conversation_cache_test (new holds() cases and the four-conversation replay at policy level) and
conversation_split_failure_test. Not covered: no model run (the batch path itself needs one); the gate is written down.
2026-10-26 18:46:05 +01:00
Niko1221 9c2d35d978 serve: /v1/responses accepts max_output_tokens of 0 and -1 (#4036)
The Responses validator still refused anything below 1 after the chat
path made max_tokens of -1 and 0 mean no cap (documented in
docs/DETAILS.md). It now accepts -1 and 0 as no cap; values below -1,
floats, booleans and strings stay a 400. Nothing else changes: the
server already passes max_output_tokens or 0 to prepare(), which gives
the answer the room left in the context.

Tests: OverHttp.test_max_output_tokens_zero_and_minus_one_mean_no_cap
(stream and non-stream, status completed) and
test_max_output_tokens_bad_values_are_400. The first fails without the
change. serve/test_responses.py 43 tests OK.
2026-10-26 18:45:04 +01:00
Niko1221 407f659763 docs: GR_V3 and PROMPT_ATTN_IMMA defaults in DETAILS.md and the 0.2.7 release-notes draft 2026-10-26 18:45:02 +01:00
ruibeikaa f213f78bae mtp: the drafter's Q8_0 draft head and projections on the Pascal GEMV, opt-in STRATA_MTP_Q8_SM60=1 (#3751)
Problem: with STRATA_Q8_SM60=1 the main model's Q8_0 matrices run on the GP100
Q8_0 kernel, but the MTP drafter's own Q8_0 draft head and projections kept the
generic kernels.

Change: with STRATA_MTP_Q8_SM60=1 (on top of STRATA_Q8_SM60=1) the drafter packs
its Q8_0 projections as packed copies, as the main model does (the prompt path's
draft_kv still reads the GGUF layout, so a copy is kept), and packs the draft
head's subset in place when every column count fits the kernel's shared-memory
planes and the data is 16-byte aligned (otherwise a packed copy). The in-place
pack writes the GGUF blocks back if the device write fails. Release unregisters
the entry without a cudaFree when the data was packed in place. New
native_q8_0_sm60_pack_quiet and native_q8_0_sm60_pack_in_place; HIP returns
false; metal/src/engine/stubs_0145.cpp gets matching stubs that answer
"nothing packed".

Replies do not depend on the drafts (they are verified); only the draft rounding
changes, because the Pascal kernel sums in another order.

Default path: mtp_q8_sm60() is false unless the env is set, so the packing calls
are skipped. The destructor now walks the tensor list once at teardown, not in a
hot loop. The SYCL tree has its own sycl/src/core/mtp.cpp, so it is not touched.

Tests: compile-only on sm_60 (CUDA 12.9) on P1004. Model gate (identical replies
with the flag on and off, decode speed on a P100) is written down in
phaseA-tests/gh027/gates/3751-mtp-q8-sm60.sh.

Not covered: no GPU run here; the contributor measured +4.4% on four GP100 dies
with all experts resident, one run per arm, not on a single streamed P100.
2026-10-26 18:43:53 +01:00
Dzmitry Senchankau 41188ec734 calibrate: an engine that dies mid-measurement drops only its candidate (#3317)
#1337 made a failed engine start in a later calibration step a candidate
that loses. An engine that started and then exited during its own rate
measurement still failed the whole run. measure_candidate() now treats
both the same: the candidate is dropped and recorded in
report["failed_starts"], the earlier results stay. It is used for the CPU
worker step, the adaptive expert tier step and the AMD shared-expert
fork. An operator cancel (CalibrationCancelled) is still raised. The first
start (steps 1 to 3) still fails the run.

src/core/verify.cpp: when adapt_gate_wait times out after 20 s, the
message now says which layer was reached (atomic and copy signal), that
the copy stream is not drained by the release, and dumps the
STRATA_VERIFY_TRACE breadcrumbs when that is on. Diagnostics only, #2188
stays open. The watchdog label is set only once a wait actually starts,
so a gate that is already open costs nothing extra (changed from the
PR, which set it on every call).

Tests: tools/test_calibrate.py 33 tests OK (3 new death tests).
Not covered: engine compile of verify.cpp is checked in the build step.
2026-10-26 18:43:46 +01:00
ruibeikaa 6a6d58ad03 iq_kernels: GP100 decode down projection staged in shared memory, opt-in STRATA_DOWN_WS=1 (#3867)
Problem: on a GP100 die (cc 6.0, no dp4a, no tensor cores) the verify window's
routed down projection for the native packs' IQ4_NL and Q2_0 rows (n_ff 640)
is latency bound. native_down_multi_kernel reads each weight as 2-byte loads,
one item at a time, straight from global memory.

Change: native_down_ws_kernel copies a block's rows (IQ4_NL 32, Q2_0 64, 11,520
bytes) into shared memory with 16-byte loads, then runs the same row dots on the
copy: same sub-warp, lane, items, float expressions and order. Only the integer
sum inside an item changes form (vmad byte selectors, prmt table lookups), the
same int8 x int8 products, so every output is native_down_multi_kernel's (expert
mode 0) bit for bit.

Gating: compiled only in the CUDA build with STRATA_EXPERIMENTAL_SM60 (not HIP,
not Metal). At run time only with STRATA_DOWN_WS=1 on a device of compute
capability exactly 6.0, IQ4_NL / Q2_0 down rows at n_ff 640, and not with
STRATA_OLD_IQ_MMVQ or STRATA_NO_SUB16_GU. Unset, the only change on the default
path is one cached boolean test per native_expert_grouped call.

docs/OLDER_GPUS.md lists the switch. The contributor's microbench numbers (one
PH402 die, Swift 1.5 IQ3_S, one-token window: IQ4_NL 72.6 us mode 8 and 54.5 us
mode 0 become 42.3 us; Q2_0 53.5 / 65.4 us become 38.6 us) were measured on a
four-GP100 rig, not here.

Tests: compile-only on sm_60 (CUDA 12.9) on P1004. The contributor's bitwise
check (native_expert_bench hash against modes 0 and 8) is not run here: it needs
a GP100 GPU run and the model's layers. The gate script is
phaseA-tests/gh027/gates/3867-down-ws.sh.

Not covered: no end-to-end measurement on a single P100 (16 GB cannot hold the
47 GB pack resident).
2026-10-26 18:43:32 +01:00
l33tm4st3r 2ac3482144 bench: the standard report names an AMD card and its ROCm runtime when only amd-smi exists (#3519)
ROCm 7 and newer ship amd-smi and no rocm-smi, and the amdgpu product_name is empty on gfx1201, so a Radeon AI PRO R9700 run was filed
under 'unknown' with 'runtime: not measured'. bench_facts reads the market name from amd-smi static --asic, the VRAM from the amdgpu sysfs
file of the PCI address amd-smi list reports, the ROCm version from the install (or amd-smi -h), and keeps the probe's card name when the
server only says 'AMD Radeon'. Ported from PR #3519 by l33tm4st3r with its tests and the BENCHMARK_PROTOCOL.md text.

Changed while porting: the amd-smi branch also took macOS (os.name is posix there), which would have hidden the Apple chip; it now skips
darwin, and a test keeps the chip.
2026-10-26 18:41:53 +01:00
Niko1221 9e3fd0f73a App: the DeepSeek card names the free disk the set needs, and the version line says "{N} GB RAM + VRAM, plus about {disk} GB of disk"
The "This computer" card said only "Fits fully in memory." for DeepSeek, with no word of the ~265 GB of disk. For a DeepSeek version it now says "Needs about {N} GB
of free disk space. What does not fit in memory is read from your SSD." (the spill line stays beside it when part of the model is read from the SSD), and the
selected version's line reads "{N} GB RAM + VRAM, plus about {disk} GB of disk". {disk} is not hard-coded: the catalog (tools/strata_mcp.py) gives each set
its `disk_gb`, the set's whole download (model files, n-gram table, drafter, metadata) plus the pack and room to work (dsv41_fit.DISK_HEADROOM_GB), so 2.00 and
2.25 differ. Same card and row markup, no design change; every other model keeps its card. Both sentences are fx80 and fx81 in the eight fix catalogs.
The Terminal's DeepSeek description makes no "fits fully in memory" claim (its lines already state the download and the free disk), so it is unchanged.
2026-10-26 18:41:17 +01:00
Niko1221 54d9363c97 CUDA Plan opt-ins become defaults where measured: STRATA_GR_V3 on a discrete 12.0 card above 13 GiB, STRATA_PROMPT_ATTN_IMMA on CUDA sm_80+ (env =0 turns each off); DeepSeek Faster mode (5 experts) by default on 2+ RDNA4 cards that hold half the experts; the route-prior/GR_V3/IMMA rules are pure functions with tests; gr_parity and fused_gr_bench pin STRATA_GR_V3=0 for their bitwise contract 2026-10-26 18:41:09 +01:00
Tsury fb966cff37 pack_gguf: accept older DeepSeek GGUF metadata names (#3955)
Older converters write the Engram scalars as engram.key_length and
engram.pad_id, and omit the sparse-attention geometry. The packer now
reads the old names only when the canonical engram.head_dim and
engram.pad_token_id are absent, and a new --config-json option fills the
omitted geometry from the source model's config.json. A value that
disagrees with metadata already in the GGUF stops the pack with a clear
message. The weights are never rewritten and a set that already has the
canonical names packs exactly as before.

Tests: LegacyModelMetadata (2 new, 5 in the file) in
tests/terminal/test_pack_gguf_engram.py.

Not covered: the author reports the older Q2_K set still needs dense
Q2/Q3 and Engram Q2_K tensor support in the engine; this does not add it.
2026-10-26 18:40:30 +01:00
Patrick Schaub a0fc88abd6 serve: timings on /v1/responses (#3291)
The final response.completed / response.incomplete event and a
non-streaming /v1/responses reply now carry the same top-level timings
object that /v1/chat/completions has. The server already put timings in
the done dict; the Responses assembler dropped them.

Tests: UsageAndStatus.test_responses (stream and non-stream).
Unset timings leave the event unchanged; response.failed has none.
2026-10-26 18:40:04 +01:00
Norm 5b14fe23f7 tests: three suite gaps a live PC showed (#3236)
test_iq_pack imports without llama.cpp's gguf-py and skips the classes that build GGUF fixtures with a named reason; the MCP discovery tests use
a free port, not whatever Strata serves on 8080; the six conversation-cache tests add the repo root to sys.path so they run from the repo root
as the suite does. Ported from PR #3236 by Buddychek (tests only, no shipped code).
2026-10-26 18:38:42 +01:00
Niko1221 f06bfc30e7 docs/TROUBLESHOOTING.md: the Windows AMD driver-reset entry is back
The 0.2.5 rewrite of the page dropped the 'Windows AMD: the driver resets' entry (v0.2.4.2 had it), although setup.py still tells a Windows gfx12
card owner to read it and tools/test_setup_amd.py (TdrPointer) checks for it, so that test failed on main. Restored from v0.2.4.2 with the
log and config locations worded for a source checkout.
2026-10-26 18:37:19 +01:00
Norm e6f598c0a2 docs: STRATA_VCVARS, /metrics/prometheus behind the API key, and the Monitor's prompt speed (#3244)
Three 0.1.42 changes no page told. Ported by hand from PR #3244 by Buddychek (the PR conflicted with the rewritten pages), each sentence
checked against the code: /metrics/prometheus needs the API key like /metrics (serve/server.py _authorized), the prompt-speed row is
(prompt_tokens - reused) / prompt_ms (apps/host/server/monitor-dashboard.mjs), STRATA_VCVARS is read by setup.find_vcvars. The compiler
text now says Visual Studio 2022 (2019 is refused since 0.2.5, #3566) and went to docs/SOURCE_INSTALL.md.
2026-10-26 18:36:43 +01:00
Niko1221 3478382eee conversation copies: the private copy streams and events are handed back to a pool when a thread ends (#3350, #3470)
conversation_detail::private_stream() and order_after_queued() kept one non-blocking stream and one event per device in
thread_local storage and never destroyed them. The layer split's later stages run each prompt chunk on a fresh
std::async thread (STRATA_HANDOFF_PIPE off, the default), and a checkpoint part is saved on that thread, so every
periodic checkpoint of every long cold prompt left a stream and an event behind on each later card: a few MiB of VRAM
per long prompt that never came back (#3470: the later card's free VRAM drops 2-6 MiB per long cold prompt, no plateau),
until cudaStreamCreate failed with "out of memory" (#3350: "conversation snapshot running-state copy: out of memory",
engine exit) on a card that was already nearly full.

A thread now hands its handles to a process-wide pool when it ends and the next thread takes them, so the number of
streams and events is the number of threads copying at once. Same stream, same flags, same contract; only the lifetime
changed. copy_streams_created() / copy_events_created() count what the process made.

conversation_copy_streams_test (CUDA, no model, skips without a device): 200 short-lived threads in a row create at most
one stream and one event; ten rounds of six concurrent threads at most six more; the same on a second device.
2026-10-26 18:36:32 +01:00
Niko1221 f62589fdbe serve: a checkpoint whose copy runs out of memory is skipped, it no longer ends the engine (#3350)
A conversation checkpoint is an optimization, like parking, but the writer treated a failed copy as fatal: "conversation
snapshot running-state copy: out of memory" (cudaErrorMemoryAllocation from the snapshot copy: the private copy stream's
first creation or the driver's staging on a nearly full card, not host RAM) made the prefill fail and the engine exit,
21 times in 4 h for one reporter and 16 in 34 h for another, each exit a reload and the in-flight requests lost.

Now, after a failed save, checkpoint_failure_is_survivable() clears the non-sticky allocation error and synchronizes the
device: when that is clean the checkpoint is skipped with a line on stderr and the request goes on (a layer split's stage
part is left empty, which already means "no checkpoint at this length"; the message-boundary / turn / root checkpoints
after a prompt chunk do the same). A device that is unwell (a sticky error, a GPU that stopped answering) still ends the
request exactly as before. Same edits in sycl/src/program/generate.cpp.

STRATA_TEST_COPY_FAIL_AT=N (tests only, like STRATA_TEST_SLOW_COPY_US): the N-th snapshot copy of the process fails with
cudaErrorMemoryAllocation, so the case can be made without a full card. Unset, nothing changes.
Not changed: "verify: instantiate: out of memory" (graph capture), which the reporter saw once.
2026-10-26 18:36:32 +01:00
Niko1221 164619c5a8 DeepSeek: the 4 x R9700 share-based defaults and the drafter cut apply when every device is RDNA4 (gfx1200/gfx1201), not to every HIP build (Impl::rdna4) 2026-10-26 18:36:12 +01:00
pschaub 3cd33d504c cmake: a CUDA build with no architecture option is for sm_120, and native builds the W4A4 unit for sm_120a (#4029, #4023)
CMakeLists.txt tested "NOT DEFINED CMAKE_CUDA_ARCHITECTURES" after enable_language(CUDA). Since CMake 3.18 that call
fills an unset variable with nvcc's default (sm_75 for CUDA 13), so the sm_120 default never applied: a hand build with
no option was an sm_75 build. The default is now set before enable_language, unless the user gave
-DCMAKE_CUDA_ARCHITECTURES or the CUDAARCHS environment variable, and the configure step prints a warning that says what
to pass for another card (a recommendation, not a refusal).

With -DCMAKE_CUDA_ARCHITECTURES=native, the one NVFP4 W4A4 unit (mmq_nvfp4_w4a4.cu) stayed on plain `native`, where
ptxas refuses the FP4 MMA on an RTX 50. `native` is now replaced by CMAKE_CUDA_ARCHITECTURES_NATIVE (the detected
architectures) before the 12x -> 12xa rewrite, as a list, so a mixed PC works too. docs/SOURCE_INSTALL.md says what
`native` and the other values build.

Release and setup builds pass their architectures explicitly and are unchanged. Configure matrix, CUDA 12.9 on Linux:
none -> 120 (+ the warning), CUDAARCHS=86 -> 86, 89, "75;86;120" (w4a4 120a only for the 12x entry), native -> detected.

Ported from PR #4029.
2026-10-26 18:35:58 +01:00
sergqwer 4a7e373da3 engine: --adapt-decay outside 0..1 falls back to 0.7 with a line on stderr (#1284)
--adapt-decay took any number. Above 1 the adaptive tier's usage counts grow without end (inf after a few hundred
adaptations) and the swap gains turn NaN, which breaks the sort that orders the swaps; a negative value flips the counts.
The value is now checked where it is parsed (include/strata/program/adapt_decay.hpp, also in sycl/): a finite value in 0..1
is used exactly as before (0 and 1 included, the range the calibration profile already allows); anything else, NaN and inf
too, runs with the default 0.7 and says so on stderr. A warning instead of the PR's exit 2: a risky pick is reported,
the engine still starts.

adapt_decay_test (25 checks) covers the range, the fallback and finite counts after 100000 adaptations.

Ported from PR #1284 (the check; the refusal became a fallback).
2026-10-26 18:35:58 +01:00
Niko1221 bb62a048b8 updater: a failed test start keeps the runtime records, not only the engine log tail (#3654)
The sandbox of a test start was deleted at once, so the PID records that show why a start failed were lost; only the last 64 KiB of the engine
log stayed. A failed start now also keeps run.json (without keys), the host, child and ready JSON records of the runtime (tokens cut to their
first 8 characters) and the host and server logs in .updates/logs/test-start-<version>-failed. The last three failed starts are kept.
Best effort, never hides the failure. Tests: tests/terminal/test_updater.py (FailedStartRecords).
2026-10-26 18:35:15 +01:00
Niko1221 484c689337 DeepSeek: the 4 x R9700 share-based defaults (small chunk for multi-card tp runs, whole-chunk slab, small staging ring, pinned copy at share >= 0.5) apply to HIP builds only; the staging ring takes a hip flag; tests for both branches 2026-10-26 18:33:37 +01:00
Niko1221 3b64abf5e0 DeepSeek drafter cumulative cut 0.5 by default on gfx1200/gfx1201 only (Model::gpus_are_gfx12); Strix Halo, other AMD and CUDA keep 0 until measured 2026-10-26 18:29:34 +01:00
Niko1221 c8596ec391 Merge f-027-marlin (DeepSeek on 4x R9700: staging ring from the room, copy pin auto, chunk 1024 where the cards hold most experts, whole-chunk attention slab, F16 gate/up activations, drafter cumulative cut on HIP, diagnostics). Conflicts resolved: attention slab = one rule (attn_slab_rows: Marlin's share >= 0.5 branch plus the automatic growth, unified memory and DS4_ATTN_SLAB=0/N untouched); staging ring = one function (staging_ring_mb: 16 MB where the cards hold the experts, Marlin's small per-device ring at 0.5..0.82, main's streamed 400..1200 MB on unified memory and under half); DS4_COPY_PIN auto = huge-page arena OR cards hold >= half; DS4_FAST_ATTN stays gfx12-only; chunk 1024 for multi-card tp runs too, but not for --threads on or one card on the default tp 2026-10-26 18:28:51 +01:00
Niko1221 bf994ca414 setup: a build folder configured with another generator is configured again instead of stopping cmake (#4038)
The manual build in docs/INSTALL.md uses CMake's default generator (Unix Makefiles on Linux) in build/, and setup --build always asks for
Ninja, so cmake stopped with 'Does not match the generator used previously'. cmake_build now reads CMAKE_GENERATOR from the folder's
CMakeCache.txt; when it is not Ninja it says so in one line and removes the cache and CMakeFiles (no other file). A Ninja folder or a new
folder is untouched. Tests: tools/test_setup_build_dir.py.
2026-10-26 18:28:23 +01:00
Niko1221 590c69a578 cmake/ggml_tq_overlay_test.cmake sets CMP0057 NEW so IN_LIST works under CMake 3.x (#3412)
A cmake -P script has no cmake_minimum_required, so CMake 3.x used the OLD behaviour and stopped at 'if given arguments ... IN_LIST: Unknown
arguments' (ctest ggml_tq_overlay with CMake 3.31). Checked with pip cmake 3.31.6 and 4.4.3 against the pinned llama.cpp: both print
'ggml_tq_overlay_test: ok'. A static test keeps the policy before the first IN_LIST in every cmake/*_test.cmake.
2026-10-26 18:27:27 +01:00
Niko1221 7507b47525 sycl/tools/build.sh exits with the build's own status (#2990)
The script ended with a grep pipeline, so a failed compile exited 0 (and, before the ocloc check was added at the end, a good build exited 1).
A build.sh && next-step chain then ran after a failure. The status of cmake --build is kept and the script exits with it. The test runs the real
script with a stub cmake on PATH (good build: 0, failed compile: 1; the old script exits 0 on the failed compile).
2026-10-26 18:26:40 +01:00
Niko1221 1be7eb77c1 tests: the setup golden compares the platform's vision binary names, and test_nvfp4_local_pack skips without pytest (#3325)
tools/test_setup_golden normalized strata.exe but not strata-vision.exe, so all 25 baseline entries failed on Linux (48 subtests). VEXE
and HVEXE are mapped to <VEXE> and <HVEXE> like EXE, and the baseline records <VEXE>. test_nvfp4_local_pack imports pytest, which is not in
requirements.txt; under python -m unittest discover the module is now skipped with a reason instead of failing to import.
2026-10-26 18:25:25 +01:00
ctr-137 800730103c tools/strata_bench.py: the redactor replaces a folder before the short user name inside it (#3619, #3326)
Substitutions were ordered by the length of their regular expression, and a user name wrapped in word bounds outranked the Strata or home
folder that contains it, so a report kept /home/<user>/Strata instead of <strata>. They are now ordered by the text they replace. Ported from
PR #3619 by ctr-137 with its test (Linux and Windows layouts).
2026-10-26 18:24:35 +01:00
Niko1221 9a0462a011 Chat: sending a message still scrolls to it when the reader had scrolled up (#4014) 2026-10-26 18:24:01 +01:00
Patrick Schaub bc939b6303 docs/SAFETY.md: strata launch updates an installed DeepSeek engine package by itself (#4028, #4022)
The page said that starting a model never downloads and that nothing checks for updates in the background. Since 0.2.4.2 strata launch with a
DeepSeek model checks the signed feed (10 s at most, silent without a connection) and replaces an older installed DeepSeek engine package.
Ported from PR #4028 by pschaub (wording from issue #4022), checked against strata_cli/launch/runner.py upgrade_dsv41_engine.
2026-10-26 18:23:38 +01:00
xxxliqu1dxxx 3cda075b51 setup (AMD, Linux): the engine probes get the engine's lib_dirs on LD_LIBRARY_PATH (#3305)
engine_env put the ROCm library folders first on PATH, which only Windows' loader reads. On AMD Linux strata --families exited 127 on
libhipblas.so.3, so setup falsely said the engine does not run Qwen3.6. On Linux and elsewhere the folders now go first on LD_LIBRARY_PATH
(an existing value is kept); Windows keeps PATH. Ported from PR #3305 by xxxliqu1dxxx (resubmission of #2914), with its tests.
2026-10-26 18:23:22 +01:00
cuijinkuo 7c9b6f3a28 MCP chat: opt-in mcp.image_results caches PNG/JPEG/WebP tool results beside the config, shows them in the web chat with an authenticated download and passes them to the vision reader (PR #2473 by cuijinkuo) 2026-10-26 18:22:31 +01:00
Demetree Kallergis cc2c628cbe setup (Mac): say when an update was asked for a model that is not installed (#3848)
metal/setup_mac.py returned right after the engine step on --update while the model questions had been asked, so choosing a model the Mac did
not have ended with no download and no word. The update now names the missing model (or image encoder) and points at ./setup.sh --setup; an
update for a model that is installed prints what it printed before. Ported from PR #3848 by demetree, with its two tests (mocked; not run on a Mac).
2026-10-26 18:22:30 +01:00
MegaRushX 3e81173d63 chat.py: fenced code blocks in streamed answers are coloured on a terminal; prose is untouched (PR #1895 by MegaRushX) 2026-10-26 18:21:36 +01:00
medking82 df6928400f Agent: the staged Harness keeps output room in small contexts (8K): a guarded, hash-checked correction of pi-ai 1.1.0's safety margin below 64K (PR #3684 by medking82) 2026-10-26 18:21:20 +01:00
medking82 1e4ae3ddbf Models page: a failed preparation is shown only on the model it belongs to, not on the next model you select (PR #3361 by medking82) 2026-10-26 18:21:08 +01:00
kossumandMIDHAT NASHAR e2e99ae54a Windows: the token default DACL grants generic rights, so a child process can still be opened after adopt_user_defaults (#3428, #3399)
adopt_user_defaults() set the token's default DACL with FA (FILE_ALL_ACCESS). That default also applies to new process objects, and the file
mask has no process query or set-information right, so a child could not be reopened by PID: the HighQoS helper timed out and
procs.start_token could not record the child. GA lets Windows map the mask to the object being created; the principals are unchanged.

Ported from PR #3428 by kossum (QoS reopen test). Test from PR #3399 by midhatn (a live child is identified and recorded) added; both fail
without the change and pass with it.

Co-authored-by: MIDHAT NASHAR <[email protected]>
2026-10-26 18:21:07 +01:00
medking82 f499922379 App: the composer's effort (Off, Low, Medium, High, Max) reaches saved API models that support reasoning, with the OpenRouter mapping; unknown support keeps the provider default (PR #3363 by medking82) 2026-10-26 18:20:52 +01:00
jubeater 14bef6bb46 setup: hand component engines to the server for validation (#4027, #4026)
_start() checked cfg['exe'] with Path.exists(), so a run config whose engine is component:strata-engine was refused as a missing file
before the server could resolve and verify it. Ported from PR #4027 by jubeater. A component reference is no longer tested as a file; model
files and literal engine paths are checked as before. ensure_engine_for, the Windows HIP runtime repair and the HIP architecture lookup
skip the source-build engine folder for a component. Eight tests (tools/test_setup_component_start.py).
2026-10-26 18:20:42 +01:00
zapabob c7833932a1 App: a model load validates and parses the same captured text of its settings file, so a file changed between two reads cannot start with unchecked settings (PR #3954 by zapabob) 2026-10-26 18:20:17 +01:00
SuperDistance d2c69e6d4b tools/strata_bench.py: read CMakeLists.txt as UTF-8, not with the locale's code page (#4011, #2792)
Ported from PR #4011 by SuperDistance. project_version() read CMakeLists.txt with Path.read_text() and no encoding, which raised
UnicodeDecodeError on Windows code pages such as cp936. The test makes read_text default to each of 14 Windows code pages. The same crash was
reported in #3235 and #3414.
2026-10-26 18:20:17 +01:00
BColsey 19bfcb33f0 Tests: the Terminal fixtures write settings where the platform keeps them and no longer swap os.name or assume a Windows ACL on Linux (PR #3782 by BColsey) 2026-10-26 18:19:55 +01:00
Niko1221 9e2c0b6eb0 docker: test that both Dockerfiles take the source commit and the source-build commands pass it (#3644) 2026-10-26 18:19:55 +01:00
jackbapa 24c3f24ae1 Tests: the ACL checks compare trustees through the SDDL alias, so an account that Windows writes as LA passes (PR #3406 by jackbapa) 2026-10-26 18:19:43 +01:00
Niko1221 7c4c20799c Residency-biased routing default is CUDA-discrete-only: read from the device properties once in Verifier::init (never first inside a graph capture), off in HIP builds and on integrated GPUs, a line in the log when it is on; the wide HC up read never defaults on in a HIP build (gfx12 reports compute capability 12.0 too) 2026-10-26 18:19:39 +01:00
Niko1221 68ba321acc launch: a model folder is found whatever its letter case, so a downloaded model is not offered again on a case-sensitive disk (#3996); a log path the person set in the config is kept (#3656); --dspark takes on|off|auto|none and the word reaches the engine (#3959) 2026-10-26 18:19:38 +01:00
Niko1221 a8755042bd Delete a model: every part of a multi-part GGUF goes with it (the run config names only the first), and a part another model names stays (#3716) 2026-10-26 18:19:38 +01:00
Roberto Jose Etcheverry Romero 255861c442 docker: the root Dockerfile passes the source commit to the Terminal stage (#3644)
.git is not in the build context, so package_terminal.py had no commit to put in the archive and the documented source build stopped.
Ported from PR #3644 by robertoetcheverryr: ARG STRATA_COMMIT / ENV STRATA_BUILD_COMMIT as in docker/Dockerfile, and the two 'From source'
commands in docs/DOCKER.md pass --build-arg STRATA_COMMIT=$(git rev-parse HEAD).

Added: .dockerignore leaves out the folders docker/entrypoint.sh creates in a mounted data/ (install, state, home, Strata-data). A dangling
data/install/third_party/llama.cpp link in a clone that had been mounted made the Terminal stage fail with a copytree error (seen in the
review thread of #3644). Not built with docker here (no docker on this PC).
2026-10-26 18:19:35 +01:00
i-Vladimir-i e0469a585f MCP chat: at the tool-round limit the model gets one final reply with no tools from the results already found, instead of the turn ending without an answer (PR #3741 by i-Vladimir-i) 2026-10-26 18:19:27 +01:00
天空Blond d398f35325 docker: the build context carries the nested tools/ scripts the Terminal bundle reads (#3974)
docker/build.sh staged only top-level tools/*.py, so the Terminal stage stopped at 'installer tool missing: tools/dsv41/tune.py'.
Ported from PR #3974 by hurui200320: the pathspec is tools/**/*.py, the no-.git fallback finds nested files, and the context test reads
INSTALLER_TOOLS and INSTALLER_SCRIPTS from packaging/package_terminal.py. Fixes the failure that images built from v0.2.4.1 and v0.2.4.2 had.
2026-10-26 18:19:09 +01:00
toel1234 dc1fd367d3 procs: an engine adopted by systemd --user is found as an orphan, and ending it waits on the start token until the process is really gone (PR #3254 by toel1234) 2026-10-26 18:19:05 +01:00
MaKla89 eb1c40038c App sharing: the shared and tunnel address answers a browser's CORS preflight and sends Access-Control headers when an API key is set; without a key nothing changes (PR #3853 by MaKla89, #3208) 2026-10-26 18:18:15 +01:00
thisisspartan cd72accac3 Web page: a request that timed out in the queue shows a Timeout badge in Recent requests (PR #3991 by thisisspartan) 2026-10-26 18:17:49 +01:00
Niko1221 4abccbb360 Merge f-027-cuda (CUDA speed work: wide HC up read on sm_120, residency-biased routing on small cards, STRATA_TRACE, release-notes draft) 2026-10-26 18:16:40 +01:00
Niko1221 f92cfa4374 Merge f-027-win (main 2d07d1c0b with the DeepSeek slab scoping 32 MB unified / 8 MB discrete plus automatic growth, the dq kernel arch choice, and the HIP-only guard for the gfx11 WMMA gemm call) 2026-10-26 18:16:34 +01:00
justindressler b3db5a1c83 serve: the lifeline identity is the child's start time, the sampler survives a bad source (#3590, #3407)
The portable lifeline watcher built a child's identity from `ps -o lstart=,stat=` and kept the whole line, so the token
changed with the process state (R while it starts, S asleep). The watcher compared its own reading with the token taken
at start, decided it was not the child it was told about and exited without ending it (test_lifeline "the child outlived
a killed server (watcher)", #3407). The identity is now the start time alone: field 22 of /proc/<pid>/stat on Linux
(a zombie has none), `ps -o lstart=` elsewhere; strata_control/procs.start_token drops the state column the same way.
A launch that is not a real Popen is not watched.

Telemetry._loop no longer ends on the first failing sample (a machine whose GPU or psutil source is unreadable): it
prints one line and keeps sampling, and NoLeakedSampler checks the thread itself instead of counting threads.

New test: the token is the same while the child runs (R) and sleeps (S); on the old code it fails on Linux
('...R' != '...S').

Ported from PR #3590.
2026-10-26 18:15:01 +01:00
Niko1221 9a1e768388 App: the source-checkout server is started without an empty STRATA_API_KEY (#3851); a card taken out of the PC no longer decides the engine make, and a PC with two makes accepts a build for either (#3892) 2026-10-26 18:14:26 +01:00
Niko1221 085dcded73 Monitor: the host reads the graphics card every 3 s only while a model runs or the Monitor is open, every 30 s otherwise, and the PCIe sample only while a model runs (#4066) 2026-10-26 18:12:22 +01:00
Righteous-AI fd8d37e93d serve: accept Codex spellings of a namespace tool (#3972)
A declared namespace.member tool (Codex MCP: mcp__filesystem.write_file) was accepted only as namespace__member (#3556).
Qwen3.8-Flash-Next also writes namespace=member (the <function= tag's equals sign ends up in the name), namespace_member
(one underscore dropped) and the bare member. Each is now an alias of the declared tool when exactly one declared tool
produces it and no declared name is the same; a name that matches nothing stays text (#3075), two namespaces that share
a member are not guessed, and a declared flat tool always wins. Requests without namespaced tools are unchanged.

serve/test_tool_name_alias.py (10 tests, 5 of the first 9 fail without the change; the tenth pins that a flat tool named
like the underscore alias wins), docs/DETAILS.md names the spellings.

Ported from PR #3972.
2026-10-26 18:12:20 +01:00
Niko1221 89a68d8829 Chat: the transcript follows the answer only while the reader has not left; a wheel notch, finger or key moving up lets go at once, in the app chat and the served web page (#4014) 2026-10-26 18:11:11 +01:00
Stefanos Anastasiou f8ff3420d9 conversation cache: never park one conversation with another's retained K/V (#1503)
With --batch and conversation parking, a restore keeps the restored K/V (ConversationCache::retain) so the next park of
the main session copies only what changed. The reuse tracked a token count (unchanged_tokens), not whose tokens, and
copy_from_slot (a batch slot giving its conversation back to the main session) replaced the main session's K/V without
dropping the retained image. The next park reused the earlier conversation's K/V for the new one's prefix: a restore of
the new conversation then answered from the other one's history.

retain() now records the ids of the conversation the K/V belongs to; a park calls take_reuse_for(live), which reuses only
the prefix live shares with those ids (nothing when they share none or the ids are unknown), and one stderr line is
printed whenever a reuse is cut. copy_from_slot drops the retained image before it replaces the main session's K/V.
Same edits in sycl/src/program/generate.cpp. The guard only cuts reuse: at most one full copy of a parked conversation's
K/V, the saved bytes are otherwise identical; no flag, no default change.

Ported from PR #1503 onto the current tree (the first-stage ids sit after the stages field so the existing aggregate
initialisers keep their meaning). conversation_cache_test gains the cross-conversation cases of the PR and the logged
sequence (restore P, a slot hands Q to the main session, park Q): another conversation never gets the K/V, a diverging
one only the shared prefix on every stage, a slot hand-back drops it, unknown identity is never reused. Removing the
ids limit makes the first case fail.
2026-10-26 18:09:44 +01:00
Niko1221 f681b9ad62 Manual mode: the look-only shell allowlist takes only options known to look (#3975)
A command on the allowlist used to run without a card with any option except -delete/-exec/-execdir/-fprint/-fls/-ok. Write modes (sort -o,
tree -o, uniq IN OUT), program-execution modes (sort --compress-program, rg --pre, git grep -O), date -s and tail -f got through, and so did
git config -e, git branch --set-upstream-to and git remote ... forms.

Each command now has a spec of the options and operands that only look (short and long, spelled in full); every other option, abbreviation
or attached form asks. git subcommands have the same specs, a branch/tag name is taken only with a list option, git config only reads with
--get. date takes only +FORMAT operands, uniq at most one operand, hostname none. less and more leave the list. Quoted texts are checked as
operands too. `du -sh` no longer trips the 'sh' word check.

Tests: apps/host/tests/gate-readonly.test.mjs (deny and allow cases for options, operands, spellings, quoting; every listed command asks for
an unknown option).
2026-10-26 18:08:17 +01:00
Niko1221andyehya 6c01f85add qwen35: a penalty for a history token outside the logit row no longer indexes past the row (#3911)
sample() applied the repeat / frequency / presence penalties to lg[id] for every token id of the last penalty_last_n
tokens of the history. A row shorter than the vocabulary (the MTP draft head picks among the first N token ids,
--draft-vocab N) made a history token with id >= N an out-of-bounds read and write on the host. A token that is not in
the row takes no penalty; the loop skips ids < 0 or >= lg.size(). A full-vocabulary row is bit-identical.

The loop and Sampling move to include/strata/qwen35/sampling.hpp (header-only) so a CPU unit test can run them:
src/qwen35/sampling_test.cpp (qwen35_sampling_test) checks nothing is written past a short row, that in-row tokens
take the penalty, and that a full row matches the unguarded loop bit for bit over 108 penalty/window mixes. Without the
guard the test dies with an access violation.

Co-authored-by: yehya <[email protected]>
2026-10-26 18:07:29 +01:00
Niko1221 3a604e0855 Merge f-027-kva (docs/KVA.md: the shipped opt-in's numbers on the R9700 and the P100, and the decode trade on fast GPUs) 2026-10-26 17:52:48 +01:00
Niko1221 6f45b24864 docs/KVA.md: the R9700 and P100 numbers for the shipped opt-in (split 36), and the trade stated plainly 2026-10-26 17:52:39 +01:00
Niko1221 063b3c5dc8 DeepSeek engine: the RDNA3 WMMA dense GEMM call in gemm() is HIP-only (wmma_gemm.cu is part of the AMD builds; main's 0eea4a3d5 called strata_pf_gemm_f16_ld unconditionally, so a CUDA build failed to link: unresolved external) 2026-10-26 17:48:36 +01:00
Niko1221 20c2f42bf1 ds4_iq.cu: the NUL character literal in dq_use_old_kernel had been written as a raw NUL byte by the merge script (nvcc: quoted string should contain at least one character); it is a backslash-zero escape again 2026-10-26 17:36:16 +01:00
Niko1221 2e1fad783d host test offline-start: point STRATA_COMPONENTS_DIR at the test's own folder so the host cannot find an installed Strata Control. On a PC with the app installed the update check went through the installed Control (no host fetch, 45 s), so the test saw 0 outside calls and failed; it failed the same on v0.2.6. With the fix both tests pass in 2-12 s. 2026-10-26 17:35:48 +01:00
Niko1221 afc45665fc Merge main 2d07d1c0b (Strix Halo DeepSeek work: fd33cb6ee, 0eea4a3d5, 3133160ed, 2d07d1c0b) into f-027-win. The attention slab: the base is STRATA_DS4_A_MB megabytes of q, 32 MB on a unified-memory GPU (where main measured it) and 8 MB on a discrete card (the 0.2.6 slab); the automatic growth (attn_slab_rows) stays on top for discrete cards with spare VRAM, DS4_ATTN_SLAB still overrides, both env names work. The dense dequantizer: dq_rows_kernel2 stays the default; a gfx11 device (gcnArchName) defaults to main's improved row-per-lane kernel, DS4_DQ_OLD=1/0 forces either. 2026-10-26 17:29:43 +01:00
Niko1221 63b6e0e834 Merge branch 'main' into f-027-cuda 2026-10-26 17:19:22 +01:00
Niko1221 259532bf2e prompt: the RAM store's post-prompt preload is off by default - it costs a 4K prompt 4.7 s and the decode that follows gains nothing
cache_settle()'s store preload (DS4_SETTLE_STORE) read 2,243 hot experts into free store slots after every chunked prompt: 4,713 ms of a 4K prompt.  Measured on the box (code fixture, prompt + 256 tokens, interleaved, 3 runs a side): with it on the decode is 13.9/14.4 s, with it off 14.2 s - no benefit - while the 4K prompt goes 31.3 -> 26.6 s (130.89 -> 153.78 tok/s).  The swap loop above it is what the decode uses (it holds the CPU lane at ~5-7 k experts a run against ~14 k without), so that keeps its budget; DS4_SETTLE_STORE=1 restores the preload.
2026-10-26 17:07:53 +01:00
Niko1221 9c059bade1 Merge f-027-kva (KVA ships off in 0.2.7: STRATA_KVA unset is the exact 0.2.6 engine; STRATA_KVA=1 approximates layers 36-47 with a 2048-token exact tail, the configuration that kept agent loops and long extraction at the exact level; geometry gate; draft fixes; opt-in STRATA_KVA_SKIP=moe) 2026-10-26 17:02:35 +01:00
Niko1221 9c8c90391d Merge f-027-hip (gfx12: decode miss split starts at pcie_frac 0.15, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55; DeepSeek matrix-core prompt attention on by default on gfx1200/gfx1201, DS4_FAST_ATTN=0 off; huge-page copy arena and its pinned DMA on only when a reserved pool holds the copy; KVA draft fixes) 2026-10-26 17:02:13 +01:00
Niko1221 bda3005082 KVA, when switched on, approximates layers 36-47 with a 2048-token exact tail (the measured configuration); STRATA_KVA_SPLIT / STRATA_KVA_SUFFIX override
On the RTX 5070 (--agent-long, 22 tasks) split 36 kept agents at 6/10 (exact 4 and 7) and extraction at 585/606 facts (exact
585) for +11% prompt speed at 14K and +18% at 30K; the split-24 default passed 0/10 agent runs and found 556 facts, a tail of
8192 with split 24 found 574. KVA stays off unless STRATA_KVA=1. docs/KVA.md gives the numbers and the reasons; kva_test
checks the opt-in's split 36 / tail 2048 and the overrides. The geometry gate is unchanged.
2026-10-26 17:02:12 +01:00
Niko1221 aeb70de349 Merge f-027-kvaoff (KVA is off by default: the app's and the Terminal's 'Faster long prompts' switch turns it on, shown for Qwen3.8-Flash-Next models only) 2026-10-26 17:01:54 +01:00
Niko1221 76cd7d7970 KVA: the one-time note says how many positions were approximated, not where the tail starts (chunks round the frontier down) 2026-10-26 16:55:44 +01:00
Niko1221 25a62e0fb3 KVA: keep the geometry gate - a model whose shape KVA was not measured on reads exactly; STRATA_KVA_ANY_MODEL=1 is the research override
The app toggle will write STRATA_KVA=1 for Qwen models in general; the split layer, the tail and the measured quality apply to
Qwen3.8-Flash-Next's shape only (48 layers, 512 experts). Any other shape (Qwen3.6-35B-A3B, ...) now logs "KVA configured but
NOT applied: this model's shape (L layers, E experts) is not one KVA was measured on" and reads exactly; the routine reasons
(short prompt, continuation) still come first. STRATA_KVA_ANY_MODEL=1 allows it, with one warning. kva_test and docs/KVA.md
cover both.
2026-10-26 16:40:59 +01:00
Niko1221 72b7696810 Faster long prompts is shown for Qwen3.8-Flash-Next models only (app and Terminal)
KVA was measured on the Flash-Next shape (48 layers, 512 experts) and the engine declines any other shape with a log reason, so the row would do nothing
for Qwen3.6-35B-A3B. Setup writes "arch" in a run config only for the families with an engine path of their own (Qwen3.6-35B-A3B: qwen35moe; DeepSeek:
deepseek41); every Qwen3.8-Flash-Next file (the original, Swift, Coder, Unsloth, Gyro, NVFP4) has none. The row is therefore shown for a model without an
"arch" on CUDA or HIP, in the app's Native engine settings and in the Terminal's Advanced table. The Radeon RX 9000 and DeepSeek rows are unchanged.
Tests: a Qwen3.6 config gets no KVA row (CUDA and HIP), the six Flash-Next family names do, and through the host fixture a model with an arch has no switch row.
2026-10-26 16:38:15 +01:00
Niko1221 e3450e4552 KVA ships OFF in 0.2.7: STRATA_KVA unset is the exact 0.2.6 engine, STRATA_KVA=1 turns it on with an 8192-token exact tail
The quality runs found lost records and invented rows in long extraction tasks and defects in agent loops with a ~10K
system prompt when most of the prompt was approximated. Opt-in now: unset (or 0) nothing runs, the draft layer included.
Switched on, the tail is 8192 tokens (STRATA_KVA_SUFFIX overrides), the safest configuration measured. A model other than
Qwen3.8-Flash-Next's geometry gets a one-line warning instead of a refusal. Kept: the untouched draft layer (cell floor
behind STRATA_KVA_DRAFT_FLOOR=1), the decline reasons, STRATA_KVA_SKIP=moe, the tests. docs/KVA.md says why it is off and
how to turn it on; kva_test checks unset = off and STRATA_KVA=1 = on with the 8192 tail.
2026-10-26 16:36:31 +01:00
Niko1221 9760cbca4b App: "Faster long prompts" (KVA) is Off by default in the model's Expert settings; On writes STRATA_KVA=1, Off removes it
The Speed row now follows the shared table's default: Faster long prompts reads "Off (recommended)" / "On" and its new sentence is "Reads very long prompts
faster by approximating part of the work. Off by default: answers about long documents can miss or mix up details." (replaces fx73; "Off (recommended)"
is fx79; translated in zh, ja, de, es, ru, pt, ko). The other two rows are unchanged ("On (recommended)" / "Off"). A STRATA_KVA=0 stored by an earlier
0.2.7 test build shows as Off. Still shown only where KVA applies (Qwen on CUDA or HIP). tests/speed-switches.test.mjs updated: the default launch has
no KVA entry, On writes STRATA_KVA=1, Off removes it, the old 0 shows as Off, and the other two switches are tested as before.
2026-10-26 16:34:01 +01:00
Niko1221 ae03ef6323 Terminal and the shared speed-switch table: Faster long prompts (Qwen KVA) is OFF by default, STRATA_KVA=1 turns it on
Owner decision (26 October, conditional on the last KVA variant results): KVA ships off. strata_control/speed_switches.py now gives every switch a default
and the environment it writes outside that default: KVA is default off and writes STRATA_KVA=1; the Radeon RX 9000 and DeepSeek switches stay on by
default and write their off entries as before. A launch in the default carries no entry. A file that holds STRATA_KVA=0 (the off switch of an earlier
0.2.7 test build) reads as off, which is what it is now; choosing Off cleans it up.
The Advanced table row says "off by default; set 1 to turn on" with the new sentence (`--set STRATA_KVA=1`; `off` or Del puts the default back).
tests/terminal/test_speed_switches.py covers the defaults, KVA on/off/old-zero, the rows and the comparison with the app's table.
2026-10-26 16:34:01 +01:00
Niko1221 3171291420 KVA: STRATA_KVA_SKIP=moe (opt-in) - the late layers of an approximated chunk run their whole attention half and skip only the MoE half
The default stays the state-only form (byte-identical). With STRATA_KVA_SKIP=moe the GDN recurrence (gate projection, output
normalization, output projection) and the QSA attention (query side, selection, attention, output projection) run in full
for layers >= split, their outputs go into the residual, and the loop leaves before the MoE half; the expert transfers of
those layers are still skipped. The KVA line says "mode moe-skip". Motivation: the extraction losses of the state-only form
come from late-layer K/V and state built from a stale residual; this keeps the attention halves' contribution in it.
kva_test covers the switch.
2026-10-26 16:18:49 +01:00
Niko1221 fd81df6972 DeepSeek engine (AMD): the matrix-core attention on by default for the chunked prompt
fast_attn_enabled() was opt-in (DS4_FAST_ATTN=1); with the prompt chunk at 1024 it is worth far more than when
it was gated: measured on 4x R9700, 2.25 bpw, 8K code prompts, three warm prompts a load, the same ids
(5705e1164a) --

  without it  1159 / 1180 / 1195 ppts   (median 1180)
  with it     1327 / 1354 / 1360 ppts   (median 1354, +14.7%)

and after this commit the un-enved default reads 1314 / 1357 / 1369 (median 1357), i.e. identical to setting the
flag, so a deployment no longer needs any environment at all.  The QK/PV sums are the matrix cores' order, which
the owner already approved for the prefill; the decode path keeps the scalar kernels.  DS4_FAST_ATTN=0 turns it
off (it is a no-op on CUDA, whose dispatch is not compiled).
2026-10-26 16:16:28 +01:00
Niko1221 879b958a3e docs: setup is expected to choose --ple-io ram on a RAM-rich Linux PC - say to check setup --plan rather than asserting it 2026-10-26 16:08:11 +01:00
Niko1221 be4450fdc3 Merge branch 'main' into f-027-cuda 2026-10-26 15:51:50 +01:00
Niko1221 67cfaf2ac6 prompt streaming: the staging ring sizes from the VRAM room (400 -> 1200 MB on Strix Halo) and the attention slab defaults to 32 MB - 4K prompt 27.3 -> 21.0 s
ring_mb_auto() returned a flat 400 MB on any box whose experts do not fit in VRAM, but on such a box the ring is exactly what lets a prompt queue its likely experts early.  400 -> 1200 MB measured on the box (settle off): the queue 3,200 -> 8,087 experts, the disk's effective rate 1.49 -> 2.44 GB/s, the compute's exposed read wait 11.4 -> 3.5 s, the 4K prompt 148.98 -> 190.98 tok/s; 16K 290.97 -> 315.01, 32K 316.92 -> 325.89.  Decode unmoved in an interleaved A/B (17.54/16.39 against 17.61 tok/s).  It is now a sixteenth of the VRAM room clamped to 400-1200 MB, so a smaller pool still loads; DS4_RING_MB still overrides.

c.A: the attention slab decides how many times each dense matrix is re-read a prompt.  8 -> 32 MB (STRATA_DS4_A_MB): 32K 325.89 -> 357.78 tok/s, 16K 315.01 -> 328.04, 8K +1.6%, 4K flat, decode unmoved (16.81 against 17.60 tok/s interleaved, noise-limited) for ~0.4 GB of VRAM.

Best measured prompt set with the two new defaults (DS4_SETTLE_MS=0;DS4_SETTLE_STORE=0, which is a prompt-only trade): 4K 195.18, 8K 258.47, 16K 328.04, 32K 357.78 tok/s.  With the settle work at its default a 4K prompt is 31.3 s = 130.89 tok/s (was 88.30).
2026-10-26 15:43:31 +01:00
Niko1221 bffb41ef47 Merge f-027-win (ctest registers ple_residency_test with the GPU-build tests) 2026-10-26 15:13:40 +01:00
Niko1221 3e2f45e2e1 ple_residency_test is registered with the GPU-build tests (it sat in the conversation-tests block, which is off by default, so ctest never ran it) 2026-10-26 15:12:23 +01:00
Niko1221 de6229fb17 Merge branch 'main' into f-027-cuda 2026-10-26 15:08:46 +01:00
Niko1221 1ee6060afb docs: the 12 GB card's prompt figures are withdrawn - a five-pair repeat read +254% and -20.5% where the first read +35% and +17%, and that box's prompt swings 218-466 tok/s between arms 2026-10-26 15:08:43 +01:00
Niko1221 cca3dd2e33 dense GEMM: the chunked prompt's gemm() prefers the engine's own RDNA3 WMMA kernel (src/prefill/wmma_gemm.cu, now linked) over hipBLAS - 4.97 vs 20-40 TFLOPS at our shapes
The dense GEMM was 11.1 s of a 4K prompt at hipBLAS's measured 4.97 TFLOPS (M 4096 N 2304 K 5120, the engine's call convention). A blocked WMMA kernel reaches 23.96 TFLOPS at that shape (128x128 tile, BK 32, 2-stage LDS, rocwmma; scripts/wmma2.cu) and the engine's own tuned kernel (BM 128 x BN 256, grouped tile order, raw wmma intrinsics) 20.8-20.9 at the wq_b shape and 38-41 at M 16384. gemm() now calls strata_pf_gemm_f16_ld with hipBLAS as the fallback (rows < 64, N < 512, K % 32, unaligned X: unchanged); STRATA_DENSE_PFGEMM=0 restores hipBLAS for an A/B. Same operands, the kernel's own k order (rounding-level).

Measured on the box (eng-028, 4E224E7F -> 467D7320, 4K prompt fixture p4k, chunk 16384): 46.4 -> 38.1 s, 88.30 -> 107.63 tok/s; with DS4_SETTLE_MS=0;DS4_SETTLE_STORE=0 27.5 s = 148.98 tok/s. 8K 134.66 -> 218.31, 16K 158.6 -> 290.97, 32K 178.9 -> 316.92 (settle off). Decode unchanged in an interleaved A/B (old 15.17/16.73, new 17.38/18.48 tok/s) with acceptance 151/164 = 92% (93/98/100/100/90 per position).

c.A (the attention slab, the reason each dense matrix is re-read 10-16x a prompt) takes STRATA_DS4_A_MB: 32 gives 107.63 -> 110.67, small because the expert reads now dominate.
2026-10-26 15:07:37 +01:00
Niko1221 4329c0d380 Merge f-027-ctx (DeepSeek-V4.1-Flash: 512K and 1M contexts in setup, the Terminal and the app up to the pack's own limit, with a 'not tested yet' note past 256K; the app now honours a chosen DeepSeek context; idea from Project Maya v1.1.28 #442 by @needmorevram) 2026-10-26 14:47:11 +01:00
Niko1221 f9fa567e1d Merge branch 'main' into f-027-cuda 2026-10-26 13:58:25 +01:00
Niko1221 6bc27ebef7 docs: the agent scenario across all three boxes 2026-10-26 13:34:58 +01:00
Niko1221 b8946f4c7b Tests: nobody gets a bigger DeepSeek context than before unless they choose it (8K stays the default in setup, the Terminal and the app; an installed model shows and loads the context it was prepared with) 2026-10-26 13:28:18 +01:00
Niko1221 c4c320a755 App: DeepSeek's context goes up to the pack's limit (512K, 1M), with a plain "not tested yet" note past 256K, in eight languages (Project Maya #442, @needmorevram)
The Models screen listed the shared sizes for DeepSeek but installed and loaded it at 8K whatever was chosen. Now the choice counts (8K stays the
default for a version that is not installed) and the list is DeepSeek's own: 8K to the limit the installed pack states in ds4.txt (max_position_embeddings,
read by the host as `contextMax`); a pack without it behaves as before (up to 256K). Past 256K the existing note slot under the model shows: "Not tested yet.
A context this long takes graphics memory from the model and reads long prompts slowly." (fx78 in the eight fix catalogs). Qwen's choices are unchanged.
src/lib/deepseek-context.ts is the app's copy of strata_control/dsv41_context.py; tests/terminal/test_dsv41_context.py compares the two.
2026-10-26 13:25:32 +01:00
Niko1221 33780a4955 DeepSeek-V4.1-Flash: 512K and 1M contexts in setup and the Terminal, as Project Maya v1.1.28 did (#442 by @needmorevram, MIT)
The model reads 1,048,576 positions natively. Setup and `strata launch` offered DeepSeek the shared list that stops at 512K and dressed anything past
256K in Qwen's "rope scaling" words. Now:
- strata_control/dsv41_context.py: the choices are 8K, 32K, 64K, 128K, 256K, 512K and 1M (labelled "512K" and "1M"), limited to the pack's own limit
  (max_position_embeddings in its ds4.txt) when it has one; a size past it (2M, or 1M on a pack that says 256K) is the one refusal; past 256K a plain
  note says it is not tested yet and what it costs.
- setup.py: --context is checked before any download and again against the pack once it is built; the install tool refuses the same sizes.
- Terminal: the context step lists DeepSeek's own sizes (and the note beside them), a typed size is checked, `--context 512k` / `1m` and 524288 / 1048576
  are accepted, plan and launch carry the note as a warning; Qwen's list and wording are unchanged.
- Engine: read, no change needed. strata-dsv41 takes --max-context as given (ds4_main.cpp) and sizes its buffers from it (ds4_model.cu); nothing clamps it
  below the pack's limit, and the rope settings come from the pack. A test keeps it that way.
- docs/DEEPSEEK41.md: the cost (graphics memory per position taken from the expert cache, slow reading of very long prompts) and that 512K / 1M are untested.
2026-10-26 13:25:32 +01:00
Niko1221 11b5a77fe9 docs: the agent scenario - two conversations nearly double throughput 2026-10-26 13:23:56 +01:00
Niko1221 f9907b6d9c docs: the routing default is +67% with two conversations in flight 2026-10-26 13:14:27 +01:00
Niko1221 d56dec89b5 Merge commit '2e6f88b7b' into f-027-hip 2026-10-26 13:04:40 +01:00
Niko1221 6c00851ac1 KVA: the draft layer is untouched by default (window 32768, cells built for approximated chunks, no floor); the cell floor is STRATA_KVA_DRAFT_FLOOR=1
Measured on a 5070 (IQ3_S) the cell floor cut decode after a 14K / 30K approximated prompt by 22 / 34 percent (drafts
accepted 100 -> 83 / 73 percent): a draft layer that reads only the exact tail loses its context. The approximated
chunks' draft cells are usable, so by default nothing about the draft layer changes: not the window, not the cells
built, not the attention. The floor machinery (MtpDrafter::set_cell_floor, window_ids_floor, KvaFloorRegistry) stays
behind STRATA_KVA_DRAFT_FLOOR=1 for research. kva_test now really covers window_start with floor 0, the registry and
wants_cell_floor.
2026-10-26 13:03:41 +01:00
Niko1221 2043b682af Merge f-027-win (DeepSeek: the chunked prompt's attention slab is automatic, up to 2048 rows only where the VRAM would not hold an expert; DS4_ATTN_SLAB=0 = the 0.2.6 rule) 2026-10-26 13:00:52 +01:00
Niko1221 e66d2758ed docs: the release notes carry the two-conversation batching gain 2026-10-26 12:46:27 +01:00
Niko1221 03056168d9 dense dequant: the chunked prompt's dq_rows_kernel packs each sub-block's 32 halves into four 16-byte stores and stages the warp's input through contiguous uint4 loads decoded at a relative index - the same values (bit-identical), 4K prefill 79.5 -> 88.52 tok/s, acceptance 97.4% 2026-10-26 12:21:18 +01:00
Niko1221 2d1db1ec6f Merge commit '9c282fbc5' into f-027-hip 2026-10-26 12:15:19 +01:00
Niko1221 7a34c60d61 KVA: leave the draft window alone; a per-session cell floor keeps the draft layer off approximated cells
The first default-on build set --mtp-window 2048 once per process whenever KVA was on, so short prompts (where KVA never
applies) lost draft context and decode speed. Now the draft layer keeps its window (32768 by default) and
MtpDrafter::set_cell_floor() names the first draft cell built from a real row: a KVA read raises it to the end of its
approximated chunks (they are neither built nor read), the draft attention starts at max(floor, pos - window)
(window_ids_floor, window_start.hpp), and KvaFloorRegistry gives every state a request continues from (live, checkpoint,
parked) the floor of its own tokens. A session KVA did not approximate has floor 0 and attends exactly as with
STRATA_KVA=0; with STRATA_KVA=0 or a process KVA is ruled out for the draft layer's code is 0.2.6's. The tail is 2048
whatever the window. Unit tests: window_start with floor 0 equals the plain window, the registry, wants_cell_floor.
2026-10-26 12:14:30 +01:00
Niko1221 50df0b95a0 Merge f-027-app (off switches for the three 0.2.7 speed features: app Native engine settings > Speed in 8 languages, Terminal Advanced rows, shared table strata_control/speed_switches.py; host tests isolated from the real components folder) 2026-10-26 12:08:29 +01:00
Niko1221 af7f7371c3 DeepSeek attention slab: the two new log lines lost their newline escape in the previous commit (a literal line break inside the string literal); no other change 2026-10-26 11:56:33 +01:00
Niko1221 bdde697225 DeepSeek engine: the chunked prompt's attention slab is automatic (DS4_ATTN_SLAB unset): up to 2048 rows where the bigger buffers take VRAM no expert would have used (the expert tier off, or every expert fits with room to spare), else the 0.2.6 rule; DS4_ATTN_SLAB=0 restores the 0.2.6 rule exactly, N rows overrides. If the automatic slab does not fit the load retries with the 256-row slab before it halves the chunk. Measured on the RTX 5070 (12 GB, 2.25 bpw): 1024 / 2048 rows gave +2% prompt but cost 29 / 83 VRAM experts, -3% decode and different greedy ids (placement), so there the rule keeps 256. attn_slab_rows() in ds4_experts_rule.hpp, 14 new checks in experts_rule_test. 2026-10-26 11:34:15 +01:00
Niko1221 eb448299c0 Merge branch 'f-027-kva' into f-027-hip 2026-10-26 11:04:09 +01:00
Niko1221 621e660604 Host tests green on this PC: the GuidedSetup very-large-model sentence gets its catalog key in all 8 languages; the Control lookup test no longer reads the real components folder
- locale-gaps: the sentence added in 72c37ea2d (RTX 30/40/50, R9700, Strix Halo; "the line below says what to expect") had no key. fx77 in the eight fix catalogs.
- packaged-control: findControl was given an env without STRATA_COMPONENTS_DIR, so it fell back to the real per-user components folder and found the
  updated Control installed on the PC (read only). The test now passes a scratch components folder.
2026-10-26 11:03:37 +01:00
Niko1221 cb2e1dbb4e Merge branch 'main' into f-027-cuda 2026-10-26 11:03:10 +01:00
Niko1221 bc5f84fd62 PLE residency policy: the load log also says the expert arena size the rule counted (the proof that the layout is initialised and the sum is not 0) 2026-10-26 10:55:20 +01:00
Niko1221 8a46ca9061 Merge f-027-int (the PLE residency fix) into f-027-kva 2026-10-26 10:55:03 +01:00
Niko1221 0ee72a5041 Merge branch 'f-027-int' into f-027-hip 2026-10-26 10:54:58 +01:00
Niko1221 ad1660aadf Merge f-027-win (PLE residency: admit the table only when RAM holds table + expert arena + 8 GiB; fixes the IQ3_S load failure, error 1453, on a 64 GB Windows PC) 2026-10-26 10:54:45 +01:00
Niko1221 472de9cdac PLE residency policy: admit the table only when RAM holds table + the expert arena + 8 GiB. The 0.2.7 rule (table + 8 GiB) admitted the 26.8 GiB table on a 64 GB Windows PC (RTX 5070, IQ3_S), where the engine then fills a 51 GiB pinned expert arena: the locked table left too little lockable memory and the expert load died with ERROR_WORKING_SET_QUOTA (1453). ple_resident_admit() (include/strata/core/ple_residency.hpp) sizes the arena from the native expert layout; unknown sizes or telemetry admit nothing. Unit test ple_residency_test. 2026-10-26 10:53:11 +01:00
Niko1221 ddd4d5d42b App: off switches for the three 0.2.7 speed features in the model's Expert settings (Native engine settings > Speed), on by default, in eight languages
Three rows under "Speed" in the native settings, in the existing row markup with the existing Select (On (recommended) / Off): Faster long prompts
(Qwen on CUDA or HIP; off writes STRATA_KVA=0), Faster answers on Radeon RX 9000 / AI PRO R9700 (Qwen on a gfx1200/gfx1201 card; off writes
STRATA_PCIE_FRAC_DEFAULT=old) and DeepSeek speed-ups (DeepSeek; off writes the six DS4_* = 0 of the contract). Each has a plain sentence saying what
turning it off does. A row is shown only where it applies, like the DeepSeek Speed row; the host decides from the model's file and the hardware report.

The switch is the model's file "env": off adds exactly its entries, on removes them, so a default launch is byte for byte what 0.2.6 started and the
Terminal, Control and the app reach the engine the same way (serve/server.py child_env). The host now shows and edits STRATA_* and DS4_* names (the
Strix Halo async entry is visible and kept); other names stay untouched. New strings are fx70..fx76 in the fix catalogs, translated in zh, ja, de, es,
ru, pt, ko. tests/speed-switches.test.mjs covers the default, each switch off and on again (same file), where each applies, and the catalogs.
2026-10-26 10:37:53 +01:00
Niko1221 3b3ea883c3 Terminal: the three 0.2.7 speed features as rows of the Advanced table, with their off values and plain words
strata_control/speed_switches.py holds the engine switch contract (Faster long prompts: STRATA_KVA=0; Radeon RX 9000 / R9700: STRATA_PCIE_FRAC_DEFAULT=old;
DeepSeek speed-ups: six DS4_* = 0), where each applies (Qwen on CUDA/HIP; Qwen on a gfx1200/gfx1201 card; DeepSeek) and the one-sentence effect text.
`strata launch` shows the environment names of the switches that apply as rows of the Advanced table (default "on", the sentence beside them); typing the
off value (`--set STRATA_KVA=0`) writes it into the run config's env, `off` takes it out again. A default launch carries none of them.
tests/terminal/test_speed_switches.py checks the names, the applicability, the config a launch writes, the rows, and that the app's table
(apps/host/server/native-settings.mjs) says the same.
2026-10-26 10:37:47 +01:00
Niko1221 750ca8c3b1 KVA on by default for a cold prompt of 4096+ tokens (Qwen3.8 on one CUDA/HIP device); STRATA_KVA=0 gives the exact 0.2.6 prompt read
The tail follows the draft layer: with KVA on and no --mtp-window given, the draft window becomes 2048 cells and the
fully computed tail 2112 (a given --mtp-window N sets the tail N + 64; a window of every cell, or STRATA_KVA_SUFFIX, is
handled and named). Layer splits, batch slots, --peer-device, pictures, the DFlash2 drafter, other model geometries and
the Metal build read exactly, with the existing "NOT applied: reason" line (quiet for continuation and short reads unless
STRATA_KVA=1). The reasons live in KvaConfig::decline and are unit tested (kva_test). Later chat turns continue from the
live session or a checkpoint and read their new tokens exactly over the cells the first read left; docs/KVA.md says so.
2026-10-26 10:14:06 +01:00
Niko1221 3c8a452900 DeepSeek copy-all: the huge-page arena (DS4_COPY_HUGETLB) is on when the reserved pool already holds the whole copy, and the pin of the copies (DS4_COPY_PIN) is on only when the whole arena is on explicit huge pages; =0 turns either off, an explicit =1 keeps the old opt-in behaviour, no pool means the ordinary arena exactly as before 2026-10-26 10:00:12 +01:00
Niko1221 1f079f9d13 DeepSeek HIP: the matrix-core prompt attention (DS4_FAST_ATTN) is on by default on gfx1200/gfx1201 only; DS4_FAST_ATTN=0 is the off switch, gfx11 and gfx1151 stay scalar unless =1 2026-10-26 10:00:11 +01:00
Niko1221 e5cfa4bbc8 docs: the miss split is the gfx12 start value again (the owner's decision 26 Oct; revert of the opt-in wording) 2026-10-26 09:58:12 +01:00
Niko1221 8d56d41768 gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55 2026-10-26 09:58:11 +01:00
Niko1221 45c9e623e2 STRATA_LOGITS_DUMP: a post-window logits tap for KL quality measurement (off by default); the sampler itself runs inside a CUDA graph capture, so the copy lives in the verify window's host-side epilogue where STRATA_DBG_NAN already copies from 2026-10-26 09:44:39 +01:00
Niko1221 053a41570c KVA: say why a configured run fell back to the full path (the observability criterion: an operator must be able to tell a configured-and-declined run from one that never asked) 2026-10-26 09:44:39 +01:00
Niko1221 5b469ee9a3 KVA: keep gdn_recurrence_variant's public signature - the SYCL tree defines that symbol, so a new parameter would break its link; the state-only form goes through a private impl 2026-10-26 09:44:39 +01:00
Niko1221 4abef6a74e KVA: what the opt-in approximate late-layer prefill does, how to enable and capture, and the measured frontier 2026-10-26 09:44:39 +01:00
Niko1221 c5cf218e54 KVA: state-only GDN recurrence (no gate projection, no output norm) behind STRATA_KVA_GDN_NORM for the A/B 2026-10-26 09:44:39 +01:00
Niko1221 011e5beced KVA: layer-type filter, per-layer transfer skip, stream-entry release on a skipped MoE half 2026-10-26 09:44:38 +01:00
Niko1221 f4012d6369 KVA: optional approximate late-layer prefill (split 24, chunk-aligned suffix, capture/oracle modes) 2026-10-26 09:44:38 +01:00
Niko1221 7dacaf8ea1 Merge f-027-r9700 (Qwen on one R9700: the PLE table kept in RAM when the PC can afford it, four bit-identical gfx12 decode switches back on) into f-027-int 2026-10-26 09:44:24 +01:00
Niko1221 ca7b3d2ca5 Merge f-026-miss (DeepSeek 0.2.7: the RAM store's spare + put, the 12 GB-cell prompt fixes, opt-in matrix-core attention on HIP) into f-027-int 2026-10-26 09:44:16 +01:00
Niko1221 7e1a7cb93c docs: the release notes carry the 12 GB routing default with its held-out evidence 2026-10-26 09:38:45 +01:00
Niko1221 a266bdfe1d markov bias in Q8: the drafter's head is quantised to Q8_0 blocks as it loads (66 MB -> 33 MB, the pack untouched) and read by a Q8 variant of markov_kernel - the markov segment 1.80 -> 1.34 ms, the drafter's round 13.46 -> 12.95, the drafts' acceptance 90-95% (the same band) 2026-10-26 09:35:56 +01:00
Niko1221 e43284572b head: 8 columns a pass for a GGUF head (every window rows<=kWin walks the 129280x5120 head once instead of twice, the same logits per column) - the drafter's fixed T=5 head 4.55 -> 2.83 ms, its round 14.49 -> 13.46; the verify window's head 3.48 -> 2.68 ms 2026-10-26 09:02:28 +01:00
Niko1221 0f66c316ee DeepSeek engine (AMD): the whole chunk's attention slab where the cards hold most of the experts
The slab rule (8 MB of q/o per device, 256 rows here) was set to keep the dense dequant's VRAM small; with a
1024-token chunk it dequantizes q_b/wo once per 256 rows - four times a chunk, 32 times an 8K prompt.
Measured on 4x R9700, 2.25 bpw, share 0.77, chunk 1024, 8K code prompt, ids 5705e1164a:

  256-row rule     1204 / 1236 / 1250 ppts, decode 77.7 tok/s
  chunk's 1024     1324 / 1353 / 1363        decode 78.1        (+9% prefill, decode unchanged)

The 1024 rows cost 268 MB a device (1% of the card).  So high-share runs take min(chunk, 1024) rows and the
old rule stays elsewhere; DS4_ATTN_SLAB still overrides.  The un-enved default verified at 1335 ppts /
77.8 tok/s with the ring at 253 MB.
2026-10-26 08:52:39 +01:00
Niko1221 4aa645c007 DeepSeek engine (AMD): a small staging ring where the cards hold most of the experts
With the 1024-token prompt chunk (the default where the cards hold most of the experts, abfe7d813) a big
staging ring is no longer the trade it was: every MB it takes is a tenth of an expert the decode keeps.
Measured on the same box, 8K code prompt + six 128-token decodes a load, ids 5705e1164a / 81d895366c:

  2000 MB  1243 ppts / 73.2 tok/s
  1000 MB  1287 / 77.5
   500 MB  1314 / 78.5
   256 MB  1324 / 78.4
   128 MB  1338 / 78.3

So high-share runs take clamp(room/device * 0.008, 128, 512) MB (248 MB here) and the old deeper ring stays
for the cases it was measured for (share < 0.5, where the cards stream most of the experts).  The un-enved
default verified at 1339 ppts / 78.1 tok/s, 42 s of summed GPU time in a 36 s pass.
2026-10-26 08:10:37 +01:00
Niko1221 88c2931963 DeepSeek engine (AMD): the small prompt chunk whenever the cards hold most of the experts, 4 cards included
prefill_chunk_default() returned 1024 only for a single-card run (own_default); a multi-card run whose cards
hold most of the experts fell through to 2048 (HIP skips the two 4096 branches) - and the rule's own datapoint
for 1024 was a 4 x R9700 run ('1024 370 tok/s, 8192 214').  Measured again here, 2.25 bpw, 4 cards, 8K code
prompt, --dspark auto, ids 5705e1164a, 4 prompts a load:

  512   1110 / 1137 / 1142 ppts (median 1123)
  1024  1229 / 1257 / 1270        (median 1243)   <- this commit's default
  2048  689 / 695 / 696           (median 692)    <- what the engine picked before
  8192  690 / 690 / 695           (median 690)    <- what the 0.2.7 configs passed

Decode unchanged (73.2 against 73.2 tok/s medians, --dspark auto).  So the owner's default 8K prompt goes from
692 to 1243 ppts (+80%) with no flag; --prefill-chunk still overrides.
2026-10-26 07:24:22 +01:00
Niko1221 85ddf96ffc Merge branch 'main' into f-027-cuda 2026-10-26 06:48:53 +01:00
Niko1221 e83b82afe5 DeepSeek engine (AMD): the drafter's cumulative confidence cut on by default (0.5) on HIP
DeepInfer's rule: verify a draft while the product of the confidences so far stays above the cut, instead of
only each draft's own floor.  The 0.2.7 notes left it off ('not measured faster on code: H100 / owner PC'),
Project Maya runs 0.5 on a Radeon 8065S (+6.5% decode, the same tokens).

Measured on 4x R9700, DeepSeek 2.25 bpw, six 128-token decodes an arm in one load, same ids
(81d895366c): 71.7 -> 76.9 tok/s (medians) with STRATA_DS4_DSPARK_CUM=0.5, and a second pair 74.1/73.6;
the default with the rule on (no env) verified at 74.5 tok/s (6 decodes).  CUDA keeps 0 (the rule was flat
there).  STRATA_DS4_DSPARK_CUM still overrides either way.
2026-10-26 06:09:02 +01:00
Niko1221 2247e5a98d DeepSeek engine (AMD): pin the CPU lane's RAM copy when the cards hold >= half the experts (DS4_COPY_PIN auto)
The prompt path copies every streamed expert from the RAM copy into the pinned staging before its DMA (the
file's own note: 6.8 GB/s through the staging against 24-28 GB/s probes).  Registering the copy lets it DMA
in place.  Three interleaved pairs on 4x R9700, 2.25 bpw (share 0.72): the second 8K prompt of a process
12.0 / 11.9 / 11.9 s against 28.8 / 13.8 / 12.3 s unpinned; decode 35.42 / 35.50 / 34.9-36.0 against
35.43 / 35.45 / 34.9-36.0 tok/s, i.e. no cost; ids unchanged.  Costs 23 s of load (registering a 50 GB
copy) - once a process, which for a server is the right trade.  Auto only when the expert share is >= 0.5
(the CPU lane is then a minority of the decode, so the pinned pages' TLB cost cannot bite); DS4_COPY_PIN=0/1
forces it, and it stays off where the 12 GB cards' decode is CPU-lane-bound (the -15% measured there).
2026-10-26 04:46:46 +01:00
Niko1221 5b969c1f7a DeepSeek engine (AMD, diagnostic): the io threads' busy time in the prompt stream line
The RAM->staging memcpy runs on the IoQueue's threads; on a warm 8K prompt (stream stage 14.3 s, 12.8 GB a
device) they are busy 4.1 s, i.e. ~3 GB/s a device - a real share, and the thing to check in the 50-95 s
first-prompt runs (where the same counters are what to read first).  DS4_PROF only.
2026-10-26 04:30:12 +01:00
Niko1221 3cc132ec51 DeepSeek engine (AMD, diagnostic): where copy_ahead's time goes in the prompt stream line (DS4_PROF only)
The first prompt after a load is 2-9x slower than the second on the 4x R9700 2.25 bpw cell and it is not
the disk (0 MB read during it), the RAM store (removing it is worse), the dense GEMM shapes (q_b+rope
1.46 s) or the compute waiting (0 ms).  The lead is the RAM->pinned-staging->DMA path (the file's own note:
6.8 GB/s through the staging against 24-28 GB/s probes); DS4_COPY_PIN=1 measured 47.9 -> 31.4 s on the
first prompt and 28.8 -> 12.0 s on the second, decode unchanged.  This commit only adds the counters
(staging / DMA issue / slot wait) to the existing prompt stream line under DS4_PROF; no behaviour change.
2026-10-26 04:07:19 +01:00
Niko1221 3b23994568 DeepSeek engine (AMD): the staging ring from each device's room (not the summed room)
No behaviour change on the 4x R9700 cell (126.8 GB of room / 4 = 31.7 GB a device, 8% -> 2000 MB, the value
measured in 76177ec34); on other card counts it is the per-device share that was meant.
2026-10-26 03:36:03 +01:00
Niko1221 1e72cdc4b9 DeepSeek engine (AMD): a staging ring sized from the VRAM room, and expert-tile knobs for the MarLin work
The prompt path streams the experts the cards cannot hold.  ring_mb_auto() gave 16 MB whenever the cards
held >= 1/1.3 of the experts, which is exactly this 2.25 bpw cell (126.8 GB of room, 156.3 GB of experts,
share 0.769): the stream then had no depth and the expert tiles waited 1.7 s inside a 5.2 s stage.  A share
of the VRAM room (5%, 400-2000 MB) when any expert spills, 16 MB when they all fit (the 0.2.6 rule, kept
for the decode it was chosen for).

Measured, 4x R9700, 2.25 plain pack, 6 experts, --dspark off, 8192-token code prompt, ids 5705e1164a:
8K warm 632 -> 675-678 ppts (+6.8%, 5 prompts a run, 3 runs), agent decode 35.9 -> 35.3 tok/s (-1.7%,
4 runs).  The profile of the warm pass: expert stage 5185 -> 4034 ms (kernels 3509 -> 2645 ms, the same
kernels now that the weights arrive in VRAM ahead of them, waits 1676 -> 1389 ms), everything else equal.

Also in this commit (opt-in, defaults byte-identical except the F16 activation path which is bit-identical):
- SetArgs::x16: gate/up take the layer's F16 copy of the activations (rows_f16 rounds exactly as the tile's
  own smem store did) - half the activation bytes, 26% faster gate/up in the standalone bench
  (box/expert_bench.cpp), +0.3% at the pass level (the stage is stream-bound).
- HIPK_CFG 6..11: tile variants for the expert GEMMs (64x128, 32x128, 128x256, 64x128/4 waves,
  128/128+down 256, 64x256).  64x128 (cfg 6) is 21% faster than 128x128 in the bench and +0.9% in the pass.
- HIPK_DBG 1/2/3: switch the MMA / the decode off in the tiles (the breakdown in 09-marlin-kernel-diary.md).
- ds4_iq_hip.inc: a UTF-8 BOM on a source file removed.
2026-10-26 03:01:27 +01:00
Niko1221 81ac4dd83b setup: a 16,384-token prompt chunk on a unified-memory PC whose pool holds ~128 GB (16K prompt 152.2 -> 158.6 tok/s, 32K 174.0 -> 179.7, 8K flat, decode unchanged); smaller unified PCs keep 8,192 2026-10-26 01:52:05 +01:00
Niko1221 44b88bc91e docs: the release notes carry the residency-biased routing default
It now ships on at 2.0 for a card of 13 GiB or less: +37.6% story / +35.2% code
decode over five interleaved pairs on an RTX 3060 12 GB, and it stays off on
bigger cards where it measured neutral to negative (l620 -1.9%/-0.5%, z440
-4.2%/+11.0%).  It changes which experts run - the owner approved that trade
for 12 GB cards - and STRATA_ROUTE_PRIOR=0 turns it off.
2026-10-26 01:01:09 +01:00
Niko1221 2395c8a53e hc: the one-kernel persistent hyper-connection read builds and runs on CUDA
The design was HIP-only: 80 resident blocks, a column slice of D each, two
grid barriers on an atomic counter, a barrier timeout so a card that cannot
hold all 80 blocks resident turns a hang into a refused start-up check.  Ported
by giving grid_sync a CUDA branch (volatile generation counter + clock64() for
the timeout, scaled to SM clocks), guarding the gfx-arch refusal to HIP and
adding a masked-shuffle helper.

On the RTX PRO 5000 Blackwell 72 GB it passes its own check - '80 blocks on 110
multiprocessors, 0 token counts timed out, max relative difference 1.8e-07
against the default read' - and measurably does NOT win: story 201.4 -> 190.6
tok/s (-5.4%), code 245.6 -> 249.6 (+1.6%) over 3 interleaved pairs.  That
matches the AMD note in the same file (an R9700 measured it neutral to 1%
slower), so it stays opt-in (STRATA_HC_PERSIST=1) and is not a default.  The
port is kept as the starting point for a CUDA-tuned block mapping.
2026-10-26 00:47:21 +01:00
Niko1221 c30656aaf9 route: residency-biased routing is on by default at 2.0 on a card of 13 GiB or less
The router's top-10 is picked again with a logit bonus for the experts already
in VRAM (route_prior.cu); the weights stay the router's.  Measured on an RTX
3060 12 GB, IQ3_XXS, 5 interleaved pairs of fresh engines: story 44.2 -> 60.9
tok/s (+37.6%), code 47.3 -> 64.0 (+35.2%); =1.0 gives +22.6% / +24.6%.  Three
more pairs against the switch itself read +23.6% / +36.9%.

It changes which experts run, so it is a default only where the expert cache
cannot hold the model, and the owner approved that trade for 12 GB cards.  It
is NOT a default on a bigger card: on an RTX PRO 5000 72 GB it measures
-1.9% / -0.5% and on 2x RTX A4000 -4.2% / +11.0% (3 pairs each); both were
re-run after this change and are unchanged (201.7 / 246.9 and 87.1 / 107.1
tok/s).  STRATA_ROUTE_PRIOR=0 turns it off and any explicit value wins.
2026-10-26 00:27:14 +01:00
Niko1221 219beb5736 DeepSeek engine: porting guards for the 12 GB work. The matrix-core attention and its dispatch are now inside #if defined(STRATA_USE_HIP) (a CUDA/sm120 build keeps the scalar path and compiles), and a failed DS4_COPY_HUGETLB allocation falls back to the ordinary arena with a warning instead of leaving that layer's experts in place (the flag is safe on a machine with no huge-page pool). HIP build verified: 8K warm 768.7 / 767.4 / 765.6 ppts with DS4_FAST_ATTN=1, ids 5705e1164a, the agent decode's 81d895366c. 2026-10-26 00:21:01 +01:00
Niko1221 32c686fa67 DeepSeek engine: the matrix-core attention's fragments load two halves a word (the scores' Q/K tiles and the values' P tile), and the online softmax runs a warp per two heads with a lane a key (16-lane shuffle max/sum) instead of a thread a head walking 16 keys. R9700 12 GB cell, 2.25 pack, 8K warm with DS4_FAST_ATTN=1: 767.7 / 765.8 ppts (10.67/10.70 s), the sparse attention stage 2516 -> 2311 ms, first prompt 465.6, with the same ids (5705e1164a, the agent decode's 81d895366c) and the same decode speed (16.49 tok/s). Quality gate on this binary (4995-position score, 2.25 plain, ref_base1): KL 0.10776 / ppl 3.389 / top1 87.89% against the scalar kernel's 0.10746 / 3.395 / 87.81% (delta 0.0003; the gate is 0.006). The softmax's sum order differs from the previous matrix-core build (a shuffle tree, not a key walk). 2026-10-25 23:43:12 +01:00
Niko1221 f05ccd3b54 DeepSeek engine: DS4_FAST_ATTN=1 runs the chunked prompt's packed-KV attention on the gfx11/gfx12 matrix cores (new sparse_attn_wmma_kernel, opt-in; the decode and verify paths stay on the scalar kernels). The matrix's rows are 16 heads of ONE query, so a block stages a 16-key tile once for all of them: Q (16 heads x 512) is the A operand of the scores, the key tile is B, and the same tile is B of the values' product; the scores' 64-dim slices are computed by the 8 warps and summed in shared, the values by a warp a 64-dim slice, the online softmax a head a thread. R9700 12 GB cell, 2.25 pack, 8K warm: 752.2 / 748.3 ppts (10.89/10.95 s) against 694.2 / 693.0 / 691.2 for the scalar kernel; the sparse attention stage 3750 -> 2516 ms; first prompt 484.8. Quality gate (4995-position score, 2.25 plain, ref_base1): KL 0.10750 / ppl 3.394 / top1 87.83% against the scalar kernel's 0.10746 / 3.395 / 87.81% (delta 0.00004; the gate is 0.006). The ids are the same (5705e1164a) and the 128-token agent decode's ids too (81d895366c); its decode speed unchanged (16.49 vs 16.87 tok/s). The QK/PV sums are the matrix cores' order, hence opt-in (the owner approved the trade for the prefill). 2026-10-25 23:29:29 +01:00
Niko1221 2dde837d9f DeepSeek engine: DS4_COPY_HUGETLB=1 puts the CPU lane's RAM copy on explicit 2 MB pages (MAP_HUGETLB from the kernel pool; the copy's RAM budget adds the pool, which MemAvailable does not count). With DS4_COPY_PIN=1 the pin then keeps 2 MB pages: on a transparent-huge-page arena the registration pins 4 KB pages and the prompt's DMA from it swings between 11.9 and 30.4 s for the same 140.2 GB stream. R9700 12 GB cell, 2.25 pack, 8K, 3 warm prompts: huge-page arena + pin + DS4_RING_MB=3500 + DS4_ATTN_SLAB=2048 -> 694.2 / 693.0 / 691.2 ppts (11.80-11.82 s), first prompt 435.4 (18.8 s), ids identical (5705e1164a); the THP arena with the pin ran 691/573/270 in one load and 665-691 in another. Copy at load 33.3 s for 156.7 GB; the pool is reserved once per boot (sysctl vm.nr_hugepages=78000). Opt-in: default path unchanged; no-pin staged stays 651. 2026-10-25 22:27:30 +01:00
Niko1221 bc6a39217a docs: 0.2.7 release notes draft from the CUDA speed work
A draft for the owner: the one engine default that changed (the wide
hyper-connection up read on a 12.0 card, +1.9% story / +0.7% code decode on
the RTX PRO 5000, 5 pairs, bitwise), the four settings that are faster where
the PC supports them (PLE in RAM, the English draft vocabulary, --prefill-pipe
on a two-card split, q4_0 KV on 12 GB with its precision cost), and the list
of what was measured and rejected so it is not retried.  No version number is
claimed: the owner decides what else 0.2.7 carries.
2026-10-25 21:26:06 +01:00
Niko1221 d39e2a7539 sm_120: the wide hyper-connection up read is on by default
STRATA_GR_UP_WIDE (one warp a row, N/2 = 1,280 blocks instead of 160)
measured on the RTX PRO 5000 Blackwell 72 GB, IQ3_S, every expert in VRAM,
5 interleaved pairs of fresh engines: story 197.6 -> 201.3 tok/s (+1.9%),
code 243.5 -> 245.1 (+0.7%).  A third run against the switch (default on vs
STRATA_GR_UP_WIDE=0) reads +1.7% / +0.8%.  The produced token ids were
identical in all 6 pairs, and fused_gr_bench's per-T hashes are the plain
read's.  The switch stays: =0 off, =1 on any card, STRATA_SM120_PERF=0 the
whole 12.0 set.  Other cards are untouched.
2026-10-25 21:21:28 +01:00
Niko1221 2465b93708 docs: the miss split is opt-in, not a default: the text can differ (4 of 6 fixed prompts within 64 greedy tokens) even though the first-token KL is 0.0013 2026-10-25 21:03:52 +01:00
Niko1221 1d00c018e1 Revert "gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55"
This reverts commit 550c6ddbf90d57a0d018878115f6d0f9fa86cb74.
2026-10-25 21:03:43 +01:00
Niko1221 1f00c1825e Merge f-027-async: synchronous verify window on Strix Halo (+31% code decode)
setup writes DS4_ASYNC_T=4 into the config env for an integrated Radeon: the engine's mailbox path is sized for cards that hold
most of the experts and costs this box 31% on code (15.9-16.3 -> 20.1-21.3 tok/s), 16% on agent with identical accepted tokens,
9% on chat; the 2.25 set (no drafter) is unchanged.  The same tokens come out.  Discrete cards keep the engine default.
2026-10-25 20:53:56 +01:00
Niko1221 63b8a8d3cc Strix Halo: run the verify window synchronously (DS4_ASYNC_T=4 in the config env), +31% code decode
The engine's windowed decode uses the mailbox ("async") path for windows up to DS4_ASYNC_T (8 by default). That default was sized
for cards that hold most of the experts; on the 128 GB gfx1151 box the 6-row verify window is faster on the synchronous path.
Measured 2026-10-25 with the 2.00 bpw set and the shipped 0.1.51 engine (greedy, 256-token sets, three interleaved reps):

- code 15.87-16.33 -> 20.14-21.28 tok/s (+31%);
- agent, with identical accepted tokens a round (5.07): 15.93 -> 18.48 (+16%);
- chat 10.33 -> 11.30 (+9%);
- 2.25 code unchanged (11.2 -> 10.9): the drafter is unloaded there, so the extra syncs have nothing to hide.

The same tokens come out; only the scheduling changes, and the drafter's own single-token windows (T=1) stay on the async path.
dsv41_config_env(cards) writes DS4_ASYNC_T=4 into the generated config for an integrated Radeon only (a discrete card keeps the
engine default); STRATA_DSV41_ASYNC_T=n overrides the value and =8 restores the default. Tests:
tests/terminal/test_dsv41_async_t.py (5 cases). No engine rebuild; the signed dsv41 package is unchanged.

Docs: the STRIX_HALO and DEEPSEEK41 measured tables and the README row now read 19-21 tok/s code, 18 agent, 11 chat.
2026-10-25 20:53:52 +01:00
Niko1221 3a26753027 STRATA_TRACE: one clock for the whole process, so a layer split's stages compare
The trace origin was thread-local, and the prompt's later stages run on their
own thread, so their timestamps could not be compared with the first stage's.
One process-wide origin makes the hand-off visible.

Found with it on z440 (Q2_0, split 24, 16,383-token prompt, 8K chunks): stage
0 starts at 0 ms and 3,267 ms, stage 1 at 3,268 ms and 5,996 ms, read ends at
8,775 ms - the two stages do overlap chunk-for-chunk, and each takes ~2.7 s
per 8K chunk for its 24 layers.  A split 30 run leaves stage 1 at 3.09 s a
chunk with 18 layers, so the per-chunk cost is not its layer count.
2026-10-25 20:52:26 +01:00
Niko1221 457d8cc791 DeepSeek engine: DS4_ATTN_SLAB=N sets the chunked prompt's attention slab rows (default unchanged: 8 MB / HQ rounded to 64). The slab loop runs the dense projections (q_b, wo_a, wo_b) per slab, and each call dequantizes the whole dense matrix: a 256-row slab dequantized the same weights 32 times for an 8K prompt (the dense dequant 916 ms and rocBLAS 867 ms of that prompt). R9700 12 GB cell, 2.25 pack, 8K warm: slab 256 -> 591.9 ppts (with the earlier fixes), slab 2048 + DS4_RING_MB=1600 -> 668.0/666.2 ppts (12.26/12.30 s), the first prompt 429.9 (from 143-402), ids identical (5705e1164a); q_b+rope 3.97 -> 1.15 s. Opt-in: the default slab is unchanged. 2026-10-25 20:45:17 +01:00
Niko1221 14d076ee2e STRATA_TRACE: the prompt chunk line carries its stage range and elapsed ms
Same diagnostic as the request-step timestamps: with a layer split the chunk
line now says which stage is starting a chunk and when, so a prompt's stage
overlap can be read off the log.  STRATA_TRACE only; no behaviour change.

Measured with it on z440 (Q2_0, split 24, 16,383-token prompt): serial reads
run stage 0's two 8,192-token chunks 3,332 ms apart and the whole read takes
8,788 ms; --prefill-pipe 1500 reads four 4,096-token chunks 1,582 ms apart and
the read takes 8,375 ms, so the stages overlap only partially at best.
2026-10-25 20:15:58 +01:00
Niko1221 13c4aebeca docs: the miss split is the gfx12 start value now (measured A/B on the shipped path: 8K +9.0%, 32K +23.6% decode) 2026-10-25 20:08:41 +01:00
Niko1221 d7ad4a2762 gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55 2026-10-25 19:57:35 +01:00
Niko1221 8abe826f70 DeepSeek engine: the chunked prompt's dense dequantizer writes its 32 values a thread through shared memory, coalesced (a warp covers 1024 contiguous values), instead of each thread writing 2 bytes every 64 - a warp's store touched 32 cache lines. Same arithmetic and the same F16/BF16 values (only the store order changes). R9700 12 GB cell, 2.25 pack, 8K warm: prompt 14474 -> 13840 ms (566 -> 591.9 ppts), the q_b+rope stage 4025 -> 3581 ms, ids identical (5705e1164a); the cold first prompt also uses it. DS4_DQ_OLD=1 keeps the old kernel for A/B. 2026-10-25 19:43:51 +01:00
Niko1221 7901bc5137 docs: RDNA4 decode miss split measured on the R9700 (--pcie-frac 0.15: +9.9% at 8K, +25.6% at 32K, first-token KL 0.0013, top-1 12/12) 2026-10-25 19:15:53 +01:00
Niko1221 91d4d7ff86 STRATA_TRACE: print elapsed milliseconds on each request step
The trace lines named the step a request reached but not how long it took to
get there, so a stall between two marks was invisible.  Each line now carries
the time since the first traced step (thread-local).  A diagnostic only: no
step, no order and no output changes without STRATA_TRACE set.

Found with it on z440 (Q2_0, layer split 24, 16,383-token prompt): the batched
read takes 8,758 ms while stage 0's own GPU timeline is 5,374 ms - stage 1 is
the slow one - and the expert-cache refill after the prompt costs a further
592 ms of the request.
2026-10-25 18:56:26 +01:00
Niko1221 2b5cadd984 Merge f-027-perf: 8,192-token prompt chunk on Strix Halo (11-15% faster prompts)
Measured on the 128 GB box with the 2.00 bpw set: 4K prompt 78.7 -> 87.2 tok/s, 8K 108.9 -> 125.1, 32K 155.2 -> 176.9;
decode unchanged (16.53 -> 16.59).  setup writes --prefill-chunk 8192 only for an integrated Radeon whose whole pool is at
least 64 GB; the escape hatch is STRATA_DSV41_NO_BIG_CHUNK=1.  Also repairs UTF-8 damage in three docs from an earlier
PowerShell edit.
2026-10-25 18:54:28 +01:00
Niko1221 e7e48be2c9 Strix Halo (gfx1151): an 8,192-token prompt chunk on the unified-memory APU (11-15% faster prompts)
The engine's prompt-chunk rule returns 1,024 when a single card holds half or more of the experts; it was calibrated on
cards that hold most of them (4 x R9700).  On the 128 GB Strix Halo (share 0.62, one shared pool) the bigger chunk wins:
measured with the 2.00 bpw set, engine 0.1.51, 2026-10-25: 4K prompt 78.7 -> 87.2 tok/s, 8K 108.9 -> 125.1, 32K 155.2 ->
176.9.  Decode is unchanged (16.53 -> 16.59): the chunk buffers take ~5 GB from the GPU expert tier (81.9 -> 76.6 GB) and the
RAM store's hit rate rises (68.6% -> 76.8%) to compensate.

dsv41_prefill_chunk_args(cards, ram) writes --prefill-chunk 8192 for an integrated Radeon only, and only when the whole pool
(the RAM Windows sees plus the BIOS carve-out) is at least 64 GB; a discrete card or a smaller unified PC keeps the engine's
choice, and STRATA_DSV41_NO_BIG_CHUNK=1 turns it off.  The chunk changes the prompt GEMMs' summation order, not the model: the
repo's chunk-size KL check reads 1024 0.1705 / 2048 0.1692 / 4096 0.1688 / 8192 0.1695.  Test:
tests/terminal/test_dsv41_prefill_chunk.py (5 cases).

Also repairs UTF-8 damage in README.md, docs/STRIX_HALO.md and docs/MODELS.md from an earlier PowerShell edit (the middots,
em dashes and arrows had been double-encoded; no text changed).  No engine rebuild is needed: the existing signed dsv41 package
accepts the flag.
2026-10-25 18:54:22 +01:00
Niko1221 00a1292531 DeepSeek engine: the post-prompt RAM-store preload skips the experts the CPU lane already holds as a RAM copy (copy_mode). In copy-all mode every routed expert is in that copy, so the preload read a second copy of the same bytes: 4152 experts, 40 GB at 6.4 GB/s, 6.2 s of every 8K prompt, with 0.0 store hits in the decode after it; with --mmap-experts off and the experts not all fitting, the in-place ones are still preloaded. R9700 12 GB cell, 2.25 pack, 8K: prompt 27918 -> 21567 ms (293 -> 380 ppts; a repeated prompt 526.6), decode 16.61 -> 16.82 tok/s, ids identical (5705e1164a). With DS4_COPY_PIN=1 the same run is 21347 ms (383.7 ppts, repeat 535.6). 2026-10-25 18:45:40 +01:00
Niko1221 4be0a7ce4d DeepSeek engine: DS4_COPY_PIN=1 DMAs the prompt's experts from the CPU lane's copies in place (cudaHostRegister, read-only with DS4_COPY_PIN_RO=1), so the flow does not copy each one to the pinned staging first; the pinned chunks are unregistered with the arena. R9700 1 card, 2.25 pack, 8K prompt: the stream's compute wait 5517 -> 0 ms, prompt 28035 -> 27392 ms (292 -> 299 tok/s), ids identical; decode unchanged (16.6 vs 16.8 tok/s). The stream itself stays ~7 GB/s (the DMA issue pattern, not the staging), so the gain is small here; on an SSD-bound box the removed wait is the point. Opt-in: default path unchanged. 2026-10-25 18:05:54 +01:00
Niko1221 1d380fad87 arch_defaults_test: the gfx12 table is nine switches now (five prompt, four decode); check the apply path against a clean environment 2026-10-25 17:47:24 +01:00
Niko1221 d18d8ba045 CUDA HC: the wide up read, one warp per row, opt-in (STRATA_GR_UP_WIDE=1)
The up projection of the hyper-connection read runs 160 blocks of 8 warps
(each warp takes 8 rows) on a 12.0 card, so 160 blocks of 192 SMs and ~6.7
warps an SM: the read is latency-bound at about 0.5 TB/s.  This kernel takes
one row per warp and 2 columns x the 4 streams a block, so the grid is
N/2 = 1280 blocks (53 warps an SM).  Every row's dot, xor tree, epilogue and
the cross-stream sum are gr_up_multi_kernel's, in its order, so the outputs
are bitwise the same; the token streams of both arms were identical in all
6 pairs measured.

Measured on l620 (RTX PRO 5000 72 GB, all experts in VRAM, IQ3_S, 3
interleaved pairs, fresh engine each): story 197.4 -> 201.3 tok/s (+1.9%),
code 243.0 -> 244.7 (+0.7%).  Off by default (STRATA_GR_UP_WIDE unset); the
q8_1 tail of S26 needs the 16-column blocks, so the wide kernel is only used
when no q8_1 image is written.
2026-10-25 17:39:28 +01:00
Niko1221 3fd5a748c7 gfx12: the four bit-identical decode switches back on (ATTN_LANECELL, ATTN_MERGE_V2, TSUM, EXPDEC), measured on p620 R9700 IQ3_S 8K: +3.5% decode, 3/3 pairs, answer md5 identical; docs: the PLE residency policy and its numbers; ple_reader_test: table_bytes 2026-10-25 17:34:42 +01:00
Niko1221 3f26ca5c2f Merge f-027-strix: DeepSeek V4.1-Flash on Ryzen AI Max (Strix Halo, gfx1151) on Windows
The Windows AMD DeepSeek engine (feed member dsv41, windows-x86_64-hip-rdna3, gfx1151) ships with these
changes: the 0.2.6 copy-all rule is off on an integrated GPU (it copied 53 GB into RAM beside the GPU's 82 GB
in one 128 GB pool and an 8K prompt stalled; the in-place path is the 0.2.5 soak path), a Windows HIP package's
rocm/bin is on the engine's library path (setup, run config, app), the unified-memory fit plan counts the BIOS
carve-out, and --feed reaches the DeepSeek engine download.  Docs, app text and tests included.

Measured on the Strix box (2.00 bpw, Faster, DSpark auto): 4K prompt 78.7 tok/s, 8K 108.9, code 17.6 tok/s,
chat 11.0.  Clean-user end-to-end from a signed feed: install, download, pack, smoke, serve, chat, tool call,
agent round trip: 5/5.
2026-10-25 17:20:42 +01:00
Niko1221 a6543d6a7f setup: accept --feed (the DeepSeek engine's package comes from it)
strata launch --feed passed the feed to setup, but setup's parser had no --feed (getattr(a, 'feed', None)
was always None), so the flag was refused with 'Unrecognized arguments' and a local or non-default feed could
never serve the DeepSeek engine (found by the clean-user test on strix, 2026-10-25).  It is now the same
option strata update has: a URL or a folder.
2026-10-25 17:06:43 +01:00
Niko1221 932bda54bd Strix Halo: the DeepSeek engine follows --feed, and the unified plan counts the BIOS carve-out
Found by the clean-user test on strix: 'strata launch --feed DIR' read the local feed for its own engine
check and then setup went to GitHub for the DeepSeek package, so a local (or any non-default) feed never
served the DeepSeek engine.  setup's get_dsv41_engine/upgrade_dsv41_engine take the feed, install_dsv41
passes --feed, and runner.setup_args hands --feed to setup (a test in test_launch_dsv41).

The unified-memory plan counted only the RAM Windows sees (64 GB on the owner's box) and ignored the 64 GB
BIOS carve-out the GPU owns: 128 GB in one pool.  uma_total_gb() adds the carve-out (or the GPU figure less
its shared part), so the plan now says about 109 GB of the 2.00 set's 132 GB stay in memory and 23 GB are read
from the SSD; the docs and the test fixtures use the 128 GB box.  The 'measured' text says 128 GB unified too.
2026-10-25 17:00:58 +01:00
Niko1221 3b34b6c7b2 Engine: keep the PLE table resident in RAM when the PC can afford it (opt-out STRATA_PLE_AUTO_RAM=0); measured on p620 R9700 IQ3_S: 8K prompt PLE 845 -> 132 ms, 2125 -> 2611 tok/s, answer md5 identical 2026-10-25 16:42:05 +01:00
Niko1221 6c70d4bc5d Strix Halo (gfx1151) on Windows: DeepSeek V4.1-Flash support and the 0.2.6 shared-memory fix
The Windows AMD DeepSeek engine package (feed member dsv41, windows-x86_64-hip-rdna3, arch gfx1151) is built from this
branch and installed as its own package; these are the code changes it needs.

- ds4_model.cu: on an integrated GPU the CPU lane's experts are no longer copied into RAM by the auto rule.  The GPU's
  expert tier and that copy are the same shared memory: on the 64 GB Strix Halo the GPU held 82 GB of experts, the rule
  saw 62 GB 'free' and copied 53 GB, and an 8K prompt did not finish in 11 minutes.  Found 2026-10-25 on strix; the
  corrected build does the same prompt at 108.9 tok/s (4K 78.7), code answers 17.6 tok/s, chat 11.0.  DS4_COPY_ALL=1
  still forces the copy for a test.
- dsv41_engine.py: lib_dirs()/start_env() put the package's BUILD.json lib_dirs on PATH/LD_LIBRARY_PATH (a Windows HIP
  package keeps hipBLAS/rocBLAS in rocm/bin, or a clean PC fails with 0xC0000135); the upgrade test-start uses it.
- setup.py: the dsv41 engine's library dirs come from the package (so the run config carries rocm/bin), the unsupported-card
  text names Ryzen AI Max on Windows, and the manual package hint names the .tar.gz the release page actually has.
- runner.py: the same library dirs for the app/Terminal path.
- dsv41_fit.py: a unified-memory plan for the official IST-DAS sets (one pool, what stays in RAM, what is read from the SSD,
  measured 14-16 tok/s on code, 10-11 on chat, 4K 78 / 8K 109), never a refusal; tools/strata_mcp.py uses it for the app
  catalog's on-this-PC line.
- Docs: STRIX_HALO.md (DeepSeek section with speeds, driver 26.10.1, memory behaviour, limits), DEEPSEEK41.md, MODELS.md,
  AI_REFERENCE.md, README.md, packaging/README.md (the Windows AMD package recipe).
- Tests: tests/terminal/test_dsv41_fit.py (7) and 4 new tests in test_dsv41_engine.py (lib dirs, feed selection for gfx1151).
2026-10-25 16:41:49 +01:00
Niko1221 4b9e05a2a8 Windows uninstaller: remove the app's own folders (control, host, node, harness) recursively; files the app wrote there while it ran (Python bytecode caches under control) are not in the installer's file list, so those folders and the program folder stayed behind after use (found on strix with a standard user after the process fix); test 2026-10-25 13:56:36 +01:00
Niko1221 af197f96db Windows uninstaller: find Strata's own running processes with the 64-bit PowerShell (the installer is 32-bit, whose Get-Process cannot read a 64-bit process's path) and with Get-Process first, WMI only as a second source: a standard user's WMI query was denied, so the host kept the folder open and it stayed behind (found on strix with a standard user) 2026-10-25 12:32:41 +01:00
Niko1221 dfaae03b00 DeepSeek engine: the RAM store no longer misses the experts the VRAM cache gives back to the CPU lane: the preload fills the slots the lane leaves over (and the 256 reserved ones, evictable first) with the VRAM experts the profile counts least and the lane experts it never saw (DS4_STORE_SPARE=0 off), and an expert leaving VRAM is copied into a store slot from its VRAM blob (put_begin/put_done, DS4_STORE_PUT=0 off) instead of being read from the SSD while the lane waits. r9700 1 card 2.00, store forced (--ssd-cache 140): decode code 23.7 -> 26.2, agent 20.7 -> 24.6 tok/s, misses 1.5/2.6 -> 0.5/0.6 a token; P2003 RTX 3060: no loss, ids identical 2026-10-25 11:54:42 +01:00
Niko1221 53548c8af4 Merge main (41cf05a06: README model order, docs only) into f-026-rel 2026-10-25 11:37:25 +01:00
Niko1221 41cf05a06a README (all 7 languages): models in the order Qwen3.8-Flash-Next, Qwen3.6-35B-A3B, DeepSeek V4.1-Flash 2026-10-25 11:35:36 +01:00
Niko1221 4f0f933941 Merge main (ed4b4db40: README, docs and icons) into f-026-rel; AGENTS.md keeps main's text with Version 0.2.6, engine 0.1.51; the installer names in the READMEs and docs are the 0.2.6 ones 2026-10-25 11:29:12 +01:00
Niko1221 ed4b4db405 docs and packaging: the owner's 2026-10-23 guide set brought in for 0.2.5 (one installer per system, engine downloaded on first start, DeepSeek in the app and the Terminal with three sets and the Speed choice, install.sh form kept, versions 0.2.5 and 0.1.50, source setup labelled as the developer path) 2026-10-25 11:28:02 +01:00
Niko1221 af8c6f8ec7 README translations (zh-CN, ja, de, fr, es, pt-BR) rebuilt in the structure of the new README; the README links source setup to SOURCE_INSTALL.md and names the three 0.2.5 installers 2026-10-25 11:28:01 +01:00
Niko1221 e99c4d2d37 AGENTS.md and AI_SETUP.md: lead an agent to the installer, install.sh/install.ps1 and strata launch --plan then --yes --no-tui; source setup only as the fallback; provider order ISTA-DASLab, Unsloth, Gyro, others 2026-10-25 11:28:01 +01:00
Niko1221 b8b4ca0e32 docs: add the guide index, the source-install page, the AI reference and the media kit (the owner's pages, updated to 0.2.5) 2026-10-25 11:28:01 +01:00
Niko1221 562a0c2ed7 Version 0.2.6: the Qwen engine packages and Metal stay 0.1.50 (no C++ change since 0.2.5: only the DeepSeek engine and a generate.cpp comment changed); Control and the DeepSeek engine are 0.1.51 as the CMake project version, the shape of 0.2.4; build_hip_linux.sh keeps PKGVER 0.1.50 2026-10-25 11:21:36 +01:00
Niko1221 79ec8e2786 Host test: the source check for 'a version that is not downloaded starts with Read pictures on' accepts the braced else that the DeepSeek 8K context fix introduced (test only) 2026-10-25 11:16:25 +01:00
Niko1221 b5d41babfa README: the owner's layout (model dropdowns with Qwen and DeepSeek marks, provider and set tables, short install, hardware and tools tables), updated for 0.2.5: DeepSeek in the app and the Terminal with three sets and the free disk each needs, 0.2.5 speeds, one installer per system; model marks from LobeHub Icons (MIT) 2026-10-25 11:14:21 +01:00
Niko1221 27a8ed96b6 Version 0.2.6: Terminal and app 0.2.6, engine and Control 0.1.51 (every Qwen and DeepSeek package, Metal included) 2026-10-25 11:14:16 +01:00
Niko1221 8aee4538f2 generate.cpp: the --spec-split comment no longer calls its windows exact (they are not bit-exact against no-split: batch-shape divergence); comment only 2026-10-25 11:14:00 +01:00
Niko1221 e9e22db6a7 Setup: the DeepSeek engine step does not stop on 'can't reach GitHub' when the update feed is a local one (STRATA_UPDATE_FEED or --feed naming a folder, file or loopback address); a transient GitHub timeout no longer fails a verify run; tests 2026-10-25 11:13:51 +01:00
Niko1221 7025a99799 WSL on AMD: setup and launch say plainly that the AMD (HIP) engines do not run under WSL yet (no /dev/kfd, the GPU bridge never finishes a command) and name the Windows app or Terminal as the next step; a warning, nothing is refused; tests 2026-10-25 11:12:59 +01:00
Niko1221 36c6dd9ef5 Windows uninstaller: stop the app, host, node and engine that run from the install folder before files are removed, so the program folder no longer stays behind after use; only processes whose image path is inside the install folder; test with a real process 2026-10-25 11:11:37 +01:00
Niko1221 4a6a5fb699 Merge f-026-copyall (DeepSeek copy-all CPU-lane experts) into f-026-rel 2026-10-25 11:10:16 +01:00
Niko1221 d51d71e7c0 Merge f-025-fix (6 owner-QA fixes) into f-026-rel 2026-10-25 11:10:13 +01:00
Niko1221 fa663fabc5 Merge f-0243-rel (721c6f7ff) into f-025-readme 2026-10-25 10:41:20 +01:00
Niko1221 721c6f7ff5 Mac: the engine bundle inside a signed .app is not removed after the first start (deleting it broke the app's signature); outside an .app and on other systems the cleanup is unchanged; test 2026-10-25 10:37:21 +01:00
Niko1221 025cf7d954 DeepSeek engine: the copy-all fit check is one tested rule (copy_all_fits) and also needs the Windows commit room (RAM + page file beside its margin), not only the free RAM, so a 64 GB PC with a 12 GB card, or a big-RAM PC with a small page file, never copies the lane in by itself; the auto store says (verbose) when it holds less than the CPU lane 2026-10-25 10:18:53 +01:00
Niko1221 5d5c26d5bc Monitor/serve: a one-token reply has no decode span, so no tok/s is shown for it (a 1-token probe read 1,250 tok/s); test 2026-10-25 09:29:40 +01:00
Niko1221 8a3ca07c4e serve: a chat request with no message is a 400 (messages must be a non-empty list), as OpenAI answers it; test 2026-10-25 09:28:47 +01:00
Niko1221 dd9b9f746d About/Updates: never show the host's placeholder engine version (strata-engine 0.1.0) while the real one is still being read; a forced status read that meets one already running waits and reads again, so the version after an engine update is not the old one; test 2026-10-25 09:27:38 +01:00
Niko1221 6b5d593639 DeepSeek engine: --mmap-experts auto copies all the CPU lane's experts into RAM at load when they fit beside 8 GiB, with no demand store (all or none; --mmap-experts on/off, --ssd-cache N and DS4_STORE_GB stand as given); the auto store logs when it holds less than the lane. 1 x R9700 + 251 GB, 2.00 pack, warm: decode code 24.5 -> 27.6, agent 21.9 -> 27.1 tok/s, same ids as --ssd-cache 0 --mmap-experts off, load 34 -> 55 s. Rule ported from Project Maya v1.1.27 (MIT), mmap-experts auto copy-all 2026-10-25 09:27:16 +01:00
Niko1221 4d6721822a Storage and Manage models: a model whose files are all gone (old config, 0.0 GB, not ready) is no longer listed as a download 2026-10-25 09:26:19 +01:00
Niko1221 ad66a758a7 Models screen: DeepSeek is installed and shown at its one 8K context (as the guided setup), not the 32K default 2026-10-25 09:25:53 +01:00
Niko1221 59599f69f8 Monitor: All GPUs leaves out a small integrated Radeon/Intel chip beside a real card (totals and count); the hardware report gives each card a stable id since each make numbers from 0; tests with the RTX 5070 + Radeon iGPU pair 2026-10-25 09:25:17 +01:00
Niko1221 a04f5505aa Merge f-0243-rel (ad638b93e) into f-025-readme 2026-10-25 02:12:48 +02:00
Niko1221 36f580a5c3 UPDATES.md: the Mac dmg keeps its Metal engine; only the Windows and Linux installers are engine-less 2026-10-25 02:12:42 +02:00
Niko1221 ad638b93e5 Merge f-0243-dsimport (the app's model-folder preview recognises official DeepSeek sets) into f-0243-rel 2026-10-25 01:07:30 +02:00
Niko1221 32ad109816 Model preview: recognise official DeepSeek-V4.1-Flash IST-DAS sets (1.75/2.00/2.25 bpw, official and first-release names, with or without the Engram) as family deepseek41 2026-10-25 01:07:05 +02:00
Niko1221 6270e2d9d6 collect_libs: a library an earlier binary's pass already put in --out is not copied onto itself (SameFileError stopped the AMD packages' second pass) 2026-10-24 22:51:19 +02:00
Niko1221 ce27d13539 Merge f-0250-tui-uma (Faster is the default Speed on shared-RAM GPUs) into f-0243-rel 2026-10-24 22:31:11 +02:00
Niko1221 2dbdea2a77 Merge f-0243-strixspeed into f-0243-rel 2026-10-24 22:13:52 +02:00
Niko1221 da1eb6bc69 Merge f-0243-copyfix into f-0243-rel 2026-10-24 22:13:51 +02:00
Niko1221 2cea5afbc5 Merge f-0243-locales into f-0243-rel 2026-10-24 22:13:51 +02:00
Niko1221 0b9539877f README and docs for the single installer: three download buttons (Windows, Linux, Mac) with the 0.2.5 file names, one plain line instead of the per-GPU sentence (7 languages); docs/AMD_HIP.md and packaging/UPDATES.md name the single installers 2026-10-24 22:08:00 +02:00
Niko1221 9551b4168d Metal and SYCL builds: the Metal engine gets stubs for the 512-bit i-quant rows native_expert.cpp now references (x86-only file), and SYCL compiles iq_avx512.cpp with -mbmi2 and -mavx512vpopcntdq like the root CMakeLists.txt 2026-10-24 22:01:11 +02:00
Niko1221 1aff6d1411 Version 0.2.5: Terminal and app 0.2.5, engine and Control 0.1.50 (every Qwen and DeepSeek package, Metal included) 2026-10-24 21:52:49 +02:00
Niko1221 3f8f9aeaa6 Translations (de, zh, ja, es, ru, pt, ko) for the 26 new fix-catalog keys: first-run card, DeepSeek note, Speed labels 2026-10-24 21:47:01 +02:00
Niko1221 684881246d DeepSeek Speed label: the default is named by the engine's own rule, integrated / unified-memory GPU (Strix Halo) or cards totalling 16 GiB or less; '(default on this PC)' on the matching choice; test 2026-10-24 21:45:44 +02:00
Niko1221 d44144d7bb English catalog keys for the first-run card, the DeepSeek note and the Speed labels (the locale-gaps test failed on the merged branch); the other seven fix catalogs carry the English text until translated 2026-10-24 21:45:44 +02:00
Niko1221 df7d407742 Merge f-025-fast (Faster is the default on shared-RAM GPUs; ZC warning) into f-0243-rel 2026-10-24 21:40:42 +02:00
Niko1221 0597aaaeb7 launch: an integrated / unified-memory GPU (Strix Halo, gfx1151) is Faster by default, as the engine's --experts auto says: the 'default on this PC' label and the plan mirror 'integrated GPU, or cards totalling 16 GiB or less'; the engine 0.1.50 guard stays; tests 2026-10-24 21:40:29 +02:00
Niko1221 c4ce40bf5e DeepSeek engine: faster mode (5 experts a word) is the default on a GPU that shares the PC's RAM (Strix Halo: decode +11..13%, prompts +5..7% against 6); the 16 GiB rule is unchanged, --experts N wins; rule test rows 2026-10-24 21:40:06 +02:00
Niko1221 add4362139 Merge branch 'f-025-strix' into f-025-fast 2026-10-24 21:39:41 +02:00
Niko1221 ff91bc3729 DeepSeek engine: DS4_STORE_ZC=1 on a GPU that shares the PC's RAM, whose budget the weights already fill, prints one plain line that it will likely fail with a GPU fault and gains nothing (no cap) 2026-10-24 21:36:27 +02:00
Niko1221 67996022cd build_dsv41_windows.bat: configure every time so the build id is the current commit 2026-10-24 21:25:48 +02:00
Niko1221 7988837183 release build scripts: a failed step is a non-zero exit (build_hip_linux.sh), a stale binary or an old configure cannot pass, and strata-dsv41 must carry the release commit (BUILD_ID for a tarball, checked in the binary) 2026-10-24 21:24:53 +02:00
Niko1221 77b1a50541 launch: flow_for reads the settings only when the context has them (a context with only a computer, as test_mixed_vendors passes, raised AttributeError after the Speed step) 2026-10-24 21:18:56 +02:00
Niko1221 902239e7f7 ds4_gg_bench --k512: the two printf strings end in a newline escape (they held literal line breaks) 2026-10-24 20:57:49 +02:00
Niko1221 c1c3eab540 Merge f-026-5070 (in-place chunk 4096 on non-HIP cards under 16 GiB) into f-0243-rel 2026-10-24 20:41:36 +02:00
Niko1221 1fac68d6f2 DeepSeek engine: the experts-read-in-place default prompt chunk is 4096 on non-HIP cards under 16 GiB (was 8192): decode +2% agent / +0.4..9% code (5 pairs on RTX 5070 and RTX 3060, 120 more experts resident), an agent turn of 3K new tokens on 16K takes the same time, a cold 8K prompt 9% slower; DS4_FEEDLOG prints each feed's time; test rows for 12 GB, 15.9 / 16 / 24 GiB, 8 GB, unknown VRAM and HIP 2026-10-24 20:41:08 +02:00
Niko1221 aa4b225237 Merge f-025-strix (gfx11 WMMA expert kernels, iq test) into f-0243-rel 2026-10-24 20:31:03 +02:00
Niko1221 bd71ea81b9 Merge f-0243-pick (prompt-path rule, post-ready warm-up) into f-0243-rel 2026-10-24 20:25:56 +02:00
Niko1221 4d8ff9646f DeepSeek engine tests: the gfx11 / gfx12 matrix-core prompt tiles are forced on and off in ds4_iq_test (launch counted, both against the CPU reference and each other); umaprobe m1/ver probes; test knobs DS4_ZC_COARSE and DS4_STORE_CHUNK_SLOTS for the zero-copy registration 2026-10-24 20:11:27 +02:00
Niko1221 536460c3d8 Copy feedback: one helper (lib/clipboard.ts copyText with a toast); Markdown and Serving use it, lib/copy-text.ts is gone (the toast store is loaded only when a code block is copied) 2026-10-24 20:05:05 +02:00
Niko1221 061a285832 Merge f-0243-gui into f-0243-rel (Chat.tsx and DropZone.tsx: the guifix clipboard helpers kept at the conflicts, copyAndTell stays for Markdown and Serving) 2026-10-24 20:03:44 +02:00
Niko1221 e752c65455 ds4_gg_bench: --k512 calls the AVX-512 rows (iq512x_rows) directly against ggml-cpu's dot, bit for bit, and says whether the CPU and type allow them 2026-10-24 20:02:50 +02:00
Niko1221 dfd1b0f291 DeepSeek engine: the prompt path is chosen from the prompt's length alone (96 tokens and more through the chunked pass, windows of 16 below), one number for every box - the same prompt always takes the same path and gives the same bits (the old per-request timing prediction flipped the logits between runs and took 7 s on a 28-token prompt); the chunked pass is warmed up after the model is ready, on a thread, so the BLAS library's ~3.6 s start is not paid by the first prompt a person types; startup line; DS4_PICK=adaptive, DS4_PICK_MIN=N, DS4_WARM=0 as switches; rule test 2026-10-24 19:56:11 +02:00
Niko1221 8b8f8a10ee Merge f-026-5070 final chunk rule (7bf6e4d3b) into f-0243-rel 2026-10-24 19:15:07 +02:00
Niko1221 7bf6e4d3b7 DeepSeek engine: one default prompt chunk rule, prefill_chunk_default(): 1024 only for a single-card default run where the GPUs hold at least half of the routed experts (4 x R9700; the R9700 worker's gpu_expert_share() from 4cde62a35), 4096 when a GGUF pack's experts are copied into RAM (--mmap-experts off) on a non-HIP card with 10 GiB or more, 8192 for the experts read in place (5070, 3060, 1 x R9700: what a launch on one card already ran), 2048 for a pack that is not a GGUF; real launch logs on the RTX 5070 and the RTX 3060 show no --tp / --threads and 'prompts in chunks of 8192'; one unit test with the 5070, 3060, 5090-class, 1 x and 4 x R9700 cases 2026-10-24 19:14:17 +02:00
Niko1221 56ac21a217 Merge f-026-5070 (prefill chunk default rule) into f-0243-rel 2026-10-24 19:00:37 +02:00
Niko1221 829913ea59 DeepSeek engine: 4096 chunk rule applies to a GGUF pack only when it is copied into RAM (--mmap-experts off) and to an explicit --tp 1 single card; the experts-read-in-place default stays 8192; other packs keep 2048 2026-10-24 19:00:07 +02:00
Niko1221 dba7d0e0ab DeepSeek engine: the prompt chunk of the experts-copied-into-RAM case (2048) and of an explicit --tp 1 single card (1024) becomes 4096 on cards with 10 GiB or more (HIP builds and smaller cards keep theirs; the in-place 8192 is unchanged); startup line; rule test incl. 12/16/24/32 GB, short-VRAM, HIP and max-context cases 2026-10-24 18:49:20 +02:00
Niko1221 535cacb052 Tests never write the owner's real Strata profile: pytest gets scratch LOCALAPPDATA/XDG state and data folders; the host test runner and the native-control test files get a scratch settings file, components folder and app data 2026-10-24 18:43:52 +02:00
Niko1221 67a6ee76f4 Engine bundle cleanup: one function (cleanupAfterInstall) writes the removed and kept lines to the host output, with a test that both reach it 2026-10-24 18:36:50 +02:00
Niko1221 a264168390 One size figure: engine sizes are decimal MB, rounded half up, in the Terminal (strata update) and the app; both tested on the same numbers 2026-10-24 18:14:36 +02:00
Niko1221 e6c5997091 DeepSeek engine: gfx11 WMMA expert kernels for the prompt sets, on by default on gfx1151 (Strix Halo: prompts +22..29% at the same KL), opt-in on other gfx11 parts; STRATA_DS4_HIPK=0 turns them off 2026-10-24 18:13:43 +02:00
Niko1221 7063f3adca Sizes in decimal MB on the first-run card and the update check (one helper); the engine bundle of a per-GPU install is removed after an update only when the same engine is complete in the store (size and SHA-256), else kept with one log line 2026-10-24 18:10:05 +02:00
Niko1221 1c69e23658 Merge f-0243-guifix final head into f-0243-rel 2026-10-24 17:50:44 +02:00
Niko1221 f613637cbd Merge f-026-5070 (experts N|auto, IQ1_M row) into f-0243-rel 2026-10-24 17:50:01 +02:00
Niko1221 4f47009d08 Merge f-0243-fuse into f-0243-rel 2026-10-24 17:48:34 +02:00
Niko1221 94a4a34b05 Merge f-026-qwen into f-0243-rel 2026-10-24 17:46:13 +02:00
Niko1221 8993997e02 Merge f-025-strix into f-0243-rel 2026-10-24 17:46:13 +02:00
Niko1221 9741a61d7b Merge f-0250-tui into f-0243-rel 2026-10-24 17:46:13 +02:00
Niko1221 06f313b019 CPU lane IQ1_M AVX-512 rows: the 64-weight step's scale words by one vpermw and the delta signs by a masked subtract (bit-identical to ggml's dot: 0 of 2.7M rows differ at 1 and 2 tokens on Zen 4 and Sapphire Rapids); +9% on the rows, the 1-token policy is unchanged 2026-10-24 17:41:33 +02:00
Niko1221 236e13b4af Settings update check: a failed check of the app itself says so in one plain line, with the raw error behind Details (it used to show only Control's new-version text) 2026-10-24 17:25:53 +02:00
Niko1221 c893eeebf1 App update: an update job an older host left running is finished when the installer has written uninstall.exe or changed the program file after the job began (the installer keeps the build time as the modified time) 2026-10-24 17:11:55 +02:00
Niko1221 9b4092a94d launch: the Speed choice and --experts need a DeepSeek engine of 0.1.50 or newer (0.1.49.2 does not accept the flag): an older or unknown engine gets no flag, no Speed step and no plan line, and no error; the version comes from the package folder, else --version; tests 2026-10-24 16:47:17 +02:00
Niko1221 3283d2ae49 DeepSeek Speed: offered and passed only when the installed DeepSeek engine is 0.1.50 or newer (an older one ends the start with unknown option --experts) 2026-10-24 16:46:06 +02:00
Niko1221 b79d6930c1 test: the own-GGUF check loads its helper by path (another tests package can already be loaded in a full tools run) 2026-10-24 16:43:46 +02:00
Niko1221 883e21bf2e tests: the app-owned settings list includes the DeepSeek Speed choice 2026-10-24 16:43:31 +02:00
Niko1221 4e4921f63e DeepSeek settings: Speed (Faster / Full quality), default shown by the engine's own 16 GB rule, passed as --experts 5|6 only when chosen 2026-10-24 16:42:29 +02:00
Niko1221 bd7acc0e6d launch: the 16 GiB rule for the Speed label is compared in whole MiB; the catalog's vram_gb is already GiB (nvidia-smi MiB / 1024, AMD bytes / 2**30), so a 16 GB card (16376 MiB) counts as 16 GiB or less; tests with exact MiB values 2026-10-24 16:39:53 +02:00
Niko1221 c4413c0b54 launch: one rule for DeepSeek's speed. Nothing is passed to the engine unless the person chose (its --experts auto decides: 5 at 16 GiB of cards or less); the 'default on this PC' words use the same 16 GiB rule summed over the cards the launch uses, and the summary reads the engine's own 'experts per word: N' line back; tests follow 2026-10-24 16:36:49 +02:00
Niko1221 bc1a3ba2ed launch: DeepSeek Speed choice, Faster (--experts 5, the default on a PC with 16 GB of graphics memory or less) or Full quality (--experts 6): a Speed step in the guided screens, --speed and --full-quality, plain words in --plan and the summary, the quality note; tests, docs 2026-10-24 16:33:51 +02:00
Niko1221 3dd11c6dc1 DeepSeek engine: --experts N (the router's top N, weights renormalised); default 5 where the cards hold 16 GiB or less in all, the model's 6 elsewhere; startup line 'experts per word: 5 (faster mode; full quality: --experts 6)'; DS4_TOPK=N stays as a test override; rule test; DS4_PROF prints the CPU lane's stage split 2026-10-24 16:31:09 +02:00
Niko1221 eb4736c45e DeepSeek engine: on a Windows integrated GPU the experts leave room for the context's later GPU allocations
The Windows commit limit decides how many experts an integrated GPU may hold (place_experts). With AMD driver 26.10.1 the limit
went from 74 GB to 133 GB on a Strix Halo, the engine put 9506 experts (86 GB) on the GPU instead of 8105, every byte of the
GPU's address space was gone, and the 32K prompt's later allocations spilled into RAM: free RAM fell to 0 and the engine died
("rocBLAS: No devices found"). The experts now leave 4 GiB + 0.2 MB a token of context above 16384 (DS4_UMA_GPU_RESERVE_GB sets
it, tests); only an integrated GPU on Windows is affected.

Strix Halo (gfx1151, 2.00 pack, shipped profile), driver 26.10.1:
  32K prompt: killed (min free RAM 0) before, 113.2 tok/s with min free 13.6 GB now; 64K prompt (70000 context): 122.2 tok/s, min free 14.4 GB
  4K / 8K prompt: 62.8 / 83.9 tok/s (64.9 / 86.2 with the 9506 experts: -3%), min free 13.4-13.8 GB
  KL, code segment, 999 positions: 0.1020 (0.1028 before)
A smaller reserve (3 GiB + 0.1 MB a token) gained 0.0% on 4K / 8K / code decode and left 0.9 GB less free at 64K, so it was not taken.
2026-10-24 16:22:28 +02:00
Niko1221 66b43d4a7f Monitor: with All GPUs, a card that cannot report a value is left out and the card that does report is named (35 C . RTX 5070) instead of blanking the figure 2026-10-24 16:09:59 +02:00
Niko1221 91eb05a228 App update: a job an older host left running is finished when the app's program file is newer than the job 2026-10-24 16:07:45 +02:00
Niko1221 4228e6ee11 chat model picker: a downloaded model that is not loaded uses the same two-line row as the API models 2026-10-24 15:59:39 +02:00
Niko1221 9caf39495c Merge f-0243-guifix into f-0243-rel 2026-10-24 15:44:30 +02:00
Niko1221 0c53372a0e Merge f-0250-tui into f-0243-rel 2026-10-24 15:44:30 +02:00
Niko1221 a3e4cb64fd test: serving never reaches the DeepSeek engine check (only launch and setup import it), and 'serve --plan runs no program' means no installer or fetch: a PC's read-only hardware probes (nvidia-smi, Get-CimInstance) no longer fail it 2026-10-24 15:26:07 +02:00
Niko1221 dc2c4229e9 test: launch_engine_first gives the runner a Qwen catalog (a bare Mock answered 'DeepSeek' to the 0.2.4 multi-file question and skipped the engine step); the launch code was right 2026-10-24 15:25:07 +02:00
Niko1221 60d856a18e DeepSeek decode: fewer launches a layer - the int8 roundings of one GEMV call in one launch (identical inputs once) and hc_mix + hc_split in one launch (the token's last mix block runs the split); same arithmetic
R9700 1 card 2.00: 21.30 -> 21.82 tok/s (+2.4%), 4 cards 35.39 -> 37.00 (+4.5%), 3 pairs each, same generated ids. DS4_QUANT_SINGLE=1 / DS4_QUANT_NODEDUP=1 go back to one launch an item / no sharing.
2026-10-24 14:47:48 +02:00
Niko1221 abab458d74 Feed reads: one more try when the first times out or the connection fails (the first wait is 12 s, the second the full 30 s); about one github.com request in 25 hung for 30 s and the first-run card said offline 2026-10-24 14:44:12 +02:00
Niko1221 9cf91f3555 Load of DeepSeek: the start wait is at most the 600 s Control allows 2026-10-24 14:36:14 +02:00
Niko1221 f30d191d54 Load of DeepSeek: the app adds no Flash-Next options (request slots, RAM budget, output clamp, resource profile, memory retry) to the DeepSeek engine and waits as long as its config says 2026-10-24 14:34:47 +02:00
Niko1221 75ee28fd3d Host: the per-model settings routes accept a dotted id; a test walks a dotted model id through list, load and unload 2026-10-24 14:33:40 +02:00
Niko1221 fe2b9f4094 Host: run config file names with a dot (DeepSeek) pass the app's own checks, so the model can be loaded 2026-10-24 14:31:43 +02:00
Niko1221 54768d25b1 test: the own-GGUF check finds tests.terminal when tools/ is run on its own 2026-10-24 14:29:49 +02:00
Niko1221 4499c69681 Installed models: an id with a dot (deepseek-v4.1-flash-ist-das-2.00bpw) is listed; the install finished but the app never showed the model 2026-10-24 14:22:39 +02:00
Niko1221 beeea0abf3 test: the Mac build-tools test follows find_tool's version check 2026-10-24 14:19:31 +02:00
Niko1221 569f7c739d test: the --kv choices check reads the last line of the error (Python 3.12 and 3.13 print the list differently) 2026-10-24 14:18:47 +02:00
Niko1221 ff84ff8164 First-run card: one plain line when the drive for the engine is short on space 2026-10-24 14:14:32 +02:00
Niko1221 61a4b6bd02 First-run card: no dash in the engine size line 2026-10-24 14:14:10 +02:00
Niko1221 192ab03813 DeepSeek-V4.1-Flash in the app: the model picker lists the three official sizes (not the 508 GB multi-card set), an 8K context only, a plain size/speed/disk note that warns and never blocks, no calibration step; the install tool accepts the own-path sizes 2026-10-24 14:11:45 +02:00
Niko1221 16c4da9443 CPU expert rows: 512-bit grid lookups by gather where a gather is cheap; one IQ3 token on the exact 512-bit rows
The 512-bit i-quant rows (IQ2_XXS/XS/S, IQ3_XXS/S) assembled their grid words from scalar loads (~30 uops a 64-value chunk). Where cpu_gather_fast()
(Intel Sapphire Rapids and later) they now load them by one gather, 32-bit for IQ3 and 64-bit for IQ2 (STRATA_IQ_GATHER=1/0 force it; Zen 4 stays scalar:
its gathers are microcoded). The same words are loaded, so every output bit is unchanged. One token of an IQ3_XXS / IQ3_S expert takes the exact 512-bit rows (ggml-cpu's
single-token AVX2 dot bit for bit) with the gather, after a first-use bit check against this ggml; STRATA_NO_IQ512X1=1 turns it off.

Xeon 8468, native_gu_rows (gate+up of an expert), us: IQ3_XXS 1 token 295 -> 219, 2 tokens 462 -> 300, 4 tokens 558 -> 389; IQ3_S 415 -> 305, 603 -> 342, 562 -> 296;
IQ2_S 2 tokens 377 -> 235, 4 tokens 437 -> 327 (one IQ2_S token stays on ggml: 239 against 234). Output checksums identical with and without.
Test iq512_gather_parity (both lookup modes): exact rows == ggml vec_dot at 1-4 tokens, engine rows within float rounding.
2026-10-24 14:09:08 +02:00
Niko1221 9030da09f9 Merge f-026-5070 into f-0243-rel 2026-10-24 14:01:05 +02:00
Niko1221 980c95a6d7 Merge f-0243-store into f-0243-rel 2026-10-24 14:01:04 +02:00
Niko1221 86a751af27 Merge f-025-strix into f-0243-rel 2026-10-24 14:01:04 +02:00
Niko1221 84b712a3d8 Merge f-0243-sm89 into f-0243-rel 2026-10-24 14:01:04 +02:00
Niko1221 db42e59930 First-run card stays until Continue; a processor's own graphics is not counted as a card 2026-10-24 13:58:36 +02:00
Niko1221 c1e80f7952 Markdown: a single newline inside a paragraph is a soft break (a space); two spaces or a backslash at the end of a line keeps a line break (#3818) 2026-10-24 13:55:42 +02:00
Niko1221 ea349eb8b8 Language: 43 interface strings that had no key (/plan /research /new hints, tuning, goal, telemetry, task and picture texts, the new fix texts) now exist in all 8 languages; the four Open hints translate their label; a test fails when a tr() literal or hint has no key (#3479, #3825) 2026-10-24 13:51:18 +02:00
Niko1221 08a5e6b34a packaging smoke: the packaged Terminal carries the installer since 0.2.4, so 'strata install' without a keyboard stops at the first question instead of saying 'no installer'; the check accepts both 2026-10-24 13:48:09 +02:00
Niko1221 353c9f2556 chat: a PDF is added to the chat folder and named in the next message; the file dialog lists PDFs and text on Linux (#3838) 2026-10-24 13:48:06 +02:00
Niko1221 ec03ae0106 First-run engine card: GPU, engine size, one Download button, progress with resume, 'I already have the file' (offline), plain states; host routes /api/engine/first-run, /job and /from-file; install-engine-file job 2026-10-24 13:47:22 +02:00
Niko1221 780a69b41b Expert settings: decimals can be typed in sampling and number fields; Ctrl+V of a screenshot reads clipboard items (#3736, #3748) 2026-10-24 13:47:18 +02:00
Niko1221 c7899f9c89 chat: Copy response and Copy message answer Copied or say they could not (with an older-copy fallback); a typed message keeps its line breaks (#3731, #3720) 2026-10-24 13:45:58 +02:00
Niko1221 e61b7024cf Models page: a neutral placeholder until the engine's first answer, not the bundled Starter (#3740) 2026-10-24 13:45:12 +02:00
Niko1221 25abbdd90c Models page: the chosen conversation context and cache format count in the memory estimate (#3832) 2026-10-24 13:43:25 +02:00
Niko1221 f750086183 Models page: the chosen conversation context and cache format count in the memory estimate (#3832) 2026-10-24 13:43:02 +02:00
Niko1221 31adebc8b7 strata components install-file: install an engine package the person downloaded (signature, every file, fit, test start, switch; no network); the engine switch is shared with the download path 2026-10-24 13:42:41 +02:00
Niko1221 4ccfd7ca30 Windows AMD/Intel cards: real memory size from the DirectX adapter list, load and memory in use from the Windows GPU counters, virtual monitors no longer listed as cards (#3727, #3836) 2026-10-24 13:40:59 +02:00
Niko1221 b750d757f6 CUDA engine selector: sm_MN machine code also covers a card sm_MK of the same major version with K >= N (CUDA cubin compatibility), so an RTX 40 card (sm_89) takes an sm_86 package; tests for the Qwen and DeepSeek packages; DeepSeek card text says RTX 30, 40 and 50 2026-10-24 13:40:27 +02:00
Niko1221 d9d033be86 launch: at 80 columns the model list keeps its verdict column (fits, tight, too big and the recommendation) instead of cutting it off; test 2026-10-24 13:40:22 +02:00
Niko1221 9c375ccb7d launch: setup's DeepSeek engine upgrade lines (the failure one 'A newer DeepSeek engine is available but could not be installed') are shown on the setup path too; test 2026-10-24 13:38:49 +02:00
chicco83 90a7aeea51 Setup wizard: show the VRAM of AMD cards on Windows from the engine's list when the host probe reports 0 (#3727, PR #3798 by chicco83) 2026-10-24 13:38:38 +02:00
Niko1221 d6c1936021 setup: a source build on Windows needs Visual Studio 2022 (2019 cannot compile the C++20 sources) and the stop says so, with the ready-made engine as the way around (#3566); tests follow 2026-10-24 13:37:35 +02:00
Niko1221 fb3bae20c6 setup: the list of installed packages remembers which system wrote it, so a .venv shared by Windows and Linux installs again instead of saying 'already installed' (#3634); test 2026-10-24 13:36:55 +02:00
Niko1221 1d20d3e2b0 chat: reopening a chat restores its workspace and the turn runs in that folder (#3754) 2026-10-24 13:36:35 +02:00
Niko1221 13a4153be4 copy-text: load the toast store only when copying, so the Markdown renderer still renders without a browser 2026-10-24 13:35:56 +02:00
Niko1221 ea897551ee folder picker: a PowerShell that ends before its window opens (security software) falls back to typing the path (#3831) 2026-10-24 13:35:54 +02:00
Niko1221 ed0c8939aa setup: a download that gets no data for two minutes says so with the percentage and the one next step, instead of a line that never moves (#3567); test 2026-10-24 13:35:43 +02:00
Niko1221 47ccec7c6a Delete a model: files Strata downloaded and packs it made go with it even in another models folder; the person's own files stay (#3716) 2026-10-24 13:35:04 +02:00
Niko1221 aef053e353 launch: the Access screen's help names --show-key and --api-key, so a person can see or set a key of their own (#3713 #3951); test 2026-10-24 13:34:18 +02:00
Niko1221 c22e5e7633 missing files: name the engine program in words once, not as component:strata-engine (#3900) 2026-10-24 13:34:11 +02:00
Niko1221 27cbbfe6d6 launch: the start waits longer for a big model by itself (10 minutes, about 8 s more per GB above 40 GB, at most 30), the server's own engine wait follows, and the runtime accepts start waits up to 1800 s (#3949 #3459). Own fix instead of PR #3817: that one needs people to set STRATA_ENGINE_READY_S; tests 2026-10-24 13:34:04 +02:00
Niko1221 97c31168aa chat model picker lists downloaded optimized models that are not loaded yet (#3814) 2026-10-24 13:33:56 +02:00
Niko1221 5faf357b49 Chat: Copy response/message/code say Copied or Couldn't copy and fall back to the older copy route (#3731); a pasted screenshot is also read from clipboard items (#3748); the user bubble keeps its line breaks (#3720) 2026-10-24 13:33:17 +02:00
Niko1221 77284b5e33 update: the test start runs the new engine with a short context and no picture reader (#3842), and an update keeps the CUDA build that runs here (#3835); launch: the summary names the engine log (#3656) and a tuned model says how to measure again (#3874); tests 2026-10-24 13:32:44 +02:00
Niko1221 4b49729a1e monitor: read Windows disk counters from one long-lived PowerShell instead of a cold one every 3 s (#3939) 2026-10-24 13:32:09 +02:00
Niko1221 014fd6b5db App update: log lines for every update step, an alert when the app came back as the old version, and Update engine works before the native snapshot has connected 2026-10-24 13:30:13 +02:00
Niko1221 5bfce4bea0 setup: the build tools come from the virtualenv first and an older system cmake (below the 3.24 CMakeLists.txt needs) counts as missing, so pip installs the right one (#3822); tests 2026-10-24 13:28:02 +02:00
Niko1221 15fdb69563 In-app update: a finished app update is no longer shown as an interrupted operation; a test update feed used once is not remembered as the feed for later checks
After 'Update Strata' installed a new app version, the host that asked for it was ended by the installer, so the job stayed saved as running and the next start turned it into a red 'Interrupted operation' alert in Settings > Update and Diagnostics. The job now records the app version it started from, and a start under a different version marks it done ('Strata is updated to X').

strata update and the app's check reused the feed used last, including a loopback address or a folder from a test of the update feed; every later check on that PC then said 'No internet connection'. Only an https address is remembered now.
2026-10-24 13:27:28 +02:00
Niko1221 ad9bb57ddc own model files: a qwen4exp file set that is no tested build but has the layout of one is used as that model with a plain notice, and setup keeps a complete GGUF of another size or hash with a warning instead of stopping (#3651 #3777); tests 2026-10-24 13:27:08 +02:00
Niko1221 b18700c8f7 Adaptive tier: the engine chooses the asynchronous tier by measurement (arena, one CUDA GPU)
Without --adapt-async on the command line, the expert arena on one CUDA GPU (--serve, adaptive tier on, no --batch, no layer
split, no pipelined windows, no peer or remote tier) starts on the blocking tier, ignores 64 steady decode windows, measures 64,
and when the tier's wait is at least 5% of a window runs 64 windows on the asynchronous tier, going back (for the session) only if
that was more than 2% slower per token. One log line at each switch. --adapt-async 0|1 and STRATA_ADAPT_ASYNC=0 keep their meaning;
the identity gates run with --adapt-every 100000 and never reach it. Decision logic and its unit test are header-only
(adapt_async_default.hpp, adapt_async_default_test).
2026-10-24 13:27:00 +02:00
Niko1221 c02a0b8b0d UPDATE.bat / update.sh: an engine that stayed older than the minimum is said as 'the engine was NOT updated' with one next step, not '[ok] Strata is updated' (#3653); test 2026-10-24 13:08:46 +02:00
demetree 7a03fc9414 strata connect: scale the app configs' compaction point with the server's context 2026-10-24 13:06:11 +02:00
Niko1221 d924beffe5 setup: a Mac with an Intel (Rosetta) Python is still an Apple Silicon Mac; the chip name and hw.optional.arm64 decide, one plain line says the Python is an Intel build (#3894); test 2026-10-24 13:06:00 +02:00
Niko1221 bdaa93edf0 launch: a setup option error ('setup.py: error: argument --kv: invalid choice') is the cause shown, not only 'could not be installed'; setup accepts --kv fp16 as DOCKER.md says (#3785); tests 2026-10-24 13:05:30 +02:00
Niko1221 1736b558bd setup: a NVIDIA driver older than CUDA 13 needs (528 to 579) runs the CUDA 12 engine by itself and says so in one line instead of stopping and naming a flag strata launch does not have (#3843); tests 2026-10-24 13:04:44 +02:00
Niko1221 0cd44323e4 Windows: a card taken out of the PC no longer decides the engine; only the display adapters Windows reports as present count when two makes are in the registry (#3892); test 2026-10-24 13:03:56 +02:00
Niko1221 e3eeb93972 DeepSeek engine: the RAM store's background fill is opt-in (DS4_STORE_BG=1): ready 9 s sooner but the first prompt waits for the same SSD and decode measured 3.7% lower 2026-10-24 12:40:19 +02:00
Niko1221 58f2acc05b DeepSeek engine: the RAM store's fill beyond the first 48 GB is read in the background after the model is ready (hottest first, behind the model's own reads); DS4_STORE_BG=0 reads it all first 2026-10-24 12:33:10 +02:00
Niko1221 6fcde7e305 DeepSeek CPU lane: compute the store hits while the misses are read, each missed expert as its read lands (DS4_MISS_SPLIT=0 restores the wait-for-all order); ExpertStore::ensure split into begin / wait_slots 2026-10-24 12:29:56 +02:00
Niko1221 b3c86c783f DeepSeek engine: the automatic RAM store may take every routed expert, not 48 GB at most, where the free RAM holds them (1 R9700 + 251 GB, 2.00 set: 12.6 -> 21.4 tokens/s); free - 12 GB and the explicit sizes unchanged 2026-10-24 11:58:35 +02:00
Niko1221 f1e41cdee2 DeepSeek engine: the automatic RAM store is sized again after the weights are on the GPU, and keeps room for the context's buffers, so a unified-memory PC (Strix Halo) keeps 12 GB free; explicit sizes unchanged 2026-10-24 11:28:09 +02:00
Niko1221 7dd5cf18e6 release notes 0.2.4.2: the picture-file and already-have-files fixes (#3927, #3917, #3914); no environment variable in the conversations line 2026-10-24 10:52:25 +02:00
Niko1221 8aacdbc7a0 release notes 0.2.4.2: the list of new files 2026-10-24 10:51:27 +02:00
Niko1221 3eabee6049 test: the restyled READMEs state the app version through their download buttons (Strata-0.2.3-...), not the old '0.2.3 · ' header line 2026-10-24 10:51:13 +02:00
Niko1221 c896f73934 release notes 0.2.4.2: the DeepSeek engine packages are 0.1.49.2, the same number the binary prints and the Control has 2026-10-24 10:36:34 +02:00
Niko1221 93a91a2767 README translations (zh-CN, ja, de, fr, es, pt-BR): brought up to the restyled English README (download buttons per system, section menu, coffee badge, plain-word DeepSeek table, shorter Using it) 2026-10-24 10:36:09 +02:00
Niko1221 12939aa64d README: restyled to the owner's GitHub layout (download buttons per system, a section menu and a coffee badge at the top, plain-word DeepSeek table with the measured 0.2.4 speeds, shorter Using it, jargon cut); DEEPSEEK41.md: 4x R9700 speed updated to the 0.2.4 measurements 2026-10-24 10:29:36 +02:00
Niko1221 dfa3f4025c DeepSeek: pack_gguf takes an automatically found mmproj only when its projector is deepseek41v (a Qwen mmproj in the shared models folder is ignored, no error: #3927 #3917); files the person already has are recognised as DeepSeek by the pinned shard names in the guided screen too, not by the generic GGUF check that called the UD n-gram shard unreadable (#3914); tests 2026-10-24 10:27:57 +02:00
Niko1221 ac08a0820d release notes 0.2.4.2: conversations and a shared system prompt are kept on disk 2026-10-24 10:12:46 +02:00
Niko1221 05041e5db0 Merge branch 'f-024-on' into f-0242-rel 2026-10-24 10:12:41 +02:00
Niko1221 3157c38c68 DeepSeek engine: an installed package older than the signed feed's is updated by itself on launch and setup (fetched and checked, put next to the old one, started once, then switched to and the old one removed; one plain line and the old one keeps running on a failure; silent without a network; never for --dsv41-package or STRATA_DSV41_EXE); tests; release notes 2026-10-24 09:35:15 +02:00
Niko1221 d2d198a492 release notes 0.2.4.2: the DeepSeek engine 0.1.49.1 (faster on AMD Zen 1/2 CPUs, answers unchanged) and how to get it 2026-10-24 09:30:42 +02:00
Niko1221 8c9bd206ce Merge branch 'f-0243-iq1m' into f-0242-rel 2026-10-24 09:29:52 +02:00
Niko1221 46bddbbb74 dsv41: BMI2-off ggml-cpu applies only to the DeepSeek engine build (STRATA_BUILD_DSV41); ds4_gg_bench --dump FILE to compare two ggml builds bit for bit (IQ1_S/IQ1_M with and without BMI2: identical at 1,2,4,8,16 tokens) 2026-10-24 09:14:06 +02:00
Niko1221 6953d896d0 dsv41 CPU lane: IQ1_M uses Strata's exact rows from one token (ggml's IQ1_M dot runs at 1.8 GB/s on AMD Zen 2: _pdep_u64 is microcoded there); portable ggml-cpu build has BMI2 off (IQ1_S 2.3 -> 37 GB/s, IQ1_M 1.8 -> 32 GB/s on an EPYC 7452) 2026-10-24 09:09:23 +02:00
Niko1221 baa8f616de DeepSeek-V4.1: conversation slots and the shared-start checkpoint on by default (STRATA_DS4_SLOT_KEEP=0 / STRATA_DS4_AUTO_PIN=0 turn them off) 2026-10-24 08:25:17 +02:00
Niko1221 ac669f199d Terminal 0.2.4.2 / Control 0.1.49.2: version bump for the DeepSeek launch and install fixes (app stays 0.2.3, engine packages unchanged); release notes 2026-10-24 06:29:42 +02:00
Niko1221 0003604859 launch: the second sentence of 'The model could not be installed.' starts with a capital; test 2026-10-24 06:29:42 +02:00
Niko1221 e168b76b8c Merge branch 'f-0242' into f-0242-rel 2026-10-24 06:28:20 +02:00
Niko1221 f53664c159 launch: a setup message that ends with a full stop does not get a second one ('engine..'); test 2026-10-24 05:43:54 +02:00
Niko1221 44f8c334de model list: the DeepSeek disk check counts only what is still to be downloaded (files already in the models folder, also under the first release's names, are done), plus the room for the pack; tests 2026-10-24 05:41:50 +02:00
Niko1221 a22d557eee launch --plan: a DeepSeek plan no longer promises a Strata engine download; test 2026-10-24 05:39:30 +02:00
Niko1221 51d23fdd1e P2003 findings: DeepSeek setup says in one line when GitHub cannot be reached (a 10 s check before any download, not minutes of retries) and no longer fetches or test-starts the Strata engine, llama.cpp source or CUDA libraries it does not use; install.sh is one function called on the last line (a half-downloaded file runs nothing) and every documented one-liner is 'curl ... -o install.sh && sh install.sh' so a failed curl cannot look like success; tests 2026-10-24 05:37:41 +02:00
Niko1221 a62ca064a3 launch: the DeepSeek engine package that runs is the newest of the running Terminal's own install (STRATA_INSTALL_DIR, else its own folder) and the install the model's settings came from (the first remembered install, which can be an older Terminal folder or another checkout); the running Terminal's own on a tie; the choice is logged; versions compare as numbers; tests 2026-10-24 05:09:58 +02:00
Niko1221 8183c6b671 docs: RTX 3060 speed with the release defaults (8 to 10 tokens/s, 149 tokens/s at 8K); DSpark stays off on 12 GB cards and adds 2-13% when turned on; README x7, DEEPSEEK41.md, release notes 2026-10-24 04:59:46 +02:00
Niko1221 c33fa7ba6d Terminal 0.2.4.1 / Control 0.1.49.1: version bump for the DeepSeek-setup hotfix (app stays 0.2.3, engine packages unchanged); release notes 2026-10-24 04:36:57 +02:00
Niko1221 b7958c044f Packaged Terminal: tools/dsv41/pack_gguf.py, smoke.py, tune.py (and strata_inspect, strata_bench, nvfp4_local_pack) are in the bundle; a test reads every tools/*.py that setup.py, strata_cli and strata_control run by path and fails when one is missing from the bundle, and starts the DeepSeek scripts from the bundle alone 2026-10-24 04:36:57 +02:00
Niko1221 06e51dfaef test: the failed-update reason is capitalised where it joins the sentence (follows 00a2cfa07) 2026-10-24 04:13:07 +02:00
Niko1221 96ca4383c7 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-24 03:17:28 +02:00
Niko1221 2ecdc2e511 launch: an installed DeepSeek model whose config names a gone engine, with no package in this install, is not 'already installed': setup runs (feed download, or one plain line with the next step) instead of the start failing on a missing file; tests 2026-10-24 02:58:01 +02:00
Niko1221 5353097c90 DeepSeek quality numbers from the final-tip scores (H100, 4,995 positions): 2.00 KL 0.170 ppl +15.5% (code +8.6%, agent +5.3%), 2.25 KL 0.107 ppl +7.3% (code +3.2%, agent +2.2%); README x7, DEEPSEEK41.md, release notes, setup and fit text 2026-10-24 02:57:14 +02:00
Niko1221 9011bd107a launch: DeepSeek runs on this install's own engine package, not the exe path baked into an installed model's config (which can come from another Strata install on the computer or name a package folder an update replaced); tests 2026-10-24 02:56:16 +02:00
Niko1221 52b104b1ac IQ1_M quality from the H100 (KL 0.217, ppl 3.80, +21%; code +8.9%, agent +10.2%, weaker on plain prose) in setup, the fit text, README x7, DEEPSEEK41.md and the release notes 2026-10-24 02:37:08 +02:00
Niko1221 00a2cfa079 launch: a reason joined after 'The engine could not be installed.' starts with a capital; test 2026-10-24 02:36:17 +02:00
Niko1221 6eaf32075c Merge f-024-maya2 (6c6267f33) into f-024-rel: Windows batch submit, flow pump, Stop during a long prompt, lane-free, opt-in switches (all in src/dsv41)
Five conflicts, resolved keeping every f-024-rel behaviour (slots pin, long-prompt skip, HIPK, ZC fix) and maya2's:
- ds4_main.cpp, request struct: both fields kept (pin=N from f-024-slots and dspark_cum from maya2).
- ds4_main.cpp, request parser: both keys kept (pin, dspark_cum/spec_cum).
- ds4_main.cpp, prompt feed loop: slots' pin_at (the read stops at N for the shared-start checkpoint 3) and maya2's Stop handling (checkpoint 1 before the prompt, set_cancel, the stop check at the top of each feed) combined: the loop checks Stop first, then sizes the feed with the pin cut (n is no longer const).
- ds4_model.cu, forward chunk loop (two places): the skip's 'run.runs(k, il)' (an early chunk skipped past the yoco layer) kept, and maya2's halt_now() / halt_end(di, err) check added after it, so a skipped layer does no halt work.
win_submit (ds4_kernels.cu) is compiled to a no-op outside _WIN32 (#if defined(_WIN32) ... else (void) s), so it is already out of the Linux CUDA and Linux HIP builds.
2026-10-24 02:35:40 +02:00
Niko1221 698a87d999 Merge branch 'f-024-tui' into f-024-rel 2026-10-24 02:34:57 +02:00
Niko1221 6c6267f33c dsv41: Maya-port defaults from the owner-PC A/Bs (ON: win_submit, flow pump, Stop, lane-free; OFF with switch: timer, Engram readers, unbuffered Engram, cold prefetch, DSpark cum)
Owner PC (RTX 5070 12 GB, Ryzen 5 7600, official 2.00 pack, shipped profile-2.00.bin, 3 alternating pairs, medians, off = every switch off):
 4K prompt, off 92.64 tok/s: win_submit 99.12 (+7.0%, 3/3 pairs), flow pump 97.37 (+5.1%, 3/3), 64 Engram readers 96.64 (+4.3%, 3/3),
   unbuffered Engram 85.56 (-7.6%, 0/3), all on 86.84 (-6.3%).
 code decode, off 4.51: timer 4.51 (1/3), 64 readers 4.41 (2/3), unbuffered 4.67 (2/3, one pair 3.83), all 4.57.
 mmap decode, off 1.95: cold-page prefetch 1.71 (-12%, 0/3).
 Cold (no profile) agreed on win_submit and flow pump (+4.5%, +2.7%, 2/2) and showed the timer +12%, readers + unbuffered +22% on decode (2/2, off arm noisy).
 Identity: --vram-experts 0, off vs all IDENTICAL (4K prompt, code decode); text equal to off in every warm arm.
 Stop during a long prompt (H100, 8K and 16K): STOP answered in 0.12 / 0.16 s, the conversation before the prompt kept, the next request identical.
 DSpark cum cut: H100 code 20.13 tok/s off vs 19.60 at 0.5, no gain.
ON: DS4_WIN_SUBMIT, DS4_FLOW_PUMP_WAIT, Stop, STRATA_DS4_LANE_FREE (a no-op on the physical-core lane).
OFF, switch turns it on: DS4_TIMER_MS=1, DS4_ENGRAM_THREADS=64, DS4_EG_DIRECT=1, DS4_IQ_PF_COLD=1, STRATA_DS4_DSPARK_CUM=0.5.
2026-10-24 02:29:57 +02:00
Niko1221 cec75af6ae launch: the Save screen shows DeepSeek's own tuning length (5-10 / 10-15 min) like the Tuning screen; walk test: save text, run paths, error paths 2026-10-24 02:29:04 +02:00
Niko1221 8ac2a8385e tests: the 1.75 bpw set also runs with --tp 1 on 4 AMD cards; release notes: set up again on 4+ AMD cards 2026-10-24 02:24:31 +02:00
Niko1221 894dc24c6c Merge branch 'f-024-mgpu' into f-024-rel 2026-10-24 02:23:57 +02:00
Niko1221 c4dcde7145 dsv41 iq test: IQ1_S set gate_up (f16) tolerance 1.5e-2 (data-order dependent, 7.1e-3..1.08e-2, same on the older build); fix a stale comment in ds4_model.cu (IQ1_S/IQ1_M/IQ4_NL have device kernels) 2026-10-24 02:18:45 +02:00
Niko1221 67d5f44bf8 dsv41 plan: four or more AMD cards run the IST-DAS sets with --tp 1 (8K prompt 370 against 196 tokens a second on 4 x R9700) 2026-10-24 02:17:25 +02:00
Niko1221 6a35eb6c5f release notes: three sets everywhere, the DeepSeek engine is separate, Mac Control 0.1.49 2026-10-24 02:15:32 +02:00
Niko1221 12d0a4a051 Merge f-dsv41-tune (c5e0e1482) into f-024-rel: the third set IST-DAS 1.75 bpw (IQ1_M) in setup, fit and the launch list
One conflict, docs/DEEPSEEK41.md (both sides edited the three-set paragraph and the file table). Resolution: f-024-rel's text kept (IQ1_M first, the model-card sentence, 533 GB for all three, the 'not a model to pick' Engram row), with the setup worker's sentence that setup recommends 1.75 where 2.00 is tight in memory or disk added to the 1.75 entry.
2026-10-24 02:09:11 +02:00
Niko1221 c5e0e14823 DeepSeek: IST-DAS 1.75 bpw (IQ1_M), the smallest official set, as model ist-das-1.75bpw: pins at ISTA-DASLab cb02422c (the 9 earlier files are unchanged there), fit numbers, setup recommends it where 2.00 is tight in memory or disk (warn, never refuse), catalog/TUI text, docs, tests 2026-10-24 02:08:43 +02:00
Niko1221 9bb00346a2 release notes: R9700 prompt numbers from the r97 command (4 cards 8K 505, 32K 818; kernels +36% / +49%) 2026-10-24 02:04:13 +02:00
Niko1221 672612ac1c DeepSeek tables: the IQ1_M (1.75 bpw, ist-das-1.75bpw) row first, 239 GB; quality 'being measured' plus ISTA-DASLab's model-card figure; README + 6 translations, DEEPSEEK41.md, release notes (placeholder until the KL is in) 2026-10-24 02:03:56 +02:00
Niko1221 f4e7685ccf Merge f-024-rel into f-024-tui 2026-10-24 01:59:28 +02:00
Niko1221 267a2f4462 launch: a Runner that cannot be built is shown on the run screen as a plain failure instead of crashing the screens; tests: launch TUI walk (every screen, every key, Qwen and DeepSeek, 80x24 and wide, run paths, error paths) 2026-10-24 01:59:28 +02:00
Niko1221 eec705cc98 release notes: the prompt-skip agreement at the final tip, 99.5% of 799 positions 2026-10-24 01:58:45 +02:00
Niko1221 b0d5f6e57c Merge branch 'f-024-slots' into f-024-rel 2026-10-24 01:52:42 +02:00
Niko1221 f2413d77e9 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-24 01:52:27 +02:00
Niko1221 4f8770d13e release notes: R9700 prompt numbers with the new kernels (4 cards 8K about 320, 32K about 750) 2026-10-24 01:52:08 +02:00
Niko1221 984f52c827 Merge branch 'f-024-rel' into f-dsv41-tune 2026-10-24 01:51:58 +02:00
Niko1221 aebdc298f9 Merge f-024-hipk (cdf8b98f5) into f-024-rel: gfx12 WMMA i-quant prompt tiles for DeepSeek V4.1 (HIP only, default on for gfx12, STRATA_DS4_HIPK=0 off)
One conflict, src/dsv41/ds4_model.cu (forward_chunks): f-024-rel has the long-prompt skip (kv_only layers), f-024-hipk added DS4_PROF stage marks around the same lines.
Resolution: the f-024-rel code kept as it is, and hipk's three prof_mark(d, 5/6/7) calls added at the same places (before the q/kv gemms, before the snapshot, after the kv_only return). They are no-ops with DS4_PROF unset.
2026-10-24 01:51:58 +02:00
Niko1221 fd5823ab83 Merge branch 'f-dsv41-cuk-zcfix' into f-024-rel 2026-10-24 01:48:40 +02:00
Niko1221 d50da4e3a1 release notes: the run-to-run difference is near-tied words, not only expert placement 2026-10-24 01:48:18 +02:00
Niko1221 624a226d13 dsv41 store zero-copy: clear the device's zc_tab entry when VRAM takes an expert over (a stale pointer into a reused slot made zc_stage copy another expert's blob into the plan: garbage, NaN routing, illegal access) 2026-10-24 01:44:40 +02:00
Niko1221 fa3252ad5d dsv41 serve: with slots off the loop is as before (no anchor copy, same continuation test on an empty engine) 2026-10-24 01:43:35 +02:00
Niko1221 9797056e4d Merge branch 'f-024-rel' into f-024-slots
# Conflicts:
#	serve/test_dsv41.py
2026-10-24 01:43:15 +02:00
Niko1221 e2361d80d2 DeepSeek-V4.1: slot keep and auto-pin default off (switches STRATA_DS4_SLOT_KEEP=1, STRATA_DS4_AUTO_PIN=1) 2026-10-24 01:39:15 +02:00
Niko1221 afd24ea6ae launch: the last lines suggest 'strata launch --stop' (and --save-preset NAME to keep the settings) when no preset was saved; --preset "<name>" only when one exists; tests 2026-10-24 01:27:07 +02:00
Niko1221 4ebba26767 pack_gguf: an Engram file ('engram' or 'deepseek41-engram') is never taken for a model shard or for the model's metadata; real header fixture and tests; setup test: the first release's files are packed as they are with no shard download 2026-10-24 01:25:23 +02:00
Niko1221 598372c3fd setup: repin the DeepSeek 2.00/2.25 sets to ISTA-DASLab's final files (flat repository 0d8f49fb, IQ2_XXS = 2.00 bpw, IQ2_XS = 2.25 bpw, DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf); the first release's local files (model-2.00bpw-..., engram-IQ4_NL.gguf, in 2.00bpw/ folders) stay accepted by their old size and SHA-256 and are never downloaded again; tests 2026-10-24 01:24:14 +02:00
Niko1221 2e477ecd40 release notes: the 99.75% is agreement between skip on and off over 799 positions 2026-10-24 01:21:59 +02:00
Niko1221 0e121c7a13 DeepSeek table: the IQ4_NL Engram table row (README and 6 translations, DEEPSEEK41.md, release notes); notes: long-prompt skip and the 32,768 thinking budget 2026-10-24 01:21:32 +02:00
Niko1221 415e618085 Merge branch 'f-dsv41-maya' into f-024-rel 2026-10-24 01:20:31 +02:00
Niko1221 dfe29bce61 serve: DeepSeek-V4.1 thinks at most 32768 tokens when neither the request nor the config names a budget
Same default as Maya #363. For the deepseek41 family only: Service.reasoning_budget() answers 32768 when the request has
no reasoning_budget_tokens / thinking_budget_tokens and the config has no reasoning_budget_tokens at all (a config value,
0 included, wins; a request's 0 turns it off). The existing wrap-up (#123) closes the thinking at the budget, and the
answer-room clamp still cuts it to leave the answer its share of max_tokens. Thinking off and other families unchanged.
Tests in serve/test_dsv41.py: the default, a request's 0 and own value, the config's value and 0, another family, and a
5000-token cap that is wrapped up and answered. serve suite: 1027 tests OK (15 skipped).
2026-10-24 01:20:04 +02:00
Niko1221 f51b80c222 README: R9700 numbers in the DeepSeek block (11 tokens/s on one card, 4x about 31, 55 to 69 with DSpark); the same block in the six translated READMEs; release notes: one-card R9700 row 2026-10-24 01:19:04 +02:00
Niko1221 17c585c402 Merge branch 'f-024-hsamsg' into f-024-rel 2026-10-24 01:18:51 +02:00
Niko1221 41e804d93c launch: the hsa_exception sentence ignores hsaKmt lines, and a real cause in the same log (out of memory, no permission on the card) still wins; tests for both 2026-10-24 01:18:16 +02:00
Niko1221 524ff90e1a docs: the final DeepSeek quants (ISTA-DASLab repository) in the release notes and DEEPSEEK41.md 2026-10-24 01:17:00 +02:00
Niko1221 86834afedc Merge branch 'f-024-hsamsg' into f-024-rel 2026-10-24 01:16:46 +02:00
Niko1221 fa77717a57 README: DeepSeek V4.1-Flash (ISTA-DASLab IQ2_XXS / IQ2_XS) in the model list, Terminal only 2026-10-24 01:16:46 +02:00
Niko1221 60d847ba33 launch: a GPU runtime start failure (hsa_exception) says the card was not ready, with a next step; the runtime text stays in the log 2026-10-24 01:16:19 +02:00
Niko1221 a3f76dfd63 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-24 01:13:51 +02:00
Niko1221 b4418e30df serve: DeepSeek-V4.1 with a short max_tokens (48-1023) caps the thinking at half of it, so a text block always arrives
/v1/messages and /v1/chat/completions with max_tokens 200-600 and the thinking on spent all of max_tokens on the
thinking: the 0.1.45 answer room starts at max_tokens 1024 and a client that gave no budget of its own got no text block.
For the deepseek41 family only, with no budget from the client or the config, the thinking is now capped at max_tokens / 2
for max_tokens from 48 up to 1023 (the existing wrap-up, </think> and the answer follow in the other half). A client's
own reasoning_budget_tokens / budget_tokens wins, "answer_room": false turns it off, thinking off is unchanged, other
families (Qwen) are untouched. Maya #363 closes the thinking in the engine instead; the server's #123 budget already does
that for every engine, so only the short-cap gap needed closing.

Tests (serve/test_dsv41.py SmallCap, fake engine that thinks without end): max_tokens 200, 400 and 600 on both APIs return a
non-empty text block with the thinking wrapped up at half; thinking off answers directly; a client budget wins; a tool
call under a short cap still parses; another family keeps the old behaviour. serve suite: 1025 tests OK (15 skipped).
2026-10-24 01:11:34 +02:00
Niko1221 8a7102ec4d dsv41: a long prompt skips layers 20-39 on all but its last 2,668 rows (STRATA_DS4_PROMPT_SKIP, on by default)
Layers 20-39 of DeepSeek-V4.1-Flash read a prompt's earlier tokens only through layer 20's compressed KV and index
keys, and each looks back one 128-token window more than the layer before. So for the answer and for the rings the
decode reads they need their own work only on the last (40 - 20) x 127 + 128 = 2,668 rows. A prompt of ~3,700 or more
new tokens on one GPU now runs its earlier chunks through layers 0-19 plus layer 20's kv_only part (the hyper-connection
pre-norm, the compressor latent, the index keys, the compressed KV) and nothing of the later layers' attention or
experts; the chunk the cut falls in runs layers 20-39 from the cut row on; later chunks run every layer. The expert flow
plans nothing for the skipped (chunk, layer)s. STRATA_DS4_PROMPT_SKIP: 1 (default) the skip, 2 only the last 128 rows
(DeepSeek's own serving; answers shift a little), 0 off. Same approach as Maya #318.

H100 emulation (64 GB cgroup, 12 GiB VRAM, store 41 GB, 3.6 GB/s SSD cap): 16K prompt 69.3 s -> 51.5 s (236 -> 318
tok/s, two alternating pairs), 4K 32.5 s -> 30.7 s; 800 teacher-forced positions after a 4,200-token skipped prompt: KL
0.00029 against the full work, top-1 against the text 88.11% against 87.98%, ppl 1.5110 against 1.5111.

--score gets DS4_SCORE_PREFIX=N (the segments joined, the first N tokens read as one prompt with the last row's logits,
the rest scored) so the skip can be scored; tools/dsv41/test_prompt_skip.py runs it against a pack when one is named.
The Maya notice file now says later Maya changes are covered by the same notice.
2026-10-24 01:11:34 +02:00
Niko1221 be26cb7995 DeepSeek-V4.1: slots on by default (2 GB cap, one week), docs 2026-10-24 01:10:21 +02:00
Niko1221 736a4136e1 serve: DeepSeek-V4.1 engine gets the slot switch and a slot folder in Strata's data folder (other families untouched) 2026-10-24 01:09:06 +02:00
Niko1221 1d31e2a1e4 tools/dsv41/slot_keep_test.py: two agents taking turns, fresh reads vs shared start vs slots vs restart vs folder cap 2026-10-24 00:59:52 +02:00
Niko1221 b8f9c63ae7 serve: DeepSeek-V4.1 marks the system prompt and tools as the shared start (pin=N) so a sub-agent goes back to it
Only the deepseek41 family; STRATA_DS4_AUTO_PIN=0 switches it off, a client's own strata_prefix wins, Qwen is untouched.
2026-10-24 00:59:52 +02:00
Niko1221 cdc3a26577 dsv41: keep a conversation another request displaces and take it back at its checkpoints (STRATA_DS4_SLOT_KEEP), shared-start checkpoint (pin=N)
A prompt that does not continue the conversation in the model first sets that one aside (position, window rings,
compressor partial groups, each KV source's compressed KV and index keys, its checkpoints), then takes back the kept
conversation it does continue, in RAM or from the slot folder, also across restarts. The latest checkpoint the prompt
still agrees with is used (end of the last prompt, before it, the anchor of the first prompt of a run, and the shared
start the server marks with pin=N). The .meta format and the prefix matching are in ds4_slotmeta.hpp with a GPU-free
test (ds4_slot_test). STRATA_DS4_REUSE=0 reads every request from token 0 (reference for tests). Off unless
STRATA_DS4_SLOT_KEEP=1.
2026-10-24 00:59:52 +02:00
Niko1221 cdf8b98f54 Merge f-024-rel d914211e6 into f-024-hipk 2026-10-24 00:58:00 +02:00
Niko1221 5be4964c2b launch for DeepSeek: the tune text says what it really does and its result lists what was timed (or that nothing could be compared); the plan names an installed DeepSeek engine ('installed, 0.1.49') instead of saying setup will download it; preset test; tests 2026-10-24 00:56:08 +02:00
Niko1221 d914211e64 release notes 0.2.4: the manual DeepSeek engine command 2026-10-24 00:41:47 +02:00
Niko1221 71c674dcdb DeepSeek engine by hand: --dsv41-package FILE (setup and strata launch) or a file put in engine-dsv41/; the package is checked (DeepSeek engine for this system, signed manifest, every file's hash) before it is installed; plain steps in the missing-engine message and docs; tests 2026-10-24 00:41:28 +02:00
Niko1221 0d38517888 dsv41 prompts: expert copies keep going while the host waits for the routing (flow_pump on the busy stream); DS4_FLOW_PUMP_WAIT=0 as before 2026-10-24 00:40:48 +02:00
Niko1221 e51e84e22a dsv41 CPU lane: two cores left free beside the host thread when the lane would take every logical CPU; STRATA_DS4_LANE_FREE=0 as before 2026-10-24 00:40:42 +02:00
Niko1221 cffd240834 dsv41 serve: Stop ends a long prompt's reading at a layer/chunk and keeps the conversation before it (set_cancel / cancelled) 2026-10-24 00:40:09 +02:00
Niko1221 062fd4a95e release notes 0.2.4: the DeepSeek engine is a separate download (automatic and by hand), Qwen engines unchanged 2026-10-24 00:38:34 +02:00
Niko1221 f73ae5a62e dsv41 DSpark: cumulative cut on the drafts (product of confidences >= 0.5); STRATA_DS4_DSPARK_CUM, request key dspark_cum, 0 off 2026-10-24 00:38:21 +02:00
Niko1221 e6e14f2714 dsv41 CPU lane: prefetch only the IQ experts' pages not in RAM (os::prefetch_cold); DS4_IQ_PF_ALL=1 as before 2026-10-24 00:38:01 +02:00
Niko1221 a6cbc3621e dsv41 Windows: Engram rows read unbuffered from their unmapped file; DS4_EG_DIRECT=0 as before, DS4_EG_CHECK compares against a buffered read 2026-10-24 00:37:44 +02:00
Niko1221 aee5db5eb1 dsv41 Windows: 64 Engram reader threads (each ReadFile is synchronous, so the threads are the queue depth); DS4_ENGRAM_THREADS overrides 2026-10-24 00:37:14 +02:00
Niko1221 611c1701d3 dsv41 Windows: 1 ms timer resolution while the engine runs; DS4_TIMER_MS=0 leaves it 2026-10-24 00:37:04 +02:00
Niko1221 5ec845f6bc dsv41 Windows: queued GPU work submitted at once on NVIDIA too (win_submit), after every flag hand-off; DS4_WIN_SUBMIT=0 turns it off 2026-10-24 00:36:40 +02:00
Niko1221 a23b5f209c dsv41: gfx12 WMMA expert tiles on by default (STRATA_DS4_HIPK=0 turns them off) 2026-10-24 00:35:47 +02:00
Niko1221 94413c7057 dsv41: gfx12 WMMA prompt tiles for the i-quant expert sets (STRATA_DS4_HIPK=1, default off) 2026-10-24 00:35:38 +02:00
Niko1221 8fda835383 dsv41: DS4_PROF expert-kernel timers on both set paths 2026-10-24 00:35:37 +02:00
Niko1221 4278304914 dsv41: finer DS4_PROF for the chunked prompt (stages, expert kernels by type) 2026-10-24 00:35:37 +02:00
Niko1221 75c57be831 release notes 0.2.4: Thanks names IST-DAS and Unsloth only 2026-10-24 00:34:09 +02:00
Niko1221 2dbb263b02 launch/setup for DeepSeek: setup installs the screens packages (textual) and launch goes on in plain lines without them; the Tuning screen shows the DeepSeek tuner's real length and what it tries; the Advanced list knows strata-dsv41's own flags; the model menu counts the disk (download + preparation) in 'fits'; docs: tuning and the separate engine package; tests 2026-10-24 00:18:48 +02:00
Niko1221 955c709d96 Merge branch 'f-024-dsml' into f-024-rel 2026-10-24 00:12:55 +02:00
Niko1221 70efffb5b6 serve: a DeepSeek-V4.1 pack that ships the V4-Flash chat template (tool_calls, no space) is rendered with the V4.1 template, so tool calls come back as real calls; STRATA_DEBUG prints the raw ids; test 2026-10-24 00:07:13 +02:00
Niko1221 027691b33d release notes 0.2.4: RTX 3060 row 6-9 tokens/s 2026-10-24 00:06:34 +02:00
Niko1221 00ce92068a setup: banner names the chosen family; the user's own home (root's included) is allowed as a data folder; a missing DeepSeek engine package is said plainly (supported cards), the developer hint only with --verbose; tests 2026-10-24 00:05:11 +02:00
Niko1221 18022ceaf3 launch: the plan says 'drafting DSpark' for DeepSeek, not MTP; test 2026-10-23 23:48:59 +02:00
Niko1221 1904c0261b dsv41 tune: with the drafter on, also try a draft block of 3 against the pack's own 2026-10-23 23:47:48 +02:00
Niko1221 642e76948e setup: shared DeepSeek files (drafter, Engram) are also looked for two folders above the set's own folder (strata launch passes .../off/2.00bpw); test 2026-10-23 23:47:30 +02:00
Niko1221 1d0a8051d5 setup: DeepSeek's engine comes from the signed feed's dsv41 member (strata_control/dsv41_engine.py: same download, resume and verification as an engine update) into engine-dsv41/, beside the engine; the flat-zip route is dropped; tests 2026-10-23 23:46:46 +02:00
Niko1221 9f04da3afc Merge branch 'f-024-rel' into f-dsv41-tune 2026-10-23 23:37:40 +02:00
Niko1221 134f24f199 setup: DeepSeek's engine is its own package (engine-dsv41/ beside the engine, never inside it): downloaded with resume and a SHA-256 check from the --prebuilt base, found before the old all-in-one place; launch's note says so; tests 2026-10-23 23:37:19 +02:00
Niko1221 6655733d5e feed: a separate dsv41 member for the DeepSeek engine packages (Feed.dsv41, pack-engine --prefix, build-feed --dsv41); Qwen engine list untouched; tests 2026-10-23 23:35:00 +02:00
Niko1221 f9f297bf90 Merge branch 'f-dsv41-tune' into f-024-rel 2026-10-23 23:33:46 +02:00
Niko1221 ec5b2bdc4e dsv41 win: --dspark-block N (at most N drafts a round) 2026-10-23 23:33:40 +02:00
Niko1221 a2aeb4ecb1 launch: a real short tune for DeepSeek-V4.1-Flash (tools/dsv41/tune.py: the drafter on or off, CPU threads; gains over 5% only, written into the launch config; any failure leaves the standard settings with a plain line); the launch no longer forces --tune skip for it; tests 2026-10-23 23:31:51 +02:00
Niko1221 3feaa2aa3f Merge branch 'f-dsv41-setup' into f-dsv41-tune 2026-10-23 23:30:35 +02:00
Niko1221 8ea7fef827 Terminal 0.2.4 / Control and engine 0.1.49: version bump for the DeepSeek V4.1-Flash release (app stays 0.2.3, Mac stays 0.1.48); release notes 2026-10-23 23:29:00 +02:00
Niko1221 36a94857a1 dsv41_fit: remove a duplicated constants block; one unit (tokens a second) 2026-10-23 23:28:05 +02:00
Niko1221 28ce98bf16 Merge f-dsv41-rel (0be62ccad) into v0.2.3.1: DeepSeek V4.1-Flash on top of the released 0.2.3 line
f-dsv41-rel was based on the 0.2.2-era tree (merge-base 1f727f4b5), so the merge brings it onto 7ddc17a38 without dropping 0.2.3 / 0.2.3.1.
Conflict resolutions (0.2.3.1's behaviour kept everywhere that is not DeepSeek):
- packaging/package_engine.py: build() keeps 0.2.3.1's check_reader / allow_stale_vision / allow_no_reader and adds require_dsv41; both the
  picture-reader check and the strata-dsv41 presence check run; the CLI call passes both sets of arguments.
- packaging/ENGINE-COMPONENT.md: 0.2.3.1's cuda13 / cuda12 rows (vision built per variant), with strata-dsv41.exe added to the cuda13 contents.
- serve/server.py: the parser is self.parser_cls (OutputParser, or the DeepSeek DSML parser) and keeps 0.2.3.1's declared_only argument.
- strata_cli/launch/cmd.py: the --plan output prints 0.2.3.1's drafting line and then the DeepSeek placement lines.
- strata_cli/launch/runner.py: plan() returns 0.2.3.1's dict with "drafting", plus "placement" for multi-file (DeepSeek) models;
  drafting_plan_line() kept; dsv41_engine() and placement() added.
2026-10-23 23:27:29 +02:00
Niko1221 0be62ccad7 Merge f-dsv41-setup 86ba33845 into f-dsv41-rel (serve: DeepSeek turn without </think> returns its text as content; pack recipe noise token/markov rank; exec bits in packages; console-safe smoke) 2026-10-23 23:00:32 +02:00
Niko1221 86ba33845f serve: a DeepSeek turn that stops without </think> and wrote no answer returns its text as the answer (streaming ends with it as content; the non-streaming OpenAI and Anthropic replies drop the identical reasoning); tests 2026-10-23 23:00:05 +02:00
Niko1221 6f507fb210 packaging: strata-dsv41 and strata-vision get execute bits in the package and in the tar (ELF header decides when packed on Windows); setup restores them on an engine copied without modes 2026-10-23 22:49:29 +02:00
Niko1221 1a7b4ae236 setup: pack recipe states --dspark-noise-token 128799 and --dspark-markov-rank 256; plain wording (nothing to download, one unit: tokens a second, the one-time SHA-256 check and its time, the remembered-folder line) 2026-10-23 22:48:21 +02:00
Niko1221 a9dd554e13 setup: the smoke reply is made printable for the console code page before it is shown 2026-10-23 21:54:52 +02:00
Niko1221 3724818db5 Merge f-dsv41-setup into f-dsv41-rel (setup installs the official IST-DAS 2.00/2.25 sets; strata-dsv41 in the engine packages) 2026-10-23 21:54:42 +02:00
Niko1221 e9546cb564 Merge f-dsv41-cpul e5f536a6c into f-dsv41-rel (AVX-512 one-token rows for Zen 4/5, bit-identical to ggml, per-type start-up self-check, chosen by the detected CPU) 2026-10-23 21:10:53 +02:00
Niko1221 8de8d90287 dsv41 smoke: utf-8 output (the reply has DeepSeek's special tokens; a Windows console code page failed on them) 2026-10-23 21:09:46 +02:00
Niko1221 e5f536a6cb dsv41 cpu lane: AVX-512 rows from 1 token on Zen 4 as well (IQ2_XXS, IQ2_XS; IQ3_XXS and IQ1_M from 2), measured 2026-10-23 21:02:09 +02:00
Niko1221 b6cc7e681b setup: DeepSeek family line names IST-DAS for the official 2-bit GGUFs and Unsloth for UD-Q8_K_XL; says who made the files of the chosen version 2026-10-23 20:41:48 +02:00
Niko1221 f2323d876d setup test + docs: no --prefill-chunk, one plain line about the engine's own chunked prompt 2026-10-23 20:37:24 +02:00
Niko1221 10af5bb1e4 setup: drop --prefill-chunk (the chunked prompt is the engine's default now); doc line 2026-10-23 20:37:14 +02:00
Niko1221 237eb667d3 Merge branch 'f-dsv41-rel' into f-dsv41-setup 2026-10-23 20:37:12 +02:00
Niko1221 d6f18babe9 Merge f-dsv41-r97 fb321d62c into f-dsv41-rel (layer-major chunked prompt as the engine default) 2026-10-23 20:12:55 +02:00
Niko1221 1747c8e538 dsv41 cpu lane: one-token AVX-512 rows - sign masks by popcount (IQ2_XXS, IQ2_XS, IQ3_XXS), IQ1_M AVX-512 rows with pdep indices; chosen per CPU (Zen 5 from 1 token) 2026-10-23 20:09:59 +02:00
Niko1221 f1cb66d09e dsv41: strata-dsv41 --version and a 'ds4: build <commit>' line with --verbose (commit from git at configure, a BUILD_ID file in an archive, or -DSTRATA_BUILD_ID) 2026-10-23 20:03:24 +02:00
Niko1221 4eedb86b4f Merge f-dsv41-h100 (store pre-grow: register the whole arena before the first kernel on Linux, fixes the --ssd-cache zero-copy hang) 2026-10-23 20:01:41 +02:00
Niko1221 e917cf5b37 Merge branch 'f-dsv41-rel' into f-dsv41-setup 2026-10-23 20:01:11 +02:00
Niko1221 c506238ce2 setup: DeepSeek official sets start with --dspark-conf 0.65 (H100 measured default) 2026-10-23 20:00:50 +02:00
Niko1221 c35710e388 dsv41: store arena pre-grown by default on Linux when the GPU maps it 2026-10-23 19:59:29 +02:00
Niko1221 fb321d62cb dsv41: the layer-major (chunked) prompt is the default on every run: chunks of 1024 for the single-card default run, staging ring 16 MB where the cards hold most of the experts 2026-10-23 19:59:13 +02:00
Niko1221 f7d7072841 dsv41: DS4_STORE_PREGROW=1 (map the whole store arena before the first kernel; cuMemHostRegister mid-decode hangs on Linux) 2026-10-23 19:58:22 +02:00
Niko1221 1c1005952a Merge f-dsv41-cuk 86bdfbd0d (HIP keeps the release grids and kernels). One conflict: the identical IQ1_M code-table HIP guard (9c46ab821 vs cuk); took cuk's wording. 2026-10-23 19:52:52 +02:00
Niko1221 41856f6594 dsv41: staging ring 64 MB when the experts are within 30% of fitting the cards (R9700 x4 plain 2.00: ratio 1.16) 2026-10-23 19:45:21 +02:00
Niko1221 0beae0bca6 dsv41: the store fill after a chunked prompt reaches the store's whole capacity, not only its allocated slots 2026-10-23 19:44:54 +02:00
Niko1221 9469412db2 setup: DeepSeek official sets start with --prefill-chunk 1024 (layer-major prompt path, measured 231 tok/s at 8K on a 12 GB PC) 2026-10-23 19:44:40 +02:00
Niko1221 a345d9f8cf Merge branch 'f-dsv41-rel' into f-dsv41-setup 2026-10-23 19:44:39 +02:00
Niko1221 a82477069b Merge branch 'f-dsv41-pkg' into f-dsv41-setup 2026-10-23 19:41:53 +02:00
Niko1221 81dd215de9 setup: the two official IST-DAS DeepSeek-V4.1-Flash sets (2.00 / 2.25 bpw) as sizes: pinned files, shared Engram + DSpark, plain pack recipe, shipped profiles, pack-info and a 16-token text check, --dspark auto; fit words from the measured numbers 2026-10-23 19:41:49 +02:00
Niko1221 4b5f8d89f2 tests + ENGINE-COMPONENT.md: strata-dsv41 in the engine package and the --require-dsv41 refusal 2026-10-23 19:39:24 +02:00
Niko1221 dc352b839b dsv41: say how the chunk staging ring was sized 2026-10-23 19:36:53 +02:00
Niko1221 e838e2beca Merge f-dsv41-cpul 02b07b686 into f-dsv41-rel (exact multi-row CPU-lane rows for IQ types, bit-identical to ggml-cpu, self-check per type, DS4_GG_IQ256=0 switch) 2026-10-23 19:35:58 +02:00
Niko1221 6053a43841 packaging: build_dsv41_windows.bat and build_dsv41_cuda.sh build strata-dsv41 for the NVIDIA release packages (sm_86 + sm_120 fat) 2026-10-23 19:35:51 +02:00
Niko1221 faaf82acd7 build_hip_linux.sh: build strata-dsv41 (own build tree) and ship it in the rx9070 package; DSV41=1|0 overrides, --require-dsv41 on the package 2026-10-23 19:35:51 +02:00
Niko1221 8fee63f4f0 package_engine: ship strata-dsv41 beside the engine (picked up like the picture readers); --require-dsv41 refuses a package or a --verify without it 2026-10-23 19:34:59 +02:00
Niko1221 7519b1c75c pack_gguf: warn loudly when --skip-tensors / --dense-override are used on an official IST-DAS set (engram_k/q/wkv present): the rebuilt pack loads but gives garbage text; the plain recipe is right 2026-10-23 19:27:17 +02:00
Niko1221 dbf3bb2e64 dsv41: the layer-major run's RAM is given back before the cache catch-up and the store fill 2026-10-23 19:24:01 +02:00
Niko1221 86bdfbd0da dsv41 iq: HIP keeps the release grids and kernels (no persistent work-item grid there) 2026-10-23 19:23:13 +02:00
Niko1221 796dbb77b4 dsv41 iq: the IQ1_M code-table copy is CUDA only (kIq1mCodes does not exist on HIP; found by building the HIP-guarded paths under nvcc with -DSTRATA_USE_HIP) 2026-10-23 19:17:14 +02:00
Niko1221 96a47b79d5 dsv41: after a chunked prompt the RAM store is filled with the prompt's hottest CPU-lane experts (a window prompt fills it through the CPU lane; the chunk path streams past it) 2026-10-23 19:14:44 +02:00
Niko1221 9c46ab821a dsv41 iq: kIq1mCodes is CUDA-only, so its use in load_tab_fast is under #if !STRATA_USE_HIP (HIP failed 'undeclared identifier kIq1mCodes': non-dependent names are looked up in the dead constexpr branch) 2026-10-23 19:13:07 +02:00
Niko1221 12595032e7 Merge f-dsv41-cuk a3d955201 into f-dsv41-rel (persistent grids only for codebooks >= 8 KiB: fixes 2.5x slowdown with 1 active expert) 2026-10-23 19:06:12 +02:00
Niko1221 a3d9552015 dsv41 iq test: golden compare also covers dynamic-count window groups (5/1/3 entries, ne_max 5; 9 groups with 4 live) 2026-10-23 19:05:14 +02:00
Niko1221 01220edb67 dsv41 iq test: DS4_BENCH_WIN (window path / DSpark verify bench: n entries a group) 2026-10-23 19:01:30 +02:00
Niko1221 02b07b686a dsv41 cpu lane: AVX-512 rows chosen by the detected CPU (Zen 5 family 0x1A: IQ2_XXS and IQ3_XXS; Zen 4 0x19: IQ2_XXS; others none), prompt chunks of 7+ tokens only 2026-10-23 18:51:19 +02:00
Niko1221 a70b006929 dsv41 iq (CUDA): persistent work-item grids only for codebooks of 8 KiB+ (IQ1_M/IQ1_S/IQ2_S), lazy codebook load, small-codebook types keep the release grids and kernels (no regression with few active experts) 2026-10-23 18:49:14 +02:00
Niko1221 4cca3fac20 dsv41: --spec-eval honours --dspark off 2026-10-23 18:44:21 +02:00
Niko1221 995ff61f3c dsv41 cpu lane: AVX-512 rows by default for IQ2_XXS prompt chunks (7+ tokens), measured on Zen 4 2026-10-23 18:43:28 +02:00
Niko1221 f5fc88c599 dsv41 iq (CUDA): token and light kernels take work items (group, row tile) round a persistent grid (any number of active groups shares all blocks), IQ1_M codes from a generated table (ds4_iq1m_codes.inc), persistence only for codebooks of 8 KiB+; many-row light sets keep the release kernel; test: --bench-tiles, DS4_BENCH_ACTIVE 2026-10-23 18:41:55 +02:00
Niko1221 428950b96c Merge catch-up: f-dsv41-win fd19f9d6b, f-dsv41-h100 c0f944e12, f-dsv41-cuk 2026-10-23 18:35:43 +02:00
Niko1221 0553ba6c3f dsv41 iq (CUDA): light sets (DSpark verify, short prompts) take the persistent grid and the IQ1_M code table for experts of <= 4 rows, 4-block occupancy kept for 8+ rows; test: --bench-light 2026-10-23 18:16:11 +02:00
Niko1221 fd19f9d6bf dsv41 win: store off the lookahead by default 2026-10-23 18:09:27 +02:00
Niko1221 b43b0bb518 dsv41: kEntMax declared before Ent 2026-10-23 18:06:26 +02:00
Niko1221 d2411af9fd dsv41: i-quant CPU-lane entries take a whole window's tokens of an expert (cap 16, was 8) for the exact multi-token rows; the chunk staging ring is 64 MB where the cards hold 90% of the experts (decode after a chunked prompt back to the windowed path's speed) 2026-10-23 18:04:51 +02:00
Niko1221 49e35e8bb7 dsv41 cpu lane: exact AVX-512 rows (iq512x_rows, 1-6 tokens and prompt tiles), opt-in DS4_GG_IQ512=1 until measured on Zen 4 2026-10-23 18:04:12 +02:00
Niko1221 c25724a222 Merge branch 'f-dsv41-cpul' into f-dsv41-r97 2026-10-23 18:01:40 +02:00
Niko1221 d689e6080d Merge f-dsv41-cuk into f-dsv41-rel (per-device light-set scratch and tensor-core check, faster IQ1_M/IQ2_S/IQ3_XXS token kernels) 2026-10-23 17:59:37 +02:00
Niko1221 cf4f8befe7 dsv41 cpu lane: take the AVX2 rows by ggml-cpu's own AVX2/FMA/F16C answer (MSVC builds have no __builtin_cpu_supports) 2026-10-23 17:59:03 +02:00
Niko1221 1cace5c864 dsv41 iq (CUDA): per-device light-set scratch and tensor-core check (stream 0 is one handle on every device: the scratch of the first GPU was reused on the second), block/sub-block loop form in the token kernels; test: --devices (in-process multi-GPU, bit-compared) 2026-10-23 17:58:59 +02:00
Niko1221 7e9b0dcc25 dsv41 cpu lane: exact AVX2 rows for 7..64+ tokens an expert (the prompt path): decode once, u8 activations, no per-token sign; IQ1_M too
T tokens in tiles of 16 and rows in groups of 8 (activation slice and accumulators in L1): 7.7-9 ns per block and token on IQ2/IQ3 rows against
28-60 for ggml's dot (x3.6-6.6), IQ1_M 16 ns (x1.8); bit-identical to ggml (ds4_gg_bench --exact at 7, 8, 16, 33, 64 tokens, all six types).
gg_cpu_gate_up takes up to 64 tokens a call (and any number in pieces); ds4_gg_bench --expert times the whole expert pipeline.
2026-10-23 17:57:53 +02:00
Niko1221 0b48e23fa5 dsv41 win: --dspark auto (not loaded when the experts do not fit the GPU), --ids @file, h100sim 2026-10-23 17:57:34 +02:00
Niko1221 f6e4cd6899 dsv41: chunked prompt for a single card (--prefill-chunk N), the expert cache learns from chunks and catches up after a prompt, chunk buffers fall back by halving, CLI feeds a whole prompt
The layer-major chunked prompt path only ran with --threads on / --tp > 1. --prefill-chunk N (N > 0) now enables it on one card too (kChunkedPromptDefault stays false until the decode gate is signed off). A chunk's routing bumps the layers' heat and CPU-lane hits, and cache_settle() runs the swap rule in rounds after the prompt so the decode that follows finds the hot experts in VRAM (8K prompt: 1699 swaps, 5.6 s). If the chunk buffers do not fit, the chunk is halved down to 256 rows, then windows of 16 with one line saying so. ds4_main: --window 0 (default) feeds a whole prompt when the chunk path is on.
2026-10-23 17:47:43 +02:00
Niko1221 c0f944e127 dsv41: DS4_PRED_DUMP / DS4_XIN_DUMP (router logits + projected router input; router inputs for the expert-similarity scan) 2026-10-23 17:44:34 +02:00
Niko1221 6aba354c16 dsv41 cpu lane: the exact AVX2 rows on by default from 2 tokens (1 on IQ3_S, 3 on IQ1_M), 5-6 tokens in one pass, IQ1_M rows rewritten
The rows are ggml-cpu's vec_dot bit for bit at any token count, so the default cannot change an answer; a one-time per-type check against the
ggml-cpu build in use turns them off if they ever differ (DS4_GG_IQ256=0 never, =1 always).  EX rows: 5 and 6 tokens in one decode pass
(were 4+rest), independent integer chains for 1-2 tokens, IQ1_M: delta dot from per-call pair sums of the activation, group signs from one
shifted load.  The main engine's iq256_rows hash is unchlanged (phaseA-tests/cpul/hash_iq256.cpp).
2026-10-23 17:42:52 +02:00
Niko1221 71abb6c163 dsv41 iq (CUDA): persistent token kernels (codebook loaded once a block) and IQ1_M decode through a 4096-entry code table (delta sign folded in): bit-identical, IQ1_M layer -15% sm_120 / -30% sm_86 2026-10-23 17:39:02 +02:00
Niko1221 cf45533cf6 dsv41 cpu lane: AVX2 multi-token i-quant rows in ggml's float order (bit-identical to ggml-cpu at 1..8 tokens), IQ1_M rows
The main engine's iq256_* rows keep their float order (EX=false, the default; hash of their outputs on all types and 1..8 tokens is
identical before/after, phaseA-tests/cpul/hash_iq256.cpp).  The new iq256x_rows (EX=true) sums as ggml-cpu does - block scale f16(d)*y.d, the
format constant after the sum, ggml's hsum - so a token's dot no longer depends on how many tokens share the pass (the greedy ids of a
DSpark run changed with DS4_GG_IQ256=1 because the old order differed in the last bits).  IQ1_M (29), single-token in ggml, gets a
multi-token AVX2 row.  ds4_gg_bench --exact: 0 differing bits for IQ1_M, IQ2_XXS/XS/S, IQ3_XXS/S at 1..8 tokens.
2026-10-23 17:28:19 +02:00
Niko1221 95994c45c5 dsv41 iq test: --bench-big (>=640 MiB working set, rotated experts: DRAM rates), --dump/--cmp (bit-exact golden compare incl. real shapes and grouped path) 2026-10-23 17:18:21 +02:00
Niko1221 ea081e3312 dsv41 rel: one IQ1_M implementation in ds4_iq.cu (the textual merge kept both; port's kernels stay), win's dequant_rows (chunked-prompt dense dequant) kept and takes IQ1_S too; strix add4 dropped with win's IQ1_M decode (port's decode has no __vadd4) 2026-10-23 17:06:08 +02:00
Niko1221 8cfe988383 Merge f-dsv41-strix into f-dsv41-rel (HIP build fixes: portable add4 for the IQ1_M decode, strata_core link for the DXGI budget, HIP build scripts, umaprobe)
No conflicts. The strix add4 edit applies to the merged IQ1_M decode; no __vadd4 is left in ds4_iq.cu.
2026-10-23 17:02:44 +02:00
Niko1221 e5c70bbaa0 Merge f-dsv41-h100 into f-dsv41-rel (--profile-run, DS4_TRACE on the chunked path, --spec-eval) 2026-10-23 17:02:30 +02:00
Niko1221 c70ae83d4c Merge f-dsv41-port into f-dsv41-rel (release integration)
Conflict decisions (win side = f-dsv41-win, port side = f-dsv41-port):
- ds4_iq.cu: kept port. Its IQ1_S/IQ1_M/IQ4_NL kernels are verified bit-exact on HIP gfx1201, sm_86 and sm_120; win's
  IQ1_M table load and supported() list are a subset (IQ1_S, IQ4_NL added).
- ds4_kernels.cu/.cuh: kept port's engram_dequant fmt int (0 MXFP8, 1 Q6_K, 2 IQ4_NL) and its iq4nl_value, replacing
  win's q6k flag and iq4nl_value; same arithmetic.
- ds4_model.cu: kept port's eg_fmt vector and its Engram type check (unknown type exits 2 instead of segfaulting);
  win's eg_q6k is gone, the duplicate IQ4_NL row branch from the textual merge is removed.
- ds4_packinfo.cpp: kept port (device types derived from iq::supported()).
- iq_expert_test.cpp: kept port (hip_or_cuda.h); win's tests/dsv41/hip_as_cuda.h removed (nothing else used it).
  The IQ1_M test-d placement is the same bytes in both versions.
- CMakeLists.txt: kept port's test-language block (HIP, CUDA or CXX for the CPU-only build); win's
  _strata_dsv41_test_lang variable dropped.
- pack_gguf.py: kept port's type table (IQ1_S/IQ1_M comment) and combined both missing-Engram checks
  (has_part from win's split Engram layout AND port's dense-override names).
2026-10-23 17:02:22 +02:00
Niko1221 f232cbfed6 dsv41 win: store test covers cancel and shrink; shrink finalizes read slots 2026-10-23 16:59:01 +02:00
Niko1221 08224f228f dsv41 win: priority SSD queue with cancellation (demand before guesses), partial GPU visibility, store shrink on RAM pressure, predsim 2026-10-23 16:53:02 +02:00
Niko1221 c8ead78419 dsv41 win: store init last, zc off by default, flow reads the store, release notes 2026-10-23 16:37:01 +02:00
Niko1221 55e1a53cfd dsv41: --spec-eval and DS4_ROUND_LOG (DSpark acceptance per round, every draft verified) 2026-10-23 16:32:59 +02:00
Niko1221 8da42765dd dsv41: DS4_TRACE also records the chunked prompt path's picks 2026-10-23 16:29:22 +02:00
Niko1221 75c9f2d81e dsv41: profile-run accepted without --ids 2026-10-23 16:25:51 +02:00
Niko1221 2e61fef4d1 dsv41 win: store auto size, profile preload, decode staging zc_tab (T=1), release plumbing 2026-10-23 16:21:59 +02:00
Niko1221 fb141be3e0 dsv41: profile-run string fix 2026-10-23 16:21:19 +02:00
Niko1221 9051f31306 dsv41: --profile-run mode (feed id segments without logits, expert profile saved as it goes) 2026-10-23 16:20:57 +02:00
Niko1221 17bc654b6d dsv41 strix: umaprobe (UMA budget, hipMalloc reach, host registration, zero-copy read bandwidth), build script 2026-10-23 16:10:12 +02:00
Niko1221 084e654bcb dsv41 on Windows HIP (gfx1151): build script for ROCm 10.2 + MSVC, Linux HIP build script, portable per-byte add for the IQ1_M decode (no __vadd4 on HIP), strata_core link for the DXGI-budgeted cudaMemGetInfo 2026-10-23 15:38:17 +02:00
Niko1221 7d2307deb0 dsv41 win: the store's slots in blob layout, GPU-visible (zero-copy mapped) with a PCIe staging copy for decode picks, node-free reserved arena 2026-10-23 15:33:54 +02:00
Niko1221 226d73b45b tools/dsv41/engram_fold.py --gate-kv-from: make the ple table the only variable, and the answer it gave
Deriving their engram.gate from their k*q and transposing their wkv scored 0.409 then 2.996 KL - both my arithmetic, not
their data (their k/q/wkv are stored transposed and the gate is not k*q in the engine's layout).  --gate-kv-from <pack>
copies those two tensors from a pack that is known good, so a pack can carry their ple table and nothing else of theirs
differ.  That is how the comparison the plan asks for was finally made (numbers in EACH-PC-PLAN/R9700-BOX.md, step 2):
their 110.6 GB IQ4_NL table scores KL 0.16787 / ppl 3.627 against 0.16502 / 3.588 for our 203 GB MXFP8 one - +0.003 KL,
inside this engine's run-to-run placement noise - while saving 92 GB of disk.

Also fixes the fold's reads: Gguf's tensor offsets are relative to the shard's data section (pack_gguf adds data_start),
so the fold must add it too; without that every tensor came out as NaN.
2026-10-23 15:10:00 +02:00
Niko1221 52543768c7 dsv41: one runtime header for the tests on both backends (HIP or CUDA), so a CUDA box needs no shim on -I
tests/dsv41/{iq,tq}_expert_test.cpp included <hip/hip_runtime.h> unconditionally, which no CUDA toolkit ships - every CUDA
box had to pass a hand-written shim on -I (found on l620 and z440, 2026-10-23, and reported as the one thing worth
upstreaming).  include/strata/dsv41/hip_or_cuda.h now provides the handful of HIP names those tests use on a HIP build and
maps them onto the CUDA runtime otherwise (hipMalloc/Free/Memcpy, device sync, the error type and the events).

Verified on r9700 that the HIP build is untouched: incremental build rc=0, ds4_iq_test --type 29 ALL OK, ds4_tq_test ALL OK.
2026-10-23 15:00:19 +02:00
Niko1221 3826885f4b tools/dsv41/engram_fold.py: transpose their Engram layout, but the fold's reader is still wrong
Their engram_k/q/wkv are transposed against the engine's layout (wkv [6144, 25600] against [25600, 6144], k/q [5120, 4]
against a [4, 5120] gate), so the fold now transposes and writes the engine's shape.  The pack builds (their IQ4_NL table,
all files present) and runs without crashing, but it scores KL 0.40915 - and that is this tool's bug, not their data: a
direct read of their BF16 engram_k through Gguf's offsets already comes out as NaN, while the engine reads the same tensors
through a pack fine, so the fault is in how this code turns an offset into bytes (the Q5_K dequant inherits it).

Do not quote 0.409 as the IQ4_NL-vs-MXFP8 comparison; that measurement is still open.  Next step is to read the tensors the
way pack_gguf writes them into tensors.txt and check one row against the engine's own dump.
2026-10-23 14:53:02 +02:00
Niko1221 e7b3f15240 dsv41 win: RAM expert store (unbuffered reads, policy, lookahead fill), Windows direct reads, dense dequant for the chunked prompt, SMT option, store test 2026-10-23 14:44:51 +02:00
Niko1221 7ddc17a38f tests: the old-launcher test expects the 0.2.3.1 version line 2026-10-23 14:37:21 +02:00
Niko1221 d62ff4a608 tools/dsv41: engram_fold.py (their k*q gate and wkv from their own shards) and pack_gguf accepts an override for the Engram tensors
Their sets keep the Engram as three per-layer tensors (blk.N.engram_k/q/wkv) while the engine wants engram.gate (k*q
folded) and engram.wkv; their engram-IQ4_NL.gguf carries only the ple table, so a pack built around their Engram stopped at
'blk.1.engram_kv.weight: missing'.  engram_fold.py folds the gate and renames the kv into a dense_requant-style folder, and
pack_gguf now treats a tensor a --dense-override supplies as present (it checked the quant's shards and the Engram set only).

Result: packs/ista_2.00_native - 48784 tensors, their IQ4_NL table, Engram layers 1,14, 2896 Engram reads for a 32-token
prompt, no crash.

NOT yet a measure of their table: it scored KL 0.40915 / ppl 4.769 against 0.16502 / 3.588 for the same set with our MXFP8
table, because their engram_k/q/wkv are stored TRANSPOSED relative to the engine's layout (their engram_wkv is [6144, 25600]
against [25600, 6144]; their k/q [5120, 4] against the [4, 5120] gate).  Copying the bytes as they are yields a valid pack
with the wrong data.  engram_fold.py still needs the transpose (gate in float; the Q5_K kv dequantised, transposed, written
BF16) - Phase 1.1's 'load their layout as published'.
2026-10-23 14:36:12 +02:00
Niko1221 488dd8459c dsv41: --pack-info's device-type list is derived from iq::supported() instead of copied
It was a hand-kept copy of the supported() switch and fell behind when IQ1_S, IQ1_M and IQ4_NL got kernels: the 2.00 set's
60 IQ1_M expert tensors (59.5 GB) were reported as 'no GPU kernel ... the CPU lane takes them' while the kernels were in the
same binary and ds4_iq_test passed them.  That is exactly the line an operator trusts before a download, so it now walks
ids 0..63, asks iq::name()/iq::supported(), and adds MXFP4 and the trellis types by hand (they are not in that switch).
Verified: packs/ista_2.00bpw_9e2528f and packs/h225 both report 'no GPU kernel: none'.  Also fixes the intake's eval call
(session16.sh takes the pack directory name, not a path - the 2.00 score had failed with an empty runs/s16_<path>.tsv).
2026-10-23 14:21:38 +02:00
Niko1221 b2d2cf22eb dsv41: the Engram reader takes IST-DAS's IQ4_NL table, and a pack with an unknown Engram type stops cleanly instead of segfaulting
Found and fixed on p620, the only box with their engram-IQ4_NL.gguf (110.6 GB). Before this, a pack built around their
file loaded for 38 s and then died in Model::Impl::eg_issue at the first token (3 of 3 runs, addr2line 0x6d3450).

- Model::Impl::load: one guard line and exit(2) for an Engram tensor whose type the reader does not handle, naming the
  layer and the type (verified with a pack retyped to IQ4_XS, rc 2; a missing embed tensor also exits 2). The guard stays
  for the next unknown type.
- eg_q6k becomes eg_fmt per layer: 0 MXFP8 ([scale, 32 codes] blocks, 264 B a row), 1 ggml Q6_K (210 B a 256), 2 ggml
  IQ4_NL (18 B a 32: an fp16 d and 16 nibble bytes), 3 an old pack's split FP8. The load loop sniffs the type from the
  pack's tensor table, eg_issue builds the row descriptor from it (144 B a row, off + id*144, no scale tensor), and
  EgReader::read() needed no new branch: its contiguous, scale-free path already served Q6_K.
- ds4_kernels.cu: engram_dequant_kernel and engram_dequant_tok_kernel take the format and add the IQ4_NL branch, a literal
  port of ggml's dequantize_row_iq4_nl (fp16 d per 18-byte block, low nibbles 0-15, high 16-31, kvalues_iq4nl).
- The Engram row slot stays cols*(hd+hd/32) for every format, so existing packs are byte-identical: --pack-info on the
  MXFP8 h225 pack and on a Q6_K zero pack is unchanged, and a Q6_K pack's 32-token ids are identical old against new.

Verified with their real table: the pack that segfaulted runs 32 tokens in 1692-1711 ms (18.1-18.3 tok/s) and produces
coherent Python (GGML_TYPE_Q2_0 = 42, GGML_TYPE_NAMES: dict[int, str]); 64 tokens on the code prompt, DSpark off 17.16
and on 23.36 tok/s (44 of 53 drafts accepted). IQ4_NL decode is bit-exact against ggml on 8 of 8 sampled rows, and the row
addresses match the pack's tensor table (data_start 320, 144 B a row).

Not covered: no non-zero MXFP8 table existed on the box to re-check that path in-engine (the Q6_K and synthetic-zero MXFP8
packs were), there is no in-engine float dump against the Python reference (the proof is formula- and address-level), and
the legacy split-FP8 format (3) is kept but untested.
2026-10-23 14:13:26 +02:00
Niko1221 7b3e2eaa0b Control 0.1.48.1 / Terminal 0.2.3.1: version bump for the Control-only hotfix of #3810 (app stays 0.2.3, engine packages stay 0.1.48); release notes 2026-10-23 14:10:33 +02:00
Niko1221 f5f205bee0 setup: the HIP ready-made-engine checks read gfx names with arch_names, not arch_numbers (an installed AMD engine was never recognised); tests for archs as a list and as a string 2026-10-23 14:07:48 +02:00
Niko1221 0edd96ccf6 setup: BUILD.json archs / vision_archs are read as a list or as the ';'-separated string the packaging writes (setup stopped on the released Windows CUDA package: int(';'), #3810) 2026-10-23 14:05:32 +02:00
Niko1221 eb6c0a23f0 dsv41 win: pack_gguf reads the IST-DAS Engram parameters from tensors, IQ1_M type, packinfo 64-bit sizes, DS4_TRACE_AHEAD, cachesim 2026-10-23 13:17:02 +02:00
Niko1221 c523890228 dsv41: a CPU-only build path, and the intake's pack recipe fixed
1. CMakeLists: -DSTRATA_BUILD_DSV41=ON was silently dropped when no GPU backend was on, because the whole DSV41 block
   sits inside if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP).  Found on p1004 (2026-10-23), whose P100 is not a DeepSeek
   target but which is exactly the box the CPU-lane A/B belongs on.  A new branch builds ds4_gg_bench with no GPU
   runtime at all.  Verified on r9700 with both backends off: configure rc=0, build rc=0, the benchmark runs IQ3_XXS at
   60.5 GB/s (ggml-cpu) against 64.1 (AVX2 rows, x1.06) and 8.02e-09 against a double reference.

2. tools/dsv41/ista_intake.py: the pack step now uses the recipe that actually builds a loadable pack for these sets -
   --engram-meta (the UD set's first shard carries the Engram hash parameters, without it pack_gguf sees no Engram layers
   and refuses per_layer_token_embd), --mmproj, the combined --skip-tensors regex, and the DSpark noise token and markov
   rank (pack_gguf fills the rest of the geometry from the drafter but not those two, so a pack came out with
   dspark_noise_token_id 0).  Verified end to end on the 2.25 set: the intake rebuilds packs/h225's content exactly -
   same 48794 tensor lines, --pack-info 'files: 10, all present', 'no GPU kernel: none', dspark geometry 5/128799/256/128/3.
2026-10-23 12:56:24 +02:00
Niko1221 5c523d55f7 dsv41: device kernels for IQ1_S, IQ4_NL and IQ1_M (the three IST-DAS types that had none)
Found and implemented on z440's CUDA bring-up, applied here and re-verified on HIP.

ds4_iq.cu: decode<T>/deq8<T> for kIQ1_S, kIQ4_NL, kIQ1_M, plus iq1_codes() (iq1s_grid_gpu's 4-bit nibbles -> the int8 codes
8*(n-1)+-1 through two __vsub4) and iq1m_d() (IQ1_M carries its F16 d in the top nibbles of its four u16 scales - there is no d
field).  kQK<T> is 32 for IQ4_NL as well as MXFP4; load_tab uses iq1s_grid_gpu / kvalues_iq4nl; DS4_IQ_SWITCH and
supported() cover the three types, so they leave the CPU lane and join the device path.

tests/dsv41/iq_expert_test.cpp: make_mat now writes IQ1_M's scale where the format keeps it (bytes 49/51/53/55, high nibbles).
Without that the derived d is random and roughly 1/32 of the blocks come out NaN, which std::max then hides.

Parity (z440, one A4000, ds4_iq_test --type 19,20,29): token gate_up 6.70e-08 / 2.04e-08 / 5.97e-08, token down 3.62e-08 /
4.71e-08 / 3.71e-08 against ggml's to_float and a double reference; CPU lane 3.4e-8..8.2e-8; F16 prompt tiles inside their
own tolerances.  Bandwidth: IQ4_NL 376 GB/s (84% of the A4000's 448, bandwidth-bound), IQ1_S 214, IQ1_M 202 (both
compute-bound at 0.92-1.10e12 weight-values/s against IQ2_XXS's 1.22e12).

Not covered: no end-to-end run with a pack that uses these expert types (none on z440); a combined-seed run put IQ1_M's
light-path gate_up at 1.10e-2 against its 1e-2 tolerance, which is the int8 activation rounding of that path (it does not
move with tensor cores off), not the dequant.
2026-10-23 12:34:16 +02:00
Niko1221 45dc5e3400 dsv41: the CUDA build of the tree needs two portable edits (found on z440's sm_86 bring-up)
1. ds4_tq.cu:262 used the two-argument __shfl_xor, which is HIP-only; the rest of the tree already uses
   __shfl_xor_sync(0xffffffffu, v, lane), which both backends have.
2. tests/dsv41/{tq,iq}_expert_test.cpp were forced to LANGUAGE HIP, so a CUDA configure failed at generate time
   ('The language HIP was requested for compilation but was not enabled'). The language now follows the enabled
   backend: HIP, CUDA, else CXX.

Verified on r9700 that the HIP build is untouched: incremental rebuild rc=0, ds4_iq_test --type 17 ALL OK, the engine
--pack-info clean. Verified on z440 (CUDA 12.9, sm_86) by the bring-up agent: HEAD builds, ds4_iq_test ALL OK for
IQ2_XXS/XXS/IQ2_XS/IQ2_S/IQ3_XXS/IQ3_S/IQ4_XS at 5e-8 against ggml, 272-369 GB/s on one A4000. IQ1_M, IQ1_S and
IQ4_NL still have no CUDA kernel (CPU lane only) - that is the Phase 5 kernel list.
2026-10-23 12:08:49 +02:00
Niko1221 c42f4f2969 dsv41 on Windows+CUDA: IQ1_M device kernels (bit-exact vs ggml), IQ4_NL Engram rows, CUDA build fixes (strtok_r, shfl_xor, test language), IST-DAS pack names, hardware bench 2026-10-23 11:57:51 +02:00
Niko1221 5c8dd32252 serve: max_tokens -1 and 0 mean no cap again, as in 0.2.2 (clients send -1 for that); documented 2026-10-23 07:57:09 +02:00
Niko1221 5964d113f8 release notes 0.2.3: hotfix heading, status/logs names, launch not setup, drop the unreleased-pack limit 2026-10-23 06:37:32 +02:00
Niko1221 fea2d02aaa docs: Strata 0.2.3 release notes 2026-10-23 06:36:54 +02:00
Niko1221 81a7260194 launch config never keeps --mtp beside "mtp": false; one line says where the pack and draft layer are read from (B10, C08 note) 2026-10-23 03:28:36 +02:00
Niko1221 3b1ab2848d Save and launch saves the preset only after the restart question is answered Restart; the question names the running model and port 2026-10-23 03:27:48 +02:00
Niko1221 80a70b86de Folder step: models Strata already keeps stay where they are (the text said they were moved) 2026-10-23 03:24:46 +02:00
Niko1221 9222b6a1d3 a failed start ends its own runtime host, checks the port first and quotes only this attempt's logs (X01) 2026-10-23 03:23:58 +02:00
Niko1221 67a48427b8 Merge f-023-int into f-023-tui2 2026-10-23 00:27:00 +02:00
Niko1221 d64ae1b0b1 tests: the smoke test reads the engine's own words from .reason, the message is plain (X01) 2026-10-23 00:26:59 +02:00
Niko1221 27c0cabd81 Merge f-023-tui1: TUI stress fixes (second launch reuse-or-ask with --restart, context change on the same port, status/logs find the launch, key kept, retune stops its own model, folder checks, Ctrl+C, --model-path wins) 2026-10-23 00:13:45 +02:00
Niko1221 2f5f7c1135 Failed engine test start gives a plain cause and next step; rollback never says the same version twice; max_tokens below 1 is a 400 (TUI stress X01, #18, #15)
X01: TestStartFailed has one text builder (TestStartFailed.plain): what did not work in plain words, that nothing was installed and the current engine is unchanged, and one next step (`strata update --force`, or a plain retry when the card was busy). The engine's own words stay in .reason, the state file and the JSON. The app's message matcher also accepts the new wording. Hints that a caller passes, such as the one for a busy launch, are used as they are, so the launch-holds-the-card line of f-023-tui1 composes with it.
#18: `components rollback` when an earlier rollback is still in force says the engine is already the previous one and what to run, instead of "0.1.45.3 -> 0.1.45.3".
#15 (K05): max_tokens, max_completion_tokens and max_output_tokens below 1 are a 400 naming the field (they used to mean "no cap"); an unknown model id is still answered by the loaded model, now documented in DETAILS.md.
2026-10-23 00:06:13 +02:00
Niko1221 3bf804c9c8 tests follow the folder check and the compact save summary 2026-10-23 00:05:00 +02:00
Niko1221 0af0f9f125 Installer: an older feed shows its real reason; a folder with accented letters unpacks (TUI stress #11 A01b/A05, #12 A04)
#11: the refusal of a feed older than the one already accepted now says both sequence numbers, the folder that holds the newer one (.updates) and one next step (use the newest release folder, or delete state.json to take the older one). install.ps1 and install.sh print the whole check output, not its first 6 lines, and no longer blame a missing signature when the check gave a reason.
#12: tar.exe got the path through the ANSI/OEM code page and failed on accented letters. The installer now runs Windows' own bsdtar (System32 first, so a GNU tar from Git for Windows cannot be picked up for a zip) with ASCII relative arguments from inside the data folder and renames the result, so no non-ASCII path ever crosses that boundary. Tested on C:\...\<U-umlaut>n<i-umlaut> test with a stub release folder; the old script fails there with "could not chdir".
2026-10-22 23:54:57 +02:00
Niko1221 ea67f6999a A missing shard is named with its file name in the CLI, the TUI and the app's plain issues (TUI stress #9, C05, #3478)
plan_import now returns missing_shards (the file names of the split set that are not in the folder). problem_words and plain_issues say "<name>-00002-of-00002.gguf is missing" instead of "some parts of the model are missing"; more than two are counted. The three call sites (models import, strata launch --model-path, the Model screen's own-files field) pass the list.
2026-10-22 23:47:37 +02:00
Niko1221 eb818c50ed Download size counts the draft layer and the picture reader; --plan no longer calls an installed engine missing; --drafting off writes no --mtp (TUI stress #6, B03, B10/E04)
#6 (A07/B03): a model that is not installed now counts the draft layer (5.2 GB) and the picture reader (0.9 GB) in the one place (modelstate.assess), so the Folder step, the confirm and --plan agree (Gyro-S 58.5 + 5.2 + 0.9 = 64.6 GB). Gyro joins the families that get the helpers. --mtp off and --drafting off drop the draft layer from the count and from what setup is told. setup.py's disk check counts the draft layer it is about to fetch.
B03: --plan keeps "engine_missing" for no engine at all; an installed engine that fails the check says so in plain words and adds engine_usable.
B10/E04: with --drafting off (or an installed config that says off) the written run config carries "mtp": false and no --mtp / --spec-min-p arguments.
2026-10-22 23:44:27 +02:00
Niko1221 092a62daaa --model-path and --data-dir win over an installed copy with the same model and context (C08) 2026-10-22 23:37:20 +02:00
Niko1221 1ed28f8863 the Folder step checks the path before going on, the Save summary shows the storage path, a denied folder is told in plain words (B05, B12) 2026-10-22 23:36:16 +02:00
Niko1221 38c7f7017d ctrl+c quits the launch screens like ctrl+q; only a running download or tune asks first (B13) 2026-10-22 23:36:15 +02:00
Niko1221 c924d3c9d7 An old Terminal that cannot hand over to a new Control is told to run the install command once more (TUI stress #4, H03)
The Terminals before 0.2.1.3 (0.2.0-beta, 0.2.1, 0.2.1.1, 0.2.1.2) install a newer Control and report "updated", but their launcher has no hand-over, so the old Control keeps running. Their updater test-starts the staged Control with --version (STRATA_NO_ACQUIRE=1, no STRATA_HANDOVER, from the .s staging folder) and shows the first 120 characters of the answer when that fails. Control now answers that start with one plain sentence and exit 7, so the old updater keeps the old Control and prints the sentence instead of "updated". Every updater that can hand over sets STRATA_HANDOVER and is not touched. Release note added.
2026-10-22 23:33:24 +02:00
Niko1221 6f8800aefe Version bump for 0.2.3: app 0.2.3, engine and Control 0.1.48
Cargo.toml and Cargo.lock, desktop and host package.json and lock, PRODUCT_VERSION, CMake project and Metal STRATA_VERSION, build_hip_linux PKGVER default, About comments, tests that pin the number, Docker image tags and docs (0.2.3-<flavor>), download names in the READMEs, AMD_HIP.md and UPDATES.md, AGENTS.md, the bug report placeholder. The engine moves too: it shares the CMake number and its source changed since 0.2.2 (DFlash2, verify and prefill paths). A new test checks every place agrees and that a 0.2.2 install (Control 0.1.47) is offered Control 0.1.48.
2026-10-22 23:33:15 +02:00
Niko1221 814716336a a second launch reuses the same choices, asks or needs --restart for other choices, and frees the port first (G04/E01) 2026-10-22 23:31:30 +02:00
Niko1221 0530a1f7c2 retune stops its own server first; engine update gives one next step when the launch holds the card (E07/H04/H06) 2026-10-22 23:22:57 +02:00
Niko1221 f6db09bae2 launch keeps the API key it made until --new-key is given (E05/G01) 2026-10-22 23:21:37 +02:00
Niko1221 217ddda0eb status and logs find the running launch when no address is given (F02) 2026-10-22 23:20:36 +02:00
Niko1221 734ada7e1b requant_gguf: check the output's free space against the plan before writing (or in --dry-run, return 3)
The release runbook promises the pack/requant builder checks free space before it writes (DEEPSEEK-RELEASE.MD 9.6 item 5);
it did not.  --dry-run now prints 'space: N GB planned, M GB free on <fs>' and returns 3 when 98% of the free space does
not cover the plan, so 'requant_dry && requant' cannot start a run that will fill the disk; a real run raises instead.
With --resume only the bytes still missing are counted against the free space.

Verified on the box with two recipes that do not fit /data today: rq3 (224.3 GB planned against 128.6 GB free) and rq_gu3
(251.2 GB against 128.6 GB) both refuse with the numbers printed, rc 3 in dry-run.  pack_gguf's own output is a few MB
(tensors.txt, engram.bin, tokenizer) and needs no guard.
2026-10-22 22:38:16 +02:00
Niko1221 a14e85d8e6 tests: the pinned DFlash2 drafter address is allowed in strata_control (only strata drafting fetch reaches it, behind acquisition.require) 2026-10-22 21:19:41 +02:00
Niko1221 996b8a4044 dsv41: a type with no device kernel is known, laid out, and kept on the CPU lane (stream H1, second half)
The rule from DEEPSEEK-RELEASE.MD 6.5.2 - every ggml type works, a tensor whose type has no GPU kernel stays on the CPU
lane with one warning instead of stopping the load - is now implemented for IQ1_S (19), IQ1_M (29) and IQ4_NL (20):

- iq::known(t) (a layout exists: block_values/block_bytes/name) is separated from iq::supported(t) (a device kernel
  exists).  type_row_bytes/type_block use known(), so a pack lays out such a tensor instead of refusing it;
  gg::type_on_device(t) is what the VRAM tier tests.
- the CPU lane takes them: gg_cpu_type() covers 10-14, 16-23 and 29 (ggml-cpu's row dots).
- a layer whose experts are such a type is flagged cpu_only at load, printed once ('no GPU kernel for it, its experts
  stay on the CPU lane') and excluded from the VRAM budget in place_experts().
- iq::block_values(IQ4_NL) = 32 (QK4_NL): the only block here that is not 256 values.  Without it the CPU lane read
  IQ4_NL at the wrong row stride (2.4e-02 and 1.4e+05 error against the reference).

Verified: ds4_iq_test --type 29,19,20 runs the CPU lane against ggml's own decode and a double reference - IQ1_M
1.6e-08 / 4.6e-08, IQ1_S 5.4e-08 / 4.0e-08, IQ4_NL 3.1e-08 / 3.2e-08, ALL OK; the device types still pass; --pack-info
reports the layer and role for each such tensor.  A full model with real IQ1_M weights has not been run (none on the
box): the arithmetic of the lane is what is verified.
2026-10-22 20:27:19 +02:00
Niko1221 164dd3bbef Merge f-023-fix3: drafting network code loads only when used, rollback tests wait for the restored start, Arc B-series runs with pictures on 2026-10-22 19:58:52 +02:00
Niko1221 6577b69d5d Intel Arc B-series (Battlemage, e.g. Arc Pro B70) runs like any Arc: pictures on by default with the CPU reader, one plain line that the card is new and how to report a problem
Owner decision 2026-10-22. setup_intel no longer switches pictures off for a B-series card; it prints the line instead (known risk: the first-picture xe reset on the B70, 2026-10-19). strata launch gets the same line as a warn-level note (Computer.new_card_note) and its picture default for an Intel engine now follows the CPU-reader rule at any VRAM size (it used the GPU reader for a 32 GB B70). Gyro adoption on Arc was already allowed in this tree (adopt.FAMILIES). Tests that pinned the opt-in are updated; docs/INTEL.md, packaging/README.md and docs/GYRO.md say B-series is supported and untested by us.
2026-10-22 19:39:27 +02:00
Niko1221 dca4217501 test_runtime: the rollback tests wait 30 s for the restored launch (5 s was shorter than a Windows engine start); the two failures and the teardown error were a timeout, the rollback itself works
Same failure on v0.2.2 (011c9e600): the failed new start is refused for its port as intended, the old config is relaunched, but it was still loading when the 5 s startup wait ended, so rolled_back was False and the left-over process held the temp folder at teardown.
2026-10-22 19:35:29 +02:00
Niko1221 cf4bef2f78 dsv41: --pack-info prints the DSpark and Engram geometry a pack needs at run time
Today a pack built by another tool had dspark_block_size 0: DSpark ran one token a round and was worth +0.8% instead of
+28-77%.  The report now shows block_size / noise_token_id / markov_rank / the drafter's expert counts, the mtp tensor
count, and the Engram layer ids, with a <- marker when the block size is 0 or there is no drafter.  Verified on h225,
c0q3hQ6_K, c0q2d5 and mxfp4 (all: block 5, noise 128799, rank 256, 1242 mtp tensors, engram layers 1,14).
2026-10-22 19:33:47 +02:00
Niko1221 84b1e9815e strata drafting: load the network code only when the command runs, so a started engine's command line does not import it (437a60676 regressed the start-path rule) 2026-10-22 19:30:23 +02:00
Niko1221 c266cfb3cb Merge f-023-int into f-023-gui 2026-10-22 18:59:33 +02:00
Niko1221 a8301a2709 Merge f-023-metalfix2: no expert-cache loan in the Metal CLI prompt path either 2026-10-22 18:42:02 +02:00
Niko1221 669023fc08 Merge f-023-strixpp: Strix Halo gets its 8192 prompt chunk back on Windows, and the HIP engine finds its own tuning table 2026-10-22 18:36:58 +02:00
Niko1221 2f3f363c33 dsv41: --pack-info checks the files the pack points at (a cleaned-up model_dir is a dead pack, not a description)
The pack is pointers: ds4.txt + tensors.txt + the weights where model_dir says.  Today /home/niko/dsv41-c0/t3 was empty
and the c0q3 pack (all IQ3_XXS) described itself perfectly while being unable to load.  --pack-info now stats every file,
compares the length with the largest tensor end, prints MISSING/SHORT lines and exits 2 when any is bad; a healthy pack
prints 'files: N, all present and long enough'.  Verified: h225 10/10, c0q3hQ6_K 19/19, c0q2d5 12/12, c0q3 fails with
six MISSING lines.
2026-10-22 18:31:53 +02:00
Niko1221 b440640929 metal: no expert-cache loan in the CLI prompt path either (the 4K+ prompt of a plain strata run aborted in ExpertCache::fill_slot_queued) 2026-10-22 18:31:04 +02:00
Niko1221 155995b65c Merge f-023-drafting: the app hides DFlash2 until the engine serves it 2026-10-22 18:07:21 +02:00
Niko1221 e890e30766 The app offers DFlash2 only once the engine serves it: the Drafting setting is Auto / MTP / Off until then (Control reports it) 2026-10-22 18:07:03 +02:00
Niko1221 7b04360f93 hip prefill: the package tuning folder lookup split the exe path on '/' only (a backslash escape lost in the string): split on both 2026-10-22 18:05:47 +02:00
Niko1221 31785cce47 pack_gguf: build a pack from a quant that was made by another tool (the IST-DAS sets)
The IST-DAS 2.25bpw GGUF differs from ours in five ways that stopped pack_gguf, each found by building its pack:

- geometry(): ALIASES for the keys they spell differently (kv_source_layer_ids, index_source_layer_ids, the indexer's
  candidate_*, dspark_target_layer_ids, engram_layer_ids).  Without the source ids the engine refused the pack
  ('layer 2: compress ratio 2 without a KV source of that ratio before it').
- --engram-meta FILE: their GGUF carries no Engram hash parameters (no ple.*/engram.layers, no token map, no
  multipliers/primes/offsets - those are tensors of integer type in their file); the parameters come from a GGUF of the
  same model that still has them, the tables from --engram.
- integer tensors (I8/I16/I32/I64/F64) are left out with a line instead of stopping the build (engram_token_map.weight,
  engram_primes.weight, ...).
- --skip-tensors REGEX: their attn_output_a is one interleaved 3-D tensor ([8,1024,4096] ne) that this engine's
  o_groups slices cannot take; our Q5_K requant of the same BF16 weights supplies it through --dense-override, which
  now also *adds* a tensor the quant leaves out (it could only replace before).
- the DSpark geometry (dspark_block_size / n_routed_experts / num_experts_per_tok / target_layer_ids / markov_rank) is
  filled from the drafter's dflash.* keys and the markov tensor's shape, and --dspark-noise-token sets the model's noise
  token (128799).  Without it d_block stayed 0 and DSpark ran one token a round: h225 measured 16.77 tok/s off vs
  25.93 on (3.16 tokens a round) after the fix, 1.01 before.
2026-10-22 18:04:57 +02:00
Niko1221 e22a3c9ba9 Merge f-023-drafting: strata launch --drafting auto|mtp|dflash2|off and the app's Drafting setting; suggestion table (Auto = MTP until a card wins everywhere); pinned drafter download and numpy packer; gated until the engine serves DFlash2 2026-10-22 17:43:34 +02:00
Niko1221 b7196c23de Drafting in the app: "Drafting: Auto (recommended) / MTP / DFlash2 / Off" beside context in the model's settings
- The Models screen gets one more column in the context row, the existing Select, with the four choices; DFlash2 is offered for Qwen3.8-Flash-Next
  (the qwen and gyro families) only. The choice is saved with the model (a setting the app owns, like Draft layer) and shown again next time.
- Before a load the host asks Control (strata drafting plan / fetch --if-needed): when the choice would use the DFlash2 drafter and its file is not
  here, it is downloaded pinned and checked and packed, with plain progress in the load. A drafter that cannot be had never stops the load: the MTP
  starts and the job says so. The download plan dialog lists the drafter's size when it is chosen.
- What ran is told in the engine's words under Details and below the row: "Drafting: MTP (DFlash2 didn't fit)" and the other three reasons, all in
  the eight languages. Off is the draft layer's own switch.
- The app never reads the suggestion table: Control decides (strata_control/drafting.py), so the Terminal and the app cannot disagree.
2026-10-22 17:41:10 +02:00
Niko1221 fcb0c2d0ec Merge f-023-int into f-023-gui 2026-10-22 17:34:22 +02:00
Niko1221andmedking82 e17efa53ce Updates: an interruption that a later finished update settled says so in the same red alert (#3578)
Logic and tests ported from #3578 by medking82. The alert stays red and in place; only its words change for the restart interruption of an update that a later update of the same component finished. Sentence-case text in all eight languages; the PR's screenshots and validation files are left out.

Co-authored-by: medking82 <[email protected]>
2026-10-22 17:33:54 +02:00
Niko1221andmedking82 3f81245864 Reuse a retained legacy model library through a guarded native data root, without moving weights (#3362)
Ported from #3362 by medking82.

Co-authored-by: medking82 <[email protected]>
2026-10-22 17:33:01 +02:00
Niko1221andG0-0000 9471faa5f0 Web page: model settings rows no longer overflow their card on narrow windows (#3601)
The overflow was found and reported in #3601 by G0-0000. Its 75% column made the inputs narrower at every width, so this keeps the current layout when it fits: the label column may shrink, long names wrap, the input keeps 360 px.

Co-authored-by: G0-0000 <[email protected]>
2026-10-22 17:32:50 +02:00
Niko1221andjkir4n 3b51719abc Web page: keep long words and wide tables inside the viewport on narrow screens (#3594)
Ported from #3594 by jkir4n.

Co-authored-by: jkir4n <[email protected]>
2026-10-22 17:32:35 +02:00
Niko1221andengharat 2ba95cf61f Web page: use an edited API key at once, even when browser storage is blocked (#3435)
Ported from #3435 by engharat.

Co-authored-by: engharat <[email protected]>
2026-10-22 17:32:35 +02:00
Niko1221 437a60676f Drafting: strata launch --drafting auto|mtp|dflash2|off, one suggestion table, and the DFlash2 drafter fetched pinned and packed on the person's PC
- strata_control/drafting.py is the one place: the choice (saved in the model's run config as "drafting"), the table card class -> suggested
  drafter with the measured evidence (Auto picks DFlash2 only where it was faster on every tested prompt; today no class qualifies, so Auto is
  the MTP everywhere), the pinned drafter (incoai/Qwen3.8-Flash-Next-DFlash2 rev a22a1744, sizes and SHA-256 of the repository), what each choice
  puts on the engine arguments, and the plain "Drafting: MTP (DFlash2 didn't fit)" words read from the engine's own log.
- serve/server.py applies the choice at every start and reports "drafting" / "drafting_note" in /metrics.
- strata launch --drafting writes it to the launch config and presets, shows it in --plan, adds the drafter to the download plan when it would
  be used, fetches it before the start (a failure starts the MTP and says so), and prints the engine's fallback line.
- strata drafting plan|fetch (JSON/JSONL for the app). tools/dflash2_pack.py now reads safetensors with numpy alone (no torch on the person's
  PC), byte for byte the same file as before (checked against the earlier version), and ships in the Terminal bundle.
- DFlash2 is for Qwen3.8-Flash-Next (qwen, gyro) only. The engine in this release runs it only in its CLI run, not with --serve, so while
  drafting.ENGINE_SERVES_DFLASH2 is False the choice starts the MTP, says "DFlash2 isn't available in this mode yet" and downloads nothing.
2026-10-22 17:27:39 +02:00
Niko1221 f1b6a2144c Merge f-023-df2: DFlash2 drafter (opt-in --draft dflash2) merged onto the combined branch, builds on CUDA, HIP and Metal 2026-10-22 17:05:26 +02:00
Niko1221 c62d980112 hip prefill: with no STRATA_HIPBLASLT_TUNING in the environment the engine takes the table beside it (tuning/<arch>-hipblaslt-<version>.txt) itself, so a start that did not come through the launcher (hand-made config, CLI) does not read prompts at half speed on a Strix Halo; STRATA_HIPBLASLT_TABLE=off keeps plain hipBLAS 2026-10-22 17:04:58 +02:00
Niko1221 9ee8b0cf79 dsv41: --pack-info (the pack as a description: type map, bytes an expert/a token, tiers, tensors with no GPU kernel)
strata-dsv41 --pack-info --pack <dir> reads only ds4.txt and tensors.txt: no device, no weights, no lock.  It prints the
layer x role type map with an expert's bytes and a token's MB (top-k picks), the type totals, all-experts GB at the mean
bpw, the dense side split into embed / head / mtp / vision / engram, and every tensor whose type has no device kernel
(IQ1_S, IQ1_M, IQ4_NL today) with the layer and role - the H1 rule's report, and the first line of every benchmark log.
Verified on c0q3 (208.1 GB at 3.06 bpw, 3.251 GB a token - the document's figures) and c0q3d5.
2026-10-22 16:57:11 +02:00
Niko1221 051db74cdf gfx1151 on Windows: a bare --prefill auto may reach 8192 again (the 6144 cap of #1630 stays for other cards); STRATA_GFX1151_CHUNK=0 keeps 6144 2026-10-22 16:55:16 +02:00
Niko1221 49c157fae3 dsv41: IQ2_XS device kernels (bit-exact vs ggml), Q5_K/Q4_K/Q6_K embedding rows on the host, i-quant expert test, CPU-lane bench reference
- ds4_iq.cu: IQ2_XS (type 17) through the same kernels as IQ2_XXS: Ty<17> (74 B, 1024-word grid),
  decode<17> and deq8<17>, block_bytes/name/dispatch.  ds4_iq_test compares the token path, the window
  path, the prompt sets and the CPU lane against ggml's own to_float + a double reference: worst
  4.6e-08 (token gate_up), 4.6e-08 (down), 8.9e-03 (F16 tiles), CPU lane 3.5e-08.  IQ2_XXS / IQ2_S /
  IQ3_XXS / IQ3_S / IQ4_XS pass the same test unchanged.
- ds4_model.cu: gg_type_of takes 17; embed.weight may be any ggml block type (Q5_K / Q4_K / Q6_K ...)
  through gg_row_to_bf16 (ggml's to_float, the same BF16 rounding as the Q8_0 path) - the IST-DAS sets'
  Q5_K token_embd needs no dense override now.
- gg_dot_bench: the A/B's pass criterion was |x-y|/|x| on single outputs, which fails on cancellation
  while the two paths agree to a few ulps; it now also compares 64 sampled rows against a double
  reference scaled by the L1 norm (worst 1.1e-08) and requires |x-y| <= 1e-6 of the output scale.
  A/B at 32 threads: single token x1.03 (IQ2_XXS) / x1.13 (IQ2_S); 4 rows x2.4 / x2.3.
- CMake: ds4_iq_test target; ds4_tq_test and ds4_tq_test_portable link ds4_gg_cpu.cpp (the ggml CPU lane
  ds4_iq_cpu.cpp refers to since 49e82263a, missing from those targets).
2026-10-22 16:53:39 +02:00
Niko1221 0c467e9d45 Merge f-023-agent: agent context pressure from measured tokens, summaries in pieces that fit, recovery from the context-window 400, preflight of oversized messages, quote-aware read-only shell detection, visible compaction, plain context errors, stale child marker after a force-kill
# Conflicts:
#	apps/host/src/screens/Chat.tsx
2026-10-22 16:52:15 +02:00
Niko1221 76ac5842ff Merge f-023-tune: Work profile CPU workers ceil(0.8 x physical cores), VRAM reserve capped at 1536 MiB on 16 GB cards (picture floor only for a GPU reader); #3493 conversation cache sized automatically after load; #3438 bench reports an engine that did not stop 2026-10-22 16:26:51 +02:00
Niko1221 5de8d26949 tests: setup no longer tips about the conversation cache (the engine sizes it, #3493) 2026-10-22 16:22:37 +02:00
Niko1221 20fed551ba dsv41: Strata AVX2 i-quant rows opt-in (DS4_GG_IQ256=1) until the engine A/B 2026-10-22 16:21:25 +02:00
Niko1221 e4b31d6f69 strata-context: drop the development log hook 2026-10-22 16:14:56 +02:00
Niko1221 b96f174775 Merge f-dflash2 (the DFlash2 drafter, opt-in) into f-023-int 2026-10-22 16:10:08 +02:00
Niko1221 f0010a607f serve: the conversation cache is sized from the free RAM when --conversation-cache-mib is not given (#3493): 40% of what is left above the parking floor, at most 8192, off under 1024, one log line; an explicit value (0 too) is kept; harness_reuse_probe.py; docs 2026-10-22 16:06:17 +02:00
Niko1221 ed0ed38e2d Merge f-023-tls: #3722 HTTPS uses the OS certificate store (truststore, certifi fallback) in setup, Control, serve and the fetch tools; the app host adds the system roots; a certificate failure says plainly what is intercepting and what to try 2026-10-22 15:43:33 +02:00
Niko1221 c4aec611b2 tls: downloads check the server against the operating system's certificates; a failed check gives a plain message (#3722)
Python's OpenSSL checks against certifi's roots and does not fetch a missing intermediate; Chrome and curl.exe use the Windows store
(schannel), which also holds the roots antivirus HTTPS scanning, a company proxy or a preview build adds. strata_control/tls.py
install() puts truststore (pinned in requirements.txt and the three runtime locks with hashes, CPython 3.12 in every archive) under
every HTTPS connection of setup, Control's transfer and component download, serve and the fetch tools, with certifi as the fallback.
A certificate failure is no longer the raw urlopen text and is not retried: it names the likely causes (antivirus web scanning, a
company or school proxy), says what to try, and never offers to skip the check.

The host: server/system-ca.mjs adds the system roots to Node's bundled list at start (Node 24.5+), and the agent process gets
NODE_USE_SYSTEM_CA=1.
2026-10-22 15:43:12 +02:00
Niko1221 3ae6b3041b A big message on a chat near the compaction threshold summarises first; the real-engine script can also run against a server that is already up 2026-10-22 15:40:55 +02:00
Niko1221 ebaeda85f1 Summarize and continue asks for the answer instead of sending the refused message a second time; agent README describes the context pieces 2026-10-22 15:37:35 +02:00
Niko1221 815256d89b Tests for the two ways to send a message that is too large (file in the chat folder, shortened preview) and the host route 2026-10-22 15:26:53 +02:00
Niko1221 b14862351b The chat checks a message against the model's window before sending it, handles /compact itself, and explains a full context in plain words
- /compact was typed into the chat and sent to the model as text (it answered "nothing to compact, this conversation is still short" at 90%
  full). The host now runs the Harness's /compact and says how full the chat was and is, measured.
- Before a message goes out the model server counts it (its own tokenizer). Fits: sent. Fits once the older messages are a summary: summarised
  first, with a line in the chat. Can never fit (a 103 KB paste is 70K tokens in a 64K window): not sent, and the card offers "Attach as a file"
  (saved in the chat folder, the assistant reads it in parts) or "Send a shortened preview".
- A refusal that remains after the Harness's own compact-and-retry is a plain sentence with the engine's text under Details, and the card
  offers "Summarize and continue" or "Start a new chat" instead of "Try sending the message again".
- Compaction shows: a status line while it runs and one plain line afterwards. The context meter follows the running turn (the host reports
  what the context holds after every model request) so it no longer reads 1% between two tool steps.
- A Harness error "lost host child is still present" reads as "the last Strata session is still closing down".
- All new sentences are in the catalog in the eight languages. The real-engine run is scripts/context-e2e.mjs.
2026-10-22 15:26:30 +02:00
Niko1221 f8f4e8a7df bench: a failed engine stop is loud and fails the run (#3438): stop_engine returns why the process may still be alive (taskkill result and exit, the 60 s and 30 s timeouts), the run prints an ERROR, writes ENGINE-NOT-STOPPED.txt, exits 6 and reports ok false 2026-10-22 15:20:59 +02:00
Niko1221 acf63bef73 app: the Work profile takes 80% of the physical cores rounded up (the engine's own count at most), keeps no more than 1536 MiB on a card of 16 GB or less, and a CPU picture reader no longer raises the reserve to 2048 (RTX 5070, Gyro-S 64K: 9 workers + 2048 33 tok/s, 5 + 1536 46) 2026-10-22 15:17:24 +02:00
Niko1221 098fd545b3 A marker left by a force-killed app no longer blocks the next load when its process number belongs to a later program
The child marker without a creation time could not tell the recorded engine from another Python or Strata program that had reused the
number, so every load said "lost host child is still present" until the marker was deleted by hand. The child is always started before
its marker is written, so a live process created after the marker's file time (with a two-second slack) is not the child. Unreadable
times still keep refusing; a recorded creation time still decides by itself. New procs.started_epoch reads a creation time on Windows,
Linux and macOS.
2026-10-22 15:02:49 +02:00
Niko1221 a7cd2de97a A look-only shell command with an & or a bracket inside quotes no longer asks for approval
The read-only check refused any command containing & ; ( ) { } before it understood quoting, so listing "C:\...\GUI Q&A" showed an
approval card. Quoted text is now set aside first (maskQuotes): an expansion inside double quotes ($ or a backtick), an unterminated
quote, a quote glued to other characters and a quoted text that starts with a dash are still not understood, so those and everything
outside quotes ask as before.
2026-10-22 14:46:52 +02:00
Niko1221 be58604387 Compaction reads the prompt size the model server measured, and a summary request always fits the window
The Harness prices a conversation at 4 characters a token; code, logs and JSON are about twice as dense. A new Harness plugin
(plugins/strata-context.mjs, arithmetic in context-budget.mjs) wraps the token meter so pressure is the last measured prompt plus what
came after, priced at the density that measurement showed. It also wraps the compaction summary request: when it would not fit the
window it is asked for in pieces (a message bigger than a piece is cut for the request, each piece builds on the summary before it, a piece
the server still refuses is halved). A chat the engine refused once (a 70K-token message in a 64K window) is no longer stuck: the next
message compacts and goes through. Tests run the real Harness against a scripted server that counts and refuses like the engine.
2026-10-22 14:40:20 +02:00
Niko1221 5701fc65ce Merge f-023-tools: tool-returned pictures with elastic vision, picture-reader start errors and per-variant packaging, Markdown images (safe sources), PR #3556 tool aliases, MCP wiring, command-tool cwd/shell/timeout, image-token limit, gate fails closed 2026-10-22 14:31:55 +02:00
Niko1221 0351ecb11d Two GTX 1080s are a working PC: an 8 GB card starts Off only beside a bigger card, and a PC whose only NVIDIA cards are Pascal uses them on the CUDA 12 engine (#3655)
recommend() (Terminal, app and shared vectors) now turns a card with 8 GB or less Off only when a bigger usable card is there or it has under 4 GB. The hardware report (tools/strata_mcp.py) applies setup's own opt-in rule for a PC with only Pascal/Volta NVIDIA cards and no usable AMD card, so those cards are no longer flagged 'older than the RTX 20 series' (engine_variant cuda12); the feed already picks the cuda12 package for sm_61 and refuses cuda13. Tests: shared gpu-roles vectors, strata_mcp hardware, feed selection for sm_61, and Tune keeping a two-card role map.
2026-10-22 13:47:41 +02:00
Niko1221 d9042a16bd Missing model files are named, with one next step (#3478)
The load used to stop with 'Some model files are missing' and nothing else, so Retry looked the same every time. Control's model list now carries the required parts it cannot find (part, file, status); the host joins them with the installer's missing GGUF files and shards, puts the names in the error (four shown, all in Details and in the load log), and the app shows 'These model files are missing: ...' with 'Press Download & load again to fetch them. Files that are already there are kept.' in all eight languages.
2026-10-22 13:29:59 +02:00
Niko1221 642f5120a7 Route change: the view transition's update callback no longer waits for requestAnimationFrame (#3673)
While the callback is pending the browser holds rendering, so the frame callback never ran and every click waited for the browser's own guard: about 4 s in Chromium (and so the WebView2 app window, reproduced in Edge 155: callback 4023 ms) and about 5 s in Firefox. The callback now resolves on the first DOM change after the hash flipped, or after 150 ms. Measured in Edge 155: callback 9-19 ms, full transition about 350 ms, with and without a 3.6 MB chat history.
2026-10-22 13:24:33 +02:00
Niko1221 98a139c3c7 Updates never cross GPU makes: Windows display adapters decide the backend, an installed engine of another make is not replaced (#3688)
On an AMD R9700 PC a leftover nvidia-smi made detection say cuda, so the update staged cuda13 and its test start failed. detect_backend (Control) and gpuBackendHere (app) now let the Windows display adapters decide first and ignore a tool for a make with no adapter. The update decision keeps the installed engine's make when the PC has a card of it, and otherwise stages nothing and says so in plain words (status wrong-make, shown by the app). update_engine refuses a package of another make before any download. --backend and STRATA_ENGINE_BACKEND are the explicit way to change make.
2026-10-22 13:21:36 +02:00
Niko1221 828a2559a9 vision: the image-token limit has a default and is enforced (I7, #3690)
The reader was clamped only when the config carried vision.max_tokens; an old, hand-edited or migrated config left it unclamped and
one picture became 93,700 image tokens. Vision.token_limit gives every reader a limit (GPU 1024, CPU 300, the numbers setup writes)
unless the config names one (a value the person set is kept, 0 still means none). strata-vision refuses a picture that comes back
over the limit plus an eighth with a plain ERR instead of passing 90 times the budget on, and prints a WARNING when its own warm-up
picture is over it. The server logs a picture that is more than 4 times the limit with the numbers to send with a report.
2026-10-22 13:10:12 +02:00
Niko1221 bf8285b8c9 Merge f-dflash2-check: the per-context cost table for the cost-aware MTP window (opt-in, STRATA_SPEC_COST), phase 0 tools and launch-script fixes 2026-10-22 13:09:05 +02:00
Niko1221 a71fd34758 agent: custom command tools run in the right shell and folder, with a time the tool can raise (T7)
shell: true meant cmd.exe on Windows (Get-Content failed), the host's own folder (python tools/foo.py found nothing) and a fixed two
minutes with silently cut output. custom-command-run.mjs: a Windows command that reads as PowerShell runs in pwsh or powershell.exe,
any other in cmd.exe as before, sh elsewhere (or the tool's named shell); the folder is the call's optional cwd, then the tool's cwd,
then the home folder, and a missing folder is a plain refusal; the time is the tool's timeoutSec (1 to 3600, default 120) and ends the
programs the command started; a cut output says so. Settings stores cwd, shell and timeoutSec for a command tool and an edit that
does not send them keeps them. No new field in the Settings screen: that needs a design decision.
2026-10-22 13:09:00 +02:00
Niko1221 7d55581527 agent: stop claiming a window opened; point the model at the present tool (T9, O6, #3576)
A launch command that printed nothing was reported as 'The program, folder or file was opened.', and the system prompt told the model
to say it is open. A browser given a file:// address also returns without an error, and the model then reported that it had previewed
the page. The card now says what is known (the command finished without an error, so the program was started; Strata cannot confirm
that a window or page opened). The prompt (and the plan view step) send local images, SVG, HTML and PDF to the present tool and
forbid claiming a window opened or a page rendered unless a tool confirmed it.
2026-10-22 13:07:18 +02:00
Niko1221 034f2fdebe agent: tool servers follow the default model, one naming rule, a dropped server is reported (T5, T6)
T5: the agent's tool servers came from local[0], the first model loaded, not the model the chat runs on. buildProfile takes
defaultLocalId (the native default instance, agent-bridge option defaultLocalId); the agent's default route follows it too.
T6: the editor and the save check accepted 64 characters while the profile dropped everything over 32 or with a dot, silently.
server/mcp-names.mjs holds the one rule (32, letters, digits, hyphen, underscore; src/lib/mcp-names.ts is its copy, a test
keeps them equal). A name saved before the rule stays saveable. mcpRows reports what it leaves out (name, no program or address,
a name that collapses to another's id); the agent status carries skippedServers and the chat shows one notice per reason before
the turn. The sentences are in all eight languages (src/locales/tools-<lang>.json).
2026-10-22 13:06:11 +02:00
Niko1221 c5b13accfd metal: serve_probe.py, the owner round 9 short/long prompt probe on the default config 2026-10-22 13:05:58 +02:00
Niko1221 4791d6b676 agent: the approval gate fails closed when it cannot read the permission mode (T4)
A session whose preset lookup threw ran the call unasked (return next()). It is treated as Manual now: a call that is not provably
read-only asks, a read goes through, and the reason is logged.
2026-10-22 13:03:11 +02:00
Niko1221andAmit Tsur 8bf178febf serve: tool-call aliases and wrapper boundaries (port of PR #3556), and an unparsed call is logged (T1)
Ported from #3556 (Amit Tsur): a declared namespaced tool written with its flat alias (mcp__server__tool for mcp__server.tool) keeps
its schema while streaming and is reported under the declared name; an ambiguous alias is no alias; a quoted incomplete wrapper
(`Format: <tool_call>... </function> and then I explain it.`) stays text; a following call opener separates concatenated calls;
streamed JSON is closed only once the wrapper or the next call confirms the boundary; a call cut by the output limit stays
incomplete. The Responses-only alias map is gone (the parser resolves aliases now).

Reconciled with the 0.2.2 envelope rule: with tools declared, a call for an undeclared name stays text on every route (the PR's
tests that expected a native call for an undeclared name now expect text, or a native call only where no tools were declared and
declared_only is off: the Responses/Codex compaction route keeps that). The aliases attribute replaces the envelope work's _alias
set, so the lenient opener (#3441) and the declared-only gate use the PR's unambiguous alias table.

New rail: a call that named a tool the request did not declare is counted (parser.unread) with the names the model wrote
(parser.unread_names), and the server's existing log line for unread calls now prints them.

Co-authored-by: Amit Tsur <[email protected]>
2026-10-22 13:01:04 +02:00
Niko1221 1fa22cd29a app: the chat renders Markdown images from data: URLs and the chat's own folder; remote pictures stay links (O3)
![alt](src) used to come out as a link with a stray '!', and a data: URI as literal text. Allowlist (src/lib/markdown-images.ts):
data: PNG/JPEG/WebP/GIF (never SVG), the host's /api/files/raw address, and a scheme-less path read through that address for the
active chat. http(s) pictures are not fetched: they show as a link with the alt text. file:, javascript:, blob:, share paths and
every other scheme show as text. One .md-image rule keeps the picture inside the bubble.
2026-10-22 12:58:22 +02:00
Niko1221 e2858b9855 packaging: the picture reader is built per CUDA variant and checked against the package's archs (I1, I2)
build_cuda_linux.sh builds strata-vision for the same architectures as the engine (cuda12 includes sm_60/61/70) and records
vision_archs in BUILD.json; the Dockerfile records it too. packaging/build_vision_windows.bat does the same for the Windows
cuda12 and cuda13 zips (they shared one reader built before Pascal support, #3669). package_engine.py refuses a CUDA package whose
reader's vision_archs do not cover its archs and a macOS package without strata-vision (#3697); --allow-stale-vision and
--no-picture-reader are the explicit exits; --stamp-vision records a hand-built reader. ENGINE-COMPONENT.md describes the release steps.
2026-10-22 12:56:30 +02:00
Niko1221 e6dbbc312a metal: no expert-cache loan for the prompt path (Prefill::relayout and the refill into given-back allocations), auto expert cache capped at 1500 slots; CI: Metal build job on a macOS runner 2026-10-22 12:55:09 +02:00
Niko1221 425bca7199 Agent watchdog: the engine's prompt-read progress counts as activity; local routes get a 60 min stream-idle timeout (#3623)
A local engine reading a long prompt sends the Harness nothing for minutes, so the 4 minute stall limit ended the turn mid-read. The host now hands the bridge the engine's /metrics progress (live.prompt_read, generated, slots); a moving mark keeps the turn alive for at most 30 minutes after the Harness last spoke. An engine that stops reporting still ends at the usual limit. API routes are unchanged.
2026-10-22 12:54:48 +02:00
Niko1221 1298a3971c Chat store: save the chat list at most once a second, serialise only changed chats, cap the saved copy (#3604, #3603)
Every streamed token stringified every chat and wrote the lot to localStorage, which grew the WebView2 renderer to gigabytes. Writes from the streaming path are now coalesced (1 s), flushed when a turn ends, is cancelled or fails, and on pagehide/hidden. Unchanged chats reuse their JSON by object identity; a field over 200k characters, a chat over 1.5M and a list over 3.5M are shortened in the saved copy only. The stored format is unchanged.
2026-10-22 12:52:53 +02:00
Niko1221 c40a03e6dc serve: a tool's picture with an elastic image encoder starts the encoder (I6); a failed encoder start carries its stderr tail, exit status and card hint (I3)
A picture a tool returned wrote to an elastic encoder that was not running (AttributeError, then a 400 that every later
turn repeated). _note_unreadable_tool_images now calls _vision_up and turns a start failure into the usual
[image omitted: why] note; Vision.encode raises a clear ValueError when no reader runs.

vision_start_error no longer returns a blank sentence: last stderr lines, the exit status in words, the seconds of a
timeout, and a kernel-image error (sm_60 on the Windows cuda12 reader, #3669) names the card/architecture and the CPU way out.
2026-10-22 12:52:37 +02:00
Niko1221 eebb870922 dflash2: reserve the drafter's head before the expert cache is sized, loud fallback lines naming the VRAM needed, Blackwell window cap 6 (draft_cap_test) 2026-10-22 12:44:45 +02:00
Niko1221 dd8201d20d metal: link stubs for hc_mean_streams and route_marginal_stats (STRATA_TAP_FILE / STRATA_ROUTE_STATS hooks of the shared Verifier; owner round 9 G6 link failure) 2026-10-22 12:44:40 +02:00
Niko1221 a4c503d7ce nv_defaults: the sm_86 speculative start-up adaptation is opt-in (STRATA_SPEC_ADAPT_SM86=1); owner round 9 measured 0.995 on the A4000s and a length-confounded 3060, so the earlier gain did not reproduce 2026-10-22 12:43:35 +02:00
Niko1221 23b3dfc2ca Merge f-0221-sycl: a Linux SYCL (Intel Arc) engine package recipe and feed entry, --version without GPU access, RUNPATH $ORIGIN; Arc PCs get an engine from the feed 2026-10-22 12:13:33 +02:00
Niko1221 8be3f916f2 Intel Arc engine package: packaging/build_sycl_linux.sh, tests, docs
No release feed ever carried a linux-x86_64-sycl engine package, because no recipe built one (the 0.1.45 package of stream split-s1 was made by
hand). build_sycl_linux.sh builds the JIT engine with oneAPI, bundles the oneAPI runtime and the Unified Runtime adapters with collect_libs.py,
sets RUNPATH $ORIGIN on every ELF file (Intel's libraries have none or a wrong one), packages it with archs intel-arc and checks the package
with --verify, binary_deps.py and an env -i strata --version. _drm_vendor_backend takes the sysfs root so the Arc detection can be tested with a
fake /sys/class/drm; tests/terminal/test_sycl_package.py covers the choice (an Arc PC with no oneAPI tools gets the package, nobody else does,
Windows gets a reason) and the recipe; the release-matrix test has the sycl package.
2026-10-22 12:12:30 +02:00
Niko1221 424b364a88 SYCL engine: strata --version answers before any GPU is touched; no SYCL device is built before main
The migrated generate.cpp had no --version (the CUDA and HIP engines answer it first thing), and three things in dpct's helper headers
built a SYCL device at program start: blas::descriptor's inline _saved_queue_ptr, the dpct::device_memory constructor (pointer table,
mem_mgr, dev_mgr) and its default-built sycl::queue member. On a PC whose user cannot open the GPU the engine therefore aborted with
'No device of requested type available' before it could print anything, which also failed the engine smoke test of strata update.
The headers are edited and sycl/tools/fixups.py reproduces each edit after a re-migration. A frozen first-token-logits run on an Arc A750
(4095-token prompt, IQ3_XXS) is byte-identical before and after.
2026-10-22 12:12:30 +02:00
Niko1221 0788a91c64 Merge f-0221-fix: #3683 test start behind a Windows venv launcher, #3670 certifi 2026.10.19, #3613/#3467 calibrate in the bundle, and the owner's round 8 Terminal fixes (real errors, cards and roles kept, --mtp off, every download planned, engine-version preflight) 2026-10-22 11:33:43 +02:00
Niko1221 ad98bf1d17 N6: Gyro on macOS needs engine 0.1.47 (the Metal TQK6/TQK7/TQ2_T kernels first shipped there) 2026-10-22 11:33:30 +02:00
Niko1221 c67180edf1 Owner round 8: show the real engine error, keep cards and roles, honour --mtp off, plan every download, check the engine version
F5: a start that did not reach ready names the stage, the cause line from the engine log, a next step (render group on Linux, smaller
context, strata update) and the log path; --no-tui and runs without a keyboard never ask 'Keep or move?' and print what was kept.
F3: a context or picture change forwards the installed cards and Main/Helper roles to setup.
N3: --mtp on|off reaches setup, and setup no longer fetches the draft layer for --mtp off.
N4/N2: --plan counts a picture reader unless one sits beside the files, names the --vision off and --mtp off switches, and counts
the ready-made pack (NVFP4) setup would fetch; setup's disk hint is --data-dir under launch.
N5/N6: STRATA_DATA_DIR/install is looked at first, --plan says when an earlier install is the one in use, and an installed engine
older than the model needs is updated or named.
N8: --plan warns when the file headers say another family or model than the one named.
N9: update --check names the package it selected.
N11: the Arc adopt path takes Gyro and the sentence has no double full stop.
2026-10-22 11:32:32 +02:00
Niko1221 49e82263ad dsv41: CPU lane takes Strata's AVX2 i-quant rows (switchable), ds4_gg_bench, pack_gguf repeatable --dense-override 2026-10-22 10:31:03 +02:00
Niko1221 736d7a840c dsv41: pack_gguf --override-experts takes several folders with roles 2026-10-22 10:16:05 +02:00
Niko1221 a539e12c97 Merge f-strix-hcq8: Strix Halo 8-bit hyper-connection weights by default (gfx1151), sm_86 start-up adaptation default, gfx1151 expert layout mode 8, env-gated DFlash2/spec instrumentation and the opt-in cost-aware MTP window 2026-10-22 10:10:32 +02:00
Niko1221 2a22daa4f0 Fix #3613 #3467: the Terminal/Control bundle ships tools/calibrate.py; a setup crash now says what it was
setup.py imports tools/calibrate.py after a tune, but INSTALLER_TOOLS did not list it, so every launch after a tune died with
ModuleNotFoundError and only 'The model could not be installed'. The list now names it, and add_installer also reads the
tools/ modules setup.py imports so the list cannot go stale again (stage-control.py uses the same list).

A new test stages the bundle the way package_terminal.py does and imports setup.py and its tools with tools/ limited to the bundle.
strata launch keeps the last 400 lines of setup's output in launch/<name>.setup.log, shows the Python error line in plain words
when setup crashed without an [X] line, and names the file.
2026-10-22 10:08:50 +02:00
Niko1221 27a5707de5 Fix #3670: certifi 2026.10.19 (drops the emSign ECC Root CA - G3 root) in requirements.txt and the three runtime locks 2026-10-22 09:38:34 +02:00
Niko1221 72d0bf667e Fix #3683: the test start no longer waits out its 10 minutes when a Windows venv launcher sits above the server
A source checkout's .venv\Scripts\python.exe starts the real interpreter as its child, so the pid in the server's ready file
was one below the pid the runtime host started, the ready file was never matched, no probe was sent and the update ended
with startup_wait_expired. The host now accepts a ready pid within three parents of its child (Windows only).

The test start also shows progress (loading, loaded and waiting for one token) and a wait that runs out says what the engine was
doing, with the kept log, instead of blaming a busy graphics card because the log mentions VRAM.
2026-10-22 09:38:34 +02:00
Niko1221 e7ded666be docs: the Strix Halo 8-bit hyper-connection line goes into a new release-notes-0.2.3.md (0.2.2 is released) 2026-10-22 09:13:17 +02:00
Niko1221 3ae2123bce dflash2: --draft-conf (window cut by the selector's cumulative probability), drafter streams at high priority on CUDA 2026-10-22 08:51:54 +02:00
Niko1221 6d4874a491 hip: 8-bit hyper-connection weights (STRATA_HC_Q8) on by default on gfx1151, STRATA_HC_Q8=0 keeps the bf16 read; one start-up line; docs and 0.2.2 release note with the KL table and the CPU-vs-GPU rounding baseline 2026-10-22 08:16:47 +02:00
Niko1221 b1ebc3d7ca spec: the cost-aware window keeps one row-cost table per context length (< 6K, < 24K, longer), seedable per bucket 2026-10-22 07:24:46 +02:00
Niko1221 5e52cf569f dsv41: pack_gguf --override-experts (mixed-bit packs from two requantized sets); stageB tooling 2026-10-22 07:03:10 +02:00
Niko1221 ff9ca03f3c dflash2: a verify window whose graph runs out of memory narrows the window instead of failing; --draft-cap, 5 by default on Strix Halo, a plain line on 12 GB NVIDIA 2026-10-22 06:24:00 +02:00
Niko1221 5278ba0043 dsv41: dense K-quant GEMV (Q5_K/Q6_K) + dense_requant, ggml-type CPU lane, imatrix builder, requant fixes (C0 experiments) 2026-10-22 04:03:32 +02:00
Niko1221 1ef68f83e7 docs: note the STRATA_HC_Q8 measurement on the Strix Halo
+8.4% on json_extract at fixed windows (53.55 -> 58.03 tok/s), 94 of 250
output tokens differ from the BF16 read, cold HC read 93.8 -> 20.3 us
(142 -> 352 GB/s).  Experimental, opt-in only, never a default.
2026-10-21 23:43:02 +02:00
Niko1221 aec327ee51 gr: fold the hyper-connection inject and gate/mean into their neighbours
gr_read launched six kernels per hyper-connection read: norm, down,
gate, mean, inject, write.  The inject projection is now the down grid's
last block (one warp per stream, the same lane stride and warp_sumf as
gr_inject_kernel), and gate + mean are one kernel (one block owns WARPS
columns for every stream, the per-stream gated values meet in shared and
are added in the old c order).  Both keep the old per-row arithmetic and
reduction order, so lo/lq, mixed and inject are bitwise unchanged; gated
is no longer written to global because nothing outside gr_read reads it.
Six launches per read become four.

This is a no-op for the default IQ3_S path on the Strix Halo: that engine
already takes the fused (staged) hyper-connection read, not gr_read.  The
change is kept for the unfused gr_read path (CUDA, other HIP
architectures and --no-fused-gr).

Verified on gfx1151: json_extract at fixed windows (--spec 4
--spec-min-p 0 --suffix-draft 0) gives an identical output line before
and after.
2026-10-21 23:42:55 +02:00
Niko1221 9fe2ad3f55 wip: gfx1151 takes mode 8 gate/up (LDS, fused SwiGLU + q8_1) and the CUDA-layout down for IQ4_NL 2026-10-21 19:55:22 +02:00
Niko1221 ba6966d83f verify: the DFlash2 tap kernel only where the DFlash2 sources are built (Metal and SYCL keep the per-row dump path) 2026-10-21 19:52:11 +02:00
Niko1221 4c953b424d wip (experiment): expert layouts compiled for every HIP arch 2026-10-21 19:46:43 +02:00
Niko1221 6f83fe2c42 dflash2: drop the cache-touch kernel (no gain), wait-commit measurement switch 2026-10-21 19:46:26 +02:00
Niko1221 2ccf57ec46 spec: faster start-up adaptation of the expert tier in speculative runs (swap every 2 verify rounds for the first 200, then the normal interval); default on sm_86 only, STRATA_SPEC_ADAPT_EVERY / STRATA_SPEC_ADAPT_WARM set it elsewhere, an explicit --adapt-every is kept 2026-10-21 19:45:52 +02:00
Niko1221 89799596a6 dflash2: fused conv+residual+norm+quant, q/k norm+rope, K/V ring write and attention-output quantization; stage skip switches and small benches 2026-10-21 16:34:27 +02:00
Niko1221 48ec4a59ac spec: cost-aware window: the first round of a width is replaced by the guess from the other widths, a running mean for the first samples 2026-10-21 16:10:56 +02:00
Niko1221 e0d79b02f9 spec: with STRATA_SPEC_COST a round far above the known cost of its width (the first use captures a graph) counts as a tenth of a sample 2026-10-21 16:04:10 +02:00
Niko1221 d45cd9d3cf spec: STRATA_SPEC_COST_TABLE seeds the row costs (a per-card table), the CLI prints the learned table at the end 2026-10-21 15:35:07 +02:00
Niko1221 5bc7f8cab3 dflash2: tiled Q8 GEMM, fused quantizers, head-major K/V ring with a prefetching attention, forked context update, PLE read-ahead, benches 2026-10-21 15:31:29 +02:00
Niko1221 b7d5ea2f82 native_expert_bench: BENCH_CAP sizes the launch for more possible groups than real ones (what the empty block rows of a verify window cost); bench only 2026-10-21 15:08:13 +02:00
Niko1221 d232721db6 spec: cost-aware chain rule keeps the best width seen and looks one or two drafts ahead (the row cost is not convex); unmeasured widths are probed a few times 2026-10-21 15:03:51 +02:00
Niko1221 5f49ec4599 spec: STRATA_SPEC_COST keeps an MTP draft only while the next row pays at the measured row cost (step-by-step chain rule in MtpDrafter, DraftPolicy::keep_draft) 2026-10-21 14:46:56 +02:00
Niko1221 54f81315bf dsv41: DS4_TRACE routing trace (env), routestats / budget model scripts live in phaseA-tests 2026-10-21 14:12:19 +02:00
Niko1221 6878105560 spec: STRATA_SPEC_COST, a cost-aware MTP window (DraftPolicy::best_rows: the window with the best expected tokens per ms from the chain's probabilities, a measured probability-to-acceptance table and the measured row costs); the trace now logs the probabilities the window was chosen from (A/B arm, off by default) 2026-10-21 14:07:25 +02:00
Niko1221 7f97c35aca DFlash2 drafter for Qwen3.8-Flash-Next: Q8_0 pack tool, GPU forward (conv, K/V ring, windowed attention, top-16 selector), permanent verify/prefill taps, --draft dflash2 --draft-model (opt-in, MTP stays the fallback) 2026-10-21 14:04:15 +02:00
Niko1221 651a56806a generate: STRATA_SPEC_TRACE, a per-round trace of the window rows, accepted drafts, round ms and MTP draft probabilities for offline window-length studies (bench only, off by default) 2026-10-21 13:46:06 +02:00
Niko1221 d276adf277 verify: STRATA_ROUTE_STATS (distinct and new-per-row routed experts per layer per window width) and a per-width GPU stage profile line for STRATA_SPEC_T_CYCLE runs (bench only, off by default) 2026-10-21 13:28:02 +02:00
Niko1221 d8f58af1b5 generate: STRATA_SPEC_T_CYCLE, a bench-only cycle of the verify window size with a per-width cost line (off by default) 2026-10-21 13:25:20 +02:00
Niko1221 dcba249d01 verify: STRATA_TAP_FILE, an env-gated dump of the hidden states at chosen layers per verified row (DFlash2 acceptance experiment; off by default, no new graph nodes without it) 2026-10-21 13:25:19 +02:00
Niko1221 5f426c38cb tools/dflash2: q8test, the drafter with Q8_0 matrices keeps its acceptance 2026-10-21 13:24:21 +02:00
Niko1221 128223891e tools/dflash2: the torch port of the DFlash2 drafter and the scripts of the acceptance experiment (research tooling) 2026-10-21 13:24:21 +02:00
Niko1221 17c9b92bc9 generate: STRATA_SPEC_T_CYCLE, a bench-only cycle of the verify window size with a per-width cost line (off by default) 2026-10-21 13:24:21 +02:00
Niko1221 d1dec64362 verify: STRATA_TAP_FILE, an env-gated dump of the hidden states at chosen layers per verified row (DFlash2 acceptance experiment; off by default, no new graph nodes without it) 2026-10-21 13:24:21 +02:00
Niko1221 011c9e600a README: RX 6000 cards get their own engine package in 0.2.2 (no installer of their own; built, not tested on a real card) 2026-10-21 11:07:39 +02:00
Niko1221 2355265fdc dsv41: pack_gguf --tq-layers (a subset of the converted layers) 2026-10-21 08:24:45 +02:00
Niko1221 76f90e5843 Fix #3425: the memory tiers count RAM + VRAM less 5 GB that Windows and the system keep (setup.py and tools/strata_mcp.py, one constant each; the app and strata launch read the engine's pick; the Mac keeps its own macOS share) 2026-10-21 07:03:33 +02:00
Niko1221 328a3f5bd2 Gyro-M text: 16-21% slower than Q2_0 everywhere (setup's model list, strata_mcp, GYRO.md intro), as README and MODELS.md say 2026-10-21 05:04:54 +02:00
Niko1221 1df3a3108b Merge f-022-int (68864c564) into f-022-gyrocuda: release notes keep both sides 2026-10-21 05:03:36 +02:00
Niko1221 daa654bd35 Gyro docs: the 0.2.2 decode numbers on the pinned files (RTX 3060, 5 pairs, merged tree against f-022-int)
Gyro-S 54.3 -> 62.5 tok/s (+15.2%), Gyro-M 50.2 -> 54.8 (+9.1%), Q2_0 66.4 -> 67.1 (untouched); against Q2_0: Gyro-S 16% -> 7% slower, Gyro-M 23% -> 16%; prompts unchanged.
2026-10-21 05:03:12 +02:00
Niko1221 68864c564e Merge f-022-specfix: first-window HIP verify hang (PLE flag before the first launch of each window graph) + 1-token prompt starts the speculative loop 2026-10-21 03:48:34 +02:00
Niko1221 2c27afec08 docs/GYRO.md: Gyro-M was 24% slower than Q2_0 before 0.2.2 (the measured table says 24) 2026-10-21 02:57:31 +02:00
Niko1221 1c9bfd2a6e Version bump for 0.2.2 (proposed): app 0.2.2, engine and Control 0.1.47
Cargo.toml and Cargo.lock, desktop and host package.json, PRODUCT_VERSION, CMake project and Metal STRATA_VERSION, build_hip_linux PKGVER default, About comment, tests that pin the number, Docker image tags and docs, download names in the READMEs, AMD_HIP.md and UPDATES.md.
2026-10-21 02:57:22 +02:00
Niko1221 be4e24d421 Fix the gfx906 query-swizzle host guard test: the dispatch expression now spans two lines and reads kernel 2026-10-21 02:57:22 +02:00
Niko1221 3a7a6955cf docs: release notes 0.2.2 cover every merged stream (GPU roles, Gyro on Arc/Mac/NVIDIA/P100, models folder, NUMA, RX 6000, API key, own files, speed definition, fixes), what 0.2.1.1-0.2.1.4 already shipped, known limits 2026-10-21 02:26:58 +02:00
Niko1221 4f864bcd2d docs: translated READMEs give Gyro-M's decode gap to Q2_0 as 16-21% (RTX 3060 12 GB: 16%, R9700: 21%), like the English README 2026-10-21 02:22:24 +02:00
Niko1221 2c8eb88196 Merge f-022-gyrocuda: Gyro trellis kernels faster on NVIDIA (8-lane rows on every card above Pascal): Gyro-S +17%, Gyro-M +11% on an RTX 3060, identical answers 2026-10-21 02:20:43 +02:00
Niko1221 e0cfb80f53 Merge f-022-engfix: #3465 NaN retry reads the prompt batched, #3441/#3075 tool-call envelopes, #3503 cuda12 package keeps Pascal/Volta, #3480 queue priming respects GPU_MAX_HW_QUEUES 2026-10-21 01:48:02 +02:00
Niko1221 86d10ed8f9 verify (HIP): the first launch of each window graph reads the PLE rows and raises the PLE flag before cudaGraphLaunch, so the launch cannot wait for a GPU that waits for the flag (first-window hang on gfx1151; STRATA_VERIFY_EARLY_PLE_FLAG=0/1/2) 2026-10-21 01:38:57 +02:00
Niko1221 1c2388876c dsv41: trellis gate/up fused in one pass sharing the activation fragments (token path 454 -> ~355 us a layer, outputs unchanged: ds4_tq_test ALL OK); optional 32-bit stream loads (no gain) 2026-10-21 01:30:01 +02:00
Niko1221 01f8c32559 Merge f-022-int into f-022-gyrocuda: Pascal path kept for cc 6.0, 8-lane trellis rows for every other CUDA card
iq_kernels.cu is f-022-int's (staged codebook, row chunks and staged activations on cc 6.0; STRATA_TQ_STAGE). On top of it, tq_sub8_on()
(CUDA, not cc 6.0, STRATA_TQ_STAGE / STRATA_TQ_SUB8 not 0) turns on the staged codebook and the staged 8-lane down rows for the other cards
and adds the 8-lane 160-dot gate/up rows (row_dot_160_sub8). RTX 3060, Gyro-M, 24 groups, 1 token: gate/up 142 us (the cc 6.0 path forced:
158, off: 225), down 77 (off: 184); the hashes of the outputs are equal in all three modes.
2026-10-21 01:16:12 +02:00
Niko1221 894f133b7d Fix #3465: after non-finite logits in a prompt window the retry reads the whole prompt batched
The reporter's prompts ended in non-finite logits in the window that reads the prompt's last few tokens (after the last turn boundary), every time and on every retry, and passed with --short-read 0. The cause on their machine (a Windows VM, sm_89, a Broadwell AVX2 CPU, IQ3_S) was not reproduced on a P100 with a Broadwell Xeon, an RTX 3060 or a Core Ultra.
The engine now reads the request that follows such a window with --short-read 0 for that one request, so the server's transparent retry succeeds instead of failing the same way. Nothing changes until a prompt window reports non-finite logits. STRATA_NAN_INJECT=1 exercises it.
2026-10-21 00:52:29 +02:00
Niko1221 8a4106a47e suffix drafter: randomized soundness test (small vocabularies, tiny tables, propose_after), clean under ASan/UBSan 2026-10-21 00:41:25 +02:00
Niko1221 4c3edf4765 generate: spec_loop_runs() names the speculative loop's start rule (+ test); STRATA_SPEC_WATCH=<s>, a stall watchdog with the stage names for the CLI's verify loop (stall report and thread dump, off by default) 2026-10-21 00:41:25 +02:00
Niko1221 f05725c332 Fix #3480: the HIP queue priming stands aside when GPU_MAX_HW_QUEUES is 1 or 2
With the runtime capped to two hardware queues the two primed queues are all of them, and the high-priority draft stream and the draft side stream share them: a 4K prompt on an RX 9060 XT read 681-753 tok/s primed and 1188-1238 unprimed. Primed only when the cap is 3 or more or unset; STRATA_HIP_QUEUE_PRIME=nn still forces it. HIP code only (gfx12, Linux).
2026-10-21 00:37:50 +02:00
Niko1221 e8cb1cba7f dsv41: tq_convert resumes (skips finished layers), pack_gguf --tq-experts takes several folders 2026-10-21 00:26:31 +02:00
Niko1221 8de1ec4165 Test: torch has one pin per Python range since #3474; compare the package set 2026-10-21 00:23:52 +02:00
Niko1221 3fbcea2f79 Merge f-022-issues2: #3488 #3502 #3490 #3507 #3424 #3501 #3504 #3425 #3469 #3437 #3510 2026-10-21 00:23:39 +02:00
Niko1221 dc16e3a7cb Fix #3510: on a card that shares the PC memory (Strix Halo) the Work profile no longer reserves a fifth of the shared pool as graphics memory 2026-10-21 00:23:09 +02:00
Niko1221 5cdb59cfa9 Fix #3469, #3437: UPDATE.bat and setup fetch the engine from the signed update feed when the release has no flat zip, and the minimum engine is 0.1.46 2026-10-21 00:21:24 +02:00
Niko1221 0cf83f610a Merge main (0.2.1.4 Terminal update: RX 9000 engine choice, download checks); engine choice keeps the RX 6000 tie-break 2026-10-21 00:20:01 +02:00
Niko1221 18b4affb87 Fix #3425: the recommendation says how much of the picked size would be read from the SSD when RAM + VRAM is only just the tier's floor 2026-10-21 00:16:23 +02:00
Niko1221 aae92ceb26 Gyro: decode speed on NVIDIA (docs, 0.2.2 notes), Gyro-M's slowdown against Q2_0 16-21%
GYRO.md gets the measured RTX 3060 numbers of the shared-memory codebook / 8-lane row kernels (Gyro-S 53.6 -> 62.8 tok/s, Gyro-M 50.6 -> 56.1, 5 interleaved pairs, identity gate identical), the release notes one line, and the places that said Gyro-M decodes about 20% slower than Q2_0 now say 16-21% (RTX 3060 12 GB after the kernels: 16%, R9700: 17-21%). The translated READMEs keep the old figure.
2026-10-21 00:14:08 +02:00
Niko1221 926e7a38b8 Merge f-022-uma-linux: Strix Halo + discrete card on Linux (no exit with PF_PAD, iGPU never last stage, UMA-aware expert cache, picture reader on a card with its own memory, nicer engine on the display chip); ports PR #3484 by John-Pi 2026-10-21 00:13:34 +02:00
Niko1221 366c8dbaae Fix #3503: the Linux cuda12 engine package carries sm_60/61/70; the source Dockerfile builds it
packaging/build_cuda_linux.sh is the recipe for the Linux cuda13 and cuda12 packages (cuda12: CUDA 12.9, 60;61;70;75;80;86;89, STRATA_EXPERIMENTAL_SM60=ON, driver 575.51). package_engine.py refuses a cuda12 package without sm_60, sm_61 or sm_70, so the 0.2.1 mistake (sm_75/86/89 only, a P100 container in a restart loop) cannot ship again.
The root Dockerfile takes the CUDA image from CUDA_IMAGE: the CUDA 12 case turns on the Pascal/Volta build when an architecture below 75 is listed, names the package cuda12 with the matching driver, and the terminal stage has git (package_terminal.py calls it).
2026-10-21 00:13:14 +02:00
Niko1221 70f4240728 Fix #3441, #3075: a drifted tool-call opener is read as the call; a call for a tool the request did not declare stays text
A <tool_call> whose opener drops the word function (<=NAME>, <function:NAME>, <NAME>) is read as <function=NAME> when NAME is a declared tool.
The answer follows the rule the reasoning already had (#804): a call counts only for a declared tool. Chat Completions and /v1/messages requests that declared no tools get the text back; the Responses route keeps its Codex compaction behaviour.
Blocks that open like a call but do not read as one are counted in the server window.
2026-10-21 00:08:55 +02:00
Niko1221 30a67051d0 setup: say when the BIOS gives a big carve-out to a Strix Halo this run does not use (the OS loses that RAM; the experts and the desktop need it) 2026-10-21 00:02:32 +02:00
Niko1221 b8b381a3b9 Fix #3504: a two-value option (control-vector-layer-range) is written and removed as two argv entries, and an empty Advanced value is no longer a bare flag 2026-10-21 00:02:01 +02:00
Niko1221 25d025fef8 docs: Strix Halo with a card of its own on Linux (RAM counted once, split order, picture reader, nice) 2026-10-21 00:00:22 +02:00
Niko1221 8895949ef7 gpu_fit: a chip that shares the system memory adds only its BIOS carve-out beside the RAM (a Strix Halo model was counted as VRAM and as RAM) 2026-10-21 00:00:22 +02:00
Niko1221 0da82e740c prefill gemm: skip the padded dequant on a device that has no padded product (a gfx12 stage of a gfx1151 + gfx1201 split)
After #3484 the process-wide STRATA_PF_PAD can be on for a gfx12 stage, which then dequantized every weight padded, found no product and dequantized it
again unpadded before the copy path. Ask strata_pf_padded_gemm_available() for the current device first; same products, same bits.
2026-10-21 00:00:22 +02:00
TheHubUser4860938 95cfbd4cc4 Fix #3501: in_trash splits backslashes, so a Windows Recycle Bin path is found on Linux too 2026-10-21 00:00:02 +02:00
Niko1221 c8b1436022 Fix #3507, #3424: first-run 'Use your own GGUF' goes to the local GGUF page instead of making the folder the data folder (which downloaded the files again); preparing own files never uses their folder as data_dir 2026-10-20 23:59:31 +02:00
Niko1221 9de04cd44e Control 0.1.46.3: version bump for the Terminal/Control-only update (engine packages unchanged) 2026-10-20 23:56:06 +02:00
Niko1221 31ca248fbe Progress words: a file being repaired after a wrong SHA-256 says so in plain words
Python only (strata_control/progress_words.py, test).
2026-10-20 23:55:59 +02:00
Niko1221 63214a5655 Downloads: every ranged answer is checked, a file with a wrong SHA-256 is repaired piece by piece instead of deleted
A 39.8 GB shard came out with the wrong SHA-256 on 0.2.1 and was deleted, 40 GB of download lost.

- every ranged answer must be a 206 with exactly the Content-Range asked for (start, end and total), a fitting
  Content-Length and the same ETag as the first answer; a changed file part-way stops the download and drops the .part
- the raw unbuffered write is looped until every byte is down (its return value was ignored)
- a .part records the address and ETag it came from (.part.src); one from another mirror or revision is dropped, not mixed
- a parallel download always has a piece list before the first byte is written; if none can be kept it uses one stream,
  so holes in the sparse file are never read as a finished prefix
- the one-stream resume accepts a 206 only at the byte asked for; another range restarts from zero
- a finished file with a pinned hash but no finish mark is checked, not marked; the rename waits for a virus scanner
- a SHA-256 mismatch re-reads the file from the server piece by piece, rewrites only the blocks that differ and checks
  again ("Some parts of the file were damaged; downloading only those parts again"); deleted only if still wrong
- update archives (strata_control/transfer.py): a resumed answer is appended only when its Content-Range starts at the byte asked for
- tests: tests/terminal/test_download_integrity.py (a local server that answers 200 for ranges, wrong ranges and totals,
  cut bodies, another ETag, 503s, flipped bytes)

Python only (setup.py, strata_control/transfer.py, tests).
2026-10-20 23:55:59 +02:00
Niko1221 86620837b2 Updates: an AMD card with an unknown gfx never gets the newest HIP build by version; read the gfx from the Windows display adapters, choose by GPU name, keep the installed variant 2026-10-20 23:55:36 +02:00
Niko1221 75d0d0b966 Fix #3490: tuning stops waiting for memory when none is being freed instead of ten minutes per candidate, and says how much is free and needed 2026-10-20 23:55:11 +02:00
Niko1221 6515aebac2 Fix #3502: a CUDA-init failure in a tuning run is not recorded as OOM (the memory guard info line no longer decides), and a command that passed before is not blacklisted by a later failure 2026-10-20 23:55:11 +02:00
Niko1221 f801953992 Linux integrated GPU: the auto expert cache counts the RAM once (cache <= RAM available - desktop reserve); the APU warning uses the same reserve
A Strix Halo's free VRAM is the GPU pool (carve-out + GTT, nearly all RAM with ttm.pages_limit raised) while the cache copies experts into that
same RAM on top of the arena, so a cache sized from the pool alone could ask for more than the OS had left (OOM killer). The auto size is now
min(GPU free, MemAvailable - max(8 GiB, 8% of RAM)); --expert-cache N is kept as asked, STRATA_UMA_RESERVE_GIB=N sets the reserve (0 = old sizing).
2026-10-20 23:53:10 +02:00
Niko1221 53752401f2 Fix #3488: a reused PID in a child marker without a start time no longer blocks every load; the engine hash check reads twice before refusing 2026-10-20 23:53:07 +02:00
johnpi ab61f892d8 prefill: a padded activation is gfx11 only, and Gemm::native copies it unpadded instead of ending the engine
STRATA_PF_PAD=1 STRATA_PF_GEMM=1 lets the two K 6144 producers write row stride 6,208 into the prompt GEMM, but the only
product that reads a padded stride is strata_pf_gemm_f16_ld, which is built into wmma_gemm.cu's gfx11 matrix-core block
and returns false everywhere else.  On an RDNA4 card Gemm::native then reached its "a padded X (ldx 6208, K 6144) needs
STRATA_PF_PAD's path" branch and called std::exit(1): under serve/ that is one engine restart per request.  Reproduced
on an RX 9070 (gfx1201, ROCm 7.2.4, kernel 7.2.11) at --vram-reserve-mib 700 and 1,536, so it was not the VRAM guard.

- wmma_gemm.cu/.h: strata_pf_padded_gemm_available() answers whether the padded product exists on the current device
  (the same gfx11 list the kernels are compiled for; false in a build with no gfx11 target).
- prefill.cpp: pf_pad() asks it before padding, and prints one line when the env asks for a product this card has not.
- gemm.cu/.hpp: a padded X that no kernel took is copied to a contiguous FP16 buffer (one cudaMemcpy2DAsync, one
  grow-only buffer per Gemm) and the ordinary paths run - the same products, the same bits.  exit(1) remains only for a
  failed allocation, with a message that says so.

Checked on the RX 9070, same 1,199-token prompt and 8 greedy tokens in every arm:
- before (0.1.45, pre-fix binary): `prefill gemm: a padded X (ldx 6208, K 6144) needs STRATA_PF_PAD's path`, exit 1.
- after, STRATA_PF_PAD=1 STRATA_PF_GEMM=1: `prefill gemm: STRATA_PF_PAD needs the padded gfx11 product, which this card
  does not run; not padding`, prompt and decode complete, exit 0, output identical to the unpadded run.
- after, with the gate forced open so that the producers really pad and Gemm::native takes the copy path: no exit, no
  "needs STRATA_PF_PAD's path", output identical to the unpadded run (13 271 248046 198 248045 74455 198 248068).
  With the gate in place STRATA_PF_GEMM=1 alone is inert on this card (4K 1,247 against 1,243 tok/s, 8K 1,330 against
  1,321, medians of 3 starts).

docs/AMD_HIP.md carries the RDNA4 bullet for it.  Built for gfx1201 (RX 9070, ROCm 7.2.4, Release): 338 of 338; ctest
77 of 79, the two failures (ple_parity without the Q2_0 GGUF here, hip_prefill_mmq_parity's APR block reading
ggml_type_name(145) with GGML_TYPE_COUNT 43) fail identically on origin/main.  The changed files are shared with the
CUDA build and were not compiled for CUDA here (no nvcc on this machine); SYCL is untouched.

(cherry picked from commit 341625013d)
2026-10-20 23:51:15 +02:00
Niko1221 034422c4d3 AMD: built-in Radeon graphics (Strix Halo) is never the last stage of a layer split, the picture reader leaves it, --gpus all leaves it out beside two cards, engine nice +5 on Linux with such a chip, split note
The #1576 memory guard compared the iGPU's BIOS carve-out with the dGPU's VRAM and could keep the iGPU as the last stage (head, draft layer, verify).
The picture reader inherited the engine's first visible card, the iGPU. setup --gpus all took every usable card including the APU.
All three are recommendations: a number list, gpu_order as_given, a manual layer split and vision.cuda_device are kept as asked.
2026-10-20 23:50:19 +02:00
Niko1221 aece46db89 generate: --spec on a native pack starts the speculative loop for a 1-token prompt (spec_pos -1 = unused, 0 is a real position) 2026-10-20 23:41:24 +02:00
Niko1221 da70196350 dsv41: tq_convert --act (input-column weighted encode from the gate-3 calibration sums) 2026-10-20 23:24:48 +02:00
Niko1221 88fcf2b443 dsv41: engine-kernel bench in ds4_tq_test (--bench) 2026-10-20 23:21:41 +02:00
Niko1221 12a3c0739e dsv41: portable (FMA) tile test target, docs for the trellis experts 2026-10-20 22:31:25 +02:00
Niko1221 1e7bcb8b6e Merge f-022-cores: start-up line lists the CPU worker processors, experimental --pool-cores, thread names, Windows no-topology fallback 2026-10-20 22:15:28 +02:00
Niko1221 7df82763e3 dsv41: tq_convert.py (re-encode a pack's MXFP4 experts with the trellis encoder into the engine's file layout) and pack_gguf --tq-experts 2026-10-20 21:52:59 +02:00
Niko1221 e5b4021c6f dsv41: trellis routed experts end to end on the device and the CPU lane (TQK6/7, TQK5X/4X computed, TQK5T/4T table): ds4_tq.cu (gfx12 WMMA + portable FMA tiles over the file layout), ds4_tq_cpu.cpp (Strata tq_cpu kernels), iq entry points route the types; test ds4_tq_test (all six types pass: device and CPU lane within 1e-6 of a double reference) 2026-10-20 21:49:29 +02:00
Niko1221 c5eb2dd923 docs: reorder the placement section (threads that are not pinned, then the numbers) 2026-10-20 21:46:39 +02:00
Niko1221 88942998c5 Merge f-022-pascaltq: Gyro trellis kernels about 30% faster on Pascal (sm_60), other cards unchanged 2026-10-20 21:44:40 +02:00
Niko1221 0255581677 Release notes 0.2.2: Gyro on a Tesla P100 2026-10-20 21:43:27 +02:00
Niko1221 9b5b758358 tq_pascal_bench: other expert geometries (a longer hidden row, shorter experts) against the per-column kernels 2026-10-20 21:43:03 +02:00
Niko1221 29c811217f Merge f-022-dlintegrity: every ranged answer is checked (206, Content-Range, length, ETag), full writes, safe resume, and a SHA mismatch repairs only the differing parts instead of deleting the file 2026-10-20 21:29:41 +02:00
Niko1221 5f81c5dc47 Gyro decode (CUDA): trellis codebook in shared memory, 8-lane rows for the 160-dot gate/up and the 40-dot down
The int8 codebook lookups of tq_load went through the global table: a warp's 32 random 2-byte reads touched ~20 L1 sectors each
(native_expert_bench, ncu: 51M sectors for 2.8M requests, L1 64% busy). The grouped gate/up and down kernels now stage the 4 KB
codebook per block (stage_iq_grid, STRATA_IQ_STAGE_GRID=0 reads it in place) and take 4 rows per warp: gate/up K=160 (new
row_dot_160_sub8) and the K=40 down (row_dot_40_sub8, as IQ4_NL), 32 rows per block so the staging is amortised and no lane idles.
The sums keep row_dot_multi's tree (lane sums of t, t+8, t+16, t+24, then the same xor steps), so the outputs are bit for bit the
old ones: native_expert_bench prints an FNV hash of the outputs now, equal for base and new on Gyro-S, Gyro-M and Q2_0 blobs.
RTX 3060, 24 groups, 1 token: Gyro-M gate/up 221 -> 140 us, down 182 -> 86 us; Gyro-S 228 -> 157 and 173 -> 89.
2026-10-20 21:24:27 +02:00
Niko1221 cbf7f1d9f9 Test: Gyro-S and Gyro-M are in the Mac model list now (Mac Gyro support) 2026-10-20 21:24:07 +02:00
Niko1221 d6a6c1c797 Merge f-dsv41-enc (trellis encoder, K4/K5 computed-code types 224/225, table versions 226/227) into f-dsv41-port; conflicting trellis files taken from f-dsv41-enc 2026-10-20 21:23:42 +02:00
Niko1221 4e83ef04d5 Merge f-022-fitauto: a model just short of fitting gets a smaller context / reserve / prompt chunk automatically (Gyro-M on 8 GB Arc) 2026-10-20 21:23:40 +02:00
Niko1221 34d7b2a3ba Fit rule for cards that are just short: lower context, reserve and prompt chunk until the start fits (Gyro-M on the 8 GB Arc A750)
strata_control/vram_fit.py estimates the VRAM left for the expert cache (calibrated on 12 A750 starts) and walks the ladder 32K -> 16K -> 8K, prompt chunk 1024, reserve down to 100 MiB. Used by sycl/setup_intel.py (setup), strata launch's recommended context and the adopt path. A config that already fits is returned unchanged; values the user chose are kept; it never refuses.
2026-10-20 21:23:25 +02:00
Niko1221 a5f54c9796 dsv41: test includes point at src/dsv41 2026-10-20 21:02:33 +02:00
Niko1221 31fa54fb68 Merge f-022-macgyro: Gyro-S and Gyro-M on Apple Silicon (Metal trellis kernels, Gyro dense tiles) 2026-10-20 20:57:36 +02:00
Niko1221 bc3f293861 Docs: Gyro-S on the Mac against the CUDA engine (KL at the CUDA noise level) 2026-10-20 20:57:15 +02:00
Niko1221 21b0fdb0d4 Merge f-022-int into f-022-macgyro: Gyro is listed as supported on Intel Arc and on the Mac (docs, seven READMEs, release notes keep both bullets) 2026-10-20 20:55:53 +02:00
Niko1221 564d0abebe Merge f-022-arcgyro: Gyro-S and Gyro-M on Intel Arc (SYCL trellis kernels, Hadamard, XMX prompt path) 2026-10-20 20:53:18 +02:00
Niko1221 413544b6d9 Merge f-022-int (e28889d12) into f-022-arcgyro 2026-10-20 20:51:47 +02:00
Niko1221 634e5826a5 docs: Gyro on Arc, pinned file and CUDA 1.4K-prompt check 2026-10-20 20:51:41 +02:00
Niko1221 c4f1e05dbd --pool-cores: a list of more than 4096 processors is refused by the parser (with a message; the pool then keeps its default placement) 2026-10-20 20:42:25 +02:00
Niko1221 e38025769d dsv41: picture test (mkimg.py) and serve_prompts GENI / text output 2026-10-20 20:40:41 +02:00
Niko1221 e2b4464917 CPU pool: read the logical-processor layout when the pool is built
The start-up line and the --pool-cores check used the layout as seen by whichever thread asked, and on Linux a thread that was
already pinned (the host, after the session pins it) sees only its own CPU. The layout is now read in the constructor.
2026-10-20 20:24:00 +02:00
Niko1221 10edc50f05 start-up: no 'workers by default' line when --pool-cores gave the list (its own line says where they run) 2026-10-20 20:20:12 +02:00
Niko1221 f353087104 docs: Gyro on Intel Arc (GYRO.md, INTEL.md, release notes 0.2.2) 2026-10-20 20:14:39 +02:00
Niko1221 79946b9c4f Gyro on Intel Arc: README and MODELS say so, ctest cases for the trellis kernels, setup's Gyro memory warning reads the real RAM on the Intel path 2026-10-20 20:13:49 +02:00
Niko1221 3d8d1079bc dsv41: gfx12 WMMA trellis GEMV (ds4_tqw.cuh, from the CUDA lab's recipe) with a parity + bench test: all K4-K7, widths 1/3/5, table and computed codes within 8e-8 of the double reference on the R9700; ~1.07 Tw/s 2026-10-20 20:09:32 +02:00
Niko1221 6c8247af60 Merge f-dsv41-serve into f-dsv41-port 2026-10-20 20:03:45 +02:00
Niko1221 4cb96acd81 docs: the CPU picture encoder stays where the OS puts it (pinning it one per core was 7 to 19 times slower on a 16-core Windows PC) 2026-10-20 20:01:56 +02:00
Niko1221 2713121725 trellis: computed codebook on the GPU encoder, README for the K5/K4 versions 2026-10-20 19:59:48 +02:00
Niko1221 ca40c88ef4 Docs: Gyro on the Mac (MAC.md, GYRO.md, MODELS.md, README, 0.2.2 release notes) 2026-10-20 19:51:16 +02:00
Niko1221 e28889d120 Merge f-022-gpuroles: Main / Helper / Off per graphics card in the app, TUI, launch, setup, server and Docker 2026-10-20 19:48:30 +02:00
Niko1221 890fee81b0 GPU roles: two identical cards recommend the highest number as Main, so the engine order stays [0, 1]; a model that keeps one card keeps setup's old pick; golden back to [0, 1] with roles only 2026-10-20 19:48:20 +02:00
Niko1221 5e633b4ff7 Metal: Gyro dense K-quant GEMV tiles (+5% decode, 5/5 pairs, same output), GYRO.md supports the Mac 2026-10-20 19:39:16 +02:00
Niko1221 c11e66aa8b tqe_table takes a type 2026-10-20 19:37:54 +02:00
Niko1221 7c11a196d5 Server: the card roles (and a stop when every card is Off) come before anything starts 2026-10-20 19:31:34 +02:00
Niko1221 9fcc101844 docs: DEEPSEEK41.md (where the model goes, install, the API) 2026-10-20 19:26:21 +02:00
Niko1221 a7c50df002 K5/K4 version 2: computed (LUT-free) codebook for TQK5X/TQK4X, point-table versions kept as TQK5T/TQK4T; CPU decoder tables per type; tests 2026-10-20 19:25:16 +02:00
Niko1221 a4c9336416 serve: engine_ready_s in the run config (DeepSeek-V4.1-Flash reads 300 GB at start; setup writes 3600) 2026-10-20 19:23:44 +02:00
Niko1221 2189884a59 dsv41: DS4_PROF events per device (invalid resource handle on several GPUs), exl3_legacy.cu in CUDA builds (reported by the CUDA lab) 2026-10-20 19:22:51 +02:00
Niko1221 f7c645c065 DeepSeek-V4.1-Flash in the catalog, setup and strata launch
- setup.py: the nine pinned shards of unsloth/DeepSeek-V4.1-Flash-1017-GGUF UD-Q8_K_XL (revision f924b94f, sizes and SHA-256 from the reference run; the DSpark drafter and picture reader pinned but not fetched), MULTI_FILE_FAMILIES, install_dsv41: plain warnings for the 508 GB download, RAM and disk (a refusal only where the disk cannot hold the files), the strata-dsv41 check before any download, --gguf-dir for files you already have, pack_gguf.py, config with --pack --max-context 8192 --eos 1 --tp 2 and the cards; menu entry and a --check line.
- strata_control/dsv41_fit.py: where the model goes on the cards and in RAM, from the measured run (6,240 experts in graphics memory predicted on 4 x 32 GB, 6,197 measured), the n-gram table's SSD note.
- tools/strata_mcp.py and strata_cli/launch: the model is in the list; strata launch --plan prints the placement (layer blocks per card, experts on the cards and on the CPU, RAM, disk, the SSD note); --model-path takes the nine shards where they are; parallel, pictures and tuning are said, not passed to an engine that does not take them.
- tests: tools/test_setup_dsv41.py (22), tests/terminal/test_launch_dsv41.py (13).
2026-10-20 19:22:11 +02:00
Niko1221 8bf77c0755 SYCL Gyro: trellis packs take the link share from a 12 GB/s reference (A750: 0.29 -> 0.48, decode 13.7 -> 15.8 tok/s, 5/5 pairs); Gyro memory warnings on an Arc read the real RAM 2026-10-20 19:21:44 +02:00
Niko1221 8c41d3611c Thread names: the Windows half lives in pool.cpp, not in a header every engine file includes
thread_name.hpp pulled <windows.h> (and WIN32_LEAN_AND_MEAN) into every file that includes aux_cpus.hpp; on Windows the function is now
defined in pool.cpp, which already includes it. The PLE reader and the direct file readers are not named (they do not link the pool).
2026-10-20 19:20:00 +02:00
Niko1221 4455c8bd1d tqenc.py: GPU lock wait with CPU fallback 2026-10-20 19:18:57 +02:00
Niko1221 2aa1ec2d56 serve, launch: "pool_cores" in the run config and strata launch --pool-cores
Both pass --pool-cores to the engine (experimental, off by default). A list that cannot be read is said and left out.
strata launch also shows the notices that the flags themselves produce (they were dropped before).
2026-10-20 19:16:17 +02:00
Niko1221 7a55995818 CPU pool: say where the workers run, --pool-cores (experimental), names for the engine's threads
The start-up now has one plain line: CPU workers on logical processors 2,4,...,30 (15 workers, one per physical core; host on 0).
It says so when two workers share a core, when the host's core is taken, and when the OS did not say which processors are
siblings. --pool-cores LIST / STRATA_POOL_CORES pins the workers to exactly the listed logical processors (off by default;
siblings, the host's processor and processors the process may not use are warned about, never refused). The default placement
is unchanged. The pool workers, the host thread and the long-lived helper threads get names (strata-pool-N, strata-host,
strata-stage, strata-adapt, ...), so ps -L, /proc and Windows thread tools show which is which. Windows without core
information no longer counts every logical processor as a core when it picks the default worker count.
2026-10-20 19:16:15 +02:00
Niko1221 82a97251ea Merge main (0.2.1.3 Terminal update); docs keep the 0.2.2 text 2026-10-20 19:13:58 +02:00
Niko1221 cbfd5b926b Merge main (0.2.1.2 engine hotfix) into the 0.2.1.3 Terminal update 2026-10-20 19:12:29 +02:00
Niko1221 2e58e320b5 MCP config description names gpu_roles 2026-10-20 19:05:28 +02:00
Niko1221 301ab7c7b4 strata launch: --gpu-roles summary line in the plan and before the start; tests for the flag, the mirror and installed roles 2026-10-20 19:04:17 +02:00
Niko1221 0fce78c1c3 Docs: Main, Helper, Off in MULTI_GPU.md (the main card is the last stage), Docker GPU_ROLES, wording fixes 2026-10-20 19:03:27 +02:00
Niko1221 095e524fca App: GPU roles in the generic Run screen (two or more cards), preview fixture can name its cards 2026-10-20 19:02:06 +02:00
Niko1221 88278af79a Gyro trellis gate/up on compute capability 6.0: the group's activations in shared memory for the block's row chunks 2026-10-20 19:00:14 +02:00
Niko1221 c864d804b6 Gyro trellis experts on compute capability 6.0: row chunks per block so the staged codebook is paid once per several rows; bench: rows of different lengths 2026-10-20 19:00:14 +02:00
Niko1221 51f418f15a Gyro trellis experts on compute capability 6.0: read-only loads for the block words, TQK6 needs three words of the stream 2026-10-20 19:00:14 +02:00
Niko1221 9f38ee338c Gyro trellis experts on compute capability 6.0: the int8 codebook staged in shared memory, the TQ down rows as 8-lane rows (bitwise the same sums) 2026-10-20 19:00:14 +02:00
Niko1221 b2f2149431 tq_pascal_bench: the Gyro trellis decode kernels on synthetic blocks (bitwise vs the per-column kernels, output checksums, --bench) 2026-10-20 19:00:14 +02:00
Niko1221 ed61ad2a9c App: GPU roles section (Main / Helper / Off per card) replaces the card option list and the expert Run on picker; host writes roles plus the legacy mirror; 46 strings in 8 locales 2026-10-20 18:58:30 +02:00
Niko1221 06e614017d Merge f-022-int (b1594fbbe) into f-022-arcgyro 2026-10-20 18:57:46 +02:00
Niko1221 2411e7cd3b SYCL Gyro: decode-once multi-entry trellis kernels, the grouped XMX prompt path takes the trellis types (prompt 40 -> 82 tok/s on the A750), tq_sycl_parity --bench; setup keeps Gyro's n-gram table on the SSD on i915 2026-10-20 18:57:15 +02:00
Niko1221 d9220bd1a5 Merge f-dsv41-cudalab (trellis GEMV lab, tqenc) into f-dsv41-port 2026-10-20 18:54:44 +02:00
Niko1221 8cfe44b71d GPU roles in strata launch (--gpu-roles, M/H/O/R/D Cards screen), setup (--gpu-roles, one roles question) and the host's TypeScript/JS rules; golden and setup tests follow the new defaults 2026-10-20 18:52:30 +02:00
Niko1221 1b47c1d076 Metal: Gyro read-ahead skips the n-gram table, Q6_K/IQ4_NL hyper-connection matrices go through BF16 before their Q8_0 copy (as the pack does for CUDA) 2026-10-20 18:51:27 +02:00
Niko1221 b1594fbbe5 Merge main (0.2.1.2 Strix Halo engine hotfix) 2026-10-20 18:46:37 +02:00
Niko1221 d251b2c0db dsv41 cudalab: gfx12 WMMA port recipe of the trellis GEMV: host emulation of the data flow (matches the reference), HIP kernel source, layout probe 2026-10-20 18:45:22 +02:00
Niko1221 5a52007005 serve: DeepSeek-V4.1 through the server (deepseek-v3 pre-tokenizer, DSML tool calls, effort words, end token, card sampling)
- tools/strata_tokenizer.py: the deepseek-v3 pre-tokenizer (llama.cpp's DEEPSEEK3_LLM list as a sequence of splits) chosen by the pack's tokenizer.json 'pre'; special tokens decode to their literal text (the DSML token is not in the byte alphabet). 20 of 20 reference prompts tokenize to the ids llama.cpp made.
- serve/dsml.py: DSML calls (the GGUF template's spelling) translated as they stream into the form OutputParser reads, so tool streaming, tool_choice and calls in the thinking work as for the other families; string=true values stay strings.
- serve/server.py: Service.set_family('deepseek41'): end token from the pack, DsmlOutputParser, forced-call opening in DSML, effort words low/medium/high/max/1..100, model name and card sampling.
- serve/test_dsv41.py (39 tests, fake engine) and fixtures from the GGUF's own template and the llama.cpp reference prompts.
2026-10-20 18:45:21 +02:00
Niko1221 4dd62738df Progress words: a file being repaired after a wrong SHA-256 says so in plain words
Python only (strata_control/progress_words.py, test).
2026-10-20 18:41:45 +02:00
Niko1221 6f703a6dbe dsv41 cudalab: computed (LUT-free) codebooks for the 15-bit trellis (1mad, 3inst, 3sum, 3single), encoder hook TQE_CODE, LUT-free GEMV kernel, code SQNR tool, pilot scripts from f-dsv41-enc 2026-10-20 18:41:32 +02:00
Niko1221 f6b4101127 metal/tests/gyro_parity.sh: the Gyro kernel checks in one script 2026-10-20 18:40:57 +02:00
Niko1221 d581286885 Hardware report: speed per NVIDIA card (SMs x clock) for the Main-card pick 2026-10-20 18:39:38 +02:00
Niko1221 0b736f7702 Gyro on the Mac: model list tests, resident wrap span without the n-gram table, trellis gate/up tile 2026-10-20 18:37:04 +02:00
Niko1221 b6825a493c Server: gpu_roles (Main last, Off excluded, hand-edited gpu wins), roles log line, every-card-Off handling 2026-10-20 18:36:30 +02:00
Niko1221 c0b0869b03 Downloads: every ranged answer is checked, a file with a wrong SHA-256 is repaired piece by piece instead of deleted
A 39.8 GB shard came out with the wrong SHA-256 on 0.2.1 and was deleted, 40 GB of download lost.

- every ranged answer must be a 206 with exactly the Content-Range asked for (start, end and total), a fitting
  Content-Length and the same ETag as the first answer; a changed file part-way stops the download and drops the .part
- the raw unbuffered write is looped until every byte is down (its return value was ignored)
- a .part records the address and ETag it came from (.part.src); one from another mirror or revision is dropped, not mixed
- a parallel download always has a piece list before the first byte is written; if none can be kept it uses one stream,
  so holes in the sparse file are never read as a finished prefix
- the one-stream resume accepts a 206 only at the byte asked for; another range restarts from zero
- a finished file with a pinned hash but no finish mark is checked, not marked; the rename waits for a virus scanner
- a SHA-256 mismatch re-reads the file from the server piece by piece, rewrites only the blocks that differ and checks
  again ("Some parts of the file were damaged; downloading only those parts again"); deleted only if still wrong
- update archives (strata_control/transfer.py): a resumed answer is appended only when its Content-Range starts at the byte asked for
- tests: tests/terminal/test_download_integrity.py (a local server that answers 200 for ranges, wrong ranges and totals,
  cut bodies, another ETag, 503s, flipped bytes)

Python only (setup.py, strata_control/transfer.py, tests).
2026-10-20 18:33:45 +02:00
Niko1221 2219cc99bf GPU roles: one shared rule for Main / Helper / Off (recommend, set, order, mirror, warnings) with shared test vectors 2026-10-20 18:32:50 +02:00
Niko1221 befdd9d3e1 tqenc.py: optional per-call GPU locks (TQENC_GPU_LOCK) 2026-10-20 18:30:12 +02:00
beplee 4681762e3c setup: FP4-Local conversion installs torch 2.14.1 on Python 3.14+ (2.9.1 has no cp315 CPU wheel), older Pythons keep 2.9.1 (#3474) 2026-10-20 18:29:35 +02:00
Niko1221andmw00 2d718defc0 dsv41: pack_gguf takes unsloth's DSpark drafter GGUF (dflash) and the mmproj vision GGUF into the pack; the vision loader reads F32 rows and gate/up halves
Co-authored-by: mw00 <[email protected]>
2026-10-20 18:28:30 +02:00
Niko1221 6b93b42877 tests: Gyro-M per-tensor parity (--experts-first on a sparse copy), moe_mm_parity takes the trellis types 2026-10-20 18:28:01 +02:00
Niko1221 d3cb2f58f1 Merge f-022-gpu0: strata launch keeps the installed card choice; engine/nvidia-smi number map; hip_ordinal and image gpu list fixes 2026-10-20 18:27:52 +02:00
Niko1221 9bb1dd2044 Keep the card a person chose: strata launch no longer re-picks GPU 0, the engine's GPU 0 is explained, old hip_ordinal and list GPUs for images
- strata launch with no --gpu replaced the installed config's own card choice (setup --gpu 1, the app's pick, "gpus_asked") by the
  fit rule's pick, which is card 0 on equal cards. The installed choice and its order are kept; the Cards step preselects it.
- The server prints one line tying the engine's own numbering ("GPU 0 / CUDA0" in its log) to nvidia-smi's, when they differ.
- A card named by the app or by strata launch drops an old AMD config's hip_ordinal, which would have sent the engine to the old card.
- A list "gpu" in an image config is a device list ("1,0"), not "[1, 0]".
2026-10-20 18:27:36 +02:00
Niko1221 9f78a77e01 Merge f-022-int into f-dsv41-port 2026-10-20 18:22:49 +02:00
Niko1221 01389205e9 dsv41: split-KV attention opt-in (DS4_ATTN_SPLIT=1): A/B on 4x R9700 shows no decode gain, KL unchanged 2026-10-20 18:22:02 +02:00
Niko1221 c3472caf8e Metal: Gyro in the Mac setup and app model lists, the n-gram table stays out of the dense copy, decode-check test
stream.cpp: load_dense skipped nothing of the n-gram table because it was always another shard; in a Gyro
file it is the first 28.8 GB of the one file (the 32 GB allocation that failed on the Mac mini).
mac_fit / setup_mac / setup.py (app path) / strata_mcp: Gyro-S and Gyro-M as the 'gyro' family of the Mac
(one file, --compat-bf16 pack, --native and --ple-gguf the same file).
expert_parity --decode-check compares ref_quant's trellis decoder with the format's definition written out bit by bit.
2026-10-20 18:20:41 +02:00
Niko1221 a63b2d4dae dsv41: --score segment file really carries end_line 2026-10-20 18:16:03 +02:00
Niko1221 f1bfcc2253 SYCL Gyro: APR pack log line in generate, second-GPU helper declines Hadamard-folded experts 2026-10-20 18:01:04 +02:00
Niko1221 8e13982710 build_swap: ldl damping 0.3 2026-10-20 17:58:45 +02:00
Niko1221 a64f1dd99b Metal: Gyro engine message fix, expert_parity prints the pipelines' limits (PIPE_INFO) 2026-10-20 17:57:19 +02:00
Niko1221andmw00 ff5577ad70 dsv41: split-KV sparse attention from the CUDA lab for decode / short verify windows (packed rows, T<=5), DS4_ATTN_SPLIT=0 off
Co-authored-by: mw00 <[email protected]>
2026-10-20 17:54:04 +02:00
Niko1221 d17c32d6a3 dsv41: --score writes end_line in the segment file (quant_quality.py compare reads it) 2026-10-20 17:51:12 +02:00
Niko1221 1c20a59675 dsv41 cudalab: split-KV sparse attention (2.9x at T=1), CPU-kernel cross-check of the trellis GEMV, build for the repo CPU kernels 2026-10-20 17:47:29 +02:00
Niko1221 02791a6d18 SYCL: trellis experts (TQ2_T, TQK6, TQK7) for decode and the prompt path, Hadamard-folded inputs; tq_sycl_parity (host replay of the kernels bit for bit, dequantizers, Hadamard) 2026-10-20 17:41:12 +02:00
Niko1221 cb725bb209 dsv41 cudalab: DeepSeek kernel parity + speed harness (mHC, router, compressor, indexer, top-k, sparse attention, Engram) over src/dsv41/ds4_kernels.cu 2026-10-20 17:39:16 +02:00
Niko1221 770292e41c dsv41 cudalab: tensor-core trellis GEMV (fp16 point table, TQL coalesced layout, prepared B fragments), K4-K7, widths 1-5 2026-10-20 17:34:48 +02:00
Niko1221 672fd99ccc Metal: trellis expert kernels (TQ2_T, TQK6, TQK7) and the block Hadamard rotation for Gyro
The decode (moe_gu / moe_down) and prompt (mm_gu2 / mm_down2) expert kernels read the three trellis types
through a unit decoder in quant.metal (the codebook from threadgroup or constant memory); had128 kernels
rotate the gate/up and down inputs of Hadamard-folded layers in the decode window and both prompt paths.
ref_quant decodes the types with the CPU engine's own state extraction; expert_parity takes real experts of a
GGUF (--gguf --layer), the pack's hadamard.txt (--had-pack) and has a bitwise rotation test (--had-test).
2026-10-20 17:31:52 +02:00
Niko1221 5b47f889b0 Test: the calibrate setup test's data_folder stand-in takes the extra arguments 2026-10-20 17:26:14 +02:00
Niko1221 5db632e72a Test: engine minimums are checked against the tree's CMake version, not a fixed 0.1.45 2026-10-20 17:26:14 +02:00
Niko1221 f5c4047e78 Test: the unsloth setup tests answer every disk-space question themselves 2026-10-20 17:26:14 +02:00
Niko1221 090121c31b dsv41 cudalab: trellis K4-K7 dequant + GEMV v1 on sm_86 with parity and speed harness (tqenc and the K5/K4 CPU decoder taken from f-dsv41-enc) 2026-10-20 17:20:22 +02:00
Niko1221 1f727f4b52 Merge f-022-left: interrupted-move prompt (app + Terminal), deterministic disk test, engine-minimum test reads the tree version 2026-10-20 17:18:00 +02:00
Niko1221 ebe4db78d4 Test: the calibrate setup test's data_folder stand-in takes the extra arguments 2026-10-20 17:17:45 +02:00
Niko1221 6b5ad4f27d trellis: proxy_ldl 2026-10-20 17:11:51 +02:00
Niko1221 dea6920b81 hip-rdna3 on Windows: Strix Halo (gfx1151) keeps the expert arena unregistered by default; strata.exe finds rocm\bin for --version (delay-loaded ROCm math DLLs, bundled by package_windows.py) 2026-10-20 17:09:46 +02:00
Niko1221 9e24f4ab32 Control 0.1.46.2: version bump for the Terminal/Control-only update (engine packages unchanged) 2026-10-20 17:01:22 +02:00
Niko1221 dcbaa992d2 launch --plan: a part that is already on disk is not listed as a download, and a draft layer or picture reader found in the data folder is not counted 2026-10-20 17:00:31 +02:00
Niko1221 79cb59c574 Updater tests reset a signature verifier left behind by an earlier test file 2026-10-20 17:00:30 +02:00
Niko1221 ae456dfe5c iq_pack: an interrupted experts.bin write continues at the next layer (finished layers are flushed and recorded, the last kept one is checked against the GGUF) instead of starting from zero; says so 2026-10-20 17:00:30 +02:00
Niko1221 176e11a597 launch: a run by flags reuses the tuning result stored for the same model, choices and computer and says so; --retune measures again 2026-10-20 17:00:29 +02:00
Niko1221 5dc151d4d3 launch: setup step 6 (packing, converting) is no longer silent - the plain sentence for what it does is shown and a 'still working, N min so far' line comes every 30 s of silence; a cancel is seen at once 2026-10-20 17:00:29 +02:00
Niko1221 0dbea601a0 Download label: the total counts every shard and the picture reader, in the data folder setup is given (--data-dir), not the first shard only 2026-10-20 17:00:29 +02:00
Niko1221 3d5594d656 launch --plan: names the model your files are (title, size, picture setting, a copyable line with --model) instead of placeholders, and says what would be downloaded and how big (0 when everything is here, the draft layer and picture reader counted, engine mentioned) 2026-10-20 17:00:28 +02:00
Niko1221 284552fe26 Own model files: links to shards are accepted with or without --model, and a problem that applies to several files is said once (problem_words), in the Terminal, the screens and 'models import'; tests for items 2 and 3 2026-10-20 17:00:28 +02:00
Niko1221 c24f2a139f Own model files: a --model-path that is not there or holds no model is said in plain words before the engine or setup start (no 'missing /Qwen...gguf', no promise of a download), also in --plan and the screens 2026-10-20 17:00:28 +02:00
Niko1221 bd4c6bf1af Setup tests follow the merges: verify_sha256 takes own=, a tight disk margin warns and a real shortfall refuses 2026-10-20 17:00:27 +02:00
Niko1221 fa56c2b387 Start: a picture reader path from an older engine folder follows the installed package (engine update or other image no longer ends in server_exited) 2026-10-20 17:00:27 +02:00
Niko1221 cf34e915c4 Update rules: Control --version waits longer and tries twice, a busy card is not held against an engine (other failures expire after a day), pre-release Control handover, a wrong PC clock says so 2026-10-20 17:00:26 +02:00
Niko1221 14be49605f Models folder: a whole drive becomes D:\Strata (and D:\Strata\Models), the person's folder is written into the settings file Control reads (so the installer accepts it whatever it is called), no link where the disk cannot hold one, plain words for a folder that is gone or not writable; the Terminal remembers a chosen --data-dir; docs: moving models to another disk and what stays on C: 2026-10-20 17:00:26 +02:00
Niko1221 6b29adad1a strata launch without a real tty (Git Bash, mintty): say so and go on with plain questions instead of refusing 2026-10-20 17:00:26 +02:00
Niko1221 e728a132d7 POSIX owner-only checks: repair our own folders whose mode drifted, note drives without modes, and name the chown command for folders another account owns 2026-10-20 17:00:25 +02:00
Niko1221 32b9f75376 Tests: loopback.py is the one allowed opener for this computer's own servers 2026-10-20 17:00:25 +02:00
Niko1221 1972af6da1 Download and prepare steps say what they are: one set of plain sentences (draft layer download, packing experts, checking files, picture reader) for the app and the Terminal; the SHA-256 check shows its own figure instead of a download stuck at 99%; a step's heading is kept in view; the .part file no longer jumps the bar 2026-10-20 17:00:23 +02:00
Niko1221 7db513297d Launch: a tuning that found the standard settings already fastest counts as tuned (no 20-minute measurement at every start) 2026-10-20 17:00:08 +02:00
Niko1221 bf2926e05a Terminal and app checker: the folder holding the model is found above the picked one, links keep their model name, one plain wording for every refusal (file name + why) 2026-10-20 17:00:07 +02:00
Niko1221 4dee320dd5 setup --gguf-dir never deletes the person's own file: a size or SHA-256 mismatch stops with the file's name and why, and the file stays 2026-10-20 17:00:07 +02:00
Niko1221 c316a7c215 Fix: Windows runtime folder refused as not owner-only 2026-10-20 17:00:06 +02:00
Niko1221 73b68dfd88 Fix #3410: a Strata folder in the Trash is not remembered or offered as an install 2026-10-20 17:00:06 +02:00
ctr-137 ce938fb8c9 Fix #2971: a tool call no longer widens the web page on phones (from PR #3156) 2026-10-20 17:00:06 +02:00
Yash Raj Pandey fb2f387407 Fix #2496: an engine program that cannot be started is a 503 with the reason, not a dropped connection (from PR #2497) 2026-10-20 17:00:05 +02:00
xrip 9d2870d187 Fix #2289: Anthropic usage reports cache_creation_input_tokens, the prompt tokens read now and kept for the next request (from PR #2290) 2026-10-20 17:00:05 +02:00
xrip b611ec5f1e Fix #2141: honour llama-server's thinking_budget_tokens (from PR #2140) 2026-10-20 17:00:04 +02:00
MEsfits ddbbda4824 Fix #3275: calibrate a dual config whose gpu is the string 0,1 (from PR #3276; a single card given as a number keeps asking for that card) 2026-10-20 17:00:04 +02:00
Niko1221 386155fa25 Fix #3370: the vision work folder is made again when a temp cleaner removed it 2026-10-20 17:00:03 +02:00
Niko1221 ff64e08bba Fix #3403: the Mac setup installs cmake and ninja before it compiles 2026-10-20 17:00:03 +02:00
Niko1221 a569d80680 Fix #3187: calibrate checks the report folder before the sweep, not after it 2026-10-20 17:00:03 +02:00
Niko1221 461d0baee7 Fix #3188: strata_bench names the port it starts its own server on 2026-10-20 17:00:02 +02:00
Niko1221 ef5d8ab946 Fix #3302: owner-only file checks accept the built-in Administrator (SDDL alias LA) 2026-10-20 17:00:02 +02:00
Niko1221 bcda2e4b0d Fix #3298: --mtp with a depth number is not a folder in the model file checks 2026-10-20 17:00:02 +02:00
Niko1221 34d6c37df3 Disk space: stop only when the bytes do not fit, warn inside the working-room margin, a resumed download's partial file is not counted twice 2026-10-20 17:00:01 +02:00
Niko1221 f207907b14 Own model files: the checker in front of setup accepts what setup --gguf-dir accepted (Unsloth split sets with a metadata-only first shard, links, several sizes in a folder, a folder above the model folder, more than 128 files); Unsloth Q5/Q6/Q8 and Swift IQ3_S are in the table 2026-10-20 17:00:01 +02:00
Niko1221 7ebdd34356 Windows file in use: every state-file swap goes through atomic.replace (retry); a sharing violation no longer says run as yourself 2026-10-20 17:00:00 +02:00
Niko1221 aa44b9ce53 No engine for this PC: say it plainly (no graphics card Strata can use, or no engine for the card), no feed address as the next step 2026-10-20 17:00:00 +02:00
Niko1221 af189129a4 Calls to this computer's own servers skip the system proxy (tuner, tune session, launch self-check) 2026-10-20 17:00:00 +02:00
Niko1221 c6a0e739be Fix #3417: the hardware report counts physical cores, so the Work profile no longer asks for more CPU workers than the CPU has cores 2026-10-20 16:59:59 +02:00
Niko1221 e52668857c API key is optional beyond this computer: warn, never refuse
The server and strata serve start on 0.0.0.0 without a key and print one plain
warning. strata launch gets --no-api-key; the Access screen makes a key by
default and t turns it off. api_key "none" in a config means no key on
purpose. Cloudflare without a key gets a stronger warning. Docs updated.
2026-10-20 16:59:59 +02:00
Niko1221 c0c3a06ce4 Hand-over: a source checkout never hands over to an installed Control 2026-10-20 16:59:58 +02:00
Niko1221 f39d951904 Terminal hands over to the Control that strata update installed
The Terminal on PATH ignored components/control/active.json, so after
strata update the old Control kept running. strata.py now checks the
pointer first (stdlib only) and runs the newer, whole Control with its
own interpreter; never an older one, never outside the control folder,
no loop (STRATA_HANDOVER), one warning and a fallback if it is broken.
strata update says the next command already runs the new Control.
2026-10-20 16:59:58 +02:00
Niko1221 968fcb70b0 Drop the cached update notice when Control, engine or app version changed 2026-10-20 16:59:57 +02:00
Niko1221 b486d6f2b4 Test: engine minimums are checked against the tree's CMake version, not a fixed 0.1.45 2026-10-20 16:58:31 +02:00
Niko1221 43127e3982 Test: the unsloth setup tests answer every disk-space question themselves 2026-10-20 16:58:14 +02:00
Niko1221 6c688faf22 Models folder: Finish or leave an interrupted move from the Storage card (host routes, card notice, 6 strings in 8 languages) 2026-10-20 16:57:27 +02:00
Niko1221 0a35538b74 Models move: remember an unfinished move in the old folder; strata models move --resume/--leave/--status; one-line notice at strata launch 2026-10-20 16:53:14 +02:00
Niko1221 c3e9a998ad dsv41: tools to turn the llama.cpp reference's 10 greedy prompts into ids and compare the engine's greedy output (serve protocol) 2026-10-20 16:50:46 +02:00
Niko1221 72f66767ca Test: the models-folder strings moved to s8339-s8416 when the locale merge renumbered them 2026-10-20 16:48:21 +02:00
Niko1221 031adf99bf Merge f-022-modelsui: change the models folder in Settings > Storage (move / keep / start empty), as in the approved mockup 2026-10-20 16:46:46 +02:00
Niko1221 e3040e9e25 Models folder: spacing and the path as code in the dialogs, a plain line instead of the mockup's 'as in step 2', a way to force the copy path for tests, INSTALL.md sentence, string test 2026-10-20 16:41:58 +02:00
Niko1221 6f1eae68ff Merge f-022-flow2: --plan shows real files and download size, plain missing-path text, total download label, heartbeat, stored tuning reused, resumable packing 2026-10-20 16:41:26 +02:00
Niko1221andmw00 a0dbad8078 dsv41: pack_gguf uses the GGUF's own chat template (fixes Maya #299), drop Maya's template and the EXL3 packer
Maya's tools/ds4/chat_template.jinja lost tool associations and later system instructions; the GGUF's embedded
template matched DeepSeek's encoding.py byte for byte on 9 cases (notes-dsv41-ref.md).

Co-authored-by: mw00 <[email protected]>
2026-10-20 16:36:35 +02:00
Niko1221 085474b8dc launch --plan: a part that is already on disk is not listed as a download, and a draft layer or picture reader found in the data folder is not counted 2026-10-20 16:35:12 +02:00
453cc0d56f dsv41: build the imported DeepSeek-V4.1 engine path in Strata (HIP gfx1201), add --score
CMake option STRATA_BUILD_DSV41 (default OFF, adds the target strata-dsv41 and a private static kernel lib; no
existing target changes). The vendored exl3 / sampler / window-grouping kernels live in namespace strata::dsv41k so
no symbol is shared with strata_kernels. Missing CUDA names for HIP come from a force-included private header
(include/strata/dsv41/hip_extra.h), the IQ CPU lane stays the stub branch, MAYA_* env aliases are gone.
New --score IDS.json --score-out RUN.tsv: teacher-forced top-64 log-probabilities in tools/quant_quality.py's
row format (the gate against the llama.cpp reference). MIT notice added to every ported file.

Co-authored-by: mw00 <[email protected]>
Co-authored-by: needmorevram <[email protected]>
2026-10-20 16:34:12 +02:00
Niko1221 1d6a040129 Settings > Storage: Models folder card with the three choices (move, keep, start empty), progress, stop, and plain errors; host routes and Control calls; 78 strings in 8 languages; tests 2026-10-20 16:29:44 +02:00
Niko1221 b73bd26a5c trellis: run_parts (engine runs that survive a failed segment) 2026-10-20 16:26:00 +02:00
Niko1221 a0e09803ac Models folder: move the model files to another folder (strata models move): copied and checked file by file, stoppable, run configs follow 2026-10-20 16:20:27 +02:00
mw00andneedmorevram bbd0a2c673 dsv41: import Project Maya's DeepSeek-V4.1 engine unmodified (v1.1.17, 8d9a9b5d)
Verbatim copy of mw00/project-maya src/ds4, tests/ds4_*, tools/ds4 and the exl3 / sampler
kernel sources it needs, MIT, with their LICENSE and a NOTICE. Not built yet; the next
commits adapt it to Strata's tree.

Co-authored-by: needmorevram <[email protected]>
2026-10-20 16:19:45 +02:00
Niko1221 2fcc25aab6 Release notes 0.2.2 draft: shared models folder and Docker flow fixes 2026-10-20 16:08:31 +02:00
Niko1221 67bb56bd1d Merge branch 'f-022-int' into f-022-intfix 2026-10-20 16:08:15 +02:00
Niko1221 d511c5102e Updater tests reset a signature verifier left behind by an earlier test file 2026-10-20 16:08:13 +02:00
Niko1221 d408fa2ff1 tools tests: the verify_sha256 stand-in takes the new own= argument (34 setup tests errored on it) 2026-10-20 16:04:05 +02:00
Niko1221 970c75c8f6 iq_pack: an interrupted experts.bin write continues at the next layer (finished layers are flushed and recorded, the last kept one is checked against the GGUF) instead of starting from zero; says so 2026-10-20 16:03:23 +02:00
Niko1221 ee8e6404b0 launch: a run by flags reuses the tuning result stored for the same model, choices and computer and says so; --retune measures again 2026-10-20 16:02:35 +02:00
Niko1221 a7eb99bba9 trellis: block-feedback (BlockLDLQ) encoder and the ldl mode of build_swap 2026-10-20 15:59:36 +02:00
Niko1221 17cb817594 launch: setup step 6 (packing, converting) is no longer silent - the plain sentence for what it does is shown and a 'still working, N min so far' line comes every 30 s of silence; a cancel is seen at once 2026-10-20 15:58:11 +02:00
Niko1221 431803abb9 STRATA_ACT_DUMP: rarer flush with the covariance on; merge_dumps keeps it 2026-10-20 15:57:45 +02:00
Niko1221 594e379291 Download label: the total counts every shard and the picture reader, in the data folder setup is given (--data-dir), not the first shard only 2026-10-20 15:56:33 +02:00
Niko1221 fd4f7d445c launch --plan: names the model your files are (title, size, picture setting, a copyable line with --model) instead of placeholders, and says what would be downloaded and how big (0 when everything is here, the draft layer and picture reader counted, engine mentioned) 2026-10-20 15:55:39 +02:00
Niko1221 226b5f5492 Own model files: links to shards are accepted with or without --model, and a problem that applies to several files is said once (problem_words), in the Terminal, the screens and 'models import'; tests for items 2 and 3 2026-10-20 15:50:48 +02:00
Niko1221 9dc1c96377 Own model files: a --model-path that is not there or holds no model is said in plain words before the engine or setup start (no 'missing /Qwen...gguf', no promise of a download), also in --plan and the screens 2026-10-20 15:50:42 +02:00
Niko1221 4e1e6bad4e Merge f-022-flow: Docker and re-run fixes from the end-to-end flow walk 2026-10-20 15:41:52 +02:00
Niko1221 3de97a8892 Setup tests follow the merges: verify_sha256 takes own=, a tight disk margin warns and a real shortfall refuses 2026-10-20 15:41:30 +02:00
Niko1221 adf7ca9729 trellis: swap comparison and greedy sanity scripts 2026-10-20 15:34:41 +02:00
Niko1221 c23a4a5827 Merge f-022-modelsdir: the app and the Terminal share one models folder 2026-10-20 15:33:01 +02:00
Niko1221 4aabf3b8f5 Release notes 0.2.2: DRAFT paragraphs for the merged streams 2026-10-20 15:31:01 +02:00
Niko1221 6ce6413606 One models folder for the app and the Terminal: each settings file keeps when the folder was chosen, the one changed last wins and the other file follows (nothing is moved), tests 2026-10-20 15:30:37 +02:00
Niko1221 4fd5b0646d setup.py: the three remaining os.replace calls use atomic.replace 2026-10-20 15:29:52 +02:00
Niko1221 703daada09 Locales: keyless-sharing, own-GGUF, ports and models-folder strings in en, de, es, ja, ko, pt, ru, zh (s8301-s8313) 2026-10-20 15:29:52 +02:00
Niko1221 d69c9c4b4f Settings > Storage shows the Documents-fallback notice (app.modelsDirNotice) with the existing storage notice line 2026-10-20 15:29:51 +02:00
Niko1221 b22206c7bc Serving: Remove key stays available while sharing; plain warning in the remove dialog 2026-10-20 15:29:51 +02:00
Niko1221 a4f7975324 Removing the API key while sharing or Cloudflare runs is allowed (the key is optional); the remove dialog warns plainly 2026-10-20 15:29:51 +02:00
Niko1221 bcf88b7383 Merge f-022-refusals: environmental refusals repair or warn instead of blocking (audit items 1-4, 6, 8-10, 12-15) 2026-10-20 15:14:01 +02:00
Niko1221 6a55bcb5e9 Host tests: sharing and the tunnel start without a key (owner decision) 2026-10-20 15:13:11 +02:00
Niko1221 c7938c2d3e Merge f-022-ownfiles: own GGUF files load again, plain download/prepare steps, chosen models folder 2026-10-20 15:05:13 +02:00
Niko1221 69a6db1904 Docs: Docker, files you already have (what is and is not downloaded, mounts, links, old volume) 2026-10-20 14:49:46 +02:00
Niko1221 1b6f782faf Start: a picture reader path from an older engine folder follows the installed package (engine update or other image no longer ends in server_exited) 2026-10-20 14:40:43 +02:00
Niko1221 215aa14f25 Update rules: Control --version waits longer and tries twice, a busy card is not held against an engine (other failures expire after a day), pre-release Control handover, a wrong PC clock says so 2026-10-20 14:40:35 +02:00
Niko1221 4fc2e140bd Models folder: a whole drive becomes D:\Strata (and D:\Strata\Models), the person's folder is written into the settings file Control reads (so the installer accepts it whatever it is called), no link where the disk cannot hold one, plain words for a folder that is gone or not writable; the Terminal remembers a chosen --data-dir; docs: moving models to another disk and what stays on C: 2026-10-20 14:38:00 +02:00
Niko1221 0b4e603cfd STRATA_ACT_DUMP: keep the accumulator alive for the exit flush 2026-10-20 14:36:38 +02:00
Niko1221 2ec3d2e381 trellis: corpus builder, original-weight fetch, swap builder for Gyro-S layer variants 2026-10-20 14:36:38 +02:00
Niko1221 a775b04695 native experts: STRATA_ACT_DUMP, per-layer second moments of the CPU expert inputs (research, env-gated, off by default) 2026-10-20 14:36:38 +02:00
Niko1221 6f99190f82 trellis: README, column-profile check of the Gyro error 2026-10-20 14:36:38 +02:00
Niko1221 4292678b8f trellis: GPU path in tqenc.py and the pilot, diagonal activation second moments (importance-weighted encode), error profile script 2026-10-20 14:36:38 +02:00
Niko1221 68a8cad3ed trellis HIP: register pressure tweaks, CU mode 2026-10-20 14:36:38 +02:00
Niko1221 c23936be78 trellis: HIP encoder (gfx1201), bit-identical to the CPU encoder; host helpers; pilot driver reads remote GGUF shards 2026-10-20 14:36:38 +02:00
Niko1221 7e9531b3da trellis: python wrapper, Hadamard helpers, .tqx container, Gyro re-encode and original-weight comparison scripts, pilot driver 2026-10-20 14:36:38 +02:00
Niko1221 512e298e6e trellis encoder: tail-biting Viterbi for TQ2_T/TQK6/TQK7 plus experimental TQK5X/TQK4X (CPU decoder reads them)
tools/trellis/tqenc.{hpp,cpp}: shift-trellis Viterbi (group-min ACS), tail-biting by a rotated free pass or the exact
search over all start groups, fp16 scale refinement, importance weights.  tqenc_test.cpp checks the streams, the
Viterbi against a naive DP, planted solutions, bit-exact decode against tq_states.hpp and the CPU kernels.
The CPU decoder gets K = 5 and 4 (ids 224, 225, experimental v1); existing types and layouts are unchanged.
2026-10-20 14:36:38 +02:00
Niko1221 740acc8934 strata launch without a real tty (Git Bash, mintty): say so and go on with plain questions instead of refusing 2026-10-20 14:34:21 +02:00
Niko1221 72fc52251b POSIX owner-only checks: repair our own folders whose mode drifted, note drives without modes, and name the chown command for folders another account owns 2026-10-20 14:34:20 +02:00
Niko1221 0c79d7db94 Tests: loopback.py is the one allowed opener for this computer's own servers 2026-10-20 14:34:20 +02:00
Niko1221 d298999c3c Download and prepare steps say what they are: one set of plain sentences (draft layer download, packing experts, checking files, picture reader) for the app and the Terminal; the SHA-256 check shows its own figure instead of a download stuck at 99%; a step's heading is kept in view; the .part file no longer jumps the bar 2026-10-20 14:29:58 +02:00
Niko1221 632b4ed283 Launch: a tuning that found the standard settings already fastest counts as tuned (no 20-minute measurement at every start) 2026-10-20 14:19:55 +02:00
Niko1221 b111d1e830 Merge f-022-issues: 14 small issue fixes + Windows runtime folder repair 2026-10-20 14:18:38 +02:00
Niko1221 56dee6b938 Terminal and app checker: the folder holding the model is found above the picked one, links keep their model name, one plain wording for every refusal (file name + why) 2026-10-20 14:17:32 +02:00
Niko1221 d101771758 App 'Use a local GGUF': take one .gguf file as well as a folder (quotes around the path allowed), hand setup the folder that really holds the files and the data folder their prepared data sits in, name the file and why when a set cannot be used 2026-10-20 14:17:31 +02:00
Niko1221 a5b1c2cefa setup --gguf-dir never deletes the person's own file: a size or SHA-256 mismatch stops with the file's name and why, and the file stays 2026-10-20 14:17:29 +02:00
Niko1221 ca1d469e9a Fix: Windows runtime folder refused as not owner-only 2026-10-20 14:13:35 +02:00
Niko1221 47d9d6c75a Fix #3410: a Strata folder in the Trash is not remembered or offered as an install 2026-10-20 14:13:35 +02:00
ctr-137 f2709679db Fix #2971: a tool call no longer widens the web page on phones (from PR #3156) 2026-10-20 14:13:34 +02:00
Yash Raj Pandey c330caf4da Fix #2496: an engine program that cannot be started is a 503 with the reason, not a dropped connection (from PR #2497) 2026-10-20 14:13:34 +02:00
xrip 7adced3db5 Fix #2289: Anthropic usage reports cache_creation_input_tokens, the prompt tokens read now and kept for the next request (from PR #2290) 2026-10-20 14:13:34 +02:00
xrip a0b6d1cd11 Fix #2141: honour llama-server's thinking_budget_tokens (from PR #2140) 2026-10-20 14:13:34 +02:00
MEsfits 774096d16b Fix #3275: calibrate a dual config whose gpu is the string 0,1 (from PR #3276; a single card given as a number keeps asking for that card) 2026-10-20 14:13:34 +02:00
Niko1221 76186bb595 Fix #3370: the vision work folder is made again when a temp cleaner removed it 2026-10-20 14:13:34 +02:00
Niko1221 e346198ff3 Fix #3403: the Mac setup installs cmake and ninja before it compiles 2026-10-20 14:13:34 +02:00
Niko1221 98faf8139e Fix #3359: the HIP picture reader is built position independent so it links on Linux 2026-10-20 14:13:34 +02:00
Yudhanjaya Wijeratne 17b93dcb7c Fix #3368: verify_q8 no longer calls the device-only __half_as_ushort from host code (CUDA 12.0/12.1) 2026-10-20 14:13:34 +02:00
Niko1221 52ecf67ef7 Fix #3187: calibrate checks the report folder before the sweep, not after it 2026-10-20 14:13:34 +02:00
Niko1221 f74ba2ea53 Fix #3188: strata_bench names the port it starts its own server on 2026-10-20 14:13:34 +02:00
Niko1221 4fb0dde729 Fix #3302: owner-only file checks accept the built-in Administrator (SDDL alias LA) 2026-10-20 14:13:34 +02:00
Niko1221 f94a34c330 Fix #3298: --mtp with a depth number is not a folder in the model file checks 2026-10-20 14:13:34 +02:00
Niko1221 c7c0e37bc4 Host ports: plain message when the main port is taken or reserved; share and tunnel take a free port instead of PORT+1 and PORT+2 2026-10-20 14:12:52 +02:00
Niko1221 2f4e42b9cf App Share and Cloudflare work without an API key and warn plainly; a key is still offered and made by default 2026-10-20 14:12:04 +02:00
Niko1221 d61a443eb8 Docker: start line names the model files when only MODEL_PATH is set; the stop text no longer promises a download 2026-10-20 14:11:50 +02:00
Niko1221 52cc6b4af0 Host: a model folder in Documents that cannot be made falls back to STRATA_HOME/models with a plain notice instead of crashing at start 2026-10-20 14:10:46 +02:00
Niko1221 31361a98bf install.ps1: ARM64 reason, TLS 1.2 and proxy sign-in, real error text, long-path unpack with tar.exe, mklink check with a launcher fallback, running version message 2026-10-20 14:09:47 +02:00
Niko1221 87b7bb2ccb install.sh on ARM Linux and Intel Mac: keep the refusal, say why and what to do 2026-10-20 14:09:47 +02:00
Niko1221 48c34f735a Docker: MODEL_PATH that is not mounted, or holds no GGUF files, ends the start in plain words before any setup work 2026-10-20 14:09:30 +02:00
Niko1221 537f8fef8e Disk space: stop only when the bytes do not fit, warn inside the working-room margin, a resumed download's partial file is not counted twice 2026-10-20 14:08:31 +02:00
Niko1221 83ae413176 Docker: an NVIDIA card the container cannot use ends the start before the model is prepared, with the --gpus all hint 2026-10-20 14:07:42 +02:00
Niko1221 80f7679b61 Docker: a volume of the older image is moved into place (no new download), run settings of another image's engine folder are made again 2026-10-20 14:07:42 +02:00
Niko1221 b8488e0d2b Own model files: the checker in front of setup accepts what setup --gguf-dir accepted (Unsloth split sets with a metadata-only first shard, links, several sizes in a folder, a folder above the model folder, more than 128 files); Unsloth Q5/Q6/Q8 and Swift IQ3_S are in the table 2026-10-20 14:07:31 +02:00
Niko1221 d0afabb6ab Windows file in use: every state-file swap goes through atomic.replace (retry); a sharing violation no longer says run as yourself 2026-10-20 14:04:57 +02:00
Niko1221 656478c68d Merge f-022-toks: one answer-speed definition everywhere (engine decode figures) 2026-10-20 14:03:44 +02:00
Niko1221 fccfd343d7 One definition of tok/s: produced tokens over decode time, from the engine's own figures
The agent footer reads the local engine's /metrics for the turn (no empty number on short replies, prompt reading speed kept apart), model APIs are timed from the first token, the monitor average and the request list use answer speeds, the footer shows the last reply's own speed, and the server's done line no longer prints a million tok/s for a one-token reply.
2026-10-20 14:03:26 +02:00
Niko1221 5a9b4f4fc4 Merge f-022-needsfmt: needs-format issue fixes (#3418, #3417, #3358 tests/build) 2026-10-20 14:01:25 +02:00
Niko1221 4509f2f997 No engine for this PC: say it plainly (no graphics card Strata can use, or no engine for the card), no feed address as the next step 2026-10-20 14:00:31 +02:00
Niko1221 f9e1fff983 Fix #3358: the ISA guard knows the trellis AVX-512 rows (tq_avx512.cpp), so isa_guard_check passes on a normal build 2026-10-20 13:59:25 +02:00
Niko1221 a6ffc04d6d Fix #3358: the HIP MMQ parity screen no longer builds a string from a null type name 2026-10-20 13:59:25 +02:00
Niko1221 a5c7abe074 Fix #3358: the trellis CPU tests skip, not fail, on a ggml without trellis types 2026-10-20 13:59:25 +02:00
Niko1221 2793e3bd33 Calls to this computer's own servers skip the system proxy (tuner, tune session, launch self-check) 2026-10-20 13:58:47 +02:00
Niko1221 1b9b768939 Fix #3417: the hardware report counts physical cores, so the Work profile no longer asks for more CPU workers than the CPU has cores 2026-10-20 13:56:13 +02:00
Niko1221 642425bf2e Fix #3418: the app finds the engine's strata-device in the component folder, so a single Radeon is numbered the way HIP numbers it (an iGPU on ordinal 0 no longer makes GPU 0 the wrong card) 2026-10-20 13:55:24 +02:00
Niko1221 cc500be1c9 Merge f-022-apikey: the API key is optional; serving without one warns, never refuses 2026-10-20 13:29:58 +02:00
Niko1221 13806c323e Tests: an RX 6000 card now has a package of its own (0.2.2), drop the old no-package text 2026-10-20 13:29:58 +02:00
Niko1221 2f361a3757 API key is optional beyond this computer: warn, never refuse
The server and strata serve start on 0.0.0.0 without a key and print one plain
warning. strata launch gets --no-api-key; the Access screen makes a key by
default and t turns it off. api_key "none" in a config means no key on
purpose. Cloudflare without a key gets a stronger warning. Docs updated.
2026-10-20 13:29:14 +02:00
Niko1221 2dfb254f7b Merge f-022-ctlswitch: the Terminal starts the Control that strata update installed 2026-10-20 13:21:46 +02:00
Niko1221 28b150f854 Hand-over: a source checkout never hands over to an installed Control 2026-10-20 13:21:46 +02:00
Niko1221 676b57bff2 Terminal hands over to the Control that strata update installed
The Terminal on PATH ignored components/control/active.json, so after
strata update the old Control kept running. strata.py now checks the
pointer first (stdlib only) and runs the newer, whole Control with its
own interpreter; never an older one, never outside the control folder,
no loop (STRATA_HANDOVER), one warning and a fallback if it is broken.
strata update says the next command already runs the new Control.
2026-10-20 13:18:07 +02:00
Niko1221 9424830516 Merge f-022-bench: strata bench run fixes from the first real run 2026-10-20 12:39:58 +02:00
Niko1221 00e8d10994 Merge f-022-idle: idle-GPU measurement (2507-0), no default change 2026-10-20 12:38:42 +02:00
Niko1221 bf69ec2da4 bench: results of the first real runs on the 0.2.1 release (RTX 3060, 2x RTX A4000), protocol text from the measured counts 2026-10-20 12:36:35 +02:00
Niko1221 7924bf65d2 MULTI_GPU: STRATA_SPLIT_OVERLAP is measured on NVIDIA now (no gain) 2026-10-20 12:36:06 +02:00
Niko1221 867d7477ff 2507-0: measure the idle GPUs with the existing switches (z440 2x A4000, P2003 RTX 3060)
tools/idle_gpu_ab.py is the 0.2.1 split A/B harness with a per-card nvidia-smi sampler. Result on the 0.2.1 engine: the two-card
split is 47-49 % idle in decode (cards take turns), the single card 5 %; STRATA_SPLIT_OVERLAP, --pipeline-windows 1/2 (with and
without the draft layer on card 0) and --batch-groups 2 give no gain worth a default. docs/MULTI_GPU.md: the overlap flag is now
measured on NVIDIA.
2026-10-20 12:35:59 +02:00
Niko1221 f5410a6feb bench: say what the short prompt and the output cap mean; measured duration replaces the 20-30 minute guess
- the plain summary and the README get a Capped column (requests that reached the 512-token cap) and leave out the
  short case's prompt rate (an 80-token prompt measures the fixed cost of a request); results.json keeps it
- BENCHMARK_PROTOCOL.md, COMMUNITY_BENCHMARKS.md, BENCH.bat and bench.sh: the standard suite took 2 to 4 minutes after
  loading on an RTX 3060 and on 2x RTX A4000, not 20 to 30
2026-10-20 12:13:07 +02:00
Niko1221 bb7e399c30 Merge main (0.2.1.1 long-path hotfix) 2026-10-20 11:39:11 +02:00
Niko1221 4bd5912695 Merge f-022-numa: NUMA-split CPU pool on multi-node machines (#3201) 2026-10-20 11:35:00 +02:00
Niko1221 8c07c0c36a Merge f-022-rx6000: RX 6000 (gfx1030/1031) engine package, built not tested on a card 2026-10-20 11:34:52 +02:00
Niko1221 9ae93f7b1f Control 0.1.46.1: version bump for the Windows long-path hotfix 2026-10-20 11:18:52 +02:00
Niko1221 9f13d1e78f Merge main (Docker commit fix) into the long-path hotfix 2026-10-20 11:15:54 +02:00
Niko1221 a0abdf7588 Engine install: survive Windows MAX_PATH (short staging folder, extended-length paths, plain file errors, deep-folder warning)
On a fresh Windows PC (long paths off) the AMD engine packages could not be unpacked: staging under .updates/staging/<12 hex> pushed the
longest hipBLASLt file past 259 characters, and the failure surfaced as 'local file operation failed (FileNotFoundError)'.

- staging is now <components>/.s/<4 hex>/engine/<folder>; Control stages the same way
- unpack, delete and verify go through extended-length paths (strata_control/longpath.py)
- OSError messages name the file and say so when the path is too long
- a plain warning (shown as a warn line by launch) when the install folder is too deep for the engine's own file loading
- tests: real 0.2.1 package path lengths for a 20 character user name
2026-10-20 11:15:20 +02:00
Niko1221 149abc896b Release notes 0.2.2 draft: NUMA pool 2026-10-20 10:54:40 +02:00
Niko1221 069384b4f6 Merge main: Docker commit fix 2026-10-20 10:51:45 +02:00
Niko1221 c644312d15 bench: fixes from the first real runs on release engines (LVM drive lookup, CUDA UMD banner, build runtime, engine log, fresh-load line, archive COMMIT)
The first runs of strata bench run on the 0.1.46 release engine (RTX 3060, 2x RTX A4000) showed:
- the release terminal archive carried no COMMIT file, so every run was nonconforming; package_terminal.py now takes
  --commit / STRATA_COMMIT and refuses to build a distributable archive when the commit is unknown
- the model drive on an LVM volume was not found (lsblk -s nests lvm -> part -> disk)
- nvidia-smi 615 prints CUDA UMD Version, so no runtime was recorded; the runtime is now the engine's build
  runtime (engine-component.json) plus the driver's supported CUDA version
- engine.log was empty (serve --quiet); it now holds the run config's log from the start of the run
- a fresh server (0 prior requests) was reported as 'could not tell whether freshly loaded'
2026-10-20 10:44:25 +02:00
Niko1221 ac71d2b32e RX 6000 package: docs, release-notes draft, Windows build comment, tuning-table test rows 2026-10-20 10:43:45 +02:00
Niko1221 c093ac263f docker/build.sh: STRATA_BUILD_COMMIT overrides the commit it passes 2026-10-20 10:39:48 +02:00
Niko1221 436fbd0a6e Docker: pass the source commit into the terminal stage (no git in the image)
package_terminal.py reads STRATA_BUILD_COMMIT first and only then asks git, and no longer crashes when git is missing. build.sh passes the commit as a build arg, the Dockerfile turns it into that env.
2026-10-20 10:39:15 +02:00
Niko1221 7274c10e9a NUMA pool: STRATA_TEST_SYSFS_ROOT lets a test give the pool a made-up two-node machine; docs 2026-10-20 10:31:13 +02:00
Niko1221 f0afedc50c Merge main: publish.sh executable 2026-10-20 10:29:56 +02:00
Niko1221 5bda973a43 Docker: make docker/publish.sh executable (the image workflow failed with Permission denied) 2026-10-20 10:29:54 +02:00
Niko1221 b5d14279aa pool_tasks_test: fix the string literals 2026-10-20 10:28:43 +02:00
Niko1221 2e12958759 pool_tasks_test: STRATA_NUMA_POOL=0 switches the split off; nothing set leaves no refusal behind 2026-10-20 10:26:57 +02:00
Niko1221 e4d22e7ef1 NUMA pool: tried by itself on a machine with 2 or more NUMA nodes, STRATA_NUMA_POOL=0 turns it off, one node unchanged 2026-10-20 10:26:01 +02:00
Niko1221 e59e64ec5d Merge f-022-notice: drop a stale update notice after a version change 2026-10-20 10:25:01 +02:00
Baptiste Rajaut 393eec80f9 fix: drop the pack and MTP files' page cache before the NUMA placement 2026-10-20 10:22:05 +02:00
Baptiste Rajaut 06425cf932 docs: update NUMA-split numbers measured with iommu=pt 2026-10-20 10:22:05 +02:00
Baptiste Rajaut afbd64bc4f docs: document the opt-in NUMA-split CPU expert pool 2026-10-20 10:22:05 +02:00
Baptiste Rajaut 2614b82c9a fix: list the pool's own worker cores for the start-up line and --aux-cpus under the NUMA split 2026-10-20 10:22:05 +02:00
Baptiste Rajaut 5b8cdaf2d9 fix: interleave an explicit pool worker count across sockets when the NUMA split is on 2026-10-20 10:22:05 +02:00
Baptiste Rajaut ea6b385df8 feat: span both sockets with the default pool workers when the NUMA split is on 2026-10-20 10:22:05 +02:00
Baptiste Rajaut 8ca9093d7d fix: print a NUMA slice as not verified when all its move_pages batches failed 2026-10-20 10:22:05 +02:00
Baptiste Rajaut ec841ca1e9 fix: leave the NUMA split plan clean when the split is refused 2026-10-20 10:22:05 +02:00
Baptiste Rajaut c4442fed95 fix: report an unverifiable NUMA placement instead of FAILED when move_pages fails 2026-10-20 10:22:04 +02:00
Baptiste Rajaut 2496795bd9 fix: refuse a lopsided NUMA split and accept only 1/test for STRATA_NUMA_SPLIT 2026-10-20 10:22:04 +02:00
Baptiste Rajaut 8d29804204 fix: word the NUMA per-node shortfall as at risk, FAILED stays for the verdicts 2026-10-20 10:22:04 +02:00
Baptiste Rajaut 1523c592a6 fix: judge NUMA placement after registration, log silent no-ops and per-node shortfall up front 2026-10-20 10:22:04 +02:00
Baptiste Rajaut f8c4410a38 fix: release the model files' cached pages before NUMA placement of the complement 2026-10-20 10:22:04 +02:00
Baptiste Rajaut df4706577e feat: place complement row slices on their NUMA node by pinned first touch (opt-in) 2026-10-20 10:22:04 +02:00
Baptiste Rajaut afd1f61cc5 test: assert slice-0/slice-1 ranges never intersect, across neighbouring slots too 2026-10-20 10:22:04 +02:00
Baptiste Rajaut 31d857f817 feat: per-NUMA-slice byte ranges of a complement slot 2026-10-20 10:22:04 +02:00
Baptiste Rajaut 4126250bf1 fix: NUMA split refuses STRATA_EXCHANGE_ROTATE=1, diag sums the slices in a split phase, mixed-mode split test 2026-10-20 10:22:04 +02:00
Baptiste Rajaut 709f8dc63d feat: per-NUMA-node task slices and counters in the CPU expert pool (opt-in) 2026-10-20 10:22:04 +02:00
Niko1221 3985c50777 Engine package for the RX 6000 series (gfx1030 / gfx1031): build variant, package choice, setup note, tests 2026-10-20 10:20:08 +02:00
Niko1221 f68ccf30c7 Drop the cached update notice when Control, engine or app version changed 2026-10-20 10:07:55 +02:00
Niko1221 90767252f4 Release notes 0.2.1: final two-card batch A/B result, drop TODOs 2026-10-20 09:40:48 +02:00
Niko1221 a7bdbfdabc Release notes 0.2.1: the #3237 drive probe is on by default (owner OK) 2026-10-20 06:27:42 +02:00
Niko1221 9944d2f7f1 Merge branch 'f-021-int' into f-021-rel 2026-10-20 05:56:29 +02:00
Niko1221 b42ec662c7 Release notes 0.2.1: the R9700 queue fix with the two-queue numbers 2026-10-20 05:56:26 +02:00
Niko1221 b35a9816c7 amd: open two GPU queues instead of five at start (keeps the fixed prompt speed, no decode loss on R9700) 2026-10-20 04:53:45 +02:00
Niko1221 d33bb832ca Release notes 0.2.1: engine 0.1.46 and feed sequence 102 (proposed), file-name and Docker TODOs resolved, tuning tables per package 2026-10-20 02:37:08 +02:00
Niko1221 3e7c5d8e1c build_hip_linux.sh: pick up lib/rocm_sysdeps when the ROCm tree has it (the gfx1151 halo package needed a patched copy in 0.2.0-beta) 2026-10-20 02:31:50 +02:00
Niko1221 894d1be1f4 Version bump for 0.2.1 (proposed): app 0.2.1, engine and Control 0.1.46
Cargo.toml and Cargo.lock, desktop and host package.json, PRODUCT_VERSION, CMake project and Metal STRATA_VERSION, build_hip_linux PKGVER default, About comment, tests that pin the number. tauri.conf.json has no version key (Tauri takes it from Cargo.toml). Download names in the READMEs, AMD_HIP.md and UPDATES.md follow.
2026-10-20 02:17:37 +02:00
Niko1221 93aa62ed64 Release notes 0.2.1: fill the iD, iE, zc and bimodal parts (measured defaults, fixes, known limits) 2026-10-20 02:14:44 +02:00
Niko1221 83027df587 Merge branch 'f-021-iD' into f-021-int 2026-10-20 02:13:21 +02:00
Niko1221 484e06b961 Linux file tier: a start-up drive probe overrides the room rule's file cache on a fast drive (#3237)
When the room rule picks the file cache but the cache holds under half of the read set, the engine now
times O_DIRECT preads (8 threads) against first touches of a mapping (3 threads) on random 1 MiB
pieces of the model file, once, and reads unbuffered when the direct path runs at 4 GB/s or more and
is 1.3x (under 30% held) / 1.5x (under 50%) faster than the cached path. The reporter's PCIe 5 NVMe
(8.18 vs 4.93 GB/s, 13% held) lands on unbuffered (2x prompts, 3-4x decode in their A/B). A 1.7-2.5
GB/s NVMe (P1004) stays on the cache, where it decodes 1.1-1.6x faster; a ratio alone could not
separate the two, hence the absolute floor. STRATA_FILE_TIER_PROBE=0 keeps the room rule alone.
Default change: proposed, not validated on a fast drive here. Decision in platform::file_tier_prefers_direct
(platform_memory_test).
2026-10-20 01:32:52 +02:00
Niko1221 ed4595ad69 Merge branch 'f-021-bimodal' into f-021-int 2026-10-20 01:20:25 +02:00
Niko1221 06d7604af7 Merge branch 'f-021-zc' into f-021-int 2026-10-20 01:07:32 +02:00
Niko1221 ecf2bdcb7f Linux: io prefetch + decode gate by default for experts.bin packs when the page cache has room for 55% of the experts
Zero-copy does not apply to experts.bin (one contiguous blob the kernels already read from the mapping), but the io_fill_ prefetch path does. RTX 3060, IQ3_XXS 43 GB pack, 32 GB cgroup: decode 1.38/1.21/1.33x (5/5 pairs), prompts 1.04/1.06x. At 16 GB the cold 4K prompt was 4-6% slower, so it stays off below the room test and the log says how to force it. Identity gates 10/10, exact_pp 4K/20K identical.
2026-10-20 01:07:19 +02:00
Niko1221 3f1fa67f14 amd: open the GPU queues in a fixed order at start (R9700 fast/slow prompt starts)
On gfx1200/gfx1201 under Linux the compute stream and the expert staging stream
landed on the same compute pipe in about half of the starts, which made 4K prompts
13% slower (p620, 1,980 vs 2,290 tok/s). Five queues (four normal, one highest) are
now opened one after the other on the main thread before any other thread starts.
STRATA_HIP_QUEUE_PRIME=0 turns it off. Answers are unchanged.
2026-10-20 01:04:48 +02:00
Niko1221 881a42adab Merge branch 'f-021-notes' into f-021-int 2026-10-20 00:48:22 +02:00
Niko1221 3616148bbf Merge branch 'f-021-iE' into f-021-int
# Conflicts:
#	apps/host/src/locales/de.json
#	apps/host/src/locales/en.json
#	apps/host/src/locales/es.json
#	apps/host/src/locales/ja.json
#	apps/host/src/locales/ko.json
#	apps/host/src/locales/pt.json
#	apps/host/src/locales/ru.json
#	apps/host/src/locales/zh.json
2026-10-20 00:48:09 +02:00
Niko1221 24e56539c5 Release notes 0.2.1 (draft): downloads table, what is new, measured defaults, fixes, privacy, known limits, TODO markers for the running streams 2026-10-20 00:47:45 +02:00
Niko1221 8e957cd53b SYCL: build.sh stops an AOT Battlemage build before the compile when ocloc is the one that crashes on the int8 kernels (Compute Runtime 26.22.38646.x, IGC 2.36.3); I8_ESIMD=OFF / STRATA_SYCL_I8_ESIMD builds without them; docs say docker build --pull (#3219) 2026-10-20 00:47:08 +02:00
Niko1221 4f6ec303b5 Merge branch 'f-021-tfix' into f-021-int 2026-10-20 00:44:55 +02:00
Niko1221 1f52a00373 apps/host staging-scripts-syntax test: use Git bash with forward-slash paths on Windows instead of the WSL launcher 2026-10-20 00:44:41 +02:00
Niko1221 e68a032a58 math test: the KaTeX render check skips when katex is not installed in a shared node_modules (#3268) 2026-10-20 00:43:04 +02:00
Niko1221 10715d856d gfx906 (HIP wave64) builds again: no Qwen3.6 engine (cuBLAS calls the compat layer does not map), GGML_USE_HIP on the W4A4 target, the trellis overlay sources and the moe_f16_kq stub in the HIP source list; all targets compile for gfx906 (#3172 #3271) 2026-10-20 00:41:38 +02:00
Niko1221 e45a9bfd3a Merge branch 'f-021-split' into f-021-int 2026-10-20 00:19:12 +02:00
Niko1221 34e7c5cb1a Merge branch 'f-021-iA' into f-021-int
# Conflicts:
#	apps/host/server/native-control.mjs
2026-10-20 00:12:03 +02:00
Niko1221 8aaf188aa9 Your own engine settings are kept: user_args, a reserve of 0 on a compute-only card, control vectors survive a load (#3215 #3270 #3216)
The app and setup rewrote the run config's "args" on every Load: the resource preference forced
--vram-reserve-mib to 1536 (2048 with pictures, a fifth of the card on "work") whatever the file said, and
a setup run (started by the Load when the context or pictures differ) dropped every option it did not
choose itself, the control vector options included.

- run config: "user_args" holds the options set by hand; "args_written" is the args as setup/the app last
  wrote them, so a hand edit of "args" is noticed and moved into user_args (strata_control/user_args.py,
  apps/host/server/native-user-args.mjs: the same rule)
- every start (serve/server.py engine_args), every setup run (setup.carry_user_args), every host write and load
  and a saved tuning put the person's options over the generated ones; the same option: theirs wins
- old files without args_written: setup keeps the options the old args had, the new ones lack and setup.py never names
  (plus the control vector group); a reserve that is not setup's own 700
- --vram-reserve-mib: any number the person set (Expert setting, user_args, the key, a value in args) is used as it
  is, 0 included; the safety default (1536/1024, 2048 with pictures, 20% on "work") stays for a card that drives a
  display or is not known to be headless; a card the probe says drives no display keeps 512 (700 with pictures);
  a 0.2.0-beta file where the app wrote the number itself is told from a changed one
- hardware: drives_display per card (nvidia-smi display_active; the Windows adapter with a desktop resolution for AMD)
- setup --vram-reserve-mib, the web settings page and `start --vram-reserve-mib` record the value as the person's
- docs/TROUBLESHOOTING.md: what is kept and what the app still chooses
2026-10-20 00:11:11 +02:00
Niko1221 22703894ec Merge branch 'f-021-iB' into f-021-int
# Conflicts:
#	apps/host/server/native-control.mjs
2026-10-20 00:06:49 +02:00
Niko1221 b1713b3ff2 docs: dual-socket CPU pool measurements (workers across sockets won on one box, lost on another), IOMMU check, no numactl pinning (#3201) 2026-10-20 00:06:38 +02:00
Niko1221 0875029d10 docs: two-card gfx906 tip, --pipeline-windows 2 (+36% decode) and --trim-stage-weights (+5% prompt) measured on 2x MI50, off by default (#3173) 2026-10-20 00:06:38 +02:00
Niko1221 952cce7752 GPU choice (0.2.1): Automatic skips cards the engine cannot run, one card sent as gpu, settings-file GPUs kept, readable dropdown, Monitor opens on assigned cards (#3203 #3195 #3256 #3221 #3211) 2026-10-20 00:06:22 +02:00
Niko1221 30c64f09d7 Test fixes after the 0.2.1 merges: the gfx1151 table parse stops before the iGPU set, the benchmark's model-card link is not a model-host fetch, docker/build.sh copies bench/prompts/standard-v1 for the Terminal package, the Docker script tests use Git bash with unix paths on Windows 2026-10-20 00:05:48 +02:00
Niko1221 cbc0e329c6 Trellis MMQ overlay follows the ggml-cuda it patches: the generated callbacks and the loader pass the precision argument when mmq.cuh has it (llama.cpp b11xxx), identical output on the pinned commit, off with a message for an unknown signature (#3217) 2026-10-20 00:03:13 +02:00
Niko1221 f680a80d88 Batch groups: PROPOSED NVIDIA two-card default is one serial group (measured +12..16% on 2x A4000); owner decides 2026-10-20 00:03:09 +02:00
Niko1221 d8a6a0e012 App settings: the Allow browser origins help says it does nothing in the app, that the app's API already allows pages of any site, and where cors_origins really goes; text in all 8 languages and in DETAILS (#3208) 2026-10-19 23:52:04 +02:00
Niko1221 3cc18d7bb7 Chat: LaTeX math is drawn with KaTeX (inline $..$ and \(..\), display $$..$$ and \[..\]); KaTeX and its woff2 fonts ship with the app and load only when an answer has a formula; money and code stay text (#3268) 2026-10-19 23:49:59 +02:00
Niko1221 1994f3acb2 FP4-Local: setup installs the pinned torch, transformers and safetensors the converter imports (requirements-nvfp4-local.txt) right before a prepare that runs; the tool names a missing package up front instead of after 20 minutes (#3227) 2026-10-19 23:44:24 +02:00
Niko1221 c911065a14 Windows installer: ask for the install folder (another drive when C: is short), keep the earlier folder in app updates and silent installs; docs name what stays on C: (#3186 #3190) 2026-10-19 23:43:47 +02:00
Niko1221 40c605f7bc Windows with a non-UTF-8 code page (Russian): PowerShell output is read as UTF-8 (Documents folder, save dialog, fonts, GPU names), Python runs in UTF-8 mode from the launchers and the host's children, and the engine and picture reader get a UTF-8 manifest so a Cyrillic model folder arrives intact (#3202) 2026-10-19 23:38:49 +02:00
Niko1221 96a9107a27 Downloads: the MTP fetch shows progress in the app and survives a flaky link (runs again, fewer connections, one writer per folder); a cancel and a killed setup end the whole tree on Windows (snapshot + the fetch ends with its parent); Gyro's low-RAM pack gets --compat-bf16 (#3257 #3214 #3193) 2026-10-19 23:38:38 +02:00
Niko1221 4e76d6c9a6 HIP prefill: no start-up hipBLAS warm-up when the cards are of more than one architecture (#3204)
f3be9743 warms the first BF16 and FP16 products of every device at start. On a layer split across a
gfx1151 and a gfx1201 card it slowed prompts about 25% (hc read of the gfx1151 stage 2.7 -> 9.0 s per
32K block; STRATA_PREFILL_WARM=0 restores it). Two cards of one architecture lose nothing (2x R9700,
interleaved pairs). The warm-up now runs only when every visible card has the same gcnArchName;
STRATA_PREFILL_WARM=1 asks for it anyway. Policy in warm_policy.hpp with warm_policy_test.
2026-10-19 23:37:11 +02:00
Niko1221 8441cfd0e2 App: a start that stopped for no VRAM left for the expert cache is tried once more with less kept free (#3192); a model on the person's own GGUF folder keeps that folder, so a reload does not download it (#3199) 2026-10-19 23:31:43 +02:00
Niko1221 9cc2a9b6aa docs: build from git on Linux (clone, setup, rebuild, trying a pull request, engine env, the monitor from a checkout, plain cmake) (#3191) 2026-10-19 23:27:09 +02:00
Niko1221 9b8436df4e tools: gguf-py is also looked up in the install folder (STRATA_INSTALL_DIR, the app's folder), so tools run by hand from the Control package find what setup.py downloaded; the error names STRATA_GGUF_PY (#3269) 2026-10-19 23:26:32 +02:00
Niko1221 c6a20b7e85 Linux: say it when a finite RLIMIT_RTTIME (PipeWire without rtkit) is the reason for an engine death by signal 9, warn at start, troubleshooting entry (#3238) 2026-10-19 23:25:48 +02:00
Niko1221 88b29d481a CPU pool: the LP-E cores (/sys/devices/cpu_lowpower/cpus) are efficiency cores, in the engine and in setup (#3262)
On Meteor Lake and Arrow Lake-H the LP-E cores are in neither the cpu_core nor the cpu_atom
list, so the pool and setup counted them as P-cores: a Core Ultra 7 255H read as 8P+8E, the
more-E-than-P default plan did not apply and 15 workers ran (-35%). A CPU in no list now falls
back to its capacity. Tests: pool_plan_test (helper), tools/test_setup_hybrid.py (fake /sys).
2026-10-19 23:25:46 +02:00
Niko1221 2efa4e067a Merge branch 'f-021-minipc' into f-021-int 2026-10-19 22:20:52 +02:00
Niko1221 eb7eac250b Radeon 780M: the matrix-core prompt set is the default on gfx1103 (integrated, Linux): prompts x1.6 at 1K and x2.0 at 4K, decode unchanged, same greedy ids; STRATA_IGPU_PROMPT_SET=0 turns it off; gfx1150/1151 and Windows stay off; docs note the MES reset issue; per-arch tests 2026-10-19 22:20:33 +02:00
Niko1221 9f5488e3b2 Merge branch 'f-021-arcvis' into f-021-int
# Conflicts:
#	docs/MODELS.md
#	strata_control/vision_reader.py
#	tools/vision/CMakeLists.txt
#	tools/vision/strata_vision.cpp
2026-10-19 21:42:12 +02:00
Niko1221 cac4816afd Intel Arc pictures: on by default for the Alchemist cards (CPU reader), Battlemage stays opt-in; SYCL GPU picture reader (strata-vision-sycl) for --vision gpu; reader rule in vision_reader.py; tests and docs 2026-10-19 21:40:50 +02:00
Niko1221 4ed8eb02fa hipBLASLt tuning table on by default on Strix Halo (gfx1151) only, the owner's choice; opt-in elsewhere 2026-10-19 21:37:42 +02:00
Niko1221 b203b5f71e arch_defaults: UMA arena is not a default (measured slower, resets not fixed): constant off, STRATA_ARENA_PIN_GIB=none stays opt-in; docs say the iGPU prompt set is opt-in with the 780M numbers 2026-10-19 21:37:25 +02:00
Niko1221 468e3a6252 Merge branch 'f-021-hip' into f-021-int 2026-10-19 21:34:18 +02:00
Niko1221 c6aa4223d9 AMD packages carry the hipBLASLt tuning tables; the engine start picks the one for the card
Each AMD engine package gets tuning/<arch>-hipblaslt-<version>.txt for the architectures it covers and the
hipBLASLt version it bundles, and records hipblaslt_version in BUILD.json (packaging/stage_tuning.py for the
Linux build scripts, tools/hip/package_windows.py for the Windows zip). package_engine.py lists the tables in
the manifest and refuses an AMD stage without a version or with a table for another version or architecture.

serve/server.py with_component_engine sets STRATA_HIPBLASLT_TUNING from the package (strata_control/
hipblaslt_tuning.py): a table only for the exact arch and version, a log line for a mismatch, an unreadable
card or a missing version, STRATA_HIPBLASLT_TABLE=on|off (or "hipblaslt_table" in the run config) as the one
switch, and a STRATA_HIPBLASLT_TUNING the person set is kept. setup.py looks in the engine folder's tuning/
first.

A table changes the rounding of the prompt's GEMMs (R9700: residual differs, first-token KL 0.016), so the
automatic choice is off (AUTO_USES_TABLE = False) until the owner decides; "on" uses the table. No gfx1201
table exists for hipBLASLt 1.2.0 and one calibrated for it was not worth shipping (+1% prompt, -2% decode).
2026-10-19 21:33:47 +02:00
Niko1221 c23a78694e Merge branch 'f-021-macvis' into f-021-int 2026-10-19 21:27:28 +02:00
Niko1221 509466a668 Mac pictures: signing smoke checks strata-vision, docs and the 7 READMEs say the Mac reads pictures
sign-and-notarize.sh starts strata-vision --version in the signed engine folders (it already signs every Mach-O).
MAC.md carries the M4 numbers (0.45 s on the GPU, 50.7 s on the CPU for a 260-token picture) and the rule.
2026-10-19 21:26:49 +02:00
Niko1221 a10777a671 Merge branch 'f-021-R' into f-021-int 2026-10-19 21:26:01 +02:00
Niko1221 bcf67d1774 Merge branch 'f-021-bench' into f-021-int 2026-10-19 21:23:46 +02:00
Niko1221 c8e5d2f3ce Docs: BENCHMARK_PROTOCOL (implemented, still draft: field mapping, Metal, versions, runner reading, SECURITY link), COMMUNITY_BENCHMARKS, TERMINAL; BENCH.bat, bench.sh and strata_bench.py point to strata bench run 2026-10-19 21:23:20 +02:00
Niko1221 6d41b37e76 strata bench run: the standard benchmark protocol (standard-v1 is the default suite); fixtures and COMMIT file in the Terminal archive; tests with a mock server 2026-10-19 21:23:20 +02:00
Niko1221 e1ef9df21c Standard benchmark: frozen fixtures (bench/prompts/standard-v1) and the tool that builds them 2026-10-19 21:23:19 +02:00
Niko1221 dab7ee72d0 arch_defaults: the integrated-GPU matrix-core prompt set is opt-in again (bit-changing, owner's call); the Radeon 780M keeps its expert arena unregistered by default on Linux (UMA mode); per-arch selection tests 2026-10-19 21:21:50 +02:00
Niko1221 6b28dbbd1e Merge branch 'f-021-dock' into f-021-int 2026-10-19 21:21:22 +02:00
Niko1221 e0e0c0545a Docker: DOCKER.md records the rocm-gfx120x card test of the 0.2.0-beta packages (Radeon AI PRO R9700) 2026-10-19 21:20:52 +02:00
Niko1221 8acee5a055 Docker: build.sh and publish.sh exit 0 after a good run; DOCKER.md records the 0.2.1 image tests
- The EXIT trap of build.sh (--ctx-only or --ctx given) and publish.sh (--work given) ended with a failing test, so dash reported exit 1 after a
  successful run. Both traps now use if/fi. Test: build.sh --ctx-only exits 0 and its context holds docs/TERMINAL_LAUNCH.md.
- docs/DOCKER.md: image sizes, the 2026-10-19 test of the images built from the 0.2.0-beta packages (cuda13 and cuda12 on an RTX A4000, ROCm images
  built and checked, wrong-card message).
2026-10-19 21:16:34 +02:00
Niko1221 dca9770ef4 Task board: Schedule | Board view on Tasks, derivation with tests, loop record, sidebar count, all 8 locales 2026-10-19 21:05:20 +02:00
Niko1221 ff18fbb868 Task board host side: dismiss mark, newest problem only, one plain line per run, pending approvals of every chat 2026-10-19 21:05:20 +02:00
Niko1221 cf4749276d Mac pictures: strata-vision with Metal built in, Mac setup and launch turn pictures on
tools/vision builds one program for the Mac: ggml's Metal backend inside, --gpu reads on the GPU, without it on the CPU. The reader
choice (strata_control/vision_reader.py) gives a Mac the GPU reader whatever its memory (the CPU reader needs ~50 s per picture:
ggml has no BF16 vector kernel on ARM). setup.py's Mac path downloads the mmproj and writes the vision section; strata launch,
the MCP server and metal/setup_mac.py default pictures on. packaging/build_vision_mac.sh builds the program for the engine package.
2026-10-19 20:59:29 +02:00
Niko1221 578bb6aa7a Docker: publish the images from 0.2.1 (workflow, signed-feed fetch, local script)
- .github/workflows/docker-images.yml: manual only (workflow_dispatch) with the release tag as input; a plan job lists the flavors the
  release has an engine package for; one job per flavor builds the image and, only with push on, pushes ghcr.io/niko1221/strata:<version>-<flavor>
  and latest-<flavor> with GITHUB_TOKEN (packages: write).
- docker/fetch-engines.py: downloads the engine package from the release and checks it against the signed update feed (feed signature, archive
  size and SHA-256, manifest hash, package signature, every file), the same code as `strata update`. Flavors cuda13, cuda12, rocm-gfx120x,
  rocm-gfx110x, rocm-gfx1151 (the halo package), sycl (only if the feed lists one).
- docker/publish.sh: the same steps by hand (fetch, build, start check, push only with --push).
- Fix: docker/build.sh did not copy docs/TERMINAL_LAUNCH.md, which package_terminal.py needs; no image could be built from a clean tree.
- Dockerfile: OCI source/url/documentation labels (ghcr links the package to the repository).
- Compose files and docs/DOCKER.md name the 0.2.1 tags on ghcr.io; DOCKER.md has a maintainers section. Tests for the scripts, the workflow and the fetch.
2026-10-19 20:57:22 +02:00
Niko1221 1073929c80 [Other] serve.test_parallel: split in two files, each about a minute
Not a hang: 42 tests against a fake engine that paces real time took 127 s in one module and passed. The yield/failure tests move to serve/test_parallel_yield.py; the shared fake-engine helpers are FakeEngineCase.
2026-10-19 20:54:34 +02:00
Niko1221 e5dc00c81e docs: integrated GPUs (mini PCs) - the matrix-core prompt set is on by default, numbers, off switches, Windows untested 2026-10-19 20:38:55 +02:00
Niko1221 1c3c6d3351 hip: STRATA_ARENA_PIN_GIB=none leaves the expert arena unregistered (UMA mode for an integrated GPU: no KFD userptr, resident through the working-set lock); opt-in, not default 2026-10-19 20:38:55 +02:00
Niko1221 ce968a2fac hip: integrated Radeons (gfx1103/gfx1150/gfx1151, UMA flag) take the matrix-core prompt set by default; STRATA_IGPU_DEFAULTS=0 and each switch =0 turn it off; unit test with mock devices 2026-10-19 20:38:55 +02:00
Niko1221 40d1a3e7d7 [TUI] strata connect --write: a test with no keyboard, and an ended input counts as no 2026-10-19 20:38:37 +02:00
Niko1221 a269d1b983 [Other] engine hash tests: mock the release probe, keep the engine folder in a temp dir
get_prebuilt has probed the release with a HEAD request since 0.2.0; the refusal tests reached GitHub, got a 404 and compiled instead of refusing. The code is right, the test was stale.
2026-10-19 20:38:04 +02:00
Niko1221 9d2cb4f6d2 [Other] --data-dir never moves the model library without a choice
A --data-dir that names a new folder used to move Models, packs and mtp into it and rewrite the saved data_dir. Now the terminal asks (keep or move), --yes and a missing keyboard keep the old library in place (still found as a root), --move-data moves, and the saved default changes only on a move, --set-default-data-dir, or a PC with no models. Tests use temp folders.
2026-10-19 20:37:29 +02:00
Niko1221 74e25e165b Merge branch 'f-021-hostA' into f-021-int
# Conflicts:
#	apps/host/src/locales/de.json
#	apps/host/src/locales/en.json
#	apps/host/src/locales/es.json
#	apps/host/src/locales/ja.json
#	apps/host/src/locales/ko.json
#	apps/host/src/locales/pt.json
#	apps/host/src/locales/ru.json
#	apps/host/src/locales/zh.json
2026-10-19 20:30:04 +02:00
Niko1221 f5690d51a8 web-ask test: one data folder per bridge so the run exits; the scheduled-task case checks the refusal and the report 2026-10-19 20:25:20 +02:00
Niko1221 e2071c2f8b Downloads: 16 connections, several pieces per request, ModelScope length probe, faster MTP fetch
A single connection to Hugging Face's CDN stalls to 1-3 MB/s for 10-30 s again and again on the
owner's route; eight connections hide it, but a request per 32 MiB piece paid TCP slow start every
time. A request now covers up to four neighbouring pieces (the piece list and resumes are unchanged),
a try that moved bytes forward does not count as a failed one, and the default is 16 connections.
A server whose HEAD gives no length (ModelScope) is asked for bytes 0-0 so it gets the connections
too. mtp_fetch reuses the CDN address, reads the 28 shard headers side by side and lets the
connections run ahead of a slow range.
2026-10-19 20:07:33 +02:00
Niko1221 f2345ebd66 The model page and first-run page show the engine's own pick for this PC (memory tiers, Gyro-M as the other choice); the fit counts the draft layer (F) 2026-10-19 19:56:02 +02:00
Niko1221 671f4573da Fit checks and slot sizing count the draft layer's ~1 GB of graphics memory while MTP is on 2026-10-19 19:56:01 +02:00
Niko1221 3d0a79ca23 SAFETY.md: update checks only on a button press, web search asks once per chat in Manual, starter list refused 2026-10-19 19:54:54 +02:00
Niko1221 f523a94227 Chat footer speed: tool-call writing time counts as decoding time, so tok/s is output tokens over decode time 2026-10-19 19:54:48 +02:00
Niko1221 267f35f777 Chat with no model: plain words from the host, nothing sent before the engine's first state is known; tests for every state 2026-10-19 19:54:48 +02:00
Niko1221 71bdfda0b3 Offline web search: a pretend-offline mode for the search plugin and a test that the model gets the plain sentence 2026-10-19 19:54:48 +02:00
Niko1221 7662bf85c4 Starter model list: with the real engine on, the host refuses to download from it (unpinned files); it is display only 2026-10-19 19:54:47 +02:00
Niko1221 44a56865f5 Manual mode asks once per chat before the first web search or page; Settings, Tools switch to skip the question (off by default) 2026-10-19 19:54:37 +02:00
Niko1221 7f037f4451 Terminal: 'already downloaded' for a model on disk; strata monitor finds a running launch without --url (K) 2026-10-19 19:48:18 +02:00
Niko1221 5c9884cdee A build for another make of graphics card says so in plain words with the right download and is not unpacked (I) 2026-10-19 19:40:57 +02:00
Niko1221 5b4db785ca Start-up line: an installed engine with no model loaded is not 'No Strata engine' (J), and the wrong-make notice wiring 2026-10-19 19:40:57 +02:00
Niko1221 321f159032 Offline: the update check says there is no internet connection in plain words; offline start and load test 2026-10-19 19:40:05 +02:00
Niko1221 54a02dc32c Settings: opening Engine and Update makes no network call; the last check is shown and a new one runs only on a button press 2026-10-19 19:40:05 +02:00
Niko1221 40bf5f5c40 Merge branch 'f-021-settings' into f-021-int 2026-10-19 19:34:47 +02:00
Niko1221 bc54d945f6 Merge branch 'f-021-crash' into f-021-int 2026-10-19 19:34:46 +02:00
Niko1221 7a2bfc52ae Models: a per-model draft layer (MTP) switch, picture-off and custom context that stick
The installed app listed every model's pictures as "off", so each Load of a picture-capable model ran
setup again, which writes the arguments (--mtp, --spec-min-p, --vision, --max-context) and the vision
section from its own choices. Hand edits and the Read pictures tick came back each time.

- host: images comes from the run config's vision section, so a prepared model loads straight away
- host: Read pictures off no longer downloads/keeps the reader; --vision is left out of the engine arguments
- host: Draft layer (MTP) setting; "MTP per request" and it are the app's own and no longer frozen
- host: a context size set by hand shows in the Context list
- "mtp": false in the run config (strata_control/draft_layer.py) is applied at every start by
  serve/server.py, kept by setup, written by `setup --mtp off` and `strata launch --mtp off`
- strata_install accepts any context size, not only the listed ones
- docs: draft layer off, and what the app owns in the run config
2026-10-19 19:29:39 +02:00
Niko1221 e88ee39d1d Docs: state exactly which refusal checks ran (Linux packages) and the Windows PATH note for --version 2026-10-19 19:18:52 +02:00
Niko1221 a5ee74ffa3 Desktop: a failed start on Windows and Mac leaves a log, a crash file and a message box instead of 0xc0000409
Reported crash (0.2.0-beta, Windows 11 26300): strata-desktop.exe exits with 0xc0000409 at 0x31a315 and no log. That offset is the
fast-fail in Rust's abort after a panic (panic = "abort"); a Tauri setup error is a panic ("Failed to setup app"), and the shell had no
log on Windows, no console (windows_subsystem), so the reason was lost.

- the run log opens first in main (%LOCALAPPDATA%\Strata\logs, the folder Save logs already reads) and a panic hook writes the
  message and place to it and to desktop-crash-<time>.log, and shows a plain message box before the abort
- setup no longer returns an error to Tauri: a host that does not start or a window that cannot be made is reported the same way
  and ends the app with exit code 1; the build error is no longer an expect
- the host's output goes to the run log (it was lost without a console)
- plain next step in the box for a WebView2 problem and for a blocked host
- STRATA_NO_DIALOG (scripted runs) and STRATA_DESKTOP_TEST_PANIC (tests) for proving the net
2026-10-19 19:17:43 +02:00
Niko1221 74652692e1 Release matrix tests: Windows rdna3 (RX 7000, Strix Halo) and Linux halo builds, no arch in two hip builds of a platform 2026-10-19 18:32:56 +02:00
Niko1221 85f77658f3 Docs: RX 7000 and Strix Halo installers (Windows rdna3, Linux strixhalo) in the READMEs, AMD_HIP, STRIX_HALO and the 0.2.0 release notes 2026-10-19 18:27:13 +02:00
Niko1221 f57c0adcb3 Merge branch 'f-020-reset' into f-020-int 2026-10-19 17:12:10 +02:00
Niko1221 3fa1b524d7 Portable monitor bundle: carries consoleinfo.py and overview.py (manifest schema 6), the plain Monitor runs when launch is not bundled, smokes accept the Overview's /metrics read, test derives the bundle's import closure 2026-10-19 16:35:51 +02:00
Niko1221 76e51103c7 Merge branch 'f-020-ci' into f-020-int 2026-10-19 16:26:40 +02:00
Niko1221 fcca2ccb84 CI smokes: serve smoke expects the server-package preflight, server smoke accepts OK (skipped=N), nested-telemetry fixture deep enough for Python 3.12+ 2026-10-19 16:23:26 +02:00
Niko1221 eac694ae4c Issue and PR reset bot: intake check, /reopen, 5-day sweep, with node tests 2026-10-19 16:10:26 +02:00
Niko1221 4caf857c76 README translations: the new app and Terminal screenshots with translated captions 2026-10-19 16:03:04 +02:00
Niko1221 e5d987a553 Merge branch 'f-020-shots' into f-020-int 2026-10-19 16:02:32 +02:00
Niko1221 dde39e8a8f README app screenshot: crop the chat footer (its speed figure is wrong, see the 0.2.1 plan) 2026-10-19 16:02:32 +02:00
Niko1221 03b75b8da9 README: new app and Terminal screenshots for 0.2.0-beta 2026-10-19 15:52:41 +02:00
Niko1221 3d1fb48090 Release notes 0.2.0: which file to download, memory tiers, faster checked downloads, the chat fixes, signed Mac 2026-10-19 15:29:09 +02:00
Niko1221 042cd201c9 Merge branch 'f-020-tr2' into f-020-int 2026-10-19 15:04:28 +02:00
Niko1221 b961c7aa8c README: use the app's own UI strings in the ja, pt-BR and zh-CN translations 2026-10-19 15:04:16 +02:00
Niko1221 6ec549d2dd README de/fr/es: use the app's own labels, French keeps English UI names; tidy colon asides 2026-10-19 15:03:27 +02:00
Niko1221 4ff5972fcb README: new Japanese, Brazilian Portuguese and Simplified Chinese translations of the English README 2026-10-19 15:02:13 +02:00
Niko1221 04a2e6a3f1 README: new German, French and Spanish translations of the 0.2.0 README 2026-10-19 15:01:57 +02:00
Niko1221 c301209636 README: the Windows AMD app is for RX 9000 / R9700 cards; other AMD cards point to AMD_HIP.md 2026-10-19 14:57:39 +02:00
Niko1221 78bba245a5 README: file-per-card download table, what-you-need before install, shorter blocks; drop the Mac damaged workaround 2026-10-19 14:57:15 +02:00
Niko1221 18f7b90ca6 Feed: the app entry may carry a pre-release version (0.2.0-beta); the strict key rejected the whole release feed; UPDATES: the release steps for the engine-less update installers, --url-prefix=, Control version 2026-10-19 14:01:29 +02:00
Niko1221 1b8e88b9e5 feed_tool: a Linux or Mac package packed on Windows keeps its programs executable (mode from the file header); test 2026-10-19 12:19:17 +02:00
Niko1221 ef59c0b555 Merge f-020-int (final 0.2.0-beta source 4ea817163) into f-020-rel 2026-10-19 11:53:08 +02:00
Niko1221 4ea8171636 Merge branch 'f-020-dl' into f-020-int 2026-10-19 11:51:12 +02:00
Niko1221 3c3d2881b4 Merge f-020-int into f-020-rel 2026-10-19 11:35:31 +02:00
Niko1221 604e23cd9a Engine-less app update: tauri.update.conf.json drops engine-bundle so one latest.json entry per OS fits every installer variant; tests for the overlay, for the engine-less update on an installed AMD engine, and for the release engine selection (CUDA 13/12 by driver and card, HIP by gfx, Metal) 2026-10-19 11:35:23 +02:00
Niko1221 41cc4b2e89 Plans: no step counter in the progress lines (owner: 'Step 2 of 5' removed like the goal's) 2026-10-19 11:35:11 +02:00
Niko1221 3d9c21509e Merge f-020-int into f-020-rel 2026-10-19 11:32:19 +02:00
Niko1221 779bc30501 Merge branch 'f-020-gui' into f-020-int 2026-10-19 11:29:58 +02:00
Niko1221 d5bcc3bd8d GUI fixes: wait for the assistant to start, no step counter on goals, files open without a chosen folder, results open in the sidebar 2026-10-19 11:29:40 +02:00
Niko1221 116adbf7d6 tester README (Linux NVIDIA): the driver line names the CUDA 13 engine's minimum, 580.65 2026-10-19 10:53:25 +02:00
Niko1221 19d5cd87fa install.ps1: name a discrete graphics card before the built-in one in the line it prints 2026-10-19 10:53:24 +02:00
Niko1221 2ca90e48ec binary_deps: a Windows DLL found only in a sub-folder counts as bundled only when BUILD.json lists that folder in lib_dirs (the RX 9000 package's rocblas.dll under rocm/bin made a plain start die with 0xC0000135) 2026-10-19 10:53:24 +02:00
Niko1221 adfb7ecb2a packaging: the Control python launcher uses no external command (dirname failed inside the AppImage on newer systems); the engine start error names a missing Windows DLL (0xC0000135) and says how to fix it 2026-10-19 10:52:51 +02:00
Niko1221 a122ba0bdf Mac signing: lessons of the first real run (entitlements plist LF everywhere, TMPDIR's trailing slash no longer makes a // path, the Terminal python check finds the python3 symlink) 2026-10-19 10:50:55 +02:00
Niko1221 04221c3e2b Download big model files over several connections
download() reads a file of 256 MiB or more in 32 MiB ranged pieces over 8 connections
(STRATA_DL_CONNECTIONS=1 restores one stream), writes each piece at its own offset,
and keeps the finished pieces in <name>.part.chunks so a stopped download continues
without fetching them again. The redirect to the signed CDN address is resolved once
and again before it expires or after a 401/403/410. A server without Range support
or without a length is read in one stream as before; an older single-stream .part
continues after its last whole piece. The progress line and the .part size the app
reads are unchanged, the SHA-256 pin is still checked on the whole file, and the
.done mark is the same. tools/mtp_fetch.py reads its ranges in parallel too.
2026-10-19 10:38:35 +02:00
Niko1221 fdb5e48616 Merge branch 'f-020-macrel' into f-020-int 2026-10-19 09:33:22 +02:00
Niko1221 af78949b17 build-feed writes install-manifest.txt from the Control packs (what install.sh and install.ps1 read); test of the release path; docs 2026-10-19 09:28:23 +02:00
Niko1221 935407d883 Mac in-app update: sign-and-notarize.sh --updater-archive packs the notarised app; feed_tool set-app-platform verifies the Tauri signature and writes the update entry
tauri build makes Strata.app.tar.gz and its .sig before sign-and-notarize.sh re-signs and notarises the app, so the published Mac update would be the unsigned app. The script now writes Strata.app.tar.gz from the stapled app on request; the Tauri key stays on the PC (npx tauri signer sign), and set-app-platform checks the .sig against tauri.conf.json's pubkey over the exact bytes and records url, signature, sha256 and size.
2026-10-19 09:18:13 +02:00
Niko1221 899d1b1b3a metal: restore the stubs for gemm_iq_f16_grouped, native_q8_0_sm60_pack/_release and swiglu_split_f16 (CUDA-only paths the Metal engine links but never runs)
The Metal engine did not link from f-020-int: prefill.cpp and native_head.cpp reference these four entry points. All answer 'nothing done'.
2026-10-19 09:17:56 +02:00
Niko1221 02a8193da7 test_strata_mcp: the Windows AMD recommend test no longer reads this PC's installed engine (hermetic; fails on a PC with a CUDA engine since e107297a8) 2026-10-19 09:07:01 +02:00
Niko1221 c7bfd131b5 Merge branch 'f-020-int' into f-020-golden 2026-10-19 08:51:07 +02:00
Niko1221 6f2a3e758d Merge branch 'f-020-sha' into f-020-int 2026-10-19 08:41:46 +02:00
Niko1221 e193161de2 Pin SHA-256 and size for every model file setup downloads; stop when the pinned version is gone
Qwen3.8-Flash-Next (ISTA), Swift 1.5, the Coder and the picture reader get fixed
hashes (REPO_PINS, checked in download()). A 404 on the pinned commit stops instead
of taking the current file; STRATA_ALLOW_UNPINNED=1 is the announced tester opt-out
(setup.py and tools/mtp_fetch.py). Swift moves to the commit that has its IQ3_S.
Tests for pins, mismatch, 404, opt-out and offline use; SAFETY.md updated.
2026-10-19 08:41:24 +02:00
Niko1221 06b6b23e51 Merge branch 'f-020-int' into f-020-golden 2026-10-19 08:38:10 +02:00
Niko1221 553c25ca7c Merge branch 'f-020-tiers' into f-020-int 2026-10-19 08:36:15 +02:00
Niko1221 8ac47258bb tests: wizard tests pick IQ3_XXS by hand now that IQ3_S is pre-selected at 64 GB 2026-10-19 08:35:13 +02:00
Niko1221 dbd1cb9af8 setup golden: re-record the run configs after pictures-on-by-default (91baead75) and the memory tiers (aa82a8863); classification in phaseA-tests/golden-020.md 2026-10-19 08:34:26 +02:00
Niko1221 94f095c893 tests: launch TUI pins follow the 64 GB tier (IQ3_S) 2026-10-19 08:33:19 +02:00
Niko1221 a4c1644a37 Merge branch 'f-020-tiers' into f-020-int 2026-10-19 08:30:20 +02:00
Niko1221 aa82a88639 Recommend by the owner's memory tiers (RAM + VRAM): 40 GB Swift 1.5 IQ2_XS with Gyro-M named, 48 GB IQ3_XXS, 64 GB IQ3_S; MODELS.md Pick by memory 2026-10-19 08:26:54 +02:00
Niko1221 9962aae5b7 Merge branch 'f-020-safety' into f-020-int 2026-10-19 08:16:03 +02:00
Niko1221 cbbdd16b43 SAFETY/README: no 'nothing leaves your PC' overclaim (web search is on by default); neutral wording for Mac signing 2026-10-19 08:16:03 +02:00
Niko1221 25492e4a6b Docs: add SAFETY.md (what Strata downloads, when it uses the internet, what stays local, how downloads are checked, what the agent may do) and link it 2026-10-19 08:15:11 +02:00
Niko1221 7e3a9fec71 tests: DOCKER.md no longer carries TODO(owner); check the not-published line instead 2026-10-19 08:13:25 +02:00
Niko1221 2a6650a364 README: memory tiers per the owner (40 GB Swift 1.5 or Gyro-M with its speed note, 64 GB IQ3_S original or Swift), low-RAM line, Gyro-S vs IQ2_XS 2026-10-19 08:11:47 +02:00
Niko1221 5e7b9a8301 Docs: plain-wording pass on the 0.2.0 text (fewer mid-sentence bolds, direct phrasing) 2026-10-19 08:01:47 +02:00
Niko1221 2b209b406d Docs consistency pass: pictures on by default on NVIDIA and AMD (Intel Arc off for now), seven monitor views, area prefix for issue and PR titles in AGENTS.md 2026-10-19 08:00:26 +02:00
Niko1221 6ff75059a9 Merge f-020-docs3 into f-020-int (0.2.0 release wiring and docs)
# Conflicts:
#	packaging/UPDATES.md
2026-10-19 07:59:35 +02:00
Niko1221 e46be1fe0a Merge f-020-docs2 into f-020-int (0.2.0 release wiring and docs) 2026-10-19 07:59:18 +02:00
Niko1221 f86ca9d1c5 Merge f-020-docs into f-020-int (0.2.0 release wiring and docs) 2026-10-19 07:59:17 +02:00
Niko1221 477fd36bce Merge f-020-release into f-020-int (0.2.0 release wiring and docs) 2026-10-19 07:59:17 +02:00
Niko1221 193cbb1fbf README redesign: logo header, clearer install per system, memory rule, Gyro for low memory, library analogy, support section
Header uses the existing brand lockup. Performance tables get bars and plain labels, with the engine versions left to DETAILS. What you need is built around 32 GB of RAM plus VRAM and links the platform docs. Troubleshooting moves below How does it work.
2026-10-19 07:55:50 +02:00
Niko1221 76e4fef398 Intel Arc pictures stay opt-in until the SYCL engine's first-picture GPU fault is fixed; UPDATES: publish as a normal release 2026-10-19 07:48:06 +02:00
Niko1221 dac635445b CONTRIBUTING.md: a Titles section (area prefix for issues and pull requests) 2026-10-19 07:42:35 +02:00
Niko1221 046bdc83d9 Titles start with their area: Area dropdown and title prefix in the issue forms, a benchmark form, a PR template line, a title-area workflow 2026-10-19 07:42:27 +02:00
Niko1221 667d3cf7d7 Intel Arc: pictures on by default through the portable CPU reader (setup_intel, shared /dev/shm for the embeddings) 2026-10-19 07:28:38 +02:00
Niko1221 4794b13779 strata launch --help states the picture rule; strata monitor is the seven-view monitor (dead six-view fallback removed), tests 2026-10-19 07:28:38 +02:00
Niko1221 09b8b54d19 Release app config: the engine travels inside every installer, signed updater artifacts overlay, macOS 15 minimum 2026-10-19 07:28:37 +02:00
Niko1221 98b865c56f Release wiring: release feed key and Tauri updater key, GitHub Releases feed address in the installers, versions 0.1.45.3 and 0.2.0-beta 2026-10-19 07:23:44 +02:00
Niko1221 91baead754 Pictures on by default on every make of card: MCP install, Qwen3.6, Unsloth and the original Flash-Next families follow the one reader rule (CPU under 24 GB, GPU from 24 GB); the CPU reader keeps no VRAM reserve 2026-10-19 07:23:44 +02:00
Niko1221 80a10f31c3 docs: details, batching and tester readmes for 0.2.0
DETAILS: pictures on by default with the 24 GB reader rule, the engine label of the prompt-chunk planner (before 0.1.45.3, not 0.1.46), the per-architecture sets measured on 3060, A4000 and P100, the N7 and ROUTE_PRIOR decisions, a pointer to the app and Terminal troubleshooting. Links to private phaseA-tests notes removed from DETAILS and BATCHING. Tester readmes: AMD pictures are on (CPU under 24 GB, GPU on a 24 GB 7900 XTX), the RX 7900 text reconciled with the community runs, Mac needs macOS 15 and has no picture reader yet.
2026-10-19 07:16:00 +02:00
Niko1221 b27c46cbd6 docs: Mac, AMD, Intel, older GPUs and multi-GPU for 0.2.0
MAC: the .dmg and the Terminal bring the Metal engine; pictures only with the source build. AMD_HIP: the 24 GB
picture rule and the measured R9700 GPU reader. INTEL: the grouped XMX label is engine 0.1.45.3 (not 0.1.48), plus
the A750 prompt-hang fix (fca3c5958; STRATA_PF_DMA_PAGEABLE=1 restores the old copy). INTEL_ARC: maintainers now test
on a B70 and an A750; where the Intel engine comes from. OLDER_GPUS: the P100 sm_60 defaults (STRATA_SM60_PERF).
MULTI_GPU: two models on two cards, one make per engine. MODELS and TROUBLESHOOTING: the Mac and Intel Arc have no
picture reader yet.
2026-10-19 07:15:14 +02:00
Niko1221 1c4970b9ac 0.2.0 release notes: the split, installers, Terminal, pictures, new defaults with measured numbers, tester fixes, known limits 2026-10-19 07:09:40 +02:00
Niko1221 26613a44f6 AGENTS, AI_SETUP, INSTALL: the app and the Terminal first, install from source as the fallback
Agents install the Terminal with the one line and run strata launch --model NAME --yes --no-tui (flags checked against
strata launch --help). Pictures follow the 24 GB rule on every make; the Mac has no picture reader yet. INSTALL gains
The app and The Terminal sections, update paths for both, and a Mac source section. AGENTS points to CONTRIBUTING.md.
2026-10-19 07:08:46 +02:00
Niko1221 afe92c1727 docs: models, NVFP4, Gyro, Qwen3.6 and troubleshooting for 0.2.0
Pictures are on by default (CPU reader under 24 GB, GPU reader from 24 GB, every make; no Mac reader yet).
Gyro and NVFP4-QAD read pictures; Gyro needs engine 0.1.45.1+ and is correct on RTX 50 since the SwiGLU guard.
The three NVFP4 versions get their app names. A Qwen3.6-35B-A3B section, the app and Terminal ways to add a model,
and engine versions on the older speed tables. Troubleshooting: no-engine warning, needs a newer engine / Update
engine, Mac xattr steps, strata doctor, Save logs. Private test-notes references removed.
2026-10-19 07:07:54 +02:00
Niko1221 ad262c80aa README: the app installers and the one-line Terminal install first; build from source moves lower
Pictures on by default (CPU reader under 24 GB, GPU reader from 24 GB, every make; none on the Mac yet), the Mac app,
the Terminal in a short section, the Anthropic base URL without /v1, two models on two cards, one make per engine.
The speed tables stay and say which engine measured them. A Contributing line links CONTRIBUTING.md.
2026-10-19 07:05:43 +02:00
Niko1221 14ec155ec7 0.2.0 text, group 3: Terminal docs (installed command, seven views, release address, connect --check json), Docker, packaging, SECURITY, issue and PR templates, test requests, historical Terminal pages 2026-10-19 07:04:59 +02:00
Niko1221 e9e08adb37 Add CONTRIBUTING.md and the draft standard benchmark protocol (owner text, unchanged) 2026-10-19 06:58:45 +02:00
Niko1221 fd0e3766a3 Merge f-split-tuiqa (95f380769) into f-split-int: TUI QA fixes (launch with flags, discovery, monitor menu, connect checks, layouts, no browser, warnings), picture reader CPU under 24 GB / GPU from 24 GB 2026-10-19 00:03:56 +02:00
Niko1221 95f3807699 Merge f-split-int d6aa6504e into f-split-tuiqa: launch warns and asks (setup stops only for NEED_ENGINE unless launch said go on), the app warns; one picture-reader rule in strata_control/vision_reader.py (CPU reader under 24 GB, GPU reader from 24 GB, every make; pictures stay on) for setup, launch and the app catalog 2026-10-19 00:03:31 +02:00
Niko1221 a2367ca55b Mac: one script to sign (Developer ID, hardened runtime) and notarise the app dmg and the Terminal archive; Mac README text for notarised and not notarised builds 2026-10-19 00:01:30 +02:00
Niko1221 e657d310e7 models import: a folder with several models says which ones it holds and asks for one file, in plain words 2026-10-18 23:33:52 +02:00
Niko1221 64af23dafc Terminal shows the app's version (0.2.0-beta) with the build number beside it; the archive README no longer contradicts itself, ships the launch doc it links and gives free-memory guidance for the system it is for; launch doc updated 2026-10-18 23:28:01 +02:00
Niko1221 bccb0f4280 Terminal wizard: the Save choices stay on screen at 60x20 and 40x15, the Advanced command has its own scroll area, key hints wrap and say what Enter does; request filter explains itself and matches LIVE/DONE; plain-word setting descriptions; tuning default explained; monitor bundle carries nav.py (T09, T11) 2026-10-18 23:23:09 +02:00
Niko1221 adc4a6822b Advanced command preview follows the wizard's context, memory format and picture choice (T09 part 1) 2026-10-18 23:00:01 +02:00
Niko1221 3d2e7f166e strata connect --check --output json runs the same checks and returns each result: ok is false and the exit code 1 when a check failed or the server is unreachable; the app list says Vendor tested (T07) 2026-10-18 22:58:57 +02:00
Niko1221 545fccf007 Terminal monitor: one seven-view model (arrows and 1-7 reach Connect, wrap, hints say 1-7), the selected tab always marked and the names kept at narrow widths, Overview titled Overview, Connect separates vendor-tested history from this server's live result, light theme for Connect and pop-ups (T06, T08, T10) 2026-10-18 22:57:36 +02:00
Niko1221 d6aa6504e0 Merge f-amd-vision (d5fda4d2f) into f-split-int: GPU picture reader for AMD (strata-vision-hip) with automatic CPU fallback; pictures on by default on AMD; packaging for Linux and Windows 2026-10-18 22:53:23 +02:00
Niko1221 d5fda4d2f3 build_windows.bat: STRATA_VISION_HIP_ARCHS 2026-10-18 22:50:16 +02:00
Niko1221 8667a3dbf7 Merge f-split-int (a3024ed10) into f-split-qa1: qa2's s3237-s3238 kept, my 21 strings renumbered s3260-s3280 2026-10-18 22:46:39 +02:00
Niko1221 b7b65edeb1 build_hip_linux.sh: VISION_HIP_DEFS 2026-10-18 22:46:29 +02:00
Niko1221 4440a303f8 Terminal: no browser opens by itself (launch config forbids it, --open is the only way), and the start message promises a browser only with --open 2026-10-18 22:46:10 +02:00
Niko1221 e107297a8f Mixed makes of graphics cards: only the cards the engine can drive are offered, never a cross-make split, one plain line when the other make has the bigger card, two models at once said to be unsupported (launch, setup, app card list) 2026-10-18 22:44:12 +02:00
Niko1221 6b8cfcd1a3 strata-vision-hip (Windows): hipblas.dll is delay-loaded and rocm\bin is put on the DLL search path first, so the reader starts from any PATH 2026-10-18 22:41:18 +02:00
Niko1221 d060783fbb Terminal: pictures are on by default when the model and the engine can read them (card for CUDA and HIP, CPU otherwise); --vision off turns them off 2026-10-18 22:39:16 +02:00
Niko1221 a3024ed10a Merge f-split-qa2c (10f3435b0) into f-split-int: Linux drag-drop through the page widget's GTK drag signals; export header test 2026-10-18 22:39:06 +02:00
Niko1221 c4345a7c03 Terminal: no engine for this computer is a warning, not a refusal: ask (default yes, --yes skips), download the model, end with the engine problem and the next step, exit 0 2026-10-18 22:33:43 +02:00
Niko1221 10f3435b00 Test: the saved Monitor picture with a model loaded is headed by the logo and the model and time line (NativeMonitor is not used by any screen) 2026-10-18 22:30:01 +02:00
Niko1221 35698ffe70 Linux: dropping a file on the window works under WebKitGTK 2.52: the shell reads the file:// addresses from the GTK drag signals, accepts the drag itself, takes the leave with every button up as the drop and hands the paths to the host 2026-10-18 22:28:58 +02:00
Niko1221 e6ab4c024d No recognised engine: the app still downloads the model and warns in its card; loading names the next step
setup.py control_engine_step stops only for strata launch (STRATA_NEED_ENGINE=1); the app warns and continues. The Models card and the first-run
download card show the owner's plain warning before and during the download; the load refusal says the same with Settings > Engine.
2026-10-18 22:28:15 +02:00
Niko1221 23b08c7620 Terminal: one discovery of existing installs and model files for the screens, models list, own-files import and launch; Gyro-S/M GGUFs are known; installed models show what is really missing (T03, T04, T05) 2026-10-18 22:22:31 +02:00
Niko1221 e23a56f2f8 ENGINE-COMPONENT.md: the picture readers in an AMD package 2026-10-18 22:19:49 +02:00
Niko1221 b0b8860d13 build_windows.bat: the HIP reader's host compiler gets the ROCm device-library path 2026-10-18 22:19:18 +02:00
Niko1221 1486151d4d Terminal: strata launch with flags opens its run screen instead of crashing, and a crash of the screens exits non-zero with a plain sentence (T02) 2026-10-18 22:11:21 +02:00
Niko1221 97cb031bf7 docs: AMD pictures run on the card by default, with the CPU reader as the automatic fallback 2026-10-18 22:04:25 +02:00
Niko1221 c59505e638 build_windows.bat: STRATA_ROCM_KEEP uses the ROCm venv as it is 2026-10-18 22:02:17 +02:00
Niko1221 34bd640a19 Merge f-split-qa2b (b3348da49) into f-split-int: the saved Monitor screenshot shows the logo alone in its top line 2026-10-18 22:01:44 +02:00
Niko1221 0f5511cfdf Merge f-split-qa2 follow-ups into f-split-int: drop guard (no navigation to a dropped file), shell-to-host drop hand-over 2026-10-18 22:01:44 +02:00
Niko1221 b3348da493 Monitor: the saved screenshot starts with the Strata logo alone, without the Monitor headline 2026-10-18 22:01:21 +02:00
Niko1221 b31c13875e AMD picture reader: the no-GPU/arch refusal only in the HIP program; the app's CPU choice starts the CPU reader program 2026-10-18 21:55:51 +02:00
Niko1221 b2e3008bfb build_hip_linux.sh: kernel data collected once 2026-10-18 21:51:33 +02:00
Niko1221 55ca7b955a launch: the Pictures screen says the AMD picture reader runs on the graphics card; build_hip_linux.sh collects libraries from the build outputs 2026-10-18 21:51:10 +02:00
Niko1221 979da9d2bc Terminal launch list: Qwen3.6 (own engine path) stays out of Flash-Next's rows 2026-10-18 21:44:49 +02:00
Niko1221 536c7f019e Windows HIP zip: build_windows.bat builds the two picture readers and package_windows.py ships them 2026-10-18 21:44:45 +02:00
Niko1221 de9101935d setup: AMD pictures on by default (the HIP reader with the CPU reader behind it, else the CPU reader), on Windows too; source build compiles the HIP reader; package_engine carries strata-vision-hip 2026-10-18 21:43:23 +02:00
Niko1221 f82fffbe25 picture reader choice for AMD engine folders and the server's fallback from the GPU reader to the CPU reader (one log line) 2026-10-18 21:39:13 +02:00
Niko1221 b808d6e442 Linux and macOS: dropping a file on the window works. WebKit gives the page the drag but not the files, so the shell keeps its drop handler, hands the paths to the host, and the page fetches each file once; a file dropped outside the chat no longer takes the window to it 2026-10-18 21:36:51 +02:00
Niko1221 75127c794e strata-vision: fix the escaped newlines 2026-10-18 21:34:28 +02:00
Niko1221 96658aebae packaging: build_hip_linux.sh builds the HIP engine package with both picture readers 2026-10-18 21:32:58 +02:00
Niko1221 fb14c354b4 strata-vision: HIP build (strata-vision-hip) for AMD GPUs; the reader refuses a GPU that is not in its build; RUNPATH $ORIGIN 2026-10-18 21:31:30 +02:00
Niko1221 43ea31544b native poll: a read that never answers is dropped after 60 s 2026-10-18 21:31:21 +02:00
Niko1221 a1788c54be Read pictures is on unless the person turned it off: an installed model with no saved choice starts checked 2026-10-18 21:30:17 +02:00
Niko1221 7f7bfa962b Merge f-split-qa3 (283258664) into f-split-int: web search back-off and Bing fallback, turn guard with a final message, /loop stops by count/time/condition, Anthropic base URL without /v1 2026-10-18 21:25:06 +02:00
Niko1221 2832586641 Chat stream: translate the app's own delta lines before the delta branch, so the answer-text code stays as merged 2026-10-18 21:17:43 +02:00
Niko1221 a04244a4be Merge f-gyro-swiglu (8837cb6f0) into f-split-int: the fused gate/up SwiGLU path takes only 80-dot rows; Gyro's trellis types fall back to the full-warp kernel (correct Gyro output on sm_120) 2026-10-18 21:13:13 +02:00
Niko1221 714ca744dc Merge f-split-int (c157ea373) into f-split-qa3 2026-10-18 21:10:12 +02:00
Niko1221 e5476b0e12 Merge f-split-int (c157ea373) into f-split-qa1: my new strings renumbered after s3236 (s3237-s3255) 2026-10-18 21:09:33 +02:00
Niko1221 fef5dfec03 Storage list full width; version list title keeps its spaces 2026-10-18 21:08:41 +02:00
Niko1221 c157ea373f Merge f-split-qa2 into f-split-int: drag-drop in the native window, files-edited count only from tool changes in the workspace, answer-text guard, dialog focus, skill enabled flag, number field checks, folder picker release, small-window composer, Monitor screenshot feedback 2026-10-18 21:07:47 +02:00
Niko1221 40b41da916 Turn guard: a search-heavy /plan that keeps failing is stopped by the app, and a full context never ends in silence
A /plan on a small model searched again after every refused search and rewrote its plan after each one until the context reached 100%,
then the turn ended with no answer and no message. The bridge now counts what the turn shows (turn-guard.mjs): 3 failed searches in a
row, 10 searches in a plan turn (25 otherwise), a plan written from 1. again 4 times, a local model's context at 93%. The turn is stopped the
way Stop stops it but ends as a normal turn: the plan it had is still proposed (the plan card says why it stopped early), any other turn
ends with one plain translated sentence with a next step (frames carry the English sentence as `key`, the page translates it, 8 languages),
and the Harness's own /compact is asked for so the next message has room. A turn that ends with no words at all always gets a message:
"I ran out of room in this chat..." at 85% of the window or more, otherwise "I stopped without writing an answer...". Settings -> Tools says
results may come from Bing when DuckDuckGo is busy. Tests: counting (pure) and three end-to-end runs through the real Harness with the scripted
model (searchloop / bloat) and the real search code on a pretend web that refuses (bridge option searchMock), no network.
2026-10-18 21:07:38 +02:00
Niko1221 31030b170a storage and version: one Downloaded models list from the real files, Check folder unloads nothing, app version shown as Strata
- tools/strata_mcp.py: disk usage from the files a model really uses (every part of a multi-part GGUF, the picture reader, packs, draft layer),
  a file named by several flags or shared by several models counted once; Delete's plan stays separate (removable_gb)
- Settings > Storage: one list and one total (optimized models and other local models), shared files as a line of their own
- Check folder on the same folder no longer unloads the loaded models (O01); an actual change still asks first
- Settings shows the app version (0.2.0-beta) as Strata with the engine beside it; Control's 0.1.45 is no longer shown as Strata
  and no longer goes into the update check as the app version
2026-10-18 20:55:06 +02:00
Niko1221 b23dad1d5f scripts/qa2-flows.mjs: browser checks for the QA fixes (dialog focus, number fields, small window, picture drop, screenshot) 2026-10-18 20:54:29 +02:00
Niko1221 8837cb6f0f gu_swiglu: the 80-dots-a-row guard for every format (Gyro's trellis gate/up rows went through the 16-lane kernel)
launch_gu_swiglu (STRATA_GU_SWIGLU, on by default on compute capability 12.0 cards since 147-verify) runs
16-lane gate/up rows that assume 80 dots a row, but its guard rejected only the K-quants. The APR trellis
types (TQ2_T / TQK6 gate/up: qk 128, ipb 8, 160 dots at n_embd 2560) went through it: half of each row,
Gyro-S answered with loops, garbage or nothing on every RTX 50 / RTX PRO Blackwell card. They now take
launch_gu's full-warp kernel, as launch_gu itself always did for them; 80-dot formats are unchanged.

l620 (RTX PRO 5000, Gyro-S through serve/server.py, the owner's QA prompts, greedy and the app's sampling):
before 0/8 correct, after 8/8 ("STRATA_OK 42", "4", the follow-up sentence, "PING"), the same as
STRATA_SM120_PERF=0 / STRATA_GU_SWIGLU=0; Gyro-S fix vs STRATA_SM120_PERF=0: identity gate 10/10, exact_pp
4K / 20K IDENTICAL (KL 0).
2026-10-18 20:54:11 +02:00
Niko1221 707eebd1dd Web search: spaced and backed-off requests, a session cache, Bing as the second keyless source, one plain message when every source refuses
The default DuckDuckGo search failed with "limiting searches from this computer" after two or three quick searches (reproduced on a
Linux box: two searches 1.2 s apart answered, the third got HTTP 202 and every DuckDuckGo page, html and lite, refused for many minutes
while it was polled). Now: browser-shaped requests (Referer, Origin, Sec-Fetch-*), one request at a time per source 2.5 to 4 s apart with
random jitter and at most 8 a minute, a 30 minute session cache per question, a per-source cooldown after a refusal that doubles from
1 to 10 minutes (Retry-After honoured) during which that source is not asked, no lite-page follow-up after a refusal, one retry for a
timeout or 5xx. When DuckDuckGo refuses, Bing's result page is used (links decoded from its redirects). When every source refuses the
tool says, in one plain sentence, that web search is limited, to try again in a minute or add a key (Exa, Perplexity or DeepSeek) in
Settings -> Tools -> Web search, and tells the model not to retry. Tests use a mocked rate-limited search and a real Bing page fixture.
2026-10-18 20:53:37 +02:00
Niko1221 f4cb8095c4 Locales: the new plain sentences (number range, Add skill from another computer, screenshot saved, folder window) 2026-10-18 20:52:35 +02:00
Niko1221 2325b2ee26 Desktop: pictures can be dropped on the chat in the Windows window (the shell's own drop handler is off, the page gets the drop); the drop box is animated 2026-10-18 20:52:05 +02:00
Niko1221 7572afc5bf Monitor: saving the screenshot says so (it is in the Downloads folder) or says it failed 2026-10-18 20:51:04 +02:00
Niko1221 4794a8067f Test fix for the message box cap 2026-10-18 20:49:39 +02:00
Niko1221 95ece0bcd2 Chat: the message box stops at 30% of the window height and scrolls inside, so Send and the toolbar stay in view in a small window 2026-10-18 20:49:29 +02:00
Niko1221 e0aa3b872a Windows folder window: the host waits for the window to report it is opening and otherwise asks for the path in words, so the button is released; Add skill from another computer says why it cannot 2026-10-18 20:48:17 +02:00
Niko1221 1237d9a5fa downloads and models: live progress, parts counted, choices kept, clear NVFP4 names, More versions list
- native state: the host's 'native' events are applied at once; the light poll (1 s while a job runs, none while hidden) is no longer
  held by a slow forced read; a read that never answers is dropped after 20 s
- install progress: every file of the model is counted (parts, picture reader beside the model folders, NVFP4 packs by their pinned sizes);
  the bar no longer sits at 99 % while the picture reader downloads; the screen names the part (part 2 of 4) and lists the parts
- a config is 'ready' only when every part of a multi-part GGUF is whole
- Models page keeps version, context, computer use, picture reading and expert choices across tabs; Read pictures is on for a version that
  is not downloaded yet; the running download is named
- version list: smaller rows, a Downloaded tag, NVFP4 / NVFP4-NVIDIA / NVFP4-QAD named by who made them with one line each
- first-run page: the four recommended versions, then a More versions list (the Models page list)
2026-10-18 20:47:43 +02:00
Niko1221 337c70d3c2 Response settings: a number outside its range (Max output tokens -1) is refused with a plain sentence in the dialog and by the host 2026-10-18 20:46:09 +02:00
Niko1221 e6aa6f317f Skills: a skill created with Skill enabled off is saved off (and keeps its trigger phrases) 2026-10-18 20:44:33 +02:00
Niko1221 969d7da7df Narrow window: a dialog opened from the menu closes the menu and takes the keyboard (focus also moves in when animation frames are held back) 2026-10-18 20:44:33 +02:00
Niko1221 c0abfdc358 /loop stops by itself: a count, a time or a stop condition in words is read when the loop is created and enforced by the scheduler
"/loop 1m say hi, stop after 3 answers" kept firing and the model answered "stopped" every minute until /loop stop. The count
("stop after 3 answers", "three replies", "5 times"), the time ("for the next 2 hours", "stop after 30 minutes") and a stop
condition ("stop when it passes", "until the file exists") are now parsed out of the line, left out of the message the model gets,
and enforced by a new scheduler (lib/loop-runner.ts): after the last allowed answer nothing is sent again, and the person is told in
one plain sentence why it ended. The scheduler also ends the loop when an answer says it is done or stopped (a bare "Stopped."
always; "Done ..." and similar when a limit was stated; DONE as the first word when a stop condition was given in words), and after
three failed answers in a row. A busy chat waits without using up the count. Tests drive the scheduler with a scripted model and a
fake clock. Strings in 8 languages.
2026-10-18 20:41:28 +02:00
Niko1221 323a11e721 Connect app: the third tab and strata connect give the server root (no /v1); the server also accepts /v1/v1/messages
Messages-API clients (coding CLIs, the SDKs) add /v1/messages to their base URL themselves. The Connect dialog showed the OpenAI-style
base that ends in /v1, so those apps posted /v1/v1/messages and got a 404. The third tab now shows the root address with a
one-line note (the base-URL variable for coding CLIs), the other tabs keep /v1. strata connect cuts a pasted /v1 back to the root for
every app and says so for coding CLIs, and strata launch and the monitor's Connect view list the Messages-API address. The server (serve
and the app's host, plus the shared listener) reads /v1/v1/messages and /v1/v1/messages/count_tokens as the real paths; nothing else
is aliased. Tested with the real Python SDK client against the mock-engine server.
2026-10-18 20:40:33 +02:00
Niko1221 f4078faf4f Answers: reasoning wrapper tags and a stray leading tag fragment never show in the answer or the Markdown export (UI side only) 2026-10-18 20:36:25 +02:00
Niko1221 e09eccc839 Files edited: only report what the turn itself wrote (a cut-short first look no longer makes the rest of the folder new files); never look in app, engine, component, AppData or temp folders 2026-10-18 20:33:55 +02:00
Niko1221 83f2a2c73a model list: Qwen3.6-35B-A3B has its three sizes (the first-run page showed no versions)
The catalog only walked setup.FAMILIES; Qwen3.6 lives in OWN_PATH_FAMILIES, so the app's Qwen3.6 page had an empty list.
strata_models now lists its sizes (download, memory, minimum engine 0.1.45, pictures) and strata_install plans them.
2026-10-18 20:33:08 +02:00
Niko1221 66ad879d18 Merge f-149-default (bac1a3e0e) into f-split-int: Linux defaults for the GGUF read in place (zero-copy page-cached experts, io prefetch with the usefulness/room gate), cgroup-aware RAM check, docs 2026-10-18 17:22:54 +02:00
Niko1221 76ceb365ca Merge f-split-launchfix (8f2f34ec4) into f-split-int: strata launch refuses at once when no usable engine exists, before any model download 2026-10-18 16:52:18 +02:00
Niko1221 8f2f34ec4e strata launch: settle the engine before any model download
strata update exits 0 when the feed has no engine for this PC (or only one for another card family), so launch went on and downloaded the model before failing with the generic engine line. ensure_engine now re-checks the engine after the update step and refuses at once with the update's own plain sentence plus one next step; setup.py in Control mode stops at the engine step for a build of another card family, an old driver, or when launch asks for an engine (STRATA_NEED_ENGINE=1), before the download step. The app without any engine keeps its model-first order.
2026-10-18 16:50:19 +02:00
Niko1221 bac1a3e0e9 Merge f-149-prefetch (docs) into f-149-default 2026-10-18 16:31:49 +02:00
Niko1221 24dc4a1dd6 docs: the 0.2.0 Linux file-tier defaults - the measured A/B against the 0.1.4x defaults 2026-10-18 16:31:49 +02:00
Niko1221 44fbe69c66 Merge f-149-prefetch (GGUF-only defaults) into f-149-default 2026-10-18 15:58:23 +02:00
Niko1221 c31367979d 149: the Linux file-tier defaults only for the GGUF read in place
An experts.bin pack (the NVFP4 packs) keeps STRATA_COLD_OVERLAP, STRATA_IO_PREFETCH and STRATA_IO_PF_GATE
off unless they are set: the defaults were measured on the GGUF read in place only. The startup line and
docs/DETAILS.md say so.
2026-10-18 15:58:22 +02:00
Niko1221 240304928b Merge f-149-prefetch (82c80ac4d) into f-149-default (from f-split-int fb7c62963) 2026-10-18 15:39:49 +02:00
Niko1221 82c80ac4d7 149: Linux defaults - zero-copy page-cached experts, io prefetch and its decode gate; cgroup RAM check counts the clean page cache
- STRATA_COLD_OVERLAP defaults to 4 on Linux (0 elsewhere), STRATA_IO_PREFETCH to 1 on Linux for the
  file tier (mapped GGUF in place or experts.bin), STRATA_IO_PF_GATE to 1 with io prefetch. Off switches:
  STRATA_COLD_OVERLAP=0, STRATA_IO_PREFETCH=0, STRATA_IO_PF_GATE=0. One startup line names them.
- cgroup_available_bytes: every clean file page of the group (memory.stat file, active + inactive; else
  inactive_file) is reclaimable, less min(half of it, limit / 64), as MemAvailable counts page cache on a
  bare host. A container whose own reads left active cache was judged short of RAM and its file tier
  switched to unbuffered reads (an 18 GiB RAM tier in a 32 GiB group: 1.2 GiB available instead of ~11).
  Unit tests with mock memory.stat / memory.max values.
- The per-request cold-reads line only with STRATA_COLD_OVERLAP or STRATA_IO_STATS set.
- docs/DETAILS.md: the 0.2.0 Linux file-tier defaults.
2026-10-18 15:39:41 +02:00
Niko1221 fb7c62963e Merge f-152-users (2e111557b) into f-split-int: CPU worker counts per physical core with a cap of 32, host socket, quota; warning above the recommendation; setup caps what it writes 2026-10-18 15:38:16 +02:00
Niko1221 8b1139d026 149: STRATA_IO_PF_GATE=1 - the decode's read-ahead only where the page cache has room and it pays (opt-in)
With STRATA_IO_PREFETCH=1, the decode's read-ahead (pread on the I/O threads, the look-ahead two layers
deep) and its pread demand reads run only while the gate is open; the prompt path is never gated.
Checked every 16 decode windows:
- room: (the tightest cgroup v2 limit - the group's non-file memory, else MemAvailable) / the experts'
  bytes; closes below STRATA_IO_PF_GATE_LO 0.55, opens above STRATA_IO_PF_GATE_HI 0.65;
- pays: closes when fewer than 128 MB were read ahead in 16 windows, or under STRATA_IO_PF_GATE_USEFUL
  (0.10) of those bytes were used; then a probe every 256 windows.
Closed, the look-ahead predicts one layer and warms with the madvise hint, as without io prefetch. The
decode thread (and the fetch threads it starts) is marked by FileExpertSource::DecodeScope. The serve
log's file-tier line says the room, the share used, the state and the flips. A request's wait for the
adaptive round in flight at its start is logged when >= 1 ms.

5 pairs (Q2_0 GGUF in place, 12 GB card emulated, cold SSD, cgroup RAM caps), mode 4 + IO_PREFETCH +
gate vs mode 4 alone: 32 GB 1.16 / 1.21 / 1.70 decode, 1.86 / 1.98 prompts; 16 GB (closed) 1.00 / 1.01 /
1.10, 1.54 / 1.78 prompts; an 18 GiB RAM tier 0.91 / 0.84 / 0.86 (its IO_PREFETCH start re-checks the
RAM inside the cgroup and switches the file tier to unbuffered - not the gate's doing).
Identity gates vs b829a3189 (fixed flags, Q2_0 and IQ3_S, default and mode 4 + IO_PREFETCH + gate with
--mmap-experts): 10/10 and exact_pp 4K / 20K IDENTICAL in all four.
2026-10-18 15:19:59 +02:00
Niko1221 7b06ea51bd Merge f-split-fb3 into f-split-int (locale keys re-added after the card-list strings) 2026-10-18 15:02:48 +02:00
Niko1221 2e111557b0 docs: how many CPU threads the engine starts and when a configured count is warned about 2026-10-18 14:59:31 +02:00
Niko1221 c47fbb3f22 Merge f-split-fb5 (cd1b3bd31) into f-split-int: the card list preselects the card a loaded model runs on; sidebar shows version and GPU 2026-10-18 14:57:18 +02:00
Niko1221 cd1b3bd31a Models: the card list preselects the card a loaded model runs on; the sidebar model card says the version and the card (Q2_0 · GPU 2) 2026-10-18 14:46:30 +02:00
Niko1221 0c7872ddc2 Merge f-split-pics3 (ea1e28436) into f-split-int: NVFP4-QAD reads pictures; setup accepts a prepared model without its checkpoint 2026-10-18 14:45:12 +02:00
Niko1221 ea1e28436f TUI test: the model without pictures is a stand-in now that Gyro reads them 2026-10-18 14:43:02 +02:00
Niko1221 261145baaf Merge f-split-fb4 (201d674e9) into f-split-int: cloudflared discovery, settings file, Gyro minimum engine, two models on two GPUs 2026-10-18 14:42:15 +02:00
Niko1221 201d674e94 Models: GPU numbering matches the other screens, friendly model names, fit figures less what the kept models use
Owner-approved for 0.2.0. The card list says (GPU n) with the same numbers as Monitor and the picker; a loaded
model is named from the catalog (Qwen3.8-Flash-Next Q2_0) in the picker, the Loaded list, the sidebar card and the
Monitor, the raw file id only as a fallback; with Run multiple models on, This computer and the card rows show what is
left after the models that stay loaded (a short muted line says so), in 8 languages. The picker rows no longer
print the raw id beside the name.
2026-10-18 14:41:11 +02:00
Niko1221 4eeccd9c00 Task handover: markdown bubble, clean chat title, long step lists capped 2026-10-18 14:39:45 +02:00
Niko1221 2dbd302038 setup: a prepared NVFP4-QAD model needs none of the checkpoint's safetensors; NVFP4-QAD reads pictures
ready_needed() drops the checkpoint's tensors/* once the model is prepared (prepared_done): the engine reads the prepared files only. have_model, ready_fetch, the already-downloaded search in other Strata folders and the disk need now use it, so a prepared model whose 95 GB of safetensors were deleted is ready instead of failing with 'not enough free disk' and a 102 GB download.
NVFP4-QAD (FP4-Local) reads pictures: vision True in setup.py and the MCP catalog; the config's vision section uses prepared/dense.gguf as the text model with the shared mmproj, like the GPTQ pack.
Tests for both; the launch test for a model without pictures now stands one in.
2026-10-18 14:32:19 +02:00
Niko1221 6149cec03e cloudflared: only Strata's own tunnel (same port or same token) blocks a start; other running connectors are listed in plain words with their PID
The coordinator's rule: the owner PC and a tester's system service run connectors for unrelated tunnels, and
quick tunnels must keep working. Running connectors are listed (pid; the command line is only matched,
never shown). A refusal only when one is clearly Strata's own: --url at Strata's port, or the same token.
Terminal (strata launch) follows the same rule. Checked with a real cloudflared on r9700.
2026-10-18 14:24:08 +02:00
Niko1221 5eea92bb09 Helper run no longer ends early when the helper reports while the chat model is still answering (the "interrupted before it finished" error) 2026-10-18 14:17:53 +02:00
Niko1221 6ad1ad69ab Settings file check: the app's own ui_* notes are not unknown keys (found by checking the configs the app wrote on r9700) 2026-10-18 14:14:18 +02:00
Niko1221 c1440388fb Two models on two GPUs in the app: say which card a loaded model uses, prefer a free card for the second, explain a failed start on a shared card, show an unload refusal beside its button, keep a failed instance out of the chat picker
Found running the AMD AppImage on r9700 (Q2_0 on card 1, IQ3_XXS on card 2, Run multiple models):
- the card list ignored loaded models (same text, same pre-selected card); a second model on a used card died
  after two slow engine starts with 'server_exited'. Now the row says 'Already used by <model>', a free card is
  pre-selected while Run multiple models is on, and a failed start there is one plain sentence (no useless retry).
- Unload refused because a chat still used the model (409) showed nothing near the button; it now shows the
  sentence and the next step in the Loaded models card.
- a model that failed to start was listed in the chat picker (disabled); it is hidden.
2026-10-18 14:13:35 +02:00
Niko1221 e2d03ac4fa scripted worker: multi script ends with its own text 2026-10-18 13:41:41 +02:00
Niko1221 b533ae5246 Skills used under an answer; Continue in a new chat on a task (owner-approved mockups), 8 languages, tests 2026-10-18 13:41:17 +02:00
Niko1221 7a70fcccbc Remove the generated Tauri schema files a cargo test run wrote 2026-10-18 13:34:44 +02:00
Niko1221 796937d485 agent README: model choice, reasoning, pictures 2026-10-18 13:34:34 +02:00
Niko1221 e932ad7eb7 Plain next step after a refused picture names the new switch 2026-10-18 13:31:33 +02:00
Niko1221 3084cd93de pics-e2e: on-by-default, OFF, silent server, agent step 2026-10-18 13:29:04 +02:00
Niko1221 14691f6e53 Control launcher: shell built-ins only (no dirname)
Inside the AppImage LD_LIBRARY_PATH holds the bundled libraries, and on Ubuntu 26.04 (uutils coreutils)
dirname fails there: 'libsystemd.so.0: version LIBSYSTEMD_254 not found'. The python/python3 launchers then
computed an empty folder and the app said Control stopped / no engine. Found with the AMD AppImage on r9700.
2026-10-18 13:24:36 +02:00
Niko1221 36e06fcd86 pictures-ui: temp path 2026-10-18 13:23:24 +02:00
Niko1221 65ac8f9106 Pictures: 8-language strings, plain-error rule narrowed, status unit test, real-browser paste/+/drop/off script 2026-10-18 13:22:51 +02:00
Niko1221 138075961f Merge f-148-intel-xmxdef (cfe65f8d1) into f-split-int2 2026-10-18 13:21:18 +02:00
Niko1221 bec4c838ab CPU pool plan: hybrid CPUs with more E-cores than P-cores default to the P-cores but the host's plus half of the E-cores (Core Ultra 7 265: 12-16 workers +12.6% decode over 19; 24 workers -44%) 2026-10-18 13:18:38 +02:00
Niko1221 af26053565 Pictures on by default everywhere: paste, shrink big pictures, one status for composer and Send, plain tool error, per-model off switch wording 2026-10-18 13:12:12 +02:00
Niko1221 cfe65f8d1a docs/sycl: PF_XMX=2 Alchemist default approved by the owner 2026-10-18 (comment and INTEL.md heading only) 2026-10-18 13:11:44 +02:00
Niko1221 ae40bd7516 Merge f-148-intel-xmxdef (12fd8cff6) into f-split-int2 2026-10-18 13:10:38 +02:00
Niko1221 79e94f37f0 OpenRouter reasoning from the model list (supported_parameters), effort sent as reasoning.effort; API pictures on by default 2026-10-18 13:07:03 +02:00
Niko1221 bca49fc5f2 Helper agents on a named model: forgiving model choice (id, provider/id, name, local), did-you-mean, plain list, guidance for every model 2026-10-18 13:05:00 +02:00
Niko1221 d631ed2da7 Gyro: the minimum engine is 0.1.45.1, not 0.1.45; the app says 'needs a newer engine' with the update step
Gyro's expert files are native_experts.txt v5 (Hadamard-folded). The engine reads v5 since the APR port
(f47b0385f); the released 0.1.45 stops at v4 (its binary and source have no v5 reader, every 0.1.45.1+ package
has). GYRO_ENGINE in setup.py and the catalog the app reads (tools/strata_mcp.py: engine in the fallback table,
engine_min on every size) now say 0.1.45.1. The host marks sizes the installed engine cannot read
(needsNewerEngine; an engine built from this checkout or with no readable version is never called too old),
refuses the download and the load with a plain sentence, and Models / first-run setup show the existing
'This model needs a newer engine' notice with Update engine and hold the buttons back.
2026-10-18 13:04:09 +02:00
Niko1221 036c8f6ec3 CPU pool plan: cap 32 (measured on the 32-core EPYC: 24 lost 2-5% decode, 32 and 48 equal, 63 oversubscribed lost 35%) 2026-10-18 13:03:10 +02:00
Niko1221 9319219623 Expert settings: Open settings file (the model's run config in the system text editor), and a plain check of it before every start
The per-model settings already live in one text file (strata-<model>.json: args, env, sampling and the
named keys). Expert settings gets an Open settings file button (Notepad, open -t, or the system's
plain-text program) with the note that changes apply on the next start. The file is checked when the
panel opens, when the window comes back into focus, and before a load: bad JSON, a wrong value, an
unknown key (the nearest real name is offered) are said in plain words with the line, in 8 languages.
A mistake holds the start back instead of an engine that dies on it; an unknown key only warns.
2026-10-18 12:59:05 +02:00
Niko1221 31f225e4fb cloudflared: find a system install (PATH and the usual places), check its version, say which one is used, and never start a second connector next to a running one
The app only looked in Strata's own connector folders, so a cloudflared installed by the system
(/usr/bin, brew, winget, a .deb) was reported as not installed. Now: own folder, component folder,
PATH, then /usr/bin, /usr/local/bin, ~/.local/bin, Homebrew, Program Files. The first copy that runs
--version and is 2022.1.0 or newer is used; the Connector step names it and its place. A cloudflared
already running (system service or started by the person) is said so plainly and not doubled.
Terminal (strata launch) gets the same search and check. Plain messages in 8 languages.
2026-10-18 12:52:12 +02:00
Niko1221 eeb05631fe tests: the Mac catalog test looked for '0 GB' in the whole computer line, which also holds the free-disk figure ('60 GB free'); it now checks the card part only 2026-10-18 12:51:17 +02:00
Niko1221 8a66da38a5 Merge f-split-pics2 (ac658f54a) into f-split-int2 2026-10-18 12:51:01 +02:00
Niko1221 a451dcd913 Merge f-148-intel (fca3c5958) into f-split-int2 2026-10-18 12:38:59 +02:00
Niko1221 ac658f54a6 Test: strata-vision-gguf-test wrapper and Gyro pictures flag 2026-10-18 12:25:37 +02:00
Niko1221 12fd8cff65 sycl: STRATA_PF_XMX=2 is the default on Arc Alchemist (opt-out STRATA_PF_XMX=0); Battlemage unchanged
A750 prompt prefill +30..39% at 4K (mode 2: 79.3 -> 110.4 tok/s, 5 of 5 rounds), +32..35% at 1-2K tokens; decode and cache slots unchanged.
Teacher-forced KL against the default 0.0002 (floor 0.0001), first-token KL 0.0017 (floor 0.0005), tasks inside the default's run-to-run spread.
Docs: INTEL.md has the numbers.
2026-10-18 12:22:23 +02:00
Niko1221 fca3c59586 sycl prefill: DMA an expert straight from the arena only when it is USM host memory (A750 prompt deadlock)
SYCL has no cudaHostRegister, so the expert arena is pageable memory even where the engine calls it pinned. A copy->memcpy of a 2.3 MB blob
from it goes through the Level Zero driver's staging path on the calling thread, which never returned on some prompts (A750, default
flags, a 2,744-token docs excerpt: the thread spun in sched_yield inside libze_intel_gpu with every engine idle, 6 of 6 runs; also seen
in a 5.3K-token needle prompt of the task suite). Only USM host allocations (the --stream-experts pinned mirror) are DMA'd directly now
(answer cached per 2 MiB bucket); every other expert goes through the stager's pinned buffers like an unpinned one.
STRATA_PF_DMA_PAGEABLE=1 restores the old direct copy.
Checks: the repro completes 3 of 3 (0 of 6 before); identical first-token logits and tokens, old vs new: A750 (frozen share, 1.4K, 2.5K and
4K prompts), B70 stock config and B70 without --stream-experts (there the new path is also faster: prefill 680-740 vs 77-456 tok/s); A750 4K
prefill 79.4 vs 79.3 tok/s, B70 stock 1246 vs 1240.
Adds sycl/tools/test_prompt_deadlock.sh with the 2,744-token prompt (passes on the fix, times out on the old build).
2026-10-18 12:19:14 +02:00
Niko1221 35af455aec Gyro catalog test: pictures are on 2026-10-18 12:16:56 +02:00
Niko1221 3fe84ec28d strata-vision reads only the vocabulary and header of the text model; Gyro reads pictures
llama.cpp refuses a GGUF whose tensor directory has a type it does not know, so the picture reader could not open a Gyro model (TQK6 / TQK7, ids 145 and 146) and Gyro was left without pictures. strata-vision now copies the key/value header into a small tensor-less GGUF and loads that vocab-only; the weights are never read or type-checked. Falls back to the file itself when the header holds a value type the copier does not handle.
Gyro-S and Gyro-M read pictures (setup.py, MCP catalog). Adds a C++ test (strata-vision-gguf-test) with a GGUF that has unknown tensor types.
2026-10-18 12:14:19 +02:00
Niko1221 83cde2ca30 Merge f-151-b70 (c765c5f87) into f-split-int2 2026-10-18 11:29:12 +02:00
Niko1221 5ece068f7e CPU expert pool: default worker count from physical cores, host socket, quota and a cap; warning for a configured count above it; setup caps what it writes 2026-10-18 11:20:20 +02:00
Niko1221 196a683d6d docs: the prompt chunk planner (file tier, Linux) - what it does, where it applies, the measured numbers, STRATA_PREFILL_HOST_PLAN=0 2026-10-18 11:14:16 +02:00
Niko1221 fd46e34d5c Merge f-150-planner (8253477bf) into f-split-int2 2026-10-18 11:13:37 +02:00
Niko1221 c765c5f87b sycl: int8 prompt experts default to auto on Battlemage (bmg-g21 / bmg-g31 by architecture id), FP16 elsewhere; STRATA_PREFILL_MMQ=0 off, =1 forced; launch failure or missing kernels fall back to the FP16 chain with one log line; load-time log line; mode-choice unit test; docs 2026-10-18 10:51:21 +02:00
Niko1221 983fa18ad3 docs: B70 follow-up measurements (GDN variant, XMX attention, attention layout, KqSplit, group size); static_assert on the group size 2026-10-18 10:31:41 +02:00
Niko1221 5bcc208a39 sycl: QSA prompt attention: probabilities in [cell][head] order with vector reads in the value phase (default, bit-identical, +0.9% prompt on the B70); STRATA_ATTN_RS=1 reduce-scatter scores measured -1.5% 2026-10-18 09:53:11 +02:00
Niko1221 0e58a652fb tests: iq_multi_parity under STRATA_KQ_SPLIT=1 also without V2K, with the fused gate/up SwiGLU, and with the sub-16 gate/up rows off
The NVIDIA worker's non-bitwise result came from a cherry-pick onto a tree without the dot-count guard in launch_gu (K-quant rows have 160 dots, the sub-16 rows assume 80); the merged tree has it and the Split path is bitwise on every route.
2026-10-18 09:50:49 +02:00
Niko1221 88c1502dc7 pt: s3158 without the word 'demora' (the no-demo catalog test matches 'demo' inside it) 2026-10-18 09:34:41 +02:00
Niko1221 97b680e13e gfx12: the WMMA prompt attention and the Q2_0 fused prompt experts are default-on on Linux only; Windows HIP builds keep them opt-in until tested there
core/gfx12_platform_defaults.hpp holds the per-platform rule (checked in arch_defaults_test); the engine log names it at the first prompt; docs/AMD_HIP.md.
2026-10-18 09:34:37 +02:00
Niko1221 a77d9fab54 Merge f-149-prefetch (2044864db) into f-split-int2 2026-10-18 09:34:20 +02:00
Niko1221 58f81f20e3 Merge f-151-amd (d9070189c) into f-split-int2
Conflicts with 148-amd's gfx12 WMMA default resolved: STRATA_HIP_WMMA unset = gfx12 on every call (148-amd), 0 = off, 1 = forced (also gfx11), auto = gfx12 with n_q >= STRATA_HIP_WMMA_MIN_Q; failed launch falls back to the FP32 kernel; the 96-slot ring follows the same state; the opt-in hint is gone.
2026-10-18 09:34:20 +02:00
Niko1221 8253477bf9 Prefill planner: request-sized chunks. A prompt that fits today's chunk keeps one chunk; a longer one reads in equal chunks of at most the plan's size (no stub tail). Buffers stay sized for today's chunk. 2026-10-18 09:20:24 +02:00
Niko1221 6aa8e25f49 docs: INTEL.md section for the fused int8 expert path (numbers, quality, crossover) 2026-10-18 09:19:38 +02:00
Niko1221 af3770c6b0 Merge f-148-amd (2053388dc, without the IQ2 fused default) into f-split-int2 2026-10-18 09:18:46 +02:00
Niko1221 40a2216b42 Merge f-148-nv into f-split-int2 2026-10-18 09:18:38 +02:00
Niko1221 2e1d62961a Merge f-split-pics into f-split-int2 2026-10-18 09:18:38 +02:00
Niko1221 b6a38b3288 Merge f-split-linuxui (83aa59fdc) into f-split-int2 2026-10-18 09:18:33 +02:00
Niko1221 d9070189c0 docs: --kv-grow on HIP with numbers, STRATA_PREFILL_HELP on two R9700 2026-10-18 08:32:28 +02:00
Niko1221 2725c23f74 NVIDIA sm_86 and sm_60: the measured bitwise decode switches on by default per architecture (nv_defaults; STRATA_SM86_PERF=0 / STRATA_SM60_PERF=0 turn them off) 2026-10-18 08:05:20 +02:00
Niko1221 3980a5ce70 HIP: virtual memory ranges (hipMemAddressReserve/Create/Map/SetAccess, 2 MiB chunks) so --kv-grow can run on ROCm 6+ (STRATA_HIP_VMM=0 turns it off) 2026-10-18 07:58:25 +02:00
Niko1221 2044864dba 149: STRATA_COLD_OVERLAP=4 (zero-copy without the overlap); no residency query for RAM-resident experts
The zero-copy pre-pass ran mincore for every CPU expert of a layer, the RAM copy's too: -2..-9% decode
with an 18 GiB RAM tier. It now skips the RAM-resident and override experts (b32: 0.98/0.97/0.91 ->
1.05/1.10/1.08 vs mode 0, 3 pairs). A file without a residency query (not Linux) never goes zero-copy.
2026-10-18 07:12:43 +02:00
Niko1221 9a01b6e3b8 sycl: int8 expert products: work-group kernels (activation tiles and codebook in local memory), STRATA_PREFILL_MMQ=auto from a measured per-GPU crossover cached per device/driver/format, per-group FP16 fallback, per-device state, clamped down-matrix reads, AOT guard for non-Xe2, docs; 4K +12.5%, 1K +20% with identical tokens 2026-10-18 07:09:40 +02:00
Niko1221 961d37cd55 gfx12: the 96-slot prompt ring follows the matrix-core attention's real state (STRATA_HIP_WMMA=1/auto on a card that has it, not failed) 2026-10-18 06:21:31 +02:00
Niko1221 2053388dc9 docs: AMD_HIP.md - the gfx12 matrix-core prompt attention and the Q2_0 fused prompt experts as defaults, with the 148-amd quality and speed numbers 2026-10-18 06:08:29 +02:00
Niko1221 ebdfaacd1c sycl: int8 DPAS expert products for IQ3_XXS and IQ2_S gate/up and Q2_0 down too (every layer type of the IQ3_S pack but one), thread tile NT=1 with MT 2/4/8, parity test over all six type pairs; STRATA_PREFILL_MMQ=1 prompts +10% (4K), +16% (2K), +20% (1K) with identical tokens 2026-10-18 05:54:34 +02:00
Niko1221 9c4516509f tests: iq_multi_parity runs with STRATA_KQ_SPLIT=1 2026-10-18 05:35:23 +02:00
Niko1221 4908ddaa49 docs: STRATA_HIP_WMMA=auto, the first-prompt warm-up, the gfx1201 hipBLASLt version note; timing: the first chunk's PLE wait on the ple line 2026-10-18 05:35:05 +02:00
Niko1221 d8471050bd Prefill planner: refit the miss share to two rigs (#2824 SATA 2x3060, P2003 NVMe cgroup sweep); read the drive class from the GGUF path 2026-10-18 05:09:14 +02:00
Niko1221 2856a8045e sycl: fused int8 DPAS expert products for the prompt path (ESIMD, Battlemage): IQ3_S gate/up + IQ4_NL down, int8 q8_1 activations, parity test, opt-in STRATA_PREFILL_MMQ=1; first correct version, not yet faster than dequantize + oneMKL 2026-10-18 05:08:50 +02:00
Niko1221 c85cc6dd19 149: io prefetch scheduling, RAM_ADAPT2, cold-read overlap and zero-copy GGUF experts (all opt-in)
- STRATA_IO_PF_SCHED=1 (with STRATA_IO_PREFETCH=1): confidence-scored read-ahead from the router
  lookahead, per-window speculative byte budget, deadlines (late jobs dropped), useful-share feedback;
  new per-request counters (read ahead MB, used before demand, late, not admitted).
- STRATA_RAM_ADAPT2=1 (with STRATA_RAM_ADAPT=n): RAM-tier promotions ranked by expected stall saved per
  byte (drive vs page cache), read on the source's I/O thread, installed at window boundaries; the
  prompt's routing seeds it with a limited weight (Prefill::on_routes).
- STRATA_COLD_OVERLAP: 1 reads a layer's staged blobs in the background while the pool computes the
  ready experts, each landed group computed as it lands; 2 measures R / C_r / C_c per layer; 3 adds
  zero-copy and 4 is zero-copy alone: an expert the page cache holds whole (GGUF in place, Linux mincore)
  is computed from the mapping through a per-job NativeFmt (ExpertJobMulti::fmt) instead of being
  assembled into a stage buffer. Same bytes, same rows, same bits.

Default (all unset) is the old code path: identity gates vs b829a3189 IDENTICAL (Q2_0 and IQ3_S,
fixed flags, 10/10 + exact_pp 4K/20K); mode 3 with --mmap-experts also 10/10 + 4K/20K IDENTICAL.
2026-10-18 05:06:29 +02:00
Niko1221 aef32e224b prefill timing: the hyper-connection write-back on its own line (it was charged to the phase before it, e.g. the wait for the first chunk's PLE rows); hip_prefill_gr_write_bench 2026-10-18 04:57:28 +02:00
Niko1221 83aa59fdc5 setup: say why the page waits while the bundled engine unpacks (8 languages) 2026-10-18 04:42:27 +02:00
Niko1221 f3be974359 prefill: warm the first BF16 and FP16 hipBLAS products on a thread at start (STRATA_PREFILL_WARM=0 skips), STRATA_COLD_TRACE diagnostic 2026-10-18 04:24:19 +02:00
Niko1221 9b75c6a127 desktop: host_command test (loopback bind forced, shutdown secret, plain refusal); Cargo.lock follows the app version 2026-10-18 04:17:21 +02:00
Niko1221 219ae436fa setup_intel: recommend --prefill 8192 for a Battlemage card in the borrowing (over 32K context) config, with the measured numbers and a test 2026-10-18 04:07:31 +02:00
Niko1221 79ac3164a0 gfx12: STRATA_HIP_WMMA=auto (prompt attention on matrix cores chosen per call, FP32 fallback on a failed launch); Split traits for Q4_K/Q5_K/Q8_0 (STRATA_KQ_SPLIT=1, bitwise, opt-in) with iq_multi_parity cases 2026-10-18 03:51:49 +02:00
Niko1221 2c6bfef271 gfx12: the native Q2_0 pack's gate/up take the fused int8 prompt experts by default (STRATA_PF_FUSED=0 keeps MMQ)
R9700 Q2_0: prompt 4K +16.7%, 20K +19.0% (5/5 pairs); teacher-forced KL 0.0004 over 6,022 positions, 0.0016 over 1,760 long-document ones.
2026-10-18 03:47:13 +02:00
Niko1221 99f6d47233 gfx12: the matrix-core prompt attention (STRATA_HIP_WMMA) is the default; STRATA_HIP_WMMA=0 turns it off
Teacher-forced KL against the old kernel on an R9700 (Q2_0, int8 KV, fixed cache so the floor is 0.0000): 0.0003 over 6,024
positions (p99 0.006, top-1 99.8%), 0.0014 over 1,760 positions of 8 long documents (top-1 98.9%); HumanEval, reasoning, agent,
needles, CJK and tool calls equal. gfx11 stays opt-in. The 96-slot prompt ring follows the kernel (qsa_hip_wmma_active()).
The start-up hint about the switch is gone.
2026-10-18 03:47:13 +02:00
Niko1221 577d644480 sycl: STRATA_FOLLOW / STRATA_FOLLOW_LOGPOS test path in the serve decode loop (teacher-forced KL), off unless set; measurement tooling 2026-10-18 03:40:58 +02:00
Niko1221 b1d4686f4a pics-e2e: optional API part, model match and context options 2026-10-18 03:25:01 +02:00
Niko1221 00d2f12a2d desktoplog: drop an unused constructor 2026-10-18 03:24:55 +02:00
Niko1221 e77f1dfc3f Prefill planner: price the host (RAM page cache, displaced experts, drive) when choosing chunk and ring (#2824)
The auto chunk scan only tested the GPU cache.  A lent slot's expert comes back from the RAM copy or the model file for the length of a
prompt, and every chunk streams the whole file set; past the page cache that set is re-read per chunk (28-45 GB per 29K prompt on
2x RTX 3060 / 32 GB / IQ2_XS at 8192-token chunks, ~1 GB at 4096).  prefill_plan.hpp is the pure model (file set, miss share fitted to
#2824, additive stream + compute + drive time); the serve scan and the generate path refine the VRAM pick with it, after the RAM copy
exists.  A set that fits the page cache keeps today's pick.  STRATA_PREFILL_HOST_PLAN=0 turns it off; --prefill N is untouched.
2026-10-18 03:21:24 +02:00
Niko1221 4306182741 tester READMEs: --save-logs, crash log location, the app version from Cargo.toml 2026-10-18 03:21:01 +02:00
Niko1221 1b003bd406 desktop shell: logs, crash log, --save-logs; the window blocks startViewTransition; one app version (Cargo.toml, 0.2.0-beta); banner while the bundled engine unpacks; diagnostics include the desktop logs 2026-10-18 03:19:07 +02:00
Niko1221 e4ed5d1a4e Pictures: untick means does not read, API refusal hint, e2e catalog check skips known false 2026-10-18 03:16:43 +02:00
Niko1221 52f50ba1f1 Pictures: plainer refusal wording, e2e reads answer tokens only 2026-10-18 03:15:43 +02:00
Niko1221 a3cf8a647b Pictures reach models: catalog marks picture-capable models, API models take image_url, import knows picture readers
The catalog no longer forces images:false in the app; a model reads pictures when it is picture-capable and the engine component carries strata-vision.
API models: a model reads pictures when the person ticks 'This model reads pictures' or the server says so (model list modalities, llama-server /props); the chat sends image_url to any OpenAI-style API that does, and says in plain words how to fix every case where it cannot.
Importing a model folder recognises the picture reader next to it and says plainly when a model type is not ours (use a Custom model API).
2026-10-18 03:14:46 +02:00
Niko1221 6c16529a84 ui: Quit dialog can be dismissed after Strata has stopped 2026-10-18 02:21:09 +02:00
Niko1221 01c8381ce6 make-tester-readme --verify: hash only the files the README names; a sentence that merely names a file is not a stale hash 2026-10-18 02:14:23 +02:00
Niko1221 efa888b143 Merge commit 'b1e0fdaa4' into f-split-int 2026-10-18 01:32:22 +02:00
Niko1221 c37a3c415b repack-appimage: write <out>.sha256 from the finished file 2026-10-18 01:30:50 +02:00
Niko1221 dd1a55f99a make-tester-readme: verify by checksum set 2026-10-18 01:30:04 +02:00
Niko1221 51c60366b8 make-tester-readme: plain replacement 2026-10-18 01:29:06 +02:00
Niko1221 1b27554b9c make-tester-readme: templates may have CRLF (archives of a Windows checkout) 2026-10-18 01:28:00 +02:00
Niko1221 c1a5b13fdf sycl: PcieAuto on SYCL, opt-in STRATA_PCIE_FRAC_DEFAULT=auto (single card, native pack): the PCIe share refined from the CPU pool's speed in the first decode windows 2026-10-18 01:27:12 +02:00
Niko1221 9d868f38e8 make-tester-readme: templates may have CRLF (archives of a Windows checkout) 2026-10-18 01:27:11 +02:00
Niko1221 3409d9aaec gfx12: the native Q2_0 pack's gate/up on the fused int8 prompt experts (STRATA_PF_FUSED=1; opt-in until KL-checked)
native_w11_kernel's gate/up path walked 256-value super-blocks only; a Q2_0 row is 64-value blocks, so gate/up of that
format now takes the same one-block-a-stage walk its down path has. R9700 Q2_0: prompt +16.7% (4K) / +19.0% (20K),
5/5 pairs; hip_prefill_fused_iq Q2_0 / Q2_0: fused vs MMQ rel RMS 1.6e-5.
2026-10-18 01:26:43 +02:00
Niko1221 20f92128c5 tester README test as unittest 2026-10-18 01:25:02 +02:00
Niko1221 b1e0fdaa4b sm_120: PF_FUSED default and per-token HC down blocks only on 80+ SM cards
The RTX 5070 check (48 SMs, IQ2_XS pack, experts partly in RAM): the
fused int8 prompt experts were 4.7% slower on 20K prompts (0/6 pairs) with
no decode gain, and one token a block row could not be told apart from
token pairs in the hyper-connection down projection (an earlier run leaned
to pairs).  So on a 12.0 card with fewer than 80 SMs the prompt experts
keep MMQ (STRATA_PF_FUSED=1 still takes the fused kernels) and the down
projection takes token pairs at every even launch (STRATA_GR_DOWN_TG=2);
12.0 cards with 80+ SMs (the RTX PRO 5000 measured) are unchanged.
2026-10-18 01:24:48 +02:00
Niko1221 a0469e9455 tester READMEs: templates with checksums computed from the files, make-tester-readme.sh (+ --verify), test 2026-10-18 01:24:05 +02:00
Niko1221 e1168345f1 desktop: tester README templates (hashes filled at build time), plain_path warning 2026-10-18 01:15:48 +02:00
Niko1221 5c5fdafb94 sycl: STRATA_FOLLOW / STRATA_FOLLOW_LOGPOS test path in the serve decode loop (teacher-forced KL), off unless set; measurement tooling 2026-10-18 01:09:08 +02:00
Niko1221 eaddf15b77 desktop: Mutex import only where used 2026-10-18 01:02:33 +02:00
Niko1221 a8fdceba5b ui: Quit Strata in the menu in browser mode (confirm, stopped screen), 8 languages; host: open the page without changing the mode 2026-10-18 01:01:32 +02:00
Niko1221 8be9e0976f desktop (Linux): the shell owns the host, the window is its own process; a crashed window moves Strata to the browser (--browser, STRATA_UI=browser, --quit, session file) 2026-10-18 00:58:54 +02:00
Niko1221 e01f766118 host: window mode vs browser mode (app-ui.mjs): Quit Strata, the shell's UI route, app updater unavailable in browser mode with a plain message 2026-10-18 00:55:37 +02:00
Niko1221 401169c02f ui: no route cross-fade (View Transitions) on WebKitGTK, which crashes its UI process on startViewTransition (null pointer in AcceleratedBackingStore::update) 2026-10-18 00:47:02 +02:00
Niko1221 b829a3189d docs: the 12.0 defaults with the final A/B (decode +14-16%, prompts +3-10%), PF_FUSED quality, the ple-io rule 2026-10-18 00:27:27 +02:00
Niko1221 b25b309703 setup: --ple-io ram on an SSD too when the RAM can spare the n-gram table (Linux)
A new prompt's first chunk waits for its n-gram rows (65,536 random SSD
reads at 4K tokens) unless the table is in RAM.  RTX PRO 5000 (Linux, NVMe,
251 GB, a fresh engine per prompt, 3 rounds, the same bits): prompt 4K
1298 -> 1059 ms (-18%), 20K 4644 -> 4223 (-9%), 98K 20447 -> 20027 (-2%),
decode +0.8%.  On an R9700 (slow prompt) the read hides behind layer 0 and
nothing changes; under 64-200 GB RAM caps (cgroup, page cache included)
ram mode equalled direct there and cost ~6 s of a cold start.

The rule: not low-RAM mode, Linux (not WSL), and RAM >= the model's own
need + the table + 16 GB (Q2_0: 93 GB, IQ3_S: 107 GB); a smaller PC keeps
the SSD reads as before.  Setup says what it did and how to undo it.
Tests: RotationalDisk gets three SSD cases; the golden configs keep the
Windows behaviour (ple_ram_platform_ok mocked).
2026-10-18 00:06:47 +02:00
Niko1221 959e6c34c4 prefill: fused int8 prompt experts by default on sm_120 (IQ2_S/IQ3_XXS/IQ3_S/IQ4_XS)
Owner decision, the AMD gfx12 rule: on a compute capability 12.0 card the
native packs' prompt experts run on the fused int8 kernels (moe_fused_iq)
for the gate/up formats measured there - the IQ3_S pack's IQ2_S, IQ3_XXS,
IQ3_S and IQ4_XS layers with IQ4_NL / Q2_0 down.  IQ2_XXS / IQ2_XS gate/up
keep MMQ until measured.  This rounds differently from MMQ:

RTX PRO 5000, IQ3_S: prompt 4K +7.0%, 20K +5.2%, 98K +3.4% (3/3 pairs);
teacher-forced KL against MMQ 0.0002 (5998 positions, noise floor 0.0000)
and 0.0010 (1760 positions after ~6K-token documents); first-token KL
against the FP16 prompt path 0.01181 vs MMQ's 0.01236 (48 prompts of
1.5K-24K tokens); HumanEval 156/164, reasoning 26/30, agent 6/6, needles
6/6, CJK 8/8, tool calls 4/5 - the same as MMQ.

STRATA_PF_FUSED=0 keeps MMQ, =1 takes every covered format (as before),
STRATA_SM120_PERF=0 drops the 12.0 default.  Other cards: unchanged.
2026-10-18 00:06:47 +02:00
Niko1221 75afe3f4a0 experts: gate/up, SwiGLU and q8_1 in one launch (bitwise; default on sm_120)
native_gu_swiglu_kernel: a 512-thread block owns 32 hidden columns of a
group; each 16-lane sub-warp computes gate row r and up row r together
(row_dot2_80_sub16: both rows' loads before the applies, each row's sums in
row_dot_80_sub16's order), the block forms h = silu(gate) * up in shared
memory, and warp k writes entry k's q8_1 block with q8_1_store.  The values
are the separate kernels' bit for bit (gate/up are the same floats, the
product is rounded on its own as in swiglu_q8_1_entries_kernel); it saves
the gate/up round trip and one launch per layer.

RTX PRO 5000 (IQ3_S, all experts resident): a layer's gate/up + SwiGLU
35.5 + 2.4 us -> 36.3 us; fixed 4-token rounds 11.92 -> 11.80 ms; default
flags, 6 interleaved pairs: decode +0.6% story, +0.9% code, +0.5% 8K
document, 6/6 pairs each, the same tokens in every case.

On by default for compute capability 12.0 (sm120_series.hpp); elsewhere
opt-in with STRATA_GU_SWIGLU=1; STRATA_GU_SWIGLU=0 turns it off.
2026-10-18 00:06:47 +02:00
Niko1221 fa726b4c9c launch: shorter model-list wording (too big, SSD N GB) so it fits 110 columns; docs and previews for the classic-console look and several cards 2026-10-17 23:53:48 +02:00
Niko1221 1d03976529 monitor: the classic Windows console switches to Cascadia Mono when installed, else a solid-block look with palette colours; several cards and CPUs in the Overview (totals, a Cards strip, sockets); whole card names in view 5 2026-10-17 23:39:05 +02:00
Niko1221 941041768b sm_120: the hyper-connection down projection with one token a block row (bitwise)
Measured on the RTX PRO 5000 against the token pairs of the previous commit,
fixed verify rounds, same text: 2 tokens 9.44 -> 9.24 ms, 3 tokens 10.81 ->
10.56, 4 tokens 11.94 -> 11.90, 5 tokens 14.04 -> 13.70.  Each (row, token)
is still summed by one warp in the same order.  STRATA_GR_DOWN_TG=2 / 0 picks
pairs / no split as before.
2026-10-17 23:05:40 +02:00
Niko1221 44997124ff mmvq_il: the sm_120 2-4 column table only on 80+ SM cards; docs
The shared table was measured on an RTX 5070 (48 SMs) together with an RTX
3060, so a small 12.0 card keeps it; the RTX PRO 5000 table (110 SMs) now
applies to 12.0 cards with 80 or more SMs (il_cc answers 1201 for them).
The 5-8 column table stays for every 12.0 card.  docs/DETAILS.md: the
measured decode gain of the sm_120 defaults, docs/MMVQ_IL_TABLE.md: the
new tables and switches.
2026-10-17 22:49:13 +02:00
Niko1221 3b4c379b68 Merge commit '7bdc1ffb7' into f-split-int 2026-10-17 22:48:56 +02:00
Niko1221 2aeb246cfc Merge commit 'ae31e3cd3' into f-split-int 2026-10-17 22:41:42 +02:00
Niko1221 ae31e3cd36 Docker: smoke-test.sh (the checks run on real cards), executable bits 2026-10-17 22:41:09 +02:00
Niko1221 8d838f770f DOCKER.md: sizes and what was tested 2026-10-17 22:38:47 +02:00
Niko1221 7bdc1ffb72 launch: the card or cards chosen this time go into every launch's own config (--gpu, the Cards step, or the fit rule), so an installed model never keeps its old card and nothing is reinstalled 2026-10-17 22:34:19 +02:00
Niko1221 a00ed935f1 Docker: GPU/GPUS take effect on a volume that already has the model, tuning wording, build id of the source build 2026-10-17 22:15:53 +02:00
Niko1221 404713f234 Dockerfile (source): no NCCL in the picture reader, so the package check passes 2026-10-17 21:59:11 +02:00
Niko1221 4895b1cfa0 Docker: keep only gguf-py and the ggml marker of the baked llama.cpp; root Dockerfile PATH fix 2026-10-17 21:55:59 +02:00
Niko1221 039d2d4609 desktop: cards-e2e.mjs, the Graphics cards choice end to end through the host API 2026-10-17 21:52:31 +02:00
Niko1221 fa6367a98f Merge branch 'f-split-int' into f-split-fit 2026-10-17 21:51:30 +02:00
Niko1221 33d37ddff6 app: the Graphics cards choice on the model page (the recommended option pre-selected, neutral style), passed on install and load 2026-10-17 21:51:30 +02:00
Niko1221 80e6d91a63 Merge commit 'a6759787c' into f-split-int 2026-10-17 21:49:11 +02:00
Niko1221 ba2c2d5fb7 sm_120: four bitwise decode switches on by default on a 12.0 card
Measured on an RTX PRO 5000 Blackwell 72 GB (IQ3_S, every expert in VRAM),
fixed 4-token verify rounds, story / code, the same text as before:
  STRATA_SIDE_MIXER=2   13.67 -> 13.15 ms a round
  STRATA_ATTN_DAILY=1   -> 13.31
  STRATA_GR_DOWN_TG=2   -> 13.34
  STRATA_GR_UP75=1      -> 13.44
  all four              13.70 -> 12.23 ms (2-token rounds 10.37 -> 9.59)
Each is bitwise equal to the kernels it replaces (their parity tests; the
identity gate).  include/strata/core/sm120_series.hpp holds the rule: on for
compute capability 12.0 unless STRATA_SM120_PERF=0; each switch's own
variable still wins in both directions on any card.  The down projection's
token pairs run at every even launch there (Turing keeps 4 and 6 only).
STRATA_ATTN_DAILY / STRATA_ATTN_MERGE_V2 and STRATA_SIDE_MIXER are now
decided per device (a layer split can mix cards).  Other cards: unchanged.
2026-10-17 21:45:38 +02:00
Niko1221 75fabbd941 mmvq_il: 5-8 column interleaved kernels and the sm_120 tables (bitwise)
A verify window of 5+ rows fell from the interleaved small-batch kernels (1-4
columns) back to native_mmvq_multi: on an RTX PRO 5000 Blackwell the dense
projections of a round jumped +2.0 ms from T=4 to T=5 (the main Q5_K head
521 -> 1011 us).  native_mmvq_il_kernel now also runs 5-8 columns from the
cp = 8 interleaved copy (1-2 rows a warp), every value bitwise native_mmvq's
(mmvq_il_parity, T 1-8, every rows choice).

Tables for compute capability 12.0, read off mmvq_il_parity --bench
--emit-table on the RTX PRO 5000 (3% rule): kIlRows_120 for 2-4 columns (it
takes the interleaved kernels in more cells than the shared table, e.g. the
head at 2 columns 507 -> 367 us) and kIlRows58_120 for 5-8 columns (the head
at 5 / 8 columns 899 -> 491 / 1178 -> 667 us).  Other cards keep their
tables and native_mmvq at 5+ columns.

STRATA_MMVQ_IL_WIDE=0 turns the 5-8 column path off; STRATA_MMVQ_IL_120=0
gives a 12.0 card the shared 2-4 column table again.
2026-10-17 21:45:28 +02:00
Niko1221 390336f68d Docker: pinned llama.cpp baked into the image (no GitHub download on first start), file consistency tests 2026-10-17 21:43:37 +02:00
Niko1221 7e490ef63c Docker: launch gets SIGINT back so docker stop ends a download in 1 s; INSTALL.md points to DOCKER.md 2026-10-17 21:26:58 +02:00
Niko1221 da34c60ba7 Merge commit '91cccc1ae' into f-split-int 2026-10-17 21:21:13 +02:00
Niko1221 3641dafe98 Docker: DOCKER.md, MODEL_PATH in compose, device check for AMD and Intel 2026-10-17 21:15:13 +02:00
Niko1221 c7f803d0db Merge commit 'e77e5550d' into f-split-int 2026-10-17 21:06:44 +02:00
Niko1221 5f1bd58e30 Docker: images built from the engine component packages (cuda13, cuda12, HIP gfx120X/gfx110X, SYCL), launch-based entrypoint, compose examples, from-source Dockerfile 2026-10-17 21:00:56 +02:00
Niko1221 f533c7faa6 docs: which cards a model uses; Cards step column widths and previews 2026-10-17 20:35:07 +02:00
Niko1221 31abd50a60 launch: the installed config must carry the chosen cards (a forced split re-runs setup), unified-memory chips count when they are the only device; setup tests follow the fit rule 2026-10-17 20:28:16 +02:00
Niko1221 91cccc1ae1 Merge branch 'f-split-testfix' into f-split-amdtester
# Conflicts:
#	apps/desktop/scripts/stage-harness.mjs
2026-10-17 20:27:24 +02:00
Niko1221 e77e5550df stage-harness: fix a broken string literal in the import check; node --check / bash -n / py compile of every build script in the host tests 2026-10-17 20:26:24 +02:00
Niko1221 15339326e9 model list: gpu_options per model from the shared fit rule (the app reads the choice from here) 2026-10-17 20:16:48 +02:00
Niko1221 1d8d830afa setup: the multi-GPU default follows the fit rule (largest card + RAM first, the split recommended only when the model does not fit one card), goldens re-recorded, setup tests 2026-10-17 20:16:02 +02:00
Niko1221 c198c25e57 launch: the Cards step uses the shared gpu_fit rule (largest card + RAM recommended, the layer split offered with plain numbers, an integrated chip ignored) 2026-10-17 20:13:33 +02:00
Niko1221 ac38793caa gpu_fit: one rule for which graphics cards a model uses (largest card + RAM first, the split offered when it does not fit, an iGPU ignored), with tests 2026-10-17 20:10:22 +02:00
Niko1221 a6759787c3 Merge f-146-int (b7505b72a, 0.1.46 engine: Volta perf PRs + QSA_WARP sm<75 fix) into f-split-146
Conflict in src/prefill/prefill.cpp (step 2b): the split rotates mixed into Xr (APR, had_gu)
before 0.1.45's quantize_scatter (NVFP4 activations keep the gather); 0.1.46 adds the pf_wmma
FP16 gather branch. Both kept: wmma_l gathers FP16 rows (and rotates Xs / the swiglu output Hh
in fp16 when the layer is Hadamard-folded), otherwise the split's rotate then quantize_scatter
or the NVFP4 gather.
2026-10-17 20:09:14 +02:00
Niko1221 b2761df626 stage-harness.mjs: the missing-imports message had raw line breaks inside a string (SyntaxError, no Harness could be staged on any OS) 2026-10-17 19:53:48 +02:00
Niko1221 bfed34873a Merge branch 'f-split-testfix' into f-split-amdtester 2026-10-17 19:53:36 +02:00
Niko1221 b7505b72a4 Merge branch 'f-146-qsawarp' into f-146-int 2026-10-17 19:23:39 +02:00
Niko1221 1b856af242 Installer: Uninstall only uninstalls; the uninstaller offers Also remove my Strata data (off by default) and lists exactly what it deletes 2026-10-17 19:17:44 +02:00
Niko1221 051d0fc9c4 Models tab: Computer use buttons level with the context dropdown, clear of the separator 2026-10-17 19:15:08 +02:00
Niko1221 447b99ba38 Folder chooser fallback in the app: plain message and a field for the path 2026-10-17 19:12:09 +02:00
Niko1221 d85bb21c6b Linux: programs that are not Strata's own get the AppImage's original environment; folder chooser tries zenity, kdialog, portal and never fails silently 2026-10-17 19:11:09 +02:00
Niko1221 d7347cdde6 Start failures: plain, correct messages from the engine's exit code (missing DLL, bad DLL, crash, blocked, file missing); low memory only with evidence; no memory retry for non-memory failures 2026-10-17 19:07:42 +02:00
Niko1221 ad3b340d30 App never adopts an older CLI install: own settings file for Control and the installer, explicit install folder, tests 2026-10-17 19:02:17 +02:00
Niko1221 37f7f03cd2 Engine packages: bundle every imported non-system library, refuse a package a clean PC could not start, PE/ELF/Mach-O import scanner and tests 2026-10-17 18:59:45 +02:00
Niko1221 cbb249d3dd Windows HIP package: ship msvcp140_atomic_wait.dll (strata.exe imports it) and the other VC++ runtime DLLs next to the exes as well as in rocm/bin 2026-10-17 18:54:32 +02:00
Niko1221 f95fb3e074 Fix agent chat on Linux and Mac: keep the Harness's own dsh-* packages when pruning other-OS binaries, verify imports at staging, plain start-failure message 2026-10-17 18:54:11 +02:00
Niko1221 6aa502d938 strata update: a feed whose only HIP engine is for another AMD card family says so in one plain line (This build of Strata is for AMD RX 7000 cards ... Ask for the RX 9000 build) 2026-10-17 18:25:27 +02:00
Niko1221 67bd2a1609 hip_archs_present: an explicit kfd folder is always read (the Windows adapter lookup only replaces the default Linux one) 2026-10-17 18:21:43 +02:00
Niko1221 fe6fa9a274 Merge commit 'db5be2aa554cb248beb70c9595b9719254373772' into f-split-int 2026-10-17 18:01:54 +02:00
Niko1221 db5be2aa55 vision-e2e: load by the installed model name 2026-10-17 14:50:28 +02:00
Niko1221 50768496f5 vision-e2e: find the installed model by id 2026-10-17 14:48:34 +02:00
Niko1221 ab5adeb79f docs: pictures in strata launch 2026-10-17 14:41:45 +02:00
Niko1221 ac5f0d19a3 desktop: vision-e2e.mjs, the picture-reading end to end test through the installed app layout 2026-10-17 14:41:19 +02:00
Niko1221 9a22b78887 setup: the Mac control path says plainly that the Mac engine has no picture reader yet 2026-10-17 14:40:09 +02:00
Niko1221 7c5e846121 Merge branch 'f-split-int' into f-split-vision 2026-10-17 14:40:04 +02:00
Niko1221 417cbab4ff packaging: the engine package carries strata-vision when the build has it 2026-10-17 14:40:04 +02:00
Niko1221 788e0b731b Merge branch 'f-split-agent8' into f-split-vision 2026-10-17 14:38:52 +02:00
Niko1221 debe8ddabb launch: a missing picture reader file is fetched again instead of starting an engine that cannot read pictures 2026-10-17 14:31:03 +02:00
Niko1221 7ecfe4a6a6 A tester build for another AMD card family: one plain line (This build is for AMD RX 9000 cards ... Ask for the RX 7000 build) in the engine status and the start error 2026-10-17 14:16:44 +02:00
Niko1221 4db866ae00 Engine packages: the folders named in BUILD.json lib_dirs go on the library path with the package folder (the Windows HIP package keeps the ROCm DLLs in rocm/bin), for serve, the binary smoke test and tuning 2026-10-17 14:16:02 +02:00
Niko1221 8c676a7ccd Windows AMD: the engine is picked from the display adapters when no GPU tool answers (an AMD card, not an integrated Radeon), the app lists AMD cards on Windows and no longer refuses the hip backend there 2026-10-17 14:10:08 +02:00
Niko1221 8b3a96779e Merge f-145-pp-volta into f-146-volta (0.1.46): the community Volta/Pascal perf-PR group (35 commits)
Merge resolution as in f-145-perfprs-withvolta: iq_kernels.cu (IQ3_XXS / IQ3_S staged sign-mask tables over the EXPDEC grid staging), prefill/kernels.cu and prefill/prefill.cpp keep both sides; CMakeLists keeps both test blocks.
2026-10-17 14:00:21 +02:00
Niko1221 c61364fc89 qsa prompt attention: STRATA_QSA_WARP=1 only forces the Turing kernels on cards that have the Turing MMA (sm_75+)
The switch made every card answer as sm_75, so a Tesla P100 (sm_60) launched the m16n8k8 prompt-attention kernels, whose sm_60 code is a __trap(): the engine died on the first long prompt with "prefill copy_i32: unspecified launch failure". A card below sm_75 now ignores the switch and runs the kernel it runs without it (the same tokens).
2026-10-17 13:49:20 +02:00
Niko1221 e29112390f launch: turning pictures on for a model that is already installed says it downloads only the picture reader, not the whole model 2026-10-17 13:48:17 +02:00
Niko1221 8d65058c9f launch: Pictures on goes through setup's Control mode (setup keeps the choice when the engine component carries strata-vision); the screen warns only when the installed engine lacks the reader 2026-10-17 13:46:42 +02:00
Niko1221 e819ba0bd0 launch: the Folder screen says that model files already stored are moved into a new folder 2026-10-17 13:10:14 +02:00
Niko1221 0b055d8316 Merge commit 'ee0f32805' into f-split-int 2026-10-17 12:04:03 +02:00
Niko1221 ee0f328053 test: the bundle keeps one real python and launchers for the other names 2026-10-17 11:58:14 +02:00
Niko1221 d4abe4b745 strata launch on a Mac: a Terminal that carries strata_control/mac_fit.py installs the model through setup.py's Control mode instead of adopting prepared files 2026-10-17 11:55:03 +02:00
Niko1221 78fc321cd3 launch: the Pictures screen says plainly that this Terminal cannot set up the picture reader yet 2026-10-17 11:52:23 +02:00
Niko1221 254ee2b9cd Control: the NVIDIA driver version also from /proc/driver/nvidia/version (Fedora has no nvidia-smi without its cuda package); an integrated AMD GPU beside a usable NVIDIA card is not offered 2026-10-17 11:27:51 +02:00
Niko1221 a93264877c metal: the opt-in HIP/Turing entry points (fused_gr_persist_init, fused_gr_read_persist, f16_range_read, gemm_f16_range_read) answer 'nothing to do' instead of stopping the engine; e2e script waits for the bundled engine 2026-10-17 11:26:20 +02:00
Niko1221 e4b885e9fa repack-appimage.sh is executable 2026-10-17 11:19:53 +02:00
Niko1221 f2fe9a9097 metal: stubs for the 0.1.45 entry points Metal does not implement yet (stubs_0145.cpp) and the Gemm::bf16 / bf16_converts_x signatures, so the engine links from the 0.1.45 tree 2026-10-17 11:18:31 +02:00
Niko1221 d2a38c9e5c Linux tester AppImage: repack-appimage.sh puts the staged engine/Node/Python/Harness back byte for byte (linuxdeploy rewrites their RPATH, which broke the bundled engine's manifest check); control status says the bundled engine is being set up; WebKit DMA-BUF renderer off on Linux; e2e script takes --backend and finds the engine by its components path 2026-10-17 11:13:50 +02:00
Niko1221 47e2343b1b launch: Pictures on means the CPU picture reader on an AMD card, and a plain warning when pictures stay off 2026-10-17 11:10:47 +02:00
Niko1221 eeeeddd3f6 metal: the CUDA-name compat layer declares the seven calls the 0.1.45 sources added (compute capability, stream priority, cudaMemcpyPeer, DisablePeerAccess), so the Metal engine builds from the 0.1.45 tree again 2026-10-17 10:54:20 +02:00
Niko1221 63ac0838bc docs/MAC.md: the app and strata launch on a Mac 2026-10-17 10:47:04 +02:00
Niko1221 ccb33b7c7c Mac install plan: no card, KV or RAM-mode options; the context follows unified memory; a Mac too small for the model is refused in plain words 2026-10-17 10:45:17 +02:00
Niko1221 78914f55be Mac app: the model list, fit words and recommendation judge a Mac by unified memory (strata_control/mac_fit.py, one definition shared with metal/setup_mac.py); setup.py in Control mode installs a Mac model with no source build and no setup_mac.py and writes component:strata-engine with the Metal flags 2026-10-17 10:42:27 +02:00
Niko1221 6c4dbe3362 launch: Pascal/Volta cards are admitted when they are the only NVIDIA cards, as setup does (V100 showed 'no usable graphics card' and no Cards step) 2026-10-17 10:41:48 +02:00
Niko1221 8df9e0afd0 Linux app: one real python in the bundle (the other names become launchers), GStreamer bundled in the AppImage, a driver too old for every engine gets one plain line naming the version to install 2026-10-17 10:33:22 +02:00
Niko1221 395ed1ac54 bundled engine: CUDA 13 and CUDA 12 builds of one version install side by side, only the ones the NVIDIA driver can run; fix the duplicate 'ready' parameter that made native-control.mjs a syntax error 2026-10-17 10:30:37 +02:00
Niko1221 13ded2d662 Merge branch 'f-split-tester' into f-split-macapp 2026-10-17 10:22:38 +02:00
Niko1221 60c4d8de1e Merge branch 'f-split-apps' into f-split-macapp 2026-10-17 10:20:35 +02:00
Niko1221 e288308b46 Installer can carry the engine: the host installs a bundled engine package (manifest hashes checked, never over a newer one, no network) before the first model list 2026-10-17 10:14:19 +02:00
Niko1221 3618b48030 stage-harness: drop musl and other-CPU prebuilds on Linux (linuxdeploy stops on koffi's musl build) 2026-10-17 09:47:07 +02:00
Niko1221 59cee65da3 serve tests: the refused-prompt test runs in strict mode (fit_max_tokens off) so a huge max_tokens is refused, not shortened 2026-10-17 07:00:49 +02:00
Niko1221 9a58a4b73a Merge f-145-int (477ac13cb, the 0.1.45 release tree) into f-split-145 2026-10-17 06:33:36 +02:00
Niko1221 f202d08c5f Merge commit '872bcd197' into f-split-int 2026-10-17 06:16:06 +02:00
Niko1221 872bcd1974 launch: see an Intel Arc card (from the kernel's device list) and recommend a context that fits its memory 2026-10-17 05:52:15 +02:00
Niko1221 6ad5596b9d tests: the Intel Arc flags follow the kernel driver in the launch settings 2026-10-17 05:12:09 +02:00
Niko1221 982f43dee5 Regenerate launch previews after the model-list change 2026-10-17 05:11:26 +02:00
Niko1221 87fbbdaaa9 Merge branch 'f-split-agent8' into f-split-int 2026-10-17 05:06:46 +02:00
Niko1221 e3bf0ad04d Time-skip: final 'Both versions are ready' line, no stale warning after B finished 2026-10-17 05:02:30 +02:00
Niko1221 b8c1602715 launch: Intel Arc recipe for prepared files (i915: --ple-io ram; xe: --stream-experts; VRAM reserve) 2026-10-17 04:35:15 +02:00
Niko1221 08297223e4 launch: on a Mac, use prepared model files with the Metal defaults (the Terminal has no Mac model installer) 2026-10-17 04:34:33 +02:00
Niko1221 64fff9e149 launch: model list shows the app's RAM+VRAM figure and the SSD part; strata monitor gets the Connect view (7); Mac env names 2026-10-17 04:31:34 +02:00
Niko1221 b799ef5f6d launch tests: top-folder install case 2026-10-17 04:06:23 +02:00
Niko1221 8cc02778e4 launch: firewall line, overview GPU name and legacy width cap, install.sh accepts an archive with a top folder, tool-call check budget, plain wording for adopted files 2026-10-17 04:05:47 +02:00
Niko1221 87efc9ace5 launch tests: macOS fixes (platform names, python 3.9 tar, no setsid/timeout/cp -l), feed platform macos-arm64, p2006 engine path 2026-10-17 04:01:23 +02:00
Niko1221 c1533d6d5f update: an engine without --version that lists its options passes the test start (with a plain note) 2026-10-17 04:00:17 +02:00
Niko1221 a693660a15 launch: fix the download question from the start screen 2026-10-17 03:41:25 +02:00
Niko1221 436fabc466 launch: ask before downloading a model that is not here; tuning table labels 2026-10-17 03:39:30 +02:00
Niko1221 7c2f878051 launch: a Mac's setup gets only the options it knows 2026-10-17 03:36:59 +02:00
Niko1221 a2bab5c8c2 launch: no choices and no terminal starts nothing 2026-10-17 03:36:37 +02:00
Niko1221 dc1aa5030a launch: docs (TERMINAL_LAUNCH.md), installer tests, plain GGUF issue words 2026-10-17 03:36:03 +02:00
Niko1221 463d6780a9 launch: write the run settings from prepared files when the installer cannot set up the card (Intel Arc) 2026-10-17 03:21:39 +02:00
Niko1221 9330989efa launch: say why the engine could not be installed 2026-10-17 03:16:46 +02:00
Niko1221 69f9cc7b9e launch: environment settings also apply to the engine's test start 2026-10-17 03:12:10 +02:00
Niko1221 b3ff0c2ebf launch: environment names keep their case in --set (IGC_..., ZES_...) 2026-10-17 03:07:25 +02:00
Niko1221 4fbd0b3fcb connect check: responses answer text excludes reasoning items 2026-10-17 03:02:20 +02:00
Niko1221 d247802094 launch: install.ps1 child scope and stderr note, tuning result line, Windows watch hint 2026-10-17 02:57:51 +02:00
Niko1221 1fe683e951 Picture input for a server on this network only when its own model list says it reads pictures (a server without a vision encoder refused whole requests) 2026-10-17 02:53:59 +02:00
Niko1221 eaa3a58172 launch: monitor tests, quit hint, check result note 2026-10-17 02:50:03 +02:00
Niko1221 7ce4b6df37 Time-skip step limit 15 minutes 2026-10-17 02:41:28 +02:00
Niko1221 f548cfd843 Time-skip: a step that runs longer than 20 minutes is stopped and left as it is with a plain note; one shared step runner for A and B 2026-10-17 02:41:20 +02:00
Niko1221 6a29939806 launch: connect --check, tester stage scripts (phaseA-tests/split/launch-tests are outside the repo) 2026-10-17 02:33:31 +02:00
Niko1221 ec06637d48 launch: installer placeholder fix, quieter plain run, flags next to --preset do not rewrite it 2026-10-17 02:26:28 +02:00
Niko1221 4e6a1b46be launch: text previews of every screen, advanced table widths 2026-10-17 02:15:12 +02:00
Niko1221 efa38a99ca monitor: the approved Overview as view 1 (state bar, 8 tiles with sparklines, bars, recent requests) from /metrics; b opens the browser 2026-10-17 02:14:32 +02:00
Niko1221 4e788fbf3e connect: the 8 tested harness configs (limits in plain words, context-aware), Connect view shows them 2026-10-17 02:11:28 +02:00
Niko1221 da4a3a4765 Merge commit 'e0a932a32' into f-split-launch 2026-10-17 02:07:56 +02:00
Niko1221 4c9423e194 launch: one-line installers (install.sh, install.ps1), local test feed builder, package_terminal installer step 2026-10-17 02:07:47 +02:00
Niko1221 4fac421d14 Time-skip sample copy only when the host cannot be reached at all; packaged test follows the Read pictures install 2026-10-17 01:55:37 +02:00
Niko1221 7b1405e104 Time-skip: view-in-browser steps never hang or fail a version (no-window rule, skip with a plain note); steps carry the unattended note 2026-10-17 01:48:50 +02:00
Niko1221 97e4dcaa1f Merge branch 'f-split-ui6' into f-split-int 2026-10-17 01:47:30 +02:00
Niko1221 c473d97109 Merge branch 'f-split-agent8' into f-split-int 2026-10-17 01:47:28 +02:00
Niko1221 4a786e03d6 Time-skip: a step that does not finish is tried once more in a fresh session; B says why it stopped 2026-10-17 01:42:17 +02:00
Niko1221 e0a932a32d Merge branch 'f-split-harness' into f-split-int 2026-10-17 01:06:34 +02:00
Niko1221 f31b6efd68 serve: shorten max_tokens to the room left instead of answering 400 (all three dialects)
Claude Code asks for max_tokens 32000 on every request and the Harness for 16384; on a 32K context the
prompt fits but prompt + max_tokens does not, and the server answered 400 "request exceeds the context
window ... requests are never truncated" before the first token.

Service.prepare() (the one place chat/completions, messages and responses all pass through) now clamps
max_tokens to ctx - CTX_SLACK - prompt when the prompt fits, logs one line, and the answer_room thinking
budget (3/4 of max_tokens) is taken from the clamped value. A prompt that alone does not fit is still the
plain 400 ("no room to answer"). --no-fit-max-tokens / "fit_max_tokens": false brings back the strict
llama.cpp 400; --fit-max-tokens stays accepted.

Tests: serve/test_server.py MaxTokens (shortened by default in the three dialects, exact fit and one over,
prompt-too-long 400, thinking budget follows, strict mode), serve/test_context_overflow.py (default
service), serve/test_monitor.py (error record uses the strict mode).
2026-10-17 00:55:18 +02:00
Niko1221 f11768e873 Time-skip: each step runs in its own agent session (the folder carries the work), so a small window never fills up across a long plan 2026-10-17 00:41:02 +02:00
Niko1221 af34f0a4b7 Reply room 30 percent of the window and an eighth headroom: a 32K window no longer compacts every few steps 2026-10-17 00:04:36 +02:00
Niko1221 64034f1674 launch: terminal archive carries the installer files (shared with the app's Control staging), legacy-console glyph and box fallback, install from a bundle runs in control mode 2026-10-16 23:43:01 +02:00
Niko1221 8251089c1f launch: pilot tests for every screen, run screen state fix, no auto focus 2026-10-16 23:40:55 +02:00
Niko1221 af4a7e5638 Merge branch 'f-split-fp4' into f-split-int 2026-10-16 23:38:52 +02:00
Niko1221 5fbd539628 launch: TUI screens, runner events, monitor Connect view, harness check, tests for flags and rules 2026-10-16 23:32:37 +02:00
Niko1221 e61e90e555 launch: settings, catalog (setup.py tables via strata_mcp, fitOn port), presets store, access, runner, flags, connect, harness drafts 2026-10-16 23:25:34 +02:00
Niko1221 17ba2d71ad A model server on this network keeps half its window for the reply (long Time-skip steps overflowed the window); Time-skip says why A failed 2026-10-16 23:18:34 +02:00
Niko1221 76707b230b Round 8: neutral Open result button, pictures uploaded to /api/chat/upload and sent as imageIds, picture-reading files note on the Models tab 2026-10-16 23:00:11 +02:00
Niko1221 0a6eced5cc Merge branch 'f-split-agent8' into f-split-ui6 2026-10-16 22:50:27 +02:00
Niko1221 490fda2657 Round 8: project folder chip above the message box, Time-skip as two agents (plan, lever with its why, A and B progress, Keep A / Keep B), working view reachable while a run has no files yet 2026-10-16 22:50:05 +02:00
Niko1221 30a367e5f4 Image limits in the validation test follow the new 4 pictures / 6 MB 2026-10-16 22:49:22 +02:00
Niko1221 c3e9480592 catalog: one plain line each for NVFP4 (best quality of the NVFP4 versions) and NVFP4-QAD (NVIDIA's FP4, trained for FP4, slightly faster on short prompts) 2026-10-16 22:44:58 +02:00
Niko1221 477ac13cbe verify: the side-projection q8_1 interleave copies need two or more columns (STRATA_SIDE_MIXER=2 on a 1-column window threw on Q2_0 packs: the queries' branch and the GDN gate branch) 2026-10-16 22:39:03 +02:00
Niko1221 c8265fce0e tests: setup's gfx1103 / gfx1150 switch sets do not copy STRATA_TOPK_REG_MIN_BLOCKS (the gfx1151 line from #2016, measured on gfx1151 only) 2026-10-16 22:39:03 +02:00
Niko1221 e6ad6c81d7 fix(HIP compat): cudaMemcpyPeer and cudaDeviceDisablePeerAccess for the #2666 peer-copy probe (peer_experts.cpp did not compile on HIP in f-145-int) 2026-10-16 22:39:02 +02:00
Niko1221 f260925adc fix(sm75 merge): stray brace after hf() in prefill/kernels.cu (#2093 #1821 resolution) 2026-10-16 22:39:02 +02:00
Niko1221 d615708a7b Time-skip v2: plan, lever, agent A to the end, agent B forked at the lever with an improvement plan, A vs B, keep one 2026-10-16 22:37:12 +02:00
Niko1221 4d7fede4ed Merge branch 'f-split-agent8' into f-split-ui6 2026-10-16 22:36:21 +02:00
Niko1221 9e2d698c9e Round 8: Models navbar family dropdown, deliverable cards with Open menu, Edited summary, grouped activity lines, text in order, drop zone, image generation hidden 2026-10-16 22:35:54 +02:00
Niko1221 7af6b6c152 Vision end to end: vision files are downloaded when asked, pictures through the agent and the plain path, uploads by multipart/data address/raw bytes, dropped files named in the next message 2026-10-16 22:32:11 +02:00
Niko1221 917d2fc345 Deleting a model: a file that is already gone counts as removed; the host verifies the files are really gone 2026-10-16 22:26:44 +02:00
Niko1221 0ac98f80ee Merge branch 'f-split-agent8' into f-split-ui6 2026-10-16 22:26:01 +02:00
Niko1221 8509870849 Round 8: compact segmented Computer use, version chevron and panel gap, download details only when useful, immediate download progress, Models screens no longer follow every monitor tick 2026-10-16 22:26:01 +02:00
Niko1221 aa936f59f0 Deliverable events, grouped activity lines, file preview/raw/attach routes 2026-10-16 22:25:51 +02:00
Niko1221 ef3902477e Per-chat project folder (agent cwd and sandbox), opening files by path in the browser, app or folder 2026-10-16 22:23:38 +02:00
Niko1221 ddfed500a9 A turn always ends: watchdog for a silent model or a tool that never returns; launches return at once 2026-10-16 22:21:57 +02:00
Niko1221 ade3e6adb5 nvfp4_local_pack ple: accept a folder with only the shards that hold the table 2026-10-16 22:02:03 +02:00
Niko1221 4736639a7e merge 5e7a965ac into f-split-int 2026-10-16 21:57:35 +02:00
Niko1221 dbbe79aae8 merge bdb04dd8d into f-split-int 2026-10-16 21:57:34 +02:00
Niko1221 bdb04dd8d6 batch-mtp on automatically for one big card (24 GB+, several slots, MTP on); off on small cards and layer splits; Expert mode MTP per request overrides 2026-10-16 21:57:18 +02:00
Niko1221 48b87d42e1 docs: NVIDIA FP4-Local in NVFP4.md 2026-10-16 21:10:27 +02:00
Niko1221 e7f7dad3e0 FP4-Local engine floor 0.1.45 (the split release's engine has the NVFP4 n-gram table) 2026-10-16 21:04:27 +02:00
Niko1221 0da73d9159 FP4-Local prepare: separate step marks (the pack's own files were overwritten), scale tensors in the checkpoint-direct pack; docs section 2026-10-16 20:06:19 +02:00
Niko1221 5e7a965ac1 Queued-message test: ask Stop again when it answers still stopping 2026-10-16 20:06:04 +02:00
Niko1221 19d16c7dbe ui-flows: Save logs is inside the View logs pop-up now 2026-10-16 19:54:56 +02:00
Niko1221 d465e37bc1 setup: NVFP4-QAD, NVIDIA's FP4-Local checkpoint downloaded pinned and prepared on this PC (nvfp4_local_pack prepare), with its own draft layer 2026-10-16 19:40:17 +02:00
Niko1221 c163678f7d merge f-split-agent6 0b871e2d7 (compaction, /research, response settings, browser and skills checks, time-skip file order, step summaries) into f-split-int 2026-10-16 19:38:02 +02:00
Niko1221 0b871e2d79 Time-skip: a big file that did not change is not listed as changed 2026-10-16 19:34:49 +02:00
Niko1221 4cca90e757 nvfp4_local_pack pack: the experts straight from the checkpoint through iq_pack (no 74 GB GGUF in between) 2026-10-16 19:32:05 +02:00
Niko1221 87d70909a8 Time-skip: the made page first in the compare list, housekeeping files (browser profile, caches) last and unread, never the whole result 2026-10-16 19:29:57 +02:00
Niko1221 7f9e719c8a nvfp4_local_pack: slim (the dense-only GGUF), convert --dense-only 2026-10-16 19:27:08 +02:00
Niko1221 f37abb1365 Tests follow the new reply room; the queued-message test waits for the goal to be running 2026-10-16 19:25:32 +02:00
Niko1221 2b54cf4fbd Model servers on this network get the thinking vocabulary of a local model (the Reasoning menu did nothing for them); effort and sampling drivers 2026-10-16 19:23:03 +02:00
Niko1221 b6b546ae13 Response settings reach agent turns (thinking budget, Expert sampling) through a local forwarding endpoint; step summaries from what the step changed; plan title heading 2026-10-16 19:09:54 +02:00
Niko1221 d42319b6c9 ple_reader_test: NVFP4 shares the I8 type with FP8; the metadata picks it 2026-10-16 19:05:11 +02:00
Niko1221 c937aeb1df /research gathers first (notes hidden), nudges once when no page was opened, then writes the answer in the fixed shape; /research in the slash menu 2026-10-16 19:04:25 +02:00
Niko1221 9c6255cfce NVIDIA Qwen3.8-Flash-Next-FP4-Local: converter (CSF scale decode, MXFP8 to Q8_0/BF16, MTP, NVFP4 PLE table) and the NVFP4 PLE row format
tools/nvfp4_local_pack.py convert|ple|mtp|verify reads NVIDIA's ModelOpt checkpoint and writes what the existing NVFP4 path takes:
the CSF block scales are decoded to plain E4M3 (experts then repack bit for bit), MXFP8 weights are dequantized exactly and take the
Q8_0/BF16 policy of nvfp4_convert, the n-gram table becomes a one-tensor GGUF of 90-byte rows, the MTP head the folder mtp_pack reads.
ngram.cpp reads that table as a new PLE format (I8 bytes, strata.ple.format = nvfp4x16).
2026-10-16 19:02:17 +02:00
Niko1221 998e9df959 /research: web search and page reading with a structured answer and sources; http driver commands for plan, mode switch and long chats 2026-10-16 18:52:45 +02:00
Niko1221 c6a86ce168 Local models compact automatically: reply room is half the window and compaction headroom fits the window 2026-10-16 18:52:45 +02:00
Niko1221 743a6b3c61 merge f-split-ui5 e9a2ca44a (round-7 UI, time-skip fixes, mode switch mid-goal, /plan, queued messages, uninstaller icon; includes agent6 eecb29593) into f-split-int 2026-10-16 18:49:39 +02:00
Niko1221 e9a2ca44a7 UI round 7: plan card, Plan sidebar, goal step and queued messages built on agent6's backend contract 2026-10-16 18:49:13 +02:00
Niko1221 89d0965408 Merge branch 'f-split-agent6' into f-split-ui5 2026-10-16 18:43:25 +02:00
Niko1221 8da1d3f8c1 Installer: the uninstaller uses the Strata icon, and the shortcuts name it explicitly 2026-10-16 18:43:25 +02:00
Niko1221 eecb29593c Plan mode (/plan) with approve/edit/reject/start-as-goal routes, goal steps with per-step events, queued messages during a goal 2026-10-16 18:42:14 +02:00
Niko1221 83c8f0b120 UI round 7: /plan card, Plan sidebar tab with step states, queued messages with cancel, floating round arrow, plain Privacy without a box 2026-10-16 18:37:30 +02:00
Niko1221 c751c23965 Merge branch 'f-split-agent6' into f-split-ui5
# Conflicts:
#	apps/host/src/components/TimeSkipPanel.tsx
2026-10-16 18:17:44 +02:00
Niko1221 2e1fcd15d9 Time-skip: run the page's own script in the A/B preview, working view in the sidebar, retry the file list; mode change applies to the next tool call mid-run 2026-10-16 18:16:56 +02:00
Niko1221 e849aef2ae UI round 7: live chat numbers with tween, fixed status-line columns, Subagents counts on title row, plain Privacy text, round latest arrow, time-skip never empty 2026-10-16 17:59:07 +02:00
Niko1221 50f973c9f9 merge f-split-agent5 60bfe0693 (agent-children stability: agent-bridge race, request-flow test wait) into f-split-int 2026-10-16 16:06:10 +02:00
Niko1221 60bfe0693a stability: helper rows always show working before completed (the held row is sent first), agent-children tests order frames by arrival not by the clock and use generous bounds, request-flow waits for a slow host start 2026-10-16 16:05:42 +02:00
Niko1221 9550318e34 merge f-split-ui4 2812c939f (Use your own GGUF: the refusal reaches the page, a second preparation no longer fails) into f-split-int 2026-10-16 15:20:24 +02:00
Niko1221 2812c939fe local-gguf-e2e.mjs: the Use your own GGUF flow through the installed layout on a box (preview, refusal, prepare, load, answer, unload) 2026-10-16 15:19:41 +02:00
Niko1221 87b90a7882 Preparing an adopted GGUF folder a second time no longer fails on the data folder
An adopted model reports the parent of the person's own GGUF folder as its data folder; the next preparation sent that as the folder for the packs and Control refused it ('a new data folder must be named Strata'). A preparation from a GGUF folder now uses Strata's own download folder. Found by the real-box check on P2003; test fails without the fix.
2026-10-16 15:17:11 +02:00
Niko1221 fa25086125 Use your own GGUF: an unsupported or incomplete GGUF folder is shown in words, not as a generic failure
Control answers a refused folder with exit 5 and the plan (ok:false, issues); the host threw that away and the page said only 'The files could not be checked'. The plan with its reasons now reaches the page, which says 'Strata can't run these files yet' and keeps the checker's codes under Details. Found by the real-box check (a Gemma vocab GGUF on P2003).
2026-10-16 14:53:25 +02:00
Niko1221 7c4b3d743e merge f-split-ui4 a7cd2b38e into f-split-int 2026-10-16 14:33:50 +02:00
Niko1221 a7cd2b38e6 Tune strings that already live in the Tune catalog are not duplicated in the main one 2026-10-16 14:33:24 +02:00
Niko1221 9cede7db08 ui-flows: Models one-view check, outcome card and an empty task navbar 2026-10-16 14:30:31 +02:00
Niko1221 35c72bcd10 Models tab in one view with the family switch, one result card for every tune outcome, the tuned values in Expert mode, glow instead of a spinning ring
- Models: the family switch (Qwen3.8-Flash-Next, Qwen3.6-35B-A3B) is back in the navbar without the word Model; version, context and computer use, the tuning line and Load/Tune sit in one card beside This computer, so the tab fits the first view; image generation moves into Expert mode
- Tune: one result card for faster, already the fastest, couldn't measure (busy PC, with plain copy and Try again), failed and stopped, driven by the host's code, reason_code and values; the same Tune dialog frame for all of them
- Tune running: the step icon glows and a small dot beeps instead of a ring spinning around a square tile; reduced motion respected
- Expert mode: Context memory and Low-RAM mode show the active tuning's values, the engine settings list what the tuning sets, and loading does not re-prepare the model because of them
- the tuning line (Using your tuning / Switch off) is a calm line with an icon
- strings in 8 languages; tests in tune-outcome.test.mjs
2026-10-16 14:30:31 +02:00
Niko1221 99cc9829a0 Other local models: the back link says Back; the running version's group can be collapsed 2026-10-16 14:30:29 +02:00
Niko1221 837f992bd9 Owner round 6: short model names, one logs button, plain Monitor sentence, Privacy notes, Tasks cleanup
- Settings > Storage: downloaded optimized models show family + quantization with the full name as a tooltip, size aligned, Delete at the end
- Engine > Diagnostics: one View logs button; Save logs lives in the logs view
- About > Privacy: six short lines across the box instead of one long paragraph
- Monitor: the request summary is a plain sentence ('In 18 requests your average output was 47.4 tok/s and prompt reading 1,332 tok/s, reusing 91K tokens.'); the screenshot leaves out Model state and Conversation cache
- Tasks: Bypass permissions replaces 'Allowed to make changes' and is the default for a new task; an opened task has an empty left navbar and no Model section, its name in the page, and a readable Markdown last answer; no deleted-tasks list, restore or suggestions; + icon opens the editor modal (Draft it with AI inside it); removing says 'Task removed'
- strings in 8 languages; tests in ui4-fixes.test.mjs
2026-10-16 14:30:29 +02:00
Niko1221 19ea30f6ab merge f-split-agent4 4b5cc08d1 (skills end to end, New skill button, agent skills folder, goal-status race fix) into f-split-int 2026-10-16 14:26:57 +02:00
Niko1221 4b5cc08d1a a goal-step status is never swallowed by the 300 ms gap (fixes the flaky /goal test); skills strings re-added to the merged catalog 2026-10-16 14:24:21 +02:00
Niko1221 ca34357b08 skills: plain card words for loading a skill (8 languages), Skills page render script 2026-10-16 14:21:33 +02:00
Niko1221 4818b1f4a1 Skills: write one in the Skills page (New skill), saved as a folder in the app's data; the whole list is written out for the agent, which scans it (customSkillDirs) and watches it, so a saved skill reaches the next message without a restart; tests 2026-10-16 14:21:32 +02:00
Niko1221 0c165b48bf merge f-split-s5 dab63e9c3 (tuning presets on Models, effective values, outcome codes, apply_saved_tuning) into f-split-int
native-control.mjs: this tree's publicState/localPreview kept, s5's tuneEffective/tuneOff added to the exported API. NativeModels.tsx: the loaded-model Unload/Ready actions kept, s5's NativeTuneBadge added above them. NativeTune.tsx: ui2's result view kept; s5's busy_pc sentence and the steady-aware answer notice added.
2026-10-16 14:19:31 +02:00
Niko1221 dab63e9c38 tuning feedback: structured outcome (faster / already_fastest / busy_pc / failed / stopped + values), tuned values readable (presets effective, settings carry them), which tuning is active and off/on, applied at every start (restart and unload/load tests) 2026-10-16 14:16:07 +02:00
Niko1221 a79e653036 merge f-split-agent3 df52a2854 (API connection output limit) into f-split-int 2026-10-16 14:12:08 +02:00
Niko1221 f9bfe8bc12 merge f-split-agent3 c99896ae6 (status, goal and time-skip events and backend, time-skip keepalive and stop, atomic.py) into f-split-int
Conflicts: locales (agent3's 52 keys renumbered s2937-s2988 in all 8 files), timeskip.mjs (ui3's image() kept, agent3's stop()/whenEnded kept, imports unioned), store.ts (ui3's comparison-open rule plus agent3's markTurn), Chat.tsx (ui3's working card and chat card kept, agent3's timeSkipTab dropped), time-skip.ts (duplicate activity field removed).
2026-10-16 14:12:08 +02:00
Niko1221 df52a28548 API models: answer room is the connection's own maximum output when the provider lists one, else a generous 32K bounded by the window minus the prompt (was a quarter of the window, at most 8,192, which cut thinking models off with no answer) 2026-10-16 14:09:37 +02:00
Niko1221 334dde73a2 merge f-split-ui3 088f8685a (live status line, reasoning panel, goal bar, tool rows, time-skip comparison) into f-split-int 2026-10-16 14:09:13 +02:00
Niko1221 c99896ae66 the real-browser test is opt-in (STRATA_TEST_BROWSER=1); HTTP driver goal bar command 2026-10-16 14:08:27 +02:00
Niko1221 04cca3bca8 status sentences carry their catalog template and values and exist in 8 languages; goal tools have plain words; per-request thinking count 2026-10-16 14:05:21 +02:00
Niko1221 20b377d30d Time-skip and tune job records are replaced with a wait for a reader (Windows 'Access is denied' when the host reads the file at that moment: it ended jobs at random) 2026-10-16 13:29:12 +02:00
Niko1221 088f8685a7 Time-skip in chat: working view with Stop, comparison sidebar only when there is an update, A versus B that adapts to the kind of file
- while a real run works, an animated A/B card shows what the assistant is doing (host activity), elapsed time and Stop (TimeSkipWorking); the sidebar no longer opens at the start and opens only when the finished run changed files; a run that changed nothing says so in the chat and has no tab
- the finished A versus B (TimeSkipCompare) keeps one window size whatever was made: text and Markdown rendered with changed words marked; HTML as two sandboxed previews (no permissions, no scripts, no outside requests) with a View code toggle; pictures side by side, click to enlarge; code and other files as a folded line diff; several files through a compact file switcher
- host: changed pictures are listed as pictures and served from the run's own folders by GET /api/timeskip/file (images only, inside the run, size-capped)
- strings in 8 languages; tests for the file kinds, word and line diffs, the HTML policy, the activity words and the picture route
2026-10-16 13:20:20 +02:00
Niko1221 3e6c354504 status test no longer depends on timing; resume following a Time-skip after a reload keeps the Stop button 2026-10-16 13:19:19 +02:00
Niko1221 b085d54b94 status tests (plain chat, real Harness file write), HTTP driver for real-model checks 2026-10-16 13:16:59 +02:00
Niko1221 ee8b23bcc7 Time-skip: streamed revision answer with a beat (a slow turn no longer ends in 'could not be reached'), Stop ends the job tree and deletes the copies, activity in the status route and SSE, sidebar only when there is an update, Stop button while it runs 2026-10-16 13:15:12 +02:00
Niko1221 7c4092dabe Chat feels alive: live status line, one-line reasoning, goal bar
- status line under the newest message: pulse, a rotating playful word only while it thinks, the real action once a tool runs, elapsed time and tokens; only the newest status frame is ever shown and it is cleared on idle, done, error and stream end (src/lib/live-status.ts)
- reasoning is one quiet line (what it is thinking about, then 'Thought for 38s'); opened it follows the newest line unless the person scrolls up
- goal bar above the input: small, icon-only pause/resume, edit, remove, from the goal frame, the SSE goal event and /api/goal (src/lib/goal.ts); resume sends goalAction:'resume' with no user bubble
- all strings in 8 languages; tests in tests/live-chat.test.mjs
2026-10-16 13:14:01 +02:00
Niko1221 965f22e3db live status frames for agent, goal and plain chat turns (thinking tokens, composing a file call, running tool, waiting for approval, helpers, goal step), goal state frames and goal bar backend (state, pause, edit, clear via the Goal Remotes; resume through the chat stream) 2026-10-16 13:11:40 +02:00
Niko1221 44a12d78be Tool rows in chat: compact lines, no dividers 2026-10-16 13:05:20 +02:00
Niko1221 aaf37247da integration after ui2: a second-try failure is shown once (details keep the first stop), error Details box is height-capped so the small tuning pop-up never scrolls, ui-flows expects the real tuner's Tune model button 2026-10-16 11:55:14 +02:00
Niko1221 c5755ed837 merge f-split-ui2 baeb4533b (approved UI designs, context fix, workspace default with rename, autotune progress) into f-split-int
Locales: both sides added keys from s2676, so ui2's 88 keys are renumbered s2849-s2936 in all 8 files. events.mjs: ui2's verbsFor target plus agent2's ask card. index.mjs: the node:path import takes both sides' names (relative, sep). The workspace default is ui2's (Documents/Strata/Workspace).
2026-10-16 11:27:06 +02:00
Niko1221 baeb4533be Approval cards in plain words, Tasks open view, This computer card, version list, and the Tune steps with a live speed
- approval cards: one plain question per tool call (server/agent/ask.mjs maps commands and files to sentences; src/lib/ask.ts holds the translated templates), a compact calm card, the exact command under Details
- Tasks: the opened task shows schedule, what it may do, last result, instructions and last answer; back link before the title like the chat's navbar; pause/resume and delete icons, Edit, Run now; pause and play icons added to the sprite
- Models: 'This computer' shows graphics memory and RAM bars with what the chosen version takes, and one plain line about the SSD (src/lib/native-fit.ts, from the catalog's memory_gb and the hardware report; the n-gram table is not counted)
- Models: one-line version list: Recommended (the fixed four), Swift, GSQ-RCO, NVFP4, Gyro, Unsloth groups, Use your own GGUF; RAM+VRAM in bold, download size muted, 'Uses SSD N GB' only where it applies
- Tune: five steps with their own animated icons and states, a live speed readout and sparkline, a result card with the gain and what it was saved as; reduced motion respected; the tuner now reports each measurement's speed
- catalog: memory_gb and mem_total per version (tools/strata_mcp.py)
- tests: ask, native-fit, tune-phase, autotune speeds, memory figures; headless checks (ui-flows ui2/ui2d, scripts/approval-flow.mjs), light and dark
2026-10-16 11:24:09 +02:00
Niko1221 ea2d565e60 merge f-split-agent2 c7b65c7f7 (/loop, /goal, /time-skip, keyless search, system-browser tool, Settings Tools, Plan refuses connected tools) into f-split-int 2026-10-16 11:08:59 +02:00
Niko1221 c7b65c7f7b tools flow script restores the settings it changes 2026-10-16 11:05:57 +02:00
Niko1221 d146b0da29 plain browser card sentences (waiting, scripts, tabs, back, close), no empty braces on cards; tools flow script for the real model 2026-10-16 11:01:55 +02:00
Niko1221 df6262f270 Merge perf-PR group sm75 (f-145-pp-sm75) 2026-10-16 10:59:08 +02:00
Niko1221 9f24934663 tuner: a run that could not measure any candidate is inconclusive, not default confirmed 2026-10-16 10:58:33 +02:00
Niko1221 21c579727c Merge perf-PR group rdna2 (f-145-pp-rdna2) 2026-10-16 10:54:10 +02:00
Niko1221 b277412eb1 Merge perf-PR group rdna34 (f-145-pp-rdna34) 2026-10-16 10:53:17 +02:00
Niko1221 cbc6e3d8f8 Merge perf-PR group cache (f-145-pp-cache) 2026-10-16 10:53:00 +02:00
Niko1221 70f8268dab Time-skip refuses the app's own folder and a folder too big to copy, in plain words in all 8 languages 2026-10-16 10:52:01 +02:00
Niko1221 2f3a5d9013 Merge perf-PR group serve (f-145-pp-serve) 2026-10-16 10:51:54 +02:00
Niko1221 da6d8d9205 Merge perf-PR group batch-1441 (f-145-pp-batch-1441) 2026-10-16 10:51:15 +02:00
Niko1221 e8b1edbcb7 Merge perf-PR group batch (f-145-pp-batch) 2026-10-16 10:49:15 +02:00
Niko1221 0848864a03 cards for the browser and the person's own tools say what happened in plain words (8 languages) 2026-10-16 10:48:49 +02:00
Niko1221 8075af7c2a calmer Time-skip wording (one note, no repeated reassurance), plain Manual reason for the person's command tools, agent README for Settings Tools 2026-10-16 10:47:46 +02:00
Niko1221 5a6f4ed5fb Merge perf-PR group cuda86 (f-145-pp-cuda86) 2026-10-16 10:46:42 +02:00
Niko1221 533f6b713f browser tool (Playwright MCP on the person's Chrome or Edge, separate empty profile; Manual asks before opening, clicking, typing, not before looking; Plan refuses), staged into the installer; Tools screen strings in all 8 languages; Time-skip result view without the sample success check; /loop words translated 2026-10-16 10:45:44 +02:00
Niko1221 926b0bbe6d Merge perf-PR group sycl (f-145-pp-sycl) 2026-10-16 10:43:56 +02:00
Niko1221 62b7d251ae Merge perf-PR group cpu (f-145-pp-cpu) 2026-10-16 10:43:55 +02:00
Niko1221 c20c4aed11 tuner: say so when the default itself is not repeatable (RTX 3060 resident mode, A750); exact knobs are still auto-picked there 2026-10-16 10:36:37 +02:00
Niko1221 24e0157b53 Settings Tools behind the screen: tool groups on/off reach the agent (restart on change), web search with no key (DuckDuckGo, polite, cached) or the person's own Exa/Perplexity/DeepSeek key in the environment only, the person's own tools (instructions, commands, tool servers) through the app's tool server, Plan refuses connected tools, keyless API connections work in the agent; browser finder 2026-10-16 10:35:36 +02:00
Niko1221 f662759046 Workspace rename strings in all 8 languages (after rebase onto df13760cf) 2026-10-16 10:23:59 +02:00
Niko1221 0d4e0dde8c Settings Tools tab (screen only, mock-up state), Time-skip trial runs on the chat's API model too 2026-10-16 10:22:09 +02:00
Niko1221 db9c322bce slash commands: /goal and /loop in the menu with plain descriptions, /loop repeats a message while the chat is open, /time-skip runs a real job on a copy of the chat folder (assistant revises, files listed, update kept or original restored on request, rollback copy); apply of a reviewed attempt in Strata Control 2026-10-16 10:22:09 +02:00
Niko1221 a655bf22b2 Context from cached prompts for agent chats, and a default workspace that is never the app folder
- agent context: the Harness reports a cached prompt as cacheReadTokens, apart from inputTokens; the context now counts input + cache read + cache write + reply of the last request (a long agent chat showed 0-3%; checked against a real Swift IQ2_XS 64K run: 8,152 tokens, matching the engine's own 8,042-token prompt + 110 reply)
- first workspace: Documents\Strata\Workspace (display name Strata) unless STRATA_WORKSPACE names one, never the app folder; a saved first workspace that is the app folder moves there and its chats follow through an alias
- workspaces can be renamed (display name only, path kept): PATCH /api/workspaces/:id, pencil in the sidebar, 8 languages
- tests: workspaces.test.mjs, cache-aware context in functional-fixes.test.mjs
2026-10-16 10:22:08 +02:00
Niko1221 39f3670d41 docs: STRATA_KV_HOST_REGISTER and our measurements for STRATA_EXCHANGE_ROTATE (#2042) and STRATA_PCIE_BALANCE (#1548)
#1498 pins the KV host copy with cudaHostRegister under WSL; the switch forces either order. Measurements: 5 interleaved pairs on an RTX 3060, an RTX A4000 and a Tesla P100.
2026-10-16 10:18:08 +02:00
Niko1221 df13760cf4 files made by a run as administrator no longer lock the user out: explicit user-SID entry and owner on create (private_mkdir, restrict_to_current_user), process default owner/DACL set to the user, self-repair of Strata's own per-user folders (APPDATA, LOCALAPPDATA, ~/.strata) on read, at host start and by 'strata repair-access', plain message when it cannot; tests; tune_session loopback allowed in the no-acquisition audit 2026-10-16 10:10:43 +02:00
Niko1221 e1dca4bff5 merge f-split-gyro b15b209a0 (Step B: Gyro trellis types through MMQ in the pinned ggml via an overlay) into f-split-int 2026-10-16 07:08:30 +02:00
Niko1221 95e4f27f17 tuner knobs from TUNING-SPEC draft 2: kv-resident 65536 at long contexts and the prefill chunk as exact candidates (prompt metric), pin-by-need/host-core/kv-grow last, no workers two below, single best knob 2026-10-16 06:42:20 +02:00
Niko1221 1ea66f60af owner decisions: goal, skill, workflow and fork tools back in the agent (short plain prompt kept); agent and chat default reasoning effort Medium, with a test per level (Off/Low/Medium/High/unset) on the chat_template_kwargs the server receives; Tune dialog switch reads 'Allow settings that change answers very slightly (same quality)' in all 8 languages, off by default, with a test 2026-10-16 06:30:44 +02:00
Niko1221 394de92915 Functional-test fixes: agent tok/s and context, plain tool rows, Thinking Off proof, profile used, no download wording for installed models, plain chat, smaller agent prompt
- agent path: live stats frames and a final tok/s from the stream timing; the context is the last request (one source)
- tool rows say what happened in words (Wrote a file, Ran a command, Couldn't write the file) with the file or command; approval cards show it too
- a launch that returned without error reports 'Started.' instead of 'nothing to show'
- agent system prompt replaced by a short plain one; goal, skill, workflow and fork tools left out (first message about 7.5k -> about 4.3k tokens)
- Models: the running model reports the profile it was started with and the page shows it; Reasoning popover names the loaded native model
- installed model with other settings: 'Apply settings and load', no size, no download wording
- plain chat: agent:false or tools:false answers directly and never runs a tool
- tests: stats/context/verbs/launch/prompt size, server-side enable_thinking=false with the agent's request shape, plain chat over HTTP, ui-flows profile scenario
2026-10-16 04:20:20 +02:00
Niko1221 b15b209a03 Step B: Gyro's trellis MMQ prompt kernels on an upstream ggml, so the packaged engine reads Gyro prompts as fast as rc2
cmake/strata_ggml_tq_overlay.cmake writes, at configure time, a copy of the pinned ggml-cuda headers (and ggml.h, quantize.cu) into
the build folder with TQ2_T / TQK6 / TQK7 routed the way IQ2_XS is: same Q8_0_16 tile layout, dot kernels and tile configs (the
fork's own configs for them are copies of IQ2_XS's), three template instances, the tile loader and tq.cuh from third_party/ggml-tq.
Only the strata_mmq target sees the copy; every other type is compiled from the untouched files. Anchors are checked: another
llama.cpp turns the overlay off with a message (FP16 prompt path). moe_mmq.cu knows the trellis block geometry itself.

The tile loader and the trellis tables are agentionai/llama.cpp's (MIT, ggml authors; Laurent Zuijdwijk), ported from
agentionai/llama.cpp 93a24e5 ggml/src/ggml-cuda/{tq.cuh,mmq-load-tiles.cuh,tq-lut-i8.cuh}.

Checked: Gyro-S and Gyro-M exact_pp 4096/20000 residual dumps IDENTICAL to rc2 (R9700 gfx1201); Q2_0 and IQ3_XXS IDENTICAL to f-split
on gfx1201 and sm_86; ctest ggml_tq_overlay.
2026-10-16 03:51:27 +02:00
Niko1221 7ff67fe772 docs: the #2340 / #2207 Q6_K decode kernels and their switches (STRATA_MMVQ_LARGE_GRF, STRATA_Q6_PLANAR) 2026-10-16 03:46:10 +02:00
Niko1221 76cdc271c0 merge f-split-s4b 0398e09af (README docs on top of 0edf5cb9f) into f-split-int 2026-10-16 03:22:26 +02:00
Niko1221 15937b7e80 tuner measures the engine a load starts: the app's request-slot choice (and fit_max_tokens) moves into buildConfig, which both the load and the tune use; test that a tune's saved config equals a load's for the same model and profile, on a 12 GB and a 48 GB card 2026-10-16 03:20:28 +02:00
Niko1221 a161098036 merge f-split-s4b 0edf5cb9f (Harness in the app chat, Manual approvals, request slots) into f-split-int
index.mjs: createNativeControl gets control, appUpdater and hardwareProbe; the agent bridge gets componentsDirectory() and the bundled Harness folder.
native-control.mjs: this tree's file kept, s4b's parallel choice, fit_max_tokens, instance parallel/mcpServers re-applied after buildConfig; its own retry around control.start is folded into load-host's: ONE second try in all, which carries the memory change and/or the drop of several-at-once, with load-host's plain sentence (FALLBACK_NOTE only when no memory change applies). Test added.
locales: both sides had added keys s2571.., so s4b's seven task keys are renumbered s2615-s2621 in all 8 files.
2026-10-16 03:17:17 +02:00
Niko1221 0398e09af5 agent README: thinking and output budget, tasks 2026-10-16 03:12:53 +02:00
Niko1221 004e22c1de merge f-split-s5 (real tuner and presets) into f-split-int
native-control.mjs: this tree's file kept (S6 updates, load retry, remove), s5's buildConfig split out of loadEngine, auto-tuner, tuneConfig/tuneStatus/autotune, autotune and recheck operations and presets re-applied.
NativeModels.tsx: s5's Tune model button (the real tuner) and NativeTune dialog; the old short-benchmark Tune button stays out. native.ts: job progress field.
2026-10-16 03:11:50 +02:00
Niko1221 9691ea2763 sycl: the planar Q6_K head kernel gets the large-GRF launch at 2-8 columns on Arc A-series (#2340 + #2207)
The planar kernel of #2340 is the wide kernel's arithmetic, but it was launched without the 256-GRF mode that #2207 gives the GGUF-layout
wide kernel, so on an Alchemist card it spilled: mmvq_bench 2560 x 248320 on an A750, 4 columns, 1.78 ms (GGUF layout, large GRF) against
15.4 ms planar, 8 columns 4.8 against 48 ms. Same rule as #2207 (STRATA_MMVQ_LARGE_GRF, default on Alchemist only), same arithmetic.
2026-10-16 03:06:25 +02:00
Niko1221 a4d20310f1 Gyro needs engine 0.1.45: it ships in the split release's engine (GYRO_ENGINE, docs, test) 2026-10-16 03:05:27 +02:00
Niko1221 0b22b8c736 tuner: fix the RAM-floor rule syntax 2026-10-16 02:48:56 +02:00
Niko1221 a3dd4b702d tuner: a candidate is refused for low free RAM only if it is worse than the base (a RAM-bound base can sit below the floor itself) 2026-10-16 02:48:30 +02:00
Niko1221 a1447b678e Merge f-145-fix3: #2622 PLE gather, #2267 config args, #2644 split share, #2637 note, #2666 P2P probe, PR #2425 opt-in 2026-10-16 02:41:38 +02:00
Niko1221 5fb8fa95b3 load retry: the same stop twice is reported once (the second try stays in the job's details); fixes the 1280x700 tuning-error pop-up that scrolled in ui-flows 2026-10-16 02:33:25 +02:00
Niko1221 bcbf2caf3f merge f-split-gyro (Gyro-S/M trellis port and catalog entries) into f-split-int 2026-10-16 02:22:37 +02:00
Niko1221 31d36db678 docs: SECOND_GPU.md - the peer tier tests peer access at start, keep headroom on both cards (#2666) 2026-10-16 02:18:30 +02:00
Niko1221 3315072467 peer experts: peer access is used only if a test copy each way arrives exact (#2666)
On 2x RTX A4000 behind a host bridge cudaDeviceCanAccessPeer says 1 and cudaDeviceEnablePeerAccess succeeds, and
cudaMemcpyPeerAsync then returns success having moved nothing (1,048,576 of 1,048,576 floats wrong in both directions
in a stand-alone test; right without peer access enabled, the driver stages it). The peer prompt path trusted it: the
peer's input rows were zeros and its results never reached the primary, the first prompt's MoE output was garbage and
the logits NaN (--peer-device 1, --peer-reserve-mib 1500..2100; STRATA_PF_PEER_COMPACT is not the cause).
PeerExperts::open now copies 64 KiB each way once after enabling access; a wrong result turns peer access off again and
takes the host route (the one STRATA_PEER_NO_P2P=1 asks for), and says so. A test that cannot run keeps the old trust.

Reported by @CaptRauti on #2666 (WDDM, 2x RTX 3060: the no-P2P route is the one they ran).
2026-10-16 01:52:55 +02:00
Niko1221 0edf5cb9fc tasks: the host's task event reaches the page (toast); real-clock task and mode flows for a real model 2026-10-16 01:49:30 +02:00
Niko1221 14acfc3b3c desktop test: the stable-origin test no longer assumes port 47831 is free (an installed Strata holds it on this PC) 2026-10-16 01:42:15 +02:00
Niko1221 529a7019ae install job: the finished job keeps the merged progress (engine part and parts); test that the packaged install job fetches the engine through Control and that a failed engine keeps the model 2026-10-16 01:36:22 +02:00
Niko1221 fd0d7fd875 tuner: the quick budget reserves only the extra base rounds and the final base 2026-10-16 01:36:19 +02:00
Niko1221 0bf5fe1ebe tests: the packaged-mode fixture allows Control's installer tool process (fr runs strata_mcp.py there); still no other process 2026-10-16 01:33:54 +02:00
Niko1221 07fe758ac9 merge f-145-int 194c26881 (model-card sampling defaults, web sampling, answer room, thinking budget) into f-split-int
serve/server.py: the model_arch line joins the existing engine == strata block next to requested_slots.
2026-10-16 01:31:17 +02:00
Niko1221 90a5d082bc merge f-split-load (pin_budget reserves on Windows) into f-split-int 2026-10-16 01:30:58 +02:00
Niko1221 fa0d99e5b7 merge f-split-load-host (one safer second try when a load fails) into f-split-int 2026-10-16 01:30:58 +02:00
Niko1221 8820fa433d merge f-split-fr (owner app UI fixes, rounds 1-5) into f-split-int
native-control.mjs: fr's file kept, S6's engine step, component updates, plain update messages and app updater re-applied on it;
native.ts: fr's types plus capabilities.componentUpdates; SettingsUpdates: fr's per-row update buttons, Update engine enabled without a model when component updates exist.
2026-10-16 01:30:48 +02:00
Niko1221 781690f397 tuner: per-request points get a control and a fresh-engine 5-round confirmation (the first real run showed sending any setting reads ~10% faster) 2026-10-16 01:25:47 +02:00
Niko1221 20e0bf2e9d direct chat: Thinking budget in the Reasoning effort menu (reasoning_budget_tokens); Off reaches the server as none 2026-10-16 01:25:09 +02:00
Niko1221 3571ceb1e4 do not track generated Tauri schemas 2026-10-16 01:22:20 +02:00
Niko1221 a4a0da0d5d app updater: the installer starts outside the app's Job Object (found by the end-to-end run), plain message when no update key exists 2026-10-16 01:22:14 +02:00
Niko1221 22265c6ac7 tuner: fix the killed-engine pattern 2026-10-16 01:16:07 +02:00
Niko1221 e62e6bf2f5 tuner: a killed engine (no message) is an out-of-memory 2026-10-16 01:15:37 +02:00
Niko1221 84ca0b6c21 tuner: five decodes per run, early drop only after a 5% loss, noisy default-confirmed says so 2026-10-16 01:11:57 +02:00
Niko1221 daa45b1e37 thinking: the Strata server reads chat_template_kwargs, not a top-level enable_thinking (Off had no effect); plain output for read-only PowerShell noise 2026-10-16 01:11:17 +02:00
Niko1221 ff62ce3fbe app: Tune strings without the duplicate Check again 2026-10-16 01:11:15 +02:00
Niko1221 7d43470f31 app: Tune dialog tidy, short Check again label; headless UI fixture for it 2026-10-16 01:10:56 +02:00
Niko1221 14168890c9 app catalog: a test for the Gyro sizes in the Qwen3.8 group 2026-10-16 01:08:57 +02:00
Niko1221 10ede85dd1 tests: skip the real-install step test when the disk is too small for setup's own check 2026-10-16 01:05:23 +02:00
Niko1221 5f054d2a2c tests: resident-mode detection 2026-10-16 01:04:17 +02:00
Niko1221 aab3810b54 tuner: the GPU cache's 'resident experts' wording is not the low-RAM resident mode 2026-10-16 01:03:41 +02:00
Niko1221 bd773bbd77 app: Tune model dialog (real tuner, saved tunings, undo) reusing the Auto-tune look; its strings in separate catalogs 2026-10-16 01:02:17 +02:00
Niko1221 35cfd187cf host: autotune and recheck operations, saved-tuning routes (Strata Control's tuner and presets, run on the config a load would use) 2026-10-16 00:58:17 +02:00
Niko1221 eaf8745697 docs/GYRO.md: setup is the install path, our measurements, credit and licenses; docs/MODELS.md lists Gyro-S and Gyro-M; tools/gyro_setup.sh (rc2's script) is replaced by setup 2026-10-16 00:56:04 +02:00
Niko1221 c0616f06be tuner: AMD architecture from the kernel for the never list 2026-10-16 00:54:20 +02:00
Niko1221 e6b3af5889 tuner: waiting for the box lock does not count against the time budget; session tests 2026-10-16 00:53:30 +02:00
Niko1221 8e2460d18c owner feedback round: unified local-GGUF page and Manage models, no Tune model, Engine settings reduced with working Save logs, update buttons only when available, GPU data in Monitor with no model, thinking dots stop, Latest spacing, calmer native status reads, local GGUF adoption via --gguf-dir, no Contrast option or request-log button 2026-10-16 00:53:05 +02:00
Niko1221 33120935a3 tuner: a queued kernel flock for the box lock, so other streams' waiters are served in order 2026-10-16 00:52:06 +02:00
Niko1221 bb33349149 tuner: memory and quiet checks run inside the box lock right before each start 2026-10-16 00:49:59 +02:00
Niko1221 f23eda8c95 auto-tuner: single-knob candidates from the measured spec, interleaved rounds with noise and drift guards, memory guards, saved presets (one store for CLI and app), re-check, strata tune auto / strata presets 2026-10-16 00:48:42 +02:00
Niko1221 194c26881a serve: Anthropic thinking.budget_tokens is a thinking budget too; docs for the answer room and the budget; test 2026-10-16 00:44:38 +02:00
Niko1221 d0e990ba78 serve: max_tokens counts the thinking - without a budget, thinking is closed early enough to leave a quarter of the cap for the answer (cap 8192 cut the thinking off with no answer); config answer_room:false opts out; tests 2026-10-16 00:38:04 +02:00
Niko1221 326e391188 merge f-split into f-split-s6 2026-10-16 00:33:37 +02:00
Niko1221 158a42d8b6 tasks run as the agent in their own session under a per-task mode, with a plain result and a notification; task editor and toast strings in all eight languages 2026-10-16 00:33:05 +02:00
Niko1221 be1a871349 agent: output budget for a local model is the window (was 8192), the Reasoning menu reaches the request, fit_max_tokens for app-loaded models, plain outcome for silent commands 2026-10-16 00:30:13 +02:00
Niko1221 5e33c4e85d Gyro: the MCP recommendation asks 4 GB of room beside the memory figure; tests for the Gyro catalog 2026-10-16 00:16:01 +02:00
Niko1221 b74d6ad3d3 stage-harness: flatten pnpm's links into one real folder per package (the installer copies files) 2026-10-16 00:15:08 +02:00
Niko1221 375f76fef0 setup: a tip for Gyro installs on the ready-made engine (the fork's prompt kernels need a source build) 2026-10-16 00:14:27 +02:00
Niko1221 02bbf51fb7 Gyro: the Hadamard row kernels go up in slices of 65535 rows (gridDim.y limit): the FP16 prompt path rotates T*K rows, which a 16K-token chunk takes past it (invalid configuration argument on a build without the fork's MMQ) 2026-10-15 23:46:50 +02:00
Niko1221 e863486567 web: the page sends no sampling until the user changes it, so the server's model-card defaults apply (stored 0.6 counts as not set); test (#2685) 2026-10-15 23:34:15 +02:00
Niko1221 945971648a desktop: ship the staged Harness as resources/harness (built by stage-harness.mjs) 2026-10-15 23:28:08 +02:00
Niko1221 365c60e7a4 Metal: stubs for the Gyro Hadamard entry points (the engine does not run trellis experts; native_expert_supported refuses them at load) 2026-10-15 23:02:00 +02:00
Niko1221 34204c4a31 Gyro: the CPU trellis rows build for arm64 (scalar kernel; the AVX units are x86-only) and the Metal build lists hadamard.cpp and tq_cpu.cpp 2026-10-15 22:47:19 +02:00
Niko1221 805a308d2e pin budget: CUDA-on-Windows default only (the HIP build keeps its pinning), docs/DETAILS.md 2026-10-15 22:45:55 +02:00
Niko1221 a8091f4d2e serve: a request with no sampling gets the model card's (thinking 1.0/0.95/20; not thinking 0.7/0.8/20 + presence 1.5; Qwen3.6 too); greedy only when asked (#2685) 2026-10-15 22:35:55 +02:00
Niko1221 ff14364de0 packaging: recipe and table row for the gfx1103 (Radeon 780M) Linux HIP engine package
Built on P2006 from the f-split tree and run through the Terminal archive: resolve (kfd gfx1103), tune, serve and client, TUI, Ctrl+C/SIGHUP, saved preset, memory back to baseline.
2026-10-15 22:35:46 +02:00
Niko1221 4e67f1f045 host: one automatic safer second try when the engine dies while loading (smaller pin / RAM budget), said in plain words and not saved; Leave 20% also sets the engine's own RAM headroom 2026-10-15 22:31:15 +02:00
Niko1221 e1d11fe727 Gyro: native_fmt knows the trellis types' geometry itself, so a ggml built from upstream (no such types; its type table is indexed by the id) is never asked about them 2026-10-15 22:28:55 +02:00
Niko1221 4ff5cd4050 pin budget: test knobs (simulated ceiling, 'driver took everything'), report lines, smaller default reserve, pin_budget_test 2026-10-15 22:27:30 +02:00
Niko1221 4f74de9cdb Keep pinned-memory headroom while the expert arena pins; retry the engine's required pinned buffers once after handing it back (Windows load failure: verify mapped staging allocation failed) 2026-10-15 22:13:37 +02:00
Niko1221 45f6111fc8 docs: MULTI_GPU.md - an unequal pair under layer-split auto (#2644, measured on z440 with the speed-scale seam), STRATA_SPLIT_UNIFORM (#2418) 2026-10-15 22:03:34 +02:00
Niko1221 ffbc606682 delete a downloaded native model: strata_remove (own files and run config, shared files kept), Delete in Models and Storage, real disk use per model 2026-10-15 21:48:51 +02:00
Niko1221 eca3aef3a2 Gyro: the CPU trellis kernels run on every CPU (scalar without AVX2: upstream ggml has no trellis vec_dot to fall back to); the SYCL build lists the new CPU sources 2026-10-15 21:47:32 +02:00
Niko1221 ba64156fd4 Catalog: Gyro-S and Gyro-M (agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) in setup, the Terminal, the app and the MCP server
Both files are one GGUF with the n-gram table inside: 58.5 GB and 92.0 GB downloads for 27.6 GB and 35 GB of VRAM + RAM together.
Neither is a default; the MCP recommendation names them only where the 2-3-bit sizes and the Coder do not fit. Pinned to the Hub
revision 1ada08422e40 (sizes and SHA-256 from its LFS pointers). Model license: Qwen Community License 1.0.
2026-10-15 21:44:02 +02:00
Niko1221 ec53cae11f request slots: fit to the expert cache the card would have (measured), hardware from the host probe, safe default with no read 2026-10-15 21:43:30 +02:00
Niko1221 697cbc6376 stage-harness: peers, safe pruning; profile: office converter off 2026-10-15 21:41:06 +02:00
Laurent 26da97bde0 docs/GYRO.md: clone the rc2 branch (Strata v0.1.40.3 + Gyro)
Ported from agentionai/Strata rc2 9ff02c9 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:35 +02:00
Laurent Zuijdwijk 701f31964b docs/GYRO.md: the CPU trellis kernels and their switches
Ported from agentionai/Strata rc2 6bfb1e4 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:35 +02:00
Laurent Zuijdwijk 5ebea67b79 native_expert_parity: check the CPU trellis kernels in the synthetic modes
For tq2_t / tqk6 / tqk7 experts (--synthetic, --synthetic-hadamard), the
gate and down rows on every ISA the CPU has against the scalar kernel bit
for bit, and the scalar kernel against ggml-cpu's vec_dot on the same
Q8_0 activations. The CPU path of check (b) already runs through the new
kernels via native_gu_rows / native_down_rows.

Ported from agentionai/Strata rc2 7d0ecd0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:35 +02:00
Laurent Zuijdwijk d0a02fe426 CPU trellis kernels: AVX2 / AVX-512 decode-and-dot for TQ2_T / TQK6 / TQK7
Expert-cache misses of the Gyro files run on the CPU workers. Until now those
rows went through ggml-cpu's vec_dot one token at a time. Strata now has its
own kernels for the three trellis types (tq_cpu.hpp), chosen at run time like
the i-quant ones:

- the 32768 x 4 int16 state table (one 8-byte lookup per four weights, built
  from the codebook in third_party/ggml/ggml-common-tq.h, now also visible to
  host code), scalar state extraction, four table rows per YMM by broadcast
  loads + VPBLENDD, VPMADDWD (AVX2) or VPDPWSSD (AVX-512 VNNI);
- the weights of a block are decoded once for all the tokens routed to the
  expert (a verify window), then each token is a few instructions;
- one fixed arithmetic for every ISA: exact int32 sums per Q8_0 sub-block,
  four float FMA chains, a fixed final sum. AVX2, AVX-512 and the scalar
  reference give the same bits, and a token gives the same bits alone or in a
  group of any size (no group-size rule needed, cf. #152).

native_gu_rows / native_down_rows take them for types 144-146 on AVX2 and
AVX-512 CPUs. STRATA_NO_TQ_KERNELS=1 restores ggml-cpu's vec_dot,
STRATA_TQ_ISA=scalar|avx2|avx512 caps the ISA. A CPU without AVX2 keeps
ggml-cpu.

tq_cpu_parity (CPU only, ctest tq_cpu_parity and tq_cpu_parity_avx2):
AVX2 / AVX-512 vs the scalar kernel bit for bit for groups of 1-8 tokens,
vs ggml-cpu's vec_dot (rel ~1e-7, float order) and vs to_float in double
(rel ~1e-5, the int16 table), plain and Hadamard-rotated, for Gyro-S
(tqk6/tqk7) and Gyro-M (tq2_t) experts through the engine entry points.
`tq_cpu_parity --bench` is the micro-benchmark (clock-calibrated cycles).

Zen 5, one thread, weights streamed from DRAM, core cycles per 128-weight
block (one token / per token in a group of three):
  ggml-cpu vec_dot, AVX2 build:     108-117 / 108-116
  ggml-cpu vec_dot, AVX-512 build:   64-66  /  61-62
  Strata AVX2:                       47-49  /  22-23
  Strata AVX-512:                    47-49  /  22
  (Q2_0's AVX2 kernel: ~20; the repacked AVX-512 Q2_0 kernel: ~7)

Ported from agentionai/Strata rc2 7c85f1d (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:35 +02:00
Laurent 25a1f6e046 APR on 0.1.40: the sub-16 gate/up rows assume 80 dots per row; trellis types (qk 128) take the full-warp kernel
v0.1.40's row_dot_80_sub16 covers (n_embd / qk) * ipb == 80, true for the 256-value i-quants but not for
TQ2_T / TQK6 / TQK7 (160 dots at n_embd 2560): half of every gate/up row was dropped (native_expert_parity
gpu rel 0.87, garbage output). Restores gpu rel 2.13e-2 / decode-once bitwise.

Ported from agentionai/Strata rc2 c4ba288 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 84507d7179 hadamard_rows: say what is wrong with a bad block before exiting
hadamard_rows / hadamard_rows_f16 still exit on an invalid block, like
check() does on a failed launch, but now through one helper whose message
names the entry point, the block, the row width and the requirement (a power
of two in [2, 1024] dividing the width), and where the mismatch comes from.

Ported from agentionai/Strata rc2 48ff8b9 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 86faf035b6 prefill: say why the FP16 path rotates H in place in fp16
Not changed to an fp32 rotation: hadamard_rows_f16 already does its
arithmetic in fp32 and rounds once on the store, so the only extra error is
the fp16 rounding of the SwiGLU output before the rotation, while an fp32
copy would cost a T*K*640 float buffer or a fused kernel on the non-MMQ
fallback. The comment states this, the precision cost and the parity bound
(relative L1 < 2e-3 against the float rotation); the gate/up input refers to it.

Ported from agentionai/Strata rc2 50a1972 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 4557f68053 iq_pack: refuse to mark a native_experts.txt other than v3/v4 as v5
The v5 rewrite for Hadamard-folded experts was a re.sub on "v[34]": on any
other header it changed nothing and still wrote hadamard.txt, an
inconsistent pack the engine refuses at load. native_experts_v5() now
rewrites v3 and v4 only and otherwise returns an error naming the version
found and the supported ones; main() prints it and exits 1 before any file
of the pack is written. Tests: both rewrites, the refusal for v2/v5/v6/v10
and a missing header, and a full pack run that must leave no
native_experts.txt or hadamard.txt behind.

Ported from agentionai/Strata rc2 235787c (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 8fd8e6da3c prefill: resolve the APR sign vectors once per device, not per layer and chunk
run_impl called hadamard_device_signs for every folded layer of every chunk,
which takes a process-wide mutex and may cudaMalloc. Prefill::init now
resolves the per-layer gate/up and down sign pointers on its device into
Impl::had_sx / had_sh, as peer_had_signs does for the peer and the layer
split's helper (P.had_sh), and run_impl only reads them. carve no longer
warms the device table on every relayout (init's lookup does that now, once);
the comments on peer_had_signs and the new members say what is resolved when.

Ported from agentionai/Strata rc2 3d191fa (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk dd32704d4d prefill: look up the APR sign vectors at the layout's widths, not 640 / N
run_impl asked hadamard_device_signs for the down projection's signs at a
literal 640 (and gate/up's at N), while carve and the peer path resolve them
at lay.fmt[l].n_ff / n_embd. The widths are part of the device copy's key, so
the two only shared an entry because the widths happen to match. All lookups
now use the layout's widths.

The other 640s next to the rotations are the row width of the prompt path's
expert buffers and products (swiglu, quantize, GEMM), fixed for this model
like N; Prefill::init now refuses a native layout whose experts are not
N x 640 instead of assuming it. No other sign or rotation width in the tree
is hardcoded (verify, peer and remote experts use the geometry).

Ported from agentionai/Strata rc2 ff463c8 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 390da5d905 Trellis numerics: say which codebook each path reads and what is bitwise
Decode reads the int8 codebook tq_lut_i8 (one global scale, 0.85% relative
RMS); the FP16 prompt path reads the fp16 codebook and the MMQ path is the
GGML fork's tile loader. The comment over tq_load / tq_apply now says that
only the decode-once and per-entry kernels are bitwise equal to each other,
and that native_expert_parity bounds the decode path against ggml's float
dequantizer at relative L1 error < 3e-2 (the prompt dequantizers at < 1e-6).
The same goes into a short "Numerics" section of docs/GYRO.md and the parity
test's header.

Ported from agentionai/Strata rc2 457047e (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 99581ed5e3 verify: carve the rotated expert input only for Hadamard-folded packs
nat_xr_ (T x N floats) was taken from the verify arena for every pack, though
only a layer whose gate/up is Hadamard-folded reads it. It is now carved only
when some layer of the native layout has had_gu, as the prompt path does with
its rotated copy; the one reader is under that layer's f.had_gu, so it never
sees the nullptr.

Ported from agentionai/Strata rc2 687b7e6 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 294c95ebbc APR packs: drop the STRATA_APR opt-in; warn only on unvalidated setups
The GPU path for Hadamard-folded experts is now validated end to end on
CUDA sm_86, sm_89 (also a two-GPU layer split and peer decode) and sm_120,
and on HIP gfx1151, so `strata generate` no longer exits without
STRATA_APR=1 (the variable is ignored if set). It logs a warning instead for
the two setups that have not been run end to end yet: a HIP GPU other than
gfx1151, and the P2P prompt peer (Prefill::set_peer).

docs/GYRO.md and tools/gyro_setup.sh no longer set STRATA_APR=1; the setup
script's AMD note now says what was validated on gfx1151.

Ported from agentionai/Strata rc2 0fabd20 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 7709a0e2c8 docs/GYRO.md: read as part of Strata (no fork clone step); render-group note for HIP containers
Ported from agentionai/Strata rc2 f9a40f8 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk cce9b16a72 docs/GYRO.md: RTX 30-series setup line, RTX 4090 validation and the two-GPU notes
Ported from agentionai/Strata rc2 98125c0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent 926171c779 docs/GYRO.md: 2026-10-03 results (5090 #2, Gyro-M, 3090, AMD smoke); fix Gyro-S size
Gyro-S is 58.5 GB (54.5 GiB). Gyro-M and AMD have now run end to end;
long-prompt prefill is measured. The measured section now has the int8
decode numbers (rc1 includes them and --prefill auto), the server
protocol, Gyro-M and the 24 GB RTX 3090; the cache-size rows stay,
labelled as the 2026-10-02 build.

Ported from agentionai/Strata rc2 c494121 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk f5b7628a7f APR port: Split<144..146>::load takes upstream's STAGE_GRID form
Upstream 0.1.39 calls the decode-once loaders as
S::template load<STAGE_GRID>(row, kbx, iqs, s_grid) (the IQ types can stage
their grid table in shared memory). The trellis loader has no grid to stage;
it takes the template parameter and the table pointer and ignores them.
Found by the first CUDA build of rc1 (CUDA 12.4, sm_89).

Ported from agentionai/Strata rc2 07e9afd (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 28651a4079 APR port: multi-GPU prompt offload (peer, layer split helper) for Hadamard-folded packs
set_peer and set_stage_helper no longer decline an APR pack. Their MMQ
share now takes the same rotations as the primary's MMQ path:

- gate/up input: the primary sends its rotated copy (m.Xr = H (s * x), made
  once for its own MMQ quantize) instead of `mixed` to the peer / helper
  card, over P2P or the mapped host route. One rotation serves both cards,
  bit for bit the same input; `mixed` still feeds the shared expert.
- down input: on the peer / helper card, hadamard_rows on H between
  mmq::swiglu and mmq::quantize, in the compact (per group) and the
  layer-sized buffers, with that card's own copy of the layer's sign
  vector (PeerPrefill::had_sh, made at setup on the peer's device).
- The fused int8 prompt kernels never take a folded layer (fused_nat), and
  bind_stage_helper already refuses layers on the fused kernels, so the
  helper's folded layers are always on MMQ with the separate
  swiglu -> rotate -> quantize passes.
- The trellis types need nothing more there: the peer gathers the blobs
  with mmq::gather_native and runs the same MMQ instances (STRATA_MMQ_TQ).

Not run on a GPU yet. Needs a 2-GPU box: Gyro-S with --peer-device (and
separately a layer split with STRATA_PREFILL_HELP=1), the first token's
logits and a greedy generation against the single-GPU run.

Ported from agentionai/Strata rc2 c64e1ec (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:19 +02:00
Laurent Zuijdwijk 72f21aca93 docs/GYRO.md: clone the rc1 branch
Ported from agentionai/Strata rc2 87fc901 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:04 +02:00
Laurent b4e477c014 APR trellis decode: int8 codebook + dp4a in tq_load / tq_apply
Port of the agention llama.cpp CUDA change (perf/tq-cuda-decode 0005): the 2048-point trellis
codebook as int8 pairs with one global scale (max |point| 3.984 -> 127; relative RMS error 0.85%
of the codebook std). A trellis step's 4 weights are two 2-byte entries = one int, so a decoded
lane (TqLane) is 4 ints instead of 16 floats (5 registers instead of 17) and tq_apply is 4 dp4a
instead of 16 int8->float conversions + 16 FMAs. load/apply stay split: decode-once and per-entry
kernels remain bitwise equal. The dequantizers (prompt path, embedding) keep the fp16 codebook.

native_expert_parity --synthetic tqk6/tqk7 tq2_t/tq2_t and --synthetic-hadamard: 0 failures (RTX
3090, CUDA 12.4); GPU rel vs reference 1.18e-2 -> 2.13e-2 (tqk6/tqk7), 2.20e-2 (tq2_t), within the
3e-2 bound; gpu decode-once vs per-entry: bitwise equal.

Ported from agentionai/Strata rc2 87bad56 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:04 +02:00
Laurent 81aef750d2 gyro_setup.sh: --prefill auto, like setup.py writes for the GSQ-RCO models
The Gyro config used --prefill 512, so the prompt path borrowed only 364 cache slots. Measured on
an RTX 3090 (24 GB, cache auto = 15,756 slots, ctx 131k), same prompts: 4K-token prompt 767 ->
1,325 tok/s, 16K 817 -> 1,870 tok/s, on par with ISTA IQ2_XS under setup.py's config (1,401 /
2,058); decode unchanged within noise.

Ported from agentionai/Strata rc2 a4d70dd (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:04 +02:00
Laurent Zuijdwijk 75d9b260da gyro_setup.sh: export STRATA_GGUF_PY for every step (mtp_pack.py needs it too)
Ported from agentionai/Strata rc2 0d0f4b0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:04 +02:00
Laurent Zuijdwijk 2fce1e1e61 gyro_setup.sh: point the HIP build at ROCm (ROCM_PATH or /opt/rocm)
Ported from agentionai/Strata rc2 73c3106 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk e05565e223 gyro_setup.sh: set up the venv first and build with its cmake/ninja (distro cmake can be < 3.24)
Ported from agentionai/Strata rc2 9a89f63 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 21aa723a2a gyro_setup.sh: skip the download when the model file is already there
Ported from agentionai/Strata rc2 ffad2f0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk ffbb367b8a docs/GYRO.md: clone from the agentionai fork
Ported from agentionai/Strata rc2 0802c90 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 5f48f323da docs/GYRO.md: drop the agent-client note
Ported from agentionai/Strata rc2 e466545 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 0ca3752ea6 Gyro: setup guide and script for the published agentionai Gyro-S/Gyro-M files
docs/GYRO.md: build against the agentionai llama.cpp fork, pack the published GGUF unchanged,
Strata's MTP runtime, server config, RTX 5090 measurements (full and 16/8 GB-sized expert caches)
and known limits. tools/gyro_setup.sh does every step on Linux (CUDA or HIP).

Ported from agentionai/Strata rc2 759675e (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk f4abf43154 APR port: count the prompt path's rotated-input buffer in moe_set_bytes
carve() takes m.Xr (T x 2560 floats, the Hadamard-rotated copy of `mixed` for
the MMQ path) from the MoE set of the shared region, but moe_set_bytes, which
sizes that region (and bytes_needed / the borrowed slots), did not count it.
With an APR pack and MMQ on, the MoE set is the region's largest, so its last
take overflowed and init failed with "device buffers for a chunk of N tokens
do not fit" at every chunk and cache size, borrowed or own buffers alike
(reported on an RTX 5090 with the published Gyro-S). Workaround on an older
build: STRATA_PREFILL_MMQ=0 (the FP16 prompt path takes no Xr).

Ported from agentionai/Strata rc2 09042dd (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 62bb21a2d7 APR port: run the published Gyro files unchanged (Q6_K embedding, Q8_0 PLE table)
- iq_kernels.cu: dq_q6_k (llama.cpp's dequantize_block_q6_K, two halves per
  thread) in dq_dispatch; embed_type_supported and iq_row_bytes take Q6_K
  (embedding only) - Gyro-S's token_embd.
- ngram.cpp: PleTable reads Q8_0 rows (170 B; PLE_ROW_BYTES_MAX raised), mapped
  and direct SSD I/O - Gyro-M's per_layer_token_embd.
- native_expert_parity: --gguf-embd (random token_embd rows via iq_embed_rows
  vs ggml to_float), --gguf-ple (random PLE rows via PleTable, both I/O modes),
  --load-check (the engine's start-up checks for a pack + GGUF, no GPU load).

gfx1151: Gyro-S token_embd (Q6_K) and Gyro-M token_embd (Q8_0): 300 rows,
0 values differ; Gyro-M PLE (Q8_0) and Gyro-S PLE (IQ4_NL): 320 rows, 0
differ, mmap and direct. --load-check on packs of both published files:
0 failures. (Measured before the rebase onto 0.1.39.)

Ported from agentionai/Strata rc2 cc287a6 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:30:56 +02:00
Laurent Zuijdwijk f7f21bc823 APR port: decode-once verify kernels and MMQ prompt path for TQ types
- iq_kernels.cu: the TQ lane dot split into tq_load (16 codebook values) and
  tq_apply; vec_dot_tq_q8_1 = apply(load), Split<144..146> + kSplit, so
  verify windows decode each weight lane once for all entries (bitwise
  equal to the per-entry kernels).
- MMQ: with STRATA_GGML_DIR = the agention fork, the fork's
  mmq-instance-{tq2_t,tqk6,tqk7} are built and moe_mmq takes the types
  (STRATA_MMQ_TQ). prefill.cpp rotates a copy of `mixed` (Xr) before
  mmq::quantize and H between swiglu and quantize; device sign vectors are
  made at prefill setup.
- hip_prefill_mmq_parity: TQ gate/up (1280x2560) and down (2560x640, odd
  blocks per row) passes; MMQ_PARITY_REPEAT; the output sentinel is now set
  with hipMemsetAsync on the test stream (a null-stream hipMemset could land
  after the MMQ kernel and make every output look unwritten).

- Rebased onto upstream 0.1.39: the fused int8 prompt kernels (STRATA_PF_FUSED)
  never take a folded layer. The multi-GPU prompt paths (set_peer,
  set_stage_helper) decline a Hadamard-folded pack here; a later commit
  gives their MMQ share the rotations.

gfx1151 (ROCm 7.2.1, STRATA_PREFILL_MMQ=ON): MMQ parity passed 3 runs x 10
repeats (rel_l2 1.4e-3..3.2e-3); native_expert_parity --synthetic and
--synthetic-hadamard tqk6/tqk7 tq2_t/tq2_t: 0 failures, decode-once bitwise
equal. CUDA 12.8 sm_120 full build compiled (incl. TQ MMQ instances) before the
rebase onto 0.1.39.

Ported from agentionai/Strata rc2 dc19786 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:30:50 +02:00
Laurent Zuijdwijk bda227817e APR port: TQ row bytes on the GPU (iq_row_bytes returned 0 for 144-146)
native_expert_layout takes gu_row / d_row / up_off / down_off from
iq_row_bytes, which had no cases for the trellis types and returned 0:
every gate, up and down row read row 0 of the blob, and up/down read the
gate matrix. The dequantizers (flat superblock indexing) were unaffected,
which is why they matched ggml bit for bit while the expert matmul did not.
native_expert_supported now also refuses a type without row bytes.

gfx1151 (ROCm 7.2.1): native_expert_parity --synthetic and
--synthetic-hadamard tqk6/tqk7 tq2_t/tq2_t: 0 failures (gpu rel = cpu rel,
cpu-gpu 5e-5..1e-3).

Ported from agentionai/Strata rc2 dae1f66 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:29:42 +02:00
Laurent Zuijdwijk fbf045eb6a APR port, stage 4: trellis decode and Hadamard rotation on the GPU paths
- third_party/ggml/ggml-common-tq.h: the fork's TQ2_T/TQK6/TQK7 block
  structs and 2048-point codebook (device table).
- iq_kernels.cu: Fmt<144..146> (the fork's tq.cuh lane decode against q8_1,
  4 q8_1 blocks per 128-block), dq_tq in dq_dispatch (prompt FP16 path,
  embedding), is_iq / row bytes; hadamard_rows / hadamard_rows_f16
  (shared-memory FWHT in the CPU's butterfly order) and
  hadamard_device_signs; native_expert_grouped rotates h when
  NativeExpertLayout::had_block is set.
- verify.cpp (local GPU), remote_experts.cpp (helper GPUs), prefill.cpp
  (FP16 fallback; folded layers never take MMQ) rotate x / h into their
  own buffers; the shared expert and the CPU doorbell copy keep x.
- native_expert_parity --synthetic-hadamard; ctests for tqk6/tqk7 and
  tq2_t when ggml is the agention fork.
- generate.cpp: APR packs run only with STRATA_APR=1 until validated end
  to end on a GPU.
- hip_backend.cmake: gfx1151 as an unvalidated architecture.

Rebased onto upstream 0.1.39: native_expert_grouped takes the separate
SwiGLU / rotate / q8_1 passes when the down weights are folded (the one-pass
swiglu_q8_1 kernel has no h to rotate); the new decode peer GPU
(peer_experts.cpp) rotates its private input copy and passes the down signs,
as remote_experts.cpp does.

Ported from agentionai/Strata rc2 6c7abde (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:29:38 +02:00
Laurent Zuijdwijk f47b0385f3 APR port, stage 3: Hadamard-folded experts on the CPU path
- hadamard.{hpp,cpp}: hadamard.txt parser, process-wide spec, in-place
  signs + normalized FWHT per block.
- expert_layout_load reads native_experts.txt v5 + hadamard.txt and sets
  NativeFmt::{had_block, had_gu, had_d, gu_signs, d_signs}; inconsistent
  packs are refused.
- native_quant_act / native_quant_h rotate before quantizing, which covers
  every CPU consumer (expert_source's doorbell copy, the pool's h).
- generate.cpp refuses Hadamard packs until the GPU paths rotate too.
- native_hadamard_test: Strata's CPU expert vs the fork's ggml graph
  (ggml_mul(signs) + llama_mul_mat_hadamard + ggml_mul_mat): bit-identical
  on real Gyro-S (TQK6/TQK7) and Gyro-M (TQ2_T) experts, also through
  ExpertPool::run_split_multi_native; synthetic Q8_0 reproduces the
  unrotated float expert to 1.2%.

Built with -DSTRATA_GGML_DIR=<agention llama.cpp> (ggml-cpu supplies the
trellis vec_dot kernels).

Ported from agentionai/Strata rc2 1587897 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:28:58 +02:00
Laurent Zuijdwijk 89febede8b APR port, stages 1-2: map, trellis type tables, prism.hadamard packing
- Type tables: PTQ1_0/TQ2_T/TQK6/TQK7 (143-146, 128-value blocks) in the C++
  and Python GGUF readers.
- iq_pack.py: validates prism.hadamard.* and writes hadamard.txt (block,
  sign vectors, folded roles per layer); native_experts.txt becomes v5 for
  such models so an engine that cannot rotate refuses the pack. Anything
  it cannot apply (dense folded weights, inverse tables, gate without up,
  Q2_0 experts, missing sign widths) is refused. 3 new unit tests.
- embd_bf16_pack.py --gguf: a GGUF's token_embd dequantized to BF16, for
  embedding types without a GPU dequantizer (Gyro-S: Q6_K).

Ported from agentionai/Strata rc2 d864b6a (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:28:47 +02:00
Niko1221 31556cd1a0 helpers wait for their turn visibly; batch-mtp policy off; helper cap max(slots,4) 2026-10-15 21:25:37 +02:00
Niko1221 322e3fdfb3 Terminal: a failed runtime start names the engine log; doctor warns when the render group is missing
- runtime start/restart/verify that fail print the run config's engine log path and its last lines (the line was only 'server_exited')
- doctor: on Linux with a SYCL or HIP run config, a /dev/dri/renderD* (and /dev/kfd) the user cannot open is GPU_DEVICE_ACCESS (warning) naming the render group;
  seen on P1006 where the engine only said 'No device of requested type available'
2026-10-15 21:14:37 +02:00
Niko1221 8401c1cdb5 stage-harness: simpler prune of files 2026-10-15 21:14:08 +02:00
Niko1221 486555d4f1 scripts: agent-flow-s4b (helpers, one-card edit, web call, Stop) for a real model 2026-10-15 21:13:23 +02:00
Niko1221 3152013186 agent README: tools, helper agents, /goal, plain words, Manual gate 2026-10-15 21:12:33 +02:00
Niko1221 1e73f53160 host: tool servers (MCP) set up for the model are connected to the agent, secrets only through the environment 2026-10-15 21:09:32 +02:00
Niko1221 b9c6c65482 engine component: a HIP package is not chosen for a card it has no code for
P2006 (Radeon 780M, gfx1103) resolved the gfx1200/gfx1201 package as usable: HIP detection looked at the PCI vendor only.
The Linux kfd topology (gfx_target_version) names each card's gfx target; a package whose archs list none of them is skipped with the reason.
Unknown targets (Windows, no kfd) are not a refusal, as with CUDA.
2026-10-15 21:08:57 +02:00
Niko1221 37cab2c76f tests: /goal over HTTP, plain unavailable notice 2026-10-15 21:07:26 +02:00
Niko1221 14d0ec7ac8 tests: request slots default, sizing, retry with one 2026-10-15 21:05:55 +02:00
Niko1221 5a00c8fb64 host: agent tools, helper agents, one approval card in Manual, /goal, plain agent states, request-slot sizing
- strata-gate plugin: Manual asks before a change (no blocked attempt first); children of a Manual chat reach the same card
- preset.mjs: the Harness standard preset minus scheduler, search without a key, unanswerable question tool, plan mode
- helper agents projected into the existing agent rows; Stop stops the helpers; the run ends when they have reported
- /goal through the Harness goal Remotes; goal rounds stream into the chat
- native-parallel.mjs: request slots by default in the installed app (fit policy, one-line switch), batch-mtp, one-slot retry
- plain wording for agent failures and tool errors, original text kept as details
- stage-harness.mjs: pruned production Harness for the installer (not run yet)
2026-10-15 21:05:15 +02:00
mes b884451697 Layer split auto: STRATA_SPLIT_UNIFORM=1 restores the uniform miss cost
0.1.41 scaled each stage's miss term by its probed link (0391e478);
0.1.44 scales from a 4x gap against the fastest link (9d41e18b, #2007).
On an x16+x4 dual-3090 rig (26.9 vs 6.4 GB/s, 4.2x) either rule moves
auto off the balanced placement (K=27 -> K=19/K=31): decode -4..-8%,
prefill +15..+19% at 10K / -6..-9% at 90K vs 0.1.40.x at matched
calibration. Opt-in restore next to STRATA_SPLIT_LINK_ABS, default off.
2026-10-15 20:33:15 +02:00
Niko1221 1696b5c959 installer: header text, branding line, no details pane 2026-10-15 20:29:43 +02:00
Niko1221 0db5d17102 chat speed is decode speed with reasoning, context shown is the running model's, Models shows Unload while loaded, VRAM reserve floor, default text size 14, installer branding, icon 2026-10-15 20:25:35 +02:00
Niko1221 85c7bfda29 Terminal archive: calibrate refuses in words, models import works, tuner runs a packaged engine and boolean flags
Found qualifying the Terminal archive on real boxes:
- engine calibrate crashed with ModuleNotFoundError (tools.calibrate / setup are not in the archive): refuse with exit 5 naming STRATA_INSTALL_DIR and tune run --mode api; --plan reports tool_available
- models import --plan failed with "fingerprint catalog is unavailable": data/gguf_fingerprints.json is now in the archive
- tune plan/run --engine <package exe> could not load the package's libraries: engine_env puts the package folder first; --engine component:strata-engine accepted
- a boolean candidate (--kv-grow) was passed as "--kv-grow 0" ("unknown argument: 0"), and the protocol harness turned "0" into a bare flag (on): flag_argv
- an unmeasured candidate's reason showed the engine usage dump; the reason line comes first now
2026-10-15 20:13:16 +02:00
Niko1221 42ffb27951 stable window origin (fixed port with fallback), run configs repaired when Control moves, app installs to Programs\Strata and migrates an old install out of the data folder 2026-10-15 20:10:06 +02:00
Niko1221 bfe0798660 install progress step as a sentence; Model files label; ui-flows details check 2026-10-15 19:51:52 +02:00
Niko1221 5df75b1d4d first-run feedback 2: plain words for failures with Details, parts-based download progress, reading size reaches all conversation text, tuning pop-up fits its error, headless UI flows, installer mock-ups
ErrorNotice and toasts map known failures (server_exited, engine missing, network, integrity, Harness missing, Control) to a sentence and a next step with the original text under Details. The install progress sums a list of parts (model files, engine) into one count with a time left. The Read pictures block is hidden when the catalog has no pictures. Control reports the last server log lines of a failed start. Reading size scales headings, code, tables and reasoning. All eight locales.
2026-10-15 19:42:08 +02:00
Niko1221 844f91c606 app: every updater message in plain words, technical reasons in details 2026-10-15 19:23:38 +02:00
Niko1221 36c6ff200e host: the model install job also gets the engine for this PC (engine-step), one combined progress total, retry message on failure 2026-10-15 19:21:21 +02:00
Niko1221 7b2082dbb0 first-run feedback: combined download progress with an ETA, catalog without pictures for app installs, RAM budget for the 20% profile, human engine-start errors, installed model preferred in Models, system theme default, NSIS ignores a remembered temp install folder 2026-10-15 19:10:26 +02:00
Niko1221 4a1ef86b0a layer split auto: STRATA_SPLIT_SPEED_SCALE scales each stage's speed, a test seam for an unequal pair on equal cards (#2644) 2026-10-15 18:50:06 +02:00
Niko1221 8a9d724e6e host: look for the Harness in the user's component folder too, not only when STRATA_COMPONENTS_DIR is set 2026-10-15 17:53:22 +02:00
Niko1221 1b6ed0898d desktop: first-run-ui.mjs, headless Chrome on the packaged host's onboarding with the real catalog 2026-10-15 17:50:11 +02:00
Niko1221 0c37d0e285 update --check: do not print a note twice 2026-10-15 17:49:49 +02:00
Niko1221 2bd0cdc4f9 installer: home() instead of a constant (callers can still move ROOT); close cancels a running install; tests for the app's install mode 2026-10-15 17:46:28 +02:00
Niko1221 31340f7279 docs: STRATA_FILE_CACHE_OLD_RULE (#1752) 2026-10-15 17:45:17 +02:00
Niko1221andKevinX8 1d6fd9f97a file tier: the old room rule on the gfx906 backend (#1752), STRATA_FILE_CACHE_OLD_RULE=1/0 on any Linux build
On an MI50 ROCm queues are suspended while the kernel reclaims around the HMM-tracked arena; the cached
reads of the new rule churn the page cache.  The gfx906 backend keeps the old room >= read_bytes rule;
other HIP cards keep the current rule (the PR applied it to every HIP build).

Co-authored-by: KevinX8 <[email protected]>
2026-10-15 17:45:01 +02:00
Niko1221 d949d600f8 updates: shell bridge tests, host test that a start never reaches the update step, install hint 2026-10-15 17:40:14 +02:00
Niko1221 5a8985620c start path: a signed package is verified with the keys this build trusts; engine-missing notices name Update engine and strata update 2026-10-15 17:38:20 +02:00
Niko1221 10a326a90c packaging: UPDATES.md, feed_tool url prefix and private key file, tests 2026-10-15 17:37:03 +02:00
Niko1221 5322caafe0 Merge branch 'f-split' into f-split-fr 2026-10-15 17:36:37 +02:00
Niko1221 11bf85d8a8 host: the packaged app gets the model catalog, the GPUs and the install job from Control's bundled installer
native-control runs Control's tools/strata_mcp.py on Control's Python (installerEnvironment: STRATA_NO_ACQUIRE stays for the
process; only the install job the user starts may fetch model files), in a writable per-user install folder. The catalog,
hardware (nvidia-smi) and recommendation come from it, the install plan and job go through it, and a failed catalog is an
empty one. Calibration stays off in the packaged app. packaged-control tests adjusted and extended.
2026-10-15 17:36:32 +02:00
Niko1221 ec42c93947 installer: setup.py and strata_mcp.py run under Control (STRATA_CONTROL_INSTALL) for the desktop app
In the app's mode the install downloads the model the user chose, prepares it with the bundled tools, installs no Python
package, fetches no engine and compiles nothing, and writes a run config naming component:strata-engine into a per-user
folder (STRATA_INSTALL_DIR). Only bundled tools run (run() refuses pip, git, cmake and anything else). The engine
component is looked up if present; a model can be downloaded before an engine is installed.
stage-control.py adds setup.py, the preparation tools (static import closure), ref/load.py and data/ to the staged Control.
2026-10-15 17:36:32 +02:00
Niko1221 5778a9ef17 update: Control bundles may carry relative links inside themselves (packer and unpacker); tests 2026-10-15 17:35:22 +02:00
Niko1221 4868f3234d updater: require the one-token probe (force the load of a lazy server), keep the test start's engine log, report what was installed when a step fails 2026-10-15 17:33:47 +02:00
Niko1221 d3c4bba4ca host: Settings update check, Update engine/Strata and rollback through Control's explicit update step; shell bridge client; updated Control is found 2026-10-15 17:28:33 +02:00
Niko1221 8ad2a62730 Merge f-split-life: engine always ends on close (app and Terminal), single app instance, Bypass is full access 2026-10-15 17:28:03 +02:00
Niko1221 e3de5044b2 procs: init adopting an orphan does not make init its starter (found on Linux: the orphan check found nothing) 2026-10-15 17:27:15 +02:00
Niko1221 13cd63567e tests: first-run update test ignores the machine's own installed models 2026-10-15 17:19:40 +02:00
Niko1221 1aa306d7ea update: the last check's result is shown as a one-line notice by components engine (no network) 2026-10-15 17:16:33 +02:00
Niko1221 912d7c1673 terminal: strata update, strata components update and rollback 2026-10-15 17:15:35 +02:00
Niko1221 bc9058a8d4 host UI: the first screen no longer crashes on an empty or failed model catalog
GuidedSetup read catalog.recommendation.family without checking that a catalog has a
recommendation; the packaged host's catalog had none and the app showed 'couldn't show this
screen'. GuidedSetup and NativeModels now tolerate no catalog, no recommendation, no families
and families without sizes. A test renders both components with each of those shapes.
2026-10-15 17:14:01 +02:00
Niko1221 c8698f6257 updates: signed feed, Ed25519 verifier, component updater with resume, staged test-start, switch and rollback 2026-10-15 17:13:41 +02:00
Niko1221 5ec49d7adf test_procs: the orphan's stdio is not the test's pipe 2026-10-15 17:10:14 +02:00
Niko1221 95654fe073 matrix: drain the pty a TUI paints to, give it a size, and tolerate an already-reaped child 2026-10-15 17:09:15 +02:00
Niko1221 b386055d52 prefill_fused_iq_test: the Q2_0 / Q2_0 pair is skipped where the device does not set it up (it is the Turing opt-in's pair only) 2026-10-15 17:08:13 +02:00
Niko1221 921668da66 installed-flow: a killed shell takes the host and its engine with it 2026-10-15 16:49:47 +02:00
Niko1221 7d0b3e7090 tests: a real ssh disconnect ends the server and its engine on a Linux box 2026-10-15 16:44:12 +02:00
Niko1221 b11004671f app matrix: a Terminal-started engine survives the app's close 2026-10-15 16:43:07 +02:00
Niko1221 1d26952f72 serve: the engine's clean-stop deadlines are named, and a test shows an engine that ignores STOP, QUIT and SIGTERM is killed 2026-10-15 16:39:14 +02:00
Niko1221 871a242721 matrix: the TUI is quit with q (typed into its console or pty) and killed; the service it watches is untouched 2026-10-15 16:38:42 +02:00
Niko1221 e09733aa9d tuning: the engine and worker a tuning sweep starts end with the sweep, however it ends 2026-10-15 16:35:49 +02:00
Niko1221 d0e9cb3e37 docs: CLOSING.md, what ends the engine on each OS and how leftovers are found 2026-10-15 16:35:15 +02:00
Niko1221 896c66180e serve: a hangup that was ignored at start (nohup) stays ignored; the matrix proves it and resets inherited signal dispositions 2026-10-15 16:32:10 +02:00
Niko1221 14038ee30c serve: an explicit --pipeline-windows 2 on two stages says when the draft layer is not on the first card (#2637)
The default takes the pipelined decode on two stages only with the draft layer on the first card (mtp_first); asked
for by hand it was taken without a word, and without mtp_first the draft chain shares the last card with its
window (2x R9700: the pipeline lost to the serial loop before 0.1.44). Recommend, never force: it stays on as
asked and a note says what it costs, why mtp_first is not on (no --mtp / --spec 2+ / draft_vocab.bin, or
STRATA_MTP_STAGE0=1 to ask for it) and how to turn it off. No default output changes.

Reported by @dinesh-chander on #2637 (closed by the reporter).
2026-10-15 16:31:46 +02:00
Niko1221 3b848837fa layer split auto: a two-GPU rig whose first pick has a tiny stage tries each card's share of the room before the three-layer rule (#2644)
On RTX 4060 Ti 16 GB + RTX 3060 12 GB (48 layers, IQ3_S) the first pick was K=2 and the #1616 retry
(every stage at least 3 layers) landed on its own edge, K=3: the 3060 ran 3 layers, 12 GB idle, 452 tok/s of
prefill against 802 at 28 layers (the prompt flows through the stages as a pipeline). For two GPUs the retry
now first asks for a placement in which each stage owns at least floor(layers x its room / all rooms) layers
(split_rules::share_floors, at least 3, trimmed to fit); the three-layer rule stays the second try and the
first pick the last, and the log says which one was taken. Nothing changes where the first pick has no
one- or two-layer stage (equal cards), and three or more GPUs keep the #1616 rule alone.

Reported by @Tworks25 on #2644, with the sweep of layer_split 12..32 (PR #2643). split_rules_test gets the
reporter's rooms (14.2 GiB and 5.50 GiB) and the identical-card and degenerate cases.
2026-10-15 16:31:45 +02:00
Niko1221 f31f24c8dc matrix: STRATA_MATRIX_LOGS keeps the server and engine logs of each case 2026-10-15 16:29:47 +02:00
Niko1221 6b83f43dc9 matrix: the fake engine claims the family the OS supports (metal on macOS) 2026-10-15 16:24:34 +02:00
Niko1221 ec73f63f28 matrix: sample GPU memory while the engine runs, so 'freed' is measured against a loaded value 2026-10-15 16:21:05 +02:00
Niko1221 22fea7b767 desktop: close-path matrix for the installed app, self-test proves the shell is in the job, installed-flow checks a killed host takes its engine 2026-10-15 16:20:45 +02:00
Niko1221 e437bc472d matrix: graceful means the clean-stop line, the engine's marks only for the fake 2026-10-15 16:06:50 +02:00
Niko1221 6aa4663bf2 matrix: a real engine's log goes to the rig 2026-10-15 16:06:02 +02:00
Niko1221 651e64ff32 matrix: real-engine mode takes a component folder and waits long enough for a model load 2026-10-15 16:05:32 +02:00
Niko1221 322790aff0 app: one instance, and closing always ends the engines this app started
Tauri single-instance plugin: a second launch focuses the open window and starts nothing. On Windows the shell puts itself in a
kill-on-close Job Object, so killing it, a crash, logoff or shutdown ends the host, Control, the owner host, the server and the
engine. The host follows its parent (STRATA_EXIT_WITH_PARENT) and takes SIGHUP and SIGBREAK down the clean shutdown; every engine
it starts is tied to the host with runtime start --die-with-pid (drain 15 s, then ended); it runs runtime cleanup once at start.
The shell waits 60 s for the host's clean stop (the host's own deadline is 45 s) before ending it.
2026-10-15 16:04:43 +02:00
Niko1221 347d255eaa runtime start: check the engine package before any process exists; a changed file is refused with exit 5 and its name 2026-10-15 15:58:32 +02:00
Niko1221 48e6d64c4f serve, control: a server's engine ends with it however it ends, leftovers are found by PID, start time and command line, runtime stop/cleanup recover a lost owner
lifeline.popen ties every engine, image encoder, MCP server and sd-cli to the server that started them (kill-on-close job on Windows,
PR_SET_PDEATHSIG from one anchor thread on Linux, a pipe watcher elsewhere). Engines and encoders get their own process group so
Ctrl+C, a closing terminal and a console Break reach the server only, which stops the engine the clean way (STOP, QUIT, wait 20 s,
terminate 5 s, kill). SIGHUP, SIGBREAK and the Windows console close/logoff/shutdown events take Ctrl+C's path; the handlers are
installed before the model loads and the browser launch no longer holds the main thread (#1872). The owner guard escalates SIGTERM
to SIGKILL after a grace period. strata_control/procs.py records every launch and finds leftovers; the next engine start, runtime
start, runtime stop and the new runtime cleanup end them after re-checking identity. runtime start --die-with-pid for app-owned
instances. tests/terminal/lifecycle_matrix.py is the close-path matrix.
2026-10-15 15:58:04 +02:00
Niko1221 865b184a0f packaging: what each target's engine package holds 2026-10-15 15:36:59 +02:00
Niko1221 c5388c78cd tests: collect_libs arch filter, ELF check, system/driver split 2026-10-15 15:35:05 +02:00
Niko1221 1bd9746f53 collect_libs: name the libraries a dlopen'd library cannot find 2026-10-15 15:31:11 +02:00
Niko1221 f116778bd5 #1825 qsa_select: the thread-per-head prompt scorer is on for the Turing series (STRATA_SM75_PERF=1) or STRATA_SCORES_THREAD=1, not a default elsewhere
Tesla P100 (sm_60) prompt read, interleaved 5 pairs each, thread vs warp scorer: 30000 tokens 339.29 vs 335.45 tok/s (1.011) and 338.98 vs 330.20 (1.027), 60000 tokens 338.88 vs 330.15 (1.026), 5/5 pairs faster each, below the +3% bar for a new default. Same bits either way. docs line; qsa_scores_parity runs with the switch on.
2026-10-15 15:25:10 +02:00
Niko1221 7e469d797c tests: a schema-2 package made by package_engine.py starts a stub engine, and a flipped library byte refuses the start 2026-10-15 15:25:09 +02:00
Niko1221 b6443f3ae0 moe_fused_iq: #2145's Q2_0 gate/up fused coverage is the Turing opt-in's only (sm_80+ with STRATA_PF_FUSED=1 is bit-identical to base again)
The z440 A4000 Q2_0 gate with STRATA_PF_FUSED=1 on both arms found the residual dump different from base: gu_covered() took Q2_0 on every CUDA card and d.ok on sm_80+ now required the Q2_0 gate/up setup.
2026-10-15 15:21:02 +02:00
Niko1221 83426781d9 Revert #2223 (STRATA_NO_PCIE_GROUP=1): no gain measured on our boxes
Interleaved A/B, 6 pairs, --pcie-frac 0 vs --pcie-frac 0 + the switch, decode ALL: RTX 3060 IQ3_XXS 53.36 vs 53.16 tok/s (0.996, 3/6 pairs faster); RTX A4000 Q2_0 61.72 vs 62.17 (1.007, 4/6). The author's own +2.2% is below our +3% bar and does not reproduce on Ampere. Declined; the PR author's commit stays in the history only as this revert's parent.
2026-10-15 15:20:52 +02:00
Niko1221 dc1bc972ba metal: find the kernel sources, ggml-common.h and default tables beside the executable first, so an engine package can move away from the source tree 2026-10-15 15:18:53 +02:00
Niko1221 21da8442f5 collect_libs: dlopen extras are named by their soname, ELF files only 2026-10-15 15:17:00 +02:00
Niko1221 ac4770f2b1 host: Bypass mode is the Harness's full access with no prompts (danger-full-access + never) 2026-10-15 15:15:03 +02:00
Niko1221 d7895e1690 collect_libs: one copy per real library file 2026-10-15 15:13:43 +02:00
Niko1221 633a87577f collect_libs: --data-arch keeps only the wanted gfx kernels 2026-10-15 15:11:44 +02:00
Niko1221 2d784fa6ff packaging: collect_libs.py gathers an engine's shared libraries (ldd closure, dlopen extras, kernel data) for Linux packages 2026-10-15 15:07:14 +02:00
Niko1221 3e8b5161be engine component: optional variant (cuda13/cuda12 of one version), equal versions prefer the newer-driver build 2026-10-15 15:05:00 +02:00
Niko1221 8101a8903b serve: an engine option written as one entry of the config's args is split, a JSON slip is said with its place (#2267)
"--ple-io mmap" as ONE element of "args" (or "--ple-io=mmap") reached the engine as a single
argument, which it stops on ("unknown argument: --ple-io mmap"); the usual form is two entries.
The server now splits such an element into the flag and its value and says so at startup
(never one in a flag's value place). A config with a JSON mistake (a missing comma between two
args) gave a traceback; it now names the file, line and column. The slow-PLE-probe warning says
how the two entries are written in a config. --ple-io mmap, ram and direct themselves parse
correctly in the engine (checked on the 0.1.45 build).

Reported by lama7 on #2267.
2026-10-15 15:04:33 +02:00
Niko1221 480cfc5f3b engine component manifest schema 2: archs, min_driver, build_id, signature hook; start hashes every file; packager takes nested files 2026-10-15 15:04:15 +02:00
Niko1221 36381540f9 ple: gather_batch reuses prefetched keys that were reordered or duplicated (#2622)
PleTable::gather_batch matched the Direct-mode prefetch slots by position, so rows
prefetched as A..F and gathered as D,E,F,A,B,C (or A gathered as A,A,A) were read a
second time. Each requested token now looks its key up among the prefetch slots and
decodes from the matching slot into its own output position; duplicates share a slot.
All outstanding tickets are drained on a failed collect (the first error is kept)
instead of dropping the rest. The decoded bytes are unchanged.

Reported by @hotschmoe, who also supplied the fixture and the patch; the patch is
used as proposed. ple_reader_test --selftest gets the reorder / duplicate / subset /
missing / over-the-window cases, compared against the mapped reader.
2026-10-15 14:59:12 +02:00
Niko1221 4c968e6199 gdn chunk / Volta recurrence: name the kernels in every device pass (wmma body guarded inside the kernel), not behind __CUDA_ARCH__ at the call sites: the dea9b6de5 guard left the host stubs undefined in a 60;70;86 build (link error) 2026-10-15 14:53:40 +02:00
Niko1221 be885026f9 Merge f-split-s3: Control bundled in the installer, local engine components, no runtime acquisition 2026-10-15 14:52:46 +02:00
Niko1221 dc87d45eb3 app: chat cards drop Demo wording, finished tools say Ran unless approved, and the Mode choice travels with the turn
Approval label is Needs your approval on every card; example-card results lose the Demo prefix. A finished agent tool shows Approved only when the person approved it. Other Demo labels (simulated-model toasts, estimates, preview test, time-skip, context panel, tasks, serving) become truthful Example/Illustrative/Simulated wording in every locale.
2026-10-15 14:49:07 +02:00
Niko1221 2496301c60 host: chat modes (Manual, Plan, Bypass) as Harness permission presets, applied per turn
The page sends the chat's mode with each turn. The bridge switches the session to our preset with the Harness's own /permission command, verifies it through the permissions projection, and fails the turn closed if it cannot. Manual is read-only with ask (a change needs a one-shot approval), Plan is read-only with never (refused, no card), Bypass is workspace-write with never. Tests run the three modes against the real Harness with the scripted worker.
2026-10-15 14:49:07 +02:00
Niko1221 24d28c3f41 packaging: document the Control component, the engine component folder and the no-acquire line 2026-10-15 14:46:06 +02:00
Niko1221 6aa3b99816 tests: the old-engine hint check accepts update.sh as well as UPDATE.bat (found running the suite on Linux) 2026-10-15 14:45:49 +02:00
Niko1221 e1ee39169d sm_60: IQ3 table staging is opt-in (STRATA_IQ_STAGE_TABLES=1), not the default on cc 6.0: no end-to-end gain on a Tesla P100 (#1424) 2026-10-15 14:33:21 +02:00
Niko1221 9546c3af26 fetch-node.sh: mktemp with an X template (GNU mktemp on Linux refused -t); installed-flow.mjs gets a real-engine mode 2026-10-15 14:33:00 +02:00
Niko1221 64d8488c67 serve: a component engine's folder goes first on the library path; with_component_engine helper + test 2026-10-15 14:30:02 +02:00
Niko1221 aecee58bdd host: Control runs without STRATA_API_KEY (an empty one is refused by the server); installed-flow.mjs drives the installed app with a stub engine 2026-10-15 14:28:32 +02:00
Niko1221 4736323d86 desktop: ship the Control component (bundled CPython + locked wheels) in the installer, set STRATA_NO_ACQUIRE and STRATA_CONTROL_DIR for the host, clean host stop through Control, self-test checks for Control and the no-fetch engine notice 2026-10-15 14:22:38 +02:00
Niko1221 2622762f89 host: Strata Control bridge (control-runtime, packaged native-control via owner runtime), acquisition.mjs gates the llama.cpp and cloudflared fetches, /api/control routes, acquire-spy tests 2026-10-15 14:18:13 +02:00
Niko1221 9ddebc1b11 control: run configs can name "component:strata-engine"; the server, doctor and calibration resolve it from the component folder; targets.py keeps the start path off the fetcher; start-through-owner-runtime tests with a fake engine package 2026-10-15 14:13:08 +02:00
Niko1221 dc859b26ff control: resolve the engine from a local component folder (no download path), strata components engine, package_engine.py; the component fetcher refuses under STRATA_NO_ACQUIRE 2026-10-15 14:05:36 +02:00
Niko1221 a7bb051154 docs: agent bridge overview 2026-10-15 13:55:47 +02:00
Niko1221 a51d311fb9 tests: pure agent bridge units (profile, environment scrub, event projection) 2026-10-15 13:55:30 +02:00
Niko1221 cfd45e0c4d host: answer the agent's own questions as unavailable so a turn never waits for a missing answerer 2026-10-15 13:54:08 +02:00
Niko1221 3df612f80a host: stop a leftover Harness from a dead host at the next start (PID record, command-line check) 2026-10-15 13:52:17 +02:00
Niko1221 2425a18705 scripts: agent-flow waits for visible output before Stop and compares tool cards across reload 2026-10-15 13:51:45 +02:00
Niko1221 178bde4ce6 scripts: agent-flow checks per-chat model and two chats at once 2026-10-15 13:49:09 +02:00
Niko1221 537333045a scripts: agent-flow drives the app through approve, Stop and reopen with screenshots 2026-10-15 13:46:03 +02:00
Niko1221 7f81445c02 ui: wire chat to agent frames, real approvals, per-chat Stop and per-chat model choice (no visual change) 2026-10-15 13:42:19 +02:00
Niko1221 a24e78f6f1 host: Stop during agent start cancels the start instead of leaking a turn 2026-10-15 13:42:18 +02:00
Niko1221 fad1ce5568 host: route agent chats through the Harness (stream, approval, stop, status) with a visible fallback 2026-10-15 13:36:58 +02:00
Niko1221 e4da566272 host: internal accessors for the agent bridge (connections with keys, ready instances) 2026-10-15 13:36:58 +02:00
Niko1221 597039b6cb host: agent bridge (sessions, approvals, Stop, resume) with scripted-worker tests 2026-10-15 13:26:08 +02:00
Niko1221 3e373187ea host: project Harness session events into the chat frames 2026-10-15 13:26:08 +02:00
Niko1221 bf63faab2d host: Harness runtime launcher, Gateway client and Strata profile generator 2026-10-15 13:26:08 +02:00
Niko1221 9627ee00ce Merge f-split-desk (Tauri Windows build: node.exe entry path, self-test layout, pinned tauri CLI) 2026-10-15 12:48:43 +02:00
Niko1221 b73ea23b68 a started server and a managed runtime's server are told not to acquire (STRATA_NO_ACQUIRE=1)
setup.start passes it to the server it spawns (subprocess.call env, or execve for Docker); the runtime host sets it in
the owned server's environment. The test checks the start's child environment.
2026-10-15 12:33:40 +02:00
Niko1221 236cca71bb host: no /usr/bin/tar, /usr/bin/env or HOME/.local/state on Windows
- engine-install.mjs extracts with tarBinary(): System32\tar.exe on Windows (bsdtar reads .zip and .tar.*), /usr/bin/tar
  elsewhere.
- util.mjs which() asks System32\where.exe on Windows (first match), /usr/bin/env which elsewhere.
- the tuning and Time-skip state default to %LOCALAPPDATA%\Strata\tuning and \time-skip on Windows
  (apps/host/server/platform-paths.mjs; POSIX keeps $XDG_STATE_HOME/strata). The Python side resolves the same folders
  (strata_control/paths.py, used by the runtime directory and the tune and timeskip commands), so the host and the
  terminal agree.
- tests: apps/host/tests/platform-paths.test.mjs (both platforms from either host) and tests/terminal/test_state_paths.py.
2026-10-15 12:31:49 +02:00
Niko1221 3e5e0c73b2 terminal: an existing install is found through settings.json and used in place
The terminal archive is a new folder with no run configs. models.resolve_install() now looks at the install folders
setup.py remembered in settings.json (installs, next to data_dir) when the archive's own folder has none, read-only,
and reports how it found them (explicit, here, remembered, default). The run configs name their engine and model
files by absolute path, so the old install and its data folder keep serving: nothing is copied, moved or created.

tests/terminal/test_migration.py builds today's layout in a temp dir (strata-<model>.json, engine/, run scripts,
.venv, a separate Strata-data with GGUF, packs and MTP, settings.json) and checks the listing, inspect, a profile and
serve --plan, a plain start and the explicit setup path leave every file, size, mtime and hash as they were.
2026-10-15 12:29:44 +02:00
Niko1221 2e324bbbe8 package_terminal: --wheelhouse builds with no internet; lock_wheels --download fetches exactly the locked wheels
A box with no DNS (p620) can build the Linux archive from a folder of wheels fetched and hash-checked elsewhere:
pip still runs --require-hashes against the lock, with --no-index --find-links.
2026-10-15 12:27:36 +02:00
Niko1221 d5be478937 runtime: the managed owner runtime on Windows (loopback TCP + token, job object, byte-range leases)
strata runtime start/status/stop/verify/restart now work on Windows. The owner transport (loopback-only TCP, 256-bit
token, state file in a private directory) was already there; the supervisor around it was fcntl/getuid/killpg only.

- strata_control/winproc.py: LockFileEx byte-range leases (shared and exclusive, on a byte beyond the data so the file
  stays readable) and process identity (OpenProcess + GetProcessTimes). Liveness never uses os.kill(pid, 0): on
  Windows that is TerminateProcess.
- runtime.py: state directory %LOCALAPPDATA%\Strata\runtime with a protected one-entry ACL (existing ones must be
  unshared); owner-only checks through private_files.is_unshared; the state file's address must be a loopback port;
  the owner host is detached with CREATE_NEW_PROCESS_GROUP|CREATE_NO_WINDOW; the server runs in a kill-on-close job
  object (serve/winjob.py, terminate() added), so the whole tree ends with the host however the host ends; a graceful
  stop closes the server's stdin pipe (--owner-fd 0, serve/owner_guard.py turns the EOF into the Ctrl+C drain path);
  the deadline kill ends the job; the ownership marker records the child's creation time, so a PID that was reused
  by another process is not mistaken for the recorded child. A PID from a state file is only compared, never signalled.
- A connect failure to a lost owner is a typed ProfileError (exit 3) instead of a raw OSError.
- `strata serve` and calibration take the activity lease on Windows too.
- tests/terminal/test_runtime.py runs on Windows (32 tests were skipped): a .cmd stands in for the fake engine, the
  signal tests have Windows variants; new tests: a second client attaches from another process, wrong/stale tokens
  cannot stop the owner, a recorded PID of an unrelated live process is never touched, a reused PID with another
  creation time is not the recorded child, the liveness check does not end its target, the directory ACL, shared and
  exclusive leases.
2026-10-15 12:25:58 +02:00
Niko1221 52d58ba32e desktop: pin the Tauri CLI, fix the packaged Windows start
The shell now builds and tests on Windows. Three fixes found by running the installed NSIS build:
Tauri's resource_dir() is a verbatim path (\?\C:\...) that node.exe cannot take as its entry script, so the host died at once and the window hung; the path is made plain.
The self-test only knew the macOS Resources layout; it now finds resources next to the exe (Windows) and under lib/Strata (Linux).
A host that exits early is reported at once instead of after the 60 s health wait.
@tauri-apps/cli 2.12.3 is a pinned devDependency with a lockfile (npm ci), and the npm scripts call it instead of cargo tauri.
2026-10-15 12:14:33 +02:00
Niko1221 66bc889072 terminal archive: bundled Python for Windows x64 and Linux x86_64, hash-locked wheels, launchers, smoke test
package_terminal.py builds the terminal archive for windows-x86_64 (.zip), linux-x86_64 and darwin-arm64 (.tar.gz)
from the pinned python-build-standalone CPython 3.12.15 (SHA-256 per target, from the release's SHA256SUMS) and a
per-target lock, packaging/locks/runtime-<target>-cp312.txt, installed with pip --require-hashes --no-deps
--only-binary=:all: at BUILD time only (pip is the interpreter's own, no upgrade; the result must equal the lock).
packaging/lock_wheels.py makes a lock. Console-script launchers that name the build path are removed; a VERSION
file and the read-only gguf_reader tool join the payload.

- bin/strata.cmd and bin/strata.ps1 (Windows), bin/strata (POSIX): the bundled runtime first, PYTHONNOUSERSITE=1,
  no PYTHONHOME/PYTHONPATH; a path with spaces works.
- START-HERE.bat, SETUP.bat, UPDATE.bat, setup.sh, update.sh use a python/ folder beside them when present (no
  Python lookup or install).
- strata --version (one line: version and interpreter), strata serve --engine mock, a typed exit-5 message when the
  server's packages are missing (strata.py preflight).
- packaging/smoke_terminal.py extracts an archive in a clean env (space in the path, PATH without Python, dead proxy)
  and runs --version, doctor, profile list, install, serve --engine mock (health, models, a chat answer).
- tests/terminal/test_packaging_pins.py: digests, lock shape and agreement with requirements.txt, launcher order.
2026-10-15 12:06:29 +02:00
Niko1221 2e3085a555 start and serve never acquire: one guard, a typed error, a test that fails on egress
setup.py mixed acquisition (pip, downloads, engine fetch and compile) with configuration and start. A start of an
installed model used to check the engine version against the release and download a newer one, and moved model
files into the data folder.

- strata_control/acquisition.py: forbidden() / require() / AcquisitionRefused (exit 5, code acquisition_refused,
  names SETUP / UPDATE / `strata install`). STRATA_NO_ACQUIRE=1 forbids a whole process.
- setup.py: start() runs inside forbidden(); download, pip_install, run, every urlopen (now _urlopen), the
  engine fetch/compile and CUDA-library installs call require() first. A plain start only says when the engine is
  older than expected (engine_update_note) and no longer touches the network; UPDATE / setup keep
  update_installed_engine. A plain start reads the data folder from settings.json and creates, moves and copies
  nothing (data_folder_in_place); setup, check, calibrate and update still gather files as before.
- strata install [--update] [setup options]: the explicit installer entry; a folder without setup.py says so.
- tests/terminal/test_no_acquisition.py: the start/serve import graph reaches no installer code, no serving module
  runs pip/git/cmake/npm or opens a URL (listed exceptions are request-time or loopback), and a real start with the
  network and every subprocess mocked makes no request and runs no installer.
2026-10-15 11:52:21 +02:00
Niko1221 9587a7a1e0 bench test: kill the lease holder's whole process tree on Windows (a venv python.exe is a launcher) 2026-10-15 11:40:42 +02:00
Niko1221 874b8850e8 host tests: the reliability and tuning-route tests use the hardware seam and a Python stub engine 2026-10-15 11:25:29 +02:00
Niko1221 135430b076 host: STRATA_TEST_HARDWARE stands in for the probes in tests; the two lifecycle tests no longer depend on the memory of the PC running them 2026-10-15 11:24:25 +02:00
Niko1221 4d988cb460 host tests: native-onboarding matches serve/server.py with either path separator 2026-10-15 11:23:23 +02:00
Niko1221 44d8858747 host tests: fileURLToPath replaces the regex that stripped the slash before a drive letter (replaces 439339ab1; the pathname form also left %20 in paths with spaces) 2026-10-15 11:22:56 +02:00
Niko1221 03d9f35ed5 components test: the execute-bit assertion is POSIX-only (NTFS has none); the setuid check stays everywhere 2026-10-15 11:21:28 +02:00
Niko1221 c24c426c06 connections test: nest deeper than any Python's JSON recursion limit (3.13 parses 1100 levels) 2026-10-15 11:21:22 +02:00
Niko1221 341228c718 calibration tests: use setup.settings_path() (APPDATA on Windows) instead of an XDG-shaped path 2026-10-15 11:21:01 +02:00
Niko1221 a0bd7b25ea control: Windows leases lock a byte past the marker so another process can still read the holder (benchmark and calibration worker leases; msvcrt locks are mandatory) 2026-10-15 11:20:43 +02:00
Niko1221 18bb5b793b bench tests: no real GPU in the telemetry fixture (its first reading raced the second run), Windows ACL check for the job receipt, lease marker size on Windows; private_files.is_unshared 2026-10-15 11:19:23 +02:00
Niko1221 b0e3b5add6 tuning: peak resident memory of the engine's process tree on Windows (Toolhelp + GetProcessMemoryInfo; ps does not exist there) 2026-10-15 11:16:30 +02:00
Niko1221 f7e12d23ce tuning tests: a Python stub engine (READY/PP/T/DONE) started as [sys.executable, stub.py] replaces the .sh stubs on every OS
strata_control.tuning.engine_command runs a .py engine through the current interpreter (the --help
inventory and the measurement harness both use it).
2026-10-15 11:16:01 +02:00
Niko1221 62413d835c time-skip: a check whose program does not exist is unknown on cmd.exe too (9009, or exit 1 with no such program on PATH); a timed-out check stops the whole process tree; tests no longer need coreutils 2026-10-15 11:14:20 +02:00
Niko1221 1ee7c7337d host: one Python resolver for every spawn (env, repo venv, py -3, python.exe on Windows; python3 elsewhere)
Tuning, Time-skip, the revise command, the native MCP client, setup and
calibration jobs and the native server all go through server/python-resolver.mjs;
the Microsoft Store alias is skipped because it must start.
2026-10-15 11:13:11 +02:00
Niko1221 01f309efef host tests: the native-control fixture fakes .venv/Scripts/python.exe on Windows 2026-10-15 11:13:11 +02:00
Niko1221 decb3f3f8d host: folder picker and skill-add work on Windows (PowerShell FolderBrowserDialog) and Linux (zenity, kdialog); a typed path replaces the dialog where none exists 2026-10-15 11:11:49 +02:00
Niko1221 8a11d125db desktop: Windows node.exe in the Tauri shell, hidden host console, and fetch-node.ps1 (win-x64, same SHA-256 pin as fetch-node.sh)
node_binary() looked for 'node' only. Not compiled here: no Rust toolchain on this PC.
2026-10-15 11:10:00 +02:00
Niko1221 f549c31d1d host: the API-key file is owner-only on Windows (a protected ACL for the current user), and the test asserts the effective ACL 2026-10-15 11:08:58 +02:00
Niko1221 5f051b83d0 control: owner-only files on NTFS (a one-entry protected ACL via ctypes) and the owner state-file check uses it on Windows
POSIX keeps 0600/0700. The address-stability test is skipped on Windows (the
placeholder address is constant by design; the listener picks the port).
2026-10-15 11:08:26 +02:00
Niko1221 515e045faf packaging: monitor rollback works on Windows (os.rename cannot replace the mkdtemp directory) 2026-10-15 11:05:28 +02:00
Niko1221 d442b0307f packaging: the monitor bundle's LICENSE check accepts CRLF; LICENSE is eol=lf in .gitattributes
A Windows checkout (autocrlf) has CRLF in LICENSE, and the bundle verify
required the file to start with b'MIT License\n'. Normalize line endings
before the check, pin the file to LF, and test a CRLF member.
2026-10-15 11:05:24 +02:00
Niko1221 439339ab14 apps/host tests: Windows file URLs in the esbuild entry points
14 test files took the path of `new URL(rel, import.meta.url).pathname`, which on
Windows is "/C:/..." and cannot be resolved (esbuild "Could not resolve"). Strip the
leading slash before the drive letter. No behavior change on macOS/Linux. Went from 14
whole-file failures to 6 real test failures (listed in the S0 notes). Easy to drop if the
owner would rather have a different fix.

Follow-up to the owner's rebuild branch strata/rebuild-2026-10 (small, test-only).
2026-10-15 10:58:10 +02:00
Niko1221 5389eb9db8 kv_hybrid_parity: the tensor-core prompt kernel is sm_70+ (base launch gate), so a refusal below sm_70 passes 2026-10-15 10:47:40 +02:00
Niko1221 31a3d5553e repo: restore the executable bit on the terminal and desktop scripts
The files were copied through Windows, which does not carry the mode; the modes are
as in the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:10 +02:00
Niko1221 e424cc940d docs/work: Phase A implementation status, qualification kit and evidence text
IMPLEMENTATION_STATUS.md, ACCEPTANCE-EVIDENCE.md, QUALIFICATION-KIT.md and the
evidence logs (text only, about 140 KB). The 170 MB of screenshots under
apps/host/docs/evidence and checkpoints are not in git; they are in
StrataSplit\evidence-archive. The Electron mentions in IMPLEMENTATION_STATUS.md are
history (Electron was replaced by Tauri); the superseded top-level desktop/ folder is
not carried over.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 cef6027dcb packaging: terminal distribution, portable monitor bundle and the acceptance matrix
packaging/package_terminal.py builds the self-contained terminal archive with its own
Python runtime, monitor_bundle.py the portable monitor, acceptance_matrix.py maps the 67
acceptance rows to evidence, README.md documents the target matrix. The built archive
(release/artifacts/*.tar.gz, 42 MB) is not in git.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 48fc04a425 tuning: job contract, history, apply/rollback and the real-API measurement harness
strata_control/tuning.py (the job contract and the HyperCore-Tune decision), tune_engine
(measures through the real API and records memory), tune_history, tune_apply (explicit,
reversible apply and rollback of a recommendation) and tune_api, with tests/tuning
against a stub engine. Review-only: nothing here starts a sweep from the UI, and the
sweep has the known gaps listed in the S0 notes (e.g. --kv-grow 0 / --expert-cache 0
command lines).

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 486120787d timeskip: strata_control/time_skip.py and tests/time_skip
The Time-skip state machine: checkpoint, revise (a bounded Harness turn), check and
decide, with a durable job record, cancellation, and the "inconclusive means ask the
person" rule. The host routes are in apps/host; the CLI entry points are in strata_cli.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 090e45adf1 integrations/harness: the Strata profile and the agent bridge
A Harness profile for Strata, a bridge that drives a Harness agent session against a
Strata server (_bridge.py, bridge_e2e.py, bridge_tests.py), the tool-execution /
approval / cancel end-to-end test (tool_flow_e2e.py) and the bounded revision turn the
Time-skip job uses (revise_once.py). Tested only against a scripted mock worker; the
Harness is not wired into the app's chat yet.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 48ae834b58 apps/desktop: Tauri 2 shell for the Strata desktop app
The Tauri shell (src-tauri Rust, 400-odd lines) that starts the Node App Host and
opens the owner's UI, with the macOS entitlements, icons and generated schemas, and
the build scripts in apps/desktop/scripts that package the app around a private Node
sidecar (digest-verified fetch). The Node binary, Rust target/ and DMGs are not in git;
src-tauri/resources/node/ keeps only its .gitignore placeholder.

Windows and Linux packaging is not working yet (the sidecar lookup expects "node" and the
scripts are bash-only); see the S0 notes.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 0b3299a019 apps/host: the owner's UI and the Node App Host
The UI source from the 14 October handoff (src/, public/, index.html, vite and
tsconfig, tests, audit-fixes, design docs) and the App Host (server/*.mjs): model
import and monitor routes, API-provider catalog and streaming (OpenAI chat, OpenAI Responses,
the Messages API), serving, tunnel, tasks, plus the host routes for Time-skip
(server/timeskip.mjs) and the tuning plan/reports. src/lib/time-skip.ts renders a real
job's own record.

The UI design is unchanged. The rebuild routed #/tuning to a new TunePanel; that is
reverted: src/screens/ModelRun.tsx and src/main.tsx are the handoff's again (#/tuning
goes where the handoff sends it), and src/components/TunePanel.tsx and
src/styles/tuning.css, which only that route used, are not carried over. Left out of
git on purpose: dist/ (build output), node_modules, and docs/evidence + docs/checkpoints
(170 MB of screenshots, kept in StrataSplit\evidence-archive).

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 a242537838 setup: split build helpers (cmake, ninja) out of the runtime requirements
requirements.txt keeps the runtime packages only; requirements-build.txt pins
cmake and ninja. setup.cmake_build installs only the helper that is missing, right
before a local engine or image-encoder build, so the ready-made engine path (and
serving) never runs pip for build tools. ram_gb() reads hw.memsize on macOS and no
longer crashes without /proc/meminfo. docs/INSTALL.md notes the split; two tests in
tools/test_setup_pins.py cover it.

From the owner's rebuild branch strata/rebuild-2026-10. setup.py merged onto 0.1.45
without conflicts (0.1.45's MIN_ENGINE and pins are unchanged).
2026-10-15 10:23:23 +02:00
Niko1221 c968d37a48 serve: owned-server contract for the terminal (ready file, owner guard, launch identity, queue deadline, stage telemetry)
New serve/ready.py, serve/owner_guard.py and serve/launch_identity.py; server.py
gains the queue wait deadline (QueueWaitExpired/QueueCancelled), the opt-in
STRATA_BATCH_TELEMETRY plumbing (BSTAT/PSTAT parsed off the engine's token stream,
dropped from request output), the engine stage-plan reader, a main(argv, ...)
entry the terminal can call, and per-replica selection hooks; telemetry.py reports
GPU names; the tests for all of it.

server.py merged by hand onto 0.1.45: one conflict, in StrataEngine._take_control, where
0.1.45's STRATA_ADMIT_SHORT_FIRST ordering and the rebuild's queue deadline both edit the
acquire step; both are kept (ordering check first, then the deadline-bounded acquire).
Every line 0.1.45 added to server.py (the #2518 pick_slot fix, the havok multi-user
switches, the Qwen3.6 parallel/vision work) is still present.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:23:23 +02:00
Niko1221 79d8ea066a engine: opt-in batch telemetry (STRATA_BATCH_TELEMETRY=1 prints BSTAT/PSTAT lines)
include/strata/core/batch_telemetry.hpp (BatchWindowSampler, StageInFlightSampler),
its unit test, and the hooks in the generate.cpp batch loops and the --batch-groups
pipelined loop. Off unless STRATA_BATCH_TELEMETRY is exactly "1": no new output, no
new state beyond two disabled samplers, so the default output is unchanged.
Host-side C++ only (no CUDA/HIP/SYCL code), so it builds on every backend.

From the owner's rebuild branch strata/rebuild-2026-10. generate.cpp and
CMakeLists.txt merged onto 0.1.45 without conflicts.
2026-10-15 10:23:23 +02:00
Niko1221 64a91dcdc0 terminal: strata_cli, strata_control, bin/strata and the terminal tests
The command-line and TUI client with profiles, doctor, monitor, serve, calibrate,
bench, the runtime owner, component registry and offline policy, plus tests/terminal,
the component/offline tests, the smoke tools, the terminal CI workflows and the
terminal docs. Tuning and Time-skip modules of strata_control follow in their own
commits (strata_cli/cli.py imports them lazily).

From the owner's rebuild branch strata/rebuild-2026-10 (1e4171e2, based on 296b59d2).
README.md and tools/calibrate.py applied as small 3-way patches.
2026-10-15 10:23:23 +02:00
Niko1221 85b75dcbea repo: .gitignore entries for the terminal/desktop split
Keep node_modules, Tauri target/, DMGs, the Node sidecar, release archives, the
built App Host dist/ and the Mac evidence screenshots out of git. The two
.terminal-*venv entries are from the owner's rebuild branch strata/rebuild-2026-10;
the rest are new for the clean integration branch.
2026-10-15 10:19:47 +02:00
Niko1221 2016b7fe01 sycl: the migrated Gemm::bf16 / gr_norm* definitions take #2091's new parameters (the shared headers changed; SYCL never uses the FP16 image)
The SYCL build of the sm_75 series failed with 'out-of-line definition of bf16 does not match any declaration'
(sycl/src/prefill/gemm.dp.cpp); the four gr_* writers would have failed at link. Gemm::bf16_converts_x() answers false.
2026-10-15 09:52:43 +02:00
Niko1221 456ec79c0a docs/NVIDIA_V100.md: STRATA_GDN_REC_WARP (#1098) 2026-10-15 09:49:14 +02:00
Niko1221 4ef3855745 gdn_chunk_parity: skip (77) on a card whose shared-memory opt-in cannot hold the kkt stage (Pascal), instead of failing 2026-10-15 09:48:05 +02:00
Niko1221 dea9b6de5d gdn chunk / Volta recurrence: name the sm_70-only kernels only in the host pass and the sm_70 device pass (a 60;70 or 70;86 build failed in the other device passes) 2026-10-15 08:55:08 +02:00
Niko1221 bcef7b7818 moe_fused_iq: #2145's host_astages() also for HIP (the shared launch() calls it; the HIP build of #2145 as submitted did not compile) 2026-10-15 08:53:39 +02:00
Niko1221andc3c 285e90aee2 qsa_prompt_attn: Q4_0 KV prompt attention on Turing, opt-in STRATA_PROMPT_ATTN_Q4_TC=1 (#1581, the Q4 attention part only)
Ported from c3c's #1581 (the native-IQ and VRAM-loan parts are not in this commit).  Default unchanged: Turing keeps returning false for q4_0.

Co-authored-by: c3c <[email protected]>
2026-10-15 08:53:32 +02:00
Niko1221 7f8e6f12f4 docs: the Turing (sm_75) kernel series and its switches (STRATA_SM75_PERF) 2026-10-15 08:53:32 +02:00
Niko1221 b30d09cb98 moe_fused, moe_fused_iq: the Turing opt-in (#2145) reads the compute capability through STRATA_EMULATE_CC like the other kernels 2026-10-15 08:53:32 +02:00
Niko1221 0639deff7d qsa_prompt_attn: #1821's register prefetch and shared K/V buffer are a template flag, on with STRATA_SM75_PERF=1 or STRATA_PA_V1_PREFETCH=1
The v1 kernel keeps its exact previous code (and 37.9 KB of shared memory) unless the flag is set.  launch_i8 picks the Turing
single-stage layout by the (emulated) compute capability, so the kernel and the launch agree under STRATA_EMULATE_CC.
2026-10-15 08:53:32 +02:00
Niko1221 7f1964a6cb sm_75 series: the bitwise Turing kernels (#2069-#2072 #2081 #2093 #2220-#2222) are opt-in with STRATA_SM75_PERF=1
No sm_75 card is available here to confirm the series, so the kernels that were on by default for compute capability 7.5
in the submitted PRs now follow one switch, include/strata/core/sm75_series.hpp.  Each keeps its own variable, which
wins in both directions.  The one-read prompt top-k (#2093) was on for every CUDA card as submitted; it is now Turing only
like the rest.  STRATA_EMULATE_CC=75 counts as a Turing card, so a 75-virtual build on an sm_86 GPU runs the series for tests.
2026-10-15 08:53:32 +02:00
xrip 6e76ae23fd #1825 qsa_select: four lanes per query in the Turing prompt scorer (4x fewer shared loads)
block_scores_thread_kernel gave a lane one (query, head) and the whole
128-value query head; ncu on a CMP 50HX showed it bound by the MIO pipe
(mio_throttle first): one shared-memory key load per 4 FMAs. Lane 0's
xor-shuffle tree splits into four independent quarters - c[j] sums groups j,
j+16, j+8, j+24, j+4, j+20, j+12, j+28 - which only meet at the end
(c0 + c2, c1 + c3, then their sum). Now four lanes share a query, lane j
holds quarter j of all four heads, a key load feeds four heads (16 FMAs),
and two xor shuffles finish the same tree. The same pairs in the same order,
so the same bits.

qsa_scores_parity on both cards: all 141 M scores identical to
block_scores_kernel; per call 1,024 queries at 140K 7.76 -> 6.17 ms (-20 %),
at 200K 10.78 -> 8.54 ms (-21 %), 2,048 queries at 40K 12.60 -> 11.57 ms.
agentbench 34K -> 190K: AGENT GATE PASS on the rerun (the first run diverged
at turn 2 - decode's known run-to-run variation; the rerun of the same binary
matched all 36 turns), incremental read -0.6 % median, about -1 % from 150K
up, decode unchanged.
2026-10-15 08:53:32 +02:00
xrip 33237ff367 #1825 qsa_select: prompt-path block scores with a thread per query head (Turing)
On cards without the TF32 scorer (sm_75) the prompt path scored every
(query, key block) with a whole warp: 20 shuffles and five 512-byte loads per
score. block_scores_thread_kernel gives each lane one (query, head), keeps the
128-float query head in registers, reads each key once from shared memory as
a broadcast, and adds the 32 four-value partials in the tree lane 0 of the
xor-shuffle sum computes. Float addition is commutative, so the same pairs in
the same tree give the same bits; the heads are summed 0..3 from 0.0f as
before; the tail block keeps block_scores_tail_kernel. STRATA_SCORES_THREAD=0
selects the old kernel.

Measured on 2x CMP 50HX (TU102, sm_75, unlocked to RTX 2080 clocks):
- qsa_scores_parity (new): 141 M scores bitwise identical on both cards
  (normal values, wide magnitudes with zeros, small integers with ties);
  per call 4.6-5.2x faster, e.g. 1,024 queries at 200K: 52.4 -> 10.8 ms,
  at 140K: 35.9 -> 7.8 ms. 199 registers, no spills.
- nsys, one +4.4K-token turn at 140K: block_scores_kernel was the top prompt
  kernel at 1,045 ms per card (17 %).
- agentbench, IQ3_XXS, layer split, 262144 context, 34K -> 190K (with the
  tokenizer cache): engine read per turn at 190K 9.25 -> 6.96 s (476 -> 632
  tok/s, +33 %), time to first token 10.49 -> 7.13 s (-32 %); median
  incremental read 576 -> 682 tok/s; decode unchanged (73.3 tok/s).
  AGENT GATE: all 36 turns word-for-word identical to the base run.
2026-10-15 08:53:32 +02:00
xrip 1d5319ae88 #1821 qsa_prompt_attn: int8 K/V prefetched in registers and sharing one buffer (Turing)
The Turing (v1) prompt attention kernel used 37.9 KB of shared memory per
128-thread block, so one block ran per SM (4 warps, 12.5 % of the warps an
SM can hold), and each 32-cell chunk gathered its scattered K and V rows
after the previous chunk had finished: ncu on a CMP 50HX showed
long_scoreboard (global load latency) as the top stall. For int8 K/V
(--kv int8, mode 1):
- the next chunk's K/V rows and scales are loaded into registers while the
  current chunk computes (Turing has no cp.async), then stored where the
  gather stored them;
- K is read only by the scores and V only by p.v, so they share one buffer:
  V is written over K after the scores' barrier (the softmax's barrier
  orders it before p.v). 37.9 -> 29.2 KB: two blocks per SM.
The same bytes in the same places and the same arithmetic, so the same bits.
Other KV modes keep their code. qsa_prompt_attn_parity now prints a checksum
of the new kernel's output, so two builds compare bitwise.

Measured on a CMP 50HX (sm_75), qsa_prompt_attn_parity, 2,048 queries:
int8 21.45 -> 12.91 ms at 40K (-40 %), 21.94 -> 13.31 ms at 140K (-39 %);
all output checksums (int8 and fp16, 4K/40K/140K) identical to the previous
kernel. agentbench 34K -> 190K (IQ3_XXS, layer split, 2x CMP 50HX): AGENT
GATE PASS (36 turns identical), incremental prompt read time -3.5 % (median
per-turn ratio 0.965), decode unchanged.
2026-10-15 08:53:32 +02:00
Rafa e3b3ba1c66 #2223 verify: STRATA_NO_PCIE_GROUP=1 drops the empty PCIe-share stage
With --pcie-frac 0 the pool never plans PCIe experts, yet every layer of a
verify window still runs the flag B wait, the staging copy and the grouped
launch over zero groups: six kernels, ~18 us a layer on an RTX 2080 Ti
(kernel-node nsys). STRATA_NO_PCIE_GROUP=1 records the window graph without
them; a plan with PCIe experts then stops the engine with a message.

Engine A/B on an RTX 2080 Ti (Flash-Next Q2_0, 512 tokens, two runs each, with the open sm_75
decode PRs applied): Spanish chat 87.8 -> 89.8 t/s, code 106.1 -> 108.5,
32K 78.4 -> 80.1, 128K 75.0 -> 76.8, 390K 74.7 -> 75.1.
2026-10-15 08:53:32 +02:00
Rafa 88a1c23f61 #2222 sm_75: two launches fewer per decode layer (bitwise)
Two kernel pairs of a verify window's layer run as one launch on Turing,
every byte as before:

- the plan's host-flag wait folded into its copy (143-amdk's
  wait_copy_i32_from_mapped, gfx12-only until now). STRATA_FUSE_WAIT on
  CUDA: 2 = the plan only (Turing default), 1 = also the CPU rows'
  copy_rows_from_mapped_after, which makes every row block poll the flag
  over PCIe and was -1.5 % on an RTX 2080 Ti, 0 = neither.
- native_quantize_q8_1_il: a projection input's q8_1 blocks and their
  interleaved copy (native_q8_1_interleave) in one kernel; an int of the
  copy is four codes gathered with two shuffles. Used for the GDN qkv and
  out-projection inputs and the QSA input and out-projection. STRATA_QIL=0 / 1
  forces it off / on (Turing default on).

RTX 2080 Ti, Flash-Next Q2_0, 512 tokens, two runs each: the plan fold
+0.6..+1.4 % decode, the fused quantize another +0.5..+1 % (code 108.0 ->
109.1 t/s). The fused quantize writes the same bytes as quantize +
interleave at 2560/6144/10240 inputs and 2-4 columns.
2026-10-15 08:53:32 +02:00
Rafa bd45500839 #2221 sm_75: Q2_0 matvec with one warp per four rows (bitwise)
The exact layout gives each Q2_0 row four warps, a half block per thread and a
cross-warp reduction. At the engine's 2560 inputs (40 blocks a row) 48 of the
128 threads have no chunk and the reduction dominates: the QSA attn_q
projection (2560x12288) at three columns took 80 us (~110 GB/s) on an
RTX 2080 Ti.

q2_0_warp_rows_kernel: lane l runs the chunks of the exact kernel's threads
l, 32+l, 64+l and 96+l in their loop order, adds the four sums as warp 0 adds
the other warps' partials, then the same warp_sum - every output is bit for
bit native_q2_0_mmvq_kernel's (one column) and launch_multi's exact layout
(two to four). 1/2/3/4 columns: 36.6/60/68/88 -> 24.8/31/31/37 us.

Default on sm_75 only; STRATA_Q2_WARP=0 keeps the old kernels, =1 takes this
one on any CUDA card.
2026-10-15 08:53:32 +02:00
Rafa 996b9f84f1 #2220 sm_75: QSA decode attention v3 for INT8 pools (bitwise)
attn_chunk_kernel<1> saturates the SM's shared-memory pipe on Turing: its
time grew linearly with the queries while the SMs were half empty (two
16-byte q reads per head per cell with a two-way bank conflict, and 12
warp_sums = 60 shuffles per cell).

attn_chunk_q8_v3_kernel computes the same operations in the same order:
  - the lane's slice of the 12 query heads is read into registers once;
  - the 12 lane sums are reduced with reduce12_gfx906 (16 shuffles, every
    sum warp_sum's pairs);
  - the probabilities are stored cell-major, so the value loop reads a
    cell's 12 as three 16-byte broadcasts.

RTX 2080 Ti, with the merge: 2051 selected cells, 1-4 queries
51.6/78.5/110/146 -> 34.5/41.0/73.0/84.9 us; 600 cells
45.9/74.6/105/137 -> 28.6/32.8/33.5/33.3 us. qsa_prompt_attn_parity
reports the same errors with STRATA_ATTN_V3=0 and 1.

Default on sm_75 only; STRATA_ATTN_V3=0 keeps the default kernel, =1 takes
this one on any CUDA card.
2026-10-15 08:53:32 +02:00
Rafa 9512b9c84b #2145 prefill: fused int8 experts on Turing (sm_75), opt-in
The fused prompt-expert kernels (moe_fused.cu for the Strata Q2_0 pack,
moe_fused_iq.cu native_kernel for the native packs) on sm_75, with
STRATA_PF_FUSED=1 STRATA_PF_FUSED_SM75=1:

- cp.async -> plain 16-byte copies (the barriers around them stay),
  prefetch.global.L2 kept;
- mma m16n8k32 / m16n8k16 s8 -> four / two m8n8k16 (exact int32 sums,
  the same d as the sm_80 instructions), MAGIC as the C operand;
- two activation stages on Turing (64 KB of shared memory);
- native_kernel takes Q2_0 gate/up (64-value blocks, a block per stage),
  so the canonical Q2_0 pack (Q2_0 / Q2_0 layers) runs fused; on Turing
  only that pair is set up;
- Turing, Q2_0: the two halves of a stage share the weight scale, so a
  row's dot is d (dx_0 dot_0 + dx_1 dot_1) - 11 % fewer instructions.

RTX 2080 Ti, one layer, 512 experts top 10 (prefill_fused_iq_test,
Q2_0 / Q2_0 pair added): 2048 tokens MMQ 12.98 ms -> fused 6.59 ms,
16384 tokens 55.08 -> 37.16 ms; fused vs MMQ rel RMS 2e-5, the same
error against the FP32 reference.  In the engine (canonical Q2_0 pack,
98K prompt, 16K chunks): prefill 58.3 -> 53.3 s; layer 0 expert rows vs
MMQ: median rel 1.6e-7.  Teacher-forced 2048 tokens at P = 16K / 32K /
64K: perplexity vs MMQ +3.4 % / -4.5 % / +0.6 % (FP16 path vs MMQ
+0.7 % / +0.1 % / +0.3 %) - trajectory noise, no systematic shift; the
verify phase (same tokens) 42.36 s MMQ, 42.28 s fused.
2026-10-15 08:53:32 +02:00
Niko1221 036d473a3c gdn: the Volta chain-split recurrence is opt-in (STRATA_GDN_REC_WARP=2), not the sm_70 default: it changes the summation order 2026-10-15 08:40:13 +02:00
Niko1221 c0d02ba96a V100 fusions (#1097) behind STRATA_FUSE_DECODE=1; pf_wmma opt-in and only when every visible card is cc 7+; WMMA path skips NVFP4 layers; docs 2026-10-15 08:39:46 +02:00
Niko1221 5333327bd5 metal: wrap_split_test: the note's newline 2026-10-15 07:02:42 +02:00
Niko1221 3ef8d7a09f metal: wrap_split_test reports run_icb instead of counting it (a bench path that misreads at some offsets of a single buffer too) 2026-10-15 07:02:42 +02:00
Niko1221 a35ac34944 metal: wrap_split_test also runs with --factor <= 1 (one buffer) as a control 2026-10-15 07:02:42 +02:00
Niko1221 38978a6dfd metal: wrap_split_test - a sparse file longer than maxBufferLength, wrapped in chunks, read at every chunk boundary (#2338) 2026-10-15 07:02:42 +02:00
hokar3361 8b3ec44b1d metal: wrap a model bigger than maxBufferLength as overlapping no-copy buffers (64 GB Macs keep IQ3_XXS resident); MoE kernels bound per layer 2026-10-15 07:02:42 +02:00
Niko1221andBlueKingMuch 2b296987b9 conversation cache: a match that reaches a pin=N prefix is taken instead of borrowed (#1164)
A borrow copies the donor's checkpoints up to the match, so a match that reached a pinned shared prefix gave the borrower the pin too. Such a match now takes the conversation as before. The borrow's guard is guard()/unguard()/guard_blocked(), and the refusal before evicting counts the pinned conversations along with the donor.

Rebased by hand onto 0.1.44.1 (evict_oldest() honours the guarded donor).

Co-authored-by: BlueKingMuch <[email protected]>
2026-10-15 07:02:42 +02:00
Niko1221andBlueKingMuch c9e571e057 serve: a new conversation that starts with a checkpoint of a parked one borrows it, the parked conversation stays (#1164)
Sibling subagents with the same system prompt and tools no longer cost each other their history: only the K/V up to the checkpoint and the checkpoint itself are copied into the session (conversation_snapshot_restore_prefix); the parked conversation stays where it is and is pinned while the outgoing one parks. A match on a parked conversation's live state or newest checkpoint is that conversation going on and is taken as before.

Rebased onto 0.1.44.1 by hand: the pinned shared prefix (#879) and evict_oldest() now also skip the borrowed donor, serials follow every erase, and park-gain (#2265) and the pin= sibling rule gate the outgoing park.

Co-authored-by: BlueKingMuch <[email protected]>
2026-10-15 07:02:42 +02:00
Niko1221 04e3481b99 docs: --conversation-cache-park-gain (#2265) 2026-10-15 07:02:42 +02:00
Niko1221 3d99819e38 setup: the engine builds pin -DSTRATA_MMQ_KQUANTS=ON too (465cd61d flipped it with STRATA_Q6K_EXPERTS, and a stale CMakeCache.txt keeps the old OFF the same way; found by qiuzan2001 on #2515) 2026-10-15 06:22:32 +02:00
Niko1221 dd08a7d0ec serve: pick_slot sends a chat's next turn to the slot that holds it, found by its turn checkpoint (#2518)
A client renders the history again for its next turn, without the reply the slot decoded or the assistant header,
so the slot's held stream (prompt + reply) is no start of the new prompt and pick_slot fell back to the least
recently used slot. With two chats in two slots and two next turns arriving together, whichever was read first
could take the other chat's slot, and the engine's slot copy then overwrote the cache the second turn needed
(0 tokens reused, a full prompt read). pick_slot now also counts the held tokens up to the last <|im_start|>
(where the engine takes its turn checkpoint) when the prompt agrees with them, and a pipelined layer split with one
slot per group (--batch 2 on two cards) no longer ignores what its slots hold.
2026-10-15 05:59:33 +02:00
Niko1221 c2215a2af3 docs/DETAILS.md: the elastic encoder at 196K context 2026-10-15 05:59:13 +02:00
Niko1221 5d83130cdb setup: the HIP engine build pins -DSTRATA_Q6K_EXPERTS=ON too (the same stale CMake cache as #2515 for the CUDA build); the CUDA configure line is tested (tools/test_setup_stale_stamp.py) 2026-10-15 05:59:13 +02:00
xrip a7d4b75443 serve: --prompt-cache-spread, checkpoints kept spread back from the conversation's end (opt-in)
Least recent use keeps the newest checkpoints and every turn adds one, so after a few short turns all 6 sit in the
last 1-2K tokens. An agent client that rewrites its history a few thousand tokens back then resumes from the system
prompt: on 2x CMP 50HX with the pi coding agent a 210,528-token prompt that was the same as the session for 204,349
tokens resumed at 9,525 and read 201,003 again (100 s).

With the flag the victim is the checkpoint whose removal leaves the smallest gap for its distance from the newest, so
the kept points thin out geometrically: after 200 turns of 1,000 tokens they sit 0, 1K, 3K, 8K, 43K and 200K before
the end, and an edit 6K back reads 8K again (least recent use: 200K), 42K back 43K. The same number of checkpoints, no
more RAM; never the root, the newest or a pinned one, the tail rule first, the plain policy without a candidate. The
resume is the usual exact-prefix match.

On the box: an 82,593-token conversation grown turn by turn, a turn ~6 steps back rewritten, resumed at 73,610 tokens
in 10.2 s against 40.0 s read in full; with the expert placement fixed (--adapt-swaps 0, --pcie-frac 0.17) the
greedy answer was the same text as the full read. conv_cache_test: the spread cases (a 200-turn simulation against
least recent use, the fallbacks). docs/DETAILS.md: a paragraph after the prompt cache's options.
2026-10-15 05:42:53 +02:00
Krypto-Whitehat d9cb22d287 Conversation cache: --conversation-cache-park-gain N stops a turn that grew a few hundred tokens from rewriting a whole snapshot (RTX 5090 Laptop, IQ3_XXS: 1,418 parks -> 1, 2,251.9 GB -> 3.26 GB host copy at 4.7 % read back; decode -3.4 %, inside the run-to-run spread)
A conversation that only advanced a few hundred tokens since its last park does not need a fresh 3 GB copy: the entry it would supersede still holds this conversation's prefix, and the next turn drops the copy anyway (drop_superseded). Reported in #2264 over 1,418 park events on one long single conversation.

The flag is opt-in and defaults to 0, which parks every turn exactly as before. What this adds is a way to ask for the gate without a rebuild: on the reporter's rig it is 3.26 GB instead of 9.79 GB per benchmark, the superseded class disappears, and the decode median moves -3.4 %, inside that arm's 92.2-124.1 tok/s spread.

parked_prefix_tokens() reports exactly the entries drop_superseded() would remove - it finds them the same way, by the deepest checkpoint the outgoing chain still holds - so the gate cannot name a different entry than the park would drop. conversation_cache_test covers it (7 checks; 4,226 pass).

Fixes #2264
2026-10-15 05:42:53 +02:00
BlueKingMuch 53f72571fb conversation cache: a conversation parked again keeps at most an eighth of each K/V buffer in reserve (was up to 16 MiB per buffer, 0.4-0.5 GB per conversation)
A conversation that was restored and is parked again keeps its unchanged K/V
pages and appends the new ones (92a3d5f). The appended segment reserved room
for later turns, growing geometrically up to 16 MiB - per buffer, and a parked
image has one per K/V layer and part (k, v, their scales, the indexer rows, the
draft layer). With Qwen3.8-Flash-Next (IQ3_S, int8 K/V) a 32K-token main agent
parked again took 1.49 GB instead of 0.97 GB, a 17K-token subagent 1.16 GB
instead of 0.74 GB. The cache counts that capacity against its budget, so with
2816 MiB the reserve decides which conversations are evicted.

The reserve is now at most an eighth of what the buffer held before the append
(at least 64 KiB, and at most 16 MiB as before). Small turns still fill the
reserve instead of adding a transfer each: the 4096-append test keeps its
segment bound, and 256 turns of 1 MiB onto a 16 MiB buffer take 28 segments
(17 before).

The docs said parking falls back to a full capture when the reserve would evict
another conversation; that check went in 6cd29d9. With the reserve bounded the
two captures differ by at most an eighth of the K/V, and the docs now say what
the reserve is.
2026-10-15 05:42:52 +02:00
qni-live b6dac80740 logit_bias: normalize a long list once per request instead of five times
A standing ban of a whole script (65,929 IDs) was normalized five times per request: in _openai, twice in
validate_request and twice in engine_key. About 200 ms of Python per request, whatever the answer length.

A list is now normalized once. The same object again in a request is answered from the last call; an equal list in
a later request is found by its compact JSON text, which tells False from 0 and "1" from 1 (== does not: [[7, False]]
== [[7, 0]]). Only a list that normalized without an error is stored. Lists under 64 entries skip the cache.
No change to what normalize, engine_key and validate_request return or raise.
2026-10-15 05:41:36 +02:00
f13a870d52 serve: tokenise the prompt from the last shared prefix, not from the start (#1852, #567)
PromptEncoder keeps the last prompts with their ids and resume points (one per
conversation-cache slot) and encodes only the text after the longest shared
special-token boundary; the ids equal a full encode. Rebased onto 0.1.44.1
(prepare() keeps the new `cancel` argument; forget_prompt on a context refusal;
count_tokens reads without storing). SECURITY.md notes the in-memory prompts.

Based on gputier's #1852 (8 commits) and blange48's #567.

Co-authored-by: Guillaume PUTIER <[email protected]>
Co-authored-by: benoit lange <[email protected]>
2026-10-15 05:41:24 +02:00
Niko1221 40a6220f24 docs: DEQUANT_WIDE on sm_86 also measured on the RTX A4000 2026-10-15 05:11:05 +02:00
Niko1221 abce866ebe docs: the CUDA opt-in switches ported for 0.1.45 (#2058 #2059 #2075 #1888 #1889 #1418 #1370 #1367 #1262 #1368 #1107), measured on sm_86; STRATA_DEQUANT_WIDE default on sm_86 2026-10-15 04:10:19 +02:00
Niko1221 5f843a926f dequant: the wide kernels (STRATA_DEQUANT_WIDE) are on by default on sm_86 too
Bit-identical (gate: 10/10 + exact_pp 4K/20K IDENTICAL with STRATA_DEQUANT_WIDE=1 on the RTX 3060 and the RTX A4000), and the time
to the first token of a 150 / 300 / 450-token prompt falls 12-20% on the RTX 3060 (5 of 5 pairs each), the prompt read +2.3%.
STRATA_DEQUANT_WIDE=0 turns it off. Arch-gated to the measured sm_86 and sm_61. Follow-up to #2178 (Techannel), the kernels this switches on;
the same stores are what #1525 (sergqwer) proposes for Q8_0 / IQ4_XS.
2026-10-15 02:38:49 +02:00
p3lusa 2783569494 setup: pin -DSTRATA_Q6K_EXPERTS=ON in the CUDA engine build
A build folder configured while the option's default was OFF keeps
STRATA_Q6K_EXPERTS=OFF in its CMakeCache.txt even after the default
flipped to ON (465cd61d): CMake does not rewrite an option already in
the cache, and source_hash() fingerprints the source files but not the
CMake option values, so setup recompiles the engine with the Q6_K
gate/up expert kernels still disabled. A model whose routed experts are
Q4_K/Q6_K (Unsloth UD-Q4_K_XL) then fails to load with:

  strata-q36: blk.34.experts are Q4_K/Q6_K, which the native expert
  kernels do not take (build with -DSTRATA_Q6K_EXPERTS=ON for Q6_K)

Passing -DSTRATA_Q6K_EXPERTS=ON explicitly overrides the stale cache
entry, so the engine always has the kernels its default build promises.
Same pin in the Dockerfile build for consistency. Not hardware-specific:
it is a CMake cache-staleness issue, independent of GPU/platform.
2026-10-15 02:29:55 +02:00
Niko1221 c8cab03503 docs/DETAILS.md: the image encoder on 16 GB cards, measured (#2406) 2026-10-15 01:13:31 +02:00
Niko1221 df44bcb712 Merge f-145-havok (#2364 multi-user series, opt-in) 2026-10-15 01:11:37 +02:00
Niko1221 45752724d9 setup: Flash-Next images 'yes' gets the CPU encoder on an NVIDIA card under 24 GB, --vision gpu kept with a note (#2406: the GPU encoder cost 3-5% of text speed on 16 GB cards, 35-60% on an RTX 4080 SUPER under Windows as reported); goldens re-recorded for the two 16 GB Unsloth configs 2026-10-15 01:11:37 +02:00
Niko1221 0408f0921c Merge f-145-q36speed (Qwen3.6 speed: decode/prompt kernels, bigger prompt chunk by default) 2026-10-14 22:23:29 +02:00
Niko1221 bb90c106b6 docs/QWEN36.md: speed section (kernels, measured R9700 / RTX 3060 numbers, the prompt chunk default and its KL, what was dropped) 2026-10-14 22:04:58 +02:00
Niko1221 7e8485a48b verify: a Q4_0 KV cache takes the batched verify window by default (STRATA_DF_QB_Q4=0 for the old path): the 6-token graph is 1950 nodes, not 2610, ids identical; + kv_window_parity test (#2403: a gfx1030 fault with q4_0 + KV streaming + MTP, not reproduced; cause unconfirmed) 2026-10-14 21:55:16 +02:00
Niko1221 8e2c70a132 metal: STRATA_QSA_SLOTS launchers decline (no Metal kernel); the shared verify.cpp links again 2026-10-14 21:42:05 +02:00
Niko1221 79e5cd1827 docs/BATCHING.md: more than 8 slots on a layer split, the opt-in admission and decode switches, and the 4x R9700 numbers (PR #2364 port) 2026-10-14 21:40:11 +02:00
Niko1221 f755a06d7c #2234 follow-up: the rows attention runs the default kernel's body from its own kernel (the default kernel's signature, and the pre-sm_75 / HIP function-pointer selections, stay as they were: HIP did not build); per-row calls where the per-row path runs a variant (lane-cell, pre-sm_75, gfx906)
batch_rows_parity also checks the one top-k launch over all rows against one call per row.
2026-10-14 21:38:26 +02:00
Niko1221 a08db82dbd Version 0.1.45: project version, Metal string and the setup minimum engine 2026-10-14 21:38:21 +02:00
Niko1221 55bae973fb qwen35: the prompt delta rule takes two barriers per token instead of three
The output sum of token t is done after token t + 1's first barrier (sk / sq double-buffered), off the recurrence's chain. Same arithmetic: the
3000-token prompt's logits and 8 greedy steps are bit-identical to the base engine at chunk 512 and 2048 (R9700: 913 -> 885 ms, 607 -> 573 ms).
2026-10-14 21:38:07 +02:00
Niko1221 4b348ad4b8 docs/QWEN36.md: the P100 row 2026-10-14 21:38:01 +02:00
Niko1221 34db47a8e1 setup: Qwen3.6 --parallel on small cards follows the measured cost (the slots may take 35% of the experts' room; 12 GB at 8K: 4, 16 GB at 32K: 2), and the text says the gain against one-at-a-time with drafts is small 2026-10-14 21:37:45 +02:00
Niko1221 e79156ae2e Merge f-145-q36 (Qwen3.6 start that cannot fit its buffers says so; vision_cmp --dump-ngl) 2026-10-14 21:36:20 +02:00
InB4DevOps fc27296dee serve: skip tool-body rescans until a closing tag arrives 2026-10-14 12:49:34 +02:00
InB4DevOps b145a4a2d6 serve: avoid exhaustive partial-tag suffix checks 2026-10-14 12:49:26 +02:00
Niko1221 c2947a9230 q36 engine: a hybrid start that cannot fit its buffers says so with the numbers
With many slots on a small card (8 sessions on a 12 GB RTX 3060, UD-Q6_K_XL) the sessions left too little VRAM for the hybrid
mode's staging pair, CPU-output buffer and expert slots, and the start died in a bare cudaMalloc failure. place_experts now adds
up what it is about to allocate and exits 1 with "not enough VRAM left for the hybrid experts' buffers and slots (N MiB needed,
M MiB free ...)" and the way out (fewer slots, a shorter --max-context, fewer --expert-slots). Only a start that would have
crashed changes; the placement and every answer are as before.

tools/qwen35/q36_vram_refusal.py: demanding starts (8 slots, long contexts, every expert as a slot) must reach READY or exit 1
with one of the engine's refusals, never a raw allocation failure.
2026-10-14 12:33:29 +02:00
Niko1221 739bc94881 tests: batch_rows_parity, #2234's one-launch kernels against the per-row launches (scores, decode attention, GDN window and commit), bitwise 2026-10-14 12:26:10 +02:00
bsvinay cbc1578a2f layer split overlap: off beside --pipeline-windows (#859)
--pipeline-windows runs its own overlapped decode loop with its own hand-offs, so
STRATA_SPLIT_OVERLAP now switches itself off when it is set (one log line says so).

Rebased onto main 1735d64: the early launch of the next stage keeps #871's graph
variant (the next stage's refresh_ar() runs before its capture and prelaunch, and
not again in its run() when it was prelaunched), resets the plan error word like
run()'s launch, and the chain path checks that word after its sync as well.
2026-10-14 12:26:10 +02:00
bsvinay 090b8e9f13 layer split overlap: the hand-off publish reads the test word only with STRATA_TEST_HANDOFF_DROP set
Without the hook the window graph passes no word, so the publish kernel does no
extra read of mapped memory per window.
2026-10-14 12:26:10 +02:00
bsvinay 2443c9225c layer split overlap: bound the GPU-side hand-off wait (STRATA_SPLIT_WAIT_MS)
Review on #936 (justxiami): the reading stage's wait_flag_ge spun with no bound,
so a lost publish would leave that stream spinning.

- wait_handoff: the reading stage's graph now waits with a bound (default 30 s,
  STRATA_SPLIT_WAIT_MS, 0 = unbounded, GPU clock as in gpu_stamp). On timeout it
  sets a mapped error word and returns, so the graph completes; the host checks
  the word after the window and fails it with an error that names the setting,
  instead of using a stale hand-off.
- The host's per-layer wait (20 s, #267) still normally ends a stalled window
  first; release_gpu_waits already raises the hand-in flag.
- Test hook STRATA_TEST_HANDOFF_DROP=N: the first stage's N-th window withholds
  its flag (a lost publish), to exercise both paths.

Only active with STRATA_SPLIT_OVERLAP=1; the default path is unchanged.
2026-10-14 12:26:10 +02:00
Niko1221andbsvinay 56146c4c49 layer split: overlap the stages' hand-off (STRATA_SPLIT_OVERLAP=1, opt-in)
Each verify window of a layer split runs its stages strictly in turn: stage 1 finishes, the host syncs its stream,
then stages and launches stage 2's graph. Under HIP the sync and the graph launch both sit on the critical path of
every window.

With STRATA_SPLIT_OVERLAP=1 the hand-off carries a ready word (mapped, one per hand-off buffer):
- the writing stage's graph ends with handoff_publish: the payload with volatile stores (RDNA keeps plain stores to
  mapped memory in L2), a system fence per thread, and the last block raises the flag;
- the reading stage's graph starts with wait_flag_ge(flag, 1);
- Verifier::run captures the next stage's window on this thread, then stages and launches it on a helper thread
  (prelaunch) while this thread serves its own layers; it then serves the next stage's layers directly and syncs
  the earlier stage once the chain is done;
- release_gpu_waits also raises the hand-in flag; an early return after the prelaunch releases the next stage.

Batch windows and --split-device 0 keep the serial order. Off by default.

Measured on 2x RX 7900 XTX (ROCm, Linux), Swift 1.5 IQ3_XXS, 512-token greedy answers, median of 8:
decode 53 -> 70.1 tok/s (code prompt), 45 -> 55.9 tok/s (prose prompt); greedy output byte-identical with the
flag at 0 and 1; quiz 47/51 both ways. Not measured on NVIDIA.

Split-overlap launch merged with 0.1.44 HostGraphLaunch (STRATA_VERIFY_ASYNC_LAUNCH): the launcher object is
kept in an optional so an early-launched window has none.

Co-authored-by: bsvinay <[email protected]>
2026-10-14 12:26:10 +02:00
Niko1221andSuperDistance 5d6bf8d414 batch-groups: decode and give way between the pieces of a prompt read (#2050, opt-in STRATA_PIPED_READ_PIECE), on top of #2326's concurrent read
Squash of SuperDistance's three commits of PR #2050 (decode between the pieces of a read on the pipelined groups, the
piece lines counting the rows the slots decoded meanwhile, the stall watchdog staying on for a read in pieces),
applied after #2326 here: the BYIELD give-way is #2326's give_way lambda, the concurrent block runs first and the
piece path serves a read that STRATA_PIPED_CONCURRENT leaves to it (pictures, configurations where it is off).

Co-authored-by: SuperDistance <[email protected]>
2026-10-14 12:26:10 +02:00
SuperDistance 073d079d27 batch-groups: the concurrent read's threads do not share the host thread's core on Linux
On Linux a thread inherits its creator's CPU mask, and the host thread is pinned to the one core the pool leaves it
(aux_cpus.hpp). The read's thread and the doorbell thread were created from it, so they shared that core with the host
thread that pumps the windows, and every window waited there for the doorbell of each layer: 6.1 rows/s during a read
on 4x RTX 5080, 4 stages (blange48 on #2326). Windows places the host with CPU Sets, which new threads do not inherit,
so it did not show there. Both threads now move off the host's core and the pool workers' CPUs (on an SMT machine with
the default pool: the idle sibling threads; without such CPUs, every CPU but the host's), and the threads that already
run on the host's CPU (the prompt path's copy threads, started by the first prompt) move the same way at a read.
--aux-cpus places them all when it is on. STRATA_PIPED_CONCURRENT_UNPIN=0 keeps the inherited CPUs (an A/B).

Measured on Linux (WSL2, Ubuntu 24.04, 2x RTX 5090, 4 streams, a cold 32K prompt; same binary, UNPIN=0 / default): the
slots decoded 25.9 and 23.7 rows/s during the read with the threads kept on the host's CPU (76-81 threads there), and
75.6 and 62.8 with this change; with the n-gram table in RAM (WSL's disk reads it 7x slower than Windows) 110.7 rows/s,
as on Windows (107.0). piped_read_join_bench.py (cold32k, followup, two_joiners; table in RAM, 2-3 runs each): 24-37
rows/s during the 30K+ reads kept on the host's CPU, 95-127 with this change, and 925-1,065 stream tokens during the
cold 32K join against 454-484. On Windows, STRATA_TEST_CONC_ON_HOST_CPU=1 (a test) puts both threads on the host
thread's CPU set first: 18.6 rows/s kept there, 105.4 with this change.
2026-10-14 11:58:16 +02:00
SuperDistance 0d47295f5a batch-groups: --remote-expert-opt without a helper cache does not keep the concurrent read off
The flag works only through a helper cache (--expert-cache-device1..3), and on a plain layer split it is inert
(MULTI_GPU.md, #1447), so the check looks at the helper caches themselves. #2275's rig runs a plain split with the
flag set, and the mode would have stayed off there for a cache it does not have.
2026-10-14 11:58:16 +02:00
SuperDistance 6c5c382b94 docs/BATCHING.md: reading beside the decoding slots, and what it measured
The opt-in's section: how the read and the windows share the GPUs, the doorbell thread, the give-way at a chunk, where
the mode is off. Measured against main (a0116b27) on 2x RTX 5090 in one window (main, this, this, main): the stops,
the streams' rate during the reads with the same answers every run, the joining prompts' TTFT, a question during a
read, the same randomised load on both, the doorbell thread's first wait as an A/B; 16 GB cards on each side; a weaker
host as well in #1906's shape; the default path's check; text equality.

tools/piped_read_join_bench.py, the client that measured it, is #2050's, unchanged (the same file as there).
2026-10-14 11:58:16 +02:00
SuperDistance 34b726e090 batch-groups: the concurrent read allocates nothing on its threads, keeps its checks and the watchdog, and waits on a launch
From a review of the concurrent read:
- With external serving batch_poll still finds a window that will not finish (its graph ended or failed with a layer
  to ring, or no layer rang for 20 s): the served count is read before the stream query and the ring count after it,
  at most one query a millisecond; the last ring's time is an atomic. batch_serve reads a window's row count after its
  served count, so a window launched while the doorbell thread was descheduled is not served with the last one's rows.
- #1996's pinned upload rings (0.1.43.1) grew a slot at its first larger upload, a pinned free and allocation on the
  read's thread: the drafter's records and the prompt path's routing tables reserve their slots before the read
  (PinnedRing::reserve, Prefill::reserve_uploads).
- Off, said once, where serving a doorbell makes a CUDA call (--pcie-mode dma, STRATA_FS_SLOTS, the helper caches) or
  the expert source recycles the buffers it hands out (the GGUF in place, unbuffered, STRATA_IO_PREFETCH); a prompt
  with pictures is read the usual way. The expert sources' read counters are atomics.
  The check is a pure function, concurrent_read_off (batch_read.hpp), with a test in batch_read_test.
- pump's all-idle branch turned the stall watchdog off when the last slot finished mid-read; it stays on while a prompt
  is read until the request ends.
- Between windows the doorbell thread slept 50 us, a whole timer tick on Windows (up to 15.6 ms): it now waits on a
  condition each launch notifies (STRATA_TEST_BELL_SLEEP_US=N, a test, restores the sleep for an A/B). With nothing
  decoding the main thread waits for the read's end or a line instead of sleeping. A pool failure the doorbell thread
  set is reported before pump clears the flag for the next window.
- The read's log line ends with the slots' rate and, per stage, its windows and the host time that set each up. The
  priority stream's cudaGetLastError runs only under the opt-in.
2026-10-14 11:58:15 +02:00
SuperDistance bc90e1acff batch-groups: read a prompt beside the decoding slots on a thread of its own (opt-in, experimental STRATA_PIPED_CONCURRENT)
With a layer split and --batch-groups > 1 a prompt read is one sp.run, so every decoding slot stops for the whole read
(#1906); #2050's pieces bound the stop but drain the pipeline at each piece. With STRATA_PIPED_CONCURRENT=1 the read
runs on a worker thread while the main thread keeps the groups going through the pipeline (pump) and takes BSTOP and
BYIELD meanwhile. The prompt's K/V goes to the main sessions and the slots' to their own; the windows' streams get the
device's highest priority on CUDA (STRATA_PIPED_CONCURRENT_PRIO=0 keeps the default).

- The slots' doorbells are served on a thread of their own that makes no CUDA call (Verifier::batch_serve, now
  batch_poll's per-layer service too, and set_external_serve). Two hangs found while building this, read from the
  stall dumps against the PDB, had #31's shape: the main thread waited for the driver's lock in cudaSetDevice while its
  holder waited for the GPU (a cudaFree in the drafter's prompt pass; cuBLASLt's lazy load of a GEMM module), and a
  window on that GPU waited for its doorbell.
- MtpDrafter::reserve_prefill captures the drafter's prompt graphs and sizes its records before the read.
- Prefill::stop_at_chunk: a BYIELD ends the run at its next chunk, also with no slot decoding; the pipeline is drained
  there and the part read gives way as #656 does. A read nobody waits for is not cut.
- The reader is joined on every way out (an exception too). Off beside STRATA_PREFILL_HELP and STRATA_SPLIT_MTP_BATCH,
  which allocate during a read.
2026-10-14 11:58:15 +02:00
SuperDistance a2de744653 PleTable: a lock around its reader state, so two threads can use one table
The prompt path gathers a chunk's PLE rows on its own thread and the batch windows gather theirs on the main thread,
from the same table (ss.ple.table). Today they never overlap - a prompt run's read-ahead ends with the run, and the
windows run between runs - but a prompt read beside running windows would race on the reader's tickets and the
prefetch slots. Every public method that touches them now holds a recursive mutex (gather holds it across its issue
and collect). Uncontended it is a few nanoseconds per call; nothing else changes.
2026-10-14 11:58:14 +02:00
Niko1221andruibeikaa 125025a770 prefill: --prefill-pipe, chunks sized for a layer split's pipeline (#1441, opt-in; b/a from setup's --calibrate)
Squash of ruibeikaa's six commits of PR #1441 (the first fixed b/a, then runtime-learned b/a, then calibrated b/a, the
segment planned once, --prefill-pipe 1 following STRATA_PREFILL_EQUAL), applied onto 0.1.44.1 with the conflicts in
generate.cpp (Options, BSlot, the lend's chunk with --prefill-head-alias) resolved.

Co-authored-by: ruibeikaa <[email protected]>
2026-10-14 11:52:32 +02:00
Niko1221 81ead832d6 docs: STRATA_CPU_IQ_ALL_EXACT1 (the exact one-token expert rows of #1415 and #1544), what was measured 2026-10-14 11:50:52 +02:00
Niko1221 d30f5d1042 docs (#1749): the wide top-k kernel on HIP, what it does and the switches that restore the old ones (R9700: 22.0 -> 0.93 ms per 256 queries at 524K; a 500K-token prompt +34%) 2026-10-14 11:46:11 +02:00
Niko1221 6612542e17 tools/qwen35: q36_lgcmp.py (two logits dumps: KL, top-1) and the multilingual draft-head prompts 2026-10-14 11:43:29 +02:00
Niko1221 b0f675be76 qwen35: the chunk rule counts what the chunk really costs (measured on the R9700)
A bigger chunk is only taken when the experts, the 0.77 GiB reserve and the chunk's buffers (0.5 MiB per token) still fit without giving the packed
decode copies back: 131072 cells -> 512, 100000 -> 2048, 32768 with --batch 4 -> 1024, --batch 8 -> 512 (hybrid as before).
2026-10-14 11:40:05 +02:00
Niko1221 93f0f48f10 qwen35: the prompt chunk defaults to the largest that fits (4096 .. 1024); the router and the shared gate in one product for decode batches; conv tile fix for batch rows
Prompt chunk (owner decision): with every expert in VRAM the auto rule picks 4096, 2048 or 1024 (the chunk's buffers cost ~0.5 MiB per token,
the sessions' caches 45 KB per cell and session; 1024 at most with --batch N so the other slots' windows are not held for a whole chunk), else 512.
The hybrid mode keeps 2048 and falls back to 1024 / 512 on a card with little room. STRATA_Q36_CHUNK=512 or --prefill 512 gives the old size.
R9700 pp4096: 512 3390, 1024 4495, 2048 5353, 4096 5750 tok/s. Not bit-identical to 512 (other GEMM tiles): see docs/QWEN36.md for the KL.
Decode batches of up to 8 rows take the router and the shared expert's gate in one product (rows of 257), as a single token already did.
k_conv_seq's tile mode switched off the history write for batch rows too (grid y is the group count there): fixed.
2026-10-14 11:24:15 +02:00
Niko1221 2bc0a7cafd gguf_reader: the placeholder note ends in a newline 2026-10-14 11:16:52 +02:00
Niko1221 94d189fca4 gguf_reader: an all-zero attention.compress_ratios list is a converter's placeholder and is treated as absent, with a one-line note (#2396: orcarouter Qwen3.8-Flash-Next-Uncensored GGUFs refused since 0.1.44); real lists are still checked in full 2026-10-14 11:16:26 +02:00
Niko1221 6f4697f662 qwen35: the prompt's gated delta rule loads the next token's inputs while it computes this one
k_gdn_seq_cols started every token with a chain of global loads; they are issued one token ahead now. Same arithmetic: the 3000-token
prompt's logits and the next 8 greedy steps are bit-identical to the previous engine at chunk 512 and 2048 (R9700: prompt 931 -> 902 ms
at chunk 512 together with the tiled conv, 616 -> 591 ms at 2048).
2026-10-14 11:05:46 +02:00
Niko1221 c9e7f6dd5a qwen35: faster decode kernels with the old kernels' bits (STRATA_Q36_VERIFY=1 compares them on real activations)
q36_kernels.cuh: one warp per row for the shared expert's and attention k/v Q8_0 products (grid capped at 1024 warps: 2048 rows as 2048
warps took 21 us, 5 us as 1024), shared gate+up+SwiGLU+q8_1 in one launch, k_moe_out with 1024 threads and the next layer's q8_1 image, k_add_rms
with the q8_1 image, router top-k and expert grouping in one launch. CUDA: the division / exp the --use_fast_math TUs they mirror use are spelled out.
Engine: prompt GDN conv in tiles of 16 tokens side by side (same arithmetic per token), --draft-vocab N (the MTP draft head over the first N rows only,
a draft proposes and the verify window decides: output unchanged), the packed Q8_0 default on Windows too (it was Linux only), STRATA_Q36_OLD_KERNELS=1
switches all of it off.
R9700, UD-Q4_K_XL: decode 82.5 -> 120.6 tok/s (greedy 10 prompts identical), --mtp 2 160 -> 174.
2026-10-14 10:52:17 +02:00
Niko1221 fb51ae6fbd iq_kernels: the V2K gate/up kernels take Qwen3.6-35B-A3B's expert geometry (n_embd 2048, n_ff 512, Q4_K/Q5_K gate+up, Q5_K down)
For that geometry only: the s27 gate/up kernel and the fused SwiGLU/q8_1, then the old down kernel (the new down kernel is slower at n_ff 512).
Flash-Next's shapes (2560/640) take the same paths as before; the path is still STRATA_EXPERT_V2K (default on gfx12).
2026-10-14 10:50:29 +02:00
Niko1221 c298fa559e qsa scorer (#1943): the exact four-lanes-per-query scorer is the default on gfx12; the test runs there too
The same kernel measured on an R9700 (gfx1201): qsa_scores_gfx1100_parity --full compares 223,664,777 scores and 11,895,800 ids bitwise equal
to the warp scorer (35 cases), the scorer stage with 256 queries is 3.6x faster at 131K and 4.0x at 240K, and a fresh prompt reads 8.7% faster at
131K tokens (4 of 4 pairs) and 16.4% faster at 230K (3 of 3), with identical answer tokens and byte-identical exact_pp residual dumps at 70K and 131K
tokens. So on gfx12 it is on for batches of >= 64 queries with >= 16,385 live blocks (65K cells and more) unless STRATA_SCORES_THREAD=0; gfx1100
stays the author's opt-in (STRATA_SCORES_THREAD=1). A one-time stderr line says when it engages by default.
2026-10-14 10:48:36 +02:00
Niko1221 7d3fd4b24f tests (#2147): test_setup_rocm_pins skips an index with no saved version list (the gfx1150 opt-in has none yet) 2026-10-14 10:48:36 +02:00
Niko1221 a61436c4da refill parts (#2078): not in gfx1151's default table (only gfx1151 measured it, and nothing here can gate that arch): opt-in STRATA_REFILL_PARTS=1, documented in section 5 2026-10-14 10:48:36 +02:00
Niko1221 b317c23f24 pcie (#2224): the PcieAuto share on HIP outside gfx12 is opt-in (STRATA_PCIE_FRAC_DEFAULT=auto), not a gfx1100 default: the author's gate was no-regression only and a second gfx1100 report saw it never engage 2026-10-14 10:48:36 +02:00
Niko1221 595686f6a1 hc persist (#2226): not offered on gfx11 and gfx12
The persistent read's barrier timeout uses __builtin_amdgcn_s_memrealtime, which gfx11 and gfx12 do not have (clang: "needs target feature
s-memrealtime"): the PR as ported failed to build for gfx1100, gfx1151 and gfx1200/gfx1201. On those parts the barrier compiles to an immediate
abort, fused_gr_persist_init refuses the read with a start-up line, and the default read is used. An R9700 (gfx1201) measured the read +0.7% / +1.0%
per window slower than the default in the PR thread, so nothing is lost. The silent, permanent barrier abort at run time that the same thread
describes is noted in docs/AMD_HIP.md.
2026-10-14 10:48:35 +02:00
Niko1221 44dbf66af5 Merge f-145-nv (#2347: NVFP4 on AVX2 CPUs: the pack-folder hint, setup note, docs) 2026-10-14 10:02:45 +02:00
Vop3p b2b5000952 hc persist results: ten interleaved pairs on 0.1.43.1 with no other switch (-6.6 % / -4.4 % ms per verify window at 4K / 32K, 10 of 10 pairs) 2026-10-14 09:59:02 +02:00
Vop3p cea3d61b66 tests: hc_persist_parity - the STRATA_HC_PERSIST=1 read against the default multi read at T = 1..8 (bounded 1e-4, finite, two runs bitwise), and hc_persist_default_off: with the variable unset it never runs; a skip where the card cannot hold the 80 blocks and on CUDA 2026-10-14 09:59:02 +02:00
Vop3p 6cbc15a81a STRATA_HC_PERSIST=1 (opt-in, HIP): the verifier's hyper-connection read as one persistent kernel
The verifier reads the hyper-connection twice per layer at 1-8 tokens: three small launches with two global
dependencies, the down projection on 41 blocks. Opt-in STRATA_HC_PERSIST=1 runs it as one kernel of 80 blocks with two
grid barriers on an atomic counter: the down projection split by column slices over all 80 blocks, a distributed
reduction, then the up projection and the same epilogue as gr_up_kernel.

gfx1030 (2x RX 6900 XT, layer split): the read 1.35-2.10x at 1-8 tokens; ms per verify window -4.2 % at 4K and
-2.8 % at 32K (one binary, the variable unset / set, 6 alternating rounds, every round the same way). Another summation order (not bitwise); against a
double-accumulation reference (STRATA_HC_PERSIST=3) as close as the default read.

Residency is decided by running every token count once at start-up with a 50 ms barrier timeout and comparing with
the default read; a timeout keeps the default read for that token count and larger ones, wave64 cards are refused.
One line per card says what was decided. Only the verifier's reads use it (one stream per card); the drafter's reads,
the prompt path, CUDA and SYCL are unchanged (CUDA and SYCL get two empty stubs). With the variable unset both entry
points return in their first line: the output is bitwise the same as main.

Measurements, the simulated smaller cards and a stand-alone timing tool:
bench/results/2026-10-12-hc-persist-gfx1030/.
2026-10-14 09:59:02 +02:00
2jztricks e5d91bf239 hip: add opt-in exact gfx1100 QSA prefill scorer 2026-10-14 09:59:02 +02:00
sa 82922f55e2 topk: compile the wide kernel on HIP too (it was CUDA-only; every deep context on AMD paid the ref radix)
block_topk_wide_kernel is plain shared-memory / atomicAdd code with no sm_90
feature, but its definition and the dispatch branch sat behind
#if !defined(__HIPCC__), so on AMD a context past the register kernel's reach
(about 135K cells) always fell back to the 4-pass ref radix.  The ids are
identical by construction (the same radix rule), which the bench states:
256/256 at 256 queries x 65537 blocks, 1/1 at nq=1.

Measured on a Ryzen AI Max+ 395 (gfx1151, ROCm 10.1), qsa_select_bench:
- nq=1, 131073 blocks (a decode window at 512K context): 2.24 -> 0.77 ms (2.9x)
- nq=256, 65537 blocks (a prompt batch at 262K):        3.16 -> 1.43 ms (2.2x)

End to end this moved the deep-context decode of a 512K-token prompt from
19-22 to 24-26 tok/s and the 512K prefill from 473 to 634 tok/s on that PC
(together with the split scorer of another change).  STRATA_TOPK_STREAM=0
still restores the register kernel.
2026-10-14 09:59:02 +02:00
Niko1221andZac Chen b0e88eb12f pcie: the in-window share refinement (PcieAuto) on HIP too — gfx1100 port of the XeStrata stabilization cluster's finding
Upstream aa7b0cb gates PcieAuto to CUDA single-GPU (#if !defined(STRATA_USE_HIP)); the
machinery itself is engine-generic (pool ms per missed expert + miss counts vs the probed
link rate), and this build's start-up H2D probe is already best-of-4 with a +-0.7% spread,
so the HIP exclusion was unmeasured-only, the same class as the ROUTE_TAIL_SKIP gate.
Same opt-outs as CUDA: layer splits excluded, STRATA_PCIE_FRAC_DEFAULT=old restores the
fixed start-up rule. Provenance: MistVVK/XeStrata commits 05309096/055cf9b0/c71cac34/
5952e8e7 (their measurement-stabilization cluster) demonstrated the failure mode and the
fix class; their exact subsystem is not portable (it is their own engine code).

(cherry picked from commit acb3deb0522a8fc351a4047d2bd11b8ab7a73dca)

Co-authored-by: Zac Chen <[email protected]>
2026-10-14 09:59:02 +02:00
Niko1221andonionbutties 01e1bc5c0e setup: write the gfx1150 speed switches for the opt-in install
The engine turns 18 exact (bit-identical) switches on by default on gfx1151
(src/core/arch_defaults.cpp) but keeps gfx1150 out of its arch defaults
(src/core/arch_defaults_test.cpp asserts arch_default_env("gfx1150").empty()).
A gfx1150 install gets the 17 that apply here - the gfx1151 set less
STRATA_HCD_EXACT, which copies the gfx1151 hipBLASLt table's solutions 1176 / 1177
and so does not fit beside a gfx1150 table - written into the config's "env"
beside STRATA_HIPBLASLT_TUNING.  The Radeon 890M report (docs/AMD_HIP.md;
bench/results/2026-10-07-community-gfx1150) measured the same greedy answers with
all 17 on and with all 17 off, decode +2% and prompts +3-6%.

The set is setup's own (SETUP_ENV): a re-run refreshes it, and it does not linger
after the opt-in is turned off.  A new test (tools/test_setup_strix_halo.py,
Gfx1150Switches) keeps GFX1150_SWITCHES equal to the gfx1151 arch-default table
minus STRATA_HCD_EXACT, keeps the engine's own default path for gfx1150 empty, and
checks the SETUP_ENV wiring.  Setup's suites pass (strix_halo 40, amd 36, choices 47).

Co-authored-by: onionbutties <[email protected]>
2026-10-14 09:59:02 +02:00
Niko1221andonionbutties 6f0505c69e setup: opt-in gfx1150 (Radeon 890M / 880M, Strix Point) as a unified-memory AMD card
STRATA_EXPERIMENTAL_GFX1150=1 takes the Radeon 890M / 880M iGPU (Ryzen AI 300,
Strix Point, gfx1150) as a unified-memory card like Strix Halo: it joins AMD_ARCHS
and the UMA handling (new is_uma_apu helper), with notes of its own.  The engine
already builds and runs for gfx1150 (0.1.40.2, #1217; tools/hip/gfx1150-hipblaslt-100401.txt
is calibrated on it); no release ships a ready-made Windows engine for it yet, so
a Windows user builds one (STRATA_HIP_ARCHS=gfx1150 tools\hip\build_windows.bat) and
starts setup with --prebuilt dist\.  Unset, the default path is unchanged: the test
that asserts gfx1150 is not in AMD_ARCHS (tools/test_setup_strix_halo.py) still passes.

Co-authored-by: onionbutties <[email protected]>
2026-10-14 09:59:02 +02:00
Niko1221andAlexisCarbonell d3c4a6de77 setup: write the gfx1103 exact speed switches for the opt-in install
The engine turns 18 exact (bit-identical) switches on by default on gfx1151 (src/core/arch_defaults.cpp) but keeps
gfx1103 out of its arch defaults (src/core/arch_defaults_test.cpp asserts arch_default_env("gfx1103").empty()).  A
gfx1103 install gets the 17 that apply here - the gfx1151 set less STRATA_HCD_EXACT, which copies the gfx1151
hipBLASLt table's solutions 1176 / 1177 and so does not fit beside a gfx1103 table - written into the config's
"env" beside STRATA_HIPBLASLT_TUNING.  One 780M (Ryzen 7 8745HS, 60 GB, GTT 50 GiB, ROCm 10.1, Q2_0, 4K prompt)
measured the same greedy token ids with the 17 on and with them off, +13% to +15% prompt and decode +3.2% (the
decode half alone); bench/results/2026-10-12-community-gfx1103.

The set is setup's own (SETUP_ENV): a re-run refreshes it, and it does not linger after the opt-in is turned off.
The gfx1103 note now names the native dot4 / WMMA kernels and docs/RADEON_780M.md, which documents the opt-in
build, the table and the measured numbers.  Setup's suites pass (strix_halo, rocm_pins, amd, choices: 143 tests).
Refs #1423.

Co-authored-by: AlexisCarbonell <[email protected]>
2026-10-14 09:59:02 +02:00
onionbutties 7aa71b08e5 hip: add a gfx1150 hipBLASLt 100500 tuning table (Radeon 890M)
Calibrated on a Radeon 890M (Strix Point, gfx1150) with
tools/hip/tune_hipblaslt --workspace-mib 32 over the 16 dense GEMM geometries of
the shipped gfx1150-hipblaslt-100401.txt at T = 64, 128, ..., 16384: two passes,
each row keeping the solution with the lowest mean of all six repetitions.  The
three rows slower than plain hipBLAS (the tiny bf16 N=1 K=2560 shape at T = 64,
128 and 256) are left out, so those buckets route to the nearest good row.  Every
remaining row beats plain hipBLAS: 1.07x to 16.7x per GEMM, geometric mean 5.0x.
The solution IDs are scoped to hipBLASLt 1.5.0 (version 100500, the ROCm
10.2.0a20260930 nightly the engine here was built with).
2026-10-14 09:59:02 +02:00
AlexisCarbonell 0fb72e8b0e hip: a gfx1103 hipBLASLt tuning table (Radeon 780M)
The Radeon 780M (gfx1103, RDNA3, 12 CUs) builds and runs (issue #1423), but had no hipBLASLt table, so the prompt's
dense GEMMs ran on plain hipBLAS.  This adds tools/hip/gfx1103-hipblaslt-100401.txt, calibrated on one 780M with
ROCm 10.1's hipBLASLt 1.4.1 (version number 100401) over the gfx1150 table's 144 GEMM cases at T = 64..16384 with
the engine's 32 MiB workspace.

Two tune_hipblaslt passes back to back; each row keeps the solution with the lowest median of all six repetitions,
and a row is left out when no candidate beats plain hipBLAS.  143 rows, 1.05-23.30x per GEMM, geometric mean 4.95x;
the one shape left out is bf16 N=1 K=2560 T=64 (best 0.039 ms against 0.024 ms on plain hipBLAS, sub-100 us
timings), where the engine falls back to hipBLASEx.

Measured on the 780M (Ryzen 7 8745HS, 60 GB, GTT 50 GiB, Q2_0 + MTP, 32K context): a 4K prompt read 90 tok/s on
plain hipBLAS against 187 with the table (medians of an interleaved A/B; with the gfx1103 exact switches and the
WMMA opt-ins, 277 at 4K and 272 at 32K).  hip_prefill_hipblaslt_gemm passes with it (launches=4 fallbacks=0) and
hip_prefill_hcd_exact_parity skips, because the table picks no 1176 / 1177 for the HC down GEMM - the gfx1150 table
gets the same skip, which is why STRATA_HCD_EXACT is not in the gfx1103 switch set.  Setup installs for gfx1103 only
with STRATA_EXPERIMENTAL_GFX1103=1 and then picks the table up when the installed hipBLASLt reports 1.4.1.

Docs, the community report (runs.csv, tune-summary.csv, the server config, the engine's start-up lines) and the
setup tables list carry the numbers.  Refs #1423.
2026-10-14 09:59:02 +02:00
Niko1221andlibratechw 1b1bd4b4a8 prefill: STRATA_GDN_CHUNKED=1 on gfx11, the prompt's DeltaNet recurrence in 16-token chunks on the matrix cores (opt-in)
The switch exists for CUDA (#1372) and is compiled out on HIP.  On gfx11 it now runs the WY form of the gated delta rule in
chunks of 16 tokens, every product one 16x16x16 WMMA (FP16 operands, FP32 accumulation, the state in the accumulators), with the
output norm and the z gate fused: a prep kernel for T and P of every chunk, then a scan with a block per value head.
It rounds differently from the FP32 recurrence, so it is opt-in; the default path is unchanged.  A gfx1100, gfx1101, gfx1102, gfx1150 or gfx1151 part, 64 tokens and
more, an output row stride divisible by 8 and no STRATA_GDN_REC_HEADS take it; anything else runs the existing kernels.

tests/hip/gdn_chunked.cpp: bounds on the FP16 output and the carried state against the existing recurrence over 0-16,385 tokens
(chunk and stride edges, a split request), a double-precision CPU check, and bit-identical output with the switch off or unset.
tests/hip/gdn_rec_head.cpp unsets the switch, as its variants 0 and 1 are the references there.  hip_gdn_chunked --bench and
--bench-default time this path and gfx1151's default kernels, one per process (the switch is read once).

gfx1151 (Ryzen AI Max+ 395, UD-IQ4_XS): a layer's recurrence + norm takes 1.60x less time than the default kernels at 16K and
32K tokens; a whole prompt 1.4-2.1% less at 4K-128K after a warm-up prompt (11 pairs each: 1.8-2.1% with the docs/STRIX_HALO.md section-5 switches,
1.4-1.5% without them, 10 or 11 of 11 faster); first-token KL against the default 0.006-0.007 on average, the same top token in
12 of 12 prompts.  Run on gfx1151 only; kernels.cu compiles for gfx1030, 1100, 1151, 1152 and 1201 (docs/AMD_HIP.md).

Co-authored-by: libratechw <[email protected]>
2026-10-14 09:59:02 +02:00
libratechw feeb8dc808 docs: STRATA_PF_F16=1 on gfx1151 (what it covers, the layer times, first-token KL) 2026-10-14 09:58:51 +02:00
libratechw ed160b461a tests: hip_prefill_fused_iq also runs with STRATA_PF_F16=1 STRATA_PF_F16_KQ=1 2026-10-14 09:58:51 +02:00
libratechw 0c2608ae97 prefill: STRATA_PF_F16=1 takes Q8_0 down without STRATA_PF_FUSED_KQ on gfx11
Q8_0 has no minimum term and the F16 kernel's output already differs from the int8 kernels', so the KQ opt-in is
not needed for it.  UD-IQ4_XS has five Q8_0-down layers (4 IQ3_S / Q8_0, 1 IQ4_XS / Q8_0): they left the fused path
for MMQ, and since fused_ring() asks every layer, the pack kept MMQ's buffers.  d_covered_on() is the one test, for
native_supported() and experts_native(); gfx12 keeps d_covered().

UD-IQ4_XS, gfx1151, 3 interleaved rounds, prompt tok/s (v0.1.42 / 65d809c6 with STRATA_PF_F16=1 / this):
4K 926.6 / 992.2 / 1054.8, 32K 1257.4 / 1364.6 / 1397.1, 128K 1202.0 / 1309.4 / 1333.6.
2026-10-14 09:58:51 +02:00
libratechw bd5cc1f27b prefill: the other native formats on gfx11's F16 matrix cores (STRATA_PF_F16=1)
native_f16_kernel takes the IQ gate/up formats (IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_XS) and the Q2_0, IQ4_NL
and Q8_0 down: load_unit / convert decode each sub-block once into an int8 weight tile in LDS with its scales as
half2, each wave turns its own rows into F16 (gufo's Q8_0 decode: sign flip, byte permute into 1024's mantissa,
one packed multiply), and the F16 WMMA accumulates all of K.  The work item, the activations staged as F16, the WMMA
loop and both epilogues move to moe_f16_common.cuh, shared with the K-quant kernels (moe_f16_kq.cu, now on it; its
output bits are unchanged).  The switch becomes STRATA_PF_F16=1 (was STRATA_PF_F16_KQ) and covers both; the K-quants
still need STRATA_PF_FUSED_KQ=1.  gfx12 keeps the int8 kernels.

hip_prefill_fused_iq (gfx1151, ROCm 7.14.1 clang 23, top 10 over 512 experts, one 128-expert launch, ms):
- IQ3_S / IQ4_NL at 16384 tokens: gate/up 11.65 -> 9.87, down 6.11 -> 4.71; layer at 8192 37.3 -> 32.2.
- every pair is faster at 4096 and 16384 (IQ2_S gate/up 14.38 -> 9.85, Q8_0 down 6.80 -> 4.79), and at 128-2048
  tokens equal or faster.
- FP32 reference rel RMS as the int8 kernels' (IQ3_S / IQ4_NL 1.456e-2 against 1.454e-2); against the int8 model
  and MMQ the F16 path sits at 2.4-2.7e-3, so both checks allow 5e-3 under STRATA_PF_F16=1 (f16_on).
- With the weights decoded to F16 in LDS (18 KB a block) gate/up was 9.83 and down 5.71.  Neither a VGPR cap nor
  dropping the sign table (4 blocks per WGP) nor decoding after the products moved gate/up; without the decode it is
  8.98, without the weight loads 9.30.
Off by default (the output differs from the int8 kernels').
2026-10-14 09:58:51 +02:00
libratechw 929010be61 prefill: gfx12 keeps the int8 kernels under STRATA_PF_F16_KQ=1
moe_f16_kq.cu's bodies are gfx11's WMMA layout (and trap on other targets); v0.1.42's gfx12 native kernels share
experts_native's launch, so the F16 path is taken only off gfx12.
2026-10-14 09:58:51 +02:00
libratechw b2492918f5 prefill: Q5_1 down on the gfx11 F16 matrix cores too (STRATA_PF_F16_KQ=1)
down_kernel takes gate/up's structure for UD-Q4_K_XL's Q5_1 down (47 of its layers): the codes and high-bit words
packed in LDS, w = q d + m per wave in F16, H staged once as F16 from its int8 blocks, FP32 outputs at the routed
rows.  The activation staging and the WMMA loop are shared with gate/up (ActStage, mma_tiles).  Q8_0 down stays on
native_w11_kernel.

hip_prefill_fused_iq (gfx1151, one layer, 16384 tokens x top 10 over 512 experts): down 7.10 -> 4.83 ms per
128-expert launch; Q4_K / Q5_1 layer 62.7 -> 53.5 ms (85.3 on the int8 kernels); FP32 reference rel RMS 1.304e-2.
2026-10-14 09:58:50 +02:00
libratechw df218bd8ae prefill: K-quant gate/up on the gfx11 F16 matrix cores (opt-in STRATA_PF_F16_KQ=1)
UD-Q4_K_XL's routed gate/up was 6/10 of the prompt-time gap to gufo on gfx1151 (128K: 23.1 s against 13.9 s).
moe_f16_kq.cu adapts gufo's paired RoutedF16GEMMKernel (97b8645f, MIT) to Strata's tables, int8 activations and H:
the Q4_K / Q5_K codes stay packed in LDS and each wave turns its own rows into F16 after the read, so the F16 WMMA
accumulates all of K in F32 with no per-block epilogue; the activations are staged once in LDS as F16 for every wave;
the epilogue writes H as int8 per 32 features for the existing down kernels.  One block per work item; two tiles of
an expert per item from 64 routed rows per expert on average.

hip_prefill_fused_iq (gfx1151, ROCm 7.14.1 clang 23, one layer, 16384 tokens x top 10 over 512 experts):
- gate/up kernel 14.63 -> 8.46 ms per 128-expert launch; Q4_K / Q5_1 layer 85.3 -> 62.7 ms, Q5_K / Q5_1 89.9 -> 66.2.
- against the FP32 reference the error is the int8 path's (rel RMS 1.305e-2 / 1.302e-2: the int8 activations
  dominate); the int8 model check allows 5e-3 under this switch (2.5e-3 measured with Q8_0 down).
- the output bits are the same with 64- and 128-row items.
Off by default; it needs STRATA_PF_FUSED=1 and STRATA_PF_FUSED_KQ=1, whose output already differs from MMQ's.
2026-10-14 09:58:50 +02:00
libratechw d573332130 prefill: refill a prompt's lent expert slots from the GGUF mapping's pieces (STRATA_REFILL_PARTS=1, gfx1151's table)
After a prompt the borrowed expert-cache slots are refilled from the experts' files.  With the GGUF read in place an
expert is three tensors, and the refill first assembled each 4 MB blob in a staging buffer (a copy on the calling thread)
and then copied that to the slot.  The pieces are contiguous in the mapping, so they can go straight to their offsets in
the slot: blob_parts() hands them out, ExpertCache::fill_slot_queued_parts copies them on the same queue.  The bytes in
the slot are the same.  Off by default; gfx1151's table turns it on.
2026-10-14 09:58:50 +02:00
Niko1221 37f2941fc2 docs/NVFP4.md: the CPU rows on a CPU without AVX-512 (AVX2 rows bit-equal to ggml's, measured on three CPUs) (#2347) 2026-10-14 09:45:18 +02:00
Niko1221 5e9bd5db4f docs/MAC.md: the M1 Max line quotes the 9,940-token prompt speed of the #2348 report, not a 24-token cold request 2026-10-14 08:05:07 +02:00
Niko1221 b1f7b4ac3a pytest: a session config home, so no test can write or migrate the user's real Strata settings (#198 data-folder migration) 2026-10-14 08:04:52 +02:00
Niko1221 3de6363749 Merge f-145-mac (the #2365 test no longer runs data_folder() for real) 2026-10-14 07:51:50 +02:00
Niko1221 98f5484c8d gr_upmix_parity: skipped (exit 77) below sm_80, where the fused read refuses by design (#1370 follow-up) 2026-10-14 07:46:45 +02:00
Niko1221andAndy Slater b558df67b4 cpu: the exact singleton rows only where ggml-cpu is itself the AVX2 build (not AVX-512, not an ISA-floor build)
STRATA_CPU_IQ_ALL_EXACT1 reproduces ggml-cpu's AVX2 single-token dot bit for bit, which only
holds if the ggml-cpu in this binary is that dot: a STRATA_ISA_FLOOR build runs ggml's AVX1/SSE
dots, and a native build on an AVX-512 CPU run with STRATA_FORCE_AVX2=1 runs its AVX-512 one.
The guard is #2215's (ggml_cpu_has_avx2 / ggml_cpu_has_avx512).

Co-authored-by: Andy Slater <[email protected]>
2026-10-14 07:20:05 +02:00
Niko1221 6eb34c7036 setup: --parallel is not offered where the card holds every expert for one session only (4 slots at 128K on a 32 GB card moved all but 15 experts per layer to the CPU: 81 tok/s against ~180) 2026-10-14 07:17:06 +02:00
Niko1221 093108c2e0 setup: the engine's own programs run with its rocm/bin on PATH (#2372: strata --families on Windows AMD showed hipblas.dll / libhipblaslt.dll not found and Qwen3.6 was refused) 2026-10-14 07:16:36 +02:00
Niko1221 8bfef49c5f q36_vision_cmp: --dump-ngl, llama.cpp's offloaded layers for a quant larger than the card 2026-10-14 07:14:54 +02:00
Niko1221 a6bdc3fb5b pf_wmma: compile the WMMA kernel bodies only for sm_70+ device passes (a multi-arch 60;70 build failed in the sm_60 pass: no nvcuda::wmma there) 2026-10-14 07:14:39 +02:00
Niko1221andtuandat3019 10552345e9 AMD HIP: the primary ordinal, and the helper caches beside the resident RAM mode
Three changes for two-card AMD PCs whose faster card does not come first in the
runtime's enumeration (Windows HIP orders the cards itself; HIP_VISIBLE_DEVICES
only filters):

- gfx1032 (RX 6600) joins the unvalidated build list - it shares the gfx1030 /
  gfx1031 / gfx1034 dp4a path.
- STRATA_PRIMARY_DEVICE=N picks the visible ordinal the engine's own session
  runs on; it is set before any context or allocation exists.  The helper
  caches take the other visible devices, the HIP arch check and the peer tier's
  device queries follow it, and the primary's session is created on the chosen
  device.  A layer split still assumes device 0 and is refused with this set.
  Unset or 0 is upstream's behavior unchanged.
- The helper-GPU caches are allowed beside the resident RAM mode: the experts a
  helper holds are left out of the RAM copy as a split stage's are, and the two
  adaptive tiers already keep each other's experts out.  RemoteExperts::finish
  spins on its stream like PeerExperts::finish - on HIP the CUDA branch's
  cudaInitDevice(ScheduleSpin) does not exist.

Measured on an RX 6800 (16 GB, gfx1030) + RX 6600 (8 GB, gfx1032), Windows 11,
32 GB of RAM, Swift 1.5 IQ3_XXS at 160K context, the resident RAM mode with the
helper's cache left out of the copy: decode 37.5 -> 44.2 tok/s on 8K-token
requests with a 3800-slot helper; the CPU pool's share of the routed experts
falls from ~2.9 to ~0.4 entries per layer-window.

Rebased onto 296b59d2 (conflicts in generate.cpp, peer_experts.cpp, hip_backend.cmake, docs/AMD_HIP.md resolved by keeping main's code and adding the PR's).  Changes: remote_caches is kept (still used by the --batch-mtp check); the spin in RemoteExperts::finish is HIP only (CUDA keeps its cudaInitDevice(ScheduleSpin) + blocking sync).  NOT COMPILED YET, not run.

Co-authored-by: tuandat3019 <[email protected]>
2026-10-14 07:14:17 +02:00
Niko1221 e8b20f2a6e setup: NVFP4 on a CPU without AVX-512 is offered, with a note on what its CPU rows are (#2347)
The menu never hid NVFP4 on an AVX2 CPU; the engine runs a native pack's rows on AVX2 kernels that are bit-equal to
ggml's. The NVFP4 menu line and the install step now say, on a CPU with AVX2 and no AVX-512, that the CPU's share of
the experts runs on AVX2 rows (measured about as fast when memory-bound, up to ~12% slower on 4-token windows).
A note only: nothing is refused or capped. tools/test_setup_menu.py: 2 tests.
2026-10-14 07:14:11 +02:00
Vop3p eb6e0f1e4d calibrate: measure the shared-expert stream fork on AMD (STRATA_SH_STREAM, off against on) and keep it per PC
The fork stays off by default on HIP (#816, #826): an RX 6800 on Windows and 4x R9700 decode 12-45% faster without it.
The same card family on another PC goes the other way (#1272: 2x RX 6900 XT on Linux), and the output is identical
either way, so it is a per-PC choice - the open item of #1272.

- tools/calibrate.py: step 6, HIP only. With everything chosen before, the engine is started twice, with
  STRATA_SH_STREAM=0 and =1 set explicitly, two rates each; "1" is kept only when it beats "0" by more than MIN_GAIN
  (the rule of the other steps). measure() returns the choice as "env"; a failed start drops that candidate (#1337).
  start_engine takes an optional env; apply_env() sets the chosen variables and removes the others of ENV_KEYS, so an
  older calibration's choice does not linger.
- setup.py: --calibrate writes "env" into the config and the settings file; a reinstall applies a saved calibration's
  "env" the same way (a calibration saved before this step leaves "env" as it is).
- docs/AMD_HIP.md: the SH_STREAM note says calibrate measures it.
- tools/test_calibrate.py: the step runs on HIP only, keeps the fork only on a win, falls back on a failed start;
  apply_env; setup writes and later removes the variable.

Measured with the real engine (0.1.42 tag, 2x RX 6900 XT gfx1030, Linux, IQ3_S layer split, setup-style config): the
whole calibration took 982 s, the new step chose STRATA_SH_STREAM=1 (off 62.2 / 61.3, on 67.2 / 66.6 tok/s, +8.3%).
Separately, with a fixed protocol, the fork's output is byte-identical to off (4 of 4 requests, one card and the split).
2026-10-14 07:12:58 +02:00
Niko1221andxjc10 4f21be8723 hip: the FP32 tiled QSA block scorer on HIP cards (opt-in STRATA_SELECT_SIMT=1, as on CUDA), tiled 32/128/16/8 on gfx103x
The scorer of 65ce329 is plain FP32 arithmetic, so it runs on any HIP card as it is; without matrix cores the
prompt path otherwise scores on the warp kernel.  Its tiling becomes template parameters: every tiling keeps each
score's fmaf chain over k = 0..127 in order and the same relu sum over the heads, so every tiling gives the same
bytes (RX 6900 XT, qsa_select_bench at 32K and 128K: 14 tilings, also on 60 and 72 of its 80 CUs through
HSA_CU_MASK, all byte-identical to the original kernel; the fastest tiling the same at every width).  CUDA keeps
the tiling it was written with; gfx103x runs 32/128/16/8: 256 queries at 128K 2.69 ms (warp kernel) -> 0.695 ms
(the original tiling) -> 0.587 ms.  FP64 gate passed, selections identical 256/256.  On the 2x RX 6900 XT layer
split a 128K-token prompt reads 7.4% faster, 32K 0.8%.  A HIP ctest runs it at 131K.

Rebased onto 296b59d2: the docs/AMD_HIP.md conflict (main's SGEMM/rocBLAS-table bullets from the ports before it) is resolved by keeping all bullets.

Co-authored-by: xjc10 <[email protected]>
2026-10-14 07:12:53 +02:00
xjc10 4b3e4d096f rocBLAS table docs: the shipped gfx1030 table was measured on an RX 6900 XT; other gfx1030 cards can make their own with tune_rocblas (reports welcome) 2026-10-14 07:12:35 +02:00
Niko1221andxjc10 57c95d5faf hip: a rocBLAS solution table for the FP16 prompt GEMMs on gfx103x (the GDN projection 7x below 1,152 tokens; short prompts -10 to -14%)
rocBLAS picks the kernel for each FP16 product from its own table by shape, and on gfx1030 that pick is far
from the best kernel the same library holds for some shapes: N = 10240 K = 2560 (the GDN input projection)
below T = 1152 runs at 5.2 TFLOPS where the best of rocBLAS's 237 solutions for n = 799 is 6x faster (not an
alignment effect: n = 800 is as slow as 799).

tools/hip/tune_rocblas enumerates the library's solutions per shape and token bucket, checks each against the
default kernel's output and that it also runs at smaller odd token counts, and writes the winners (and
`default` rows bounding their reach) as tools/hip/<arch>-rocblas-<version>.txt; Gemm::f16_inplace runs the
table's solution for a shape (STRATA_ROCBLAS_TUNING: a file, or the directory from which the engine takes the
file of its own rocBLAS build; the version string, tweak hash included, is checked). One tiny default product
at startup opens rocBLAS's kernel library (a solution-index call before that fails although the index is
valid); a refused solution falls back to the default kernel for that call, after three refusals for good.
setup points the engine at tools/hip when a table for the card's architecture exists. Shipped: the gfx1030
table for ROCm 10.0.0's rocBLAS 5.6.0.8d1ae90e (112 rows, 76 with a solution).

Measured (RX 6900 XT, 6f32ec0 + #835 + this, temperature 0, the same answers with and without): a 799-token
prompt's GPU time 3,329 -> 2,985 ms (-10.3%), 495 tokens 2,477 -> 2,139 (-13.6%), 8,874 tokens 11,978 ->
11,459 (-4.3%); a PCIe-bound 1,108-token prompt unchanged (its saving reappeared as `wait copy`). ctest 66 of
70 (the 4 failures are outside this change); hip_prefill_rocblas_gemm, the new smoke test, passes with 0
fallbacks. Builds on #835. bench/results/2026-10-05-rdna2-rocblas-table has the per-run lines and the tuner's
summary.

(cherry picked from commit e35e103604)

Rebased onto 296b59d2: conflicts in setup.py (SETUP_ENV gained STRATA_ROCBLAS_TUNING beside the 0.1.44 entries) and docs/AMD_HIP.md (kept both sections) resolved.

Co-authored-by: xjc10 <[email protected]>
2026-10-14 07:12:35 +02:00
Benjamin Larsson 2f44bf51dd STRATA_HC_REQ8: hyper-connection projections requantized to int8 + fp32 scale per 32 at load
The rebased #704 under the name asked for there (upstream's STRATA_HC_Q8 now
reads the GGUF's Q8_0 bytes, a different thing).  Opt-in.  The four BF16 hc_*
projections per layer (1,200 MiB) are skipped by the loader and uploaded as
int8 codes with one fp32 scale per 32 values (675 MiB); the freed VRAM goes to
the expert cache.

- Q8 is a template parameter of the down/up kernels (single- and multi-token):
  the BF16 instantiations compile exactly as before, the launch sites pick the
  Q8 ones when w_down / w_up carry scales.  The inject rows stay BF16.
- The staged / v3 / upmix / PF_HCDOWN / gfx906 split paths read BF16 rows and
  are skipped for it; the prompt path expands the codes into a bf16 scratch for
  its GEMM.
- The loader seeks with 64-bit offsets.
- A layer split keeps BF16 with a warning instead of exiting.
2026-10-14 07:11:52 +02:00
Niko1221 0c808eed00 RDNA2 SGEMM prompt GEMMs (#1123) are opt-in: STRATA_RDNA2_SGEMM=1 (gfx103x) / =all (any AMD card)
Fix-up to the ported #1123 commits.  The route is not bit-identical to the native call
(the author measured at most 4.3e-6 of the largest output apart), so unset keeps the native
path on every card, gfx103x included; =1 enables it on gfx103x devices only (a mixed layer
split keeps the native call on the other cards), =all on every AMD device for A/B arms and
tests.  The buffers are no longer freed under an in-flight GEMM: the stream is synchronised
before the FP32 copies grow, as the sm_60 FP32 route does.  docs/AMD_HIP.md says so.
2026-10-14 07:11:51 +02:00
vallicgrr 6ccdb14167 prefill gemm: a padded BF16 X (ldx, STRATA_PF_PAD) keeps the native call on the RDNA2 SGEMM route 2026-10-14 07:11:04 +02:00
vallicgrr 6dffb74a1f tests: hip_prefill_gemm crosses an RDNA2 SGEMM slice boundary (T 3300, K 10240; FP16 beta 1 with a padded ldy, and BF16) 2026-10-14 07:11:04 +02:00
Niko1221andSanghunYun d7298883b8 Let the resident RAM copy follow decode routing (opt-in)
Port of PR #1838 (commit fb868db01) onto 0.1.44.1. Test file merged with #2134's foresight test (both kept). replace_resident now drops a Foresight slot for the outgoing expert while its RAM place is rewritten (#2134's copies read those places).

Co-authored-by: SanghunYun <[email protected]>
2026-10-14 07:11:02 +02:00
Niko1221andvallicgrr 62a98cd353 docs: RDNA2 prompt GEMMs as SGEMMs (measured on an RX 6800), and a Windows gfx1030 report
Rebased onto 296b59d2: the docs/AMD_HIP.md conflict (the gfx1031/gfx1150/gfx1152 bullets main gained) is resolved by keeping main's text.

Co-authored-by: vallicgrr <[email protected]>
2026-10-14 07:10:58 +02:00
Rafa ad0582c3d4 #2146 tests: pa_bench (the prompt attention on a recorded selection); STRATA_QSA_DUMP_LAYER / _STEPS
STRATA_QSA_DUMP_LAYER=i keeps one QSA layer, STRATA_QSA_DUMP_STEPS=1 appends
the steps to each record.

pa_bench: the prompt sparse attention (qsa_prompt_attn_batch) on a
recorded selection - a STRATA_QSA_DUMP record of one layer and chunk with
its steps appended - random q and int8 K/V; median time and an output
hash for A/B of kernel variants (PA_BENCH_OUT=file writes the output).
2026-10-14 07:10:57 +02:00
Rafa 5d4c786760 #2146 sm_75: v2 prompt attention in one stage, three blocks per SM (bitwise)
On Turing the v2 int8 prompt attention (STRATA_PA_V2T=1) stages the next
chunk in registers, so its second shared-memory stage bought nothing - and
at 45.6 KB a block only one fit an SM (64 KB): four warps on four
schedulers, every latency exposed (ncu: 12.5 % warps active, tensor pipe
8 %).  Smem2T: one K/V stage, and the q.k partials of cell group nt
(8 cells) written over the 512 bytes of the warp's own K slice that held
those cells (read by then, a __syncwarp between) instead of their own
8.4 KB.  19.7 KB a block: three per SM (168 registers fit three).  The
same arithmetic in the same order - bitwise the same output.

RTX 2080 Ti, a recorded 130K selection (QSA layer 5, last chunk: 15871
queries x 2051 cells, tests/cuda/pa_bench.cpp): 113.8 -> 68.5 ms, same
output hash.  Engine (Q2_0 pack, 16K chunks, fused experts): 98K prefill
53.3 -> 49.9 s, 4K 3.54 -> 3.42 s; first logits bitwise identical at 98K
and 4K, the same generated text.
2026-10-14 07:10:57 +02:00
Rafa 0410d62aa9 #2092 sm_75: the v2 int8 prompt attention with register staging instead of cp.async (STRATA_PA_V2T=1)
On Turing the int8-KV prompt attention ran v1 because v2 stages its K/V
slices with cp.async. Here a warp loads its next slice into registers and
stores it into its other stage after the current chunk (the same bytes in
the same places); the MMAs are the m16n8k8 pairs v1 already uses. Accuracy
as v1 (qsa_prompt_attn_parity vs FP64: 3.7e-6 / 4.9e-6 / 4.5e-6 / 3.8e-6 of
3.5-3.7). RTX 2080 Ti, ms per chunk v1 -> v2: 32K x 1024 queries 13.0 -> 7.6,
64K x 512 6.7 -> 4.8, 1.5K 6.2 -> 5.0, 2.1K x 256 3.2 -> 2.3. Opt-in until
the engine A/B.
2026-10-14 07:10:56 +02:00
Rafa 8bdef963cc #2091 sm_75: the hyper-connection xn16 written as the FP16 its tensor-core GEMMs read (bitwise)
With STRATA_BF16_TC the prompt path converted xn16 (T x 10240) from BF16 to
FP16 before each of its two GEMMs (down and inject), in 16M-element slices:
~2.8 s of a 130K prompt read on an RTX 2080 Ti. The four xn16 writers now
store gemm.cu's conversion of the BF16 value (bf_to_hf_conv: the same clamp
and rounding as bf16_to_f16_kernel), and Gemm::bf16(..., x_f16) skips the
conversion on the tensor-core path (same slices, same GEMMs) and converts
back to BF16 on any other. Only when the stage runs BF16 on FP16 tensor
cores, without the BF16X2 low parts or a padded xn16 stride; the MTP draft's
own read keeps BF16. STRATA_XN_F16=0 turns it off.

First logits after a 98K prompt: bitwise identical; prompt read 1431 -> 1473
tok/s; a 4K run writes the same tokens.
2026-10-14 07:10:56 +02:00
imanu d976db3454 #2081 dequant: the prompt projections' BF16 dequant with 16-byte stores on sm_75 (bit-exact; STRATA_TC_SM75=0 keeps the per-group kernel)
The per-group dequant kernel writes 2 bytes every 64 bytes; 0.1.43 coalesced only the Q8_0 path (#1720). On compute
capability 7.5 the BF16 projections of the prompt path take a kernel that writes 16 bytes per thread; the values are
the same (a conversion, no arithmetic), so the output is bit for bit the old kernel's. Other cards are untouched.
2026-10-14 07:10:51 +02:00
Niko1221andvallicgrr 8f949956b2 HIP RDNA2 (gfx103x): run the prompt path's 16-bit GEMMs as FP32 SGEMMs
The rocBLAS the engine ships has tuned gfx1030 kernels for FP16->FP16, int8 and
FP32->FP32 only; every FP16/BF16-input, FP32-output GEMM (rocBLAS HS/BS) runs a
fallback kernel at ~5 TFLOPS. Widening the inputs to FP32 is exact and SGEMM
accumulates in FP32 like the native call.

Nothing passes through FP16, so unlike STRATA_HIP_PROMPT_F16=1 (#835) the
route cannot overflow: an FP16 GEMM output above 65504 becomes inf and can
spread as NaN, while here it stays a finite FP32 value. It is slower than the
FP16 route, and not bit-identical to the native call (at most 4.3e-6 of the
largest output apart on random inputs).

RX 6800, Coder IQ1_M, fresh 12K-token prompts (4 each, A/B in one binary via
STRATA_RDNA2_SGEMM=0): prompt 312 -> 452 tok/s, GPU timeline 37.2 -> 25.5 s
(gdn 9.4 -> 2.8 s, qsa proj 3.6 -> 1.4 s, hc read 3.85 -> 2.84 s). Decode
unchanged. A 9-task code-repair benchmark: 9/9 first try on both, model time
458 -> 374 s.

Default on for gfx103x only; N < 64 keeps the native call; if the FP32
buffers cannot be allocated the native call runs.

Rebased onto 296b59d2: the conflict in Gemm::f16 and gemm.hpp (the CUDA FP16 algorithm fallback and the sm_60 STRATA_DENSE_F32 block that main gained) is resolved by keeping both; the RDNA2 call sits before the CUDA/HIP GemmEx call.

Co-authored-by: vallicgrr <[email protected]>
2026-10-14 07:10:39 +02:00
Rafa d5b80b2e65 #2093 sm_75: the prompt path's top-k with one full read of the scores (identical ids)
block_topk_wide_kernel reads a query's scores six times (four radix passes,
a count, the emit): past ~115K cells that is DRAM traffic. block_topk_sample_kernel
takes a bound from every 32nd block (the KS-th largest of those keys), reads
all blocks once keeping those at or above it, and when they hold at least
`width` cells computes block_topk_kernel's threshold, `above` and tie budget
over them and emits them in block order (bitonic sort): every block at or
above the threshold is a candidate, so the ids are the reference's.
Otherwise the query is flagged and block_topk_wide_kernel<true> runs it.
qsa_topk_parity --selftest and qsa_topk_active_parity: identical ids
(continuous, 64-level, equal scores - the last ones take the fallback).

RTX 2080 Ti, 256 prompt queries under a 512K capacity: 131K 0.51 -> 0.45 ms,
262K 0.95 -> 0.59, 400K 1.48 -> 0.71, 520K 1.84 -> 0.79. Used above 28,672
active blocks; STRATA_TOPK_SAMPLE=0 keeps the wide kernel.
2026-10-14 07:10:32 +02:00
Rafa 050229ecda #2072 sm_75: the decode QSA selection at long context, split top-k and halving scorer (bitwise)
A verify window's 1-8 queries scored and selected on a handful of SMs; at
420K cells on an RTX 2080 Ti the two took ~6 ms of a ~52 ms window.

- block_scores_halve_kernel: the decode scorer's warp sums as one
  recursive-halving reduction (each value pairs lane l with l ^ o at offsets
  16..1, the butterfly's additions operand for operand), the lane's query
  values in registers (shared-memory reads were the limit, ncu mio_throttle),
  5-8 queries as two calls of up to 4. Same bits (qsa_select_bench
  QSA_BENCH_DECODE=1 hashes, 32K-524K, 1-8 queries). 420K: 1 / 3 / 4 / 6 / 8
  queries 117 / 265 / 347 / 510 / 774 -> 101 / 118 / 125 / 238 / 255 us.
  STRATA_SCORES_HALVE=0 keeps the old kernel.
- qsa_block_topk_split: the decode top-k over chunks of 2,048 blocks in six
  launches (four radix passes into per-query global histograms, a count, the
  emit); every CTA takes each digit from the complete histogram with
  block_topk_kernel's rule, so the ids are identical (qsa_topk_parity
  --selftest: continuous, 64-level and equal scores, 2K-524K). Decode calls
  at 420K: 0.21 -> 0.033-0.064 ms; short contexts unchanged within 0.01 ms.
  STRATA_TOPK_SPLIT=0 keeps the wide kernel (=1: on any card, for tests).

Both on sm_75 only by default.
2026-10-14 07:10:28 +02:00
Rafa 3f691eb345 #2071 sm_75: STRATA_SELECT_SIMT=2, the tiled FP32 block scorer with 32 x 128 tiles
block_scores_simt2_kernel: 8 x 8 products per thread, twice the blocks per
CTA and the next K chunk in registers; every dot is the same fmaf chain over
k = 0..127 and the same relu sum, so its scores are bitwise those of
STRATA_SELECT_SIMT=1 (qsa_select_bench QSA_BENCH_HASH=1, 131K-510K).
RTX 2080 Ti, 256 queries: 131K 1.21 -> 0.95 ms, 262K 2.01 -> 1.87,
400K 3.00 -> 2.82, 510K 3.77 -> 3.57 (the warp kernel: 7.7 / 15.3 / 23.2 /
29.6 ms). Opt-in like =1 (another summation order than the warp kernel).
2026-10-14 07:10:27 +02:00
Breno Perucchi 653f011360 prefill: gather experts in groups on short prompts too (resident and staged)
Below stream_all_min() the MMQ path gathered every expert with its own launch
and, for a staged one, its own wait and release event. The staged walk now
groups like the streamed walk: its ring is STAGE_GRP (32) slots, two groups, and
it stages at most 15 entries ahead, so a group's staged slots are not refilled
before the group is gathered (one wait on its last copy, one event releasing
them all). stage_slots() counts those 32 slots wherever the ring is counted or
taken, so a prompt borrows every byte it writes.
STRATA_PREFILL_GROUP_GATHER=0 turns grouping off everywhere, as before.
2026-10-14 07:10:25 +02:00
Rafa 80872d4349 #2070 sm_75: hc up kernel with batched row loads for 3+ tokens, down token pairs for T 4/6
gr_up75_kernel: two batches of 4 rows of w_up loaded before the dots, the
token activations read once per batch, the 4 xor reductions interleaved and
the row tails held 4 rows to a uint4 - bitwise gr_up_multi_kernel (gr_parity,
gr_multi_parity, fused_gr_bench hashes). Used from 3 tokens on sm_75
(STRATA_GR_UP75). The staged down kernel takes a token group from
blockIdx.y; sm_75 runs T 4 and 6 as pairs (STRATA_GR_DOWN_TG).

fused_gr_bench cold, RTX 2080 Ti, whole read: T3 53.2 -> 50.2 us,
T4 64.1 -> 57.6, T5 76.2 -> 69.3, T6 86.1 -> 70.9; T1-2 unchanged.
2026-10-14 07:10:24 +02:00
Rafa 26855986e1 #2069 sm_75: grouped native experts default to mode 4 (down: mode 0 in windows of 4+ tokens)
native_expert_bench on real Q2_0 rows of Flash-Next, RTX 2080 Ti, bitwise
equal to mode 0 in every case: 10 groups x 1 entry 47.3 -> 41.4 us,
20 x 1 83.7 -> 71.5, 30 x 1 120.2 -> 100.2, 20 x 2 124.2 -> 111.4. The down
projection of mode 4 loses with 3+ entries per group (12 x 3 36.5 -> 40.8,
16 x 4 56.9 -> 68.9), so cap_groups > 30 keeps mode 0 for it.
STRATA_SM75_EXP=0 keeps the CUDA layout; STRATA_EXP_MODE still overrides.
2026-10-14 07:10:23 +02:00
Niko1221andsergiywith 3f2db5e2c5 decode: the adaptive tier's evictions reach the device's residency table before the next window when the tier does not wait (STRATA_ADAPT_NOWAIT=1, STRATA_ADAPT_LAG > 1)
adapt() marks an evicted expert non-resident in host_res at once; d_res follows in apply_pending, once the round's
copies have landed.  When the tier does not wait for them (STRATA_ADAPT_NOWAIT=1, or STRATA_ADAPT_LAG > 1) the next
window runs before that.  doorbell_publish_res reads d_res, sees the evicted expert as resident, and when every routed
expert of a layer looks resident it does not publish the layer's activation.  The pool plans from host_res and
computes the evicted expert on the CPU from the activation published before.  Now every verifier launch (run,
run_slot_rows, batch_launch, pl_launch; a layer split's stages too) runs a pre-launch hook.  It patches the device
tables from a list of changed entries: res_patch, a small kernel on the verifier's stream reading mapped memory; the
whole table when the list overflows or a device does not map it at the host's address.  A failed whole-table upload in
apply_pending leaves its entries to the hook.  Only where the tier does not wait.  --batch-groups with a layer split
publishes every activation instead: set_always_publish, which now also turns E-6's device plan off.
--pipeline-windows keeps its fence.  STRATA_ADAPT_EVICT_SYNC=0 restores the old behaviour.  STRATA_ADAPT_CHECKRES=1
reads the tables back before each window, compares them with host_res, and counts the layer-windows that computed a
CPU expert with no activation published.

RTX 5090, IQ2_XS, --expert-cache 12000, 32K explain, 600 tokens: NOWAIT=1 34 of 222 windows with mismatched tables
(2470 entries), 1 such layer-window -> 0 / 0 / 0; LAG=2 55 of 223 (3222 entries: every swap), 5 -> 0 / 0 / 0.  LAG=2
is reproducible (two runs of each arm, the same tokens), and the fix changes its answer from token 228.  The hook's
cost under NOWAIT (6297 slots, 3 pairs): ms a round -1.1% chat, -1.2% 32K, noise.  The default (the tier waits)
installs no hook: tokens and first-token logits identical to d5ea7133 (p2k 200 tokens, 32K explain 600).

Port of PR #1517 (commit eb4796045) onto 0.1.44.1. Conflicts with the 0.1.43 adapt overlap gate and the tail-skip kind -2 resolved; set_adapt_gate and set_always_publish kept side by side. Docs line added.

Co-authored-by: sergiywith <[email protected]>
2026-10-14 07:10:19 +02:00
Niko1221 80bed76c8a tools/test_setup_mac: the --source test no longer runs setup.data_folder() for real
The real data_folder() moves the model files of this checkout and of the data folder named in the user's settings.json into --data-dir and saves that folder as the new data folder. With a TemporaryDirectory the models are deleted on exit and settings.json is left pointing at a deleted directory (found on the M4 and on the PC: the test wrote %APPDATA%/Strata/settings.json and ~/.config/strata/settings.json). The test is about --source, so data_folder is mocked.
2026-10-14 07:10:16 +02:00
Rafa 004efd4606 #2069 sm_75: interleaved small-batch matvec table for Turing (kIlRows_75)
The 2-4 column dense projections of the verify windows ran the plain multi-column kernels on Turing, which are
bound by dp4a throughput there from 2 columns on. mmvq_il_parity --bench --emit-table on an RTX 2080 Ti (SM 1650
MHz) gives a table like the Volta/Pascal ones: IQ4_XS 2560x10240 at 4 columns 81.1 -> 37.4 us, Q4_K 74.5 -> 39.1,
the Q5_K head 1671 -> 1045 us. Bitwise equal (the decode writes the same tokens). No one-column table: the
engine's dedicated single-column kernels sum in another order, for a 0.5% gain. STRATA_SM75_IL=0 turns it off.

Q2_0 512K resident, 4K prompt: one-shot decode 52.6 -> 55.5 tok/s; serve steady state ~69 -> ~77 tok/s.
2026-10-14 07:10:01 +02:00
sergiywith af0e9d3c1b prompt attention: STRATA_PROMPT_ATTN_IMMA=1 - the int8-KV kernel on INT8 tensor cores (opt-in, closer to FP32)
prompt_attn_i8v3_kernel: QK^T and PV as mma m16n8k32 on the int8 K / V codes as stored; the other operand in three
exact 8-bit parts (q scaled per head and 64-dim group, p * v_scale per row and chunk; X = rint(x * 65536) split into
h, l signed and m unsigned bytes, a u8 x s8 MMA): 24 bits, exact int32 sums, combined in FP32.  One K and one V stage
per warp, 12-row partials, launch bounds (128, 3): ~25.5 KB, 3 blocks an SM.  sm_80+ (STRATA_PA_SM80); Turing,
Volta and HIP keep their kernels.  Unset: v2, byte-identical.

qsa_prompt_attn_parity: against FP64 IMMA 2.15e-6 - 2.64e-6, v2 2.78e-6 - 3.30e-6, the FP32 kernel 1.90e-6 - 2.51e-6.
IQ2_XS, first-token KL to the FP32 kernel (STRATA_PROMPT_ATTN_OLD=1): median 0.0027 (IMMA) against 0.013 (v2);
2K 0.0026 against 0.0248.  1.35-1.53x a launch; a 32K prompt in 8K chunks 358 -> 312 ms of attention.
2026-10-14 07:09:47 +02:00
Niko1221 f288cbfea9 F16_RANGE counters (#1149): HIP only, and the gfx906 compat layer maps cudaMemcpyFromSymbol
Fix-up to the ported #1149 commits.  The range counters and the saturating to_f16
are compiled for HIP only, so the CUDA kernels (hf_sat in the SwiGLU product, the
embedding image) stay exactly as they were, with no flag read; set_f16_range /
f16_range_read are no-ops there.  strata_hip.h (the STRATA_HIP_GFX906 compat layer,
which goes first on that backend's include path) gets the cudaMemcpyFromSymbol
mapping the HIP shim already has, so the counters also compile for gfx906.
2026-10-14 07:09:41 +02:00
Niko1221andsergiywith 04be241a5a decode: STRATA_PCIE_BALANCE=1 picks each layer's PCIe count from measured costs
The verify window reads pcie_frac of a layer's missed experts over PCIe, a share
the link probe sets; the CPU's speed does not enter it.  With a slow CPU the GPU
spins on the pool's rows: IQ2_XS with one AVX2 pool worker (emulated) runs
16.8-17.2 ms a round at the probe's 0.55.

Opt-in, STRATA_PCIE_BALANCE=1 (one GPU, no batch slots, no --pcie-frac):
- wait_flag_ge_stamped: the flag spin kernels also write the GPU clock as the
  plan arrives, as the PCIe part starts and as the GPU waits for the CPU's rows
  (no extra launch; captured only with the switch on);
- the host times its pool per step; after each window, least squares with
  forgetting fit the pool's a + c n_cpu + d n_pcie (d: the link's DRAM reads
  slowing the pool - on a DRAM-bound pool moving experts to PCIe saves nothing)
  and the GPU's g0 + g n_pcie;
- a layer then reads over PCIe the m of its misses that minimizes the longer
  side; every 64th layer that would keep them all on one side moves one, so
  both fits stay measured.

IQ2_XS, RTX 5090 + 9950X3D, chat to the end of the turn:
- one AVX2 pool worker: 16.77 / 17.17 -> 12.98 / 12.82 ms a round
  (fit: CPU 42-60 us an expert, GPU 28 us a PCIe expert; 0.8 of the misses
  move);
- the 9950X3D's 16 workers: 3 pairs 12.19 -> 12.17 (pool DRAM-bound, few move).
Off (the default) the logits and 32 tokens equal v0.1.39 (2K, fixed cache).

Port of PR #1548 (commit a513394c4) onto 0.1.44.1. Conflicts with the 0.1.43 tail skip, PCIE_MIN1 and the fused waits resolved: the fused wait+copy launches stay off while the balance stamps are on. Docs line added.

Co-authored-by: sergiywith <[email protected]>
2026-10-14 07:09:32 +02:00
sergiywith cffe31d43e prefill: opt-in STRATA_HC_UPMIX=1 on CUDA - the hyper-connection up projection with gr_mix_r as its epilogue (sm_80+)
The up GEMM of each hyper-connection read writes `gated` (T x 10240 FP32, 336 MB at an 8K chunk) and gr_mix_r reads
it back with R at ~96% of the bandwidth - only fewer bytes help.  gr_upmix_cuda_kernel (128 tokens x 32 columns x 4
streams a block, K = 320 in cp.async-staged slices of 32, BF16 mma.m16n8k16 with FP32 accumulation, the mix in
registers in gr_mix_r's order writing all its outputs) never writes or reads `gated`.  Behind the gfx11 switch;
sm_80+ per device (STRATA_EMULATE_CC honoured), chunks of >= 33 tokens; else the pair.  Unset: unchanged.

gr_upmix_parity (new ctest): bitwise the cuBLAS + gr_mix_r pair at T >= 33 on an RTX 5090 (cuBLAS 13.3), <= 1.8e-7 at
T <= 32, both as close to FP64.  1.42x / 1.81x / 2.19x the pair at 2K / 8K / 32K; IQ2_XS: an 8K prompt -4.4%, 32K
-3.9% (the hc read -19% / -24%).  Logits identical with it on (600, 2K, 32K); the 32K greedy tokens moved only through
the time-driven suffix-draft policy (identical with --suffix-draft 0).
2026-10-14 07:09:30 +02:00
Niko1221andsergiywith c675f5b787 prefill: quantize each token's MoE input once and scatter it to its k rows (the same bytes)
mmq::quantize(m.mixed, m.src_dev, ...) read and quantized every token's row of `mixed` once per routed expert (K =
10).  mmq::quantize_scatter uses llama.cpp's quantize_scatter_mmq_q8_1_cuda (the pinned quantize.cu strata_mmq
already compiles): each token quantized once, its block written to all of its rows through slot_dev, the inverse of
src_dev.  The same q8_1 bytes, so the default; STRATA_QUANT_GATHER=1 keeps the gather.  No new device code (CUDA and
HIP compile the same kernels); SYCL's prefill.cpp untouched.

prefill_mmq_scatter_test (new ctest): every MMQ q8_1 layout (D4, DS4, D2S6), T = 1..2099: 0 differing bytes.  IQ2_XS
600 / 2K / 32K and Q2_0 2K against e8ca9afd: identical first-token logits and 32 greedy tokens.  RTX 5090 (96 MB L2):
the quantizer 2x at 2K, even at 8K (the chunk's input fits in L2), 1.4-1.9x at 32K; a 32K prompt with
--prefill auto:32768: 77.5 -> 58.1 ms of quantization.

Co-authored-by: sergiywith <[email protected]>
2026-10-14 07:09:21 +02:00
ruibeikaa a87fb80e13 sm_60: stage the IQ3 tables by default on compute capability 6.0 only
The staged IQ3_XXS / IQ3_S tables trade a per-block copy and barrier for
cheaper lookups, which pays only where the grouped expert kernels are
decode-bound. On GP100 (6.0: no __dp4a, 70-140 GB/s) it does; on an RTX 3090
Ti (sm_86), forced on with STRATA_IQ_STAGE_TABLES=1, the same real Swift 1.5
IQ3_S layers (20 experts, 24 entries) took 3-13% longer (71-76 -> 75-84 us;
those kernels read at 490-665 GB/s there) and IQ3_XXS was unchanged, the
output bitwise the same. 6.1 cards (P40, GTX 10) have __dp4a and are
unmeasured, so the default now asks for 6.0 rather than any 6.x;
STRATA_IQ_STAGE_TABLES=1 still turns it on anywhere.

docs/OLDER_GPUS.md says so, and that neither STRATA_Q8_SM60 nor the tables
is measured on 6.1: both build for sm_61, and tools/q8_sm60_check.cu passes
on an RTX 3090 Ti, where the kernel's dot is the hardware __dp4a.
2026-10-14 07:09:00 +02:00
ruibeikaa ac35eebf30 sm_60: stage IQ3_S's codebook and sign masks in shared memory too
IQ3_S was the one i-quant gate/up format whose codebook the grouped expert
kernels still read from global memory (IQ2_XXS / IQ2_XS / IQ2_S stage theirs
everywhere, IQ3_XXS on 6.x since the first commit of this branch). Its 512
codebook words are now staged per block with two 256-word tables of byte
masks - IQ3_S keeps one explicit sign byte per 8 values, so the tables hold
the masks of a sign byte's bits 0-3 and 4-7 - applied as
(g ^ mask) + (mask & 0x01010101): every iq3s_grid byte is odd, so a negated
byte never carries into the next. Bitwise the same output (iq_multi_parity
and native_grouped_parity on a GP100 die, with and without
STRATA_IQ_STAGE_GRID_MMVQ=1).

One GP100 die, a verify window's layer of real Swift 1.5 IQ3_S experts
(20 experts, 24 entries): 517 -> 393 us with Q2_0 down, 492 -> 369 us with
IQ4_NL down. Swift 1.5 IQ3_XXS (13 of 48 layers IQ3_S), four GP100 dies,
1M context, decode tok/s, medians of three interleaved engine starts:
36.0 -> 37.2 (10.7K document), 36.3 -> 37.4 (cached), 43.1 -> 44.5
(reasoning), 41.6 -> 43.0 (code), with the same drafts accepted.

The switch covers both formats and is renamed accordingly:
stage_tables_on(), STRATA_IQ_STAGE_TABLES=1|0 (was STRATA_IQ_STAGE_GRID18).
Still the default on compute capability 6.x only.
2026-10-14 07:08:59 +02:00
ruibeikaa 080abf86a4 tools/q8_dense_gguf.py: convert every shard that holds dense matrices
The tool converted only the shard it was given and hard-linked the others, which is enough for Flash-Next (every
natively served projection and the head are in shard 1) but not for Swift 1.5 IQ3_XXS, which keeps 78 of them in
shard 1 and 223 in shard 2: one run left three quarters of them in their K- / i-quants, silently, and the kernel
only runs Q8_0 matrices. Now any shard of the model may be named; every shard that holds such a matrix is written
as before (same tensor-info length, Q8_0 blocks appended, offsets checked) and the others are hard-linked.

Checked: Swift 1.5 IQ3_XXS in one run gives both shards byte-identical to two single-shard runs (301 tensors);
Flash-Next IQ3_XXS converts 301 tensors in shard 1 and links shard 2.
2026-10-14 07:08:59 +02:00
Niko1221andruibeikaa 6e29a5d314 sm_60: a Q8_0 decode GEMV for Pascal GP100 (STRATA_Q8_SM60=1, opt-in)
GP100 has no __dp4a, and on a GP100 die the verify window's dense projections read their weights at 40-80 GB/s
of ~730. With STRATA_Q8_SM60=1 every Q8_0 dense matrix (any shape) and the head get the packed planes of
STRATA_Q8_PACKED and run src/kernels/cuda/q8_sm60.cuh: a warp (half / quarter warp for narrow rows) per row,
16-byte weight loads through the read-only cache, the window's q8_1 activations copied once per workgroup into
shared memory as an int8 plane plus float scales, STRATA_DP4A (vmad on sm_60), a shuffle sum with no barrier per
row, persistent workgroups (4 per SM, STRATA_Q8_SM60_BPS). Shapes it declines (> 48 KiB of shared memory: a
6144-wide row with 7-8 columns) keep the usual kernel. Not bitwise the exact kernels (summation order); the
packed copies cost their size in VRAM (Flash-Next: 2.9 GiB of dense projections over the stages, 0.6 GiB head).

The GSQ-RCO packs have no Q8_0 dense matrices: tools/q8_dense_gguf.py writes a shard 1 whose natively served
projections and head are Q8_0, keeping the tensor-info block's length so every expert keeps its offset (the native
pack is reused) and hard-linking the other shards beside it.

PH402 SKU 200 (four GP100 dies), clocks locked at 1050 MHz, Flash-Next IQ3_XXS, 32K, layer split 12,24,36,
--prefill 2048, greedy, decode tok/s (medians, interleaved starts): GSQ-RCO shard 28.3, Q8_0 shard with
STRATA_Q8_PACKED 31.1, Q8_0 shard with STRATA_Q8_SM60 38.9 (+37%); per window dense 19.2 -> 11.2 ms, head
4.25 -> 2.1 ms; prompt reads unchanged. tools/q8_sm60_check.cu: 57 shape x column cases against a CPU reference,
max relative error < 1e-5. docs/OLDER_GPUS.md has the table, the conditions and how to use it.

Co-authored-by: ruibeikaa <[email protected]>
2026-10-14 07:08:58 +02:00
Niko1221 f01b989264 sycl: mmvq_bench checks the STRATA_Q6_PLANAR copy and a gathered subset bitwise against the GGUF-layout result (#2340) 2026-10-14 07:08:52 +02:00
Francesco Albano 051b970303 cpu pool: --host-core sibling, the host on the first core's other hardware thread (hybrid CPUs too)
Windows sends a GPU's interrupts to one logical processor, usually the first,
and the host thread spins there (--host-core first, the default). --host-core
last moves it to the last physical core, but not on a hybrid CPU, whose first
core is its best P-core, so a 12th-14th gen Intel or a Core Ultra keeps the
host on logical processor 0 with the interrupts.

--host-core sibling (or STRATA_HOST_CORE=sibling) keeps the host on that same
first core but on its other hardware thread. The interrupts keep the first
logical processor, which nothing else is pinned to, and the workers keep
every other core, since they take one logical processor per core (with
--pool-affinity auto / p-cores, which also use the P-cores' siblings, the
host's sibling is taken out of their list). Without an SMT sibling it is
--host-core first. It works the same on Linux (the sibling from sysfs), where
the interrupts are usually spread anyway.

detect_cpu_topology records the host core's sibling (CpuTopology::host_sibling)
in each layout; pool_affinity_test checks, in every affinity mode, that
sibling puts the host there with no worker on its core, and that it is first
without SMT. The startup log names the placement.
2026-10-14 07:08:35 +02:00
xjc10 c211cb3ebe hip gfx103x FP16 prompt: STRATA_F16_RANGE counts what enters and leaves FP16; the range measured (outputs peak at 410 of 65504) and the FP32 route compared
The README's range check said "none beyond 65504" over 110K tokens without
the numbers. The engine now has the counter: STRATA_F16_RANGE=1 records,
per device, the largest |value| and the count beyond 65504 (or not finite)
where a value enters or leaves FP16 - the activation images (hf_sat, the
image writers' one funnel), the BF16 weights converted to FP16
(bf16_to_f16_rows) and the FP16 GEMM outputs as they are widened
(widen_rows_f16: an output rocBLAS wrote beyond the range is Inf there) -
and prints one line per prompt. Off, the cost is one flag read per value;
on, a 34.4K-token prompt read at 889.8 tok/s against 888.5. to_f16 (the
hc image for the FP16-weight GEMMs, the PLE key image) saturates like the
other image writers now, so every FP16 image is under the same rule. The
HIP shim gains cudaMemcpyFromSymbol.

Measured on one RX 6900 XT over 466K tokens in nine prompts (corpus
slices, two prompts built to push activations - delimiter floods, a token
repeated 4,000 times, 6,000-character runs, base64, five scripts - and
four batches of real requests): outputs peak at 409.5 (160x below 65504),
activations at 112.2, the BF16 weights at 11.56; nothing beyond the range,
nothing non-finite. The prompts built to push activations stayed below
the ordinary corpus. One model (GSQ-RCO IQ3_S); the table is in the
README's range section.

The README also gains the same-card comparison with #1006, which takes
these GEMMs as FP32 SGEMMs: 8.4K / 32.8K / 104K-token prompts 412 / 455 /
456 tok/s on the old path, 622 / 742 / 750 with #1006, 730 / 900 / 904
with this change (the card's FP32 rate against its FP16 rate), with the
phase times and what the SGEMM route covers that this change does not.

(cherry picked from commit e780c8581e)
2026-10-14 07:08:33 +02:00
xjc10 d0838c6c3a tests: hip_prefill_gemm runs the FP16-io prompt GEMMs (set_f16_io) on any HIP card, and its BF16 inputs are no longer zero
The test ran only the FP32-out GEMMs, so the gfx103x prompt path's FP16-out
GEMM (f16_inplace, widen_rows_f16) and bf16()'s W conversion through the
scratch were not under ctest. Every shape now runs a second time with
set_f16_io(true), forced on whatever HIP card runs the test: X as the FP16
image for the BF16 products, a scratch of 64 rows so a 96-row weight converts
in two slices (the second written at Y + 64), ldy > N, an odd N (257), T = 1,
and a beta = 1 f16() call, which keeps the FP32-out GEMM. FP16-out results are
checked against the double reference at FP16's rounding (6e-4 relative per
element plus 1e-4, rel L2 < 1e-3); the row padding and the guards around Y
must be untouched, as before. The FP32-out runs keep their tolerance.

Found while adding them: the BF16 inputs were all zero. hip_bfloat16(float)
produced data 0 in this test's build (ROCm 10.0, gfx1030; not traced further),
so the two BF16 cases passed on nothing (rel L2 printed 0). The BF16 bits are
now made by hand (round to nearest even on the float's bits); with real inputs
the BF16 cases pass at rel L2 1.9e-07 (FP32 out) and 2.2e-04 (FP16 out).

On an RX 6900 XT (gfx1030, ROCm 10.0): 11 of 11 pass.

(cherry picked from commit 4c6c9fe374)
2026-10-14 07:08:32 +02:00
libratechw b6ccf873b8 prefill: the register top-k from 16K cells of context on gfx1151 (STRATA_TOPK_REG_MIN_BLOCKS, gfx1151's table sets 4096)
On HIP a prompt batch with fewer than 7,168 blocks (28K cells of context) took the original top-k kernel, a line measured on
gfx1201 before the cooperative scan.  With the scan the register kernel is faster from about 15.5K cells on gfx1151 (2,048 queries:
original / register kernel 0.57 / 0.82 ms at 13K, 1.01 / 0.87 at 16K, 2.4 / 0.93 at 20K, 3.9 / 1.00 at 24K).  The line is now
STRATA_TOPK_REG_MIN_BLOCKS (default 7168, as before) and gfx1151's table sets 4096.  The selected cells are the same
(qsa_topk_parity, also run with the register kernel always on: ctest qsa_topk_parity_reg_always).
2026-10-14 07:08:28 +02:00
0FL01 557b79e78f test: cover exact multi-row and fused HIP paths on gfx906 2026-10-14 07:08:27 +02:00
ruibeikaa 96cb96aced sm_60: stage IQ3_XXS's codebook and sign masks in shared memory (grouped expert kernels)
Ported from shinbunbun/llama-cpp-p100-patches (MIT), patches 29 `mmvq-iq3xxs-grid-smem` and 30
`mmvq-ksigns-smem`: the grouped expert kernels stage the 256-word iq3xxs_grid and two 128-word tables of per-byte
sign masks once per block, and apply a mask as (g ^ mask) + (mask & 0x01010101) instead of the per-lookup
unpack_ksigns / __vcmpne4 / __vsub4 chain (iq3xxs_grid's bytes are all >= 4, so no carry crosses a byte).

Bitwise the same output: checked end to end on a PH402 (GP100), greedy, three prompts x three interleaved rounds,
identical replies with and without. Measured only on GP100, where it is within noise (+0-4% decode), so it is the
default on compute capability 6.x only; STRATA_IQ_STAGE_GRID18=1 / 0 forces it on / off on any card.
2026-10-14 07:08:24 +02:00
Warabii1248 1327c0916c fused_gr: say in the comment that CUDA opts in to the register split norm 2026-10-14 07:08:24 +02:00
libratechw 6ef23b8966 prefill: the QSA top-k digit by a cooperative scan (same ids)
block_topk_reg_kernel picked each radix digit with thread 0 walking up to 255 bins while the other threads waited.
Threads 0-255 now prefix-sum the bins in descending order, and the one where the count above reaches `width` takes the
digit (bin 0 when none of 255..1 does). The digit and the count above are the same, so the selected ids are the same.

Measured on a Ryzen AI Max+ 395 (gfx1151, ROCm 7.14.1), qsa_topk_parity with 2,048 queries (CTX 2048 10 <ctx> 1),
ms per call:
  32K  2.19 -> 1.11 (-49%)
  64K  2.54 -> 1.63 (-36%)
  128K 4.83 -> 3.79 (-22%)
qsa_topk_parity --selftest: 19 PASS. UD-Q4_K_XL end to end, 3 interleaved rounds: the same text in every round; the
128K prompt moved +0.2%, inside the noise, so the gain is the kernel's and not yet visible end to end.

CUDA (RTX 3060, Windows, CUDA 13.0, sm_86), same call, ms: 4K 2.12 -> 0.975, 8K 2.04 -> 1.03, 16K ~1.9 -> ~1.1, 32K ~2.0 -> 1.27, 64K 2.53 -> 2.15, 128K 7.29 -> 7.29 (no change); qsa_topk_parity --selftest passes, ids identical in every case. Not built: SYCL.
2026-10-14 07:08:23 +02:00
Niko1221andnekomario28 719b733e01 dq_q2_0: keep the Q2_0 signed-zero fix on every HIP arch/version (#1540, #1474)
The negative-scale-times-zero workaround was gated to an arch list (gfx1012 on
HIP < 7, gfx1151, gfx1200/gfx1201 since 0ff45cffb); the list only ever chased
the newest report. The workaround writes the scale's sign bit for code 1, which
is value-identical on targets whose compiler never folds: keep it on every HIP
build. Ported from PR #1540 onto the tree that already carries the gfx1200/1201
entries; the comment text is adjusted.

Co-authored-by: nekomario28 <[email protected]>
2026-10-14 07:08:23 +02:00
Warabii1248 fec7d1cae2 fused_gr: STRATA_GR_NORM_FAST=1 on CUDA too (opt-in; the register split norm is bitwise the default; decode +0.5-0.6% on a V100 + PG500 split) 2026-10-14 07:08:23 +02:00
Warabii1248 ab7c977216 verify: say in the comments that sm_7x takes argmax_rows with STRATA_ARGMAX_ROWS_SM70=1 2026-10-14 07:08:23 +02:00
Warabii1248 83081577e5 sm_70: the multi-block greedy pick (argmax_rows) as an opt-in, STRATA_ARGMAX_ROWS_SM70=1 (bitwise the one-block pick; decode +0.7-0.8% on a V100 + PG500 split) 2026-10-14 07:08:22 +02:00
Warabii1248 ab5ea838de pf_wmma: say in the comment where the WMMA2 tile layout comes from 2026-10-14 07:08:12 +02:00
Warabii1248 dca6a6532f pf_wmma: STRATA_PF_WMMA2=1 (opt-in), a second Volta prompt-expert kernel for IQ2_S gate/up and IQ4_NL down - the dequantization spread over the whole block from a raw stage in shared memory (bitwise dq_iq2_s / dq_iq4_nl; prompt +2.5-4.3% with the IQ3_XXS pack on a V100 + PG500 split) 2026-10-14 07:08:12 +02:00
Warabii1248 4dc3206402 pf_wmma: set the tiles shared-memory attribute on each card, not once per process (a layer split stopped on its second card: "the WMMA expert tiles could not launch") 2026-10-14 07:08:11 +02:00
Warabii1248 caa3c8425a pf_wmma_parity: run on every sm_70+ card in one process, in turn (a layer split launches the tiles on its second card after the first) 2026-10-14 07:08:11 +02:00
g0 2d9b1b348a pf_wmma: take the FP16 prompt path for IQ4_NL down too (mmq_dt == 20) 2026-10-14 07:08:10 +02:00
g0 cb73a0538a pf_wmma: IQ4_NL (type 20) WK=128 lane for the down product (K=640); parity cases 2026-10-14 07:08:10 +02:00
NoxestNoz 140f011030 bench/results: V100 prompt experts on FP16 tensor cores (+9.6% / +10.3% prompt speed); docs/NVIDIA_V100.md 2026-10-14 07:08:09 +02:00
NoxestNoz fb433c3a57 pf_wmma_parity: gemm_iq_f16_grouped against a double reference over the FP16 path's weights 2026-10-14 07:08:09 +02:00
Niko1221 cb2740a8c2 prefill: Volta runs the prompt experts on FP16 tensor-core tiles, a group per launch (STRATA_PF_WMMA)
gemm_iq_f16_grouped: one launch per MMQ expert group, a block = 128 weight rows of one
expert x up to 256 of its token rows, each weight superblock dequantized once into shared
memory by dq_dispatch (the FP16 path's formulas), FP16 activations, WMMA with FP32 sums,
the next activation tile prefetched into registers.  sm_70 defaults to it for the layers
MMQ takes and for IQ1_M (which rides the MMQ grouping); STRATA_PF_WMMA=0 keeps MMQ.

Co-authored-by: NoxestNoz <[email protected]>
(merge-resolved onto 0.1.44.1: the MMQ gather now zeroes the group tail itself and the quantizer takes a scale buffer)
2026-10-14 07:08:08 +02:00
Xiaomi2008 1ab0fbd02a --pool-affinity measure: on a hybrid CPU, time the all and p-cores pool layouts at start and take the faster (opt-in)
The pool's default layout puts a worker on every physical core, and a layer's
experts are done when the slowest worker is. On a Core i7-13700K (8 P + 8 E
cores) the E-cores make the verify window wait, by an amount that changes from
run to run.

`--pool-affinity measure` (hybrid CPUs, native packs) times both layouts on
the pack's own experts before the pool is built: two experts of four tokens
per layer through run_split_multi_native, every layer in turn, the host
spinning ~150 us between two layers, three interleaved blocks per layout,
about a second in all. p-cores is taken unless all is more than 10% faster.
One line says what it read and took. It is one reading at start and is not
repeated.

Measured on this branch (0.1.42 + this change), Core i7-13700K, RTX 5090,
64 GB, Windows 11, Swift 1.5 IQ3_XXS, 128K context, one 8K prompt and the
same 1,500 forced tokens (--spec-follow), three interleaved rounds:
  default (all, 15 workers)   146.6 / 151.8 / 158.0 tok/s, pool 6.3-7.6 ms per window
  --pool-affinity measure     200.0 / 201.5 / 204.8 tok/s, pool 2.1-2.2 ms (read all at 16.4-17.6 ms, took p-cores)
  --pool-affinity p-cores     200.3 / 203.9 / 204.9 tok/s, pool 2.1 ms
Over four server starts of one day on 0.1.41 it read all at 14.8, 21.9 and
25.3 ms against 6.1-6.2 ms for p-cores, and once all at 6.6 ms against 8.6 ms,
where it took all.

Without the value nothing new runs: the option parses as before and the
function is not called. The layout changes no result, only where threads run.
Built for CUDA only (sm_120, CUDA 13.0.88, MSVC 14.50 with nvcc's
--allow-unsupported-compiler); HIP and SYCL were not compiled (the change is
in generate.cpp, which both build). Not measured: other hybrid CPUs, Linux.
2026-10-14 07:08:06 +02:00
aangelinsf fde1d795ef SYCL verify: batch windows keep the doorbell nodes under the lean window
0.1.43 made the decode window lean under STRATA_VERIFY_NO_HOST with the device
plan (46a77907): the doorbell publish, flag waits and plan copy are dropped
because no host answers them. capture_batch() records batch windows through the
same record_window(), so a batch graph was also recorded lean, but the host batch
path (run_slot_rows / batch_poll) still waits for each layer's doorbell unless the
stage is all-resident. On a stage with mirrored experts the graph finished with
the ring silent: "verify batch: layer 0 never rang (graph finished)", then
UR_RESULT_ERROR_UNINITIALIZED and exit 139 at the first shared window.

Leave lean off while recording a batch window (batch_rec_). The solo window
stays lean. Batch tokens are identical to a run with STRATA_VERIFY_LEAN=00.
2026-10-14 07:07:51 +02:00
W1nge ccba2d5fd3 core: write reduced helper output directly to mapped host memory 2026-10-14 07:07:49 +02:00
Zack ea08bd6e81 decode: size --lookup-chain's window cap from the cap in force, not the stale mtp_max_t
With the default --suffix-draft 3 the suffix block above already raises --spec by 2
(4 -> 6) and writes mtp_max_t = 4. The --lookup-chain block then computed
min(mtp_max_t + K, 8) = min(4 + K, 8), so with --spec 4:

  K=1 -> max(6, 5) = 6   (unchanged: the flag does nothing)
  K=2 -> max(6, 6) = 6   (unchanged: the flag does nothing)
  K=3 -> max(6, 7) = 7

Measured on 0.1.40 by reading the cap back from /metrics engine.spec after a restart
per row; K=1 and K=2 were indistinguishable from not passing the flag.

Size from the cap actually in force instead, so K adds up to K of headroom while the
kVerifyMaxT = 8 ceiling and the mtp_max_t bound on the MTP's own drafts are preserved.
Behavior is unchanged when --suffix-draft is 0 or when the cap is already at 8.
2026-10-14 07:07:36 +02:00
yjamil8 c41c0ebbd1 KV streaming under WSL: pin the K/V host copy with cudaHostRegister
Under WSL2 the NVIDIA driver caps what cudaHostAlloc(Mapped) can pin at about
1 GiB in all, so the KV streaming host copy (layer.cpp) cannot be allocated
and setup turns streaming off there: the whole KV cache must sit in VRAM,
beside the expert cache. cudaHostRegister of ordinary memory is not capped the
same way; the expert arena already registers tens of GiB with it on the same
machines (pinned.cu).

kv_host_alloc() maps anonymous memory and registers it Portable|Mapped, before
any page is touched, when /dev/dxg exists (WSL2, detected as generate.cpp's
under_wddm() does). Elsewhere cudaHostAlloc stays first. Either way the other
method is the fallback when the first is refused, and
STRATA_KV_HOST_REGISTER=1/0 forces the order. On native Linux and Windows the
behaviour is unchanged unless cudaHostAlloc is refused (by inspection; not run
there). The failure message no longer says WSL pins only 1 GiB.

Measured on one RTX 5090, Ryzen 9 9950X3D, WSL2 (157 GiB), Windows driver
591.86, IQ3_S, --max-context 1048576 --rope-scaling yarn --rope-scale 4
--kv int8 --kv-resident 32768, with this commit:
- start: "the K/V in 12.38 GiB of pinned RAM"; expert cache 11,609 slots
- tools/needle_bench.py: 4/4 found, 128K and 1M (1,015,070 prompt tokens) at
  depths 10 and 90
- decode 99-124 tok/s with 0.5-1M tokens in context (4 runs); 150-191 tok/s
  medians from 55-token prompts to 128K (single runs 113-196); prefill about
  4,200 tok/s for a cold 1M prompt
kv_stream_parity still passes; it allocates its own host pools, so it checks
the streaming kernels, which this change does not touch.

Not tested: layer split or remote experts under WSL, Docker Desktop.
setup.py still never writes --kv-resident on WSL and removes it there
(upgrade_config); until that changes it has to be added by hand.

Port of PR #1498 (commit 79b43be18).
2026-10-14 07:07:32 +02:00
Evan Lightcap f2e142f444 resident RAM mode: a budget keeps the prompt path's lend region when it has room (#1683)
pin_cache_complement set lend_from_slot = -1 whenever --resident-budget-gib was given, so the experts of the cache
slots the prompt path borrows were in neither VRAM nor RAM during a prompt and were read from the model files on every
chunk, and again when the slots were refilled - even with room left in the budget. The budget branch now appends the
lend region's experts after the ranked ones, from the last slot down (the order the borrowing takes them), while they
fit; the budget stays a hard cap, and the ones past it keep the file fallback. The start-up line reports the count, and
the lend warning names a larger budget as the remedy when one is set.

Measured on an RX 6900 XT 16 GB (HIP gfx1030), Ryzen 9 5950X, 125 GB RAM, UD-Q4_K_XL, --resident-budget-gib 71 (the
copy held all 63.47 GiB outside the cache, 7.5 GiB unused; 1,231 lent slots = 3.60 GiB), engine 0.1.42, one run per
case, greedy: file reads of the lent experts per request 0.5-2.6 GB -> 0; SSD reads per fresh 8K / 16K prompt 2.8 /
1.9 GB -> 0.3 GB; prompt-path host staging 0.8-2.1 s -> 0.06-0.5 s per request; fresh prompts 887 / 7.8K / 15.8K
tokens 180 -> 194, 633 -> 676, 677 -> 698 tok/s. Decode unchanged.

Port of PR #1896 (commit 29063eb2b).
2026-10-14 07:07:32 +02:00
Xiaomi2008 fa1d1d1ebb STRATA_EXCHANGE_ROTATE: rotation for packs whose layers differ in expert block size (spares kept per size)
Rotation hands the evicted expert's exchange buffer over as its place in the
RAM copy instead of copying the bytes there. It required one block size for
all layers and fell back to the copy path on any other pack: Swift 1.5 IQ3_S
has seven sizes (1,510,400 to 2,662,400 bytes) and IQ3_XXS nine.

A swap exchanges two experts of one layer, so the slot the promoted expert
leaves has the size the evicted one needs. ExchangeStorage now keeps its
spares per block size (initialize_sized): an exchange takes a spare of its
layer's size and hands back one of the same size. commit() refuses a spare of
another size and an expert of another layer. initialize() keeps the uniform
geometry and its checks. exchange_buffer() takes the layer; reserve_exchanges()
takes the number of trailing buffers that only stage copies (--adapt-async's
second half), kept in the largest size alone. With --adapt-async 1 a rotated
round flips in the tick that sees the copies in: there is nothing for the job
to move, so the round ends one verify window sooner.

Still opt-in (STRATA_EXCHANGE_ROTATE=1); unset, nothing changes. The exchange
allocation grows to one spare of every size per swap of a round: 1.06 GiB of
page-locked RAM on IQ3_S with --adapt-swaps 80 (1.26 GiB with --adapt-async 1)
against 0.20 and 0.40 GiB.

Measured on a Core i7-13700K, RTX 5090 32 GB, 64 GB, Windows 11, Swift 1.5
IQ3_S, 128K, --resident-experts, --adapt-every 1 --adapt-swaps 80, p-cores:
  one 8K prompt, the same 1,500 forced tokens, blocking tier, rotation off / on:
    161.6-165.1 tok/s (five runs) / 184.9-188.7 (four runs); two runs of each
    arm with the same cache size (10,681 slots) wrote identical --window-hashes
    for all 434 windows
  --serve, 23 coding-agent turns, greedy, three interleaved rounds, blocking tier:
    147.8 / 159.2 / 158.4 (the build before) -> 175.2 / 184.9 / 172.8
  the same turns with --adapt-async 1, one binary, two rounds:
    166.0 / 170.1 -> 185.6 / 182.5
The tier swaps about 30 experts per verify window on this box (44% of the
experts fit in VRAM), so the copy path moved about 50 MiB of RAM per window.

tests/core/exchange_storage_test.cpp gains a three-size case (6,144 exchanges)
and passes built alone with MSVC 14.50. Built for CUDA only (sm_120, CUDA
13.0.88, MSVC 14.50 with nvcc's --allow-unsupported-compiler); HIP and SYCL
were not compiled (expert_source.cpp and generate.cpp are built by both). Not
measured: Linux, AMD cards, a layer split.

Port of PR #2042 (commit 8db375018).
2026-10-14 07:07:31 +02:00
Katsumi Takeuchi 9de548d198 sycl opt-in: STRATA_IQ3S_WIDE=1, a wide mmvq kernel for the dense IQ3_S matrices
The multi kernel reads the dense IQ3_S matrices (attn_q, attn_gate, ssm_out, attn_k, the shared experts' gate/up)
at 76-114 GB/s on the B70. The new kernel has the shape of the Q4_K..IQ4_XS wide kernels. unitrace, one card,
IQ3_XXS, T=1: worst attn_k (512 rows) 77 -> 123 GB/s, best attn_q (12288 rows) 114 -> 307 GB/s; these matrices take
0.60 ms of kernel time per token instead of 1.36 ms.

Unset or 0: the multi kernel, output identical to main. With the option the greedy output parts from the default at
token 198 of 256 (the sub-blocks are summed in another order).
2026-10-14 07:07:25 +02:00
libratechw edf61f18e9 prefill: the VGPR-capped gfx11 fused expert kernels are on again for gfx1151 on clang 23, except IQ2_XS and Q8_0 (#1180)
Since 0.1.40.2 the waves_per_eu(8) variant of native_w11_kernel is uncapped on clang 22 and newer, because clang 22
computes wrong results with it on gfx1100. On gfx1151 with clang 23 (ROCm 7.14.1) the capped kernels give the same
bits and take less time, as alexdns1 measured in #1180. The cap now applies there again, per format
(hip_prefill_fused_iq, one layer, 2048 / 3584 / 8192 tokens, the same output bits hash in every case):
- gate/up IQ2_XXS / IQ3_XXS / IQ3_S / IQ4_XS and down Q2_0 / IQ4_NL: capped, 8-15% less time per layer.
  IQ2_S gate/up is capped too, but its LDS leaves room for 3 blocks per WGP and the compiler keeps its 238 VGPRs;
  IQ2_S / Q2_0 takes 2-4% less time from the Q2_0 down.
- IQ2_XS gate/up: not capped. Under the cap it spills 104 VGPRs and takes 61-66% more time.
- Q8_0 down: not capped. It spills 18 VGPRs and took -1.9% to +1.9% time (with Q4_K gate/up).
The test is the clang major version: other clang 23 builds are not measured, and clang 24 and newer keep the clang 22
rule. The template argument is the same in the host and every device pass; only the device pass for gfx1151 decides
the attribute (nw_capped). Other architectures, other compilers and -DSTRATA_W_LB=N behave as before: the gfx1100
disassembly is identical to before (kernel names aside). The file builds for gfx1012 / 1030 / 1100 / 1150 / 1151 /
1200 / 1201 and with STRATA_HIP_GFX906 (gfx906); no CUDA build here, and the changed lines are inside
`#if defined(__HIPCC__)`. SYCL has its own copy of this file, not touched. The path is opt-in (STRATA_PF_FUSED=1), so
default output is unchanged.

End to end on a Ryzen AI Max+ 395 (128 GB, gfx1151), UD-IQ4_XS, the section-5 switches of docs/STRIX_HALO.md with
--prefill 16384, 3 interleaved rounds of fresh servers: a 128K prompt read 1,126.1 -> 1,153.4 tok/s (+2.4%; +1.8%,
+2.4%, +2.6% per round). At 4K (868.0 -> 894.9, +3.1%) and 32K (1,182.9 -> 1,207.0, +2.0%) the medians moved too, but
within the spread of 3 runs. Decode unchanged; the same text for all 9 prompts.
2026-10-14 07:07:17 +02:00
libratechw acc94b2171 prompt attention (gfx11 / gfx12 WMMA): issue a chunk's reads before its compute
prompt_attn_wmma_kernel interleaved a chunk's global reads with its
compute: the V slice was staged first, then each 16-cell tile loaded its K
scale and fragments between the q.k WMMAs, so a wave waited on each read
in turn (a thread trace on gfx1151: 56% of the kernel's time in
s_waitcnt vmcnt, 0.4% in WMMA). The chunk's reads now go out together
(fetch), then the same compute runs on them (chunk). The values and the
order of the arithmetic are unchanged.

gfx1151, ROCm 7.14.1 SDK (clang 23), hip_prompt_attn_wmma with
STRATA_PA_FAST=1, three interleaved runs against the unchanged kernel,
ms per chunk: 32K context / 2,048 queries 12.50 -> 11.29 (-9.7%),
1,500 / 1,500 3.39 -> 3.20, 2,100 / 256 1.30 -> 1.23. The output hashes
(NEWHASH) match with STRATA_PA_FAST=1 and =0. gfx12 compiles the same
change; it was not run there.
2026-10-14 07:07:13 +02:00
Andy Slater db0b4d315c sycl/A770: wide MMVQ kernels at 2-8 columns in large-GRF mode on Arc A-series
The wide kernels (Q6_K a2 and the generic native_mmvq_wide) spill at 2-8 columns with 128
registers per thread on the A770. They now launch with grf_size<256> there. Large GRF halves the
threads resident per Xe core, so it is the default on Alchemist only (strata::intel_gpu_gen);
Battlemage and other devices, and 1 column on every device, launch the same kernels as before.
STRATA_MMVQ_LARGE_GRF=1 forces it on, =0 off.

mmvq_bench now prints a checksum of the output (FNV-1a over the float bits) so the two modes can
be compared bit for bit.

Measured on an Arc A770 16 GB (xe, oneAPI 2026.1 JIT), same day. mmvq_bench Q6_K, 2560x2560 / 2560x10240 /
10240x2560, 1-8 columns: 1 column unchanged; 2 columns 1.3-2.0x, 4 columns 2.3-3.1x, 6 columns 3.0-5.0x, 8 columns
3.7-7.4x faster (e.g. 2560x10240 at 4 columns 289.3 -> 92.9 us, 74 -> 232 GB/s of weights); checksums identical
with STRATA_MMVQ_LARGE_GRF=0 and =1 in all 15 cases. End to end, a0116b27 + #2190 + #2193 + #2203 with and
without this commit, Coder IQ1_M, 128K context, int8 KV, --resident-experts, STRATA_VERIFY_STEPPED=1, 8K prompt,
256 tokens: decode 23.47 / 23.50 / 23.73 / 23.77 -> 25.99 / 27.10 / 27.49 tok/s (+10-17%), prompt unchanged,
output ids identical where the expert placement matched. Not built or run on Battlemage.
2026-10-14 07:07:12 +02:00
yi-sudo f629fb0417 Batch decode: one launch for all rows in QSA attention and GDN recurrence (opt-in)
With --batch N the verify window ran QSA block scoring, top-k and decode
attention as one launch per row, and the GDN recurrence (window and 1-row
commit) as one launch per slot.  At 4-8 live slots these serial launches are
most of the window's cost beyond the expert reads.

Two env switches, both off by default (off = the same launches as before):

  STRATA_BATCH_ATTN_ROWS=1   block scores, top-k and decode attention for all
                             rows of the window in one launch each; every row
                             reads its own slot's indexer keys and K/V pools
                             (passed by value as kernel params, so the graph
                             captured per row layout stays valid).  A slot that
                             streams KV (kv_mode 1) keeps the per-row path.
  STRATA_BATCH_GDN_GROUPS=1  GDN recurrence in the window and the 1-row commit
                             for all slots in one launch (grid.y = slot group).
                             Conv commits stay per slot.

Same arithmetic per row: 4 concurrent temperature-0 prompts give the same
bytes with and without the switches (--batch 4 and --batch 8).

RTX PRO 6000 Blackwell, IQ3_S, all experts in VRAM, a0116b2, same day,
total tok/s (docs/BATCHING.md has the table):
  --batch 4, 4 clients: 268.4 -> 311.7
  --batch 8, 8 clients: 308.4 -> 374.8 (reserve 6144 MiB)
Window: 4 rows 14.3 -> 12.2 ms, 8 rows 24.6 -> 19.9 ms.

Built and run with CUDA 13.0 (sm_120) only.  The .cu files are also in the
HIP build, which was not built here (no ROCm).  The SYCL port has its own
copies of these files and is untouched.
2026-10-14 07:07:10 +02:00
imanu 8e68aa5879 OPUS-MTP-MMVQ multi-colonna con attivazioni riusate fra le righe del blocco (STRATA_B6_MMVQ_ROWS) e micro-bench
Q4_K, Q5_K, Q6_K e IQ4_XS: il frammento di attivazione di ogni colonna si carica una volta per blocco
di pesi e serve tutte le righe del blocco; le scale Q6_K stanno in registro. Stesso ordine di accumulo e
di riduzione: uscite identiche bit per bit al layout esatto e alla chiamata a colonna singola
(b6_mmvq_bench, memcmp su entrambe le schede). Default spento.
2026-10-14 07:07:09 +02:00
Stuart Chapin 77ddc2ff14 cpu: IQ2_S in the bit-exact singleton rows; the singleton path only below the #152 rule
Stacked on #1415. IQ2_S (type 22, 122 gathered) joins STRATA_CPU_IQ_ALL_EXACT1's list: ggml's AVX2
iq2_s dot sums the same int32 lanes as row_dot_ggml_one and ends with 0.125f * hsum_float_8.
The singleton gate now also requires nt < native_gu_mt_min, so with STRATA_IQ_MT_MIN=1 a lone token
keeps the multi-token kernel and native_expert_parity's width invariance passes with the switch on.
2026-10-14 07:07:08 +02:00
leadgtr7 6befb3d65d sycl mmvq: Q6_K decode kernels no longer ALU-bound; output heads as aligned planes; int4 activation loads
- Q6_K wide kernel (and the planar one): the six-bit offset (v - 32) was dpct's per-byte saturating char4 subtract, a per-byte loop on Xe;
  no byte can saturate, so a 4-op SWAR form is bit-exact. Q6_K dense matvecs 59.5 -> 30.8 us (1 row), 77.4 -> 45.5 us (4 rows).
- STRATA_Q6_PLANAR (default on, =0 off): the Q6_K LM head and the MTP draft head are converted at load to ql/qh/scales/d planes (same bytes, the
  GGUF-layout allocation is freed: no VRAM change) so a lane's loads are single aligned 16-byte loads; the draft head subset is gathered from the planar
  head. native_q6_k_mmvq serves every registered matrix at any column count. Bitwise equal outputs.
- ld_i4_a4: the Q8_1 activation qs (4-byte aligned) as one dword-vector load in the Q4_K / Q5_K / IQ4_XS wide kernels (Q5_K 4 rows 81 -> 68 us).
2026-10-14 07:07:08 +02:00
Niko1221 db568409fc tools: batch_test.py takes up to 32 prompts (folds in havok2-htwo's batch_test16.py and batch_test32.py) 2026-10-14 07:07:06 +02:00
InB4DevOps 8ee2005dae cpu: add bit-exact AVX2 singleton expert rows 2026-10-14 07:07:04 +02:00
KarlGrier 1a2f95c332 QSA decode attention: split each query's cells across blocks on the int8 mma path (STRATA_QSA_SPLIT_ATTN=1, opt-in)
With STRATA_QSA_SPLIT_ATTN=1 the verify window's and the draft layer's QSA attention run through the prompt path's int8
tensor-core kernel with each query's cells split among up to 20 blocks, and the splits are merged with the log-sum-exp
rescale. qsa_decode_attn_split() returns false (nothing launched) unless the KV is int8 on sm_80 or newer, or when its
partials would not fit the caller's scratch, and on HIP; the caller then keeps qsa_decode_attn_batch(). The sums run in
another order, so with the switch on the output is a rounding-level change. Unset, nothing changes.

The idea is from Eddoursul's fork (fd23d91); the kernel is written against this tree's qsa_prompt_attn.cu.
2026-10-14 07:06:38 +02:00
KarlGrier ffc30b8d76 Verify window: the mixer's input-only work on graph branches (STRATA_SIDE_MIXER=1/2, opt-in)
STRATA_SIDE_MIXER=1 runs the GDN gates and the QSA indexer (key projection, tail snapshot, query chain, block scores,
top-k) on a branch of the window's graph beside the attention's projections, joined before the recurrence and the
attention; =2 also runs the GDN gate projection and the QSA queries' chain on a second branch. A branch that projects
reads F4's interleaved copy, written on the main stream before the fork, and qkv is recorded before the branch so its grid
is dispatched first. Every result is bitwise the same as in line; STRATA_DF_BRANCH is off while this is on. Unset, the
window runs in line as before.

The idea is from Eddoursul's fork (34a35db, b063c13).
2026-10-14 07:06:34 +02:00
imanu f7eaf2de9a Verify window: the routed experts' gate/up with the group's q8_1 rows in shared memory (STRATA_EXPDEC=1, opt-in, bitwise); RTX 2080 Ti gate/up kernel -11/-23% at T 1-4, decode +1.1% on a 131k layer split
native_expert_grouped already decodes each weight part once per block, but the apply re-reads every entry's q8_1 block from global memory with 9 scattered 4-byte loads, so an extra row of the window costs ~60% of a whole expert. expdec_gu_kernel stages the group's entries in shared memory (two conflict-free 16-byte loads per call, the fp16 scale as a float), takes rb rows per block and stages the IQ3 codebooks too; the same lane does the same calls in the same order with the same operations (apply_u = apply on the block's words), so the output is bitwise the default kernels' (expdec_bench: identical on every cell, 2 cards). Gate/up IQ2_XXS/IQ2_XS/IQ3_XXS/IQ3_S/IQ2_S at n_embd 2560; the down (Q2_0, IQ4_NL) behind STRATA_EXPDEC_DOWN=1 (in the engine the Q2_0 down was ~10% slower). Knobs STRATA_EXPDEC_R/_DR/_MINB/_RB/_DRB/_MINBLK/_D20; one log line per (role, format) at the first launch. docs/EXPERTDEC.md.
2026-10-14 07:06:21 +02:00
Alex Sadleir 79376a2732 cpu: the AVX2 expert-row dots keep their accumulators in registers too
q5_1_dot and q8_0_dot in kq_avx2.cpp held their per-token sums in the same
runtime-indexed __m256 acc[MAXT] that made bf16_rows_dot_multi spill: with the
array live the compiler leaves it in memory, and every token's FMA reads and
writes a stack slot.  Both are now template <int NT>, dispatched 1..4 and, past
that, run four tokens per pass with the row's bytes re-read from L1 - the same
idiom the router change above and kq256_gu_rows already use.

Ryzen 7 9800X3D (Zen 5), GCC 15.2.0, build/native_expert_parity --synthetic
(ggml-quantized weights and activations), one thread pinned to core 0, at the
verify window's four tokens:

  Q5_1 down rows [2560 x 640]: 190-197 us -> 155-156 us  (1.24x)
  Q8_0 down rows [2560 x 640]: 135 us     -> 104 us      (1.30x)

Same sums: 0 of 36 token-sets differ in any bit against ggml's own vec_dot over
1..8 tokens, both before and after; native_expert_parity reports 0 failures.

Left alone: q4k_dot keeps its runtime-nt array.  Its token accumulators are not
what spills - the unpatched function carries 8 ymm stack ops for the sums
against 24 (q5_1) and 59 (q8_0); what q4k_dot spills is its weight-side unpack
arrays (q4l/q4h/scale_l/scale_h), which token chunking does not touch and
re-does.  The same change measured +5% at four tokens and -4..-7% at six to
eight, so it is a wash at the window and a small regression above it, and q4k is
the gate/up format that runs most.  The AVX1 path, the iq_avx2/q2_avx2 rows and
the engine's dispatch are untouched.

Backends: CUDA and HIP do not compile this file.  The SYCL port does; not built
on this machine (no icpx here) - the router change in this branch built there.

Tests: native_expert_parity --synthetic q4_K/q5_1 and q4_K/q8_0 (0 failures,
0 bits differ), iq_avx2_parity, iq_avx1_parity, q8k_quant_parity,
router_dot_parity - all exit 0.
2026-10-14 07:06:19 +02:00
Alex Sadleir 2a669f3c29 cpu: the AVX2 router dot keeps its accumulators in registers (2.1x on a Zen 3)
bf16_rows_dot_multi held its per-token sums in __m256 acc[MAXT] indexed by a
variable. With eight 256-bit accumulators live GCC also unrolled the column loop
and left the whole array in memory, so every token's FMA read and wrote a stack
slot (vmovaps 0xc0(%rsp) ... vfmadd213ps 0xe0(%rsp)). On a Ryzen 7 5700X3D one
layer's router [512 x 2560] at 4 tokens took 0.41 ms against 0.25 ms for the AVX1
copy in kq_avx1.cpp that kept its accumulators in registers, so every AVX2 CPU ran
the slower of its two kernels (router_dot_parity --cpu 2, median of 3). Pragma
GCC unroll 1 on the column loop changes nothing; the variable-indexed array is
what forces the array into memory.

Four tokens per pass, in four named accumulators. A row above four tokens is read
a second time, from L1: nt=8 measures 0.37 ms, the same as the spilled form. On
the same CPU the AVX2 kernel goes 0.41 -> 0.19 ms at 4 tokens and nt=2 0.36 ->
0.16 ms, and it now runs 1.3x ahead of the AVX1 copy instead of 1.6x behind it.
The loop body carries no stack slot any more (was 21 memory ops per iteration).

Untouched: the arithmetic and the reduction. Each token's FMA chain still runs
over the same columns in the same order and ends in the same hsum_float_8, so
every sum is bit-identical - the output for nt 1/2/4/8 at cols 2560/5120 hashes
the same before and after (FNV-1a over the whole out[]), and router_dot_parity
reports 0 failures with the AVX2 rel error unchanged at 3.16e-07.

Left alone: the expert row kernels (iq_avx2.cpp, q2_avx2.cpp, kq256_gu_rows,
kq256_rows) and the AVX1 path. The iq_avx2_parity bench over four format pairs x
1..8 tokens moved within 1.6% (noise), and the CPU parity tests (8 targets) pass.
The router stays what it was, an estimate for which experts to prefetch. CUDA and
HIP do not compile this file; the SYCL port does, and it builds there under icpx
2026.1 with no new warnings and no spills either.

The AVX1 copy in kq_avx1.cpp kept a comment saying it reads its rows "exactly as the AVX2
kernel does"; with the row loop chunked four tokens at a time that is no longer the
AVX2 kernel's shape, so the clause is dropped there.  Comment only - no code change.
2026-10-14 07:06:18 +02:00
imanu 731528beba QSA decode attention: PR #540's two-cells-per-step chunk kernel on every CUDA card and a bounded merge (STRATA_ATTN_DAILY=1, opt-in, bit-exact); parity harness
attn_chunk_kernel_pre75 (PR #540 by sskver: two cells per step with a 12-shuffle reduction, cell-major
probabilities, V loaded four at a time) is compiled in every CUDA build, not only with STRATA_EXPERIMENTAL_SM60, and
STRATA_ATTN_DAILY=1 selects it on any card for the decode attention (single query and batch: verify window and MTP
drafter) together with attn_merge_v2_kernel (Hardin22), a merge that skips the chunks the chunk kernel marks empty.
Bit for bit with the default kernels by construction: reduce12 adds the same pairs as warp_sum's butterfly at every
level, the V cells fold in the same order with the same fmaf, the masked cells stay out; the merge does the same fmaf
in the same order. Shared memory 15,872 B like the default kernel (spt shares sq's storage). STRATA_ATTN_MERGE_V2=0/1
and STRATA_ATTN_PRE75=0 switch the two halves separately. HIP unchanged.

qsa_merge_daily_parity (tests): memcmp of both kernels against the default ones on four KV formats, widths at the
chunk borders and non-resident pages; --bench times the chunk kernels back to back.
2026-10-14 07:06:09 +02:00
imanu 5b6c93987f QSA decode: the window's block scores with the queries in registers and six shuffles per block (STRATA_QSA_ROWS=1, opt-in, bitwise); RTX 2080 Ti 101 -> 55 us a call at T=4, decode +3.2/+0.1% on a 131k layer split
In a 131k decode block_scores_multi_kernel costs ~20 us more per call for every row of the verify window: not the keys (read once for all the queries) but the shared-memory and shuffle pipes, 4 x LDS.128 of the query slices and 20 shuffles per (block, query). block_scores_rows_kernel<NQ> keeps the NQ <= 4 queries' slices in registers and reduces the four heads' lane products with a scatter that keeps warp_sum's pairings (xor 16 and 8 leave one head per lane, then 4, 2, 1): every addition adds the same two operands of the same level, so scores and top-k ids are bitwise the multi kernel's. qsa_rows_bench measures the window's QSA kernels row by row and, with --rows, checks the parity (16 cases per card, 0 differ) and prints both kernels' microseconds: 2080 Ti 47/57/78/101 -> 36/38/49/55 us at T 1-4, 3060 73/109/152/194 -> 63/76/98/130. docs/QSAROWS.md.
2026-10-14 07:06:00 +02:00
ATIVX928 f1a9eaa47e gdn chunk: the smem opt-in and the workspace are per device (layer-split fix)
justxiami reported (#1098) a 4xV100 layer split dying with
STRATA_GDN_CHUNK=1.  Reproduced on 2xV100: the second card launched the
chunked kernels at the 48 KB default (a process-global std::call_once set
the opt-in on the first card only) and the engine exited with
"gdn_chunk wmma: invalid argument".

Two more per-process assumptions in the same path:
- gdn_chunk_cc() cached the first card's capability for the whole process
- the chunked workspace was thread_local with no destructor: the per-request
  stage threads leaked ~50 MB per card per request (the second card ran out
  of memory on the next prompt) and one card's pointers could reach another

Now: the opt-in and the capability are recorded per device (each attempt
clears the runtime error, and a refused wmma opt-in falls back to the FMA
path), and the workspace is one per (thread, device) with a destructor.
Verified on 2xV100 32K: STRATA_GDN_CHUNK=1/2 run to completion over repeated
rounds (1: 1900 tok/s prefill, 2: 1766), gdn_chunk_parity 0 failures.
2026-10-14 07:05:19 +02:00
ATIVX928 fd13adf248 bench (v100): gdn_chunk_bench - the GDN prompt recurrence variants and the chunked path, real shapes
CUDA events over one gdn_recurrence call per side (S=128, 16 q/k heads, 48 v heads, C=10240,
T=1024/4096/8192), all four paths and the max abs diff between the chunked wmma output and the
pipelined recurrence.  RESULTS.md records the table and the error bounds for
bench/results/2026-10-02-gdn-chunk (measured on the V100-SXM2-16GB, CUDA 12.8, sm_70 build).
2026-10-14 07:05:18 +02:00
ATIVX928 c0a4a9e657 tests (v100): gdn_chunk_parity - the chunked and Volta GDN recurrences vs a double-precision host reference
GPU, synthetic, no model.  Compares through the real dispatch (STRATA_GDN_CHUNK /
STRATA_GDN_REC_WARP A/B) at T=64/128/200/320 - 200 exercises the < 64-token tail - and asserts
measured bounds, not equality: the recurrence is the FP32 floor (y abs 1.1e-6, state 3.0e-8), the
chain-split and the chunked FMA path are FP32-level (~3e-6), the wmma chunked path is the
f32->f16 staging level (y abs 2.3e-3, state 8.9e-5).  The sync-free warp recurrence is asserted
BIT-EXACT against gdn_rec_cols_pipe (memcmp of y and the final state).  A NaN or inf anywhere
fails the check outright - an unordered comparison would otherwise leave the max at zero.

On main's 0.1.38 recurrence the out norm keeps its raw FP32 output in place and writes only the
FP16 copy, so the kernel comparisons use that raw output (ref_full's yraw_out) and the norm is
checked once through its FP16 store.
2026-10-14 07:05:18 +02:00
ATIVX928 7c4ff2de72 prefill (v100): the chunked GDN prompt recurrence (FlashQLA gdn-chunk) + Volta recurrence variants
The prompt GDN recurrence is a per-token chain; on a V100 that chain is what the 36 GDN layers
cost.  This adds the chunked form (cumsum / kkt / fwd over 64-token chunks, chunk-serial scan,
the state in registers - the gdn-chunk-sm70 structure of llama.cpp-v100, hand-written CUDA):
a chunk is a triangular solve and a batch of matmuls instead of 64 sequential rank-1 steps.
Two variants: an FP32-only FMA scan and an m16n16k16 wmma one with the reference's f32->f16
staging.  Both are FP32-level, NOT bit-exact (the sums are reordered; the wmma path stages
q/k/v and the state through f16) - gdn_chunk_parity bounds them.

Also two Volta variants of the recurrence itself: a sync-free warp-shuffle reduction (BIT-EXACT
vs gdn_rec_cols_pipe, but measured 2.6x slower - kept as the bitwise control) and a 4-accumulator
chain split of the pipelined kernel (FP32-level, measured +6-7% at T=4096/8192, the V100 default
now).  All of this is compiled only into the STRATA_EXPERIMENTAL_SM60 build and dispatched only
at cc==7.0; sm_80 and up keep the trunk's kernel bit-for-bit.  The env switches
(STRATA_GDN_CHUNK=1/2 chunked, STRATA_GDN_REC_WARP=0/1/2) still override.

Measured on the V100 (bench/gdn_chunk_bench, real shapes, one gdn_recurrence call): the chunked
path is correct but 4-6x SLOWER than the recurrence as implemented (4-warp CTAs underfeed the
tensor cores - see bench/results/2026-10-02-gdn-chunk/RESULTS.md), so it stays opt-in.
2026-10-14 07:05:18 +02:00
ATIVX928 d284809fb0 V100: operator fusions for the decode GEMV / MoE paths, bitwise
Fusions (all bit-exact vs the pipelines they replace; bench/micro/fusions_parity.cu
checks every one bitwise and times the before/after at decode shapes):

- gate/up dual GEMV + SwiGLU in ONE kernel (llama.cpp's mul_mat_gated):
  s_gemv_pair_silu, one read of the activation for both projections and the
  SwiGLU in the epilogue - the up intermediate never lands in memory.  Rows are
  accumulated by the shared row functions (s_rowdev.cuh) exactly as the
  standalone kernels accumulate them.  shared_expert uses it whenever both
  projections are S-family (bitwise checked for all three silu kinds).
- silu(gate)*up + the intermediate quantizer fused (was 2 kernels):
  swilu_quantize_q8_0/_scaled/_q8_K, warp-parallel swilu then the verbatim
  quantizer body.  Wired into the S2 expert entry points and shared_expert.
- every activation image of x in one pass (was 2-3 kernels): quantize_act_images
  produces block_q8_0 + block_q8_K + bf16 from one read of x.  Wired into
  gdn_layer, qsa_layer and moe_shared.
- multi-weight bf16 GEMV (llama.cpp's mul_mat_multi): bf16_gemv_pair computes
  two matrices from one read of x; used for ssm_alpha/ssm_beta and for the
  indexer's q/k projections (q and k now one launch).

The call sites are gated on STRATA_EXPERIMENTAL_SM60 (the Pascal/Volta build),
so the ready-made engine runs exactly the pipelines it did before; the fused
kernels are the same code either way.

Measured (V100, CUDA events, n_embd 2560 / FF 640, 200 reps):
  expert intermediate (swilu+q8_0)   15.7 -> 11.6 us  M=1,  18.8 -> 11.4  M=8
  activation images (q8_0+q8_K+bf16) 46.2 -> 30.4 us
  dual GEMV + swilu (gate,up)        30.4 -> 14.8 us
  bf16 pair (alpha,beta)             24.2 -> 14.4 us  M=1,  23.7 -> 14.1  M=8

Parity: fusions_parity 0 failures (bitwise), plus s_gemv/s_gemv_q8k/s2_gemv_q8/
s2_expert_grouped/shared_expert/quantize_act/bf16_gemv/elementwise/mmvq_multi/
iq_multi/native_expert parity suites all pass unchanged.
2026-10-14 07:05:13 +02:00
ATIVX928 1e14e6469e kernels (sm_70): the prompt attention's latency pipeline (mma884 kernel)
Port of oldlocal commit 4672cf9 ("kernels (sm_70): the prompt attention's
latency pipeline - 2.2x over the FP32 kernel at 32K") onto the #600
prompt_attn_v70_kernel, the m8n8k4 Volta kernel now inline in
qsa_prompt_attn.cu.  Still gated on STRATA_EXPERIMENTAL_SM60, so the
ready-made engine keeps exactly the code it had.

Two of the commit's three changes port; the third (splitting the dependent
WMMA chains into f0/f1 and fh/fl) is already inherent to the mma884 kernel -
its scores run two tg[rb] chains and its p.v four tmp[rb][nt] chains - so
porting it would only change the summation order.  The two ported, measured
on a V100-SXM2-16GB (CUDA 12.8):

* the staging loop issues every global load of a thread before any of its
  stores (2-4 in flight), instead of paying the HBM latency per piece;
* the next chunk's ids/page-table lookups double-buffer (row[2][CH]) and run
  just before the previous p.v, where their latency hides.

The original commit measured, on the #371 kernel, 2,048 queries of a 32K
selection: 34.4 -> 15.8 ms (2.18x) int8, 33.5 -> 15.4 ms (2.17x) fp16 KV,
35.6 -> 20.3 ms (1.76x) K8V4 (its q4_0 V dequantizes to FP16 at staging, the
sm_75/80 kernel's same trade).

This port, same 2,048 queries of a 32K selection (qsa_prompt_attn_sm70_bench,
which also gained the K8V4 pools): the Volta kernel goes int8 12.11 -> 10.25 ms,
fp16 KV 25.70 -> 17.39 ms, K8V4 25.71 -> 24.12 ms, so against the FP32 decode
kernel int8 rises 1.73x -> 2.05x and fp16 0.80x -> 1.18x (K8V4, 0.85x -> 0.91x,
keeps main's #600 dequant-to-FP16 trade).

qsa_prompt_attn_parity passes all four cases.  An A/B against the unpatched
kernel shows the kernel's output is unchanged - every FP64 and new-vs-old
figure is identical before and after (int8 5.08e-6, fp16 5.29e-6 vs FP64,
within the 4x gate).
2026-10-14 07:05:12 +02:00
ATIVX928 132faae7cf kv_q8: the gather decodes 8 int8 per thread (bitwise, +11% on the 32k microbench)
The experimental build (-DSTRATA_EXPERIMENTAL_SM60=ON, Pascal/Volta) gains a
kv_gather_q8_wide_kernel: one uint2 (8 B) load of codes, one uint4 (16 B) store of
the dequantized fp16 and one scale per thread instead of two.  The 8 values lie in
one 64-value group and the arithmetic is unchanged ((float) code * scale ->
f16_from_f32), so the output is byte-identical.  The ready-made engine keeps exactly
the 4-wide kv_gather_q8_kernel and launches it unchanged, so its gather is bit for
bit the release engine's.  On a device below sm_75 the wide kernel is selected at
run time inside the experimental build (STRATA_Q8_GATHER_WIDE=0 opts out).

Measured on 2x Tesla V100-SXM2-16GB (SM 7.0), CUDA 12.8, one card: +11.4% on the
32k gather microbench (15.85 -> 14.05 us/call, 401 -> 452 GB/s).  kv_q8_parity
stays OK at worst INT8-vs-FP16 error 0.590 quantization steps.
2026-10-14 07:05:04 +02:00
ATIVX928 6a5be385c6 audit: where the int8 KV read decodes, and whether it is vectorized 2026-10-14 07:05:04 +02:00
Niko1221 8ccc4a0fe6 Verifier::ar_live and no Prefill::cuda_err_ in the hand-off worker (the two pieces of the unported STRATA_PIPE_OVERLAP series that STRATA_ADMIT_MERGE_FIRST and STRATA_HANDOFF_PIPE name) 2026-10-14 07:03:56 +02:00
havok2-htwo ed4fa42a80 serve: STRATA_ADMIT_SHORT_FIRST=<tokens> admits short prompts before long ones (opt-in)
New requests took the control lines in whatever order the lock fell, so a 500-token chat question could wait
behind several agent reads. With the env, a prompt of at most that many tokens goes first (arrival order within
each class); a longer one that waited STRATA_ADMIT_MAX_WAIT_S (30 s) counts as short. A request already in
acquire() gives the lines back when a shorter one came. Only the order changes, never what is read.

Ported from havok2-htwo/Strata 08830ddf onto 0.1.44.1: _take_control has no image-request line here, so its
gate is only the ordering; the first admission passes new=True.
2026-10-14 07:02:42 +02:00
Niko1221 ed7c7f339d engine: a pack folder without native_experts.txt says so before the AVX-512 refusal (#2347)
The refusal 'this CPU cannot run the expert kernel' belongs to the canonical Q2_0 pack. An NVFP4, i-quant or
Unsloth pack is a native pack (native_experts.txt in the --pack folder) and its rows have AVX2 kernels, but a
config whose --pack points at the wrong folder reads like a CPU that is too old. On a CPU without AVX-512 the
engine now says the file is missing, and where it sits when it is one level down (<pack>/pack).
2026-10-14 07:02:11 +02:00
havok2-htwo 1d9758f897 prefill: STRATA_HANDOFF_PIPE=P hands a layer split's chunk on in P pieces, gated by events, on a persistent thread
A layer split hands each prompt chunk's rows (21 MB per 512 tokens) to the next card through a pinned
host buffer: one D2H, a host stream sync, a new std::async thread, then one H2D before the next stage's
first kernel.  The 3090s here have no P2P (topo: CNS) and cudaMemcpyPeerAsync is driver-staged and
slower (3.4 ms vs 3.2 ms for D2H + H2D), so the copy stays on the host route; nsys showed 3.8-4.0 ms per
512-row hop, of which ~1-1.4 ms was the thread start alone.

With STRATA_HANDOFF_PIPE=P (2..16, opt-in; unset = unchanged):
- the sending stage records an event when its chunk's work is done and one after each of P row-piece
  downloads into the same pinned buffer, and waits on the host only for the first (on_stage_chunk's
  checkpoint parts and the error check need the state, not the copy);
- the next stage waits on the GPU for each piece's event (cross-device cudaStreamWaitEvent) before that
  piece's upload, so its uploads overlap the remaining downloads (separate PCIe links); a smaller
  receiving chunk waits for every piece, then copies as before;
- the next stage's chunks run on one worker thread per sending stage, started and pinned once, whose
  packaged_task futures carry the same results and exceptions as std::async's (it clears the CUDA error
  state after each job, as a thread's end did).  STRATA_HANDOFF_PIPE_WORKER=0 keeps std::async (A/B).

Same bytes in the same rows before the next stage's first kernel: exact.  No device memory (events only,
the existing two pinned buffers per stage).  Measured (T16-143l5, P=8): 512-row hops 3.8-4.0 -> 1.85-1.89
ms, 143-row hops 1.6 -> 1.0 ms; reads 1.2K about -1.5 %, 150 about -1 %, 9K (agent, --kv-resident) -3 %;
decode rows/s unchanged.  batch_test16/32 (greedy and temperature 0.7), batch_interleave_test and
batch_overlap_test (+ the stage fault arm) byte-/token-identical to the reference dumps; an agent-config
run with checkpoint parts byte-identical to the reference binary.
2026-10-14 07:00:13 +02:00
havok2-htwo 6959adf404 tests: qsa_slots_parity skips where a slots launcher declines by design
STRATA_SCORES_MULTI=0, the pre-sm_75 attention (STRATA_EXPERIMENTAL_SM60 builds on a
V100/P100) and the gfx906 query swizzle make qsa_block_scores_slots or
qsa_decode_attn_slots decline; the engine keeps its per-row calls there, so a decline
is no failure. A declined case prints SKIP, the test exits 77 when every case was
declined, and STRATA_SCORES_MULTI=0 skips up front.
2026-10-14 07:00:13 +02:00
havok2-htwo a71ee46cd2 verify: STRATA_QSA_SLOTS=1 runs a batch window's QSA block scores, top-k and attention as one launch over its rows
In a batch window every row reads its own slot's indexer keys and K/V, so each QSA layer ran the
selection and attention once per row: four rows (16 slots) or eight (32 slots) times qsa_block_scores,
qsa_block_topk and qsa_decode_attn_batch, each launch small (the top-k on one CTA, the attention on
cap/64 x 2 blocks).  With the switch, per window (up to 8 rows per launch):

- block_scores_slots_kernel: grid (256, rows), blockIdx.y runs the single-query block_scores_multi_kernel
  body with that row's slot keys (block_scores_multi_kernel is now a wrapper around the same body)
- qsa_block_topk over the rows in one call (it only reads the rows' scores and steps)
- attn_chunk_slots_kernel: attn_chunk_kernel's body with the row's own QsaAttnPools (an 8-entry table
  passed by value, blockIdx.z picks), then the existing batched merge

Each row is computed by the same kernel body with the same reduction order as its own call.  The
launchers decline (the per-row calls run) where that call would pick another kernel: STRATA_SCORES_MULTI=0,
the pre-sm_75 / gfx103x attention, the gfx906 query swizzle.  A streamed KV cache keeps the per-row
resolve-then-attend loop.  Default (switch off): unchanged.

Checked on 4x RTX 3090, T16-143gsc setup (layer split 12,24,37, int8 KV):
- SASS (sm_86) of every existing kernel in qsa_select.cu and of the default attn_chunk_kernel<M> and the
  merge is identical before/after; the opt-in lane-cell instantiations differ only in register allocation
- new qsa_slots_parity (ctest, also with STRATA_ATTN_LANECELL=1): fp16, int8, q4_0 and int8-K/q4_0-V
  pools, 1/2/4/8 rows of different slots with their own page tables and contexts 37..32768: scores, ids
  and attention bitwise identical to one call per row
- batch_test16/32 (greedy and t=0.7), batch_interleave_test and batch_overlap_test (STRATA_PIPE_OVERLAP=1)
  token-identical to the build without the change; with the switch off batch_test16 is byte-identical too
- batch decode rows/s (engine level, 30 s, back to back with the reference): 16 slots 683 -> 726 (+6.3 %),
  32 slots 750 -> 802 (+6.9 %); STRATA_VERIFY_PROFILE "QSA projections + attention" per stage-window
  0.89 -> 0.52 ms (16) and 1.52 -> 0.79 ms (32)

HIP: the new code uses only what the existing kernels use; SYCL has its own sources and only sees the new
declarations.  Neither was built here.
2026-10-14 07:00:13 +02:00
Niko1221 3bfa088e46 setup: Qwen3.6 offers --parallel on small cards too (the price said: a request alone ~5% slower, 4 together ~1.5x), and asks how many requests at once; Enter keeps one at a time 2026-10-14 07:00:10 +02:00
havok2-htwo 2f4fe20a9a serve: STRATA_SLOT_CKPT_MOVE honours the #879 flush (no resume from a tainted slot's checkpoint)
Upstream's non-finite flush only marks running slots `tainted`; their checkpoints stayed.  With
STRATA_SLOT_CKPT_MOVE a running slot's checkpoints are a resume source (active_source) and a slot's
moved turn checkpoint can be restored into the main session (lent_slot), so a request right after the
flush, logged as "reading this prompt from token 0", still continued from a pre-flush checkpoint.

- the flush drops a running slot's checkpoints too (only with STRATA_SLOT_CKPT_MOVE, the one reader of
  them; they go back to the checkpoint pool)
- active_source and the lent_slot search skip tainted slots

Checked with STRATA_NAN_INJECT=600 on the T16-142gsc setup (next turns of chats still running in
slots): before, the two requests after the flush resumed 1628 tokens from slots 0 and 8 ("gave back ...
its turn checkpoint"); now both read 1661 tokens from token 0.  A next turn before the flush still
resumes from its running slot.  Default path (switch off) unchanged.
2026-10-14 07:00:09 +02:00
havok2-htwo f32663a8b3 tools: batch_interleave_test runs on a pipelined batch (no give-way required there) and checks a turn checkpoint taken from a slot
On a layer split with more than one group a batch read never gives way to a
BYIELD, so step 4 always failed with 'no such free slot' and steps 5-7 never
ran.  --yield auto now expects the give-way only on a non-pipelined batch;
there D is read whole and E admitted beside it, both still equal to solo.
New step 7b: A ends in slot 1, --flush unrelated solo GENs, then F2 can only
start from slot 1's turn checkpoint.  --dump writes every token list for an
A/B of two builds.
2026-10-14 07:00:09 +02:00
havok2-htwo bec9edf1e0 prefill: decide STRATA_MMQ_GROUP_SORT per layer once, in the MMQ plan
mmq::whole_tiles walks every visible GPU and every tile width through the
MMQ config table; the sort hook called it twice for each MoE layer of each
prompt chunk, on the host path between the routed-ids download and the
expert launches.  Its verdict depends only on the layer's weight formats and
the device set, so MmqPlan now keeps it as a per-layer flag (taken only when
STRATA_MMQ_GROUP_SORT=1 and STRATA_MMQ_RESIDENT_SORT_NE is unset) and logs
how many layers qualify.  Same order, same bits; without the switch nothing
new runs.
2026-10-14 07:00:09 +02:00
havok2-htwo 69a05bfb59 verify: STRATA_BATCH_SELF_COMMIT=1 lets a one-row-per-slot batch window keep its rows itself
A pipelined batch window is followed on its stage's stream by the layout's commit graph, and the stage counts
as done only after both. When every row of the window is another slot's one token, that commit is
unconditional (each slot keeps its one row; commit_slot_prefixes allows nothing else), and most of the
graph only repeats what the window wrote: it puts each indexer tail back from its snapshot and appends the
same row again, and copies each PLE history back from the window's own snapshot of it. What it adds is the
GDN conv history and recurrence state, which the window ran without writing.

With the switch, such a window advances the GDN conv history (gdn_conv_l2_multi with commit) and state
(gdn_step_norm_multi with n_keep = 1, the kernel the commit graph runs for a one-row group) itself, as a
one-token solo window does, and takes no tail or history snapshots. No commit graph is captured for the
layout; batch_launch and commit_slot_prefixes launch none, layout_captured and ensure_batch_layouts do not
ask for one. Layouts with proposal rows (--batch-mtp) keep the commit graph. With the switch off every path
is unchanged.

Measured on 4x RTX 3090, UD-IQ4_XS, T16-142fcb (split 12,24,37), decode of 16 / 32 slots, 30 s each,
reference and new build back to back: 653.7 -> 686.0 rows/s (+4.9 %) and 714.1 -> 752.6 rows/s (+5.4 %);
the group step period 24.5 -> 23.3 ms and 44.85 -> 42.6 ms. STRATA_VERIFY_PROFILE: the window's GDN
recurrence takes 0.09 ms (16) / 0.18 ms (32) longer per stage-window with the state written in it.

batch_test16 (16 slots, a 1,200-token prompt) is token-identical to the reference dump; batch_test32
32/32 identical and its dump equal to the switch-off run; batch_overlap_test (STRATA_PIPE_OVERLAP=1) passes
and is token- and RESUME-identical to the switch-off run; batch_interleave_test parts 1-3 identical (its
part 4 fails at the same point with the reference build).
2026-10-14 07:00:08 +02:00
havok2-htwo 62fc7c8a3c prefill: STRATA_MMQ_GROUP_SORT=1 sorts the full MMQ groups by row count, keeping the partial group
STRATA_MMQ_RESIDENT_SORT_NE=1 puts a fully resident layer's experts in row-count order, so each
16-expert group's max_rows (its tile padding) is close to its experts' own. It also moves the hottest
experts into the last, partial group, whose few tiles take MMQ's stream-k split: the K boundaries move
with the group's tile count and the prompt's bits change.

The new switch sorts only order[0, n - n % 16] (mmq_resident_sort_span) and leaves the partial group's
experts, and so its rows, where they were. A full group's launch has 160 (gate/up) or 320 (down) tiles
per column tile; mmq::whole_tiles checks on every visible GPU that every such launch runs one block per
tile (tile efficiency >= 90 %, NVIDIA) and that all tile widths share the I/K tiles and SRAM layout. Then
each expert's rows get the same bits in any full group whatever its group-mates and max_rows. Where it does
not hold (e.g. 108 or 128 SMs, AMD's stream-k grid) the switch does nothing. RESIDENT_SORT_NE takes
precedence when both are set; with neither the code path is unchanged.

prefill_mmq_tab_test part 4 runs a layer's groups as prefill.cpp does (src-gathered q8_1 rows, absolute
bounds, the group's max_rows) in ID order and in both sorted orders, for IQ3_S / IQ4_XS gate/up and
IQ4_NL / Q8_0 down: with the partial group kept every token's rows are bitwise equal (16 cases); sorting
the partial group too changes 133-1881 rows per case. 890 comparisons, 0 failed (RTX 3090).

Measured on 4x RTX 3090, UD-IQ4_XS, T16-142fcb (split 12,24,37, every expert resident), read time,
medians of 6, back to back: 150-token prompt 145.8 -> 136.2 ms, 1,200-token prompt 428.6 -> 363.4 ms
(RESIDENT_SORT_NE: 134.9 / 358.6 ms). The final residuals of five prompts (300-1,200 tokens,
STRATA_PREFILL_DUMP_R_ALL every position and STRATA_PREFILL_DUMP_R fp32) are byte-identical to the
reference build; with RESIDENT_SORT_NE every row differs. batch_test16 (16 slots, a 1,200-token prompt)
is token-identical to the reference dump.
2026-10-14 06:59:57 +02:00
havok2-htwo a923889bd2 generate: review fixes for the admission switches
STRATA_SLOT_CKPT_MOVE: an active slot is a slot source only when the main
drafter's rows come along or are not drafted from (no --mtp, --batch-mtp,
or a batch admission).  The slot copy takes the main layers' cells only, so
a solo GEN beside active slots would draft from another conversation's
drafter rows below its start; it reads its prompt as with the switch off.

STRATA_CKPT_THREADS: a checkpoint thread widens its CPU mask before it
goes to the aux CPUs, so the mask the aux placement restores (a request
with aux_cpus=0) is the wide one, not the host thread's single CPU.

STRATA_ADMIT_MERGE_FIRST: a merged first window counts as one row in the
STRATA_DECODE_TIMING window statistics (its prompt rows are not decode rows).
2026-10-14 06:59:57 +02:00
havok2-htwo ca239cdabc generate: STRATA_SLOT_COPY_BATCHSYNC=1 queues the direct slot copy and synchronizes each device once
The direct slot copy (STRATA_SLOT_COPY_DIRECT) waited for each of its ~116 copies on the private copy
stream.  With the switch, conversation_checkpoint_copy and conversation_kv_copy only queue them (new
`queued` argument, default off), and the slot copy's closing sync_all is the one wait per device.  A
streamed K/V state (kv_mode != 0) has its residency refresh on the legacy stream, which is not ordered
after a non-blocking stream: conversation_detail::order_after_queued records an event on the private
stream and makes the legacy stream wait for it first.  The copies stay on the private non-blocking stream.

Same copies in the same order on one stream: tokens are identical.  Measured on 4x RTX 3090 (T16-142,
24 admissions): the slot copy takes 0.5 ms instead of 1.3 ms; batch_test16 16/16, batch_overlap_test
and batch_interleave_test token-identical to the references; a --kv-resident 20480 run with a 21K
prompt is identical with the switch off and on.
2026-10-14 06:59:57 +02:00
havok2-htwo 139d702ebd generate: STRATA_SLOT_CKPT_MOVE=1 moves a batch admission's turn checkpoint into the slot
A batch admission copied the main chain's longest checkpoint into the slot (~112 MB of host vectors, 7.6 ms, ~60 ms
on fresh pages).  With the switch, the request's own turn checkpoint (or one it took out of the slot it is admitted
to) moves instead; a root, a pinned prefix and any other checkpoint the chain shares are still copied.

The chain then no longer holds it, so the request path also looks in the slots:
- while the main session holds its tokens (live, its cells below it), a slot's checkpoint is restored on the main
  session as a chain one would be, with no K/V copy; when that slot is the one the request is admitted to, it moves
  back into the chain first (the admission drops the slot's checkpoints), else the slot keeps it and the admission
  copies it as it copied the chain's;
- an active slot's checkpoints are a slot source like an idle slot's (its windows write its cells from p on): a
  second request with the same history after the main session moved on restores instead of reading it all;
- a slot source that is the admit slot moves its checkpoint into the chain instead of copying it.

The switch also keeps the dropped checkpoints' storage as STRATA_CKPT_POOL=1 does: a moved checkpoint frees
nothing at the next prune, so without it every save faulted in fresh pages (17 -> 38-60 ms, more than the copy saved).

Off by default: nothing changes.  A session SAVE or a parked conversation right after an admission lacks the moved
turn checkpoint (the slot holds it).
2026-10-14 06:59:25 +02:00
havok2-htwo bebeb791a4 generate: STRATA_CKPT_POOL=1 saves checkpoints into dropped ones' vectors, STRATA_CKPT_THREADS=1 saves a split's parts at once
STRATA_CKPT_POOL=1 (off by default): the serve loop keeps the running-state
vectors of the checkpoints it drops (LRU eviction, the tail one, the prune at
a new request, a batch slot's old one), at most two, and the next checkpoint
or slot copy goes into them.  A save then resizes nothing: no zero-fill of
~118 MB of fresh vectors.  Every byte is overwritten by the copy.

STRATA_CKPT_THREADS=1 (off by default): a layer split's checkpoint saves the
first stage's part on the calling thread and every later stage's on a
persistent thread of its own (the copies' private streams are per thread, so
short-lived threads would leak them).  The threads leave the host thread's one
CPU first (to the aux CPUs with --aux-cpus, else to every CPU the process may
use): on it the copies only took turns.

Measured on a 4-stage split (T16, 24 batch admissions per arm): a turn
checkpoint takes 17.2 ms today, 10.4 ms with the pool and 5.6 ms with both;
the batch tests, the overlap and interleave tests and a driver run give the
same tokens with the switches on as off.
2026-10-14 06:59:13 +02:00
havok2-htwo 0cf1a9b7c8 generate: STRATA_ADMIT_MERGE_FIRST=1 reads a batch admission's last prompt tokens and its first token in one window
A batch admission read its header through the verify windows (T=4 + T=2 for the
usual 6 tokens) and then its first token in a window of the last prompt token
alone. With the switch the last windowed part leaves its last tokens (up to the
verifier's max_t, 8, the last prompt token included) to the first window: the
part's earlier tokens are read as before, the first window commits every row,
the drafter gets the rows of the prompt tokens in it as the prompt windows give
them, and only row T-1's output is emitted. Row T-1 sits at the first window's
position with the same history and Philox draw, and a row computes what it
computes in any window while every expert is in VRAM, so the token and the
state are the same.

Only for a BGEN admission with every expert of every stage in VRAM (no loan,
no swap pending, the zero-doorbell graphs), and not with --batch-mtp, the
pipelined windows, pictures, the elastic K/V, a peer device, remote caches,
foresight swaps, STRATA_LOGPOS or a checkpoint at the last token. The merged
window takes no drafter q (STRATA_SPEC_PROB), adds nothing to the draft
counts, and the "PP <n-1>" line is printed before it. Verifier::max_t() is new.

Off by default: the same path as before. Tokens (4x RTX 3090, split
12,24,36, 16 slots, STRATA_IQ_MT_MIN=1): 24 admissions of prompts of 3 to 65
tokens and chat headers with and without an assistant prefill (merged T 3 to
8), greedy, seeded and with penalties, on vs off identical; batch_test16 16/16
and equal to the earlier dump; batch_overlap_test token- and RESUME-identical;
batch_interleave_test equal; 47 sampled driver requests equal.

Measured (24 admissions, ~131 and ~1195 tokens): header windows 23.5 plus
first window 11.5 ms become one 7-token window of 27.6 ms, 7.4 ms less per
admission (send to first batch token 246.8 -> 239.4 ms short, 656.1 -> 648.5
long). A window row costs ~2.4 ms up to T=6 and ~4 ms at T=7 and 8 (T=1 11.4,
T=7 28.6, T=8 32.1 ms).
2026-10-14 06:58:52 +02:00
havok2-htwo a62bd38eba generate: STRATA_PIPE_BYIELD_NODRAIN=1 drops a BYIELD without draining the pipelined groups
A pipelined read never gives way (batch_read_piece returns 0 when piped), so a
BYIELD that reaches the main loop is for a read that has ended and is dropped.
The main loop still drained the groups for it first, as for a request, unless
STRATA_PIPE_OVERLAP was set. With the switch the BYIELD is dropped as it comes.

A BYIELD the server sends during the read is queued and taken right after BADM,
before any group is restarted, so that drain costs nothing today (0 groups in
flight, 0.0 ms). Only a BYIELD that arrives once the groups run again (the
server reads the control lines late) paid for it: measured with one sent 60 ms
after BADM, drain 26 ms with 4 groups in flight and the first rows 50 ms after
it, a 23.5 ms gap in the slots' rows and about 14 rows lost (at 576 rows/s). With
the switch the longest gap is 7.1 ms, as in plain decode.

Off by default: the same path as before. Tokens: batch_overlap_test (a BYIELD
during B's read) on vs off token- and RESUME-identical, batch_interleave_test
output byte-identical, batch_test16 16/16, and 41 sampled requests of the
admission driver (8 with a BYIELD while 4 groups were in flight) equal.
2026-10-14 06:58:39 +02:00
havok2-htwo bc761d06b1 generate: STRATA_TRACE times a batch admission's phases
One stderr line per batch admission ("strata trace: admission of slot N: ...")
splits it into the drain before BGEN, the prompt (batched read, header windows,
the checkpoints' device syncs, copies into RAM and eviction, the rest), the first
window, the slot copy, freeing the slot's old state and the copy of the turn
checkpoint into the slot. Each checkpoint gets its own line, and with pipelined
groups "first rows N ms after the admission" times the refill after BADM. A drain
for another line says how many groups were in flight and what it cost.

Only with STRATA_TRACE set; without it the log and the tokens are unchanged.

Measured (4x RTX 3090, split 12,24,36, 16 slots in 4 groups, 24 admissions):
drain 25-26, header windows 23.5, checkpoint 17.3 (sync 0.1, save 17.2), first
window 11.5, slot copy 1.3, slot freed 0.0, checkpoint copy 7.8 (60 on a slot's
first fill), refill after BADM 23.7 ms.
2026-10-14 06:58:16 +02:00
havok2-htwo 6ab4c31243 generate: more than 8 batch slots are held to 8 rows per group window once the slot sessions that fit are known
The >8-slot cap ran only before the slot sessions were carved. When fewer
sessions fit than asked for, the groups were resolved again with no row bound:
auto could pick a group of more than kVerifyMaxT slots (PGroup's rows overrun,
every window refused) and a --batch-groups N that no longer divides the slots
fell back to one serial group whose windows reach only the first 8 slots.

slots_in_windows() picks the most slots whose groups, resolved as
batch_groups::resolve does (a number given, auto, the default's one group for
--adapt-async 1), divide them with at most kVerifyMaxT rows each and at most
one group's worth per stage. It runs before the groups are resolved and again
on the slots that fit (the extra sessions are dropped like the per-stage trim);
the pump refuses a group over kVerifyMaxT rows as a safety net. The hand-off
buffers hold one row per slot only for pipelined groups, so --batch-mtp keeps
upstream's kVerifyMaxT rows. Log and comment wording without the g3 tags.
2026-10-14 06:57:02 +02:00
havok2-htwo 574dcbf9c3 docs: STRATA_MMQ_DIRECT needs -DSTRATA_MMQ_TAB=ON; its VRAM and build cost; the hack is no bound; engine exactness
- The build option is off by default and why (36-38 MiB of VRAM per GPU under EAGER, ~70% of the upstream MMQ
  instances' compile time per arch); the nine types with a pointer table.
- The skip-gather hack's 225/577 ms are no bound: it changes the routing and does less work.
- Engine exactness as measured: first-window logits (STRATA_DUMP_FIRST_LOGITS) and greedy tokens byte-identical,
  2 vs 1 vs 0, on the 4-GPU split with --prefill 2048 (the streamed walk), with 9 GB less expert cache per GPU
  (mixed in-place, copied and mid-group-flushed groups), and on one GPU with --expert-cache-per-layer.
2026-10-14 06:56:59 +02:00
havok2-htwo 02d0c23a59 docs: STRATA_MMQ_DIRECT (opt-in) and the STRATA_MMQ_TAB build option, with the measurements 2026-10-14 06:56:59 +02:00
havok2-htwo a2d979b999 prefill mmq tab: moe_mmq_tab.cuh carries ggml's MIT notice
The file mirrors parts of llama.cpp's ggml-cuda mmq.cuh; like the other
ggml-derived kernels (native_moe.cu etc.) it now includes the copyright and
permission notice the MIT license asks for. Comment only.
2026-10-14 06:56:49 +02:00
havok2-htwo c2b44ca5b8 expert cache: slot_guard() reports the guard applied; a test for slot_tail on uniform and sized caches
- ExpertCache::slot_guard() returns the guard open_sized applied (guard_, reset by close), not the request: 0 for
  uniform slots, when closed, and on SYCL, whose open_sized applies none.
- tests/core/expert_cache_slot_guard_test: slot_stride, slot_room and slot_tail on a uniform cache (empty off_,
  including the last slot) and on sized caches with and without the 256 B guard; no guard gives today's offsets;
  close and reopen.
- generate: the "+256 B guard per sized slot" line only for a sized cache; a uniform one says it has no guard.
- prefill: STRATA_MMQ_DIRECT_LOG=1 (with STRATA_MMQ_DIRECT=1) prints each stage's totals of experts read in place,
  resident and streamed after every prompt, so an engine run can show which walk read what.
2026-10-14 06:56:49 +02:00
havok2-htwo a5ad4077ef prefill mmq tab: STRATA_MMQ_TAB off by default, the nine tested types only, not on gfx906
Review fixes for the pointer-table MMQ build:
- STRATA_MMQ_TAB defaults to OFF, like STRATA_MMQ_KQUANTS: the engine loads every kernel at start
  (CUDA_MODULE_LOADING=EAGER), and the nine tab instances took 36-38 MiB of VRAM per GPU (sm_86, measured with
  a cudaMemGetInfo probe linked with and without the tab objects; equal under LAZY), with STRATA_MMQ_DIRECT unset
  too. prep_group_kernel is compiled only with STRATA_MMQ_TAB, so a build without it has the same device code as
  3adce85 (the 58 cubins and PTX of build/strata are byte-identical to engine/strata-g3b's).
- The fingerprint block and the tab TUs are skipped on STRATA_HIP_GFX906: that build compiles no .cu from here, so
  the define, the uncompiled TUs and an unlinkable prefill_mmq_tab_test no longer appear.
- Tab instances only for Q2_0 IQ2_XXS IQ2_XS IQ2_S IQ3_XXS IQ3_S IQ4_NL IQ4_XS Q8_0, the types prefill_mmq_tab_test
  checks bit for bit; Q5_0 and the K-quants (STRATA_ORCA_Q4KS_MMQ, STRATA_MMQ_KQUANTS) keep the gather.
- overread_bytes takes the largest over-read of any config, not the over-read of the largest K tile (the over-read
  is not monotonic in the tile length).
- The test fails (instead of skipping) when a tested type has no tab, checks that Q5_0/K-quants report none, and
  pins the sm_86 over-reads (0, 72, 136 B); the dead K-quant scale branch is gone.
- SYCL's moe_mmq.dp.cpp gets the STRATA_MMQ_DIRECT stubs (false/0), for a future migrated prefill.cpp.
- moe_mmq.hpp points at docs/DETAILS.md.
2026-10-14 06:56:49 +02:00
havok2-htwo 0f10feb4d2 prefill_mmq_tab_test: pad the identity row map for memcheck
MMQ loads a whole tile's ids, and its stream-k fixup one per column tile of max_rows, past an
expert's last row (upstream behaviour on both paths; the values are not used). With the map
padded, compute-sanitizer memcheck runs the whole test clean: 0 errors, 874/874 bitwise equal.
2026-10-14 06:56:49 +02:00
havok2-htwo 20f75a7474 prefill mmq tab: the table as 16 restrict kernel parameters, so the weights keep LDG.CONSTANT loads
A pointer taken from a dynamically indexed array parameter (mmq_x_tab by value) made nvcc copy
the table to local memory and read the weights through generic LD.E loads instead of
upstream's LDG.E.*.CONSTANT: the products ran 1.2-2.5x slower than upstream's in the unit test
(Q8_0 down 2.2-2.6x, IQ4_NL 1.7-2.1x, IQ3_S 1.2-1.35x), which ate most of the gather saving.
The kernel now takes x0..x15 as `const char * __restrict__` parameters (like upstream's x)
and picks an expert's with a select chain: same SASS load mix as upstream (no local memory),
kernel time within 1% of upstream over every type and J in prefill_mmq_tab_test (69.7 vs
69.3 ms in total), and still 874/874 comparisons bitwise identical.
2026-10-14 06:56:49 +02:00
havok2-htwo 7d9b50dc1a prefill: STRATA_MMQ_DIRECT=1 reads resident experts in their cache slots (pointer-table MMQ)
mmq_plan marks a native layer direct when the pointer-table MMQ covers both products, gate/up
reads nothing past its matrix, the down over-read (72 B IQ4_NL, 136 B Q8_0) fits the 256 B
guard, and the blob is exactly [gate | up | down] in MMQ's matrix sizes; it says once how many
layers qualify.

In the MoE walk (both the staged and the streamed walk) compute() takes the expert's cache slot:
a resident expert whose slot has room for the over-read (slot_tail) and a 16-byte aligned blob
is not gathered - the group's pointer tables point at its gate|up and down in the slot, and the
guard goes into the group's GuardTab. Streamed, peer and unguarded experts are gathered as
before, at compact positions c = 0..ns-1 (gather_native, or gather_native_group with its
mid-group flushes unchanged); the end-of-group flush runs even when the last entry was direct.
At the group end one prep_group launch zeroes the direct experts' guards and the tails after
position ns, replacing the two memsets; both products run with the same n, bounds, ids,
max_rows and buffers plus .tab. STRATA_MMQ_DIRECT=0 (default) and 2 run the existing body.

PrefillStats::experts_direct, said on the generate summary when non-zero.
2026-10-14 06:56:48 +02:00
havok2-htwo b82cdb1090 expert cache: a slot guard for STRATA_MMQ_DIRECT (set_slot_guard), the sizing sites use slot_stride
ExpertCache gets inline set_slot_guard / slot_guard / slot_stride / slot_room / slot_tail; the
4-argument open_sized declaration is unchanged (the SYCL engine defines it out of line against
the same header and ignores the guard). open_sized lays slots out at slot_stride(blob, guard);
blob_ stays the largest blob, so fills, verify_slot and the HIP staging read no further.
slot_tail works on uniform caches too (through slot_offset).

generate.cpp: slot_guard = Prefill::slot_guard_bytes() (mmq::direct_slot_guard(): 256 when
STRATA_MMQ_DIRECT is 1 or 2 on a pointer-table build, else 0) for native packs; the seven
round-256 slot costs (split-skip-if-fits, split placement, per-layer sum, profile sizing,
shrink_to, cache_bytes, stage sizing) use ExpertCache::slot_stride(b, slot_guard), and the
primary and stage caches get set_slot_guard before they open. Guard 0: the same arithmetic as
before, so the default layout is unchanged byte for byte.
2026-10-14 06:56:15 +02:00
havok2-htwo 86f677b4a9 prefill mmq: pointer-table MMQ (STRATA_MMQ_TAB), prep_group, the bitwise test
A Strata-side copy of llama.cpp's MMQ outer scheduler (mul_mat_q, launch_mul_mat_q,
mul_mat_q_switch_J; fallback=false only) in src/prefill/moe_mmq_tab.cuh: each expert's base
address comes from a 16-entry table passed by value instead of x + z * stride_channel_x.
The launch geometry (J, tiles, stream-k grid and k split, fixup) is computed from the same
mmq_args, and process_tile / load_tiles / vec_dot / write_back / the fixup kernel are
upstream's own templates, so the results are the same bits. Instantiated per type from
mmq_tab_instance.cu.in in strata_mmq (same flags); moe_mmq.cu sees declarations only.

CMake turns it off unless the SHA-256 of mmq.cuh's mirrored region (mul_mat_q_process_tile
through mul_mat_q_case) is a verified one: 3cf03257 (the fetched pin; the vendored tree's
mmq.cuh is byte-identical). HIP and SYCL get nothing; Product::tab is a default-null field.

Also: overread_bytes, tab_fits, direct_mode (STRATA_MMQ_DIRECT, read once, 0 unless the tab
is built), direct_slot_guard, and prep_group (one launch: zero the direct experts' guards and
the two group tails). Nothing calls them yet; the default path is unchanged.

tests/cuda/prefill_mmq_tab_test.cu (STRATA_BUILD_TESTS): 874 comparisons on an RTX 3090, all
bitwise identical - IQ3_S gate/up, IQ4_NL and Q8_0 down (1..16 experts, rows
{0,1,7,8,9,31,64,100,512,1000}, every J 8..128, stream-k fixup on and off, max_rows above the
largest count), every other instanced type, mixed direct/gathered groups (compact c != q,
mid-group flush, last entry direct); negative control: guards left as fp16 NaN give non-finite
down outputs.
2026-10-14 06:56:14 +02:00
havok2-htwo dd5443646c batch: device-to-device slot copy for admissions and slot resumes
The main sessions and the batch slot sessions of a stage live on the same device and are
carved alike, so copy_to_slot / copy_from_slot (without a checkpoint) now copy the running
state and the QSA K/V directly (conversation_checkpoint_copy, conversation_kv_copy:
the snapshot code's private non-blocking copy stream, one device sync before and after) instead of the
checkpoint/K-V image round trip through host memory with a sync per layer. Same bytes into
the same places in the same order; the residency refresh for streamed K/V is shared with
conversation_kv_restore. Opt-in: STRATA_SLOT_COPY_DIRECT=1 (default: the host path,
as upstream).

4x RTX 3090, UD-IQ4_XS, --batch 16, split 12,24,36 + --trim-stage-weights: admission copy
65.6 -> 34.0 ms average (pure copy ~8 ms); tools/batch_test16.py 16/16 and
batch_interleave_test.py (incl. next turn from its slot) identical to solo.
2026-10-14 06:55:40 +02:00
havok2-htwo ee2a907e30 batch: more than 8 slots with a layer split (8 rows per pipeline group), hand-off sized per slot
Measured on 4x RTX 3090 (UD-IQ4_XS): 16 and 32 slots decode bit-identical to solo
(tools/batch_test16.py / batch_test32.py); 16 slots 603 tok/s batch decode vs 345 with 8.

Rebased on 0.1.42: the stages' batch row check (batch_rows_error, upstream 48f2c47d) bounds
hbase + S by the hand-off rows; set_handoff_rows passes the per-slot size there (only above 8 slots).
2026-10-14 06:55:26 +02:00
Niko1221andDemetree Kallergis b8ec9bbf94 telemetry: the Mac's Monitor cards - GPU load and VRAM from IOKit, the rest said out loud
On a Mac the Monitor tab's GPU cards were empty: telemetry had NVML (NVIDIA) and the amdgpu
sysfs files (#301) and nothing else, so the Metal backend read nothing.  The Apple GPU's load
and memory are in the IOKit registry's IOAccelerator nodes (ioreg -l -w0 -r -c
IOAccelerator, 22 ms), and the device's memory budget is the same Metal property the engine
reads (metal/src/mtl_rt.mm's recommended_working_set, the engine's "GPU working set" log line
- 26,800,603,136 bytes here, which is 25.0 GiB).

_Metal (a new reader beside _Nvml and _Amd): the index'th accelerator node's PerformanceStatistics
give Device Utilization % (older chips: the renderer or the tiler) and the in-use system
memory; the total is the working set, read through the Metal framework and objc_msgSend (no
PyObjC).  The temperature and power sensors are not readable
without root (powermetrics), and an Apple Silicon GPU sits on the die, so there is no PCIe
link: the cards show a dash and static gpu_note says why, the same way #1380's note does for a
Windows AMD card.  gpu_reader takes `amd` first, so the AMD trees still read on a Mac for the
tests, and a missing selector is asked for before it is sent (an unanswered one raises an
Objective-C exception no ctypes call can catch).

Service.free_vram_mib now goes through gpu_reader instead of _Nvml directly, so the Mac gets
that reading too (it used to return None there).

Measured on an M1 Max (macOS 26.6.2): through the server, GET /metrics during a request -
gpu_name "Apple M1 Max", gpu_vram_total_mib 25559, gpu_util 74-77 % while decoding (9 % once
idle), gpu_mem_used 10.5 GiB in use of 24.96 GiB, cpu 12.6 %; the plain-text
/prometheus-format metrics carry the same series.  serve/test_server.py: MacTelemetry
(fixture registry: two accelerators, index selection, renderer/tiler fallback, a raising
runner, the note and the samples) plus AmdTelemetry, TelemetryNoDisks, Monitor and the full
serve/test_server.py (318 tests) pass on the same box.

What it leaves alone: the NVIDIA and AMD paths (a test-only gpu_reader order change on
macOS), and the web app - the cards read the series it already had.

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
nekomario28 be225ae20c setup: honour --source on a Mac, whose hand-over skips setup.py's parser (#2363)
setup.py returns from mac_setup() before its own argparse runs, so a Mac never
sees the --source that sets STRATA_SOURCE, and the Mac parser did not define it
either: the flag the download failure recommends landed in parse_known_args'
leftovers and was dropped with "not used on a Mac".

Define it in the Mac parser and set STRATA_SOURCE from it, mirroring setup.py.
2026-10-14 06:52:38 +02:00
Niko1221andDemetree Kallergis 2e98a86344 mac docs: the two prompt opt-ins an M1 Max cannot run (measured)
Recorded with the hardware they were measured on, per the measured-numbers rule, because
neither can be fixed here: mm_gu3's fault is a threadgroup-count threshold of the
cooperative-tensor path on this GPU (the fix shape would change the M5's register-operand
GEMM), and router_route's race has no acquire atomic to fix it with on this Metal
(device-scope atomics are relaxed-only, macOS 26.6). Both are opt-in and off by default;
the engine's M1 defaults are the v2 expert GEMMs and the separate gemv + moe_route, which
run on this box.

Measured on an M1 Max (macOS 26.6.2, Q2_0): mm_gu3 with 35 threadgroups passes and 40
faults (data-independent; the engine's tile 16x32x32 SG 4 and the v3 test's 190 tiles both
fault). router_route's logits are bitwise gemv's, but two identical runs give different
ids, weights and plan.

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
Niko1221andDemetree Kallergis ad7345311a moe_mm_parity: the default tile that did not exist, and --v3's v1 argument list
Two stale defaults, both of which kept the test from covering what the engine ships.

The default tile was TM 16 TN 64 TK 16, but the v1 kernel set only instantiates TK 64: the
default run died at pipeline creation on any GPU (measured on an M1 Max:
"mtl: no function mm_gu_42_16_64_16_2"). TK is now 64, which both the v1 and the v2 sets
instantiate, so the default run and --v2 both work; the engine's v2 tile for Q2_0
(16,64,16,2) stays reachable with --tile.

--v3 bound the v1 kernels' argument list ({W, x, H, stok, tiles, ma}) to the v3 kernels:
tiles was read as the sorted-token buffer and MmArgs as raw int2 pairs, so every stride,
n_embd and expert id was garbage. On the M1 Max that is a deterministic GPU
command-buffer error in the GEMM pass; with the right arguments the v3 kernels run, and a
separate M1 limit shows (≥40 threadgroups of mm_gu3 fault, 35 pass, data-independent)
that the engine does not hit: gu_v3 = dn_v3 = coop_k4, which is false on the M1.
v3 now binds {W, XS, HS, tiles, ma} as the engine does, fills XS for v3 too (v3 is v2's
gather plus the register-operand GEMMs), and binds HS - not H - to down3 with DSG
simdgroups.

Measured on an M1 Max (macOS 26.6.2, Q2_0, T 256, 64 experts): default (v1) PASS, rel L2
2.53e-06 vs the fp16-operand reference and 2.37e-04 vs exact; --v2 PASS, 4.42e-06 and
2.37e-04, also with the engine's tile (16,64,16,2).

Leaves alone: no kernel, engine or other test change.

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
Niko1221andDemetree Kallergis 831ba47afd metal: the M1 Max prompt kernels - a select with a device load in the arm loses the threadgroup
qsa_attn_mm and qsa_flash (metal/kernels/qsa_prompt.metal) fill their single-simdgroup
matmul2d right inputs with

    rk[i] = id >= 0 ? half(float(kq[row * QD + k]) * float(ksc[...])) : half(0.0f);

and that shape - a select with the device load in the arm - loses the threadgroup on an M1
Max: every dispatch of the two kernels fails with
kIOGPUCommandBufferCallbackErrorInnocentVictim, deterministically, and they run only under
MTL_SHADER_VALIDATION=1. qsa_prompt_parity dispatches qsa_attn_mm the way the file intends
(it is the matrix-unit attention of a prompt chunk) and faulted on its first dispatch before
this change. The engine reaches the same fills through qsa_flash, which it dispatches under
STRATA_METAL_PROMPT_QSA=flash; its default on an M1/M4 is the fp32 qsa_attn, which never
touches these kernels - that is why the default prompt path kept running while the parity
test could not, and why this is measured on the test rather than on a request.

The row is now clamped with max(id, 0), the load is unconditional, and the select is on the
value. For an id >= 0 the value is what it was; for an id < 0 it was discarded either way, so
the results do not change.

Measured on an M1 Max (macOS 26.6.2, Metal compiler 32023): qsa_prompt_parity ALL PASS -
qsa_attn_mm vs qsa_attn rel L2 3.09e-04 (pos0 0) and 3.56e-04 (pos0 30000), qsa_flash 3.72e-04
and 4.22e-04, qsa_score_mm worst rel 2.66e-06 (T 96, within budget and with 2051 selected
cells a token); before the change the test faulted on its first dispatch and reported nothing.

The compile failure that started this - qsa_flash's cooperative_tensor copy, "cannot
initialize a parameter of type 'void *' with an rvalue of type 'const void *'" - is already
fixed in main (#2323); this change is the runtime half of the same two kernels' M1 Max
bring-up.

What it leaves alone: the COOP_K4 path (a function constant: compiled out of the M1/M4
pipelines), qsa.metal, and every host file; no CUDA, HIP or SYCL backend builds these kernels.
docs/MAC.md records the M1 Max with the measured tok/s (the hardware's medians, from the
benchmark report: chat decode 10.6, 4K prompt 223.7, 16K prompt 228.6 tok/s).

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
Kingdon Barrett f037612bad fix(metal): the prompt's buffers get their own allocation when no arena chunk is big enough for them
stream.cpp: the standalone-mmap fallback (the prompt's buffers as their own allocation) sat inside the chunk loop behind the slot-class check, so it only ran when a chunk was big enough for the prompt's 1.60 GB of buffers. When the arena shrank below that at the start (the 24 GB M2: chunks of 0.8-1.0 GB), every chunk was skipped and the prompt stopped with 'no arena region for the prompt buffers'. Both standalone sites - a fitting chunk whose slots a class cannot spare, and no fitting chunk at all - now share standalone_region(); the fitting-chunk carve and the slot-class rule are unchanged.

This is the half of #2323 that 0.1.44.1 could not see: the hotfix's reporter ran a 96 GB M2 Max, where the arena's chunks are several GB and the prompt region always fits. On a 16-24 GB Mac running --mmap-experts, 0.1.44.1 still stops at the first prompt with 'metal prompt: no arena region for the prompt buffers'; this makes the standalone allocation run there too.

Verified on a 24 GB M2 (macOS 26.6.2, Qwen3.8-Flash-Next GSQ-RCO-Q2_0 streaming): the fallback line appears where the error used to be, prompts run end-to-end (3.7-4.1 tok/s steady in web chat, no crashes). That run predates the standalone_region() refactor, which moves the same statements into one function both call sites use.

docs/MAC.md: the small-arena note says the prompt's buffers also get their own allocation when no chunk of the arena is big enough for them.
2026-10-14 06:52:38 +02:00
Tsury e69cbfa568 fix(serve): preserve explicit false against reasoning defaults 2026-10-14 06:52:38 +02:00
nekomario28 69fe40553d tools: poll the server address the config binds, not loopback (#2330)
strata_bench starts the server on the config's `host` but polled readiness at
a hardcoded 127.0.0.1, so a LAN bind (192.168.1.10 in the report) printed
"still loading" while the server was answering on that address.

Use the configured host, mapping the wildcard binds to loopback and bracketing
IPv6 literals; an explicit --url still wins. Patch from the report, with the
mapping lifted into server_base so a test can reach it.
2026-10-14 06:52:37 +02:00
nekomario28 a091cdf681 serve: average the throughput gauges over the window between scrapes (#2341)
/metrics/prometheus exposes llamacpp:prompt_tokens_seconds and
llamacpp:predicted_tokens_seconds as averages of the cumulative totals, so an
idle server keeps advertising the speed of its last request.

llama.cpp reads those two names from a bucket it resets on every scrape ("the
gauges are averaged over the window between two scrapes", server_metrics::bucket
in tools/server/server-common.h) and reports 0 when the bucket is empty, so a
llama.cpp dashboard on Strata froze at the last value where llama.cpp decays to
0. Keep a bucket per scrape and expose its delta.
2026-10-14 06:52:37 +02:00
Niko1221 db4dd65e56 Merge f-q36-vp (Qwen3.6 parallel requests and pictures) 2026-10-14 06:51:43 +02:00
Niko1221 1a900ae160 Merge f-14412 (setup: UD-Q5_K_XL metadata-only shard check, #2346) 2026-10-14 06:51:43 +02:00
Niko1221 6ca7e92968 setup: a metadata-only GGUF shard that ends at its header is whole (#2346: Unsloth's UD-Q5_K_XL 00001 is 10,946,618 bytes, its aligned data start 10,946,624) 2026-10-14 03:17:43 +02:00
Niko1221 95eefd51f0 tools/qwen35: q36_vision_proto says to start the engine without --mtp for its cache check 2026-10-14 02:40:00 +02:00
Niko1221 1e13cbc8a5 qwen35: keep the base engine's end-of-answer accounting (hybrid text identical), a stop keeps only x, the held tokens list every accepted draft
An earlier commit capped the accepted drafts at the end of an answer; that is an exact state, but it moved the hybrid mode's expert-slot
counts, so a sequence of requests with --mtp on a card that does not hold every expert differed from the base engine from the fourth
request on (found by comparing the 10 parity prompts against the base goldens on the 12 GB card).  Now the speculative step commits what
the base engine committed; a stop seen before the state moves keeps only x (the server's hand-over of a stopped request to a batch slot then
reuses the held state: its text equals the run alone), and the held tokens list x and every accepted draft, so a conversation that continues
from them matches the state exactly.  Setup says what parallel measured on the 12 GB card; docs/QWEN36.md updated.
2026-10-14 02:28:13 +02:00
Niko1221 a7027766ff docs/QWEN36.md: the long picture, two pictures, reads beside decoding slots, the hand-over; the picture comparison takes several pictures 2026-10-14 01:54:21 +02:00
Niko1221 296b59d2f7 Merge f-1441-perf (Responses/Codex honours reasoning_effort; setup keeps the installed size on a re-run) into f-1441 2026-10-14 01:51:12 +02:00
Niko1221 3921f57eea qwen35: a conversation's held tokens match its state exactly (no drafts past the end of an answer or a stop), reads give the decoding slots windows, pictures x2 check
spec_step takes no draft past an end-of-answer token or past the room max_new leaves, and a stop asked for before the state moves keeps
only x: every token emitted but the last is in the state, so a conversation that continues from the held tokens (the server's solo ->
slot -> solo hand-over, a chat's next turn) neither misses nor reads a token twice.  Before, an answer ending inside accepted drafts left
the state ahead of the held tokens (found by tools/qwen35/q36_serve_handoff.py: the first request's text now equals its run alone).
STRATA_Q36_READ_WINDOWS sets the decode windows of the other slots between two chunks of a prompt read (default 1).  The mtmd dump takes
several pictures; the engine logs each request's reuse; a 1,024-token test picture.
2026-10-14 01:47:11 +02:00
Niko1221 0ca1dbbaa2 pool_tasks_test: skipped on a CPU without AVX (its activation quantiser is the AVX one) 2026-10-14 01:19:42 +02:00
Niko1221 944c88314b Merge f-1441-mac (Metal: gdn_step fits 512 threads, every pipeline's thread limit checked, qsa_prompt compiles on the macOS 26.3 headers; #2323) into f-1441 2026-10-14 01:19:41 +02:00
Niko1221 dd9cbce1a2 Version 0.1.44.1: project version and the setup minimum engine 2026-10-14 01:19:41 +02:00
Niko1221 9aa674edf8 serve: the illegal-instruction note says what the CPU reports (a missing AVX/AVX2/FMA/F16C, or that it has them all and the engine is at fault) 2026-10-14 01:19:41 +02:00
Niko1221 d981a51013 serve: a crashed engine start is not retried as a GPU runtime start; the start error names how the process ended (signal / NTSTATUS) and the CPU's AVX flags after an illegal instruction (#2324) 2026-10-14 01:19:41 +02:00
Niko1221 0b6fc1cec7 nvfp4 CPU rows: no static initialisers in the AVX2/AVX-512 files (an illegal instruction before main on a CPU without AVX/FMA, #2324); check_isa_guard flags AVX-class static initialisers 2026-10-14 01:19:41 +02:00
nekomario28 ab638a5684 qwen35: widen F16 by hand so upload_f32 builds on CUDA 12.1
upload_f32 runs on the host and widens F16 with
__half2float(__ushort_as_half(...)), but __ushort_as_half is a __device__
intrinsic: nvcc 12.1 and 12.4 refuse the call ("calling a __device__ function
from a __host__ function is not allowed") and q36_engine.cu does not compile.

Widen the bits here instead, the way the neighbouring bf16_to_f32 already
does for BF16.
2026-10-14 01:19:41 +02:00
Niko1221 de9115fac7 setup: run again over an installed model keeps its size; the three env entries 0.1.44 made setup's own stay the user's in a GSQ-RCO config
0.1.44 made IQ3_S the recommended Qwen size on a PC with 64 GB or more (IQ3_S: 9 to 15% slower decode than IQ3_XXS, more RAM).
SETUP.bat / --setup on a folder that already holds a model of the family now pre-selects the installed size (and --yes takes it);
the menu marks it "(installed)" next to "(recommended)", every size stays a choice, a first install takes the recommendation.

SETUP_ENV gained STRATA_MMQ_KQ, STRATA_HC_Q8 and STRATA_ROUTE_TAIL_SKIP, which setup writes for Unsloth's files only; a setup
run again dropped a user's own value of them in any other config (a hand-set STRATA_ROUTE_TAIL_SKIP=0).  They are setup's only
in an Unsloth config.
2026-10-14 01:16:56 +02:00
Niko1221 5d08c753c2 serve: the config's reasoning_effort reaches the Responses API too; reasoning_effort false is a level; serve/test_effort_paths.py pins every request form's prompt
/v1/responses (Codex) read only the Chat settings shared with apps, so the config's "reasoning_effort" (#1641) never applied
there (it always thought at xhigh); Service.effort_default() is now the one place that says which level a request that names
none gets, for the chat, Messages and Responses routes.  openai_to_messages dropped "reasoning_effort": false (an `or` between
the two fields); effort_kwargs already treats false as thinking off.

The new test renders the prompt of every way a client sets the level (OpenAI reasoning_effort / reasoning.effort /
chat_template_kwargs, Messages API output_config.effort / thinking budgets at the 2048 and 8192 limits / disabled, Responses),
the config key, the shared Chat settings and their precedence, the web app's request shape, tools and a conversation.
2026-10-14 01:16:56 +02:00
Niko1221 cfc7686c00 tools/qwen35: the Qwen3.6 pictures and parallel-requests gates (engine protocol harness, solo / batch identity, protocol edges, llama.cpp picture comparison, throughput A/B) 2026-10-14 01:11:42 +02:00
Niko1221 a9adfa5dad Qwen3.6-35B-A3B: setup asks for pictures and recommends parallel requests; docs
Setup: the images question for qwen36 (GPU encoder on an NVIDIA card of 16 GB or more, the CPU encoder otherwise, AMD under Linux on the
CPU; --vision gpu kept as asked with a note), the pinned mmproj (size and SHA-256) beside the GGUF, the vision section and --vision in
the config; --parallel N is written as asked with what a slot costs, and the recommendation is said where every expert still fits.
docs/QWEN36.md: pictures and several requests at once with the measurements.
2026-10-14 01:11:11 +02:00
Niko1221 7419037255 qwen35: --batch-mtp (each slot verifies an MTP proposal in the batch window) and a clear refusal when the slots' caches cannot fit
A slot's window carries its next token and, for the greedy slots that fit beside one row per slot, the proposal of its MTP block;
the proposals for the next window come right after the window from its own hidden states (the first from the state the prompt
left).  Grouped GDN rows (a slot's rows in sequence on its own state), verify windows and their commit run per slot; every slot's
tokens equal its solo run.  The engine also says what the sessions take and stops with the numbers when they cannot fit.
2026-10-14 01:11:11 +02:00
Niko1221 b4ef48e941 qwen35: sessions, parallel requests (--batch N) and pictures (--vision, M-RoPE) in the Qwen3.6 engine
One conversation state per session; --batch N keeps N of them and decodes one token of each in a single window (rows of
several sessions in one forward, bit-equal to the solo rows), reads a prompt between the other slots' windows and takes a
conversation over from the session that holds its start.  GENI / BGENI pictures: the encoder's rows replace the embeddings
of the <|image_pad|> cells, positions as llama.cpp's mtmd (interleaved M-RoPE 11/11/10, text after a picture continues at
p + max(nx, ny)).  tools/qwen35/q36_mtmd_dump.cpp is llama.cpp's side of the picture check.
2026-10-14 00:46:16 +02:00
Niko1221 5ce999455b Metal: gdn_step fits its launch size, every dispatch checks maxTotalThreadsPerThreadgroup, qsa_prompt compiles on macOS 26.3 (#2323) 2026-10-13 23:21:28 +02:00
2333 changed files with 428131 additions and 11487 deletions

No files matched your search

+5
View File
@@ -17,6 +17,11 @@ __pycache__/
/mtp/
/models/
/Strata-data/
# the folders docker/entrypoint.sh creates in a mounted data/ (a link to llama.cpp, the models, the keys): never part of a build (#3644)
/data/install/
/data/state/
/data/home/
/data/Strata-data/
third_party/llama.cpp/
third_party/*.zip
*.done
+18
View File
@@ -4,9 +4,27 @@ bench/prompts/** -text
# launchers keep their platform line endings wherever the repository is cloned
*.sh text eol=lf
*.bat text eol=crlf
*.cmd text eol=crlf
*.ps1 text eol=crlf
bin/strata text eol=lf
# copies of real packs' native_experts.txt, parsed byte for byte by tests/core/expert_layout_test.cpp
tests/data/native_experts/** -text
# the quant yardstick's text set: tokenized byte for byte (set.ids.json), never convert its line endings
data/quant_quality/** -text
# the license text is shipped and checked as LF text on every checkout
LICENSE text eol=lf
# container scripts run in Linux: always LF
docker/*.sh text eol=lf
Dockerfile text eol=lf
docker/Dockerfile text eol=lf
docker/compose/* text eol=lf
# Apple tools (codesign, plutil) read these: LF only, wherever the repository is cloned
*.plist text eol=lf
# vendored KaTeX (serve/web/vendor): the files are hashed byte for byte (PROVENANCE.json), never convert their line endings.
serve/web/vendor/** -text
+26
View File
@@ -0,0 +1,26 @@
name: Benchmark
description: Share measurements from your machine, or report a problem with the benchmark tools
title: "[Benchmark] "
labels: ["benchmark"]
body:
- type: markdown
value: |
The title must start with **[Benchmark]**. Keep it, and add the GPU and model after it.
To share a full report, follow the [community benchmark guide](https://github.com/Niko1221/Strata/blob/main/docs/COMMUNITY_BENCHMARKS.md) and open a results-only pull request. The standard protocol is a [draft](https://github.com/Niko1221/Strata/blob/main/docs/BENCHMARK_PROTOCOL.md); existing tools do not implement it yet. Include poor results and failed requests.
- type: textarea
id: what
attributes:
label: What did you measure, or what went wrong
description: Hardware (GPU and VRAM, CPU, RAM, OS), Strata version, model and quantization, the exact command, and the numbers. Paste the report or link the pull request.
validations:
required: true
- type: dropdown
id: reproduction-status
attributes:
label: Reproduction status
options:
- Measured locally
- Unconfirmed (I did not run it)
validations:
required: true
+74 -10
View File
@@ -1,37 +1,101 @@
name: Bug report
description: Something in Strata doesn't work as documented
title: "[Bug]: "
title: "[Area] "
labels: ["bug"]
body:
- type: markdown
value: |
Thanks for the report.
Thanks for the report. Please read [CONTRIBUTING.md](https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md) first.
Please search the [open issues](https://github.com/Niko1221/Strata/issues?q=is%3Aissue+state%3Aopen) first — this may already be tracked.
Search the [open and closed issues](https://github.com/Niko1221/Strata/issues?q=is%3Aissue) first: this may already be tracked, and new evidence belongs in that thread.
Write only what you saw. Do not guess at steps, errors, hardware or severity.
**The title must start with its area:** [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other]. Choose the same one below.
- type: dropdown
id: area
attributes:
label: Area
description: Start the issue title with the same area in square brackets, for example "[Engine] decode stops after 2 minutes". Pick Other if none fits.
options:
- "[Engine] (speed, kernels, models, the server and API)"
- "[GUI] (the desktop app)"
- "[TUI] (the Terminal: strata launch, connect, monitor)"
- "[Benchmark] (measurements and reports)"
- "[Docs] (documentation)"
- "[Other] (anything else, including setup and installation)"
validations:
required: true
- type: dropdown
id: how
attributes:
label: Desktop app, Terminal or source?
options:
- Desktop app (Windows, Linux or Mac)
- Terminal (the `strata` command)
- Source (START-HERE.bat, setup.sh or a build)
- Docker
- Other
validations:
required: true
- type: textarea
id: what
attributes:
label: What happened
description: What you did, what you got, and what you expected instead. Paste the exact error text if there is one. If there's a set of steps or commands that reproduces it, put them here too.
description: What you did, what you got, and what you expected instead. Paste the exact error text if there is one.
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Smallest steps or command that reproduce it
description: The shortest list of steps that shows the problem. If you cannot make it happen again, say so here.
validations:
required: false
- type: dropdown
id: reproduction-status
attributes:
label: Reproduction status
options:
- Reproduced locally
- Unconfirmed static analysis (I read code or ran a scanner; I did not see it fail)
validations:
required: true
- type: input
id: version
attributes:
label: Strata version
description: "In the app: Settings > About. In the Terminal: `strata --version`. From source: the About tab of the web page, or `engine` in `GET /v1/status`, or the git commit."
placeholder: "Strata 0.2.7 · engine 0.1.52"
validations:
required: true
- type: input
id: hardware
attributes:
label: Strata version, GPU, OS
description: The engine version is on the About tab, or in `GET /v1/status` under `engine`.
placeholder: "0.1.31 · RTX 5070 12GB · Windows 11"
label: Graphics card, memory and system
description: The card and its VRAM, the RAM of the PC, and the operating system.
placeholder: "RTX 5070 12 GB · 64 GB RAM · Windows 11"
validations:
required: true
- type: input
id: model
attributes:
label: Model, size and settings
description: The model and size you started, the context, and whether pictures were on. Write what is relevant.
placeholder: "Qwen3.8-Flash-Next IQ2_XS · 128K context · pictures on"
validations:
required: false
- type: textarea
id: log
attributes:
label: Engine log
description: Drag in `strata-<model>.log` from your Strata folder. Drag and drop works directly into this box.
label: Logs
description: |
The app: Settings > Engine > Diagnostics > View logs > Save logs, then drag the file in. The Terminal: paste the output of `strata doctor`. From source: drag in `strata-<model>.log` from your Strata folder.
Remove API keys, passwords and private text first.
validations:
required: false
- type: markdown
value: |
A couple of extras that help a lot:
- Check the [troubleshooting table](https://github.com/Niko1221/Strata/blob/main/docs/DETAILS.md#troubleshooting) first, many reports are already solved there.
- Check the [troubleshooting page](https://github.com/Niko1221/Strata/blob/main/docs/TROUBLESHOOTING.md) first, many reports are already solved there.
+1 -1
View File
@@ -1,5 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Troubleshooting and known issues
url: https://github.com/Niko1221/Strata/blob/main/docs/DETAILS.md#troubleshooting
url: https://github.com/Niko1221/Strata/blob/main/docs/TROUBLESHOOTING.md
about: Symptom-to-fix table for crashes, out of memory, hangs, and port problems.
+27 -4
View File
@@ -1,18 +1,41 @@
name: Feature request
description: Suggest something new or an improvement
title: "[Feature]: "
title: "[Area] "
labels: ["enhancement"]
body:
- type: markdown
value: |
Explain what you want in plain language.
Explain what you want in plain language. For a large feature, a change to a default or a refactor, please agree the scope here first. Read [CONTRIBUTING.md](https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md).
**The title must start with its area:** [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other]. Choose the same one below.
- type: dropdown
id: area
attributes:
label: Area
description: Start the issue title with the same area in square brackets, for example "[Engine] decode stops after 2 minutes". Pick Other if none fits.
options:
- "[Engine] (speed, kernels, models, the server and API)"
- "[GUI] (the desktop app)"
- "[TUI] (the Terminal: strata launch, connect, monitor)"
- "[Benchmark] (measurements and reports)"
- "[Docs] (documentation)"
- "[Other] (anything else, including setup and installation)"
validations:
required: true
- type: textarea
id: idea
attributes:
label: What do you want to do?
description: Describe the goal rather than the implementation. If there's a workaround you use today, mention it.
description: Describe the goal rather than the implementation. Say who needs it and why.
validations:
required: true
- type: textarea
id: workaround
attributes:
label: What do you do today instead?
description: The workaround you use now, if there is one.
validations:
required: false
- type: markdown
value: |
All of this counts: model and architecture support, the server and API, the web app, multi-GPU and layer split, images and vision, MCP servers, setup and installation, and performance.
All of this counts: model and architecture support, the server and API, the app and the Terminal, multi-GPU and layer split, images and vision, MCP servers, setup and installation, and performance.
+26 -3
View File
@@ -1,11 +1,27 @@
name: Question
description: Ask how to use Strata, or share a result
title: "Question: "
title: "[Area] "
labels: ["question"]
body:
- type: markdown
value: |
Not sure how to do something, or want to share numbers from your machine? Ask here.
Not sure how to do something, or want to share numbers from your machine? Ask here. To share a full benchmark, see [CONTRIBUTING.md](https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md#share-a-benchmark).
**The title must start with its area:** [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other]. Choose the same one below.
- type: dropdown
id: area
attributes:
label: Area
description: Start the issue title with the same area in square brackets, for example "[Engine] decode stops after 2 minutes". Pick Other if none fits.
options:
- "[Engine] (speed, kernels, models, the server and API)"
- "[GUI] (the desktop app)"
- "[TUI] (the Terminal: strata launch, connect, monitor)"
- "[Benchmark] (measurements and reports)"
- "[Docs] (documentation)"
- "[Other] (anything else, including setup and installation)"
validations:
required: true
- type: textarea
id: question
attributes:
@@ -13,10 +29,17 @@ body:
description: What you're trying to do. Include your version, GPU, and OS if it affects the answer.
validations:
required: true
- type: textarea
id: tried
attributes:
label: What you tried
description: What you already did and where you got stuck.
validations:
required: false
- type: textarea
id: numbers
attributes:
label: Results (optional)
description: Only if you ran a benchmark or measured speed. Model, quant, context, and the tok/s you get.
validations:
required: false
required: false
+27 -9
View File
@@ -1,12 +1,30 @@
## Title
<!-- If Applicable, reference the GitHub issue -->
Issue: Resolves #
**The title must start with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], for example "[Engine] fix decode stop on long prompts" or "[GUI] keep the model list after a tab switch".**
## Summary
<!-- Quick Summary of changes -->
<!--
Read CONTRIBUTING.md first. Lead with the problem and the result, not a file-by-file list.
Placeholders below are not test results: replace them with what you observed, or write "none".
-->
## What changed
<!-- Specifics on files changed, and what changes were made there -->
## Problem and result
<!-- What triggered this change. How it behaves before and after. -->
## Extra Notes
<!-- Any extra notes, delete if there are none -->
## Related discussion
<!-- The issue URL and the maintainer's agreement on scope, when the change needs one (features, refactors, defaults). Write "none" if not applicable. -->
## Approach
<!-- Why this fix. Compatibility effects. Keep unrelated cleanup out of the PR. -->
## Validation
<!-- Exact commands, the revision and the environment tested, what you saw. Keep the three lists apart. -->
**Passed:**
<!-- command, revision/environment, observed result, log -->
**Failed:**
<!-- command and failure, or "none observed" -->
**Not run:**
<!-- check, why, and what stays unverified (for example CUDA, HIP, SYCL or Metal builds you could not do) -->
## Limits
<!-- Known limitations or remaining work. -->
+218
View File
@@ -0,0 +1,218 @@
'use strict';
// Format check for issues and pull requests (see CONTRIBUTING.md and .github/ISSUE_TEMPLATE/*.yml).
// Pure functions only: no network, no filesystem. Used by intake.yml and reopen.yml through reset-bot.js.
const AREAS = ['Engine', 'GUI', 'TUI', 'Benchmark', 'Docs', 'Other'];
const TITLE_RE = new RegExp('^\\[(' + AREAS.join('|') + ')\\] \\S');
const NO_ANSWER = /^(_no response_|none|n\/a|na|-|todo|tbd|\.|\?)$/i;
const NOT_RUN_LABELS = /^\*\*(passed|failed|not run)\s*:?\*\*\s*:?/gim;
// Version the reopened work must be tested on.
const CURRENT_VERSION_RE = /\b0\.2\.\d+|0\.2\.0-beta/i;
// Any version or commit.
const ANY_VERSION_RE = /\bv?\d+\.\d+(\.\d+)*(-[a-z0-9.]+)?\b|\b[0-9a-f]{7,40}\b|\bcommit\b/i;
const HARDWARE_RE = /\b(rtx|gtx|geforce|quadro|tesla|radeon|rx ?\d|instinct|arc|a\d{3,4}|b\d{2,3}|apple|m[1-5]( pro| max| ultra)?|nvidia|amd|intel|gpu|cpu|vram|\d+ ?gb|ram|windows|linux|ubuntu|debian|fedora|arch|macos|mac os|wsl)\b/i;
const REVISION_RE = /0\.2\.\d|beta|\bmain\b|\b[0-9a-f]{7,40}\b/i;
function stripComments(text) {
return String(text || '').replace(/<!--[\s\S]*?-->/g, '');
}
function hasAreaPrefix(title) {
return TITLE_RE.test(String(title || ''));
}
// Issue forms render as "### Label\n\nvalue". Returns { label: value } for those headings.
function parseSections(body) {
const out = {};
let cur = null;
for (const line of stripComments(body).split(/\r?\n/)) {
const m = /^#{1,6}\s+(.+?)\s*#*\s*$/.exec(line);
if (m) {
cur = m[1].trim();
out[cur] = '';
} else if (cur !== null) {
out[cur] += line + '\n';
}
}
for (const k of Object.keys(out)) out[k] = out[k].trim();
return out;
}
function realText(value) {
const t = String(value || '').replace(NOT_RUN_LABELS, '').replace(/\s+/g, ' ').trim();
return NO_ANSWER.test(t) ? '' : t;
}
function findSection(sections, re) {
for (const k of Object.keys(sections)) if (re.test(k)) return { key: k, value: sections[k] };
return null;
}
// Required parts of each issue form: [display name, heading regex, minimum characters of real text].
const FORMS = {
bug: {
detect: /what happened/i,
parts: [
['Area', /^area$/i, 2],
['Desktop app, Terminal or source?', /desktop app, terminal or source/i, 2],
['What happened', /what happened/i, 20],
['Reproduction status', /reproduction status/i, 2],
['Strata version', /strata version/i, 3],
['Graphics card, memory and system', /graphics card|memory and system/i, 3],
],
needsCurrentVersion: 'Strata version',
},
feature: {
detect: /what do you want to do/i,
parts: [
['Area', /^area$/i, 2],
['What do you want to do?', /what do you want to do/i, 20],
],
},
question: {
detect: /your question/i,
parts: [
['Area', /^area$/i, 2],
['Your question', /your question/i, 15],
],
},
benchmark: {
detect: /what did you measure/i,
parts: [
['What did you measure, or what went wrong', /what did you measure/i, 30],
['Reproduction status', /reproduction status/i, 2],
],
needsCurrentVersionInBody: true,
},
};
function checkIssue({ title, body, labels = [], mode = 'intake' }) {
const missing = [];
if (!hasAreaPrefix(title)) {
missing.push('Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] decode stops after 2 minutes".');
}
const text = stripComments(body);
const sections = parseSections(text);
const labelSet = new Set(labels.map((l) => String(l).toLowerCase()));
const needCurrent = mode === 'reopen';
let formKey = null;
for (const [key, f] of Object.entries(FORMS)) {
if (findSection(sections, f.detect)) { formKey = key; break; }
}
if (formKey) {
const form = FORMS[formKey];
for (const [name, re, min] of form.parts) {
const s = findSection(sections, re);
if (!s || realText(s.value).length < min) missing.push('Fill in "' + name + '".');
}
if (needCurrent && form.needsCurrentVersion) {
const s = findSection(sections, /strata version/i);
if (s && realText(s.value) && !CURRENT_VERSION_RE.test(s.value)) {
missing.push('Re-test on Strata 0.2.0-beta and put that version in "' + form.needsCurrentVersion + '". If it no longer happens, say so in a comment instead.');
}
}
if (needCurrent && form.needsCurrentVersionInBody && !CURRENT_VERSION_RE.test(text)) {
missing.push('Say which Strata version you measured on. Re-measure on 0.2.0-beta if you can.');
}
return { ok: missing.length === 0, missing, kind: formKey };
}
// Free-form text (older issues, blank issues).
const plain = realText(text);
const lenient = labelSet.has('enhancement') || labelSet.has('question');
if (plain.length < (lenient ? 40 : 80)) {
missing.push('Describe it in the issue body: what happened, what you expected, and the exact error if there is one. The issue forms (New issue > Bug report) list what to include.');
}
if (!lenient) {
if (!ANY_VERSION_RE.test(text)) missing.push('Add the Strata version (Settings > About, or `strata --version`).');
else if (needCurrent && !CURRENT_VERSION_RE.test(text)) missing.push('Re-test on Strata 0.2.0-beta and name that version in the issue. If it no longer happens, say so in a comment instead.');
if (!HARDWARE_RE.test(text)) missing.push('Add your graphics card with its VRAM, your RAM and your operating system.');
}
return { ok: missing.length === 0, missing, kind: 'free-form' };
}
function checkPull({ title, body, mode = 'intake' }) {
const missing = [];
if (!hasAreaPrefix(title)) {
missing.push('Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] fix decode stop on long prompts".');
}
const text = stripComments(body);
const sections = parseSections(text);
const problem = findSection(sections, /problem|result|summary|what.*change/i);
const validation = findSection(sections, /validation|testing|tests?\b|how.*tested/i);
let valText = '';
if (problem || validation) {
if (!problem || realText(problem.value).length < 20) missing.push('Fill in "Problem and result": what triggered the change, and how it behaves before and after.');
if (!validation || realText(validation.value).length < 20) {
missing.push('Fill in "Validation": the exact commands, the revision and environment tested, what you saw. Keep Passed, Failed and Not run apart.');
} else {
valText = validation.value;
}
} else {
const plain = realText(text);
if (plain.length < 150) {
missing.push('Describe the pull request: the problem and the result. The pull request template shows the parts (Problem and result, Related discussion, Approach, Validation, Limits).');
}
if (!/(problem|result|before|after|fix|because|bug)/i.test(plain)) missing.push('Add "Problem and result": what triggered the change, and how it behaves before and after.');
if (!/(validat|tested|passed|ran |built|build|not run)/i.test(plain)) missing.push('Add "Validation": the exact commands, the revision and environment tested, and what you saw.');
valText = plain;
}
if (mode === 'reopen' && valText && !REVISION_RE.test(valText)) {
missing.push('Say in Validation which revision you tested: rebase on main (or 0.2.0-beta) and re-run your checks.');
}
return { ok: missing.length === 0, missing, kind: 'pull' };
}
function checkItem({ isPull, ...rest }) {
return isPull ? checkPull(rest) : checkIssue(rest);
}
function checklist(missing) {
return missing.map((m) => '- [ ] ' + m).join('\n');
}
const CONTRIB = 'https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md';
function intakeComment(isPull, missing) {
return [
'Thanks for the ' + (isPull ? 'pull request' : 'issue') + '. To keep the tracker readable, every item needs an area prefix in the title and the parts of the ' + (isPull ? 'pull request template' : 'issue form') + ' filled in. This one is missing:',
'',
checklist(missing),
'',
'Edit the ' + (isPull ? 'pull request' : 'issue') + ' (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment `/reopen`.',
'',
'Details: [CONTRIBUTING.md](' + CONTRIB + ')',
].join('\n');
}
function reopenMissingComment(isPull, missing) {
return [
'Not reopened yet. Still missing:',
'',
checklist(missing),
'',
'Edit the ' + (isPull ? 'pull request' : 'issue') + ' and comment `/reopen` again. Details: [CONTRIBUTING.md](' + CONTRIB + ')',
].join('\n');
}
function reopenOkComment(isPull) {
return 'Reopened, thank you. A maintainer will pick it up from here.' + (isPull ? ' If it conflicts with main, rebase it first; that speeds up review.' : '');
}
function staleComment(isPull) {
return [
'Closing this ' + (isPull ? 'pull request' : 'issue') + ' because it still did not follow the format after 5 days. That is housekeeping, not a rejection.',
'',
'To bring it back: fix the title (area prefix) and the missing parts, then comment `/reopen`. A check runs and reopens it, or tells you what is still missing. See [CONTRIBUTING.md](' + CONTRIB + ').',
].join('\n');
}
module.exports = {
AREAS, TITLE_RE, hasAreaPrefix, parseSections, stripComments, realText,
checkIssue, checkPull, checkItem, checklist,
intakeComment, reopenMissingComment, reopenOkComment, staleComment,
};
+192
View File
@@ -0,0 +1,192 @@
'use strict';
const test = require('node:test');
const assert = require('node:assert');
const c = require('./item-check');
const bot = require('./reset-bot');
const bugForm = (over = {}) => {
const f = {
'Area': '[Engine] (speed, kernels, models, the server and API)',
'Desktop app, Terminal or source?': 'Terminal (the `strata` command)',
'What happened': 'Decode stops after two minutes with "CUDA error 700". I expected it to continue.',
'Smallest steps or command that reproduce it': '_No response_',
'Reproduction status': 'Reproduced locally',
'Strata version': 'Strata 0.2.0-beta · engine 0.1.45.3',
'Graphics card, memory and system': 'RTX 5070 12 GB · 64 GB RAM · Windows 11',
...over,
};
return Object.entries(f).map(([k, v]) => `### ${k}\n\n${v}`).join('\n\n');
};
test('title prefix', () => {
assert.ok(c.hasAreaPrefix('[Engine] fix decode'));
assert.ok(c.hasAreaPrefix('[Docs] x'));
assert.ok(!c.hasAreaPrefix('Engine fix'));
assert.ok(!c.hasAreaPrefix('[engine] fix'));
assert.ok(!c.hasAreaPrefix('[Engine]'));
assert.ok(!c.hasAreaPrefix('[Engine] '));
assert.ok(!c.hasAreaPrefix('[Area] foo'));
assert.ok(!c.hasAreaPrefix(''));
assert.ok(!c.hasAreaPrefix(undefined));
});
test('filled bug form passes in both modes', () => {
for (const mode of ['intake', 'reopen']) {
const r = c.checkIssue({ title: '[Engine] decode stops', body: bugForm(), labels: ['bug'], mode });
assert.deepStrictEqual(r.missing, []);
assert.strictEqual(r.kind, 'bug');
}
});
test('bug form: missing parts are named', () => {
const r = c.checkIssue({ title: 'decode stops', body: bugForm({ 'Strata version': '_No response_', 'What happened': 'bad' }) });
assert.ok(!r.ok);
assert.ok(r.missing.some((m) => /title/i.test(m)));
assert.ok(r.missing.some((m) => m.includes('What happened')));
assert.ok(r.missing.some((m) => m.includes('Strata version')));
});
test('reopen needs a 0.2 version, intake does not', () => {
const body = bugForm({ 'Strata version': 'Strata 0.1.44' });
assert.ok(c.checkIssue({ title: '[Engine] x y', body, mode: 'intake' }).ok);
const r = c.checkIssue({ title: '[Engine] x y', body, mode: 'reopen' });
assert.ok(!r.ok);
assert.ok(r.missing.some((m) => m.includes('0.2.0-beta')));
});
test('html comments and placeholders do not count', () => {
const r = c.checkIssue({ title: '[Engine] x y', body: bugForm({ 'What happened': '<!-- describe -->' }) });
assert.ok(r.missing.some((m) => m.includes('What happened')));
});
test('feature and question forms', () => {
const f = '### Area\n\n[GUI] (the desktop app)\n\n### What do you want to do?\n\nLet me pin a model to the top of the list in the app.';
assert.ok(c.checkIssue({ title: '[GUI] pin models', body: f, mode: 'reopen' }).ok);
assert.ok(!c.checkIssue({ title: '[GUI] pin models', body: '### Area\n\n[GUI]\n\n### What do you want to do?\n\n_No response_' }).ok);
const q = '### Area\n\n[Other]\n\n### Your question\n\nHow do I move the model folder to another drive?';
assert.ok(c.checkIssue({ title: '[Other] model folder', body: q }).ok);
});
test('free-form issue', () => {
const good = 'Strata 0.2.0-beta on an RTX 3060 12 GB with 32 GB RAM on Linux: the server hangs when I send two requests at once. Expected both to finish.';
assert.ok(c.checkIssue({ title: '[Engine] hang on two requests', body: good, mode: 'reopen' }).ok);
assert.ok(!c.checkIssue({ title: '[Engine] hang', body: 'it crashes', mode: 'intake' }).ok);
const old = good.replace('0.2.0-beta', '0.1.40');
assert.ok(c.checkIssue({ title: '[Engine] hang', body: old, mode: 'intake' }).ok);
assert.ok(!c.checkIssue({ title: '[Engine] hang', body: old, mode: 'reopen' }).ok);
assert.ok(c.checkIssue({ title: '[Other] idea', body: 'It would help if the app remembered my last model between runs, today I pick it again.', labels: ['enhancement'], mode: 'reopen' }).ok);
});
const prBody = (over = {}) => {
const p = {
problem: 'Decode stopped on prompts above 8k because the stop check read the wrong length. Now it continues.',
validation: '**Passed:** `ctest -R decode`, commit abcdef1, RTX 3060, 12 of 12.\n\n**Failed:** none observed\n\n**Not run:** HIP build',
...over,
};
return `## Problem and result\n<!-- hint -->\n${p.problem}\n\n## Related discussion\nnone\n\n## Approach\nFix the length.\n\n## Validation\n${p.validation}\n\n## Limits\nnone`;
};
test('filled PR passes', () => {
assert.deepStrictEqual(c.checkPull({ title: '[Engine] fix stop', body: prBody(), mode: 'reopen' }).missing, []);
});
test('untouched PR template fails', () => {
const tpl = '## Problem and result\n<!-- What triggered -->\n\n## Validation\n<!-- cmds -->\n\n**Passed:**\n<!-- x -->\n\n**Failed:**\n\n**Not run:**\n';
const r = c.checkPull({ title: '[Engine] fix stop', body: tpl });
assert.ok(!r.ok);
assert.ok(r.missing.some((m) => m.includes('Problem and result')));
assert.ok(r.missing.some((m) => m.includes('Validation')));
});
test('PR: empty body and bad title', () => {
const r = c.checkPull({ title: 'fix', body: null });
assert.ok(!r.ok);
assert.ok(r.missing.length >= 2);
});
test('PR reopen needs a revision', () => {
const body = prBody({ validation: '**Passed:** I tried it and it worked for me on my machine, looked fine.' });
assert.ok(c.checkPull({ title: '[Engine] x y', body, mode: 'intake' }).ok);
assert.ok(!c.checkPull({ title: '[Engine] x y', body, mode: 'reopen' }).ok);
});
test('free-form PR', () => {
const body = 'This fixes a crash when the context is above 8k, because the length was read from the wrong field. Before it aborted, after it continues. I built it on CUDA and ran the decode test, passed.';
assert.ok(c.checkPull({ title: '[Engine] fix crash', body }).ok);
assert.ok(!c.checkPull({ title: '[Engine] fix crash', body: 'fixes stuff' }).ok);
});
test('messages', () => {
const m = c.intakeComment(false, ['a', 'b']);
assert.ok(m.includes('- [ ] a') && m.includes('/reopen') && m.includes('CONTRIBUTING.md'));
assert.ok(c.staleComment(true).includes('/reopen'));
});
// handlers with a fake GitHub client
function fake(calls, extra = {}) {
const rest = {
issues: {
getLabel: async () => ({}),
createLabel: async (a) => calls.push(['createLabel', a.name]),
addLabels: async (a) => calls.push(['addLabels', a.labels]),
removeLabel: async (a) => calls.push(['removeLabel', a.name]),
createComment: async (a) => calls.push(['comment', a.body]),
update: async (a) => calls.push(['update', a.state]),
get: async () => ({ data: extra.live }),
listEvents: async () => {},
listForRepo: async () => {},
},
pulls: { get: async () => ({ data: { mergeable_state: 'clean' } }) },
};
return { rest, paginate: async (fn) => (fn === rest.issues.listEvents ? (extra.events || []) : (extra.items || [])) };
}
const core = { info() {} };
const repo = { owner: 'o', repo: 'r' };
test('intake labels and comments once', async () => {
const calls = [];
const item = { number: 1, title: 'bad', body: '', labels: [], user: { type: 'User' }, author_association: 'NONE' };
await bot.intake({ github: fake(calls), context: { repo, payload: { action: 'opened', issue: item } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['addLabels', 'comment']);
calls.length = 0;
item.labels = [{ name: 'needs-format' }];
await bot.intake({ github: fake(calls), context: { repo, payload: { action: 'edited', issue: item } }, core });
assert.deepStrictEqual(calls, []);
});
test('intake clears the label when edited into shape', async () => {
const calls = [];
const item = { number: 1, title: '[Engine] x y', body: bugForm(), labels: [{ name: 'needs-format' }], user: { type: 'User' } };
await bot.intake({ github: fake(calls), context: { repo, payload: { action: 'edited', issue: item } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['removeLabel', 'comment']);
});
test('reopen: ignores strangers, reopens the author when it passes', async () => {
const issue = { number: 5, state: 'closed', user: { login: 'amy' }, labels: [] };
let calls = [];
const comment = { body: '/reopen', user: { login: 'bob', type: 'User' }, author_association: 'NONE' };
await bot.reopen({ github: fake(calls), context: { repo, payload: { issue, comment } }, core });
assert.deepStrictEqual(calls, []);
const live = { number: 5, title: '[Engine] x y', body: bugForm(), labels: [] };
comment.user.login = 'amy';
await bot.reopen({ github: fake(calls, { live }), context: { repo, payload: { issue, comment } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['update', 'removeLabel', 'comment']);
assert.strictEqual(calls[0][1], 'open');
calls = [];
live.title = 'no prefix';
await bot.reopen({ github: fake(calls, { live }), context: { repo, payload: { issue, comment } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['comment']);
assert.ok(calls[0][1].includes('Not reopened'));
});
test('sweep closes only old flagged items', async () => {
const calls = [];
const old = new Date(Date.now() - 6 * 86400000).toISOString();
const fresh = new Date(Date.now() - 1 * 86400000).toISOString();
const mk = (n, created) => ({ number: n, title: 'bad', body: '', labels: [{ name: 'needs-format' }], created_at: created });
const gh = fake(calls, { items: [mk(1, old), mk(2, fresh)] });
await bot.sweep({ github: gh, context: { repo }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['comment', 'update']);
});
+164
View File
@@ -0,0 +1,164 @@
'use strict';
// GitHub-side handlers for intake.yml, reopen.yml and the 5-day sweep.
// They take the actions/github-script objects ({ github, context, core }) and never touch PR code:
// everything comes from the event payload and the REST API.
const check = require('./item-check');
const LABEL = 'needs-format';
const STALE_DAYS = 5;
const MAINTAINER = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']);
// Items a maintainer closed on purpose stay closed.
const BLOCK_LABELS = new Set(['wontfix', 'invalid', 'duplicate', 'declined']);
function labelNames(item) {
return (item.labels || []).map((l) => (typeof l === 'string' ? l : l.name));
}
async function ensureLabel(github, context) {
const { owner, repo } = context.repo;
try {
await github.rest.issues.getLabel({ owner, repo, name: LABEL });
} catch (e) {
if (e.status !== 404) throw e;
await github.rest.issues.createLabel({
owner, repo, name: LABEL, color: 'fbca04',
description: 'Title or description does not follow CONTRIBUTING.md yet',
});
}
}
async function removeLabel(github, context, number) {
const { owner, repo } = context.repo;
try {
await github.rest.issues.removeLabel({ owner, repo, issue_number: number, name: LABEL });
} catch (e) {
if (e.status !== 404) throw e;
}
}
function itemFromPayload(context) {
const p = context.payload;
return p.pull_request || p.issue;
}
function isPullItem(item, context) {
return Boolean(context.payload.pull_request || item.pull_request);
}
// issues: opened/edited/reopened, pull_request_target: opened/edited/reopened
async function intake({ github, context, core }) {
const { owner, repo } = context.repo;
const item = itemFromPayload(context);
if (!item || (item.user && item.user.type === 'Bot')) return;
if (MAINTAINER.has(item.author_association) && context.payload.action === 'opened') {
// Maintainer items are not nagged on creation.
return;
}
const isPull = isPullItem(item, context);
const labels = labelNames(item);
const hasLabel = labels.includes(LABEL);
const res = check.checkItem({ isPull, title: item.title, body: item.body, labels, mode: 'intake' });
core.info('check ' + (isPull ? 'PR' : 'issue') + ' #' + item.number + ': ' + (res.ok ? 'ok' : res.missing.join(' | ')));
if (res.ok) {
if (hasLabel) {
await removeLabel(github, context, item.number);
await github.rest.issues.createComment({ owner, repo, issue_number: item.number, body: 'Thanks, this follows the format now.' });
}
return;
}
if (context.payload.action === 'edited' && !hasLabel) return; // edits only matter for items already flagged
if (hasLabel) return; // one note only
await ensureLabel(github, context);
await github.rest.issues.addLabels({ owner, repo, issue_number: item.number, labels: [LABEL] });
await github.rest.issues.createComment({ owner, repo, issue_number: item.number, body: check.intakeComment(isPull, res.missing) });
}
// issue_comment created, body starts with /reopen
async function reopen({ github, context, core }) {
const { owner, repo } = context.repo;
const comment = context.payload.comment;
const issue = context.payload.issue;
if (!/^\s*\/reopen\b/.test(comment.body || '')) return;
if (comment.user.type === 'Bot') return;
const isAuthor = comment.user.login === issue.user.login;
const isMaint = MAINTAINER.has(comment.author_association);
if (!isAuthor && !isMaint) {
core.info('ignored /reopen from ' + comment.user.login + ' (not the author or a maintainer)');
return;
}
const isPull = Boolean(issue.pull_request);
const say = (body) => github.rest.issues.createComment({ owner, repo, issue_number: issue.number, body });
const labels = labelNames(issue);
const wasOpen = issue.state === 'open';
if (labels.some((l) => BLOCK_LABELS.has(l.toLowerCase())) && !isMaint) {
await say('This one was closed on purpose (' + labels.filter((l) => BLOCK_LABELS.has(l.toLowerCase())).join(', ') + '). Please add your new evidence in a comment and a maintainer will decide.');
return;
}
if (wasOpen && !labels.includes(LABEL)) {
core.info('already open, nothing to do');
return;
}
// The bot reads the live item, not the possibly stale event copy.
const live = (await github.rest.issues.get({ owner, repo, issue_number: issue.number })).data;
const res = check.checkItem({ isPull, title: live.title, body: live.body, labels: labelNames(live), mode: 'reopen' });
core.info('reopen check #' + live.number + ': ' + (res.ok ? 'ok' : res.missing.join(' | ')));
if (!res.ok) {
await say(check.reopenMissingComment(isPull, res.missing));
return;
}
if (!wasOpen) {
try {
await github.rest.issues.update({ owner, repo, issue_number: live.number, state: 'open' });
} catch (e) {
await say('The check passed, but GitHub would not reopen it (' + (e.message || e.status) + '). ' + (isPull ? 'If the branch was deleted, restore it or open a new pull request and link this one.' : 'Please open a new issue and link this one.'));
return;
}
}
await removeLabel(github, context, live.number);
let extra = '';
if (isPull) {
try {
const pr = (await github.rest.pulls.get({ owner, repo, pull_number: live.number })).data;
if (pr.mergeable_state === 'dirty') extra = '\n\nIt has conflicts with main now: please rebase it.';
} catch (e) { /* best effort */ }
}
await say(check.reopenOkComment(isPull) + extra);
}
async function labelTime(github, owner, repo, number, fallback) {
const events = await github.paginate(github.rest.issues.listEvents, { owner, repo, issue_number: number, per_page: 100 });
let t = null;
for (const e of events) {
if (e.event === 'labeled' && e.label && e.label.name === LABEL) t = e.created_at;
}
return t || fallback;
}
// scheduled: close items that kept the needs-format label for more than 5 days
async function sweep({ github, context, core }) {
const { owner, repo } = context.repo;
const cutoff = Date.now() - STALE_DAYS * 86400 * 1000;
const items = await github.paginate(github.rest.issues.listForRepo, { owner, repo, state: 'open', labels: LABEL, per_page: 100 });
core.info(items.length + ' open items with ' + LABEL);
for (const item of items) {
const since = new Date(await labelTime(github, owner, repo, item.number, item.created_at)).getTime();
if (since > cutoff) continue;
const isPull = Boolean(item.pull_request);
// Re-check first: an edit that fixed it should have cleared the label already; do not close good items.
const res = check.checkItem({ isPull, title: item.title, body: item.body, labels: labelNames(item), mode: 'intake' });
if (res.ok) {
await removeLabel(github, context, item.number);
continue;
}
await github.rest.issues.createComment({ owner, repo, issue_number: item.number, body: check.staleComment(isPull) });
await github.rest.issues.update({ owner, repo, issue_number: item.number, state: 'closed', state_reason: 'not_planned' });
core.info('closed #' + item.number);
}
}
module.exports = { intake, reopen, sweep, LABEL, STALE_DAYS };
+114
View File
@@ -0,0 +1,114 @@
name: Docker images
# Builds the Strata container images of one release from its engine packages and pushes them to ghcr.io.
# Run by hand only (Actions tab -> Docker images -> Run workflow). Nothing starts it on its own.
#
# For each flavor the job downloads the engine package from the release and checks it against the signed update feed
# (docker/fetch-engines.py: the feed's Ed25519 signature, the archive's SHA-256, the manifest hash, the package signature,
# every file's hash), builds the image with docker/build.sh and, only when "push" is on, pushes
# ghcr.io/niko1221/strata:<version>-<flavor> and ghcr.io/niko1221/strata:latest-<flavor>
# The first run with "push" off builds everything without publishing it (a dry run).
on:
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag whose engine packages go into the images (for example v0.2.7)'
required: true
type: string
flavors:
description: 'Flavors to build, comma separated (empty: every flavor the release has a package for)'
required: false
default: ''
type: string
push:
description: 'Push the images to ghcr.io (off: build only, nothing is published)'
required: true
default: false
type: boolean
latest:
description: 'Also move the latest-<flavor> tags (turn off for a pre-release or a test)'
required: true
default: true
type: boolean
source_ref:
description: 'Git ref of the Dockerfile and the Terminal sources (empty: the release tag)'
required: false
default: ''
type: string
permissions:
contents: read
env:
IMAGE: ghcr.io/niko1221/strata
jobs:
plan:
runs-on: ubuntu-24.04
outputs:
flavors: ${{ steps.list.outputs.flavors }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.source_ref || inputs.release_tag }}
- name: Which flavors does the release have a package for
id: list
env:
WANT: ${{ inputs.flavors }}
TAG: ${{ inputs.release_tag }}
run: |
set -eu
args="--release-tag $TAG --list --skip-missing"
[ -n "$WANT" ] && args="$args --flavors $WANT"
python3 docker/fetch-engines.py $args > list.tsv
cat list.tsv
flavors=$(awk -F'\t' '$3 != "SKIP" {printf "%s\"%s\"", (n++ ? "," : ""), $1}' list.tsv)
[ -n "$flavors" ] || { echo "the release has no engine package for any requested flavor"; exit 1; }
echo "flavors=[$flavors]" >> "$GITHUB_OUTPUT"
image:
needs: plan
runs-on: ubuntu-24.04
timeout-minutes: 120
permissions:
contents: read
packages: write
strategy:
fail-fast: false
max-parallel: 3
matrix:
flavor: ${{ fromJSON(needs.plan.outputs.flavors) }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.source_ref || inputs.release_tag }}
- name: Free disk space (the runner has about 14 GB; a package plus its image needs about 10 GB)
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL || true
docker system prune -af >/dev/null 2>&1 || true
df -h /
- name: Log in to ghcr.io
if: ${{ inputs.push }}
env:
REGISTRY_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
run: printf '%s' "$REGISTRY_TOKEN" | docker login ghcr.io -u "$REGISTRY_USER" --password-stdin
- name: Fetch, verify, build (and push when push is on)
env:
TAG: ${{ inputs.release_tag }}
FLAVOR: ${{ matrix.flavor }}
PUSH: ${{ inputs.push && '--push' || '' }}
LATEST: ${{ !inputs.latest && '--no-latest' || '' }}
run: |
docker/publish.sh --release-tag "$TAG" --flavors "$FLAVOR" --image "$IMAGE" $PUSH $LATEST
- name: Summary
if: ${{ always() }}
run: |
{
echo "### ${{ matrix.flavor }}"
echo "- release: ${{ inputs.release_tag }}, pushed: ${{ inputs.push }}"
echo "- image: \`$IMAGE:${INPUT_TAG#v}-${{ matrix.flavor }}\`"
} >> "$GITHUB_STEP_SUMMARY"
env:
INPUT_TAG: ${{ inputs.release_tag }}
+58
View File
@@ -0,0 +1,58 @@
name: Intake
# New issues and pull requests that do not follow CONTRIBUTING.md get the needs-format label and one friendly note.
# Items that keep the label for 5 days are closed by the daily sweep, with a note on how to /reopen.
#
# pull_request_target is used instead of pull_request on purpose: a pull request from a fork gets a read-only
# GITHUB_TOKEN under pull_request, so it could not be labelled or commented on. It is safe here because this
# workflow never checks out or runs the pull request's code. It checks out the default branch only (the bot
# scripts) and reads the title and description from the event payload and the REST API.
on:
issues:
types: [opened, edited]
pull_request_target:
types: [opened, edited]
schedule:
- cron: '17 4 * * *'
workflow_dispatch:
permissions:
contents: read
issues: write
pull-requests: write
concurrency:
group: intake-${{ github.event.issue.number || github.event.pull_request.number || 'sweep' }}
cancel-in-progress: false
jobs:
check:
if: github.event_name == 'issues' || github.event_name == 'pull_request_target'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: .github/scripts
persist-credentials: false
- uses: actions/github-script@v7
with:
script: |
const bot = require('./.github/scripts/reset-bot.js');
await bot.intake({ github, context, core });
sweep:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: .github/scripts
persist-credentials: false
- uses: actions/github-script@v7
with:
script: |
const bot = require('./.github/scripts/reset-bot.js');
await bot.sweep({ github, context, core });
+53
View File
@@ -0,0 +1,53 @@
name: Metal build
# Compile and link the Metal engine on a macOS runner (no GPU needed: the kernels are compiled at run time). The shared host files
# (src/core/verify.cpp, src/prefill, src/program/generate.cpp) call kernel entry points that exist only for CUDA / HIP until Metal gets a
# stub or an implementation; a missing one is a link error here, not on a user's Mac (owner round 9, G6).
on:
pull_request:
paths:
- 'src/**'
- 'include/**'
- 'metal/**'
- 'third_party/**'
- 'CMakeLists.txt'
- '.github/workflows/metal-build.yml'
workflow_dispatch:
permissions:
contents: read
jobs:
metal-link:
# the macOS 26 SDK: metal/src/mtl_rt.mm names MTLGPUFamilyMetal4 (and MTLCompileOptions.mathMode, macOS 15) behind @available,
# which is a run-time check; the names must still be in the SDK the engine is compiled against
runs-on: macos-26
timeout-minutes: 45
env:
# setup.py's LLAMA_CPP_COMMIT and metal/setup_mac.py's metal-cpp pin
LLAMA_CPP_COMMIT: 3cf03257f219afbe7334045ff7c6a06ac68c627d
METAL_CPP_URL: https://developer.apple.com/metal/cpp/files/metal-cpp_26.zip
METAL_CPP_SHA256: 4df3c078b9aadcb516212e9cb03004cbc5ce9a3e9c068fa3144d021db585a3a4
steps:
- uses: actions/checkout@v4
- name: Tools
run: brew install ninja
- name: metal-cpp
run: |
curl -fsSL "$METAL_CPP_URL" -o metal-cpp.zip
echo "$METAL_CPP_SHA256 metal-cpp.zip" | shasum -a 256 -c -
unzip -q metal-cpp.zip -d _mcpp
- name: llama.cpp (ggml) at the pinned commit
run: |
mkdir llama.cpp && cd llama.cpp
git init -q
git remote add origin https://github.com/ggml-org/llama.cpp.git
git fetch -q --depth 1 origin "$LLAMA_CPP_COMMIT"
git checkout -q FETCH_HEAD
- name: Configure
run: |
cmake -S metal -B build-metal -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DMETAL_CPP_DIR="$PWD/_mcpp/metal-cpp" -DSTRATA_GGML_DIR="$PWD/llama.cpp"
- name: Build and link the engine
run: cmake --build build-metal --target strata
+38
View File
@@ -0,0 +1,38 @@
name: Reopen
# An author comments /reopen on their closed issue or pull request. The check runs and the item is reopened,
# or the bot replies with what is still missing. GitHub does not let authors reopen items a maintainer closed,
# so this workflow does it for them.
#
# issue_comment always runs the workflow file from the default branch with a write-capable token, which is what
# we need for fork authors too. It never checks out or executes the pull request's code: it reads the live
# title and description through the REST API. Only the author of the item or a maintainer can trigger it.
on:
issue_comment:
types: [created]
permissions:
contents: read
issues: write
pull-requests: write
concurrency:
group: reopen-${{ github.event.issue.number }}
cancel-in-progress: false
jobs:
reopen:
if: startsWith(github.event.comment.body, '/reopen') && github.event.comment.user.type != 'Bot'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: .github/scripts
persist-credentials: false
- uses: actions/github-script@v7
with:
script: |
const bot = require('./.github/scripts/reset-bot.js');
await bot.reopen({ github, context, core });
+21
View File
@@ -0,0 +1,21 @@
name: Scripts test
on:
pull_request:
paths:
- '.github/scripts/**'
- '.github/workflows/intake.yml'
- '.github/workflows/reopen.yml'
permissions:
contents: read
jobs:
node-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- run: node --test .github/scripts/item-check.test.js
@@ -0,0 +1,65 @@
name: Minimal serving runtime
on:
push:
paths:
- .github/workflows/terminal-minimal-server.yml
- requirements.txt
- strata.py
- strata_cli/**
- strata_control/**
- serve/**
- tests/terminal/test_runtime.py
- tools/minimal_server_smoke.py
- tools/strata_tokenizer.py
pull_request:
paths:
- .github/workflows/terminal-minimal-server.yml
- requirements.txt
- strata.py
- strata_cli/**
- strata_control/**
- serve/**
- tests/terminal/test_runtime.py
- tools/minimal_server_smoke.py
- tools/strata_tokenizer.py
workflow_dispatch:
permissions:
contents: read
jobs:
fake-engine:
name: Linux / runtime requirements only
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: actions/setup-python@v7
with:
python-version: '3.11'
- name: Install only pinned runtime requirements
shell: bash
run: |
python -m venv "$RUNNER_TEMP/strata-runtime-venv"
"$RUNNER_TEMP/strata-runtime-venv/bin/python" -m pip install -r requirements.txt
- name: Check fake-engine serving without terminal packages
shell: bash
run: |
python - <<'PY'
import os
import subprocess
from pathlib import Path
report = Path(os.environ['RUNNER_TEMP']) / 'minimal-server-smoke.json'
python = str(Path(os.environ['RUNNER_TEMP']) / 'strata-runtime-venv' / 'bin' / 'python')
subprocess.run([python, 'tools/minimal_server_smoke.py',
'--report', str(report)], check=True)
print(report.read_text())
PY
- name: Run full server suite without terminal packages
shell: bash
run: |
"$RUNNER_TEMP/strata-runtime-venv/bin/python" -m unittest discover -s serve -p 'test_*.py'
@@ -0,0 +1,115 @@
name: Portable monitor client
on:
push:
paths:
- .github/workflows/terminal-portable-monitor.yml
- packaging/monitor_bundle.py
- strata.py
- strata_cli/**
- strata_control/**
- tools/minimal_cli_smoke.py
- tools/portable_monitor_smoke.py
- tools/portable_monitor_pty_smoke.py
- requirements-monitor.txt
- LICENSE
pull_request:
paths:
- .github/workflows/terminal-portable-monitor.yml
- packaging/monitor_bundle.py
- strata.py
- strata_cli/**
- strata_control/**
- tools/minimal_cli_smoke.py
- tools/portable_monitor_smoke.py
- tools/portable_monitor_pty_smoke.py
- requirements-monitor.txt
- LICENSE
workflow_dispatch:
permissions:
contents: read
jobs:
client:
name: ${{ matrix.os }} / Python ${{ matrix.python }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python: ['3.10', '3.11']
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python }}
- name: Check source CLI without optional packages
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
report = Path(os.environ['RUNNER_TEMP']) / 'minimal-cli-smoke.json'
subprocess.run([sys.executable, 'tools/minimal_cli_smoke.py',
'--report', str(report)], check=True)
print(report.read_text())
PY
- name: Build and smoke test without site packages
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
bundle = Path(os.environ['RUNNER_TEMP']) / 'strata-monitor.pyz'
report = bundle.with_name('monitor-stdlib-smoke.json')
subprocess.run([sys.executable, 'packaging/monitor_bundle.py',
'build', str(bundle)], check=True)
subprocess.run([sys.executable, 'tools/portable_monitor_smoke.py',
'--bundle', str(bundle), '--report', str(report)], check=True)
print(report.read_text())
PY
- name: Install optional Textual dependency
if: matrix.python == '3.11'
run: python -m pip install -r requirements-monitor.txt
- name: Smoke test packaged interactive Overview
if: matrix.python == '3.11'
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
bundle = Path(os.environ['RUNNER_TEMP']) / 'strata-monitor.pyz'
report = bundle.with_name('monitor-interactive-smoke.json')
subprocess.run([sys.executable, 'tools/portable_monitor_smoke.py',
'--bundle', str(bundle), '--interactive',
'--report', str(report)], check=True)
print(report.read_text())
PY
- name: Check POSIX terminal restoration
if: matrix.os == 'ubuntu-latest' && matrix.python == '3.11'
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
bundle = Path(os.environ['RUNNER_TEMP']) / 'strata-monitor.pyz'
report = bundle.with_name('monitor-pty-smoke.json')
subprocess.run([sys.executable, 'tools/portable_monitor_pty_smoke.py',
'--bundle', str(bundle), '--report', str(report)], check=True)
print(report.read_text())
PY
+28
View File
@@ -0,0 +1,28 @@
name: Title area
# Checks that a pull request or issue title starts with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
# It reads the title only: no checkout, no labels, no comments, no write access.
on:
pull_request:
types: [opened, edited, reopened, synchronize]
issues:
types: [opened, edited, reopened]
permissions: {}
jobs:
title:
runs-on: ubuntu-latest
steps:
- name: Check the title
env:
TITLE: ${{ github.event.pull_request.title || github.event.issue.title }}
run: |
case "$TITLE" in
"[Engine] "?*|"[GUI] "?*|"[TUI] "?*|"[Benchmark] "?*|"[Docs] "?*|"[Other] "?*)
echo "Title OK: $TITLE" ;;
*)
echo "::error::The title must start with its area and then a short description. Allowed areas: [Engine], [GUI], [TUI], [Benchmark], [Docs], [Other]. Example: \"[Engine] fix decode stop on long prompts\". Your title: $TITLE"
exit 1 ;;
esac
+28
View File
@@ -48,6 +48,7 @@ build-vision-cuda12.bat
# setup.py outputs (engine, downloads) and release builds
/engine/
/engine-sycl/
/engine-cuda12/
/build-release*/
/dist/
@@ -61,3 +62,30 @@ build-vision-cuda12.bat
/.ccache/
/.rocm-win/
# Optional MCP raster cache
*.tool-media/
/.tool-media/
# local Windows HIP packaging output
dist-hip/
# Terminal / Desktop split: local venvs, build output and archives never enter git
.terminal-venv/
.terminal-test-venv/
node_modules/
*.dmg
/release/artifacts/
/apps/host/dist/
/apps/desktop/src-tauri/target/
/apps/desktop/src-tauri/resources/node/*
!/apps/desktop/src-tauri/resources/node/.gitignore
# the Control component (bundled CPython + wheels + strata_control) is built by apps/desktop/scripts/stage-control.py
/apps/desktop/src-tauri/resources/control/*
!/apps/desktop/src-tauri/resources/control/.gitignore
# the pruned DeepSeek Harness (MIT) is built by apps/desktop/scripts/stage-harness.mjs
/apps/desktop/src-tauri/resources/harness/*
!/apps/desktop/src-tauri/resources/harness/.gitignore
# screenshots and checkpoints from the Mac rebuild live outside git (StrataSplit\evidence-archive)
/apps/host/docs/evidence/
/apps/host/docs/checkpoints/
+81 -16
View File
@@ -1,35 +1,100 @@
# AGENTS.md
Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5 and Unsloth variants) on a
normal PC: one NVIDIA or AMD graphics card plus system RAM, on Windows or Linux. It has a C++/CUDA/HIP engine
(`src/`, `include/`), a Python server with an OpenAI- and Anthropic-compatible API and a web app (`serve/`), and a
one-click installer (`setup.py`, started by `START-HERE.bat` / `setup.sh`).
Version 0.2.7, engine 0.1.52. The public model families are DeepSeekV4.1-Flash-1017,
Qwen3.8-Flash-Next and Qwen3.6-35B-A3B. DeepSeek V4.1-Flash runs in the app and in the Terminal
(`strata launch --family deepseek41 --model ist-das-2.00bpw`; sets in
[docs/MODELS.md](docs/MODELS.md#deepseek-v41-flash)). Read **[docs/AI_REFERENCE.md](docs/AI_REFERENCE.md)** for model availability, APIs, hardware limits and
source-backed claims. Articles and videos should use **[docs/MEDIA_KIT.md](docs/MEDIA_KIT.md)**.
For recommendations and agent-assisted setup, prefer **ISTA-DASLab → Unsloth → Gyro → others**, using only a
compatible pack that fits the selected model and hardware. ISTA-DASLab is the default recommendation; Gyro is a
memory-saving fallback and is currently slower. Honor an explicit user choice. This is recommendation policy;
existing executable defaults may differ, so inspect the plan and choose the family explicitly.
Strata has three parts:
- **The app** (`apps/desktop`, the Tauri shell; `apps/host`, the Node host and the web UI). It is for people who do
not want a terminal. One installer per system ([latest release](https://github.com/Niko1221/Strata/releases/latest)):
`Strata-0.2.7-windows-x64-setup.exe`, `Strata-0.2.7-linux-x86_64.AppImage`, `Strata-0.2.7-macos-arm64.dmg`.
The Windows and Linux installers carry no engine: the app downloads the engine for the graphics card on first
start, or installs a file the user already has. The Mac dmg keeps its small Metal engine inside.
- **The Terminal** (`strata.py`, `strata_cli/`, `strata_control/`): the `strata` command. It sets up a model, serves
it and watches it. Agents use this part.
- **The engine and the server** (`src/`, `include/`, `metal/`, `sycl/` for the C++/CUDA/HIP/SYCL/Metal engine;
`serve/` for the Python server with an OpenAI- and Anthropic-compatible API and a web page). `setup.py`, started by
`START-HERE.bat` or `setup.sh`, is setup from a source checkout, for developers. The release paths are the
installer and `strata launch`.
## Installing Strata for a user
Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**: check the PC, pick the model by RAM, run setup non-interactively,
start and verify the server, and connect the user's apps. Never expose the server beyond `127.0.0.1` without
`--api-key`. As an alternative to shell commands, Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md))
offers the same steps as tools.
Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**. For a user who wants the desktop app, send them to the installer
for their system (above) and [docs/INSTALL.md](docs/INSTALL.md). For an agent that has a shell, use the Terminal; the
short version is:
1. Install the Terminal. It needs no administrator rights.
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh # Linux, Mac
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex # Windows PowerShell
```
On Windows, open a new terminal afterwards so that `strata` is on the PATH.
2. Check what would happen. Nothing is changed:
```sh
strata launch --plan --family qwen --model q2_0
```
3. Set up and serve without questions. It downloads the engine for the card and the model (tens of GB), starts the
server, prints the addresses and returns. The server keeps running.
```sh
strata launch --family qwen --model q2_0 --yes --no-tui
```
Useful flags (all of them: `strata launch --help`): `--family`, `--model-path` (files the user already has),
`--data-dir`, `--context 32k|64k|128k`, `--vision off|on|gpu|cpu`, `--parallel N`, `--gpu 0,1`,
`--access local|lan|cloudflare|tailscale`, `--port`, `--api-key-env NAME`, `--new-key`, `--save-preset NAME`,
`--speed faster|full` (DeepSeek), `--output json`. `strata launch --status` shows what is serving, `strata launch --stop` stops it, and
`strata launch --show-key` prints the API key.
4. Connect the user's apps: `strata connect` lists them, `strata connect <app> --check` sends one prompt and one tool
call the way that app does.
Keep the server on `127.0.0.1` unless the user asks for more. Beyond it an API key is recommended: `--access lan` makes one for you, and `--no-api-key` serves without one (a warning, never a refusal; on a public tunnel the warning is stronger). Source setup (`START-HERE.bat`, `setup.sh`) is the fallback for a developer or for a system with no release package; it is
documented in [docs/SOURCE_INSTALL.md](docs/SOURCE_INSTALL.md). Those wrappers exist in a source checkout, not in the
released app. Do not tell an app user to find a `.bat` file.
Source setup does not make an unsupported model or backend compatible. Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md)) offers the same steps as tools.
## Working on the code
- How the engine works, every measured number, the API and all settings: [docs/DETAILS.md](docs/DETAILS.md) and
the [paper](docs/paper/Strata-Paper.pdf).
- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md).
- Setup's own tests run without a GPU or downloads: `python tools/test_setup_<name>.py` (for example
`tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
- Keep the docs' style: plain words, measured numbers with what they were measured on, no claims without a
measurement.
- The Terminal: [docs/TERMINAL.md](docs/TERMINAL.md) and [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). The app,
the packages and the update feed: [packaging/README.md](packaging/README.md) and [packaging/UPDATES.md](packaging/UPDATES.md).
- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md);
Mac (Metal): [docs/MAC.md](docs/MAC.md).
- Tests: the Terminal and Control tests are in `tests/terminal` (`python -m pytest tests/terminal`). The app's UI
tests run with `npm test` in `apps/host`. Setup's own tests run without a GPU or downloads:
`python tools/test_setup_<name>.py` (for example `tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
- Keep the docs' style: plain words, short sentences, written for non-technical users first. Give measured numbers
with what they were measured on. Make no claim without a measurement.
## Contributing a change or report
- Search the open issues and pull requests first, and add to a thread that already covers your point.
- Open an issue with the form that fits (bug report, feature request or question).
- One change per pull request. Say what it changes and what it leaves alone.
The rules are in **[CONTRIBUTING.md](CONTRIBUTING.md)**. Read it before you open an issue or a pull request, and
follow it whether a person or an agent writes the text. The short version:
- Start every issue and pull request title with its area: `[Engine]`, `[GUI]`, `[TUI]`, `[Benchmark]`, `[Docs]` or
`[Other]`, for example `[Engine] fix …`. A check on GitHub fails a pull request whose title has no area.
- Search the open and closed issues and pull requests first, and add to a thread that already covers your point.
- Agree the scope in an issue before a large feature, a refactor or a change to a default.
- One problem per pull request. Lead with the problem and the result, then the approach, then the validation
(passed, failed and not run, kept apart) and the limits.
- A new feature is opt-in, and the default path stays byte-identical to the last release. Say how you checked.
- Build every backend a file touches (CUDA, HIP, SYCL) before asking for review.
- Change a default only where you measured it faster, and show the numbers with what they were measured on.
- Never submit measurements, logs or reproduction steps you did not produce. Mark a finding from reading code as
unconfirmed static analysis.
- A report from hardware the maintainers do not have is welcome. Follow
[docs/COMMUNITY_BENCHMARKS.md](docs/COMMUNITY_BENCHMARKS.md), compare against a same-day run of the build you are
testing, and say what you did not test.
+3
View File
@@ -2,6 +2,7 @@
rem Benchmark this Strata install the way it is used and write a report you can share (docs/COMMUNITY_BENCHMARKS.md).
rem It starts the model with your settings (or uses the one already running), takes about 10 minutes and changes nothing.
rem Options: --clients 4 (several requests at once), --long (a 32K prompt), --compare <older results folder>.
rem This is the quick speed check. The standard benchmark (docs/BENCHMARK_PROTOCOL.md), whose results other people can compare with, is `strata bench run`.
setlocal
title Strata - benchmark
cd /d "%~dp0"
@@ -10,6 +11,8 @@ if not exist ".venv\Scripts\python.exe" (
pause
exit /b 1
)
echo Quick speed check. For results other people can compare with (the standard benchmark, a few minutes), run: .venv\Scripts\python.exe strata.py bench run
echo.
".venv\Scripts\python.exe" tools\strata_bench.py %*
set "RC=%ERRORLEVEL%"
pause
+938 -21
View File
File diff suppressed because it is too large. Load diff
+122
View File
@@ -0,0 +1,122 @@
# Contributing to Strata
**AI assistance is welcome. The contributor is responsible for understanding the change and supplying real evidence.**
These rules apply to issues, pull requests, benchmark reports, and the agents helping prepare them.
You should be able to explain every changed line and answer review questions about it.
## Before starting
1. Search [issues](https://github.com/Niko1221/Strata/issues) and
[pull requests](https://github.com/Niko1221/Strata/pulls), including closed discussions. Add new evidence to an
existing thread when it covers your problem.
2. Read [AGENTS.md](AGENTS.md) and the documentation for the component you want to change.
3. Get maintainer agreement on scope before substantial features, refactors, architecture changes, or changes to
defaults. Describe the user problem, proposed behavior, and tradeoffs in an issue first. Agreement on scope
does not guarantee a merge. Focused fixes and benchmark reports can go straight to a PR.
4. Keep one problem per PR. Update that PR after feedback; do not open competing attempts for the same change.
Do not resubmit a declined proposal under a different title or through another agent. If new evidence changes
the situation, bring it to the original discussion and let a maintainer decide whether to reopen it.
## Report a problem people can investigate
Use the matching [issue form](https://github.com/Niko1221/Strata/issues/new/choose). A bug report should include:
- The smallest steps or command that reproduce the problem, and the expected and actual behavior.
- Strata version or commit, OS, GPU and VRAM, and relevant model, quantization, and configuration.
- The exact error and relevant logs, with credentials and private content removed.
- Whether you reproduced it yourself. Label a finding based only on reading code or a scanner as
**unconfirmed static analysis**, and explain the suspected path without presenting it as an observed failure.
Do not invent reproduction steps, errors, affected hardware, or severity. If you cannot reproduce something,
say what you inspected and what would verify the hypothesis. Feature requests should explain a concrete user
need and the current workaround. Questions should say what you tried and where you got stuck.
Example structure, with placeholders to replace with actual observations:
```text
Problem: <one observed behavior>
Environment: <commit, OS, GPU/VRAM, model/quant, relevant settings>
Reproduce: <exact command or short sequence>
Expected: <documented or intended behavior>
Actual: <observed behavior and exact error>
Evidence: <sanitized log attachment>
Reproduction status: <reproduced locally OR unconfirmed static analysis>
```
## Titles
Every issue and pull request title starts with its area in square brackets, then a short description: `[Engine]`, `[GUI]`, `[TUI]`,
`[Benchmark]`, `[Docs]` or `[Other]`. For example `[Engine] fix decode stop on long prompts` or `[GUI] keep the model list after a tab
switch`. The issue forms ask for the area and a check on the title fails when the prefix is missing.
## Prepare a focused pull request
Lead with the problem and resulting behavior. Explain why the approach is needed, the important limits, and
any compatibility effects. Link the issue and scope agreement when applicable. A file-by-file narration or
an AI-generated summary of the diff is not a substitute for this explanation.
- Keep unrelated formatting, cleanup, dependency changes, and speculative abstractions out of the PR.
- Add documentation where it changes what a user or developer needs to know. Avoid repeated summaries,
speculative roadmaps, and generated documentation with no actionable information.
- Preserve existing behavior outside the intended change. New features are opt-in; follow the default-path
and performance requirements in [AGENTS.md](AGENTS.md).
- Read the complete diff, including generated files, before submitting. Remove debugging output, secrets,
private paths, and accidental files.
- Prefer a small relevant regression check over tests that only restate the implementation. Documentation-only
changes need link, example, and consistency checks; they do not require GPU builds.
## Show the validation you actually performed
List exact commands, the revision and environment tested, and observed outcomes. Keep **passed**, **failed**,
and **not run** separate. Include relevant output or a log attachment. A command you suggest running is not
evidence that it passed; a CI run on a different revision is not validation of the submitted code.
Build every backend a code change touches, including CUDA, HIP, and SYCL where applicable, before requesting
review. For changes that need hardware execution, report that execution separately from compilation. A mock,
static check, or successful build does not establish hardware correctness. If you lack a required environment,
keep the PR as a draft, identify the missing checks, and ask for targeted help in the same thread. Do not claim
compatibility you have not tested.
Performance claims need before/after runs on the same hardware and workload, with every changed setting
identified. Preserve the individual runs, failures, and variability. Follow the
[community measurement guide](docs/COMMUNITY_BENCHMARKS.md); standard submissions will use the
[benchmark protocol](docs/BENCHMARK_PROTOCOL.md) once it is activated. One fast run does not justify changing a
default or claiming a speedup for other machines.
Example PR body; placeholders are not test results:
```text
Problem and result: <trigger; behavior before and after the change>
Related discussion: <issue URL and scope agreement, when required>
Approach: <why this fix; relevant compatibility effects>
Validation:
Passed: <exact command, revision/environment, observed result, log>
Failed: <exact command and failure, or none observed>
Not run: <check, reason, and what remains unverified>
Limits: <known limitations or remaining work>
```
## Rules for AI-assisted work
Use agents to help investigate, implement, and explain work that you can review. Before submitting, verify
their claims against the source, actual command output, or a cited primary reference.
Never submit fabricated measurements, invented test results, fake logs, or copied claims you have not checked.
Do not turn a repository scan into a batch of speculative issues or PRs. Consolidate related findings, establish
their effect, and submit the useful evidence. Do not present an agent's confidence or another model's approval
as validation. Keep review replies concise and address the requested change or evidence directly.
Maintainers may request a smaller change, return a PR to draft, or close a submission that lacks evidence,
duplicates existing work, or ignores agreed scope. A smaller, understandable contribution with honest limits
is easier to review and maintain.
## Share a benchmark
A results-only PR should contain the human report, raw measurements, sanitized configuration, and supporting
logs. Keep it separate from engine changes and include poor results and failed requests.
The [current community benchmark guide](docs/COMMUNITY_BENCHMARKS.md) remains available for measurements made
with today's tools. The [new protocol](docs/BENCHMARK_PROTOCOL.md) defines the draft standard and future approval
workflow. Its draft status must remain visible: existing reports are not automatically standard or approved.
+98 -105
View File
@@ -1,119 +1,112 @@
# syntax=docker/dockerfile:1
# Strata, built FROM SOURCE (NVIDIA CUDA 13). The engine is compiled here, packaged as an engine component package and then
# installed into the same runtime image as the ready-made ones (docker/Dockerfile): same Terminal, same entrypoint, same /data volume.
#
# Strata: Qwen3.8-Flash-Next on NVIDIA GPUs (RTX 30/40/50, 12+ GB VRAM; two or
# three cards can share one model, 8 GB each - docs/MULTI_GPU.md).
# Most people want the ready-made images instead (nothing compiled): docs/DOCKER.md.
#
# The engine is compiled during docker build, so the first container start only
# downloads the model (~70 GB) and starts the server. docker build has no GPU,
# so the CUDA architectures are fixed here instead of read from nvidia-smi: the
# engine is a fat binary with a cubin per listed arch, and the runtime picks the
# one matching your card. Narrow CUDA_ARCHITECTURES to your card for a faster
# build; a card outside the set needs a rebuild with its own arch.
# docker build -t strata:src . # RTX 20/30/40/50 (compiles for each, ~25 min)
# docker build -t strata:src --build-arg CUDA_ARCHITECTURES=86 . # one card only (faster); 86 = RTX 30, 89 = RTX 40, 120 = RTX 50
# docker build -t strata:src-cuda12 --build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 \
# --build-arg CUDA_ARCHITECTURES="60;61;70;75;80;86;89" --build-arg BUILD_VISION=0 . # GTX 10 / P100 / V100 (driver 575+)
# docker run --rm --gpus all -p 127.0.0.1:8080:8080 --ulimit memlock=-1 --shm-size 2g --stop-timeout 120 \
# -v strata-data:/data -e MODEL=IQ3_XXS -e API_KEY=change-me-please-123 strata:src
#
# Build:
# docker build -t strata .
# docker build -t strata --build-arg CUDA_ARCHITECTURES=89 . # RTX 40 only
# docker build has no GPU, so the CUDA architectures are given here instead of read from nvidia-smi. The engine is a fat binary with
# a machine-code part per listed architecture (plus PTX for the newest, which later cards can JIT), and the runtime picks the one
# matching your card. Narrow CUDA_ARCHITECTURES to your card for a faster build. Native code for the CPU that builds the image is NOT
# used (STRATA_PORTABLE=ON), so the image runs on any x86-64 CPU with AVX2.
#
# Run (host needs an NVIDIA driver >= 580 and nvidia-container-toolkit):
# docker run --rm --gpus all \
# -p 8080:8080 \
# --ulimit memlock=-1 \
# -v strata-data:/data \
# -e MODEL=IQ2_XS \
# strata
#
# Setup choices are env vars, read by docker-entrypoint.sh: FAMILY, MODEL, CONTEXT,
# VISION (no | yes | cpu), KV (int8 | q4_0 | k8v4), GPU (one card) or GPUS ("0,2"
# or "all", with LAYER_SPLIT), LOW_RAM (auto | on | off), HOST, PORT, API_KEY,
# GGUF_DIR (GGUF files you already have), RESIDENT_BUDGET_GIB, KV_STREAMING.
#
# Only the model files, the prepared pack, the MTP layer and the install config
# live in the /data volume; the engine is part of the image. Strata loads 32-62 GB
# into RAM, so a capped container needs -e LOW_RAM=on: setup.py reads the RAM from
# /proc/meminfo, which here is the host's total, not the container's limit. Add an
# API key before exposing the port to a network: -e API_KEY=<secret>. Pass
# -e REINSTALL=1 to change the model settings later.
#
# --gpus all on a host with two usable cards: setup takes both (the layer split is
# its recommended default). Pin one card with -e GPU=0, or name them with
# -e GPUS=0,2. A volume set up for one card switches to the pair on its first start
# on a two-card host unless GPU or GPUS pins it. LOW_RAM=on runs on one card.
# Plain legacy-builder features only (no heredocs, no --mount). AMD (HIP) and Intel (SYCL) source builds: docs/AMD_HIP.md, docs/INTEL.md.
FROM nvidia/cuda:13.0.0-devel-ubuntu24.04
# STRATA_EXECV=1: setup.py replaces itself with the server, so the server is PID 1
# and docker stop's SIGTERM reaches it (see setup.start). Normal Linux starts, which
# don't set it, keep spawning the server as a child.
ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 LANG=C.UTF-8 STRATA_EXECV=1
ARG CUDA_IMAGE=nvidia/cuda:13.0.0-devel-ubuntu24.04
ARG RUNTIME_BASE=ubuntu:24.04
# ---- 1. compile the engine and make its component package
FROM ${CUDA_IMAGE} AS engine-build
ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 PATH=/venv/bin:$PATH
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential ca-certificates curl git libatomic1 libgomp1 \
python3 python3-pip python3-venv unzip \
build-essential ca-certificates git libgomp1 libatomic1 python3 python3-pip python3-venv unzip \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/strata
WORKDIR /src
COPY . .
# RTX 20 (75), RTX 30 (86), RTX 40 (89), RTX 50 (120), plus 80 for A-series. CMakeLists
# refuses anything below 75. BUILD_VISION=0 skips the image encoder build.
ARG CUDA_ARCHITECTURES=75;80;86;89;120
ARG CUDA_ARCHITECTURES="75;80;86;89;120"
ARG BUILD_VISION=1
ARG ENGINE_VERSION=
RUN python3 -m venv /venv && /venv/bin/pip install --no-cache-dir -r requirements.txt -r requirements-build.txt
# llama.cpp at the pinned commit (ggml for the engine, mtmd for pictures), exactly as setup.py fetches it
RUN /venv/bin/python -c "import setup; print(setup.get_llama_cpp())"
RUN set -e; \
real=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do real="$real;${a}-real"; done; real=${real#;}; \
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
sm60=OFF; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do if [ "$a" -lt 75 ]; then sm60=ON; fi; done; \
/venv/bin/cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_TESTS=OFF \
-DSTRATA_EXPERIMENTAL_SM60=$sm60 \
-DSTRATA_NATIVE_EXPERTS=ON -DSTRATA_PORTABLE=ON -DSTRATA_Q6K_EXPERTS=ON -DSTRATA_MMQ_KQUANTS=ON \
"-DCMAKE_CUDA_ARCHITECTURES=${real};${last}-virtual" -DSTRATA_GGML_DIR=/src/third_party/llama.cpp; \
/venv/bin/cmake --build build --target strata strata_kq -j "$(nproc)"; \
if [ "$BUILD_VISION" = "1" ]; then \
/venv/bin/cmake -G Ninja -S tools/vision -B build-vision -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=/src/third_party/llama.cpp \
-DSTRATA_VISION_CUDA=ON -DSTRATA_PORTABLE=ON -DGGML_CUDA_NCCL=OFF "-DCMAKE_CUDA_ARCHITECTURES=${real};${last}-virtual"; \
/venv/bin/cmake --build build-vision --target strata-vision -j "$(nproc)"; \
fi
RUN set -e; \
mkdir -p /stage /engine; \
cp build/strata /stage/; \
[ -f build/libstrata_kq.so ] && cp build/libstrata_kq.so /stage/ || true; \
[ -f build-vision/bin/strata-vision ] && cp build-vision/bin/strata-vision /stage/ || true; \
ver="${ENGINE_VERSION:-$(sed -n 's/^[[:space:]]*project([^)]*VERSION[[:space:]]*\([0-9][0-9.]*\).*/\1/p' CMakeLists.txt | head -1)}"; \
cudafull=$(nvcc --version | sed -n 's/.*release \([0-9][0-9]*\.[0-9][0-9]*\).*/\1/p' | head -1); cudamajor=${cudafull%%.*}; \
if [ "$cudamajor" = 13 ]; then mindriver=580.65; else mindriver=575.51; fi; \
vision_archs=""; [ -f /stage/strata-vision ] && vision_archs=", \"vision_archs\": \"$CUDA_ARCHITECTURES\""; \
echo "{\"version\": \"$ver\", \"source\": \"docker build\", \"archs\": \"$CUDA_ARCHITECTURES\", \"cuda\": \"$cudafull\", \"portable\": true${vision_archs}}" > /stage/BUILD.json; \
python3 packaging/collect_libs.py --binary build/strata --search /usr/local/cuda/lib64 --out /stage --also 'libcublasLt.so*'; \
archs=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do archs="$archs --arch sm_$a"; done; \
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
python3 packaging/package_engine.py --src /stage --backend cuda --version "$ver" --out /engine $archs --arch "compute_$last" \
--min-driver nvidia=$mindriver --variant cuda$cudamajor --platform linux-x86_64 --build-id "docker-source-${ver}-cuda${cudafull}" --include SYSTEM-LIBS.txt --include 'libstrata_kq*' \
--include strata-vision; \
for d in /engine/*/; do python3 packaging/package_engine.py --verify "$d"; python3 packaging/binary_deps.py "$d"; done
RUN python3 -m venv .venv \
&& .venv/bin/pip install --no-cache-dir --upgrade pip \
&& .venv/bin/pip install --no-cache-dir -r requirements.txt \
&& chmod +x setup.sh docker-entrypoint.sh
# llama.cpp at the pinned commit, then the engine and the image encoder, built
# exactly the way setup.py builds them: native code for the CPU that builds the
# image, so build it on the PC it runs on. BUILD.json is what setup.py reads to
# decide whether an engine is current: source=local with a matching src hash
# means the first start reuses it instead of recompiling.
RUN .venv/bin/python - <<'PYEOF'
import json, os, pathlib, shutil
import setup
llama = setup.get_llama_cpp()
nvcc, _ = setup.find_nvcc()
arch = os.environ.get("CUDA_ARCHITECTURES", "75;80;86;89;120").strip().strip('"').replace(",", ";")
vision = "gpu" if os.environ.get("BUILD_VISION", "1") == "1" else "none"
setup.cmake_build(setup.ROOT, setup.ROOT / "build", "strata",
["-DSTRATA_ENABLE_CUDA=ON", "-DSTRATA_BUILD_TESTS=OFF",
f"-DCMAKE_CUDA_ARCHITECTURES={arch}", f"-DCMAKE_CUDA_COMPILER={nvcc}",
f"-DSTRATA_GGML_DIR={llama}"], None, "build-strata.bat")
if vision != "none":
setup.cmake_build(setup.ROOT / "tools" / "vision", setup.ROOT / "build-vision", "strata-vision",
[f"-DLLAMA_DIR={llama}", "-DSTRATA_VISION_CUDA=ON", "-DSTRATA_PORTABLE=OFF",
f"-DCMAKE_CUDA_ARCHITECTURES={arch}", f"-DCMAKE_CUDA_COMPILER={nvcc}"], None, "build-vision.bat")
eng = setup.ROOT / "engine"
eng.mkdir(exist_ok=True)
shutil.copy2(setup.ROOT / "build" / setup.EXE, eng / setup.EXE)
if (setup.ROOT / "build" / setup.KQ_LIB).exists(): # the K-quant MMQ prompt library (UD-Q4/Q5/Q6_K_XL files)
shutil.copy2(setup.ROOT / "build" / setup.KQ_LIB, eng / setup.KQ_LIB)
if vision != "none":
shutil.copy2(setup.ROOT / "build-vision" / "bin" / setup.VEXE, eng / setup.VEXE)
bindir = pathlib.Path(nvcc).parent
meta = {"source": "local", "version": setup.source_version(),
"archs": [int(a.split("-")[0]) for a in arch.split(";") if a.split("-")[0].isdigit()], "vision": vision,
"cuda_dirs": [str(d) for d in (bindir, bindir / "x64", bindir.parent / "lib64") if d.is_dir()],
"src": setup.source_hash(setup.ENGINE_SOURCES),
"vision_src": setup.source_hash(setup.VISION_SOURCES) if vision != "none" else None}
(eng / "BUILD.json").write_text(json.dumps(meta, indent=1))
PYEOF
# the cmake trees are build-time only; the engine itself is what the container needs
RUN rm -rf build build-vision
# ---- 2. the Terminal (the release archive, built here)
FROM ${RUNTIME_BASE} AS terminal
ARG STRATA_COMMIT=
ENV STRATA_BUILD_COMMIT=${STRATA_COMMIT}
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends python3 ca-certificates git && rm -rf /var/lib/apt/lists/*
COPY . /src/
RUN python3 /src/packaging/package_terminal.py --out /out --cache /cache \
&& mkdir -p /opt/strata \
&& tar -xzf /out/strata-terminal-*-linux-x86_64.tar.gz -C /opt/strata \
&& mv /opt/strata/strata-terminal-* /opt/strata/terminal \
&& /opt/strata/terminal/bin/strata --version
# llama.cpp at the commit setup.py pins: fetched here, so the first start does not depend on GitHub. Only gguf-py (the model preparation
# tools) and the file setup.py looks for (ggml/CMakeLists.txt) are kept: 170 MB become a few
RUN cd /opt/strata/terminal && STRATA_CONTROL_INSTALL=1 STRATA_INSTALL_DIR=/opt/strata/seed PYTHONNOUSERSITE=1 \
/opt/strata/terminal/python/bin/python3 -c "import setup; print(setup.get_llama_cpp())" \
&& rm -rf /opt/strata/seed/third_party/*.zip* \
&& cd /opt/strata/seed/third_party/llama.cpp \
&& find . -mindepth 1 -maxdepth 1 ! -name gguf-py ! -name ggml ! -name 'LICENSE*' -exec rm -rf {} + \
&& find ggml -mindepth 1 -depth ! -type d ! -name CMakeLists.txt -delete && find ggml -mindepth 1 -type d -empty -delete \
&& du -sh /opt/strata/seed
# ---- 3. the runtime image (the same as docker/Dockerfile's, FLAVOR=cuda)
FROM ${RUNTIME_BASE}
ARG STRATA_VERSION=dev
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends tini ca-certificates libgomp1 libatomic1 libnuma1 zlib1g \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/*
COPY --from=terminal /opt/strata/terminal /opt/strata/terminal
COPY --from=terminal /opt/strata/seed /opt/strata/seed
COPY --from=engine-build /engine/ /opt/strata/components/engine/
COPY docker/entrypoint.sh docker/healthcheck.sh /opt/strata/
COPY docker/strata-wrapper.sh /usr/local/bin/strata
RUN chmod 0755 /opt/strata/entrypoint.sh /opt/strata/healthcheck.sh /usr/local/bin/strata
ENV STRATA_COMPONENTS_DIR=/opt/strata/components STRATA_INSTALL_DIR=/data/install XDG_STATE_HOME=/data/state \
XDG_CONFIG_HOME=/data/config HOME=/data/home PORT=8080 LANG=C.UTF-8 PYTHONUNBUFFERED=1 PYTHONNOUSERSITE=1 STRATA_IN_DOCKER=1 \
NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=compute,utility
VOLUME ["/data"]
EXPOSE 8080
# /health is answered before the API key gate, so it works with or without one.
# The port only opens after the model loads (1-3 minutes, longer on a first run),
# so the start period is generous: a too short one marks a still-loading container
# unhealthy and a restart policy would kill it mid-download.
HEALTHCHECK --interval=30s --timeout=5s --start-period=600s --retries=3 \
CMD curl -fs "http://127.0.0.1:${PORT:-8080}/health" || exit 1
ENTRYPOINT ["./docker-entrypoint.sh"]
LABEL org.opencontainers.image.title="Strata" com.strata.flavor="cuda" com.strata.build="from-source" org.opencontainers.image.version="${STRATA_VERSION}"
HEALTHCHECK --interval=30s --timeout=8s --start-period=2h --retries=3 CMD ["/opt/strata/healthcheck.sh"]
STOPSIGNAL SIGTERM
ENTRYPOINT ["/usr/bin/tini", "--", "/opt/strata/entrypoint.sh"]
CMD ["run"]
+132 -183
View File
@@ -1,239 +1,188 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · **Deutsch** · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<h3 align="center">Leistungsstarke KI auf deinem eigenen Computer.</h3>
<p align="center">Chatten, programmieren und mit Bildern arbeiten, lokal.</p>
<p align="center"><b>Ein KI-Modell mit 125 Milliarden Parametern auf deinem eigenen Gaming-PC</b><br>
NVIDIA- oder AMD-Grafikkarte (ab 12 GB) · Windows oder Linux · kostenlos und Open Source</p>
<p align="center">
<a href="#modelle">Modelle</a> · <a href="#installation"><b>Installation</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#tools">Tools</a> ·
<a href="#hilfe">Hilfe</a> · <a href="#strata-unterstützen">Unterstützen</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Ein Voxel-Pagodengarten, den das Modell in Strata geschrieben hat, läuft im Browser"></a><br>
<sub>Ein Voxel-Pagodengarten aus einem einzigen Prompt, auf einer RTX 5070 mit Strata (IQ3_S, 128K Kontext) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">ganzes Video (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Strata herunterladen"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT-Lizenz"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Strata auf Buy Me a Coffee unterstützen"></a>
</p>
Strata lässt **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** auf einem normalen PC laufen.
Das ist ein großes, kluges KI-Modell, das sonst meist einen Server braucht. Es chattet, schreibt Code, versteht
Bilder und arbeitet mit deinen Apps und Coding-Agents zusammen. Nichts verlässt deinen PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <b>Deutsch</b> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
## Wie schnell ist es?
**Kostenlos, Open Source und ganz in deiner Hand.** Nutze Strata auf dem Desktop oder mit deinen liebsten Coding-Tools. Kein Konto nötig.
Wir haben es auf zwei ganz normalen Gaming-PCs gemessen. Ein Token ist etwa ¾ eines Wortes.
## Modelle
- **Schreibt Antworten:** wie schnell die Antwort in einem kurzen Chat erscheint. 60 Tokens pro Sekunde sind
schneller, als du lesen kannst.
- **Liest deinen Prompt:** wie schnell es aufnimmt, was du schickst (hier ein Dokument, Code oder Chatverlauf mit
32K Tokens).
Wir empfehlen **zuerst ISTA-DASLab**, danach **Unsloth** und andere Anbieter, soweit ein passendes Modell in deinen Speicher passt.
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Größe | Schreibt Antworten | Liest deinen Prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 Tokens/s | 2,650 Tokens/s |
| **IQ2_XS** | 79 Tokens/s | 2,090 Tokens/s |
| **IQ3_XXS** | 62 Tokens/s | 1,750 Tokens/s |
| **IQ3_S** | 53 Tokens/s | 1,620 Tokens/s |
| **Coder** | 55 Tokens/s | 2,180 Tokens/s |
</td><td>
| Größe | Schreibt Antworten | Liest deinen Prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 Tokens/s | 1,160 Tokens/s |
| **IQ2_XS** | 52 Tokens/s | 1,110 Tokens/s |
| **Coder** | 44 Tokens/s | 1,420 Tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 mit Engine 0.1.36, die anderen Zeilen mit 0.1.26 (4K Antworten, 32K Prompts). Die vollständigen
Tabellen stehen in [DETAILS.md](docs/DETAILS.md#speed-measured). Eine Karte mit mehr VRAM ist schneller: Eine
RTX 3090 (24 GB) sollte etwa 100-140 Tokens pro Sekunde schreiben. Lange Chats und andere Karten:
[Tempo jedes Modells](docs/MODELS.md#how-fast-is-each-size), [Ergebnisse aus der Community](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata ist kostenlos. Wenn es auf deinem PC gut läuft, hält ein Kaffee die Arbeit daran am Laufen.</sub></p>
## Was du brauchst
| | |
| Anbieter | Auswahl |
| --- | --- |
| **Grafikkarte** | **NVIDIA** GeForce RTX 20, 30, 40 oder 50 Serie, oder **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 oder RX 6800 / 6900 Serie. Sie braucht **12 GB VRAM oder mehr**. |
| **RAM** | 32 GB oder mehr. Dein RAM entscheidet, [welches Modell](#welches-modell-soll-ich-nehmen) passt. Mit 64 GB laufen alle Größen. |
| **Festplatte** | Etwa 80 GB frei. Nimm wenn möglich eine SSD: Der erste Start geht dann viel schneller. |
| **System** | Windows 10 / 11 oder Linux und ein aktueller Grafiktreiber von NVIDIA oder AMD. |
| **ISTA-DASLab · empfohlen** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) zum Programmieren. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Geringerer Speicherbedarf, aber langsamer. |
| **Weitere Optionen** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
Alles andere richtet der Installer ein. Zwei oder drei Karten können sich das Modell teilen ([Multi-GPU](docs/MULTI_GPU.md)).
[Größen, Speicherbedarf und Kompatibilität vergleichen](docs/MODELS.md).
Experimentell, von Community-Mitgliedern auf ihren eigenen Rechnern geschrieben und getestet:
</details>
- **Ältere Grafikkarten** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Ältere GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, unter Linux aus dem Quellcode gebaut: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, unter Linux aus dem Quellcode gebaut: [Strix Halo](docs/STRIX_HALO.md).
- **Ältere Prozessoren ohne AVX2**: Sie funktionieren, aber langsam. [Ältere CPUs](docs/INSTALL.md#older-cpus-experimental).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
Die vollständige Liste: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Das sind die derzeit dokumentierten Pakete für dieses Modell.
## Installieren
[Modellanleitung und Kompatibilität](docs/QWEN36.md).
### Lass deine KI es einrichten
</details>
Nutzt du einen KI-Coding-Assistenten (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Füge dort diesen Text ein:
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
In der **Desktop-App** und im **Terminal**. Was nicht in den Arbeitsspeicher passt, wird von deiner SSD gelesen. Darum brauchst du viel freien Speicherplatz.
| Set (IST-DAS) | Download | Freier Speicherplatz nötig |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | etwa 245 GB |
| 2.00 bpw | 257 GB | etwa 265 GB |
| 2.25 bpw | 274 GB | etwa 280 GB |
Läuft auf NVIDIA RTX 30, 40 und 50 (Windows und Linux) und auf AMD Radeon AI PRO R9700 (Linux). Das Setup lädt die Dateien und die DeepSeek-Engine herunter und startet dann das Modell.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Er prüft deine Grafikkarte, deinen RAM und deine Festplatte und wählt das passende Modell. Dann installiert und
startet er es und sagt dir, wie du deine Apps verbindest. KI-Tools können Strata auch über seinen
[MCP-Server](docs/MCP_SERVER.md) installieren, starten und stoppen.
Gemessen mit dem 2.00-bpw-Set auf Strata 0.2.5. „Ausgabe“ ist das Tempo der Antwort, „Prompt“ zeigt, wie schnell Strata deine Eingabe liest.
### Oder mach es selbst
| Hardware | Ausgabe | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4 bis 6 Token/s | etwa 110 Token/s (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9 bis 11 Token/s | etwa 155 Token/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21 bis 25 Token/s | 320 bis 360 Token/s (8K) |
| 4x Radeon AI PRO R9700 | 74 bis 77 Token/s | 400 bis 450 Token/s (8K) |
[Lade Strata herunter](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) und entpacke es (oder nutze
`git clone`). **Windows:** Doppelklick auf **`START-HERE.bat`**. **Linux:** Führe im Strata-Ordner **`./setup.sh`** aus.
[Speicher, Festplatte und Details](docs/DEEPSEEK41.md) · [Versionshinweise](https://github.com/Niko1221/Strata/releases)
**Mac (Apple Silicon, M1 bis M5, macOS 15 oder neuer):** Installiere einmal Apples Command Line Tools (`xcode-select --install` im Terminal; Xcode selbst brauchst du nicht). Öffne dann das Terminal im Strata-Ordner und führe **`./setup.sh`** aus. Es baut die Mac-Engine auf deinem Mac (etwa 7 Minuten), stellt dieselben Fragen, lädt das Modell und startet es. Beim nächsten Mal startet `./setup.sh` sofort. Welche Größe zu deinem Mac passt und wie schnell es ist: [docs/MAC.md](docs/MAC.md).
</details>
Die Schritte sind für NVIDIA und AMD gleich. Der Installer erkennt deine Karte und richtet die passende Engine
dafür ein. Er stellt dir ein paar Fragen:
<a id="the-other-sizes"></a>
- welches Modell und welche Größe,
- wie viel Kontext (wie viel Text sich das Modell merkt),
- ob es Bilder lesen soll.
Unsicher, was passt? [Wähle ein Modell für deinen Computer](docs/MODELS.md#pick-by-memory).
Drück jedes Mal Enter, um die empfohlene Antwort zu nehmen. Dann lädt er das Modell herunter (etwa 70 GB) und
startet es. Bricht der Download ab, starte ihn einfach neu: Er macht dort weiter, wo er aufgehört hat. Dein Browser
öffnet die Strata-App unter `http://127.0.0.1:8080`.
## Installation
> **Während das Modell startet, kann dein PC 1-3 Minuten langsam sein oder nicht reagieren** (beim ersten Mal am
> längsten). Strata lädt 35-55 GB in deinen RAM und reserviert einen Teil davon für die Grafikkarte. Das ist normal.
> Warte und schließ das Fenster nicht. Im Fenster siehst du, was Strata gerade tut.
<a id="option-a-the-strata-app"></a>
**Beim nächsten Mal** startest du wieder `START-HERE.bat` (oder `./setup.sh`). Es startet sofort und lädt nichts
doppelt herunter. Schließ das Fenster, um das Modell zu stoppen. `UPDATE.bat` (`./update.sh`) aktualisiert Strata,
ohne es zu starten. Updates, Docker, mehrere Karten, wo die Dateien landen und alle Optionen:
[docs/INSTALL.md](docs/INSTALL.md).
### Desktop-App
## Welches Modell soll ich nehmen?
**[Strata für Windows, Linux oder Mac herunterladen →](https://github.com/Niko1221/Strata/releases/latest)**
Der Installer empfiehlt eins passend zu deinem RAM. Dasselbe Modell gibt es in mehreren Größen, mehr oder weniger
stark komprimiert. Kleinere Größen sind schneller. Größere sind etwas klüger.
Installiere die App, wähle ein Modell und leg mit dem Chat los. Es gibt einen Installer pro System. Beim ersten Start lädt die App die Engine für deine Grafikkarte herunter. [Hilfe zum Installer](docs/INSTALL.md#the-app).
| Dein RAM | Nimm | Warum |
| --- | --- | --- |
| **32 GB** | **Coder** | passt in 32 GB und ist für Code gemacht (mit einer 24-GB-Karte laufen auch Q2_0 und IQ2_XS) |
| **48 GB** | **IQ2_XS** (oder Q2_0, das schnellste) | die größeren Größen passen nicht |
| **64 GB** | **IQ2_XS** (empfohlen) oder IQ3_XXS / IQ3_S | alle Größen passen; IQ3_S ist das beste und das langsamste |
| **96 GB oder mehr** | **IQ3_S** oder Unsloths UD-IQ4_XS (~4 Bit) | Platz für die größten Größen, auch wenn alles andere offen ist |
<sub>Dateien: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
- **[Coder](docs/MODELS.md#coder):** eine Version für Code, bei der die Hälfte der Experten entfernt wurde. Sie
erreicht 91 % des SWE-bench-Verified-Werts des vollen Modells (von ihren Autoren gemessen) und passt in 32 GB RAM.
Außerhalb von Code ist sie schwächer, auch bei chinesischem und anderem CJK-Text (#438). Nimm dafür Q2_0, IQ2_XS
oder IQ3_S, die alle Experten behalten.
- **[Swift 1.5](docs/MODELS.md#swift-15):** ein Fine-Tune, das viel kürzer nachdenkt, bevor es antwortet. Du
bekommst die Antwort früher, bei etwa gleicher Qualität.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** die ~4-Bit-Version von Unsloth, in der Qualität
zwischen IQ3_S und UD-Q4_K_XL. 94 GB Download. Mit weniger als ~80 GB RAM liest Strata einen Teil davon während
der Antwort von der SSD, dort ist es also langsamer (eine NVMe-SSD hilft).
- **[Unsloth UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL und Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):** 4 bis 8 Bit, näher am vollen Modell
(93-95 % das gleiche nächste Token wie Q8_0). Schnell, wenn alle Experten in den RAM passen (~88 GB für UD-Q4_K_XL
bis ~136 GB für Q8_0); mit weniger RAM liest Strata einen Teil davon während der Antwort von der SSD, was viel
langsamer ist (7-8.5 Tokens/s für UD-Q4_K_XL auf einem PC mit 64 GB).
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** Das richtest du von Hand
ein. Es steht nicht im Menü des Installers.
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Strata-Desktop-App mit Chat und Live-Codevorschau nebeneinander">
<br><sub>Chat, Dateien und Live-Vorschau in einem Fenster.</sub>
</p>
Größen, Downloads und was wohin passt: [docs/MODELS.md](docs/MODELS.md). Um später ein weiteres Modell
hinzuzufügen, starte `SETUP.bat` (Linux: `./setup.sh --setup`).
<a id="option-b-the-terminal-one-line"></a>
## So benutzt du es
### Terminal-App
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="Der Monitor-Tab der Strata-App neben einem Coding-Agent"><br>
<sub>Der <b>Monitor</b> der Strata-App (links), während ein Coding-Agent den Pagodengarten aus dem Video schreibt (rechts)</sub></p>
Lieber im Terminal? Installiere Strata und starte dann **`strata launch`**.
- **Im Browser:** Öffne `http://127.0.0.1:8080`. Dort gibt es **Chat**, einen Live-**Monitor** für das Modell und
deine GPU/CPU/RAM und **About** mit den Einstellungen und Adressen.
- **Deine Apps und Coding-Agents:** Füge einen „OpenAI-kompatiblen“ Anbieter mit der Basis-URL
**`http://127.0.0.1:8080/v1`** hinzu. Jeder API-Key und jeder Modellname funktioniert.
- Apps, die die API von Anthropic nutzen: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI und andere Apps, die die OpenAI Responses API nutzen: `/v1/responses`
([Einrichtung](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Nachdenken:** Wähle **aus, niedrig, mittel oder hoch** im Chat-Menü oder unter „Reasoning Effort“ in deiner
App. Aus ist am schnellsten. Hoch ist am besten für schwierige Fragen.
- **Bilder:** Antworte im Setup mit Ja auf „Images?“. Klick dann im Chat auf **Picture** oder hänge Bilder in deiner
App an. AMD-Karten lesen Bilder unter Linux über den Prozessor; unter Windows geht das noch nicht.
- **Vom Handy oder einem anderen PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Setz immer einen
Key.
- **Eine Anfrage nach der anderen:** Standardmäßig beantwortet Strata eine Anfrage, die anderen warten. Um mehrere
gleichzeitig zu beantworten, setz `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). Auf einer 12-GB-Karte wird
dadurch jede Antwort langsamer.
- **Lange Prompts:** Strata liest die erste Nachricht eines Chats komplett, etwa 1 Minute pro 30,000 Tokens.
Folgenachrichten starten in Sekunden.
<details open>
<summary><b>Installationsbefehle</b></summary>
Mehr: [wo deine Chats gespeichert werden](docs/INSTALL.md#where-things-are-stored), [die API](docs/DETAILS.md#using-it).
**Linux / Mac**
### Optional: mehr Tempo mit einer Grafikkarte, wenn das ganze Modell in den RAM passt
Zwei Opt-in-Einstellungen (von RubenTorney, #1904 und #1905). Sie helfen, wenn du eine Grafikkarte nutzt und dein RAM alle Experten hält (Stratas Arena-Modus, zum Beispiel Q2_0 oder IQ3_XXS auf einem PC mit 64 GB). Öffne deine Konfiguration `strata-<model>.json` im Strata-Datenordner ([wo](docs/INSTALL.md#where-things-are-stored)) und füge hinzu:
- **Schnellere Ausgabe:** `"--adapt-async", "1"` in die Liste `"args"`. Der Experten-Cache wird dann in einem Hilfs-Thread aktualisiert, statt die Antwort anzuhalten. Gemessen auf einer RTX 5070 (Windows, Q2_0): +4-5 % Ausgabe, +7-10 % bei Code und langen Dokumenten. Antworten können leicht vom Standard abweichen (sie bleiben von Lauf zu Lauf gleich).
- **Schnelleres Prompt-Lesen unter Windows:** `"STRATA_PIN_BY_NEED": "1"` im Block `"env"`. Windows kann nur etwa den halben RAM für die Grafikkarte sperren; damit werden zuerst die Experten gesperrt, die die Karte wirklich braucht. Gemessen auf der RTX 5070: +4,4 % Prompt-Lesen.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Starte Strata danach neu (`START-HERE.bat` oder `./setup.sh`). Zum Rückgängigmachen entferne die beiden Einträge.
**Windows PowerShell**
## Etwas ist schiefgelaufen?
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
- **Mein PC ist beim ersten Start von Strata eingefroren.** Das ist normal, während das Modell lädt. Warte und
schließ das Fenster nicht. Nach 10 Minuten immer noch eingefroren? Starte den PC neu, schließ andere Programme und
versuch es noch einmal, oder nimm eine kleinere Größe.
- **Es hat beim Herunterladen oder Installieren aufgehört.** Starte `START-HERE.bat` (oder `./setup.sh`) noch
einmal. Es macht dort weiter, wo es aufgehört hat.
- **Es ist sehr langsam und die Festplattenlampe blinkt ständig, oder es meldet „the engine stopped
unexpectedly“.** Dein PC hat nicht genug freien RAM. Schließ andere Programme (Browser brauchen viel) oder nimm
eine kleinere Größe (Q2_0 oder IQ2_XS).
- **Es meldet, dass Port 8080 schon belegt ist.** Strata läuft bereits. Such nach seinem Fenster.
Öffne unter Windows danach ein neues Terminal. Dann:
Weitere Probleme und ihre Lösungen: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Kommst du nicht weiter?
Öffne ein [Issue](https://github.com/Niko1221/Strata/issues) und hänge `strata-<model>.log` aus dem Strata-Ordner
an. Du hast ein Sicherheitsproblem gefunden? Melde es vertraulich: [SECURITY.md](SECURITY.md).
```sh
strata launch
```
## Wie funktioniert es?
Folge den Anweisungen, um ein Modell herunterzuladen und zu starten. [Terminal-Anleitung](docs/TERMINAL_LAUNCH.md).
Modelle wie dieses laufen normalerweise auf Servern mit Hunderten Gigabyte Grafikspeicher. Deine Grafikkarte hat
12-24 GB. Strata bringt das Modell trotzdem unter, indem es **die Arbeit auf deinen ganzen PC verteilt**. Stell dir
eine Küche vor: Was du ständig brauchst, bleibt auf der Arbeitsfläche, der Rest wartet in der Vorratskammer.
</details>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Die 24,576 Experten des Modells: die meistgenutzten auf der Grafikkarte, alle im RAM, eine Nachschlagetabelle auf der SSD"></p>
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Strata-Terminal-App mit Modellaktivität, Tempo, Speicher und letzten Anfragen">
<br><sub>Behalte Modell und Hardware im Blick, während es läuft.</sub>
</p>
- **Das Modell ist ein Team aus 24,576 kleinen Spezialisten („Experten“).** Jedes Wort braucht nur 10 davon.
- **Deine Grafikkarte** hält die paar tausend Experten, die am häufigsten gebraucht werden. **Dein RAM** hält alle,
und **dein Prozessor** rechnet gleichzeitig am Rest. **Deine SSD** hält eine große Nachschlagetabelle.
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Ein kleiner Helfer rät die nächsten Wörter; das große Modell prüft alle auf einmal und behält die richtigen"></p>
[Aus dem Quellcode bauen](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Vollständige Installationsanleitung](docs/INSTALL.md)
- **Raten, dann prüfen:** Ein kleiner Helfer rät die nächsten paar Wörter. Das große Modell prüft sie alle auf
einmal. Du bekommst dieselbe Antwort, nur 1.6-1.8x schneller.
- **Lange Texte werden in großen Stücken gelesen** (bis zu 8,192 Tokens auf einmal), mit über 1,000 Tokens pro
Sekunde.
<a id="what-you-need"></a>
Die längere Erklärung: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Jeder Teil und seine Zahlen:
[die Details](docs/DETAILS.md#how-it-works) und das [Paper](docs/paper/Strata-Paper.pdf).
## Hardware
## Danksagung und Lizenz
| Unterstützte Hardware | Anleitung |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [NVIDIA-Setup](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [AMD-Setup](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimentell** | [RX-6000-Setup](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel-Setup](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 oder neuer** | [Mac-Setup](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix-Halo-Setup](docs/STRIX_HALO.md) |
Das Modell ist [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) vom Qwen-Team. Komprimiert
wurde es von [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
und Unsloth. Strata nutzt Teile von [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Alle Danksagungen:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata ist Open Source unter der [MIT-Lizenz](LICENSE). Ein
paar Teile und alle Modelle haben eigene Lizenzen ([welche](docs/HOW_IT_WORKS.md#license)).
<a id="using-it"></a>
## Tools
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, und weitere über die OpenAI- und Anthropic-kompatiblen APIs.
Führe **`strata connect`** aus, um die Einrichtung für dein Tool zu sehen. [Verbindungsanleitung](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## Hilfe
[Alle Anleitungen](docs) · [Fehlerbehebung](docs/TROUBLESHOOTING.md) · [Fehler melden](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [So funktioniert es](docs/HOW_IT_WORKS.md) · [Mitmachen](CONTRIBUTING.md)
## Strata unterstützen
Strata ist kostenlos und Open Source. Wenn es dir nützt, kannst du die Entwicklung unterstützen:
**Hilf mit, lokale KI für alle offen zu halten.** Ein Kaffee hilft, Entwicklung und Tests auf mehr Hardware zu finanzieren.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: Strata unterstützen" height="48"></a>
[MIT-Lizenz](LICENSE) · [Danksagung und Modelllizenzen](docs/HOW_IT_WORKS.md#credits) · [Datenschutz](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Für KI-Tools und automatische Leser: <a href="AGENTS.md">technische Referenz</a> · <a href="docs/AI_SETUP.md">Setup-Anleitung</a>.</sub>
**Sponsoring, Hardware-Partnerschaften oder Zusammenarbeit:** [[email protected]](mailto:[email protected]).
+149 -194
View File
@@ -1,233 +1,188 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · **Español** · [Português](README.pt-BR.md)
<h3 align="center">IA potente en tu propio ordenador.</h3>
<p align="center">Chatea, programa y trabaja con imágenes, en local.</p>
<p align="center"><b>Ejecuta un modelo de IA de 125 mil millones de parámetros en tu propio PC gaming</b><br>
Tarjeta gráfica NVIDIA o AMD (12 GB o más) · Windows o Linux · libre y de código abierto</p>
<p align="center">
<a href="#modelos">Modelos</a> · <a href="#instalación"><b>Instalación</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#herramientas">Herramientas</a> ·
<a href="#ayuda">Ayuda</a> · <a href="#apoya-strata">Apoyar</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Un jardín con una pagoda de vóxeles que escribió el modelo de Strata, funcionando en el navegador"></a><br>
<sub>Un jardín con una pagoda de vóxeles, hecho con un solo prompt en una RTX 5070 con Strata (IQ3_S, contexto de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vídeo completo (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Descargar Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licencia MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Apoya Strata en Buy Me a Coffee"></a>
</p>
Strata ejecuta **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** en un PC normal. Es un
modelo de IA grande e inteligente que normalmente necesita un servidor. Conversa, escribe código, entiende imágenes
y funciona con tus aplicaciones y agentes de programación. Nada sale de tu PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <b>Español</b> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
## ¿Qué velocidad tiene?
**Gratis, de código abierto y tuyo para ejecutar.** Usa Strata en tu escritorio o con tus herramientas de programación favoritas. No necesitas cuenta.
Lo medimos en dos PC gaming normales. Un token equivale más o menos a ¾ de una palabra.
## Modelos
- **Escribe respuestas:** lo rápido que aparece la respuesta en un chat corto. 60 tokens por segundo es más rápido de lo que puedes leer.
- **Lee tu prompt:** lo rápido que procesa lo que le envías (aquí, un documento, código o historial de chat de 32K tokens).
Recomendamos **primero ISTA-DASLab**, después **Unsloth** y otros proveedores, siempre que haya un modelo compatible que quepa en tu memoria.
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Tamaño | Escribe respuestas | Lee tu prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| Tamaño | Escribe respuestas | Lee tu prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 con el motor 0.1.36, las demás filas con 0.1.26 (respuestas de 4K, prompts de 32K). Las tablas
completas están en [DETAILS.md](docs/DETAILS.md#speed-measured). Una tarjeta con más VRAM es más rápida: una
RTX 3090 (24 GB) debería escribir unos 100-140 tokens por segundo. Chats largos y otras tarjetas:
[velocidad de cada modelo](docs/MODELS.md#how-fast-is-each-size), [resultados de la comunidad](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata es gratis. Si funciona bien en tu PC, un café ayuda a seguir trabajando en él.</sub></p>
## Qué necesitas
| | |
| Proveedor | Opciones |
| --- | --- |
| **Tarjeta gráfica** | **NVIDIA** GeForce serie RTX 20, 30, 40 o 50, o **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 o serie RX 6800 / 6900. Necesita **12 GB de VRAM o más**. |
| **RAM** | 32 GB o más. Tu RAM decide [qué modelo](#qué-modelo-elijo) cabe. Con 64 GB funcionan todos los tamaños. |
| **Disco** | Unos 80 GB libres. Usa un SSD si puedes: el primer arranque es mucho más rápido. |
| **Sistema** | Windows 10 / 11 o Linux, y un controlador gráfico actual de NVIDIA o AMD. |
| **ISTA-DASLab · recomendado** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) para programar. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Usa menos memoria, pero es más lento. |
| **Otras opciones** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
El instalador prepara todo lo demás. Dos o tres tarjetas pueden repartirse el modelo ([multi-GPU](docs/MULTI_GPU.md)).
[Compara tamaños, necesidades de memoria y compatibilidad](docs/MODELS.md).
Experimental, escrito y probado por miembros de la comunidad en sus propios equipos:
</details>
- **Tarjetas gráficas antiguas** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, compilado desde el código fuente en Linux: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, compilado desde el código fuente en Linux: [Strix Halo](docs/STRIX_HALO.md).
- **Procesadores antiguos sin AVX2**: funcionan, pero despacio. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
La lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Son los paquetes documentados actualmente para este modelo.
[Guía del modelo y compatibilidad](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
En la **aplicación de escritorio** y en el **Terminal**. Lo que no cabe en la memoria se lee desde tu SSD, así que hace falta mucho espacio libre en disco.
| Conjunto (IST-DAS) | Descarga | Espacio libre necesario |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | unos 245 GB |
| 2.00 bpw | 257 GB | unos 265 GB |
| 2.25 bpw | 274 GB | unos 280 GB |
Funciona en NVIDIA RTX 30, 40 y 50 (Windows y Linux) y en AMD Radeon AI PRO R9700 (Linux). La instalación descarga los archivos y el motor de DeepSeek, y después inicia el modelo.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Medido con el conjunto de 2.00 bpw en Strata 0.2.5. «Salida» es la velocidad de la respuesta; «Prompt» es la rapidez con la que Strata lee tu texto.
| Hardware | Salida | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB de RAM | 4 a 6 tokens/s | unos 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB de RAM | 9 a 11 tokens/s | unos 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB de RAM | 21 a 25 tokens/s | 320 a 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 a 77 tokens/s | 400 a 450 tokens/s (8K) |
[Memoria, disco y detalles](docs/DEEPSEEK41.md) · [Notas de la versión](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
¿No sabes qué cabe? [Elige un modelo para tu ordenador](docs/MODELS.md#pick-by-memory).
## Instalación
### Deja que tu IA lo instale
<a id="option-a-the-strata-app"></a>
¿Usas un asistente de programación con IA (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Pega esto en él:
### Aplicación de escritorio
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
**[Descargar Strata para Windows, Linux o Mac →](https://github.com/Niko1221/Strata/releases/latest)**
Instala la aplicación, elige un modelo y empieza a chatear. Hay un instalador por sistema. En el primer arranque, la aplicación descarga el motor para tu tarjeta gráfica. [Ayuda con el instalador](docs/INSTALL.md#the-app).
<sub>Archivos: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Aplicación de escritorio de Strata con un chat y una vista previa del código en vivo, una junto a otra">
<br><sub>Chat, archivos y vista previa en vivo en una sola ventana.</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### Aplicación de Terminal
¿Prefieres la terminal? Instala Strata y ejecuta **`strata launch`**.
<details open>
<summary><b>Comandos de instalación</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Revisa tu tarjeta gráfica, tu RAM y tu disco, y elige el modelo que cabe. Luego lo instala, lo arranca y te explica
cómo conectar tus aplicaciones. Las herramientas de IA también pueden instalar, arrancar y detener Strata con su
[servidor MCP](docs/MCP_SERVER.md).
**Windows PowerShell**
### O hazlo tú mismo
[Descarga Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) y descomprímelo (o usa `git clone`).
**Windows:** haz doble clic en **`START-HERE.bat`**. **Linux:** ejecuta **`./setup.sh`** en la carpeta de Strata.
**Mac (Apple Silicon, M1 a M5, macOS 15 o posterior):** instala una vez las Command Line Tools de Apple (`xcode-select --install` en Terminal; no hace falta Xcode). Luego abre Terminal en la carpeta de Strata y ejecuta **`./setup.sh`**. Compila el motor de Mac en tu Mac (unos 7 minutos), hace las mismas preguntas, descarga el modelo y lo arranca. La próxima vez, `./setup.sh` arranca enseguida. Qué tamaño cabe en tu Mac y qué velocidad tiene: [docs/MAC.md](docs/MAC.md).
Los pasos son los mismos para NVIDIA y AMD. El instalador detecta tu tarjeta y prepara el motor adecuado para ella.
Te hace unas pocas preguntas:
- qué modelo y qué tamaño,
- cuánto contexto (cuánto texto tiene presente el modelo),
- si debe entender imágenes.
Pulsa Enter cada vez para usar la respuesta recomendada. Después descarga el modelo (unos 70 GB) y lo arranca. Si la
descarga se detiene, vuelve a ejecutarlo: sigue donde lo dejó. Tu navegador abre la aplicación de Strata en
`http://127.0.0.1:8080`.
> **Mientras el modelo arranca, tu PC puede ir lento o dejar de responder durante 1-3 minutos** (más la primera vez).
> Strata carga 35-55 GB en tu RAM y reserva una parte para la tarjeta gráfica. Es normal. Espera y no cierres la
> ventana. La ventana muestra lo que está haciendo Strata.
**La próxima vez**, vuelve a ejecutar `START-HERE.bat` (o `./setup.sh`). Arranca enseguida y no descarga nada dos
veces. Cierra su ventana para detener el modelo. `UPDATE.bat` (`./update.sh`) actualiza Strata sin arrancarlo.
Actualizaciones, Docker, varias tarjetas, dónde se guardan los archivos y todas las opciones: [docs/INSTALL.md](docs/INSTALL.md).
## ¿Qué modelo elijo?
El instalador te recomienda uno según tu RAM. El mismo modelo viene en varios tamaños, más o menos comprimidos. Los
tamaños pequeños son más rápidos. Los grandes son algo más inteligentes.
| Tu RAM | Elige | Por qué |
| --- | --- | --- |
| **32 GB** | **Coder** | cabe en 32 GB y está hecho para programar (con una tarjeta de 24 GB también funcionan Q2_0 e IQ2_XS) |
| **48 GB** | **IQ2_XS** (o Q2_0, el más rápido) | los tamaños más grandes no caben |
| **64 GB** | **IQ2_XS** (recomendado), o IQ3_XXS / IQ3_S | caben todos los tamaños; IQ3_S es el mejor y el más lento |
| **96 GB o más** | **IQ3_S**, o el UD-IQ4_XS de Unsloth (~4 bits) | espacio para los tamaños más grandes con todo lo demás abierto |
- **[Coder](docs/MODELS.md#coder):** una versión para programar a la que se le quitó la mitad de los expertos. Alcanza
el 91% de la puntuación SWE-bench Verified del modelo completo (medido por sus autores) y cabe en 32 GB de RAM. Es
más flojo fuera del código, también con chino y otros textos CJK (#438). Para eso, elige Q2_0, IQ2_XS o IQ3_S, que
conservan todos los expertos.
- **[Swift 1.5](docs/MODELS.md#swift-15):** un ajuste fino que piensa mucho menos tiempo antes de responder. Recibes
la respuesta antes, con más o menos la misma calidad.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** la versión de ~4 bits de Unsloth, entre IQ3_S y
UD-Q4_K_XL en calidad. Una descarga de 94 GB. Con menos de ~80 GB de RAM, Strata lee una parte desde el SSD
mientras responde, así que ahí es más lento (un SSD NVMe ayuda).
- **[Unsloth UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL y Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):** de 4 a 8 bits, más cerca del modelo completo
(93-95 % el mismo token siguiente que Q8_0). Rápidos cuando todos los expertos caben en la RAM (~88 GB para
UD-Q4_K_XL hasta ~136 GB para Q8_0); con menos RAM, Strata lee una parte desde el SSD mientras responde, lo que es
mucho más lento (7-8.5 tokens/s para UD-Q4_K_XL en un PC con 64 GB).
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** se instala a mano. No está en
el menú del instalador.
Tamaños, descargas y qué cabe dónde: [docs/MODELS.md](docs/MODELS.md). Para añadir otro modelo más adelante, ejecuta
`SETUP.bat` (Linux: `./setup.sh --setup`).
## Cómo usarlo
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="La pestaña Monitor de la aplicación de Strata junto a un agente de programación"><br>
<sub>El <b>Monitor</b> de la aplicación de Strata (izquierda) mientras un agente de programación escribe el jardín de la pagoda del vídeo (derecha)</sub></p>
- **En el navegador:** abre `http://127.0.0.1:8080`. Tiene **Chat**, un **Monitor** en vivo del modelo y de tu
GPU/CPU/RAM, y **About** con los ajustes y las direcciones.
- **Tus aplicaciones y agentes de programación:** añade un proveedor "compatible con OpenAI" con la URL base
**`http://127.0.0.1:8080/v1`**. Sirve cualquier clave de API y cualquier nombre de modelo.
- Aplicaciones que usan la API de Anthropic: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI y otras aplicaciones que usan la API Responses de OpenAI: `/v1/responses`
([configuración](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Razonamiento:** elige **off, low, medium o high** en el menú del chat o en el "reasoning effort" de tu aplicación.
Off es lo más rápido. High es lo mejor para preguntas difíciles.
- **Imágenes:** responde que sí a "Images?" en la instalación. Luego haz clic en **Picture** en el chat, o adjunta
imágenes en tu aplicación. Las tarjetas AMD leen imágenes en Linux a través del procesador; en Windows todavía no.
- **Desde tu móvil u otro PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Pon siempre una clave.
- **Una petición cada vez:** por defecto Strata responde una petición y las demás esperan. Para responder varias a la
vez, pon `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). En una tarjeta de 12 GB, cada respuesta va más lenta.
- **Prompts largos:** Strata lee entero el primer mensaje de un chat, más o menos 1 minuto por cada 30,000 tokens. Los
mensajes siguientes empiezan en segundos.
Más: [dónde se guardan tus chats](docs/INSTALL.md#where-things-are-stored), [la API](docs/DETAILS.md#using-it).
### Opcional: más velocidad con una sola tarjeta cuando todo el modelo cabe en la RAM
Dos ajustes opcionales (de RubenTorney, #1904 y #1905). Ayudan si usas una tarjeta gráfica y tu RAM guarda todos los expertos (el modo arena de Strata, por ejemplo Q2_0 o IQ3_XXS en un PC con 64 GB). Abre tu configuración `strata-<model>.json` en la carpeta de datos de Strata ([dónde](docs/INSTALL.md#where-things-are-stored)) y añade:
- **Generación más rápida:** `"--adapt-async", "1"` en la lista `"args"`. La caché de expertos se actualiza entonces en un hilo auxiliar en vez de pausar la respuesta. Medido en una RTX 5070 (Windows, Q2_0): +4-5 % de generación, +7-10 % en código y documentos largos. Las respuestas pueden diferir un poco de las por defecto (son iguales de una ejecución a otra).
- **Lectura del prompt más rápida en Windows:** `"STRATA_PIN_BY_NEED": "1"` en el bloque `"env"`. Windows solo puede bloquear cerca de la mitad de la RAM para la tarjeta; esto bloquea primero los expertos que la tarjeta necesita de verdad. Medido en la RTX 5070: +4,4 % en la lectura del prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Reinicia Strata (`START-HERE.bat` o `./setup.sh`) después del cambio. Para deshacerlo, quita las dos entradas.
En Windows, abre una terminal nueva después. Luego:
## ¿Algo salió mal?
```sh
strata launch
```
- **Mi PC se congeló la primera vez que arrancó Strata.** Es normal mientras carga el modelo. Espera y no cierres la
ventana. ¿Sigue congelado después de 10 minutos? Reinicia el PC, cierra otros programas y vuelve a intentarlo, o
elige un tamaño más pequeño.
- **Se detuvo durante la descarga o la instalación.** Vuelve a ejecutar `START-HERE.bat` (o `./setup.sh`). Sigue
donde se detuvo.
- **Va muy lento y la luz del disco no deja de parpadear, o dice "the engine stopped unexpectedly".** Tu PC no tiene
suficiente RAM libre. Cierra otros programas (los navegadores usan mucha), o elige un tamaño más pequeño (Q2_0 o
IQ2_XS).
- **Dice que el puerto 8080 ya está en uso.** Strata ya está en marcha. Busca su ventana.
Sigue las indicaciones para descargar e iniciar un modelo. [Guía del Terminal](docs/TERMINAL_LAUNCH.md).
Más problemas y sus soluciones: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). ¿Sigues atascado? Abre un
[issue](https://github.com/Niko1221/Strata/issues) y adjunta `strata-<model>.log` de la carpeta de Strata. ¿Has
encontrado un problema de seguridad? Infórmalo en privado: [SECURITY.md](SECURITY.md).
</details>
## ¿Cómo funciona?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplicación de Terminal de Strata con la actividad del modelo, la velocidad, la memoria y las peticiones recientes">
<br><sub>Observa tu modelo y tu hardware mientras se ejecuta.</sub>
</p>
Los modelos como este suelen funcionar en servidores con cientos de gigabytes de memoria gráfica. Tu tarjeta gráfica
tiene 12-24 GB. Strata hace que el modelo quepa **repartiendo el trabajo por todo tu PC**. Piensa en una cocina: lo
que usas todo el rato se queda en la encimera, y el resto espera en la despensa.
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Los 24,576 expertos del modelo: los más usados en la tarjeta gráfica, todos en la RAM y una tabla de consulta en el SSD"></p>
[Compilar desde el código fuente](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Guía de instalación completa](docs/INSTALL.md)
- **El modelo es un equipo de 24,576 pequeños especialistas ("expertos").** Cada palabra solo necesita 10 de ellos.
- **Tu tarjeta gráfica** guarda los pocos miles de expertos que más se usan. **Tu RAM** los tiene todos, y **tu
procesador** trabaja con el resto al mismo tiempo. **Tu SSD** guarda una gran tabla de consulta.
<a id="what-you-need"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Un pequeño ayudante adivina las siguientes palabras; el modelo grande las comprueba todas a la vez y se queda con las correctas"></p>
## Hardware
- **Adivinar y comprobar:** un pequeño ayudante adivina las siguientes palabras. El modelo grande las comprueba todas
a la vez. Recibes la misma respuesta, 1.6-1.8x antes.
- **Los textos largos se leen en trozos grandes** (hasta 8,192 tokens cada vez), a más de 1,000 tokens por segundo.
| Hardware compatible | Guía |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [Instalación en NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Instalación en AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimental** | [Instalación en RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Instalación en Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 o posterior** | [Instalación en Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Instalación en Strix Halo](docs/STRIX_HALO.md) |
La explicación larga: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte y sus números:
[los detalles](docs/DETAILS.md#how-it-works) y el [artículo](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Créditos y licencia
## Herramientas
El modelo es [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), del equipo de Qwen. Lo
comprimieron [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
y Unsloth. Strata usa partes de [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Todos los créditos:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata es de código abierto con la [licencia MIT](LICENSE).
Algunas partes y cada modelo tienen sus propias licencias ([cuáles](docs/HOW_IT_WORKS.md#license)).
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, y más, mediante las API compatibles con OpenAI y Anthropic.
## Apoya a Strata
Ejecuta **`strata connect`** para ver las instrucciones de tu herramienta. [Guía de conexión](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
Strata es libre y de código abierto. Si te resulta útil, puedes apoyar su desarrollo:
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
## Ayuda
[Todas las guías](docs) · [Solución de problemas](docs/TROUBLESHOOTING.md) · [Informar de un error](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [Cómo funciona](docs/HOW_IT_WORKS.md) · [Contribuir](CONTRIBUTING.md)
## Apoya Strata
**Ayuda a mantener la IA local abierta a todos.** Un café ayuda a financiar el desarrollo y las pruebas en más hardware.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: apoya Strata" height="48"></a>
[Licencia MIT](LICENSE) · [Créditos y licencias de los modelos](docs/HOW_IT_WORKS.md#credits) · [Privacidad](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Para herramientas de IA y lectores automáticos: <a href="AGENTS.md">referencia técnica</a> · <a href="docs/AI_SETUP.md">instrucciones de instalación</a>.</sub>
**Patrocinios, alianzas de hardware o colaboraciones:** [[email protected]](mailto:[email protected]).
+148 -198
View File
@@ -1,238 +1,188 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · **Français** · [Español](README.es.md) · [Português](README.pt-BR.md)
<h3 align="center">Une IA puissante sur votre propre ordinateur.</h3>
<p align="center">Discutez, programmez et travaillez avec des images, en local.</p>
<p align="center"><b>Faites tourner un modèle d'IA de 125 milliards de paramètres sur votre propre PC de jeu</b><br>
Carte graphique NVIDIA ou AMD (12 Go ou plus) · Windows ou Linux · gratuit et open source</p>
<p align="center">
<a href="#modèles">Modèles</a> · <a href="#installation"><b>Installation</b></a> ·
<a href="#matériel">Matériel</a> · <a href="#outils">Outils</a> ·
<a href="#aide">Aide</a> · <a href="#soutenir-strata">Soutenir</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Un jardin de pagode en voxels écrit par le modèle de Strata, qui tourne dans le navigateur"></a><br>
<sub>Un jardin de pagode en voxels, généré en une seule requête sur une RTX 5070 avec Strata (IQ3_S, contexte de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vidéo complète (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Télécharger Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licence MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Soutenir Strata sur Buy Me a Coffee"></a>
</p>
Strata fait tourner **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** sur un PC normal. C'est
un grand modèle d'IA, très capable, qui demande d'habitude un serveur. Il discute, écrit du code, lit des images et
fonctionne avec vos applications et vos agents de code. Rien ne quitte votre PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <b>Français</b> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
## Quelle est sa vitesse ?
**Gratuit, open source, et à vous de l'exécuter.** Utilisez Strata sur votre bureau ou avec vos outils de programmation préférés. Aucun compte nécessaire.
Nous l'avons mesurée sur deux PC de jeu ordinaires. Un token correspond à environ ¾ d'un mot.
## Modèles
- **Écrit les réponses :** la vitesse à laquelle la réponse s'affiche dans une courte discussion. 60 tokens par seconde, c'est plus rapide que votre lecture.
- **Lit votre prompt :** la vitesse à laquelle il absorbe ce que vous envoyez (ici un document, du code ou un historique de discussion de 32K tokens).
Nous recommandons **d'abord ISTA-DASLab**, puis **Unsloth** et d'autres fournisseurs, lorsqu'un modèle compatible tient en mémoire.
<table>
<tr><th>NVIDIA : RTX 5070 (12 Go), Ryzen 5 7600, 64 Go de RAM</th><th>AMD : RX 9070 XT (16 Go), Ryzen 9 3900X, 47 Go de RAM</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Taille | Écrit les réponses | Lit votre prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| Taille | Écrit les réponses | Lit votre prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA : Q2_0 avec le moteur 0.1.36, les autres lignes avec 0.1.26 (réponses de 4K, prompts de 32K). Les tableaux
complets sont dans [DETAILS.md](docs/DETAILS.md#speed-measured). Une carte avec plus de VRAM va plus vite : une
RTX 3090 (24 Go) devrait écrire environ 100-140 tokens par seconde. Longues discussions et autres cartes :
[vitesse de chaque modèle](docs/MODELS.md#how-fast-is-each-size), [résultats de la communauté](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata est gratuit. S'il tourne bien sur votre PC, un café aide à poursuivre le travail.</sub></p>
## Ce qu'il vous faut
| | |
| Fournisseur | Choix |
| --- | --- |
| **Carte graphique** | **NVIDIA** GeForce RTX série 20, 30, 40 ou 50, ou **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 ou série RX 6800 / 6900. Il faut **12 Go de VRAM ou plus**. |
| **RAM** | 32 Go ou plus. Votre RAM décide [quel modèle](#quel-modèle-choisir-) tient. Avec 64 Go, toutes les tailles tournent. |
| **Disque** | Environ 80 Go libres. Prenez un SSD si possible : le premier démarrage est bien plus rapide. |
| **Système** | Windows 10 / 11 ou Linux, avec un pilote graphique NVIDIA ou AMD à jour. |
| **ISTA-DASLab · recommandé** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder : IQ1\_M](docs/MODELS.md#coder) pour programmer. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Utilise moins de mémoire, mais plus lent. |
| **Autres options** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
L'installateur s'occupe de tout le reste. Deux ou trois cartes peuvent se partager le modèle ([multi-GPU](docs/MULTI_GPU.md)).
[Comparer les tailles, la mémoire nécessaire et la compatibilité](docs/MODELS.md).
Expérimental, écrit et testé par des membres de la communauté sur leurs propres machines :
</details>
- **Cartes graphiques plus anciennes** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT) : [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, compilé depuis les sources sous Linux : [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, compilé depuis les sources sous Linux : [Strix Halo](docs/STRIX_HALO.md).
- **Processeurs plus anciens sans AVX2** : ça marche, mais lentement. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
La liste complète : [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Ce sont les paquets actuellement documentés pour ce modèle.
[Guide du modèle et compatibilité](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
Dans l'**application de bureau** et dans le **Terminal**. Ce qui ne tient pas en mémoire est lu depuis votre SSD : il faut donc beaucoup d'espace disque libre.
| Jeu (IST-DAS) | Téléchargement | Espace disque libre nécessaire |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | environ 245 GB |
| 2.00 bpw | 257 GB | environ 265 GB |
| 2.25 bpw | 274 GB | environ 280 GB |
Fonctionne sur les NVIDIA RTX 30, 40 et 50 (Windows et Linux) et sur l'AMD Radeon AI PRO R9700 (Linux). L'installation télécharge les fichiers et le moteur DeepSeek, puis démarre le modèle.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Mesuré avec le jeu 2.00 bpw sur Strata 0.2.5. « Sortie » est la vitesse de la réponse, « Prompt » la vitesse à laquelle Strata lit votre texte.
| Matériel | Sortie | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB de RAM | 4 à 6 tokens/s | environ 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB de RAM | 9 à 11 tokens/s | environ 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB de RAM | 21 à 25 tokens/s | 320 à 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 à 77 tokens/s | 400 à 450 tokens/s (8K) |
[Mémoire, disque et détails](docs/DEEPSEEK41.md) · [Notes de version](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Vous ne savez pas ce qui tient ? [Choisir un modèle pour votre ordinateur](docs/MODELS.md#pick-by-memory).
## Installation
### Laissez votre IA l'installer
<a id="option-a-the-strata-app"></a>
Vous utilisez un assistant de code IA (Claude Code, Cursor, Codex, GitHub Copilot, ...) ? Collez-lui ceci :
### Application de bureau
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
**[Télécharger Strata pour Windows, Linux ou Mac →](https://github.com/Niko1221/Strata/releases/latest)**
Installez l'application, choisissez un modèle et commencez à discuter. Il y a un seul installateur par système. Au premier lancement, l'application télécharge le moteur de votre carte graphique. [Aide pour l'installateur](docs/INSTALL.md#the-app).
<sub>Fichiers: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Application de bureau Strata avec une discussion et un aperçu du code en direct côte à côte">
<br><sub>Discussion, fichiers et aperçu en direct dans une seule fenêtre.</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### Application Terminal
Vous préférez le terminal ? Installez Strata, puis lancez **`strata launch`**.
<details open>
<summary><b>Commandes d'installation</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Il vérifie votre carte graphique, votre RAM et votre disque, et choisit le modèle qui convient. Ensuite il l'installe,
le démarre et vous explique comment connecter vos applications. Les outils d'IA peuvent aussi installer, démarrer et
arrêter Strata grâce à son [serveur MCP](docs/MCP_SERVER.md).
**Windows PowerShell**
### Ou faites-le vous-même
[Téléchargez Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) et décompressez-le (ou faites un `git clone`).
**Windows :** double-cliquez sur **`START-HERE.bat`**. **Linux :** lancez **`./setup.sh`** dans le dossier Strata.
**Mac (Apple Silicon, M1 à M5, macOS 15 ou plus récent) :** installez une fois les Command Line Tools d'Apple (`xcode-select --install` dans le Terminal ; Xcode lui-même n'est pas nécessaire). Ouvrez ensuite le Terminal dans le dossier Strata et lancez **`./setup.sh`**. Il compile le moteur Mac sur votre Mac (environ 7 minutes), pose les mêmes questions, télécharge le modèle et le démarre. La fois suivante, `./setup.sh` démarre tout de suite. Quelle taille convient à votre Mac et sa vitesse : [docs/MAC.md](docs/MAC.md).
Les étapes sont les mêmes pour NVIDIA et AMD. L'installateur détecte votre carte et installe le bon moteur pour elle.
Il vous pose quelques questions :
- quel modèle et quelle taille,
- combien de contexte (la quantité de texte que le modèle garde en tête),
- s'il doit lire les images.
Appuyez sur Entrée à chaque fois pour garder la réponse recommandée. Ensuite il télécharge le modèle (environ 70 Go)
et le démarre. Si le téléchargement s'arrête, relancez-le : il reprend là où il s'était arrêté. Votre navigateur
ouvre l'application Strata à l'adresse `http://127.0.0.1:8080`.
> **Pendant le démarrage du modèle, votre PC peut ralentir ou ne plus répondre pendant 1-3 minutes** (plus longtemps la première fois).
> Strata charge 35-55 Go dans votre RAM et en réserve une partie pour la carte graphique. C'est normal. Attendez, et
> ne fermez pas la fenêtre. La fenêtre montre ce que fait Strata.
**La fois suivante**, relancez `START-HERE.bat` (ou `./setup.sh`). Il démarre tout de suite et ne télécharge rien
deux fois. Fermez sa fenêtre pour arrêter le modèle. `UPDATE.bat` (`./update.sh`) met Strata à jour sans le démarrer.
Mises à jour, Docker, plusieurs cartes, emplacement des fichiers et toutes les options : [docs/INSTALL.md](docs/INSTALL.md).
## Quel modèle choisir ?
L'installateur en recommande un selon votre RAM. Le même modèle existe en plusieurs tailles, plus ou moins
compressées. Les petites tailles sont plus rapides. Les grandes sont un peu plus intelligentes.
| Votre RAM | Prenez | Pourquoi |
| --- | --- | --- |
| **32 Go** | **Coder** | il tient dans 32 Go et il est fait pour le code (avec une carte de 24 Go, Q2_0 et IQ2_XS tournent aussi) |
| **48 Go** | **IQ2_XS** (ou Q2_0, le plus rapide) | les plus grandes tailles ne tiennent pas |
| **64 Go** | **IQ2_XS** (recommandé), ou IQ3_XXS / IQ3_S | toutes les tailles tiennent ; IQ3_S est le meilleur et le plus lent |
| **96 Go ou plus** | **IQ3_S**, ou l'UD-IQ4_XS d'Unsloth (~4 bits) | de la place pour les plus grandes tailles, avec tout le reste ouvert |
- **[Coder](docs/MODELS.md#coder) :** une version pour le code, avec la moitié des experts en moins. Elle atteint 91 %
du score SWE-bench Verified du modèle complet (mesuré par ses auteurs) et tient dans 32 Go de RAM. Elle est moins
bonne en dehors du code, y compris pour le chinois et les autres textes CJK (#438). Pour ces cas, prenez Q2_0,
IQ2_XS ou IQ3_S, qui gardent tous les experts.
- **[Swift 1.5](docs/MODELS.md#swift-15) :** un fine-tune qui réfléchit bien moins longtemps avant de répondre. Vous
avez la réponse plus tôt, avec à peu près la même qualité.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs)** : la version ~4 bits d'Unsloth, entre IQ3_S et
UD-Q4_K_XL en qualité. Un téléchargement de 94 Go. Avec moins de ~80 Go de RAM, Strata en lit une partie depuis
le SSD pendant qu'il répond, il y est donc plus lent (un SSD NVMe aide).
- **[Unsloth UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL et Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0) :** de 4 à 8 bits, plus proches du modèle
complet (93-95 % le même token suivant que Q8_0). Rapides quand tous les experts tiennent dans la RAM (~88 Go pour
UD-Q4_K_XL jusqu'à ~136 Go pour Q8_0) ; avec moins de RAM, Strata en lit une partie depuis le SSD pendant qu'il
répond, ce qui est bien plus lent (7-8.5 tokens/s pour UD-Q4_K_XL sur un PC avec 64 Go).
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs) :** vous l'installez à la main.
Il n'est pas dans le menu de l'installateur.
Tailles, téléchargements et ce qui tient où : [docs/MODELS.md](docs/MODELS.md). Pour ajouter un autre modèle plus
tard, lancez `SETUP.bat` (Linux : `./setup.sh --setup`).
## Utilisation
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="L'onglet Monitor de l'application Strata à côté d'un agent de code"><br>
<sub>Le <b>Monitor</b> de l'application Strata (à gauche) pendant qu'un agent de code écrit le jardin de pagode de la vidéo (à droite)</sub></p>
- **Dans le navigateur :** ouvrez `http://127.0.0.1:8080`. Vous y trouvez **Chat**, un **Monitor** en direct du
modèle et de votre GPU/CPU/RAM, et **About** avec les réglages et les adresses.
- **Vos applications et agents de code :** ajoutez un fournisseur « OpenAI-compatible » avec l'URL de base
**`http://127.0.0.1:8080/v1`**. N'importe quelle clé API et n'importe quel nom de modèle fonctionnent.
- Applications qui utilisent l'API d'Anthropic : `http://127.0.0.1:8080/v1/messages` (Claude Code :
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI et les autres applications qui utilisent l'API OpenAI Responses : `/v1/responses`
([configuration](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Réflexion :** choisissez **off, low, medium ou high** dans le menu du chat ou dans le « reasoning effort » de
votre application. Off est le plus rapide. High est le meilleur pour les questions difficiles.
- **Images :** répondez oui à « Images? » pendant l'installation. Ensuite cliquez sur **Picture** dans le chat, ou
joignez des images dans votre application. Les cartes AMD lisent les images sous Linux via le processeur ; sous
Windows, pas encore.
- **Depuis votre téléphone ou un autre PC :** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Définissez
toujours une clé.
- **Une requête à la fois :** par défaut, Strata répond à une requête et les autres attendent. Pour répondre à
plusieurs en même temps, mettez `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). Sur une carte de 12 Go, chaque
réponse devient alors plus lente.
- **Longs prompts :** Strata lit en entier le premier message d'une discussion, environ 1 minute pour 30,000 tokens.
Les messages suivants démarrent en quelques secondes.
Plus d'infos : [où sont stockées vos discussions](docs/INSTALL.md#where-things-are-stored), [l'API](docs/DETAILS.md#using-it).
### Optionnel : plus de vitesse sur une seule carte quand tout le modèle tient en RAM
Deux réglages optionnels (de RubenTorney, #1904 et #1905). Ils aident si vous utilisez une seule carte graphique et que votre RAM contient tous les experts (le mode arena de Strata, par exemple Q2_0 ou IQ3_XXS sur un PC de 64 Go). Ouvrez votre configuration `strata-<model>.json` dans le dossier de données de Strata ([où](docs/INSTALL.md#where-things-are-stored)) et ajoutez :
- **Génération plus rapide :** `"--adapt-async", "1"` dans la liste `"args"`. Le cache d'experts est alors mis à jour dans un thread auxiliaire au lieu de mettre la réponse en pause. Mesuré sur une RTX 5070 (Windows, Q2_0) : +4-5 % de génération, +7-10 % sur le code et les longs documents. Les réponses peuvent légèrement différer de celles par défaut (elles restent identiques d'une exécution à l'autre).
- **Lecture du prompt plus rapide sous Windows :** `"STRATA_PIN_BY_NEED": "1"` dans le bloc `"env"`. Windows ne peut verrouiller qu'environ la moitié de la RAM pour la carte ; ce réglage verrouille d'abord les experts dont la carte a vraiment besoin. Mesuré sur la RTX 5070 : +4,4 % en lecture du prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Redémarrez Strata (`START-HERE.bat` ou `./setup.sh`) après la modification. Pour annuler, retirez les deux entrées.
Sous Windows, ouvrez ensuite un nouveau terminal. Puis :
## Un problème ?
```sh
strata launch
```
- **Mon PC s'est figé au premier démarrage de Strata.** C'est normal pendant le chargement du modèle. Attendez, et ne
fermez pas la fenêtre. Toujours figé après 10 minutes ? Redémarrez le PC, fermez les autres programmes et
réessayez, ou choisissez une taille plus petite.
- **Il s'est arrêté pendant le téléchargement ou l'installation.** Relancez `START-HERE.bat` (ou `./setup.sh`). Il
reprend là où il s'était arrêté.
- **C'est très lent et le voyant du disque clignote sans arrêt, ou il affiche « the engine stopped unexpectedly ».**
Votre PC n'a pas assez de RAM libre. Fermez les autres programmes (les navigateurs en utilisent beaucoup), ou
choisissez une taille plus petite (Q2_0 ou IQ2_XS).
- **Il dit que le port 8080 est déjà utilisé.** Strata tourne déjà. Cherchez sa fenêtre.
Suivez les invites pour télécharger et démarrer un modèle. [Guide du Terminal](docs/TERMINAL_LAUNCH.md).
Autres problèmes et leurs solutions : [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Toujours bloqué ? Ouvrez une
[issue](https://github.com/Niko1221/Strata/issues) et joignez `strata-<model>.log` depuis le dossier Strata. Vous
avez trouvé un problème de sécurité ? Signalez-le en privé : [SECURITY.md](SECURITY.md).
</details>
## Comment ça marche ?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Application Terminal de Strata affichant l'activité du modèle, la vitesse, la mémoire et les requêtes récentes">
<br><sub>Suivez votre modèle et votre matériel pendant qu'il tourne.</sub>
</p>
Les modèles comme celui-ci tournent d'habitude sur des serveurs avec des centaines de gigaoctets de mémoire
graphique. Votre carte graphique a 12-24 Go. Strata fait tenir le modèle en **répartissant le travail sur tout votre
PC**. Pensez à une cuisine : ce que vous utilisez tout le temps reste sur le plan de travail, et le reste attend dans
le placard.
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Les 24,576 experts du modèle : les plus sollicités sur la carte graphique, tous en RAM, une table de correspondance sur le SSD"></p>
[Compiler depuis les sources](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Guide d'installation complet](docs/INSTALL.md)
- **Le modèle est une équipe de 24,576 petits spécialistes (les « experts »).** Chaque mot n'en demande que 10.
- **Votre carte graphique** garde les quelques milliers d'experts les plus utilisés. **Votre RAM** les contient tous,
et **votre processeur** travaille sur les autres en même temps. **Votre SSD** contient une grande table de
correspondance.
<a id="what-you-need"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Un petit assistant devine les mots suivants ; le grand modèle les vérifie tous d'un coup et garde les bons"></p>
## Matériel
- **Deviner, puis vérifier :** un petit assistant devine les quelques mots suivants. Le grand modèle les vérifie
tous d'un coup. Vous obtenez la même réponse, 1.6-1.8x plus tôt.
- **Les longs textes sont lus par gros morceaux** (jusqu'à 8,192 tokens à la fois), à plus de 1,000 tokens par seconde.
| Matériel pris en charge | Guide |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [Installation NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Installation AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · expérimental** | [Installation RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Installation Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 ou plus récent** | [Installation sur Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Installation Strix Halo](docs/STRIX_HALO.md) |
L'explication plus longue : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Chaque partie et ses chiffres :
[les détails](docs/DETAILS.md#how-it-works) et l'[article](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Crédits et licence
## Outils
Le modèle est [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), de l'équipe Qwen. Il a été
compressé par [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
et Unsloth. Strata utilise des parties de [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Tous les
crédits : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata est open source sous [licence MIT](LICENSE).
Quelques parties et chaque modèle ont leur propre licence ([lesquelles](docs/HOW_IT_WORKS.md#license)).
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, et d'autres via les API compatibles OpenAI et Anthropic.
Lancez **`strata connect`** pour voir la configuration de votre outil. [Guide de connexion](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## Aide
[Tous les guides](docs) · [Dépannage](docs/TROUBLESHOOTING.md) · [Signaler un bogue](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [Comment ça marche](docs/HOW_IT_WORKS.md) · [Contribuer](CONTRIBUTING.md)
## Soutenir Strata
Strata est gratuit et open source. S'il vous est utile, vous pouvez soutenir son développement :
**Aidez à garder l'IA locale ouverte à tous.** Un café aide à financer le développement et les tests sur davantage de matériel.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee : soutenir Strata" height="48"></a>
[Licence MIT](LICENSE) · [Crédits et licences des modèles](docs/HOW_IT_WORKS.md#credits) · [Confidentialité](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Pour les outils d'IA et les lecteurs automatiques : <a href="AGENTS.md">référence technique</a> · <a href="docs/AI_SETUP.md">instructions d'installation</a>.</sub>
**Parrainages, partenariats matériel ou collaborations :** [[email protected]](mailto:[email protected]).
+148 -189
View File
@@ -1,229 +1,188 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · **日本語** · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<h3 align="center">あなた自身のパソコンで、高性能な AI を。</h3>
<p align="center">チャット、コーディング、画像の処理を、すべてローカルで。</p>
<p align="center"><b>1,250 億パラメータの AI モデルを、手元のゲーミング PC で動かす</b><br>
NVIDIA または AMD のグラフィックカード(12 GB 以上) · Windows または Linux · 無料・オープンソース</p>
<p align="center">
<a href="#モデル">モデル</a> · <a href="#インストール"><b>インストール</b></a> ·
<a href="#ハードウェア">ハードウェア</a> · <a href="#ツール">ツール</a> ·
<a href="#ヘルプ">ヘルプ</a> · <a href="#strata-を支援する">支援</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Strata で生成した、五重塔のある庭。ボクセルで表現され、ブラウザーで動作している"></a><br>
<sub>五重塔のある庭をボクセルで表現。RTX 5070 で動かした Strata(IQ3_S、コンテキスト長 128K)に、1 回のプロンプトで作成を依頼 ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">動画を見る(全編 49 秒)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Strata をダウンロード"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT ライセンス"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Buy Me a Coffee で Strata を支援する"></a>
</p>
Strata を使うと、**[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** を一般的な PC で動かせます。
通常はサーバーを必要とする大規模で高性能な AI モデルで、チャットやコードの生成、画像の読み取りができます。
アプリやコーディングエージェントとも連携でき、データは PC の外に送信されません。
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <b>日本語</b> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
## どのくらい速い?
**無料でオープンソース。自分の環境で動かせます。** デスクトップで、またはお好みのコーディングツールと一緒に Strata を使えます。アカウントは不要です。
一般的なゲーミング PC 2 台で速度を測定しました。英語の場合、1 トークンはおよそ ¾ 語に相当します。
## モデル
- **生成速度:** 短いチャットで回答を生成する速度です。毎秒 60 トークンなら、人が読むよりも速く文章が表示されます。
- **プロンプトの読み込み:** 入力した内容を読み込む速度です。ここでは、文書やコード、チャット履歴など 32K トークン分を読み込ませています。
まず **ISTA-DASLab** をおすすめします。次に **Unsloth** などの提供元です。メモリに収まる互換モデルがある場合に選べます。
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| モデルサイズ | 生成速度 | プロンプトの読み込み |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| モデルサイズ | 生成速度 | プロンプトの読み込み |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA の測定には、Q2_0 のみエンジン 0.1.36、それ以外は 0.1.26 を使用しています(回答 4K、プロンプト 32K)。測定結果の詳細は
[DETAILS.md](docs/DETAILS.md#speed-measured) をご覧ください。VRAM の容量が大きいカードほど高速で、RTX 3090(24 GB)なら
毎秒およそ 100-140 トークンの生成速度が見込めます。長いチャットや他のカードでの性能については、[モデルごとの速度](docs/MODELS.md#how-fast-is-each-size)や
[コミュニティの測定結果](docs/COMMUNITY_BENCHMARKS.md)を参照してください。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata は無料で使えます。お使いの PC で快適に動いたら、コーヒー 1 杯分のご支援が開発を続ける励みになります。</sub></p>
## 必要なもの
| | |
| 提供元 | 選択肢 |
| --- | --- |
| **グラフィックカード** | **NVIDIA** GeForce RTX 20、30、40、50 シリーズ、または **AMD** Radeon RX 7900 XT / XTX、RX 7800 XT / 7700 XT、RX 9060 XT、RX 9070 / 9070 XT、Radeon AI PRO R9700、RX 6800 / 6900 シリーズ。**VRAM 12 GB 以上**が必要です。 |
| **RAM** | 32 GB 以上。[利用できるモデル](#どのモデルを選べばいい)は RAM の容量によって異なります。64 GB あれば、どのサイズも利用できます。 |
| **ディスク** | 約 80 GB の空き容量。SSD を使うと初回の起動が大幅に速くなるため、おすすめです。 |
| **OS** | Windows 10 / 11 または Linux。NVIDIA または AMD の最新のグラフィックドライバーも必要です。 |
| **ISTA-DASLab · おすすめ** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S。[Coder: IQ1\_M](docs/MODELS.md#coder)(コーディング向け)。 |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)。メモリ使用量は少ないですが、動作は遅くなります。 |
| **その他の選択肢** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
その他の必要なソフトウェアは、インストーラーが用意します。2 枚または 3 枚のカードにモデルを分散させることもできます([マルチ GPU](docs/MULTI_GPU.md))。
[サイズ、必要なメモリ、互換性を比べる](docs/MODELS.md)。
以下のハードウェアにも試験的に対応しています。いずれもコミュニティのメンバーが手元のマシンで実装・テストしたものです。
</details>
- **旧世代のグラフィックカード**(Tesla P40 / V100、GTX 10、Radeon VII / MI50、RX 6700 XT、RX 5500 XT):[旧世代 GPU の対応状況](docs/OLDER_GPUS.md)。
- **Intel Arc**(Linux でソースからビルド):[Intel Arc の対応状況](docs/INTEL_ARC.md)。
- **AMD Ryzen AI Max(Strix Halo)**(Linux でソースからビルド):[Strix Halo の対応状況](docs/STRIX_HALO.md)。
- **AVX2 非対応の旧世代 CPU**:動作しますが、速度は遅くなります。[旧世代 CPU の対応状況](docs/INSTALL.md#older-cpus-experimental)を参照してください。
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
必要な環境の一覧は、[docs/INSTALL.md](docs/INSTALL.md#what-you-need)をご覧ください。
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0。現在、このモデルで文書化されているパックです。
[モデルガイドと互換性](docs/QWEN36.md)。
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
**デスクトップアプリ**と**ターミナル**で使えます。メモリに収まらない分は SSD から読み込むため、空きディスク容量がたくさん必要です。
| セット(IST-DAS) | ダウンロード | 必要な空き容量 |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | 約245 GB |
| 2.00 bpw | 257 GB | 約265 GB |
| 2.25 bpw | 274 GB | 約280 GB |
NVIDIA RTX 30・40・50 シリーズ(Windows と Linux)と、AMD Radeon AI PRO R9700(Linux)で動きます。セットアップがファイルと DeepSeek エンジンをダウンロードし、モデルを起動します。
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Strata 0.2.5 で 2.00 bpw のセットを使って測定しました。「出力」は回答の速さ、「プロンプト」は Strata が入力を読み込む速さです。
| ハードウェア | 出力 | プロンプト |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4〜6 トークン/秒 | 約 110 トークン/秒 (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9〜11 トークン/秒 | 約 155 トークン/秒 (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21〜25 トークン/秒 | 320〜360 トークン/秒 (8K) |
| 4x Radeon AI PRO R9700 | 74〜77 トークン/秒 | 400〜450 トークン/秒 (8K) |
[メモリ、ディスク、詳細](docs/DEEPSEEK41.md) · [リリースノート](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
どれが動くかわからない場合は、[お使いのパソコンに合うモデルを選ぶ](docs/MODELS.md#pick-by-memory)。
## インストール
### AI にセットアップしてもらう
<a id="option-a-the-strata-app"></a>
Claude Code、Cursor、Codex、GitHub Copilot などの AI コーディングアシスタントを使っている場合は、次の文を貼り付けてください。
### デスクトップアプリ
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
**[Windows、Linux、Mac 用の Strata をダウンロード →](https://github.com/Niko1221/Strata/releases/latest)**
アプリをインストールし、モデルを選んで、チャットを始めましょう。インストーラーはシステムごとに1つです。初回起動時に、アプリがお使いのグラフィックスカード用のエンジンをダウンロードします。[インストーラーのヘルプ](docs/INSTALL.md#the-app)。
<sub>ファイル: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="チャットとライブのコードプレビューを並べて表示した Strata デスクトップアプリ">
<br><sub>チャット、ファイル、ライブプレビューを1つのウィンドウで。</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### ターミナルアプリ
ターミナルのほうが好みですか? Strata をインストールして、**`strata launch`** を実行します。
<details open>
<summary><b>インストールコマンド</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
AI がグラフィックカード、RAM、ディスクを確認して、PC に合ったモデルを選びます。インストールと起動を行い、
アプリとの接続方法も案内します。また、Strata の [MCP サーバー](docs/MCP_SERVER.md) を使えば、
AI ツールから Strata のインストール・起動・停止を行えます。
**Windows PowerShell**
### 自分でやる
[Strata をダウンロード](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)して展開します(`git clone` でも構いません)。
**Windows:** **`START-HERE.bat`** をダブルクリックします。**Linux:** Strata フォルダーで **`./setup.sh`** を実行します。
**Mac(Apple シリコン、M1〜M5、macOS 15 以降):** まず一度だけ Apple の Command Line Tools をインストールします(ターミナルで `xcode-select --install`。Xcode 本体は不要です)。次に Strata フォルダーでターミナルを開き、**`./setup.sh`** を実行します。Mac 用エンジンをお使いの Mac 上でビルドし(約 7 分)、同じ質問をしてからモデルをダウンロードして起動します。次回からは `./setup.sh` ですぐに起動します。どのサイズが Mac に合うかと速度:[docs/MAC.md](docs/MAC.md)。
NVIDIA でも AMD でも手順は同じです。インストーラーがカードを検出し、対応するエンジンを用意します。
続いて、次の項目を設定します。
- 使用するモデルとサイズ
- コンテキストの長さ(モデルが覚えておけるテキストの量)
- 画像を読み取る機能を使うかどうか
各項目で Enter を押すと、推奨設定が選ばれます。設定が終わるとモデル(約 70 GB)をダウンロードし、起動します。
ダウンロードが途中で止まった場合は、もう一度実行すると続きから再開できます。起動が完了すると、ブラウザーで
`http://127.0.0.1:8080` が開き、Strata の画面が表示されます。
> **モデルの起動中は、1-3 分ほど PC の動作が遅くなったり、応答しなくなったりすることがあります**(初回は特に時間がかかります)。
> Strata が 35-55 GB のデータを RAM に読み込み、その一部をグラフィックカード用に固定(ピン留め)しているためです。
> 正常な動作なので、ウィンドウに表示される処理状況を確認しながら、閉じずにお待ちください。
**次回からは** `START-HERE.bat`(または `./setup.sh`)を実行するだけですぐに起動できます。2 回目以降は、ダウンロード済みのファイルを再度取得する必要はありません。
モデルを停止するにはウィンドウを閉じてください。`UPDATE.bat`(`./update.sh`)を使うと、Strata を起動せずに更新できます。
更新方法や Docker、複数のカードの使い方、ファイルの保存先、各種オプションについては、[docs/INSTALL.md](docs/INSTALL.md)をご覧ください。
## どのモデルを選べばいい
インストーラーが RAM の容量に合ったモデルをおすすめします。同じモデルでも、圧縮率によって複数のサイズが用意されています。
サイズが小さいほど高速に動作し、大きいほど回答の品質が少し高くなります。
| RAM | おすすめのモデル | 選ぶ理由 |
| --- | --- | --- |
| **32 GB** | **Coder** | 32 GB で動作するコーディング向けモデル(24 GB のカードがあれば Q2_0 と IQ2_XS も動作) |
| **48 GB** | **IQ2_XS**(速度優先なら Q2_0) | これより大きいサイズはメモリに収まらない |
| **64 GB** | **IQ2_XS**(おすすめ)、または IQ3_XXS / IQ3_S | どのサイズも利用可能。IQ3_S は回答の品質が最も高い一方、速度は最も遅い |
| **96 GB 以上** | **IQ3_S**、または Unsloth の UD-IQ4_XS(約 4-bit) | 他のアプリを開いたままでも、最大サイズのモデルを動かせるだけの余裕がある |
- **[Coder](docs/MODELS.md#coder):** エキスパートを半分に減らしたコーディング向けモデルです。SWE-bench Verified ではフルモデルのスコアの
91% を達成しており(開発者による測定)、32 GB の RAM で動作します。ただし、コード以外の用途では性能が劣り、中国語などの CJK 言語の処理も苦手です(#438)。
こうした用途には、すべてのエキスパートを残した Q2_0、IQ2_XS、IQ3_S を選んでください。
- **[Swift 1.5](docs/MODELS.md#swift-15):** 回答前の思考時間が大幅に短くなるようファインチューニングしたモデルです。
回答の品質をほぼ保ったまま、より短い時間で答えが返ってきます。
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth による約 4-bit のモデルです。回答の品質は IQ3_S と UD-Q4_K_XL の中間です。
ダウンロード容量は 94 GB です。RAM が約 80 GB 未満の場合、回答の生成中にモデルの一部を SSD から読み込むため、速度が落ちます(NVMe SSD を使うと改善します)。
- **[Unsloth UD-Q4_K_XL / UD-Q5_K_XL / UD-Q6_K_XL / Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):** 4〜8 ビットで、フルモデルにより近いモデルです
(次のトークンが Q8_0 と一致する割合は 93〜95%)。すべてのエキスパートが RAM に収まる場合(UD-Q4_K_XL で約 88 GB、
Q8_0 で約 136 GB)は高速です。RAM が足りない場合は回答の生成中に一部を SSD から読み込むため、大幅に遅くなります
(RAM が 64 GB の PC で UD-Q4_K_XL は毎秒 7-8.5 トークン程度)。
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** 手動でのセットアップが必要です。
インストーラーのメニューからは選べません。
各サイズのダウンロード情報や必要なメモリ容量は、[docs/MODELS.md](docs/MODELS.md)をご覧ください。後から別のモデルを追加する場合は、
`SETUP.bat`(Linux:`./setup.sh --setup`)を実行してください。
## 使い方
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="Strata の Monitor タブ(左)と、コードを生成するコーディングエージェント(右)"><br>
<sub>動画に登場する五重塔の庭を作成中の画面。左が Strata の <b>Monitor</b>、右がコードを生成しているコーディングエージェント</sub></p>
- **ブラウザーで使う:** `http://127.0.0.1:8080` を開きます。チャット用の **Chat**、モデルや GPU/CPU/RAM の状態をリアルタイムで確認できる **Monitor**、
設定や接続先のアドレスを確認できる **About** の各タブがあります。
- **アプリやコーディングエージェントと連携する:** 「OpenAI-compatible」プロバイダーを追加し、ベース URL に
**`http://127.0.0.1:8080/v1`** を指定します。API キーとモデル名には任意の値を指定できます。
- Anthropic API を使うアプリ:`http://127.0.0.1:8080/v1/messages`(Claude Code では
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080` を設定)。
- Codex CLI など、OpenAI Responses API を使うアプリ:`/v1/responses`
([設定方法](docs/DETAILS.md#the-responses-api-and-codex-cli))。
- **思考(Thinking)の設定:** チャットのメニュー、または連携先アプリの「reasoning effort」で **off、low、medium、high** から選びます。
off が最も速く、難しい質問には high が向いています。
- **画像を読み取る:** セットアップ時に「Images?」で yes を選びます。その後、チャットの **Picture** ボタンをクリックするか、連携先のアプリで画像を添付してください。
AMD のカードを使う場合、Linux では CPU で画像を読み取ります。Windows ではまだ画像の読み取りに対応していません。
- **スマートフォンや別の PC から使う:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>` で起動します。API キーは必ず設定してください。
- **リクエストは 1 つずつ:** 標準設定では 1 つずつ処理し、それ以外のリクエストは順番を待ちます。同時に処理するには、
`"parallel": 2` を設定してください([BATCHING.md](docs/BATCHING.md))。12 GB のカードでは、リクエストごとの回答速度が落ちます。
- **長いプロンプトを送る:** チャットの最初のメッセージは全文を読み込むため、30,000 トークンあたり約 1 分かかります。
2 通目以降は数秒で応答が始まります。
[チャットの保存場所](docs/INSTALL.md#where-things-are-stored)や[API の詳細](docs/DETAILS.md#using-it)も参照してください。
### オプション:モデル全体が RAM に収まる場合、GPU 1 枚でさらに高速化
2 つのオプトイン設定です(RubenTorney 氏による #1904 と #1905)。GPU を 1 枚使い、RAM にすべてのエキスパートが収まる場合(Strata の arena モード。例:64 GB の PC で Q2_0 や IQ3_XXS)に効果があります。Strata のデータフォルダーにある設定ファイル `strata-<model>.json`([場所](docs/INSTALL.md#where-things-are-stored))を開き、次を追加します:
- **生成の高速化:** `"args"` リストに `"--adapt-async", "1"`。エキスパートキャッシュの更新が回答を止めずに補助スレッドで行われます。RTX 5070(Windows、Q2_0)での実測:生成 +4〜5%、コードや長文では +7〜10%。回答はデフォルトとわずかに異なる場合があります(実行ごとには同じです)。
- **Windows でのプロンプト読み込みの高速化:** `"env"` ブロックに `"STRATA_PIN_BY_NEED": "1"`。Windows はGPU 用に RAM の約半分しか固定できないため、この設定で GPU が本当に必要とするエキスパートから先に固定します。RTX 5070 での実測:プロンプト読み込み +4.4%。
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
変更後は Strata を再起動してください(`START-HERE.bat` または `./setup.sh`)。元に戻すには、2 つの項目を削除します。
Windows では、インストール後に新しいターミナルを開きます。そのあと:
## うまくいかないとき
```sh
strata launch
```
- **初回起動時に PC が応答しなくなる。** モデルの読み込み中は、一時的に応答しなくなることがあります。ウィンドウを閉じずにお待ちください。
10 分たっても応答しない場合は、PC を再起動して他のアプリを閉じてからやり直すか、モデルのサイズを小さくしてください。
- **ダウンロードやインストールが途中で止まる。** `START-HERE.bat`(または `./setup.sh`)をもう一度実行してください。
中断したところから再開できます。
- **動作が極端に遅く、ディスクのアクセスランプが点滅し続ける。または「the engine stopped unexpectedly」と表示される。** RAM の空き容量が不足しています。
ブラウザーなど、RAM を多く使う他のアプリを閉じるか、小さいモデル(Q2_0 または IQ2_XS)を選んでください。
- **ポート 8080 が使用中と表示される。** Strata がすでに起動しています。起動済みのウィンドウを探してください。
画面の案内に従って、モデルをダウンロードして起動します。[ターミナルガイド](docs/TERMINAL_LAUNCH.md)。
その他の問題と対処方法は、[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)をご覧ください。解決しない場合は
[Issue](https://github.com/Niko1221/Strata/issues) を作成し、Strata フォルダー内の `strata-<model>.log` を添付してください。
セキュリティ上の問題は、[SECURITY.md](SECURITY.md)の手順に従って非公開で報告してください。
</details>
## しくみ
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="モデルの動作、速度、メモリ、最近のリクエストを表示する Strata ターミナルアプリ">
<br><sub>動作中のモデルとハードウェアの状態を確認できます。</sub>
</p>
この規模のモデルは通常、数百ギガバイトの GPU メモリを搭載したサーバーで動かします。一方、手元のグラフィックカードのメモリは
12-24 GB 程度です。Strata は **PC 全体で処理を分担する** ことで、こうした環境でもモデルを動かせるようにしています。
台所で、よく使うものは調理台に置き、それ以外は食品庫にしまっておくような仕組みです。
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="24,576 個のエキスパートのうち、よく使うものをグラフィックカードに配置。全エキスパートは RAM に、ルックアップテーブルは SSD に保持"></p>
[ソースからビルド](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [インストールガイド(全文)](docs/INSTALL.md)
- **モデルは「エキスパート」と呼ばれる 24,576 個の小さな処理単位から構成されています。** 1 語の生成に必要なのは、このうち 10 個だけです。
- **グラフィックカード** に、使用頻度の高い数千個のエキスパートを置きます。**RAM** にはすべてのエキスパートを保持し、
**CPU** が残りの処理を並行して担当します。**SSD** には大きなルックアップテーブルを保存します。
<a id="what-you-need"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="小さな補助モデルが次の数語を予測し、大きなモデルがまとめて検証。正しい予測だけを採用する"></p>
## ハードウェア
- **予測してから検証:** 小さな補助モデルが次の数語を予測し、大きなモデルがまとめて検証します。
回答を変えずに、生成速度を 1.6-1.8 倍に高められます。
- **長いテキストはまとめて読み込みます**(一度に最大 8,192 トークン)。読み込み速度は毎秒 1,000 トークンを超えます。
| 対応ハードウェア | ガイド |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 シリーズ** | [NVIDIA のセットアップ](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900、RX 9060 / 9070、AI PRO R9700** | [AMD のセットアップ](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · 試験的** | [RX 6000 のセットアップ](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel のセットアップ](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 以降** | [Mac のセットアップ](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo のセットアップ](docs/STRIX_HALO.md) |
仕組みについては、[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md)で詳しく説明しています。各処理の詳細や測定値は、
[技術資料](docs/DETAILS.md#how-it-works)と[論文](docs/paper/Strata-Paper.pdf)をご覧ください。
<a id="using-it"></a>
## クレジットとライセンス
## ツール
使用しているモデルは、Qwen チームが開発した [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) です。
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)、Unsloth が提供する圧縮版を利用しています。
Strata は [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) の一部も使用しています。クレジットの一覧は
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits)をご覧ください。Strata は [MIT License](LICENSE) で公開されていますが、一部の構成要素と
各モデルには、それぞれ別のライセンスが適用されます([ライセンスの詳細](docs/HOW_IT_WORKS.md#license))。
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**。OpenAI 互換と Anthropic 互換の API を通じて、ほかのツールも使えます。
**`strata connect`** を実行すると、お使いのツールの設定方法が表示されます。[接続ガイド](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect)。
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## ヘルプ
[すべてのガイド](docs) · [トラブルシューティング](docs/TROUBLESHOOTING.md) · [不具合を報告](https://github.com/Niko1221/Strata/issues/new/choose) ·
[ベンチマーク](docs/COMMUNITY_BENCHMARKS.md) · [仕組み](docs/HOW_IT_WORKS.md) · [貢献する](CONTRIBUTING.md)
## Strata を支援する
Strata は無料で使えるオープンソースソフトウェアです。役に立ったら、開発へのご支援をお願いします。
**ローカル AI をみんなに開かれたものに。** コーヒー1杯が、開発と、より多くのハードウェアでのテストの支えになります。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: Strata を支援する" height="48"></a>
[MIT ライセンス](LICENSE) · [クレジットとモデルのライセンス](docs/HOW_IT_WORKS.md#credits) · [プライバシー](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>AI ツールと自動リーダー向け: <a href="AGENTS.md">技術リファレンス</a> · <a href="docs/AI_SETUP.md">セットアップ手順</a></sub>
**スポンサー、ハードウェア提携、共同開発のご相談:** [[email protected]](mailto:[email protected]).
+153 -203
View File
@@ -1,243 +1,193 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
**English** · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<h3 align="center">Run powerful AI on your own computer.</h3>
<p align="center">Chat, code and work with images, locally.</p>
<p align="center"><b>Run a 125-billion-parameter AI model on your own gaming PC</b><br>
NVIDIA or AMD graphics card (12 GB or more) · Windows or Linux · free and open source</p>
<p align="center">
<a href="#models">Models</a> · <a href="#install"><b>Install</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#tools">Tools</a> ·
<a href="#help">Help</a> · <a href="#support-strata">Support</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">full video (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Download Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Support Strata on Buy Me a Coffee"></a>
</p>
Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** on a normal PC. This is a
large, smart AI model that usually needs a server. It chats, writes code, reads pictures and works with your apps
and coding agents. Nothing leaves your PC.
<p align="center"><sub><b>English</b> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
## How fast is it?
**Free, open source, and yours to run.** Use Strata on your desktop or with your favorite coding tools. No account needed.
We measured it on two ordinary gaming PCs. A token is about ¾ of a word.
## Models
- **Writes answers:** how fast the reply appears in a short chat. 60 tokens per second is faster than you can read.
- **Reads your prompt:** how fast it takes in what you send (here a 32K-token document, code or chat history).
We recommend **ISTA-DASLab first**, then **Unsloth** and other providers, where a compatible model fits.
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Size | Writes answers | Reads your prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| Size | Writes answers | Reads your prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 with engine 0.1.36, the other rows with 0.1.26 (4K answers, 32K prompts). The full tables are in
[DETAILS.md](docs/DETAILS.md#speed-measured). A card with more VRAM is faster: an RTX 3090 (24 GB) should write
about 100-140 tokens per second. Long chats and other cards: [speed of each model](docs/MODELS.md#how-fast-is-each-size),
[community results](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata is free. If it runs well on your PC, a coffee keeps the work on it going.</sub></p>
## What you need
| | |
| Provider | Choices |
| --- | --- |
| **Graphics card** | **NVIDIA** GeForce RTX 20, 30, 40 or 50 series, or **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series. It needs **12 GB of VRAM or more**. |
| **RAM** | 32 GB or more. Your RAM decides [which model](#which-model-should-i-pick) fits. 64 GB runs every size. |
| **Disk** | About 80 GB free. Use an SSD if you can: the first start is much faster. |
| **System** | Windows 10 / 11 or Linux, and a current graphics driver from NVIDIA or AMD. |
| **ISTA-DASLab · recommended** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) for coding. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Lower memory use but slower. |
| **Other options** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
The installer sets up everything else. Two or three cards can share the model ([multi-GPU](docs/MULTI_GPU.md)).
[Compare sizes, memory needs and compatibility](docs/MODELS.md).
Experimental, written and tested by community members on their own machines:
</details>
- **Older graphics cards** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, built from source on Linux: [Intel Arc](docs/INTEL_ARC.md).
- **Apple Silicon Macs (M1 to M5)**, built from source by `./setup.sh` (the Metal engine): [Mac](docs/MAC.md).
- **AMD Ryzen AI Max (Strix Halo)**, built from source on Linux: [Strix Halo](docs/STRIX_HALO.md).
- **Older processors without AVX2**: they work, but slowly. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
The full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. These are the currently documented packs for this model.
[Model guide and compatibility](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
In the **desktop app** and the **Terminal**. What does not fit in memory is read from your SSD, so it needs a lot of free
disk space.
| Set (IST-DAS) | Download | Free disk needed |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | about 245 GB |
| 2.00 bpw | 257 GB | about 265 GB |
| 2.25 bpw | 274 GB | about 280 GB |
Runs on NVIDIA RTX 30, 40 and 50 series (Windows and Linux), AMD Radeon AI PRO R9700 (Linux), and AMD Ryzen AI Max — Strix Halo
(Windows). Setup downloads the files and the DeepSeek engine, then starts the model.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Measured with the 2.00 bpw set on Strata 0.2.5 (Strix Halo: 0.2.7). Output is the speed of the answer, prompt is how fast Strata reads your input.
| Hardware | Output | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4 to 6 tokens/s | about 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9 to 11 tokens/s | about 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21 to 25 tokens/s | 320 to 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 to 77 tokens/s | 400 to 450 tokens/s (8K) |
| Ryzen AI Max+ 395 (Strix Halo), 128 GB unified | 19 to 21 tokens/s (code), 11 (chat) | 87 tokens/s (4K), 125 tokens/s (8K) |
[Memory, disk and details](docs/DEEPSEEK41.md) · [Release notes](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Not sure what fits? [Choose a model for your computer](docs/MODELS.md#pick-by-memory).
## Install
### Let your AI set it up
<a id="option-a-the-strata-app"></a>
Do you use an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Paste this into it:
### Desktop app
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
**[Download Strata for Windows, Linux or Mac →](https://github.com/Niko1221/Strata/releases/latest)**
Install the app, choose a model, and start chatting. There is one installer for each system. On first start the app
downloads the engine for your graphics card. [Help with the installer](docs/INSTALL.md#the-app).
<sub>Files: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Strata desktop app with a chat and live code preview side by side">
<br><sub>Chat, files and a live preview in one window.</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### Terminal app
Prefer the terminal? Install Strata, then run **`strata launch`**.
<details open>
<summary><b>Install commands</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
It checks your graphics card, RAM and disk and picks the model that fits. Then it installs and starts it and tells
you how to connect your apps. AI tools can also install, start and stop Strata through its
[MCP server](docs/MCP_SERVER.md).
**Windows PowerShell**
### Or do it yourself
[Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or `git clone` it).
**Windows:** double-click **`START-HERE.bat`**. **Linux:** run **`./setup.sh`** in the Strata folder.
**Mac (Apple Silicon, M1 to M5, macOS 15 or newer):** install Apple's Command Line Tools once
(`xcode-select --install` in Terminal; Xcode itself is not needed). Then open Terminal in the Strata folder and run
**`./setup.sh`**. It builds the Mac engine on your Mac (about 7 minutes), asks the same questions, downloads the model
and starts it. Next time, `./setup.sh` starts it right away. Which size fits your Mac and how fast it is:
[docs/MAC.md](docs/MAC.md).
The steps are the same for NVIDIA and AMD. The installer finds your card and sets up the right engine for it. It
asks you a few questions:
- which model and which size,
- how much context (how much text the model keeps in mind),
- whether it should read pictures.
Press Enter each time for the recommended answer. Then it downloads the model (about 70 GB) and starts it. If the
download stops, run it again: it continues where it left off. Your browser opens the Strata app at
`http://127.0.0.1:8080`.
> **While the model starts, your PC can be slow or stop responding for 1-3 minutes** (longest the first time).
> Strata loads 35-55 GB into your RAM and locks part of it for the graphics card. This is normal. Wait, and don't
> close the window. The window shows what Strata is doing.
**Next time**, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close
its window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker,
several cards, where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md).
**Is it fast on your PC?** Run `BENCH.bat` (`./bench.sh`): about 5-10 minutes, it measures chat, 4K/16K prompts and an agent-style session with your own settings and writes a report to share ([how](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
## Which model should I pick?
The installer recommends one for your RAM. The same model comes in several sizes, compressed more or less. Smaller
sizes are faster. Larger sizes are a bit smarter.
| Your RAM | Take | Why |
| --- | --- | --- |
| **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) |
| **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit |
| **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits; IQ3_S is the best and the slowest |
| **96 GB or more** | **IQ3_S**, or Unsloth's UD-IQ4_XS (~4-bit) | room for the largest sizes with everything else open |
- **[Coder](docs/MODELS.md#coder):** a coding version with half of the experts removed. It reaches 91% of the full
model's SWE-bench Verified score (measured by its authors) and fits 32 GB of RAM. It is weaker outside code,
including Chinese and other CJK text (#438). For those, take Q2_0, IQ2_XS or IQ3_S, which keep every expert.
- **[Swift 1.5](docs/MODELS.md#swift-15):** a fine-tune that thinks for a much shorter time before it answers. You
get the answer sooner, at about the same quality.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth's ~4-bit version, between IQ3_S and
UD-Q4_K_XL in quality. A 94 GB download. With less than ~80 GB of RAM, Strata reads part of it from the SSD
while it answers, so it is slower there (an NVMe SSD helps).
- **[Unsloth UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL and Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):**
4- to 8-bit, closer to the full model (93-95% the same next token as Q8_0). Fast where every expert fits in RAM
(~88 GB for UD-Q4_K_XL up to ~136 GB for Q8_0); with less RAM Strata reads part of them from the SSD while it
answers, which is much slower (7-8.5 tokens/s for UD-Q4_K_XL on a 64 GB PC).
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** you set it up by hand. It is
not in the installer's menu.
Sizes, downloads and what fits where: [docs/MODELS.md](docs/MODELS.md). To add another model later, run
`SETUP.bat` (Linux: `./setup.sh --setup`).
## Using it
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="The Strata app's Monitor tab next to a coding agent"><br>
<sub>The Strata app's <b>Monitor</b> (left) while a coding agent writes the pagoda garden from the video (right)</sub></p>
- **In the browser:** open `http://127.0.0.1:8080`. It has **Chat**, a live **Monitor** of the model and your
GPU/CPU/RAM, and **About** with the settings and addresses.
- **Your apps and coding agents:** add an "OpenAI-compatible" provider with the base URL
**`http://127.0.0.1:8080/v1`**. Any API key and any model name work.
- Apps that use Anthropic's API: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI and other apps that use the OpenAI Responses API: `/v1/responses`
([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Thinking:** choose **off, low, medium or high** in the chat menu or in your app's "reasoning effort". Off is the
fastest. High is best for hard questions.
- **Pictures:** say yes to "Images?" in setup. Then click **Picture** in the chat, or attach pictures in your app.
AMD cards read pictures on Linux through the processor; on Windows they can't yet.
- **From your phone or another PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Always set a key.
- **One request at a time:** by default Strata answers one request, and the others wait. To answer several at once,
set `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). On a 12 GB card this makes each answer slower.
- **Long prompts:** Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up
messages start in seconds.
More: [where your chats are stored](docs/INSTALL.md#where-things-are-stored), [the API](docs/DETAILS.md#using-it).
### Optional: more speed on one card when the whole model fits in RAM
Two opt-in settings (by RubenTorney, #1904 and #1905). They help when you run one graphics card and your RAM holds
every expert (Strata's arena mode, for example Q2_0 or IQ3_XXS on a 64 GB PC). Open your config,
`strata-<model>.json` in the Strata data folder ([where](docs/INSTALL.md#where-things-are-stored)), and add:
- **Faster decode:** `"--adapt-async", "1"` to the `"args"` list. The expert cache then updates on a helper thread
instead of pausing the answer. Measured on an RTX 5070 (Windows, Q2_0): +4-5% decode, +7-10% on code and long
documents. Answers can differ slightly from the default (they stay the same from run to run).
- **Faster prompt reading on Windows:** `"STRATA_PIN_BY_NEED": "1"` in the `"env"` block. Windows can lock only about
half the RAM for the graphics card; this locks the experts the card actually needs first. Measured on the RTX 5070:
+4.4% prompt reading.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Restart Strata (`START-HERE.bat` or `./setup.sh`) after the change. To undo, remove the two entries.
On Windows, open a new terminal afterwards. Then:
## Something went wrong?
```sh
strata launch
```
- **My PC froze the first time Strata started.** This is normal while it loads the model. Wait, and don't close the
window. Still frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.
- **It stopped while downloading or installing.** Run `START-HERE.bat` (or `./setup.sh`) again. It continues where
it stopped.
- **It's very slow and the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does
not have enough free RAM. Close other programs (browsers use a lot), or pick a smaller size (Q2_0 or IQ2_XS).
- **It says port 8080 is already in use.** Strata is already running. Look for its window.
Follow the prompts to download and start a model. [Terminal guide](docs/TERMINAL_LAUNCH.md).
More problems and their fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an
[issue](https://github.com/Niko1221/Strata/issues) and attach `strata-<model>.log` from the Strata folder. Found a
security problem? Report it privately: [SECURITY.md](SECURITY.md).
</details>
## How does it work?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Strata terminal app showing model activity, speed, memory and recent requests">
<br><sub>Watch your model and hardware while it runs.</sub>
</p>
Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has
12-24 GB. Strata makes the model fit by **sharing the work across your whole PC**. Think of a kitchen: the things
you use all the time stay on the counter, and the rest waits in the pantry.
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
[Build from source](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Full install guide](docs/INSTALL.md)
- **The model is a team of 24,576 small specialists ("experts").** Each word needs only 10 of them.
- **Your graphics card** keeps the few thousand experts that are used most often. **Your RAM** holds all of them,
and **your processor** works on the rest at the same time. **Your SSD** holds a big lookup table.
<a id="what-you-need"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
## Hardware
- **Guess, then check:** a small helper guesses the next few words. The big model checks them all at once. You get
the same answer, 1.6-1.8x sooner.
- **Long texts are read in big pieces** (up to 8,192 tokens at a time), at over 1,000 tokens per second.
| Supported hardware | Guide |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 series** | [NVIDIA setup](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [AMD setup](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimental** | [RX 6000 setup](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel setup](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 or newer** | [Mac setup](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo setup](docs/STRIX_HALO.md) |
The longer explanation: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Every part and its numbers:
[the details](docs/DETAILS.md#how-it-works) and the [paper](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Credits and license
## Tools
The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was
compressed by [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
and Unsloth. Strata uses parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). All credits:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few
parts and every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, and more through the OpenAI- and Anthropic-compatible APIs.
Run **`strata connect`** to see setup instructions for your tool. [Connection guide](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## Help
[All guides](docs) · [Troubleshooting](docs/TROUBLESHOOTING.md) · [Report a bug](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [How it works](docs/HOW_IT_WORKS.md) · [Contribute](CONTRIBUTING.md)
## Support Strata
Strata is free and open source. If it is useful to you, you can support its development:
**Help keep local AI open to everyone.** A coffee helps fund development and testing on more hardware.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: support Strata" height="48"></a>
[MIT license](LICENSE) · [Credits and model licenses](docs/HOW_IT_WORKS.md#credits) · [Privacy](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>For AI tools and automated readers: <a href="AGENTS.md">technical reference</a> · <a href="docs/AI_SETUP.md">setup instructions</a>.</sub>
**Sponsorships, hardware partnerships or collaborations:** [[email protected]](mailto:[email protected]).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
+192
View File
@@ -0,0 +1,192 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
<h3 align="center">Uruchom potężną sztuczną inteligencję na własnym komputerze.</h3>
<p align="center">Rozmawiaj, programuj i pracuj z obrazami — lokalnie.</p>
<p align="center">
<a href="#modele">Modele</a> · <a href="#instalacja"><b>Instalacja</b></a> ·
<a href="#sprzęt">Sprzęt</a> · <a href="#narzędzia">Narzędzia</a> ·
<a href="#pomoc">Pomoc</a> · <a href="#wesprzyj-stratę">Wsparcie</a>
</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Pobierz Stratę"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licencja MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Wesprzyj Stratę na Buy Me a Coffee"></a>
</p>
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <b>Polski</b></sub></p>
**Bezpłatna, z otwartym kodem źródłowym i do uruchomienia u siebie.** Używaj Straty na swoim komputerze albo razem z ulubionymi narzędziami do programowania. Konto nie jest potrzebne.
## Modele
Najpierw polecamy **ISTA-DASLab**, potem **Unsloth** i innych dostawców — o ile zgodny model mieści się w pamięci.
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Dostawca | Do wyboru |
| --- | --- |
| **ISTA-DASLab · polecany** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) do programowania. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Zużywa mniej pamięci, ale działa wolniej. |
| **Inne opcje** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
[Porównaj rozmiary, wymagania pamięciowe i zgodność](docs/MODELS.md).
</details>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. To obecnie udokumentowane pakiety dla tego modelu.
[Przewodnik po modelu i zgodność](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
W **aplikacji na komputer** i w **Terminalu**. To, co nie mieści się w pamięci, jest czytane z dysku SSD, więc potrzebujesz
dużo wolnego miejsca na dysku.
| Zestaw (IST-DAS) | Do pobrania | Potrzebne wolne miejsce |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | około 245 GB |
| 2.00 bpw | 257 GB | około 265 GB |
| 2.25 bpw | 274 GB | około 280 GB |
Działa na kartach NVIDIA RTX serii 30, 40 i 50 (Windows i Linux), AMD Radeon AI PRO R9700 (Linux) oraz AMD Ryzen AI Max — Strix Halo
(Windows). Instalacja pobiera pliki i silnik DeepSeek, a potem uruchamia model.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Zmierzone na zestawie 2.00 bpw w Strata 0.2.5 (Strix Halo: 0.2.7). „Wyjście” to szybkość odpowiedzi, „prompt” — jak szybko Strata czyta twój tekst.
| Sprzęt | Wyjście | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB RAM | 4–6 tokenów/s | około 110 tokenów/s (4K) |
| RTX 3060 12 GB, 46 GB RAM | 9–11 tokenów/s | około 155 tokenów/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB RAM | 21–25 tokenów/s | 320–360 tokenów/s (8K) |
| 4x Radeon AI PRO R9700 | 74–77 tokenów/s | 400–450 tokenów/s (8K) |
| Ryzen AI Max+ 395 (Strix Halo), 128 GB pamięci współdzielonej | 19–21 tokenów/s (kod), 11 (czat) | 87 tokenów/s (4K), 125 tokenów/s (8K) |
[Pamięć, dysk i szczegóły](docs/DEEPSEEK41.md) · [Informacje o wydaniach](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Nie wiesz, co się zmieści? [Wybierz model do swojego komputera](docs/MODELS.md#pick-by-memory).
## Instalacja
<a id="option-a-the-strata-app"></a>
### Aplikacja na komputer
**[Pobierz Stratę dla Windows, Linuxa lub Maca →](https://github.com/Niko1221/Strata/releases/latest)**
Zainstaluj aplikację, wybierz model i zacznij rozmawiać. Dla każdego systemu jest jeden instalator. Przy pierwszym
uruchomieniu aplikacja pobiera silnik dla twojej karty graficznej. [Pomoc z instalatorem](docs/INSTALL.md#the-app).
<sub>Pliki: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Aplikacja Strata na komputer: czat i podgląd kodu na żywo obok siebie">
<br><sub>Czat, pliki i podgląd na żywo w jednym oknie.</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### Aplikacja Terminal
Wolisz terminal? Zainstaluj Stratę, a potem uruchom **`strata launch`**.
<details open>
<summary><b>Polecenia instalacji</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
**Windows PowerShell**
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
W Windows otwórz potem nowe okno terminala. Następnie:
```sh
strata launch
```
Postępuj zgodnie z podpowiedziami, aby pobrać i uruchomić model. [Przewodnik po Terminalu](docs/TERMINAL_LAUNCH.md).
</details>
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplikacja Terminal Straty pokazująca aktywność modelu, szybkość, pamięć i ostatnie zapytania">
<br><sub>Obserwuj model i sprzęt w trakcie pracy.</sub>
</p>
<a id="build-from-source"></a>
[Budowanie ze źródeł](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Pełny przewodnik instalacji](docs/INSTALL.md)
<a id="what-you-need"></a>
## Sprzęt
| Obsługiwany sprzęt | Przewodnik |
| --- | --- |
| **NVIDIA GeForce RTX serii 20 / 30 / 40 / 50** | [Konfiguracja NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Konfiguracja AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · eksperymentalnie** | [Konfiguracja RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Konfiguracja Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 lub nowszy** | [Konfiguracja Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Konfiguracja Strix Halo](docs/STRIX_HALO.md) |
<a id="using-it"></a>
## Narzędzia
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw** i inne — przez API zgodne z OpenAI i Anthropic.
Uruchom **`strata connect`**, aby zobaczyć instrukcję konfiguracji dla swojego narzędzia. [Przewodnik po połączeniach](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## Pomoc
[Wszystkie przewodniki](docs) · [Rozwiązywanie problemów](docs/TROUBLESHOOTING.md) · [Zgłoś błąd](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Testy wydajności](docs/COMMUNITY_BENCHMARKS.md) · [Jak to działa](docs/HOW_IT_WORKS.md) · [Współtworzenie](CONTRIBUTING.md)
## Wesprzyj Stratę
**Pomóż zachować lokalną sztuczną inteligencję otwartą dla wszystkich.** Kawa pomaga finansować rozwój i testy na większej liczbie sprzętu.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: wesprzyj Stratę" height="48"></a>
[Licencja MIT](LICENSE) · [Podziękowania i licencje modeli](docs/HOW_IT_WORKS.md#credits) · [Prywatność](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Dla narzędzi AI i automatycznych czytników: <a href="AGENTS.md">dokumentacja techniczna</a> · <a href="docs/AI_SETUP.md">instrukcja instalacji</a>.</sub>
**Sponsoring, partnerstwa sprzętowe i współpraca:** [[email protected]](mailto:[email protected]).
+148 -194
View File
@@ -1,234 +1,188 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · **Português**
<h3 align="center">IA poderosa no seu próprio computador.</h3>
<p align="center">Converse, programe e trabalhe com imagens, localmente.</p>
<p align="center"><b>Rode um modelo de IA de 125 bilhões de parâmetros no seu próprio PC gamer</b><br>
Placa de vídeo NVIDIA ou AMD (12 GB ou mais) · Windows ou Linux · gratuito e de código aberto</p>
<p align="center">
<a href="#modelos">Modelos</a> · <a href="#instalação"><b>Instalação</b></a> ·
<a href="#hardware">Hardware</a> · <a href="#ferramentas">Ferramentas</a> ·
<a href="#ajuda">Ajuda</a> · <a href="#apoie-o-strata">Apoiar</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Um jardim com pagode em voxels que o modelo do Strata escreveu, rodando no navegador"></a><br>
<sub>Um jardim com pagode em voxels, feito com um único prompt numa RTX 5070 com o Strata (IQ3_S, contexto de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vídeo completo (49 s)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="Baixar o Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="Licença MIT"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="Apoie o Strata no Buy Me a Coffee"></a>
</p>
O Strata roda o **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** num PC comum. É um modelo
de IA grande e inteligente que normalmente precisa de um servidor. Ele conversa, escreve código, entende imagens e
funciona com seus apps e agentes de programação. Nada sai do seu PC.
<p align="center"><sub><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <b>Português</b> · <a href="README.pl.md">Polski</a></sub></p>
## Qual é a velocidade?
**Gratuito, de código aberto e seu para executar.** Use o Strata no desktop ou com suas ferramentas de programação favoritas. Não precisa de conta.
Medimos em dois PCs gamer comuns. Um token equivale a cerca de ¾ de uma palavra.
## Modelos
- **Escreve respostas:** a velocidade com que a resposta aparece numa conversa curta. 60 tokens por segundo é mais rápido do que você consegue ler.
- **Lê seu prompt:** a velocidade com que ele absorve o que você envia (aqui, um documento, código ou histórico de conversa de 32K tokens).
Recomendamos **primeiro a ISTA-DASLab**, depois a **Unsloth** e outros provedores, quando um modelo compatível cabe na memória.
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB de RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB de RAM</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| Tamanho | Escreve respostas | Lê seu prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| Tamanho | Escreve respostas | Lê seu prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 com o engine 0.1.36, as outras linhas com 0.1.26 (respostas de 4K, prompts de 32K). As tabelas
completas estão em [DETAILS.md](docs/DETAILS.md#speed-measured). Uma placa com mais VRAM é mais rápida: uma
RTX 3090 (24 GB) deve escrever cerca de 100-140 tokens por segundo. Conversas longas e outras placas:
[velocidade de cada modelo](docs/MODELS.md#how-fast-is-each-size), [resultados da comunidade](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>O Strata é gratuito. Se ele roda bem no seu PC, um café ajuda a manter o trabalho nele.</sub></p>
## Do que você precisa
| | |
| Provedor | Opções |
| --- | --- |
| **Placa de vídeo** | **NVIDIA** GeForce RTX séries 20, 30, 40 ou 50, ou **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 ou séries RX 6800 / 6900. Ela precisa de **12 GB de VRAM ou mais**. |
| **RAM** | 32 GB ou mais. Sua RAM decide [qual modelo](#qual-modelo-devo-escolher) cabe. Com 64 GB, roda qualquer tamanho. |
| **Disco** | Cerca de 80 GB livres. Use um SSD se puder: a primeira inicialização fica bem mais rápida. |
| **Sistema** | Windows 10 / 11 ou Linux, e um driver de vídeo atualizado da NVIDIA ou da AMD. |
| **ISTA-DASLab · recomendado** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) para programar. |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Usa menos memória, mas é mais lento. |
| **Outras opções** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
O instalador cuida de todo o resto. Duas ou três placas podem dividir o modelo ([multi-GPU](docs/MULTI_GPU.md)).
[Compare tamanhos, necessidades de memória e compatibilidade](docs/MODELS.md).
Experimental, escrito e testado por membros da comunidade nas próprias máquinas:
</details>
- **Placas de vídeo mais antigas** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, compilado a partir do código-fonte no Linux: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, compilado a partir do código-fonte no Linux: [Strix Halo](docs/STRIX_HALO.md).
- **Processadores mais antigos sem AVX2**: funcionam, mas devagar. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
A lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. Esses são os pacotes documentados atualmente para este modelo.
[Guia do modelo e compatibilidade](docs/QWEN36.md).
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
No **aplicativo de desktop** e no **Terminal**. O que não cabe na memória é lido do seu SSD, por isso é preciso muito espaço livre em disco.
| Conjunto (IST-DAS) | Download | Espaço livre necessário |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | cerca de 245 GB |
| 2.00 bpw | 257 GB | cerca de 265 GB |
| 2.25 bpw | 274 GB | cerca de 280 GB |
Roda em NVIDIA RTX 30, 40 e 50 (Windows e Linux) e em AMD Radeon AI PRO R9700 (Linux). A instalação baixa os arquivos e o motor do DeepSeek e depois inicia o modelo.
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
Medido com o conjunto de 2.00 bpw no Strata 0.2.5. “Saída” é a velocidade da resposta; “Prompt” é a rapidez com que o Strata lê o seu texto.
| Hardware | Saída | Prompt |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB de RAM | 4 a 6 tokens/s | cerca de 110 tokens/s (4K) |
| RTX 3060 12 GB, 46 GB de RAM | 9 a 11 tokens/s | cerca de 155 tokens/s (4K) |
| 1x Radeon AI PRO R9700, 251 GB de RAM | 21 a 25 tokens/s | 320 a 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 a 77 tokens/s | 400 a 450 tokens/s (8K) |
[Memória, disco e detalhes](docs/DEEPSEEK41.md) · [Notas da versão](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
Não sabe o que cabe? [Escolha um modelo para o seu computador](docs/MODELS.md#pick-by-memory).
## Instalação
### Deixe sua IA instalar
<a id="option-a-the-strata-app"></a>
Você usa um assistente de programação com IA (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Cole isto nele:
### Aplicativo de desktop
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
**[Baixar o Strata para Windows, Linux ou Mac →](https://github.com/Niko1221/Strata/releases/latest)**
Instale o aplicativo, escolha um modelo e comece a conversar. Há um instalador por sistema. Na primeira vez que abre, o aplicativo baixa o motor da sua placa de vídeo. [Ajuda com o instalador](docs/INSTALL.md#the-app).
<sub>Arquivos: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Aplicativo de desktop do Strata com um chat e uma prévia do código ao vivo lado a lado">
<br><sub>Chat, arquivos e prévia ao vivo em uma só janela.</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### Aplicativo de Terminal
Prefere o terminal? Instale o Strata e execute **`strata launch`**.
<details open>
<summary><b>Comandos de instalação</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Ele verifica sua placa de vídeo, RAM e disco e escolhe o modelo que cabe. Depois instala, inicia e explica como
conectar seus apps. Ferramentas de IA também podem instalar, iniciar e parar o Strata pelo
[servidor MCP](docs/MCP_SERVER.md) dele.
**Windows PowerShell**
### Ou faça você mesmo
[Baixe o Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) e descompacte (ou use `git clone`).
**Windows:** clique duas vezes em **`START-HERE.bat`**. **Linux:** rode **`./setup.sh`** na pasta do Strata.
**Mac (Apple Silicon, M1 a M5, macOS 15 ou mais recente):** instale uma vez as Command Line Tools da Apple (`xcode-select --install` no Terminal; o Xcode em si não é necessário). Depois abra o Terminal na pasta do Strata e rode **`./setup.sh`**. Ele compila o motor do Mac no seu Mac (uns 7 minutos), faz as mesmas perguntas, baixa o modelo e o inicia. Da próxima vez, `./setup.sh` inicia na hora. Qual tamanho cabe no seu Mac e qual a velocidade: [docs/MAC.md](docs/MAC.md).
Os passos são os mesmos para NVIDIA e AMD. O instalador encontra sua placa e configura o engine certo para ela.
Ele faz algumas perguntas:
- qual modelo e qual tamanho,
- quanto contexto (quanto texto o modelo guarda na memória),
- se ele deve entender imagens.
Aperte Enter em cada uma para usar a resposta recomendada. Depois ele baixa o modelo (cerca de 70 GB) e o inicia.
Se o download parar, rode de novo: ele continua de onde parou. Seu navegador abre o app do Strata em
`http://127.0.0.1:8080`.
> **Enquanto o modelo inicia, seu PC pode ficar lento ou parar de responder por 1-3 minutos** (mais na primeira vez).
> O Strata carrega 35-55 GB na sua RAM e reserva parte dela para a placa de vídeo. Isso é normal. Espere e não
> feche a janela. A janela mostra o que o Strata está fazendo.
**Da próxima vez**, rode `START-HERE.bat` (ou `./setup.sh`) de novo. Ele inicia na hora e não baixa nada duas
vezes. Feche a janela para parar o modelo. `UPDATE.bat` (`./update.sh`) atualiza o Strata sem iniciá-lo.
Atualização, Docker, várias placas, onde ficam os arquivos e todas as opções: [docs/INSTALL.md](docs/INSTALL.md).
## Qual modelo devo escolher?
O instalador recomenda um de acordo com sua RAM. O mesmo modelo vem em vários tamanhos, mais ou menos comprimidos.
Os tamanhos menores são mais rápidos. Os maiores são um pouco mais inteligentes.
| Sua RAM | Escolha | Por quê |
| --- | --- | --- |
| **32 GB** | **Coder** | cabe em 32 GB e é feito para código (com uma placa de 24 GB, Q2_0 e IQ2_XS também rodam) |
| **48 GB** | **IQ2_XS** (ou Q2_0, o mais rápido) | os tamanhos maiores não cabem |
| **64 GB** | **IQ2_XS** (recomendado), ou IQ3_XXS / IQ3_S | todos os tamanhos cabem; IQ3_S é o melhor e o mais lento |
| **96 GB ou mais** | **IQ3_S**, ou o UD-IQ4_XS da Unsloth (~4-bit) | espaço para os maiores tamanhos com todo o resto aberto |
- **[Coder](docs/MODELS.md#coder):** uma versão para programação com metade dos experts removida. Ela alcança 91%
da pontuação do modelo completo no SWE-bench Verified (medido pelos autores) e cabe em 32 GB de RAM. É mais fraca
fora do código, inclusive em chinês e outros textos CJK (#438). Para isso, escolha Q2_0, IQ2_XS ou IQ3_S, que
mantêm todos os experts.
- **[Swift 1.5](docs/MODELS.md#swift-15):** um fine-tune que pensa por bem menos tempo antes de responder. Você
recebe a resposta mais cedo, com quase a mesma qualidade.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** a versão de ~4 bits da Unsloth, entre IQ3_S e
UD-Q4_K_XL em qualidade. Um download de 94 GB. Com menos de ~80 GB de RAM, o Strata lê uma parte dele do SSD
enquanto responde, então ali ele fica mais lento (um SSD NVMe ajuda).
- **[Unsloth UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL e Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):** de 4 a 8 bits, mais próximos do modelo
completo (93-95% o mesmo próximo token que o Q8_0). Rápidos quando todos os especialistas cabem na RAM (~88 GB para
o UD-Q4_K_XL até ~136 GB para o Q8_0); com menos RAM, o Strata lê uma parte do SSD enquanto responde, o que é bem
mais lento (7-8.5 tokens/s para o UD-Q4_K_XL num PC com 64 GB).
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** você configura à mão. Ele
não está no menu do instalador.
Tamanhos, downloads e o que cabe onde: [docs/MODELS.md](docs/MODELS.md). Para adicionar outro modelo depois, rode
`SETUP.bat` (Linux: `./setup.sh --setup`).
## Como usar
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="A aba Monitor do app Strata ao lado de um agente de programação"><br>
<sub>O <b>Monitor</b> do app Strata (à esquerda) enquanto um agente de programação escreve o jardim com pagode do vídeo (à direita)</sub></p>
- **No navegador:** abra `http://127.0.0.1:8080`. Lá tem o **Chat**, um **Monitor** ao vivo do modelo e da sua
GPU/CPU/RAM, e o **About** com as configurações e os endereços.
- **Seus apps e agentes de programação:** adicione um provedor "OpenAI-compatible" com a URL base
**`http://127.0.0.1:8080/v1`**. Qualquer chave de API e qualquer nome de modelo funcionam.
- Apps que usam a API da Anthropic: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI e outros apps que usam a OpenAI Responses API: `/v1/responses`
([configuração](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Raciocínio:** escolha **off, low, medium ou high** no menu do chat ou no "reasoning effort" do seu app. Off é o
mais rápido. High é o melhor para perguntas difíceis.
- **Imagens:** responda sim para "Images?" na instalação. Depois clique em **Picture** no chat, ou anexe imagens no
seu app. Placas AMD entendem imagens no Linux usando o processador; no Windows, ainda não.
- **Do seu celular ou de outro PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Sempre defina uma chave.
- **Uma requisição por vez:** por padrão, o Strata responde uma requisição e as outras esperam. Para responder
várias ao mesmo tempo, defina `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). Numa placa de 12 GB, isso deixa
cada resposta mais lenta.
- **Prompts longos:** o Strata lê a primeira mensagem de uma conversa inteira, cerca de 1 minuto a cada 30,000
tokens. As mensagens seguintes começam em segundos.
Mais: [onde suas conversas ficam salvas](docs/INSTALL.md#where-things-are-stored), [a API](docs/DETAILS.md#using-it).
### Opcional: mais velocidade com uma placa quando o modelo inteiro cabe na RAM
Duas configurações opcionais (de RubenTorney, #1904 e #1905). Elas ajudam quando você usa uma placa de vídeo e sua RAM guarda todos os experts (o modo arena do Strata, por exemplo Q2_0 ou IQ3_XXS num PC com 64 GB). Abra sua configuração `strata-<model>.json` na pasta de dados do Strata ([onde](docs/INSTALL.md#where-things-are-stored)) e adicione:
- **Geração mais rápida:** `"--adapt-async", "1"` na lista `"args"`. O cache de experts passa a ser atualizado numa thread auxiliar em vez de pausar a resposta. Medido numa RTX 5070 (Windows, Q2_0): +4-5% de geração, +7-10% em código e documentos longos. As respostas podem diferir um pouco do padrão (ficam iguais de uma execução para outra).
- **Leitura do prompt mais rápida no Windows:** `"STRATA_PIN_BY_NEED": "1"` no bloco `"env"`. O Windows só consegue travar cerca de metade da RAM para a placa; isto trava primeiro os experts de que a placa realmente precisa. Medido na RTX 5070: +4,4% na leitura do prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Reinicie o Strata (`START-HERE.bat` ou `./setup.sh`) depois da mudança. Para desfazer, remova as duas entradas.
No Windows, abra um novo terminal depois. Em seguida:
## Algo deu errado?
```sh
strata launch
```
- **Meu PC travou na primeira vez que o Strata iniciou.** Isso é normal enquanto ele carrega o modelo. Espere e não
feche a janela. Ainda travado depois de 10 minutos? Reinicie o PC, feche outros programas e tente de novo, ou
escolha um tamanho menor.
- **Parou durante o download ou a instalação.** Rode `START-HERE.bat` (ou `./setup.sh`) de novo. Ele continua de
onde parou.
- **Está muito lento e a luz do disco não para de piscar, ou aparece "the engine stopped unexpectedly".** Seu PC
não tem RAM livre suficiente. Feche outros programas (navegadores usam muita), ou escolha um tamanho menor (Q2_0
ou IQ2_XS).
- **Diz que a porta 8080 já está em uso.** O Strata já está rodando. Procure a janela dele.
Siga as instruções para baixar e iniciar um modelo. [Guia do Terminal](docs/TERMINAL_LAUNCH.md).
Mais problemas e como resolver: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Ainda sem solução? Abra uma
[issue](https://github.com/Niko1221/Strata/issues) e anexe o `strata-<model>.log` da pasta do Strata. Encontrou um
problema de segurança? Informe em particular: [SECURITY.md](SECURITY.md).
</details>
## Como funciona?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Aplicativo de Terminal do Strata mostrando a atividade do modelo, a velocidade, a memória e as solicitações recentes">
<br><sub>Acompanhe seu modelo e seu hardware enquanto ele roda.</sub>
</p>
Modelos como este normalmente rodam em servidores com centenas de gigabytes de memória de vídeo. Sua placa de vídeo
tem 12-24 GB. O Strata faz o modelo caber **dividindo o trabalho pelo PC inteiro**. Pense numa cozinha: o que você
usa o tempo todo fica na bancada, e o resto espera na despensa.
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Os 24,576 experts do modelo: os mais usados na placa de vídeo, todos na RAM, uma tabela de consulta no SSD"></p>
[Compilar a partir do código-fonte](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Guia de instalação completo](docs/INSTALL.md)
- **O modelo é uma equipe de 24,576 pequenos especialistas ("experts").** Cada palavra precisa de só 10 deles.
- **Sua placa de vídeo** guarda os poucos milhares de experts usados com mais frequência. **Sua RAM** guarda todos
eles, e **seu processador** trabalha no resto ao mesmo tempo. **Seu SSD** guarda uma grande tabela de consulta.
<a id="what-you-need"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Um pequeno ajudante adivinha as próximas palavras; o modelo grande confere todas de uma vez e fica com as certas"></p>
## Hardware
- **Adivinhar e conferir:** um pequeno ajudante adivinha as próximas palavras. O modelo grande confere todas de
uma vez. Você recebe a mesma resposta, 1.6-1.8x mais cedo.
- **Textos longos são lidos em grandes pedaços** (até 8,192 tokens por vez), a mais de 1,000 tokens por segundo.
| Hardware compatível | Guia |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50** | [Instalação para NVIDIA](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [Instalação para AMD](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · experimental** | [Instalação para RX 6000](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Instalação para Intel](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 ou mais novo** | [Instalação para Mac](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Instalação para Strix Halo](docs/STRIX_HALO.md) |
A explicação mais longa: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte e seus números:
[os detalhes](docs/DETAILS.md#how-it-works) e o [artigo](docs/paper/Strata-Paper.pdf).
<a id="using-it"></a>
## Créditos e licença
## Ferramentas
O modelo é o [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), da equipe Qwen. Ele foi
comprimido por [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
e Unsloth. O Strata usa partes do [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Todos os créditos:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). O Strata é de código aberto sob a [MIT License](LICENSE).
Algumas partes e todos os modelos têm licenças próprias ([quais](docs/HOW_IT_WORKS.md#license)).
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, e outras, pelas APIs compatíveis com OpenAI e Anthropic.
Execute **`strata connect`** para ver as instruções da sua ferramenta. [Guia de conexão](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect).
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## Ajuda
[Todos os guias](docs) · [Solução de problemas](docs/TROUBLESHOOTING.md) · [Relatar um bug](https://github.com/Niko1221/Strata/issues/new/choose) ·
[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [Como funciona](docs/HOW_IT_WORKS.md) · [Contribuir](CONTRIBUTING.md)
## Apoie o Strata
O Strata é gratuito e de código aberto. Se ele é útil para você, você pode apoiar o desenvolvimento:
**Ajude a manter a IA local aberta para todos.** Um café ajuda a financiar o desenvolvimento e os testes em mais hardware.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee: apoie o Strata" height="48"></a>
[Licença MIT](LICENSE) · [Créditos e licenças dos modelos](docs/HOW_IT_WORKS.md#credits) · [Privacidade](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>Para ferramentas de IA e leitores automáticos: <a href="AGENTS.md">referência técnica</a> · <a href="docs/AI_SETUP.md">instruções de instalação</a>.</sub>
**Patrocínios, parcerias de hardware ou colaborações:** [[email protected]](mailto:[email protected]).
+148 -177
View File
@@ -1,217 +1,188 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="340">
</picture>
</p>
[English](README.md) · **简体中文** · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<h3 align="center">在你自己的电脑上运行强大的 AI。</h3>
<p align="center">本地聊天、写代码、处理图片。</p>
<p align="center"><b>在你自己的游戏电脑上运行 1250 亿参数的 AI 模型</b><br>
NVIDIA 或 AMD 显卡(12 GB 及以上)· Windows 或 Linux · 免费开源</p>
<p align="center">
<a href="#模型">模型</a> · <a href="#安装"><b>安装</b></a> ·
<a href="#硬件">硬件</a> · <a href="#工具">工具</a> ·
<a href="#帮助">帮助</a> · <a href="#支持-strata">支持</a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Strata 的模型写出的体素宝塔花园,在浏览器中运行"></a><br>
<sub>体素宝塔花园,一次提示生成,在 RTX 5070 上用 Strata 运行(IQ3_S,128K 上下文)·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">完整视频(49 秒)</a></sub></p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases/latest"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?label=download&color=0ab887" alt="下载 Strata"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT 许可证"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="https://img.shields.io/badge/Buy_me_a_coffee-FFDD00?logo=buymeacoffee&logoColor=000" alt="在 Buy Me a Coffee 上支持 Strata"></a>
</p>
Strata 能在普通电脑上运行 **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**。这是一个又大又聪明的
AI 模型,通常要用服务器才能跑。它能聊天、写代码、看图片,还能配合你的应用和编程智能体一起工作。所有数据都留在你的电脑上。
<p align="center"><sub><a href="README.md">English</a> · <b>简体中文</b> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a> · <a href="README.pl.md">Polski</a></sub></p>
## 速度有多快?
**免费、开源,由你自己运行。** 在桌面上使用 Strata,或搭配你喜欢的编程工具。无需账号。
我们在两台普通的游戏电脑上做了测试。一个 token 大约相当于 ¾ 个英文单词。
## 模型
- **写回答:** 短对话中回复出现的速度。每秒 60 个 token 已经比你的阅读速度还快。
- **读提示:** 读入你发送内容的速度(这里是一份 32K token 的文档、代码或聊天记录)。
我们**首先推荐 ISTA-DASLab**,其次是 **Unsloth** 和其他提供方,前提是有内存放得下的兼容模型。
<table>
<tr><th>NVIDIA:RTX 5070(12 GB)、Ryzen 5 7600、64 GB 内存</th><th>AMD:RX 9070 XT(16 GB)、Ryzen 9 3900X、47 GB 内存</th></tr>
<tr><td>
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.8-Flash-Next</b></summary>
| 规格 | 写回答 | 读提示 |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| 规格 | 写回答 | 读提示 |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA:Q2_0 用的是引擎 0.1.36,其他各行用的是 0.1.26(4K 回答,32K 提示)。完整表格见
[DETAILS.md](docs/DETAILS.md#speed-measured)。显存越大的显卡越快:RTX 3090(24 GB)写回答应该能达到每秒约
100-140 个 token。长对话和其他显卡的数据:[各模型的速度](docs/MODELS.md#how-fast-is-each-size)、
[社区测试结果](docs/COMMUNITY_BENCHMARKS.md)。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="请我喝杯咖啡" height="50"></a><br>
<sub>Strata 是免费的。如果它在你的电脑上跑得不错,请我喝杯咖啡,让这个项目继续下去。</sub></p>
## 你需要什么
| | |
| 提供方 | 可选项 |
| --- | --- |
| **显卡** | **NVIDIA** GeForce RTX 20、30、40 或 50 系列,或 **AMD** Radeon RX 7900 XT / XTX、RX 7800 XT / 7700 XT、RX 9060 XT、RX 9070 / 9070 XT、Radeon AI PRO R9700 或 RX 6800 / 6900 系列。需要 **12 GB 或以上显存**。 |
| **内存** | 32 GB 或以上。内存大小决定能装下[哪个模型](#该选哪个模型)。64 GB 可以运行所有规格。 |
| **硬盘** | 约 80 GB 可用空间。尽量用 SSD:第一次启动会快很多。 |
| **系统** | Windows 10 / 11 或 Linux,以及 NVIDIA 或 AMD 的最新显卡驱动。 |
| **ISTA-DASLab · 推荐** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S。[Coder:IQ1\_M](docs/MODELS.md#coder),适合写代码。 |
| **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 |
| **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)。占用内存更少,但速度更慢。 |
| **其他选项** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) |
其他的都由安装程序搞定。两到三张显卡可以分担同一个模型([多 GPU](docs/MULTI_GPU.md))。
[比较大小、内存需求和兼容性](docs/MODELS.md)。
实验性支持,由社区成员在自己的机器上编写和测试:
</details>
- **较老的显卡**(Tesla P40 / V100、GTX 10、Radeon VII / MI50、RX 6700 XT、RX 5500 XT):[较老的 GPU](docs/OLDER_GPUS.md)。
- **Intel Arc**,在 Linux 上从源码构建:[Intel Arc](docs/INTEL_ARC.md)。
- **AMD Ryzen AI Max(Strix Halo)**,在 Linux 上从源码构建:[Strix Halo](docs/STRIX_HALO.md)。
- **不支持 AVX2 的老处理器**:能用,但很慢。[较老的 CPU](docs/INSTALL.md#older-cpus-experimental)。
<details>
<summary><img src="docs/media/models/qwen.svg" width="22" alt="" align="middle"> &nbsp;<b>Qwen3.6-35B-A3B</b></summary>
完整列表:[docs/INSTALL.md](docs/INSTALL.md#what-you-need)。
**Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0。这些是目前文档中收录的该模型的版本。
[模型指南和兼容性](docs/QWEN36.md)。
</details>
<details>
<summary><img src="docs/media/models/deepseek.svg" width="22" alt="" align="middle"> &nbsp;<b>DeepSeekV4.1-Flash-1017</b></summary>
可在**桌面应用**和**终端**中使用。内存放不下的部分会从 SSD 读取,所以需要大量空闲磁盘空间。
| 版本(IST-DAS) | 下载大小 | 所需空闲磁盘 |
| --- | ---: | ---: |
| 1.75 bpw | 239 GB | 约245 GB |
| 2.00 bpw | 257 GB | 约265 GB |
| 2.25 bpw | 274 GB | 约280 GB |
支持 NVIDIA RTX 30、40 和 50 系列(Windows 和 Linux),以及 AMD Radeon AI PRO R9700(Linux)。安装程序会下载模型文件和 DeepSeek 引擎,然后启动模型。
```sh
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
```
以下为 Strata 0.2.5 上 2.00 bpw 版本的实测结果。“输出”是回答的速度,“提示词”是 Strata 读取你输入内容的速度。
| 硬件 | 输出 | 提示词 |
| --- | ---: | ---: |
| RTX 5070 12 GB, 64 GB 内存 | 4 到 6 tokens/s | 约 110 tokens/s(4K) |
| RTX 3060 12 GB, 46 GB 内存 | 9 到 11 tokens/s | 约 155 tokens/s(4K) |
| 1x Radeon AI PRO R9700, 251 GB 内存 | 21 到 25 tokens/s | 320 到 360 tokens/s (8K) |
| 4x Radeon AI PRO R9700 | 74 到 77 tokens/s | 400 到 450 tokens/s (8K) |
[内存、磁盘和详细信息](docs/DEEPSEEK41.md) · [发行说明](https://github.com/Niko1221/Strata/releases)
</details>
<a id="the-other-sizes"></a>
不确定哪个放得下?[为你的电脑选择模型](docs/MODELS.md#pick-by-memory)。
## 安装
### 让你的 AI 帮你装
<a id="option-a-the-strata-app"></a>
你在用 AI 编程助手吗(Claude Code、Cursor、Codex、GitHub Copilot 等)?把下面这段粘贴给它:
### 桌面应用
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
**[下载适用于 Windows、Linux 或 Mac 的 Strata →](https://github.com/Niko1221/Strata/releases/latest)**
安装应用,选择模型,就可以开始聊天。每个系统只有一个安装程序。首次启动时,应用会下载适合你显卡的引擎。[安装程序帮助](docs/INSTALL.md#the-app)。
<sub>文件: `Strata-0.2.7-windows-x64-setup.exe` · `Strata-0.2.7-linux-x86_64.AppImage` · `Strata-0.2.7-macos-arm64.dmg`</sub>
<p align="center">
<img src="docs/media/app-chat.png" width="900" alt="Strata 桌面应用,聊天和实时代码预览并排显示">
<br><sub>聊天、文件和实时预览都在一个窗口里。</sub>
</p>
<a id="option-b-the-terminal-one-line"></a>
### 终端应用
更喜欢终端?安装 Strata,然后运行 **`strata launch`**。
<details open>
<summary><b>安装命令</b></summary>
**Linux / Mac**
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
它会检查你的显卡、内存和硬盘,选出合适的模型。然后安装并启动它,再告诉你怎么连接你的应用。AI 工具也可以通过
Strata 的 [MCP 服务器](docs/MCP_SERVER.md)来安装、启动和停止 Strata。
**Windows PowerShell**
### 或者自己动手
[下载 Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) 并解压(或者用 `git clone`)。
**Windows:** 双击 **`START-HERE.bat`**。**Linux:** 在 Strata 文件夹里运行 **`./setup.sh`**。
**Mac(Apple 芯片,M1 到 M5,macOS 15 或更新版本):** 先安装一次 Apple 的 Command Line Tools(在终端里运行 `xcode-select --install`;不需要 Xcode 本身)。然后在 Strata 文件夹里打开终端,运行 **`./setup.sh`**。它会在你的 Mac 上编译 Mac 版引擎(大约 7 分钟),问同样的问题,下载模型并启动。下次运行 `./setup.sh` 会马上启动。哪个大小适合你的 Mac、速度如何:[docs/MAC.md](docs/MAC.md)。
NVIDIA 和 AMD 的步骤完全一样。安装程序会识别你的显卡,并装好对应的引擎。它会问你几个问题:
- 用哪个模型、哪个规格,
- 上下文多大(模型能记住多少文字),
- 是否要识别图片。
每次直接按回车就是推荐选项。之后它会下载模型(约 70 GB)并启动。如果下载中断了,再运行一次即可:它会从中断的地方继续。
浏览器会打开 Strata 应用,地址是 `http://127.0.0.1:8080`。
> **模型启动时,你的电脑可能会变慢或卡住 1-3 分钟**(第一次最久)。
> Strata 会把 35-55 GB 加载到内存,并为显卡锁定其中一部分。这是正常的。请耐心等待,不要关闭窗口。
> 窗口里会显示 Strata 正在做什么。
**下次使用时**,再运行一次 `START-HERE.bat`(或 `./setup.sh`)。它会马上启动,已下载的东西不会重复下载。关闭它的窗口就能停止模型。
`UPDATE.bat`(`./update.sh`)只更新 Strata,不启动。更新、Docker、多张显卡、文件存放位置以及所有选项:
[docs/INSTALL.md](docs/INSTALL.md)。
## 该选哪个模型
安装程序会根据你的内存推荐一个。同一个模型有几种规格,压缩程度不同。规格越小越快,越大越聪明一些。
| 你的内存 | 选择 | 原因 |
| --- | --- | --- |
| **32 GB** | **Coder** | 32 GB 装得下,而且专为代码打造(如果显卡是 24 GB,Q2_0 和 IQ2_XS 也能跑) |
| **48 GB** | **IQ2_XS**(或 Q2_0,最快) | 更大的规格装不下 |
| **64 GB** | **IQ2_XS**(推荐),或 IQ3_XXS / IQ3_S | 所有规格都装得下;IQ3_S 最好,也最慢 |
| **96 GB 或以上** | **IQ3_S**,或 Unsloth 的 UD-IQ4_XS(约 4-bit) | 开着其他程序也能放下最大的规格 |
- **[Coder](docs/MODELS.md#coder):** 编程版本,去掉了一半专家。它达到完整模型 SWE-bench Verified 分数的 91%(由其作者测得),
32 GB 内存就能装下。代码以外的能力较弱,包括中文和其他中日韩文本(#438)。这类用途请选 Q2_0、IQ2_XS 或 IQ3_S,
它们保留了所有专家。
- **[Swift 1.5](docs/MODELS.md#swift-15):** 一个微调版本,回答前思考的时间短得多。你能更早拿到答案,质量基本不变。
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth 的约 4-bit 版本,质量介于 IQ3_S 和 UD-Q4_K_XL 之间。
下载 94 GB。内存少于约 80 GB 时,Strata 回答时要从 SSD 读取其中一部分,所以会更慢(NVMe SSD 有帮助)。
- **[Unsloth UD-Q4_K_XL、UD-Q5_K_XL、UD-Q6_K_XL 和 Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):** 4 到 8 位,更接近完整模型(下一个 token
与 Q8_0 相同的比例为 93-95%)。所有专家都能放进内存时速度很快(UD-Q4_K_XL 约需 88 GB,Q8_0 约需 136 GB);内存不够时,
Strata 回答时要从 SSD 读取一部分,会慢很多(64 GB 的电脑上 UD-Q4_K_XL 每秒 7-8.5 个 token)。
- **[OrcaRouter 的 Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** 需要手动设置,不在安装程序的菜单里。
规格、下载以及各配置能装下什么:[docs/MODELS.md](docs/MODELS.md)。以后想再添加模型,运行
`SETUP.bat`(Linux:`./setup.sh --setup`)。
## 使用方法
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="Strata 应用的 Monitor 标签页,旁边是一个编程智能体"><br>
<sub>Strata 应用的 <b>Monitor</b>(左),编程智能体正在写视频里的宝塔花园(右)</sub></p>
- **在浏览器里:** 打开 `http://127.0.0.1:8080`。里面有 **Chat**(聊天)、实时 **Monitor**(监控模型和你的
GPU/CPU/内存),以及 **About**(设置和地址)。
- **你的应用和编程智能体:** 添加一个“OpenAI 兼容”的提供商,base URL 填 **`http://127.0.0.1:8080/v1`**。
API key 和模型名随便填都行。
- 使用 Anthropic API 的应用:`http://127.0.0.1:8080/v1/messages`(Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`)。
- Codex CLI 和其他使用 OpenAI Responses API 的应用:`/v1/responses`
([设置方法](docs/DETAILS.md#the-responses-api-and-codex-cli))。
- **思考:** 在聊天菜单或应用的“reasoning effort”里选择 **off、low、medium 或 high**。off 最快,
high 最适合难题。
- **图片:** 在安装时对“Images?”选是。然后在聊天里点 **Picture**,或在你的应用里附上图片。
AMD 显卡在 Linux 上通过处理器识别图片;在 Windows 上暂时还不行。
- **从手机或另一台电脑访问:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`。一定要设置 key。
- **一次处理一个请求:** 默认情况下 Strata 一次只回答一个请求,其他请求排队等待。想同时回答多个,
设置 `"parallel": 2`([BATCHING.md](docs/BATCHING.md))。在 12 GB 显卡上,这会让每个回答变慢。
- **长提示:** Strata 会完整读入对话的第一条消息,大约每 30,000 个 token 需要 1 分钟。之后的消息几秒内就开始回答。
更多:[聊天记录存在哪里](docs/INSTALL.md#where-things-are-stored)、[API](docs/DETAILS.md#using-it)。
### 可选:整个模型都能放进内存时,单卡再提速
两个可选设置(来自 RubenTorney 的 #1904 和 #1905)。适用于只用一张显卡、并且内存能放下所有专家的情况(Strata 的 arena 模式,例如 64 GB 电脑上的 Q2_0 或 IQ3_XXS)。打开 Strata 数据文件夹里的配置文件 `strata-<model>.json`([在哪里](docs/INSTALL.md#where-things-are-stored)),加入:
- **生成更快:** 在 `"args"` 列表里加 `"--adapt-async", "1"`。专家缓存改在辅助线程里更新,不再暂停回答。在 RTX 5070(Windows,Q2_0)上实测:生成 +4-5%,代码和长文档 +7-10%。回答可能与默认设置略有不同(但每次运行都一样)。
- **Windows 上读取提示更快:** 在 `"env"` 块里加 `"STRATA_PIN_BY_NEED": "1"`。Windows 只能为显卡锁定大约一半的内存;这个设置会先锁定显卡真正需要的专家。在 RTX 5070 上实测:读取提示 +4.4%。
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
修改后重启 Strata(`START-HERE.bat` 或 `./setup.sh`)。要撤销,删掉这两项即可。
在 Windows 上,安装后请打开一个新的终端。然后运行:
## 出问题了?
```sh
strata launch
```
- **Strata 第一次启动时电脑卡死了。** 加载模型时这是正常的。请耐心等待,不要关闭窗口。
10 分钟后还卡着?重启电脑,关掉其他程序再试一次,或者换一个更小的规格。
- **下载或安装时中断了。** 再运行一次 `START-HERE.bat`(或 `./setup.sh`)。它会从中断的地方继续。
- **非常慢,硬盘灯一直闪,或者提示“the engine stopped unexpectedly”。** 你的电脑可用内存不够。
关掉其他程序(浏览器很占内存),或者换一个更小的规格(Q2_0 或 IQ2_XS)。
- **提示 8080 端口已被占用。** Strata 已经在运行了。找一下它的窗口。
按提示下载并启动模型。[终端指南](docs/TERMINAL_LAUNCH.md)。
更多问题和解决办法:[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)。还是解决不了?提交一个
[issue](https://github.com/Niko1221/Strata/issues),并附上 Strata 文件夹里的 `strata-<model>.log`。发现了安全问题?
请私下报告:[SECURITY.md](SECURITY.md)。
</details>
## 它是怎么工作的?
<p align="center">
<img src="docs/media/terminal-monitor.png" width="900" alt="Strata 终端应用,显示模型活动、速度、内存和最近的请求">
<br><sub>模型运行时,随时查看模型和硬件状态。</sub>
</p>
这类模型通常运行在拥有数百 GB 显存的服务器上。而你的显卡只有 12-24 GB。Strata 让模型装得下的办法是
**把工作分摊到整台电脑上**。可以想象一个厨房:常用的东西放在台面上,其余的放在储藏室里。
<a id="build-from-source"></a>
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="模型的 24,576 个专家:最常用的在显卡上,全部在内存里,一张查找表在 SSD 上"></p>
[从源码构建](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [完整安装指南](docs/INSTALL.md)
- **这个模型是由 24,576 个小专家(“experts”)组成的团队。** 每个词只需要其中 10 个。
- **你的显卡** 存放最常用的几千个专家。**你的内存** 存放全部专家,
**你的处理器** 同时处理其余的部分。**你的 SSD** 存放一张大查找表。
<a id="what-you-need"></a>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="一个小助手猜测接下来的几个词;大模型一次性全部检查,保留正确的"></p>
## 硬件
- **先猜,再检查:** 一个小助手先猜接下来的几个词,大模型再一次性检查它们。答案完全一样,但快 1.6-1.8 倍。
- **长文本分大块读入**(每次最多 8,192 个 token),速度超过每秒 1,000 个 token。
| 支持的硬件 | 指南 |
| --- | --- |
| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 系列** | [NVIDIA 安装](docs/INSTALL.md#what-you-need) |
| **AMD Radeon RX 7700 / 7800 / 7900、RX 9060 / 9070、AI PRO R9700** | [AMD 安装](docs/AMD_HIP.md) |
| **AMD Radeon RX 6800 / 6900 · 实验性** | [RX 6000 安装](docs/AMD_HIP.md#rdna2-gfx1030) |
| **Intel Arc · Linux** | [Intel 安装](docs/INTEL_ARC.md) |
| **Apple Silicon · M1 或更新** | [Mac 安装](docs/MAC.md) |
| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo 安装](docs/STRIX_HALO.md) |
更详细的解释:[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md)。每个部分及其数据:
[详细说明](docs/DETAILS.md#how-it-works)和[论文](docs/paper/Strata-Paper.pdf)。
<a id="using-it"></a>
## 致谢与许可证
## 工具
模型是 Qwen 团队的 [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)。它由
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)
和 Unsloth 压缩。Strata 使用了 [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) 的部分代码。完整致谢:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits)。Strata 以 [MIT 许可证](LICENSE)开源。少数部分和每个模型
有各自的许可证([具体是哪些](docs/HOW_IT_WORKS.md#license))。
**Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**,也可以通过兼容 OpenAI 和 Anthropic 的 API 使用更多工具。
运行 **`strata connect`**,查看你所用工具的设置方法。[连接指南](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect)。
<a id="guides"></a>
<a id="how-fast-is-it"></a>
<a id="how-does-it-work"></a>
<a id="something-went-wrong"></a>
## 帮助
[全部指南](docs) · [故障排查](docs/TROUBLESHOOTING.md) · [报告问题](https://github.com/Niko1221/Strata/issues/new/choose) ·
[基准测试](docs/COMMUNITY_BENCHMARKS.md) · [工作原理](docs/HOW_IT_WORKS.md) · [参与贡献](CONTRIBUTING.md)
## 支持 Strata
Strata 免费且开源。如果它对你有用,欢迎支持它的开发:
**帮助本地 AI 保持开放。** 一杯咖啡能帮助我们在更多硬件上开发和测试。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="请我喝杯咖啡" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee:支持 Strata" height="48"></a>
[MIT 许可证](LICENSE) · [致谢和模型许可证](docs/HOW_IT_WORKS.md#credits) · [隐私](docs/SAFETY.md)
<a id="option-c-let-your-ai-set-it-up"></a>
<sub>面向 AI 工具和自动阅读器: <a href="AGENTS.md">技术参考</a> · <a href="docs/AI_SETUP.md">安装说明</a></sub>
**赞助、硬件合作或其他合作:** [[email protected]](mailto:[email protected]).
+55 -3
View File
@@ -23,8 +23,9 @@ this PC only. The details and every setting are in [docs/DETAILS.md](docs/DETAIL
- **Where it listens.** `127.0.0.1` by default. `--host 0.0.0.0` (or `"host"` in `strata-<model>.json`) opens it to
your network, and the server then warns when no API key is set.
- **API key.** `"api_key"` in the run config (or `STRATA_API_KEY`) is required on `/v1/*` and on every endpoint
that shows the model's state, requests or answers (`/status`, `/metrics`, `/settings`, `/mcp`, `/props`, `/slots`,
`/api/requests`, `/config`) and on every `POST`. It is compared in constant time. Set one before you open the
that shows the model's state, requests or answers (`/status`, `/metrics`, `/metrics/prometheus`, `/settings`,
`/mcp`, `/props`, `/slots`, `/api/requests`, `/config`) and on every `POST`. It is compared in constant time.
Set one before you open the
server to your network or put a tunnel in front of it.
- **Host check (DNS rebinding, 0.1.38).** Without an API key the server answers only requests whose `Host` is a name
it knows (`localhost`, an IP address, the address it listens on and, when it listens beyond this PC, this PC's
@@ -39,6 +40,57 @@ this PC only. The details and every setting are in [docs/DETAILS.md](docs/DETAIL
- **MCP tools** are opt-in: only the servers you put in the run config, only for requests from Strata's own page
that ask for them. They run with your user's rights, and the model decides when to call them.
- **The request monitor** (`/api-monitor`, which keeps the last prompts and answers in memory) is off unless
`"api_monitor": true` is set.
`"api_monitor": true` is set. Apart from it, the server keeps in its process memory the text and tokens of its
last few prompts, whatever `"api_monitor"` says, to encode the next turn faster; no endpoint shows them, and they
are gone when the server stops.
## The desktop app and the Terminal
Strata comes in three parts that share one engine: the **desktop app**, the **Terminal** (`strata`) and the **engine** that runs
the model. The section above is about the engine's HTTP server. This is what the other two add.
- **The app's host listens on this PC only.** The app serves its page from a local host process on `127.0.0.1` (port 8791 unless
changed). A request to its `/api/` endpoints that carries a browser `Origin` from any other site gets 403. The app shell holds a
random secret (32 characters or more, compared in constant time) that is needed to quit the host or tell it which window is open,
and the host gives its own helper processes a second random secret. By default the models the app starts are served on this PC only.
- **Sharing is a separate, key-protected listener.** "Share with other devices" (and the Cloudflare tunnel) will not start until an
API key is set. It opens a different listener that answers only `/v1/models`, `/v1/chat/completions` and `/v1/messages`, never the
app's page or its host endpoints.
- **App and Terminal talk through an owner channel.** One owner process per installation starts and ends the engine, and a second
client (the Terminal) attaches to it. On macOS and Linux that is a private Unix socket in a folder only you can open. On Windows it
is a TCP listener on `127.0.0.1` that answers only requests carrying a 256-bit token kept in an owner-only folder. That is a weaker
guarantee than a socket other users cannot open: on Windows the token is what protects it, as with any loopback service that has a
token (`strata_control/owner_transport.py`). The bridge between the app shell and the host (used for the app update) is also loopback and protected by a token.
- **Starting a model and serving never download anything.** Downloads and updates are one explicit step: `strata update`, or "Update
engine" and "Update Strata" in the app's Settings. Models and saved tuning are never replaced by an update.
- **Updates come from a signed feed.** The release publishes `strata-feed.json` and `strata-feed.json.sig` (Ed25519). An installed
Strata checks the signature against the public key it carries (`strata_control/update_keys.json`), refuses a feed older than the last
one it accepted (a replayed old feed) or expired, and checks every engine package against its SHA-256, its signed manifest and the
card in your PC before it test-starts it and switches to it. Any failure keeps the engine you had. The app installer itself is signed
with a separate key (Tauri's minisign key) that the app and the Terminal both check. The private keys are kept offline by the owner.
See [packaging/UPDATES.md](packaging/UPDATES.md).
### How far to trust `curl | sh` and `irm | iex`
The Terminal installs with one line:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Piping a script into a shell runs whatever the server sends, so it deserves the same care as running any downloaded program.
What the script does is short and written at its top: it fetches the Terminal archive named in the release's `install-manifest.txt`,
checks its SHA-256, unpacks it into your home folder (no administrator rights, nothing outside your home), checks the release's signed feed
with the Terminal it just unpacked and removes it again if that fails, and puts `strata` on your PATH.
Those checks have a limit. The checksum comes from the same release as the archive, and the public key that checks the feed arrives inside
that archive, so for the first install all trust rests on HTTPS and on the `Niko1221/Strata` GitHub release itself. The checks catch a
damaged or half-finished download and a feed that was not signed by the release key; they do not protect you if the release has been replaced
by someone who controls it. Later updates are checked against the key already on your PC, which is a stronger position.
If that is not enough: save the script (`curl -fsSLO https://github.com/Niko1221/Strata/releases/latest/download/install.sh`), read it, and
run it from the file; or install the desktop app or build from source instead and check the code you build.
Strata has not had an outside security audit yet.
What Strata downloads, when it uses the internet, what stays on your PC and how downloads are checked is written up in [docs/SAFETY.md](docs/SAFETY.md).
+13
View File
@@ -2,8 +2,13 @@
rem Strata for Windows: the first run installs everything and starts the model; later runs just start it.
rem Needs only an NVIDIA or AMD graphics driver. Python is installed for your user account if it is missing (no admin needed).
setlocal
rem Python reads and writes text as UTF-8, not in the Windows code page (cp1251, cp866, ...): a folder or user name with Cyrillic or other letters stays intact (#3202).
set "PYTHONUTF8=1"
title Strata
cd /d "%~dp0"
rem A bundled Python runtime (the python\ folder of the terminal archive) is used when it is there: nothing to find,
rem install or download. SETUP.bat and UPDATE.bat run this file, so they use it too.
if exist "python\python.exe" goto bundled
if exist ".venv\Scripts\python.exe" goto run
call :findpy
@@ -36,6 +41,14 @@ echo Could not create the Python environment in .venv
pause
exit /b 1
:bundled
set "PYTHONNOUSERSITE=1"
set "PYTHONHOME="
set "PYTHONPATH="
"python\python.exe" setup.py %*
if errorlevel 1 pause
exit /b
:run
".venv\Scripts\python.exe" setup.py %*
if errorlevel 1 pause
+2 -1
View File
@@ -2,7 +2,8 @@
rem Update Strata without starting the model (#475): the newest code (git pull, when this folder is a git clone), then
rem what START-HERE.bat does before a start - the engine (a new one when this version needs it), the Python packages,
rem each installed model's settings and draft subset. The model files are not touched. Start the model later with
rem START-HERE.bat. Options are passed on to setup.py.
rem START-HERE.bat. Options are passed on to setup.py. The bundled Python (python\ beside this file) is used when
rem there is one: START-HERE.bat, which this file runs, picks it.
setlocal
title Strata - update
cd /d "%~dp0"
+232
View File
@@ -0,0 +1,232 @@
{
"name": "strata-desktop",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "strata-desktop",
"version": "0.1.0",
"devDependencies": {
"@tauri-apps/cli": "2.12.3"
}
},
"node_modules/@tauri-apps/cli": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli/-/cli-2.12.3.tgz",
"integrity": "sha512-f7/LsY74SVb1ChmOPw1ajOpxii0GSzDAgmWHRci8JUYuuai04rXsSbFobpmB0sP2DdlvMD/B5ZotEn8GOGMRUg==",
"dev": true,
"license": "Apache-2.0 OR MIT",
"bin": {
"tauri": "tauri.js"
},
"engines": {
"node": ">= 10"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/tauri"
},
"optionalDependencies": {
"@tauri-apps/cli-darwin-arm64": "2.12.3",
"@tauri-apps/cli-darwin-x64": "2.12.3",
"@tauri-apps/cli-linux-arm-gnueabihf": "2.12.3",
"@tauri-apps/cli-linux-arm64-gnu": "2.12.3",
"@tauri-apps/cli-linux-arm64-musl": "2.12.3",
"@tauri-apps/cli-linux-riscv64-gnu": "2.12.3",
"@tauri-apps/cli-linux-x64-gnu": "2.12.3",
"@tauri-apps/cli-linux-x64-musl": "2.12.3",
"@tauri-apps/cli-win32-arm64-msvc": "2.12.3",
"@tauri-apps/cli-win32-ia32-msvc": "2.12.3",
"@tauri-apps/cli-win32-x64-msvc": "2.12.3"
}
},
"node_modules/@tauri-apps/cli-darwin-arm64": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-arm64/-/cli-darwin-arm64-2.12.3.tgz",
"integrity": "sha512-xmVNoJOOKzKLj7GLHXmXLimRby9M42aBIJa0sB80N7o24eKV5JiVmToKTRLVH9yhL05GZ/B6rSSOmbzNFJwBAA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-darwin-x64": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-x64/-/cli-darwin-x64-2.12.3.tgz",
"integrity": "sha512-URd+0usfmUQQyb5kNPiKt2OT8gp7oGU7QR+qOppB6e35mpVBFFVNhWc9oWb3p0u71fDJ09DpF4T6m9xYxoIB8g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-arm-gnueabihf": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm-gnueabihf/-/cli-linux-arm-gnueabihf-2.12.3.tgz",
"integrity": "sha512-0kwUVx2LbmOoTRlwF1oDww7th4/NSluv4ZBQ6KRZsFKg4g0TlTcIiJ0VehqkQYUeC9xIT1Mk6XtF52mI155VSQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-arm64-gnu": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-gnu/-/cli-linux-arm64-gnu-2.12.3.tgz",
"integrity": "sha512-gKSGDx6fx+HBLQmG6guChFdisbrCSdx9Yli+j5teOa64gm1CeM+C6NLru5Pj3ZkkEep05BmCFjxvX+cjqAycZQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-arm64-musl": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.12.3.tgz",
"integrity": "sha512-Yr59qMy/gcvAfBuFdCGlAmGiLsYWMFDbsP4bAI6iNBzcJTGrYWZGeiQhGDo29MQwfF/NUBqyM2qljXu5tZ6wjQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-riscv64-gnu": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-riscv64-gnu/-/cli-linux-riscv64-gnu-2.12.3.tgz",
"integrity": "sha512-jKr0/REBMnLKC0OwPQZkyge0mKDaNZ0b/AbfGwVyCgFfwS730EVLfVDo3nt7MUu6jJaytRq08+xSj5qfy+U1+Q==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-x64-gnu": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-gnu/-/cli-linux-x64-gnu-2.12.3.tgz",
"integrity": "sha512-gTkEqJAGCMROGanxHnzaJzQ+/rXl39HWVg0/hDODS1wDWBGqoceqGC6Qm1+mQbyT11xnsB8mKu/gyVID/YpFIw==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-x64-musl": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-musl/-/cli-linux-x64-musl-2.12.3.tgz",
"integrity": "sha512-T2dTxL1svRym71fZtaW4pZYXbthbfFsEJr1NeOUg/p7laWXZbUt/8DMSZf8recu9f7+1F/Lv00HtzsEu6vDO9w==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-win32-arm64-msvc": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-arm64-msvc/-/cli-win32-arm64-msvc-2.12.3.tgz",
"integrity": "sha512-eoiQwpEZWbpsVHcM9BAzqqe4TCHszvr3rsM7pUQ4ofLK0lF1i1d/jX0Qo5DYMqcrUtwQuDdDuOCbcT0Izf1VwQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-win32-ia32-msvc": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-ia32-msvc/-/cli-win32-ia32-msvc-2.12.3.tgz",
"integrity": "sha512-kE+KUg6Dbbkgvcw0m99wJVIToFQzJqpMBAXah74WqxaLzssNwSTUccrHp6+QVzMhjQP330tvekFmnQF+0NbE6Q==",
"cpu": [
"ia32"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-win32-x64-msvc": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-x64-msvc/-/cli-win32-x64-msvc-2.12.3.tgz",
"integrity": "sha512-L7wndWAgX2mLW7alswpL3d1R0P9FN2WYBgHaQkAYEBxHHGkY4vFX9CjcRQ1LAxDAMIJxrY7/aDziFJeChS84NQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
}
}
}
+20
View File
@@ -0,0 +1,20 @@
{
"name": "strata-desktop",
"productName": "Strata",
"version": "0.2.7",
"private": true,
"description": "Strata Desktop - local AI, tuned to one model.",
"scripts": {
"build:ui": "npm --prefix ../host run build",
"typecheck:ui": "npm --prefix ../host run typecheck",
"test:ui": "npm --prefix ../host test",
"tauri": "tauri",
"dev": "npm run build:ui && tauri dev",
"build": "npm run build:ui && tauri build",
"build:app": "npm run build:ui && tauri build --bundles app",
"dmg": "npm run build:ui && tauri build --bundles dmg"
},
"devDependencies": {
"@tauri-apps/cli": "2.12.3"
}
}
+361
View File
@@ -0,0 +1,361 @@
#!/usr/bin/env python3
"""The installed Windows app, closed every way, with a stub engine: nothing may be left running.
python apps/desktop/scripts/app-close-matrix.py --install <folder the installer wrote> [--scenarios all] [--report out.json]
For each case it starts the REAL installed `strata-desktop.exe` (its window opens), loads a stub engine through the host's API
(profile + `runtime start` under Control's owner runtime, exactly the app's path), records every process of the tree (the app,
the host, Control, the owner host, the server, the engine), ends the app one way, and asserts:
* no process of the tree is left, the engine's port is free, the runtime lease can be taken again;
* the engine got the clean stop where the way of closing allows one (normal close, window X);
* the next load of the same slot works (a lost owner's state is cleaned, not a blocker).
It also proves a second launch focuses the open window and starts nothing, and that one app runs two engines and closes both.
Everything per-user lives under a scratch folder; the stub engine is a script. No real engine ever runs.
"""
from __future__ import annotations
import argparse
import ctypes
import json
import os
import subprocess
import sys
import tempfile
import time
import urllib.error
import urllib.request
from ctypes import wintypes
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(ROOT))
import psutil # noqa: E402
from strata_control import procs, runtime # noqa: E402
from tests.terminal.lifecycle_matrix import install_fake_engine, port_open # noqa: E402
if os.name != "nt":
raise SystemExit("this matrix drives the Windows installer's app")
user32 = ctypes.WinDLL("user32", use_last_error=True)
WM_CLOSE = 0x0010
def main_window(pid: int) -> int | None:
found = []
@ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
def visit(hwnd, _):
owner = wintypes.DWORD()
user32.GetWindowThreadProcessId(hwnd, ctypes.byref(owner))
if owner.value == pid and user32.IsWindowVisible(hwnd):
buf = ctypes.create_unicode_buffer(256)
user32.GetWindowTextW(hwnd, buf, 256)
if buf.value.strip() == "Strata":
found.append(hwnd)
return True
user32.EnumWindows(visit, 0)
return found[0] if found else None
class Scratch:
def __init__(self, root: Path, install: Path):
self.root, self.install = root, install
self.components = root / "components"
self.models = root / "models"
self.mark = root / "mark"
for d in (self.components, self.models, self.mark, root / "local", root / "home", root / "appdata", root / "webview"):
d.mkdir(parents=True, exist_ok=True)
install_fake_engine(self.components)
from tools.strata_tokenizer import BYTE_TO_UNICODE
tok = self.models / "tokenizer"
tok.mkdir()
tokens = [BYTE_TO_UNICODE[i] for i in range(256)] + ["ab", "<|im_start|>", "<|im_end|>", "<|endoftext|>"]
(tok / "vocab.json").write_text(json.dumps({t: i for i, t in enumerate(tokens)}))
(tok / "merges.txt").write_text("a b")
(tok / "token_type.json").write_text(json.dumps([0] * 257 + [3, 3, 3]))
for name in ("stub", "stub2"):
(self.models / f"strata-{name}.json").write_text(json.dumps({
"exe": "component:strata-engine", "backend": "cuda", "tokenizer": str(tok), "args": [], "cwd": str(root),
"model_name": name, "host": "127.0.0.1", "port": 0}))
self.env = {**os.environ, "STRATA_COMPONENTS_DIR": str(self.components), "STRATA_PROFILE_DIR": str(root / "profiles"),
"STRATA_RUNTIME_DIR": str(root / "runtime"), "STRATA_INSTALL_DIR": str(self.models),
"STRATA_MODELS_DIR": str(root / "models-unused"), "STRATA_ENGINE_BACKEND": "cuda",
"STRATA_HOME": str(root / "home"), "APPDATA": str(root / "appdata"), "LOCALAPPDATA": str(root / "local"),
"WEBVIEW2_USER_DATA_FOLDER": str(root / "webview"), "FAKE_ENGINE_MARK": str(self.mark),
"FAKE_ENGINE_SLOW_STEPS": "0", "PYTHONDONTWRITEBYTECODE": "1"}
self.env.pop("STRATA_API_KEY", None)
for key in ("STRATA_COMPONENTS_DIR", "STRATA_PROFILE_DIR", "STRATA_RUNTIME_DIR", "LOCALAPPDATA", "STRATA_ENGINE_BACKEND",
"STRATA_NO_ACQUIRE", "FAKE_ENGINE_MARK"):
os.environ[key] = self.env.get(key, "1") # this process reads the same runtime and records folders
class App:
def __init__(self, scratch: Scratch):
self.s = scratch
self.proc = subprocess.Popen([str(scratch.install / "strata-desktop.exe")], cwd=scratch.install, env=scratch.env,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
self.base = None
def host(self):
for child in psutil.Process(self.proc.pid).children(recursive=True):
if child.name().lower() == "node.exe":
return child
return None
def wait_up(self, seconds=90):
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
if self.proc.poll() is not None:
raise RuntimeError(f"the app exited with {self.proc.returncode}")
try:
host = self.host()
if host:
for c in host.net_connections(kind="tcp"):
if c.status == "LISTEN" and c.laddr.ip == "127.0.0.1":
base = f"http://127.0.0.1:{c.laddr.port}"
try:
if urllib.request.urlopen(base + "/api/health", timeout=2).status == 200 and main_window(self.proc.pid):
self.base = base
return
except OSError:
pass
except psutil.Error:
pass
time.sleep(0.5)
raise RuntimeError("the app did not come up")
def call(self, path, method="GET", body=None):
req = urllib.request.Request(self.base + path, method=method, data=None if body is None else json.dumps(body).encode(),
headers={"content-type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=120) as r:
return json.loads(r.read() or b"null")
except urllib.error.HTTPError as exc:
raise RuntimeError(f"{method} {path} -> {exc.code} {exc.read()[:300]!r}") from None
def load(self, model, multiple=False):
self.call("/api/native?refresh=1")
started = self.call("/api/native/operations", "POST", {"kind": "load", "args": {"model": model, **({"multiple": True} if multiple else {})}})
deadline = time.monotonic() + 180
while time.monotonic() < deadline:
state = self.call("/api/native")
job = next((j for j in state["jobs"] if j["id"] == started["id"]), None)
if job and job["state"] != "running":
if job["state"] != "done":
raise RuntimeError(f"load {model}: {job['state']} {job.get('error')}")
return state
time.sleep(0.3)
raise RuntimeError("load did not finish")
def tree_of(app: App, slots=("app-1",)) -> tuple[list[procs.Ident], list[int]]:
idents, ports = [], []
members = [psutil.Process(app.proc.pid), *psutil.Process(app.proc.pid).children(recursive=True)]
pids = {p.pid for p in members}
for slot in slots:
state = runtime._read_state(slot)
if state:
pids.add(state["host_pid"])
for r in procs.records():
if r.get("profile") in slots: # the app's own slots; a Terminal service is not part of the app's tree
pids.add(r["pid"])
for pid in pids:
token = procs.start_token(pid)
if token:
idents.append(procs.Ident(pid, token, tuple(procs.command_line(pid) or [])))
return idents, ports
def survivors(idents):
return [f"{i.pid} {' '.join(i.cmd)[:80]}" for i in idents if procs.start_token(i.pid) == i.token]
def wait_gone(idents, seconds):
deadline = time.monotonic() + seconds
left = survivors(idents)
while left and time.monotonic() < deadline:
time.sleep(0.25)
left = survivors(idents)
return left
def lease_free() -> bool:
try:
fd = runtime.acquire_activity_lease(shared=False)
except Exception: # noqa: BLE001
return False
runtime._unlock_file(fd)
os.close(fd)
return True
def control(scratch: Scratch, *argv: str, timeout: int = 300) -> dict:
py = scratch.install / "control" / "python" / "python.exe"
out = subprocess.run([str(py), str(scratch.install / "control" / "strata.py"), *argv, "--output", "json"],
env={**scratch.env, "PYTHONNOUSERSITE": "1", "STRATA_NO_ACQUIRE": "1"},
capture_output=True, text=True, timeout=timeout)
try:
return json.loads(out.stdout)
except ValueError:
return {"ok": False, "raw": out.stdout[-300:] + out.stderr[-300:]}
def stop_cli(scratch: Scratch, slot: str) -> dict:
py = scratch.install / "control" / "python" / "python.exe"
out = subprocess.run([str(py), str(scratch.install / "control" / "strata.py"), "runtime", "stop", "--profile", slot, "--wait", "30",
"--output", "json"], env={**scratch.env, "PYTHONNOUSERSITE": "1", "STRATA_NO_ACQUIRE": "1"},
capture_output=True, text=True, timeout=120)
try:
return json.loads(out.stdout)
except ValueError:
return {"state": "?", "raw": out.stdout[-200:] + out.stderr[-200:]}
def case(name: str, install: Path, how: str) -> dict:
with tempfile.TemporaryDirectory(prefix="strata-app-matrix-", ignore_cleanup_errors=True) as tmp:
saved = dict(os.environ)
scratch = Scratch(Path(tmp), install)
app = App(scratch)
problems, note = [], ""
try:
app.wait_up()
slots = ("app-1", "app-2") if how == "two-engines" else ("app-1",)
terminal = None
if how == "window-x-terminal-engine":
# an engine the Terminal started (its own profile, no tie to the app) must survive the app's close
control(scratch, "profile", "create", "terminal-1", "--from-config", str(scratch.models / "strata-stub2.json"))
terminal = control(scratch, "runtime", "start", "--profile", "terminal-1", "--wait", "60")
if terminal.get("state") != "ready":
raise RuntimeError(f"the Terminal's service did not start: {terminal}")
terminal_pids = {r["pid"] for r in procs.records(profile="terminal-1")} | {runtime._read_state("terminal-1")["host_pid"]}
state = app.load("stub")
if how == "two-engines":
state = app.load("stub2", multiple=True)
engine_ports = [i["port"] for i in state["instances"]]
idents, _ = tree_of(app, slots)
kinds = sorted({("engine" if "engine" in " ".join(i.cmd) else "server" if "serve.server" in " ".join(i.cmd) else
"owner" if "strata_control.runtime" in " ".join(i.cmd) else "app" if "strata-desktop" in i.cmd[0].lower()
else "host" if "node" in i.cmd[0].lower() else "other") for i in idents})
note = f"tree {len(idents)} processes: {', '.join(kinds)}; engine ports {engine_ports}"
t0 = time.monotonic()
if how == "normal-close":
subprocess.run(["taskkill", "/PID", str(app.proc.pid)], capture_output=True)
elif how in ("window-x", "two-engines", "window-x-terminal-engine"):
hwnd = main_window(app.proc.pid)
if not hwnd:
raise RuntimeError("no main window")
user32.PostMessageW(hwnd, WM_CLOSE, 0, 0)
elif how == "kill-app":
subprocess.run(["taskkill", "/F", "/PID", str(app.proc.pid)], capture_output=True)
elif how == "host-crash":
host = app.host()
subprocess.run(["taskkill", "/F", "/PID", str(host.pid)], capture_output=True)
elif how == "second-launch":
second = subprocess.Popen([str(install / "strata-desktop.exe")], cwd=install, env=scratch.env,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
try:
code = second.wait(timeout=30)
except subprocess.TimeoutExpired:
second.kill()
problems.append("the second launch did not exit")
code = None
time.sleep(2)
hosts = [p for p in psutil.process_iter(["name", "pid"]) if p.info["name"].lower() == "node.exe"
and (p.parent() and p.parent().pid == app.proc.pid)]
apps = [p for p in psutil.process_iter(["name"]) if p.info["name"].lower() == "strata-desktop.exe"]
if app.proc.poll() is not None:
problems.append("the first app went away")
if len(apps) != 1:
problems.append(f"{len(apps)} app processes (want 1)")
if len(hosts) != 1:
problems.append(f"{len(hosts)} hosts under the first app (want 1)")
after, _ = tree_of(app, slots)
new = [i for i in after if i.pid not in {x.pid for x in idents}]
if new:
problems.append(f"the second launch started: {[' '.join(i.cmd)[:60] for i in new]}")
note += f"; second launch exit code {code}, app processes {len(apps)}, hosts {len(hosts)}"
subprocess.run(["taskkill", "/PID", str(app.proc.pid)], capture_output=True)
if how == "host-crash":
# the shell and its window stay (nothing here restarts a dead host); the host, Control's owner, the server and
# the engine must not
idents = [i for i in idents if any(k in " ".join(i.cmd) for k in ("strata_control.runtime", "serve.server", "engine", "node.exe"))]
left = wait_gone(idents, 90)
if left:
problems.append(f"processes left: {left}")
for port in engine_ports:
deadline = time.monotonic() + 10
while port_open(port) and time.monotonic() < deadline:
time.sleep(0.2)
if port_open(port):
problems.append(f"engine port {port} still open")
if terminal is None and not lease_free():
problems.append("lease still held")
marks = sorted(p.name for p in scratch.mark.iterdir())
if terminal is not None:
status = control(scratch, "runtime", "status", "--profile", "terminal-1")
alive_pids = [p for p in terminal_pids if procs.start_token(p)]
if status.get("state") != "ready" or len(alive_pids) != len(terminal_pids):
problems.append(f"closing the app disturbed the Terminal's service: {status.get('state')}, alive {alive_pids} of {sorted(terminal_pids)}")
note += f"; the Terminal's service is {status.get('state')} after the app closed"
stopped = control(scratch, "runtime", "stop", "--profile", "terminal-1", "--wait", "30")
if stopped.get("state") != "stopped" or [p for p in terminal_pids if procs.start_token(p)]:
problems.append(f"the Terminal's service did not stop on request: {stopped}")
time.sleep(1)
if not lease_free():
problems.append("lease still held after the Terminal's service stopped")
if how in ("normal-close", "window-x", "two-engines", "second-launch", "window-x-terminal-engine") and not ({"quit", "eof"} & set(marks)):
problems.append(f"the engine was not stopped the clean way (marks {marks})")
seconds = round(time.monotonic() - t0, 1)
if how == "host-crash":
subprocess.run(["taskkill", "/PID", str(app.proc.pid)], capture_output=True)
note += "; the app shell stays open after a host crash (by design today)"
# the slot must be usable again: stopping a lost owner's slot cleans its state; nothing blocks the next load
for slot in slots:
res = stop_cli(scratch, slot)
already = "not managed or is stopped" in json.dumps(res) # nothing was left to stop: the clean outcome
if res.get("state") not in ("stopped",) and not already:
problems.append(f"runtime stop {slot} after the close: {res}")
if runtime._read_state(slot) is not None:
problems.append(f"state of {slot} left behind")
return {"name": f"app-{how}", "ok": not problems, "problems": problems, "seconds": seconds, "marks": marks, "note": note}
except Exception as exc: # noqa: BLE001
return {"name": f"app-{how}", "ok": False, "problems": [f"{type(exc).__name__}: {exc}"] + problems, "note": note}
finally:
for p in psutil.process_iter(["name", "exe"]):
try:
if (p.info["exe"] or "").lower().startswith(str(install).lower()):
p.kill()
except psutil.Error:
pass
for r in procs.records():
try:
psutil.Process(r["pid"]).kill()
except psutil.Error:
pass
os.environ.clear(); os.environ.update(saved)
SCENARIOS = ["normal-close", "window-x", "window-x-terminal-engine", "kill-app", "host-crash", "second-launch", "two-engines"]
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--install", required=True)
ap.add_argument("--scenarios", default="all")
ap.add_argument("--report")
a = ap.parse_args()
names = SCENARIOS if a.scenarios == "all" else a.scenarios.split(",")
results = []
for how in names:
r = case(how, Path(a.install), how)
results.append(r)
print(("PASS " if r["ok"] else "FAIL ") + r["name"], r.get("seconds", ""), r.get("problems") or "", r.get("note", ""), flush=True)
if a.report:
Path(a.report).write_text(json.dumps({"platform": "windows", "engine": "fake", "results": results}, indent=1), encoding="utf-8")
return 0 if all(r["ok"] for r in results) else 1
if __name__ == "__main__":
raise SystemExit(main())
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env node
/**
* The Graphics cards choice of the app, end to end on a box with two cards, through the installed app layout (packaged host + bundled Control + a real
* engine component): the model list carries the options (one rule, strata_control/gpu_fit.py), a model that fits one card recommends that card, then
* load with ONE card -> answers, only that card holds the model; load with BOTH cards (the layer split) -> answers, both cards hold the model.
*
* node cards-e2e.mjs --install <folder with node/ host/ control/> --components <folder holding engine/<package>> --work <scratch> \
* --family qwen --model IQ3_XXS --data <data folder that has the model files> [--smi "nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits"]
*/
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readdirSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
for (const need of ['install', 'components', 'work', 'model']) if (!args[need]) { console.error('missing --' + need); process.exit(2); }
const install = resolve(args.install), work = resolve(args.work), components = resolve(args.components), family = String(args.family ?? 'qwen'), modelId = String(args.model);
const node = join(install, 'node', 'node'), hostEntry = join(install, 'host', 'server', 'index.mjs');
const dirs = { models: join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home'), xdg: join(work, 'xdg'), appdata: join(work, 'appdata') };
for (const d of [dirs.models, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
let failed = false;
const record = (name, ok, detail = '') => { if (!ok) failed = true; console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 500) : ''}`); };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.models,
STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models-unused'), APPDATA: dirs.appdata, XDG_CONFIG_HOME: dirs.xdg, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(node, [hostEntry], { cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'],
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
async function waitJob(id, seconds) {
let last = '';
for (let i = 0; i < seconds * 2; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.summary !== last) { last = job.summary; console.log(' ... ' + String(last).slice(0, 120)); }
if (job && job.state !== 'running') return { job, state };
await delay(500);
}
throw new Error('operation did not finish');
}
const smi = () => {
const cmd = String(args.smi ?? 'nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits').split(' ');
const r = spawnSync(cmd[0], cmd.slice(1), { encoding: 'utf8' });
return (r.stdout ?? '').split('\n').map(s => Number(s.trim())).filter(Number.isFinite);
};
async function chatPong(instance) {
const chat = await call('/v1/chat/completions', 'POST', { model: instance?.apiModelId ?? modelId.toLowerCase(), max_tokens: 400, stream: false, temperature: 0, messages: [{ role: 'user', content: 'Reply with exactly one word: pong' }] });
const m = chat.body?.choices?.[0]?.message ?? {};
return { ok: chat.status === 200 && /pong/i.test(`${m.content ?? ''}`), text: `${m.content ?? ''}`.slice(0, 60), status: chat.status };
}
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { /* not yet */ } if (!up) await delay(150); }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
const packages = existsSync(join(components, 'engine')) ? readdirSync(join(components, 'engine')) : [];
record('an engine component is installed', packages.length > 0, packages.join(', '));
// ---- the model list carries the choice
const native = (await call('/api/native?refresh=1')).body;
const gpus = native.hardware?.gpus ?? [];
record('the host sees two or more graphics cards', gpus.filter(g => g.usable !== false).length >= 2, JSON.stringify(gpus.map(g => ({ i: g.index, n: g.name, gb: g.vram_gb }))));
const size = native.catalog?.families?.find(f => f.family === family)?.sizes.find(s => s.model === modelId);
const opt = size?.gpu_options;
record('the model list offers the cards choice for this model', !!opt && opt.options.length >= 2, JSON.stringify(opt?.options?.map(o => ({ key: o.key, cards: o.cards, verdict: o.verdict, rec: o.recommended }))));
record('a model that fits one card recommends the single largest card (the split is offered, not forced)', opt?.recommended === 'single' && opt.options.find(o => o.key === 'split'), `recommended=${opt?.recommended}`);
const single = opt?.options.find(o => o.key === 'single')?.cards ?? [0];
const both = opt?.options.find(o => o.key === 'split')?.cards ?? [0, 1];
// ---- Download and prepare with the recommended card
const values = { family, model: modelId, context: 32768, vision: 'no', gpu: single[0], ...(args.data ? { data_dir: resolve(args.data) } : {}) };
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: values });
record('the install operation starts', started.status === 200 || started.status === 202, JSON.stringify(started.body).slice(0, 160));
const done = await waitJob(started.body.id, 3600);
record('Download and prepare finishes', done.job.state === 'done', `${done.job.state} ${done.job.error ?? done.job.summary}`);
const installed = (await call('/api/native?refresh=1')).body.installed?.models?.find(m => String(m.model).toLowerCase() === modelId.toLowerCase());
const model = installed?.model ?? modelId.toLowerCase();
for (const [label, cards] of [['ONE card', single], ['BOTH cards (layer split)', both]]) {
const before = smi();
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model, gpus: cards } });
const loaded = await waitJob(load.body.id, 900);
const instance = loaded.state.instances[0];
record(`load with ${label} starts the engine`, loaded.job.state === 'done' && instance?.status === 'ready', `${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}; gpus=${JSON.stringify(instance?.gpus)}`);
record(`the instance runs on exactly the chosen card(s)`, JSON.stringify([...(instance?.gpus ?? [])].sort()) === JSON.stringify([...cards].sort()), `want ${JSON.stringify(cards)} got ${JSON.stringify(instance?.gpus)}`);
await delay(3000);
const used = smi();
const holding = used.map((mb, i) => (mb - (before[i] ?? 0)) > 1000 ? i : -1).filter(i => i >= 0);
record(`only the chosen card(s) hold the model (graphics memory in use: ${used.join(', ')} MiB)`, JSON.stringify(holding) === JSON.stringify([...cards].sort()), `holding ${JSON.stringify(holding)}`);
const pong = await chatPong(instance);
record(`it answers (${label})`, pong.ok, `${pong.status} ${JSON.stringify(pong.text)}`);
const unloaded = await call(`/api/native/instances/${instance?.id}/unload`, 'POST', {});
record(`unload (${label})`, unloaded.status === 200, unloaded.status);
await delay(4000);
}
}
try { await main(); } catch (e) { record('run', false, e.stack ?? e); }
host.kill();
await delay(1500);
console.log(failed ? 'CARDS-E2E FAIL' : 'CARDS-E2E PASS');
process.exit(failed ? 1 : 0);
+72
View File
@@ -0,0 +1,72 @@
# Fetch and verify the private Node runtime the Windows desktop bundle ships (win-x64).
#
# The PowerShell twin of fetch-node.sh: same pinned version, same SHA-256 pin, same refusal to
# install anything whose digest does not match. Not committed output: node.exe is a ~90 MB binary.
#
# Usage:
# powershell -NoProfile -ExecutionPolicy Bypass -File apps\desktop\scripts\fetch-node.ps1
# ... -File fetch-node.ps1 -Check # verify what is already installed
# ... -File fetch-node.ps1 -FromFile node.zip # verify and install a zip you already have (offline)
#
# The Node version must satisfy the App Host's own engines.node (>= 24.12.0).
[CmdletBinding()]
param(
[switch]$Check,
[string]$FromFile = '',
[string]$Dest = ''
)
$ErrorActionPreference = 'Stop'
$NodeVersion = if ($env:STRATA_NODE_VERSION) { $env:STRATA_NODE_VERSION } else { '24.12.0' }
$Target = 'win-x64'
# SHA-256 of node-v24.12.0-win-x64.zip from nodejs.org's SHASUMS256.txt; keep equal to fetch-node.sh.
$Expected = '9c125f61ae947b52e779095830f9cac267846a043ef7192183c84016aaad2812'
if (-not $Dest) { $Dest = Join-Path $PSScriptRoot '..\src-tauri\resources\node' }
$Installed = Join-Path $Dest 'node.exe'
function Fail([string]$Message) { [Console]::Error.WriteLine("fetch-node: $Message"); exit 1 }
if ($Check) {
if (-not (Test-Path -LiteralPath $Installed)) { Fail "not installed at $Installed" }
$got = (& $Installed --version).Trim()
Write-Output "fetch-node: $Installed is $got"
if ($got -ne "v$NodeVersion") { Fail "expected v$NodeVersion" }
exit 0
}
$Archive = "node-v$NodeVersion-$Target.zip"
$Work = Join-Path ([IO.Path]::GetTempPath()) ("strata-node-" + [Guid]::NewGuid().ToString('N'))
New-Item -ItemType Directory -Path $Work | Out-Null
try {
$Zip = Join-Path $Work $Archive
if ($FromFile) {
Copy-Item -LiteralPath $FromFile -Destination $Zip
} else {
$Url = "https://nodejs.org/dist/v$NodeVersion/$Archive"
Write-Output "fetch-node: $Url"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -UseBasicParsing -Uri $Url -OutFile $Zip
}
$Actual = (Get-FileHash -Algorithm SHA256 -LiteralPath $Zip).Hash.ToLowerInvariant()
if ($NodeVersion -ne '24.12.0') {
# another version has no recorded digest: say so rather than implying a verification
[Console]::Error.WriteLine("fetch-node: WARNING - no recorded digest for v$NodeVersion; this download is NOT authenticated.")
[Console]::Error.WriteLine("fetch-node: computed sha256 $Actual - record it in this script and fetch-node.sh.")
} elseif ($Actual -ne $Expected) {
Fail "REFUSING - $Archive hashes $Actual, expected $Expected"
} else {
Write-Output 'fetch-node: sha256 verified against the recorded digest'
}
Add-Type -AssemblyName System.IO.Compression.FileSystem
$zipFile = [IO.Compression.ZipFile]::OpenRead($Zip)
try {
$entry = $zipFile.GetEntry("node-v$NodeVersion-$Target/node.exe")
if (-not $entry) { Fail "node.exe is not inside $Archive" }
New-Item -ItemType Directory -Force -Path $Dest | Out-Null
[IO.Compression.ZipFileExtensions]::ExtractToFile($entry, $Installed, $true)
} finally { $zipFile.Dispose() }
Write-Output "fetch-node: installed at $Installed"
} finally {
Remove-Item -LiteralPath $Work -Recurse -Force -ErrorAction SilentlyContinue
}
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env bash
# Fetch and verify the private Node runtime the desktop bundle ships.
#
# The App Host is an ordinary Node program, and the packaging requirement is that an end user
# supplies no runtime of their own. The desktop bundle therefore carries a pinned standalone Node
# beside the host.
#
# It is NOT committed. It is a ~117 MB platform binary: committing it would bloat every clone, it
# cannot be reviewed in a diff, and it is not source. This script is the reproducible substitute —
# it downloads the official build for the target and refuses to install anything whose SHA-256 does
# not match the recorded digest.
#
# Usage:
# apps/desktop/scripts/fetch-node.sh # host platform
# apps/desktop/scripts/fetch-node.sh --target linux-x64
# apps/desktop/scripts/fetch-node.sh --check # verify what is already installed
#
# The Node version must satisfy the App Host's own `engines.node` (>= 24.12.0).
set -euo pipefail
NODE_VERSION="${STRATA_NODE_VERSION:-24.12.0}"
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEST="$HERE/../src-tauri/resources/node"
# Published SHA-256 of each official nodejs.org archive, recorded so a tampered or truncated
# download is refused rather than shipped. Add a row when bumping the version.
# Taken from nodejs.org's own SHASUMS256.txt for v24.12.0, which is published over HTTPS beside
# the archives. Recorded here so the fetch is verified against a value that does not come from the
# same response as the payload.
digest_for() {
case "$1" in
darwin-arm64) echo "319f221adc5e44ff0ed57e8a441b2284f02b8dc6fc87b8eb92a6a93643fd8080" ;;
linux-x64) echo "6159227e0af7d7c3c6bb2fa900452b04a6cb8841a702a79acc613209d70b04d0" ;;
linux-arm64) echo "9b2a2eeb98a8eb37361224e2a1d060300ad2dd143af58dfdb16de785df0f1228" ;;
win-x64) echo "9c125f61ae947b52e779095830f9cac267846a043ef7192183c84016aaad2812" ;;
*) echo "" ;;
esac
}
target() {
if [ "${1:-}" = "--target" ]; then echo "${2:?--target needs a value}"; return; fi
case "$(uname -s)-$(uname -m)" in
Darwin-arm64) echo darwin-arm64 ;;
Linux-x86_64) echo linux-x64 ;;
Linux-aarch64) echo linux-arm64 ;;
MINGW*|MSYS*|CYGWIN*) echo win-x64 ;;
*) echo "unsupported host: $(uname -s)-$(uname -m)" >&2; exit 2 ;;
esac
}
TARGET="$(target "$@")"
EXE="node"
case "$TARGET" in win-*) EXE="node.exe" ;; esac
INSTALLED="$DEST/$EXE"
if [ "${1:-}" = "--check" ]; then
if [ ! -x "$INSTALLED" ]; then echo "fetch-node: not installed at $INSTALLED" >&2; exit 1; fi
got="$("$INSTALLED" --version)"
echo "fetch-node: $INSTALLED is $got"
case "$got" in "v$NODE_VERSION") exit 0 ;; *) echo "fetch-node: expected v$NODE_VERSION" >&2; exit 1 ;; esac
fi
# The official archive for this target.
case "$TARGET" in
darwin-arm64) ARCHIVE="node-v$NODE_VERSION-darwin-arm64.tar.gz" ;;
darwin-x64) ARCHIVE="node-v$NODE_VERSION-darwin-x64.tar.gz" ;;
linux-x64) ARCHIVE="node-v$NODE_VERSION-linux-x64.tar.gz" ;;
linux-arm64) ARCHIVE="node-v$NODE_VERSION-linux-arm64.tar.gz" ;;
win-x64) ARCHIVE="node-v$NODE_VERSION-win-x64.zip" ;;
*) echo "fetch-node: no archive known for $TARGET" >&2; exit 2 ;;
esac
URL="https://nodejs.org/dist/v$NODE_VERSION/$ARCHIVE"
EXPECTED="$(digest_for "$TARGET")"
WORK="$(mktemp -d "${TMPDIR:-/tmp}/strata-node.XXXXXX")" # GNU mktemp needs the X template; BSD accepts it too
trap 'rm -rf "$WORK"' EXIT
echo "fetch-node: $URL"
curl -fsSL -o "$WORK/$ARCHIVE" "$URL"
ACTUAL="$(shasum -a 256 "$WORK/$ARCHIVE" 2>/dev/null | cut -d' ' -f1 || sha256sum "$WORK/$ARCHIVE" | cut -d' ' -f1)"
if [ -n "$EXPECTED" ]; then
if [ "$ACTUAL" != "$EXPECTED" ]; then
echo "fetch-node: REFUSING — $ARCHIVE hashes $ACTUAL, expected $EXPECTED" >&2
exit 1
fi
echo "fetch-node: sha256 verified against the recorded digest"
else
# Without a recorded digest the download cannot be authenticated. Say so loudly and keep the
# computed value so it can be recorded, rather than implying a verification that did not happen.
echo "fetch-node: WARNING — no recorded digest for $TARGET; this download is NOT authenticated." >&2
echo "fetch-node: computed sha256 $ACTUAL — record it in $(basename "${BASH_SOURCE[0]}") to enable verification." >&2
fi
mkdir -p "$DEST"
case "$ARCHIVE" in
*.zip) unzip -q -j "$WORK/$ARCHIVE" "node-v$NODE_VERSION-win-x64/node.exe" -d "$DEST" ;;
*) tar xzf "$WORK/$ARCHIVE" -C "$WORK"
cp "$WORK/node-v$NODE_VERSION-$TARGET/bin/node" "$DEST/node" ;;
esac
chmod +x "$DEST/$EXE"
echo "fetch-node: installed $("$DEST/$EXE" --version) at $DEST/$EXE"
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env node
/**
* The first run of the installed app, headless: the host as the Tauri shell starts it (STRATA_PACKAGED=1, STRATA_NO_ACQUIRE=1,
* the staged Control, nothing installed, nothing in the per-user folders) and Chrome on its onboarding.
*
* node apps/desktop/scripts/first-run-ui.mjs [--control <staged control>] [--node <node.exe>] [--work <scratch>] [--shots <dir>]
*
* Checks, one line each (exit 1 if any fails):
* the host answers; /api/native has a real catalog (families with sizes), a recommendation and the GPUs of this PC
* (from Control's bundled installer on Control's Python); the page shows "Let's get started", and after it the model
* list with sizes (no "couldn't show this screen"); no page error; the install plan for the recommended model is a
* plan (nothing is downloaded or started: a plan changes nothing); the spy sees no pip/git/compiler/engine fetch.
* Per-user folders (STRATA_HOME, APPDATA, LOCALAPPDATA, the components and install folders) are all under --work.
* No model is downloaded and no engine runs.
*/
import { spawn } from 'node:child_process';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') ? true : all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const repoHost = resolve(here, '../../host');
const staged = resolve(here, '../src-tauri/resources');
const control = resolve(args.control ?? join(staged, 'control'));
const nodeExe = resolve(args.node ?? join(staged, 'node', process.platform === 'win32' ? 'node.exe' : 'node'));
const work = resolve(args.work ?? mkdtempSync(join(tmpdir(), 'strata-first-run-')));
const shots = resolve(args.shots ?? join(work, 'shots'));
mkdirSync(work, { recursive: true }); mkdirSync(shots, { recursive: true });
const spy = pathToFileURL(join(repoHost, 'tests/helpers/acquire-spy.mjs')).href;
const spyLog = join(work, 'spy.log'); writeFileSync(spyLog, '');
const { chromium } = await import(pathToFileURL(join(repoHost, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome', '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find(existsSync);
let failed = false;
const check = (ok, label, detail = '') => { console.log(`${ok ? 'ok ' : 'FAIL'} ${label}${detail ? ' ' + detail : ''}`); if (!ok) failed = true; };
const delay = ms => new Promise(r => setTimeout(r, ms));
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
for (const need of [nodeExe, join(control, 'strata.py'), join(repoHost, 'dist', 'index.html')]) if (!existsSync(need)) { console.error(`missing ${need} (run stage-control.py, fetch-node and "npm run build" in apps/host first)`); process.exit(2); }
const dirs = { home: join(work, 'home'), appdata: join(work, 'appdata'), local: join(work, 'local'), components: join(work, 'local', 'Strata', 'components') };
for (const d of Object.values(dirs)) mkdirSync(d, { recursive: true });
const port = await freePort();
const env = {
...process.env,
STRATA_PORT: String(port), STRATA_DIST: join(repoHost, 'dist'), STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_CONTROL_DIR: control, STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models'), STRATA_COMPONENTS_DIR: dirs.components,
APPDATA: dirs.appdata, LOCALAPPDATA: dirs.local, XDG_CONFIG_HOME: join(work, 'xdg'), XDG_DATA_HOME: join(work, 'xdgdata'),
STRATA_PROFILE_DIR: join(work, 'profiles'), STRATA_RUNTIME_DIR: join(work, 'runtime'), ACQUIRE_SPY_LOG: spyLog,
};
delete env.STRATA_INSTALL_DIR;
mkdirSync(env.STRATA_MODELS_DIR, { recursive: true });
const host = spawn(nodeExe, [...(args['no-spy'] === undefined ? ['--import', spy] : []), join(repoHost, 'server', 'index.mjs')], { cwd: repoHost, env, stdio: ['ignore', 'pipe', 'pipe'] });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const base = `http://127.0.0.1:${port}`;
const api = async (path, init) => { const r = await fetch(base + path, init); return { status: r.status, body: await r.json().catch(() => null) }; };
let browser;
try {
let up = false;
for (let i = 0; i < 150 && !up; i++) { try { up = (await api('/api/health')).status === 200; } catch { await delay(200); } }
check(up, 'the packaged host starts', `port ${port}`);
if (!up) throw new Error(hostLog.slice(-2000));
const native = await api('/api/native?refresh=1');
const state = native.body;
check(native.status === 200 && state?.available === true, 'the host reads Control', state?.error ?? '');
const sizes = (state?.catalog?.families ?? []).reduce((n, f) => n + (f.sizes?.length ?? 0), 0);
check((state?.catalog?.families?.length ?? 0) > 0 && sizes >= 3, 'the catalog has real families and sizes', `${state?.catalog?.families?.length} families, ${sizes} sizes`);
check(!!state?.catalog?.recommendation, 'the catalog has a recommendation', JSON.stringify(state?.catalog?.recommendation ?? null).slice(0, 120));
check(Array.isArray(state?.hardware?.gpus), 'the hardware lists GPUs', (state?.hardware?.gpus ?? []).map(g => `${g.name} ${g.vram_gb} GB`).join('; ') || 'none on this PC');
check(state?.installDir && realpathSync.native(state.installDir).toLowerCase().startsWith(realpathSync.native(work).toLowerCase()), 'models go to a writable per-user folder, not the bundle', state?.installDir);
const hw = (await api('/api/hardware')).body;
console.log('info host hardware probe:', JSON.stringify((hw?.gpus ?? []).map(g => ({ name: g.name, backend: g.backend, vramGb: g.vramGb, unified: g.unified }))));
check(!state?.installerError, 'the bundled installer answers', state?.installerError ?? '');
browser = await chromium.launch({ executablePath: CHROME, headless: true });
const page = await browser.newPage({ viewport: { width: 1400, height: 950 } });
const problems = [];
page.on('pageerror', e => problems.push(`pageerror: ${e.message}`));
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('text=/Let.s get started|Choose a version|Connect your Strata/i', { timeout: 30000 }).catch(() => {});
await page.screenshot({ path: join(shots, '1-welcome.png') });
const broken = await page.locator('text=/couldn.t show this screen/i').count();
check(broken === 0, 'the first screen renders (not "Strata couldn\'t show this screen")');
const start = page.getByRole('button', { name: /get started/i });
check(await start.count() > 0, 'the welcome step offers "Let\'s get started"');
if (await start.count()) await start.first().click();
await page.waitForSelector('text=/Choose a version/i', { timeout: 30000 }).catch(() => {});
await delay(1200);
await page.screenshot({ path: join(shots, '2-models.png') });
check(await page.locator('text=/couldn.t show this screen/i').count() === 0, 'the model step renders');
check(await page.getByText(/Choose a version/i).count() > 0, 'the model step lists the versions');
const labels = await page.locator('[role=radio]').allInnerTexts();
check(labels.length >= 3, 'the versions are the real catalog\'s sizes', labels.map(l => l.replace(/\s+/g, ' ').slice(0, 40)).join(' | '));
check(labels.some(l => /\d+\.\d\s*GB/.test(l)), 'each version shows its download size');
check(problems.length === 0, 'no page error', problems.join('; '));
const recommended = state?.catalog?.recommendation;
if (recommended?.model) {
const plan = await api('/api/native/plan', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ family: recommended.family, model: recommended.model, context: recommended.context, data_dir: join(work, 'Strata-data') }) });
check(plan.status === 200 && !!plan.body?.plan, 'the install plan for the recommended model is a plan (nothing downloaded)', plan.body?.summary?.slice(0, 100) ?? JSON.stringify(plan.body).slice(0, 150));
check(!existsSync(join(work, 'Strata-data', 'models')), 'a plan wrote no model files');
}
} catch (error) {
check(false, 'the run', String(error?.message ?? error).slice(0, 600));
} finally {
await browser?.close().catch(() => {});
host.kill();
await new Promise(r => host.once('exit', r)); // only the PID this script started
const attempts = readFileSync(spyLog, 'utf8').split('\n').filter(Boolean);
check(attempts.length === 0, 'the spy saw no pip, git, compiler, engine or internet access', attempts.slice(0, 3).join(' '));
console.log(`shots in ${shots}`);
if (!args.work) rmSync(work, { recursive: true, force: true, maxRetries: 3 });
}
process.exit(failed ? 1 : 0);
+133
View File
@@ -0,0 +1,133 @@
/**
* Turn a pnpm "isolated" node_modules (a virtual store in `.pnpm` full of links) into an ordinary hoisted tree of real folders.
*
* Why: the installer copies the Harness folder file by file. pnpm's layout is thousands of links (junctions on Windows); a copy
* that follows them duplicates every package once per dependent and one that does not breaks on the first dangling link. A hoisted
* tree has each package once (nested only where two versions of the same name are needed), no links, and Node resolves it
* exactly as it resolves any npm install.
*
* Algorithm (the one npm uses, in short): the packages the root names come first, at the top. Then, walking each placed package's
* own dependencies: look the name up from the package's own folder upwards; if the same instance is already visible, nothing to
* do; if a DIFFERENT instance of that name is visible, nest this one inside the dependent; if none is visible, hoist it to the top.
*
* @param {string} out folder holding `node_modules`
* @param {{skip?: RegExp}} [options] package names never copied (the caller removes them anyway)
* @returns {{packages:number, copied:number, nested:number}}
*/
import { cpSync, existsSync, lstatSync, mkdirSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync } from 'node:fs';
import { dirname, join, relative, sep } from 'node:path';
export function flattenPnpm(out, { skip = null } = {}) {
const nm = join(out, 'node_modules');
const store = join(nm, '.pnpm');
const flatRoot = join(out, 'node_modules.flat');
rmSync(flatRoot, { recursive: true, force: true });
mkdirSync(join(flatRoot, 'node_modules'), { recursive: true });
const manifest = dir => { try { return JSON.parse(readFileSync(join(dir, 'package.json'), 'utf8')); } catch { return null; } };
const nameOf = dir => manifest(dir)?.name ?? null;
/**
* What makes two instances "the same package": name and version. pnpm keeps one copy per set of peer packages (the same workspace
* package under many hashed folders); in one hoisted tree they are one package.
*/
const identity = dir => { const m = manifest(dir); return m ? `${m.name}@${m.version}` : dir; };
/** The sibling links of a store instance: its dependencies, by name -> real instance folder. */
function storeDeps(instance, name) {
const container = name.startsWith('@') ? dirname(dirname(instance)) : dirname(instance);
const found = new Map();
for (const e of readdirSync(container, { withFileTypes: true })) {
if (e.name === '.bin') continue;
const entries = e.name.startsWith('@') ? readdirSync(join(container, e.name), { withFileTypes: true }).map(x => [`${e.name}/${x.name}`, join(container, e.name, x.name)]) : [[e.name, join(container, e.name)]];
for (const [dep, p] of entries) {
if (dep === name) continue;
try { if (lstatSync(p).isSymbolicLink() || existsSync(p)) found.set(dep, realpathSync(p)); } catch { /* a dangling link: the package was removed on purpose */ }
}
}
return found;
}
/** Every instance in the store by package name, for packages that arrived without a store entry (copied peers). */
const byName = new Map();
if (existsSync(store)) {
for (const id of readdirSync(store, { withFileTypes: true })) {
if (!id.isDirectory()) continue;
const container = join(store, id.name, 'node_modules');
if (!existsSync(container)) continue;
for (const e of readdirSync(container, { withFileTypes: true })) {
if (e.name === '.bin') continue;
const dirs = e.name.startsWith('@') ? readdirSync(join(container, e.name)).map(x => join(container, e.name, x)) : [join(container, e.name)];
for (const d of dirs) {
let real; try { real = realpathSync(d); } catch { continue; }
if (!real.startsWith(store)) continue; // a link to a workspace package outside the store is handled through the instance table
const n = nameOf(real);
if (n && !byName.has(n)) byName.set(n, real);
}
}
}
}
const placed = new Map(); // folder (in the flat tree) -> real instance folder it holds
let copied = 0, nested = 0;
const visibleFrom = (chain, name) => {
for (const dir of chain) {
const slot = join(dir, 'node_modules', ...name.split('/'));
if (placed.has(slot)) return { slot, instance: placed.get(slot) };
}
return null;
};
const copyInto = (slot, instance) => {
mkdirSync(dirname(slot), { recursive: true });
cpSync(instance, slot, { recursive: true, dereference: true, filter: f => !relative(instance, f).split(sep).includes('node_modules') });
placed.set(slot, identity(instance));
copied++;
};
const queue = []; // { slotDir (folder of the placed package), chain (folders searched upwards), instance, name }
const rootChain = [flatRoot];
const place = (name, instance, chain, forceHere) => {
if (skip?.test(name)) return;
const seen = visibleFrom(chain, name);
if (seen && seen.instance === identity(instance)) return;
const hoist = !seen && !forceHere;
const base = hoist ? flatRoot : chain[0];
const slot = join(base, 'node_modules', ...name.split('/'));
if (placed.has(slot)) return; // another instance already sits where this one would go (it nests below, handled by the caller's chain)
copyInto(slot, instance);
if (!hoist && base !== flatRoot) nested++;
queue.push({ slotDir: slot, chain: [slot, ...(hoist ? rootChain : chain)], instance, name });
};
// 1. the packages the root names
const rootEntries = [];
for (const e of readdirSync(nm, { withFileTypes: true })) {
if (e.name === '.pnpm' || e.name === '.bin' || e.name.startsWith('.')) continue;
const entries = e.name.startsWith('@') ? readdirSync(join(nm, e.name)).map(x => [`${e.name}/${x}`, join(nm, e.name, x)]) : [[e.name, join(nm, e.name)]];
for (const [name, p] of entries) rootEntries.push([name, p]);
}
for (const [name, p] of rootEntries) {
let real; try { real = realpathSync(p); } catch { continue; }
place(name, real, rootChain, true);
}
// 2. their dependencies, breadth first
while (queue.length) {
const { slotDir, chain, instance, name } = queue.shift();
let wants;
if (instance.startsWith(store)) wants = storeDeps(instance, name);
else {
// a package copied in without a store entry: its declared requirements, taken from the store by name when not visible yet
const m = manifest(instance) ?? {};
wants = new Map();
for (const dep of [...Object.keys(m.dependencies ?? {}), ...Object.keys(m.peerDependencies ?? {}), ...Object.keys(m.optionalDependencies ?? {})]) {
if (visibleFrom(chain, dep)) continue;
const real = byName.get(dep);
if (real) wants.set(dep, real);
}
}
for (const [dep, real] of wants) place(dep, real, chain, false);
}
rmSync(nm, { recursive: true, force: true });
renameSync(join(flatRoot, 'node_modules'), nm);
rmSync(flatRoot, { recursive: true, force: true });
return { packages: placed.size, copied, nested };
}
+52
View File
@@ -0,0 +1,52 @@
/** Strata's output-budget correction for the staged pi-ai copy. */
import { createHash } from 'node:crypto';
import { lstatSync, readFileSync, realpathSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
export const HARNESS_BUDGET_PATCH = Object.freeze({
id: 'strata-pi-ai-context-safety-v2',
package: '@earendil-works/pi-ai',
version: '1.1.0',
file: 'node_modules/@earendil-works/pi-ai/dist/api/simple-options.js',
beforeSha256: '9b3cc0ae6daa86fdc050bc0cb0f3f986401e64ccddbde6757183c9f8ad411be2',
});
const sha256 = bytes => createHash('sha256').update(bytes).digest('hex');
const oldClamp = 'const available = model.contextWindow - estimateContextTokens(context).tokens - CONTEXT_SAFETY_TOKENS;';
// Small windows also need room after tool results; a fixed margin must not
// consume all remaining estimated space. Keep the old 64K+ calculation.
const newClamp = `const remaining = model.contextWindow - estimateContextTokens(context).tokens;
const safety = model.contextWindow >= 65536 ? CONTEXT_SAFETY_TOKENS
: Math.min(CONTEXT_SAFETY_TOKENS, Math.floor(model.contextWindow / 16), Math.max(0, Math.floor(remaining / 2)));
const available = remaining - safety;`;
/**
* Patch only a flattened, disposable staged Harness. Returns the BUNDLE.json
* provenance entry; the caller records it when writing the bundle manifest.
* A repeat application or any unknown package/source fails before writing.
*/
export function patchStagedHarnessBudget(stagedRoot) {
const fail = message => { throw new Error(`Harness output-budget patch: ${message}`); };
const root = realpathSync(resolve(stagedRoot));
const packageDir = join(root, 'node_modules', '@earendil-works', 'pi-ai');
const target = join(root, HARNESS_BUDGET_PATCH.file);
let manifest, before;
try {
manifest = JSON.parse(readFileSync(join(packageDir, 'package.json'), 'utf8'));
// Flattening must have removed links; never write through a shared source file.
if (realpathSync(target) !== target || !lstatSync(target).isFile() || lstatSync(target).nlink !== 1)
fail('expected an unlinked staged module');
before = readFileSync(target);
} catch (error) {
fail(`cannot read a standalone staged pi-ai dependency (${error.message})`);
}
if (manifest?.name !== HARNESS_BUDGET_PATCH.package || manifest?.version !== HARNESS_BUDGET_PATCH.version)
fail('expected @earendil-works/pi-ai 1.1.0; review the patch before staging a different dependency');
if (sha256(before) !== HARNESS_BUDGET_PATCH.beforeSha256)
fail('unexpected simple-options.js SHA-256; review the dependency source before staging');
const source = before.toString('utf8');
if (source.split(oldClamp).length !== 2) fail('expected exactly one known context clamp');
const after = Buffer.from(source.replace(oldClamp, newClamp));
writeFileSync(target, after);
return { ...HARNESS_BUDGET_PATCH, afterSha256: sha256(after) };
}
+79
View File
@@ -0,0 +1,79 @@
/**
* Which files and folders of the staged Harness are dropped, and the check that nothing the Harness imports is gone.
*
* The 2026-10-17 tester report: on Linux and Mac every agent chat failed ("session/create ... waiting for workflowEngine").
* Cause: the old rule "drop any folder named after another operating system" also removed the Harness's own JavaScript package
* `@deepseek-ai/dsh-win32-process`. It is plain JS (no binary), and `dsh-subprocess-local` and `dsh-sandbox-local` import it by
* name on every platform, so those two plugins failed to load, and everything waiting on them (shell, subprocess, ptcRuntime,
* workflowEngine) never started. Windows kept it (the rule differs there), so the Windows tests passed.
*/
import { existsSync, readFileSync, readdirSync } from 'node:fs';
import { join, sep } from 'node:path';
export const OTHER_OS = {
win32: /(^|[\\/._-])(darwin|linux|android|freebsd|musl)([\\/._-]|$)/i,
darwin: /(^|[\\/._-])(win32|linux|android|freebsd|musl)([\\/._-]|$)/i,
// Linux: the AppImage bundler (linuxdeploy) resolves the libraries of every ELF file under resources and stops on one it
// cannot satisfy (koffi's musl build wants libc.musl). The app is glibc x86_64, so musl and other-CPU builds are dropped.
linux: /(^|[\\/._-])(win32|darwin|android|freebsd|musl|arm64|aarch64|armv7|arm|ia32|riscv64|ppc64|s390x|loong64)([\\/._-]|$)/i,
};
const NATIVE_FOLDER = /prebuild|bin|native|platform|-(win32|darwin|linux)/i;
/** The Harness's own JavaScript packages (`@deepseek-ai/dsh-*`) are code every platform imports by name; never drop one. */
export function isHarnessCodePackage(path) {
const parts = path.split(sep);
const i = parts.lastIndexOf('@deepseek-ai');
return i >= 0 && i === parts.length - 2 && parts[i + 1].startsWith('dsh-');
}
/** True when the folder `path` (a directory named `name`) is another operating system's prebuilt binary and can go. */
export function isOtherOsBinaryFolder(platform, name, path) {
const rule = OTHER_OS[platform] ?? OTHER_OS.linux;
if (isHarnessCodePackage(path)) return false;
return rule.test(name) && NATIVE_FOLDER.test(path);
}
/** Imports the Windows copy (which works) is also without: browser-side or dev-only bundles, and the document converter the app drops on purpose. */
const KNOWN_ABSENT = new Set([
'@deepseek-ai/libreoffice-kit', '@deepseek-ai/dsh-client-ui-dockkit', '@deepseek-ai/dsh-client-web',
'@deepseek-ai/dsh-experimental-webworker-runtime', '@deepseek-ai/dsh-remote-mock', '@deepseek-ai/dsh-client-test-runtime',
]);
function jsFiles(dir, out = []) {
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isSymbolicLink()) continue;
if (e.isDirectory()) jsFiles(p, out);
else if (/\.(m?js|cjs)$/.test(e.name)) out.push(p);
}
return out;
}
/**
* Every bare `@deepseek-ai/<package>` the staged Harness's own packages import (static `from "x"` and `import("x")`) must exist
* in the staged node_modules. Returns the list of `{from, missing}` problems (empty = good). Plugins that are only loaded on
* another platform still count: Node resolves a static import whether or not the code runs.
*/
export function missingHarnessImports(harnessDir) {
const scope = join(harnessDir, 'node_modules', '@deepseek-ai');
if (!existsSync(scope)) return [{ from: scope, missing: '@deepseek-ai (no installed packages)' }];
const problems = [];
const seen = new Set();
const re = /(?:from|import)\s*\(?\s*["'](@deepseek-ai\/[^"'/]+)(?:\/[^"']*)?["']/g;
for (const pkg of readdirSync(scope)) {
const lib = join(scope, pkg, 'lib');
if (!existsSync(lib)) continue;
for (const file of jsFiles(lib)) {
const text = readFileSync(file, 'utf8');
for (const m of text.matchAll(re)) {
const name = m[1];
const key = `${pkg}>${name}`;
if (seen.has(key) || KNOWN_ABSENT.has(name)) continue;
seen.add(key);
if (!existsSync(join(harnessDir, 'node_modules', ...name.split('/')))
&& !existsSync(join(scope, pkg, 'node_modules', ...name.split('/')))) problems.push({ from: `${pkg}`, missing: name });
}
}
}
return problems;
}
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env node
/**
* Prove the INSTALLED app layout can start and stop an engine with nothing acquired, using a stub engine.
*
* node apps/desktop/scripts/installed-flow.mjs --install <folder the installer wrote> [--work <scratch>] [--spy <acquire-spy.mjs>]
*
* It starts the installed host with the installed node.exe exactly as the Tauri shell does (STRATA_PACKAGED=1,
* STRATA_NO_ACQUIRE=1; Control found beside the host), against a scratch component folder holding a fake engine
* package and a scratch install folder holding one run config. Everything per-user (profiles, runtime state, host data,
* components) lives under --work, so the real profile of whoever runs this is never read or written.
*
* Steps (each one printed, the run exits 1 if any fails):
* host health, Control status (engine usable, acquiring forbidden), models from Control, load through the host
* (profile + `runtime start` under the owner runtime), a chat answer from the stub engine, a second client attaches to
* the same owner/instance, unload stops it, no engine for this PC is a notice with nothing started, the Tauri-style
* clean shutdown stops the engines the host started, a hard-killed host takes its engine with it.
* With --spy the host runs under tests/helpers/acquire-spy.mjs and the run fails on any attempt to reach the internet or
* run an installer.
*
* REAL mode (a test box with a GPU): --components <folder holding engine/<package>> --models <folder with strata-<id>.json
* whose engine is "component:strata-engine"> --model <id> [--backend cuda]. The same host and Control, a real engine and a
* real model: load, a real answer, ownership, unload, the missing-engine notice, load again, clean shutdown. The stub-only
* steps (the marker file, the hard-kill report) are left out.
*/
import { spawn, spawnSync } from 'node:child_process';
import { createHash, randomBytes } from 'node:crypto';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, writeFileSync, chmodSync } from 'node:fs';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
if (!args.install) { console.error('usage: installed-flow.mjs --install <folder> [--work <dir>] [--spy <acquire-spy.mjs>]'); process.exit(2); }
const install = resolve(args.install);
const isWin = process.platform === 'win32';
const node = join(install, 'node', isWin ? 'node.exe' : 'node');
const hostEntry = join(install, 'host', 'server', 'index.mjs');
const controlDir = join(install, 'control');
const python = isWin ? join(controlDir, 'python', 'python.exe') : join(controlDir, 'python', 'bin', 'python3');
const real = !!args.components;
const modelId = String(args.model ?? 'stub');
const work = resolve(args.work ?? mkdtempSync(join(tmpdir(), 'strata-installed-flow-')));
mkdirSync(work, { recursive: true });
const dirs = { components: real ? resolve(args.components) : join(work, 'components'), models: real ? resolve(args.models) : join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home') };
// the runtime and profile folders are created by Control itself, with the owner-only ACL it insists on
for (const [name, d] of Object.entries(dirs)) if (!['runtime', 'profiles'].includes(name) && !(real && ['components', 'models'].includes(name))) mkdirSync(d, { recursive: true });
const steps = [];
const record = (name, ok, detail = '') => { steps.push({ name, ok: !!ok, detail: String(detail) }); console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + detail : ''}`); };
const delay = ms => new Promise(r => setTimeout(r, ms));
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const sha = buf => createHash('sha256').update(buf).digest('hex');
// ------------------------------------------------------------------------------ fixtures
const baseEnv = {
...process.env,
STRATA_COMPONENTS_DIR: dirs.components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime,
STRATA_INSTALL_DIR: dirs.models, STRATA_ENGINE_BACKEND: String(args.backend ?? 'cuda'), STRATA_HOME: dirs.home, STRATA_MODELS_DIR: real ? join(work, 'models-unused') : dirs.models,
APPDATA: join(work, 'appdata'), XDG_CONFIG_HOME: join(work, 'xdg'), PYTHONDONTWRITEBYTECODE: '1',
};
delete baseEnv.STRATA_API_KEY;
mkdirSync(baseEnv.APPDATA, { recursive: true });
const mark = join(work, 'engine-ran.json');
const FAKE_ENGINE = `import os, sys, json
from pathlib import Path
Path(os.environ['FAKE_ENGINE_MARK']).write_text(json.dumps({'pid': os.getpid(), 'script': os.path.abspath(__file__), 'argv': sys.argv[1:]}))
print('READY 4096 stop', flush=True)
for line in sys.stdin:
if line.strip() == 'QUIT': break
if line.startswith(('GEN ', 'BGEN ')):
slot = int(line.split()[1]) if line.startswith('BGEN ') else None
print('T 97', flush=True)
print('DONE 1 1 1.0 1.0 length 0 0 0', flush=True)
if slot is not None: print(f'BADM {slot} 0', flush=True)
`;
function installFakeEngine() {
const platform = isWin ? 'windows-x86_64' : 'linux-x86_64';
const folder = join(dirs.components, 'engine', `0.1.45-${platform}-cuda`);
mkdirSync(folder, { recursive: true });
writeFileSync(join(folder, 'engine_main.py'), FAKE_ENGINE);
const entry = isWin ? 'engine.cmd' : 'engine';
writeFileSync(join(folder, entry), isWin
? `@echo off\r\n"${python}" "%~dp0engine_main.py" %*\r\n`
: `#!/bin/sh\nexec "${python}" "$(dirname "$0")/engine_main.py" "$@"\n`);
if (!isWin) chmodSync(join(folder, entry), 0o755);
const files = [entry, 'engine_main.py'].map(p => { const b = readFileSync(join(folder, p)); return { path: p, sha256: sha(b), bytes: b.length }; });
writeFileSync(join(folder, 'engine-component.json'), JSON.stringify({ schema_version: 1, id: 'strata-engine', version: '0.1.45', platform, backend: 'cuda', entrypoint: entry, files }));
return folder;
}
function makeTokenizerAndConfig() {
const tok = join(dirs.models, 'tokenizer');
mkdirSync(tok, { recursive: true });
const script = `import json, sys\nfrom tools.strata_tokenizer import BYTE_TO_UNICODE\ntok = sys.argv[1]\ntokens = [BYTE_TO_UNICODE[i] for i in range(256)] + ['ab', '<|im_start|>', '<|im_end|>', '<|endoftext|>']\njson.dump({t: i for i, t in enumerate(tokens)}, open(tok + '/vocab.json', 'w'))\nopen(tok + '/merges.txt', 'w').write('a b')\njson.dump([0] * 257 + [3, 3, 3], open(tok + '/token_type.json', 'w'))\n`;
const done = spawnSync(python, ['-c', script, tok], { cwd: controlDir, env: { ...baseEnv, PYTHONNOUSERSITE: '1' }, encoding: 'utf8' });
if (done.status !== 0) throw new Error('tokenizer fixture failed: ' + done.stderr);
writeFileSync(join(dirs.models, 'strata-stub.json'), JSON.stringify({ exe: 'component:strata-engine', backend: 'cuda', tokenizer: tok, args: [], cwd: work, model_name: 'stub', host: '127.0.0.1', port: 0 }));
}
const control = (...argv) => spawnSync(python, [join(controlDir, 'strata.py'), ...argv, '--output', 'json'], { cwd: controlDir, env: { ...baseEnv, PYTHONNOUSERSITE: '1', STRATA_NO_ACQUIRE: '1', FAKE_ENGINE_MARK: mark }, encoding: 'utf8', timeout: 120000 });
const controlJson = (...argv) => { const r = control(...argv); let json = null; try { json = JSON.parse(r.stdout.slice(r.stdout.indexOf('{'))); } catch { /* not json */ } return { code: r.status, json, text: r.stdout + r.stderr }; };
const alive = pid => { try { process.kill(pid, 0); return true; } catch { return false; } };
// ------------------------------------------------------------------------------ the host
let host = null, port = 0, base = '', spyLog = null, shutdownToken = randomBytes(32).toString('hex');
async function startHost() {
port = await freePort(); base = `http://127.0.0.1:${port}`;
const hostArgs = [...(args.spy ? ['--import', pathToFileURL(resolve(args.spy)).href] : []), hostEntry];
if (args.spy) { spyLog = join(work, 'acquire-spy.log'); writeFileSync(spyLog, ''); }
host = spawn(node, hostArgs, {
cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'], windowsHide: true,
env: { ...baseEnv, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1',
STRATA_NO_ACQUIRE: '1', STRATA_SHUTDOWN_TOKEN: shutdownToken, FAKE_ENGINE_MARK: mark, STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()),
...(spyLog ? { ACQUIRE_SPY_LOG: spyLog } : {}) },
});
let log = '';
host.stdout.on('data', c => { log += c; }); host.stderr.on('data', c => { log += c; });
host.hostLog = () => log;
for (let i = 0; i < 200; i++) { try { if ((await fetch(base + '/api/health')).ok) return true; } catch { /* not yet */ } await delay(150); }
return false;
}
/** The host started the way the Tauri shell starts it, by a stand-in shell (a node process) that can then be killed hard. */
async function startHostViaShell() {
port = await freePort(); base = `http://127.0.0.1:${port}`;
const spec = { command: node, args: [hostEntry], options: {
cwd: join(install, 'host'), stdio: 'ignore', windowsHide: true,
env: { ...baseEnv, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1',
STRATA_NO_ACQUIRE: '1', STRATA_SHUTDOWN_TOKEN: shutdownToken, STRATA_EXIT_WITH_PARENT: '1', FAKE_ENGINE_MARK: mark,
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } } };
const shell = spawn(node, ['-e', "const s=JSON.parse(process.env.HOST_SPEC);const c=require('node:child_process').spawn(s.command,s.args,s.options);console.log(c.pid);setInterval(()=>{},1000);"],
{ env: { ...process.env, HOST_SPEC: JSON.stringify(spec) }, stdio: ['ignore', 'pipe', 'inherit'], windowsHide: true });
const hostPid = await new Promise(r => shell.stdout.once('data', c => r(Number(String(c).trim()))));
for (let i = 0; i < 200; i++) { try { if ((await fetch(base + '/api/health')).ok) return { shell, hostPid }; } catch { /* not yet */ } await delay(150); }
return { shell, hostPid, failed: true };
}
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
async function waitJob(id, seconds = 120) {
for (let i = 0; i < seconds * 5; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.state !== 'running') return { job, state };
await delay(200);
}
throw new Error('operation did not finish');
}
async function stopHost(hard = false) {
if (!host || host.exitCode !== null) return;
if (hard) { if (isWin) spawnSync('taskkill', ['/PID', String(host.pid), '/T', '/F'], { windowsHide: true }); else host.kill('SIGKILL'); }
else host.kill();
for (let i = 0; i < 100 && host.exitCode === null; i++) await delay(100);
}
async function main() {
for (const need of [node, hostEntry, python, join(controlDir, 'strata.py')]) if (!existsSync(need)) { record('installed layout', false, `missing ${need}`); return; }
record('installed layout', true, install);
if (real) {
const packages = existsSync(join(dirs.components, 'engine')) ? readdirSync(join(dirs.components, 'engine')) : [];
record('fixtures', packages.length > 0 && existsSync(join(dirs.models, `strata-${modelId}.json`)), `engine package(s) ${packages.join(', ')} in ${dirs.components}; run config strata-${modelId}.json in ${dirs.models}`);
} else {
installFakeEngine();
makeTokenizerAndConfig();
record('fixtures', true, `stub engine package in ${dirs.components}; run config strata-stub.json`);
}
if (!(await startHost())) { record('host healthy', false, host.hostLog().slice(-600)); return; }
record('host healthy', true, base);
const status = (await call('/api/control/status')).body;
record('Control status: bundled, engine usable, acquiring forbidden', status.control?.python?.startsWith(controlDir) && status.engine?.usable === true && status.acquisition?.forbidden === true,
`python=${status.control?.python}; engine=${status.engine?.engine?.version}; forbidden=${status.acquisition?.forbidden}`);
const native = (await call('/api/native?refresh=1')).body;
const model = native.installed?.models?.find(m => m.model === modelId);
record('models come from Control', native.available && model?.ready === true && native.packaged === true, `available=${native.available}; model ready=${model?.ready}; ${native.summary ?? native.error}`);
// ---- load
const started = (await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body;
const loaded = await waitJob(started.id, 400);
const instance = loaded.state.instances[0];
record('load through the host starts the engine under Control', loaded.job.state === 'done' && instance?.status === 'ready', `job=${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}`);
let ran = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]), script: listed[0].trim().split(/\s+/)[1] } : null;
record('the engine process running is the binary from the component folder', !!ran, listed[0] ? listed[0].trim().slice(0, 160) : 'no engine process under the component folder');
} else {
record('the engine that ran is the one in the component folder', ran && ran.script.toLowerCase().startsWith(dirs.components.toLowerCase()), ran?.script ?? 'engine never ran');
}
const runtime = controlJson('runtime', 'status', '--profile', 'app-1');
record('a second client sees the same owner and instance (ownership)', runtime.json?.state === 'ready' && !!runtime.json.owner_id && !!runtime.json.instance_id && runtime.json.host_pid !== host.pid,
`state=${runtime.json?.state}; owner=${String(runtime.json?.owner_id).slice(0, 8)}; instance=${String(runtime.json?.instance_id).slice(0, 8)}; owner host pid ${runtime.json?.host_pid} is not the app host (${host.pid})`);
// ---- answer
const chat = await call('/v1/chat/completions', 'POST', real
? { model: instance?.apiModelId ?? modelId, messages: [{ role: 'user', content: 'Reply with exactly one word: pong' }], max_tokens: 200, temperature: 0, reasoning_effort: 'none', stream: false }
: { model: instance?.apiModelId ?? modelId, messages: [{ role: 'user', content: 'hi' }], max_tokens: 4, stream: false });
const message = chat.body?.choices?.[0]?.message ?? {};
const answer = `${message.content ?? ''}${message.reasoning_content ?? ''}`; // the stub emits token 97 ("a"); the server files it as reasoning
record(real ? 'the real model answers through the host (the answer text is checked)' : 'the engine answers through the host (the stub token, not an error)',
chat.status === 200 && (real ? /pong/i.test(answer) : answer === 'a'), `status=${chat.status}; answer=${JSON.stringify(answer.slice(0, 200))}${chat.status === 200 ? '' : ' ' + JSON.stringify(chat.body).slice(0, 160)}`);
// ---- unload
const unloaded = await call(`/api/native/instances/${instance.id}/unload`, 'POST', {});
await delay(500);
const after = controlJson('runtime', 'status', '--profile', 'app-1');
for (let i = 0; i < 100 && ran && alive(ran.pid); i++) await delay(100); // a real engine takes a moment to release the GPU
record('unload stops the engine', unloaded.status === 200 && after.json?.state === 'stopped' && (!ran || !alive(ran.pid)), `status=${unloaded.status}; runtime=${after.json?.state}; engine pid ${ran?.pid} alive=${ran ? alive(ran.pid) : 'n/a'}`);
// ---- no engine for this PC
const hidden = join(dirs.components, real ? 'engine.hidden-by-flow' : 'engine.gone');
renameSync(join(dirs.components, 'engine'), hidden);
rmSync(mark, { force: true });
const missing = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id);
record('no engine for this PC is a notice and nothing is started', missing.job.state === 'error' && /No engine for this PC/.test(missing.job.error) && /never downloads/.test(missing.job.error) && !existsSync(mark),
`${missing.job.state}: ${String(missing.job.error).slice(0, 160)}`);
const refused = await waitJob((await call('/api/native/operations', 'POST', { kind: 'install', args: { family: 'qwen', model: 'IQ2_XS' } })).body.id);
record('installing from the app is refused (Terminal only)', refused.job.state === 'error' && /Terminal/.test(refused.job.error), refused.job.error);
const binary = await call('/api/engine/binary', 'POST');
record('the llama.cpp download route is refused', binary.status === 409 && /never downloads/.test(binary.body?.error ?? ''), `${binary.status} ${binary.body?.error}`);
// ---- clean shutdown (what the Tauri shell does on exit)
renameSync(hidden, join(dirs.components, 'engine'));
rmSync(mark, { force: true });
const again = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id, 400);
let ran2 = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran2 = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]) } : null;
}
record('load again after the engine package returns', again.job.state === 'done', again.job.error ?? '');
const wrong = await fetch(base + '/api/control/shutdown', { method: 'POST', headers: { 'x-strata-shutdown': 'nope' } });
record('shutdown without the shell\'s secret is refused', wrong.status === 403, String(wrong.status));
const exited = new Promise(r => host.once('exit', code => r(code)));
const stopping = await fetch(base + '/api/control/shutdown', { method: 'POST', headers: { 'x-strata-shutdown': shutdownToken } });
const code = await Promise.race([exited, delay(90000).then(() => 'timeout')]);
for (let i = 0; i < 100 && ran2 && alive(ran2.pid); i++) await delay(100);
const final = controlJson('runtime', 'status', '--profile', 'app-1');
record('clean shutdown stops the engine the app started and exits', stopping.status === 202 && code === 0 && final.json?.state === 'stopped' && (!ran2 || !alive(ran2.pid)), `status=${stopping.status}; host exit=${code}; runtime=${final.json?.state}; engine alive=${ran2 ? alive(ran2.pid) : 'n/a'}`);
// ---- hard kill: an engine whose app host dies ends with it (the owner follows the host: runtime start --die-with-pid)
if (!(await startHost())) { record('host restarts', false, host.hostLog().slice(-400)); return; }
await call('/api/native?refresh=1');
rmSync(mark, { force: true });
const third = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id, 400);
let ran3 = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran3 = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]) } : null;
record('(real) the engine is up before the host is killed', !!ran3, listed[0] ? listed[0].trim().slice(0, 120) : 'no engine process');
}
await stopHost(true);
for (let i = 0; i < 600 && ran3 && alive(ran3.pid); i++) await delay(100); // the owner notices within a second, drains, stops
const orphan = controlJson('runtime', 'status', '--profile', 'app-1');
record('a hard-killed app host takes its engine with it', third.job.state === 'done' && (!ran3 || !alive(ran3.pid)), `engine alive=${ran3 ? alive(ran3.pid) : 'n/a'}; runtime=${orphan.json?.state ?? orphan.json?.error?.message}`);
const cleanup = controlJson('runtime', 'stop', '--profile', 'app-1');
record('nothing is left to stop afterwards', /stopped|not managed/.test(JSON.stringify(cleanup.json)) && controlJson('runtime', 'status', '--profile', 'app-1').json?.state === 'stopped', JSON.stringify(cleanup.json).slice(0, 160));
// ---- the shell itself is killed (what the desktop shell's crash or kill looks like to the host): the host leaves, and
// its engine with it, with nothing announced. Windows also has the job object; this is the path that works everywhere.
const started2 = await startHostViaShell();
if (started2.failed) { record('host starts under a stand-in shell', false, ''); try { started2.shell.kill('SIGKILL'); } catch { /* gone */ } }
else {
await call('/api/native?refresh=1');
rmSync(mark, { force: true });
const fourth = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id, 400);
let ran4 = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran4 = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]) } : null;
}
if (isWin) spawnSync('taskkill', ['/PID', String(started2.shell.pid), '/F'], { windowsHide: true }); else started2.shell.kill('SIGKILL');
for (let i = 0; i < 100 && alive(started2.hostPid); i++) await delay(100);
const hostGone = !alive(started2.hostPid);
for (let i = 0; i < 900 && ran4 && alive(ran4.pid); i++) await delay(100);
const engineGone = !ran4 || !alive(ran4.pid);
record('a killed shell takes the host and its engine with it', fourth.job.state === 'done' && hostGone && engineGone,
`host pid ${started2.hostPid} gone=${hostGone}; engine pid ${ran4?.pid} gone=${engineGone}; runtime=${controlJson('runtime', 'status', '--profile', 'app-1').json?.state}`);
controlJson('runtime', 'stop', '--profile', 'app-1');
}
await finishSpy();
}
async function finishSpy() {
if (spyLog) {
const attempts = readFileSync(spyLog, 'utf8').split('\n').filter(Boolean);
record('no acquisition attempt by the host during the whole run (spy)', attempts.length === 0, attempts.join(' | '));
}
}
try { await main(); }
catch (error) { record('flow', false, error.stack ?? error.message); }
finally {
try { const hid = join(dirs.components, 'engine.hidden-by-flow'); if (real && existsSync(hid) && !existsSync(join(dirs.components, 'engine'))) renameSync(hid, join(dirs.components, 'engine')); } catch { /* reported by the run */ }
await stopHost(true);
controlJson('runtime', 'stop', '--profile', 'app-1');
controlJson('runtime', 'stop', '--profile', 'app-2');
}
const failed = steps.filter(s => !s.ok);
writeFileSync(join(work, 'installed-flow.json'), JSON.stringify({ ok: failed.length === 0, install, work, steps }, null, 1));
console.log(`\n${steps.length - failed.length}/${steps.length} ok; report ${join(work, 'installed-flow.json')}`);
if (!args.keep && !args.work && failed.length === 0) { try { rmSync(work, { recursive: true, force: true }); } catch { /* leave it */ } }
process.exit(failed.length ? 1 : 0);
@@ -0,0 +1,73 @@
#!/usr/bin/env node
/**
* Mock-ups of the Windows installer in Strata's own look: four screens (welcome, where, installing, done), light and dark, drawn with
* the app's built CSS (apps/host/dist) and brand files, then photographed with Chrome. Nothing here is built or shipped: it is what
* the owner is shown before an installer is chosen.
*
* node apps/desktop/scripts/installer-mockups.mjs [--out <folder>]
*/
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const host = resolve(here, '../../host');
const out = resolve(args.out ?? join(here, '../installer-mockups'));
mkdirSync(out, { recursive: true });
const dist = join(host, 'dist');
const css = readdirSync(join(dist, 'assets')).filter(f => f.endsWith('.css')).map(f => readFileSync(join(dist, 'assets', f), 'utf8')).join('\n');
const logo = theme => readFileSync(join(dist, 'brand', `strata-lockup-${theme}.svg`), 'utf8');
const { chromium } = await import(pathToFileURL(join(host, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const check = '<svg class="st-icon" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12.5l4.5 4.5L19 7.5"/></svg>';
const folder = '<svg class="st-icon" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M3 7a2 2 0 0 1 2-2h4l2 2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2z"/></svg>';
const screens = {
welcome: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="onboard-welcome-copy"><h1>State of the art. At home.</h1><p>Strata runs AI on your own computer.<br>Setting it up takes a few minutes.</p>
<div class="onboard-benefits"><span>${check}Private</span><span>${check}Fast</span><span>${check}Easy to use</span></div></div>
<footer class="onboard-footer onboard-footer--welcome"><button class="st-btn st-btn--primary">Install Strata</button><small>Installs for you only. No administrator needed.</small></footer>`,
where: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="mock-body"><h2 class="mock-title">Where should Strata go?</h2><p class="mock-sub">This is the right place for most people.</p>
<div class="onboard-folder">${folder}<div><strong>Your Strata folder</strong><small>C:\\Users\\Alex\\AppData\\Local\\Strata</small></div><button class="st-btn st-btn--sm">Change</button></div>
<p class="mock-note">About 310 MB. Your models and chats are kept somewhere else and are never removed by an update.</p></div>
<footer class="onboard-footer"><button class="st-btn st-btn--ghost">Back</button><span></span><button class="st-btn st-btn--primary">Install</button></footer>`,
installing: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="mock-body"><h2 class="mock-title">Installing Strata…</h2><p class="mock-sub">Copying the app and its engine room. You can keep working meanwhile.</p>
<div class="onboard-progress" style="--p:62%"><progress max="100" value="62"></progress></div>
<div class="onboard-progress-meta"><span>62% · 190 / 310 MB</span><div class="onboard-progress-actions"><span>About 1 min left</span><button>Cancel</button></div></div></div>
<footer class="onboard-footer"><span></span><span class="mock-hint">Next: choose your first model inside Strata.</span><button class="st-btn st-btn--primary" disabled>Installing…</button></footer>`,
done: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="onboard-welcome-copy"><h1>Strata is ready.</h1><p>Open it to pick your first model. Strata downloads it for you and gets it ready.</p></div>
<footer class="onboard-footer onboard-footer--welcome"><button class="st-btn st-btn--primary">Open Strata</button><small>A shortcut was added to your Start menu and desktop.</small></footer>`,
};
const page = (name, theme) => `<!doctype html><html data-theme="${theme}"><head><meta charset="utf-8"><style>${css}
body{background:${theme === 'dark' ? '#0b0d0e' : '#e9edef'};margin:0;display:grid;place-items:center;min-height:100vh}
.mock-window{width:720px;border-radius:14px;overflow:hidden;background:var(--st-surface,#fff);border:1px solid var(--st-line,#dde);box-shadow:0 30px 80px #0003}
.mock-titlebar{height:34px;display:flex;align-items:center;justify-content:space-between;padding:0 14px;font-size:12px;color:var(--st-ink-muted);background:var(--st-surface-2,#f3f5f6);border-bottom:1px solid var(--st-line-soft,#e6e9eb)}
.mock-logo svg{height:22px;width:auto;display:block}
.mock-body{padding:8px 36px 28px}.mock-title{font-size:28px;margin:16px 0 8px}.mock-sub{color:var(--st-ink-muted);margin:0 0 22px;line-height:1.6}
.mock-note{color:var(--st-ink-muted);font-size:12px;margin-top:16px;line-height:1.6}.mock-hint{font-size:12px;color:var(--st-ink-muted)}
.onboard-progress progress{width:100%}
.guided-onboarding{min-height:430px;display:flex;flex-direction:column}.onboard-footer{margin-top:auto}
</style></head><body><div class="mock-window"><div class="mock-titlebar"><span>Strata setup</span><span>– ☐ ✕</span></div>
<div class="setup guided-onboarding" data-step="${name === 'welcome' || name === 'done' ? 1 : 2}">${screens[name](theme)}</div></div></body></html>`;
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
const written = [];
for (const theme of ['light', 'dark']) {
for (const name of Object.keys(screens)) {
const tab = await browser.newPage({ viewport: { width: 860, height: 620 }, deviceScaleFactor: 1 });
await tab.setContent(page(name, theme));
await tab.waitForTimeout(250);
const file = join(out, `${name}-${theme}.png`);
await tab.screenshot({ path: file });
written.push(file);
await tab.close();
}
}
await browser.close();
console.log(written.join('\n'));
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env node
/**
* "Use your own GGUF" end to end on a box, through the installed app layout (packaged host + bundled Control + a real engine component):
* preview a folder of a supported model, preview an unsupported GGUF (must be refused in plain terms), prepare the supported one
* (pack, from the folder, nothing copied), load it, get a chat answer, unload, shut down, and check nothing is left running.
*
* node local-gguf-e2e.mjs --root /data/ui4-e2e --good /data/141-gyro/models/IQ2_XS --bad /data/ui4-e2e/bad [--context 8192] [--backend cuda|metal]
* (the app folder is <root>/app: node/, host/, control/ ...; <root>/components gets the engine, from <root>/app/engine-bundle when the app carries one)
*/
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const root = resolve(args.root);
const good = args.good, bad = args.bad;
const context = Number(args.context ?? 8192);
const app = join(root, 'app');
const dirs = { components: join(root, 'components'), install: join(root, 'install'), profiles: join(root, 'profiles'), runtime: join(root, 'runtime'), home: join(root, 'home'), xdg: join(root, 'xdg'), appdata: join(root, 'appdata') };
for (const d of [dirs.install, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
const steps = [];
let failed = false;
const record = (name, ok, detail = '') => { steps.push({ name, ok: !!ok, detail: String(detail).slice(0, 400) }); console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 300) : ''}`); if (!ok) failed = true; };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: dirs.components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.install, STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(root, 'Strata-data'), STRATA_ENGINE_BACKEND: args.backend ?? 'cuda', XDG_CONFIG_HOME: dirs.xdg, APPDATA: dirs.appdata, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(join(app, 'node', 'node'), [join(app, 'host', 'server', 'index.mjs')], { cwd: join(app, 'host'), stdio: ['ignore', 'pipe', 'pipe'], detached: false,
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(app, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1', STRATA_EXIT_WITH_PARENT: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()), STRATA_WORKSPACE: join(root, 'workspace') } });
mkdirSync(join(root, 'workspace'), { recursive: true });
let hostLog = ''; host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => { const r = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) }); return { status: r.status, body: await r.json().catch(() => null) }; };
async function waitJob(id, seconds) {
const until = Date.now() + seconds * 1000;
let last = '';
while (Date.now() < until) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.state !== 'running') return { job, state };
if (job && job.summary !== last) { last = job.summary; console.log(' ...', String(job.summary).slice(0, 140)); }
await delay(2000);
}
throw new Error('operation did not finish in time');
}
const engineProcs = () => spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(join(dirs.components, 'engine')) && /\/strata( |$)/.test(l));
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { await delay(150); } }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
let status = (await call('/api/control/status')).body;
for (let i = 0; i < 180 && status?.engineSetup; i++) { await delay(1000); status = (await call('/api/control/status')).body; } // the engine that came with the app is copied on the first start
record('Control status: bundled, engine usable', status?.engine?.usable === true, `engine=${status?.engine?.engine?.version} ${status?.engine?.notice ?? ''}`);
await call('/api/native?refresh=1');
// ---- the unsupported GGUF: a plain refusal
const badPreview = await call('/api/native/local/preview', 'POST', { path: bad });
const badIssues = badPreview.body?.preview?.issues ?? badPreview.body?.issues ?? [];
const badStatus = badPreview.body?.preview?.file_set_status;
console.log(' bad preview:', badPreview.status, JSON.stringify(badPreview.body).slice(0, 400));
record('an unsupported GGUF is refused, nothing adopted', badPreview.status === 200 ? (badStatus !== 'plausible' || !badPreview.body.preview.candidate_selection) : badPreview.status >= 400, `status=${badPreview.status}; file_set_status=${badStatus}; issues=${JSON.stringify(badIssues)}`);
// ---- the supported folder
const goodPreview = await call('/api/native/local/preview', 'POST', { path: good });
const selection = goodPreview.body?.preview?.candidate_selection;
console.log(' good preview:', JSON.stringify(goodPreview.body).slice(0, 500));
record('a folder of a supported model is recognised', goodPreview.status === 200 && goodPreview.body?.preview?.file_set_status === 'plausible' && !!selection, `family=${selection?.family}; model=${selection?.model}; files=${goodPreview.body?.preview?.file_count}`);
if (!selection || args.only === 'preview') return;
// ---- prepare (what the page's button does): install from the folder, nothing copied or downloaded
const native0 = (await call('/api/native?refresh=1')).body;
const catalogSize = native0.catalog?.families?.find(f => f.family === selection.family)?.sizes.find(s => s.model === selection.model);
record('the catalog lists the recognised version', !!catalogSize, catalogSize ? `id=${catalogSize.id}` : `no ${selection.family}/${selection.model} in the catalog`);
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: { family: selection.family, model: selection.model, gguf_dir: good, context, vision: 'no' } });
record('prepare starts', started.status < 300 && !!started.body?.id, `status=${started.status} ${JSON.stringify(started.body).slice(0, 200)}`);
if (!started.body?.id) return;
const prepared = await waitJob(started.body.id, 3600);
record('prepare finishes', prepared.job.state === 'done', `${prepared.job.state}: ${prepared.job.error ?? prepared.job.summary}`);
if (prepared.job.state !== 'done') return;
const native1 = (await call('/api/native?refresh=1')).body;
const modelId = catalogSize?.id ?? `${selection.family === 'qwen' ? '' : selection.family + '-'}${String(selection.model).toLowerCase()}`;
const entry = native1.installed?.models?.find(m => m.model === modelId);
record('the model is ready and points at the folder, not a copy', entry?.ready === true, `model=${modelId}; ready=${entry?.ready}; dataFolder=${entry?.dataFolder}`);
if (!entry?.ready) return;
// ---- load
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } });
const loaded = await waitJob(load.body.id, 1800);
const instance = loaded.state.instances[0];
record('load starts the engine from the component folder', loaded.job.state === 'done' && instance?.status === 'ready', `job=${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}`);
if (instance?.status !== 'ready') return;
record('one engine process is running', engineProcs().length === 1, engineProcs().map(l => l.trim().slice(0, 140)).join(' | '));
// ---- a chat answer, checked
const chat = await fetch(base + '/api/chat/stream', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ sessionId: 'e2e-chat', message: 'Reply with exactly one word: pong', history: [], nativeModelId: instance.id, thinking: 'Off', agent: false, workspaceId: null }) });
let text = '', done = null;
for await (const chunk of chat.body) {
for (const line of Buffer.from(chunk).toString('utf8').split('\n')) {
if (!line.trim()) continue;
try { const f = JSON.parse(line); if (f.type === 'delta') text += f.text ?? ''; if (f.type === 'done') done = f; if (f.type === 'error') text += ` [error: ${f.message}]`; } catch { /* partial line */ }
}
}
record('the model answers through the app (the answer text is checked)', chat.status === 200 && /pong/i.test(text), `status=${chat.status}; answer=${JSON.stringify(text.slice(0, 120))}; tokensOut=${done?.tokensOut}; tps=${done?.tps}`);
// ---- unload, shutdown, nothing left
const unload = await call(`/api/native/instances/${instance.id}/unload`, 'POST');
await delay(3000);
record('unload stops the engine', unload.status < 300 && engineProcs().length === 0, `status=${unload.status}; engines left=${engineProcs().length}`);
}
try { await main(); } catch (e) { record('the flow', false, e?.stack ?? e); }
host.kill();
for (let i = 0; i < 100 && host.exitCode === null; i++) await delay(100);
if (host.exitCode === null) host.kill('SIGKILL');
await delay(2000);
const left = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => (l.includes(dirs.install) || l.includes(join(app, 'host'))) && !l.includes('ps -eo'));
record('nothing of this run is left running', left.length === 0, left.join(' | ').slice(0, 300));
writeFileSync(join(root, 'e2e-result.json'), JSON.stringify({ steps, hostLogTail: hostLog.slice(-2000) }, null, 1));
console.log(failed ? 'FAILED' : 'ALL OK');
process.exit(failed ? 1 : 0);
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
# Build the Strata DMG without a Finder session.
#
# Tauri's default DMG step runs an AppleScript that asks Finder to arrange the window (icon
# positions, background). Finder AppleEvents time out in a non-GUI or sandboxed context — observed
# here as `Finder got an error: AppleEvent timed out. (-1712)` — so the default path cannot be
# relied on for an automated build. The upstream `create-dmg` script already supports exactly this
# case via `--skip-jenkins`, which produces a plain, correct DMG with the app and the Applications
# drop link and no cosmetic arrangement.
#
# Usage: scripts/make-dmg.sh [output.dmg]
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DESKTOP="$(cd "$HERE/.." && pwd)"
BUNDLE="$DESKTOP/src-tauri/target/release/bundle"
APP="$BUNDLE/macos/Strata.app"
VERSION="$(grep -m1 '^version = ' "$DESKTOP/src-tauri/Cargo.toml" | cut -d'"' -f2)" # the one place the app version is written
OUT="${1:-$BUNDLE/dmg/Strata_${VERSION}_aarch64.dmg}"
if [ ! -d "$APP" ]; then
echo "error: $APP does not exist; run 'cargo tauri build --bundles app' first" >&2
exit 1
fi
WORK="$(mktemp -d -t strata-dmg)"
trap 'rm -rf "$WORK"' EXIT
# A staging folder is what create-dmg copies into the image: the app, and nothing else.
#
# The Applications drop link is created by `--app-drop-link` below. Adding one here as well makes
# create-dmg fail with "File exists", because it links into the volume it is assembling.
mkdir -p "$WORK/stage"
cp -R "$APP" "$WORK/stage/"
# Reuse Tauri's vendored create-dmg if the bundler left it behind, otherwise take it from the
# bundle cache. Building it yourself is not necessary and not expected.
SCRIPT="$BUNDLE/dmg/bundle_dmg.sh"
if [ ! -x "$SCRIPT" ]; then
echo "error: $SCRIPT not found; run 'cargo tauri build --bundles dmg' once to vendor it" >&2
exit 1
fi
mkdir -p "$(dirname "$OUT")"
rm -f "$OUT"
# --skip-jenkins: no Finder AppleScript. Everything else is the same code path Tauri uses.
"$SCRIPT" \
--volname "Strata $VERSION" \
--volicon "$BUNDLE/macos/Strata.app/Contents/Resources/Strata.icns" \
--app-drop-link 400 200 \
--skip-jenkins \
--no-internet-enable \
"$OUT" "$WORK/stage"
echo
echo "built: $OUT"
ls -lh "$OUT" | awk '{print "size: " $5}'
shasum -a 256 "$OUT"
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env node
/**
* The app and installer icon, refined from the existing Strata mark (apps/host/public/brand/strata-mark-*.svg and the current
* icons/icon.png): the same three rounded bars on the same dark tile, in the same colours. Nothing is redrawn. What changes is
* how it is cut for small sizes: at 16-48 px the bars are snapped to whole pixels and made a little bolder and wider so they
* stay crisp instead of blurring; from 64 px up the geometry is the mark's own proportions.
*
* node apps/desktop/scripts/make-icons.mjs [--out <folder>] writes icon-<size>.png for every size and icon.ico
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const host = resolve(here, '../../host');
const out = resolve(args.out ?? join(here, '../src-tauri/icons'));
mkdirSync(out, { recursive: true });
const { chromium } = await import(pathToFileURL(join(host, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const TILE = '#161a1d', LIGHT = '#eef1f3', GREEN = '#34d399'; // the colours of the current icon.png
/** Integer geometry for the small sizes: bars on whole pixels, centred. Returns [{x,y,w,h,fill}]. */
function hinted(size) {
const bar = Math.max(2, Math.round(size * 0.14)), gap = Math.max(2, Math.round(size * 0.1)), mid = bar + 1;
const total = 2 * bar + mid + 2 * gap, top = Math.floor((size - total) / 2);
const even = w => (w % 2 === size % 2 ? w : w + 1);
const wide = even(Math.round(size * 0.5)), long = even(Math.round(size * 0.76));
return [
{ x: (size - wide) / 2, y: top, w: wide, h: bar, fill: LIGHT },
{ x: (size - long) / 2, y: top + bar + gap, w: long, h: mid, fill: GREEN },
{ x: (size - wide) / 2, y: top + bar + gap + mid + gap, w: wide, h: bar, fill: LIGHT },
];
}
/** The mark's own proportions (from icon.png at 512): bars 34% / 50% / 34% wide, 9% high, on a tile with 21.5% corners. */
function proportional(size) {
const k = size / 512;
return [
{ x: 170 * k, y: 156 * k, w: 173 * k, h: 46 * k, fill: LIGHT },
{ x: 128 * k, y: 232 * k, w: 256 * k, h: 48 * k, fill: GREEN },
{ x: 170 * k, y: 309 * k, w: 173 * k, h: 46 * k, fill: LIGHT },
];
}
function svg(size) {
const small = size <= 48;
const bars = (small ? hinted(size) : proportional(size));
const radius = small ? Math.round(size * 0.22) : size * 0.215;
const rects = bars.map(b => `<rect x="${b.x}" y="${b.y}" width="${b.w}" height="${b.h}" rx="${small ? Math.min(b.h / 2, 1.5 + size / 32) : b.h / 2}" fill="${b.fill}"${small ? ' shape-rendering="crispEdges"' : ''}/>`).join('');
return `<svg xmlns="http://www.w3.org/2000/svg" width="${size}" height="${size}" viewBox="0 0 ${size} ${size}"><rect width="${size}" height="${size}" rx="${radius}" fill="${TILE}"/>${rects}</svg>`;
}
const SIZES = [16, 20, 24, 32, 40, 48, 64, 96, 128, 256, 512, 1024];
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
const png = {};
for (const size of SIZES) {
const page = await browser.newPage({ viewport: { width: size, height: size }, deviceScaleFactor: 1 });
await page.setContent(`<!doctype html><body style="margin:0;background:transparent">${svg(size)}</body>`);
png[size] = await page.screenshot({ omitBackground: true, clip: { x: 0, y: 0, width: size, height: size } });
await page.close();
writeFileSync(join(out, `icon-${size}.png`), png[size]);
}
await browser.close();
// The .ico: one PNG image per size (16-256), the sizes Windows asks for at 100-250% scaling.
const ICO = [16, 20, 24, 32, 40, 48, 64, 96, 128, 256];
const head = Buffer.alloc(6); head.writeUInt16LE(0, 0); head.writeUInt16LE(1, 2); head.writeUInt16LE(ICO.length, 4);
let offset = 6 + 16 * ICO.length;
const entries = [], blobs = [];
for (const size of ICO) {
const data = png[size], e = Buffer.alloc(16);
e.writeUInt8(size === 256 ? 0 : size, 0); e.writeUInt8(size === 256 ? 0 : size, 1); e.writeUInt8(0, 2); e.writeUInt8(0, 3);
e.writeUInt16LE(1, 4); e.writeUInt16LE(32, 6); e.writeUInt32LE(data.length, 8); e.writeUInt32LE(offset, 12);
offset += data.length; entries.push(e); blobs.push(data);
}
writeFileSync(join(out, 'icon.ico'), Buffer.concat([head, ...entries, ...blobs]));
writeFileSync(join(out, 'icon.png'), png[512]);
console.log('icons in', out, ICO.join(','));
@@ -0,0 +1,66 @@
#!/usr/bin/env node
/**
* The installer's pictures, drawn from the app's own design: the light surface and ink colours of the app (src/styles tokens), the
* brand lockup and mark files, GT Walsheim Bold for the words (baked into the pictures, because NSIS cannot use a web font).
*
* node apps/desktop/scripts/make-installer-images.mjs [--out <folder>] [--preview <folder>]
*
* Writes sidebar.bmp (164x314, the welcome and finish pages) and header.bmp (150x57, the other pages) as 24-bit BMPs, and with
* --preview the same as PNGs at 3x.
*/
import { spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { tmpdir } from 'node:os';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const host = resolve(here, '../../host');
const out = resolve(args.out ?? join(here, '../src-tauri/nsis'));
mkdirSync(out, { recursive: true });
const { chromium } = await import(pathToFileURL(join(host, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const font = pathToFileURL(join(host, 'public/fonts/GTWalsheimPro-Bold.woff2')).href;
const fontMedium = pathToFileURL(join(host, 'public/fonts/GTWalsheimPro-Medium.woff2')).href;
const lockup = readFileSync(join(host, 'public/brand/strata-lockup-light.svg'), 'utf8');
const mark = readFileSync(join(host, 'public/brand/strata-mark-light.svg'), 'utf8');
// the app's light tokens
const SURFACE = '#f9fafb', INK = '#171b1e', MUTED = '#5b646b', ACCENT = '#10b981', LINE = '#e3e8ea';
const css = `@font-face{font-family:GT;font-weight:700;src:url(${font})}@font-face{font-family:GT;font-weight:500;src:url(${fontMedium})}
*{box-sizing:border-box}body{margin:0;font-family:GT,'Segoe UI',sans-serif;color:${INK};background:${SURFACE}}`;
const pages = {
sidebar: { w: 164, h: 314, html: `<div style="width:164px;height:314px;padding:22px 18px;display:flex;flex-direction:column;background:linear-gradient(180deg,#f9fafb 0%,#eef6f2 100%);border-right:1px solid ${LINE}">
<div style="height:26px">${lockup.replace('<svg', '<svg style="height:26px;width:auto;display:block"')}</div>
<div style="margin-top:auto">
<div style="font-weight:700;font-size:22px;line-height:1.12;letter-spacing:-.01em">State of<br>the art.<br>At home.</div>
<div style="margin-top:10px;font-weight:500;font-size:11px;line-height:1.5;color:${MUTED}">AI on your own computer.<br>Private. Fast. Easy.</div>
<div style="margin-top:16px;height:3px;width:34px;border-radius:2px;background:${ACCENT}"></div>
</div></div>` },
header: { w: 150, h: 57, html: `<div style="width:150px;height:57px;display:flex;align-items:center;justify-content:flex-end;padding-right:12px;background:${SURFACE}">${lockup.replace('<svg', '<svg style="height:22px;width:auto;display:block"')}</div>` },
};
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
const work = join(tmpdir(), 'strata-installer-images'); mkdirSync(work, { recursive: true });
for (const [name, page] of Object.entries(pages)) {
for (const scale of [1, 3]) {
const file = join(work, `${name}.html`);
writeFileSync(file, `<!doctype html><meta charset="utf-8"><style>${css}</style><body>${page.html}</body>`);
const tab = await browser.newPage({ viewport: { width: page.w, height: page.h }, deviceScaleFactor: scale });
await tab.goto(pathToFileURL(file).href);
await tab.evaluate(() => document.fonts.ready);
await tab.waitForTimeout(200);
const png = join(work, `${name}@${scale}.png`);
await tab.screenshot({ path: png, clip: { x: 0, y: 0, width: page.w, height: page.h } });
await tab.close();
if (scale === 3 && args.preview) { mkdirSync(resolve(args.preview), { recursive: true }); writeFileSync(join(resolve(args.preview), `${name}.png`), readFileSync(png)); }
}
}
await browser.close();
// 24-bit BMP, the format NSIS wants: render at 3x, shrink with a good filter
const py = `import sys\nfrom PIL import Image\nfor name in ('sidebar','header'):\n im = Image.open(sys.argv[1] + '/' + name + '@3.png').convert('RGB')\n w, h = im.width // 3, im.height // 3\n im.resize((w, h), Image.LANCZOS).save(sys.argv[2] + '/' + name + '.bmp', 'BMP')\n`;
const done = spawnSync('python', ['-c', py, work, out], { encoding: 'utf8' });
if (done.status !== 0) { console.error(done.stderr); process.exit(1); }
console.log('wrote', join(out, 'sidebar.bmp'), join(out, 'header.bmp'));
@@ -0,0 +1,85 @@
#!/usr/bin/env bash
# Write a tester's README with the checksums and sizes of the files it names, computed from the files themselves.
#
# apps/desktop/scripts/make-tester-readme.sh <template> <out.txt> --appimage <file name> --files-dir <dir> [--files-dir <dir> ...]
#
# Why: a README that carries a hand-pasted sha256 goes stale the moment the file is rebuilt (the Ubuntu 26.04 tester compared the file
# with the old build's hash). The template holds placeholders instead, and this script fills them from the real files:
# @APPIMAGE@ the AppImage's file name
# @APP_VERSION@ the app version, read from apps/desktop/src-tauri/Cargo.toml
# @SHA256:<name>@ sha256 of the file <name> found in one of the --files-dir folders
# @SIZE_GB:<name>@ its size in GB with one decimal
# @INCLUDE:<file>@ (alone on a line) the contents of <file> from the template's folder, with the same placeholders
# It refuses to write the README when a named file is missing or a placeholder is left over, so a stale or empty hash cannot ship.
# The same check is done by `--verify <readme> --files-dir <dir>`: every "<64 hex> <name>" or "sha256 <64 hex>" pair that names a file
# present in the folders must match that file.
set -euo pipefail
usage() { echo "usage: $0 <template> <out.txt> --appimage <name> --files-dir <dir> [--files-dir <dir> ...]" >&2; echo " $0 --verify <readme> --files-dir <dir> [...]" >&2; exit 2; }
VERIFY=""; TEMPLATE=""; OUT=""; APPIMAGE=""; DIRS=()
if [ "${1:-}" = "--verify" ]; then VERIFY="${2:?readme}"; shift 2; else TEMPLATE="${1:?template}"; OUT="${2:?out}"; shift 2 || usage; fi
while [ $# -gt 0 ]; do
case "$1" in
--appimage) APPIMAGE="$2"; shift 2 ;;
--files-dir) DIRS+=("$2"); shift 2 ;;
*) usage ;;
esac
done
[ ${#DIRS[@]} -gt 0 ] || usage
find_file() { local d; for d in "${DIRS[@]}"; do [ -f "$d/$1" ] && { echo "$d/$1"; return 0; }; done; return 1; }
sha_of() { sha256sum "$1" | awk '{print $1}'; }
size_gb() { awk -v b="$(stat -c %s "$1" 2>/dev/null || stat -f %z "$1")" 'BEGIN{printf "%.1f", b/1000000000}'; }
if [ -n "$VERIFY" ]; then
# every checksum in the README must be the checksum of a file that is here, and a line that names a file and carries a checksum must carry that file's checksum
hashes=""; bad=0; seen=0
for d in "${DIRS[@]}"; do for f in "$d"/*; do [ -f "$f" ] && grep -qF "$(basename "$f")" "$VERIFY" && hashes="$hashes $(sha_of "$f")"; done; done # only files the README names are hashed (a folder may hold big unrelated files)
readme_hashes="$(grep -Eo '[0-9a-f]{64}' "$VERIFY" | sort -u || true)"
for h in $readme_hashes; do
seen=$((seen + 1))
case " $hashes " in *" $h "*) ;; *) echo "STALE: the README lists $h, which is the checksum of no file in: ${DIRS[*]}" >&2; bad=$((bad + 1)) ;; esac
done
for d in "${DIRS[@]}"; do for f in "$d"/*; do
[ -f "$f" ] || continue
grep -qF "$(basename "$f")" "$VERIFY" || continue
# a line that names this file AND carries a checksum must carry this file's checksum (a sentence that merely mentions the file is fine)
line_hash="$(grep -F "$(basename "$f")" "$VERIFY" | grep -Eo '[0-9a-f]{64}' | head -1 || true)"
if [ -n "$line_hash" ] && [ "$line_hash" != "$(sha_of "$f")" ]; then
echo "STALE: $(basename "$f") is $(sha_of "$f"), the README says $line_hash" >&2; bad=$((bad + 1))
fi
done; done
echo "$seen checksum(s) in the README checked, $bad stale"
[ "$bad" = 0 ] && [ "$seen" -gt 0 ]
exit
fi
[ -n "$APPIMAGE" ] || usage
TDIR="$(cd "$(dirname "$TEMPLATE")" && pwd)"
# the app version has one source (the same file Tauri and the shell use)
APP_VERSION="$(grep -m1 '^version = ' "$TDIR/../src-tauri/Cargo.toml" | cut -d'"' -f2)"
[ -n "$APP_VERSION" ] || { echo "make-tester-readme: no version in $TDIR/../src-tauri/Cargo.toml" >&2; exit 1; }
TMP="$(mktemp)"; trap 'rm -f "$TMP" "$TMP.2"' EXIT
# includes first (one level), then the placeholders
while IFS= read -r line || [ -n "$line" ]; do
if [[ "$line" =~ ^@INCLUDE:([^@]+)@$ ]]; then tr -d '\r' < "$TDIR/${BASH_REMATCH[1]}"; else printf '%s\n' "$line"; fi
done < <(tr -d '\r' < "$TEMPLATE") | sed "s|@APPIMAGE@|$APPIMAGE|g; s|@APP_VERSION@|$APP_VERSION|g" > "$TMP"
# @SHA256:name@ and @SIZE_GB:name@
while IFS= read -r token; do
kind="${token#@}"; kind="${kind%%:*}"; name="${token#@*:}"; name="${name%@}"
f="$(find_file "$name")" || { echo "make-tester-readme: no file named $name in: ${DIRS[*]} (a README must not carry a hash it did not compute)" >&2; exit 1; }
case "$kind" in
SHA256) value="$(sha_of "$f")" ;;
SIZE_GB) value="$(size_gb "$f")" ;;
*) echo "make-tester-readme: unknown placeholder $token" >&2; exit 1 ;;
esac
# replace every occurrence of this exact token (plain text, no pattern characters)
text="$(cat "$TMP"; printf x)"; text="${text%x}"
printf '%s' "${text//"$token"/$value}" > "$TMP"
done < <(grep -Eo '@(SHA256|SIZE_GB):[^@ ]+@' "$TMP" | sort -u)
if grep -Eq '@[A-Z0-9_]+[:@]' "$TMP"; then echo "make-tester-readme: placeholders left over:" >&2; grep -En '@[A-Z0-9_]+[:@]' "$TMP" >&2; exit 1; fi
cp "$TMP" "$OUT"
echo "wrote $OUT"
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env node
/**
* Attached pictures reaching a model, end to end on a test box, through the installed app layout (packaged host + bundled Control + an engine
* component that carries strata-vision):
* 1. the model catalog marks the picture-capable models (images: true) when the picture reader is installed;
* 2. API models: a llama-server with a small picture model (--mmproj) is saved as a Custom model API; the host works out that it reads pictures,
* and a real picture goes out as image_url and is answered. A server without a picture reader, a model set "not to read pictures", and an API
* of another style each get a plain refusal (409) that says how to fix it; ticking "This model reads pictures" lets the picture through;
* 3. a local model (unless --skip-local): Download and prepare with Read pictures ON, load, and a picture through the app's own chat.
*
* node pics-e2e.mjs --install <node/ host/ control/> --components <engine components> --work <scratch> --image shapes.png \
* Also: --silent <url> (a server whose model list says nothing about pictures: ON by default), --agent (the same picture through the agent; needs a Harness)
* --vlm http://127.0.0.1:18090/v1 --vlm-model smolvlm --text-only http://127.0.0.1:18091/v1 [--skip-local --family qwen --model IQ3_XXS --data <folder> --device cpu]
*/
import { spawn } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, readdirSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
for (const need of ['install', 'components', 'work', 'image']) if (!args[need]) { console.error('missing --' + need); process.exit(2); }
const install = resolve(args.install), work = resolve(args.work), components = resolve(args.components);
const want = String(args.match ?? args.model ?? 'IQ3_XXS').toLowerCase(), contextSize = Number(args.context ?? 32768), skipLocal = args['skip-local'] === true, device = String(args.device ?? 'cpu'), family = String(args.family ?? 'qwen'), modelId = String(args.model ?? 'IQ3_XXS');
const node = join(install, 'node', 'node'), hostEntry = join(install, 'host', 'server', 'index.mjs');
const dirs = { models: join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home'), xdg: join(work, 'xdg'), appdata: join(work, 'appdata') };
for (const d of [dirs.models, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
let failed = false;
const record = (name, ok, detail = '') => { if (!ok) failed = true; console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 600) : ''}`); };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.models,
STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models-unused'), APPDATA: dirs.appdata, XDG_CONFIG_HOME: dirs.xdg, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(node, [hostEntry], { cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'],
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
const png = 'data:image/png;base64,' + readFileSync(resolve(args.image)).toString('base64');
const QUESTION = 'Name each shape in this picture with its colour, and read out any text.';
/** One chat turn through the app's own endpoint: the plain-text answer, or the refusal (status + message). */
async function chat(extra, message = QUESTION) {
const response = await fetch(base + '/api/chat/stream', { method: 'POST', headers: { 'content-type': 'application/json' },
body: JSON.stringify({ message, agent: false, images: [{ url: png, name: 'shapes.png' }], ...extra }) });
if (response.status !== 200) { const b = await response.json().catch(() => ({})); return { status: response.status, message: b.error ?? b.message ?? '' }; }
let text = '', error = '';
for (const line of (await response.text()).split('\n')) {
if (!line.trim()) continue;
let f; try { f = JSON.parse(line); } catch { continue; }
if (f.type === 'error') error = f.message ?? 'error';
else if (typeof f.text === 'string' && f.type === 'delta') text += f.text;
}
return { status: 200, text, error };
}
const saveApi = async (name, baseUrl, models, protocol = 'openai-chat') => {
const existing = (await call('/api/model-providers')).body.connections.find(c => c.name === name);
return (await call('/api/model-providers', 'POST', { ...(existing ? { id: existing.id } : {}), kind: 'custom', provider: null, name, baseUrl, protocol, apiKey: '', models })).body;
};
async function readsAfter(connectionName, id, want, seconds = 20) {
for (let i = 0; i < seconds * 2; i++) {
const c = (await call('/api/model-providers')).body.connections.find(x => x.name === connectionName);
const m = c?.models.find(x => x.id === id);
if (m && m.reads === want) return { c, m };
await delay(500);
}
const c = (await call('/api/model-providers')).body.connections.find(x => x.name === connectionName);
return { c, m: c?.models.find(x => x.id === id) };
}
async function waitJob(id, seconds) {
let last = '';
for (let i = 0; i < seconds * 2; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.summary !== last) { last = job.summary; console.log(' ... ' + String(last).slice(0, 120)); }
if (job && job.state !== 'running') return { job, state };
await delay(500);
}
throw new Error('operation did not finish');
}
const names = low => /(red|blue|green|circle|square|triangle|strata|42)/.test(low);
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { /* not yet */ } if (!up) await delay(150); }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
// ---- 1. the catalog
const packages = existsSync(join(components, 'engine')) ? readdirSync(join(components, 'engine')) : [];
const reader = packages.some(p => existsSync(join(components, 'engine', p, 'strata-vision')));
const native = (await call('/api/native?refresh=1')).body;
record('the engine component carries the picture reader and the host knows it (visionReader)', reader && native.visionReader === true, `component=${reader} host=${native.visionReader}`);
const rows = (native.catalog?.families ?? []).flatMap(f => (f.sizes ?? []).map(s => ({ family: f.title ?? f.family, size: s.id, images: s.images, capable: s.imagesCapable })));
console.log(' catalog (family / size / images / imagesCapable):');
for (const r of rows) console.log(` ${String(r.family).padEnd(26)} ${String(r.size).padEnd(30)} images=${r.images} capable=${r.capable}`);
record('some models in the catalog read pictures', rows.some(r => r.images === true), `${rows.filter(r => r.images).length} of ${rows.length}`);
record('every picture-capable model reads pictures when the reader is installed', rows.every(r => r.capable === r.images), '');
const flash = rows.filter(r => /flash-next/i.test(`${r.family} ${r.size}`) && !/gyro|qad/i.test(`${r.family} ${r.size}`));
record('the Qwen3.8-Flash-Next models are marked images: true', flash.length > 0 && flash.every(r => r.images === true), flash.map(r => r.size + '=' + r.images).join(' '));
// ---- 2. API models
if (args.vlm) {
const smol = args['vlm-model'] ?? 'smolvlm';
await saveApi('Small picture model', args.vlm, [{ id: smol, name: 'SmolVLM' }]);
const found = await readsAfter('Small picture model', smol, true);
record('the host works out that the llama-server reads pictures (no tick needed)', found.m?.reads === true, JSON.stringify(found.m));
const conn = found.c;
const sent = await chat({ remoteModel: { connectionId: conn?.id, modelId: smol } });
console.log(` answer from the API model: ${JSON.stringify((sent.text ?? sent.message ?? '').slice(0, 300))}`);
record('the picture goes to the API model and is answered', sent.status === 200 && !sent.error && (sent.text ?? '').trim().length > 0 && names((sent.text ?? '').toLowerCase()), `status=${sent.status} err=${sent.error ?? ''}`);
if (args['text-only']) {
await saveApi('Text only server', args['text-only'], [{ id: 'plain', name: 'Plain model' }]);
const t = await readsAfter('Text only server', 'plain', false);
record('a server without a picture reader is known not to read pictures', t.m?.reads === false, JSON.stringify(t.m));
const refused = await chat({ remoteModel: { connectionId: t.c.id, modelId: 'plain' } });
console.log(` refusal: ${JSON.stringify(refused.message)}`);
record('a picture for it is refused in plain words that say how to fix it', refused.status === 409 && /Read pictures with this model/.test(refused.message ?? '') && /Settings/.test(refused.message ?? ''), `${refused.status}`);
const ticked = await saveApi('Text only server', args['text-only'], [{ id: 'plain', name: 'Plain model', vision: true }]);
const after = (await call('/api/model-providers')).body.connections.find(x => x.name === 'Text only server');
record('ticking "Read pictures with this model" is saved and makes the model read pictures', after?.models[0]?.vision === true && after?.models[0]?.reads === true, JSON.stringify(after?.models[0]));
const through = await chat({ remoteModel: { connectionId: after.id, modelId: 'plain' } });
record('with the tick the picture is sent on (the server itself decides what to do with it)', through.status === 200 && (through.error ?? '') !== '' ? /turn off/.test(through.error) : through.status === 200, `${through.status} ${through.message ?? through.error ?? ''}`);
}
await saveApi('Small picture model', args.vlm, [{ id: smol, name: 'SmolVLM', vision: false }]);
const off = await readsAfter('Small picture model', smol, false);
const notRead = await chat({ remoteModel: { connectionId: off.c.id, modelId: smol } });
record('a model switched OFF refuses pictures and says where to turn them on again', notRead.status === 409 && /not to read pictures/.test(notRead.message ?? '') && /Settings/.test(notRead.message ?? ''), `${notRead.status} ${notRead.message ?? ''}`);
await saveApi('Small picture model', args.vlm, [{ id: smol, name: 'SmolVLM', vision: null }]);
// unknown is ON by default: a server whose list and /props say nothing is sent the picture
if (args.silent) {
await saveApi('Silent server', args.silent, [{ id: 'quiet', name: 'Quiet model' }]);
const q = await readsAfter('Silent server', 'quiet', true, 8);
record('a server that says nothing about pictures is ON by default', q.m?.reads === true && q.m?.vision === null, JSON.stringify(q.m));
const sentQuiet = await chat({ remoteModel: { connectionId: q.c.id, modelId: 'quiet' } });
record('and the picture is sent to it (not refused)', sentQuiet.status === 200, `${sentQuiet.status} ${sentQuiet.message ?? sentQuiet.error ?? ''}`);
}
await saveApi('Other style', 'https://api.anthropic.com/v1', [{ id: 'claude-x', name: 'Claude' }], 'anthropic');
const other = (await call('/api/model-providers')).body.connections.find(x => x.name === 'Other style');
const style = await chat({ remoteModel: { connectionId: other.id, modelId: 'claude-x' } });
console.log(` other style: ${JSON.stringify(style.message)}`);
record('an API of another style says plainly why pictures stay in the preview', style.status === 409 && /OpenAI/.test(style.message ?? ''), `${style.status}`);
}
if (skipLocal) return;
// ---- 3. a local model
const values = { family, model: modelId, context: contextSize, vision: 'yes', loadVision: device, ...(args.data ? { data_dir: resolve(args.data) } : {}) };
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: values });
record('the install operation starts', started.status === 200 || started.status === 202, JSON.stringify(started.body).slice(0, 200));
const done = await waitJob(started.body.id, 3600);
record('Download and prepare finishes', done.job.state === 'done', `${done.job.state} ${done.job.error ?? done.job.summary}`);
const refreshed = (await call('/api/native?refresh=1')).body;
const model = refreshed.installed?.models?.find(m => [m.id, m.model, m.tag].some(x => String(x ?? '').toLowerCase() === want));
record('the model is prepared with its picture reader', !!model?.visionPrepared, JSON.stringify({ ready: model?.ready, visionPrepared: model?.visionPrepared }));
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: model?.model ?? want } });
const loaded = await waitJob(load.body.id, 900);
const instance = loaded.state.instances[0];
record('the model loads with the picture reader', loaded.job.state === 'done' && instance?.status === 'ready' && instance?.vision === true, `${loaded.job.state} ${loaded.job.error ?? ''} vision=${instance?.vision}`);
record('the model loaded with pictures ON without being asked per load (default)', instance?.vision === true, '');
const t0 = Date.now();
let local = { status: 0 };
for (let i = 0; i < 2 && !(local.text ?? '').trim(); i++) local = await chat({ nativeModelId: instance?.id, maxTokens: 4096 });
console.log(` answer from the local model (${Math.round((Date.now() - t0) / 1000)} s): ${JSON.stringify((local.text ?? local.message ?? '').slice(0, 300))}`);
const low = (local.text ?? '').toLowerCase();
record('the local model answers about the picture (names the shapes)', local.status === 200 && /red|circle/.test(low) && /blue|square/.test(low) && /green|triangle/.test(low), `status=${local.status} err=${local.error ?? ''}`);
record('the local model reads STRATA 42', low.includes('42'));
// the same picture through the agent (the Harness route must declare image input: the "does not declare image input" error of the tester)
if (args.agent) { // needs a Harness beside the host (the installed layout has one; a bare test box does not)
const t1 = Date.now(); let a = { status: 0 };
for (let i = 0; i < 2 && !/red|circle|square|triangle/.test((a.text ?? '').toLowerCase()); i++) a = await chat({ nativeModelId: instance?.id, agent: true, sessionId: 'pics-agent-' + i, mode: 'plan', maxTokens: 4096 });
console.log(` answer through the agent (${Math.round((Date.now() - t1) / 1000)} s): ${JSON.stringify((a.text ?? a.message ?? '').slice(0, 300))}`);
record('the agent route reads the picture too', a.status === 200 && /red|circle|square|triangle/.test((a.text ?? '').toLowerCase()) && !/does not declare image input/.test(a.text ?? ''), `status=${a.status} err=${a.error ?? ''}`);
}
// OFF: loading with Read pictures off refuses a picture in plain words and says what to do
await call(`/api/native/instances/${instance?.id}/unload`, 'POST', {});
const loadOff = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: model?.model ?? want, vision: 'no' } });
const loadedOff = await waitJob(loadOff.body.id, 900);
const off2 = loadedOff.state.instances[0];
record('loaded with Read pictures OFF', loadedOff.job.state === 'done' && off2?.vision === false, `${loadedOff.job.state} vision=${off2?.vision}`);
const refusedLocal = await chat({ nativeModelId: off2?.id, maxTokens: 64 });
console.log(` refusal: ${JSON.stringify(refusedLocal.message)}`);
record('a picture for the OFF model is refused with a next step', refusedLocal.status === 409 && /Read pictures/.test(refusedLocal.message ?? '') && /Models/.test(refusedLocal.message ?? ''), `${refusedLocal.status}`);
const unloaded = await call(`/api/native/instances/${off2?.id}/unload`, 'POST', {});
record('unload', unloaded.status === 200, unloaded.status);
}
try { await main(); } catch (e) { record('run', false, e.stack ?? e); }
host.kill();
await delay(1500);
console.log(failed ? 'PICS-E2E FAIL' : 'PICS-E2E PASS');
process.exit(failed ? 1 : 0);
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# Put the bundled payloads back to the bytes they were staged with, inside a finished AppImage.
#
# apps/desktop/scripts/repack-appimage.sh <in.AppImage> <out.AppImage> [--resources <src-tauri/resources>] [--check]
#
# Why: Tauri hands the AppDir to linuxdeploy, which treats every ELF file under usr/lib as a library to fix up: it rewrites the RPATH of
# the engine (strata, libstrata_kq.so, libcudart), of the bundled Python and Node and of the Harness's native modules. The files then no
# longer match the SHA-256 and size their manifests list (engine-component.json), so the host refuses the bundled engine, and the
# installer's own integrity promise ("what we staged is what runs") is broken. This script unpacks the AppImage, copies the staged
# folders (control, harness, node, engine-bundle) over the AppDir byte for byte, and packs it again with the same runtime.
#
# --check does not write anything: it lists every engine-bundle file whose bytes differ from the manifest in the AppImage.
set -euo pipefail
IN="$(realpath "${1:?in.AppImage}")"; OUT="${2:-}"
RES="$(cd "$(dirname "${BASH_SOURCE[0]}")/../src-tauri/resources" && pwd)"
CHECK=0
shift 2 || true
while [ $# -gt 0 ]; do
case "$1" in
--resources) RES="$(cd "$2" && pwd)"; shift 2 ;;
--check) CHECK=1; shift ;;
*) echo "unknown option $1" >&2; exit 2 ;;
esac
done
OFFSET="$("$IN" --appimage-offset)"
WORK="$(mktemp -d -t strata-repack.XXXXXX)"
trap 'rm -rf "$WORK"' EXIT
unsquashfs -no-progress -o "$OFFSET" -d "$WORK/AppDir" "$IN" >/dev/null
# the product folder under usr/lib (Tauri names it after productName)
LIB="$(find "$WORK/AppDir/usr/lib" -maxdepth 2 -type d -name engine-bundle -printf '%h\n' | head -1)"
[ -n "$LIB" ] || LIB="$(find "$WORK/AppDir/usr/lib" -maxdepth 2 -type d -name control -printf '%h\n' | head -1)"
[ -n "$LIB" ] || { echo "no Strata resource folder in the AppImage" >&2; exit 1; }
if [ "$CHECK" = 1 ]; then
bad=0
while IFS= read -r f; do
rel="${f#$RES/}"
if ! cmp -s "$f" "$LIB/$rel"; then echo "differs: $rel"; bad=$((bad + 1)); fi
done < <(find "$RES/engine-bundle" "$RES/node" -type f ! -name .gitignore)
echo "$bad file(s) differ from what was staged"
[ "$bad" = 0 ]
exit
fi
[ -n "$OUT" ] || { echo "give out.AppImage" >&2; exit 2; }
for d in control harness node engine-bundle; do
[ -d "$RES/$d" ] || continue
rsync -aL --checksum --delete --exclude .gitignore "$RES/$d/" "$LIB/$d/"
done
# The launcher records the environment the AppImage was started from, before its own hooks add the AppImage's libraries and GTK/GIO
# settings (LD_LIBRARY_PATH, GTK_*, XDG_DATA_DIRS ...). The host gives it back to every program that is not Strata's own (zenity,
# kdialog, xdg-open, the assistant's shell): apps/host/server/external-env.mjs. Without it a system program loads the AppImage's libssl
# and dies ("OPENSSL_3.2.0 not found"). The line goes right after `this_dir=...`, ahead of every hook.
if ! grep -q STRATA_ORIGINAL_ENV "$WORK/AppDir/AppRun"; then
sed -i '/^this_dir=/a # Strata: the environment this AppImage was started from (given back to programs that are not Strata'"'"'s own)\nexport STRATA_ORIGINAL_ENV="$(env -0 | base64 -w0)"' "$WORK/AppDir/AppRun"
grep -q STRATA_ORIGINAL_ENV "$WORK/AppDir/AppRun" || { echo "could not add the environment record to AppRun" >&2; exit 1; }
fi
mksquashfs "$WORK/AppDir" "$WORK/app.squashfs" -root-owned -noappend -comp zstd -b 131072 -no-progress >/dev/null
head -c "$OFFSET" "$IN" > "$OUT"
cat "$WORK/app.squashfs" >> "$OUT"
chmod +x "$OUT"
# The checksum is computed here, from the finished file, every time: a README or a note must take it from this file
# (apps/desktop/scripts/make-tester-readme.sh) and never from an older build.
( cd "$(dirname "$OUT")" && sha256sum "$(basename "$OUT")" > "$(basename "$OUT").sha256" )
echo "repacked: $OUT ($(stat -c %s "$OUT") bytes)"
cat "$OUT.sha256"
+202
View File
@@ -0,0 +1,202 @@
#!/usr/bin/env python3
"""Stage the Control component the installer ships: src-tauri/resources/control.
python apps/desktop/scripts/stage-control.py # build the terminal bundle, stage it
python apps/desktop/scripts/stage-control.py --archive X.zip # stage an archive you already built
python apps/desktop/scripts/stage-control.py --wheelhouse DIR # build with no network (wheels from DIR)
python apps/desktop/scripts/stage-control.py --check # verify what is staged
What lands in `control/`: the pinned CPython 3.12.15 (python-build-standalone, digest-checked), the hash-locked wheels
(`packaging/locks/runtime-<target>-cp312.txt`, installed with --require-hashes at build time), strata_control,
strata_cli, serve, tools/strata_tokenizer.py + gguf_reader.py, and BUNDLE.json (version, interpreter and lock digests,
the digest of the archive it came from). It is the same bundle as the Terminal archive (packaging/package_terminal.py):
one Python runtime-core for both products. The app then never runs pip, npm, git or a compiler to start or serve.
This script is a BUILD step. It may download the interpreter and the wheels (verified); the app never does.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import shutil
import subprocess
import sys
import tempfile
import zipfile
import tarfile
from pathlib import Path
HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[2]
OUT = HERE.parent / "src-tauri" / "resources" / "control"
sys.path.insert(0, str(ROOT / "packaging"))
import package_terminal as pt # noqa: E402
def sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1 << 20), b""):
digest.update(block)
return digest.hexdigest()
def build_archive(work: Path, wheelhouse: str | None) -> Path:
command = [sys.executable, str(ROOT / "packaging" / "package_terminal.py"), "--out", str(work)]
if wheelhouse:
command += ["--wheelhouse", wheelhouse]
subprocess.run(command, check=True, cwd=str(ROOT))
archives = sorted(work.glob("strata-terminal-*.zip")) + sorted(work.glob("strata-terminal-*.tar.gz"))
if len(archives) != 1:
raise SystemExit(f"expected one archive in {work}, found {archives}")
return archives[0]
def extract(archive: Path, destination: Path) -> Path:
"""Unpack the archive; returns the folder that holds python/ and strata.py."""
scratch = destination.parent / (destination.name + ".incoming")
shutil.rmtree(scratch, ignore_errors=True)
scratch.mkdir(parents=True)
if archive.name.endswith(".zip"):
with zipfile.ZipFile(archive) as zf:
for member in zf.namelist():
target = (scratch / member).resolve()
if scratch.resolve() not in target.parents and target != scratch.resolve():
raise SystemExit(f"unsafe member in {archive.name}: {member}")
zf.extractall(scratch)
else:
with tarfile.open(archive) as tf:
tf.extractall(scratch, filter="data")
(inner,) = [p for p in scratch.iterdir() if p.is_dir()]
return inner
#: What the app's installer needs on top of the Terminal bundle: the model catalog, GPU probe and install job. The list and the copy
#: live in packaging/package_terminal.py (the Terminal archive carries the same files, for `strata launch`).
INSTALLER_FILES, INSTALLER_TOOLS, INSTALLER_DATA = pt.INSTALLER_FILES, pt.INSTALLER_TOOLS, pt.INSTALLER_DATA
local_imports = pt.local_imports
def add_installer(folder: Path) -> dict:
"""Copy the installer's files from the source tree into the staged Control (after the Terminal bundle is in place)."""
return pt.add_installer(folder, ROOT)
#: What the other names of the interpreter become: Tauri copies a symlink as a full copy of its target, so `python`, `python3` and
#: `python3.12` in a bundle would be three real files (68 MB each on Linux). Only `python3.12` stays the real one.
#: The launcher runs inside the AppImage's environment (LD_LIBRARY_PATH holds the bundled libraries), where an external program such as
#: `dirname` can fail to start (Ubuntu 26.04's coreutils: "libsystemd.so.0: version LIBSYSTEMD_254 not found"), so it uses shell built-ins
#: only. The first version called `dirname`; that left the folder empty and "Control stopped" on such a computer.
LAUNCHER = """#!/bin/sh
# Strata: this name only starts the real interpreter beside it (the bundle keeps one copy, not one per name). Built-ins only: no dirname.
case "$0" in */*) dir=${0%%/*} ;; *) dir=. ;; esac
here=$(cd "$dir" && pwd)
exec "$here/%s" "$@"
"""
def dedupe_python(folder: Path) -> list[str]:
"""Make every other name of the interpreter in `python/bin` a small launcher of the real binary (Linux and macOS; Windows has one
python.exe). A symlink would not survive the bundling, and a plain copy is what costs the space. Returns the names changed."""
bindir = folder / "python" / "bin"
if not bindir.is_dir():
return []
real = bindir / f"python{pt.PYTHON_VERSION.rsplit('.', 1)[0]}"
if not real.exists():
return []
if real.is_symlink(): # the real file is a link itself: make the target the real one
target = real.resolve()
real.unlink()
shutil.move(str(target), str(real))
changed = []
for name in ("python", "python3"):
path = bindir / name
if not (path.is_symlink() or (path.is_file() and path.stat().st_size == real.stat().st_size)):
continue
path.unlink()
path.write_bytes((LAUNCHER % real.name).encode("utf-8"))
path.chmod(0o755)
changed.append(name)
return changed
def write_bundle_record(folder: Path, archive: Path) -> dict:
manifest_path = archive.with_name(archive.name.rsplit(".", 2 if archive.name.endswith(".tar.gz") else 1)[0] + ".manifest.json")
manifest = json.loads(manifest_path.read_text(encoding="utf-8")) if manifest_path.exists() else {}
target = manifest.get("target") or pt.host_target()
record = {
"schema_version": 1, "component": "strata-control",
"version": (folder / "VERSION").read_text(encoding="utf-8").strip(),
"target": target,
"python": {"version": pt.PYTHON_VERSION, "build": pt.PBS_RELEASE, "sha256": pt.PYTHON_PINS[target]["sha256"]},
"lock": manifest.get("lock"),
"installed": manifest.get("installed"),
"archive": {"name": archive.name, "sha256": sha256(archive), "bytes": archive.stat().st_size},
"acquires_at_runtime": False,
"installer": add_installer(folder),
}
(folder / "BUNDLE.json").write_text(json.dumps(record, indent=1) + "\n", encoding="utf-8")
return record
def check(folder: Path) -> int:
target = pt.host_target()
python = folder / pt.PYTHON_PINS[target]["python"]
problems = []
for need in (python, folder / "strata.py", folder / "strata_control", folder / "serve" / "server.py", folder / "BUNDLE.json",
folder / "setup.py", folder / "tools" / "strata_mcp.py", folder / "tools" / "iq_pack.py", folder / "data" / "draft_vocab.bin"):
if not need.exists():
problems.append(f"missing {need}")
if problems:
print("\n".join(problems), file=sys.stderr)
return 1
env = {"PYTHONNOUSERSITE": "1", "PYTHONDONTWRITEBYTECODE": "1", "STRATA_NO_ACQUIRE": "1", "SYSTEMROOT": __import__("os").environ.get("SYSTEMROOT", "")}
done = subprocess.run([str(python), str(folder / "strata.py"), "--version"], capture_output=True, text=True, env=env,
cwd=str(folder), timeout=60)
print(done.stdout.strip() or done.stderr.strip())
if done.returncode:
return done.returncode
# the installer: its catalog, a GPU probe and its imports work on the bundled Python alone (nothing is installed or fetched)
probe = ("import sys; sys.path.insert(0, %r); import importlib.util as u; s = u.spec_from_file_location('s', %r); m = u.module_from_spec(s); "
"s.loader.exec_module(m); import iq_pack, strata_pack, pack_index, mtp_fetch; "
"print('installer ok: %%d models, %%d families' %% (len(m.MODELS), len(m.FAMILIES)))") % (str(folder / "tools"), str(folder / "setup.py"))
env["STRATA_CONTROL_INSTALL"] = "1"
again = subprocess.run([str(python), "-c", probe], capture_output=True, text=True, env=env, cwd=str(folder), timeout=120)
print(again.stdout.strip() or again.stderr.strip()[-600:])
return again.returncode
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--archive", type=Path, help="stage this terminal archive instead of building one")
ap.add_argument("--wheelhouse", help="build with the wheels in this folder (no network)")
ap.add_argument("--out", type=Path, default=OUT)
ap.add_argument("--check", action="store_true", help="verify the staged component and exit")
a = ap.parse_args()
if a.check:
return check(a.out)
with tempfile.TemporaryDirectory(prefix="strata-stage-control-") as work:
archive = a.archive.resolve() if a.archive else build_archive(Path(work), a.wheelhouse)
keep = [p for p in a.out.iterdir() if p.name == ".gitignore"] if a.out.exists() else []
if a.out.exists():
for child in a.out.iterdir():
if child.name != ".gitignore":
shutil.rmtree(child, ignore_errors=True) if child.is_dir() else child.unlink()
a.out.mkdir(parents=True, exist_ok=True)
inner = extract(archive, a.out)
record = write_bundle_record(inner, archive)
for child in inner.iterdir():
shutil.move(str(child), str(a.out / child.name))
shutil.rmtree(inner.parent, ignore_errors=True)
launched = dedupe_python(a.out)
if launched:
print(f"python: {', '.join(launched)} are launchers of the one real interpreter")
print(f"staged {record['component']} {record['version']} ({record['target']}) into {a.out}")
return check(a.out)
if __name__ == "__main__":
raise SystemExit(main())
+181
View File
@@ -0,0 +1,181 @@
#!/usr/bin/env node
/**
* Stage the DeepSeek Harness (pinned, MIT) as the installer's `harness/` folder: a pruned PRODUCTION install, built at
* build time, so nothing is fetched or compiled on the person's PC.
*
* node apps/desktop/scripts/stage-harness.mjs --src <built Harness checkout> [--out apps/desktop/src-tauri/resources/harness]
*
* <built Harness checkout> is a clone of the pinned commit after `pnpm install --frozen-lockfile` and `pnpm run build:artifacts`
* (see phaseA-tests/notes-split-s4a.md, step 1). The script never touches it: it runs `pnpm deploy --legacy --prod` into
* the output folder, removes what a running Harness does not read (sources, tests, maps, docs, other operating systems'
* prebuilt binaries, the LibreOffice document converter and the speech engine, which the app does not use), keeps the
* MIT licence and third-party notices, writes BUNDLE.json, and finally boots the staged copy once as proof.
*
* The host finds the result at `<install>/harness` (server/agent/harness-runtime.mjs), or in the component folder.
*/
import { spawnSync } from 'node:child_process';
import { cpSync, existsSync, mkdirSync, readFileSync, readdirSync, realpathSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { join, relative, resolve, sep } from 'node:path';
import { createHash } from 'node:crypto';
import { flattenPnpm } from './flatten-pnpm.mjs';
import { isOtherOsBinaryFolder, missingHarnessImports } from './harness-prune.mjs';
import { patchStagedHarnessBudget } from './harness-budget.mjs';
const args = process.argv.slice(2);
const opt = (name, fallback) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : fallback; };
const src = resolve(opt('src', process.env.STRATA_HARNESS_SRC ?? ''));
const out = resolve(opt('out', join(import.meta.dirname, '..', 'src-tauri', 'resources', 'harness')));
if (!opt('src', process.env.STRATA_HARNESS_SRC) || !existsSync(join(src, 'apps', 'cli', 'lib', 'bin.js'))) {
console.error('Give --src <a built Harness checkout> (apps/cli/lib/bin.js must exist).');
process.exit(2);
}
const platform = process.platform;
const run = (cmd, argv, cwd, extra = {}) => {
const r = spawnSync(cmd, argv, { cwd, stdio: 'inherit', shell: false, ...extra });
if (r.status !== 0) { console.error(`${cmd} ${argv.join(' ')} failed (${r.status ?? r.error})`); process.exit(1); }
};
// The output folder in the repo carries a .gitignore that keeps the staged files out of git: keep it across the clean-up.
const keepIgnore = existsSync(join(out, '.gitignore')) ? readFileSync(join(out, '.gitignore')) : null;
rmSync(out, { recursive: true, force: true });
mkdirSync(out, { recursive: true });
const pnpm = platform === 'win32' ? ['corepack.cmd', ['pnpm']] : ['corepack', ['pnpm']];
run(pnpm[0], [...pnpm[1], 'deploy', '--legacy', '--prod', '--filter', '@deepseek-ai/dsh', out], src, { shell: platform === 'win32' });
// --- peers -------------------------------------------------------------------------------------------------------------
// `pnpm deploy --prod` leaves out peer dependencies, but the Harness loads some of them by name at run time (for example the
// plugin-group helper that app-boot imports). Walk the installed packages; every dependency or peer that cannot be
// resolved from the output is copied from the source checkout's own node_modules, and its own requirements after it.
function packageJsonsUnder(dir, out2 = []) {
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isSymbolicLink()) continue;
if (e.isDirectory()) packageJsonsUnder(p, out2);
else if (e.name === 'package.json' && p.includes(`${sep}node_modules${sep}`)) out2.push(p);
}
return out2;
}
function requirementsOf(file) {
try { const j = JSON.parse(readFileSync(file, 'utf8')); return [...Object.keys(j.dependencies ?? {}), ...Object.keys(j.peerDependencies ?? {}), ...Object.keys(j.optionalDependencies ?? {})]; } catch { return []; }
}
/** The checkout's own workspace packages (packages/**, vendor/*, apps/*) by name: where a peer that is one of ours comes from. */
const workspace = new Map();
function indexWorkspace(dir, depth = 0) {
if (depth > 4) return;
for (const e of readdirSync(dir, { withFileTypes: true })) {
if (!e.isDirectory() || e.isSymbolicLink() || e.name === 'node_modules' || e.name.startsWith('.')) continue;
const p = join(dir, e.name);
const manifest = join(p, 'package.json');
if (existsSync(manifest)) { try { workspace.set(JSON.parse(readFileSync(manifest, 'utf8')).name, p); } catch { /* not a package */ } }
indexWorkspace(p, depth + 1);
}
}
for (const top of ['packages', 'vendor', 'apps']) if (existsSync(join(src, top))) indexWorkspace(join(src, top));
const resolvable = name => existsSync(join(out, 'node_modules', ...name.split('/')));
const queue = [...new Set(packageJsonsUnder(join(out, 'node_modules')).flatMap(requirementsOf))].filter(n => !resolvable(n));
const added = [];
while (queue.length) {
const name = queue.shift();
if (resolvable(name)) continue;
const from = workspace.get(name) ?? join(src, 'node_modules', ...name.split('/'));
if (!existsSync(from)) continue; // an optional package for another platform, or not installed in the source: nothing to copy
const to = join(out, 'node_modules', ...name.split('/'));
mkdirSync(join(to, '..'), { recursive: true });
const realFrom = realpathSync(from);
cpSync(realFrom, to, { recursive: true, dereference: true, filter: f => !relative(realFrom, f).split(sep).includes('node_modules') });
added.push(name);
queue.push(...requirementsOf(join(to, 'package.json')).filter(n => !resolvable(n)));
}
if (added.length) console.log(`peer packages added: ${added.join(', ')}`);
// --- the browser tool server ---------------------------------------------------------------------------------------------
// Settings -> Tools can switch on a browser the assistant drives (Playwright's MCP server, Apache-2.0) on the person's own Chrome or Edge,
// so no browser is downloaded. The server and the two Playwright libraries it needs are not part of the Harness's production closure;
// they are copied from the source checkout's package store (~20 MB) and the host starts it with Node (server/agent/browser-probe.mjs).
const pnpmStore = join(src, 'node_modules', '.pnpm');
function storePackage(name, version) {
if (!existsSync(pnpmStore)) return null;
const prefix = `${name.replace('/', '+')}@`;
const hit = readdirSync(pnpmStore).filter(n => n.startsWith(prefix) && (!version || n === `${prefix}${version}` || n.startsWith(`${prefix}${version}_`))).sort().at(-1);
const dir = hit ? join(pnpmStore, hit, 'node_modules', ...name.split('/')) : null;
return dir && existsSync(dir) ? dir : null;
}
const browserDir = storePackage('@playwright/mcp');
if (browserDir) {
const wanted = JSON.parse(readFileSync(join(browserDir, 'package.json'), 'utf8')).dependencies ?? {};
const parts = [['@playwright/mcp', browserDir], ...['playwright', 'playwright-core'].map(n => [n, storePackage(n, wanted[n])])];
if (parts.every(([, dir]) => dir)) {
for (const [name, dir] of parts) {
const to = join(out, 'node_modules', ...name.split('/'));
mkdirSync(join(to, '..'), { recursive: true });
cpSync(realpathSync(dir), to, { recursive: true, dereference: true, filter: f => !relative(dir, f).split(sep).includes('node_modules') });
}
console.log(`browser tool server added: ${parts.map(([n]) => n).join(', ')}`);
} else console.log('browser tool server NOT added: Playwright libraries missing from the package store');
} else console.log('browser tool server NOT added: @playwright/mcp is not in the source checkout');
// --- one real folder per package, no links -------------------------------------------------------------------------------
// The installer copies files; pnpm's layout is links. See flatten-pnpm.mjs. The two packages the app does not use are never copied.
const flat = flattenPnpm(out, { skip: /libreoffice-kit|sherpa-onnx/i });
console.log(`flattened: ${flat.packages} placed (${flat.nested} nested for a second version)`);
// --- prune ------------------------------------------------------------------------------------------------------------
/** Whole packages the app never loads: office-document conversion (hundreds of MB) and speech input. */
const DROP_PACKAGES = [/libreoffice-kit/i, /sherpa-onnx/i];
/** Only when sitting directly in a package's root (a library may have a runtime folder called `doc`, e.g. yaml/dist/doc). */
const DROP_DIRS = new Set(['test', 'tests', '__tests__', 'docs', 'website', 'benchmarks', 'examples', '.github']);
const DROP_FILES = /\.(map|md|markdown|pdb|tsbuildinfo)$/i;
/** Licences and notices travel with the files they cover. */
const KEEP_FILES = /^(LICENSE|LICENCE|NOTICE|THIRD_PARTY)/i;
let removed = 0;
function prune(dir) {
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isSymbolicLink()) continue;
if (e.isDirectory()) {
const inLib = p.split(sep).includes('lib');
if ((DROP_DIRS.has(e.name) && !inLib && existsSync(join(dir, 'package.json'))) || DROP_PACKAGES.some(r => r.test(e.name)) || isOtherOsBinaryFolder(platform, e.name, p)) {
rmSync(p, { recursive: true, force: true }); removed++; continue;
}
prune(p);
} else if (DROP_FILES.test(e.name) && !KEEP_FILES.test(e.name)) {
rmSync(p, { force: true }); removed++;
}
}
}
prune(out);
// Strata's provenance-recorded output-budget correction applies only to this copy.
const patches = [patchStagedHarnessBudget(out)];
// --- licence: the MIT text and the third-party notices travel with the files ---------------------------------------------
for (const name of ['LICENSE', 'THIRD_PARTY_NOTICES.md']) {
if (existsSync(join(src, name))) cpSync(join(src, name), join(out, name));
}
const cli = JSON.parse(readFileSync(join(out, 'package.json'), 'utf8'));
let commit = process.env.STRATA_HARNESS_COMMIT ?? null;
if (!commit) { const r = spawnSync('git', ['rev-parse', 'HEAD'], { cwd: src, encoding: 'utf8' }); if (r.status === 0) commit = r.stdout.trim(); }
function size(dir) { let t = 0, n = 0; for (const e of readdirSync(dir, { withFileTypes: true })) { const p = join(dir, e.name); if (e.isDirectory()) { const s = size(p); t += s.bytes; n += s.files; } else if (e.isFile()) { t += statSync(p).size; n++; } } return { bytes: t, files: n }; }
const total = size(out);
writeFileSync(join(out, 'BUNDLE.json'), JSON.stringify({
component: 'deepseek-harness', version: cli.version, commit, license: 'MIT', platform, node: process.version,
bytes: total.bytes, files: total.files, entry: 'apps/cli/lib/bin.js', builtAt: new Date().toISOString(),
patches,
note: 'Pruned production install (pnpm deploy --prod). No package manager runs on the user\'s PC.',
}, null, 2));
if (keepIgnore) writeFileSync(join(out, '.gitignore'), keepIgnore);
// --- proof 1: nothing the Harness imports by name was pruned ------------------------------------------------------------
// (2026-10-17: the other-OS rule once removed @deepseek-ai/dsh-win32-process, which every platform imports; Linux/Mac agent chats died.)
const lost = missingHarnessImports(out);
if (lost.length) { console.error(['staged Harness is missing imported packages:', ...lost.map(l => ` ${l.from} -> ${l.missing}`)].join(String.fromCharCode(10))); process.exit(1); }
console.log('import check: every @deepseek-ai package the Harness imports is staged');
// --- proof 2: the staged copy answers --version and boots the web profile once ----------------------------------------
const bin = join(out, 'apps', 'cli', 'lib', 'bin.js');
const entry = existsSync(bin) ? bin : join(out, 'lib', 'bin.js');
const v = spawnSync(process.execPath, [entry, '--version'], { encoding: 'utf8' });
if (v.status !== 0) { console.error('staged Harness does not start:', v.stderr); process.exit(1); }
console.log(`staged ${cli.version} (${commit?.slice(0, 8) ?? 'no commit'}) in ${out}: ${(total.bytes / 1048576).toFixed(0)} MiB, ${total.files} files, ${removed} pruned items; --version -> ${v.stdout.trim()}`);
console.log(`sha256(BUNDLE.json)=${createHash('sha256').update(readFileSync(join(out, 'BUNDLE.json'))).digest('hex')}`);
+721
View File
@@ -0,0 +1,721 @@
#!/usr/bin/env node
/**
* Headless UI flows of the installed app against a packaged host with a FAKE installer and a FAKE engine (nothing is
* downloaded, no real engine runs; the real Control starts the fake engine through the owner runtime):
*
* node apps/desktop/scripts/ui-flows.mjs [--control <staged control>] [--node <node.exe>] [--shots <dir>] [--only download,models,tune,fail,theme]
*
* download the onboarding download: ONE combined bar over both files, an ETA in minutes left of Stop, Finish later closes the
* pop-up with the download still running, the Models screen shows the same progress, Stop stops it
* models an installed model is the one Models selects and loads, without a download pop-up
* tune the tuning step offers no picture reading, tunes, writes a RAM budget for "leave 20%", opens chat; reading size
* fail an engine that stops while loading: the error is in words and the pop-up does not scroll
* theme the app follows the system light/dark setting
* Per-user folders are all under a scratch folder. Exit 1 if a check fails.
*/
import { spawn, spawnSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import { copyFileSync, cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, rmdirSync, symlinkSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') ? true : all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const repoHost = resolve(here, '../../host');
const staged = resolve(here, '../src-tauri/resources');
const stagedControl = resolve(args.control ?? join(staged, 'control'));
const nodeExe = resolve(args.node ?? join(staged, 'node', process.platform === 'win32' ? 'node.exe' : 'node'));
const work = mkdtempSync(join(tmpdir(), 'strata-ui-flows-'));
const shots = resolve(args.shots ?? join(work, 'shots'));
mkdirSync(shots, { recursive: true });
const only = new Set(String(args.only ?? 'download,models,profile,ui2,ui2d,tune,fail,delete,theme,restart,perf,polish,stop,latest').split(','));
const isWin = process.platform === 'win32';
const { chromium } = await import(pathToFileURL(join(repoHost, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const delay = ms => new Promise(r => setTimeout(r, ms));
const sha = buf => createHash('sha256').update(buf).digest('hex');
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
let failed = false;
const check = (ok, label, detail = '') => { console.log(`${ok ? 'ok ' : 'FAIL'} ${label}${detail ? ' ' + detail : ''}`); if (!ok) failed = true; };
// ------------------------------------------------------------------ the app folder: the real Control with a fake installer
const control = join(work, 'control');
mkdirSync(control, { recursive: true });
for (const name of ['python', 'strata_control', 'strata_cli', 'serve']) symlinkSync(join(stagedControl, name), join(control, name), 'junction');
for (const name of ['strata.py', 'VERSION', 'BUNDLE.json', 'requirements.txt']) copyFileSync(join(stagedControl, name), join(control, name));
cpSync(join(stagedControl, 'tools'), join(control, 'tools'), { recursive: true });
const python = isWin ? join(control, 'python', 'python.exe') : join(control, 'python', 'bin', 'python3');
const FAKE_INSTALLER = String.raw`
import json, os, sys, time
from pathlib import Path
ROOT = Path(sys.argv[sys.argv.index('--root') + 1]) if '--root' in sys.argv else Path('.')
STATE = Path(os.environ['FAKE_STATE']); STATE.mkdir(parents=True, exist_ok=True)
RATE = float(os.environ.get('FAKE_RATE_GB_S', '0.5'))
def size(model, gb, ram, mid):
return {'model': model, 'about': model, 'download_gb': gb, 'ram_needed_gb': ram, 'experts_gb': ram - 8, 'on_this_pc': 'fits', 'installed': False, 'images': True, 'experimental': False, 'id': mid}
CATALOG = {'summary': 'fake', 'families': [{'family': 'qwen', 'title': 'Qwen3.8-Flash-Next', 'about': 'x', 'experimental': False, 'images': True, 'sizes': [
size('Q2_0', 66.4, 48, 'q2_0'), size('IQ3_XXS', 75.8, 60, 'iq3_xxs'), size('IQ3_S', 83.6, 62, 'iq3_s')]}],
'contexts': [8192, 32768, 65536], 'recommendation': {'family': 'qwen', 'model': 'IQ3_XXS', 'title': 'Qwen3.8-Flash-Next IQ3_XXS', 'context': 32768}, 'context_note': ''}
HARDWARE = {'ram_gb': 63.1, 'cpu': {'name': 'cpu', 'cores': 12}, 'gpus': [{'index': 0, 'name': 'NVIDIA GeForce RTX 5070', 'vram_gb': 11.9, 'usable': True}]}
TOTAL = 83.6
def job():
f = STATE / 'job.json'
if not f.exists(): return None
j = json.loads(f.read_text())
if j.get('cancelled'):
return {'running': False, 'exit_code': 1, 'error': 'stopped', 'result': 'failed'}
done = min(TOTAL, RATE * (time.time() - j['start']))
return {'running': True, 'step': '5/7: downloading Qwen3.8-Flash-Next IQ3_S', 'model': 'IQ3_S',
'download': {'file': 'model files', 'done_gb': round(done, 3), 'total_gb': TOTAL, 'remaining_gb': round(TOTAL - done, 3), 'percent': int(100 * done / TOTAL)}}
def call(name, a):
if name == 'strata_models': return CATALOG
if name == 'strata_remove':
cfg = ROOT / ('strata-%s.json' % a['model'])
if a.get('confirm') and cfg.exists(): cfg.unlink()
return {'summary': 'Deleted', 'plan': {'gb': 83.6}, 'failed': []}
if name == 'strata_status': return {'summary': 'fake', 'installed': {'models': [{'model': 'iq3_s', 'disk_gb': 83.6, 'disk_bytes': 83600000000}]}, 'hardware': HARDWARE, 'recommendation': CATALOG['recommendation'], 'install': job(), 'running': []}
if name == 'strata_install':
if a.get('cancel'):
f = STATE / 'job.json'
if f.exists(): j = json.loads(f.read_text()); j['cancelled'] = True; f.write_text(json.dumps(j))
return {'summary': 'stopped', 'install': job()}
if a.get('confirm'):
(STATE / 'job.json').write_text(json.dumps({'start': time.time()}))
return {'summary': 'started', 'plan': {'download_gb': TOTAL}, 'install': job()}
if not a: return {'summary': 'progress', 'install': job()}
return {'summary': 'Plan', 'plan': {'title': 'Qwen3.8-Flash-Next IQ3_S', 'download_gb': TOTAL, 'disk_short': None}}
raise Exception('unknown tool ' + name)
for line in sys.stdin:
line = line.strip()
if not line: continue
m = json.loads(line)
if 'id' not in m: continue
method = m.get('method')
if method == 'initialize': res = {'protocolVersion': '2024-11-05'}
elif method == 'tools/list': res = {'tools': [{'name': n} for n in ('strata_models', 'strata_status', 'strata_install', 'strata_remove')]}
else:
p = m['params']
res = {'structuredContent': call(p['name'], p.get('arguments') or {})}
sys.stdout.write(json.dumps({'jsonrpc': '2.0', 'id': m['id'], 'result': res}) + '\n'); sys.stdout.flush()
`;
writeFileSync(join(control, 'tools', 'strata_mcp.py'), FAKE_INSTALLER);
const FAKE_ENGINE = `import os, sys, json, time
from pathlib import Path
if os.environ.get('FAKE_ENGINE_FAIL'):
print('memory guard: 5.1 GiB of RAM free is under the 6 GiB limit (limit 63.1 GiB)', flush=True)
sys.exit(3)
ctx = '4096'
if '--max-context' in sys.argv: ctx = sys.argv[sys.argv.index('--max-context') + 1]
print('READY %s stop' % ctx, flush=True)
for line in sys.stdin:
if line.strip() == 'QUIT': break
if line.startswith(('GEN ', 'BGEN ')):
slot = int(line.split()[1]) if line.startswith('BGEN ') else None
t0 = time.time(); n = 0
for ch in os.environ.get('FAKE_REPLY', 'a'):
print('T %d' % ord(ch), flush=True); n += 1
if os.environ.get('FAKE_TOKEN_DELAY'): time.sleep(float(os.environ['FAKE_TOKEN_DELAY']))
print('DONE %d 12 300.0 %.1f length 0 0 0' % (n, max(1.0, (time.time() - t0) * 1000)), flush=True)
if slot is not None: print(f'BADM {slot} 0', flush=True)
`;
// one scenario = one fresh host with its own per-user folders
async function scenario(name, { installed = false, env = {} } = {}, body) {
if (!only.has(name)) return;
console.log(`--- ${name}`);
const dir = join(work, name);
const dirs = { home: join(dir, 'home'), appdata: join(dir, 'appdata'), local: join(dir, 'local'), components: join(dir, 'local', 'Strata', 'components'), install: join(dir, 'local', 'Strata', 'install'), state: join(dir, 'fake-state'), profiles: join(dir, 'profiles'), runtime: join(dir, 'runtime') };
for (const d of [dirs.home, dirs.appdata, dirs.local, dirs.components, dirs.install, dirs.state, join(dir, 'models')]) mkdirSync(d, { recursive: true });
const baseEnv = { ...process.env, STRATA_COMPONENTS_DIR: dirs.components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(dir, 'models'), APPDATA: dirs.appdata, LOCALAPPDATA: dirs.local, USERPROFILE: dirs.home, XDG_CONFIG_HOME: join(dir, 'xdg'), XDG_DATA_HOME: join(dir, 'xdgdata'), PYTHONDONTWRITEBYTECODE: '1', STRATA_ENGINE_BACKEND: 'cuda', FAKE_STATE: dirs.state, ...env };
delete baseEnv.STRATA_API_KEY; delete baseEnv.STRATA_INSTALL_DIR;
if (installed) {
const platform = isWin ? 'windows-x86_64' : 'linux-x86_64';
const folder = join(dirs.components, 'engine', `0.1.45-${platform}-cuda`);
mkdirSync(folder, { recursive: true });
writeFileSync(join(folder, 'engine_main.py'), FAKE_ENGINE);
const entry = isWin ? 'engine.cmd' : 'engine';
writeFileSync(join(folder, entry), isWin ? `@echo off\r\n"${python}" "%~dp0engine_main.py" %*\r\n` : `#!/bin/sh\nexec "${python}" "$(dirname "$0")/engine_main.py" "$@"\n`);
const files = [entry, 'engine_main.py'].map(p => { const b = readFileSync(join(folder, p)); return { path: p, sha256: sha(b), bytes: b.length }; });
writeFileSync(join(folder, 'engine-component.json'), JSON.stringify({ schema_version: 1, id: 'strata-engine', version: '0.1.45', platform, backend: 'cuda', entrypoint: entry, files }));
const tok = join(dirs.install, 'tokenizer');
mkdirSync(tok, { recursive: true });
const script = `import json, sys\nfrom tools.strata_tokenizer import BYTE_TO_UNICODE\ntok = sys.argv[1]\ntokens = [BYTE_TO_UNICODE[i] for i in range(256)] + ['ab', '<|im_start|>', '<|im_end|>', '<|endoftext|>']\njson.dump({t: i for i, t in enumerate(tokens)}, open(tok + '/vocab.json', 'w'))\nopen(tok + '/merges.txt', 'w').write('a b')\njson.dump([0] * 257 + [3, 3, 3], open(tok + '/token_type.json', 'w'))\n`;
const made = spawnSync(python, ['-c', script, tok], { cwd: control, env: { ...baseEnv, PYTHONNOUSERSITE: '1' }, encoding: 'utf8' });
if (made.status !== 0) throw new Error('tokenizer fixture: ' + made.stderr);
writeFileSync(join(dirs.install, 'strata-iq3_s.json'), JSON.stringify({ exe: 'component:strata-engine', backend: 'cuda', tokenizer: tok, args: ['--max-context', '32768', '--pool-workers', '12'], cwd: dir, model_name: 'stub', host: '127.0.0.1', port: 0, kv: 'int8' }));
}
const port = env.FIXED_PORT ? Number(env.FIXED_PORT) : await freePort(), base = `http://127.0.0.1:${port}`;
const host = spawn(nodeExe, [join(repoHost, 'server', 'index.mjs')], { cwd: repoHost, stdio: ['ignore', 'pipe', 'pipe'], windowsHide: true,
env: { ...baseEnv, STRATA_PORT: String(port), STRATA_DIST: join(repoHost, 'dist'), STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1', STRATA_CONTROL_DIR: control, STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let log = ''; host.stdout.on('data', c => { log += c; }); host.stderr.on('data', c => { log += c; });
const api = async (path, init) => { const r = await fetch(base + path, init); return { status: r.status, body: await r.json().catch(() => null) }; };
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
try {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await api('/api/health')).status === 200; } catch { await delay(150); } }
check(up, `${name}: the packaged host starts`);
await api('/api/native?refresh=1');
await body({ base, api, browser, dirs, shot: (page, n) => page.screenshot({ path: join(shots, `${name}-${n}.png`) }), log: () => log });
} catch (error) {
check(false, `${name}: the flow`, String(error?.stack ?? error).slice(0, 700) + '\n' + log.slice(-600));
} finally {
await browser.close().catch(() => {});
host.kill(); await new Promise(r => host.once('exit', r));
}
}
const newPage = async (browser, base, options = {}) => {
const context = await browser.newContext({ viewport: { width: 1400, height: 950 }, ...(process.env.SCHEME ? { colorScheme: process.env.SCHEME } : {}), ...options });
const page = await context.newPage();
const problems = [];
page.on('pageerror', e => problems.push(e.message));
page.problems = problems;
await page.goto(base, { waitUntil: 'domcontentloaded' });
return page;
};
const clickText = async (page, re, timeout = 30000) => { const b = page.getByRole('button', { name: re }).first(); await b.waitFor({ timeout }); await b.click(); };
const noScroll = async page => page.evaluate(() => {
const root = document.querySelector('.guided-onboarding');
const bad = [];
for (let el = document.querySelector('.onboard-tune-stage, .onboard-prepare-stage, .onboard-model-stage') ?? root; el && el !== document.body; el = el.parentElement) {
const style = getComputedStyle(el);
if (/(auto|scroll)/.test(style.overflowY) && el.scrollHeight > el.clientHeight + 1) bad.push(`${el.className || el.tagName}: ${el.scrollHeight} > ${el.clientHeight}`);
}
return bad;
});
// ------------------------------------------------------------------------------------------------ download
await scenario('download', {}, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /get started/i);
await page.getByText(/Choose a version/i).waitFor({ timeout: 30000 });
await page.locator('[role=radio]').filter({ hasText: 'IQ3_S' }).first().click();
await clickText(page, /^Continue/);
await clickText(page, /Download model/i, 60000);
await page.locator('.onboard-progress progress').waitFor({ timeout: 30000 });
// ONE bar: sample the percent while it runs, it only goes up
const samples = [];
let eta = '';
for (let i = 0; i < 40 && !eta; i++) {
const state = (await api('/api/native')).body;
const d = state.jobs.at(-1)?.result?.install?.download;
if (d) samples.push(d.done_gb);
eta = (await page.locator('.onboard-progress-actions span').first().textContent({ timeout: 1000 }).catch(() => '')) ?? '';
await delay(1000);
}
await shot(page, '1-wizard-progress');
check(/About \d+ (min left|h)/.test(eta), 'the wizard shows about how many minutes are left', eta.trim());
const order = await page.evaluate(() => { const a = document.querySelector('.onboard-progress-actions'); const kids = a ? [...a.children].map(k => k.tagName + ':' + Math.round(k.getBoundingClientRect().left)) : []; return kids; });
check(order.length === 2 && order[0].startsWith('SPAN') && order[1].startsWith('BUTTON') && Number(order[0].split(':')[1]) < Number(order[1].split(':')[1]), 'the time left sits left of Stop', order.join(' '));
check(samples.length >= 3 && samples.every((v, i) => i === 0 || v >= samples[i - 1]), 'one combined count that only goes up', samples.map(v => v.toFixed(1)).join(', '));
const meta = await page.locator('.onboard-progress-meta span').first().textContent();
check(/GB/.test(meta ?? '') && /83\.6/.test(meta ?? ''), 'the bar says done / total over the whole model', meta);
// close the pop-up while it downloads
await clickText(page, /Finish later/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByText(/Preparing your model/i).waitFor({ timeout: 15000 }).catch(() => {});
await delay(1500);
await shot(page, '2-models-progress');
await page.locator('.native-progress').first().screenshot({ path: join(shots, 'download-3-models-progress-card.png') }).catch(() => {});
check(await page.locator('.guided-onboarding').count() === 0, 'the setup pop-up closed');
check(await page.getByText(/Preparing your model/i).count() > 0, 'the Models screen shows the running download');
const still = (await api('/api/native')).body.jobs.at(-1);
check(still?.state === 'running', 'the download keeps running in the background', still?.state);
const modelsEta = await page.locator('.native-progress .onboard-progress-actions span').first().textContent({ timeout: 15000 }).catch(() => '');
check(/About \d+ (min left|h)/.test(modelsEta ?? ''), 'the Models screen shows the time left too', (modelsEta ?? '').trim());
await clickText(page, /^Stop$/);
await delay(2500);
check((await api('/api/native')).body.jobs.at(-1)?.state !== 'running', 'Stop stops it');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ models
await scenario('models', { installed: true }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
await delay(1500);
await shot(page, '1-models');
await page.locator('.version-trigger').first().waitFor({ timeout: 20000 }).catch(async () => { await shot(page, '0-models-missing-summary'); console.log('info page text:', JSON.stringify((await page.locator('body').innerText()).slice(0, 500))); });
const summary = await page.locator('.version-trigger').first().textContent({ timeout: 5000 }).catch(() => '');
// the Models tab fits the first view: nothing to scroll while no list is open and Expert mode is closed
if (await page.getByRole('button', { name: /^Close expert mode$/ }).count()) { await page.getByRole('button', { name: /^Close expert mode$/ }).click(); await delay(500); }
await shot(page, '1b-models-one-view');
const overflow = await page.evaluate(() => { const el = document.querySelector('.main__body'); return el ? el.scrollHeight - el.clientHeight : 0; });
check(overflow <= 2, 'the Models tab fits the first view without scrolling', String(overflow) + ' px over');
check(await page.locator('.family-switch').count() === 1, 'the model family switch is in the navbar');
check(await page.locator('h1').filter({ hasText: /^Model$/ }).count() === 0, 'and the navbar does not say "Model"');
check(/IQ3_S|High quality/.test(summary ?? ''), 'Models selects the model that is on this PC, not the recommendation', (summary ?? '').replace(/\s+/g, ' ').trim());
check(!/IQ3_XXS|Balanced/.test(summary ?? ''), 'the recommendation (IQ3_XXS) is not what is selected');
const load = page.getByRole('button', { name: /^Load model$/ }).first();
await load.click();
await delay(1200);
check(await page.getByRole('dialog').filter({ hasText: /Download and prepare/i }).count() === 0, 'loading an installed model opens no download pop-up');
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { const s = (await api('/api/native')).body; ready = s.instances.some(x => x.status === 'ready'); const err = s.instances.find(x => x.status === 'error'); if (err) throw new Error('load failed: ' + err.error); await delay(1000); }
if (!ready) { const st = (await api('/api/native')).body; console.log('info jobs', JSON.stringify(st.jobs.map(j => [j.kind, j.state, j.error ?? j.summary])), 'instances', JSON.stringify(st.instances.map(i => [i.status, i.error]))); await shot(page, '2-models-after-click'); }
check(ready, 'the model loads (a fake engine under the real Control)');
await page.reload({ waitUntil: 'domcontentloaded' });
await page.getByRole('button', { name: /^Unload$/ }).first().waitFor({ timeout: 30000 }).catch(() => {});
await shot(page, '3-models-loaded');
check(await page.getByRole('button', { name: /^Unload$/ }).count() > 0, 'after a reload the Models screen shows the model as loaded, with Unload');
check(await page.getByRole('button', { name: /^Load model$/ }).count() === 0, 'and no longer offers Load model');
await page.getByRole('button', { name: /^Unload$/ }).first().click();
let gone = false;
for (let i = 0; i < 40 && !gone; i++) { gone = (await api('/api/native')).body.instances.length === 0; await delay(500); }
check(gone, 'Unload stops the engine');
await page.getByRole('button', { name: /^Load model$/ }).first().waitFor({ timeout: 15000 }).catch(() => {});
check(await page.getByRole('button', { name: /^Load model$/ }).count() > 0, 'and Load model is back');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ the profile used, the popover, the pop-up
await scenario('profile', { installed: true }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
await delay(1500);
const work = () => page.getByRole('button', { name: /Leave 20% for my work/ }).first();
const full = () => page.getByRole('button', { name: /Full performance/ }).first();
await work().click();
check(await work().getAttribute('aria-pressed') === 'true', 'Leave 20% is chosen before loading');
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { const s = (await api('/api/native')).body; ready = s.instances.some(x => x.status === 'ready'); await delay(500); }
check(ready, 'the model loads');
const inst = (await api('/api/native')).body.instances.find(x => x.status === 'ready');
check(inst?.resourceProfile === 'work', 'the running model reports the profile it was started with', String(inst?.resourceProfile));
await delay(1500);
check(await work().getAttribute('aria-pressed') === 'true' && await full().getAttribute('aria-pressed') === 'false', 'after the load the page still shows Leave 20%, not Full performance');
await page.reload({ waitUntil: 'domcontentloaded' });
await page.getByRole('button', { name: /^Unload$/ }).first().waitFor({ timeout: 30000 });
await delay(1000);
check(await work().getAttribute('aria-pressed') === 'true', 'and after a reload it is still the profile in use');
await shot(page, '1-profile-used');
// the reasoning popover names the loaded model
await page.evaluate(() => { location.hash = '#/chat'; });
const effort = page.getByRole('button', { name: /^Reasoning effort/ }).first();
await effort.waitFor({ timeout: 30000 });
await effort.click();
const header = await page.locator('.reasoning-menu-model strong').first().textContent({ timeout: 10000 });
check(header && !/No model loaded/i.test(header), 'the Reasoning popover names the loaded model, not "No model loaded"', String(header));
await shot(page, '2-reasoning-popover');
await page.keyboard.press('Escape');
// an installed model asked for with other settings: the pop-up never speaks of downloading
await page.evaluate(() => { location.hash = '#/models'; });
await page.getByRole('button', { name: /^Unload$/ }).first().waitFor({ timeout: 30000 });
await page.getByRole('button', { name: /^Unload$/ }).first().click();
await page.getByRole('button', { name: /^Load model$/ }).first().waitFor({ timeout: 30000 });
await page.getByRole('combobox', { name: 'Conversation context' }).first().click();
const options = page.getByRole('option');
const pick = (await options.allTextContents()).findIndex(t => /^8K$/.test(t.trim()));
if (pick >= 0) await options.nth(pick).click();
await page.getByRole('button', { name: /^Load model$|^Checking/ }).first().click();
const dialog = page.getByRole('dialog').first();
await dialog.waitFor({ timeout: 15000 }).catch(() => {});
const text = (await dialog.textContent().catch(() => '')) ?? '';
await shot(page, '3-apply-dialog');
if (text) {
check(!/Download and prepare|Download . load|approximately|[0-9] GB/i.test(text), 'for a downloaded model the pop-up has no download wording or size', text.replace(/\s+/g, ' ').slice(0, 600));
check(/Apply/.test(text), 'it says what happens: apply settings and load');
} else console.log('info the settings were the saved ones, so loading needed no pop-up at all');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ version list, This computer, Tune, a task
for (const scheme of ['light', 'dark']) await scenario(scheme === 'light' ? 'ui2' : 'ui2d', { installed: true, env: { FAKE_TOKEN_DELAY: '0.02', FAKE_REPLY: 'I am thinking about it. </think> ' + 'Here is the answer, word by word. '.repeat(12), STRATA_TUNE_NO_QUIET_CHECK: '1' } }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base, { colorScheme: scheme });
await clickText(page, /Go straight to the app/i);
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
await delay(1500);
// the version list: Recommended (the fixed four), then folded groups, one line per model
console.log('info triggers', await page.locator('.version-trigger').count(), await page.getByRole('button', { name: 'Model version' }).count());
await page.locator('.version-trigger').first().click();
await delay(300);
console.log('info expanded after click', await page.locator('.version-trigger').first().getAttribute('aria-expanded'));
await delay(800); await shot(page, `0-after-click-${scheme}`);
console.log('info menu', await page.evaluate(() => { const m = document.querySelector('.version-menu'); const t = document.querySelector('.version-trigger'); return JSON.stringify({ menu: m ? m.getBoundingClientRect() : null, parent: m ? [m.parentElement.className, getComputedStyle(m.parentElement).display, getComputedStyle(m.parentElement).visibility, m.parentElement.style.cssText.slice(0, 120)] : null, expanded: t?.getAttribute('aria-expanded'), roles: [...document.querySelectorAll('[role=menu]')].length }); }));
await page.locator('.version-menu').waitFor({ timeout: 10000 });
const heads = await page.locator('.version-head span:first-child, .version-group__head').allTextContents();
const names = await page.locator('.version-menu > .version-row .version-row__name strong').allTextContents();
console.log('info version list', JSON.stringify(heads.map(t => t.replace(/\s+/g, ' ').trim())), JSON.stringify(names));
check(heads[0]?.trim() === 'Recommended', `${scheme}: the list starts with Recommended`);
check(names.length >= 1 && names.length <= 4, `${scheme}: Recommended shows only the picks this catalog has (at most the fixed four)`, names.join(' | '));
check(heads.some(t => /Swift models|GSQ-RCO models|Gyro models|NVFP4 models|Unsloth models/.test(t)), `${scheme}: the other models are in folded groups`);
check(heads.at(-1)?.includes('Use your own GGUF'), `${scheme}: the last row is Use your own GGUF`);
const tall = await page.locator('.version-row').evaluateAll(els => els.filter(e => e.getBoundingClientRect().height > 44).length);
check(tall === 0, `${scheme}: every model is one line`, String(tall));
await shot(page, `1-versions-${scheme}`);
await page.keyboard.press('Escape');
// This computer
const pc = page.locator('.pc-card');
await pc.waitFor({ timeout: 10000 });
const pcText = ((await pc.textContent()) ?? '').replace(/\s+/g, ' ');
check(/This computer/.test(pcText) && /(Fits fully in memory|read from the SSD)/.test(pcText), `${scheme}: This computer says in one line whether it fits`, pcText.slice(0, 200));
check(await pc.locator('.pc-meter').count() >= 1, `${scheme}: it shows the memory bars`);
check(await pc.locator('.chip, .st-pill, [class*="badge"]').count() === 0, `${scheme}: no chips or badges in it`);
await shot(page, `2-computer-${scheme}`);
// Tune: five steps, a live speed, a result card
await page.getByRole('button', { name: /^Tune model$/ }).first().click({ timeout: 10000 });
await delay(1500); await shot(page, `2b-tune-dialog-${scheme}`);
// The fake engine has no other setting worth trying, so its tune passes through "trying" too fast to be seen. The page is shown the progress a
// real tune reports (Control's status file: stage, speeds) for the running view; the start, the result and every other call are real.
let fakePhase = 'run';
await page.route('**/api/native*', async route => {
const response = await route.fetch(); let json = null;
try { json = await response.json(); } catch { return route.fulfill({ response }); }
for (const job of json?.jobs ?? []) if (job.kind === 'autotune' && fakePhase === 'done') { job.state = 'done'; job.progress = null; job.result = { outcome: 'improved', gain_pct: 18, base: { decode_tok_s: 41.2 }, chosen: { decode_tok_s: 48.6 }, preset: { id: 'fast', name: 'Fast · 64K' }, answers: { class: 'exact', fields: [] } }; } else if (job.kind === 'autotune' && job.state === 'running') job.progress = { stage: 'rounds', text: null, minutesLeft: 9, runsDone: 3, runsPlanned: 6, elapsedS: 300, step: 'Trying: more experts kept in memory (round 1 of 2)', speeds: [{ label: 'base', tok_s: 41.2 }, { label: 'Fewer helper threads', tok_s: 40.1 }, { label: 'More experts kept in memory', tok_s: 46.8 }] };
return route.fulfill({ response, json });
});
await page.getByRole('button', { name: /Start tuning|Tune again/ }).first().click({ timeout: 15000 });
await page.locator('.tune-step').first().waitFor({ timeout: 60000 });
check(await page.locator('.tune-step').count() === 5, `${scheme}: the tuner shows its five steps`);
let sawRun = false, sawSpeed = false;
for (let i = 0; i < 400; i++) {
const done = await page.locator('.tune-result, .tune-progress').count();
if (i % 4 === 0) { const jobs = (await api('/api/native')).body?.jobs ?? []; const j = jobs.find(x => x.kind === 'autotune'); if (j) console.log('info tune job', j.state, JSON.stringify(j.progress ?? null).slice(0, 300)); }
if (await page.locator('.tune-step[data-s=run]').count()) sawRun = true;
const big = await page.locator('.tune-live__big').first().textContent().catch(() => '');
if (/[0-9]/.test(big ?? '')) sawSpeed = true;
if (sawRun && sawSpeed) { await delay(700); await shot(page, `3-tune-running-${scheme}`); break; }
if (done) break;
await delay(600);
}
check(sawRun, `${scheme}: a step is shown as running`);
check(sawSpeed, `${scheme}: the live speed shows a number while it runs`);
if (sawSpeed) { fakePhase = 'done'; await page.locator('.tune-outcome').waitFor({ timeout: 15000 }).catch(() => {}); check(/18% faster/.test((await page.locator('.tune-outcome__body h3').textContent().catch(() => '')) ?? ''), `${scheme}: the result card shows the gain as its headline`); check(/Fast/.test((await page.locator('.tune-outcome__facts').textContent().catch(() => '')) ?? ''), `${scheme}: and what it was saved as`); }
await shot(page, `4-tune-result-${scheme}`);
await page.getByRole('button', { name: /^Close$/ }).first().click({ timeout: 10000 }).catch(() => {});
// a task: the opened task is skimmable, with pause / delete / Edit / Run now
const made = await api('/api/tasks', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ title: 'Open Downloads folder', prompt: 'Open my Downloads folder and tell me what is new.', frequency: 'weekdays', at: '08:00', weekday: 1, timeZone: 'Europe/Ljubljana', mode: 'manual', enabled: true }) });
const id = made.body?.id ?? made.body?.task?.id;
check(!!id, `${scheme}: a task is created`, JSON.stringify(made.body).slice(0, 120));
await page.evaluate(i => { location.hash = '#/tasks/' + i; }, id);
await page.getByRole('button', { name: /^Edit$/ }).first().waitFor({ timeout: 15000 });
console.log('info header buttons', await page.getByRole('button', { name: /^Pause / }).count(), await page.getByRole('button', { name: /^Delete / }).count(), await page.getByRole('button', { name: /Run now/ }).count(), JSON.stringify(await page.locator('.task-header-actions button').evaluateAll(b => b.map(x => (x.getAttribute('aria-label') ?? x.textContent ?? '').trim()))));
check(await page.getByRole('button', { name: /^Pause / }).count() === 1 && await page.getByRole('button', { name: /^Delete / }).count() >= 1 && await page.getByRole('button', { name: /Run now/ }).count() >= 1, `${scheme}: pause, delete, Edit and Run now are in the header`);
check(await page.locator('.main__back').count() === 0 && await page.locator('.main__title-line .task-state').count() === 0, `${scheme}: nothing sits on the left of an opened task's navbar`);
const factsText = ((await page.locator('.task-facts').textContent()) ?? '').replace(/\s+/g, ' ');
check(/Weekdays/.test(factsText) && /Schedule/i.test(factsText) && /Last result/i.test(factsText), `${scheme}: schedule, what it may do and the last result are at the top`, factsText.slice(0, 160));
await shot(page, `5-task-${scheme}`);
await page.getByRole('button', { name: /^Pause / }).first().click();
await page.getByRole('button', { name: /^Resume / }).first().waitFor({ timeout: 10000 });
check(true, `${scheme}: Pause becomes Resume`);
check(page.problems.length === 0, `${scheme}: no page error`, page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ tune + reading size
await scenario('tune', { installed: true, env: { FAKE_TOKEN_DELAY: '0.02', FAKE_REPLY: ['I am thinking about it.', '</think>', '', 'Here is a list:', '', '- one', '- two', '', '## Heading', '', 'Some `code` and **bold** text.', '', '| a | b |', '|---|---|', '| 1 | 2 |', '', '```', 'code line', '```', ''].join(String.fromCharCode(10)) } }, async ({ base, api, browser, dirs, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /get started/i);
await page.getByText(/Choose a version/i).waitFor({ timeout: 30000 });
await page.locator('[role=radio]').filter({ hasText: 'IQ3_S' }).first().click();
await clickText(page, /^Continue/);
await page.getByText(/Leave 20% for my work/i).waitFor({ timeout: 30000 });
await delay(800);
await shot(page, '1-tune-step');
check(await page.getByText(/Read pictures when this model loads/i).count() === 0, 'the tuning step offers no picture reading for an app install');
await clickText(page, /Tune & start chatting/i);
let done = false, job = null;
for (let i = 0; i < 120 && !done; i++) { job = (await api('/api/native')).body.jobs.findLast?.(j => j.kind === 'tune') ?? null; done = !!job && job.state !== 'running'; await delay(1000); }
check(job?.state === 'done', 'tuning finishes without a vision error', job?.error ?? job?.state);
const cfg = JSON.parse(readFileSync(join(dirs.install, 'strata-iq3_s.json'), 'utf8'));
check(cfg.args.includes('--resident-budget-gib'), '"leave 20%" wrote a RAM budget for the experts', cfg.args.join(' '));
const gib = Number(cfg.args[cfg.args.indexOf('--resident-budget-gib') + 1]);
check(gib + 10 <= 63.1 * 0.8, 'the budget keeps the model within 80% of the PC RAM', `${gib} GiB`);
// chat + reading size
await page.getByRole('textbox', { name: /Message Strata/i }).waitFor({ timeout: 60000 });
await page.getByRole('textbox', { name: /Message Strata/i }).fill('hello');
await page.keyboard.press('Enter');
await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 60000 });
await page.locator('.st-msg--assistant .st-bubble p').first().waitFor({ timeout: 60000 }).catch(async () => console.log('info reply so far:', JSON.stringify(await page.locator('.st-msg--assistant').first().innerText()).slice(0, 300)));
await page.waitForFunction(() => !document.querySelector('.st-msg--assistant .caret'), null, { timeout: 90000 }).catch(() => {});
// the chat's telemetry chips: speed and context
const chips = page.locator('.telemetry__chip');
const chipTexts = await chips.allInnerTexts();
console.log('info chips', JSON.stringify(chipTexts));
let speedText = '', contextText = '';
for (let i = 0; i < await chips.count(); i++) {
await chips.nth(i).click();
await delay(400);
const pop = await page.locator('.telepop').last().innerText().catch(() => '');
if (/tok\/s|speed/i.test(pop) && !speedText) speedText = pop;
if (/context/i.test(pop) && !contextText) contextText = pop;
await page.keyboard.press('Escape');
}
const tps = Number(speedText.match(/([\d.]+)\s*tok(?:ens)?\/s/)?.[1] ?? NaN);
check(tps > 15 && tps < 120, 'the speed shown is the decode speed (the engine paced at ~50 tokens a second)', speedText.replace(/\s+/g, ' ').slice(0, 160));
check(/32\.8K|32K|32,768|32768/.test(contextText) && !/128K/.test(contextText), 'the context shown is the one the model runs with (32K), not 128K', contextText.replace(/\s+/g, ' ').slice(0, 160));
const sizes = [];
for (const px of [12, 14, 16, 22]) {
await page.evaluate(v => { const k = 'strata.prefs.v2'; const cur = JSON.parse(localStorage.getItem(k) || '{}'); localStorage.setItem(k, JSON.stringify({ ...cur, contentFontSize: v })); }, px);
await page.reload({ waitUntil: 'domcontentloaded' });
await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 30000 });
await page.screenshot({ path: join(shots, `tune-3-chat-${px}px.png`) });
sizes.push(await page.evaluate(() => { const b = document.querySelector('.st-msg--assistant .st-bubble'); const t = b.querySelector('p, .md-li, span') ?? b; const u = document.querySelector('.st-msg--user .st-bubble'); const all = {}; for (const el of b.querySelectorAll('*')) { if (!el.childNodes.length || ![...el.childNodes].some(n => n.nodeType === 3 && n.textContent.trim())) continue; all[el.tagName.toLowerCase() + (el.className ? '.' + String(el.className).split(' ')[0] : '')] = parseFloat(getComputedStyle(el).fontSize); } const extra = {}; for (const [k, q] of Object.entries({ reasoning: '.st-msg--assistant .st-collapse__body', codeblock: '.st-msg--assistant .st-code pre', codehead: '.st-msg--assistant .st-code__head' })) { const e = document.querySelector(q); if (e) extra[k] = parseFloat(getComputedStyle(e).fontSize); } Object.assign(all, extra); const chain = []; for (let e = b.querySelector('p'); e && e !== b; e = e.parentElement) chain.push(e.tagName + '.' + String(e.className).split(' ')[0] + '=' + getComputedStyle(e).fontSize); all.chain = chain; globalThis.__all = all; return [parseFloat(getComputedStyle(b).fontSize), parseFloat(getComputedStyle(t).fontSize), parseFloat(getComputedStyle(u).fontSize), all]; }));
}
console.log('info chain', JSON.stringify(await page.evaluate(() => { const el = document.querySelector('.st-msg--assistant .st-bubble p'); const out = []; for (let e = el; e && !e.classList.contains('st-msg'); e = e.parentElement) out.push(e.tagName + '.' + String(e.className).split(' ')[0] + '=' + getComputedStyle(e).fontSize); return out; })));
// which visible text does NOT follow the reading size? (same elements at 12 and 22)
const survey = async px => { await page.evaluate(v => { const k = 'strata.prefs.v2'; const cur = JSON.parse(localStorage.getItem(k) || '{}'); localStorage.setItem(k, JSON.stringify({ ...cur, contentFontSize: v })); }, px); await page.reload({ waitUntil: 'domcontentloaded' }); await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 30000 }); await delay(600);
return page.evaluate(() => { const out = {}; for (const el of document.querySelectorAll('main *, .chat *, textarea')) { if (![...el.childNodes].some(n => n.nodeType === 3 && n.textContent.trim()) && el.tagName !== 'TEXTAREA') continue; const r = el.getBoundingClientRect(); if (!r.width || !r.height) continue; const key = el.tagName.toLowerCase() + '.' + String(el.className).split(' ')[0] + ' <' + (el.parentElement?.className ?? '').toString().split(' ')[0] + '>'; out[key] = parseFloat(getComputedStyle(el).fontSize); } return out; }); };
const a = await survey(12), b = await survey(22);
// a fresh app (nothing saved) uses 14 px
await page.evaluate(() => localStorage.removeItem('strata.prefs.v2'));
await page.reload({ waitUntil: 'domcontentloaded' });
await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 30000 });
check(await page.evaluate(() => getComputedStyle(document.querySelector('.st-msg--assistant .st-bubble')).fontSize) === '14px', 'the default reading size is 14 px');
const fixed = Object.keys(a).filter(k => k in b && a[k] === b[k]);
console.log('info text that does not follow the reading size:', JSON.stringify(fixed.map(k => k + '=' + a[k])));
await shot(page, '2-chat-22px');
console.log('info sizes', JSON.stringify(sizes), JSON.stringify(await page.locator('.st-msg--assistant .st-bubble').first().innerHTML()).slice(0, 400));
check(sizes[0][1] === 12 && sizes[1][1] === 14 && sizes[2][1] === 16 && sizes[3][1] === 22 && sizes[0][2] === 12 && sizes[3][2] === 22, 'the reading size changes the text size of the conversation', JSON.stringify(sizes));
// the slider itself
await page.evaluate(() => { location.hash = '#/chat?settings=1'; });
const slider = page.getByLabel('Reading text size');
if (await slider.count()) {
await slider.first().focus();
for (let i = 0; i < 4; i++) await page.keyboard.press('ArrowRight');
await delay(500);
const now = await page.evaluate(() => getComputedStyle(document.querySelector('.st-msg--assistant .st-bubble') ?? document.body).fontSize);
check(parseFloat(now) === 18, 'moving the slider changes the conversation text at once', now);
const box = await slider.first().boundingBox();
await page.mouse.click(box.x + box.width * 0.9, box.y + box.height / 2);
await delay(400);
const dragged = await page.evaluate(() => [getComputedStyle(document.querySelector('.st-msg--assistant .st-bubble') ?? document.body).fontSize, getComputedStyle(document.documentElement).getPropertyValue('--st-content-fs'), JSON.parse(localStorage.getItem('strata.prefs.v2') || '{}').contentFontSize]);
check(parseFloat(dragged[0]) >= 20 && parseFloat(dragged[2]) >= 20, 'clicking the slider with the mouse changes it too, and it is saved', dragged.join(' '));
} else console.log('info the settings slider was not reachable by hash here; the stored size was checked above');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ delete
await scenario('delete', { installed: true }, async ({ base, api, browser, dirs, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { ready = (await api('/api/native')).body.instances.some(x => x.status === 'ready'); await delay(1000); }
check(ready, 'a model is loaded');
await clickText(page, /Manage models/i);
await page.getByRole('button', { name: /^Delete$/ }).first().click();
await delay(600);
await shot(page, '1-delete-confirm');
const dialog = await page.getByRole('dialog').last().innerText();
check(/Delete this version\?/.test(dialog) && /Remove 83\.6 GB of downloaded files\. You can download this version again\./.test(dialog), 'the confirm dialog uses the existing words and the real size', dialog.replace(/\s+/g, ' ').slice(0, 600));
check(/running model will be unloaded/i.test(dialog), 'and says the loaded model is unloaded first');
await page.getByRole('button', { name: /^Delete version$/ }).click();
let state = null;
for (let i = 0; i < 40; i++) { state = (await api('/api/native')).body; if (state.instances.length === 0 && !state.installed?.models?.some(m => m.model === 'iq3_s')) break; await delay(500); }
check(state.instances.length === 0, 'the loaded model was unloaded');
check(!existsSync(join(dirs.install, 'strata-iq3_s.json')), 'the installer removed the run config of the model');
check(!state.installed?.models?.some(m => m.model === 'iq3_s'), 'the model is no longer listed as downloaded');
await shot(page, '2-after-delete');
});
// ------------------------------------------------------------------------------------------------ perf
await scenario('perf', { installed: true }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { ready = (await api('/api/native')).body.instances.some(x => x.status === 'ready'); await delay(1000); }
check(ready, 'a model is loaded');
await delay(2500);
const requests = [];
const started = new Map();
page.on('request', r => started.set(r, Date.now()));
page.on('requestfinished', r => requests.push([r.url().replace(base, '').slice(0, 60), Date.now() - (started.get(r) ?? Date.now())]));
await page.evaluate(() => { window.__long = []; new PerformanceObserver(l => { for (const e of l.getEntries()) window.__long.push(Math.round(e.duration)); }).observe({ entryTypes: ['longtask'] }); });
const t0 = Date.now();
await page.evaluate(() => { location.hash = '#/chat'; });
await page.getByRole('textbox', { name: /Message Strata/i }).waitFor({ timeout: 30000 });
const open = Date.now() - t0;
await delay(1500);
const long = await page.evaluate(() => window.__long);
console.log('info chat opened in', open, 'ms; long tasks', JSON.stringify(long), 'requests', JSON.stringify(requests.filter(r => r[1] > 100)), 'all', requests.length);
check(open < 1500 && Math.max(0, ...long) < 300, 'Models to Chat opens at once, with no long block', open + ' ms, longest task ' + Math.max(0, ...long) + ' ms');
});
// ------------------------------------------------------------------------------------------------ polish
await scenario('polish', { installed: true }, async ({ base, api, browser, dirs, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
check(await page.getByRole('button', { name: /^Tune model$/ }).count() === 1, 'the Models tab has the Tune model button (the real tuner, not the old short benchmark)');
await clickText(page, /Manage models/i);
await delay(500);
const rows = await page.locator('.native-manage-row').count();
check(rows === 1, 'Manage models lists only the downloaded model, not the catalog', rows + ' row(s)');
await shot(page, '1-manage-models');
await page.getByRole('button', { name: /^Done$/ }).click();
await page.getByRole('button', { name: /Other local models/i }).click().catch(async () => { await clickText(page, /Manage models/i); await clickText(page, /Use a local GGUF/i); });
await page.getByText(/Folder with the GGUF files/i).first().waitFor({ timeout: 15000 }).catch(() => {});
await shot(page, '2-local-gguf');
check(await page.getByText(/Folder with the GGUF files/i).count() > 0, 'the local GGUF page opens in the same design as Models');
await page.getByPlaceholder(/Full path to the folder/i).fill(join(dirs.install, 'does-not-exist'));
await page.getByRole('button', { name: /^Check files$/ }).click();
await delay(3000);
check(await page.locator('.error-notice').count() > 0, 'a folder that cannot be used says so in words');
await shot(page, '2b-local-gguf-problem');
await page.evaluate(() => { location.hash = '#/chat?settings=1'; });
await delay(1500);
const text = await page.locator('body').innerText();
check(!/Contrast/.test(text), 'Settings has no Contrast option');
check(!/View request log/.test(text), 'and no "View request log" button');
await page.getByText(/^Engine$/).first().click().catch(() => {});
await delay(1500);
await shot(page, '3-settings-engine');
const engine = await page.locator('body').innerText();
check(/Update component/.test(engine) && /Diagnostics/.test(engine), 'Settings > Engine has Update component and Diagnostics');
check(!/Performance|Model settings|Calibrate|Engine installation/.test(engine), 'and nothing else');
await page.getByRole('button', { name: /^View logs$/ }).click(); // the logs open in a pop-up that holds Save logs
await page.getByRole('button', { name: /^Save logs$/ }).click();
await page.getByText(/Saved to/).first().waitFor({ timeout: 15000 }).catch(() => {});
const savedText = await page.getByText(/Saved to/).first().innerText().catch(() => '');
const savedPath = savedText.replace(/^Saved to\s*/, '').trim();
check(!!savedPath && existsSync(savedPath), 'Save logs writes a file to Downloads and says where', savedPath);
await shot(page, '4-logs-saved');
await page.evaluate(() => { location.hash = '#/monitor'; });
await delay(6000);
await shot(page, '5-monitor');
const mon = await page.locator('body').innerText();
check(/°C/.test(mon), 'the Monitor shows GPU data with no model loaded', (mon.match(/.{0,30}°C.{0,20}/)?.[0] ?? 'no temperature text').replace(/\s+/g, ' '));
});
// ------------------------------------------------------------------------------------------------ stop
await scenario('stop', { installed: true, env: { FAKE_TOKEN_DELAY: '0.1', FAKE_REPLY: 'thinking '.repeat(30) } }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { ready = (await api('/api/native')).body.instances.some(x => x.status === 'ready'); await delay(1000); }
if (!ready) { const st = (await api('/api/native')).body; console.log('info jobs', JSON.stringify(st.jobs.map(j => [j.kind, j.state, j.error ?? j.summary])), JSON.stringify(st.instances.map(i => [i.status, i.error]))); await shot(page, '0-stop-notloaded'); }
check(ready, 'a model is loaded');
await page.evaluate(() => { location.hash = '#/chat'; });
const box = page.getByRole('textbox', { name: /Message Strata/i });
await box.waitFor({ timeout: 30000 });
await box.fill('hello');
await page.keyboard.press('Enter');
await page.locator('.typing').first().waitFor({ timeout: 30000 });
await delay(1500);
await page.getByRole('button', { name: /^Stop/i }).first().click().catch(async () => { await page.keyboard.press('Escape'); });
await delay(4000);
const dots = await page.locator('.typing').count();
await shot(page, '1-after-stop');
check(dots === 0, 'after Stop in the middle of thinking the three dots are gone', dots + ' left');
});
// ------------------------------------------------------------------------------------------------ latest
await scenario('latest', {}, async ({ base, browser, shot }) => {
const page = await newPage(browser, base);
const now = Date.now();
await page.evaluate(now => {
const messages = [];
for (let i = 0; i < 40; i++) { messages.push({ id: 'u' + i, role: 'user', content: 'question ' + i, at: now }, { id: 'a' + i, role: 'assistant', content: 'answer number ' + i + ' with some text to fill the line and more of it.', at: now, tokens: 20, inputTokens: 5 }); }
localStorage.setItem('strata.sessions.v1', JSON.stringify([{ id: 's_latest', title: 'long', updatedAt: now, createdAt: now, messages }]));
localStorage.setItem('strata.active-session.v1', 's_latest');
}, now);
await page.reload({ waitUntil: 'domcontentloaded' });
await delay(1500);
if (await page.getByRole('button', { name: /Go straight to the app/i }).count()) { await clickText(page, /Go straight to the app/i); await delay(1500); }
await page.goto(base + '/#/chat', { waitUntil: 'domcontentloaded' });
await page.getByRole('textbox', { name: /Message Strata/i }).waitFor({ timeout: 30000 });
await delay(800);
await page.locator('.main__body.chat').first().waitFor({ timeout: 15000 }).catch(async () => { await shot(page, '0-latest-missing'); console.log('info body', JSON.stringify((await page.locator('body').innerText()).slice(0, 300))); });
await page.locator('.main__body.chat').first().evaluate(el => { el.scrollTop = 3000; });
await delay(600);
await page.mouse.wheel(0, -400);
await delay(600);
await page.locator('.main__body.chat').first().evaluate(el => { el.scrollTop = 0; });
await delay(800);
const gaps = await page.evaluate(() => { const b = document.querySelector('.jumplatest'), c = document.querySelector('.main__body.chat'); if (!b || !c) return null; const r = b.getBoundingClientRect(), cr = c.getBoundingClientRect(); return { top: Math.round(r.top - cr.top), bottom: Math.round(cr.bottom - r.bottom) }; });
console.log('info chat dom', JSON.stringify(await page.evaluate(() => { const c = document.querySelector('.main__body.chat'); return { msgs: document.querySelectorAll('.st-msg').length, scrollH: c?.scrollHeight, clientH: c?.clientHeight, top: c?.scrollTop, jump: document.querySelectorAll('.jumplatest').length }; })));
await shot(page, '1-latest-button');
check(!!gaps && gaps.top >= 12 && gaps.bottom >= 12, 'the Latest button keeps a gap from the top navigation and from the bottom', JSON.stringify(gaps));
});
// ------------------------------------------------------------------------------------------------ fail
await scenario('fail', { installed: true, env: { FAKE_ENGINE_FAIL: '1' } }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /get started/i);
await page.getByText(/Choose a version/i).waitFor({ timeout: 30000 });
await page.locator('[role=radio]').filter({ hasText: 'IQ3_S' }).first().click();
await clickText(page, /^Continue/);
await clickText(page, /Tune & start chatting/i, 30000);
await page.getByText(/ran low on memory/i).first().waitFor({ timeout: 120000 });
await delay(1000);
await shot(page, '1-tune-error');
const text = await page.locator('.guided-onboarding').innerText();
check(/ran low on memory/.test(text), 'the error says in words that the PC ran low on memory');
check(/Leave 20% for my work/.test(text) && /low-RAM/.test(text), 'and what to try');
check(!/server_exited|memory guard/.test(text), 'no code or log line in the main text');
check(!/vision files/i.test(text), 'no picture-reading error');
const bad = await noScroll(page);
check(bad.length === 0, 'the tuning pop-up does not scroll when an error shows', bad.join('; '));
await page.locator('.error-notice summary').first().click();
await delay(500);
await shot(page, '2-tune-error-details');
const details = await page.locator('.error-notice pre').first().innerText();
check(/server_exited/.test(details) && /exited before/.test(details), 'Details holds the code and the reason the server logged', details.replace(/\s+/g, ' ').slice(0, 120));
const badOpen = await noScroll(page);
check(badOpen.length === 0, 'and the pop-up still does not scroll with Details open', badOpen.join('; '));
await page.setViewportSize({ width: 1280, height: 700 });
await delay(600);
const small = await noScroll(page);
check(small.length === 0, 'nor in a small window (1280x700), Details open', small.join('; '));
await shot(page, '3-tune-error-small-window');
void api;
});
// ------------------------------------------------------------------------------------------------ restart
// The app keeps its theme, text size, language and chat list in the browser's storage, which belongs to the window's origin
// (127.0.0.1:<port>). The shell now uses the same port every launch: stop the host, start it again on that port, reopen the
// window with the same browser profile, and everything is still there.
if (only.has('restart')) {
console.log('--- restart');
const fixed = await freePort();
const profile = join(work, 'browser-profile');
const run = async (body) => scenario('restart', { env: { FIXED_PORT: String(fixed) } }, async ctx => {
const context = await chromium.launchPersistentContext(profile, { executablePath: CHROME, headless: true, viewport: { width: 1200, height: 800 } });
try { await body(context, ctx); } finally { await context.close(); }
});
only.add('restart');
await run(async (context, { base }) => {
const page = context.pages()[0] ?? await context.newPage();
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.evaluate(() => { localStorage.setItem('strata.prefs.v2', JSON.stringify({ theme: 'dark', contentFontSize: 19, language: 'de' })); localStorage.setItem('strata.sessions.v1', JSON.stringify([{ id: 'chat-1', title: 'My chat' }])); });
});
await run(async (context, { base }) => {
const page = context.pages()[0] ?? await context.newPage();
await page.goto(base, { waitUntil: 'domcontentloaded' });
const saved = await page.evaluate(() => ({ prefs: JSON.parse(localStorage.getItem('strata.prefs.v2') || '{}'), chats: localStorage.getItem('strata.sessions.v1'), theme: document.documentElement.dataset.theme }));
check(saved.prefs.contentFontSize === 19 && saved.prefs.language === 'de', 'text size and language survive a restart on the same port', JSON.stringify(saved.prefs));
check(saved.theme === 'dark', 'the theme survives too', saved.theme);
check(/My chat/.test(saved.chats ?? ''), 'and the chat list', saved.chats);
});
}
// ------------------------------------------------------------------------------------------------ theme
await scenario('theme', {}, async ({ base, browser, shot }) => {
for (const scheme of ['dark', 'light']) {
const page = await newPage(browser, base, { colorScheme: scheme });
await delay(1500);
const theme = await page.evaluate(() => document.documentElement.dataset.theme);
check(theme === scheme, `a fresh app follows the system (${scheme})`, theme);
await shot(page, `1-${scheme}`);
await page.context().close();
}
});
// the junctions into the staged Control go first (rmdir on a junction removes only the link)
for (const name of ['python', 'strata_control', 'strata_cli', 'serve']) { try { rmdirSync(join(control, name)); } catch { /* already gone */ } }
try { rmSync(work, { recursive: true, force: true, maxRetries: 3 }); } catch (e) { console.log('info scratch folder left behind:', work); }
console.log(`shots in ${shots}`);
process.exit(failed ? 1 : 0);
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env node
/**
* Picture reading in the APP, end to end on a test box, through the installed app layout (packaged host + bundled Control + a real engine component
* that carries strata-vision):
* Download and prepare with "Read pictures" ON (the picture-reader file is fetched, the vision settings are written), load the model with the picture
* reader, send a real picture through the host's own /v1 endpoint and check that the answer names what is in it, unload, shut down.
*
* node vision-e2e.mjs --install <folder with node/ host/ control/> --components <folder holding engine/<package with strata-vision>> \
* --work <scratch> --family qwen --model IQ3_XXS --data <data folder that has (or will get) the model files> --image shapes.png [--device cpu|gpu]
*
* The picture has a red circle, a blue square, a green triangle and the text "STRATA 42". The run exits 1 if any step fails.
*/
import { spawn } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, readdirSync, statSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
for (const need of ['install', 'components', 'work', 'model', 'image']) if (!args[need]) { console.error('missing --' + need); process.exit(2); }
const install = resolve(args.install), work = resolve(args.work), components = resolve(args.components), family = String(args.family ?? 'qwen'), modelId = String(args.model);
const device = String(args.device ?? 'cpu');
const node = join(install, 'node', 'node'), hostEntry = join(install, 'host', 'server', 'index.mjs'), controlDir = join(install, 'control');
const dirs = { models: join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home'), xdg: join(work, 'xdg'), appdata: join(work, 'appdata') };
for (const d of [dirs.models, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
let failed = false;
const record = (name, ok, detail = '') => { if (!ok) failed = true; console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 600) : ''}`); };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.models,
STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models-unused'), APPDATA: dirs.appdata, XDG_CONFIG_HOME: dirs.xdg, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(node, [hostEntry], { cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'],
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
async function waitJob(id, seconds) {
let last = '';
for (let i = 0; i < seconds * 2; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.summary !== last) { last = job.summary; console.log(' ... ' + String(last).slice(0, 120)); }
if (job && job.state !== 'running') return { job, state };
await delay(500);
}
throw new Error('operation did not finish');
}
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { /* not yet */ } if (!up) await delay(150); }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
const packages = existsSync(join(components, 'engine')) ? readdirSync(join(components, 'engine')) : [];
const reader = packages.some(p => existsSync(join(components, 'engine', p, 'strata-vision')));
record('the engine component carries the picture reader (strata-vision)', reader, packages.join(', '));
const status = (await call('/api/control/status')).body;
record('Control: engine usable', status.engine?.usable === true, `engine=${status.engine?.engine?.version}`);
// ---- Download and prepare, "Read pictures" on (the app sends vision "yes" for the download)
const values = { family, model: modelId, context: 32768, vision: 'yes', loadVision: device, ...(args.data ? { data_dir: resolve(args.data) } : {}) };
const plan = await call('/api/native/plan', 'POST', values);
record('the plan for the download is made', plan.status === 200, JSON.stringify(plan.body).slice(0, 300));
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: values });
record('the install operation starts', started.status === 200 || started.status === 202, JSON.stringify(started.body).slice(0, 200));
const done = await waitJob(started.body.id, 3600);
record('Download and prepare finishes', done.job.state === 'done', `${done.job.state} ${done.job.error ?? done.job.summary}`);
const native = (await call('/api/native?refresh=1')).body;
console.log(' installed models: ' + JSON.stringify((native.installed?.models ?? []).map(m => ({ id: m.id, model: m.model, ready: m.ready, visionPrepared: m.visionPrepared, images: m.images, config: m.config }))));
const want = modelId.toLowerCase();
const model = native.installed?.models?.find(m => [m.id, m.model, m.tag].some(x => String(x ?? '').toLowerCase() === want));
record('the model is installed with the picture reader prepared (visionPrepared)', !!model?.visionPrepared, JSON.stringify({ ready: model?.ready, visionPrepared: model?.visionPrepared, images: model?.images }));
const configFile = model?.configPath ?? (model?.config ? join(dirs.models, model.config) : null);
let cfg = {};
try { cfg = JSON.parse(readFileSync(configFile, 'utf8')); } catch { /* reported below */ }
const mm = cfg.vision?.mmproj;
record('the run settings carry the vision entry and --vision, and the mmproj file is on disk',
!!cfg.vision && (cfg.args ?? []).includes('--vision') && !!mm && existsSync(mm) && statSync(mm).size > 800e6,
`vision=${cfg.vision ? 'yes' : 'no'} --vision=${(cfg.args ?? []).includes('--vision')} mmproj=${mm ? Math.round(statSync(mm).size / 1048576) + ' MB' : 'none'}`);
// ---- load with the picture reader
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: model?.model ?? modelId.toLowerCase(), vision: device } });
const loaded = await waitJob(load.body.id, 900);
const instance = loaded.state.instances[0];
record('load starts the engine with the picture reader', loaded.job.state === 'done' && instance?.status === 'ready', `${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}; vision=${instance?.vision}`);
record('the host says the loaded model reads pictures', !!instance?.vision, `vision=${instance?.vision}`);
// ---- a real picture through the host's own /v1 endpoint
const png = readFileSync(resolve(args.image)).toString('base64');
const t = Date.now();
let answer = '', finish = '';
for (const budget of [1536, 4096]) {
const chat = await call('/v1/chat/completions', 'POST', { model: instance?.apiModelId ?? modelId, max_tokens: budget, stream: false, messages: [{ role: 'user', content: [
{ type: 'text', text: 'Describe this picture: name each shape with its colour and read out any text.' },
{ type: 'image_url', image_url: { url: 'data:image/png;base64,' + png } }] }] });
const choice = chat.body?.choices?.[0] ?? {};
answer = choice.message?.content ?? '';
finish = choice.finish_reason ?? String(chat.status);
if (answer.trim() || finish !== 'length') break;
}
console.log(` answer (${Math.round((Date.now() - t) / 1000)} s, finish=${finish}): ${JSON.stringify(answer.slice(0, 300))}`);
const low = answer.toLowerCase();
record('the answer names the red circle', /red|circle/.test(low));
record('the answer names the blue square', /blue|square/.test(low));
record('the answer names the green triangle', /green|triangle/.test(low));
record('the answer reads STRATA 42', low.includes('42'));
// ---- unload
const unloaded = await call(`/api/native/instances/${instance?.id}/unload`, 'POST', {});
record('unload', unloaded.status === 200, unloaded.status);
}
try { await main(); } catch (e) { record('run', false, e.stack ?? e); }
host.kill();
await delay(1500);
console.log(failed ? 'VISION-E2E FAIL' : 'VISION-E2E PASS');
process.exit(failed ? 1 : 0);
+2
View File
@@ -0,0 +1,2 @@
# Rust build output. Regenerated by `cargo tauri build`; never committed.
/target/
+5612
View File
File diff suppressed because it is too large. Load diff
+32
View File
@@ -0,0 +1,32 @@
[package]
name = "strata-desktop"
version = "0.2.7"
description = "Strata Desktop — local AI, tuned to one model."
edition = "2021"
rust-version = "1.77"
license = "MIT"
# Linux: the drop of a file on the window is read straight from the GTK drag signals (see hook_linux_drops in main.rs); the versions are the ones Tauri itself uses.
[target.'cfg(target_os = "linux")'.dependencies]
gtk = "0.18"
webkit2gtk = "2.0"
[build-dependencies]
tauri-build = { version = "2", features = [] }
[dependencies]
tauri = { version = "2", features = ["tray-icon", "image-png"] }
tauri-plugin-shell = "2"
# Desktop notifications for an approval that waits (#3477); the tray icon above is Tauri's own.
tauri-plugin-notification = "2"
tauri-plugin-updater = "2"
tauri-plugin-single-instance = "2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
[profile.release]
codegen-units = 1
lto = true
opt-level = "s"
panic = "abort"
strip = true
+3
View File
@@ -0,0 +1,3 @@
fn main() {
tauri_build::build()
}
+15
View File
@@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<!-- Tauri uses the system WKWebView, so this bundle needs no JIT entitlement of its own;
the web content process gets what it needs from the OS. The App Host runs as an ordinary
child process and needs no special rights. Kept minimal on purpose. -->
<key>com.apple.security.files.user-selected.read-write</key>
<true/>
<key>com.apple.security.network.client</key>
<true/>
<key>com.apple.security.network.server</key>
<true/>
</dict>
</plist>
@@ -0,0 +1 @@
{"core":{"default_permission":{"identifier":"default","description":"Default core plugins set.","permissions":["core:path:default","core:event:default","core:window:default","core:webview:default","core:app:default","core:image:default","core:resources:default","core:menu:default","core:tray:default"]},"permissions":{},"permission_sets":{},"global_scope_schema":null},"core:app":{"default_permission":{"identifier":"default","description":"Default permissions for the plugin.","permissions":["allow-version","allow-name","allow-tauri-version","allow-identifier","allow-bundle-type","allow-register-listener","allow-remove-listener","allow-supports-multiple-windows"]},"permissions":{"allow-app-hide":{"identifier":"allow-app-hide","description":"Enables the app_hide command without any pre-configured scope.","commands":{"allow":["app_hide"],"deny":[]}},"allow-app-show":{"identifier":"allow-app-show","description":"Enables the app_show command without any pre-configured scope.","commands":{"allow":["app_show"],"deny":[]}},"allow-bundle-type":{"identifier":"allow-bundle-type","description":"Enables the bundle_type command without any pre-configured scope.","commands":{"allow":["bundle_type"],"deny":[]}},"allow-default-window-icon":{"identifier":"allow-default-window-icon","description":"Enables the default_window_icon command without any pre-configured scope.","commands":{"allow":["default_window_icon"],"deny":[]}},"allow-exit":{"identifier":"allow-exit","description":"Enables the exit command without any pre-configured scope.","commands":{"allow":["exit"],"deny":[]}},"allow-fetch-data-store-identifiers":{"identifier":"allow-fetch-data-store-identifiers","description":"Enables the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":["fetch_data_store_identifiers"],"deny":[]}},"allow-identifier":{"identifier":"allow-identifier","description":"Enables the identifier command without any pre-configured scope.","commands":{"allow":["identifier"],"deny":[]}},"allow-name":{"identifier":"allow-name","description":"Enables the name command without any pre-configured scope.","commands":{"allow":["name"],"deny":[]}},"allow-register-listener":{"identifier":"allow-register-listener","description":"Enables the register_listener command without any pre-configured scope.","commands":{"allow":["register_listener"],"deny":[]}},"allow-remove-data-store":{"identifier":"allow-remove-data-store","description":"Enables the remove_data_store command without any pre-configured scope.","commands":{"allow":["remove_data_store"],"deny":[]}},"allow-remove-listener":{"identifier":"allow-remove-listener","description":"Enables the remove_listener command without any pre-configured scope.","commands":{"allow":["remove_listener"],"deny":[]}},"allow-set-app-theme":{"identifier":"allow-set-app-theme","description":"Enables the set_app_theme command without any pre-configured scope.","commands":{"allow":["set_app_theme"],"deny":[]}},"allow-set-dock-visibility":{"identifier":"allow-set-dock-visibility","description":"Enables the set_dock_visibility command without any pre-configured scope.","commands":{"allow":["set_dock_visibility"],"deny":[]}},"allow-supports-multiple-windows":{"identifier":"allow-supports-multiple-windows","description":"Enables the supports_multiple_windows command without any pre-configured scope.","commands":{"allow":["supports_multiple_windows"],"deny":[]}},"allow-tauri-version":{"identifier":"allow-tauri-version","description":"Enables the tauri_version command without any pre-configured scope.","commands":{"allow":["tauri_version"],"deny":[]}},"allow-version":{"identifier":"allow-version","description":"Enables the version command without any pre-configured scope.","commands":{"allow":["version"],"deny":[]}},"deny-app-hide":{"identifier":"deny-app-hide","description":"Denies the app_hide command without any pre-configured scope.","commands":{"allow":[],"deny":["app_hide"]}},"deny-app-show":{"identifier":"deny-app-show","description":"Denies the app_show command without any pre-configured scope.","commands":{"allow":[],"deny":["app_show"]}},"deny-bundle-type":{"identifier":"deny-bundle-type","description":"Denies the bundle_type command without any pre-configured scope.","commands":{"allow":[],"deny":["bundle_type"]}},"deny-default-window-icon":{"identifier":"deny-default-window-icon","description":"Denies the default_window_icon command without any pre-configured scope.","commands":{"allow":[],"deny":["default_window_icon"]}},"deny-exit":{"identifier":"deny-exit","description":"Denies the exit command without any pre-configured scope.","commands":{"allow":[],"deny":["exit"]}},"deny-fetch-data-store-identifiers":{"identifier":"deny-fetch-data-store-identifiers","description":"Denies the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":[],"deny":["fetch_data_store_identifiers"]}},"deny-identifier":{"identifier":"deny-identifier","description":"DenLine truncated
@@ -0,0 +1 @@
{}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 433 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1004 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 771 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<foreground android:drawable="@mipmap/ic_launcher_foreground"/>
<background android:drawable="@color/ic_launcher_background"/>
</adaptive-icon>
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 949 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<color name="ic_launcher_background">#FFFFFF</color>
</resources>
Binary file not shown.
Loaded 100 of 2333 files, more files were not shown because too many files have changed in this diff. Show more